hdits4itk-2016_tech_eval_crit_(003).docx

TECHNICAL EVALUATION PLAN/CRITERIA

PROJECT TITLE: Help Desk and Information Technology Support for the Insight Toolkit 2016 (HDITS4ITK-2016)

Peer review of the proposals will be accomplished by a Technical Evaluation Group (TEG) in compliance with NIH policy and procedure.

The technical evaluation criteria to be applied follow. Subcategories of each evaluation criteria carry equal weight.

Criteria

Points

1. Understanding the Requirement and Technical Approach (scientific, technical, or analytical approach to achieve project objectives, including a demonstrated understanding of potential problems.)

The proposal must demonstrate a thorough understanding of the requirements of the Statement of Work and describe an approach that will demonstrate the achievement of timely and acceptable performance. The proposal must present a comprehensive statement of the problem, scope, and purpose of the project to demonstrate an understanding of the requirements from a management and technical standpoint.

Specifically the proposal must demonstrate how the offeror will meet the specific tasks in the statement of work including:

(a) Maintain and monitor communications with the ITK user community. This includes the running of mailing lists/list-serves and responding user questions and complaints. It also includes conducting and moderating weekly teleconference meetings for ITK developers, keeping minutes and maintaining meeting agendas.

(b) Identify and correct bugs and other software flaws in ITK.

(c) Contribute documentation, tutorial materials, and courseware. Organize outreach activities such as one or two symposia, tutorials, or workshops in conjunction with related conferences per year.

(d) Produce semi-annual software releases of ITK.

(e) Perform all software and documentation development under the style guidelines and architectural rules of the Insight Toolkit.

(f) Maintain the ITK software repository using a source code control system. The contractor will support ITK through Git until and unless the ITK developer community adopts a different system.

(g) Support the quality of ITK software through the use of software code review tools. The contractor will promote software quality in ITK by supporting Gerrit until and unless the ITK developer community adopts a different system.

2. Qualifications and Availability of Proposed Personnel (demonstrated evidence of the qualifications, experience, and availability of professional and technical personnel comprising the necessary project staff.)

Each proposal must demonstrate and document the relevant experience, education, and availability of developers to meet the needs of the project. Some of the traits NLM is seeking include:

· Strong knowledge of C++ object-oriented programming.

· Demonstrated experience (of at least a year) in supporting a Git and Gerrit software revision control and review system (verified through peer-reviewed publication or the existence of an established Github site).

· In-depth knowledge of the ITK software internals (verified through a response to a written or oral inquiry or a track record of responses and answers on the existing ITK users and developers mailing lists).

· Demonstrated understanding of collaborative multi-institution open-source software development principles (verified through documented participation of the offeror in the Insight Software Consortium or as part of some other open-source community project).

· Scientific depth in image processing methods (demonstrated through peer-reviewed papers with research investigators named as authors).

· Commitment to developing software under this contract under the style rules and architectural design of the Insight Toolkit.

Offerors should map named personnel to technical tasks in section I, making initial staff assignments of software developers to specific programming objectives.

3. Institutional Experience/Commitment (Commitment to the generation of public software data resources for medical image analysis) NLM is seeking confirmation that the offeror is committed to building and releasing open-source software, especially in working with software under the ITK license or similar unrestricted copyrights. Support of this experience and commitment should be expressed through (a) documentation of previous releases of ITK software, (b) a letter of support from the offeror’s Dean or Chief Operating Officer or similar official stating the offeror’s commitment to releasing patent privileges and copyrights of work performed under this requirement to the public domain, and/or (c) the text of the copyright or license, documented or clearly cited for its source, used by the offeror in previous releases of open source software and/or public domain data.

4. Proposed Facilities and Equipment (availability and proposed utilization.)

The contractor should have a demonstrated ongoing active research interest and an in-house program in clinical or biomedical information processing or the use of medical computing in health services research. This requirement is necessary to ensure the contractor has the established background and knowledge required to make practical advances in these complex areas of research. NLM wishes to establish relationships with institutions or organizations that have independent/complementary research interests to ensure continued development towards the eventual goal of comprehensive algorithm coverage in ITK.

The contractor should also have access to members of the principal user groups for which ITK is intended (i.e., computer software designers and toolbuilders for health sciences practitioners, educators, health services researchers, biomedical researchers) who can be used to test the effectiveness of the structures and models developed.

The proposal must describe the availability and proposed utilization of appropriate facilities and equipment required to successfully perform work in the proposed application area. This includes access to source data and validation instrumentation.

5. Past Performance

The evaluation will be based on information provided as part of the technical proposal. Additional information may be obtained by the contracting officer/contracting officer representative from references provided by the offeror as well as other relevant past performance information obtained from other sources known to the government.

Evaluation of Past Performance should include previous experience with Grants and other assistance vehicles as well as with previous contracts. Grants and contracts arising from other agencies such as DARPA and NSF will likely be part of the histories and should be evaluated without prejudice.

Total Points Allowable