DRAFT - BRAES Performance Work Statement April - DRAFT.docx

PERFORMANCE WORK STATEMENT

Biomedical Research Application Evaluation and Support

SECTION 1.0: GENERAL INFORMATION

0. Purpose The purpose of this contract is to award a support contract for biomedical research application evaluation and support for the Congressionally Directed Medical Research Programs (CDMRP) and U.S. Army Medical Research and Development Command (USAMRDC). This support encompasses electronic receipt of intramural and extramural pre- and full research applications, application compliance, scientific peer review, and maintaining/enhancing the government-owned, contractor-operated electronic biomedical research application portal (eBRAP). A major deliverable of this support contract is a compiled summary document containing the evaluative comments and scores of appropriate experts in accordance with criteria outlined in a funding opportunity announcement (FOA) for each compliant application submitted to that FOA.

Supporting the tasks identified below requires an experienced and cohesive team of scientific, technical, and administrative personnel who can demonstrate a comprehensive understanding of the requirements and have proven capabilities to provide biomedical research application evaluation and support, produce quality deliverables, and remain within the required timelines while meeting cost, schedule, and performance requirements.

1.2 Background

The United States Congress directed the Department of Defense (DoD) to manage appropriations totaling over $15.9 Billion, since fiscal year 1992 (FY92), for peer reviewed competitive programs directed toward targeted medical research initiatives. The United States Army Medical Research and Development Command (USAMRDC) is responsible for administering these targeted research program appropriations.

The CDMRP is an office within the USAMRDC that implements funding and serves as the executive management agent for a majority of the biomedical research programs directed by Congress in addition to providing support for the core (presidential budget) DoD and Army medical research program areas.

The DOD Breast Cancer Research Program was established in FY92 by Appropriations Conference Committee Report No 102-328, which provided $25 Million (M) for research on breast cancer screening and diagnosis for military women and their family members. In 1993 grassroots efforts by the breast cancer survivors and advocacy communities led to a FY93 congressional appropriation of $210M for peer reviewed breast cancer research.

The CDMRP sought the advice of the National Academy of Sciences Institute of Medicine (IOM) on how to best invest the $210M congressional appropriation. While the IOM made several important recommendations, a pivotal one outlining a two-tier review process for evaluating and selecting research applications became a cornerstone within CDMRP. This novel two-tiered process involves reviewing every research application for scientific and technical merit as well as programmatic relevance.

The first tier of the two-tier review process is peer review. Peer review is a criterion-based process where applications are evaluated based on their scientific and technical merit. Applications are evaluated by scientific discipline, specialty area, or award mechanism by scientific, technical and consumer peer reviewers.

The second tier of review is programmatic review. Applications are programmatically reviewed by members of the programs’ Programmatic Panel. Programmatic review is a comparison-based process in which applications from multiple research areas compete in a common pool. Programmatic review balances the potential outcomes and risks of scientifically meritorious applications to select those that have the highest potential to help achieve the vision and goals of the respective program (https://cdmrp.army.mil/about/2tierRevProcess.

Application submission is a two-step process consisting of (1) a pre-application submission through the CDMRP-provided electronic receipt system (eBRAP), and (2) an application submission through Grants.gov (http://www.grants.gov/) for extramural organizations or through eBRAP (https://eBRAP.org) for intramural organizations. In addition to serving as a portal for pre-application and application receipt, other eBRAP functions include retrieving and processing applications from Grants.gov, conducting compliance review of pre-applications and applications, sending email notifications to users/subscribers, and serving as a portal to upload award and regulatory documents.

For more information on CDMRP, go to https://cdmrp.army.mil and review materials provided on the website.

The Government reserves the right to include scientific peer review of applications for other federal agencies. These efforts may be in conjunction with this Command’s funding opportunities or a separate task. The Contractor shall provide all necessary qualified personnel to provide for the management of scientific peer review for each of the selected scientific programs.

1.3 Period of Performance

The anticipated period of period of this contract is 60 months from date of award.

1.4 Contract Kick-Off Meeting

The Contractor shall contact the Contracting Officer’s Representative (COR) to set up a post-award conference with the COR, USAMRDC designated staff, Contracting Officer and Contract Specialist within 30 calendar days after the start date of the contract.

SECTION 2.0: GOVERNMENT-FURNISHED PROPERTY AND SERVICES

2.1 General Information

The contractor must provide all facilities, equipment and materials required to support this effort.

2.2 Government-Furnished Property

The government will provide the appropriate government furnished equipment (GFE) for those contractors who require network access. Determination of what GFE will be provided will be made at the task order level.

2.3 Government-Furnished Services

Determination of what government-furnished services will be provided to the contractor will be determined at the task order level.

2.4 Government-Mandated Training

All government mandated training (i.e., ethics, HIPAA, security, etc.) is required for all individuals based on the required access to either Fort Detrick or the network, including contractors as noted in Section 3.0 below. All government mandated training shall be government furnished and shall be completed at the contractor’s facility, during duty hours, and in accordance with established timelines.

SECTION 3.0: CONTRACTOR REQUIREMENTS

3.1 Contractor Personnel Minimum Requirements

3.1.1 The contractor must have a signed letter of commitment from the proposed key personnel in response to this solicitation and must be available to begin work at the start of the contract. Key personnel positions are the Project Director, Deputy Project Director, Scientific Review Manager, Consumer Reviewer Administrative Manager, eBRAP Lead, and the Meetings Manager.

3.1.2 For contractor personnel, including sub-contract employees, requiring access to Army installations, facilities or controlled area access areas must complete the following (note that if access is not required, then personnel are not required to complete):

3.1.2.1 All contractor employees, including sub-contract employees, shall complete AT Level I training within 30 calendar days after contract start date. The contractor shall submit certificates of completion for each affected contractor employee and sub-contractor employee to the COR and CDMRP Training Officer within 5 calendar days after completion of training by all employees and sub-contracting personnel. Training shall be updated on an annual basis and certificates shall be provided to the same POCs. AT Level I awareness training is available at https://jko.jten.mil/courses/atl1/launch.html.

3.1.2.2 All contractor employees, including sub-contract employees, shall comply with applicable installation, facility, and area commander installation and facility access and local security policies and procedures. The contractor shall also provide all information required for background checks to meet installation access requirements to be accomplished by the installation Security Office. The contractor workforce must comply with all personnel identity verification requirements as directed by DoD, HQDA, and local policy. In addition to the changes otherwise authorized by the changes clause of this contract, should the Force Protection Condition (FPCON) at any individual facility or installation change, the Government may require changes in contractor security matters or processes.

3.1.2.3 All contractor employees working IA/IT functions must comply with DoD and Army training requirements in DoD 8570.02, DoD 8570.01-M and AR 25-2.

3.1.3 All contractor employees, including sub-contract employees, requiring access to a government information system must complete the following (note that if access is not required, then personnel are not required to complete):

3.1.3.1 Contractor personnel shall be designated as ADP III at a minimum as defined by Department of Defense Regulation 5200.2R (for reference, review http://www.dtic.mil/whs/directives/corres/pdf/52002r_0187/p52002r.pdf). Each employee must also be a United States citizen. This is necessary so the contractor employees may be granted access to the Fort Detrick domain. The CDMRP will initiate the process with the appropriate personnel at Fort Detrick for the successful offeror’s employees to obtain a National Agency Clearance and Inquiries (NACI). If contractor personnel are unable to receive a favorable NACI and/or be a United States citizen, that individual may not be employed under this contract.

3.1.3.2 Contractor employees shall be registered in the Army Training Certification Tracking System (ATCTS) at commencement of services and must successfully complete the DoD Cyber Awareness Challenge training prior to access to the information system and then annually thereafter. The CDMRP Training Officer will register contractor employees and will be responsible for tracking and uploading all training certificates into ATCTS.

3.1.3.3. All Contractor employees and associated sub-Contractor employees with access to a Government information system must complete an Acceptable Use Policy (AUP). The CDMRP Training Officer will be responsible for uploading a copy of the AUP to the employee’s ATCTS account.

3.1.3.4. All Contractor employees and associated sub-Contractor employees with access to a Government information system must complete the initial Health Insurance Portability and Accountability Act (HIPAA) and Privacy Act Security Training before being granted access to the network. The Contractor and sub-Contractor employees shall then complete this training annually thereafter.

3.1.3.5 All contractor and sub-contractor employees with access to a government information system must complete Level I OPSEC training within 30 calendar days of employment under the contract and then annually thereafter (by the anniversary of their initial completion date). The contractor shall submit certificates of completion for each affected contractor and sub-contractor employee to the COR and CDMRP Training Coordinator within 5 calendar days of training completion.

3.1.3.6. Per DoD 8570.01-M, DFARS 252.239.7001, and AR 25-2, the Contractor employees supporting these functions shall be appropriately certified upon contract award. The baseline certification as stipulated in DoD 8570.01-M must be completed upon contract award. Personnel who are required to have privileged level access to perform their duties must have the IAT Level 1 commercial certifications listed below at time of hire. In order to meet IAT Level 1 requirements, the proposed personnel must have one of the following CompTIA CE certifications to meet the Baseline Certification requirements: A+, Network+, or Security+ certification. Personnel proposed must also enroll in the continuing education program for those commercial certifications required in the IA Baseline Certification. Proposed personnel are also required to have a Computing Environment (CE) commercial certification (list of certifications that fulfill this requirement is extensive but most commercial certifications suffice). Note that the same IA Baseline Certification cannot fulfill the CE certification requirement. Costs for these certifications shall be the responsibility of the Contractor. However, if Government regulations change during the period of performance of this contract, the Government will be financially responsible for the costs for the Contractors’ personnel to comply with the new regulations. Personnel who require privileged level access must also complete a Privileged Level Access agreement with copy provided to the CDMRP Training Manager for upload to the employee’s ATCTS account. Personnel who require privileged level access will also require Appointment Orders drafted by the CDMRP Information Assurance Manager (IAM) and signed by the CDMRP Director with a copy provided to the CDMRP Training Manager for upload to the employee’s ATCTS account.

SECTION 4.0: PERFORMANCE OBJECTIVES

The CDMRP requests contractor personnel to provide biomedical research application evaluation and support by performing the following tasks. In addition, the Contractor shall provide a secure contractor-owned, contractor-operated web-based electronic system which includes hosting of such system. The system shall support activities and capabilities specific to peer review, including but not limited to: document viewing, facilitation of various meeting formats including virtual meetings, individual reviewer and panel scoring and written evaluations, and conducting concurrent review in multiple meetings and/or panels.

NOTE: Pre-applications and applications are submitted to a Program Announcement (PA); pre-applications/pre-proposals and applications/proposals are submitted to a Broad Agency Announcement (BAA).

4.1 Administrative Requirements

General support for science administration and management includes support and tasks that are applicable to all CDMRP programs. Program-specific support shall be supported under individual task orders, including but not limited to scientific review management, project management, and consumer reviewer administration.

4.2 Conduct Compliance Review

Compliance shall be conducted on pre- and full applications submitted to a PA and pre- and full applications/proposals submitted to a program-specific BAA. Compliance review shall consist of an eBRAP automated process for issues common across all FOAs (e.g. project narrative page limit) as well as a manual process for other issues that are FOA-specific.

4.2.1 In accordance with the Compliance Standard Operating Procedure developed in consultation with CDMRP, the General Application Instructions (GAI) and PA or General Submission Instructions and BAA (as applicable) shall be used as the authoritative document for conducting Administrative Action/Compliance Review of pre-applications/proposals and full applications/proposals submitted in response to the funding opportunities.

4.2.2 Original pdf files submitted via government electronic processes (Grants.gov and eBRAP) shall be administratively reviewed for inclusion of appropriate components as identified in each respective FOA

4.2.3 Provide a compliance specification sheet/checklist in accordance with each respective PA or program-specific BAA within 10 business days after FOA release.

4.2.4 Using eBRAP, provide listings by research program of pre-applications/pre-proposals with administrative compliance issues and with administrative modifications based on compliance guidance in each respective FOA within the following deadlines: 1) For issues undergoing automated compliance review, provide listing within 24 hours following pre-application/pre-proposal receipt in eBRAP; and 2) For other issues undergoing manual compliance review, provide listing no later than two (2) business days if 250 pre-applications/pre-proposals or less were received for a specific FOA and no later than four (4) business days if greater than 250 pre-applications/pre-proposals were received for a specific FOA.

4.2.5 Using eBRAP, provide listings of full applications/proposals with administrative compliance issues and applications/proposals with administrative modifications based on compliance guidance in each respective FOA within the following deadlines: 1) For issues undergoing automated compliance review, provide listing by research program of full applications/proposals with administrative compliance issues determined via eBRAP’s automated compliance process no later than two (2) business days following the verification deadline specified in the FOA; 2) For issues undergoing manual compliance review, provide a listing of applications/proposals with compliance issues and administrative modifications no later than the following number of days after the verification deadline in eBRAP: a) within three (3) business days for less than 100 applications/proposals received under a specific FOA; b) within five (5) business days for 101 to 250 applications/proposals received under a specific FOA; and c) no later than seven (7) business days for >250 applications/proposals received under a specific FOA.

4.2.6 Provide a report by research program and FOA of pre-applications/pre-proposals and applications/proposals that contain names of Programmatic Panel members (in sections other than “References Cited”) to be delivered no later than one (1) business day after compliance review is completed.

4.2.7 Provide a report of all Principal Investigators (PIs) by their respective eBRAP-generated application/proposal log numbers, including application/proposal title, for PIs that have submitted multiple applications/proposals to the same program and have submitted applications/proposals to multiple programs including the USAMRDC and program-specific BAA (include two (2) years) within the same fiscal year for which receipt of applications/proposals has occurred. This report shall be provided on the same day that application compliance is complete.

4.2.8 Using eBRAP, generate notifications to PIs and business officials/authorized organizational representatives informing them of their pre-application/pre-proposal or applications/proposals non-compliance status and the reasons for its non-compliance.

4.2.8.1 Prepare a draft of the annual template notification letter, in accordance with Army Regulations, for non-compliant notification and deliver to the eBRAP task order COR within 20 business days following release of the first fiscal year FOA. Within five (5) business days after receipt of COR input, edit and forward letter to the COR.

4.2.8.2 Using eBRAP, inform the appropriate CDMRP Program Manager or POC that non-compliant letters are available for review.

4.2.8.3 Ensure non-compliant letters have been automatically posted in eBRAP following concurrence from the approval authority.

4.3 Reporting Requirements

4.3.1 Provide ad hoc reports documenting peer review data as requested.

4.3.2 Perform data analyses and ad hoc reporting to support the CDMRP’s management activities such as the Inquiry Review Process.

4.3.3 Prepare briefing materials and reports on peer review processes, procedures, and outcomes as requested.

4.3.4 Provide information for the CDMRP Internal Control process and Congressional or other inquiries specific to peer review as requested.

4.3.6 Other reporting requirements shall be added and negotiated by task order to include a Cost to Complete (CTC) which will be due 60 days before the task order expires.

4.3.7 Provide a quarterly report consisting of a summary of program deliverables, an assessment of processes and procedures in place, and process improvements identified, as applicable. Any recommendations affecting the performance work statement will be accomplished through the execution of a contract modification.

4.4 Technical Requirements

4.4.1 Provide a secure contractor-owned, contractor-operated and hosted web-based electronic system that supports activities and capabilities specific to peer review, including but not limited to: document viewing, facilitation of various meeting formats including virtual meetings, individual reviewer and panel scoring and written evaluations, and conducting concurrent review in multiple meetings and/or panels. The contractor shall be responsible for the maintenance, operation, and hosting of the system. Such a system will have accessibility for the CDMRP Program Managers and Government observers/representatives. At minimum, the system must incorporate the following capabilities:

· Exchange of data via Web Services with the Government’s backend business system, Electronic Grants System (EGS) or other systems as needed and abiding by an interconnection agreement

· Display meeting documents (e.g. agenda, forms, briefings, booklets, graphs, documents, etc.)

· Display application review documents (e.g. application, budget, abstracts, statement of work, etc.)

· Display application information (e.g. title, org, PI, budget, assigned reviewers)

· Block viewing of specific applications by reviewers who have a conflict of interest (COI)

· Capture COIs for each reviewer

· Capture pre-meeting comments/votes/scores

· Capture meeting comments/votes/scores

· Share panel comments with other attendees

· Capture meeting notes from multiple sources

· Real time viewing, sorting, and ordering of interim and final scores

· Access for CDMRP Program Manager and other designated individuals to input notes and view progress, reports, notes, and documents

4.5 General Program Assistance

4.5.1. Attend and participate, as requested by the COR, in program Integrated Program Team (IPT) meetings and activities, assist with setting program milestones, provide input on the content for fiscal year PAs (one PA per award mechanism), program-specific BAA’s, and the USAMRDC BAA (one per fiscal year), and participate in other pertinent business as requested (note this is typically for CDMRP; other customers shall specifically request attendance at their standard meetings at the task order level).

4.5.2. Provide scientific and technical peer review advice, plans, recommendations, and innovative approaches when defining business process improvements.

4.5.3. Provide support for pre-application/pre-proposal screening as requested. Tasks shall include recruiting reviewers with appropriate expertise, conducting an online individual pre-application review, and providing compiled scores and critiques. Details and deliverables will be specified within the program-specific task order(s).

4.5.4 Prepare and Plan for Peer Review of Applications/Proposals

4.5.4.1. For each task order, within 10 business days after FOA release, prepare a Deliverables/Receivables Tracking Sheet (D/RTS) outlining every deliverable/receivable and due date.

4.5.4.2. Provide review specification sheets (RSS) according to each FOA for review 10 days prior to pre-application receipt, including but not limited to peer review panels (PRP) details, reviewer assignments, composition of the summary statements, and peer reviewed application files to be delivered.

4.5.4.3. Establish, manage, and enforce policies, procedures, and risk mitigation plans for Conflicts of Interest (COI) and the protection of confidential and proprietary information.

4.5.4.4. Establish process(es) to ensure reviewer’s understanding and agreement to adhere to the offeror’s policies on COI and to maintain the integrity of confidential and proprietary information. Such agreement includes disclosure of any COIs, whether real or apparent. Participation in foreign talent programs or other sponsorships should be included in such disclosures. Establish management approaches on enforcing policies, including reviewer confidentiality breaches.

4.5.4.5. The offeror is expected to employ state-of-the-art strategies and/or technologies and best practices for preventing unauthorized disclosure of confidential and proprietary information.

4.5.4.6. Develop a mitigation plan for handling COIs in the review process.

4.5.5. Conduct Peer Review of Applications/Proposals

It is anticipated that there shall be four (4) possible modalities for conducting scientific peer review: 1) onsite, in person peer review panels (PRPs); 2) video/telephonic PRPs; 3) online-only PRPs; and 4) online individual peer reviews (formerly called “mail” reviews). The peer review modality shall be decided upon at the issuance of the task order for each requirement. Each peer review modality shall be executed using the web-based contractor-owned, contractor-operated electronic system. It is expected that the contractor will utilize best practices and state-of-the-art technology to execute each peer review modality as applicable. Additional 3rd party applications that may be used to augment meetings whether onsite on virtually must meet security standards to ensure protection of procurement sensitive and government data.

Typically a PRP consists of a Scientific Review Officer (SRO) or equivalent administrator, a chairperson with appropriate experience with peer review and scientific discipline related to the PRP topic, scientist/clinician reviewers, and consumer reviewers. Other specialist reviewers and PRP configurations may be considered at the request of the task order POC or at the recommendation of the contractor.

The specific roles and responsibilities of the Scientific Review Manager, SRO, chairperson, and any other contractor staff in coordinating the PRPs shall be described in detail by the offeror/contractor.

The offeror is expected to employ state-of-the-art strategies and/or technologies and best practices for identifying and recruiting appropriate subject matter experts to serve as PRP chairpersons and scientist/clinician reviewers.

Deliver a preliminary report of recruited reviewers, chairpersons, and assigned SROs (by program and PRP) within 10 business days of application/proposal receipt, and deliver a final report 15 business days prior to the peer review meeting.

4.5.5.1. Recruit a chairperson for each PRP. Chairpersons shall have appropriate experience with peer review, qualifications commensurate with leading the panel discussions, and expertise in the specific discipline related to the PRP topic. As requested, provide the Chairperson candidates’ names and biosketches to the task order POC for review and comment.

4.5.5.2. Recruit PRP scientist/clinician and specialist reviewers. Conduct recruitment initiatives of scientist/clinician reviewers for each research program. Scientist/clinician reviewers shall possess appropriate education, training and experience relevant to task order requirements, applications assigned for review, and to the intent of the award mechanism. The majority of scientist/clinician reviewers shall have recent experience as a PI, Co-PI or manager of an extramurally funded research program or effort. Some program-specific task orders will require reviewers with experience in military medical research, the military health system, military treatment facilities and/or deployment related medical care. The contract company may recruit individuals serving as active duty military or civilian federal employees with the requisite expertise to serve as peer reviewers for compensation. This may be done as long as the individual recruited is paid in their capacity as a contract employee or sub-contractor, and not in their official Government capacity, and the company or individual provides a written ethics opinion addressing any financial conflicts of interest and representation restrictions related to the transaction. When an active duty military or civilian federal employee is identified as PRP member, the contractor shall notify the individual that they are to take leave in order to participate on the PRP. The contractor shall document that notifications were provided in the Recruited Reviewers Report I.

4.5.5.3. Up-to-date biosketches, including past, current, and pending support, of recruited reviewers shall be made available to the Government as requested.

4.5.5.4. Develop and maintain a Consumer Nomination process. Provide consumer nomination materials to the COR for review annually. Coordinate as needed with the CDMRP website COR to enhance advertising for consumer reviewer nominations.

4.5.5.5. Recruit consumer reviewers. Develop and/or update a Consumer Outreach Plan for each program as requested in the program-specific task order and after consultation with the CDMRP Program Manager. Provide the plan no later than 20 business days following Vision Setting or Strategic Planning Meeting for each respective program as applicable. Maintain contacts with a broad representation of active consumer advocacy/support organizations. Prepare and distribute information to recruit consumer reviewers. Solicit nominations from consumer/advocacy support groups for upcoming peer reviews, evaluate nominations, and select consumer reviewers according to CDMRP guidance.

4.5.5.6. Devise and implement strategies for recruiting consumer reviewers representative of the community affected by the disease/condition, including representation of disproportionally minority populations/groups, military veterans, and affected populations.

4.5.5.7. Provide updates to the task order POC during the consumer recruitment period on consumer recruitment accomplishments, barriers to recruitment, etc. on a monthly basis or as needed.

4.5.5.8. Assign and provide application review materials to the recruited reviewers, allowing an appropriate amount of time to conduct a thorough review while meeting COR-approved timelines as agreed upon in the deliverables/receivables tracking sheet (D/RTS).

4.5.5.9. Conduct in person PRP meetings. It is anticipated that PRP meetings shall be held after receipt of applications/proposals from the CDMRP and in accordance with timelines in the D/RTS. Variance from this timeframe shall be agreed to by the COR and contractor. Once the number of PRPs have been determined, final approval must be given by the COR.

4.5.5.10. Conduct video/telephonic PRP meetings. It is anticipated that there shall be two (2) to three (3) scientist/technical reviewers and one (1) consumer reviewer per application (any deviation must be discussed with the appropriate task order POC prior to final assignments). The contractor shall make all logistical arrangements necessary for the video/telephonic review and meetings. Deliverable timeline shall be included in the D/RTS and shall be negotiated before the review commences.

4.5.5.11. Conduct online-only PRP. It is anticipated that there shall be two (2) to three (3) scientist/technical reviewers and one (1) consumer reviewer per application as applicable (any deviation must be discussed with the appropriate task order POC prior to final assignments). Online-only PRPs will not be convened in a meeting for panel discussions. Following initial scoring, an online panel discussion among peer reviewers and the Chairperson shall be moderated by the PRP’s Scientific Review Officer, as needed. The online discussion may be conducted to clarify score disparities and to provide a second iteration as needed before finalizing scores. The contractor shall make all logistical arrangements necessary for the online review and discussions. Deliverable timelines shall be included in the D/RTS and shall be negotiated before the review commences.

4.5.5.12. Conduct online individual application/proposal peer reviews. It is anticipated that there shall be two (2) to five (5) scientist/technical reviewers and one (1) consumer reviewer per application as applicable. Their reviews shall be conducted individually and not discussed among each other as a PRP. Final reviewer assignment shall be discussed with the task order POC prior to assignment.

4.5.5.13. In general, no more than 60 applications shall be reviewed by any PRP without prior written approval of the COR; however, any deviations from the standard will be specified in each program task order PWS. The minimum number of applications per PRP shall be coordinated with the appropriate CDMRP Program Manager. In general, each scientist/clinician PRP member shall be responsible for reviewing eight (8) to ten (10) applications (with a per-PRP member average of seven (7) or greater); any exceptions must have prior approval of the COR. Each consumer reviewer shall be responsible for reviewing no more than 20 applications, with their reviews focused primarily on specific application components; any exceptions must have prior approval of the COR. However specific requirements for reviewer assignments, shall be detailed in each program specific task order.

4.5.5.14. Ensure each PRP maintains a quorum. A quorum shall consist of at least 80% of all PRP members (including COIs but exclusive of SROs and other administrative staff) for each PRP in order for discussions to proceed without prior approval. For example, if the PRP consists of 10 members, no more than two (2) members may be out for any reason, including COIs, travel reasons, or other brief absences from the PRP without approval of the COR or respective Task Order POC. Ad hoc reviewers are considered PRP members only for the applications for which they are participating.

4.5.5.15. Furnish information summary sheet(s) of PRP member degrees, expertise, Department/Institution, PRP assignment, and USAMRDC peer review history.

4.5.5.16. Coordinate reviews of application/proposal resubmissions, as requested in program-specific task orders. Resubmission reviews may be conducted in multiple modalities, but shall require the same reviewers who provided a technical review of the original application/proposal. A D/RTS specific to the resubmission review shall be developed. Format of the resubmission review deliverables shall be consistent with established deliverable formats.

4.5.5.17. If requested in the program-specific task orders, provide a report of PRP member demographics, including but not limited to gender, ethnic diversity, academic rank, military rank, active duty status, military dependent, Veteran status, education/degrees, affiliation with Department of Defense and/or Department of Veterans Affairs, and institutional association in aggregate and by PRP within the program D/RTS.

4.5.5.18. Provide a peer review debriefing report following each peer review or as specified in the program-specific task order. The report shall consist of a summary of all comments made by reviewers during peer review debriefings.

4.5.6. Evaluate Scientific Research Applications

4.5.6.1. Perform a technical review of all applications providing evaluative critiques and utilizing the scoring process (e.g., overall and individual criteria scores) established for each funding opportunity according to the appropriate task order. Standard deviations shall be required on each application’s overall score. The offeror must describe quality control processes and/or strategies that will be employed to ensure that reviewer critiques are free of factual errors. The offeror must also describe best practices that will be utilized to ensure that reviewer critiques are consistent with the criteria scores and overall score, and that critiques are consistent within each distinct evaluation criterion (i.e. avoid criteria bleeding).

4.5.6.2. Provide an expedited (triage) review report within 2 business days prior to the peer review meeting. Coordinate with the CDMRP Program Manager on the final expedited (triage) review parameters.

4.5.6.3. Provide finalized scoring reports in both Microsoft Excel and PDF within one (1) business day of the peer review meeting completion.

4.5.6.4. Provide reports comparing pre-meeting and final (post-meeting) scores per application, assigned reviewer, criteria, and PRP within one (1) business day of the peer review meeting, as requested by program-specific task order.

4.5.6.5. Provide score distribution reports by panel and by award mechanism within two (2) business days after delivery of final score report, as requested by program specific task order.

4.5.6.6. Provide a checklist with administrative notes within five (5) business days after the peer review meeting.

4.5.6.7. Provide a report with the names, degrees, and organizations of all reviewers per program per fiscal year for web posting within 30 business days after the last peer review meeting for a program-specific task order.

4.5.6.8. If the Government determines that the peer review resulted in factual errors in the summary statement or procedural errors in the process at the fault of the contractor, an application re-review shall be conducted at the expense of the contractor. Each application shall be re-reviewed by a PRP of scientist/technical and consumer reviewers, with appropriate administrative support, within 30 business days (unless a different timeline is approved by the COR) of the application re-review decision. The decision regarding the modality of the re-review shall be determined by the CDMRP Program Manager. A D/RTS specific to each re-review shall be developed. Format of the re-review deliverables shall be consistent with established deliverable formats.

4.6. Deliver Summary Statements

4.6.1. Provide a summary statement for each peer reviewed research application to document the peer review findings. Provide identifying information for the application, all scoring data and statistics, budget data, any peer review recommendations, and a written critique (summarizing strengths and weaknesses as established by the PRP) and additional files as requested (e.g., abstracts, statements) for each application summary.

4.6.2. Furnish the draft summary statements as requested per task order and as they become available in the appropriate part in the summary statement preparation process. The COR reserves the right to request rewrites based on this review. Percentage of draft summary statements requested will depend on the program, funding opportunity, scoring distribution, and number of applications.

4.6.3. Deliver summary statements in batches (as identified by the respective CDMRP Program Manager) per each D/RTS. Deliver the first batch of summary statements no later than 15 business days and deliver the second batch of summary statements no later than five (5) business days prior to the appropriate programmatic review meeting.

4.6.4. Data and File Deliverables

Provide all data and files associated with pre-applications, full original and peer reviewed applications/proposals and results of peer reviews via secure web services meeting government established technical specifications. Data and files shall be transferred to CDMRP electronic business management systems, primarily the EGS or other systems as needed to meet defined timelines for each program which will be established and agreed to annually within a D/RTS. The contractor shall employ quality control procedures to ensure that all data and files delivered into the Government systems meets the highest level of data integrity, consistency and accuracy.

Data and files include:

· Administrative notes transferred to EGS

· Original pre-application data and files

· Original full application data and files

· Full application budget report

· Compliant version of pre-application data

· Compliant version of pre-application files

· Compliant version of full application files (Peer Reviewed Application Files I)

· Compliant version of full application data

· Compliant version of full application files (Peer Reviewed Application Files II)

· Application Score Data

4.6.4.1. The contractor will be expected to abide by a system interconnection agreement for exchange of data/files between the contractor owned system and EGS (or other Gov. application/resource as needed). Agreement will specify details to include security considerations, documentation, technical controls and standards, configuration control, communication plans, and roles and responsibilities.

4.6.4.2. The contractor must agree to a data use policy which will define and limit how application/proposal data/files may be used and how long application/proposal data will be retained in the contractor-owned and contractor-operated system.

4.6.4.3. The contractor shall establish quality control processes to ensure that data/files delivered to the government meet standards of accuracy, integrity and consistency. Issues found with data after delivery must be corrected and redelivered.

4.7 Training

4.7.1. Furnish a coordinated training and orientation plan for all reviewer participants at the beginning of each fiscal year. The training and orientation plan shall incorporate separate sub-plans for training and orienting new scientist and technical reviewers, ad-hoc reviewers and consumer reviewers within a master timeline for accomplishing the proposed training. Each sub-plan shall identify learning objectives outlined in measurable, behavioral terms, including process or intermediate demonstrations of terminal objective achievement and outcome performance evaluation tools/audits that correspond to the learning objectives. Understanding of the contractor’s policies for COIs and protection of confidential and proprietary information shall also be included. Each plan shall furnish proposed timelines for each audience, including educational materials, using portable, alternative and web-based media to the extent possible, and incorporating targeted levels of learning appropriate to each audience. Provide periodic updates to training materials specific to the program to ensure materials are kept up-to-date. The training and orientation programs shall be approved by the COR.

4.7.2. Schedule, coordinate and provide an appropriate number of program-specific webinar orientations for 1) Scientific Review Officers; and 2) all reviewers (for all types of review processes). Orientations shall be developed in consultation with appropriate CDMRP personnel. Provide orientation materials to appropriate CDMRP personnel no later than 10 business days before the scheduled orientation. Additional teleconferences after distribution of review materials may be held as appropriate.

4.7.3. Design and implement consumer orientation sessions and provide support to consumer reviewers pre-meeting, on-site/in real-time and post meeting (as appropriate). Attend and participate in a dry run meeting as requested.

4.7.4. Iterative Peer Review

For applications/proposals to the USAMRDC intramural solicitations, the Government may request the contractor conduct peer review via multiple, iterative steps. It is anticipated that applications/proposals shall be received through the Government’s eBRAP system and abide by the peer review criteria in the solicitations. Iterative peer reviews shall be negotiated as a separate task order and are anticipated to be required for no more than 500 intramural applications/proposals each fiscal year.

If the iterative peer review approach is expanded to extramural applications, it is anticipated to be limited in scope and number of funding opportunities. The Government shall negotiate iterative peer reviews in program-specific task orders.

4.7.5. Participate in Working Groups

4.7.5.1. Participate in CDMRP program evaluation activities and sub-committee activities as requested. Some large efforts required for these sub-committee activities shall be negotiated as a separate task order. Provide science management advice on program evaluation issues as requested (such as product identification, evaluation projects, and portfolio analysis).

4.7.5.2. Participate in other CDMRP working groups as requested. Provide science management advice as requested.

4.7.6. Attend Conferences/Meetings

4.7.6.1. Provide a list of the conferences and meetings the contractor staff desires to attend, not including CDMRP business meetings. This request shall include a justification for attendance at each conference/meeting per attendee. The list shall be delivered 60-90 calendar days prior to the conference/meeting date for COR approval. Any changes to the list shall be approved by the COR (e.g., additions at a later date).

4.7.6.2. Conference and meeting attendance shall be for reviewer recruitment purposes only. Provide a summary of the recruitment results, to include number of contacts made and number of reviewers added to the recruitment list, within 20 business days of the conference or meeting conclusion.

4.8 Logistical Requirements for Onsite, In-Person Meetings

4.8.1 Secure hotel arrangements and reservations that satisfy the requirements of the onsite PRP’s. Inform the respective CDMRP program manager of planned and finalized venue when making the meeting arrangements within three business days of final venue selection.

4.8.1.1 Reserve appropriate number of meeting rooms in support of the meeting and supply any necessary audio/visual/IT support and equipment needed to conduct the meeting.

4.8.2 Arrange travel reservations for all non DOD/government peer review participants.

4.8.3 Coordinate meals for all peer review attendees, including government representatives. Peer review meeting attendees shall pay for all meals provided in accordance with the Joint Travel Regulations.

4.8.4 Provide all logistical, administrative, and information technology support during all peer review meetings. Provide contingency/backup plans for unexpected disruption.

4.8.5 Coordinate with appropriate USAMRDC personnel to ensure meeting locations meet the current Force Protection Requirements. These requirements include, but are not limited to:

4.8.5.1 Each meeting room/ auditorium/ etc. must have doors lockable from inside the room;

4.8.5.2 Each meeting room must have more than one exit;

4.8.5.3 Location hosts must agree to not advertise the meeting’s association with the U.S. Government in any manner, e.g. names on bulletin boards, signs posted in lobbies or conference room doors, internet sites, etc.

4.8.5.4 Facility must have organic or dedicated parking areas;

4.8.5.5 If the event will start or end during the hours of darkness or low-level daylight, the parking areas and avenues to and from the facility must be well-lit;

4.8.5.6 Each meeting room must have a working land-line telephone;

4.8.5.7 Each meeting room must have a serviceable fire extinguisher.

4.8.6 The following are preferred/ desired attributes for meeting venues:

4.8.6.1 Dedicated security force;

4.8.6.2 Access-controlled parking areas;

4.8.6.3 Access control (human, electronic, or key) at entrances to facility;

4.8.6.4 Geographical separation from highly-trafficked pedestrian areas;

4.8.6.5 Located in near vicinity of at least a Level III Trauma Center;

4.8.6.6 Closed-circuit Television (recorded) security system.

4.9 Operate, Maintain, and Enhance eBRAP (https://eBRAP.org)

The electronic Biomedical Research Application Portal (eBRAP) is a custom developed system, which allows for application submission and processing, document collection, and information dissemination. It serves as the front-end interface for the research community with the CDMRP, USAMRDC and other DoD organizations business processes. It is a government-owned, contractor-operated system with all appropriate approvals and waivers to exist outside of the military network (.mil).

eBRAP’s main areas of support focus on submission of pre-applications, direct submission of intramural DOD proposals, interface with Grants.gov for retrieval, and processing of extramural/intramural applications and administrative review for compliance. It is also the system for submission of award management information and files (e.g. invoices, regulatory documents, technical reports, quad charts) over the lifetime of research awards; eBRAP also interfaces with the back-end business system, EGS (a CDMRP/USAMRDC government-owned, contractor-operated system).

The Government will oversee all aspects of eBRAP operations and development. Specific POCs defined below may include COR, Task Order COR, Program Manager, Business Systems Manager or other Government personnel, which shall be defined in SOPs based on activity.

4.9.1 Project Management

4.9.1.1 Provide project management and application administration services, overseeing development, enhancements, operations, and technical support activities including the following: Project planning, timeline and milestones development, generation/maintenance of project documentation including roadmap, deliverables, incident reports, project plans, performance metrics (e.g., time to resolution etc.) architectural and technical documents, standard operating procedures, user guides for front and backend functionality etc. as deemed necessary to ensure transparency, compliance and requirements of the Government.

4.9.1.2 Oversee daily operations, development, maintenance, and support activities in coordination with Government POCs.

4.9.1.3 Document issues and incidences, track progress towards approved milestones, and provide regularly scheduled updates and reports with agreed upon metrics (i.e. time to resolution, number of issues, tickets). Status report with metrics and roadmap shall be provided 3 business days prior to monthly technical status meeting with government.

4.9.1.4 Ensure quality control for all aspects of project execution, application development, system configurations, and data management based on industry and Government standards and expectations specified by the government.

4.9.1.5 Attend regularly scheduled and ad hoc meetings with key stakeholders (i.e, Configuration Control Boards, Integrated Project Team, and Sprint Planning) as requested to support CDMRP and other eBRAP government points of contact. Provide briefings, system demonstrations, and documentation as requested.

4.9.2 Program Support

4.9.2.1 Utilize established SOPs and timelines to configure eBRAP for each FOA for draft review and testing, modify as necessary, and complete final posting upon approval by Government POCs. Maintain master eBRAP FOA set up document of all configuration parameters (i.e.