Additional Scope of Work Documentation v2 Final.pdf

Department of Homeland Security (DHS)

Peer and Merit Review Services

Scope of Work

1.1 Overview

The Department of Homeland Security (DHS) Science and Technology (S&T), Office of

University Programs (OUP) requires an experienced Contractor support for its well-established processes for managing and conducting subject matter experts (SMEs) peer and merit reviews.

The Contractor must:

• Possess a substantial (e.g., several thousand names and vitas) existing database of relevant SMEs from a variety of scientific and engineering disciplines applicable to homeland security-related research.

• Be experienced and skilled in organizing, supporting, and implementing on-site (in-person), off-site (web-based), and mixed (in-person and web-based) peer and merit reviews of S&T financial assistance proposals and programs using the above-referenced

SMEs.

• Be experienced and skilled in organizing and facilitating multiple types of merit reviews including peer reviews, ad-hoc reviews, focus groups, workshops, large meetings and study groups of SMEs and professional practitioners using existing web-based interaction tools.

• Have experience providing DHS S&T or similar large scientific organizations with administrative, management, and logistics support for the facilitation of peer and merit review events to include support for reviewer travel coordination, arrangement, reimbursement, and reviewer service fee payment. NOTE: The Contractor must have an established system to facilitate reviewer payment that has existing protections to safeguard Personally Identifiable Information and Sensitive Personally Identifiable

Information. The Contractor will hold full responsibility for safeguarding and maintaining this information.

• Have experience providing organization and facilitation of a multi-day, multi-session annual program review of university research and education programs with federal, state, local, and private sector stakeholders participating.

• Have experience planning, scheduling, facilitating, and documenting the results from workshops and meetings for scientific members and experts of the federal, academic, and non-profit and industry communities.

The DHS S&T OUP mission relies on independent and objective expert reviews to ensure applicants to OUP’s solicitations have the capability to produce high quality scientific and engineering knowledge and products that are relevant to DHS’s mission.

The Contractor will support multiple types of peer and merit review activities. The primary aims are as follows:

1) ensure DHS S&T awards the most qualified proposals for grants and cooperative agreements;

2) ensure DHS supports research in the highest priority DHS mission areas, and

3) enable DHS S&T to assess performance on a regular basis.

OUP will be a primary customer for this RFI. However, the vehicle will also be available throughout DHS S&T to conduct similar peer and merit review activities where needed. OUP requires SME merit and peer review services and administrative, management, and logistics support to facilitate the following types of activities:

• Center of Excellence (COE) Selection/Re-competition Peer Reviews, Site Reviews, and

Mid-term Reviews

• Minority-Serving Institutions (MSI) Peer Reviews

• Education Program Peer Reviews

• Annual Program Review Summit Topic grant competitions

• Periodic (biennial or triennial) reviews of all OUP programs, goals, and accomplishments

• Ad-hoc research workshops, research needs assessments, and reviews on specific cross-cutting subjects

DHS estimates that, per year, the Contractor will arrange and facilitate for OUP up to:

• Eight (8) peer/merit review activities (5-35 participants per review) in support of the

Office of University Programs’ COE, Education and MSI programs.

• Three (3) research-related workshops/meetings in support of the COE, Education, and

MSI programs (30-200 participants).

Workshops

OUP conducts focused workshops in discrete areas to identify highest priority research in existing and emerging areas. Workshop topics are based on customer interests, the necessity to understand the research and development taking place in a topic area or the need to ensure originality and avoid duplication. Workshops and meetings may be ad-hoc, may consist of student scholarship and fellowship recipients, and may be in or out of the Washington, D.C.

metropolitan area.

Reviews

The Contractor’s support and management of a comprehensive review process is necessary to ensure confidential and fair evaluation of proposals without reviewer bias or conflict-of-interest.

Conducting merit reviews with maximum process integrity is essential to ensure that awards are made only to the best research projects/institutions, and that funding decisions will stand up to any scrutiny and challenges. DHS S&T depends on peer review involvement from the scientific community. The Contractor must be able to identify conflicts-of-interest and to elicit reviewers’ acknowledgment of such potential biases when they exist. The Contractor must also have experience with the necessary processes to maintain confidentiality of the proceedings and results both during and after a review process, and to provide the fairest and most competent peer review possible.

The Contractor must be able to ensure that SME peer and merit reviews enable independent experts from a number of fields to perform an in-depth assessment of proposed projects prior to funding. To facilitate this, the Contractor must have existing access to a wide range of SMEs in each of the many fields in which DHS S&T is involved. A significantly broad selection of experts is needed to assure the reviews will provide objective information from technical experts not affiliated with the applicants, i.e., who do not have a conflict-of-interest. Final decisions related to awarding or extending of assistance agreements or taking DHS courses of action as result of the review services under this contract will be done by DHS federal employees.

The Contractor must have experience managing successful peer and merit reviews to validate the scientific merit of the proposed research tasks. In accordance with the Federal Advisory

Committee Act (FACA) (5 U.S.C. Appendix 2; 41 C.F.R. § 102-3 et seq.) the contractor shall not have the reviewers develop consensus scores or results. The Contractor must provide objective evidence that supports the federal program office decisions on activities that were or were not selected for funding. Finally, the Contractor must be experienced in facilitating the generation of insights or unique questions from panels of independent SMEs that may not have otherwise been considered prior to the distribution of funds.

The ability of the Contractor to ensure the integrity of the process and provide high-quality results will be a key criterion in this contract. All aspects of the expenditure of taxpayer funds on

DHS S&T funded research or education projects are open to public scrutiny. The review and selection processes may have high visibility and widespread interest and must be unimpeachable.

Collectively, these activities ensure OUP is supporting the right research, selecting the best performers, facilitating communication among scientists, DHS Components and Homeland

Security Enterprise partners, connecting results with end-users, and assessing performance on a regular basis. The COEs are centered on topic-specific research areas that span multiple disciplines and performers. The Annual Program Review Summit is an opportunity to present and review the full body of research conducted through COEs. The event brings together academic, federal, and private sector individuals to discuss discoveries to date, enable transition to use and explore future research directions.

Review Service Fee Payment Support

Provide review support to OUP in the area of service fee payment on a Cost-Reimbursable basis.

Upon finalization of each review the contractor will issue a service fee payment to each reviewer for services rendered. The fee amount and not to exceed limit will be established by DHS S&T

OUP.

Direct Support includes:

• Disseminate and collect all necessary paperwork for reviewer payment including a full description of the service requirements by the reviewer to receive payment and instructions for requesting and receiving payment at the conclusion of the review.

• Generate and deliver payments to reviewer in the form of a check or ACH payment, whichever is most cost advantageous.

Review Travel Support

Provide review support to OUP in the area of reviewer travel coordination, arrangement and reviewer travel expense reimbursement on a Cost- Reimbursable basis. Reviewer travel preparation will be completed in accordance with the Federal Travel Regulations (FTR).

Direct Support includes:

• Prepare travel for all non-local reviewers as needed based on the FTR.

• Non-local is defined as travel outside of the 50-mile radius of the review location.

• Provide reimbursement to reviewers for actual expenses incurred during travel based on the FTR.

Once the reviewer agrees to participate the performer should establish an agreement with the reviewer outlining the guidelines and obligations and confirming there is no Certification of

Insurance (COI) to ensure expectations are maintained for all parties. Once a review is complete and the DHT S&T OUP Program Manager and Contracting Office Representative (COR) approve the work of reviewer, the COR will authorize the performer to pay the reviewer. This will be a cost reimbursable expense supplemented with monthly reporting that includes at minimum the review name, reviewer’s email, and payment amount. The contractor will maintain sole responsibility for all security and privacy requirements to safeguard and maintain required information. DHS S&T will not have access to or maintain any data regarding the reviewer payment process.

NOTE: Reviewers may choose to prepare their own travel arrangements within the FTR guidelines. The Contractor, with approval from DHS S&T, shall establish a not-to- exceed agreement per reviewer. The agreement at minimum will include line-item estimates for lodging and transportation based on the GSA per diem guidelines and city pair fares as a baseline. The

Contractor will provide reimbursement to reviewers for actual expenses incurred during travel based on the FTR and the agreement. DHS S&T may choose to utilize a flat fee option that includes a standard reimbursement for travel and the service fee.

1.2 Center of Excellence Selection Review Process

For COE reviews, DHS receives proposals in response to Funding Opportunity Announcements

(FOA) posted on the federal government’s grant solicitation website www.grants.gov . DHS

S&T uses a three-phase selection review process to review these proposals and ultimately select institutions to become COE. This review process is described in the following paragraphs.

Phase 1: Scientific Quality Review (External Review): A panel of external SMEs from academia, non-profit research foundations, industry, and federal, state, and local agencies conducts this review. The panelists must have expertise and/or experience in the academic disciplinary areas relevant to the FOA topic (e.g., explosives detection and mitigation, border security, etc.). The panelists provide individual ratings (using electronic means) on selected review criteria and overall ratings (i.e., across reviewers) using a variety of central tendency or scoring rules, (e.g., mean, median, mode, “skating” or “diving” scores). Each proposal received in response to the

FOA receives a final rating and outcome justification.

http://www.grants.gov/

Phase 2: Department of Homeland Security Internal Relevancy Review: As soon as possible following the external review, the Selection Manager convenes an Internal Review Panel of DHS

SMEs to review proposals transmitted from the external review phase.

NOTE: Only the best proposals will be transmitted from the External Review to the Internal

Review Panel.

The Internal Review Panel will focus on a different set of evaluation criteria specified in the

FOA. These criteria are designed to identify the relevance of proposals to S&T’s mission, relative need for the research, and cohesion with other research projects funded by S&T or other agencies. The Internal Review Panel will focus on a range of criteria for proposed research, education, and/or outreach programs and the applicants’ demonstrated capabilities to deliver research and development results. The Internal Review Panel follows the same process as the external review and results in a final rating and justification for each proposal received from the external review panel.

Phase 3: Site Visit Review: A site visit review only applies when selecting or re-competing a

COE. The site visit review team will evaluate proposals transmitted from the internal review phase. Reviewers will determine the extent to which the applicant’s proposal and any site visit materials address the criteria stated in the FOA. The site visit team will focus on the applicant’s ability to lead and manage interdisciplinary efforts addressing a national scope on homeland security

Biennial COE Program Reviews

DHS conducts an evaluation of each COE every two to three years at both theme and project levels, as well as the effectiveness of the COE leadership team. The review has a structure similar to the COE competition reviews evaluating scientific quality, relevance, and leadership/management. DHS S&T OUP will utilize the results of each phase of this review, the

COE Principal Investigator on the future direction of the COE.

1.3 Education and MSI Programs

Education and MSI grants are subject to a two-phase review process (Phases 1 and 2 described above) to ensure close fit to DHS’ long-term scientific and technological workforce development needs. Competitions for education and MSI programs relevant to this solicitation take place every other year.

Solicitations are sought once every two years from qualified and accredited Minority Serving

Institutions (MSIs) Granting two-year Associate Degrees, four-year Bachelor’s Degrees, and

Graduate Degrees to establish homeland security scientific leadership programs in areas critical to homeland security. The DHS SLAs are intended to establish and support the development of

MSI education programs in HS-STEM disciplines that are required to prepare students for careers in research and development of the technology. The proposed SLA programs should include coordinated teaching initiatives, curriculum development, mentoring, and undergraduate scholarships. These internal MSI programs should be augmented by establishing collaborative relationships with the DHS COEs, Department of Energy National Laboratories, and other government laboratories engaged in innovative Homeland Security research and development to access their expertise and to provide venues for summer study. The evaluation and selection of awardees follows phases one (1) and two (2) of the above COE selection review process described.

In support of the review processes for the COE, Education and MSI Programs described above, the Contractor will be expected to provide the following services, in combinations appropriate to the event at hand:

• Assist in collecting white papers and full applications via electronic means from www.grants.gov and other sources and organize these for review by SMEs.

• Help DHS staff identify potential SMEs, scientists, and engineers in research areas related to the DHS mission to serve on merit review panels/study groups.

• Maintain a contractor owned pre-existing electronic database of DHS-relevant SMEs and continually update this database as additional experts are identified.

o Also maintain the ability to provide OUP with a suggested list of SME areas based on the subject area of the review.

• Conduct reviews of relevant published and unpublished literature to identify SMEs qualified for peer and merit reviews.

• Assist DHS staff in identifying specific academic and other experts suitable to review focused research project proposals, COE proposals, Education, and MSI proposals, and

COE progress by evaluating research quality, productivity, educational quality, management practices, outreach to, and interactions with customers, transition of research results, and involvement of minority serving institutions, among others.

• Provide technical support, such as reviews of publications, to support the integrity of the review process.

• Screen qualified reviewers for specific proposals and assist DHS in assigning proposals to panels, when more than one panel is running concurrently. Set-up proposals for electronic delivery and distribute to appropriate personnel and reviewers.

• Provide trained moderators to facilitate the discussion of proposals with maximum objectivity and other activities with maximum productivity.

• Establish a productive work environment for reviewers and provide technical and logistical instruction to peer and merit review panelists for the meeting. Prepare merit review procedures as directed by DHS; provide technical guidance to ensure reviews follow established policies and procedures.

• Provide, operate and conduct training for DHS approved existing, web-based portal and software programs for electronic (non-face-to-face) reviews, data collection, reviewer ratings, written justifications, and the tabulation and scoring of input from multiple reviewers. This web-based tool must include a minimum:

o The ability to aggregate individuals’ ratings, via electronic means, on selected review criteria and average ratings using a variety of central tendency or scoring rules (e.g., mean, median, mode, “skating” or “diving” scores). Calculate an http://www.grants.gov/ average rating, median rating, mode rating, and outcome justification from the peer review for each proposal received.

• Provide a quality control plan to ensure a pre-assessment is conducted on reviewer comments to confirm they are commensurate to scores and meet baseline requirements established by OUP. Reviewer comments that are overly generalized or vague provide little value.

• Manage peer and merit review meetings to foster rigorous professional discussions about the merits of each proposal reviewed, whether face-to-face, via teleconference, videoconference, or webinar or more than one of these simultaneously.

• Ensure confidentiality of SMEs chosen to participate in review panels, identify potential conflicts-of-interest among reviewers and refer these to the DHS program office for disposition.

• Provide administrative services including notetaking, coordination and distribution of written materials, transcription, and production of an executive summary of reviewers’ ratings.

• Provide advanced reporting of the results and provide documentation and reports of outcome of the review.

o Reporting should at minimum include organized detailed narrative and scoring for the reviewed components o Advanced visual analytics and visual depiction of review results

• Provide logistical and management support to DHS personnel which may include identifying and securing meeting space to conduct the reviews.

1.4 Annual Program Review, Conferences, Workshops, and Meetings

The Contractor may provide Annual Program Review (COE Summit) support. This event presents applicable DHS COE research and solicits feedback on these efforts from attendees consisting of experts from academia, federal laboratories, industry and state and local government officials. This event is tentative pending funding and leadership approval.

Workshops and meetings support all DHS S&T functional areas including the COE, Education, MSI, and Focused Research Project Programs. Workshops and meetings will bring together a range of scientific expertise from academia, federal laboratories, industry and the federal government and engage them on DHS S&T research and education initiatives, collaborations and

DHS mission critical issues.

In support of these, the Contractor will be expected to provide the following services appropriate to the event at hand:

• Arrange for and provide audio-visual, archiving, and webcasting equipment and services, as necessary.

• Communicate with registrants.

• Conduct site visits and meet with venue personnel (non-government sites); make recommendations on venue selection.

• Develop evaluations to include performance metrics and success benchmarks.

• Develop, deploy, maintain, and administer a web site that is capable of providing registration functions, displaying venue and local area information, participant information, event photos, agendas, and other pertinent information to include materials from prior years’ events.

• Manage abstract competitions.

• Manage and perform event clean up and closeout activities.

• Order supplies, materials, equipment, and room setups; work with suppliers.

• Organize, schedule, and conduct planning meetings at intervals prescribed by DHS personnel.

• Prepare event budget, plans, notices, registration forms, invites, brochures, agendas, signs, banners, and name badges and submit to OUP management for approval.

• Provide administrative services, including notetaking, registration of attendees, and coordination of written materials, electronic feedback mechanisms, and executive summaries.

• Provide support in the planning process, determine sequence of events, and set the tone of the event.

• Recommend and correspond with speakers/exhibitors.

• Recruit, train, and provide an orientation for support staff.

• Provide logistical and management support to DHS personnel, which may include identifying and securing meeting space

DHS conferences and related activities must be conducted as efficiently and cost-effectively as possible. Therefore, Contractors must follow DHS guidelines when planning events to include cost effective location and venue. Contractors must provide documentation that conference related costs are minimized. Where appropriate, DHS will include this documentation in conference approval requests submitted to DHS Leadership. This includes any costs incurred by related support contracts.