SampleTaskOrder2.pdf

Logistics Support for the Center for Global Health

Task Order Title – Proposal for Feasibility Study for the Evaluation of Affordable Cancer

Technologies for Global Health (ACTs) Program

A. Task Order Number – Sample Task Order 2

Background and Task Order Statement of Work:

The NCI Affordable Cancer Technologies (ACTs) supports the development of cancer-relevant technologies suitable for use in low- and middle-income countries (LMICs). Specifically, the program seeks to fund the adaptation, application, and validation of existing or emerging technologies into a new generation of user-friendly, low-cost technologies for imaging, detecting, diagnosing, preventing, and/or treating cancers in LMICs.

Because the NCI Affordable Cancer Technologies (ACTs) program is a new program, the implementation of an effective evaluation framework and strategy to adequately assess the program is both timely and necessary. The purpose of the study described herein is thus twofold: (1) to determine what processes and aspects of the ACTs program are objectively assessable and what outcomes are quantifiable, taking into account the unique aspects such as the program’s structure and trans-divisional nature, and (2) to design such an evaluation in a manner that is comprehensive, timely, and meaningful to future iterations of the program. To achieve these objectives, the Center for Global Health (CGH) is seeking to hire contract support to conduct both a feasibility study (Phase I) and subsequent outcome evaluation (Phase II) for the program. Several critical elements will affect both whether an outcome evaluation can be conducted and the nature of that outcome evaluation.

• Outcomes. Any conceptual framework for the ACTs program should be clear about the outcomes. It is too early to assess long-term outcomes, as the first grants were awarded in 2014 and the program extends into 2022; shorter-term outcomes relating to the generation and dissemination of knowledge, creation of scientific collaborations, initial regulatory approvals, and pre-commercial successes/commercialization plans will be measurable within the two years.

Evaluation should seek to establish outcomes for both the UH2 and UH3 portions and to disambiguate each phase of the work to the maximum extent possible. Noting that many projects end after the UH2 phase, it is important to evaluate the contribution of this phase of the work independently.

• Other Research Opportunities. The NIH offers a large array of other funding opportunities (i.e., IMAT, Academic-Industrial Partnerships, SBIR for Global Health, Fogarty’s mHealth program) to support research focused on technology translation for devices and assays. The availability of these other funding opportunities to ACTs investigators may leverage their own efforts. Thus, understanding the interplay between these other opportunities and the ACTs-funded researchers is important in evaluating the ACTs program.

Moreover, the global health technologies space receives research funding support from beyond the NIH and it is important to understand the potential interplay between this opportunity and the activities of other key stakeholders. Therefore, the scope of any thorough evaluation must include other funding agencies, including NGOs and non-US agencies (e.g., Wellcome Trust, Bill and Melinda Gates Foundation).

• Research Environment. Increasingly, researchers are encouraged to be more collaborative, working across disciplines and institutions. These arrangements are expected to be more effective in bringing intellectual resources to bear on research questions. Individuals now also have access to a variety of instrumentation, including bioinformatics tools that were not available a decade ago. The research environment has changed and become more complex. Describing the availability and use of this environment to ACTs -funded researchers is important to characterize for the purposes of the outcome evaluation.

Moreover, significant components of each ACTs project are conducted in low-resource settings.

As such, many of the resources typically available to researchers, such as bio-repositories, bioinformatics tools/clusters, highly-trained clinical research support staff, and modern laboratory settings may be unavailable to investigators.

• Submission process/review. Because the ACTs program requires a special application with a 30-page research strategy, outlining both the UH2 and UH3 components, letters of support, timelines, and milestones, it is worth evaluating whether or not all of this content results in stronger applications and to what degree this additional information adds utility to the review process. This should be assessed through a survey of applicants (both successful and unsuccessful) and reviewers to determine its utility.

• Award Mechanism. The ACT program makes use of an unusual mechanism, namely the

UH2/UH3 Phase Innovation Cooperative Agreement. Because this is not one of the more traditional mechanisms, it is worth examining whether the program is strengthened through its use. Specifically, the necessity of the UH2 phase should be addressed. The phase transition needs to be examined with regards to this program to determine if it increased the quality of research supported by the NCI. Finally, the use of a cooperative agreement also needs to be addressed. Did the additional involvement of NCI staff add to the resultant scientific output to a degree commensurate with the increased labor expense? This should be assessed through interviews with PDs, the SRO, PIs, and unfunded applicants.

Phase I Feasibility Study Design – 6 Months

The feasibility study used to inform the ACTs evaluation will focus on five broad research questions that should be addressed in the feasibility study report:

1. What evaluation questions are answerable, and, when answered, will they provide meaningful and useful information about the ACTs program? (Based on the objectives of the ACTs program as stated above, what specific evaluation questions should guide the study? Will these questions provide meaningful data that will allow for the assessment of progress made toward ACTs program goals?)

2. What measures can be used to answer the evaluation questions? (What performance measures will reveal whether the ACTs program goals are being achieved?)

3. What data are needed to answer the evaluation questions? (Are data currently available? If not, what steps are required to obtain the necessary data?)

4. What is the most appropriate and cost-effective method for collecting and analyzing the data for the measures? (What type(s) of research methods can be employed to collect and analyze the

ACTs program data? How will these method(s) be implemented? What are the limitations, if any, of these method[s]?)

5. How much time would be required to collect and analyze the data? (Given the type[s] of research methods employed and the availability of data, how long will it take to collect and analyze the ACTs evaluation data?

To support the proposed tasks, NCI staff will provide the following items for the evaluation team:

• List of RFAs for the program

• Bi-weekly meetings with the COR and ATC program lead

• Search terms to aid research into the program and technological space

• Package of informational materials used to advertise the program

• Access to application files via the Impact II database

Answers to the above questions will be pursued via an evaluation construct and strategy that incorporated the following elements:

• Clarification of Issues and Objectives. The COR will be designated as the lead point of contact for the evaluation and for subsequent interactions between the evaluation contractor/staff and the ACTs program. The contractor will meet with the COR to discuss at least, but not limited to the following items and to inform the development of the evaluation plan:

– The program to be evaluated and its goals

– The information sought from the evaluation

– How the results of the evaluation will be used

– The timeliness of the evaluation

The kick off and wrap up meetings will be held in-person at the NCI Shady Grove facilities and should be scheduled in coordination with the ATC Program Lead and COR.

Deliverable: Meeting summaries, scheduling of bi-weekly calls with NCI staff.

• Identification and Review of Relevant Literature. The contractor shall conduct a comprehensive review of relevant literature or related studies, using the search terms provided by NCI, to determine the feasibility of an evaluation and to inform the evaluation design.

Deliverable: literature review, annotated bibliography, archive of papers within bibliography/review

• Development of Study Questions. The contractor shall develop a proposed set of study questions for the evaluation to answer. The questions will consider which inputs, processes, outputs, outcomes, and implementation effects are most useful to learn about. Questions will address the stated purpose of the evaluation and the achievement of the specific objectives of the evaluation.

Deliverable: section of final report and recommendations

• Identification of Key Variables. The contractor shall determine what information is needed to answer each study question, including variables that describe the program resources and activities, population characteristics, and external factors.

Deliverable: section of final report and recommendations

• Development of a Conceptual Framework for the Evaluation. In consultation with the ACTs program staff, the contractor shall develop a conceptual framework to illustrate how the ACTs program is intended to work to achieve its goals, including a logic model of the framework. This framework should address the six questions posed for the feasibility design, and summarize the identified elements that will affect evaluation. It should describe the general methodology recommended for evaluation, including population, time, output and impact metrics. It should provide a logic model that will guide the evaluation and identify data gathered and analyzed for the feasibility study that will aid evaluation outcomes.

Deliverable: slides (full deck and deck using NCI template), section of final report, logic model for evaluation, raw data, graphics and slides

• Establishment of a Plan for Data Collection and Analysis. The contractor shall determine the types of data that would be used to answer the study questions and identified feasible performance and comparison measures. Strategies will be determined to ensure data integrity and to address ethical considerations such as burden on respondents, confidentiality, and security of gathered information.

Deliverable: written plan, slides

• Recommendation for and Pursuit of Evaluation Design. The contractor shall recommend and pursue

(an) evaluation design(s), as agreed upon by the COR and ACT program staff. Specifically, a multistage approach will be developed for conducting the evaluation. The first stage will involve review of grant applications and analysis of data on their overall and subcomponent ratings. A literature review shall also be performed, with the resulting information used to generate a protocol for conducting interviews with NCI staff members associated with each of the three ACTs areas.

Staff members shall be interviewed about objectives, process, and the projects that have been funded. The primary aim underlying such interviews will be to describe the ACTs program as an intervention and to understand why certain funding decisions were made. Stakeholders may also more easily be identified through these interviews. The second stage shall involve interviewing nine

ACTs principal investigators (PIs). These interviews will be aimed at understanding how grantees operate in the field, the extent to which they collaborate, their perception of the eventual outcomes of their work, their timeframes for development and next steps, and how such grantees use institutional and other resources in their ACTs -sponsored research.

Deliverable: Written proposal, raw data, timeline

• Preparation of materials for OMB Submission Package. The contractor shall work with COR and

ACT program staff to assist in the development of a submission package for the review of evaluation tools, methodology, and burden by the Office of Management and Budget. Package materials include a finalized evaluation survey, program summary, invitation letter, and summary of methodology, and should be modified or reformatted from existing deliverables to meet package requirements.

Deliverable: Evaluation survey, program summary, invitation letter, and summary of methodology, and modified or reformatted from existing deliverables

B. Reporting Requirements and Deliverables:

The Contractor shall begin work immediately upon receipt of the Phase I items listed above and provided by NCI staff, with an initial meeting to occur within three business days of receipt of those data. For each task, the Contractor shall prepare a mock deliverable up front for review by NCI (COR and ACT staff) then work towards creating the final deliverables. The project manager shall hold teleconferences with NCI staff after delivering intermediary and final deliverables.

A. Reporting

The Contractor shall hold bi-weekly reporting calls with the COR and ACT project manager(s) to discuss progress on the evaluation tasks. Upon completion of the evaluation tasks, the Contractor shall provide a summary report that outlines key findings. The audience for all documentation is the

NCI program staff.

B. Completion

The final stage will close out and confirm the completion of the work with NCI, as well as to perform any final administrative actions. The Contractor shall submit the final version of all deliverables in the format specified by NCI (PDF, Excel, MS Word, or other).

Phase I Deliverables (also articulated in the SOW)

The Contractor shall provide the following deliverables to the project manager within the designated timeline:

Deliverables Timeline

Meeting with the project manager and NCI staff (in person) within 2 weeks of award of TO

Develop and finalize project work plan within a month of TO award

Literature Review, annotated bibliography, archive of papers within bibliography/review within six weeks of TO award

Development of research questions, identification of key variables and metrics, identification of stakeholders

3 months after TO award

Development of an Evaluation Framework and plan for data collection and analysis

(draft version)

5 months after TO award

Proposal and Recommendations for the pursuit of an ACT program evaluation (draft version)

5 months after TO award

Preparation of materials for OMB package 5.5 months after TO award

Final Version of NCI/CGH deliverables

(slides, executive summary)

No later than 6 months after TO award

Final version of all document deliverables No later than 6 months after TO

Award

Wrap up in person meeting No later than 6 months after TO award

Teleconferences with project manager and other NCI staff on project updates

Bi-Weekly after TO kickoff

C. Proposal Due Date: Please review the Base IDIQ Solicitation.

D. Offerors Questions: Please review the Base IDIQ Solicitation.

E. Type of Task Order: Firm Fixed Price

F. Invoicing and Payment: See the base award for invoicing and payment instructions.

G. Anticipated Number of Task Order Awards: 1

Task Order Period of Performance:

The Phase I period of performance is XXXXXX (6 months). The work will take place off-site at the

Contractor’s facility. The Contractor shall be available during regular business hours by phone, e-mail, and fax.

H. Technical Proposal Instructions: NONE

I. Business Proposal Instructions:

a. Business Proposal COVER SHEET: Submit the following:

• Solicitation or contract number

• Task order number

• Name and telephone number of point of contact

• Name, address, and telephone number of contract administration office (if applicable)

• Date of submission

• Name, title, and signature of authorized representative

b. Budget: Provide a summary of proposed cost and/or price (Attachment 1); profit or fee (as applicable); and total.

J. Evaluation Factors for Award

Please review the Base IDIQ Solicitation.