Horizon Scanning RFP Questions

RECOVERY - RFP AHRQ-10-10003 – Horizon Scanning System

Amendment 1

The purpose of this modification is to respond to questions received from potential offerors. The date and time for receipt of proposals remains unchanged by this amendment.

Horizon Scanning RFP Questions Solicitation Questions

1) Will the offeror/prime contractor be able to claim the dollar value of small business subcontracts awarded by first tier subcontractors to second tier subcontractors as part of meeting the prime contract Small Business Subcontracting Plan requirements?

No, prime contractors can only include direct subcontractors in their subcontract goals and reporting.

2) Does AHRQ anticipate awarding 1 contract or multiple contracts?

AHRQ anticipates it will award one (1) contract from this solicitation (AHRQ-10-10003).

3) Under what circumstances would AHRQ award multiple contracts?

AHRQ currently has no plans to award more than one (1) contract for horizon scanning under this particular solicitation (AHRQ-10-10003).

4) Attachment 5 indicates that this work is for a 3 year base and 2 options year; however on page 1 of the RPF it states that “ a firm, fixed-price contract is contemplated for a period of two years, with three 1-year option periods.” Please confirm that the period of performance is a 2 year base period with three 1-year option periods.

Attachment 5 is a sample only to show the format to submit proposed prices. The period of performance for this requirement is a two-year base period with three one-year options.

5) Can the government provide an estimate for the level-of-effort or cost for this work?

(Please clarify the available funding amounts for the base period and for the option years for this contract.)

(What is the estimated level of effort related to the Horizon Scanning RFP?)

Offerors should propose a level of effort and price that shows their understanding of the requirement and their unique approach to accomplishing the goals and objectives of the horizon scanning project.

Eligibility Questions

6) Is this solicitation open to USA based organizations only, not international organizations?

In order to receive this contract, an organization must be able to meet the reporting requirements required by Section 1512(c) of the American Recovery and Reinvestment Act of 2009. The reporting requirements include submission of quarterly reports on the use of ARRA funds including, among other things, an estimate of the number of jobs created and the number of jobs retained in the United States as a result of the Recovery Act funds.

7) Will AHRQ consider a proposal from an Offeror that is not an EPC? Specifically would you be interested in reviewing a proposal from a partnership of a university with a strong research program in information retrieval and a medical university with an informatics program – as well as key staff with long, strong experience in horizon scanning as well as EPC experience?

There is no requirement that Offerors be an AHRQ Evidence-Based Practice Center (EPC). Proposals from Offerors with and without an affiliation with an EPC will be considered.

8) Are AHRQ’s Evidence-based Centers (EPCs) and Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Network members eligible to prime this work? Are they eligible to be on a team as a subcontractor?

AHRQ’s Evidence-based Centers (EPCs) and Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Network members are eligible to prime this contract. They are also eligible to be on a team as a subcontractor.

9) In section L10.B.3 (Key Personnel) It states the Project Director shall have a doctoral or medical degree and that the Project Manager must have a master’s degree in a health and human services-related specialty. Can experience be used to substitute for advanced education requirements? If so, what is the equivalence requirement the government would like for us to use?

For the Project Director, experience is NOT a substitute for the advanced education requirements. For the Project Manager, an additional three (3) years’ experience in a health and human service-related field may be a substitute for the Master’s degree requirement. Eligible Project Managers, therefore, may have thirteen (13) years’ total experience in a health and human service-related field in lieu of a Master’s-level degree.

General Horizon Scanning Questions

10) To which horizon scanning services does AHRQ currently subscribe, if any?

AHRQ currently does not subscribe to any horizon scanning service.

11) In addition to Euroscan, are there other examples of horizon scanning in the United States that could serve as a partial model for the envisioned horizon scanning system?

Euroscan is cited in the solicitation as an example of current efforts to conduct healthcare horizon scanning. Offerors should not interpret this as an endorsement of any horizon scanning method or methods being used by any of the Euroscan member organizations. Offerors shall describe in their technical proposals the specifics and rationale for any proposed horizon scanning method(s) and/or system(s).

12) Is the purpose of the forecast simply to identify potentially high impact technologies, or will it be used by AHRQ for separate purposes (such as policy setting)? If the forecast will be used by AHRQ for other purposes, how does AHRQ plan to use them?

The purpose of the forecasting is to identify potentially high impact healthcare technologies in order to inform comparative effectiveness research investments at AHRQ and the broader research community. The results from horizon scanning will NOT be used for policy setting by AHRQ.

13) Could you provide some examples of “behavioral changes” and “delivery system strategies” that would qualify as emerging technologies to be tracked in a horizon scanning system?

Technologies that would be tracked by potential Offerors may differ based on differences in horizon scanning methods. Nevertheless, examples of different behavioral changes and delivery system strategies can be found in the IOM report titled, “Initial National Priorities for Comparative Effectiveness Research” (http://www.iom.edu/Reports/2009/ComparativeEffectivenessResearchPriorities.aspx).

14) The term analyzing “relational data” appears multiple times in the section on Key Personnel and Staffing Plan (p 75) but isn’t defined in the proposal. Please provide a definition for this.

Relational data, or a relational dataset, includes information on various healthcare technologies identified in horizon scanning in which the included technologies can be grouped or organized in logical ways based on common characteristics in order to summarize and/or highlight linkages among the various technologies.

Horizon Scanning Methods Questions

15) How automated (machine-based vs. human based)) is the envisioned horizon scanning system?

AHRQ has no requirements on the extent to which the horizon scanning system is machine-based or human-based. This is left up entirely to Offerors and their horizon scanning methods, the details of which should be included in submitted proposals.

16) In what ways will the Horizon Scanning system be able to draw upon the current research executed by the Centers for Education and Research on Therapeutics (CERTs) program, Evidence-Based Practice Centers (EPCs) and the DEcIDE network?

The AHRQ Horizon Scanning System will be a separate program from the CERTs, EPCs, and DEcIDE network. Research produced by these respective programs is primarily on healthcare technologies that have already diffused and entered into healthcare practice. The purpose of horizon scanning is to identify technologies that are at the forefront of innovation and that have yet to disseminate widely into healthcare practice. Nevertheless, Offerors may utilize horizon scanning methods that incorporate and/or utilize other AHRQ products (including research findings from CERTS, EPCs, or the DEcIDE network). As required in the Technical Proposal Instructions, Offerors’ methods should be described in their respective proposals.

17) Will any of the DEcIDE databases referenced on page 12 of the RFP be made available to offerors for use in the execution of this contract? If any of the DEcIDE databases will be made available, can you provide a description of the data contained in the databases?

AHRQ will not make any databases used in DEcIDE network research or any AHRQ grant-funded research available to any Offerors or the eventual Contractor. Access to and use of all databases for horizon scanning shall be handled by the Contractor.

18) If expert panels are proposed as a part of a scanning system methodology, would such committees need to be organized as federal advisory committees?

No.

19) Are there any functional requirements prepared, if so, can they be made available?

No functional requirements have been prepared.

20) How will respective groups access Horizon and the CE data? What are the firewall considerations? What are the roles, responsibilities, and access rules for Horizon?

The results of the Contractor’s activities shall be reported to the Government through the reporting requirements outlined in Task 4 of the Statement of Work. There are no requirements to give other Government contractor groups access to any horizon scanning dataset that is being maintained for the purposes of this Contract.

21) Is there an existing formula or methodology for determining the cost impact of identified healthcare technologies that the contractor should utilize (Task 2, Page 19)?

There are no requirements that the Contractor utilize any particular formula or methodology for determining the cost impact of identified healthcare technologies. The forecasting methodology proposed by Offerors should, however, be described in their respective proposals.

22) Are specific technologies/tools preferred for the database and analysis capabilities that will underpin the horizon scanning system?

The requirements of the AHRQ Horizon Scanning System have been provided in Task 2 of the solicitation. Otherwise, AHRQ has no preferences on specific technologies or tools that will underpin the horizon scanning system.

23) Is it allowable for offerors to purchase data from commercial knowledge bases on a subscription basis?

Offerors are allowed to propose use of commercial knowledge bases in their horizon scanning methodology. Examples of commercial knowledge bases include proprietary datasets and marketing research. For those resources available through Agency enterprise licenses, the Contractor must adhere to all data use agreements, data restrictions, and contractual obligations as a support agent for this project. For those resources provided by the Offeror organization, the Contractor will be responsible for securing any data rights necessary to carry out processes outlined in their proposed methods. Where appropriate, the Offeror shall provide letters of support confirming access to data sources. The Contractor is also required to ensure the inclusion of data outputs in deliverables in negotiations, subsequent assignment of rights to the Government, and/or granting of non-fee licensing use of the data resources for Government purposes. As required in the solicitation (Task 2), Offerors must describe their methods for incorporating any proposed outside resources, including commercial knowledge bases, into their horizon scanning methods.

24) Will any federal databases (FDA, NIH) be made available to support the Horizon Scanning system?

Access to federal databases shall be obtained by the Contractor. AHRQ is not responsible for making any federal databases available.

25) What is the effective upper limit on the number of technologies included in the Horizon Scanning Status Update?

AHRQ has no requirements on the number of technologies (i.e., upper or lower limits) that should be included in the horizon scanning status updates. The choice of how many technologies to monitor in horizon scanning is left up to the Contractor and should be based on the methodology described in its proposal.

26) What elements are expected to be forecasted? (E.g., number of procedures per year? Number of patients affected?)

The purpose of the forecasting methodology is to identify technologies that have the highest potential impact on clinical care, the healthcare system, patient outcomes, and cost. As part of their proposals, Offerors are required to propose metrics for measuring “high impact” (see Subtask 2.2). Therefore, the exact elements that will be forecast are up to Offerors and should be described in their respective proposals.

27) Describe further what AHRQ envisions the indexing and linking framework looking like (please give an example) – is it series of cross-referenced keywords in a separate document (like an author index to a Journal or Periodical)?

The Contractor shall maintain information collected from horizon scanning in a relational dataset. Offerors are required to submit as part of their proposal a description of the data elements they will collect on identified healthcare technologies as well as a description of how they will link identified technologies using these data elements. Examples of potential data elements are provided in the solicitation (page 73-74). Linkages across technologies could potentially be based on keywords, though it is at the discretion of the Offeror to propose their own process of logically linking technologies identified in horizon scanning.

Reporting Questions

28) What is the expected length (word count), not including bibliography, of the High Impact Technology reports, Target Technology reports, and Existing Technology reports?

There is no required or expected length for the High Impact Technology, Target Technology, or Existing Technology reports. The length of each of these respective reports is left to the discretion of the Contractor and shall be based on their horizon scanning and forecasting methodologies as well as their proposed report formats.

29) Are forecasts required for HIT reports, Target Technology reports and Existing Technology reports or just HIT reports?

Forecasting is required for High Impact Technology Reports, Target Technology Reports, and Existing Technology Reports. Forecasting is not required for Horizon Scanning Status Updates.

30) How are the 280 high impact reports to be delivered each quarter – by PDF? Posted to a shared site? HTML or XML posted to a website? HTML/XML posted to a database/document repository? That is, what format will the reports be delivered in? (e.g., MS WORD, PDF, HTML/XML)? (this speaks to the skill set needed on our end) All reports, including High Impact Technology Reports, should be submitted to the AHRQ Project Officer via email as Microsoft Word documents. All reports submitted to AHRQ shall be Section 508 compliant.

Additionally, the statement of there being “280 high impact reports to be delivered each quarter” is incorrect. One High Impact Report shall be delivered to the AHRQ Project Officer every three months starting at six (6) months after the EDOC.

31) The contractor will be required to have the capacity to produce up to 20 Target Technology Reports per month with a 2 week turn around. Is this an expected “infrequent peak load” estimate or is this expected to be the “typical load” estimate?

Through this contract, AHRQ will have the ability to request up to 20 Target Technology Reports per month. This does not mean that AHRQ will request 20 Target Technology Reports each month. Nevertheless, Offerors should demonstrate their capacity to meet this requirement each month of the contract. The actual number of reports requested each month may vary, based on the information needs of AHRQ programs.

32) Are requirements for Existing Technology Reports likely to mirror requirements for HIT and Target Technology reports in format and scope of content, or are they likely to resemble a more lengthy Health Technology Assessment?

The format and scope of content of Existing Technology Reports, Target Technology Reports, and High Impact Technology Reports may be similar, though Offerors shall describe their rationale for selected approaches. The formatting of each should be detailed in the Offeror’s proposal. None of these reports, however, should resemble a more lengthy Health Technology Assessment. The reports required from the horizon scanning project are meant to be quick assessments of the contextual landscape of new healthcare technologies and not meant to be in-depth analyses of existing healthcare technologies.

33) The RFP requires the contractor to “maintain the capacity” to produce a specific number of reports of various types. If fewer than the specific number are requested, can the contractor re-allocate the budgeted funds to other tasks within the contract?

As noted in the solicitation, the Contractor shall maintain the capacity to produce a pre-specified number of reports. If fewer than this pre-specified number are requested by the Government, then the Contractor shall allocate the budgeted funds to the actual conduct of horizon scanning (i.e., the identification, monitoring, and impact forecasting of healthcare technologies).

34) Is it correct to assume that providing Horizon Scanning Reports as 508 documents is adequate to provide these reports to the public (Task 2, Page 19)?

All reports should be submitted to the AHRQ Project Officer as 508 compliant Microsoft Word documents. Reports will then be made available to the public through the Effective Health Care website.

35) Attachment 6, page 117 in the RFP indicates two Task 7 Management/Progress Reports as deliverables in Month 19. Please confirm that only one Management/Progress Report is due in Month 19.

Only one management/progress report is due in Month 19. This was a typo.

Public Posting Questions

36) Which reports (High Impact Technology/ Target Technology/Existing Technology/Horizon Scanning Status Update) will be made available to the public?

AHRQ will make Horizon Scanning Status Updates, High Impact Technology Reports, Target Technology Reports, and Existing Technology Reports available to the public.

37) How does AHRQ plan to make these reports available to the public?

Reports from the horizon scanning project will be made available to the public through the AHRQ Effective Health Care website (www.effectivehealthcare.ahrq.gov).

38) How frequently will AHRQ make these reports available to the public?

Reports will be made available to the public shortly after the AHRQ Project Officer approves them. Effort will be made by the Government to post approved reports as quickly as possible.

39) Does AHRQ intend for the Horizon Scanning contractor to build a gov website to house the horizon scanning reports, as part of this contract? If so, are there any requirements for this website? If not, is AHRQ’s intention to provide access to the horizon scanning reports via the Effective Health Care website?

AHRQ does not intend for the Horizon Scanning contractor to build any website to house reports from the horizon scanning project. All reports from horizon scanning will be made available to the public via the Effective Health Care website (www.effectivehealthcare.ahrq.gov).

Expert Meeting Logistics

40) In Subtask 5.4.1- Meeting Logistics, it states that the contractor shall handle all logistics and support for the expert meeting and arrange travel and hotel reservations for all meeting participants and reimburse non-federal attendees for travel expenses in accordance with federal regulations. Will these meetings be held in the Washington DC Metro area? Is the contractor also responsible for securing the venue for these meetings or is it anticipated that the meetings will take place at a government facility?

As stated in Subtask 5.4, the expert meeting “shall take place on-site at AHRQ in Rockville, MD.”

41) Is there any other travel anticipated for this work outside of the expert meetings identified in Subtask 5.4.1?

Travel to AHRQ in Rockville, MD is required for the kick-off meeting (Subtask 1.1) and the expert meeting (Subtask 5.4). No additional travel is required. However, the Offeror’s proposed horizon scanning methodology may involve additional travel requirements, the details and costs of which should be described in its technical and business proposals, respectively.

Proposal Requirements

42) If a Firm certifies as part of their Business Proposal Volume that it does not engage in lobbying activities, must Attachment 2, SF LLL-A be submitted?

The form does not need to be returned if the firm does not engage in lobbying activities; but a statement to that effect must be included in the Business Proposal.

43) The level of detail indicated in Section L.13 A and Attachment 5 appears inappropriate for the Fixed Price Labor portion of this effort, which is based on the deliverables indicated in both the SOW and Attachment 6 “Invoice Schedule.” Will the Government stipulate that responding in the format indicated in Attachment 5 may include Fixed Price Labor as an all inclusive cost per year, with a break out of direct costs and indirects applied only presented for non-labor elements?

No. While this is a firm fixed price requirement, this section asks for cost breakdowns to show how the firm fixed price was determined. Due to the nature of work and the variety of technical approaches that may be offered, the Government requires this level of detail to evaluate how the offeror arrived at their firm fixed price. It is necessary to see direct labor costs exclusive of indirect costs for all proposed personnel so it can be verified that direct labor rates do not exceed the salary rate limitation under the Consolidated Appropriations Act of 2010, Public Law 111-117, and so that labor rates can be reviewed for reasonableness.

44) Can you please confirm that the SB standard for the purpose of subcontracting for this procurement is 500 employees?

The size standard is $7.0 million in average annual receipts.

45) Section L.10.B.4.e states “Provide an organizational chart and Program Evaluation Review Technique (PERT) chart showing all tasks (i.e., a staffing plan) “. Section L.10.b.4.h states “Provide a person-level task-loading chart (to include the efforts of consultants and subcontractors) and an organizational chart indicating clear lines of authority… “. These two requirements appear to be redundant other than the specification in L.10.B.4.e of a PERT chart. If these are not duplicate requirements, please explain how they are different. Is it correct to interpret Section L.10.B.4.e to require an organizational chart depicting the staffing plan and lines of authority, and a separate PERT chart loading all tasks to be performed by the leader responsible for the function as presented in the organization chart? Typically PERT charts are task rather than person-level driven.

In response to L.10.B.4.e and L.10.b.4.h of the Technical Proposal Instructions, Offerors shall include the following in their proposals: 1) a staffing plan that describes how the project will be staffed; 2) an organizational chart indicating clear lines of authority; and 3) a list of tasks required to complete the project and the individuals assigned to each task. Offerors shall present this information graphically against the progression of time, such as in a Gantt chart or PERT chart.

46) The technical proposal instructions indicate that “Appendices are to be provided electronically in MS Office format on CD, in the same quantity as the technical proposal.” Please clarify if the appendices should ONLY be electronically submitted and NOT printed in hard copy. If we are required to submit electronic copies on CD, please confirm that you require 11 CDs (one per hard copy of the proposal). Also, please let us know if it would be permissible for us to include the appendices in a pdf format on CD rather than in an MS Office format?

(Section L.10 page 71 states that “Appendices are to be provided electronically in MS Office format on CD, in the same quantity as the technical proposal.” Is the offeror required to submit only the appendices on a CD or should the CD include the full volume for the technical proposal in MS Office format?)

Offerors shall submit an original and eleven (11) copies of their technical proposals. The original technical proposal shall also include the appendices in hard copy. The appendices for the eleven (11) copies of the technical proposal shall be included on 11 CDs. These eleven (11) CDs should include electronic copies of the full volume, including the main body of the technical proposal and the appendices. These electronic documents may be in a MS Office format or in a PDF format.

47) On the first page of the RFP it indicates that the offerors shall submit items A (i.e., Technical Proposal) through C. There are two items labeled C. Should the last item, Business Proposal, be labeled D to indicate it is a piece that is separate from the Small Disadvantaged Business Participation Plan?

This was a typo. The last item should be labeled D.

48) In Section L.11 (p.78) (Past Performance), it states “A list of the last five (5) contracts and subcontracts completed during the past three years and all contracts and subcontracts currently in process that include services similar or relevant to the requirements of this solicitation.” Please clarify how many contracts (minimum and maximum) are required as part of the offeror’s past performance volume.

There is no minimum or maximum number. Offerors should include the amount they deem necessary to show relative experience and past performance. It is to the Offeror’s advantage to show as much positive past performance and relative experience as possible, but the Government realizes that this may be extensive and an Offeror might not want to supply all its past performance for the Government to evaluate.

49) In Section L.11 (p.79) (Past Performance), it states “The attached Past Performance Questionnaire and Contractor Performance Form (Attachment 1) shall be completed by those contracting organizations listed in (1) above.” Please clarify how many Past Performance Questionnaires must be completed and submitted for this opportunity.

There is no pre-determined number. Please submit the amount the offeror deems necessary for adequate evaluation.

50) Will the government accept email delivery of the Past Performance Questionnaire sent to the AHRQ contracting officer?

The Government will accept email delivery of the Past Performance Questionnaire sent directly to the AHRQ Contracting Officer from the individual who is completing the questionnaire and not from the organization that is being graded.

Miscellaneous Requirements Questions

51) When the RFP asks for horizon scanning artifacts to be delivered to AHRQ at the conclusion of the contract, how should large, proprietary, subscription-based or limited-access datasets (or extracts from these datasets) be handled? For example, what is AHRQ’s expectation for artifacts derived from the Surveillance Epidemiology and End Results datasets, Medicare Standard Analytic File, etc. Is it sufficient that AHRQ simply receives a list of these large or protected, limited-access resources?

All horizon scanning activities performed under contract to AHRQ should be reflected in the horizon scanning artifacts delivered to AHRQ at the conclusion of the contract. This includes, but is not limited to, documentation of all horizon scanning protocols and methods developed under Task 2 of Statement of Work; documentation of any changes or refinements to these processes; and documentation of all horizon scanning activities performed while under contract to AHRQ. Proprietary subscription-based or limited-access datasets (or extracts from these data sets) should be excluded from the final deliverable to AHRQ. The Contractor shall, however, identify all resources and provide a description of how they were used in all horizon scanning activities performed while under contract to AHRQ.

52) How frequently will the contractor evaluate and report on the Identification and Monitoring protocol? How frequently will the contractor evaluate and report on the Forecasting Methodology?

The Contractor is not required to evaluate or report on the Identification and Monitoring Protocol or the Forecasting Methodology during the two-year base period of the contract. The requirements of Subtask 6.1 and Subtask 6.2 call for the Contractor to develop Evaluation Plans. If the Government chooses to exercise any option years to the Contract, then the Contractor shall be required to conduct a single evaluation of their Identification and Monitoring Protocol (see Subtask 8.1) and their Forecasting Methodology (see Subtask 8.2) based on the evaluation plan developed as part of Subtasks 6.1 and 6.2. The Government, however, is not obligated to exercise any option years.

53) The contractor is to implement a plan to incorporate important findings from the literature review and expert panel to improve the AHRQ HHSS within 27 months of the EDOC. This is outside of the initial 2 year (24 month) term of the contract. What does AHRQ mean by this requirement?

Task 8 requirements, including Subtask 8.3 (Horizon Scanning System Update), are required in the event that the government exercises any option years.