NCHS_Q&A_for_RFP_75D301-19-Q-69740.pdf

CDC/DDPHSS/NCHS/DHNES Gaining NHANES Respondent Cooperation (75D301-19-Q-69740)

RFP Questions and NCHS Answers

1Q. Is this a new requirement or is there an incumbent vendor? If available, can you please send along the incumbent contract number?

1A. This is a new requirement with no incumbent vendor. This is independent project to observe a separate data collection contractor implementing CDC procedures, but it is not a follow-on.

2Q. Could NCHS provide a range of the funding available for the contract?

2A. No, the Government will not provide a range of the funding available. However, the following table can provide guidance to Government estimated level of effort. This estimate does not create contractual requirements and each offeror is required to provide a technical proposal which is clearly supported by the proposed business solution that provides best value to the

Government.

CLIN Estimated PoP (range) Estimated LoE (in hours)

CLIN 0001 Literature Review: F2F HH surveys

April 01, 2019 – June 30, 2019 162~198

CLIN 0002 Literature Review: Changing population attitudes

April 01, 2019 – June 30, 2019 216~264

CLIN 0003 Evaluation of NHANES gaining cooperation protocol

April 01, 2019 – June 30, 2019 1,458~1,782

CLIN 0004 Identify behavioral barriers to participation

July 1, 2019 – March 31, 2020 1,998~2,442

CLIN 0005 Identify gaining cooperation’s best practices

April 1, 2019 – March 31, 2020 1,080~1,320

CLIN 0006 Interview and observe

NHANES field staff

April 1, 2019 – March 31, 2020 1,080~1,320

3Q. Page 2, Section B, should the labor hours associated with the kickoff meeting, conference calls, and debriefing be budgeted under CLIN 0007? If not, in which CLIN should they be budgeted?

3A. CLIN 0007 will be a Cost-Reimbursement Line Item. Proposed cost should include all labor hours associated with kickoff meeting, conference calls, and debriefing. This would include any projected travel associated specifically with any proposed meetings.

4Q. Page 2, Section B, should travel to the KO meeting and debriefing meeting be budgeted under CLIN

0007 or CLIN 0008? Should travel for observations in CLIN 0006 be budgeted under CLIN 0006 or

CLIN 0008?

4A. CLIN 0007: Any projected cost in association with Kickoff meeting and debriefing meeting should be proposed in CLIN 0007 along with any labor hours associated with kickoff meeting, conference calls, and debriefing.

CLIN 0008: All other projected travel expenses. For the business proposal, this should be the estimated cost for three hypothetical sites; Miami, FL, Sioux City, IA, and Los Angeles, CA.

Upon award and signing of Non-Disclosure agreements, up to six site visits (PSUs), and for each PSU up to three trips, for a total of no more than 18 trips from CLIN 0006 and any other proposed travel needs not falling under CLIN 0007 will be negotiated at the kick-off meeting.

Both CLIN 0007 and 0008 will be awarded as Cost Reimbursement Line Items.

5Q. Will all of the materials, protocols, manuals, ERB submissions needed for the work under CLINs

0003 & 0004 be provided by NCHS at the time of award?

5A. Yes, all materials, protocols, manuals, and the current ERB-approved data collection protocol for NHANES will be provided by NCHS.

6Q. For CLIN 0004 on page 21, does NCHS intend for individuals in the NHANES sample households to be used for randomized controlled trials?

6A. Either the NHANES sample or a stand-alone sample of non-NHANES participants, or both, can be used for the randomized controlled trials. If the NHANES sample is selected for the test, randomization can occur at the primary sampling unit level, the segment level, or the household level and not at the participant level. In other words, different households may receive different materials. All participants in the house will receive the same materials.

7Q. On page 25, “3) Information Security Categorization” text, can NCHS clarify the statement in bold that reads “… the vendor is not permitted to process, store or transmit PII.” We would like clarification about how this statement relates to the work under CLIN 0004, which includes conducting randomized controlled trials.

7A. The conceptual design of the randomized controlled trial does not require access to PII of

NHANES participants. Vendor may test materials on non-NHANES participants. For example, the vendor may enroll participants in a test as part of a pilot that does not include NHANES participants. If during the performance period access to PII is required, NCHS will handle this step.

8Q. On page 21, Section V, the first paragraph says that “All tasks are Firm Fixed Price except CLIN

0006 and CLIN 0007, which are cost-reimbursable.” Should the text in bold refer to CLIN 0007 and

CLIN 0008 as being cost-reimbursable instead of CLINs 0006 & 0007?

8A. This was an editing error. The text should have referred to CLIN 0007 and 0008 for cost-reimbursable travel line items, and has been corrected in the revised RFP.

9Q. Page 24, Section 2C states compliance with HHS IS2P. Is compliance with NIST SP 800-53 rev. 4 required? Can NCHS clarify the requirements related to submitting a system security plan (SSP) and obtaining an Authority to Operate (ATO) for this contract? If an ATO is required, under which CLIN should the labor hours be budgeted for the preparation of the documentation and working with CDC to achieve the ATO?

9A. Inclusion of this language is CDC required. While the contractor shall keep confidential all information they become privy to, there will be no technology developed by the contractor, nor will the contractor process/store/transmit government information, or access any government information systems. Further, contractors will be entering into a Non-Disclosure Agreement that prohibits them from recording or sharing any information with unauthorized individuals. As such, demonstrated compliance with NIST SP 800-53 rev. 4 is not required; nor are System Security Plan and Authority to

Operate required.

10Q. Page 28, B.2, Role Based Training (RBT), can NCHS provide additional details on what the training would entail and which roles or positions on the contractor’s team would be required to complete RBT?

10A. Inclusion of this language is CDC required. Not applicable to this proposal. The program manager has determined that contractor personnel will not have significant security responsibilities.

11Q. Will the four projects included in the Similar Experience section described on page 44, Section C2, also be the ones used to evaluate Past Performance as described beginning on page 46, Section D?

11A. Similar Experience is part of the offerors’ technical proposal, and offerors should provide examples of any projects performed that show relevant experience and success to the specific requirements of the solicitation. Past Performance is a separate risk assessment performed on all

Government contracts for work previously performed with the Federal Government. Offerors are not required to submit projects for Past Performance, and if prior Federal Government experience is lacking, the risk is assessed as “Neutral”.

12Q. Page 25, Section 5c refers CUI compliance. Is compliance to NIST SP 800-171 required for this project as it is only applicable if CUI is processed, stored, or transmitted by nonfederal organizations using nonfederal systems? If a SSP and ATO is required under which CLIN should the labor hours be budgeted for the preparation of the documentation and working with CDC to achieve the ATO?

12A. Inclusion of this language is CDC required. Contractors are encouraged to mention their information protection capabilities, which might include how their organizations implement NIST SP

800-171 protections. However, demonstrated compliance with NIST SP 800-171 is not required since the contractor will not process/store/transmit government information or access government systems.

Further, contractors will be entering into a Non-Disclosure Agreement and will not be permitted to collect PII or any other sensitive information or relay it to any unauthorized individuals.

13Q. Could the government provide an expected schedule of deliverables? This would help us understand how various deliverable dates may overlap.

13A. CLIN007’s kick-off meeting will be followed by CLIN1, 2, 3. These CLINs can overlap.

Upon the completion of these CLINs, work on CLIN4 begins. CLIN5 can occur before CLIN4 or after the completion of CLIN6. CLIN6 will concurrently occur with other CLINs as it requires travel to different locations at different times. The timing of these different steps will be determined during the kick-off meeting. The RFP has been revised to include projected ranges for each CLIN to provide additional clarity (see Answer to Question 2 above).

However, Offerors are encouraged to submit a technical solution which creates best value for the

Government, so may propose an alternative schedule should it provide efficiencies and meet the stated goals of the RFP.

14Q. Is there an anticipated level of effort for CLIN 001, 002, 003, 004, and 005?

14A. See answer to Question 2.

15Q. Should the bidder assume that the reports produced in each CLIN are separate deliverables?

Specifically, are the reports of literature review findings in CLINs 0001 and 0002 separate deliverables?

15A. Yes, as written each CLIN is a separate task with a separate anticipated deliverable. However, Offerors are encouraged to submit a technical solution which creates best value for the Government, so may propose an alternative schedule should it provide efficiencies and meet the stated goals of the RFP.

16Q. In order to inform cost estimates for CLIN 0004:

16a. Can the government clarify whether the randomized control trails described on page 21 will be implemented under this contract? If yes, could the government provide any assumptions that could be used to estimate costs for the randomized control trials? For example, will these trails be built into the

NHANES administration or should these be designed as separate and stand-alone samples and studies?

How many trails are anticipated?

16a. Two types of trials can be designed: (1) Trials on non-NHANES participants as stand-alone samples and studies, and/or (2) Trials on NHANES participants. For the latter, NCHS will handle the implementation of the trial.

16b. Should we assume a full OMB submission and approval will be required to conduct the trials or will this request be submitted as part of a generic clearance?

16b. Both of these trials or tests would be submitted as part of a generic clearance.

17Q. Should we assume that the kick off meeting will be held in Atlanta, GA?

17A. No, The kick-off meeting will be held at NCHS which is headquartered in Hyattsville, Maryland. Any planned in-person meetings NCHS staff will occur at NCHS headquarters.

18Q. On p. 21 of RFP, CLIN 0004 is described as including “design and collect randomized controlled trials (RCT) to evaluate the effectiveness of these newly developed materials in increasing response rates”. There are specifications about obtaining ERB and OMB approvals and that the final report will summarize findings. To help us better assess the Statement of Work (SOW), please provide the following clarifications:

18a. Does this CLIN include the development of one RCT design, with submission and approvals of packages for both ERB and OMB, and collection and analysis of data during the specified period of performance (POP) of 6 months for CLIN 0004? If not, what is the SOW for the 6 months of

CLIN 0004?

18a. As indicated in Q16, Two types of trials can be designed: (1) Trials on non-NHANES participants as stand-alone samples and studies, and/or (2) Trials on NHANES participants. For the latter, NCHS will handle the implementation of the trial. Both of these trials or tests would be submitted as part of a generic OMB clearance. ERB packages will be developed with the Vendor. The POP estimate has been modified in the RFP to provide better guidance for proposals.

18b. How many RCT trials are expected (i.e., “trials” is plural in RFP)?

18b. The vendor can recommend the appropriate design and number of trials based on their experience. One or more trials can be conducted.

18c. What would be the expected sample size for each RCT group?

18c. The vendor can recommend the appropriate design and number of participants based on their experience.

18d. Would randomization occur at the respondent or MEC level?

18d. Randomization can occur in a stand-alone sample of non-NHANES participants or if the trial was part of the NHANES sample, it can occur on the primary sampling unit level, the segment level, or the household level, and not on the participant/respondent level.

19Q. For Similar Experience, can the contractor cite current data collection projects or projects that have finished data collection (e.g., response rate is known) but are not yet completed based on the POP of the contract?

19A. Yes, ongoing work may be cited for Similar Experience, but if it is, please provide a verifiable point of contact so that confirmation of project success may be acquired by CDC.

20Q. Which aspects of advance arrangements do you anticipate this evaluation covering?

20A. The advance post card, advance letter, hard-to-contact letters, appointment reminder letters, and other materials sent to NHANES participants.

21Q. Please describe the content and availability of paradata for this effort.

21A. No PII will be shared with the vendor. However, some aggregate paradata information can be shared. For example, we can provide information on numbers of hard-to-contact letters that were sent out in a year and the outcomes of households receiving the letters.

22Q. Please provide a description of the methods studies that NCHS has conducted to date, related to improving respondent cooperation. Please include results, impact of these studies, whether they are currently implemented in the field, and references to presentations or publications from these efforts.

22A. Four projects or materials may be relevant. Two are complete, one is ongoing, and one will be implemented within the next 12 months. All materials can be found on www.reginfo.gov.

The first completed project was related to some materials developed for NHANES and related focus group testing and can be found at: https://www.reginfo.gov/public/do/PRAViewIC?ref_nbr=201506-

0920-003&icID=230189

A “make a plan” postcard that was developed and deployed in the field to assist in reducing broken appointment. Information can be found here:

https://www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=201802-0920-009

Click on “Make a Plane” tracker 1 and tracker 2

The ongoing project is an NHANES incentive pilot. Information related to this project can be found here: https://www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=201809-0920-002

Please click on “Incentive Pilot Detail” for detailed information on the design of the pilot.

A social media pilot will be implemented in the next 12 months. More information can be found here:

https://www.reginfo.gov/public/do/PRAViewIC?ref_nbr=201708-0920-025&icID=233763

23Q. Please describe any methods studies related to respondent engagement, recruitment, refusal conversion, and participation that will be in the field during this work. Please provide references to presentations or publications from these efforts.

23A. See answer to Q22.

http://www.reginfo.gov/ https://www.reginfo.gov/public/do/PRAViewIC?ref_nbr=201506-0920-003&icID=230189 https://www.reginfo.gov/public/do/PRAViewIC?ref_nbr=201506-0920-003&icID=230189 https://www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=201802-0920-009 https://www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=201809-0920-002 https://www.reginfo.gov/public/do/PRAViewIC?ref_nbr=201708-0920-025&icID=233763

24Q. For CLIN 0003, can CDC describe the type and number of print, radio, and other media publicity pieces that will be evaluated?

Print: advance postcard, advance letter, hard-to-contact letters, etc. – up to 20 different materials.

Radio: standard radio script

NHANES ERB-approved protocol

NHANES gaining cooperation training manual

NHANES related brochures – up to 20 different brochures

Templates for letters of endorsement and press release – up to 10

25Q. For CLIN 0003, can CDC expand on what is meant by “outreach and recruitment materials and procedures” to provide descriptions of the content of each and the number of pages if these materials are in addition to the others listed in CLIN 0003?

25A. See answer to Q24.

26Q. CLIN 0003 describes materials the contractor will review and evaluate. For costing purposes, can you please provide the special population brochures, refusal letters, and any materials that are sent to participants with the remuneration?

26A. See the answer to question 24 for a list of materials that will be provided. For remuneration-specific materials, please note that currently remuneration is provided in-person after the completion of the physical examination in the mobile exam center (MEC) by the MEC manager.

However, we are currently conducting a remuneration pilot with print materials accompanying the remuneration. More information on the pilot and associated materials can be found here:

27Q. CLIN 0004 requests the contractor “design and conduct randomized controlled trials” for the new materials. It is unclear who will conduct/implement the RCTs. Will the contractor awarded under this

RFP be a) designing the RCT only, b) designing and analyzing data from the RCT, or c) designing, implementing the RCT in the field, and analyzing data from the RCT? If a) or b) will the current

NHANES field contractor, Westat, conduct the RCT as part of the existing field contract, as an option to their contract, or will someone else implement the RCT in the field?

27A. See answers to questions 16 and 18.

28Q. In CLIN 0004, is the randomized controlled trial separate from the observations specified in CLIN

0006?

28A. Yes, each CLIN represents a separate task with separate deliverables.

However, Offerors are encouraged to submit a technical solution which creates best value for the

Government, so may propose an alternative schedule should it provide efficiencies and meet the stated goals of the RFP.

29Q. For CLIN 0004, the RFP states that "The contractor shall provide assistance on NCHS ERB and

OMB approval as requested." Can CDC clarify the level of effort anticipated for this task? Also, can

CDC clarify the amount of time (calendar days) the OMB clearance process is anticipated to take from submission to receipt of OMB clearance?

29A. The level of effort depends on what will be proposed. However, all tests/trials/pilots can be submitted as part of a generic clearance. These clearances can be completed in 1-4 months depending on what is being proposed.

30Q. For CLIN 0004, to better estimate costs, can CDC specify the number of packages of new materials that will be required? For example, materials currently exist for elderly potential participants, Spanish-speaking, Asian Americans, and others which could require development of at least 4 different recruiting packages (general population, elderly, Spanish-speaking, and Asian Americans).

30A. Tailored materials for specific populations may be preferable. The four categories suggested above are appropriate. Additional population-specific materials may be needed and these will be determined after the review of current materials and findings from the literature on best practices.

31Q. on Page 22, under CLIN 0005, can CDC clarify if a separate literature review is required to

“identify best practices regarding gaining entry to, and cooperation of, respondents in locked buildings, gated communities, and other hard-to-reach addresses”?

31A. These communities have been especially hard-to-reach and enroll. Identifying best practices for this specific population is needed. The literature review requested here can be a detailed, thoughtful, and thorough expansion upon the review mentioned in CLIN 0001.

32Q. For CLIN 0006, which aspects of MEC operations do you anticipate this evaluation covering?

32A. No evaluation of MEC operations will be covered, however, recruitment to the MEC is covered under gaining cooperation. Additionally, it should be noted that the response rate drop off, or attrition, between the in-home interview and the MEC have been historically low and are generally around ~3 percentage points.

33Q. For CLIN 0006, the field observation sites selected are noted for cost purposes. After contract award you may choose alternative sites. Please describe the criteria that will be used for selection, and the desired characteristics of the final sites selected.

33A. Rural field observations sites versus Urban field observations sites, and the timing of the operations at a specific location (i.e. opening of operations, mid operations, closing of stand). The timing of operations is critical as it involves observing the spectrum of eager- to hard-to-enroll participants.

34Q. For CLIN 0006, the costing estimate, vendors are to assume 3 one week trips per PSU for a total of

9 trips. How many observation visits will be conducted per trip? How many observations of the field office should be made, both at the PSU level and at the home office? Do you have a minimum number of HH screening and HH interviews that should be observed?

34A. The number of observations per week vary and are dependent on participants’ schedules, appointments, and the timing of field operations at a site. We are unable to provide an exact number at this point. However, generally speaking, the first two weeks at any location are “high screening” weeks with about 150-200 screening interviews per week, and 20-30 in-home interviews. As time progresses, about 50-75 screeners are completed per week, and 50-100 in-home interviews. These are rough estimates.

35Q. For CLIN 0006, similarly, will any observations be overlapping (two on the same day in one week), requiring more than one team of observers?

35A. One team per site should be sufficient. However, there will likely be multiple observations in 1 day.

36 Q. CLIN 0006 / CLIN 0008 specifies that 3 trips should be estimated in the contractor’s cost. Does

CDC intend to modify this contract if additional trips are required?

36A. CLIN 0006 /CLIN 0008 request projected costs for 3 hypothetical site visits. Each site is projected to have 3 separate trips. That is a minimum of 9 total trips. The RFP does state that up to 6 sites, each site requiring up to 3 trips, for a potential total of 18 trips. Should any additional trips beyond the requested 3 be deemed necessary, the contract would be modified accordingly. The labor cost for each visit will be awarded as Firm-Fixed Price under CLIN 0006, the travel cost as Cost-Reimbursment under CLIN 0008.

37Q. Page 47 states, “Past performance will be evaluated for only those offerors remaining in the competitive range after completion of the review of the technical proposals using the 4 technical evaluation factors.” Only 3 technical evaluation factors are listed on page 44. Is there a 4th evaluation factor, and if so, can you please describe it?

37A. There are 3 technical factors; technical approach, similar experience, and management approach. Cost/price is the 4th factor, which is always a factor in evaluation of Federal contracts, but should not have been included in the description of past performance technical factors.

38Q. Page 44 states, “Selection of an Offeror for contract award will be based on an evaluation of proposals against four (4) factors. The factors in order of importance are: technical, past performance, management and cost.” Does past performance here mean the “Similar Experience” section of the technical proposal?

38A. On page 44, it should have read “similar experience” and not “past performance”. Past performance is a separate risk evaluation than the requested similar experience as a technical factor.

This ambiguity has been corrected in the revised RFP.

39Q. This effort is marked as a total small business set-aside and no limitations on subcontracting are noted. Can CDC confirm that there are no requirements regarding the percentage of work that must be performed by the small business?

39A. This is a total small business set-aside under FAR clause 52.219-6, therefore there is a percentage of the work that must be performed by small business, and that percentage is at least 50%, per FAR clause 52.219-14.

40Q. Will the contractor need to work with the Collaborating Center for Questionnaire Design and

Evaluation Research (CCQDER)?

40A. No, but NCHS may invite CCQDER team members to participate in this effort.

41Q. The answers to questions will potentially change a contractor’s technical approach and costing.

Will CDC provide a two-week extension and move the due date to March 21, 2019?

41A. With the desired start date of April 01, 2019, it is deemed that an extension to March 21, 2019 would be excessive and put timely contract award at risk. However, a one-week extension to

March 14, 2019 has been granted.

42Q. Section B, Page 2 - The RFP identifies CLIN 0006 as Firm Fixed Price and CLINs 0007 and 0008 as Cost Reimbursable. However, Section D, V. Work Breakdown Structure (pg. 21) states “All tasks are

Firm Fixed Price except CLIN 0006 & 0007, which are cost-reimbursable.” Can the Government please identify which section is correct?

42A. See answer to question 3 and 4.

43Q. Section B.2, Page 3 - On page 3, the RFP indicates the contract with be FFP, but in Section B.8 –

Period of Performance (same page), the RFP says “The period of performance shall be determined under each individual task order through the mutual agreement of the parties involved.” Does the government intend to issue an FFP contract or another type of vehicle where task orders will be executed for each

CLIN?

43A. This is a scrivener’s error. The RFP has been updated to state “each individual task at the line item level”.

44Q. Section D.5, Page 20-21 - To what extent will the vendor currently providing NHANES survey services be available to respond to questions, provide materials, and coordinate the scheduling of interviews where NHANES field interviewers will be observed?

44A. All communications related to the request of materials from the vendor currently providing

NHANES survey services, or related to scheduling, must go through NCHS staff. NCHS will coordinate the transfer of questions and answers, will provide materials, and will be in charge of coordinating field observations. For CLIN 6: communications with current field staff should be limited to understanding the current gaining cooperation protocol and to identify barriers and facilitators to participation. For example, the vendor can create semi-structured interviews to ask all field interviewers questions related to the protocol. Please note that the purpose of the field observations is to assess the effectiveness of current protocols used to recruit participants and gain cooperation. The scope of the observations is limited to this task.

45Q. Does CDC restrict use of cloud computing systems for work on this opportunity?

45A. Yes, cloud computing is restricted for this opportunity as no cloud computing is allowed currently by NHANES.