Amendment_1_-_Questions_from_Offerors_-_final.pdf

Solicitation # 19-223-SOL-00022: Multi-Method Consumer Perception Investigations Amendment 1

Technical Questions (Program Office):

1. (REF C.4 of the RFP) Does FDA anticipate conducting mail surveys? Telephone surveys? Or just online and in-person surveys?

FDA anticipates conducting online and in-person surveys.

2. (REF C.4 of the RFP) The government refers to stimuli in multiple functional areas. Can the government elaborate as to whether these stimuli are existing products that will require procurement and use within the respective functional areas, or will the contractor be required to generate/create previously non-existing stimuli. If the later, can the government provide an example of the type of stimuli that may be requested for generation/creation?

There is potential for either approach to be utilized. FDA shall provide either stimuli examples or actual stimuli (e.g. tobacco products and/or their labeling and/or advertising) to be used in the copy tests in the information sheets. After receiving the information sheet from FDA, the Contractor shall locate and purchase each stimulus within U.S. commerce.

Most of the stimuli will be created from existing products and/or their labeling and/or advertising. If/when the contractor is required to generate non-existing stimuli, the stimuli will be tobacco products, their labeling and/or advertising. The format may be electronic images or physical products/labeling/advertising produced for in-person use.

3. (REF C.4 of the RFP: First bullet under Functional Area 1 & 2: Copy Tests and Focus Groups) Does stimuli refer to only tobacco products or does it also include things like sample warning messages, tobacco prevention campaign messages and materials? Will FDA provide all stimuli or will contractor be responsible for creating or editing stimuli?

Stimuli refer to tobacco products, their labeling and/or advertising. See response to Question 2 of this Section.

4. (REF H.7 of the RFP: 508 compliance) Given that no products will be delivered prior to award, does the offeror have to complete a Product Assessment for the proposal? If so, what product should be selected for assessment?

Given that no products will be delivered prior to award, the offeror should not complete a Product Assessment for the proposal.

5. (Reference Pricing Worksheet/Proposed Price List Tab) What ages are FDA using to define young adult and adult? Is FDA considering targeting youth under 18 and if so what is the minimum age?

For this contract, FDA is referring to persons 18-24 years of age as young adults and those 25 years of age or older as adults. The specific relevant requirements will be defined at the task order level.

6. (REF C.4 of the RFP) RFP Section C.4, Functional Areas, states that “the Contractor shall provide copy test services that may include stimuli construction, sample selection, and data collection on a national basis (with the capability of collecting data via internet).” Do you anticipate that some copy tests will have to be conducted by means other than the internet, e.g., in person using physical materials? If so, can you specify the number of copy tests to be done in person?

Under this section, FDA only anticipates conducting online copy tests.

7. The parameters for the focus group ODC costs stipulate: “Focus Group (20 total - 4 per year), 4 locations, 8 focus groups per location, segmented by young adult vs adult, urban vs rural, for two focus group per group (young adult urban, young adult rural, adult urban, adult rural) per location.” Can the Government confirm that “4 per year” means 4 sets of focus group per year, with each set occurring in a different geographical location, and each set consisting of 8 separate focus groups, for a total of 32 actual focus groups held per year?

Do you expect that a total of 4 distinct geographical locations will suffice for the entire 5-year period of the contract, or that there may be 4 different locations each year, or up to 20 different locations over the 5 year period? Do you have any expectation or requirement for the geographical dispersion of these locations, e.g., can more than one location be within commuting distance of the same urban area? For purposes of costing the focus group ODCs, should offerors assume that all the focus groups will be done in person; if not, can you specify the number of groups to be done in person and the number to be done by electronic means such as via internet?

FDA assumes 4 per year to mean 4 sets of focus groups. It is anticipated that all focus groups will be conducted in person. While these requirements will be defined more explicitly at the task order level, given that the travel NTE amount has been set at a high level, we are expecting that this project will include focus groups in different regions of the United States.

However, the specifications on the Pricing Worksheet are provided so that all offerors will provide pricing for the same work.

8. The parameters for the in-depth interviews include 9 total—2 per year (years 1, 3 ,4, 5) and 1 per year (year 2); 4 locations; and 10–12 interviews per location. Can you confirm that “9 total” means 9 sets of interviews and each set consisting of 10-12 interviews? Do you expect that a total of 4 distinct geographical locations will suffice for the entire 5-year period of the contract, or that there may be 4 different locations each year, or up to 20 different locations over the 5-year period? Do you have any expectation or requirement for the geographical dispersion of these locations, e.g., can more than one location be within commuting distance of the same urban area? For purposes of costing the in-depth interview ODCs, should offerors assume that all the interviews will be done in person; if not, can you specify the number of interviews be done in person and the number to be done by electronic means such videoconferencing, as allowed for in RFP Section C.4, Functional Area 3?

FDA assumes 9 total to mean 9 total sets. Each set consists of 10-12 interviews per location for a total of 40-48 interviews per set. There may be different locations for each set of in-depth interviews. In other words, set 1 may consist of 4 locations that are not the same as the locations required for set 2. The type of product being studied will determine the locations needed and are normally not within commuting distance. FDA anticipates that all in-depth interviews will be conducted in person. While these requirements will be defined more explicitly at the task order level, given that the travel NTE amount has been set at a high level, we are expecting that this project will include in-depth interviews in different regions of the United States. However, the specifications on the Pricing Worksheet are provided so that all offerors will provide pricing for the same work.

9. RFP Section M, Evaluation Criteria, Section III, Management/Staffing Approach, states that “The Government will evaluate the extent to which the Offeror’s proposal demonstrates a sound … Approach for communication and collaboration with stakeholders.” The RFP does not identify stakeholders. Can you identify the specific stakeholders for purposes of the work to be done under this contract?

For this contract, FDA identifies stakeholders to include the government, the contractor, and any subcontractors.

10. Can FDA clarify if there are any online streaming or in-person observation requirements for either the focus group, IDI, or eye-tracking tasks?

FDA would like both online streaming and in-person observation for focus groups, IDI and eye-tracking tasks.

11. The RFP says: “Create an article database to house all full text articles retrieved for literature reviews. The database shall be housed on contractor servers, and transferred to FDA servers upon contract completion.” For these literature reviews, will FDA require delivery of full-text copies of all articles evaluated for relevance, or just articles deemed relevant and included in the review?”

FDA requires delivery of the articles deemed relevant and included in the review.

12. Are the entirety of the copy test sessions in-person or can they be virtual?

FDA anticipates all copy tests to be conducted online with stimuli presented in a virtual/online environment.

13. Would the contractor be creating the 3-D images for the copy test sessions or would they be provided by FDA?

FDA anticipates the contractor creating the 3-D images for the copy test sessions.

14. Do you mean that for each year there would be a total of 6 copy test studies each consisting of 3,200 copy test sessions for a total of 19,200 copy test sessions?

For 6 copy tests, the 3,200 refers to participants. However, the specifications on the Pricing Worksheet are provided so that all offerors will provide pricing for the same work. The actual number or copy tests and copy test sessions will be specified at the task order level.

15. Do you mean that for each year there would be a total of 4 focus group studies each consisting of 32 focus group sessions for a total of 128? (4 locations, 8 focus groups per location = 32 focus groups per study, 32 groups by 4 studies per year = 128)

The specifications on the Pricing Worksheet are provided so that all offerors will provide pricing for the same work. The actual number or copy tests and copy test sessions will be specified at the task order level.

16. Do you mean that for each year there would be a total of 2 in-depth interview studies each consisting of 40-48 interviews for a total of 80-96 interviews? (4 locations, 10-12 interviews per location = 40-44 interviews per study, 40-44 interviews by 2 studies per year = 80-96 interviews)

See response to Question 15 of this Section.

17. (REF Section C.3 of the RFP) Will FDA's "desired target population" always consist of Adults legally able to obtain tobacco and nicotine-containing products?

See response to Question 5 of this Section.

18. (REF Section C.3 of the RFP) Will FDA RIHSC only consider IRB Authorization Agreements for Copy Test tasks? All other functional areas require that the coordinator engage FDA RIHSC to obtain prior approvals.

FDA would consider IRB Authorization Agreements for all tasks that require human subject’s protection.

19. (REF Section C.3 and Section K.3 of the RFP) Will the FDA RIHSC review or otherwise rule if the IRB of record rules that the task is exempt from human subject regulations per 45 CR 46.104?

Yes.

20. (REF Section C.3 of the RFP) Will FDA RIHSC consider a single IRB Authorization Agreement to capture the scope of all Copy Test tasks awarded under the contract?

FDA RIHSC will not consider a single IRB Authorization Agreement to capture the scope of all Copy Test tasks awarded under the contract.

21. (REF Section K.3 of the RFP) Section K.3(c) states that "activities which involve human subjects in one or more of the categories set forth in 45 CFR 46.101(b)(1)-(6) are EXEMPT from complying with 45 CFR part 46. Should the citation be updated to be 45 CFR 46.104 (d)(1)-(8), per the 2018 Common Rule?

The citation referenced is current as of the most recent version of the HHSAR, which was last revised in December 2015. While the common rule has since taken effect, to change this provision would require a deviation. We are working with the appropriate personnel to determine what changes are necessary to conform with the Common Rule and will incorporate once the appropriate language and corresponding deviation has been approved.

22. (REF Section K.3 of the RFP) Section H.10(c) states that "contractors involving other agencies or institutions in activities considered to be engaged in research involving human subjects..." Can you confirm if Section H.10 should have the word "Non-Exempt" in front of the word "research?"

The referenced language is the current HHSAR clause. To change this clause would require a deviation. We are working with the appropriate personnel to determine what changes are necessary and will incorporate once the appropriate language and corresponding deviation has been approved.

23. (REF Section H.9 (A.8) of the RFP) Contract Documentation references Appendix D for baseline deliverables. Can FDA provide the link to this Appendix D?

Appendix D is a post-award security requirement. Offerors should not include it in their proposals.

24. (REF Section H.9 (G.1) of the RFP)

Can FDA provide a reachable link for the Enterprise Performance Life Cycle (EPLC) Project documentation listed on page 30?

This is a post-award requirement. No external link is available.

25. (REF Section H.9 (G.6) of the RFP) Can FDA provide a reachable link to the FDA eDepart system listed on page 30?

This is a post-award requirement. No external link is available.

26. (REF Section H.9 (A.3) of the RFP) With respect to H.9.A.3, can FDA please clarify the types of data FDA intends the requirement for FISMA Moderate (with Privacy) compliance to cover?

This requirement covers all human subjects data and all other data that contains personally identifiable information (PII).

Proposal Format Questions:

1. (REF K.3 and K4 of RFP) Is an active FWA assurance number documented in a letter and acceptable Assurance of Compliance?

This would be an acceptable form of complying with this requirement. Please be sure that the FWA is active and that it is listed under your company’s name. If it is listed under another name (such as a “doing business as” (DBA)) please be sure to make note of that in your proposal.

2. (REF K.4 of RFP) Do proposed research protocols have to be approved by the offeror’s IRB prior to award?

Because the specific research requirements will be defined at the task order level, and in some cases developed as part of the task order, it is not expected that the research protocols would be approved by the IRB prior to award.

3. (REF B.4 of RFP) The government provides labor categories that are anticipated throughout the life of the contract. Are offerors limited to these labor categories when responding to Task Order issued under the contract?

To evaluate and compare like proposals, price totals will be calculated in accordance with Section M.1 (V) of the solicitation by multiplying the government provided hours by the proposed rates associated with those labor categories only. However, offerors may propose other labor categories and shall include hourly rates associated with those categories, so the rates can be evaluated for price realism and reasonableness.

4. (REF L.2 Volume II Management #4) Please clarify if the government requires resumes for just key personnel, or for all personnel.

If all personnel, are representative resumes acceptable where multiple employees in a labor category may be assigned to the project?

Detailed resumes for labor categories designated by the government as key personnel shall be provided as well as resumes for individuals identified by the offerors as integral parts of their proposal and whose expertise is expected to be considered a strength of their proposal. Representative resumes may be provided for labor categories where multiple employees might be considered for the project (i.e. more junior technical staff).

This was not the intent and resumes should only be provided for key personnel.

Accordingly, Paragraphs 4 and 5 of Section L.2 Volume II Management/Staffing Approach are revised as follows

4. The offeror shall provide its staffing plan and identify key personnel, integral to the performance of this requirement. The staffing plan should consist of individuals with the necessary education, experience and skills necessary to complete the work of the SOW and who meet the minimum education/experience requirements set forth in Attachment 1.

Offerors shall submit resumes of all key personnel proposed, but (resume length should be limited to 2-3 pages) for all non-technical personnel. Resumes should not only reference academic qualifications, but also length and variety of experience in performing similar tasks, and relevant training experience. The offeror shall explain how the technical skills and qualifications of staff remain current and relevant.

5. The offeror shall demonstrate that qualified resources are available at the start of and throughout the period of performance. The offeror shall clearly indicate whether the resumes submitted are those of current staff (prime or sub-contractor) or are identified to be hired and shall provide letters of commitment for key personnel and sub-contractor personnel.

5. (REF RFP page 40) Is the government open to removing 52.222-41 Service Contract Labor Standards if all contract proposed personnel are bona fide executive, administrative, or professionals?

The clause was not selected because all personnel are expected to executive/administrative/professional. FAR 52.212-5 includes those clauses by default, but unless they are checked they do not apply to the contract.

6. (REF Attachment 2 – Pricing Worksheet) It is helpful that the government provided estimates of the overall ODC and travel budgets associated with the contract. Will the government consider providing estimates of the travel and ODC costs associated with each of the activities that it provides parameters for?

Namely an estimate of travel and ODC costs for each of the following: Copy Test, Focus

Group, In-depth Interview, Narrow Literature Review, Comprehensive Literature Review, and Eye Tracking Studies?

The parameters for the individual functional areas have been provided in the attachment but will be addressed more specifically at the task order level. While the plug-in value for ODCs and Travel will be applied for the purpose of comparing like proposals during source selection, per Section L, Volume III and Section M.1 (V) the government is interested in seeing your individual proposed ODC costs as a means of determining your understanding of the requirement. It would also be helpful to know if the offeror has an approved purchasing system in accordance with FAR 44.3 (documentation confirming the approved purchasing system shall be provided if the offeror indicates they have one).

7. There is no project labor category for a graphic designer, and yet stimuli development, which will include graphic design, is a major component of several of the tasks that are likely to come out under this RFP. Should we add a graphic designer category?

See response to Question 3 of this Section.

8. Other than the three past performance questionnaires, Section L.3, Proposal Instructions, does not make any provision for the offeror to present information about the full extent of organizational experience, capabilities, and capacity that are relevant to the diverse activities and functional areas covered by the scope of work. While three past-performance projects will provide some of this information, there are likely to be a number of relevant experiences and capabilities that would not be covered by three projects, particularly non-personnel resources that the Offeror can provide. How would you like such information to be included in the proposal?

In accordance with Section M.1(IV) of the RFP:

The Government will evaluate the extent to which the Offeror’s proposal demonstrates recent (within the last five (5) years) and successful performance of work similar in scope and size (dollar amount) to the requirements set forth in the statement of work.

Offerors are cautioned that the Government may use data provided by each offeror as well as data obtained from possible other sources such Government past performance databases (e.g. The Department of Defense’s (DOD) Past Performance Information Retrieval System (PPIRS)) in the evaluation of past performance. Offerors are urged to verify accuracy of all data provided. Past Performance information regarding predecessor companies, key personnel who have relevant experience, and subcontractors that will perform major or critical aspects of the requirement may be provided as past performance submissions. The Government is not responsible for ensuring that a sufficient number of questionnaires are submitted.

The Government will evaluate the three references provided with the questionnaires and expect that to be sufficient information to adequately evaluate past performance but does reserve the right to evaluate other information.

9. RFP Section L.2, Proposal Instructions, Volume II – Technical, Management/Staffing Approach, #7, states: “If required, the Offeror shall submit a subcontracting plan (see Attachment 4).” Please confirm that any subcontracting plan may be included as an attachment to Volume II and does not count in the 30-page maximum for Volume II. To estimate a contract value for the small business subcontracting plan, can offerors assume that the plug-in estimated expenses for non-labor other direct costs and travel ($6,500,000 and $500,000, respectively) set in Section B.4 will be equally distributed across all 5 contract years?

It does not count against the 30-page maximum for Volume II and can be included as an attachment. Also, for the purpose of the subcontracting plan, it can be assumed that the plug in values are evenly distributed across all 5 years.

10. The proposal instructions for Volume III – Price state: “Plug-in, Estimated Expenses for Non- Labor Other Direct Costs and travel set in Section B.4, at $6,500,000.00, and $500,000.00 respectively and will cover the entire duration of the contract. These expenses will be applied at the task order level and proper documentation in accordance with FAR 31 will be required to support any expenses. Offerors shall include these expenses, as is, as a part of their total overall price. However, for the purpose of evaluation, the offeror shall include pricing data adequate to establish the realism and reasonableness of the estimated ODC costs (i.e. per unit costs, applicable indirect rates [and audited rate agreements], fee, material handling rate, etc.). Offerors should consider the parameters described in Attachment 2 – Pricing Worksheet in order to demonstrate the anticipated ODC costs necessary to perform the work of this contract.”

In addition to the plug-in values for ODCs, Attachment 2 – Pricing Worksheet lists a large number of parameters for ODC costs for each of the functional areas. Should offerors include in this price proposal the prices for all the ODCs associated with the copy test, focus groups, in-depth interviews, narrow literature reviews, comprehensive literature reviews, and eye tracking studies listed at bottom of the second worksheet in Attachment 2, or will offerors price those at the time of task order proposals?

At the time of the Task Order proposal.

If those ODCs should be included in our price volume, where should those items be presented (in the Notes and Assumptions worksheet of Attachment 2, at the bottom of the Proposed Price List of Attachment 2, or elsewhere)?”

Attachment 2 to be replaced with “Amendment 1 – Revised Attachment 2 – Pricing Worksheet.” Itemized ODC costs should be listed under the Non-Labor Ceiling and if any other notes or assumptions need to be provided, they should be provided in the Notes/Assumptions tab or in a separate Budget Narrative Document.

If these ODC costs are to be fully detailed, can you provide further guidance on what format this information should be provided to enable the Government to assess “the realism and reasonableness of the estimated ODC costs”?

Examples of supporting information would be information about the basis of unit price estimates and quantity estimates (i.e. catalog pricing, actual quotes, narrative of assumptions used in arriving at the estimated quantities and per unit value, narrative linking the ODC to the specific functional area/task it supports).

If these details are to be provided now, can the Government confirm that, as the $6,500,000 ODC and $500,000 travel costs are estimated plug-in amounts, there is no expectation that the detailed ODC pricing data will sum to exactly these amounts?

The plug-in values are intended to allow for comparison of like proposals and to ensure offerors do not propose unrealistically low pricing in this area for the purpose of having a more favorable price proposal. The individual ODC prices do not need to be sum up to these values exactly, as the individual prices will not be included in the resultant award, just the plug-in value. See also answer to question 6 from this Section.

11. Would a completed CPARS be acceptable as an alternative to completed past performance questionnaire for project references?

For the purpose of providing consistency in evaluation, offerors shall ensure their references submit the past performance questionnaire before the proposal due date.

12. Can FDA clarify if there are any restrictions in the Volume II – Technical submission for font style, font size, or margins, and if these requirements would differ for graphics or tables?

A page is defined as 8.5 X 11 inches, single- spaced, with one-inch margins in type no smaller than 12-point font. Font sizes for graphics and tables can be smaller, but offerors should use caution not to reduce sizes so much that the graphics or tables are not easily readable.

13. Is there a format or template that should be used to provide the detailed unit cost for the estimated ODCs that are required for evaluation purposes and outlined in Attachment 2?

As stated in question 10 of this section, Attachment 2 to be replaced with “Amendment 1 – Revised Attachment 2 – Pricing Worksheet.” Itemized ODC costs should be listed under the Non-Labor Ceiling and if any other notes or assumptions need to be provided, they should be provided in the Notes/Assumptions tab or in a separate Budget Narrative Document.

14. Would the FDA allow submission of CPARS evaluations in lieu of past performance questionnaires?

See response to Question 11 of this Section.

15. FAR 52.219-4 indicates that there is a price evaluation preference for HUBZone small business concerns. Will the government consider including a similar preference for Small Disadvantaged Business concerns?

As prescribed in FAR 19.1309(b), the clause at FAR 52.219-4 is required to be included in full and open competitions. There is not a specific FAR requirement for a similar price preference to be provided to any other small business classification. Therefore, it is the only price preference the government is considering for this award. However, in accordance with 52.219-4(b)(ii) this pricing preference will not be applied against “otherwise successful offers from small business concerns.”

16. We anticipate that additional labor categories will be needed. Are these categories allowed to be included into the proposal? If so, how would these categories be evaluated from the price proposal perspective? Is there additional information that would need to be included for additional labor categories?

17. Where do we submit pricing for assumed ODCs for evaluation purposes? The pricing worksheet does not include an area to input pricing for itemized ODCs. Should we create a separate sheet with pricing in addition to the assumptions worksheet?

See response to Questions 10 and 13 of this Section.

18. (REF Section L of the RFP) Are there any file size restrictions for the email submittal?

Though larger files are acceptable, it is advisable to keep the file size below 20MB if possible in order to avoid any undue delays in getting the attachment through the security screenings at the Government infrastructure. The onus is on the contractor to ensure their proposal is received timely (see FAR 52.212-1(f) for more information regarding late proposals).

19. (REF Section C.5 of the RFP) Can additional labor categories be proposed after award?

20. (REF Section L.2, Management and Staffing Approach of the RFP) Line item (7) refers to a subcontracting plan (Attachment 4). Is the subcontracting plan excluded from the technical proposal page restrictions?

See response to Question 9 of this Section.

21. (REF Section L.2, Management and Staffing Approach of the RFP) Can the Offeror submit pdfs as an acceptable format for proposal documents?

PDFs are an acceptable format for proposal documents.

22. (REF Attachment 2 Pricing Worksheet) Attachment 2 (Pricing Worksheet) will be completed for the IDIQ. Should pricing for parameters described in Attachment 2 be provided in separate Pricing Worksheets by functional area?

Pricing for the individual functional areas will be provided at the task order level. See also answer to Question 10 of this Section.

23. (REF Section L.2, Management and Staffing Approach of the RFP) Would it be acceptable to provide letters of commitment from a subcontractor’s organization as opposed to subcontractor personnel/individuals?

Letters of commitment are required for individuals being proposed as Key personnel (whether prime or subcontractor employees). See also response to Question 4 of this section.