ATTACHMENT L-2 Sample Task 1 (Disclosure) - Sections L and M 6-21-2021.pdf

ATTACHMENT L-2 – SAMPLE TASK 1 “DISCLOSURE” SECTIONS L AND M

Sample Task 1 (Disclosure) – Submission Instructions and Evaluation Criteria

Overview

This document describes the materials that Offerors must submit to demonstrate their capability to address the requirements of the Sample Task, and how the submitted materials will be evaluated by the SEC.

The SEC reserves the right to not award Sample Task 1 or to award Sample Task 1 to one or more Offerors as it deems fit as a result of proposals received in response to this solicitation.

RFP Section L.9.4 Submission Requirements for Sample Task 1:

An Offeror’s proposal must demonstrate the Offeror’s ability to provide the survey support services described in ATTACHMENT C-2 “Sample Task 1 Disclosure – Description” proposed to meet or exceed the requirements criteria, and represent the Best Value to the Government.

Tab A - Cover Letter, Table of Contents (Not included in Page count)

A cover letter shall accompany the proposal to set forth any information that the Offeror wishes to bring to the attention of the Government. The cover letter shall also stipulate that the Offeror’s proposal is predicated upon all the terms and conditions of this RFP. In addition, it must contain a statement that the Offeror’s proposal acceptance period is valid through 90 calendar days.

Information is to be presented at the summary level and shall only include general information about the proposal and the Offeror’s understanding of, and its capability to meet the requirements of the solicitation.

Tab B: Research Proposal and Technical Approach (10 page limit)

The Offeror shall demonstrate its knowledge of the Sample Task requirements as defined in ATTACHMENT C-2 “Sample Task 1 Disclosure – Description”. Specifically, the Offeror shall propose an approach to successfully performing all of the requirements in that Sample Task, across all stages of the research. This document shall be no more than 10 pages long, and shall describe research activities at all four proposed stages of the Sample Task.

Offerors shall identify and describe specific techniques, capabilities, and methods to complete each requirement of the Sample Task, and to demonstrate their organizational capability for conducting the work in a high-quality manner that is responsive to the SEC’s needs and interests.

At each anticipated research stage, Offerors shall describe one or more specific approaches that they can deliver (e.g., “participants will be recruited from the [name of survey panel]”) rather than providing general statements regarding a research approach (e.g., “participants will be recruited from a survey panel”). Offerors shall contextualize their response to the Sample Task bearing in mind the overall intentions of the IDIQ as a whole; in doing so they may, for example, need to add labor categories over the life of the project.

Contractors shall demonstrate a deep understanding of the context of POSITIER’s approach, the perspectives of OIAD within the SEC, and the technical needs of the research. Research proposals shall explicitly incorporate best practices of research methodology and design and thoroughly discuss pros and cons of the research approach. The proposal shall not simply regurgitate the description of the Sample Task, but demonstrate a deep understanding of the technical issues involved in a research project of the kind described in the Sample Task.

The SEC will evaluate the likelihood of successful completion of the Sample Task in light of Offerors’ relevant prior work, as identified in their submission to the IDIQ.

Tab C: Management Approach (5 page limit)

Contractors shall propose a management structure that will enable successful execution of the project. In addition to demonstrating that the requested competencies will be available, the description of the management structure shall appropriately incorporate expertise into the day-to-day management of the project. Important questions for contractors to answer include: How will the project be managed? How will the integrity of the research be preserved? How will subject matter and methodological expertise be brought into the project? How constant is the need for subject matter expertise? Can sufficient expertise be incorporated with short-term experts, or is long-term expert key personnel required? How will the contractor address needs for data analysis and programming? Will data analysts be expected at all stages of the project, even when analysis is not immediate (e.g., during the research design phase)?

Quality control considerations for the Sample Task shall be illustrated. Contractors shall describe a quality control process for ensuring all research and data collection is faithfully executed according to the specifications jointly developed with OIAD Principal Investigator(s). This includes the fidelity of data collection, faithful execution of randomization conditions, accuracy of transmitted data, inclusion of relevant data (including metadata, “click data”, etc.), and so forth. Offeror’s quality control considerations for the Sample Task shall be illustrated as part of the Research Proposal and Technical Approach document.

An Offeror’s ability to adapt to change is crucial to the success of this project. POSITIER research endeavors are inherently iterative in the sense that advanced stages of research are typically designed based on the findings of earlier stages. Contractors shall describe their organization’s ability to be flexible shall research results suggest deviations from the anticipated plan (for example, if initial qualitative testing requires additional rounds of qualitative study prior to moving to a quantitative data collection stage).

Tab D: Staffing - Key Personnel, Subject Matter Experts, and Technical Expertise (5 page limit not including CVs/resumes)

Key Personnel - Contractors shall submit the qualifications and background of key personnel (including CVs/resumes), with the goal of illustrating that they can bring appropriate expertise to the project to ensure high-quality execution. Key personnel must, at minimum, include a research lead, qualitative researcher, and a project management lead.

1. Research Lead: This individual is expected to be the primary point of contact for OIAD on research content, including participant sampling, survey and experimental design, and data analysis. Given that a Qualitative Researcher is identified separately, the expertise of the Research Lead shall be in quantitative methods. In particular, required competencies for this individual include one or more of the following: background in household finance, decision sciences, survey and experimental design, or econometrics.

The Research Lead is expected to have a minimum of eight (8) years’ experience leading social science research projects at a professional, post-Ph.D. level, including supervising teams that include scholars with graduate level training in the social sciences.

2. Qualitative Researcher: This individual is expected the primary point of contact for issues related to qualitative testing, as identified in Stage 1 of the Sample Task. In particular, this individual must have demonstrated experience in interview methods and other qualitative research. The Qualitative Researcher is expected to have a minimum of four

(4) years’ experience with in-depth interviewing.

3. Project Management Lead: This individual is expected to be the primary point of contact for issues related to contracting and procurement. The project management lead may or may not be the same individual as the research lead. The Project Management Lead is expected to have a minimum of five (5) years’ experience handling large, social science research projects that involve Federal Government clients.

Submitted CVs/resumes for key personnel shall be five (5) pages maximum for each individual.

Subject Matter Expertise - Contractors shall demonstrate an ability to acquire subject matter expertise as the subject matter of the Sample Task is identified and shall provide indicative CVs/resumes and letters of commitment. Sufficient subject matter and technical/methodological expertise shall be demonstrated to be incorporated into the project’s management structure to ensure highest quality research products. If subject matter experts are not existing, permanent employees of the contractors, contractors shall include letters of intent. These letters, if applicable, shall include confirmation that no conflict of interest exists that would preclude participation in an SEC research project.

Subject matter expertise relates to execution of a specific research project. To address this requirement, contractors shall seek to demonstrate expertise in areas listed in the “Sample Task 1

– Description” document. In particular, subject matter experts may be required in terms of financial products, disclosure design, and topical subjects that may arise about the inclusion of new information in disclosures (as per, for example, testing of Form CRS conducted in 2018, which would require knowledge of such topics as conflicts of interest with respect to investment advice, and disclosure effectiveness). The SEC is not looking for generalists with some familiarity in these subject areas, but rather for individuals that can contribute to the forefront of academic research on these topics and who have sufficient technical expertise to be informative in a policy discussion. Accordingly, subject matter experts could include a mix of academic experts and experts in operational aspects of the markets, products or assets.

Finally, contractors shall be mindful of the joint academic and policy needs of the research endeavor, and shall seek to recruit subject matter experts who can address both audiences.

Submitted CVs/resumes for subject matter experts shall be five (5) pages maximum for each individual.

Technical Expertise - There are at least two dimensions of technical expertise: methodological and subject matter expertise.

1. On the methodological front, the contractor shall demonstrate expertise in conducting quantitative and qualitative research of the nature discussed in the Sample Task, such as structured interviewing, participant sampling, survey and experimental design, data analysis and the preparation of academic submissions at top-tier academic journals in economics, decision sciences, econometrics, and/or public policy. Contractors shall demonstrate an ability to, or strategy for, acquiring additional research competencies as needed. These competencies are expected to be illustrated in contractors’ research proposals.

2. The SEC is interested in how subject matter expertise will be included in the management and operational structure of the project team, and how additional expertise will be brought into the project if needed. Contractors shall provide information regarding how the central requirements for this project that are expected to be filled by key personnel or specific subject matter experts. If applicable, contractors shall provide information on their past/previous experience working with those experts.

Price Proposal – Shall be submitted as part of Volume I; therefore, refer to Tab D of Volume I in Section L.9.1 of the solicitation.

RFP Section M. Evaluation Factors for Sample Task 1

There are four (4) factors that the SEC will consider as evaluation factors. The four factors below are listed in descending order of importance, with the most important factor in evaluation being the quality of the research proposal and technical approach (Factor 1). Factor 2 is more important than Factor 3. Factors 1 through 3 when combined are more important than Factor 4.

The Price factor is not weighted. The price factor may become more significant in contributing to the source selection decision if two or more proposals receive comparable evaluations in Factors 1 through 3.

Factor 1: Research Proposal and Technical Approach

The SEC will evaluate the Offeror’s proposed research proposal/ technical approach to completing the Sample Task to determine the Offeror’s ability to successfully complete the scope of the Sample Task Order.

Factor 2: Staffing

Offerors will be evaluated on the education level, relevant experience, and availability of key personnel and SMEs (i.e., is the proposed individual an existing, bona fide employee of the Offeror, or will employment of the individual be contingent upon award of the contract?).

Sufficient subject matter and technical/methodological expertise is required and will be reviewed to ensure the individuals possess the appropriate level of education and relevant experience to ensure the necessary technical positions can be filled by the Offeror and are incorporated into the project’s management structure to ensure highest quality research products. The government will review the Offerors level of existing technical expertise and if applicable, the previous work done with these personnel.

Factor 3: Management Approach

The SEC will review the Offeror’s response to ensure the management structure is rational and appropriate for successful completion and management of the Sample Task. The government will review the Offeror’s ability to develop and implement a comprehensive quality control process. The government will also review the Offeror’s flexibility/ability to adapt to change.

Factor 4: Price

Each Offeror’s price proposal for Sample Task 1 will be evaluated, but will not be assigned a rating. The evaluation of the sample price proposal will include accuracy, completeness, and reasonableness.

Note: Failure to offer ceiling rates for all labor categories and all contract periods will result in offer ineligibility. This means that the omission of a rate for just a single category will result in a material non-conformity in the proposal.