ATTACHMENT C-1 - Section C - DESCRIPTIONS-SPECIFICATIONS-STATEMENT OF WORK.pdf

ATTACHMENT C-1

Section C - DESCRIPTIONS/SPECIFICATIONS/STATEMENT OF WORK

FOR THE IDIQ

Contents Introduction

Background

History and Accomplishments

Objectives of this IDIQ

Clarification of Roles

Vision

Specific Tasks

Task Area 1: Quantitative Data Collection

Subtask 1.1: Survey Data Collection

Subtask 1.1.1: Online Surveys from a Nationally Representative Panel

Subtask 1.1.2: In-Person Surveys

Subtask 1.1.3: Telephone Surveys

Subtask 1.1.4: Leading edge data collection capacities

Subtask 1.1.5: In-Person Laboratory Research

Subtask 1.2: Human Attentional Data Collection

Subtask 1.3: Administrative Data Access

Subtask 1.3.1: Identify and Procure Administrative Data

Subtask 1.3.2: Identify and Obtain Retail Investment Account Administrative Data (optional)

Subtask 1.4: Field Testing

Task 2: Qualitative Research

Subtask 2.1: In-Depth Interviews

Subtask 2.2: Focus Groups

Subtask 2.3: Ethnographic Research (optional)

Task Area 3: Research-Related Technical Assistance

Subtask 3.1: Statistical Technical Support

Subtask 3.1.1: Survey and Experimental Statistical Support

Subtask 3.1.2: Mathematical Economics Technical Support

Subtask 3.1.3: Other Coding Support

Subtask 3.2: On-Site Research Support

Subtask 3.2.1: On-Site Research Assistants

Subtask 3.2.2: On-Site Data Analyst

Subtask 3.2.3: On-Site Statistician Assistance

Subtask 3.3: Short-Term Research Support Services

Task 4: Topical Subject Matter Expertise

Subtask 4.1: On-Site Long Term Technical Expert Program

Subtask 4.2: Short Term Subject Matter Experts

Subtask 4.3: Academic Consultant Support Services

Introduction This is a multiple-award Indefinite-Delivery, Indefinite-Quantity (IDIQ) contract to provide Quantitative Data Collection, Qualitative Research, Research-Related Technical Support, Program Management Support and Related Services for the U.S. Securities and Exchange Commission (SEC). The Office of the Investor Advocate (OIAD) is the business owner for this award and will be the primary user of the IDIQ; however, other offices within the SEC may use this vehicle. Upon the award of this contract, individual Task Orders will be issued for distinct scopes of work.

The research project will support the SEC’s research housed within OIAD, known as POSITIER (Policy Oriented Stakeholder and Investor Testing for Innovative and Effective Regulation).

POSITIER is a program of research unique in the Federal government in that it focuses on the collection of original multi-mode data, including experiments, with the desire to generate both policy and academic research outputs. POSITIER seeks to provide research that is at the frontiers of household finance, behavioral economics, and decision science. Moreover, POSITIER is unique in that it is largely an in-sourced model where contractors work hand-in-hand with OIAD Principal Investigators, who are principal authors of the research undertaken.

That is to say, that data collected and research conducted under this contract is largely to assist OIAD Principal Investigators and other SEC divisions in the execution of research projects that they have developed in order to fulfill the SEC’s mission and statutory mandate.

As the project is primarily a high-quality policy and academic investor research endeavor aimed at studying household finance and decision-making, as well as other issues and actors in financial markets, primary contractors are expected to demonstrate a professional capacity with such research. Tasks and subtasks outline specific capacity demands that successful contractors will be expected to demonstrate in contractor proposals and subsequent task orders. In a few cases, specific subtasks are identified as “Optional” – that is, they are preferred but not required.

Nevertheless, the SEC seeks to gain access to contractors that have these capabilities and encourages offerors to develop subcontracting arrangements that will allow for meeting these requirements.

Contractors will be expected to maintain high standards of excellence with appropriate quality control and an appropriate management structure to govern implementation of task orders stemming from this IDIQ. Additional requirements and instructions will be provided as part of awarded task orders.

Background

History and Accomplishments

Section 915 of the Dodd-Frank provisioned for the establishment of an Office of the Investor Advocate (OIAD) within the SEC. The Office was formally established on February 24, 2014, when the Commission’s First Investor Advocate was appointed. OIAD advances the interests of investors by: providing a voice for investors, assisting retail investors, studying investor behavior, and supporting the SEC’s Investor Advisory Committee. While the Investor Advocate reports to the Chair of the SEC, it has an independent role and has a mandate for direct reporting to Congress.1

The SEC is seeking several contractors to assist in the continued development and implementation of a program of data collection for use in monitoring developments in household finance, better understanding conditions in financial markets, improving regulatory effectiveness and improving performance and efficacy of SEC activities more broadly. After a contract award in late FY 2016, OIAD officially launched POSITIER in March 2017 at an evidence summit attended by over 12,000 participants from over 100 countries. POSITIER is not a contract and it is not a short-term project. Rather, it is an organizational architecture and organizational infrastructure for high-quality rapid cycle data collection for policy and academic research. OIAD has sought to create a capacity for meeting not only the requirements, but also the fullest spirit of legislation and executive actions that have aimed to improve evidence generation in government (even in instances where the SEC is not required to fulfill mandates in full, OIAD has sought ways to integrate best practices from these approaches into SEC policymaking),2 as well as the visions

1 See https://www.sec.gov/page/investor-advocate-landing-page for more information. The Dodd-Frank Act specifically outlined OIAD’s functions in Section 915: ‘(4) FUNCTIONS OF THE INVESTOR ADVOCATE.—The Investor Advocate shall— (A) assist retail investors in resolving significant problems such investors may have with the Commission or with self-regulatory organizations; ‘‘(B) identify areas in which investors would benefit from changes in the regulations of the Commission or the rules of self-regulatory organizations; ‘‘(C) identify problems that investors have with financial service providers and investment products; ‘‘(D) analyze the potential impact on investors of— ‘‘(i) proposed regulations of the Commission; and ‘‘(ii) proposed rules of self-regulatory organizations registered under this title; and ‘‘(E) to the extent practicable, propose to the Commission changes in the regulations or orders of the Commission and to Congress any legislative, administrative, or personnel changes that may be appropriate to mitigate problems identified under this paragraph and to promote the interests of investors.

2 See, in particular:

https://www.sec.gov/page/investor-advocate-landing-page articulated by think tanks and academia in this sphere.3 The initiative has sought to revolutionize the way the SEC conducts its policymaking activity, and it receives authority to do so from a host of executive orders and legislative reforms of the past decade.4 Under OIAD’s mandate for conducting investor research, it manages and conducts investor testing commensurate with determining the potential impact of SEC policies on investors.5

Over the past five years, OIAD has conducted extensive data collection using its existing IDIQ contracting vehicle, has articulated a vision for concrete reforms that would lead to greater use of evidence in SEC policymaking, and has provided a guiding light on innovation and evidence-generation to other agencies. As of December 2020, after about three and a half years of data collection, POSITIER had completed nearly 50 experiments in addition to 23 nationally representative probability-based surveys, 4 quota sample survey projects, and 9 qualitative research studies.6 This output is further remarkable because, excepting a round of high frequency surveys initiated last year, survey instruments and experimental designs have by-and-large been unique. This pace is in stark contrast to typical government data collection lifecycles for which each project would generally run some 18-60 months. OIAD has in-sourced major aspects of research undertakings in order to expedite work and maintain the fidelity of research.

The work that has been completed has been a partnership between academia and government as well as stakeholders within the SEC, contractors external to the SEC, and the broader investor protection community. While there are many tenants of POSITIER’s philosophy, one key component of the operational structure is a flexible standing capacity (with major elements embedded in this IDIQ) to enable the rapid and cost-effective deployment of high quality research tools. This standing capacity seeks to eliminate major bottlenecks in the evidence generation https://www.congress.gov/bill/115th-congress/house-bill/4174 https://obamawhitehouse.archives.gov/sites/default/files/omb/memoranda/2015/m-15-15.pdf https://obamawhitehouse.archives.gov/sites/default/files/omb/memoranda/2014/m-14-06.pdf https://obamawhitehouse.archives.gov/sites/default/files/omb/inforeg/memos/2015/behavioral-science-insights-and-federal-forms.pdf https://obamawhitehouse.archives.gov/the-press-office/2015/09/15/executive-order-using-behavioral -science-insights-better-serve-american 3 https://www.hamiltonproject.org/papers/predictive_analytics_rapid-cycle_evaluation_improve_program_outcomes 4 https://www.whitehouse.gov/the-press-office/2015/09/15/executive-order-using-behavioral-science-insights-better-serve-ameri can https://www.whitehouse.gov/sites/default/files/omb/memoranda/2013/m-13-17.pdf

5 Testing of the form articulated here is in fact specifically permitted and encouraged in the Dodd-Frank Act, under Section 912: (bolding of selected text is provided for emphasis) SEC. 912. CLARIFICATION OF AUTHORITY OF THE COMMISSION TO ENGAGE IN INVESTOR TESTING. Section 19 of the Securities Act of 1933 (15 U.S.C. 77s) is amended by adding at the end the following:

(e) EVALUATION OF RULES OR PROGRAMS.—For the purpose of evaluating any rule or program of the Commission issued or carried out under any provision of the securities laws, as defined in section 3 of the Securities Exchange Act of 1934 (15 U.S.C. 78c), and the purposes of considering, proposing, adopting, or engaging in any such rule or program or developing new rules or programs, the Commission may—

(1) gather information from and communicate with investors or other members of the public;

(2) engage in such temporary investor testing programs as the Commission determines are in the public interest or would protect investors; and

(3) consult with academics and consultants, as necessary to carry out this subsection.

(f) RULE OF CONSTRUCTION.—For purposes of the Paperwork Reduction Act (44 U.S.C. 3501 et seq.), any action taken under subsection

(e) shall not be construed to be a collection of information.

6 https://www.sec.gov/files/sec-investor-advocate-report-on-activities-2020.pdf https://www.congress.gov/bill/115th-congress/house-bill/4174 https://obamawhitehouse.archives.gov/sites/default/files/omb/memoranda/2015/m-15-15.pdf https://obamawhitehouse.archives.gov/sites/default/files/omb/memoranda/2014/m-14-06.pdf https://obamawhitehouse.archives.gov/sites/default/files/omb/inforeg/memos/2015/behavioral-science-insights-and-federal-forms.pdf https://www.whitehouse.gov/the-press-office/2015/09/15/executive-order-using-behavioral-science-insights-better-serve-american https://www.whitehouse.gov/sites/default/files/omb/memoranda/2013/m-13-17.pdf process. OIAD staff are widely regarded as thought leaders in the national policy community on issues of evidence and particular subject matters under study, and the model and ethos developed over the past several years have strongly influenced evidence work at other agencies, SROs and other entities domestically and internationally.7

Objectives of this IDIQ This IDIQ anticipates building on the accomplishments of the past several years and adding new capacities and flexibilities. The data collection program will include several main components:

quantitative research (often with embedded experiments); qualitative research; technical assistance and supplemental research and data support services. Some of these other services help to augment OIAD’s capacity, provide for the direct acquisition of vendor data, or seek to help ameliorate bottlenecks in the production process.

OIAD envisions that the next phase of POSITIER will continue to complement the SEC’s existing data resources as well as externally available public use data. While in initial stages, POSITIER focused largely on areas of immediate data scarcity at the SEC (in particular, surveys of households and individuals), more recently the data collection has focused on the use of administrative data, document analytics and integration of multiple modalities of quantitative data collection (for instance, the integration of survey, qualitative and administrative data). Future work may expand to work on other populations of interest to OIAD and the SEC, or building other research tools.

Moreover, while our experiments have largely been conducted using surveys, we are strongly interested in expanded use of field and laboratory experiments. Experiments, surveys and qualitative research methods are expected to remain mainstays of our work.

Contractors who work with OIAD on this IDIQ shall have skills and practices at the forefront of research and adhere to rigorous standards of data collection. For example, within our quantitative data collection task area, we anticipate continuing to collect high-quality survey data. In many cases this data is intended to ultimately provide data assets to the research community and public at large that are considered trustworthy, reliable and highly useful on par with other well-regarded public use micro data sets. In most cases, we tend to favor national probability sample surveys, but have found non-probability surveys useful in some circumstances. In addition, given that this contract may last for up to ten (10) years, for the Commission to remain at the leading edge, successful applicants shall be forward thinking in the development of new data collection, research and analytic tools. For example, were the research community to develop new modalities for survey data capture such as different delivery/contact methods, use of new technologies, or ways to measure individual behavior, the successful contractor shall acquire the capacity to deploy such approaches in a reasonable timeframe. As such, the contractor shall remain at the leading edge of research methods.

7 For more information about OIAD and POSITIER, applicants are strongly encouraged to read OIAD’s Annual Report on Activities (December) to Congress from 2018-2020, particularly the chapter on investor testing and the Investor Advocate’s statements.

Funding levels are anticipated in future fiscal cycles that would allow broadening the portfolio to complete additional research projects. The IDIQ arrangement provides a flexible procurement vehicle so that some initiatives can be lined up with SEC policymaking activity as topics of interests clarify over time. Applicants shall adapt to and incorporate new technologies and techniques as they emerge in the research community over the contract period.

Contractor capacity to deliver high-quality results with strong quality control procedures is paramount. Particularly with respect to data collection, interested contractors shall have a strong track record of obtaining statistically viable academic/scientific-quality data, as per for example, collection of widely-used public-use micro datasets.

OIAD is also seeking technical assistance components that will enable program execution. In terms of technical assistance, contractor teams shall have a strong bench of expertise to draw on in key areas (e.g., household finance, behavioral economics, labor economics, decision sciences, econometrics, survey methods, and qualitative research). Contractors shall also have methods for recruiting and bringing in specialized subject matter expertise in key areas when needed (e.g., but not limited to, ESG issues, wealth measurement, mutual funds, retirement accounts, discrete choice modeling, machine learning-based OCR, bespoke econometric estimation techniques); OIAD often expects such subject matter experts to augment OIAD’s substantial existing capacities and contractors shall leverage academic networks to fulfill these needs. At times, more generalized technical assistance will be employed by OIAD to deliver specific stages of the research production process (e.g., conducting literature reviews, conducting power analysis, writing reports, editing reports).

Clarification of Roles Contractors will be expected to work closely with OIAD staff on all aspects of implementation, including both project management and technical areas. This includes, for example, joint development of survey instruments and selection of survey sampling frames. The SEC may, at its discretion, in-source any stage of the research process (for example, the SEC may decide to provide a completely drafted experimental protocol or survey to the contractor for data collection and implementation).

Once COVID-19 restrictions limiting on-site personnel are lifted, for some components of this program, it will be necessary to provide OIAD-acceptable personnel on location at SEC headquarters in Washington.

OIAD’s Principal Investigator(s) will direct all technical aspects of all undertakings. A contracting officer within the SEC will support the Principal Investigators(s) and serve as the lead for resolving contract-specific issues. Deputy Principal Investigators may be nominated by contractors for specific undertakings, and at the discretion of the Principal Investigators, may be granted such status.

Information within the SEC is generally compartmentalized and access is typically granted only when appropriate and necessary. OIAD has special privileges in this regard as provisioned under Dodd Frank, and moreover, may be expected to use research to critique the work of other organizations within the SEC. The result is that, unless specifically directed by the Principal Investigator or authorized individuals, the contractors shall only communicate with or share data with the primary client or designated individuals within the commission. Contractors shall not assume that other parties within the SEC have the appropriate authorization for accessing data or information about any project undertakings. Due to the non-public nature of some research activities, research projects require a high degree of discretion.

Execution of this project requires access to potentially sensitive information, and as such the contractor must comply with all laws pertaining to collection and retention of sensitive personal information by the Federal Government. When collected, Personally Identifiable Information (PII) shall not be transmitted to the SEC.

For all activity, contractors will be responsible for receiving Institutional Review Board (IRB) approval, as well as meeting privacy requirements and providing informed consent to participants.

Vision The vision of POSITIER is to provide research, data assets and evidence that will allow the SEC to make information-centered decisions about potential policies as well as evaluate the effects of past policies that are enacted. On a more fundamental level, it will enable the Office of the Investor Advocate to fulfill its mission. The objective for this IDIQ is to provide the POSITIER research unit with tools to generate data, research and evidence, and augment POSITIER’s capacity to deliver on its mission.

Specific Tasks Capabilities that the SEC is seeking as part of this IDIQ are divided up into several distinct Task Areas. The SEC commits to an overall level of support for the IDIQ as a whole, but not for specific Task Areas. As the SEC’s needs grow over the life of the contract, contractors shall be able to adapt and provide support in any additional task areas that deal with research, qualitative and quantitative research, personnel support, technical support, staff augmentation, and any other areas that may be needed in support of executing the POSITIER program.

While subtask areas are enumerated separately, these task areas are best regarded as not completely distinct activities. That is, contractors shall be able to integrate multiple capacities simultaneously;

for example by integrating quantitative and qualitative data collection. The task areas identified below are those that exists today; however, the SEC envisions that additional task areas that fit within the overall scope and objectives of this requirement may be needed at some point throughout the life of the contract and contractors shall be able to address and support those task areas at the task order level.

Task Area 1: Quantitative Data Collection For this task area, the SEC’s objective is to access first-class data collection techniques and technologies that are at the leading edge of current practice and academic research. In particular, contractors shall be able to adapt to and incorporate new technologies and techniques as they emerge in the research community over the contract period. The SEC has a specific interest in integrating data from multiple sources to provide a more refined picture of household finance and financial markets – for example, the integration of survey, administrative and transactional data. As such, one of the SEC’s s objectives is to partner with contractors who have the ability to integrate data, e.g., integration of quantitative and qualitative data collection. The contractor shall have capacities that will allow for quantitative data to be collected on other populations of interest (e.g., portfolio managers, or institutional investors).

Subtask 1.1: Survey Data Collection All of the subtasks under Subtask 1.1 involve survey data collection (including surveys conducted online, in-person, via telephone, etc.). For all of the surveys described under Subtask 1.1, the contractor shall be able to embed experimental conditions into the survey instrument (i.e. conduct “survey experiments”) as this is an essential feature of the projects. In many cases these experiments may be confined to simple A/B testing, but in other cases there may be a desire for highly complex experiments (i.e. “lab-style experiments” implemented on a survey instrument; for example, behavioral choice experiments). The SEC considers that successful execution will include operationalizing experimental conditions, capacities to ensure that experimental conditions are adhered to during the study, and delivering incentive payments that may vary for particular respondents (e.g., by providing a bonus payment to respondents that have made certain choices in an experimental scenario). Although not expected to be the norm, the SEC may also be interested in delivering non-cash reward incentive payments to respondents such as financial assets (e.g., respondents may be given a choice between a cash payment, a gift card, or certain types of financial assets like a savings bond or mutual fund).

For all survey data collection projects completed under Subtask 1.1, raw data and cleaned response data will be returned to the SEC in a suitable format to be identified at the task order level. The SEC requires that contractors be able to clean data to remove any PII.

The SEC is interested in limiting potential spillover effects from one data collection exercise to another, as well as exploring how repeat participation may affect quantitative responses. As such, contractors shall have the ability to exclude participants from future studies (if requested by the SEC) or conversely, recruit participants repeatedly to examine responses over time. For all survey data collection projects completed under Subtask 1.1, contractors shall be able to track participation and non-participation using unique participant identifiers.

The SEC is interested in fielding nationally representative surveys using probability-based samples, as well as specialized surveys attuned to particular niche populations of interest (e.g., owners of particular types of financial assets, or a particular type of financial professional).

Contractors shall have the ability to employ non-probability based research when required (e.g., via non-probability panels, or using quota sampling methods).

The contractor shall be responsible for obtaining Institutional Review Board (IRB) approval for surveys and survey experiments. The SEC will be responsible for securing OMB approval for data collection when necessary.

For all subtasks under subtask 1.1, the contractor shall:

• Work directly with the Principal Investigators to develop a plan for data collection

• Develop a timeline and plan for execution of research projects

• Provide sufficient project management support to ensure successful execution of projects

• Integrate technical expertise (subject-matter expertise and/or methodological expertise) into project design and execution

• Identify an appropriate sampling frame for a given population of interest

• Obtain informed consent from survey respondents

• Establish data collection protocols and monitor implementation

• Conduct adequate pre-testing of survey instruments and modalities

• Deliver fixed and variable incentive payments to respondents per SEC specifications

• Develop sufficient materials to keep survey respondents informed and build confidence in the credibility, reliability, and confidentiality of all survey responses. This may include the development of informational brochures and/or websites for potential respondents to use to assess that the survey project is in fact an officially sanctioned project

• To obtain the highest possible survey response rate, the contractor will develop various contacting protocols to guide the research. The contractor will track each survey observation, whether ultimately interviewed or not, and provide sufficient data related to the execution of the survey.

• Appropriately document data, questionnaires, and data collection procedures, for potential release as a public use microdataset.

• Clean data of errors and PII

• Transmit data to OIAD in an acceptable format

• Conduct analysis of data as needed and directed by OIAD PIs

• Wherever possible, integrate such data collections with data collected in other task areas

• Deliver data according to pre-specified data delivery requirements8

8 These requirements are expected to include the following:

• Final datasets will be provided to OIAD including the following: sample weights, a cleaned data file, a searchable data codebook with frequencies and documentation, and documentation of data collection, all meta-data, and administrative data. OIAD requests data in .csv, .xls, STATA, SAS and R formats.

o Data on partial respondents and screened out respondents o Demographic data included, with date of collection o A unique panelist identifier o Randomization variables (when applicable) o Meta-data o Any PII MUST be removed o Copy of final implemented survey instrument, including skip logic (pdf document) o Codebook, checked to make sure it includes all variables in the data, in plain language o R code that may be used to replicate data cleaning, and other relevant steps in construction of the final dataset, as well as any analysis. The contractor will also provide R code with convenience functions that allow the user to: read in the dataset;

calculate estimates of simple statistics from the dataset with appropriate standard errors (mean, median, mode, and selected quantiles). All code shallshall be suitably commented. The contractor may rely on existing R infrastructure for completing this deliverable.

• A field Report containing the following information:

o Recruitment, retention, completion, and response rates o Sample quality breakdown o Field period o Description of any oversamples or special sampling o Description of how weights were estimated, and a description each weight provided (scaling, use with what population) o Incentive amount paid o Description of any data collection anomalies

Subtask 1.1.1: Online Surveys from a Nationally Representative Panel Under subtask 1.1.1, SEC’s objective is to conduct high-quality, nationally representative, probability-based online surveys that adapt appropriately to computer and mobile devices. The contractor will have an already recruited panel of survey respondents for which basic data has been obtained, and within which respondents are accustomed to survey participation in an online format.

The individual or household panel shall have been constructed so as to be nationally representative, including (but not limited to) address-based sampling panels.

The contractor shall have the capacity to transform a survey questionnaire instrument into a suitable online format, with associated methods for minimizing error and expediting completion of the survey (e.g., drop-down menus, auto-fill forms). In addition, the contractor shall have the capacity to collect background “meta-data” as part of the survey (e.g., whether the survey was completed on a computer or mobile device, time to complete, “click” data). Generally, the SEC has an interest in receiving data on partial completions.

The contractor shall have the capacity to target selected demographic (e.g., gender, age groups), economic (e.g., work status, high income or wealth, or potentially high income or wealth groups), geographic, and/or other groups for certain projects, with appropriate reweights for selected purposes. As such, OIAD seeks information on contractor capacities for conducting such targeted sampling or any constraints in their ability to do so. As part of this subtask, OIAD may also seek statistical support for targeting and weighting.

For subtask 1.1.1, the contractor shall fulfill all of the requirements listed immediately under subtask 1.1, and:

• Recruit survey respondents from an established panel, and merge new survey responses with existing background data

• Provide technical advice regarding sampling issues so that an appropriate sample may be drawn

• Program and pre-test survey questionnaires so minimize error and expedite survey completion

• Collect meta-data

• Target recruitment of selected respondents, including associated statistical support

• Develop respondent weights that allow for nationally representative point estimates and variance estimates, relevant to a particular population of interest.

Subtask 1.1.2: In-Person Surveys For certain research projects, the SEC may conduct in-person surveys of individuals, organizations or households or other populations of interest.

For subtask 1.1.2, the contractor shall fulfill all of the requirements listed immediately under subtask 1.1, and:

• Work from an established sampling frame to select a representative sample from the population of interest, given operational desires, and statistical specifications

• Recruit and hire survey enumerators, and train them on appropriate data collection methodologies

• Prepare technical hardware and software for data collection for use by enumerators, to expedite data collection and minimize errors

• Provide travel and logistical support to survey enumerators

• Provide for suitable health and safety protections for all survey enumerators.

• Spanish language capacity may be important for some surveys

Subtask 1.1.3: Telephone Surveys For certain research projects, the SEC may be interested in conducting telephone surveys of individuals or households.

For subtask 1.1.3, the contractor shall fulfill all of the requirements listed immediately under subtask 1.1, and:

• Identify an appropriate sampling frame for a given population of interest

• Work from an established sampling frame to select a representative sample from the population of interest, given operational desires, and statistical specifications

• Provide technical advice regarding sampling issues so that an appropriate sample may be drawn

• Recruit and hire survey enumerators, and training them on appropriate data collection methodologies

• To obtain the highest possible survey response rate, the contractor will develop various contacting protocols to guide the survey field research. The contractor will track each survey observation, whether ultimately interviewed or not, and provide sufficient data related to the execution of the survey.

• Prepare technical hardware and software for data collection for use by enumerators, in order to expedite data collection and minimize errors

• Spanish language capacity may be important for some surveys.

Subtask 1.1.4: Leading edge data collection capacities The SEC is interested in remaining at the leading edge of research and data collection methods and adapting and adopting new technologies and methods of data collection as they emerge. These approaches might include alternative delivery modalities for online surveys (e.g., SMS-based surveys, iPhone/Android-based apps, or “gamified” data collection methods). The SEC may also be interested in collecting data frequently, including daily or multiple times a day (e.g.,, diary studies, experience sampling methods), or attempts to integrate other types of data collection (e.g., biomarker data). SEC does not expect contractors to anticipate all possible research and technological innovations over the next few years. Instead, the contractor shall have the capacity to adapt, adopt new research methods, and integrate new methods into the research process.

For Subtask 1.1.4, the contractor shall fulfill all of the requirements listed immediately under subtask 1.1, and:

• Identify an appropriate sampling frame for a given population of interest

• Work from an established sampling frame to select a representative sample from the population of interest, given operational desires, and statistical specifications

• Provide technical advice regarding sampling issues so that an appropriate sample may be drawn

• Recruit and hire survey enumerators, and train them on appropriate data collection methodologies

• Establish data collection protocols and monitoring implementation

• To obtain the highest possible survey response rate, the contractor will develop various contacting protocols to guide the research. The contractor will track each survey observation, whether ultimately interviewed or not, and provide sufficient data related to the execution of the survey.

• Prepare technical hardware and software for data collection, in order to expedite data collection and minimize errors

• Provide travel and logistical support to survey enumerators, if appropriate

• Collect all data, clean data of errors, transmit data to OIAD in an acceptable format

• Spanish language capacity may be important for some surveys

Subtask 1.1.5: In-Person Laboratory Research The SEC is interested in performing behavioral laboratory experiments to study particular aspects of investor or household behavior. In such experiments, consistent with the extant academic literature that relies on such experiments, study subjects may be randomly allocated into selected groups and exposed to varying stimuli, or they may be asked to participate in a game or simulation that mirrors some real world activity. Under this subtask, the contractor shall have an available laboratory facility that can hold groups of participants for these studies (either directly or through a subcontractor relationship with an appropriate industry or academic lab).

In many instances, simulations that are required for in-person laboratory research may be implemented by exposing participants to a computer simulation or asking them to play a customized online game (see papers in experimental economics or political science for examples) in which participants will interact with each other. The contractor will provide support to implement such simulations in a way that minimizes data entry error. The contractor will have experience with developing lab experiments of this type. In other cases, in-person laboratory research may be used for individual decision-making tasks.

To accommodate group- or individual-level research, the contractor will have the capacity to deliver a wide range of coding services related to conducting experiments and simulations under this subtask.

For Subtask 1.1.5, the contractor shall fulfill all of the requirements listed immediately under subtask 1.1 and:

• Build customized software solutions as needed for the purpose of implementing experimental simulations in a laboratory or online setting, including the types of software available for group-level interactions (e.g.,, oTree, zTree) or individual decision-making studies (e.g.,, Qualtrics, Flash, etc.)

• Recruit participants for each study.

• Host groups of people for in-person laboratory research, including specifying details on:

o The number of workstations available in a given session o The types of workstations available for participants (e.g.,, iPads, desktop computers) o Methods for ensuring participants’ privacy o Methods for tracking participation and non-participation

• Conduct error checking or testing of computer code, including cleaning and validation

• Advise SEC staff on selected methods

• Integrate simulation code with other solutions

Subtask 1.2: Human Attentional Data Collection Under this subtask, the SEC’s objective is to collect information on visual and psychophysiological responses during decision making tasks or while viewing financial information. The SEC seeks to conduct these studies using best-practices in the field of attention research to generate both high methodological rigor and high substantive impact. While sample sizes compared to other methods may be smaller for these studies, samples shall be developed to rigorously test the focal research question. The SEC is interested in contractors’ capacity and experiences integrating attentional data with other qualitative and quantitative data, if applicable.

For subtask 1.2, the contractor shall:

• Work directly with the Principal Investigators to develop a plan for data collection

• Develop a timeline and plan for execution of research projects

• Provide sufficient project management support to ensure successful execution of projects

• Identify an appropriate sampling frame for a given population of interest

• Obtain informed consent from respondents

• Establish data collection protocols and monitor implementation

• Conduct adequate pre-testing of study stimuli

• Deliver fixed and variable incentive payments to respondents

• Develop sufficient materials to keep survey respondents informed and build confidence in the credibility, reliability, and confidentiality of all survey responses. This may include the development of informational brochures and/or websites for potential respondents to use to assess that the survey project is in fact an officially sanctioned project

• Appropriately document data, questionnaires, and data collection procedures, for potential release as a public use microdataset.

• Recruit and hire study administrators, and train them on appropriate data collection methodologies

• Design and program high power, high validity experiments, including both between- and within-subjects experiments

• Review academic and industry literature to operationalize focal constructs and develop tasks, procedures, and protocols that will maximize validity. This may include incentive compatible paradigms

• Conduct time sensitive stimulus presentation experiments, including:

o Accurate stimulus presentation, event coding, and response time measurement o High temporal resolution (>60hz) recording capability o Stimulus display software capable of advanced randomization procedures o Pupillometry measures o Participant calibration o Extraction, cleaning, and preprocessing data for export to general statistics packages (e.g.,, R, Stata, etc.)

• Gather survey data to be merged with visual data

• Create and use accurate area-of-interest regions for static and dynamic stimuli

• Collect visual data through mobile/wearable platforms (e.g.,, at home webcams, glasses, etc.). This could be in combination with data collection procedures from other subtasks

• Clean data of errors and PII

• Store, compress, and transmit data files to OIAD in an acceptable form

• Conduct analysis of data as needed and directed by SEC PIs

Subtask 1.3: Administrative Data Access Under Subtask 1.3, the SEC’s objective is to gain access to administrative data that may be of utility in conducting particular studies.9 Administrative data is typically generated for administrative (not research) purposes. Financial transaction account records, school records, social security or tax records may be examples of administrative data.10

For subtask 1.3, the contractor shall:

• Work directly with the Principal Investigators to develop a plan for data collection

• Work with the Principal Investigator(s) to determine an appropriate source of data

• Identify potential research uses for administrative data

• Assess the reliability of administrative data that are obtained

• Ensure the security of administrative data assets

• Seek to integrate administrative data sources into other data collection methods. For example, merging such data with survey data, or using administrative data to design a survey’s sampling frame.

Subtask 1.3.1: Identify and Procure Administrative Data The SEC’s objective is to identify and procure various administrative datasets on various topics, receive technical assistance identifying and using such datasets, and integrating such data with other SEC datasets (e.g., survey data). The contractor shall provide such data to the SEC and shall certify that they will not retain a copy for their own internal use unless the data are publicly available; the contractor is a sanctioned intermediary for providing the particular dataset and is providing the data from its own archives; or the contractor is the originator of the data. The contractor will ensure that any data with personally identifiable information is secured in appropriate manner.

Under Subtask 1.3.1, the contractor shall fulfill all the requirements listed immediately under Subtask 1.3, and:

• Have established relationships with potential administrative data providers for the collection of data particular to POSITIER’s research objectives (i.e. household finance).

9 https://obamawhitehouse.archives.gov/sites/default/files/omb/memoranda/2014/m-14-06.pdf. See also: Murray-Ryan Administrative Data Commission 10 See, for example: https://www.povertyactionlab.org/sites/default/files/documents/AdminDataCatalog.pdf https://obamawhitehouse.archives.gov/sites/default/files/omb/memoranda/2014/m-14-06.pdf https://www.povertyactionlab.org/sites/default/files/documents/AdminDataCatalog.pdf

• Have established methods of integrating such data with other data to be collected in this

IDIQ

• Remove PII and related data prior to transmittal to SEC

• Ensure for privacy and data security protections for all data collected

• Retain a copy of this data only insofar as it is a means of advancing SEC’s research needs and not for their own usage (excepting cases where the offeror or subcontractor is an originator of the data)

Subtask 1.3.2: Identify and Obtain Retail Investment Account Administrative Data (optional) Under Subtask 1.3.2, the SEC’s objective is to identify and procure data assets that use retail investment account administrative data. The contractor shall provide such data to the SEC and shall not retain a copy for its own internal use unless the data are publicly available; the contractor is a sanctioned intermediary for providing the particular dataset and is providing the data from its own archives; or the contractor is the originator of the data. The contractor shall ensure that any data with personally identifiable information is secured in appropriate manner. The SEC has a particular interest in data aggregators with detailed transactional and account information, and integration with other data sources collected as part of Task 1.

Under Subtask 1.3.2, the contractor shall fulfill all the requirements listed immediately under Subtask 1.3, and:

• Have established relationships with potential retail investment account data providers for the collection of data particular to POSITIER’s research objectives (i.e. household finance).

• Have established methods of integrating such data with other data to be collected in this

IDIQ

• Remove PII and related data prior to transmittal to SEC

• Ensure for privacy and data security protections for all data collected

• Retain a copy of this data only insofar as it is a means of advancing SEC’s research needs and not for their own usage (excepting cases where the offeror or subcontractor is an originator of the data)

Subtask 1.4: Field Testing Subtask 1.4 focuses on field research. The contractor shall facilitate and ensure successful completion of a variety of field tests of different forms. These could include tests that are primarily online/email-based, such as variation in messaging to investors, tests that require paper mailers, or tests that deliver structural changes to natural environments. Given the variety of field tests that may be possible, contractors shall have a variety of experiences and capacities to deploy different types of tests.

For subtask 1.4, the contractor shall:

• Work directly with the SEC Principal Investigators to develop a plan for data collection

• Develop a timeline and plan for execution of research projects

• Work with the SEC Principal Investigators to determine an appropriate arena for conducting a field test, and to facilitate a relationship with interested entities, if applicable

• Identify potential sources of data that are already collected, or easily collected, by interested entities, and that are amenable for statistical analyses

• Assess the reliability of administrative data that are obtained, if applicable

• Determine whether interventions shall be delivered via stratification or subgroups

• Ensure the fidelity and reliability of any interventions that are delivered

• Track “treatment” for the purposes of comparing Intent-to-Treat and Treatment-on-the-

Treated analyses

• Track and analyze spillover effects, if any

• Collect surveys or other custom data points on respondents, if requested

• Ensure the security and confidentiality of data assets associated with the field test

• Ensure participant confidentiality and privacy, including reducing any associated risks of participation

• Serve as an intermediary between OIAD and potential implementing parties, potentially providing a safe space for the sharing of research plans, data and analysis results

• Transmit data to SEC removed of PII

Task 2: Qualitative Research Qualitative research provides an important way to obtain supplemental information relating to survey responses, assisting with survey development, and augmenting SEC understanding of investor behavior, preferences, and attitudes.

For all of the subtasks under Task 2, the contractor shall be responsible for recruiting study participants and organizing all logistical aspects of the qualitative research (e.g.,, event space, travel accommodations, computer equipment, etc.). The contractor shall have established protocols for recruiting individuals that participate that are representative of key demographic, economic, geographic, or occupational groups that SEC may specify for a specific study. The contractor shall have the capacity to recruit representative research participants.

For all of the Subtasks under Task 2, the contractor shall:

• Work with the SEC PIs in design of the research endeavor

• Develop a timeline and plan for the execution of the research project

• Identify an appropriate sampling frame for a given population of interest

• Establish data collection protocols and monitor implementation

• Draft and pre-test research instruments

• Collect quantitative data on potential participants, where necessary, for screening purposes or to understand participants’ background

• Deliver fixed and variable incentive payments to respondents. In particular, OIAD may choose to offer variations on participant incentive payment structures. For example, participants may be offered payment types (e.g., cash, gift cards, in-kind gifts, financial instruments), payment based on task performance or other characteristics, and varying durations of payment. Such variations will help OIAD to study risk preferences and investment participation costs.

• Develop sufficient materials to keep participants informed and build confidence in the credibility, reliability, and confidentiality of all responses. This may include the development of informational brochures and/or websites for potential respondents to use to assess that the project is in fact an officially sanctioned project.

• Appropriately document data, questionnaires, and data collection procedures.

• Provide all transcripts.

• Utilize natural language processing capabilities (as appropriate) to analyze themes from participant transcripts

Subtask 2.1: In-Depth Interviews The SEC’s objective under Subtask 2.1 is to gain the capability to conduct individual, in-depth interviews (IDIs) using a structured or semi-structured interview protocol. SEC may desire that IDIs are conducted in multiple formats, including in-person, online, or over the phone. Contractors shall have experience building appropriate rapport with participants so as to elicit information about potentially sensitive financial topics.

For Subtask 2.1, the contractor shall fulfill…