ATTACHMENT C-3 Sample Task 2 (Survey) - Description AMENDMENT 00001.docx

ATTACHMENT C-3 – SAMPLE TASK 2 “SURVEY” DESCRIPTION

Sample Task 2 “Investor Survey”: Description

Sample Task Order Tasks: For Pricing purposes assume a 12 month Base Period and four nine 12 month long Option Periods.

Overview This document describes a Sample Task. The SEC reserves the right to not award Sample Task 2 or to award Sample Task 2 to up to one Offeror at the award of the IDIQ or shortly thereafter, using the assumptions and proposal prepared in response to this sample task.

Offerors shall prepare a proposal and cost estimate that meets the task as described, but can discuss concerns with the task description proposed and suggest alternatives.

Background:

This Sample Task is intended to help OIAD collect and analyze information that would be valuable in studying investor behavior and decision-making, and also obtaining general insights pertinent to the study of household finance.

An OIAD Principal Investigator has been identified for this task area. For the preparation of a response to this Sample Task order, assume that OIAD will provide the following:

1. Oversight of all sampling process, and survey data collection;

2. Approval of all personnel involved in a technical role;

3. Initial drafts of survey instruments (although the contractor will be asked to provide technical assistance to refine or further develop questions);

4. Where applicable, a description of experimental conditions and experiment protocol;

5. Where appropriate, approval for the public release of the data after a period of 1 year from data collection; and

6. Oversight on the preparation of any reports of survey findings

Collection of Household Survey Data OIAD is seeking a contractor to conduct a series of nationally-representative probability-based household surveys in multiple waves as part of an “OIAD Investor Survey.” One individual (household head or alternate), is expected to answer questions about the household in each wave.

Contractors are requested to propose a cost-effective and methodologically sound survey modality for collecting the data. That is, the surveys might be deployed online, in-person, over the phone, or using some other alternative modality at the contractor’s discretion. However, contractors should understand that sample design is of high importance (hence, OIAD requests a probability sample rather than a quota sample). At the same time, while data quality is a fundamental concern, in selection of the survey deployment modality, OIAD recognizes that there may be a steep tradeoff between precision and cost-effectiveness. Contractors should provide adequate justification for the chosen modality, including a discussion of both advantages and limitations.

The survey requires drawing a probability sample using appropriate sampling procedures. The contractor will be responsible for obtaining a sampling frame. The intended sample frame is all US households (both investors and non-investors); however, OIAD is interested in oversampling high net worth households and households with higher levels of mutual fund ownership/participation in financial markets (for the purposes of this sample task the contractor can make an appropriate assumption about whether this refers to the intensive or extensive margin of participation). Proposals should discuss the pros and cons of this approach, bearing in mind that the final sampling plan will be refined in cooperation with OIAD after award.

OIAD intends to conduct 1 annual survey and 4 quarterly surveys in the base year, each lasting approximately 15-20 minutes. Option years, if executed, will follow the same structure. The annual survey will consist of questions for which low-frequency changes typically occur (e.g., marital status, financial literacy). The quarterly surveys will consist primarily of questions for which higher frequency changes may be expected to occur (e.g., net worth, employment status, investment behavior, expectations), in addition to a limited number of non-repeated questions that would change in every quarterly survey (herein referred to as “modules”). Survey data is expected to be longitudinal (i.e. longitudinal with appropriate augmentation to deal with attrition, and periodic sample refresh).

The contractor will conduct survey sample selection in cooperation with OIAD, will hire and train all survey enumerators, prepare any computer interface equipment for accurate and expedited enumeration, and prepare a cleaned dataset. The contractor must already have computer-assisted interviewing capability. The contractor will detail quality control procedures that will be implemented in order to ensure the quality of coding of the survey instrument and data collection.

OIAD has prepared draft questionnaires (in English) for use in the surveys. The draft questionnaires involve complex skip logic. Many questions involve the collection of very detailed, complex information about the household’s asset holdings, and may require any enumerators to be suitably qualified for the collection of such data. Offerors should assume that each survey will have two random assignments with multiple conditions for minor experiments. For example, one random assignment consisting of two conditions might place a set of questions at the front of the survey instrument for half of the subjects and in the back of the survey for the other half of subjects. A second random assignment might have a set of multiple conditions presenting alternative text or information to a fraction of respondents prior to answering a question series.

Offerors will assume that on one survey instrument per year, a complex survey experiment will be built into the survey with approximately 8 experimental conditions as well as additional random assignments (e.g., the ordering of choice options, or the selection of items on a menu). For an example of a study design of this type, see Scholl (2019).

The surveys will be initiated in January-March of 2022. OIAD anticipates that Spanish language capacity will be needed to faithfully execute the survey, and the contractor is requested to provide a back-translated Spanish-language version of the survey instrument.

In addition to the contractor’s normal token incentive payment to incentivize participation, OIAD desires to provide an additional incentive payment to make the annual complex experiment incentive compatible. Respondents will receive POINTS for their performance on the complex experimental task. At the close of the survey, respondents will receive compensation commensurate with POINTS as a bonus payment.

The contractor should target a 45 percent contact rate, 60 percent completion rate and look to obtain complete responses from approximately 3,500 surveys for each of the annual and quarterly surveys.

Meta-data for each survey will be collected, including click and time-stamp data, as appropriate to the modality. Any calls should be recorded for quality control purposes. Enumerators and managers should review interviews with quality control staff for improved performance.

The contractor will report progress to the Principal Investigator weekly. The contractor will provide a verbal report and a weekly call agenda. A more detailed reporting is required for lower frequency reports. Specific format can be agreed to between the SEC and the contractor. The Principal Investigator will plan for unannounced visits to monitor data collection based on the contractor’s stated periods of operation.

OIAD may exploit the longitudinal structure of the survey for additional study. As such, the contractor will retain contact records for survey respondents for 18 months following the completion of this data collection project. This contact information will not be shared with any other party except OIAD, and only on request.

All data collection must be approved by an Institutional Review Board (IRB) prior to fielding of the data. IRB approval will be the responsibility of the contractor.

Survey Qualitative Pre-Testing The contractor will perform a pretest of the base survey instruments on approximately 25 respondents. OIAD representatives will be present during the pretest. For non-repeated question modules, the contractor will provide a similar test on approximately 25 respondents of the module approximately 6-8 weeks prior to fielding. After the pretest is conducted, the contractor will advise OIAD on the results and the contractor and OIAD will jointly decide on any changes to the survey instrument and sampling strategy. For the purposes of this Sample Task, the Offerors may assume that relevant approvals have been obtained from the OIAD side for the pre-testing and deployment of the survey.

Administrative Data Integration OIAD has an extremely strong interest in integration of survey data with administrative data. Offerors are expected to propose a methodology that has the potential for achieving such integration. In particular, OIAD is interested in administrative data records with one or more of the following data elements:

· Investment account data (e.g., investment holdings, transactions, investment options)

· Retirement account information (e.g., investment holdings, transactions, investment options)

· Other financial account data

· Employment, income and earnings data

· Other household balance sheet information (e.g., non-investment portions of the household balance sheet, non-financial investment components of net worth) While administrative data integration is highly promising, OIAD also recognizes that it also may be difficult to achieve in some respects. Contractors should discuss specific data assets that will be employed to pursue data integration, any restrictions on potential access to administrative data, recruitment limitations, and other issues in this area. Contractors should provide an expected match rate for each data source.

Summary of Findings:

At the end of the base year and every subsequent year, the contractor will produce an annual report of survey findings that will present summary statistics and graphics of all non-experimental data collected. The report will provide disaggregates by key demographic groups (e.g., race, age, education, net worth). The report should outline any key findings of the survey data.

OIAD may have an interest in preparing academic research papers based on the data collected. The Offeror’s team should have the capacity to serve as co-authors on such research papers. However, no such co-authorship is promised or guaranteed.

Additional Information Offerors should be aware that the specifics of this Sample Task are indicative and that the Task Order, if awarded, will ultimately be a time and materials contract. The participant groups that are oversampled, the number of experiments, the specific topics for data collection, the frequency of data collection, and other aspects of this research are subject to the priorities and requirements of OIAD.

Subject Matter Expertise Surveys may cross into niche subject matter domains. As such, offerors shall have the ability to identify and bring on subject matter experts to augment survey development capacities.

On-Site Research Assistance Upon request, the contractor will provide research-related technical assistance to support this data analysis and coding, either at initial reporting stages or for subsequent analysis (i.e., research assistants, data analysts, statisticians, or other technical professionals). OIAD may request such support be provided on-site or off-site, and on a part-time or full-time basis.

Data Deliverables The final dataset will be provided to OIAD including the following: sample weights, a cleaned data file, a searchable data codebook with frequencies and documentation, and documentation of data collection, all meta-data, and administrative data. OIAD requests data in .csv, .xls, STATA, SAS and R formats.

The contractor will provide R code that may be used to replicate data cleaning, and other relevant steps in construction of the final dataset, as well as any analysis. The contractor will also provide R code with convenience functions that allow the user to: read in the dataset; calculate estimates of simple statistics from the dataset with appropriate standard errors (mean, median, mode, and selected quantiles). All code should be suitably commented. The contractor may rely on existing R infrastructure for completing this deliverable.

After a period of approximately one year, the contractor may be requested to work with OIAD to publish the data to an FTP site for public use. The published data must be stripped of all identifying information and adjusted accordingly in order to eliminate the possibility that individuals in the dataset can be identified. The contractor shall elaborate appropriate protocols for executing this deliverable in the response to this Sample Task. The contractor will provide adequate documentation to allow data users to conduct analysis.

The contractor will provide a field report for each survey instance undertaken summarizing: population of interests, sampling design, sampling issues, completion and response rates, a summary of the survey questions, a summary of any experimental design or other experimental conditions, and full documentation of any abnormalities in the data collection process.

Additional data delivery requirements include:

1. Data on partial respondents and screened out respondents

2. Demographic data included, with date of collection

3. A unique panelist identifier

4. Randomization variables (when applicable)

5. Meta-data

6. Any PII MUST be removed

7. Copy of final implemented survey instrument, including skip logic (pdf document)

8. Codebook, checked to make sure it includes all variables in the data, in plain language

9. A field Report containing the following information:

a. Recruitment, retention, completion, and response rates

b. Sample quality breakdown

c. Field period

d. Description of any oversamples or special sampling

e. Description of how weights were estimated, and a description each weight provided (scaling, use with what population)

f. Incentive amount paid

g. Description of any data collection anomalies