ATTACHMENT C-2 Sample Task 1 Disclosure Description AMENDMENT 00001.docx

ATTACHMENT C-2 SAMPLE TASK 1 “DISCLOSURE” DESCRIPTION

50310221R0014 AMENDMENT 0001

Sample Task 1 “Disclosure” – Description

Sample Task Order Tasks: For Pricing purposes assume a 12 month Base Period and four nine 12 month long Option Periods.

Objective For this Sample Task 1, applicants must complete a Research Proposal/Technical Approach and Pricing of this task as a part of their proposal.

Although the Sample Task does not fully specify all aspects of the work, applicants are requested to prepare a proposal for the task as it is presented and make assumptions only where necessary to complete a tender. Separately, the applicant may offer a critique of the proposed design, including potential areas of cost savings, and any alternative suggestions the applicant may wish to make. For example, the contractor may choose to propose additional pretesting steps if they believe that such pretesting would be useful for completion of the Sample Task. This separate critique is not required.

The SEC reserves the right to award this project as a task order to zero or more IDIQ-qualified contractor(s) at the award of the IDIQ or shortly thereafter, using the assumptions and proposal prepared in response to this sample task.

Background The SEC regulates important aspects of the financial industry. One important area of consideration is the disclosure of key aspects of mutual funds to prospective and current retail clients. On one hand, providing too little information may deny retail investors important information that affects their investment decisions. On the other hand, some retail investors may be overwhelmed if more information is provided. Disclosure effectiveness is a key public policy goal, and this Sample Task is intended to help the SEC’s Office of the Investor Advocate (OIAD) to collect and analyze information that would be valuable in making policies consistent with establishing a more effective disclosure regime.

Information on mutual funds, ETFs, and other securities can deal with many topics. Across mandatory disclosures and non-mandatory information (i.e., the broader information environment) some topics that may be of the interest to OIAD and the SEC include:

· Information on specific investment vehicles or types of securities, including that on:

· Risks

· Fees/expenses associated with investing

· Performance information, including comparative information

· Asset allocation, including potential changes in allocation(s) over time

· Background on financial professionals, such as investment advisers or broker-dealers, and potential conflicts of interest

· Advertisements or commercials for securities

· Topical areas including:

· ESG-related considerations

· Corporate governance considerations

· Whether information is fraudulent or not

· Digital assets

Responding firms may assume that this Sample Task deals with one or more of these areas.

Responding firms may assume that an OIAD Principal Investigator has been identified for this Sample Task. In addition, to prepare a proposal, responding firms may assume that OIAD will provide the following:

1. Oversight of all sampling processes and data collection, including both qualitative and quantitative data;

2. Approval of key personnel;

3. Preparation of academic paper(s) that result from the undertaking;

4. Where appropriate, approval for the public release of the data after a period of 1 year from data collection; and

5. Drafts of survey instruments and interview questions, unless otherwise stated.

Sample Task Description Stage 1: Qualitative Testing of Alternative Disclosure Designs and Survey Instrument OIAD seeks a contractor to assist in the development and execution of a disclosure testing study that will enable inferences about mutual fund investors’ reactions to disclosures and whether different designs affect those reactions. In preparation for a quantitative data collection phase, OIAD seeks a contractor to perform qualitative testing, corresponding to Subtask 2.1 of the IDIQ. The objectives of this qualitative testing are primarily to:

1. Assess general reactions to the disclosure documents that OIAD provides; and,

2. Assess whether survey questions and instructions from the quantitative instrument are clear and understandable to participants.

The contractor will perform a pretest of the disclosure documents and survey instrument using an in-depth interview approach. The estimated sample size is approximately 45 respondents, spread across three distinct disclosure designs. In all cases, the contractor will facilitate access for OIAD representatives and OIAD representatives may be present during all pretesting sessions.

To assess whether qualitative reactions differ across investors with different levels of financial sophistication, the contractor will sample investors with varying levels of mutual fund literacy, financial literacy, and/or financial assets. In preparation for conducting the pretest, the contractor shall advise OIAD on potential approaches for identifying and recruiting these individuals. Information requested by OIAD to inform the approach includes the following:

1. Use of existing contractor data (if applicable) versus a new screening survey to collect variables that can assist in identifying investors with varying levels of financial sophistication

2. Appropriate variables to use to measure financial sophistication

3. The level of incentive payments provided to participants to recruit them to the pretest, and whether those incentives vary across participants with different levels of financial sophistication After the pretest is conducted, the contractor will provide a written summary of the results and advise OIAD on recommended changes. Then, OIAD and the contractor will jointly decide on any changes to the survey instrument and disclosure documents.

Stage 2: Quantitative Testing of Alternative Disclosure Designs Background and Sample In a second stage of the research, OIAD seeks a contractor to deploy an online survey to a household panel of respondents that is nationally representative of mutual fund holders. Work conducted under this stage of the Sample Task closely corresponds to Subtask 1.1.1 of the IDIQ; however, it is likely to draw upon Task Area 3 of the IDIQ (and may also draw upon Task Area 4).

The survey requires drawing a probability sample using appropriate sampling procedures. The contractor will be responsible for obtaining a sampling frame after refining the population of interest in cooperation with OIAD based on contractor recommendations and any anticipated logistical issues. OIAD anticipates that the final sample will contain approximately 3,000 respondents, who will be allocated across three disclosure conditions. OIAD also desires stratification along some measure of financial sophistication, and may request that respondents who are not involved in making financial decisions (or investment decisions) in their household are excluded from the study. The contractor will conduct sample selection in cooperation with OIAD, and conduct power analysis to ensure adequate sample size for projected analysis by group and by treatment condition.

Anticipated Survey Content and Quality Control Procedures The online survey shall contain an embedded survey experiment that varies in display according to the disclosure documents refined as a result of the Stage 1 research. Participants shall be allocated randomly to the different disclosure conditions; however, in the case of a stratified sample, they will be allocated randomly within strata.

The survey shall contain, at the minimum, the following:

1. Screening questions to ensure the sample meets OIAD requirements

2. Instructions about upcoming survey content

3. Presentation of mutual fund information, with information varying according to disclosure condition

4. Questions designed to assess participants’ reactions to disclosure documents they viewed. These are likely to be structured as multiple choice questions measuring comprehension. In addition, there may be questions to assess data quality (e.g., respondents’ reported inability to view stimuli)

5. Behavioral choice questions that ask participants to make investment decisions across a set of mutual funds, with information about the mutual funds presented according to participants’ disclosure condition

6. Questions to collect other background characteristics of respondents, including demographics (if not available from existing data) The contractor will program the study, including adapting the disclosure documents to an online format. The contractor will also ensure the fidelity of the study. To that end, the contractor will provide OIAD a description of its quality control procedures that describes areas of the survey that have been tested, including skip logic and display across different device types. The contractor will also collect meta-data measures, including device type, time on survey, and time spent on specific questions, and merge that information into the final data set. Upon completion, the contractor will deliver a field report that fully details all steps taken.

In addition to standard incentive payments, OIAD desires to provide an additional incentive payment to the “behavioral choice” questions described above. In particular, respondents will make choices between mutual funds and have variable payments calculated based on the selections they made. At the close of the survey, participants will receive incentives that vary based on these decisions.

The contractor will report progress to the Principal Investigator weekly on progress made under this stage of the research, including information about programming, recruitment metrics, and unexpected events (if any). Specifically, the contractor will provide an oral report and a weekly call agenda. More detailed reporting would be required for lower frequency reports. Specific format can be agreed to between the SEC and the contractor.

Funding permitting, OIAD may have an interest in exploiting a longitudinal structure for the survey in subsequent rounds. As such, the contractor will retain contact records for survey respondents for 18 months following the completion of this data collection project. This contact information will not be shared with any other party except OIAD.

Data and Code Delivery

The contractor will use multiple imputation methods to deal with nonresponse issues in the construction of a final dataset. The final dataset as well as the pre-imputation dataset will be provided to OIAD including the following: sample weights, a cleaned data file, a searchable data codebook with frequencies and documentation, and documentation of data collection. OIAD requests data in .csv, STATA and R formats.

The contractor will provide statistical code that may be used to replicate data cleaning, multiple imputation, and other relevant steps in construction of the final dataset. The contractor will also provide code with convenience functions that allow the user to: read in the dataset; calculate estimates of simple statistics from the dataset with appropriate standard errors (mean, median, mode, and selected quantiles); and estimate simple regressions using Ordinary Least Squares Regression, Logistic Regression, Multinomial Logistic Regression, Ordered Logit, Conditional Logit. All code should be suitably commented.

After a period of approximately one year, the contractor will work with OIAD to publish the data to an FTP site for public use. The published data must be stripped of all identifying information and adjusted accordingly in order to eliminate the possibility that individuals in the dataset can be identified. The contractor shall elaborate appropriate protocols for executing this deliverable in the response to this Sample Task.

Stage 3: Reporting Separate from data delivery occurring under Stage 2, which should happen as soon as possible, OIAD may request additional support to facilitate the reporting of results under this Sample Task. Such support will take two forms, and will generally correspond to Task Areas 3 and 4 of the IDIQ.

Reporting to a Technical/Academic Audience In this report, the contractor will perform statistical data analysis as directed by OIAD and prepare materials that would be appropriate for an academic style journal article on this research. As part of this task, contractors will provide OIAD all materials necessary to recreate the statistical analysis contained in such reports, including raw data files, a data dictionary, and statistical/data analysis code. Contractors should also preparer and deliver a documentation file that clearly describes the order in which programs should be run and the input and output files required for each file. After delivering those materials to OIAD, contractors shall prepare a 30-45 minute walkthrough presentation to summarize that information and answer any questions that OIAD staff may have. In addition, contractors should anticipate that subsequent research questions may arise as a result of initial data analysis and investigation, which may engender additional requests from OIAD. As such, there is likely to be an initial reporting stage and subsequent analysis. Upon request, the contractor will provide research-related technical assistance to support this data analysis and coding, either at initial reporting stages or for subsequent analysis (i.e., research assistants, data analysts, statisticians, or other technical professionals). OIAD may request such support be provided on-site or off-site, and on a part-time or full-time basis.

While the full details of an analysis plan would be developed as part of this Sample Task, reporting firms may assume that the initial report would contain, at the minimum:

1. Stage 1 reporting, including analysis and appropriate displays (i.e., tables or figures) of the following information:

a. A summary of the demographic and financial characteristics of interview participants

b. A summary of qualitative reactions to the disclosure documents

c. A summary of areas of confusion in the survey instrument, and corresponding changes to address those changes (if any)

d. An appendix with the interview protocol

2. Stage 2 reporting, including analysis and appropriate displays (i.e., tables or figures) of the following information:

a. Detailed statistics on the recruitment procedures, including number of potential participants contacted, cooperation and response rates, and resulting sample

b. An assessment of the overall representativeness of the sample, relative to the target population

c. A determination of whether any data quality, programming, or display errors may have affected the results of the study

d. Between-participant comparisons to assess whether disclosure conditions affected outcomes including: comprehension, decision quality, and decision-making time

e. Subgroup and/or moderation analyses to determine whether the effects of disclosure conditions were affected by demographic characteristics or financial sophistication

f. An appendix with the survey instrument

Reporting to a Non-Technical Audience In this short report, no longer than 5 pages, contractors will prepare a non-technical version of the conclusions of this research that would be appropriate for a policy audience. The report shall contain highlights of findings from both Stage 1 and Stage 2 of this Sample Task.

Branding All materials generated and transmitted to OIAD are OIAD work products, and as such, no material transmitted to OIAD shall contain any contractor-specific branding.

Stage 4: Field Testing Contractors shall describe their capacity for performing field testing relevant to this Sample Task, corresponding to subtask 1.4 of the IDIQ. OIAD reserves the right to allocate more or less resources to this Stage of the research depending on the proposals received and the results obtained in prior stages of research.

In general, OIAD’s objective for field testing is to assess the effectiveness of different disclosures described under Stage 1 and Stage 2 of this research. Field testing would most likely take the form of randomized assignment to different disclosure designs, with appropriate tracking of investor outcomes. These outcomes may include: attention to investment information, number and/or frequency of changes to investments, decision quality of investment choices, or investment outcomes (i.e., gains or losses in account balances).

Given the challenges associated with performing a successful field test, OIAD is interested in learning about different participant samples that contractors may be able to access and the most promising approaches for deploying a field experiment. Of particular interest would be samples that allow for the integration of administrative data as described in the IDIQ under Subtask 1.3. For instance, contractors who have access to an opt-in panel of investment-related transactions may be able to deliver different disclosures to those participants. In contrast, other contractors may believe it is more feasible to have participants make real investment decisions through an online portal, and subsequently deliver different information about those investments.

In all cases, contractors are encouraged to describe a methodology that is appropriate for the samples that they believe are possible, including any associated sampling, recruitment, ethical, or other issues. Contractors should further discuss any sample size or recruitment issues that might affect analysis of the field test. The details of any field test would be finalized only with OIAD’s approval.