RFP_Attachment_L-4_SAMPLE_TASKS_6.30.16.pdf

ATTACHMENT L-4 – SAMPLE TASKS

Sample Task Order Tasks: For Pricing purposes assume a 12 month Base Period and four 12 month long Option Periods for Example Task 1 only.

In order to determine the capability of responding firms for conducting the activities included in the SOW at a high level of performance and to enable cost comparisons across qualified vendors, contractors are requested to respond to the tasks offered below. For the Example Task 1, applicants must complete a technical approach and pricing of this task as a part of their proposal. An additional task is offered as critique assignment – for this task, contractors are requested to prepare a brief research and operational critique of the proposed task (maximum 1,500 words), but NOT a full technical approach and costing. The purpose of the critique task is to get additional insight into contractor capability and to provide information to OIAD that will be useful for later task orders while reducing the burden on contractors.

Although the example tasks provide hypothetical examples, with numerous problematic components and aspects not specified, applicants are requested to prepare a proposal for the task as it is presented and make assumptions only where necessary to complete a tender for the sample task. Separately, the applicant may offer a critique of the proposed design, including potential areas of cost savings, and any alternative suggestions the applicant may wish to make.

For the Critique Task, only the short critique is requested.

Applicants are requested to provide separate responses to the tasks as if these are offered as separate task orders.

Background:

The SEC regulates important aspects of the financial industry. One important area of consideration disclosure of key aspects of mutual funds to prospective and current retail clients.

On one hand, provision of too little information may deny retail investors of varying degrees of sophistication of important information related to risks, conflicts of interest, fees and other properties that affect their effective return on investment. On the other hand, some retail investors may be overwhelmed by too much information, and some recent research has suggested that disclosures seem to prompt certain financial intermediaries to give more biased advice to clients than is otherwise the case. Disclosure effectiveness is a key public policy goal, and this task order is intended to help OIAD to collect and analyze information that would be valuable in making policies consistent with establishing a more effective disclosure regime, and also obtaining general insights pertinent to the study of household finance.

A co-Principal Investigator has been identified for this topical study area. For the preparation of the example task orders, assume that OIAD will provide the following:

1. Oversight of all sampling process, focus groups, and auditing of survey data field collection;

2. Approval of all personnel involved in a technical role;

3. Preparation of academic paper(s) that result from the undertaking;

4. Where appropriate, approval for the public release of the data after a period of 1 year from data collection; and

5. Design of survey instruments and focus group questions, unless otherwise stated.

Example Task 1: Telephone Survey:

OIAD is seeking a contractor to conduct a nationally-representative household telephone survey as the first wave of what is anticipated to be the “OIAD Annual Household Investor Survey.”

Each completed survey is anticipated to last approximately 60 minutes. One individual (household head or alternate), is expected to answer questions in each household about the household.

The survey requires drawing a probability sample using appropriate sampling procedures. The contractor will be responsible for obtaining a sampling frame after refining the population of interest in cooperation with OIAD based on contractor recommendations and any anticipated logistical issues. OIAD desires oversampling of two key populations: high net worth households, and also persons aged 18-30 living in multigenerational households.

The contractor will conduct sample selection in cooperation with OIAD, will hire and train all survey enumerators, prepare any computer interface equipment for accurate and expedited enumeration, and prepare a cleaned dataset. The contractor must already have computer-assisted telephone interviewing capability.

OIAD has prepared a draft questionnaire (in English) for use in the survey. The survey contains a base module that is expected to be used in future rounds and an additional module with a selected set of study questions. Many questions involve the collection of very detailed, complex information about the household’s asset holdings, and will require enumerators to be well-versed in the subject matter and classification issues involved in the collection of such data.

A survey experiment is built into the survey with 2 experimental conditions and a control condition. The experimental conditions will consist of reading an approximately 100 word statement to the respondent at a point in the survey; the control group will have no statement.

The survey will be conducted in January-March of 2017 [NOTE: this timeframe is indicative for the purpose of this example task]. OIAD anticipates that Spanish language capacity will be needed to faithfully execute the survey, and the contractor is requested to provide a back-translated Spanish-language version of the survey instrument.

The contractor will perform a pretest of the survey instrument on approximately 60 respondents.

OIAD representatives will be present during the pretest. After the pretest is conducted, the contractor will advise OIAD on the results and OIAD will jointly decide on any changes to the survey instrument and sampling strategy. [Assume that relevant approvals have been obtained from the OIAD side for the pre-testing and deployment of the survey.]

In addition to the contractor’s normal token incentive payment to incentivize participation, OIAD desires to provide an additional incentive payment to make key features of the survey salient. The two treatment conditions will require the enumerator to read a script. Later in the survey, respondents (from both treatment and control groups) will be asked 5 questions about investing and will receive POINTS for each correct response. At the close of the survey, respondents will receive (POINTS * POINTS) dollars as a bonus payment (that is, the squared number of accumulated points).

The contractor should target a 45 percent contact rate, 60 percent completion rate and look to obtain complete responses from approximately 3,500 surveys.

Calls should be recorded for quality control purposes. Enumerators and managers should review interviews with quality control staff for improved performance.

The contractor will report progress to the Principal Investigator weekly. The Principal Investigator will plan for unannounced visits to the call center to monitor progress based on the contractor’s stated periods of operation.

Funding permitting, OIAD has an interest in exploiting a longitudinal structure for the survey in subsequent rounds. As such, the contractor will retain contact records for survey respondents for 18 months following the completion of this data collection project. This contact information will not be shared with any other party except OIAD, and only on request.

The contractor will use multiple imputation methods to deal with nonresponse issues in the construction of a final dataset. The final dataset as well as the pre-imputation dataset will be provided to OIAD including the following: sample weights, a cleaned data file, a searchable data codebook with frequencies and documentation, and documentation of data collection. OIAD requests data in .csv, .xls, STATA, SAS and R formats.

The contractor will provide R code that may be used to replicate data cleaning, multiple imputation, and other relevant steps in construction of the final dataset. The contractor will also provide R code with convenience functions that allow the user to: read in the dataset; calculate estimates of simple statistics from the dataset with appropriate standard errors (mean, median, mode, and selected quantiles); and estimate simple regressions using Ordinary Least Squares Regression, Logistic Regression, Multinomial Logistic Regression, Ordered Logit, Conditional Logit. All code should be suitably commented. The contractor may rely on existing R infrastructure for completing this deliverable.

After a period of approximately one year, the contractor will work with OIAD to publish the data to an FTP site for public use. The published data must be stripped of all identifying information and adjusted accordingly in order to eliminate the possibility that individuals in the dataset can be identified. The contractor shall elaborate appropriate protocols for executing this deliverable in the response to this task order.

Critique Task: Focus Group Research:

For this task Respondents are asked to provide comments and critique of the following example task. Comments and critique should be no more than 1,500 words in length, and should not include a full technical approach or cost estimates, but should include any perceived issues related to research design or operationalizing the research. The more specific research objectives of the focus groups and randomized experiments within the focus groups is to estimate the treatment effect of various information treatments on subject understanding of mutual fund fees and to Understand, qualitatively, the reasons why mutual fund fees and their importance are not well understood.

OIAD seeks a contractor to conduct focus group research in three cities (San Diego, Chicago, Fort Lauderdale) in October-December 2016 [NOTE: this timeframe is indicative for the RFP only]. The subject of the focus groups is to understand retail investor knowledge and understanding of mutual fund fee disclosures.

Approximately 40 mutual fund holding subjects should participate in each location. Following check-in and a brief orientation, half of the subjects will be assigned to study Group A and half will be assigned to Group B. In both groups, subjects will take a 15 minute basic survey to obtain their demographic and socioeconomic profile, financial literacy, numeracy, expectations, risk aversion and other factors.

In the Fort Lauderdale location, approximately half of the group (~20) will be seniors/retirees.

The other half (~20) will participate in a Spanish-language focus group and be comprised of individuals who designate their primary language as Spanish. In the Fort Lauderdale location, the seniors group and the Spanish language group will each be approximately evenly split into Group A and Group B. In the other two locations, study subjects will be drawn from a broad cross-section of the prime working-age (25-54) adult population.

Group A: will be shown a 5 minute video describing pertinent criteria for selecting mutual funds.

The video was developed under a previous task order in cooperation with a major advertising firm. Following the video, members of Group A will be provided with an information packet with information similar to the video content, but in a much more extended and detailed form.

Subjects will be provided with 5 minutes to review the information in the packets.

Group B: will not be shown a video, but will be provided with the information packet and 5 minutes to review the information.

Following the information review period, it will be explained that participants will now be entrusted to allocate a hypothetical $10,000 portfolio among three stock mutual funds and three bond mutual funds. Participants will be told that they will receive a bonus based on the performance of the actual net investment returns of their portfolio over the next month and again over the next 12 months. As a practical matter, they will be informed that their bonus will be between $0 and $150 at each interval. Subjects will be allotted to a private terminal to allocate the funds so that the time required and the content of allocation cannot be gleaned from other subject participants.

Computer terminals will present information about the funds in the form of the prospectus and summary prospectus, Morningstar ratings, and past performance over 1-, 5-, and 10-year horizons. Subjects will have up to 25 minutes to review and allocate money to the funds.

Following the computer activity, Group A subjects and Group B subjects will each be broken up into groups of approximately 10 persons each (no mixing of A and B members) for a focus group discussion on the subject of mutual fund fees and investments. These discussions will be facilitated by the contractor using a set of prompts developed in partnership with the Principal Investigator. The prompts will seek to elicit a better understanding of the reasons for the subjects’ decisionmaking regarding the portfolio game, as well as their understanding of investments more generally. This discussion should take approximately 35 minutes.

Following the led discussion, participants will be debriefed on the subject for approximately 5 minutes. Facilitators should seek to answer any follow-up questions, and seek to reinforce any of the educational components of the information treatments that are relevant.

The contractor will keep an accurate transcription of the discussion that takes place in the focus group, along with appropriate time stamps. In addition to a transcript, the contractor will provide a longitudinal dataset version of the transcript (with identifiers as participant and time) in MS Excel. The contractor will provide a brief report regarding the focus group undertakings, detailing any abnormalities or deviations from the data collection protocol, as well as a synopsis of the main findings that arise from the led discussion.

Follow up: At 3 months after the experiment and 12 months, the contractor will contact subjects to verify their current mailing address for mailing of the incentive payment. They will also ask a series of brief follow up questions related to their own personal portfolio allocation decisions since the close of the focus group, as well as their understanding of mutual fund fees. Calls are expected to last approximately 15 minutes.