9531CB24R0005 Managed Online Research Support.pdf

SEE ADDENDUMIS CHECKED

CODE 18a. PAYMENT WILL BE MADE BY

CODE

FACILITYCODE

17b. CHECK IF REMITTANCE IS DIFFERENT AND PUT SUCH ADDRESS IN OFFER

OFFEROR

CODE 16. ADMINISTERED BYCODE

X

SIZE STANDARD:

% FOR:SET ASIDE:UNRESTRICTED ORCFP

RFPIFB

10. THIS ACQUISITION ISCODE

RFQ

14. METHOD OF SOLICITATION

13b. RATING

NAICS:

SMALL BUSINESS

03/20/2024 1000 ET

03/04/2024

PATRICK SISK

(No collect calls)

INFORMATION CALL:

FOR SOLICITATION 8. OFFER DUE DATE/LOCAL TIMEb. TELEPHONE NUMBERa. NAME

4. ORDER NUMBER3. AWARD/ 6. SOLICITATION

9531CB24R0005

5. SOLICITATION NUMBER

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL ITEMS 1. REQUISITION NUMBER PAGE OF

1 74 OFFEROR TO COMPLETE BLOCKS 12, 17, 23, 24, & 30

TELEPHONE NO.

17a. CONTRACTOR/

15. DELIVER TO

WASHINGTON DC 20552

1700 G STREET, NW

OFFICE OF PROCUREMENT

9. ISSUED BY

7.

2. CONTRACT NO.

EFFECTIVE DATE

18b. SUBMIT INVOICES TO ADDRESS SHOWN IN BLOCK 18a UNLESS BLOCK BELOW

ISSUE DATE

DELIVERY FOR FOB DESTINA-

TION UNLESS BLOCK IS

MARKED

11.

SEE SCHEDULE

12. DISCOUNT TERMS

THIS CONTRACT IS A

RATED ORDER UNDER

DPAS (15 CFR 700)

13a.

SERVICE-DISABLED

VETERAN-OWNED

SMALL BUSINESS

HUBZONE SMALL

BUSINESS

8(A)

Consumer Financial Protection Burea

WOMEN-OWNED SMALL BUSINESS

(WOSB) ELIGIBLE UNDER THE WOMEN-OWNED

SMALL BUSINESS PROGRAM

EDWOSB

24.

AMOUNT

23.

UNIT PRICE

22.

UNIT

21.

QUANTITY

20.

SCHEDULE OF SUPPLIES/SERVICES

19.

ITEM NO.

Managed Online Research Support

(Use Reverse and/or Attach Additional Sheets as Necessary)

HEREIN, IS ACCEPTED AS TO ITEMS:

DATED . YOUR OFFER ON SOLICITATION (BLOCK 5),

INCLUDING ANY ADDITIONS OR CHANGES WHICH ARE SET FORTH

COPIES TO ISSUING OFFICE. CONTRACTOR AGREES TO FURNISH AND DELIVER

ARE

ARE

31c. DATE SIGNED

27b. CONTRACT/PURCHASE ORDER INCORPORATES BY REFERENCE FAR 52.212-4. FAR 52.212-5 IS ATTACHED. ADDENDA

31a. UNITED STATES OF AMERICA (SIGNATURE OF CONTRACTING OFFICER)

30c. DATE SIGNED 31b. NAME OF CONTRACTING OFFICER (Type or print)

ALL ITEMS SET FORTH OR OTHERWISE IDENTIFIED ABOVE AND ON ANY ADDITIONAL

SHEETS SUBJECT TO THE TERMS AND CONDITIONS SPECIFIED.

27a. SOLICITATION INCORPORATES BY REFERENCE FAR 52.212-1, 52.212-4. FAR 52.212-3 AND 52.212-5 ARE ATTACHED. ADDENDA

26. TOTAL AWARD AMOUNT (For Govt. Use Only)

OFFER

STANDARD FORM 1449 (REV. 2/2012)

Prescribed by GSA - FAR (48 CFR) 53.212

ARE NOT ATTACHED.

ARE NOT ATTACHED.

AUTHORIZED FOR LOCAL REPRODUCTION

PREVIOUS EDITION IS NOT USABLE

30b. NAME AND TITLE OF SIGNER (Type or print)

30a. SIGNATURE OF OFFEROR/CONTRACTOR

28. CONTRACTOR IS REQUIRED TO SIGN THIS DOCUMENT AND RETURN

25. ACCOUNTING AND APPROPRIATION DATA

29. AWARD OF CONTRACT:

REF.

x

32e. MAILING ADDRESS OF AUTHORIZED GOVERNMENT REPRESENTATIVE

32c. DATE 32b. SIGNATURE OF AUTHORIZED GOVERNMENT REPRESENTATIVE

ACCEPTED, AND CONFORMS TO THE CONTRACT, EXCEPT AS NOTED:

32a. QUANTITY IN COLUMN 21 HAS BEEN

RECEIVED INSPECTED

40. PAID BY39. S/R VOUCHER NUMBER38. S/R ACCOUNT NUMBER

37. CHECK NUMBER

FINALPARTIAL

36. PAYMENT

FINALPARTIAL

35. AMOUNT VERIFIED

CORRECT FOR

34. VOUCHER NUMBER33. SHIP NUMBER

COMPLETE

32g. E-MAIL OF AUTHORIZED GOVERNMENT REPRESENTATIVE

42d. TOTAL CONTAINERS42c. DATE REC'D (YY/MM/DD)

42b. RECEIVED AT (Location)

42a. RECEIVED BY (Print)

41c. DATE41b. SIGNATURE AND TITLE OF CERTIFYING OFFICER

41a. I CERTIFY THIS ACCOUNT IS CORRECT AND PROPER FOR PAYMENT

STANDARD FORM 1449 (REV. 2/2012) BACK

24.

AMOUNT

23.

UNIT PRICE

22.

UNIT

21.

QUANTITY

20.

SCHEDULE OF SUPPLIES/SERVICES

19.

ITEM NO.

32f. TELEPHONE NUMBER OF AUTHORIZED GOVERNMENT REPRESENTATIVE

32d. PRINTED NAME AND TITLE OF AUTHORIZED GOVERNMENT REPRESENTATIVE

74 2 of

3 | P a g e

REQUEST FOR PROPOSALS

9531CB24R0005

MANAGED ONLINE RESEARCH SUPPORT

TABLE OF CONTENTS

SECTION A – SF 1449

SECTION B – SCHEDULE OF SERVICES

B.1 Background Information B.2 Definitions B.3 Scope B.4 Period of Performance B.5 Statement of Objectives B.6 Requirements B.7 Constraints and Risks B.8 Deliverables

SECTION C – CONTRACT CLAUSES

C.1 FAR 52.212-4, Contract Terms and Conditions-Commercial Items C.2 Addendum to FAR 52.212-4 C.3 FAR 52.212-5, Contract Terms and Conditions Required to Implement Statutes and

Executive orders

SECTION D – CONTRACT DOCUMENTS, EXHIBITS OR ATTACHMENTS

Reserved

SECTION E – SOLICITATION PROVISIONS AND INSTRUCTIONS TO OFFERORS

E.1 52.212-2, Evaluation-Commercial Items E.2 Solicitation Provisions E.3 Far 52.252-1 Solicitation Provisions Incorporated by Reference E.4 Addendum to FAR 52.212-1

SOLICITATION ATTACHMENTS AND EXHIBITS

Attachment 1 – Incidence Table Attachment 2 – Offeror’s Price Model Attachment 3 – Non-Disclosure Agreement

9531CB24R0005 Managed Online Research Support – Table of Contents

4 | P a g e

Attachment 4 – Good Faith Effort Attachment 5 – Organizational Conflict of Interest Attachment 6 – Subcontracting Plan

9531CB24R0005 Managed Research Support

5 | P a g e

SECTION B – SCHEDULE OF SERVICES

B.1 BACKGROUND INFORMATION

1.1 This is a Request for Proposal (RFP) under Federal Acquisition Regulation (FAR) Part 12 to establish a single Indefinite Delivery/Indefinite Quantity (IDIQ) contract to establish Managed Online Research Support services for the Consumer Financial Protection Bureau (CFPB or Bureau). CFPB intends to award a single IDIQ contract from this RFP. The Bureau reserves the right to issue multiple awards or no award. Individual contracts will be issued to the Proposer/Proposers who submit(s) responses that represent the best value to the Government, as defined in this RFP.

1.2 Following contract award, the Contractor must undergo a cybersecurity review by CFPB to ensure its information systems used to carry out contract services meet Federal requirements. See the Cybersecurity/Information Security (February 2024) clause for more details.

1.3 On July 21, 2010, the Dodd-Frank Wall Street Reform and Consumer Protection Act (the

"Act") created the Consumer Financial Protection Bureau (CFPB). Section 1013(b)(1) of the Consumer Financial Protection Act, 12 U.S.C. § 5493(b)(1), establishes the Office of Research (“OR”) and assigns to it the responsibility of researching, analyzing, and reporting on topics relating to the Bureau’s mission, including developments in markets for consumer financial products and services, consumer awareness, and consumer behavior.

Experimentation provides researchers with control over data collection processes, which provides several benefits. For example, researchers can maintain the assumptions of a given social science model, and the ability to alter these assumptions, which enables research that would either be less precise or impossible using administrative data. In addition, the results of experiments can be replicated and tested over different specifications of the environment or in the underlying model, implying that this type of research can, relatively easily, be expanded upon by other researchers or tested for robustness. Finally, experimentation allows researchers to conduct research that is timely and relevant to the Bureau’s greatest needs. This is in contrast to research resulting from natural experiments, the timing and topics of which cannot be influenced by Bureau researchers. While it is important to recognize that controlled experimentation itself has limitations, it is also clear that it plays an important role in the advancement of economic understanding.

Managed Research Support Section B – Schedule of Services

6 | P a g e

Over the last decade, much social science research has transitioned from in-person laboratories and computer labs to online laboratories in which information is exchanged via the World Wide Web (the internet). There are at least four benefits of this approach.

First, typical online participant pools include individuals with a greater diversity of demographics than do typical in-person participant pools, especially those maintained by typical colleges and universities. In addition, online participant pools are typically larger than university pools, online studies are less expensive than typical university studies, and online studies can be run more quickly than typical university studies.

B.2 DEFINITIONS

A. Online panel. A collection of individuals who volunteered to be contacted for the purpose of participating in research via the internet.

B. Participant. A participant is any individual who participates in a CFPB study under this

Contract, regardless of whether they complete the study.

C. Study. Work conducted under this contract will be organized as “studies.” Studies will include tasks for individual participants such as surveys, as well as tasks for multiple participants such as games (e.g., a prisoners' dilemma game) or economic simulations (e.g., an auction).

D. Study Request. A Study Request will describe the requirements of the study, including required characteristics of participants and tasks for the Contractor.

E. Task. Work the Contractor conducts under each study is delineated as tasks. Recruitment of participants for a study is an example of a task.

F. Institutional Review Board (IRB): An IRB is a type of committee that applies research ethics by reviewing the methods proposed for research to ensure that they are ethical. Such boards are formally designated to approve (or reject), monitor, and review behavioral research involving humans.1

G. Paperwork Reduction Act (PRA): PRA is a federal law that seeks to limit the burden Federal agencies impose on the public for their information requirements. The CFPB is required to submit a PRA package with details about the information collection for Office of Management and Budget (OMB) review and approval. PRA approval is required prior to implementing a study.

1 Definition derived from Wikipedia: https://en.wikipedia.org/wiki/Institutional_review_board.

7 | P a g e

H. Managed Research Services: Managed Research Services are services where the Contractor manages and carries out a study on behalf of CFPB researchers based on requirements specified by CFPB researchers in a Study Request.

B.3 SCOPE

Under this contract, a Contractor shall assist CFPB in executing studies via the internet. CFPB researchers will determine the research design and recruitment requirements for each study. Some studies will require convenience samples, while other studies will require samples that match certain demographics profiles. Depending on the requirements of a study, participants may be paid a fixed amount, or they may be paid an amount that depends on their choices and/or the choices of other participants in the study. CFPB researchers will create study materials and make them accessible to participants through the internet.

Under this contract, the Contractor will only carry out support tasks, including working with CFPB staff to set up an online study for distribution; recruiting and screening for high-quality participants;

and paying participants. In addition to these support tasks, the Contractor may also be asked to provide support to obtain Institutional Review Board (IRB) approval, and match study data with administrative data, if necessary.

The CFPB is not seeking Contractor services to build a custom system for CFPB staff to carry out the support tasks described above; the support tasks will be carried out by the Contractor and rely on the Contractor’s existing systems and processes that currently carry out these types of tasks. The CFPB is also not seeking Contractor services related to designing or managing the overall research project, such as research study design, survey or data hosting, data processing, or data analysis; these tasks will be carried out by the CFPB.

Work conducted under these contracts will be organized as “Task Orders” or “studies.” Work the Contractor conducts under each study will be delineated as “tasks.” The ordering information for work conducted under each contract is as follows:

3.1 Ordering Information for Managed Research Services

At the beginning of each study, CFPB will send the Contractor a “Study Request” which will describe the requirements of the study, including required characteristics of participants and tasks for the Contractor. Each study will follow the same basic workflow:

8 | P a g e

A. CFPB sends the Contractor a Study Request.

B. CFPB and the Contractor discuss and finalize the Study Request.

C. CFPB uses the Study Request to put forth a task order After the Task Order is awarded, CFPB will work with the contractor to carry out tasks specific to the study.

D. The Contractor obtains IRB approvals, if necessary.

E. For each wave of data collection (including pre-tests/pilots):

a. The Contractor recruits and screens participants.

b. The Contractor delivers eligible screened participants to the study web address.

F. The Contractor shall pay participants. If necessary, the CFPB informs the Contractor which individuals completed the study and specifies the amount each participant should be paid.

G. The Contractor shall provide the CFPB with de-identified matched administrative data, including responses to screener questions, if necessary.

CFPB and the Contractor shall be in communication through each of these stages to confirm progress, clarify questions, and resolve any issues that arise. If necessary, the CFPB and Contractor will also meet to debrief after the study.

3.2 Study Requests

To initiate a study under this contract, CFPB will send the Contractor a “Study Request”. A Study Request will describe the requirements of the study, including required characteristics of participants and tasks for the Contractor. The Contractor will offer proposed changes to the Study Request and confirm the cost of Contractor services to support the study. CFPB and the Contractor will then discuss and finalize the Study Request. Each Study Request will include, at least:

A. The required demographics of participants B. The required number of participants C. Suggested recruitment materials and suggested screener questions/techniques, if applicable D. An estimate of study length E. A description of the web addresses for the study, including whether the addresses will be individualized, and examples of the web address(es) for the study F. The structure and timing of data collection, including (1) whether there will be a pilot/pre-test and the number of participants required, (2) the number of waves of data collection and the number of participants required for each, and (3) the approximate

9 | P a g e timing of data collection waves and the pilot/pre-test, if relevant (e.g., the number of days between the completion of a pilot and wave one of data collection).

G. A description of how participant payments will be determined, including whether participants will be paid a flat payment or whether payment will depend on participants’ choices. The description will also include minimum, maximum, and expected payment amounts.

H. Required data matching (if applicable), including, data from the Contractor’s screening process; and including (optional) supplemental data matching.

The first Task Order under this IDIQ will be used to order Study Requests for all studies. The first Task Order will also include the option to order Study Feasibility services when incidence and feasibility of specific groups are not already known and requires additional work by the Contractor to obtain this information. Once a Study Request for a study is finalized and a proposal is obtained, CFPB will issue a separate Task Order to carry out the work specified in the Study Request for the study.

B.4 PERIOD OF PERFORMANCE

The base period of performance for this contract shall be one (1) year, plus four (4) one-year option periods. In accordance with FAR 52.217-8, this contract may be extended for up to six (6) months beyond the final option period.

B.5 STATEMENT OF OBJECTIVES

A. Implementation - The Contractor shall enable CFPB to implement online studies quickly.

Each stage of each study is implemented quickly, within the timeframes listed in the deliverables table (see section B.8 Deliverables). In addition, after each round of data collection, when CFPB is ready, the next round is begun quickly, if applicable. The Contractor’s solution shall clearly provide CFPB the ability to implement online studies within this time frame.

B. Forecasting Prices - The Contractor shall provide enough information for the CFPB to accurately forecast the prices of studies before submitting a Study Request. The Contractor’s solution, Price Model, and pricing tables (rate cards), and, if applicable, estimates of incidences of demographic groups in the Contractors’ online panel(s) shall provide the means for CFPB to estimate studies’ costs prior to submitting a Study Request.

10 | P a g e

C. Technical Support for CFPB - The Contractor shall provide information, as requested, on the technical details of its processes or systems used to carry out support tasks related to a study. This information can assist CFPB in creating Study Request materials and completing PRA paperwork.

D. Technical Support for Participants – Participants are able to receive technical assistance, as needed, to help them successfully navigate issues they may encounter when accessing a study, accessing or completing screening materials, or with receiving payment. The Contractor’s solution shall demonstrate the ability to provide technical support to participants.

E. Participation - The Contractor shall recruit high-quality participants. The Contractor’s solution shall clearly demonstrate the ability to recruit and screen high-quality individuals who are human beings (i.e., not robots or “bots”) with demographics accurately reported, and who answer questions attentively and honestly.

F. Online Panels - Online panels shall meet the needs of a variety of studies. The Contractor’s solution shall clearly demonstrate the ability to access to one or more online panels from which it can recruit and screen for participants. These panels allow CFPB to meet several goals, depending on the needs of the study:

a. Recruit quickly. The Contractor shall quickly recruit high-quality participants from a convenience sample.

b. Recruit based on demographics. The Contractor shall recruit high-quality participants based on sex, race, or age; descriptors of their situations such as status as a veteran, status as a student, employment status, or household income; or descriptors of their experiences such as current or previous uses of financial products like mortgage loans, credit cards, or savings accounts. The Contractor can use this capability to create diverse samples or samples that match the demographics of certain populations (e.g., American adults).

c. Obtain large sample sizes. The Contractor shall recruit large samples of high-quality participants from both convenience samples and samples selected based on demographics.

G. Payment amounts are flexible - The Contractor’s solution shall clearly demonstrate the ability to pay participants amounts according to the needs of each study, including for studies in which participants’ payments are determined by their choices.

H. Payment Time - The Contractor’s solution shall demonstrate the ability to deliver participant payments quickly, within the timeframe specified in the deliverables table (see section B.8 Deliverables).

11 | P a g e

I. Matched Data - Matched data helps CFPB understand how participants compare to the full sample and/or population. The Contractor’s solution shall demonstrate ability to quickly provide accurate matched data, including, at least, whatever data is used to recruit and screen for participants, after each round of data collection.

B.6 REQUIREMENTS

6.1 Required Capabilities

The Contractor shall possess the following capabilities.

6.1.2 Online Panels

The Contractor shall recruit participants from one or more online panels. The Contractor shall be familiar with the demographics of the members of each online panel it uses, and shall be able to provide estimates of incidences and sample sizes for each of the following demographic groups in each of the panels it intends to use in service of this Contract:

A. Sex. Male and female B. Age. 18-29 years old, 30-39, 40-49, 50-59, and 60 years old or older C. Race. White, Black/African American, Asian-American, and American Indian or Alaska

Native D. Hispanic/Latinx and non-Hispanic/non-Latinx E. Education. Less than a college degree, associate’s degree or technical/vocational degree, bachelor’s degree, graduate or professional degree (for example, MA, MSW, MD, JD, PhD)

F. Annual Household Income. Less than $20k, between $20k and $29,999, $30k-$39,999, $40k-$59,999, $60k-$79,999, $80k-$99,999, $100k-$150k, greater than $150k

In addition, in order to confirm the cost of each study before the study is implemented, the Contractor shall be able to provide incidences of demographic groups specified in each study.

The Contractor shall also have a method to determine incidence and study feasibility of a specific group where the incidence and feasibility of the requested participants are not already known within the Contractor’s existing panel(s).

12 | P a g e

6.1.3 Recruitment

All participants recruited by the Contractor shall meet the following criteria:

A. They joined the online panel(s) and participate in studies voluntarily.

B. They are accustomed to participating in studies via the internet. This means, at least, that each member has access to a reliable internet connection and access to a modern browser.

C. They are accustomed to participating in studies that take up to 30 minutes to complete.

D. They are over the age of 18 and reside within the United States.

6.1.4 Screening

The Contractor shall recruit and screen individuals from online panel(s) for participation in CFPB studies. If the Contractor has information about its panel(s)’ members’ demographics and quality (as defined below), then the Contractor may recruit the appropriate individuals using this information rather than screening using additional data collections such as surveys. Screening criteria will be of two types:

A. Demographics. The Contractor shall screen respondents based on respondents’ characteristics, including descriptors of themselves such as sex, race, or age; descriptors of their situations such as status as a veteran, status as a student, employment status, or household income; or descriptors of their experiences such as current or previous uses of financial products like mortgage loans, credit cards, or savings accounts.

B. Participant quality. The Contractor shall screen out low-quality respondents. For example, the Contractor shall be reasonably able to screen out “bots,” respondents from “server farms,” respondents attempting to “retake” the study (either after completing the study once or after already being screened), as well as respondents demonstrating a lack of care or attention in their responses to the screener.2

2 “Bots” are computer programs that automatically complete studies. “Server farms” are used to bypass online panel location restrictions or otherwise disguise their identity. Both are described in Chmielewski and Kucker (2019).

Available at https://journals.sagepub.com/doi/abs/10.1177/1948550619875149

Individuals attempting to retake a study after already completing the study or after being excluded by screening are described in Wessling, Huber, and Netzer (2017). Available at https://academic.oup.com/jcr/article/44/1/211/3737805 https://journals.sagepub.com/doi/abs/10.1177/1948550619875149 https://academic.oup.com/jcr/article/44/1/211/3737805

13 | P a g e

C. The Contractor’s platform shall provide an option for CFPB to exclude participants who completed previous CFPB studies on that platform.

The CFPB does not anticipate asking screening questions in addition to those asked by the Contractor. In the event that a study would require this, this requirement would be detailed in the Study Request and discussed with the Contractor.

6.1.5 Participant Payment

CFPB researchers will specify how participants payments will be determined for each study.

Although some studies may specify fixed payments for all participants who complete a study, other studies will determine payments as a function of participants’ choices in the study. The Contractor must therefore be able to take payment instructions from the CFPB after a participant completes a study and pay participants individualized amounts in accordance with those instructions. In addition, before the study is complete, the Contractor shall confirm with CFPB that participants were paid as instructed.

How participants need to be paid will depend on the type of study:

A. Studies where a participant’s payment is based on their choices/answers in the study. For example, a study might offer different payment incentives based on different options the participant can choose. In this example, one participant might make choices that lead to a $5 payment, while another makes choices that lead to a $6 payment. In these types of studies, the vendor needs to be able to compensate the participant in a specific dollar amount based on the participant’s choices/answers in the study.

B. Studies where a participant’s payment is not impacted by their choices/answers in a study.

Depending on the nature of the recruitment process, the panel may mix for recruitment sources and compensation types, particularly with hard to recruit samples. The vendor has flexibility to recommend a compensation amount and type, and these can vary across participants for the same study. For example, a vendor may recruit some participants using dollar-amounts and others using sweepstakes as compensation.

Measures showing attention, effort, and sincere intention to complete the task are described by Kennedy et al.

(2020). Available at https://www.cambridge.org/core/journals/political-science-research-and-methods/article/shape-of-and-solutions-to-the-mturk-quality-crisis/521AEEB9A9753D5C6038440BD123826C https://www.cambridge.org/core/journals/political-science-research-and-methods/article/shape-of-and-solutions-to-the-mturk-quality-crisis/521AEEB9A9753D5C6038440BD123826C https://www.cambridge.org/core/journals/political-science-research-and-methods/article/shape-of-and-solutions-to-the-mturk-quality-crisis/521AEEB9A9753D5C6038440BD123826C

14 | P a g e

Although the Contractor must be able to perform studies of type (A) above with at least one of its proposed panels, not all of its panels need to meet this requirement.

6.1.6 Matched Participant Data

6.1.6.1 The Contractor shall provide to the CFPB all data collected to determine participants’ eligibility for a study. This includes data collected for individuals who were determined to be eligible as well as those who were not. These data shall include unique identifiers to allow the CFPB to match these data to participants’ responses in the study but shall be otherwise de-identified.

6.1.6.2 If the Contractor screens volunteers using information collected prior to the study (e.g., demographic information collected from all members of an online panel), then the Contractor shall provide:

A. For each screened attribute, the nature of the data used (e.g., survey data or administrative data), a list of all possible values, and, if applicable, individual survey items.

B. For each screened attribute, summary statistics (i.e., mean, minimum, maximum, quartiles, variance, population/sample size) for the entire sampling frame (i.e., those who received, or had access to, the recruitment materials). If a study uses more than one screened attribute, then, upon request, the Contractor shall provide conditional summary statistics.

C. For each screened attribute, individual values (e.g., survey responses) for all individuals who responded to the study’s recruitment materials. As above, these data shall include a unique identifier to allow for matching but shall otherwise be de-identified.

6.1.7 Personnel

A single person may fill multiple roles if that person meets those roles’ requirements. However, contractor may not invoice for multiple roles filled by a single person.

Key Personnel – Research Scientist The Research Scientist shall be responsible for reviewing Study Requests to ensure each study is feasible. The Research Scientist shall have significant experience running online studies, including studies that include tasks for individual participants, such as surveys; as well as studies that include tasks for multiple participants, such as games (e.g., a prisoners’ dilemma game) or economic simulations (e.g., an auction).

15 | P a g e

Non-Key Personnel – Research Associate The Research Associate shall be responsible for carrying-out most tasks required of the Contractor, including providing documentation, as needed, in support of CFPB’s submissions to OMB and providing matched data. The Research Associate shall have experience running online studies, recruiting and screening participants, and paying participants.

6.2 Added-value Capabilities

The following items would add value to the Bureau but are not required. They will help CFPB identify the highest value Offers.

6.2.1 Online Panels

The Contractor provides estimates of the incidence of the following groups in its online panels:

A. Those who are, or those who identify as, current students of an institution of higher education

B. Those who are, or those who identify as, borrowers with outstanding higher education loans C. Those who are, or those who identify as, active-duty members of the United States Armed

Forces or as members of the National Guard or Reserves D. Those who are, or those who identify as, veterans of the United States Armed Forces or as members of the National Guard or Reserves E. Those who have access to a computer/laptop or smart phone to complete studies.

F. Additional demographics or characteristics data about members of its online panels related to financial behaviors and decisions that can be used for targeted recruitment of participants.

6.2.2 Recruitment and Screening

A. The Contractor can provide a sample of participants that matches the demographics of certain populations. For example, the Contractor can provide a sample that matches the demographics of American adults, as measured by the U.S. census. This capability may increase the external validity of CFPB studies run under this Contract.

B. The Contractor can recruit, screen, and distribute the study link to eligible participants within half a business day of the CFPB confirming a study is ready to launch.

6.2.3 Matched Participant Data

16 | P a g e

The Contractor provides additional demographics data about members of its online panels. This allows CFPB researchers to create more succinct study instruments and therefore increases the value of this Contract. In its response to this RFP, the Contactor would provide a list of available demographics. Each Study Request would then specify a subset of this list to be provided for the study. After data collection for a study is complete, the Contractor would then provide data as specified above in section 3.1 (e.g., summary statistics for the full sampling frame, and individual values for all recruited individuals).

6.2.4 Personnel

Research Scientist

A. The Research Scientist has experience running online studies using the Qualtrics platform.

B. The Research Scientist has experience running online studies using oTree software.

Research Associate

A. The research associate has experience running online studies using the Qualtrics platform.

B. The Research Associate has experience running online studies using oTree software.

6.2.5 Institutional Review Board (IRB) Approval

The Contractor may be required to obtain all approvals required to conduct CFPB studies involving human subjects, including obtaining approval from its IRB, if applicable. If IRB is necessary, it will be noted in the Study Request.3

B.7 CONSTRAINTS AND RISKS

7.1 Budget Constraint

The budget for this contract, base and all options, is not expected to exceed $1,824,000.00. The budget estimate includes the amounts that will be paid to participants, including payments that depend on participant choices. The budget ceiling should cover both the vendor’s proposed pricing model and the amounts participants will be paid for the duration of the contract. This budget estimate is based on completion of up to 20 studies over the

3 At the Contractor’s request, CPFB will provide the Contractor with the artifacts to facilitate approvals. These may include, for example, a description of the experiment.

17 | P a g e life of the contract. The actual number of studies and effort may vary depending on the needs of the CFPB.

7.2 Contractor shall adhere to all Security and Non-Disclosure information within the clauses of this RFP.

7.3 All deliverables must be 508 compliant.

B.8 DELIVERABLES

The following deliverables are required under this contract and shall be submitted to the Contracting Officer’s Representative (COR) and Program Manager for the Study Request (PM).

The deliverable should be submitted electronically, unless specified otherwise.

8.1 Contract

The following is a list of deliverables related to this contract.

Deliverable Due Date and Format Kickoff meeting Within 10 business days after award, the Contractor shall meet in-person or remotely with CFPB staff.

Final Quality Control Plan

(QCP)

Within five (5) business days of receiving CFPB feedback, the Contractor shall make adjustments to the QCP. The final QCP shall be delivered within one (1) month after the award.

Intermittent Project Status Reviews and Study Debriefs

As requested, or as necessary, to successfully carry out a study.

The format will depend on the request and can include meetings (in-person or remote), phone calls, and by email.

8.2 Study Task Orders

The following is a list of deliverables related to each study conducted under the contract. To provide a sense of the type of work conducted for the deliverable, examples of possible work requested are included in italics.

18 | P a g e

Deliverable Due Date and Format Study Feasibility, if applicable

If applicable, within eight (8) business days of receiving a request, provide incidences for proposed groups, maximum sample sizes for proposed groups, and other information that will allow CFPB to confirm the cost of running a specific study.

Study Feasibility is intended as an option when a specific group is requested for a study, but the incidence and feasibility of these participants is not already known within the Contractor’s existing panel. This will not be necessary if the incidences and maximum sample sizes are already known by the Contractor, from the initial proposal, or other studies executed under this contract.

Finalized Study Request

1. Initial Study Feedback: Within three (3) business days of receiving a Study Request, the Contractor shall respond with proposed changes, if any, and a proposal for Contractor services to support the study. Examples of work include:

Contractor provides recommendation on proposed changes, which panel(s) to use, recruitment method, and confirms details of payment method and readily available data.

2. Study Meeting: Within five (5) business days of receiving a Study Request, the Contractor shall meet in-person or remotely with CFPB staff to discuss the contractor’s feedback and finalize the Study Request.

3. Finalized Study Request: If changes to materials such as the recruitment or screening materials are required after the Study Meeting, then the Contractor shall make these changes and send the finalized Study Request within five (5) business days of receiving the requested changes.

IRB Approval, if applicable

Prior to PRA approval from OMB (OMB approval typically takes a minimum of 8 weeks), the Contractor shall provide confirmation of IRB approval to carry out the study.

19 | P a g e

Deliverable Due Date and Format Examples of work include: Contractor coordinates and completes submission for IRB review. The CFPB provides the Contractor with information or materials, as needed, for the Contractor to submit the IRB request for approval.

High-Quality Participants

Unless otherwise specified in the finalized Study Request, the Contactor shall begin recruiting for the first wave of data collection (or the pilot/pre-test, if applicable) no later than two

(2) business days after the CFPB confirms a study is ready to launch. The Contractor shall email a confirmation when recruitment has started.

In most cases, the CFPB will directly observe when participants complete a study. In cases where a Contractor is needed to confirm completes, the Contractor shall send a confirmation within the timeframe(s) specified in the Study Request.

Confirmation of participant payment

Within one (1) business day of confirmed participant completion of the study, the Contractor shall pay the participant the amount specified by the CFPB and send confirmation of participant payment. The confirmation shall allow CFPB to confirm the date and amount of payments made for each participant who completed the study but should not include any direct-identifying PII.

De-identified demographic and/or other matched participant data, if applicable

Provided within timeframe as applicable to the Study Request.

The format of data will be determined during Study Request Meeting. The method of secure transfer will be determined after the contract is awarded.

8.3 Inspection - All deliverables will be inspected for content, completeness, accuracy and conformance to order, contract or agreement requirements by the COR, or other authorized representative designated in the order, contract or agreement. Inspection may include validation of information or inspection of the deliverables, as specified in the order, contract or agreement. The scope and nature of this inspection will be sufficiently comprehensive to ensure the completeness, quality and adequacy of all deliverables.

20 | P a g e

8.4 Quality Control Plan (QCP) – The contractor shall create and deliver a complete QCP that identifies how the contractor plans to meet the requirements of the contract relative to its commitment to quality. The contractor QCP shall describe the way in which the contractor will verify and validate the approach that will be taken to ensure the highest quality of the methodology and work products. The QCP shall be approved by the Government and maintained by the contractor to ensure the quality of the project deliverables are met. The contractor shall submit the QCP within 30 days after date of award. Please note that all offerors shall provide a draft QCP as part of its proposal submission.

21 | P a g e

SECTION C CONTRACT CLAUSES

C.1 FAR 52.212-4, CONTRACT TERMS AND CONDITIONS-COMMERCIAL ITEMS

(Oct 2018) is hereby incorporated by reference.

C.2 ADDENDUM TO FAR 52.212-4

2.1 FAR 52.204-21 BASIC SAFEGUARDING OF COVERED CONTRACTOR

INFORMATION SYSTEMS (JUN 2016).

(a) Definitions. As used in this clause– “Covered contractor information system” means an information system that is owned or operated by a contractor that processes, stores, or transmits Federal contract information.

“Federal contract information” means information, not intended for public release, that is provided by or generated for the Government under a contract to develop or deliver a product or service to the Government, but not including information provided by the Government to the public (such as on public websites) or simple transactional information, such as necessary to process payments.

“Information” means any communication or representation of knowledge such as facts, data, or opinions, in any medium or form, including textual, numerical, graphic, cartographic, narrative, or audiovisual (Committee on National Security Systems Instruction (CNSSI) 4009).

“Information system” means a discrete set of information resources organized for the collection, processing, maintenance, use, sharing, dissemination, or disposition of information (44 U.S.C.

3502).

“Safeguarding” means measures or controls that are prescribed to protect information systems.

(b) Safeguarding requirements and procedures.

(1) The Contractor shall apply the following basic safeguarding requirements and procedures to protect covered contractor information systems. Requirements and procedures for basic safeguarding of covered contractor information systems shall include, at a minimum, the following security controls:

(i) Limit information system access to authorized users, processes acting on behalf of authorized users, or devices (including other information systems).

(ii) Limit information system access to the types of transactions and functions that authorized users are permitted to execute.

(iii) Verify and control/limit connections to and use of external information systems.

(iv) Control information posted or processed on publicly accessible information systems.

(v) Identify information system users, processes acting on behalf of users, or devices.

(vi) Authenticate (or verify) the identities of those users, processes, or devices, as a prerequisite to allowing access to organizational information systems.

Section C – Contract Clauses

22 | P a g e

(vii) Sanitize or destroy information system media containing Federal Contract Information before disposal or release for reuse.

(viii)Limit physical access to organizational information systems, equipment, and the respective operating environments to authorized individuals.

(ix) Escort visitors and monitor visitor activity; maintain audit logs of physical access;

and control and manage physical access devices.

(x) Monitor, control, and protect organizational communications (i.e., information transmitted or received by organizational information systems) at the external boundaries and key internal boundaries of the information systems.

(xi) Implement subnetworks for publicly accessible system components that are physically or logically separated from internal networks.

(xii) Identify, report, and correct information and information system flaws in a timely manner.

(xiii)Provide protection from malicious code at appropriate locations within organizational information systems.

(xiv) Update malicious code protection mechanisms when new releases are available.

(xv) Perform periodic scans of the information system and real-time scans of files from external sources as files are downloaded, opened, or executed.

(2) Other requirements. This clause does not relieve the Contractor of any other specific safeguarding requirements specified by Federal agencies and departments relating to covered contractor information systems generally or other Federal safeguarding requirements for controlled unclassified information (CUI) as established by Executive Order 13556.

(c) Subcontracts. The Contractor shall include the substance of this clause, including this paragraph (c), in subcontracts under this contract (including subcontracts for the acquisition of commercial items, other than commercially available off-the-shelf items), in which the subcontractor may have Federal contract information residing in or transiting through its information system.

2.2 FAR 52.216-18 ORDERING (OCT 1995).

(a) Any supplies and services to be furnished under this contract shall be ordered by issuance of delivery orders or task orders by the individuals or activities designated in the Schedule.

Such orders may be issued from date of award through five years, unless modified IAW

FAR 52.217-8.

23 | P a g e

(b) All delivery orders or task orders are subject to the terms and conditions of this contract.

In the event of conflict between a delivery order or task order and this contract, the contract shall control.

(c) If mailed, a delivery order or task order is considered “issued” when the Government deposits the order in the mail. Orders may be issued orally, by facsimile, or by electronic commerce methods only if authorized in the Schedule.

2.3 ADDITIONAL ORDERING INFORMATION

Task orders may be requested by issuing a Performance Work Statement (PWS), Statement of Objectives (SOO) or Statement of Work (SOW).

Task orders utilizing a PWS or SOO will have performance measures either designated by the Bureau or the Contractor. All orders shall include quality assurance monitored by both the Bureau and the Contractor.

Contract Type

IDIQ Task Orders shall be issued on a Firm Fixed Price, Fixed Unit Price, Time & Materials, or Labor Hour basis with cost reimbursable Other Direct Costs (ODCs) (or as a combination of these contract types).

2.4 FAR 52.216-19 ORDER LIMITATIONS (OCT 1995).

(a) Minimum order. When the Government requires supplies or services covered by this contract in an amount of less than $5,000.00, the Government is not obligated to purchase, nor is the Contractor obligated to furnish, those supplies or services under the contract.

(b) Maximum order. The Contractor is not obligated to honor-

(1) Any order for a single item in excess of $750,000.00;

(2) Any order for a combination of items in excess of $1,900,000.00; or

(3) A series of orders from the same ordering office within fifteen (15) days that together call for quantities exceeding the limitation in paragraph (b)(1) or (2) of this section.

(c) If this is a requirements contract (i.e., includes the Requirements clause at subsection 52.216-21 of the Federal Acquisition Regulation (FAR)), the Government is not required

24 | P a g e to order a part of any one requirement from the Contractor if that requirement exceeds the maximum-order limitations in paragraph (b) of this section.

(d) Notwithstanding paragraphs (b) and (c) of this section, the Contractor shall honor any order exceeding the maximum order limitations in paragraph (b), unless that order (or orders) is returned to the ordering office within five (5) days after issuance, with written notice stating the Contractor’s intent not to ship the item (or items) called for and the reasons. Upon receiving this notice, the Government may acquire the supplies or services from another source.

2.5 FAR 52.216-22 INDEFINITE QUANTITY (OCT 1995).

(a) This is an indefinite-quantity contract for the supplies or services specified, and effective for the period stated, in the Schedule. The quantities of supplies and services specified in the Schedule are estimates only and are not purchased by this contract.

(b) Delivery or performance shall be made only as authorized by orders issued in accordance with the Ordering clause. The Contractor shall furnish to the Government, when and if ordered, the supplies or services specified in the Schedule up to and including the quantity designated in the Schedule as the “maximum.” The Government shall order at least the quantity of supplies or services designated in the Schedule as the “minimum.”

(c) Except for any limitations on quantities in the Order Limitations clause or in the Schedule, there is no limit on the number of orders that may be issued. The Government may issue orders requiring delivery to multiple destinations or performance at multiple locations.

(d) Any order issued during the effective period of this contract and not completed within that period shall be completed by the Contractor within the time specified in the order. The contract shall govern the Contractor’s and Government’s rights and obligations with respect to that order to the same extent as if the order were completed during the contract’s effective period; provided, that the Contractor shall not be required to make any deliveries under this contract after 12/31/2029.

2.6 FAR 52.217-8 OPTION TO EXTEND SERVICES (NOV 1999).

The Government may require continued performance of any services within the limits and at the rates specified in the contract. These rates may be adjusted only as a result of revisions to prevailing labor rates provided by the Secretary of Labor. The option provision may be exercised more than once, but the total extension of performance hereunder shall not exceed 6 months. The Contracting Officer may exercise the option by written notice to the Contractor within 60 days of award

25 | P a g e expiration.

2.7 FAR 52.217-9 OPTION TO EXTEND THE TERM OF THE CONTRACT (MAR

2000).

a. The Government may extend the term of this contract by written notice to the Contractor within sixty (60) days; provided that the Government gives the Contractor a preliminary written notice of its intent to extend at least thirty (30) days before the contract expires. The preliminary notice does not commit the Government to an extension.

b. If the Government exercises this option, the extended contract shall be considered to include this option clause.

c. The total duration of this contract, including the exercise of any options under this clause, shall not exceed five sixty-eight (68) months.

2.8 FAR 52.219-13 NOTICE OF SET-ASIDE OF ORDERS (NOV 2011).

The Contracting Officer will give notice of the order or orders, if any, to be set aside for small business concerns identified in 19.000(a)(3) and the applicable small business program. This notice, and its restrictions, will apply only to the specific orders that have been set aside for any of the small business concerns identified in 19.000(a)(3).

2.9 FAR 52.252-2 CLAUSES INCORPORATED BY REFERENCE (FEB 1998).

This contract incorporates one or more clauses by reference, with the same force and effect as if they were given in full text. Upon request, the Contracting Officer will make their full text available. Also, the full text of a clause may be accessed electronically at this address:

https://www.acquisition.gov/.

FAR 52.203-12 Certification and Disclosure Regarding Payments to Influence Certain Federal Transactions (Sept 2007) 52.203-17 Contractor Employee Whistleblower Rights and Requirement to Inform Employees of Whistleblower Rights (Apr 2014).

FAR 52.204-4 Printed or Copied Double-Sided on Postconsumer Fiber Content Paper (May 2011).

26 | P a g e

FAR 52.204-25 - Prohibition on Contracting for Certain Telecommunications and Video Surveillance Services or Equipment (Aug 2020).

FAR 52.204-9 Personal Identity Verification of Contractor Personnel (Jam 2011).

FAR 52.204-13 System for Award Management Maintenance (Oct 2018).

FAR 52.204-18 Commercial and Government Entity Code Maintenance (Jul 2016).

FAR 52.213-2 Invoices (Apr 1984).

FAR 52.216-32 Task-Order and Delivery-Order Ombudsman (Sept 2019). (See CFPB Local Clauses for Ombudsman information) FAR 52.217-6 Option for Increased Quantity (Mar1989).

FAR…