DRAFT OCC Quantiative Trust Survey PWS.docx

Performance Work Statement (PWS) Quantitative Survey on Trust in Banking and Bank Supervision

1.0 Introduction

The OCC requires the Contractor to conduct a survey to collect data on various dimensions of attitudes and experiences of trust in banking and bank supervision from a large, nationally representative sample of U.S. consumers. The purpose of the quantitative survey is to better understand, measure, and track consumers’ trust in banking and bank supervision over time. This effort is a continuation of a Focus Groups Study on Consumer Trust in Banking conducted previously by the OCC. The related PWS for the qualitative research study can be found here.

2.0 Objectives

The primary objective of this PWS is for the Contractor to conduct a nationally representative quantitative survey data collection on trust in banking and bank supervision. As stated in the Request for Comment on Trust in Banking and Bank Supervision (88 FR 37917), the OCC intends to publish the results of the survey in an OCC report to inform policymakers and other stakeholders about the trends and drivers of consumer trust in banking and bank supervision. The OCC, at its own discretion, may decide to use the collected survey data for other purposes not expressed in this PWS.

The Contractor must support the entire lifecycle of the survey data collection which includes:

1. Provide an expert review of the Quantitative Trust Survey Instrument (QTSI) Draft prepared by the OCC. This includes, but is not limited to, providing suggestions for the QTSI, developing the coding instructions, and leveraging results from the OCC’s focus groups study on consumer trust in banking to ensure that the QTSI will meet the goals stated in the OCC’s request for comment on trust in banking and bank supervision (88 FR 37917).

2. Develop cognitive testing protocols, recruit participants, administer cognitive testing, summarize findings and recommendations in a written document, and revise the QTSI in consultation with the OCC.

3. (Optional) Conduct a field test of the QTSI Draft to assess measurement quality, completion time, and response rate of each question.

4. Prepare a Final QTSI.

5. Develop a statistical survey plan (SSP) representative of the United States adult population and oversample any specific sub-group(s) identified.

6. Recruit survey participants in accordance with the requirements described in Task B under section 3.2 below.

7. Administer the Final QTSI and collect the raw data responses.

8. Produce a codebook that maps the responses of the QTSI into a numerical database conducive for statistical analysis using a common statistical software package.[footnoteRef:2] [2: Currently, the OCC uses SAS and Stata to conduct our statistical analysis, but the OCC reserves the right to change the statistical software package by the time the contract is awarded. ]

9. Electronically deliver the raw survey data without personal identifiable information (PII) in a secure mode consistent with the codebook developed.

10. Support the finalization of any U.S. Office of Management and Budget (OMB) packages necessary to administer any of the tasks described above.

3.0 Scope of Work

This contract’s scope must provide for all services and/or products necessary and proper to conduct a survey that meets the objectives in Section 2.0.

3.1 Project Administration

The OCC will appoint a contract officer’s representative (COR) who will be the primary point of contact for the Contractor. The Contractor must work closely with the OCC throughout the life cycle of the survey, from trust survey instrument design and testing to the collection and delivery to the OCC of the survey data. The Contractor must take the lead in executing project tasks with technical direction from the COR. The OCC may take up to two weeks to review final versions of quantitative trust survey deliverables.

Definitions

1. Throughout Sections 1-10 of this document, “Survey information” means all qualitative and quantitative data generated during the survey, preserved in any format (e.g., electronic files, audio and video recording, printout, and survey forms).

2. Throughout Sections 1-10 of this document, “Documentation” means all presentation decks and reports produced during the survey period (e.g., Project Plan and SSP) as well as any artifacts communicating testing results (e.g., Cognitive Testing, Field Tests) and survey evaluation and summary reports.

Kickoff Meeting

Within five business days of the contract award date, the Contractor must schedule and attend a kick-off meeting. Attendees must include the Contractor’s project manager, any required specialized professionals, representatives of any major subcontractors, the COR, and other representatives that the OCC deems necessary. The kick-off meeting will be held at the OCC HQ, virtually, or via conference call. The OCC and the Contractor will discuss the survey objectives and important milestones. The Contractor should include other important agenda items as necessary to meet the expectations identified in the PWS. All key personnel are required to have read the requirements and expectations identified in the PWS prior to the kick-off meeting such as the OCC’s Focus Groups Study on Consumer Trust in Banking Report.

Deliverable (see Section 6, Table 1):

1. Kickoff Meeting Agenda and Key Personnel List

Project Management Plan

After the Kickoff Meeting with the OCC, the Contractor must prepare a draft project management plan covering all tasks, working assumptions, and a schedule of deliverables required in this PWS. In addition, the plan must include status meetings and frequency with time schedules specified for each task described in this PWS. Revisions to the substance of the plan require agreement from both the OCC and the Contractor.

Deliverables (see Section 6, Table 1):

1. Written Draft Project Management Plan

2. Final Project Management Plan

Project Status Meetings and Reporting

To facilitate the successful completion of the survey, the Contractor must provide to the OCC project status reports via bi-weekly pre-scheduled meetings prior to the finalization of the QTSI and weekly update meetings when the QTSI is fielded. The Bi-Weekly/Weekly Project Status Report will include a breakdown of incurred costs, work accomplished, planned action items, outstanding issues, decisions made, and emerging risks that may delay the project timeline. The Contractor shall deliver the Bi-Weekly/Weekly Project Status Report to the OCC two business days before the meeting. Within two business days after the meeting, the Contractor shall prepare and deliver the Bi-Weekly/Weekly Project Status Meeting Minutes that will include summarizing topics discussed, tracking of action items, significant decisions made, issues resolved, and items with decisions required. The COR reserves the right to set additional meetings with the Contractor to address any issues and/or concerns related to the progress of the survey, and to cancel pre-scheduled meetings if necessary.

Deliverables (see Section 6, Table 1):

1. Bi-Weekly/Weekly Project Status Report

2. Bi-Weekly/Weekly Project Status Meeting Minutes

Technical Meetings and Reporting

The Contractor shall hold separate technical meetings with the OCC to discuss plans, identify issues, and reach consensus associated with technical topics, such as QTSI design, Cognitive Testing, Field Testing, SSP, discussions on testing results, as well as Survey Field responses and the analysis. The Contractor shall set up these technical meetings as required under the Tasks A-D described below. Key technical professionals must be present in these meetings. The OCC requires a Technical Meeting Agenda document be submitted two (2) business days before each meeting. This document must describe at a minimum: Topical Plan and Updates, Issues, Action Items and Decisions Made. Technical Meeting Minutes, must be submitted within two (2) business days after each technical meeting held, describing meeting summary with action items, technical issues discussed, and decisions made.

Deliverables (see Section 6, Table 1):

1. Technical Meeting Agenda

2. Technical Meeting Minutes

3.2 Quantitative Trust Survey Tasks and Services

This section discusses the four tasks, Task A – QTSI Development and Testing, Task B – Statistical Survey Plan, Task C – QTSI Deployment, and Task D – Data Provision and Survey Response Analysis.

To perform these tasks, the Contractor must use appropriate personnel with adequate education, skills and experiences as discussed in Section 7.0 Key Personnel. All written deliverables required under the following tasks will undergo at least two reviews, an initial draft review and final review. It is expected that no substantive changes should be required for the draft and the final deliverable version provided to the OCC.

Task A. QTSI Development and Testing

The survey intends to measure customers’ overall perceptions and levels of trust in banking and by specific banking products and services. Task A will consist of several subtasks: A1) QTSI Draft Review and Feedback, A2) Survey Instrument Coding, A3) QTSI Cognitive Testing, A4) Survey Instrument Field Test (optional), and A5) Finalize the QTSI and Codebook Document.

The OCC will provide the Contractor with the OCC’s Focus Groups Study on Consumer Trust in Banking Report and the QTSI Draft after contract award.

Subtask A1. QTSI Draft Review and Feedback

The Contractor shall review the OCC’s Focus Groups Study on Consumer Trust in Banking Report as background information and any other relevant data and research materials to provide feedback on the QTSI Draft. The review must assess the validity of the survey instrument and include recommendations to ensure that the instrument can meet the objectives and goals of the survey. The review will also include identifying any coding instruction errors or coding questions. Any change recommendations for the QTSI Draft must be discussed with the OCC Survey Team. The Contractor shall present its findings to the OCC, discuss areas where the QTSI Draft can be improved, and provide an improvement action plan with details. Once the COR approves a final draft of the survey instrument in writing, the Contractor shall code the survey for initial deployment and allow OCC employees to test the online survey.

Deliverable (see Section 6, Table 1):

1. QTSI Draft Review and Feedback

Subtask A2. Survey Instrument Coding

Upon approval of the QTSI Draft by the OCC, the Contractor must code the Revised QTSI for initial deployment and allow OCC employees to test the online survey. The coding process must convert question responses into numeric values. The survey coding may be more complicated than average, as it will involve some randomization of questions and response options to eliminate sequence or order bias. The OCC expects the average respondent will take a median time of 25 minutes to complete the survey.

Deliverables (see Section 6, Table 1):

1. Coding of the Revised QTSI.

2. Coding Documentation that reflects the coding instructions.

3. Draft of the Codebook Document that maps the question responses to the numeric values reflected in the coded dataset.

Subtask A3. QTSI Cognitive Testing

The Contractor must develop the Cognitive Testing Protocol Draft, specifying the testing methods, the number of interviewers and their qualifications (e.g., experience and training) conducting interviews with participants, the number of analysts (if any) and their proficiency and experience in taking notes during the interview session or preparing interview discussion notes based on audio/video recordings, and the number and type of participants/respondents necessary to recruit for interviews to test survey instrument validity. The Contractor must provide the necessary materials for Office of Management and Budget (OMB) approval, recruit participants, administer the cognitive tests, and provide a summary of findings and recommendations in a written document, and accordingly revise the QTSI in consultation with the OCC. Flexibility is allowed to propose different cognitive testing methods with the associated cost-benefit implications for each method.

After the OCC’s approval of the cognitive testing protocol, the Contractor must finalize the Cognitive Testing Protocol and provide any necessary materials for OMB approval. After OMB approval, the Contractor must recruit participants and conduct cognitive testing. The Contractor must seek OCC pre-approval for each cognitive test. The number of cognitive test(s) will not exceed four rounds. For each round of cognitive testing conducted, the Contractor must provide a written Cognitive Testing Summary Report summarizing the findings and recommendations to revise the QTSI Draft. The Contractor must allow a member of the OCC Survey Team to be present for each of the cognitive tests conducted.

The Contractor must conduct cognitive testing session(s) in accordance with the Final Cognitive Testing Protocol approved by the COR. All video and audio recordings must be sent to the OCC and the Contractor may retain a copy to prepare Interim Discussion Notes based on the findings observed in each round of the testing sessions. In addition, the Contractor shall hold virtual meetings to discuss with the OCC any interim findings after each round of cognitive testing and seek approval from the COR to proceed to the next round of testing.

Deliverables (see Section 6, Table 1):

1. Cognitive Testing Protocol Draft.

2. Final Cognitive Testing Protocol.

3. Cognitive Testing Summary Report.

4. Revised QTSI Draft based on Cognitive Testing Results.

5. Audio and Video (A/V) recordings

Subtask A4. Survey Instrument Field Test (Optional).

After the OCC approves the revised QTSI based on cognitive testing results, the Contractor must provide any necessary materials for OMB approval for survey instrument field testing. Once the OMB approves of the QTSI, the Contractor will deploy the survey in the field to obtain 100 completed QTSI responses. The purpose of this test is to gauge the median time for survey completion, response incidence (i.e., the number of people who responded to answer the survey), and completion rates (i.e., percentage of surveyed people who completed the survey), and determine whether any questions have response issues. Additionally, the Contractor shall conduct data quality checks and provide a description of how the data quality assurance will be performed for both the “Field Test Survey” and the “Final Survey.” (The Data Quality Assurance plan must be provided to the OCC even if the Field Test option is not exercised by the Contracting Officer (CO).)

The deployed survey will display the required OMB control number, expiration date, and descriptive paragraph. This information is required for both the “Field Test Survey” and the “Final Survey.” This information will be provided by the COR in the survey instrument.

The Contractor shall prepare the field-test dataset and report with details on question-specific response times, rate of completion, and any QTSI change recommendations. The report should include any flaws identified in specific questions and their remedies as well as suggested recommendations regarding survey instrument length to ensure a 25-minute median completion time. This field test is an optional task and shall only be performed by the Contractor if approved by the OCC CO.

Deliverables (see Section 6, Table 1):

1. Materials needed for OMB approval for QTSI field testing.

2. Data Quality Assurance plan.

3. Field-test Dataset and Report

Subtask A5. Finalize the QTSI and Codebook Document

After a careful review of the field-test data and the analysis, the OCC may modify the QTSI to edit the survey length, flow, or question clarity. With the final approval from the OCC in writing, the Contractor shall finalize the QTSI, coding instructions, and the Codebook Document.

Deliverable (see Section 6, Table 1):

1. Final QTSI

2. Codebook Document

3. Coding Instructions Document

4. Materials needed for OMB approval for final QTSI field deployment.

Task B. Statistical Survey Plan (SSP)

The Contractor must obtain approximately 10,000 completed survey responses. The Contractor shall create an SSP that demonstrates how the Contractor will attain the target number of completed survey responses while meeting the overall objectives and the relevant parameters of the survey. These parameters include, but are not limited to, sample size sufficiency and representativeness, allocation of different demographic groups (e.g., by age, race/ethnicity, education, and income), pre-established panel usage, survey mode, and timely completion of tasks specified in the PWS according to the time schedule agreed upon between the OCC and the Contractor.

To provide further clarity on what is required within the SSP, the contractor at a minimum must include two Subtasks: B1) Survey Sampling Methodology, and B2) Non-response Follow-up Plan. Each of these two Subtasks is described below in the PWS.

The Contractor shall hold at least two meetings with the OCC to discuss the overall approach to the SSP to ensure agreement and clarity on the plan before it is finalized. Accordingly, the Contractor must present the SSP Draft in the first of these two meetings, providing details on all three components under Subtask B1: 1) Statistical analysis for determining appropriate sample size, 2) sampling frame, 3) potential use of pre-established survey panel and an additional component under Subtask B2, 4) non-response follow-up plan. Following the first meeting, the Contractor shall hold meeting(s) to reach consensus on the issues and concerns raised by the OCC in the prior meeting(s) and finalize the SSP. The Contractor shall present the Final SSP for OCC’s approval prior to implementation. For planning purposes, the Contractor may need to plan for more than two meetings (if the OCC requests additional meetings) to discuss and resolve issues that may arise between the discussion on the Draft and the Final Statistical Survey Plan over a period of two weeks.

Deliverables (see Section 6, Table 1):

1. Statistical Survey Plan (SSP) Draft.

2. Final SSP.

Subtask B1. Survey Sampling Methodology (SSM)

The survey sampling method must be probability-based sampling. Using this method, the Contractor shall provide a detailed plan with analysis on the minimum size of the sample, sample allocation of potential respondents across various population segments, and weighting scheme to comply with the OCC’s objective of conducting a nationally representative survey.

In particular, the Contractor must address the following three components in its analysis of Survey Sampling Methodology (SSM).

1. Statistical Analysis for Determining Appropriate Sample Size

2. Sampling Frame, Survey Sample Allocation, and Weighting Scheme

3. Potential Use of Pre-established Survey Panel

The Contractor shall present the SSM to the OCC for review and approval. The Contractor must address OCC’s feedback, action items, and recommendations before the SSM is finalized.

Deliverables (see Section 6, Table 1):

1. SSM Draft that includes all the components of SSM specified above.

2. Final SSM that incorporates OCC’s feedback and consensus reached upon between the OCC and the Contractor on all the components of SSM specified above Below we discuss additional details on the three components of SSM mentioned under Subtask B1.

B1.1. Statistical Analysis for Determining Appropriate Sample Size

The Contractor must conduct statistical analysis to determine the appropriate sample size to be statistically representative of the U.S. adult population. The Contractor shall have the flexibility to propose survey samples of alternative sizes and the related costs and benefits to deploy the same QTSI and collect responses, fulfilling all the requirements and expectations laid out in this PWS.

The survey samples of alternative sizes (e.g., 5,000 and 7,500 complete responses) must be supported by statistical analysis. If the proposed alternative sample sizes are within the 10,000 complete responses, the Contractor shall advise on the relative merits of choosing a smaller sample size. If the statistical analysis suggests a sample size (e.g., 12,500 complete responses) that exceeds 10,000, the Contractor shall provide additional justification on the extent of potential sampling errors and biases that might occur if a smaller sample size than the suggested one (i.e., 12,500) is chosen.

The Contractor shall present to the OCC an analysis of the alternative sample sizes for OCC’s review and hold discussions for a consensus with the OCC on the exact plan for the sample size. The Contractor shall present the Final SSM to the OCC for its approval before finalizing the SSM and deploying the Final QTSI to the field.

B1.2. Survey Sampling Frame, Sample Allocations, and Weighting Scheme

The Contractor must deploy a nationally representative quantitative trust survey. To this end, it is important to cover consumers from diverse communities and those having different levels of experience with various types of banking products and services and with different types of banking institutions. Therefore, in order to create the sampling frame, the survey recruitment strategy and the SSM must satisfy the following eligibility criteria. [footnoteRef:3] [3: The five characteristics of consumers described may change based on the results and recommendations from the OCC’s focus groups study on consumer trust in banking. ]

Eligibility Criteria of Consumers:

1. From underserved geographic areas: rural, urban, and potentially credit-underserved (i.e., areas with high proportions of consumers with very little or no credit bureau information)

2. With diverse demographic characteristics: by age (young, middle, senior), level of income (low, medium, high), education level (below and above high-school degree) and/or level of financial literacy, and race/ethnicity (e.g., Black and Hispanic).

3. With different levels of banking experience: fully banked, unbanked, and underbanked[footnoteRef:4] [4: Definition: See 2021 FDIC National Survey of Unbanked and Underbanked Households: 2021 FDIC National Survey of Unbanked and Underbanked Households.]

4. Having relationships with various types of financial institutions: banks (national and state), credit unions, fintech, and other non-bank institutions

5. With different levels of experience with various types of consumer banking products and services (BPS): deposit checking/saving accounts, overdraft services, payments and money transfer services, and different types of loans (mortgage, credit card, personal loans, auto loans, other), to mention a few.

The Contractor shall describe in detail the sampling frame, survey allocation, and the weighting scheme. The Contractor shall have the flexibility to use its own sampling frame (i.e., the list of potential respondents from which the survey sample respondents will be drawn) or use a pre-established survey panel (described below in B1.3) or a combination of both. The SSM Draft must include evidence that the sampling frame and sample allocation for key demographic groups is representative of the adult population of the United States. The Contractor must compare each of the key demographic groups, including by gender, age, race/ethnicity, education, income, region, metro/non-metro area, marital status, and household income.

The Contractor shall discuss and reach a consensus with the OCC about any issues and concerns before finalizing the sample allocation to key demographic groups. The OCC’s Focus Group Study on Consumer Trust in Banking Report may provide additional guidance on sample allocation. If the OCC chooses to cover specific demographic groups (e.g., by gender, race/ethnicity, income, education, age, and/or geography) in the survey sample in a proportion different from the national population, the Contractor shall provide a specific and actionable plan with appropriate weighting scheme for administering the survey to achieve OCC’s objectives of conducting a survey to reflect a nationally representative sample. The COR will notify the Contractor two weeks before the SSM Draft is finalized and approved by the OCC.

Since some of these subgroups of the population may be difficult to reach, the OCC anticipates the potential need to over-sample these subgroups to enhance participation in the survey. Additionally, the plan may include specific community partners with whom the Contractor anticipates working with to gather survey responses. Consistent with the objective of a nationally representative survey, the Contractor shall detail the strategy for sampling from each of the key demographic groups and develop appropriate sampling weights to make valid statistical inferences of the underlying population.

The Contractor must develop a set of screening questions based on the proposed five eligibility criteria described earlier. The screeners will also help the OCC evaluate survey parameter fulfillment in a more objective and transparent manner.

B1.3. The Use of a Pre-established Survey Panel

In developing the sampling frame (i.e., potential list of respondents), the Contractor may use a pre-established survey panel (e.g., Census Household Pulse Survey)[footnoteRef:5] for the purposes of gathering survey responses if it meets the stated survey objectives and scope outlined in Subtasks B1-B2. If a pre-established survey panel is used, the Contractor shall include detailed analysis on the justification for the inclusion of a pre-established panel in the sampling methodology. [5: Household Pulse Survey (census.gov), https://www.census.gov/data/experimental-data-products/household-pulse-survey.html.]

Subtask B2. Non-Response Follow-up Plan

To maximize the response rate and obtain the target number of survey respondents, the SSP must describe a follow-up strategy and additional respondent payments needed to pursue potential non-respondents. The Contractor shall pay the offered respondent payments to all respondents only for completed surveys. The OCC expects such payment may increase with accessibility difficulties and participant tardiness in responding. To be able to execute a respondent payment structure consistently, the Contractor shall define “difficult to access” groups of consumers by characteristics such as less than high-school education, racial/ethnic minority group, certain age groups, and low-income households, to mention a few. Since the OCC has decided to use an online survey, the Contractor shall arrange alternatives mode of survey administration (e.g., via telephone or mail) for those who are “difficult to access” or do not have internet access. The Contractor shall ensure the selection of alternative modes of survey administration does not create bias in the consumers’ responses.

Per OCC expectations, the Contractor shall include a detailed plan on alternative modes of survey administration, the respondent payment structure, and the definition of likely non-respondents (i.e., “difficult to access”) to facilitate the smooth execution and timely completion of the survey. The SSP must also include a provision for collecting demographic and socio-economic variables on non-respondents to better understand the underlying factors driving non-responses.

The Contractor shall present to the OCC the non-response follow-up plan in the SSP Draft for OCC review. After review and approval of the SSP by the OCC, the Contractor shall include the Final non-response follow-up plan in the Final SSP and execute the Final SSP as agreed upon between the OCC and the Contractor.

Task C. QTSI Deployment

This task involves administering the survey and collecting the data in accordance with the Final QTSI and SSP (refer to Task A-B and Subtasks) that are approved by the OCC. The Contractor must test the online survey and allow the OCC to do the same before deploying the Final QTSI to the field. The Contractor must execute the entire Final QTSI and SSP as agreed upon with the OCC. The Contractor must complete the fielding of the QTSI within a maximum of three (3) months. The Contractor must provide IDs that allow matching to the future survey responses. There is no explicit sampling frame for repeat respondents.

The Contractor shall provide weekly updates on the progress of data collection, non-response follow-up, issues encountered, and discuss the expected challenges that shall be resolved proactively so that the survey can be completed per the timeline agreed between the COR and the Contractor.

Deliverable (see Section 6, Table 1):

1. QTSI Deployment Weekly Updates Meeting Minutes

2. Future survey response IDs.

Task D. Data Provision and Survey Response Analysis

Under this task, the Contractor shall provide evaluation and statistical evidence on survey data collection, deliver survey response data, and provide post-survey feedback and recommendations, as discussed below, to the OCC after the Final QTSI fielding is completed. There are three subtasks, D1) An Evaluation of Survey Responses with Statistical Evidence on Pre-specified Parameters, D2) Survey Responses Data Delivery, and D3) Post-survey Feedback and Recommendations.

Subtask D1. Survey Responses Evaluation with Statistical Evidence on Pre-specified Parameters

The Contractor shall provide the OCC with an evaluation report containing the statistical analysis that demonstrates that the gathered survey responses meet the specified objectives of the survey, and that any required parameters and the number of responses constitute a statistically valid sample. The Contractor shall compare the response rates and total responses for each of the key demographic groups, including, for example, by gender, age, race/ethnicity, education, income, region, metro/non-metro area, marital status, and household income. The analysis should include both respondents and non-respondents to understand any biases that may have driven the responses and the results.

Deliverable (see Section 6, Table 1):

1. Evaluation report of survey data collection.

Subtask D2. Survey Response Data Delivery

The Contractor shall deliver the Survey Response Data to the OCC electronically (the file format shall be specified and agreed upon with the OCC prior to delivery). Consistent with the Final Codebook Document, the dataset must be cleaned and coded with appropriate values and variable labels that can be easily analyzed in a statistical package (e.g., Stata). The dataset must also contain the response times for each respondent, demographic and socio-economic variables of respondents, as well as individuals who began the survey but did not complete it (“break-offs”) and panel members who received email solicitations to participate in this survey, but did not respond. The Survey Responses Data must also include the appropriate sample weights for the entire sample to enable appropriate inferences about the underlying respondent population. Additional details and the specific format of electronic delivery will be discussed and finalized by the OCC and the Contractor at the time of contract award.

Deliverables (see Section 6, Table 1):

1. Survey Response Data Delivery: Individual-level Survey Data, coded with weights, and excluding the respondents’ names or address. The coded structured data must be ready for statistical analysis in standard statistical software (e.g., Stata).

2. Survey Response Codebook Document: mapping the values or labels of responses from the survey instrument to the values or labels assigned in the raw survey data.

Subtask D3. Post-survey Feedback and Recommendations

The Contractor shall provide a presentation on post-survey feedback and recommendations. The presentation must include, but is not limited to, lessons learned, any obstacles encountered, reasons for any deviations from the original plan, adopted solutions, and future recommendations. The presentation must also include analysis of key survey statistics such as recruitment rate, profile rate, completion rate, and cumulative response rate.

Deliverable (see Section 6, Table 1):

1. Presentation on Post-survey Feedback and Recommendations.

4.0 Facilities, Staffing, Equipment, and Respondent Payments for Conducting Quantitative Trust Survey

The Contractor shall provide all the physical resources and staffing to complete the services for each of the Tasks and Subtasks specified above. All the work shall be performed at the Contractor’s facilities. The Contractor shall provide necessary logistics related to recruiting survey participants, project coordination, and survey administration. The Contractor shall ensure the availability of all office supplies, equipment, computer hardware and software, internet connections and associated services, and facilities needed to perform the Tasks and Subtasks specified above. Finally, the Contractor shall make necessary arrangements for respondent payments to all respondents only for completed surveys.

5.0 Period of Performance

The OCC expects the kick-off meeting of the Quantitative Survey on Trust in Banking and Bank Supervision to be at the beginning of 2025. The dates and timelines presented reflect an estimated projection. The final dates will be communicated, established, and agreed upon by both the OCC and the Contractor at the beginning of the work. The Contractor is required to complete all the work and deliver all results by the dates listed in Table 1 below. The timeline presented below is based on the Contractor starting the work within five (5) business days of the contract award date. Table 1 identifies and describes deliverables with acceptance criteria and due dates.

Once the QTSI (Task A), and SSP (Task B) are finalized, the OCC will submit the Paperwork Reduction Act (PRA) Clearance package for OMB Approval. The PRA package submission is expected to occur approximately 16-18 weeks after the Kick-off meeting with the Contractor. The OMB review period and approval is projected to be 60 days from the date of PRA Clearance submission. The PRA Clearance submission package must include the Final QTSI and SSP.

Prior to the OCC’s submission of the PRA Clearance package, the Contractor shall complete the following milestones: the Project Management Plan, the QTSI Development and Testing described under Task A, and the SSP described under Task B in Section 3.0. Both Task A and Task B have many Subtasks that need to be completed as well.

Assuming the OCC receives OMB approval by the end of July 2025, the Contractor should be able to launch the survey in mid-August 2025 and field the survey for approximately three months. The whole process and the Trust Survey will be concluded in December 2025.

6.0 Summary of Deliverables, Acceptance Timeline, Due Date, and Acceptable Criteria for Contract Deliverables

The timeline of contract deliverables, milestones, and acceptability criteria of each deliverable are given in Table 1 below.

Table 1 – Timeline with respect to Contract Deliverables

Kickoff Meeting Agenda/Key Personnel List:

No later than five (5) business days after the Contract is awarded by the OCC.

COR and OCC Team Lead

Five (5) business days after the kick-off meeting.

COR and OCC Team Lead

COR and OCC Team Lead

Coding of Revised QTSI, Coding Instruction Document and Draft Codebook Document

COR and OCC Team Lead.

COR and OCC Team Lead

Cognitive Testing Summary Report

1. Provide clear results of each round of the cognitive testing session.

2. Contain well-reasoned actionable advice and recommendations on appropriate wording of survey questions, ordering of responses within a question, and the ordering of questions.

3. Discuss how each round of testing has addressed the issues identified in the previous round to improve the gap between the intent of the question and the interpretation by the interviewees participating in the testing interview.

4. Include all interim findings and discussions with the OCC about recommendations for appropriate revisions to the QTSI Draft.

Five (5) business days after the OCC approves the final findings of the Contractor’s last round of Cognitive Testing.

COR and OCC Team Lead.

Data Quality Assurance Plan (The Data Quality Assurance plan must be provided to the OCC even if the Field Test option is not exercised by the CO.)

Ten (10) business days after online Operation Test is completed and discussed with the OCC Survey Team.

COR and OCC Team Lead.

Ten (10) business days after online Operation Test is completed and discussed with the OCC Survey Team.

COR and OCC Team Lead.

COR and OCC Team Lead.

SSP Draft must include analysis on all Subtasks (i.e., Subtask B1-Survey Sampling Methodology (SSM), Subtask B2-Nonresponse Follow-up Plan, and the required deliverables described in Section 3.2 (B1-B2). The Subtask B1-SSM has three components described in Section 3.2 (B1.1), 3.2 (B1.2), and 3.2 (B1.3). Each of these components has its own deliverables and the SSP Draft must meet those requirements as well. The SSP must be discussed with the OCC to address any issues/concerns before finalizing the Draft.

Final SSP must be written in detail with clarity and be easy-to-understand. It should include analysis of the sample size, sampling frame, sample allocation, weighting scheme, screening questions, follow-up strategy, and survey mode of execution. The Contractor shall carefully review the deliverable requirements explicitly stated in relevant tasks, subtasks, and components under Section 3.2 (B1-B2).

Five (5) business days after the Final QTSI and Codebook Document is finalized and delivered to the OCC.

COR and OCC Team Lead.

OMB Approval Package

COR and OCC Team Lead.

COR and OCC Team Lead.

COR and OCC Team Lead.

Two (2) weeks after the Survey Response Data is delivered.

COR and OCC Team Lead.

During the kick-off meeting, the Contractor shall discuss if there are any challenges with meeting the proposed timeline above; the contractor shall identify challenges and propose revised delivery date(s) within ten (10) days of the kick-off meeting. Any proposals shall fit within the temporal parameters defined in Section 4.0. The OCC will ultimately provide approval to revised proposals and the final timeline.

7.0 Key Personnel: Educational, Experience, and Skill Requirements.

The Contractor shall identify key personnel with specialized skill sets and experience to perform certain tasks described in this PWS. These include the QTSI Draft review, cognitive testing protocol development, analyzing the patterns in responses, writing questions, developing sampling methodology, and project management. The OCC recognizes it is likely that the Contractor may identify one personnel who can perform multiple specialized tasks described above. For example, a survey sample specialist/statistician may have also training, skills, and experience in cognitive testing, survey instrument design, and questions writing. The OCC anticipates that the successful performance of this work will require the special expertise and associated qualifications listed in the table below.

Table 2: Key Personnel: Educational, Experience, and Skills Requirements

Lead Researcher /Survey Lead Review and Evaluate OCC’s Focus Groups Study on Trust in Banking Report and QTSI Draft, develop Cognitive Testing Protocol and analyze testing results, develop Field Testing and Survey Instrument Design, and Writing Questions.

· Master's Degree in a field of behavioral science (e.g., Economics, Psychology, Cognitive Science, or similar fields)

· At least three (3) years of recent experience conducting a variety of quantitative survey techniques, including survey instrument design, writing questions, cognitive testing protocol development, and field testing.

· At least three (3) years of recent experience in designing, ordering, and contextualizing survey instruments and writing questions related to consumer financial services and products and/or experience reaching low- and moderating income consumers and other difficult to reach consumer segments.

· At least two reports with a description of their role and contributions to reflect their mastery in quantitative analysis, organizing and interpreting findings, and writing reports in a clear, concise, and effective manner.

In addition to the minimum qualifications, preferred/highly valued key personnel will have one or more of the following:

· Ph.D. in a field of behavioral science (e.g., Economics, Psychology, Cognitive Science, or similar fields) relevant to the work under this contract.

· At least six (6) years recent experience conducting a variety of quantitative survey techniques, including survey instrument design, cognitive testing protocol development, and field testing.

· At least one (1) year of experience working at the current contract firm that is submitting this proposal.