2020_RECS_PWS_Final_revised.pdf

8930319REI000036.0000

Attachment 1

PERFORMANCE WORK STATEMENT (PWS)

THE U.S. ENERGY INFORMATION ADMINISTRATION (EIA)

OFFICE OF ENERGY STATISTICS

2020 Residential Energy Consumption Survey (RECS): Household Survey

1. PURPOSE:

Energy Information Administration’s (EIA) Residential Energy Consumption Survey

(RECS) program is responsible for collecting, analyzing, and disseminating timely, detailed information about how energy is being used within the residential sector of the economy. This includes data on the fuels used in homes, equipment and appliance stocks, household behaviors associated with energy use, and total and disaggregated consumption and expenditures. EIA sponsors multiple data collections as part of the

RECS program and utilizes detailed analyses to produce meaningful information to stakeholders and the data user community. The results are used for program evaluation and planning; to track product adoption and research emerging markets; and to project future energy needs in the sector. Results are disseminated across a variety of platforms for novice, intermediate, and expert data users and include: data tables, a microdata file, detailed documentation, summary reports, featured articles, conference presentations, webinars, and social media campaigns.

Under this task order, the Contractor shall provide all personnel, equipment, supplies, facilities, transportation, tools, materials, systems, supervision, and other items and non-personal services, except for those items specified as government furnished property and services, necessary to collect and process detailed information about residential energy characteristics in the United States as defined in this Performance Work Statement

(PWS). The Contractor shall perform to the standards in this task order. The Government shall not exercise any supervision or control over the contract service providers performing the services herein. Such contract service providers shall be accountable solely to the Contractor who, in turn is responsible to the Government.

2. SCOPE:

Services included in this task order draw from four major EOPIV service areas: Program

Management, Survey Operations for EIA’s Non-Supply Based Surveys, Energy Analysis and Modeling, and Organizational Support. Services shall support execution of the 2020

RECS Household Survey. The specific activities to be conducted and expected accomplishments are fully detailed below in Section 5. Performance Objectives and

Section 7. Specific Tasks.

3. PERIOD AND PLACE OF PERFORMANCE:

3.1 Period of Performance:

The period of performance shall be for approximately 30 months from the date of the award.

3.2 Place of Performance:

The principal place of performance under this task order will be at the Contractor’s facilities. The Contractor may be required to travel to the Department of Energy (DOE)

Headquarters, field offices in the DOE complex and other agencies of Federal, State and

Local governments, in support of work under this task order.

4. BACKGROUND and Overview:

The RECS is a nationwide, benchmark data collection that supports EIA’s energy statistics and analysis programs on residential energy use. EIA began collecting and disseminating household energy usage data in 1978. The 2020 RECS will be the fifteenth iteration of EIA’s household energy usage study.

EIA collects detailed data on residential demand in two phases. The household and housing unit energy characteristics data are gathered from a sample of households during the RECS Household Survey phase. In addition to energy characteristics and usage indicators, the Household Survey questionnaire captures fuel and supplier information from respondents, which serves as the frame for the follow-on Energy Supplier Survey

(ESS) phase. EIA annualizes the data collected from the ESS and, in conjunction with the household characteristics, produces disaggregated energy end-use estimates.

This request is for work on the Household Survey alone. Subsequent phases of the RECS, which include the ESS and end-use analysis, are to be considered in the proposed design of the household phase and response to this request for work. For example, the ESS data collection relies on quality responses to the Household Survey fuel questions and to the request for customer account numbers.

The target population for RECS consists of primary, occupied housing units in the 50 states and the District of Columbia during the reference period, which for this work is the

2020 calendar year. The population excludes vacant homes, seasonal homes, and common areas in apartment buildings.

EIA intends to utilize self-administered web and mail questionnaires as the primary modes for the 2020 RECS Household Survey data collection. Since 2009 RECS, EIA utilized in-person interviewing almost exclusively to collect the Household Survey data from respondents Computer-Assisted Personal Interviewing (CAPI) since 1997. Prior to

1997, paper administration was used.

EIA has conducted significant research in transitioning the RECS to self-administered modes of collection. Between the 2009 and the 2015 RECS studies, EIA conducted a series of pilot surveys to test the feasibility of alternatives to the in-person method.

Specifically, EIA conducted three field tests using self-administered web and mail data collection modes. Results of these experiments revealed that web and mail modes could produce results of comparable quality to the traditional, in-person studies. Subsequent to the third pilot test (the “National Pilot”), EIA decided to use these new web and mail modes to complete the official 2015 RECS Household Survey. Approximately two-thirds of the final responding cases for the 2015 RECS Household Survey were collected using web or mail, with the remaining third collected using the traditional CAPI instrument.

The proven cost efficiencies and effectiveness of web and mail modes will allow EIA to conduct the largest study in the history of the RECS program. To date, the 2009 RECS, which produced estimates for the U.S., census regions, census divisions, and 16 individual states from a responding sample of 12,083 households, was the largest study in program history. EIA expects that the 2020 RECS design and data collection will yield a responding sample large enough to produce estimates for up to 25 states, in addition to estimates for the U.S., census regions, and census divisions.

While geography (states) forms the basis of key subpopulations for the 2020 RECS study, additional subpopulations are of interest for RECS data users. Therefore, EIA also intends to improve the quality of estimates for these other two areas: households using natural gas, propane, and fuel oil and for key household electricity measures.

Beginning with 2009 RECS, EIA and its contractor began using the United States Postal

Service (USPS) Delivery Sequence File (DSF) as the primary source for constructing a

Household Survey frame. EIA expects the DSF to be used as the primary source for the

2020 RECS Household Survey frame. A sample of housing units that meets or exceeds precision targets for average household energy consumption and other key RECS estimates will be drawn from the frame. The optimal design will consider tradeoffs in quality, feasibility, and cost.

All 2020 RECS Household Survey data shall be acquired and collected under the

Confidential Information Protection and Statistical Efficiency Act (CIPSEA) of 2002, which allows and requires EIA and its agents, including data collection contractors, to protect identifiable data from disclosure or release to unauthorized parties for nonstatistical purposes. Therefore, as an integral part of the task, the Contractor shall maintain confidentiality of the data at all times and provide CIPSEA training and certification for all contracting staff.

5. PERFORMANCE OBJECTIVES:

The Contractor shall achieve the following performance objectives:

Manage all aspects of the 2020 RECS Household Survey including planning, scheduling, risks, quality assurance, confidentiality, security, budgeting, cost control, and reporting.

Perform robust quality control to meet project requirements.

Provide monthly cost and status reports to the Program Office as well as daily progress reports during data collection.

Design a study to yield 15,000 to 18,000 completed surveys.

Develop a frame construction, sample design, data collection, and weighting strategy to yield quality estimates of key household energy metrics and subpopulations for occupied, primary housing units/households. The responding sample shall yield the following relative standard error (RSE) precision levels:

o For “average total site household energy consumption”

US: 1%

All census regions: 2%

All census divisions: 3%

20-25 individual states: 4% o For “average site household electricity consumption”

US: 1%

All census regions: 2%

All census divisions: 3% o For “average household natural gas consumption”

US: 1%

All census regions: 2%

All census divisions: 3% o For “average household propane consumption”

US: 3%

All census regions: 4% o For “average household fuel oil consumption”

US: 3%

Northeast census region only: 4%

Utilize the USPS Delivery Sequence File (DSF) as the basis for the RECS

Household Survey sample frame and propose methods that account for DSF frame overcoverage and undercoverage.

Conduct questionnaire and contact material pretesting and usability activities.

Implement innovative solutions to collect data and/or estimate housing unit square footage.

Collect RECS Household Survey data using self-administered web and mail questionnaires and achieve an unweighted response rate of at least 40%

(AAPOR RR3).

Implement a data management and metadata strategy to track the data lifecycle and improve capabilities for internal and external users. This strategy shall include a sample (case) management system that integrates sample tracking and survey control features with the survey instruments.

Design specifications for and execute an item imputation strategy for missing

RECS Household Survey responses.

Deliver frame, sample, response, imputed, weighting, and survey control/analytics files to the Program Office before, during, and following the data collection period, as needed.

Provide plans and final documentation for all phases of the 2020 RECS

Household Survey.

Develop and maintain an inventory of materials used to conduct the 2020

RECS Household Survey. At the end of the performance period, the

Contractor shall transfer the inventory as well as all survey materials and documentation, both hard copy and electronic files, to a location specified by the Program Office.

6. OPERATING CONSTRAINTS:

6.1 CIPSEA:

EIA and its Contractor shall protect 2020 RECS Household Survey information under

Title V of the E-Government Act, Confidential Information Protection and Statistical

Efficiency Act of 2002 (CIPSEA). The Contractor shall comply will all applicable provisions of CIPSEA.

6.2 Quality Control:

The Contractor shall develop and maintain an effective quality control program to ensure services are performed in accordance with this PWS. The Contractor shall develop and implement procedures to identify, prevent, and ensure non-recurrence of defective services. The quality control program assures that the work complies with the requirements of the contract.

6.3 Physical Security:

The Contractor shall be responsible for safeguarding all government equipment, information and property provided for Contractor use.

6.4 Post-Award Conference

The Contractor agrees to attend a post-award conference convened by the Contracting

Officer (CO) in accordance with Federal Acquisition Regulation Subpart 42.5. This conference will be scheduled as the first session of the Task Order kick-off meeting/project planning workshop specified under Subtask 1.1 of Section 7: Specific

Tasks.

6.5 Key Personnel:

This Task Order requires no specifically named Key Personnel.

6.6 Identification of Contractor Employees:

All contract personnel attending meetings, answering Government telephones, and working in other situations where their contractor status is not obvious to third parties are required to identify themselves as such to avoid creating an impression in the minds of members of the public that they are Government officials. They must also ensure that all documents or reports produced by contractors are suitably marked as contractor products or that contractor participation is appropriately disclosed.

6.7 Contractor Travel:

Anticipated contractor travel may be required in the performance of contract support.

Costs for transportation, lodging, meals, and incidental expenses incurred by contractor personnel on official company business are allowable, subject to the limitation contained in Federal Travel Regulations, prescribed by the General Services Administration, for travel in the contiguous United States.

Costs for transportation may be based on mileage rates, actual costs incurred, or on a combination thereof, provided the method used results in a reasonable charge. Costs for lodging, meals, and incidental expenses may be based on per diem, actual expenses, or a combination thereof, provided the method used results in a reasonable charge.

To avoid possible subsequent disallowance or dispute based on unreasonableness, unallocability or unallowability under the specific cost principles of travel costs, an advance travel agreement between the contractor and the designated COR shall be negotiated in writing before or during a contract performance. Ordinarily, all travel shall be approved by the COR in writing via memorandum or e-mail prior to the Contractor incurring any travel expenses and prior to any departure. However, due to the unpredictability of travel occurrence, frequencies, and location based in the performance of task requirements, contractors are allowed to travel within the confines of the negotiated advance travel agreement.

A negotiated advance travel agreement must be incorporated into applicable contracts and shall contain a statement of its applicability, mode of transportation, and duration.

6.8 Other Direct Costs:

The Government anticipates that execution of the 2020 RECS Household Survey will require ODCs and that both the types and amount of ODCs will vary based on the

Contractor’s proposed technical approach. ODCs types may include, but are not limited to travel, printing, mailing, shipping, computing, communication, respondent incentives, data purchases, and translation services.

6.9 Organizational Conflict of Interest:

Contractor and subcontractor personnel performing work under this contract may receive, have access to or participate in the development of proprietary or source selection information (e.g., cost or pricing information, budget information or analyses, specifications or work statements, etc.) or perform evaluation services which may create a current or subsequent Organizational Conflict of Interests (OCI) as defined in FAR

Subpart 9.5. The Contractor shall notify the Contracting Officer immediately whenever it becomes aware that such access or participation may result in any actual or potential OCI and shall promptly submit a plan to the Contracting Officer to avoid or mitigate any such

OCI. The Contractor’s mitigation plan will be determined to be acceptable solely at the discretion of the Contracting Officer and in the event the Contracting Officer unilaterally determines that any such OCI cannot be satisfactorily avoided or mitigated, the

Contracting Officer may affect other remedies as he or she deems necessary, including prohibiting the Contractor from participation in subsequent contracted requirements which may be affected by the OCI.

6.10 Subtask List and Dates

Subtask # Subtask Title Optional Y/N Start Date End Date

Plan and Manage 2020 RECS Household Survey N Start of award End of Task Order

Develop RECS Household Survey Frame and Select Sample N

2 weeks after award July 2020

Review, Test, Draft, and Finalize Contact Materials and Web and Mail Instruments N

2 weeks after award June 2020

Develop Data Management and Metadata Strategy N

2 weeks after award August 2020

Collect and Deliver RECS Household Survey Data N July 2020 April 2021

Perform Missing Item Imputation N November 2020 July 2021

Develop Weighting Adjustment Strategy and Calculate Replicate Weights N January 2021 September 2021

Deliver Final Report and Perform Closeout Activities N June 2021 February 2022

Expand 2020 RECS Household Survey Scope to Include Estimation for all 50 states and DC Y July 2020 April 2021

Adjust and/or Extend Data Collection Y December 2020 June 2021

Conduct Rental Agent Survey Y TBD TBD

Develop Methods for Multifamily Buildings Study Y TBD TBD

Conduct Pilot Multifamily Buildings Study Y TBD TBD

Note: Start and end dates in the Subtask table reflect the Government’s best estimate for the start and end date for the subtask activities. These dates may be revised as project planning phases of the Task Order progress.

7. SPECIFIC TASKS:

The Contractor shall perform the following specific tasks.

Subtask 1: Plan and Manage the 2020 RECS Household Survey

The Contractor shall provide Contract and Task Order management support services.

This includes but is not limited to the management and oversight of all activities performed by Contractor personnel, including subcontractors, to satisfy the requirements identified in this PWS. The Contractor shall identify a Task Order Manager and Project

Controller who will provide management, direction, administration, quality assurance, and leadership for the execution of this Task Order.

The Contractor shall undertake the day-to-day management of the 2020 RECS Household

Survey, including project management control. Project management control shall include developing a baseline project schedule; managing all Contractor and subcontractor activities; scheduling project meetings; storing and archiving project materials; and maintaining statuses of completion of all elements of the schedule, including milestones and deliverables. Project management control shall also include identifying and estimating risk exposure with regard to schedule, quality, and cost for certain risk scenarios. The Contractor shall work with the Program Office to develop and maintain project management assets (e.g. planning documents, project schedule, risk plan, risk register, staffing plan, and communication plan) sufficient to support this project management control.

Subtask 1.1: Task Order kickoff and project planning workshop

The Contractor shall schedule and coordinate a Task Order kick-off meeting at a location approved by the Program Office to be held within three weeks of contract award. The meeting will include an introduction between the key Contractor and Program Office personnel and an opportunity to discuss key project management documents, security issues, reporting procedures, and meeting schedules. At a minimum, the attendees shall include the Contractor’s key personnel, representatives from EIA program and staff office(s), and the COR for this Task Order. Following the meeting, the Contractor shall provide a List of Attendees and Minutes.

The Contractor shall provide a draft Monthly Status Report (MSR) at the kickoff meeting.

Subtask 1.2: Prepare project management planning documents

The Contractor shall develop a series of project management planning documents detailing how the Contractor plans to execute and monitor all work required in this Task

Order.

The Task Order kickoff and project planning workshop shall result in the following baselined project management documents:

Staffing Plan: list of planned contractor and subcontractor staffing and hours by month for all subtasks and in total

Communication Plan: list of key roles, staff assignments, and appropriate contact information for the Contractor and Program Office, and how the Contractor and

Program Office will interact and resolve issues throughout the Task Order

Work Breakdown Structure (WBS) and Project Schedule: a detailed list of tasks, durations, milestones, deliverables, and dependencies of the work required to meet requirements for each subtask

Risk Plan and Risk Register: description of how risk management will be performed and a detailed register of potential project risks, impacts, and management strategies

The Contractor shall provide a draft of the WBS at the planning workshop to facilitate discussion of the detailed subtasks. If necessary, additional planning workshops may be scheduled to complete the baselined versions of these documents.

The Program Office shall review and approve the baselined versions of the above project management planning documents. The initial baseline documents may be updated as necessary throughout the performance of the Task Order. The Contractor shall make revised versions of the documents available as they are updated throughout the performance of the Task Order and deliver final versions at the end of the Task Order.

Revisions may include, but are not limited to, updates to activity completion status, further decomposition of later stage activities, and updates to the risk register.

Subtask 1.3: Plan for security, confidentiality, and quality

To the extent that the work under this task requires that the Contractor or the Program

Office have access to information collected in accordance with the Confidential

Information Protection and Statistical Efficiency Act of 2002 (Title V of the E-

Government Act of 2002, Public Law 107-347) (CIPSEA), the Contractor and the

Program Office shall treat such information as confidential and use the information exclusively for statistical purposes as set forth in the Act.

The Contractor shall design and implement procedures for protecting the confidentiality of respondent identification as specified in the Contract. These procedures must be compatible with the codes and standards of the Council of American Survey Research

Organizations and the American Association of Public Opinion Research as well as

CIPSEA. The Contractor shall submit a Data Security and Confidentiality Plan to the

Program Office detailing how the data confidentiality procedures shall be implemented.

The confidentiality procedures shall include the need for the Contractor to:

Secure all data that would allow identification of survey respondents

Provide for secure transmission of all identifiable sample and respondent data

Train affected personnel in the procedures to be followed in maintaining confidentiality

Plan for disaster recovery in the event of a system shutdown

Provide secure areas for processing the data and locked files for storage of returned hard copy materials. The procedures must also be designed to allow

Program Office personnel access to the confidential data through secure data transmission.

Secure the necessary CIPSEA confidentiality pledges. All Contractor employees that have access to identifiable information collected in accordance with this Act shall undergo training in the rights and responsibilities conveyed by the Act and the penalties for noncompliance, and shall sign a written certification that they have taken the training and understand and agree to comply with its provisions.

The Contractor shall also submit a Quality Assurance Plan (QAP) to demonstrate the means by which the Contractor will ensure data quality and confidentiality, and establish and track performance metrics/measures.

Subtask 1.4: Provide Monthly Status Reports

The Contractor shall produce and deliver a Monthly Status Report (MSR) in accordance with the EOP IV base Contract reporting requirements. The MSR shall be delivered via e-mail by the 10th of each month to the EIA Contracting Officer (CO) (or designee) and to other Program Office personnel as specified in the Communication Plan.

The MSR is due on the 10th of each month for the previous calendar month. This monthly report shall include for each subtask:

List of accomplishments

Brief description of work in progress

Items requiring resolution

Additional pertinent items

Additionally, the MSR shall contain a section on the financial status of the project, monthly projections for the life of the Task Order, and discussions of any significant deviations from the planned project budget.

Subtask 1.5: Convene monthly project status meetings

The Contractor shall provide an agenda and convene monthly project status meetings to discuss key decisions, performance on each phase of the project, the risk register, and the project schedule. Attendees should include the Task Order Manager, Project Controller, COR, Program Office management staff, and other key personel as needed. Meetings shall be by teleconference or in person, as needed. The Contractor shall provide a List of

Attendees and Minutes within 3 business days.

Subtask 1.6: Convene technical status meetings

The Contractor shall provide an agenda and convene weekly (or more often, as required) technical status meetings with the Program Office and shall provide a List of Attendees and Minutes within 3 business days. Separate meetings may be required for sample design, imputation, weighting, and any other specialized aspects of the project. Meetings shall be by teleconference or in person, as needed.

Subtask 1.7: Document key decisions

The Contractor shall maintain an inventory of all correspondence that details key decisions for this Task Order that are not captured in formal planning or report documents. This includes memos, emails, and summaries of telephone conversations.

Subtask 2: Develop RECS Household Survey Frame and Select Sample

The Contractor shall develop a Frame and Sample Design Plan. The plan shall describe how the Contractor will develop a frame construction, sample design, data collection, and weighting strategy to yield quality estimates of key energy metrics for occupied, primary housing units/households in the United States. Occupied, primary housing units excludes vacant homes, seasonal homes, group quarters, and common spaces in apartment buildings.

The frame construction, sample design, and data collection shall yield approximately

15,000 to 18,000 completed surveys. As part of the study design, the Contractor shall target at least some completed interviews in every state, including those not explicitly targeted for estimation. Requiring coverage in all states ensures at least some households from each state are included in EIA’s national estimates of household energy use.

The frame construction, sample design, data collection, and weighting strategy shall yield quality estimates of key household energy metrics and subpopulations for occupied, primary housing units/households. The responding sample shall yield the following RSE precision levels:

For “average total site household energy consumption” o US: 1% o All census regions: 2% o All census divisions: 3% o 20-25 individual states: 4%

For “average site household electricity consumption” o US: 1% o All census regions: 2% o All census divisions: 3%

For “average household natural gas consumption” o US: 1% o All census regions: 2% o All census divisions: 3%

For “average household propane consumption” o US: 3% o All census regions: 4%

For “average household fuel oil consumption” o US: 3% o Northeast census region: 4%

RSEs achieved for the 2009 and 2015 RECS are provided as a reference document for this PWS.

The Contractor shall use 2015 or any prior RECS data, EIA’s State Energy Data System

(SEDS) residential sector data, American Community Survey (ACS) data, or alternative data series or methods as potential sample design sources for RSEs and average consumption. “Total site household energy consumption” shall be defined as the sum of site electricity consumption, natural gas consumption, propane consumption, and fuel oil consumption. Averages for natural gas, propane, and fuel oil shall be calculated based on the households using the fuel.

The 20-25 states targeted for individual estimation will be as follows:

The 16 states reported for the 2009 RECS: MA, NY, PA, NJ, IL, MI, WI, MO, VA, GA, FL, TN, TX, CO, AZ, and CA

4-9 additional states chosen based on geographic diversity, climate diversity, housing unit population, a unique subpopulation of interest, and/or state energy policies

Proposals shall include an initial list of states and explanations for why they were proposed. The actual number and list of states will be finalized during sample design in consultation with Program Office staff.

In addition to state selections, the Contractor shall propose a sample design solution that balances the number of in-scope completes, frame and sample design quality, and operational feasibility, while attaining the RSE precision targets. EIA will consider all these dimensions when evaluating proposals on technical merit.

The Contractor shall utilize the USPS Delivery Sequence File (DSF) as the primary source for the 2020 RECS Household Survey sample frame. Merging auxiliary housing unit address lists and other methods may be used to supplement the DSF. As part of the

Frame and Sample Design Plan, the Contractor shall include a comprehensive assessment of the frame quality (e.g. duplication and undercoverage), as well as solutions to account for frame error. The Contractor shall consider tradeoffs in frame quality, operational feasibility, coverage of subpopulations, and cost when proposing a frame construction strategy.

Traditionally, the RECS sample designs have used multi-stage sampling where primary sampling units (PSU) and secondary (segment) geographic clusters were selected prior to the ultimate housing unit selections. PSUs and segments were selected within explicit geographic domains, which were identified as states or groups of states. These multi-stage design elements facilitated in-person interviewing, which required clustering to control data collection costs, and facilitated in-person listing to construct all or portions of the sampling frame. For the 2020 RECS Household Survey the Contractor shall use web and mail questionnaires for data collection. By using self-administered web and mail instruments, as well as continued use of the DSF as the primary sample frame source, the

PSU and segment design (i.e., geographic-clustered design) may not be necessary.

The sample design shall meet all the sampling error requirements (i.e., RSEs) while minimizing the design effects with optimal sample size. Geographic domains, once again defined as individual states or groups of states, shall be required as a key element of the sample design. (The geographic domain definitions are provided for reference in the 2009 and 2015 RECS sampling documents that are supplements to this PWS.) Within each geographic domain the Contractor shall propose a stratification strategy that utilizes information about housing units/households that are correlated with the average consumption targets. Potential stratification variables, either at the housing unit or area level (e.g., census tract or block-group level), should include, but are not limited to main heating fuel, housing unit type, income, household size, urbanicity, heating degree days, and cooling degree days.

EIA expects the in-scope response rate using web and mail to be lower than the response rate achieved for prior rounds of RECS that used in-person interviewing exclusively.

EIA’s National Pilot study using web and mail modes achieved about a 40% response rate (AAPOR RR3, https://www.aapor.org/AAPOR_Main/media/publications/Standard-

Definitions20169theditionfinal.pdf), and EIA expects the 2020 RECS Household Survey to achieve a 40% or higher rate of response. The Contractor shall consider this U.S.-level response rate objective, as well as differential response across geographic or non-geographic subpopulations when designing the sample.

The Frame and Sample Design Plan shall account for the influence of any factors, such as the survey-taking climate, on response rates, precision levels, housing unit eligibility rates, and sampling rates across geographies. The plan shall also consider planning for nonresponse bias mitigations that would support execution of a nonresponse bias study.

All considerations and assumptions shall be described in the Frame and Sample Design

Plan.

The Contractor shall deliver draft and final versions of the Frame and Sample Design

Plan to the Program Office. The Contractor shall consult with the Program Office and the

Program Office will approve all major elements of the plan before it is executed.

The Contractor shall prepare all necessary files for the construction of the frame and the sample at each stage. The Contractor shall deliver preliminary and final sample files to the Program Office for review and approval. In consultation with the Program Office, the

Contractor may be required to also deliver full or partial frame files. All frame and sample files shall contain all relevant data, including unique identifiers, weights, geographic indicators, and stratification variables.

Subtask 3: Review, Test, Draft, and Finalize Contact Materials and Web and Mail

Instruments

The web and mail instruments used for the RECS National Pilot (and subsequently used to complete the official 2015 RECS Household Survey study) will be used as the model for collecting 2020 RECS Household Survey data. EIA expects the format of the web and https://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf https://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf mail instruments to be similar to the National Pilot formats, and EIA expects the time to complete each instrument to remain at or about 30 minutes.

The housing unit contact materials used for the RECS National Pilot shall also be used as the model for 2020 RECS Household Survey contact materials. These materials include letters, postcards, informational brochures, and envelopes.

The Contractor shall perform a review of the contact materials, contact protocols, and instruments and shall work in conjunction with the Program Office to identify areas for improvement. The Program Office will also recommend questions for deletion and propose new questions. The Contractor and the Program Office will coordinate to develop an appropriate testing plan to update the RECS Household Survey contact materials and questionnaires. The plan may call for various testing methods, including expert reviews, focus groups, cognitive or other lab testing, pilot tests, and/or crowdsourcing. The plan may also include evaluation and testing of Spanish translations of questionnaires and contact materials.

Of particular importance during the protocol and instrument testing phase is updating questions that capture housing unit square footage data. For much of the history of RECS, square footage estimates were derived from interviewer measurements. Working with the respondent, interviewers determined the shapes and relative sizes of each level of the home. Interviewers then used this information to measure appropriate dimensions (e.g.

lengths and widths) of the home. Official total, heated, and cooled square footage values were calculated from these dimension measurements. (Prior RECS Household Surveys also asked respondents to provide a point estimate of their home’s square footage, but these estimates were not used for official statistics.) Self-administered web and mail modes do not allow for the traditional measurement protocol. Therefore, the 2020 RECS

Household Survey must incorporate an alternative methodology for estimating housing unit square footage. The Contractor shall work with the Program Office to design, test, and implement an appropriate square footage estimation methodology.

The Contractor shall submit a Contact Materials and Instrument Review and Testing Plan and a Contact Materials and Instrument Review and Testing Report to the Program

Office. Based on the results of review and testing, the Contractor shall work in conjunction with the Program Office to draft and finalize all contact materials, questionnaire content, and data collection instruments. This includes translating questionnaires and contact materials in Spanish, as needed. The Contractor shall work in conjunction with the Program Office to draft, test, and finalize the web instrument systems that will be used to collect and store household characteristics data. Although the

Program Office will not prescribe the systems used to collect and track responses, including the mail response data entry system, the Program Office shall approve the data collection systems before they are put into production.

Subtask 4: Develop Data Management and Metadata Strategy

The 2020 RECS study will draw on numerous data sources for frame and sample design, allow for responses using multiple modes, and utilize several statistical tools (e.g., SAS and Excel). To meet program quality goals and to facilitate internal and external usability of the data, the Program Office and its Contractor will need to develop a comprehensive data management and metadata strategy that will be implemented across all phases, systems, and tools of the study.

The Contractor shall coordinate with the Program Office to develop a comprehensive

Data Management and Metadata Plan. The plan shall address how the Contractor will collect, store, manipulate, transfer, and describe data across every phase of the study.

This includes frame variables, questionnaire variables, derived variables, paradata, metadata, and data flags. The Contractor shall also work with the Program Office to consider how the strategy will facilitate post-collection processing including editing, imputation, and dissemination. EIA expects that the initial plan will be updated throughout the project life cycle.

The Data Management and Metadata Plan shall contribute to EIA’s ultimate goal of

Data Documentation Initiative (DDI) compliance for the RECS program.

Subtask 5: Collect and Deliver RECS Household Survey Data

The scope of this subtask includes using self-administered web and mail questionnaires to collect household characteristics data from a nationwide sample of housing units/households, as well as related activities that immediately precede or follow the actual data collection. These related activities include printing materials, finalizing the sample (case) management system, producing data collection reports, and conducting nonresponse bias analysis. The Contractor shall deliver draft and final versions of a Data

Collection Plan to the Program Office that describes how the Contractor plans to execute the activities in this subtask. The Contractor shall consult with the Program Office and the Program Office will approve all major elements of the plan before it is executed. The final plan shall balance quality, cost, and overall impact to the RECS program and its stakeholders.

Since the 1997 RECS, EIA has utilized a supplemental Rental Agent Survey as a follow-on data collection effort for some household survey respondents (e.g., households renting an apartment) who may be unsure about or unable to provide quality responses for key data items. Rental Agent Survey responses, captured using a combination of in-person and telephone interviewing, have only been used as a resource to edit Household Survey responses. A Rental Agent Survey is not required for the 2020 RECS Household Survey data collection. However, a Rental Agent Survey may be proposed and will be considered as an optional subtask. In place of a Rental Agent Survey, EIA invites innovative solutions to improve the quality of household characteristics data for renters. These solutions may include the use of linked administrative or other auxiliary data sources.

Subtask 5.1: Develop and print all survey materials

The Contractor shall work in conjunction with the Program Office to finalize proofs for all data collection materials including letters, postcards, informational brochures, envelopes, and paper questionnaires. Unless otherwise directed, the Contractor shall be responsible for printing and mailing all materials. The Contractor shall be responsible for maintaining all outgoing and incoming correspondence related to the 2020 RECS

Household Survey data collection.

Subtask 5.2: Sample (case) management system

The Contractor shall employ an automated system for tracking the status of all 2020

RECS Household Survey sampled cases. The system shall be capable of producing reports of data collection activities and results. The system shall track contact attempts and dates, completion modes, completion dates and times, and appropriate paradata that will be used to analyze the data collection effort.

Before the onset of data collection, the Contractor shall perform full systems integration testing across the sample (case) management system, data collection instruments, and data entry systems. Immediately after and throughout data collection, the Contractor shall monitor whether all systems are working as expected.

Subtask 5.3: Conduct 2020 RECS Household Survey data collection

EIA expects that 2020 RECS Household Survey data collection will begin in September

2020. The Contractor shall develop an effective contact and incentive strategy using self-administered web and mail questionnaires only. Each sampled housing unit/household shall be contacted by mail invitation. The Contractor may use USPS or a commercial service to deliver mail correspondence to sampled housing units. The initial invitation may be preceded by a pre-invitation postcard and may include an invitation to complete the survey via web only, mail only, or both. Note that the contact protocol and any proposed incentives are subject to approval by the Office of Management and Budget

(OMB).

The Contractor shall develop an effective sequencing and timing of nonresponse follow-up contacts after the initial mail invitation. The Contractor may propose an appropriate responsive or adaptive design that accounts for differential response across subpopulations.

The Contractor shall work in conjunction with the Program Office to determine specifications for the acceptable level of questionnaire completeness. For the purposes of this study, each response shall be dispositioned as “Complete” or “Incomplete.”

The data collection effort shall yield 15,000 to 18,000 completed, in-scope cases and at least a 40% response rate (AAPOR RR3). Every state and the District of Columbia shall be represented in the final set of completed cases. The Program Office expects the main data collection effort to yield a respondent sample that is sufficient to meet the precision targets outlined in Subtask 2.

Subtask 5.4: Provide RECS Household Survey performance and cost statistics

During the field period, the Contractor shall provide daily performance statistics to monitor the progress of data collection. A Performance and Cost Statistics Report

Template shall be developed in conjunction with and agreed to by the Program Office prior to the onset of data collection. This report shall reveal key data collection metrics at the national, census region, census division, and domain (e.g., state) level. The report shall also provide differentiation by key characteristics as determined by the Program

Office and the Contractor. A preliminary list of metrics in this report shall include, but is not limited to:

Completes by mode

Data collection cost per complete

Representativity of entire responding sample and key subpopulations, as compared to known residential sector sources (e.g., ACS)

Undeliverable or returned addresses

Out-of-scope responses

Incomplete responses

Subtask 5.5: Conduct nonresponse bias and data quality analysis

For much of the history of the RECS program, final response rates were at or above 80%.

These rates were at a level where nonresponse bias of key estimates was not a concern and at a level where a nonresponse bias study was not required according to OMB guidelines. In fact, EIA and its Contractors did not routinely perform robust nonresponse bias analysis.

As realized during the RECS National Pilot (and preceding pilot tests), a self-administered RECS data collection was not expected to achieve a response rate close to

80%. However, extensive analysis performed as part of the RECS National Pilot study and as part of the official 2015 RECS project revealed that lower response rates were not correlated with widespread nonresponse bias for key estimates. Nonresponse weighting adjustments and poststratification were used as the primary methods to account for nonresponse bias where it existed. Similarly, weighting adjustments and poststratification were also used to account for frame coverage error.

EIA expects that the 2020 RECS Household Survey should yield a similar response rate as the National Pilot effort, which was about 40%. EIA also expects that the levels of coverage error and nonresponse bias will be similar. However, due to possible changes in the frame, alterations to the sample design and selection methods, and changes in the survey-taking environment the Contractor shall closely monitor the representativity of the responding sample throughout data collection. At the conclusion of data collection, the

Contractor shall perform a comprehensive nonresponse bias analysis. This analysis may include, but is not limited to detailed comparisons with external sources or comparisons with frame variables. As a result of this study, the Contractor shall propose appropriate methods, if needed, to account for any coverage error or nonresponse bias. Additional field collection to support this nonresponse bias analysis, or to extend data collection to alleviate some nonresponse bias should be proposed as an optional subtask.

The Contractor shall also monitor and evaluate additional aspects of 2020 RECS

Household Survey data quality, including possible mode effects and measurement error.

Nonresponse bias and data quality activities and results shall be captured in a

Nonresponse Bias and Data Quality Analysis Plan and Nonresponse Bias and Data

Quality Analysis Report.

Subtask 5.5: Transfer 2020 RECS Household Survey data to the Program Office

The Contractor shall transfer Household Survey respondent data to the Program Office on a periodic basis. The Program Office suggests a routine data transfer every three weeks with each file being noncumulative. The files shall incorporate coding and other specifications provided by the Program Office, and shall be seamlessly linked to other files and variables (e.g., frame and sample files) associated with this study.

The first data file transfer shall include the first (approximately) 100 respondent cases, including a mix of both web and mail questionnaires. This delivery will be used to test the integrity of survey instruments and ensure that all file specifications are being adhered to.

The Program Office and the Contractor shall agree on an appropriate file format for each delivery. SAS has been used as a primary file format in prior rounds of RECS and will likely be the format of choice for this study. Additional formats, as appropriate, may be considered.

At the conclusion of all data collection activities, the Contractor shall transfer a final, cumulative, respondent data file.

Subtask 6: Perform Missing Item Imputation

The Contractor shall perform initial data editing and cleaning in the form of data checks incorporated into the web instrument and adjudicating anomalous mail responses (e.g., illegible responses) during mail questionnaire data entry. Program Office staff will be responsible for the majority of data editing subsequent to data collection.

The Contractor shall be responsible for imputing missing items after editing is complete.

Missing items include blank responses, “don’t know” responses, refusals, illegible written responses, and responses deleted during the editing process.

For prior RECS studies, the Program Office and its Contractors have used hot-deck imputation for most, if not all, item imputation. For the 2015 RECS, a variation of the hot-deck method (the predictive mean neighbor method) was used to impute missing square footage data. The Contractor shall review prior RECS item imputation methods and specifications and develop an item imputation plan for the 2020 RECS Household

Survey. The Contractor shall either update the traditional hot-deck imputation methods and specifications, propose a different imputation method, or use a combination of methods. The imputation procedures shall be designed to preserve the variance structure of the estimates and maintain questionnaire integrity, such as skip patterns. All imputation processes must be readily reproducible by the Program Office and all necessary data and programming codes must be delivered to the Program Office. The

Contractor shall deliver draft and final Item Imputation Plans.

The Contractor shall deliver draft and final imputed data files during and after the imputation process. The Program Office expects item imputation to be an iterative process requiring effective collaboration across organizations and multiple rounds of review and approval. The files shall incorporate coding and other specifications provided by the Program Office, and shall be seamlessly linked to other files and variables (e.g., frame and sample files, respondent data files) associated with this study.

Subtask 7: Develop Weighting Adjustment Strategy and Calculate Replicate

Weights

The Contractor shall calculate base weights, interim weights at each stage of adjustment, final weights, and replicate weights to be used for variance estimation. The appropriate weights shall be calculated and assigned to each sampled unit and to each responding case. The Contractor shall deliver draft and final Weighting Plans. All weighting processes must be readily reproducible by the Program Office and all necessary data and programming code must be delivered to the Program Office.

Base weights shall be calculated as part of the sample design and selection subtask and delivered to the Program Office as part of the sample file deliverable.

Subtask 7.1: Calculate adjusted weights

The base weights shall be adjusted to account for…