Attachment A - SOW.pdf

Statement of Work (SOW) U.S. Agency for Global Media International Audience Research Program

Survey Data Quality Control Services Blanket Purchase Agreement

1. BACKGROUND

The United States Agency for Global Media (“the agency” or USAGM) supports research into the media habits and opinions on media of foreign publics through its International Audience Research Program. The program, operated by the agency’s Office of Policy and Research (OPR), funds research in more than 90 countries, utilizing a variety of media research industry methodologies.

2. OBJECTIVE

These services are key to enabling a wider range of internal and external research consumers to easily draw insights from the agency’s survey data. They enable management of the agency’s nationally representative survey projects to be more efficient. They also enable the agency to aggregate data from country-level surveys into relevant regional and global reporting.

3. SCOPE

The agency requires data management services to ensure the quality of survey data, standardized data collection and unification, in support of a survey database.

The agency expects ten (10) full surveys and twenty (20) omnibus surveys each fiscal year.

4. SPECIFIC TASKS

1.0 Media Survey Data Quality Control

Each fiscal year, the agency will require the contractor to perform data quality control on survey projects. For each survey, the contractor shall perform the following tasks:

1.1 Review Questionnaire: Ensuring the drafted questionnaire adheres to standardized question tags, variables, and necessary filters to produce key performance indicators, as well as aggregate reporting. The agency’s core media questionnaire, when implemented in the field by survey enumerators, has an average duration of 18 minutes. The questionnaire varies minimally between countries, and mostly in filter logic, number of response options in multi-choice questions.

Deliverable: Questionnaire Review Log, a spreadsheet containing discovered errors for survey, including issues with survey metadata, instructions and script logic. Due:

three (3) days after agency supplies penultimate, customized draft with specific country inputs.

1.2 Create Data Codebook: Based on standardized variable names and labels.

Deliverable: Data Codebook based on questionnaire

1.3 Review Derived Variable Plan: In cooperation with the agency, review the accuracy of derived variable plan (DV plan) for each survey, ensuring it aligns with the agency’s survey data dictionary.

Deliverable: Reviewed derived variable plan, contributing comments/suggestion for edits to the DV plan spreadsheet. Review is due two days after the agency supplies a finalized DV plan.

1.4 Script Testing

Review scripts after questionnaires have been programmed into CAPI (or CATI) system, ensuring that question wording, order, and filter logic of the programmed survey matches the drafted questionnaire and that the contact sheet contains the required information for proper respondent selection and subsequent QC. Script Testing also includes review of dummy data.

Deliverable: Script Review Log, a spreadsheet documenting the errors found during thorough review of field contractor’s scripted survey. First round of review is due three days after field contractor provides access to the survey script. Up to three rounds of script review and one round of dummy data review may be necessary for each project

1.5 Review Interim Data

1.5.1 Review datasets after the first week of fieldwork or when 200 observations are achieved to ensure the filter logic of the programmed survey matches the filter logic of the drafted questionnaire

1.5.2 Ensure proper recording of required contact data in the survey program

Deliverable: Interim Data Review Log, a spreadsheet documenting errors found in the interim data that point to scripting errors of the survey. Due one (1) day after contractor delivers interim data.

1.6. Field Data Review

1.6.1 Interviewer analysis: Checking that interviewer performance (average interview duration, number of users among participants, etc.) does not contain overly anomalous patterns that might indicate data collection errors

1.6.2 Ensuring the delivered fielded dataset adheres to survey codebook.

Deliverable: Field Data Review Log, a spreadsheet documenting issues in the fielded data, including but not limited to: survey data points falling outside of the statistical norm for media usage patterns, illogical responses; survey metadata showing interviewers taking too long or too short a time to administer questionnaire, or interviewers conducting abnormally few or many interviews in a single workday. First review due three (3) days after contractor delivers field dataset.

1.7 Produce final datasets containing the agency’s standardized key performance indicators, all drawn from the Derived Variable Plan for each survey. After Derived Variables are created in dataset, the contractor shall review the new variables against the survey questionnaire, to ensure alignment and accuracy (bases are accurate).

Produce topline tables containing preliminary results from fielded survey data.

Deliverable: A final dataset containing the survey’s fielded variables, as well as the agency’s derived variables. As analysts work with data, it’s possible that adjustments to the derived variable plans occur, in which case, the contractor will be expected to update the survey dataset with revised derived variables and re-ingest data in the survey database. Multiple revisions may be necessary.

2.0 Survey Data Processing and Unification

The contractor shall support the agency’s mission to access media survey data efficiently by performing the following tasks on an ongoing basis:

2.1 Ensure adherence to the agency’s core media questionnaire, by maintaining the master questionnaire bank, which contains standardized English language question text, question tags, variable labels, and filter instructions.

2.2 Develop and communicate nomenclature for tagging new or country-specific questions, to be added to the agency’s core media questionnaire.

2.3 Work with agency analysts to develop SPSS syntax and software programming code that applies the agency’s unique, custom variable definitions (aka derived variables) to field datasets (field datasets delivered by field research contractors). This action produces final datasets from which key performance measures can be drawn.

Deliverable: Data Dictionary, containing all field and derived variables, labels, and corresponding question tags, delivered once annually.

5. DELIVERABLES

Task No. Deliverable Due

1.1 Media Questionnaire Review Log Two days after agency supplies country-specific questionnaire

1.2 Data Codebook One day after questionnaire is finalized

1.3 Derived Variable Plan Edits Two days after agency supplies DV plan

1.4 Scripting Review Log Three days after the field vendor delivers survey script, up to three rounds of script review may be necessary

1.5 Interim Data Review One day after field vendor delivers interim data.

1.6 Field Data Review Log Three days after field vendor delivers survey field dataset. Once issues have been addressed by field vendor, contractor shall review dataset again. Up to three rounds of review, until field vendor has answered all questions or address data quality issues, may be necessary.

1.7 Produce final datasets containing

field and derived variables

Three days after field vendor delivers final field dataset.

2.0 Data Dictionary, with updated

variables succinctly highlighted for users to quickly see what has changed

Once each year after all of the surveys for fiscal year are complete.

6. GENERAL ACCEPTANCE CRITERIA

General quality measures, as set forth below, will be applied to each work product received from the contractor under this SOW.

1. Accuracy: Work Products shall be accurate. Final and interim deliverables should accurately reflect the information provided in previous projects phases. Survey codebooks shall adhere to finalized, designed questionnaires. Syntax for creating derived variables in datasets shall reflect derived variable plans. Derived variables must be accurate to the bases obtained through surveys. Deliverables shall have limited errors and exclude basic errors that drain the Government staff’s time through multiple rounds of review and correction.

2. Clarity: Work Products shall be clear and concise. Logs generated through the review of survey scripts, questionnaires, dummy, pilot, and interim, and field data should communicate substantive errors clearly, so that the Government can efficiently understand issues and effectively communicate necessary revisions to project teams. Logs should exclude irrelevant information and adhere narrowly to scope of requirements. Written products should be concise and clear.

3. Consistency to requirements: All work products must satisfy the requirements of this SOW and or the quality assurance plan.

4. Format: Work Products shall be delivered electronically using file transfer protocols. File names should adhere to the agency’s naming conventions for research project artifacts.

Files shall be maintained in folders that have a clear, organized structure, and support efficient sharing by multiple stakeholders.

5. Timeliness: Work Products shall be submitted on or before the due date specified in this

SOW or submitted in accordance with a later scheduled date determined by OPR.

7. INSPECTION AND ACCEPTANCE CRITERIA:

The contractor shall work under the technical direction of OPR through the Contracting Officer’s Representative (COR). The contractor will submit weekly reports to OPR describing the work performed. Anticipated delays must be communicated to the COR and program managers as early as possible. Final inspection and acceptance of all work performed, reports, and other deliverables will be performed by the COR.

8. OTHER DIRECT COSTS

The contractor may expect to incur other direct costs (ODCs) for this requirement. Should materials/ODC expenses become necessary, the contractor must first contact the COR, in writing, prior to the performance of any services, provide estimated costs, including quotes, with its approval request. Allowable and reasonable costs incurred by the contractor for ODCs will be reimbursed.

All materials purchased by the contractor (if authorized) for the use or on the behalf of the Government shall become the property of the Government. The transfer of materials shall be documented by the contractor; in addition to an accounting of all materials consumed during the performance of individual elements of the contract, furnish the Government a copy of such documents with the Monthly Progress Reports.

9. TRAVEL REQUIREMENTS

No travel is required under this contract.

10. SECURITY CLEARANCE

U.S. citizens and foreign nationals that reside in the U.S. (if any) and are assigned to this contract shall be required by the Contracting Officer (CO) at the very least to be able to attain Public Trust.

Contractor personnel assigned to this contract are not authorized to view, process, store, transmit, receive, or handle classified information.

The U.S. contractor personnel be required to complete a background investigation through the Electronic Questionnaires for Investigation Processing system. It shall be the contractor’s responsibility to ensure the required information is completed accurately. If at any time the public trust certification is withdrawn, the contractor employee shall be removed from the contract, at the contractor’s expense, and can no longer be granted access to Sensitive But Unclassified information. If a contractor employee is terminated for cause, the contractor shall provide a prompt notification to the CO and the COR.

11. PERIOD OF PERFORMANCE

This is a single-award contract with a period of performance of up to five (5) years (based period + four (4) optional periods) from the date of contract award.

12. SCHEDULE

The contractor may telework remotely as agreed to by the USAGM CO or the COR. The work schedule will not exceed 40 hours each week and may be part-time; work is not required to be performed during regular business hours.

13. PLACE OF PERFORMANCE

Services are expected to be executed within USAGM network/infrastructure/platforms utilizing virtual work capabilities. Periodic meetings with USAGM staff will be required, but can be conducted in- person or via teleconference.

14. DISCLOSURE OF INFORMATION

Information made available to the contractor by OPR and/other USAGM Entities for the performance or administration of this effort shall be used only for those purposes and shall not be used in any other way without the written agreement of the COR.

The contractor agrees to assume responsibility for protecting the confidentiality of Government records, which are not public information. Each employee shall be notified in writing by the contractor that such information may be disclosed only for a purpose and to the extent authorized herein.

15. GOVERNMENT DATA RIGHTS

All products (i.e. reports, surveys, training, etc.) produced under this contract shall be the property of the Government and marked as such. Products may not be released to third parties without expressed permission of the Government.

16. LIMITED USE OF DATA

Performance of this effort may require the contractor to access and use data and information proprietary to a government agency or Government contractor, which is of such a nature that its dissemination or use, other than in performance of this effort, would be adverse to the interests of the Government and/or others.

Contractor and/or contractor personnel shall not divulge, or release data or information developed or obtained in performance of this effort, until made public by the Government, except to authorized Government personnel or upon written approval of the CO. The contractor may not use, disclose, or reproduce proprietary data that bears a restrictive legend, other than as required in the performance of this effort. Nothing herein shall preclude the use of any data independently acquired by the contractor without such limitations or prohibit an agreement at no cost to the Government between the contractor and the data owner which provides for greater rights to the contractor.

17. CONTRACTOR IDENTIFICATION

The contract performance may require contractor personnel to attend meetings with Government personnel and the public, work within Government office, and/or utilize Government electronic mail (email). Contractor personnel must take the following actions to identify themselves as non- Federal employees:

1. Use an email signature block that shows name, the office being supported, and the company affiliation,

2. Clearly identify themselves and their contractor affiliation in meetings,

3. Identify their affiliation in Agency emails and phone listings whenever contractor personnel are included in such listings, and

4. Shall not utilize USAGM logos or indicia on business cards.