Attachment 1 - Statement of Work_Revised.pdf

Statement of Work (SOW) U.S. Agency for Global Media International Audience Research Program

Survey Data Management Services

1. BACKGROUND

The United States Agency for Global Media (“the agency” or USAGM) supports research into the media habits and opinions on media of foreign publics through its International Audience Research Program. The program, operated by the agency’s Office of Policy and Research (OPR), funds research in more than 90 countries, utilizing a variety of media research industry methodologies.

2. OBJECTIVE

These services are key to enabling a wider range of internal and external research consumers to easily draw insights from the agency’s survey data. They enable management of the agency’s nationally representative survey projects to be more efficient. They also enable the agency to aggregate data from country-level surveys into relevant regional and global reporting.

3. SCOPE

The agency requires data management services to ensure the quality of survey data, standardized data collection and unification, and maintenance of a survey data database.

4. SPECIFIC TASKS

1.0 Media Survey Data Quality Control

Each fiscal year, the agency will require the contractor to perform data quality control on 15-25 media survey projects. For each survey, the contractor shall perform the following tasks:

1.1 Review Questionnaire: Ensuring the drafted questionnaire adheres to standardized question tags, variables, and necessary filters to produce key performance indicators, as well as aggregable reporting. The agency’s core media questionnaire, when implemented in the field by survey enumerators, has an average duration of 18 minutes.

Deliverable: Questionnaire Review Log, a spreadsheet containing comments and recommendations for survey with accurate instructions and script logic. Due: three (3) days after agency supplies penultimate, customized draft with specific country inputs.

1.2 Create Data Codebook: That is based on standardized variable names and label.

Deliverable: Data Codebook based on questionnaire

1.3 Review Derived Variable Plan: In cooperation with the agency, review the accuracy of derived variable plan (DV plan) for each survey, ensuring it aligns with the agency’s survey data dictionary.

Deliverable: Reviewed derived variable plan, contributing comments/suggestion for edits to the DV plan spreadsheet. Review is due two days after the agency supplies a finalized DV plan.

Deliverable: Script Review Log, a spreadsheet documenting the errors found during thorough review of field contractor’s scripted survey. First round of review is due three days after field contractor provides access to the survey script. Up to three rounds of script review may be necessary for each project.

1.4 Script Testing: Review scripts after questionnaires have been programmed into CAPI (or CATI) system, ensuring that question wording, order, and filter logic of the programmed survey matches the drafted questionnaire and that the contact sheet contains the required information for proper respondent selection and subsequent QC.

Script Testing also includes review of dummy data.

Deliverable: Script Review Log, a spreadsheet documenting the errors found during thorough review of field contractor’s scripted survey. First round of review is due three days after field contractor provides access to the survey script. Up to three rounds of script review and one round of dummy data review may be necessary for each project.

1.5 Check Survey Translations: Contractor will ensure that the scripted translated questionnaire (1-4 languages, depending on the country) matches the instructions and question text of the English questionnaire (see attachment of USAGM languages).

Deliverable: Translation Pattern Check Log, a spreadsheet documenting the errors found during pattern checks of the field contractor’s translation. The first round of review is due 1-2 days after field contractor provides survey translations.

1.5 Review Interim Data

1.5.1 Review datasets after the first week of fieldwork or when 200 observations are achieved to ensure the filter logic of the programmed survey matches the filter logic of the drafted questionnaire

1.5.2 Ensure proper recording of required contact data in the survey program

Deliverable: Interim Data Review Log, a spreadsheet documenting errors found in the interim data that point to scripting errors of the survey. Due one (1) day after contractor delivers interim data.

1.6 Field Data Review

1.6.1 Interviewer analysis: Checking that interviewer performance (average interview duration, number of users among participants) does not contain overly anomalous patterns that might indicate data collection errors

1.6.2 Ensuring the delivered fielded dataset adheres to survey codebook.

Deliverable: Field Data Review Log, a spreadsheet documenting issues in the fielded data, including but not limited to: survey data points falling outside of the statistical norm for media usage patterns, illogical responses; survey metadata showing interviewers taking too long or too short a time to administer questionnaire, or interviewers conducting abnormally few or many interviews in a single workday. First review due three (3) days after contractor delivers field dataset.

1.7 Produce final datasets containing the agency’s standardized key performance indicators, all drawn from the Derived Variable Plan for each survey. Produce topline tables in CSV or Excel file format containing preliminary results from fielded survey data.

Deliverable: A final dataset containing the survey’s fielded variables, as well as the agency’s derived variables. As analysts work with data, it’s possible that adjustments to the derived variable plans occur, in which case, the contractor will be expected to update the survey dataset with revised derived variables and re-ingest data in the survey database. Multiple revisions may be necessary.

2.0 Survey Data Processing and Unification

The contractor shall support the agency’s mission to access media survey data efficiently by performing the following tasks on an ongoing basis:

2.1 Ensure adherence to the agency’s core media questionnaire, by maintaining the master questionnaire bank, which contains standardized English language question text, question tags, variable labels, and filter instructions.

2.2 Develop and communicate nomenclature for tagging new or country-specific questions, to be added to the agency’s core media questionnaire.

2.3 Work with agency analysts to develop SPSS syntax and software programming code that applies the agency’s unique, custom variable definitions (aka derived variables) to field datasets (field datasets delivered by field research contractors). This action produces final datasets from which key performance measures can be drawn.

Deliverable: Data Dictionary, delivered once annually.

3.0 Survey Database Development and Maintenance

The agency currently maintains a survey database containing key performance indicators for all countries, networks, major brands, and languages, as well as meta-data for each media survey dataset.

For each completed media survey, the contractor shall ingest the dataset including field and derived variables into the database, including appropriate metadata: population statistics, weighted percentages, and unweighted Ns. This happens on a rolling basis as survey projects are completed, although based on survey data delivery schedules the bulk of the work will take place in the July through September and January through March, so that the agency can update dashboards in real time.

The contractor shall work with the agency and networks to maintain database inputs through continuous data quality control and unification. The contractor will coordinate with OPR analysts to ensure the database includes key variables, creating new syntax and drawing new variables (as requested) into the database.

In addition to 15-25 full media surveys each year, the agency commissions up to 30-40 omnibus surveys that must also be ingested into the survey database. The 15-25 media surveys each have an average duration of 25 minutes, and contain approximately 475 field variables. The 30-40 omnibus surveys each have a duration of 4-6 minutes and each contain approximately 150 field variables.

Although data quality control is not necessary for omnibus surveys, the contractor will similarly use derived variable plans for omnibus surveys and produce survey datasets containing key performance indicators.

Deliverable: Three days after each fielded survey dataset (full survey or omnibus) is finalized (derived variables created and added to each dataset), updated comma-separated values (CSV) with newest survey data ingested into the database.

4.0 Consulting on Effective Data Management

The contractor, while working with the agency’s questionnaires and survey dataset, will advise OPR on better practices and tools for managing survey data, with the aim of automating as many parts of the process as possible. OPR will work with the contractor to ensure that they fully understand the agency’s survey research operation, including the reporting needs from OPR’s analysts, as well as networks’ research analysts.

4.1 The agency will require the contractor to schedule monthly workshops with the agency’s OPR to review and implement new business practices. These workshops may be virtual

4.2 In addition, the contractor will organize and participate in 2-4 workshops each year, involving the research stakeholders at the agency’s research networks, in order to seek feedback on proposed new business practices and gain a better understanding of the network’s reporting needs. These workshops may be virtual, but in-person in Washington, DC is preferred.

Deliverable(s): Five (5) days after each monthly workshop, 1-5 page written report summarizing feedback and suggesting actionable steps.

5. DELIVERABLES

SOW

Task No.

Align with Activities under Section “Tasks”

Due

1.1 Media Questionnaire Review Log Two days after agency supplies country-specific questionnaire

1.2 Data Codebook One day after questionnaire is finalized

1.3 Derived Variable Plan Edits Two days after agency delivers DV plan

1.4 Scripting Review Log Three days after field vendor delivers survey script, up to three rounds of script review may be necessary

1.5 Translation Pattern Check Log Two days after the field vendor provides survey translations

1.6 Interim Data Review One day after field vendor interim data.

1.7 Field Data Review Log Three days after field vendor delivers survey field dataset, up to three rounds of review may be necessary

1.8 Produce final datasets containing field

and derived variables

Three days after field vendor delivers final field dataset.

2.0 Data Dictionary, with updated variables

succinctly highlighted for users to quickly see what has changed

Once each year after all of the surveys for the fiscal year are complete.

3.0 Updated database containing newly

finalized survey data

Three days after a survey dataset is finalized.

4.0 Written recommendations for SOP

improvements, focused on continuous improvement of agency’s data management practice.

Due five days after each workshop.

GENERAL ACCEPTANCE CRITERIA

General quality measures, as set forth below, will be applied to each work product received from the contractor under this SOW.

1. Accuracy: Work Products shall be accurate. Final and interim deliverables should accurately reflect the information provided in previous projects phases. Survey codebooks shall adhere to finalized, designed questionnaires. Syntax for creating derived variables in datasets shall reflect derived variable plans. Datatables containing reach and key performance metrics shall contain accurate formulae and results checked to ensure that formulae align with agency’s approved calculation and policies. Datatables shall have limited errors and exclude basic errors that drain the Government staff’s time through multiple rounds of review and correction.

2. Clarity: Work Products shall be clear and concise. Logs generated through the review of survey scripts, questionnaires, dummy, pilot, and interim, and field data should communicate substantive errors clearly, so that the Government can efficiently understand issues and effectively communicate necessary revisions to project teams. Logs should exclude irrelevant information and adhere narrowly to scope of requirements. Written products should be concise and clear.

3. Consistency to requirements: All work products must satisfy the requirements of this SOW and or the quality assurance plan.

4. Format: Work Products shall be delivered electronically using file transfer protocols. File names should adhere to the agency’s naming conventions for research project artifacts.

Files shall be maintained in folders that have a clear, organized structure, and support efficient sharing by multiple stakeholders.

5. Timeliness: Work Products shall be submitted on or before the due date specified in this SOW or submitted in accordance with a later scheduled date determined by OPR.

6. INSPECTION AND ACCEPTANCE CRITERIA:

The contractor shall work under the technical direction of OPR through the Contracting Officer’s Representative (COR). The contractor will submit weekly reports to OPR describing the work performed. Anticipated delays must be communicated to the COR and program managers as early as possible. Final inspection and acceptance of all work performed, reports, and other deliverables will be performed by the COR.

7. OTHER DIRECT COSTS

The contractor may expect to incur other direct costs (ODCs) for this requirement. Should materials/ODC expenses become necessary, the contractor must first contact the COR, in writing, prior to the performance of any services, provide estimated costs, including quotes, with its approval request. Allowable and reasonable costs incurred by the contractor for ODCs will be reimbursed.

All materials purchased by the contractor (if authorized) for the use or on the behalf of the Government shall become the property of the Government. The transfer of materials shall be documented by the contractor; in addition to an accounting of all materials consumed during the performance of individual elements of the contract, furnish the Government a copy of such documents with the Monthly Progress Reports.

8. TRAVEL REQUIREMENTS

Travel to work with key members of the agency’s OPR may be required under this contract.

9. SECURITY CLEARANCE

U.S. citizens and foreign nationals that reside in the U.S. (if any) and are assigned to this contract shall be required by the Contracting Officer (CO) at the very least to be able to attain Public Trust. Contractor personnel assigned to this contract are not authorized to view, process, store, transmit, receive, or handle classified information.

The U.S. contractor personnel be required to complete a background investigation through the Electronic Questionnaires for Investigation Processing system. It shall be the contractor’s responsibility to ensure the required information is completed accurately. If at any time the public trust certification is withdrawn, the contractor employee shall be removed from the contract, at the contractor’s expense, and can no longer be granted access to Sensitive But Unclassified information. If a contractor employee is terminated for cause, the contractor shall provide a prompt notification to the CO and the COR.

10. PERIOD OF PERFORMANCE

This is a single-award contract with a period of performance of up to five (5) years (based period + four (4) optional years) from the date of contract award.

11. SCHEDULE

The contractor may telework remotely as agreed to by the USAGM CO or the COR. The work schedule will not exceed 40 hours each week and may be part-time; work is not required to be performed during regular business hours.

12. PLACE OF PERFORMANCE

Services are expected executed within USAGM network/infrastructure/platforms utilizing virtual work capabilities. Period meetings with USAGM staff will be required but can be conducted in-person or via teleconference. Travel funds will be provided for no more than four in-person work sessions for one staff member, lasting no more than two (2) business days.

13. DISCLOSURE OF INFORMATION

Information made available to the contractor by OPR and/other USAGM Entities for the performance or administration of this effort shall be used only for those purposes and shall not be used in any other way without the written agreement of the COR.

The contractor agrees to assume responsibility for protecting the confidentiality of Government records, which are not public information. Each employee shall be notified in writing by the contractor that such information may be disclosed only for a purpose and to the extent authorized herein.

14. GOVERNMENT DATA RIGHTS

All products (i.e. reports, surveys, training, etc.) produced under this contract shall be the property of the Government and marked as such. Products may not be released to third parties without expressed permission of the Government.

15. LIMITED USE OF DATA

Performance of this effort may require the contractor to access and use data and information proprietary to a government agency or Government contractor, which is of such a nature that its dissemination or use, other than in performance of this effort, would be adverse to the interests of the Government and/or others.

Contractor and/or contractor personnel shall not divulge, or release data or information developed or obtained in performance of this effort, until made public by the Government, except to authorized Government personnel or upon written approval of the CO. The contractor may not use, disclose, or reproduce proprietary data that bears a restrictive legend, other than as required in the performance of this effort. Nothing herein shall preclude the use of any data independently acquired by the contractor without such limitations or prohibit an agreement at no cost to the Government between the contractor and the data owner which provides for greater rights to the contractor.

16. CONTRACTOR IDENTIFICATION

The contract performance may require contractor personnel to attend meetings with Government personnel and the public, work within Government office, and/or utilize Government electronic mail (email). Contractor personnel must take the following actions to identify themselves as non- Federal employees:

1. Use an email signature block that shows name, the office being supported, and the company affiliation,

2. Clearly identify themselves and their contractor affiliation in meetings,

3. Identify their affiliation in Agency emails and phone listings whenever contractor personnel are included in such listings, and

4. Shall not utilize USAGM logos or indicia on business cards.

Appendix 1: USAGM Broadcast Languages

1. Afan Oromo

2. Albanian

3. Amharic

4. Arabic

5. Armenian

6. Azerbaijiani

7. Bambara

8. Bangla

9. Bashkir

10. Belarussian

11. Bosnian

12. Bulgarian

13. Burmese

14. Cantonese

15. Chechen

16. Creole

17. Crimean Tatar

18. Dari

19. English

20. French

21. Georgian

22. Hausa

23. Hungarian

24. Indonesian

25. Kazakh

26. Khmer

27. Kinyarwanda

28. Kirundi

29. Korean

30. Kurdish

31. Kyrgyz

32. Lao

33. Lingala

34. Macedonian

35. Mandarin

36. Montenegrin

37. Ndebele

38. Pashto

39. Persian

40. Portuguese

41. Rohingya

42. Romanian

43. Russian

44. Sango

45. Serbian

46. Shona

47. Sindhi

48. Somali

49. Spanish

50. Swahili

51. Tajik

52. Tatar

53. Thai

54. Tibetan

55. Tigrigna

56. Turkish

57. Turkmen

58. Ukrainian

59. Urdu

60. Uyghur

61. Uzbek

62. Vietnamese

63. Wolof

SOW_ Survey Data Management Project 11