RFQ - 951700-25-Q-0009.pdf

RFQ # 951700-25-Q-0009

Survey Data Quality Control Services

RFQ Number # 951700-25-Q-0009

From:

United States Agency for Global Media (USAGM), Office of Policy and Research (OPR)

For:

Blanket Purchase Agreement (BPA)

This is a 100% small business set-aside

Conducted under Federal Acquisition Regulation (FAR) 13

Issued by:

United States Agency for Global Media (USAGM)

North American Industry Classification System (NAICS): 541720

Date of RFQ Release January 14, 2025

SUPPLIES/SERVICES/PRICES

See Attachment C – Price Schedule

PART 1 GENERAL INFORMATION

Description of Services/Introduction:

The contractor shall provide all personnel, equipment, tools, materials, supervision, and other items and non-personal services necessary to deliver data management services as defined in this Statement of Work (SOW) (Attachment A). The contractor shall perform to the standards in this contract.

Background:

The United States Agency for Global Media (“the agency” or USAGM) supports research into the media habits and opinions on media of foreign publics through its International Audience Research Program.

The program, operated by the agency’s Office of Policy and Research (OPR), funds research in more than 90 countries, utilizing a variety of media research industry methodologies.

Objectives:

These services are key to enabling a wider range of internal and external research consumers to easily draw insights from the agency’s survey data. They enable management of the agency’s nationally representative survey projects to be more efficient. They also enable the agency to aggregate data from country-level surveys into relevant regional and global reporting.

Scope:

The agency requires data management services to ensure the quality of survey data, standardized data collection and unification, in support of a survey database.

The agency expects ten (10) full surveys and twenty (20) omnibus surveys each fiscal year.

Period of Performance:

The period of performance for this Blanket Purchase Agreement will be a 12-month base period and four

(4) 12-month option periods from date of BPA award. Each call order may typically run for one full year (12 months) from the date of each BPA order execution, as signed by the Contracting Officer.

The Government reserves the right to extend the term of this contract at the prices set forth in accordance with the terms and conditions contained in clause 52.217-9 entitled, “Option to Extend the Term of the Contract”.

General Information

Quality Control:

The contractor shall develop and maintain an effective quality control program to ensure services are performed in accordance with this SOW. The contractor shall develop and implement procedures to identify, prevent, and ensure non-recurrence of defective services. The contractor’s quality control program is the means by which the contractor assures that the work complies with the requirement of the contract. After acceptance of the quality control plan, the contractor shall receive the contracting officer’s acceptance in writing of any proposed change to the QC system.

Quality Assurance:

The government shall evaluate the contractor’s performance under this contract in accordance with the Quality Assurance Surveillance Plan. This plan is primarily focused on what the Government must do to ensure that the contractor has performed in accordance with the performance standards. It defines how the performance standards will be applied, the frequency of surveillance, and the minimum acceptable defect rate(s).

Government Remedies:

The contracting officer shall follow FAR 52.212-4, “Contract Terms and Conditions Commercial Items” or 52.246-4, “Inspection of Services-Fixed Price” for contractor’s failure to perform satisfactory services or failure to correct non-conforming services.

Recognized Holidays:

The contractor is not required to perform services on the following federal Holidays:

New Year’s Day Labor Day

Martin Luther King Jr.’s Birthday Columbus Day

President’s Day Veteran’s Day

Memorial Day Juneteenth Day

Thanksgiving Day Independence Day

Christmas Day

Hours of Operation/Place of Performance:

The Contractor is expected to work off-site at a location of its own convenience but must be available to meet with the federal government in person, at the discretion of the Contracting Officer Representative (COR). The Contractor shall meet at least once for each Call Order with the COR, preferably in Washington, DC, at the discretion of the COR. The COR may determine that future meeting be conducted via teleconference calls or webinars. The Government will provide conference space for these meetings.

Type of Contract:

The government anticipates an award of a Firm Fixed Price (FFP) type contract for the Blanket Purchase Agreement; the type of contract for calls placed against the Blanket Purchase Agreement will be determined at the call order level.

Security Requirements:

Physical Security.

The contractor shall be responsible for safeguarding all government property provided for contractor use.

At the close of each work period, government facilities, equipment, and materials shall be secured.

Key Control.

Not Applicable

Lost Keys

Not Applicable

Key Use

Not Applicable

Lock Combinations.

Not Applicable

Post Award Conference/Periodic Progress Meetings:

The Contractor agrees to attend any post award conference convened by the contracting activity or contract administration office in accordance with Federal Acquisition Regulation Subpart 42.5. The contracting officer, Contracting Officers Representative (COR), and other Government personnel, as appropriate, may meet periodically with the contractor to review the contractor's performance. At these meetings, the contracting officer will apprise the contractor of how the government views the contractor's performance and the contractor will apprise the Government of problems, if any, being experienced.

Appropriate action shall be taken to resolve outstanding issues. These meetings shall be at no additional cost to the government.

Contracting Officer Representative (COR):

A COR will be delegated separately.

Contract Manager:

The contractor shall provide a contract manager who shall be responsible for the performance of the work. The name of this person and an alternate who shall act for the contractor when the manager is absent shall be designated in writing to the contracting officer. The contract manager or alternate shall have full authority to act for the contractor on all contract matters relating to daily operation of this contract.

Identification of Contractor Employees:

All contract personnel attending meetings, answering Government telephones, and working in other situations where their contractor status is not obvious to third parties are required to identify themselves as such to avoid creating an impression in the minds of members of the public that they are Government officials. They must also ensure that all documents or reports produced by contractors are suitably marked as contractor products or that contractor participation is appropriately disclosed. Contractor personnel will not be required to obtain badges.

SPECIFIC TASKS

Basic Services:

Upon issuance of a Call Order, the Contractor may be requested to complete services including:

1.0 Media Survey Data Quality Control

Each fiscal year, the agency will require the contractor to perform data quality control on survey projects.

For each survey, the contractor shall perform the following tasks:

1.1 Review Questionnaire

Ensuring the drafted questionnaire adheres to standardized question tags, variables, and necessary filters to produce key performance indicators, as well as aggregate reporting. The agency’s core media questionnaire, when implemented in the field by survey enumerators, has an average duration of 18 minutes. The questionnaire varies minimally between countries, and mostly in filter logic, number of response options in multi-choice questions.

Deliverable: Questionnaire Review Log, a spreadsheet containing discovered errors for survey, including issues with survey metadata, instructions and script logic. Due: three (3) days after agency supplies penultimate, customized draft with specific country inputs.

1.2 Create Data Codebook

Based on standardized variable names and labels.

Deliverable: Data Codebook based on questionnaire

1.3 Review Derived Variable Plan

In cooperation with the agency, review the accuracy of derived variable plan (DV plan) for each survey, ensuring it aligns with the agency’s survey data dictionary.

Deliverable: Reviewed derived variable plan, contributing comments/suggestion for edits to the DV plan spreadsheet. Review is due two days after the agency supplies a finalized DV plan.

1.4 Script Testing

Review scripts after questionnaires have been programmed into CAPI (or CATI) system, ensuring that question wording, order, and filter logic of the programmed survey matches the drafted questionnaire and that the contact sheet contains the required information for proper respondent selection and subsequent QC. Script Testing also includes review of dummy data.

Deliverable: Script Review Log, a spreadsheet documenting the errors found during thorough review of field contractor’s scripted survey. First round of review is due three days after field contractor provides access to the survey script. Up to three rounds of script review and one round of dummy data review may be necessary for each project

1.5 Review Interim Data

1.5.1 Review datasets after the first week of fieldwork or when 200 observations are achieved to ensure the filter logic of the programmed survey matches the filter logic of the drafted questionnaire

1.5.2 Ensure proper recording of required contact data in the survey program

Deliverable: Interim Data Review Log, a spreadsheet documenting errors found in the interim data that point to scripting errors of the survey. Due one (1) day after contractor delivers interim data.

1.6. Field Data Review

1.6.1 Interviewer analysis: Checking that interviewer performance (average interview duration, number of users among participants, etc.) does not contain overly anomalous patterns that might indicate data collection errors

1.6.2 Ensuring the delivered fielded dataset adheres to survey codebook.

Deliverable: Field Data Review Log, a spreadsheet documenting issues in the fielded data, including but not limited to; survey data points falling outside of the statistical norm for media usage patterns, illogical responses; survey metadata showing interviewers taking too long or too short a time to administer questionnaire, or interviewers conducting abnormally few or many interviews in a single workday. First review due three (3) days after contractor delivers field dataset.

1.7 Produce final datasets containing the agency’s standardized key performance indicators, all drawn from the Derived Variable Plan for each survey. After Derived Variables are created in dataset, the contractor shall review the new variables against the survey questionnaire, to ensure alignment and accuracy (bases are accurate). Produce topline tables containing preliminary results from fielded survey data.

Deliverable: A final dataset containing the survey’s fielded variables, as well as the agency’s derived variables. As analysts work with data, it’s possible that adjustments to the derived variable plans occur, in which case, the contractor will be expected to update the survey dataset with revised derived variables and re-ingest data in the survey database. Multiple revisions may be necessary.

2.0 Survey Data Processing and Unification

The contractor shall support the agency’s mission to access media survey data efficiently by performing the following tasks on an ongoing basis:

2.1 Ensure adherence to the agency’s core media questionnaire, by maintaining the master questionnaire bank, which contains standardized English language question text, question tags, variable labels, and filter instructions.

2.2 Develop and communicate nomenclature for tagging new or country-specific questions, to be added to the agency’s core media questionnaire.

2.3 Work with agency analysts to develop SPSS syntax and software programming code that applies the agency’s unique, custom variable definitions (aka derived variables) to field datasets (field datasets delivered by field research contractors). This action produces final datasets from which key performance measures can be drawn.

Deliverable: Data Dictionary, containing all field and derived variables, labels, and corresponding question tags, delivered once annually.

Deliverable Schedule

Task No. Deliverable Due

1.1 Media Questionnaire Review Log Two days after agency supplies country-specific questionnaire

1.2 Data Codebook One day after questionnaire is finalized

1.3 Derived Variable Plan Edits Two days after agency supplies DV plan

1.4 Scripting Review Log Three days after the field vendor delivers survey script, up to three rounds of script review may be necessary

1.5 Interim Data Review One day after field vendor delivers interim data.

1.6 Field Data Review Log Three days after field vendor delivers survey field dataset. Once issues have been addressed by field vendor, contractor shall review dataset again. Up to three rounds of review, until field vendor has answered all questions or address data quality issues, may be necessary.

1.7 Produce final datasets containing

field and derived variables

Three days after field vendor delivers final field dataset.

2.0 Data Dictionary, with updated variables

succinctly highlighted for users to quickly see what has changed

Once each year after all of the surveys for fiscal year are complete.

Instructions, Conditions, and Notices to Bidders

Provisions that are incorporated by reference (by Citation Number, Title, and Date), have the same force and effect as if they were given in full text.

SUBMISSION: The requirements are Volumes I - Technical, II - Price. Files shall not contain classified data. The intent of this standardized format is not to limit creativity. Creativity is permitted within the parameters below to allow the Government a more fair and easy assessment of offers.

Point of Contact (POC)

The Contracting Officer (CO) and Contract Specialist (CS) are the sole POC’s for this Procurement.

Contracting Officer: Karoline E. Duffy

Contract Specialist: Christopher Capers

Submission of Questions:

Questions must be submitted no later than January 16, 2025, at 12:00pm ED to CCapers@usagm.gov and KDuffy@usagm.gov via the Pre-submission Inquiries spreadsheet (Attachment D).

The subject line should read:

RFQ # 951700-25-Q-0009 – Project Title: Survey Data Quality Control Services

Delivery of Quotations

Quotations must be submitted no later than January 29, 2025, at 12:00PM ED to CCapers@usagm.gov and KDuffy@usagm.gov.

The subject line should read: RFQ # 951700-25-Q-0009 – Project Title: Survey Data Quality Control Services.

QUOTATION SUBMISSION:

The Vendor’s quotation shall be submitted in accordance with the Delivery of Quotation section of this RFQ. The Vendor’s quotation shall consist of two separate volumes. Files shall not contain classified data.

The parameters below allow the Government a more fair and easy assessment of quotes.

Quotations that do not follow the Email Subject Line format listed above will NOT be accepted.

Quotations sent by commercial package delivery and hand delivery will NOT be accepted.

mailto:KDuffy@usagm.gov mailto:KDuffy@usagm.gov

A. Format. The submission shall be clearly indexed and logically assembled. Each volume shall be appropriately numbered and clearly identified with the date and solicitation number in the header and/or footer and shall begin at the top of each page. A Table of Contents must be included. Documents shall use the following page setup parameters.

Margins – Top, Bottom, Left, Right – 1”

Gutter – 0”

From Edge – Header, Footer – 0.5”

Page Size, Width – 8.5”

Page Size, Height – 11”

The following additional restrictions apply: Each paragraph shall be separated by at least one blank line.

A standard, 12-point minimum font size applies. Times New Roman or other legible font is required.

Tables, charts, and illustrations may use a reduced font size not less than 8-point and may be landscape.

B. File Packaging. Quotations shall be submitted in two volumes: Volume I, Technical Quotation.

Volume II, Price Quotation.

C. Page Count. The page count is as follows:

Quotation Page Count Volume I - Technical

Factor 1 - Technical Approach

Factor 2 - Key Personnel

Factor 3 - Relevant Expertise

Excluding covers, title pages, executive summaries, and table of contents, the text of the narrative is Not to Exceed (NTE) 15 pages, included in this page limit are:

- Technical Approach,

- Description of the Key Personnel, and their qualifications/experience

- Relevant experience.

Resumes are not counted toward the 15- page limit.

Volume II - Price No page limitation

Those pages that exceed the page counts for each volume or attachment will not be evaluated. If there are more than forty-five (45) printed lines of text on any page, the sum of the additional lines will be removed from back to front of the particular volume or attachment and will not be evaluated.

D. Content Requirements. All information shall be confined to the appropriate volume. The applicant shall confine submissions to essential matters, sufficient to define the quotation details, in a concise manner, to permit a complete and accurate evaluation of each Quotation. Each volume shall consist of a Table of Contents, Summary Section and the Narrative discussion. The Summary Section shall contain a brief abstract of the volume. Proprietary information shall be clearly marked.

EVALUATION FACTORS FOR AWARD

VOLUME I - TECHNICAL

The Offeror shall provide a technical quotation that will be evaluated to determine whether the Offeror’s methods and approaches have adequately and completely considered, defined, and satisfied the requirements specified in the SOW and RFQ and whether the Offeror’s methods and approach to meeting the RFQ requirements provide the Government with a high level of confidence of successful completion within the required schedule. No pricing information is to be provided in the Technical Quotation.

Adjectival Ratings

The technical submission will be rated using the adjectival ratings below.

Adjectival Rating Description

Outstanding

Proposal demonstrates an exceptional approach and understanding of the requirements and contains multiple strengths and/or at least one significant strength, and risk of unsuccessful performance is low.

Good

Proposal indicates a thorough approach and understanding of the requirements and contains at least one strength or significant strength, and risk of unsuccessful performance is low to moderate.

Acceptable

Proposal meets requirements and indicates an adequate approach and understanding of the requirements, and risk of unsuccessful performance is no worse than moderate.

Marginal Proposal has not demonstrated an adequate approach and understanding of the requirements, and/or risk of unsuccessful performance is high.

Unacceptable

Proposal does not meet requirements of the solicitation and, thus, contains one or more deficiencies and is un-awardable, and/or risk of performance is unacceptably high.

Factor 1 - Technical approach

A strong proposal will detail an approach that will substantially enhance the quality of the deliverables as detailed in the SOW. The narrative should include:

• Description of survey data quality control processes, including comprehensive details about pre- and post-field activities, and outlining where software and human review may be utilized to identify survey errors or data quality issues.

• Description of the quality control checks the Offeror would utilize for data collection through CAPI and CATI modes. Checks would reveal data anomalies that could signal interviewer, administration, or respondent errors.

• Tools, including off-the-shelf programs and proprietary applications, that would be enlisted in providing services requested in the SOW.

• Discussion of deliverables, particularly how they will be formatted. While datasets must be submitted as .sav files, there is flexibility in the format of review logs and topline tables. Example templates have been provided (Attachments E.1 - E.6), but the government is open to other formats.

Factor 2 - Key personnel

The Government will evaluate the Offeror’s proposed key personnel, looking for a minimum seven (7) years of relevant experience for the Project Manager and five (5) years of relevant experience for the Developer. Offerors should submit resumes or CVs of proposed key personnel (maximum of 3 pages) listed below. Please note, if one staff member will be responsible for performing the duties of two positions, please ensure that is clear in the proposal and that requirements for both positions are included in the CV. As part of their proposed services, the Offeror may propose additional personnel as deemed necessary to support these services, and their experience should be covered in the proposal.

Project Manager

The Project Manager will have primary responsibility for ensuring that all tasks under this contract are of the highest quality and are completed on time within budgeted amounts. The Project Manager will manage workflow among team members, sub-contractors (if applicable), and others contributing to the work performed under this SOW. The Project Manager shall have full authority to act for the Contractor on all contract matters relating to daily operation of this contract.

Minimum Qualifications:

• Bachelor’s Degree or higher

• At least 7 years of project management experience overseeing international, population-based research.

• At least 7 years of experience performing and overseeing quality control of survey data gathered through face-to-face (CAPI), telephone (CATI), or/web-based methods (CAWI).

• At least 7 years of experience with administrative responsibilities including meeting deadlines, achieving targets, and overseeing the preparation and submission of required reports to client

• At least 7 years of experience building and maintaining working relations with stakeholders

• Fluent in English, both written and speaking

Developer(s)

The Developer(s) will have primary responsibility for creating the programming code and syntax that supports efficient, accurate review of survey data and creation of derived variables from field variables.

Minimum Qualifications:

• Bachelor’s Degree Equivalent or higher

• At least 5 years of experience working with statistical software such as SPSS, Stata, R, Python

• At least 5 years of experience developing software applications or coding that supports automated data quality control operations

Factor 3 - Relevant experience

The Government will evaluate the Offeror's relevant experience, determining the extent to which previous projects and clients provide a strong foundation for accomplishing the tasks and deliverables of this SOW.

• Working with CAPI and CATI data collected from populations outside of the United States.

• Performing quality control measures on CAPI and CATI survey data.

• Substantial experience with checking and identifying errors in survey scripts programmed into CAPI and CATI administration systems.

• Reviewing and unifying datasets featuring population-based research, particularly representative, quantitative data.

• Using software programming to automate data transformation activities.

• Advising organizations on data management best practices and transformation

• Contractor shall submit relevant examples of data dictionaries, codebooks, scripting, data quality control deliverables on similar, previous projects. The examples do not have to have actual data; rather they should be examples of deliverables.

PAST PERFORMANCE EVALUATION

The Government’s intent is to evaluate each Offeror’s past performance, including quality, schedule, management, and regulatory compliance to determine whether the Offeror consistently delivered quality services in a timely manner. The Government will evaluate relevant and recent past performance. The information presented in the quotation, together with information from any other sources available to the Government may provide input for the evaluation. Lack of past performance will be treated neither favorably nor unfavorably.

This factor will be evaluated based on information obtained concerning the Offerors’ performance on contracts, consistent in scope and complexity with this solicitation.

“Recent” is defined as work that has been performed within the last three (3) years from the date of this solicitation.

The Government will perform an independent determination of relevancy of the Offeror’s past performance (which includes, if applicable, the extent of its critical subcontractors’ or teaming partners’ involvement) based on the data provided or obtained. The Government may not necessarily agree with the Offeror’s opinion of relevancy. Each previous/current contract provided in the response will receive one of the relevancy ratings listed below:

Past Performance Relevancy Assessment Rating Description

Relevant Present/past performance effort involved similar scope and magnitude of effort and complexities this solicitation requires.

Not Relevant Present/past performance effort involved little or none of the scope and magnitude of effort and complexities this solicitation requires.

Offerors are to note that, in conducting this assessment, the Government reserves the right to use data provided by the Offeror, and data obtained from other sources, including, but not limited to: Contractor Performance Assessment Reporting Systems (“CPARS”), or similar systems of other Government departments and agencies; Defense Contract Management Agency (“DCMA”); interviews with program managers and Contracting Officers, and other sources known to the Government, including commercial sources. This information may include data on efforts performed by other divisions, critical subcontractors, or teaming contractors, if such resources will be brought to bear or significantly influence the performance of the proposed effort. The Government may consider as relevant efforts performed for agencies of the federal, state, or local governments and commercial customers. More recent performance will have greater impact than less recent performance. More relevant performance will have greater impact than less relevant effort. OFFERORS DO NOT NEED TO SUBMIT ANY INFORMATION FOR

THIS FACTOR.

In evaluating the quality of each offeror’s relevant and recent past performance, the following performance confidence ratings will be used:

Performance Confidence Assessment Ratings Rating Description

High Confidence Based on the Offeror’s performance record, the government has high confidence the Offeror will successfully perform the required effort, without significant performance problems.

Satisfactory Confidence

Based on the Offeror’s performance record, the government has satisfactory confidence the Offeror will successfully perform the required effort. Any performance problems would be expected to be resolved without significant attention by the Government.

Neutral/Unknown Confidence No relevant performance record was found or is available and therefore the offeror is not evaluated favorably or unfavorably on past performance.

Little Confidence

Based on the Offeror’s performance record, substantial doubt exists that the Offeror will successfully perform the required effort. The resolution of performance problems would be expected to require significant attention by the Government.

No Confidence Based on the Offeror’s performance record, extreme doubt exists that the Offeror will successfully perform the required effort.

VOLUME II – PRICE QUOTATION

The Offeror shall provide pricing for each task on an Excel document for the base and each option period.

Offerors shall utilize the pricing sheet provided (Attachment C (2 tabs)). Responses received for Volume II – Price that do not utilize the USAGM provided pricing sheet may not be considered for award.

Quotations should include the Offeror’s best pricing; including all discounts, as there may not be another opportunity to offer further discounts.

BASIS FOR AWARD

The major evaluation criteria for this solicitation are the technical factors which encompass Factor 1 (Technical Approach), Factor 2 (Key Personnel), Factor 3 (Relevant Experience), and Factor 4 (Past Performance), and the Price factor. Although the technical factors are of paramount consideration in the award of the contract, price is also important to the overall contract award decision. All technical evaluation factors, other than price, are equal to each other, and when combined, are significantly more important than price. In any case, the Government reserves the right to make award(s) to that offeror whose quotation provides the best overall value to the Government.

Offerors are advised that award will be made to that offeror whose quotation provides the combination of features that offers the best or greatest overall value to the Government. The Government is more concerned with obtaining performance capability superiority rather than lowest overall cost. However, the Government will not make an award at a significantly higher overall cost to the Government to achieve only slightly superior performance. Overall cost to the Government may become the ultimate factor for award of a contract as quotations become more equal based on other factors.

The evaluation will be based on the demonstrated capabilities of the prospective Contractors in relation to the needs of the project as set forth in the solicitation. The merits of each quotation will be evaluated carefully. Each quotation must document the feasibility of successful implementation of requirements of the SOW. Offerors must submit information sufficient to evaluate their quotations based on the detailed criteria.

UNREALISTIC QUOTATION

Unrealistically low or unreasonably high proposed prices may be grounds for eliminating a quotation from the competition. Such a quotation may be viewed as indicative of lack of understanding of the Government’s desired objective. The Government may reject any quotation that is determined to be unrealistic of requirement commitments, including terms and conditions, and evaluation criteria or is unrealistically low or unreasonably high in price, such that the quotation is deemed to reflect an inherent lack of competence or failure to comprehend the complexity and risks of the requirement.

The Government reserves the right, but is not required, to conduct a price realism analysis to determine whether the proposed labor rates are realistic for the work to be performed and to allow for recruitment and maintenance of a skilled workforce. If the government decides to conduct a price realism analysis, additional information from the offeror(s) may be necessary.

In addition, Vendors are cautioned against submitting a materially unbalanced quotation. Quotations that are determined to be materially unbalanced can be rejected. The Government will analyze quotations to determine whether they are unbalanced with respect to prices or separately priced line items. A quotation is materially/mathematically unbalanced if it is based on prices that are significantly less than the price for some contract tasks and significantly overstated in relation to price for others. A quotation is materially/mathematically unbalanced if:

· There is a reasonable doubt that the quotation would result in the lowest overall price to the Government, even though it is the lowest evaluated quotation or

· The quotation is so grossly unbalanced that its acceptance could result in the Government initially overpaying for a portion of the work performed.

Solicitation Attachments:

Statement of Work (SOW) Attachment A Terms and Conditions / Clauses Attachment B Price Schedule Attachment C Pre-Submission Inquires Spreadsheet Attachment D Templates (Examples) Attachment E (E.1 - E.6)