Sections B-M Final.pdf

International Audience Research Project (IARP) BBG50-S-11-4564-RJ

July 07, 2011 Page 1 of 70

PART I – THE SCHEDULE

SECTION B – SUPPLIES OR SERVICES AND PRICES/COST

B.1 WORK BREAKDOWN STRUCTURE

The BBG/IBB‘s Office of Contracts wishes to competitively award one or more Performance-based, Indefinite Delivery Indefinite Quantity (IDIQ) Contracts. It wishes to make multiple awards, but will make a single award if appropriate. The Contract will be implemented through

Firm-Fixed- Priced (FFP) Task Orders with price determined either based on a fixed price negotiation with a single awardee or competition among the multiple awardees of the Contract.

B.2 IDIQ TASK ORDER WORK AND LABOR RATES

a. The Contractor shall provide all necessary direct/indirect labor, materials, equipment, supervision, management, travel, subcontracts, and any other personnel or material resources that are necessary for the Contractor to perform all IDIQ Task Orders that are issued to the Contractor by an authorized Ordering Official (i.e., the Contracting Officer

(CO) and the Contracting Officer Technical Representative (COTR)).

b. The IDIQ individual Task Orders to be performed by the Contractor will be in five Task

Areas. These five Task Areas are described in Section C.3.3 of the Statement of Work

(SOW), and the ordering procedure and required documentation for individual Task

Orders are stated in Section H.4 (Procedures for Initiating, Negotiating, and Issuing Task

Orders) of this document.

c. Individual prices of IDIQ Task Orders will be determined by either of two ways depending on the number of awards made:

a. Single - Negotiated with the Contractor (based on the arithmetic product of negotiated man-hours multiplied by fully-burdened, fixed hourly labor rates accepted by the Government at time of Contract award.)

b. Multiple - Competed among the multiple awardees of the Contract.

Please note where the Task Order work necessitates the use of a subcontractor(s) for any required fieldwork or other work that is part of the Task Order, the Contractor‘s proposed price for the Task Order should include such subcontracting work.

B.3 PRICING

Offerors must provide offered prices that include Firm-Fixed-Labor Rates. The Rates must include all costs and profits. The Rates must be applied to the hours proposed for the types of personnel that will participate in varying degrees, depending on which task area that is being ordered and the particular circumstances of each such task, such as its complexity and performance period in performing each ordered Task Area.

B.4 BASE PERIOD PRICES

July 07, 2011 Page 2 of 70

B.4.1. Base Year Firm-Fixed-Labor Rates (October 2011 – September 2012)

Offerors shall determine and provide the labor category of employees needed to successfully support each of the five (5) identified task areas in the SOW (Section C), with firm-fixed labor

(hourly) rates. Template for providing labor categories is provided below; add/remove rows to template as needed.

1. Performance Metrics and Process Improvement Consulting

Labor Category Firm- Fixed Hourly Rate

2. Data Collection/Fieldwork, Data Processing, Quality Control

3. Data Analysis, Reporting

July 07, 2011 Page 3 of 70

4. Media Consulting

5. Customized Web-Based Data Portal

July 07, 2011 Page 4 of 70

B.5 OPTION YEAR I PRICES (October 2012 – September 2013)

B.5.1. Option Year 1 Firm-Fixed-Labor Rates

July 07, 2011 Page 5 of 70

July 07, 2011 Page 6 of 70

B.6 OPTION YEAR 2 PRICES (October 2013 – September 2014)

B.6.2. Option Year 2 Firm-Fixed-Labor Rates

July 07, 2011 Page 7 of 70

July 07, 2011 Page 8 of 70

B.7 OPTION YEAR 3 PRICES (October 2014 – September 2015)

B.7.2. Option Year 3 Firm-Fixed-Labor Rates

July 07, 2011 Page 9 of 70

July 07, 2011 Page 10 of 70

B.8 OPTION YEAR 4 PRICES (October 2015 – September 2016)

B.8.2. Option Year 4 Firm-Fixed-Labor Rates

July 07, 2011 Page 11 of 70

July 07, 2011 Page 12 of 70

SECTION C

DESCRIPTION/SPECIFICATIONS/WORK STATEMENT (SOW)

C.1 INTRODUCTION

The Broadcasting Board of Governors (BBG) is an independent Federal entity that is responsible for U.S.-funded non-military international broadcasting. It derives its authority from The

International Broadcasting Act of 1994, 22 U.S.C 6201 et seq., The Foreign Affairs Reform and

Restructuring Act of 1998, Pub. 105-277 §§ 1301 et seq., The Smith-Mundt Act of 1948, Pub. L.

80-402, as well as other statutes and executive orders. Based in Washington, DC, the BBG oversees a combination of federal and private broadcast entities. On the federal side, BBG oversees, through an umbrella organization called the International Broadcasting Bureau (IBB), the Voice of America (VOA), and Radio and TV Marti (referred to collectively as the Office of

Cuba Broadcasting (OCB)). VOA is headquartered in Washington, DC. OCB is located in

Miami, FL.

The BBG also issues and supervises grants to Radio Free Europe/Radio Liberty (RFE/RL, Inc.)

located in Prague, Czech Republic, Radio Free Asia (RFA, Inc.) located in Washington, DC, Alhurra and Radio Sawa, which arepart of the Middle East Broadcasting Network (MBN), headquartered in Springfield, Virginia. While consistent with the BBG‘s overall mission to promote freedom and democracy through ―accurate, objective and balanced news and information‖, each of these entities has its own unique mission under the law, and collectively they account for over 162 million regular listeners and viewers around the world. Broadcasts take place in 59 languages, and programs are delivered via an increasingly complex array of multi-media platforms, including AM and FM radio, shortwave, satellite and terrestrial TV, wired and mobile Internet (including social media), SMS, and even face-to-face events.

The BBG, like all federal agencies, is required to show evidence of performance and effectiveness. Common measures such as audience reach and reliability of programming have long served the BBG in assessing the impact of our media on populations. Research is also used to drive strategic and program planning within the agency. Information about media markets, trends in technology adoption and audience demographics and preferences inform these efforts.

Since 2001, the International Audience Research Program (IARP) contracts have supplied the agency with the business intelligence required for both performance measurement and planning through intensive study of broadcaster markets and audiences.

In order to meet the continuing needs of the agency, the Government intends to issue a

Performance-based, Indefinite Delivery Indefinite Quantity (IDIQ) IARP contract to begin

July 07, 2011 Page 13 of 70

Fiscal Year 2012. The choice of a Performance-based IDIQ Contract, in which Contractor(s) will propose how to best meet agency objectives through activities and tasks, the number and value of which have yet to be determined, is intended to offer the agency the flexibility it requires in a global research program that must be responsive to fast-changing environments.

The Contractor(s) selected will provide the BBG with regular, current and accurate audience and market research from approximately 50-75 countries per year through a combination of surveys, qualitative and evaluative research, secondary data acquisition, analysis, data integration and reporting, as well as consulting services.

Consulting services will focus on augmenting the agency‘s current performance measures and performance reporting. Historically, the BBG has limited itself to measuring and reporting the size of the audience (as percentage of national population) at a moment in time, and audience opinions about the relative quality of BBG content. Citing audience size by counting individual consumers of the product, however, does not describe the viral life of that product, or distinguish between myriad levels of engagement practiced by different audience segments. The challenge becomes even greater when agency stakeholders seek to measure any effects beyond journalistic impact,e.g, fostering freedom or democracy, promoting peace or pluralism, providing a forum for dialogue, increased understanding of current events, or any number of greater ambitions of publicly supported broadcasting. Because measures should track missions, differences in mission between broadcast entities result in measurement challenges for the BBG as a whole, and the combined research program that supports its strategic planning.

While current measures will continue to be vital to a new research program, the goal is to go beyond them in ways that will help the agency and its broadcasters understand more about how sub-national markets are functioning, motivations behind the media behavior of target (in addition to existing) audiences, the impact of media beyond the individual consumer, creating compelling content that is market-appropriate, extending reach across target markets, and responding to the increasing and changing requirements for performance measures by Ffederal oversight authorities.

C.2 BACKGROUND

C.2.1 Statutory Requirements for Audience Research

Applicable law requires that the BBG conduct audience research. These laws include:

The International Broadcasting Act of 1994, 22 U.S.C 6201 et seq., which requires the BBG to:

Review language services and make decisions to add or delete language services at least annually;

Ensure that the quality of broadcasts conforms to applicable statutory standards;

July 07, 2011 Page 14 of 70

Ensure that the broadcasts reach significant audiences; and

Use audience research as the basis for its decision-making.

The Government and Performance Results Act of 1993 (GPRA), Pub. L. 103-62 mandates that all federal entities set goals and establish performance measures to indicate progress or lack of progress in meeting these goals. These results must be reported to Congress annually. At this time, three of the four GPRA performance measures for BBG depend at least in part on in-country surveys: ―Audiences Reached‖ (measured as the percentage of a population reached directly by BBG content at least weekly), ―Credibility‖ (measured as the percentage of those reached weekly that rate programming as trustworthy) and ―Understanding‖ (measured as the percentage of those reached weekly that claim programming has increased their understanding of current events, or other defined program elements). Although guidance from the Office of

Management and Budget to the agency is still pending at this time, recent amendment to the

GPRA further increase the requirement of Federal agencies to prioritize the use of performance-related data in the interest of program improvement. The Government Accountability Office

(GAO), who frequently audits implementation of GPRA, requires that these measures be generated by means of a bona fide sample with all appropriate documentation as to sampling error and design effects, if clustering is used, where feasible. As part of its performance review process, BBG‘s Board also requires the agency to complete its annual evaluation of all programming streams (radio, TV, and Internet) of each entity based partly on consumer/outside expert review of one sample of programming, and peer review of a second sample. With the nature of broadcasting and media consumption changing so rapidly, the fragmentation of media audiences is now a reality in nearly all but the most undeveloped markets, and the increasing interest in demonstrating effectiveness in support of BBG and broadcaster missions, the agency foresees the possibility of amending GPRA performance measures during the life of a new research contract. BBG plans to award contract(s) that will be flexible enough to allow for the design of new methods of measuring and reporting performance.

The Performance and Accountability Report (PAR) is required to satisfy the reporting requirements of six pieces of legislation enacted from 1986-2002, including GPRA and the

Reports Consolidation Act. It combines performance and audited financial information intended to assist Congress, the President and the public in assessing the BBG‘s performance relative to its mission and stewardship of the resources entrusted to it. In addition to the BBG performance indicators required by GPRA, the PAR tracks former Program Assessment Rating Tool (PART) measures where applicable, including audience size by region, awareness of U.S. International

Broadcasting, and annual transmission hours. It also tracks cost per audience member, the development of BBG‘s broadcast distribution networks and signal strength, indicators which audience research informs indirectly.

July 07, 2011 Page 15 of 70

The combined impact of these legislative and other forces affecting U.S. International

Broadcasting has greatly enhanced the importance of audience research. While such research should always be structured flexibly enough to respond to a changing marketplace, performance reporting requires that research findings be available on time and in BBG specified formats, since the results must meet certain predetermined milestones every year. These milestones include:

At least one program review for each language service each year by the broadcasters;

Mandated annual deadlines for delivery of GPRA and PAR documents to the Office of

Management and Budget( OMB) and Congress;

At least annual Language Service Review by the Board.

Other milestones may be required in FY 2013 as a result of the GRPA.

Research in foreign markets is difficult and there is a need for improved sampling, survey quality control, intelligent weighting (based on available demographic information), instrument design, and reporting. Further, increased use of the Internet, television and mobile phones for news presents constant challenges to researchers. Alternatively, in other countries that are increasingly closed to outside researchers, methods for estimating critical broadcast research variables require continuous development and ingenuity.

More information on the structure of U.S. International Broadcasting can be found at the following websites:

BBG – www.bbg.gov

VOA – www.voanews.com

Radio/TV Marti – www.martinoticias.com

RFE/RL – www.rferl.org

RFA –www.rfa.org

Radio Sawa – www.radiosawa.com

Alhurra TV – www.alhurra.com

July 07, 2011 Page 16 of 70

C.2.2 Objectives

The BBG requires a program of regular, current, accurate audience research provided by a

Contractor(s). The objectives of this research program are:

To satisfy the requirements of the agency‘s statutory framework, which stipulates that

BBG use audience research as a basis for decision-making. (This decision-making includes general oversight, as well as annual reports on performance to OMB and

Congress and an annual review of broadcast languages to consider additions and deletions of programming and language services, as described above.);

To meet the needs of professional broadcasting management. (The BBG recognizes that current, accurate, comprehensive audience research is essential to professional broadcasting, and that the language services of all the broadcasting entities must have access to a wide range of audience and market research to, among other things: discern media habits and how they are changing, identify and/or describe target audiences, understand content preferences, provide input to format changes, transmission requirements, and distribution opportunities, including local FM/AM and television affiliations);

To enable BBG to have consistent, accurate, reliable measurements of broadcast effectiveness and quality so that the components of a GPRA mandated Strategic Plan and

Performance Plan can be enacted and applied at all levels of management in all the entities;

To support the BBG in its collaborative relationships with other international broadcasters and U.S. Government agencies in the field of shared research;

To provide objective research in timely manner in the direct support of approximately 75 individual program reviews annually.

C.3 SCOPE OF WORK

C.3.1. Type of Contract

In order to meet the research needs of the BBG, the Government intends to issue a Performance-based Indefinite Delivery Indefinite Quantity (IDIQ) Contract or Contracts, implemented primarily through Firm-Fixed- Priced (FFP) Task Orders. In a Performance-Based Contract, the

Government specifies requirements in terms of performance objectives and the Contractor(s) proposes 'how to' best satisfy those objectives. Task Orders involving field research, other

July 07, 2011 Page 17 of 70 significant activities, or subcontracts will be issued on a FFP in order to control costs. The

Government is considering awarding to multiple firms who would compete for Task Orders. If the Government chooses to award to only one firm, the prices of Task Orders will be determined by cost or price analysis, as appropriate.

Awards will be based on the Government‘s evaluation of all Offerors‘ written proposals. The

Government will select the offer(s) that represent best value to the Government.

C.3.2 Technical Requirements

The Contractor(s) shall implement audience and market research for the BBG and all the broadcasting organizations that it supervises, to include a total of approximately 75 language services broadcasting worldwide. Each of the language services and the BBG's Office of

Strategic Planning and Performance Measurement will require varying combinations of quantitative, qualitative and evaluative research, analysis and research-based consulting services.

These combinations will be specified in Task Orders issued under the contract. The Contractor(s) will in every case be responsible for quality control and for delivery of the resulting research findings in specified formats and by specified dates (See Section C.3.5, ―Review Schedule‖).

Precise project milestones, deliverable formats and deliverable deadlines will be outlined in each

Task Order. Specifically, the Contractor(s) is required to:

Provide consulting services in the area of overall performance measurement to accommodate changes to GPRA and other federal requirements;

Meet regularly with BBG Research staff, Broadcaster management and other agency stakeholders to understand changing research requirements;

Appraise the value of existing audience research to the design and implementation of future research and consulting services;

Conduct, directly or through subcontract arrangements, fieldwork related to data collection, quality control, data processing and analysis of research findings in accordance with schedules outlined in the Task Orders;

Ensure all intellectual property rights clauses as well as mandatory federal clauses flow-down to subcontracts.

Report research findings into highly consumable, condensed 'programmer's packages' that allows stakeholders to quickly identify the most relevant information about their service and market;

July 07, 2011 Page 18 of 70

Provide expert consultation on turning research findings into actionable program improvements to broadcasters of a selection of specified language services;

Provide real-time statistical analysis to support planned and ad-hoc agency research functions, either through on-site analysts and/or a service bureau model able to answer data queries by agency staff;

Provide a variety of aggregate monthly, semi-annual and annual reports for BBG in specified formats for the purposes of agency performance reporting;

Supportthe provision of a web-based interactive research portal that provides multiple levels of secure access to all research deliverables, project documentation and data query application(s) to stakeholders inside the agency, and in partnership organizations (e.g.

other U.S. Government agencies, international broadcasters);

Make a variety of formal and informal in-person or virtual presentations on research findings to agency staff;

Provide special consultations as required.

B. Further detail on the above items is provided in Section C.3.3 of the SOW.

C. The estimated combined total value of the Contract or Contracts is $50,000,000 over 5 years, with an annual floor for each Contract awarded (minimum spend) of $5,000.

C.3.3 Contract Task Areas

The IARP will require a host of research, analysis and reporting tasks that fall into five (5) Task

Areas. Task Orders can consist of activities from one or multiple Task Areas. All but Task Area

2 are highly consultative, either in the beginning of a task or throughout the performance of a task. They require the Contractor(s) to engage with agency staff in order to understand stakeholder needs regarding the specified topic area, and make recommendations based on the

Contractor(s) expertise.

1. Performance Metrics and Process Improvement Consulting

As described in Section C.2.1 and subsequent sections, GPRA requires annual, research-driven performance reporting. The agency currently uses four research-based performance indicators for its combined GPRA/PAR reporting: Reach (how many individuals have consumed BBG content once a week or more, expressed as a percentage of national populations and/or numerically as

‗size‘); Credibility (the percentage of the weekly audience that finds BBG content ‗very‘ or

‗somewhat‘ trustworthy); Understanding (the percentage of the weekly audience that reports increased understanding of current events or other related topics as a result of consuming BBG

July 07, 2011 Page 19 of 70 content); and Quality (a combination of external and internal content review and mission-compliance of specific language programming).

Three of these indicators, reach (or size), credibility and quality, have been used to gauge international broadcasting performance for decades, and have been required research output of the IARP contract since 2001. The fourth, understanding, was developed more recently at the request of the Board in an attempt to gauge impact of agency activities beyond strict journalistic output, yet in keeping with agency and broadcaster missions.

Ten years later, and in anticipation of new GPRA guidance with increased focus on process improvement, the agency is very interested in reviewing and possibly amending its performance metrics, and hence questions asked in surveys. Although it is unlikely that the ―weekly reach‖ and some measure of credibility, with baselines established by representative national surveys, will recede as core performance indicators, the agency still seeks consulting services to help answer the following questions:

What is the right set of indicators for the future?

What are the best methods for measuring performance using the chosen indicators?

How far along the logical framework can the agency measure beyond journalistic output to programmatic outcomes, or even impact?

This Task Area has traditionally demanded the greatest effort and resources in past contracts, and despite the growth in importance of the other Task Areas under a new procurement, it will likely remain the largest Task Area. In order to meet the objectives described in Section C.2.2, the

IARP will require primary research from approximately 60 countries and territories. Many of the populations under study are difficult to reach, either because of geography, lack of infrastructure, political instability, or repression. Some are in active conflict zones. The following is a representative list of countries and territories covered by some type of survey or qualitative research within the IARP in a typical year:

Afghanistan, Albania, Armenia, Azerbaijan, Bahrain, Bangladesh, Belarus, Bosnia, Burma, Burundi, Cambodia, China, Colombia, Cuba, Democratic Republic of Congo, Egypt, Ethiopia, Georgia, Ghana, Haiti, Hong Kong, India, Indonesia, Iran, Iraq, Jordan, Kazakhstan, Kosovo, Kuwait, Kyrgyzstan, Laos, Lebanon, Macedonia, Moldova, Montenegro, Morocco, Mozambique, Nepal, Nigeria, Niger, North Korea, Pakistan, Palestinian Territories, Russia, Rwanda, Saudi Arabia, Serbia, Somalia, Syria, Tajikistan, Tanzania, Thailand, Tibet, Turkey, Turkmenistan, UAE, Ukraine, Uzbekistan, Vietnam, Zimbabwe

July 07, 2011 Page 20 of 70

Types of field-based research in this Task Area will include surveys (face-to-face and phone), qualitative, evaluative, and observational research, as well as alternative methodological approaches suggested by the Contractor(s) and acceptable to the agency. Fieldwork will be scheduled specifically to accommodate readiness for the annual program review of each of the

Agency‘s approximately 75 language services in accordance with the broadcast entities‘ schedules, as illustrated in Section C.3.5.

Data entry and data processing activities will follow structures that are consistent with final deliverable formats as established in Task Orders, to include data files in SPSS format (IBM software originally named Statistical Package for the Social Sciences), derivative Excel spreadsheets, standard frequency tables, and interview transcripts where required.

Quality Control activities will strictly adhere to the Quality Control/Quality Assurance Plan proposed by the Contractor(s) as accepted by the Government (see Section C.3.4). The agency‘s staff of professional researchers are involved in monitoring research activities at all levels, from instrument design and verification, sample design, fieldwork observation including respondent contact, guidelines for training of enumerators and other field staff, data verification down to the individual record and weighting decisions.

3. Data Analysis, Reporting

In support of both performance reporting and improving understanding of media markets and target audiences, a wide variety of data analysis and reporting tools and approaches will be required under the IARP, the combination varying with each Task Order. The agency will continue to require certain analytics as standard, broadly outlined above. Many of these lend themselves to automation in both analysis and reporting. Additional analytics will be required however, particularly in projects where there is a driving strategic interest or where a broadcast service is facing challenges such as declining reach or an unsustainable shift in audience demographics. The agency is interested in suggestions by the Contractor(s) about the type of analytics that will address these challenges and help answer questions such as the following:

What characteristics beyond demographics correlate with high or low consumption of

BBG and/or its competitors‘ products?

What are the characteristics of someone who multiplies the effect of a broadcast through forwarding or discussing its content with others?

What are the strong BBG brand equities that might best be leveraged in specific markets?

How does access to growing media platforms correlate to consumption of BBG content across markets?

July 07, 2011 Page 21 of 70

What does it mean when someone says their understanding has been increased through the consumption of BBG content?

What do the trends say about the best distribution strategies in stable markets? Volatile markets?

The above questions are only examples when considering the needs of approximately 75 different broadcast services and other BBG stakeholders. They will all be predicated, however, on appropriate research planning and instrument development. To achieve a desired analysis as output, the appropriate inputs must exist. In this way, Task Area 3 is also about customized research design and therefore both precede and overlap with activities in Task Area 2.

The reporting of research data and analytic findings will consist of both standardized and custom products.

Standard reporting: Performance measures, media access and consumption data, station awareness levels, pre-defined content preferences by population or demographics can be automatically reported in any number of predictable formats, to minimally include: executive summary reports or tables, the Language Service Review book, derivative spreadsheets commonly used in resource allocation planning.

Custom Reporting: There will also be requirements for custom reporting based on the needs of various stakeholders throughout the Contract term. These may include anything from ad-hoc analytic reports for a broadcast service that wants more detailed information about its audience or market to roll-up presentations of research at Executive ―Deep Dives‖, where representatives from multiple Federal agencies delve into all available data sources about a country or region of intense interest.

Programmer’s Packages: Every language service is required to participate in an annual review of its operations. The precise research inputs to these reviews vary, but a description of its audience and market are central to the process. To help broadcasters get what they need out of research and analysis, each will receive a Programmer‘s Package developed by the Contractor(s).

This document will highlight performance metrics, market data and trends, audience characteristics and, if applicable, program recommendations. It will be short, highly consumable, combine survey, qualitative and other data as required, and focus on actionable findings. It will address the problem that broadcasters and others have historically had with too much data and not enough meaning.

4. Media Consulting

No research, no matter how high quality and compelling, is worth conducting if the data does not increase users' understanding of their subject and provide inputs to actionable program improvements. Not only is this a standard in Program Management, demonstration of

July 07, 2011 Page 22 of 70 improvements is likely to be increasingly required by federal oversight authorities. The

Contractor(s) will be required to provide expert analysis of the media data and offer informed strategic thinking regarding BBG media platforms and content. The effort is intended to assist broadcast service directors in distilling the data into specific action items that can be employed across television, radio, and new media. Not all broadcast services for which research is conducted will need specific recommendations for change in a given research cycle, but many will. Some broadcast services will benefit from recommendations focused on singular issues, such as format length, others will benefit from a more concerted focus on a multi-media delivery strategy or a combination of content issues.

Recommendations coming out of this type of consulting will need to be briefed at multiple levels, from the language-specific services they concern, to managers responsible for implementing programs and executives conducting strategic planning.

5. Customized Web-Based Data Portal

The BBG has an increasing need to store and distribute customizable audience and market data as a core element of a more robust agency-wide knowledge management system. Under current practices, source data, derivative data, analytic reports, meta-data (including research management tracking) and data query functions reside in multiple locations with unpredictable access requirements. Within this Task Area, the Contractor(s) will be required to assist agency identify usability parameters and narrow down key functionality in order to design a data portal and possible data query tool. The agency is undergoing a re-organization of its technical capacity, and potential tasks under this Task Area range from IT consulting, design and/or provision of one or more elements of a data portal.

Minimally, the agency must have access to the following functionality within its own content and document management system:

An archive containing all data source files, with Administrator ability to upload documents in SPSS and other formats;

Varied-level access to all documents and data;

Documents organized logically on the portal;

A data query tool allowing staff of varying technical ability to easily locate survey variables, frequencies and cross-tabulations viewable in multiple formats - these might be canned (pre-calculated) or calculated in real-time by the user (TBD);

A data export function;

July 07, 2011 Page 23 of 70

Wiki-page for research-related commentary;

Project management/tracking functions.

Appropriate licensing where applicable;

Unlimited rights to all data collected and delivered under the Contract;

Security features consistent with agency standards.

C.3.4. Research Management and Administrative Requirements

Management Capacity

The Government's intent is to issue Task Orders that reflect a Performance-Based Contracting approach. In a Performance-Based Contract, the Government shall specifies requirements in terms of performance objectives and the Contractor(s) shall proposes 'how to' best satisfy those objectives. Task Orders, primarily negotiated and issued at the level of broadcast language, will include tasks from one or more of the five (5) Task Areas outlined above, in the combination that best suits the stated performance objectives. In totality, the agency expects all Task Orders under this Contract to produce between 40-75 surveys as well as qualitative or observational research projects; hundreds of hours of analytic and consulting services annually; and either a discrete or integrated web-based data portal in the first year. The Contractor(s) must be able to perform up to $10,000,000 of Task Orders annually, as well as provide required services with full continuity should the Task Orders total less.

In order to effectively support this Contract and the key and program staff assigned to it, the

Contractor(s) is required to provide deep and experienced back-room management, using robust and tested financial and accounting software, contract tracking and IT systems. Experienced

Contract specialists must be able to create and enforce subcontracts across the globe with proper flow-down of Contract terms. A network of field teams must already be in place, through ownership, subsidiary or subcontract relationships, in dozens of countries, including those with hard to reach populations. Support staff must have back-up personnel when unable to perform tasks related to Contract and subcontract management.

Aside from dedicated finance, accounting, contracts and administrative support staff, the

Contractor(s) will also have available the following expertise available to its project staff working on this Contract, whether as employees or contractors:

Demographers/statisticians

Survey methodologists

Qualitative research specialists

July 07, 2011 Page 24 of 70

Business/process improvement/change management consultants

Writers/editors

Programmers/network support technicians

Although not a requirement, the Government expects and encourages the Offerors to enter into business arrangements that will adequately support the requirements of this Contract.

If two or more firms form a business arrangement and receive Task Orders under this

Contract, the agency staff working on the Contract expects to have access to staff with specialized expertise, whether they are associated with the prime or subordinate firm.

Quality Assurance/Quality Control

The Contractor(s) will propose a detailed research and management quality assurance/quality control plan for acceptance by the Government, and follow it strictly. The plan must describe the frequency and means of communication throughout the organizational hierarchy on different types of projects and how its corporate organizational structure will support the successful implementation of Task Orders. Minimally, the plan must address the following:

Making assignments within a complex teaming arrangement (if applicable);

Tracking Task Order and project status;

Cost control mechanisms;

Remediation in the case of project delays or failure;

Instrument management & protocol for translation and back-translation;

Enumerator training methods;

Ratio of supervisory field staff (and their roles) to enumerators;

Fielded sample integrity;

Interview back-checking protocol;

Data processing oversight;

List of statistical tests and verification checks on data to identify errors or fraud;

Remedial steps in cases of error or fraud;

Verification of analytical findings;

July 07, 2011 Page 25 of 70

Report editing and proofing;

Software testing and maintenance

C.3.5 Program Review Schedule

IARP audience data and attendant analysis are critical inputs to many activities central to the mission of the agency, but none more urgently than Program Review. Indeed, the scheduling of all research projects, and hence most Task Orders, are predicated on the schedules of approximately more than 75 Program Reviews that take place in a 12-month period. It is expected that most, but not all, Task Orders, will be negotiated and issued at the level of

―Program Review‖, which usually corresponds with a broadcast language or family of languages.

For example, a Task Order for Hausa will be required to provide all the research products and services determined as required inputs to the annual Hausa Program Review. With an average of

1-2 reviews per week, the precise scheduling of research inputs is critical. On the following page is a typical 12-month schedule for program reviews across the entities.

Annual Language Service Review is typically conducted once per year, with a publication of findings on the agency's website. GPRA and PAR reports are submitted once per year and strategic reviews can take place anytime at the discretion of the Board of Governors.

Note: The Draft Program Review Schedule for 2012 may be found in Section J, Attachments.

July 07, 2011 Page 26 of 70

C.3.6 Performance Standards and Delivery Parameters

All data and deliverables that result from research performed under this Contract remain the property of the U.S. Government and shall not be provided to any other parties without the express prior written approval of the BBG, nor used for any other purposes outside those specified in the Contract by the Contractor(s). In this Contract, "data" means recorded information, regardless of form or the media on which it may be recorded. The term includes technical data and computer software. The term does not include information incidental to contract administration, such as financial, administrative, cost or pricing, or management information.

The Contractor(s) is required to carry out audience research, data analysis and consulting services according to the requirements outlined in each Task Order. However, certain minimal technical standards will apply to Task Order activities and resulting deliverables, as outlined below:

Quantitative Surveys

All quantitative surveys must ensure instrument integrity. The agency uses a questionnaire that has evolved over time, but retains all key performance measures, and results in interviews lasting approximately 45 minutes. Every survey will require some adaptation to the market, implemented by the Contractor(s) and approved by a Research

Director, translated into the major languages to be used in fieldwork, and subject to a non-disclosure agreement. The first time under the Contract a questionnaire is used in a market, it will be back-translated in its entirety into English to ensure accuracy.

Subsequent surveys in that market will only have any changes to the questionnaire back-translated into English. A record of all interviews that are conducted in a language for which there is no translation (not uncommon in Africa) must be kept and coded in the final data file;

Under no circumstances can any questionnaire be altered in any manner, including, but not limited to adding questions or removing or relocating questions, without the prior written permission of the BBG;

Sampling must be efficiently designed to ensure the greatest representation of the population under study as possible, while not exceeding the number of interviews necessary for such representation. The sample should be randomized at every strata

(excepting, for example, the purposive selection of a location for a booster sample), from the primary sampling unit down to the individual respondent. Sampling points should be

July 07, 2011 Page 27 of 70 selected using the PPS (probability proportional to size) when possible, using the most robust frame data available. Respondents should be selected via the Kish grid method;

Contractor(s) must suggest a post-weighting approach for all data, with a goal of not exceeding 10 as a weight factor;

Every survey must follow the previously defined and accepted Quality Control Plan outlining Contractor and subcontractor communication, the training to be received by field teams of interviewers and supervisors, acceptable field interviewer/supervisor ratios, field observation and/or training conducted by Contractor personnel, data processing plan

(staffing, location, mode (e.g. double entry, scanning, CATI)), data validation plan; The

Contractor(s) is solely responsible for ensuring subcontractor(s) compliance with survey standards and practices;

Every survey will result in a full and complete SPSS data set including, for all cases, (i) respondent ID, date of interview, length of interview, relevant geographic data; (ii) a codebook specifying variable location and names; (iii) technical reporting, detailing a complete and precise definition of the survey universe, every stage of selection, achieved

vs. expected sample, weighting schema if used, margin of error, description of Quality

Control/Quality Assurance Plan as implemented, and any unique field conditions that may have influenced the survey results;

Survey practices must conform to accepted industry standards, as codified by organizations such as Council of American Survey Research Organizations (CASRO), European Society for Opinion and Marketing Research (ESOMAR), American

Association for Public Opinion Research (AAPOR) and Conference of International

Broadcast Audience Researchers (CIBAR)(update pending fall 2011).

Data Purchases

The Contractor(s) is expected to be aware of the availability of commercial data for purchase in every market in which the BBG operates, make recommendations as to the value of such data to the IARP and be able to serve as a supplier of that data where appropriate.

In all cases where the Contractor(s) suggest the use of purchased data, the Contractor(s) shall notify BBG of its intent and request approval from BBG prior to purchasing the data.

Additionally, when purchased data is reproduced in reports to the BBG, the Contractor(s) shall clearly identify those portions of the report containing purchased data and shall provide to the

BBG sufficient license rights to allow BBG to use and redistribute the purchased data as needed.

July 07, 2011 Page 28 of 70

Qualitative, Evaluative and Observational Studies

All such studies must outline very clear criteria for participant recruitment by the

Contractor, in consultation with the agency. Criteria will typically go beyond demographics such as age, gender and education, but also include media and consumer behavior, location in a social or professional network. No changes in recruitment criteria can be made without BBG approval;

All instruments (e.g. questionnaires, moderator or interviewer guides, diaries, etc.)

developed and stimulus material (e.g. broadcast content, consumer electronics) used by the Contractor(s) must be approved by the BBG and translated into the appropriate language for the study;

Reporting must combine appropriate levels of primary data and analysis, tabulated questionnaire responses if used, raw transcripts, English transcripts, video recordings, linguistic content analysis or other form of typology, meta-data on research methods, summary analysis, etc.

Data Portal

Data and deliverables produced under this Contract will be available through a web-based portal to users with a variety of levels of access, both inside and outside the agency. It will comport with the design and functionality accepted by the Government and described in any Task Orders under Task Area 5.Minimally, it should include:

An archive containing all data source files, with Administrator ability to upload documents in SPSS and other formats;

Download and printing rights;

Varied-level access to all documents and data;

Documents organized logically on the portal;

A data query tool allowing staff of varying technical ability to easily locate survey variables, frequencies and cross-tabulations viewable in multiple formats - these may be canned (pre-calculated) or calculated in real-time by the user (TBD);

A data export function;

Wiki-page for research-related commentary; and

Project management/tracking functions.

July 07, 2011 Page 29 of 70

Written agency permission for anything less than unlimited rights, including existing works.

All software, including open source, must undergo a license review by the agency.

Performance Reporting

The Contractor(s) is expected to maintain copies of all authoritative data for the purposes of calculating performance measures as input to GPRA/PAR and other performance reporting. Calculations will be made according to BBG instruction and will include a combination of recently measured audiences and statistically sound audience projections;

Research conducted for a broadcaster will result in a single, highly digestible product, or

―programmer‘s package‖ that integrates the most relevant findings from all research products that quickly identifies the broadcaster‘s place in its market(s).

Performance and market data input directly and regularly into the annual Language

Service Review Briefing Book, found at:

http://media.voanews.com/documents/2010_AnnLanguageServiceReviewBriefingBook.p df

Consulting Services

The Contractor(s) shall provide: (i) the exact personnel proposed when required to provide consulting services; and (ii) all deliverables outlined in a Task Order for such services. No substitutions of personnel can be made on consulting tasks without the prior written permission of the BBG.

July 07, 2011 Page 30 of 70

Addenda

How Audience Research is Being Provided Currently

Funding for the International Audience Research Program (IARP) is appropriated to the agency and managed centrally by the BBG. At the beginning of each fiscal year, the Board's Research

Manager assesses the funds available and while working with the Research Directors of each entity, develops a "research plan', which identifies needs in the form of very specific research products, serving the needs of individual language services as well as strategic initiatives of the

BBG Board. When different broadcast entities (e.g. RFA and VOA) need data about a shared language or country, a single survey typically supplies both, while specialized qualitative projects (e.g. focus groups or depth interviews) serve individual broadcasters. The details of planning at the project level are the responsibility of entity Research Directors. Besides planning, Research Directors vet and convey contract deliverables, typically consisting of raw data files, research reports, briefs and presentations, to entity senior management and the journalists and production staff who create broadcast content.

The projects planned in advance typically consist of the following types, with nearly all projects resulting in the delivery of written reports, ranging from 15-75 pages in length:

Representative Surveys (National, Urban or Capital City): The aim of these surveys is to provide a comprehensive picture of general media use in the area of study, with specific media consumption and behavioral data for the relevant international broadcaster(s). Samples are stratified and randomized at every level, from the primary sampling unit down to the individual respondent. Sampling points are selected using PPS (probability proportional to size) when possible, excepting a purposeful selection of the capital or primary urban area, using the most robust sampling frame available for the population. This may include recent census data, but often relies on alternative data, such as 20-year census projections, or Demographic and Health, World Bank economic or other surveys based on a sample of the target population. BBG surveys typically interview 1000-3000 respondents, using a questionnaire that consists of a core set of standard media behavior questions, broadcaster-specific modules, and a range of ad-hoc questions that may be relevant to the context. The latter may contain attitudinal questions that help provide more nuanced data for audience segmentation and typography. Most surveys are conducted face-to-face in respondents' homes using paper coding, although phone surveys using

Computer Aided Telephone Interviewing (CATI) technology are also used in the right circumstances, such as in countries with very high telephony combined with severe security challenges. Face-to-face surveys last an average of 45-60 minutes, while phone surveys run an average of 15-20 minutes. These surveys provide three of the performance measures currently required by GRPA/PAR reporting.

July 07, 2011 Page 31 of 70

Analytics used for written survey reports typically include:

The three major performance metrics as outlined above;

Access and consumption rates of major media by platform (TV vs. internet);

Exposure to branded channels (e.g. top 10 radio and TV stations);

Frequency of more specific types of new media behavior (e.g. texting vs. listening to radio on mobile phones);

Frequency and source of news consumption;

Levels of awareness of International Broadcasters;

Levels of interest in types of news and information;

Standard demographics, X-tabbed with all the above when sample warrants;

Secondary research on political, economic and cultural influences as context for media behavior.

Non-representative Surveys (Travelers): Traveler surveys are used when it is not possible for political reasons to conduct face-to-face or CATI surveys of an entire population. Rather than attempting to survey within a country, researchers interview nationals traveling outside the country. Although baseline population data is gathered in order to design a sample that most appropriately distributes sampling points across a diverse selection of travelers (business travelers, students, guest workers, tourists, refugees), or quota-sampling used to achieve an appropriate demographic distribution, these surveys cannot quantify a probability of selection or range of error, and therefore,…