Solicitation_12125_Amend1.pdf

PAGES

15A. NAME

AND

ADDRESS

OF

OFFEROR

SEC. PAGE(S) SEC. PAGE(S)

(Date) (Hour)

CALENDAR DAYS

14. ACKNOWLEDGMENT OF AMENDMENTS

(The offeror acknowledges receipt of amend-ments to the SOLICITATION for offerors and related documents numbered and dated:

(Type or Print)

SOLICITATION, OFFER AND AWARD 1. THIS CONTRACT IS A RATED ORDER

UNDER DPAS (15 CFR 700)

RATING

PAGE OF

1 70

2. CONTRACT NO.

There are no clauses/provisions included in this section.

3. SOLICITATION NO.

2010-N-12125

4. TYPE OF SOLICITATION

SEALED BID (IFB)

X NEGOTIATED (RFP)

5. DATE ISSUED

6. REQUISITION/PURCHASE

NO.

HCVJB2X2-2010-81285

7. ISSUED BY CODE 2536 8. ADDRESS OFFER TO (If other than Item 7) Centers for Disease Control and Prevention (PGO)

Acquisition and Assistance Branch 1 2920 Brandywine Road, MS E-15 Atlanta, GA 30341-5539

Approved as to Form and Legality: _____________________________ NOTE: In sealed bid solicitations “offer” and “offeror” mean “bid” and “bidder.”

SOLICITATION

9. Sealed offers in original and 3 handcarried, in the depository located in See Block 7 until 10:00 AM local time August 3, 2010 copies for furnishing the supplies or services in the Schedule will be received at the place specified in Item 8, or if

CAUTION -- LATE Submissions, Modifications, and Withdrawals: See Section L, Provision No. 52.214-7 or 52.215-1. All offers are subject to all terms and conditions contained in this solicitation.

10. FOR INFORMATION

CALL:

A. NAME

Sherrie N. Randall

B. TELEPHONE (NO COLLECT CALLS)

AREA CODE NUMBER: EXT:

(770) 488-2866

C. E-MAIL ADDRESS

srandall@cdc.gov

11. TABLE OF CONTENTS

(x) DESCRIPTION (x) DESCRIPTION

PART I – THE SCHEDULE PART II – CONTRACT CLAUSES

X A SOLICITATION/CONTRACT FORM 1 X I CONTRACT CLAUSES 42

X B SUPPLIES OR SERVICES AND PRICES/COSTS 2 PART III - LIST OF DOCUMENTS, EXHIBITS AND OTHER ATTACH.

X C DESCRIPTION/SPECS./WORK STATEMENT 4 X J LIST OF ATTACHMENTS 51

X D PACKAGING AND MARKING 25 PART IV – REPRESENTATIONS AND INSTRUCTIONS

X E INSPECTION AND ACCEPTANCE 26 REPRESENTATIONS, CERTIFICATIONS, AND

X F DELIVERIES OR PERFORMANCE 27 X K OTHER STATEMENTS OF OFFERORS 52

X G CONTRACT ADMINISTRATION DATA 31 X L INSTRS., CONDS., AND NOTICES TO OFFERORS 57

X H SPECIAL CONTRACT REQUIREMENTS 34 X M EVALUATION FACTORS FOR AWARD 67

OFFER (Must be fully completed by offeror) NOTE: Item 12 does not apply if the solicitation includes the provisions at 52.214-16, Minimum Bid Acceptance Period.

12. In compliance with the above, the undersigned agrees, if this offer is accepted within calendar days (60 calendar days unless a different period is inserted by the offeror) from the date for receipt of offers specified above, to furnish any or all items upon which prices are offered at the price set opposite each item, delivered at the designated point(s), within the time specified in the schedule.

13. DISCOUNT FOR PROMPT PAYMENT

(See Section I, Clause No. 52-232-8)

10 CALENDAR DAYS

20 CALENDAR DAYS

30 CALENDAR DAYS

AMENDMENT NO. DATE AMENDMENT NO. DATE

CODE FACILITY 16. NAME AND ADDRESS OF PERSON AUTHORIZED TO SIGN OFFER

15B. TELEPHONE NO.

AREA CODE NUMBER EXT.

15C. CHECK IF REMITTANCE ADDRESS

IS DIFFERENT FROM ABOVE - ENTER

SUCH ADDRESS IN SCHEDULE.

17. SIGNATURE

18. OFFER DATE

AWARD (To be completed by Government)

19. ACCEPTED AS TO ITEMS NUMBERED 20. AMOUNT

22. AUTHORITY FOR USING OTHER THAN FULL AND OPEN COMPETITION:

21. ACCOUNTING AND APPROPRIATION

10 U.S.C. 2304(c)( ) 41 U.S.C. 253(c)( ) 23. SUBMIT INVOICES TO ADDRESS SHOWN IN (4 copies unless otherwise specified)

ITEM

24. ADMINISTERED BY (If other than Item 7) CODE 25. PAYMENT WILL BE MADE BY CODE 434 Centers for Disease Control and Prevention (FMO)

PO Box 15580 404-498-4050 Atlanta, GA 30333-0080

26. NAME OF CONTRACTING OFFICER (Type or print)

27. UNITED STATES OF AMERICA

(Signature of Contracting Officer)

28. AWARD DATE

IMPORTANT -- Award will be made on this form, or on Standard Form 26, or by other authorized official written notice.

AUTHORIZED FOR LOCAL REPRODUCTION STANDARD FORM 33 (REV. 9-97)

PREVIOUS EDITION IS UNUSABLE Prescribed by GSA

FAR (48 CFR) 53.214©

K

Section B - Supplies Or Services And Prices/Costs

ITEM SUPPLIES / SERVICES ESTIMATED

COST

FIXED FEE TOTAL

EST. CPFF

0001 Program Eval Services - Base Year

Provide program evaluation services to assist the CDC's Division of HIV/AIDS Prevention/Capacity Building Branch for the Monitoring and Quality Control of the Capacity Building Assistance Program, as outlined in the Statement of Work.

Option 1 CBA Monitoring Items:

COST

FIXED FEE TOTAL

EST. CPFF

0002 Program Eval Services-Option Year I

Provide program evaluation services to assist the CDC's Division of HIV/AIDS Prevention/Capacity Building Branch for the Monitoring and Quality Control of the Capacity Building Assistance Program as outlined in the Statement of Work

Option 2 CBA Monitoring Items:

COST

FIXED FEE TOTAL

EST. CPFF

0003 Program Eval Services Option Year II

Provide program evaluation services to assist the CDC's Division of HIV/AIDS Prevention, Capacity Building Branch for the Monitoring and Quality Control of the Capacity Building Assistance (CBA) Program: Increasing Health Capacity to Provide Evidence-based Interventions and HIV Prevention Strategies.

Option 3 CBA Monitoring Items:

COST

FIXED FEE TOTAL

EST. CPFF

0004 Program Eval Services Option Year III

Monitoring and Quality Control of the Capacity Building Assistance (CBA) Program: Increasing Health Capacity to Provide Evidence-based Interventions and

Option 4 CBA Monitoring Items:

COST

FIXED FEE TOTAL

EST. CPFF

0005 Program Eval Services Option Year IV

Monitoring and Quality Control of the Capacity Building Assistance (CBA) Program: Increasing Health Capacity to Provide Evidence-based Interventions and

B.1. Type of Contract As stated in FAR, 52.216-1, Types of Contract, The Government contemplates award of a Cost Plus Fixed Fee contract resulting from this solicitation.

Section C - Description/Specification/Work Statement C.I. Background and Need

CDC’s Division of HIV/AIDS Prevention (DHAP) provides community-based organizations with more than $300 million annually to conduct HIV prevention activities. In addition, DHAP’s Capacity Building Branch (CBB) provides funds for Capacity Building Assistance (CBA) services in which considerable resources are allocated to serve community-based organizations (CBOs), health departments, and community planning groups nationwide.

The Capacity Building Branch provides national leadership and support for capacity building assistance to ensure that DHAP’s grantees have the knowledge, skills and technology to effectively and efficiently conduct HIV prevention across the United States and its territories. Capacity building assistance means the provision of free (not for fee) information, training, technical assistance (TA), and technology transfer to individuals, organizations and communities to improve their capacity in the delivery and effectiveness of evidence-based interventions and core public health strategies for HIV prevention. CBA services do not include the direct delivery of HIV prevention services.

CBA is provided to support grantees in the implementation, monitoring and evaluation of evidence-based HIV prevention interventions and programs; building organizational infrastructure; and community mobilization to decrease stigma and increase HIV testing in high risk communities. DHAP has funded a CBA program since 1999.

Prior to that, the program was not called CBA, it was called the National and Regional Minority Organizations (NRMO) Program. In all, DHAP has funded some form of capacity building strategy for DHAP grantees for approximately 12 years.

Even though CBA has been funded for over 10 years, there has been no systematic assessment of CBA services provided to CBO and Health Department (HD) customers. No monitoring process to determine whether CBA customers were satisfied with CBA services has been initiated. In addition, there has been no evaluation of whether CBA services result in improved HIV prevention practice. Since these data are lacking, the CBA providers do not have adequate feedback on the quality of their work and may continue on a course for 4 or 5 years with no customer feedback to inform CBA quality assurance and to direct CBA program improvement. The CBB, as the major leader in CBA, is in the position to conduct such an evaluation and use this information to improve capacity building efforts by CBB grantees.

This contract will require expertise in evaluating capacity building assistance for public health programs to include the following: design of evaluation, quality control tools, protocols, quantitative and qualitative data collection and analyses, and the use of the Statistical Package for Social Science (SPSS) and qualitative data analysis software.

C.2. Project Objective

The purpose of this contract is to provide program evaluation services to assist DHAP/CBB in the monitoring and evaluation of the CBA Cooperative Agreement Program which serves community-based organizations (CBOs) directly and indirectly funded by CDC, health departments and their community planning groups (CPGs), and other community stakeholders providing HIV prevention services targeting high risk and racial/ethnic minority populations to increase their capacity to implement evidence-based prevention practice. Program evaluation services will include program evaluation tool development, quantitative and qualitative data collection and analyses, and reports summarizing findings from the program evaluation activities.

The purpose of this program is to: 1) monitor and evaluate DHAP/CBB’s Capacity Building Assistance (CBA) Cooperative Agreement Program for community-based organizations (CBOs), health departments and their funded CBOs and community planning groups (CPGs) to increase their capacity to implement evidence-based HIV prevention practice; and 2) use the results to improve CBA services and thereby increase the capacity of CBOs and health departments funded by CDC to achieve the HIV prevention goals set forth in their CDC cooperative agreements.

In September 2009, CBB funded a new round of five-year cooperative agreements with 30 CBA providers.

Focusing on CBA services provided under the new cohort of CBA providers, this project would be the first comprehensive monitoring, evaluation, and quality control strategy to determine if DHAP customers are satisfied with CBA services and to determine if capacity has been developed as evidenced by prevention program outputs.

This CBA monitoring, evaluation, and quality control project seeks to answer the following primary monitoring, evaluation, and quality control questions: (1) How satisfied are CBB’s CBA customers with the CBA services that they receive?; (2) Do the customers of CBB, who receive CBA services, consider these services to be of benefit to them as they develop the capacity to implement evidence-based prevention practice?; (3) How can customer feedback be utilized to improve CBA services over time?; and (4) How do CBA services impact evidence-based HIV prevention practice?

C.3. Scope of Work

The contractor shall provide program evaluation services to assist the Capacity Building Branch in the monitoring, evaluation, and quality control of the Capacity Building Assistance (CBA) Cooperative Agreement Program which serves community-based organizations (CBOs) directly and indirectly funded by CDC, health departments and their community planning groups (CPGs), and other community stakeholders providing HIV prevention services targeting high risk and racial/ethnic minority populations to increase their capacity to implement evidence-based prevention practice.

The contractor shall provide program evaluation services to assist CBB in answering the following primary evaluation questions: (1) How satisfied are CBB’s CBA customers with the CBA services that they receive?; (2) Do the customers of CBB, who receive CBA services, consider these services to be of benefit to them as they develop the capacity to implement HIV evidence-based prevention practice?; (3) How can customer feedback be utilized to improve CBA services over time?; and (4) How do CBA services impact evidence-based prevention practice?

C.4. Technical Requirements

The contractor shall work independently and not as an agent of the Government to furnish all the necessary services, qualified personnel, material, and facilities to perform the required tasks, to complete the following tasks for the Base Year and each Option Year as indicated below:

C.4.1. BASE YEAR: September 10, 2010 through September 9, 2011

The components under the base year are as follows:

Across all components, the base year will focus on developing and refining the design of the project including data collection to inform the evaluation design, design of instruments, and piloting the data collection instruments.

Component 1 will focus on refining the consumer satisfaction evaluation design, conducting a literature review to refine the evaluation design, and piloting the data collection tools for consumer satisfaction.

Tasks and activities will include refining evaluation design and data collection tools, data extraction and data collection through web-based surveys, paper surveys, telephone interviews, and data analyses.

Component 2 will focus on developing the outcomes and impacts evaluation designs, conducting a literature review, convening an advisory group, and developing data collection instruments. Also, preparation and submission of materials for OMB clearance for the outcomes and impacts evaluation will be a priority during the base year.

In performance of the above components, the contractor shall:

Task 1.00 Orientation to the Project

Subtask 1.01: Review documents and materials about the CBA program, providers, and services as well as the designs for components 1 and 2 provided by the CDC Project Officer.

The CDC Project Officer will provide orientation information, which will include documents about the CBA program and its consumers, CBA Providers, CBA Request and Information System (CRIS), Training Events Calendar (TEC), Diffusion of Effective Behavioral Interventions (DEBI) Project, and other information as appropriate. The Funding Opportunity Announcement under which the CDC funded the CBA Cooperative Agreements and a copy of each funded CBA agency’s program plan will also be made available to the contractor by the CDC. In addition, the CDC will provide the contractor with information about the designs of Components 1 and

2. CDC Project Officer will provide draft data collection tools, such as facilitated discussion questions, post-training surveys, and web-based surveys.

Task 2.00 Refine Evaluation Design (Component 1)

Subtask 2.01: Review and refine the evaluation design and the detailed evaluation plan for Component 1. CDC Project Officer will provide draft evaluation design. The evaluation design includes collection of demographic data, pre/post comparison of consumer satisfaction, data entry, and data coding. Qualitative and quantitative data analyses will include statistical analyses such as t-tests, logistic regression, and analysis of variance to assess level of consumer satisfaction.

Subtask 2.02: Submit the finalized evaluation design and detailed evaluation plan.

Task 3.00 Literature Review of Consumer Satisfaction of CBA Services (Component 1)

Subtask 3.01: Conduct a literature review of consumer satisfaction with CBA services including published evaluations of consumer satisfaction with CBA or related services, indicators of consumer satisfaction, and evaluation design and methods for consumer satisfaction. The literature review will include no more than 100 articles published in peer reviewed journals in the US in English. The query will include Cochrane Library, PubMed, PsychINFO, PsychNet (APA), Google Scholar, and Ovid databases.

Subtask 3.02: Submit a report summarizing the findings of the literature review including citations and copies of articles.

Task 4.00 Consumer Satisfaction Surveys for Technical Assistance (TA) Requested Via CRIS (Component 1)

The TA consumer satisfaction data collection instruments will include: 1) the Participant Information Form (PIF);

and 2) the follow-up TA satisfaction surveys in a web-based and telephone format. Different surveys may be used for different types of consumers (e.g., community-based organizations, health departments) and different types of CBA services (e.g., organizational infrastructure, HIV prevention interventions and strategies, community mobilization). Approximately 500 customers will receive TA services each year. Both the web-based and telephone customer satisfaction surveys are totally voluntary and provide feedback to the CDC on the performance of CDC cooperative agreement grantees.

Participant Information Form (PIF): The PIF is self-administered paper survey that will be completed by TA consumers at the TA event. The CDC Project Officer will provide approximately 500 hard copies of the completed PIFs to the contractor within 60 days of the TA event. The contractor shall provide data entry and analyses of the PIF data.

Follow-up TA satisfaction Survey: CDC’s CBA Request Information System (CRIS) application will email the web-based, follow-up TA satisfaction survey to TA consumers, approximately 45 days after the TA event. If less than 80% respond to web-based surveys, the contractor shall administer telephone surveys with those TA customers who did not respond to the web-based, follow-up TA satisfaction survey. The contractor shall make attempts to increase the follow-up TA satisfaction survey response rate to a minimum of 80% each year.

The CDC Project Officer will provide to the contractor the web-based survey data in a format compatible with SPSS and ATLAS.ti software. In the base year of the project, data collection and analyses will be used to pilot and refine the data collection instruments and processes.

Subtask 4.01: Conduct telephone reminders to nonresponders of the web-based follow-up TA satisfaction survey and conduct telephone interviews using the follow-up TA satisfaction surveys to nonresponders, if the response to the web-based, follow-up TA satisfaction surveys falls below 80%. The contractor shall conduct approximately 350 telephone interviews to bring the response rate up to 80%. The Project Officer will provide a list of nonresponders with contact information to the contractor. The contractor shall make a maximum of 8 telephone calls to each TA customer.

The telephone reminders can be counted as one of the 8 calls. The telephone reminder can be used to remind nonresponders to complete the web-based survey; to schedule an interview; or to conduct the interview. One telephone reminder per nonresponder (training participant) is estimated for the contractor to conduct.

Subtask 4.02: Develop a draft plan to conduct data entry and quantitative and qualitative analyses of the TA consumer satisfaction data (from the PIF and the follow-up TA satisfaction survey) to satisfy the customer satisfaction monitoring questions provided by the CDC Project Officer. The contractor shall use quantitative and qualitative analysis methods accepted by the CDC Project Officer, SPSS (quantitative data analysis software), and ATLAS.ti (qualitative data analysis software). The plan may include using information about specific CBA requests from CRIS to understand levels of satisfaction (e.g., CBA action plans, post-CBA service information). The contractor shall finalize the plan with input from the CDC Project Officer.

Subtask 4.03: Implement the final data entry and analyses plan for the TA consumer satisfaction survey data.

Subtask 4.04: Provide a draft interim report summarizing the findings of the qualitative and quantitative analyses of the TA consumer satisfaction data and making recommendations for potential refinements to the data collection instruments and processes. The contractor shall submit an outline of the draft report to the CDC Project Officer for approval. The contractor shall finalize the report with input from the CDC Project Officer. This interim report will be used to refine the data collection instruments and for quality assurance purposes for the data collection and analyses.

Subtask 4:05: Refine the data collection instruments and processes to assess TA consumer satisfaction, as necessary, with input from the CDC Project Officer.

Task 5.00 Training Consumer Satisfaction (Component 1)

Training Consumer Satisfaction tools will include: 1) self-administered participant information form (PIF); 2) self-administered questionnaire at the end of a training event (i.e., post-training satisfaction questionnaire), and 2) follow-up training satisfaction survey in web-based and telephone formats. Different follow-up surveys may be used for different types of consumers (e.g., community-based organizations, health departments) and different types of trainings. The web-based and telephone customer satisfaction surveys are totally voluntary and provide feedback to the CDC on the performance of CDC cooperative agreement grantees.

Participant Information Form (PIF): The PIF is self-administered paper survey that will be completed by training consumers at the training event. There will be approximately 8,600 completed PIFs. The CDC Project Officer will send electronic summaries of 5,000 completed PIFs to the contractor within four weeks of each training event. The electronic PIF summaries will be in Microsoft Excel (quantitative data) and Word (qualitative data). The CDC Project Officer will provide remaining 3,600 completed PIFs as hard copies to the contractor for data entry within four weeks. The contractor shall provide quantitative and qualitative data analyses from the PIFs across trainings.

Post-Training Consumer Satisfaction Questionnaires: The post-training consumer satisfaction questionnaires will be administered at the end of the training. There will be approximately 8,600 completed post-training consumer satisfaction questionnaires. The CDC Project Officer will send electronic summaries of 5,000 completed post-training consumer satisfaction questionnaires to the contractor within four weeks of each training event. The electronic summaries will be in Microsoft Excel (quantitative data) and Word (qualitative data). The CDC Project Officer will provide remaining 3,600 completed post-training consumer satisfaction questionnaires as hard copies to the contractor for data entry within four weeks. The contractor shall provide quantitative and qualitative data analyses from the post-training consumer satisfaction questionnaires across trainings.

Follow-up Training Satisfaction Surveys: CDC’s CBA Request Information System (CRIS) application will email the web-based, follow-up training satisfaction survey to training consumers, approximately 90 days after each training event. If less than 80% respond to web-based surveys, the contractor shall administer the telephone version of the survey to those training customers who did not respond to the web-based, follow-up training satisfaction survey. The contractor shall make attempts to increase the follow-up training satisfaction survey response rate to a minimum of 80% of these training customers each year. The contractor shall conduct approximately 3,500 telephone interviews to bring the response rate up to 80%. The CDC Project Officer will provide to the contractor the web-based survey data in a format compatible with SPSS and other ATLAS.ti software. In the base year of the project, data collection and analyses will be used to pilot and refine the data collection instruments and processes.

Subtask 5.01: Conduct telephone reminders to nonresponders of the web-based follow-up training satisfaction survey and conduct telephone interviews using the telephone version of the surveys to nonresponders, if the response to the web-based surveys falls below 80%. The contractor shall conduct approximately 3,500 telephone interviews to bring the response rate up to 80%. The Project Officer will provide a list of nonresponders with contact information to the contractor. The contractor shall make a maximum of 8 telephone calls to each training customer.

The telephone reminders can be counted as one of the 8 calls. The telephone reminder can be used to remind nonresponders to complete the web-based survey; to schedule an interview; or to conduct the interview. One telephone reminder per nonresponder (training participant) is estimated for the contractor to conduct.

Subtask 5.02: Develop a draft plan to conduct data entry and quantitative and qualitative analyses of the PIF, post-training satisfaction, and follow-up training satisfaction data to satisfy the evaluation questions provided by the CDC Project Officer. The contractor shall use quantitative and qualitative analysis methods accepted by the CDC Project Officer , SPSS, and ATLAS.ti software. The contractor shall finalize the plan with input from the CDC Project Officer.

Subtask 5.03: Implement the final data entry and analyses plan for the consumer satisfaction survey data

Subtask 5.04: Provide a draft interim report summarizing the findings of the qualitative and quantitative analyses of the post-training satisfaction and follow-up training satisfaction data and making recommendations for potential refinements to the data collection instruments and processes. The contractor shall submit an outline of the draft report to the CDC Project Officer for acceptance. The contractor shall finalize the report with input from the CDC Project Officer. This interim report will be used to refine the data collection instruments and for quality assurance purposes for the data collection and analyses.

Subtask 5:05: Refine the data collection instruments and processes to assess training consumer satisfaction, as necessary, with input from the CDC Project Officer.

Task 6.00 CBA Provider Satisfaction Meeting (Component 1)

The CBA provider satisfaction meeting will be a facilitated discussion focusing on the CBA providers’ satisfaction with the CBA Program, guidance received from CDC, and collaborations with researchers, other CBA providers, and CDC/Capacity Building Branch Diffusion Teams. At least 2 staff from each CBA provider shall be in attendance, and they will pay for their own travel costs. There will be approximately 34 CBA provider organizations. This meeting will take place in Atlanta, Georgia. This meeting will be integrated into the meeting between the CDC and the CBA grantees at their annual grantee meeting.

Subtask 6.01: Develop draft plans to conduct the meeting with CBA providers during annual grantee meetings in which the CBA Providers will already be attending. The CDC Project Officer will provide discussion questions for the meeting. The contractor shall provide note taking of the meeting with sufficient detail for qualitative data analyses and logistics meeting including logistics (e.g., an agenda, space and audio/visual equipment, note taking, and provision of a facilitator, letters of invitation, attendee lists, etc.). The contractor shall finalize the plan for the CBA provider satisfaction meeting with input from the CDC Project Officer.

Subtask 6.02: Implement the final plan for CBA provider satisfaction meeting for each meeting.

Subtask 6.03: Provide qualitative analyses of the data collected at the meeting using qualitative analysis methods accepted by the CDC Project Officer and ATLAS.ti software.

Subtask 6.04: Provide one report for the meeting summarizing the findings of the qualitative analyses of the meeting data. The reports will be used for quality assurance purposes and to improve the CBA program and service delivery.

Task 7.00 Literature Review of Outcome and Impact Evaluation of CBA Services (Component 2)

Subtask 7.01: Conduct a literature review of outcome and impact evaluation of CBA services (technical assistance and training) including published evaluations of outcome and impact of CBA or related services, indicators of CBA outcome and impact, and evaluation design and methods for CBA outcome and impact. The literature review will include no more than 100 articles published in peer reviewed journals in the US in English. The query will include Cochrane Library, PubMed, PsychINFO, PsychNet (APA), Google Scholar, and Ovid databases.

Subtask 7.02: Provide a report summarizing the findings of the literature review, including citations and copies of articles.

Task 8.00 Convene an Advisory Group of Content Experts (Component 2)

Subtask 8.01: Identify 8-10 content experts in CBA outcome and impact evaluation to serve on an advisory group for Component 2. The contractor shall utilize the findings of the literature review to inform the identification of content experts for the advisory group. Content experts will have doctorate degrees in the behavioral sciences (e.g., anthropology; sociology; community, clinical, counseling, or social psychology; health education; health promotion) and at least one peer reviewed journal article in the measurement of agency or community capacity. The CDC Project Officer will make recommendations for content experts and their qualifications and areas of expertise.

Subtask 8.02: Develop a draft plan to conduct 1 Advisory Group meeting with 8-10 CBA outcome and impact evaluation content experts and the CDC Project Officer, including logistics (e.g., travel and lodging for Advisory Group members, agenda development, space and audio/visual equipment, note taking, and provision of a facilitator, letters of invitation, attendee lists, etc.). The contractor shall provide detailed note taking of the meetings proceedings. The contractor shall finalize the plan for the Advisory Group meeting with input from the CDC Project Officer.

Subtask 8.03: Implement the final plan for the Advisory Group meeting.

Subtask 8.04: Provide a summary of the meeting notes that will be used to assist in the development of the evaluation design of the outcomes and impacts of capacity building assistance provided by the CBA cooperative agreement grantees.

Subtask 8.05: Provide a report summarizing the meeting notes that will also include recommendations to the CDC on strategies to evaluate the outcome and impact of the capacity building assistance provided by the CBA cooperative agreement grantees.

Task 9.00 Develop Evaluation Design for Outcome Evaluation of CBA Services (Component 2)

Subtask 9.01: Develop a draft evaluation design to assess three overarching evaluation questions:

1) To what extent did CBA services (e.g., technical assistance and training) build the capacity of health departments and their CBOs to implement effective HIV prevention programs?

2) To what degree did CBA services (e.g., technical assistance and training) build the capacity of directly-funded CBOs to implement effective HIV prevention programs?

3) To what extent did CBA services diffuse community mobilization models and impact community mobilization to increase access to and utilization of HIV testing services?

The evaluation design will include but is not limited to refined evaluation questions, methodology, data sources, qualitative and quantitative data collection, and qualitative and quantitative data analyses. The contractor shall utilize the findings of the literature review and the Advisory Board meeting reports to inform the evaluation design.

The CDC Project Officer will provide information about possible existing CDC data sources (e.g., CRIS) that may be utilized in addition to new data collection. The outcome evaluation of the CBA program will use customers of CBA services as the primary source of information. This is not a contract to conduct research and therefore there is no comparison or control group condition. The focus of the outcome evaluation is to determine if actual customers of the CBA services have increased capacity to implement effective HIV prevention programs and increased capacity to mobilize communities to increase access to and utilization of HIV testing services. The agencies that will report as to whether their capacity has been increased are customers of the CBA program and their contact information and the type of capacity building assistance they requested and received should be integrated into the outcome evaluation design.

Subtask 9.02: Meet weekly and then as needed with the CDC Project Officer by telephone to incorporate CDC staff input into the evaluation design. The CDC Project Officer will provide guidance during the development of the evaluation design.

Subtask 9:03: Finalize the evaluation design with input from the CDC Project Officer.

Task 10.00 Develop Evaluation Design for Impact Evaluation of CBA Services (Component 2)

Subtask 10.01: Develop a draft evaluation design to assess three overarching evaluation questions:

1) To what extent did CBA services (e.g., technical assistance and training) impact health departments’ capacity building and support of their CBOs’ implementation of effective HIV prevention programs for men who have sex with men (MSM) at risk for acquiring or transmitting HIV infection?

2) To what extent did CBA services (e.g., technical assistance and training) impact CDC directly funded CBOs’ implementation of effective HIV prevention programs African American women at risk for acquiring or transmitting HIV infection?

3) To what extent did CBA services (e.g., community mobilization) impact access to and utilization of HIV testing services in communities that are disproportionately impacted by HIV disease and also receiving community mobilization CBA services?

The evaluation design will include but is not limited to refined evaluation questions, measurable indicators, methodology, data sources, and qualitative and quantitative data collection and analyses. The contractor shall utilize the findings of the literature review and the Advisory Board meeting reports to inform the evaluation design. The CDC Project Officer will provide information about possible existing CDC data sources (e.g., CRIS) that may be utilized in addition to new data collection. The three questions posed in the impact evaluation of CBA services are refinements of the three questions posed in the outcome evaluation of CBA services. The distinction for purposes of this contract is that outcome evaluation of the CBA program will focus on whether capacity has been increased for the customers of the CBA services whereas the impact evaluation of the CBA program will focus on whether that increased capacity improved prevention and HIV testing services for three populations identified by the CDC’s Division of HIV/AIDS prevention as priority populations for HIV prevention services. These include: all MSM but especially African American and Latino MSM and African American heterosexual women.

Subtask 10.02: Meet weekly and then as needed with the CDC Project Officer by telephone to incorporate CDC staff input into the evaluation design. The CDC Project Officer will provide guidance during the development of the evaluation design.

Subtask 10.03: Finalize the evaluation design with input from the CDC Project Officer.

Task 11.00 Develop Evaluation Tools for Outcome Evaluation of CBA Services (Component 2)

Subtask 11.01: Develop draft outcome evaluation and data collection tools for the outcome evaluation of CBA services. The CDC Project Officer will provide guidance in the development of these tools. The contractor shall focus the outcome evaluation on the customers of CBA services. The tools may be web-based surveys.

Subtask 11.02: Pilot and finalize outcome evaluation and data collection tools with input the CDC Project Officer.

Task 12.00 Develop Evaluation Tools for Impact Evaluation of CBA Services (Component 2)

Subtask 12.01: Develop draft evaluation tools for the impact evaluation of CBA services. The CDC Project Officer will provide guidance in the development of these tools.

Subtask 12.02: Pilot and finalize impact evaluation and data collection tools with input the CDC Project Officer.

Task 13.00 OMB Package for the Outcome and Impact Evaluation of CBA Services (Component 2)

Subtask 13.01: Assist the CDC Project Officer in the preparation and submission of the OMB package for the outcome and impact evaluation of CBA Services. The outcome and impact evaluations will not be implemented until OMB acceptance has been obtained. If there are follow-up questions from the OMB, the contractor shall assist the CDC Project Officer in preparing responses and re-submitting the OMB acceptance package.

C.4.2. OPTION 1:

Across all components, Option 1 will focus on data collection and periodic data analyses for quality control purposes and to provide information to improve the CBA program and CBA service delivery.

Component 1 will focus on assessing consumer satisfaction of the CDC Project Officers and collecting consumer satisfaction data using the evaluation design and tools developed in the base year. Also, a quality assurance plan for CDC staff and CBA grantees will be developed to use the consumer satisfaction data to improve the CBA program and service delivery. Tasks and activities will include planning and conducting focus groups, web-based surveys, telephone interviews; data entry and analyses; and the development of the quality assurance plan.

Component 2 will focus on implementing the outcomes and impact evaluation designs and using the data collection tools developed in the base year. Tasks and activities will include data collection, data analyses, and writing interim reports that summarize the findings from Option 1.

Task 1.00 Consumer Satisfaction Surveys for Technical Assistance (TA) Requested Via CRIS (Component 1)

The TA consumer satisfaction data collection instruments will include: 1) the Participant Information Form (PIF);

and 2) the follow-up TA satisfaction surveys in a web-based and telephone format. Different surveys may be used for different types of consumers (e.g., community-based organizations, health departments) and different types of CBA services (e.g., organizational infrastructure, HIV prevention interventions and strategies, community mobilization). Approximately 500 customers will receive TA services each year. Both the web-based and telephone customer satisfaction surveys are totally voluntary and provide feedback to the CDC on the performance of CDC cooperative agreement grantees.

Participant Information Form (PIF): The PIF is self-administered paper survey that will be completed by TA consumers at the TA event. The CDC Project Officer will provide approximately 500 hard copies of the completed PIFs to the contractor within 60 days of the TA event. The contractor shall provide data entry and analyses of the PIF data.

Follow-up TA satisfaction Survey: CDC’s CBA Request Information System (CRIS) application will email the web-based, follow-up TA satisfaction survey to TA consumers, approximately 45 days after the TA event. If less than 80% respond to web-based surveys, the contractor shall administer telephone surveys with those TA customers who did not respond to the web-based, follow-up TA satisfaction survey. The contractor shall make attempts to increase the follow-up TA satisfaction survey response rate to a minimum of 80% each year.

The CDC Project Officer will provide to the contractor the web-based survey data in a format compatible with SPSS and ATLAS.ti software.

Subtask 1.01: Conduct telephone reminders to nonresponders of the web-based follow-up TA satisfaction survey and conduct telephone interviews using the follow-up TA satisfaction surveys to nonresponders, if the response to the web-based, follow-up TA satisfaction surveys falls below 80%. The contractor shall conduct telephone interviews to bring the response rate up to 80%. The Project Officer will provide a list of nonresponders with contact information to the contractor. The contractor shall make a maximum of 8 telephone calls to each TA customer.

The telephone reminders can be counted as one of the 8 calls. The telephone reminder can be used to remind nonresponders to complete the web-based survey; to schedule an interview; or to conduct the interview. One telephone reminder per nonresponder (training participant) is estimated for the contractor to conduct.

Subtask 1.02: Implement the final data entry and analyses plan (developed in the base year) for the TA consumer satisfaction survey data (from the PIF and the follow-up TA satisfaction survey).

Subtask 1.03: Provide a draft interim report summarizing the findings of the qualitative and quantitative analyses of the TA consumer satisfaction data for each year. The contractor shall submit an outline of the draft report to the CDC Project Officer for acceptance. The contractor shall finalize the report with input from the CDC Project Officer. This interim reports will be used for quality assurance purposes and to improve the CBA program and service delivery.

Task 2.00 Training Consumer Satisfaction (Component 1) Training Consumer Satisfaction tools will include: 1) self-administered participant information form (PIF); 2) self-administered questionnaire at the end of a training event (i.e., post-training satisfaction questionnaire), and 2) follow-up training satisfaction survey in web-based and telephone formats. Different follow-up surveys may be used for different types of consumers (e.g., community-based organizations, health departments) and different types of trainings. The web-based and telephone customer satisfaction surveys are totally voluntary and provide feedback to the CDC on the performance of CDC cooperative agreement grantees.

Participant Information Form (PIF): The PIF is self-administered paper survey that will be completed by training consumers at the training event. There will be approximately 8,600 completed PIFs. The CDC Project Officer will send electronic summaries of 5,000 completed PIFs to the contractor within four weeks of each training event. The electronic PIF summaries will be in Microsoft Excel (quantitative data) and Word (qualitative data). The CDC Project Officer will provide remaining 3,600 completed PIFs as hard copies to the contractor for data entry within four weeks. The contractor shall provide quantitative and qualitative data analyses from the PIFs across trainings.

Post-Training Consumer Satisfaction Questionnaires: The post-training consumer satisfaction questionnaires will be administered at the end of the training. There will be approximately 8,600 completed post-training consumer satisfaction questionnaires. The CDC Project Officer will send electronic summaries of 5,000 completed post-training consumer satisfaction questionnaires to the contractor within four weeks of each training event. The electronic summaries will be in Microsoft Excel (quantitative data) and Word (qualitative data). The CDC Project Officer will provide remaining 3,600 completed post-training consumer satisfaction questionnaires as hard copies to the contractor for data entry within four weeks of the training event. The contractor shall provide quantitative and qualitative data analyses from the post-training consumer satisfaction questionnaires across trainings.

Follow-up Training Satisfaction Surveys: CDC’s CBA Request Information System (CRIS) application will email the web-based, follow-up training satisfaction survey to training consumers, approximately 90 days after each training event. If less than 80% respond to web-based surveys, the contractor shall administer the telephone version of the survey to those training customers who did not respond to the web-based, follow-up training satisfaction survey. The contractor shall make attempts to increase the follow-up training satisfaction survey response rate to a minimum of 80% of these training customers each year. The contractor shall conduct approximately 3,500 telephone interviews to bring the response rate up to 80%. The CDC Project Officer will provide to the contractor the web-based survey data in a format compatible with SPSS and other ATLAS.ti software.

Subtask 2.01: Conduct telephone reminders to nonresponders of the web-based follow-up training satisfaction survey and conduct telephone interviews using the telephone version of the surveys to nonresponders, if the response to the web-based surveys falls below 80%. The contractor shall conduct approximately 3500 telephone interviews to bring the response rate up to 80%. The Project Officer will provide a list of nonresponders with contact information to the contractor. The contractor shall make a maximum of 8 telephone calls to each training customer.

The telephone reminders can be counted as one of the 8 calls. The telephone reminder can be used to remind nonresponders to complete the web-based survey; to schedule an interview; or to conduct the interview. One telephone reminder per nonresponder (training participant) is estimated for the contractor to conduct.

Subtask 2.02: Implement the final data entry and analyses plan (developed during the base year) for the training consumer satisfaction data.

Subtask 2.03: Provide a draft interim report summarizing the findings of the qualitative and quantitative analyses of the post-training satisfaction and follow-up training satisfaction data. The contractor shall submit an outline of the draft report to the CDC Project Officer for acceptance. The contractor shall finalize the report with input from the CDC Project Officer. This interim reports will be used for quality assurance purposes and to improve the CBA program and service delivery.

Task 3.00 CDC Project Officer Focus Groups (Component 1)

Subtask 3.01: Develop draft plans to conduct 2 focus groups to assess CDC Prevention Program Branch (PPB) Project Officer satisfaction with the CBA program and CBA services (i.e., individual CBA and trainings). There are approximately 30 PPB Project Officers who will be divided into 10-15 people per focus group. The contractor shall provide note taking of the focus groups with sufficient detail for qualitative data analyses and logistics (e.g., an agenda, letters of invitation, attendee lists, etc.). Focus groups will be held at a CDC campus in Atlanta. The contractor shall finalize the plans for the CDC PPB Project Officer focus groups with input from the CDC Project Officer. The CDC Project Officer will provide evaluation questions to guide the plans and focus group facilitators.

Subtask 3.02: Refine and pilot questions for the CDC PPB Project Officer focus groups with input from the CDC Project Officer. CDC Project Officer will provide draft questions for the focus groups.

Subtask 3.03: Implement the final plan for the CDC PPB Project Officer focus groups.

Subtask 3.04: Provide qualitative analyses of the data collected at the focus groups using qualitative analysis methods accepted by the CDC Project Officer and ATLAS.ti software.

Subtask 3.05: Provide a report summarizing the findings of the qualitative analyses of the focus group data. The reports will be used for quality assurance purposes and to improve the CBA program and service delivery.

Task 4.00 Quality Assurance Plan for CDC staff and CBA Providers (Component 1)

Subtask 4.01: Develop a draft quality assurance plan whereby consumer satisfaction information gathered from the evaluation will be shared with the CDC Program Consultants, who oversee the CBA Providers; the CBA providers;

and the Capacity Building Branch partners to improve CBA services. The contractor shall utilize information from the interim reports from Components 1 and 2 to identify quality assurance issues and make recommendations for resolving the quality assurance issues and improving CBA services. The CDC Project Officer will provide input and guidance for the plan.

Subtask 4.02: Provide finalized the quality assurance plans based on the feedback from the CDC Project Officer.

Task 5.00 Data Collection for Outcome Evaluation of CBA Services (Component 2)

Subtask 5.01: Implement the final plan to collect quantitative and qualitative data as outlined in the Outcome Evaluation of CBA services evaluation design (developed in the base year).

Task 6.00 Data Collection for Impact Evaluation of CBA Services (Component 2)

Subtask 6.01: Implement the final plan to collect quantitative and qualitative data as outlined in the Impact Evaluation of CBA services evaluation design (developed in the base year).

Task 7.00 Data Analyses for Outcome Evaluation of CBA Services (Component 2)

Subtask 7.01: Finalize the draft plan to analyze the quantitative and qualitative data as outlined in the Outcome Evaluation of CBA services evaluation design (developed in the base year), based on preliminary data collection, if necessary. The contractor shall finalize the plan with input from the CDC Project Officer.

Subtask 7.02: Implement the final plan to analyze the quantitative and qualitative data for the Outcome Evaluation of CBA services.

Subtask 7.03: Provide a draft interim report summarizing the findings of the qualitative and quantitative analyses of the Outcome Evaluation of CBA services data. The contractor shall submit an outline of the report to the CDC Project Officer for acceptance. The contractor shall finalize the report with input from the CDC Project Officer.

The interim reports will be used for quality assurance purposes and to improve the CBA program and service delivery.

Task 8.00 Data Analyses for Impact Evaluation of CBA Services (Component 2)

Subtask 8.01: Finalize the draft plan to analyze the quantitative and qualitative data as outlined in the Impact Evaluation of CBA services evaluation design (developed in the base year), based on the preliminary data collection, if necessary. The contractor shall finalize the plan with input from the CDC Project Officer.

Subtask 8.02: Implement the final plan to analyze the quantitative and qualitative data for the Impact Evaluation of CBA services data.

Subtask 8.03: Provide a draft interim report summarizing the findings of the qualitative and quantitative analyses of the Impact Evaluation of CBA services data.