Performance_Work_Statement_FNS_Pilot_6-18-2015.pdf

Attachment I

PERFORMANCE WORK STATEMENT

EVALUATION OF THE USDA PILOT PROJECT FOR THE PROCUREMENT OF

UNPROCESSED FRUITS AND VEGETABLES

1.0 INTRODUCTION

The U.S. Department of Agriculture (USDA), Food and Nutrition Service (FNS) plans to conduct a study to evaluate the impacts of the USDA Pilot Project for the Procurement of

Unprocessed Fruits and Vegetables, conducted in cooperation with USDA’s Agricultural

Marketing Service (AMS). The study is designed to collect data from the eight (8) State agencies participating in the pilot program in each school year. It is the intent to issue a contract for two years of data collection (School Year 2015-16 and School Year 2016-17) with optional data collection to occur the following two school years, if the pilot project continues. The study will provide quantitative and qualitative information on the impact of the pilot program on the following: (1) quantity and cost of each type of fruit and vegetable purchased in pilot States; (2) the pilot States’ unprocessed fruit and vegetable procurement operations and States’ perceptions of the benefits and challenges of those procurements.

2.0 BACKGROUND

The following section provides background information including a brief overview of the Food

Distribution (FD) Programs and The USDA Pilot Project for the Procurement of Unprocessed

Fruit and Vegetables.

Food Distribution Programs

The Food Distribution Division (FDD) of USDA’s Food and Nutrition Service (FNS) coordinates the distribution of USDA Foods available to more than 94,000 public and private nonprofit school food authorities (SFAs) that provide meals to students. School food authorities

(SFAs) participating in the National School Lunch Program receive “entitlement” commodities for lunches served. School food authorities (SFAs) also can receive “bonus” commodities, as they are available through USDA’s price support and surplus removal programs. School food authorities (SFAs) do not receive a separate entitlement for USDA Foods served in the School

Breakfast Program, however school food authorities (SFAs) often serve USDA Foods in the

School Breakfast Program.

A link to the FD Programs homepage is provided below:

Food Distribution Division: http://www.fns.usda.gov/fdd/food-distribution-programs

The Pilot Project for the Procurement of Unprocessed Fruits and Vegetables

As required by Section 4202 of the Agricultural Act of 2014 (the Farm Bill) the U.S. Department of Agriculture’s (USDA’s) Agricultural Marketing Service (AMS) and Food and Nutrition

Service (FNS) are conducting a pilot project in eight States to provide them flexibility for using http://www.fns.usda.gov/fdd/food-distribution-programs

USDA Foods entitlement dollars for the procurement of unprocessed fruits and vegetables

(including products that are fresh cut, such as sliced apples, baby carrots, and shredded lettuce) by permitting each State to: (1) use multiple suppliers and products established and qualified by the Secretary of Agriculture; and (2) allow geographic preference, if desired, in the procurement of the products under this pilot.

The Farm Bill required that the selection of pilot States be based on demonstrated commitment and increased effort of the identified State’s farm to school program, the quantity and variety of growers of local fruits and vegetables in the State, and the degree to which the State contains a sufficient quantity of local educational agencies of various population sizes and geographic locations. At least one pilot project must take place in each of the following regions: 1) the

Pacific Northwest Region; 2) the Northeast Region; 3) the Western Region; 4) the Midwest

Region; and 4) the Southern Region. The eight States that have been selected to participate in this pilot program are California, Connecticut, Michigan, New York, Oregon, Virginia, Washington, and Wisconsin. Pilot States were selected in December 2014 and States began receiving deliveries in March 2015.

Statutory Authority

Section 4202 of the Agricultural Act of 2014 (P.L. 113-79, the Farm Bill) amended Section 6 of the Richard B. Russell National School Lunch Act (42 U.S.C. 1755(f)) to provide for the USDA

Pilot Project for the Procurement of Unprocessed Fruit and Vegetables.

AMS communicates information about its procurement activities on its website and through the

AMS CP News email listserv. To subscribe to AMS CP News, visit www.ams.usda.gov/commoditypurchasing.

A link to additional information about the Pilot Program on the FNS and AMS webpages, respectively, are provided below:

http://www.fns.usda.gov/fdd/pilot-project-procurement-unprocessed-fruits-and-vegetables http://www.ams.usda.gov/AMSv1.0/ams.fetchTemplateData.do?template=TemplateN&page=CP

DPilotProjectUnprocessedFV.

3.0 SCOPE OF WORK

The contractor shall conduct a study to evaluate the impacts of the USDA Pilot Project for the

Procurement of Unprocessed Fruits and Vegetables, conducted in cooperation with USDA’s

Agricultural Marketing Service (AMS). The study is designed to collect data from the eight (8)

State agencies participating in the pilot program in each school year. It is the intent to issue a contract for two years of data collection (School Year 2015-16 and School Year 2016-17) with optional data collection to occur the following two school years, if the pilot project continues.

The study will provide quantitative and qualitative information on the impact of the pilot program on the following: (1) quantity and cost of each type of fruit and vegetable purchased in pilot States; (2) the pilot States’ unprocessed fruit and vegetable procurement operations and

States’ perceptions of the benefits and challenges of those procurements http://www.ams.usda.gov/commoditypurchasing http://www.fns.usda.gov/fdd/pilot-project-procurement-unprocessed-fruits-and-vegetables http://www.ams.usda.gov/AMSv1.0/ams.fetchTemplateData.do?template=TemplateN&page=CPDPilotProjectUnprocessedFV http://www.ams.usda.gov/AMSv1.0/ams.fetchTemplateData.do?template=TemplateN&page=CPDPilotProjectUnprocessedFV

PART A -- STUDY OBJECTIVES AND RESEARCH QUESTIONS

The goal of the USDA Pilot Project for the Procurement of Unprocessed Fruit and Vegetables is to develop additional opportunities for school food authorities (SFAs) to purchase unprocessed fruits and vegetables with USDA Foods entitlement funding, while using pre-existing commercial distribution channels and school relationships with growers, produce wholesalers, and distributors. The pilot supports the use of locally-grown foods in school meal programs using entitlement funds. Please note that the DoD Fresh program, under which USDA partners with

Department of Defense’s Defense Logistics Agency (DoD-DLA) to purchase and to deliver fresh fruit and vegetables to school food authorities (SFAs) and other nutrition programs, remains available to eligible recipients.

The primary objective of this study is to assess the challenges and benefits presented by Section

4202 of the Agricultural Act of 2014 (the Farm Bill). The legislation requires that:

“Each participating State shall submit to the Secretary a report on the success of the pilot project in the State, including information on —

(i) the quantity and cost of each type of fruit and vegetable received by the State under the pilot project; and

(ii) the benefit provided by those procurements in conducting school food service in the

State, including meeting school meal requirements.’’

FNS has two major objectives for the study:

To provide data on the impact of the pilot project on the quantity and cost of each type of fruit and vegetable purchased in pilot States;

To provide data on the impact of the pilot on States’ unprocessed fruit and vegetable procurement operations and their perceptions of the benefits or challenges of those procurements.

These broad objectives include, but are not limited to, the following research questions. The research questions are categorized by priorities as described in the legislation. Within these categories, research questions are divided into those for which data to answer the question currently exist and those for which primary data must be collected.

The contractor must possess a solid understanding of the legislation and regulations related to

FNS Food Distribution programs, current program operations, and recently released policies and guidance material. In addition, the contractor should be familiar with recent research conducted on these programs. Links to these various elements of the Food Distribution Programs and associated research can be found on the FNS website (http://www.fns.usda.gov).

The study will require the successful contractor to conduct primary data collection and to utilize a mix of extant data sources. Historical data owned by FNS that could be provided to the contractor may include:

http://www.fns.usda.gov/ the value and quantity of all fruit and vegetable products and those tagged as local purchased through the USDA Foods and DoD Fresh programs by State for each school year;

total entitlement dollars spent through the DoD Fresh and USDA Foods programs by

State for each school year;

Descriptive data on farm to school efforts for all school districts for each school year from the USDA Farm to School Census (http://www.fns.usda.gov/farmtoschool/census#/)

Data showing where (by State) USDA Foods products were processed and packed

(http://www.fns.usda.gov/fdd/food-purchase-resources). This information in combination with a participating State’s order information would allow an analysis of the unprocessed products ordered through USDA Foods from that State and would assist in deciphering local foods availability through USDA Foods.

Contractors should submit comprehensive proposals that detail how existing data sources and primary data collection will be utilized to address each of the study’s objectives and research questions. These proposals must address in detail the study design, methodology, analysis strategy and reporting structure that will provide FNS with the most accurate, actionable data and guidance on the impact of the pilot program.

Objective Research Question Existing Data

Source

Variable of Interest

Status

Measure cost of each type of fruit and veg. received by each State in pilot.

How much did each State spend on unprocessed fruits and vegetables, by specific type of produce, received through this pilot in SY 2014-2015? In SY

2015-16?

Contractor SDA

Reporting

Template

State fruit and veg.

spending by produce type

To be collected by AMS;

provided to contractor by

FNS

Measure cost of each type of fruit and veg. received by each State in pilot.

How much did each State spend on unprocessed fruits and vegetables, by specific type of produce, procured before this pilot in SY 2013-14 through

USDA Foods?

Web Based Supply

Chain

Management

System (WBSCM)

State fruit and veg.

USDA

Foods spending by produce type

Existing (for multiple prior years);

provided to contractor by

FNS

Measure cost of each type of fruit and veg. received by each State in pilot.

How much did each State spend on unprocessed fruits and vegetables, by specific type of produce, procured before this pilot in SY 2013-14 through

DoD Fresh?

Fresh Fruits and

Vegetables Order

Receipt System

(FFAVORS)

State DoD

Fresh spending by produce type

Existing (for multiple prior years);

provided to contractor by

FNS

http://www.fns.usda.gov/farmtoschool/census#/ http://www.fns.usda.gov/fdd/food-purchase-resources

Variable of Interest

Status

Measure quantity of each type of

FV received by each State in pilot.

What quantities of each type of produce were received by each State through this pilot in SY

2014-15? In SY 2015-16?

Contractor SDA

Reporting

Template

Number of cases or pounds purchased by produce type by

State

To be collected by AMS;

provided to contractor by

FNS

Measure quantity of each type of fruit and veg.

received by each

State in pilot.

What quantities of each type of produce were received by each State before the pilot in SY

2013-14 through USDA

Foods?

WBSCM Number of cases or pounds purchased by produce type by

State

Existing (for multiple prior years);

provided to contractor by

FNS

Measure quantity of each type of fruit and veg.

received by each

State in pilot.

What quantities of each type of produce were received by each State before the pilot in SY

2013-14 through DoD

Fresh?

FFAVORS Number of cases or pounds purchased by produce type by

State

Existing (for multiple prior years);

provided to contractor by

FNS

Measure impact of pilot on conducting school food service in each State, including implementing school meal requirements.

What are the 8 State agencies’ overall perspectives on the pilot in each school year? On specific processes?

No existing source Opinions of pilot by

State

To be collected by contractor

Measure impact of pilot on conducting school food service in each State, including implementing school meal requirements.

Does this pilot impact State agencies’ ability to utilize

USDA Foods entitlement funding? How? What are the States’ perceptions of the pilot’s impact on the share/allocation of USDA

Foods entitlement funding among SFAs? What are the

States’ perceptions of this pilot’s impact on SFA’s ability to obtain readily usable products?

WBSCM; no existing source for qualitative

Percentage of entitlement dollars spent by

State;

Opinions of pilot by

State

For quantitative data, existing

(for multiple prior years);

provided to contractor by

FNS;

qualitative data to be collected by contractor

Variable of Interest

Status

Measure impact of pilot on conducting school food service in each State, including implementing

Does this pilot impact

States’ ability to implement the meal pattern in each school year? How?

No existing source List of qualitative impacts, detailed description of impacts and prevalence by State

To be collected by contractor

Measure impact of pilot on conducting school food service in each State, including implementing

What challenges and/or benefits did States encounter in each school year?

No existing source List of challenges and prevalence by State

To be collected by contractor

Gather data on

State implementation/ operation and perceptions of pilot.

What are the reasons why

State agencies participated in this pilot? Why did

States apply?

No existing source List of reasons by

State

To be collected by contractor

Gather data on

State implementation/ operation and perceptions of pilot.

What are States’ perceptions of USDA

Foods and DoD Fresh in providing access to local fruits and vegetables in each school year?

No existing source Opinions of programs by State

To be collected by contractor

Gather data on

State implementation/ operation and perceptions of pilot.

What additional technical assistance do States require as part of this pilot? From

HQ? From Regions?

No existing source List of needs by

State

To be collected

Variable of Interest

Status

Gather data on

State implementation/ operation and perceptions of pilot.

Did each Pilot State and

District (all, some, no districts within State) spend all of its USDA

Foods entitlement in

School Year 2013-2014? If not, why? Has the pilot impacted entitlement spending for SY 2014-15?

SY 2015-16? How? How does each State track entitlement funding?

WBSCM for State entitlement percent; State data for District percentage; no existing source for qualitative

Percent of entitlement spent by

State and

District;

perceptions of impact by State

WBSCM to be provided to contractor by

FNS; State quantitative and qualitative to be collected by contractor

Gather data on

State implementation/ operation and perceptions of pilot.

Does each State (all, some, no districts within State) receive all of the USDA

Foods products it would like to receive in each school year? If not, why?

Has the pilot impacted this? How?

No existing source Perceptions of impact by State

To be collected by contractor

Gather data on

State implementation/ operation and perceptions of pilot.

Does each State (all, some, no districts within State) use geographic preference option in order to purchase unprocessed foods that have been locally grown, raised, or produced in each school year? If so, how is geographic preference used and for what products? Has the pilot impacted this?

How?

No existing source Description of practices by State

To be collected by contractor

Gather data on

State implementation/ operation and perceptions of pilot.

How does each State define local as it relates to their procurement of unprocessed fruits and vegetables? Please describe in general terms how this definition is used in your

State before and during the pilot.

Farm to School

Census for SFA-level definitions;

No existing source for State-level perceptions

Description of practices by State

To be collected

Variable of Interest

Status

Gather data on

State implementation/ operation and perceptions of pilot.

Please describe barriers to procuring local products or reasons why your State (all, some, no districts within

State) does not purchase local unprocessed fruits and vegetables in each school year? Has the pilot impacted this? How?

Farm to School

Census for SFA-level barriers; No existing source for

State-level perceptions

Lists of barriers by

State

To be collected by contractor

Gather data on

State implementation/ operation and perceptions of pilot.

How does your State (all districts within State) plan to procure unprocessed fruits and vegetables under the pilot in each school year?

No existing source List of plans by

State

To be collected by contractor

Gather data on

State implementation/ operation and perceptions of pilot.

What are States’ perceptions of any changes in unprocessed fruit and veg. purchases from within participating States (intra-

State purchases) before and during the pilot? What is the nature of those changes? Do States have quantitative data on intra-

State unprocessed fruit and veg. purchases before and during the pilot? If so, please provide these data.

No existing source Opinions by State;

any available

State-level quantitative data on intra-State unprocesse d fruit and veg.

purchases

To be collected by contractor

Gather data on

State implementation/ operation and perceptions of pilot.

What are the number and characteristics of contractors participating in this pilot in each school year? (This will be compared with current

USDA Foods and DoD

Fresh contractors.)

AMS database Counts and percentages of contractors by State of product origin and

State of product delivery;

Contractor size

To be collected by AMS;

provided to contractor by

FNS

Pilot Contractor Qualifications

AMS will maintain a list of “eligible contractors” from which States and school food authorities

(SFAs) may procure products under the pilot project. This list is updated on an ongoing basis, and contractorcontractor may apply for placement on the list at any time. To become eligible, interested contractors must provide evidence to AMS that minimum requirements have been met.

Contractors should thoroughly review the AMS Pilot Project website:

(http://www.ams.usda.gov/AMSv1.0/ams.fetchTemplateData.do?template=TemplateN&page=C

PDPilotProjectUnprocessedFV ) and maintain current knowledge of contractor requirements.

Procurement/Delivery

States or school food authorities (SFAs) participating in the pilot project may contract with one of the eligible contractors through a competitive procurement process and utilize entitlement funds for the value of the unprocessed fruits and vegetables received.

Payment Process

The reports submitted by participating contractors will be processed by AMS for payment. FNS will receive these spreadsheets from AMS and FNS will subsequently provide these spreadsheets to the successful contractor. However, AMS or FNS will not conduct any data cleaning or analysis and FNS will submit the spreadsheets to the Contractor as received.

Reporting Requirement

States selected for the pilot project will keep records of product deliveries from eligible contractors. Contractors will submit invoices to AMS for products delivered under the pilot and

States will reconcile these deliveries on a monthly basis. Below is the template that contractors and States use for that monthly report.

Contractor/SDA Reporting Template: Pilot Project for Procurement of Unprocessed

Fruits and Vegetables

Name:

Contact:

Email Address:

Phone Number:

Contractor

Name

Recipient

SFA

City Product Description

Product

State of

Origin

Quantit y

Total

Cost

Delivery

Date

Contractor

Y School X

City

Y

Lettuce Green

Boston 24 Ct State Z 15 $196.80 4/15/2015 http://www.ams.usda.gov/AMSv1.0/ams.fetchTemplateData.do?template=TemplateN&page=CPDPilotProjectUnprocessedFV http://www.ams.usda.gov/AMSv1.0/ams.fetchTemplateData.do?template=TemplateN&page=CPDPilotProjectUnprocessedFV

As AMS receives these completed invoices from contractors monthly, AMS staff will provide the data that have been collected over the course of the entire school year to FNS for State review and reconciliation. FNS will provide these data to the selected contractor in the fall of each year in which the pilot is conducted (fall 2015, 2016; if the pilot continues, fall 2017, 2018). With a maximum of eight (8) States participating in the pilot in any school year, FNS anticipates that the selected contractor will receive approximately eight (8) spreadsheets for each school year, in

Microsoft Excel format. Each Excel workbook will include all annual data for one State and will include a summary tab and tabs for each month.

USDA possesses historical data on States’ fruit and vegetable purchases made prior to the pilot through USDA’s Food Distribution Programs. These data are stored in a number of USDA’s proprietary databases including the Web Based Supply Chain Management System (WBSCM), used for USDA Foods purchases and the Fresh Fruits and Vegetables Order Receipt System

(FFAVORS), used for DoD Fresh purchases. These historical data from WBSCM and

FFAVORS will also be provided to the selected contractor, for use in providing comparisons of quantities, costs, variety of products, and variety and number of contractors for fruit and vegetable procurements before and after the pilot’s implementation. The successful contractor will provide a detailed proposal outlining the ways in which these data may be utilized to provide

FNS with accurate estimates of the pilot project’s impact on the quantity and cost of fruits and vegetables purchased in the pilot States.

Examples of the data sets that will be provided to the successful contractor are presented in

Appendices as attachments to this Statement of Work.

PART B: STUDY DESIGN AND METHODOLOGY

Contractors shall propose a design and methodology that address the study objectives and will answer the research questions, above. The following instructions and constraints are provided to enable Contractors to propose the best design and methodology:

This study will be limited to the maximum of eight (8) State agencies that are participating in the Pilot Project for the Procurement of Unprocessed Fruit and

Vegetables in each school year. Fewer than 8 States may participate at any given time.

Only authorized staff at these State agencies should be contacted for participation.

Because the study is limited to the eight (8) State agencies participating in the pilot in each school year, OMB clearance will not be required.

Contractors shall specify their approach(s) to gaining cooperation with State agencies.

In the fall of each study year, FNS shall provide the successful contractor with the following raw, uncleaned quantitative data:

o Approximately eight (8) completed Contractor/SDA templates outlining the dollar value, quantity and description of all fruit and vegetable purchases made through the pilot which have been collected from all pilot States over the course of the entire school year. Each of the eight Excel spreadsheets (1 per State) containing completed contractor/SDA templates will be separated into 12 tabs, one tab for deliveries in each month of the year. Each spreadsheet contains data collated by delivery. That is, each template outlines the dollar value, quantity, and description of all fruit and vegetable purchases made through the pilot which have been collected from one pilot State over the course of the entire school year. The data must be aggregated to perform analysis. No unit price or standard unit of measure is provided in the template. Product descriptions can vary by contractor and no item numbers are supplied. The contractor shall be prepared to properly analyze all data, including data that vary based on pilot implementation. For example, the number of SFA’s in each State in the pilot currently ranges from 15 – 58.

Moreover, the deliveries to these participating SFA’s are from local contractors and, therefore, may reflect seasonal changes in timing, quantity and selection;

o The dollar value and quantity of all fruit and vegetable products and those tagged as local purchased through the USDA Foods and DoD Fresh programs by State for each school year;

o Total entitlement dollars spent through the DoD Fresh and USDA Foods programs by State for each school year (namely SY 2014-15, SY 2015-16 and SY

2016-17; if the pilot continues, then SY 2017-18 and SY 2018-19);

o Descriptive data on farm to school efforts for all school districts for each school year from the USDA Farm to School Census

(http://www.fns.usda.gov/farmtoschool/census#/ ) o Data showing where (by State) USDA Foods products were processed and packed

(http://www.fns.usda.gov/fdd/food-purchase-resources ). This information in combination with a participating State’s order information would allow an analysis of the unprocessed products ordered through USDA Foods from that

State and would assist in deciphering local foods availability through USDA

Foods.

o

Contractors shall clean and organize these raw data and compile analytic files suitable for statistical analysis. The contractor shall conduct any necessary technical assistance with

States to verify ambiguous responses.

Contractors shall specify a design and methodologies for collecting primary qualitative data from the eight (8) State agencies that are participating in the Pilot in each school year. Using State-of-science qualitative methodologies, which may include, for example, in-depth interviews, focus groups, and State administrative record review, the proposed design must fully address all qualitative research objectives and questions outlined above.

Contractors shall detail methods to utilize technological advances in qualitative data collection that maximize quality and minimize cost, for example, web-based data collection methods and qualitative analysis software programs (NVivo is the qualitative analysis package requested by FNS).

Contractors shall discuss their approach and plans for collecting and integrating all data sources used in the study. For the study’s required primary, qualitative data collection, contractors shall specify an approximate schedule of data collection points, bearing in mind school-year calendars, the pilot’s implementation schedule and operational structure, and the need to deliver a final report acceptable to FNS by the deadline in the http://www.fns.usda.gov/farmtoschool/census#/ http://www.fns.usda.gov/fdd/food-purchase-resources deliverable schedule. For the study’s required use of extant, quantitative data, contractors shall specify their approach and approximate schedule for integrating data received from

FNS with the primary data collected by the contractor, bearing in mind school-year calendars, the pilot’s implementation schedule and operational structure, and the need to deliver a final report acceptable to FNS by the deadline in the deliverable schedule.

Contractors shall discuss their approach and plans for analyzing all the data sources used in the study. Contractors shall discuss both quantitative (descriptive and inferential statistics) and qualitative analyses (e.g., content analyses of interviews, focus groups, administrative record review, etc.) to be included in their approach, including a detailed explanation of the steps, rationale and any assumptions underlying all analyses.

Contractors shall specify variables they will use in their analysis to help explain the outcomes (e.g., quantity and cost of unprocessed fruit and vegetable purchases before and in pilot, States’ perceptions of pilot, pilot’s impact on school food service/procurement operations and meeting school meal requirements, etc.) and discuss why these variables were chosen and how they might be relevant. Contractors shall discuss their procedures for successfully integrating and utilizing the relevant, extant State-level, aggregate, and administrative data with primary data collected under their approach and how these will be integrated into their analyses.

Contractors shall specify the minimum detectable difference (at 95% confidence, 80% power) their proposed approach will obtain for all quantitative estimates of the pilot’s impact.

Contractors shall specify a timeline for the study that satisfies all requirements. For illustration purposes only, FNS has provided a Gantt-chart of a possible timeframe in

Appendix A.

See both instructions to contractors and the evaluation criteria for further guidance on developing a proposed study design and methodology.

4.0 Government Furnished Information

Government Furnished Information (GFI)

Vendor SDA Reporting Template

CT FV Ordered Through USDA Foods SY 13-14

Local Purchases by State SY 12

Item 527 FFAVS 113 and FFAV 213-Local Reports

Item 799 FFAVS306-Local Item Summary By Month and State July 2012-June 2013

Item 800 Monthly USDA Report SY12-13

Item 963 FFAVS512-Fed Entitlement Spent By Funds-SY13

2015 USDA Farm to School Census Questionnaire

5.0 TASKS

This section is intended to be illustrative of the tasks, products and deliverables of the proposed study.

The tasks and deliverables listed here are required minimums. Contractors can add to the tasks as necessary to support their proposed work plan. For all tasks, FNS will provide feedback on deliverables within the following timeframe, which must be built into all contractors' proposals:

(1) for all deliverables under 25 pages, 15 working days or three weeks, whichever is longer; (2) for all deliverables over 25 pages, 6 weeks.

Task 1.1 Prepare a Revised Study, Analysis and Management Plan

The contractor shall prepare a plan for conducting the study. This plan shall be based on the contractor’s proposal, responses to the best-and-final submission, as well as discussions during the initial technical meeting. The plan shall include a discussion of research objectives and their related research questions and identification of variables and their relationship to research questions. This revised document shall incorporate information obtained from FNS about the availability of specific data at the Federal and State levels and be tailored to meet the needs of the study. It should indicate what data are available from extant data sources and what data will require primary data collection. The plan shall describe how the contractor will obtain the cooperation of respondents, and how data will be collected, integrated and analyzed.

The Revised Study and Analysis Plan shall not change the Scope of Work nor have a substantive impact on the depth or quality of research questions answered or overall anticipated costs. This document shall serve as the framework for the study.

Data Collection

Plans for data collection shall include the following:

research questions to be addressed;

data sources for each research question, including both extant data and primary data;

description of instruments to be used to collect data;

how data will be collected, validated and stored;

procedures to be used to maximize respondent cooperation/response rates;

procedures for dealing with data collection problems.

Data Analysis

The plan shall comprise quantitative and qualitative descriptive, comparative and outcome/impact analyses. The analysis plan shall describe outcome variables that will be addressed and the analytic techniques that will be employed. A discussion of how minimum detectable differences among key quantitative variables will be dealt with shall be included. A detailed plan for analysis of qualitative data, including methods for developing coding schemes, uncovering recurring themes, and synthesizing findings shall also be included. Plans for data preparation and cleaning, data entry and processing, database design, content analysis/coding, and creation of public-use files shall be included. Finally the plan shall include sample table shells that show how the results will be presented.

Deliverable 1.1.1: Draft Study Plan

Deliverable 1.1.2: Revised Study Plan

Task 1.2 Develop Data Collection Instruments

Using the Revised Study and Analysis Plan as a framework, the contractor shall develop all qualitative data collection instruments and associated qualitative data collection instructions required for the study. The contractor shall justify the use of each data collection instrument with respect to precision, respondent burden, response rate and cost. The recruitment, training, monitoring, and management of data collectors shall be included.

The contractor shall develop all qualitative data collection instruments, including interview questionnaires, focus group scripts, administrative record review protocols and any other proposed instruments, along with detailed field procedures. The draft instruments and field procedures shall be submitted to FNS for review within 8 weeks of the approval of the revised

Study Plan. FNS will provide comments within 15 working days of the receipt of the draft documents for documents totaling fewer than 25 pages and within 6 weeks for documents totaling more than 25 pages. The contractor shall revise the instruments based on feedback from

FNS and re-submit them within 15 working days. A memo describing the final instruments and field procedures is due to FNS and must be approved by FNS before data collection training can begin.

The contractor shall revise the data collection instruments and instructions developed for the initial study year in the following study year(s), based on information gathered during study administration.

Deliverable 1.2.1: Draft Data Collection Instruments and Procedures

Deliverable 1.2.2: Revised Data Collection Instruments and Procedures

Deliverable 1.2.3: Memo and Final Version of Data Collection Instruments and Procedures

Task 1.3 Identify State Agency Contacts and Recruit State Agency Participants

The contractor shall, in accordance with the Revised Study and Analysis Plan, assemble respondents at the maximum of eight (8) State agencies participating in the pilot project in each school year. The contractor shall work with FNS to obtain contact information for the State

Agency staff and shall contact State Agencies to inform them of the purposes of the data collection efforts.

Each study year, the contractor shall update this contact information.

A brief summary report shall be submitted to FNS each study year that documents this process and characteristics of the final sample.

Deliverable 1.3.1 List of State Agency contacts

Deliverable 1.3.2: Letter to State Staff

Deliverable 1.3.3: Summary Report on Study Sample

Task 1.4 Prepare Data Collection Training Package and Data Collection Manuals

The contractor shall develop a data collection training manual to accompany the data collection instruments for use by both the field personnel and data processors. This training manual shall include, at a minimum, an overview of the study, data collection instruments, procedures for data collection, and simulated data collection exercises. This data collection training manual shall be submitted to FNS for review. FNS will provide comments within 15 working days of the receipt of the draft document if it is fewer than 25 pages and within 6 weeks if it is more than 25 pages.

The final data collection training manual shall be revised to accommodate FNS comments and submitted to FNS for final approval.

Deliverable 1.4.1: Draft data collection manual

Deliverable 1.4.2: Revised data collection manual

Deliverable 1.4.3: Final data collection manual

Task 1.5 Select and Train Data Collectors

The contractor shall select and train qualified field staff to collect the qualitative data required for the evaluation. The staff shall demonstrate ability and experience to administer the interview and/or focus group instruments developed in Task1.3. This training shall be conducted in the

Washington, D.C. area so that FNS, at its discretion, may attend and participate. In addition, DVD videos of the training shall be created by the contractor and submitted to FNS to facilitate replication of the training.

Data collector training shall be carried out no sooner than two weeks before the interviews begin.

The contractor shall submit to FNS a memorandum summarizing the results of each training session (e.g., evidence of staff capacity to collect study data, along with plans for data collector replacement if necessary) within a week of completing the session(s).

Deliverable 1.5.1: Memo Summarizing Data Collector Training

Task 1.6 Collect Study Data

Following approval of the data collection plan and instruments, the contractor shall arrange with the appropriate personnel at the State agencies to collect primary study data, in each of the study years. Data collection shall proceed according to the plan set forth in the Revised Data

Collection Plan. (As noted above, FNS will provide extant data to the contractor during the fall of each study year.) Weekly status reports shall be submitted to FNS for the duration of this task.

These reports shall include the progress that has been made, problems encountered and attempted solutions. If problems are encountered that cannot be resolved by the contractor, FNS shall be notified immediately. At the completion of the data collection activity, the contractor shall prepare a brief memorandum summarizing the data collection effort. If possible, FNS would prefer to have read-only access to the data management system used to monitor the status of the data collection. Moreover, the contractor shall provide a log of the number and subject of all respondents’ calls to the contractor’s helpdesk for any field data collection instruments and shall immediately alert FNS to any problems with the data collection identified by respondents in those calls.

All interviews and focus groups will be recorded and transcribed. The transcriptions will be loaded into qualitative software (NVivo) for coding. At the midpoint of the coding (50% complete), the contractor will schedule a meeting with FNS to discuss coding and findings to date and solicit FNS feedback. Coding files to that point will be submitted to FNS for review one week prior to the meeting. When all coding is complete, the contractor will submit all coding and analysis notes to FNS in an appropriate format to be agreed upon at contract award, depending upon the qualitative software used.

Deliverable 1.6.1: Weekly Data Collection Progress Reports

Deliverable 1.6.2 Log of All Helpdesk Calls

Deliverable 1.6.3: Data Collection Summary Memo

Deliverable 1.6.4: Mid-point Coding and Analysis Notes

Deliverable 1.6.5: Final Coding and Analysis Notes

Task 1.7 Create Study Data Base and Analyze Study Data

As the study data are received in each study year, the contractor shall undertake procedures to build a data base and create analytic files consistent with the procedures described in the Revised

Study and Analysis Plan. Quality control measures should be implemented to ensure the integrity of the data.

The contractor shall conduct analyses for each study year in accordance with the approved

Revised Study and Analysis Plan. If changes in the analytic framework have occurred since the approval of this plan, the contractor shall submit a Memorandum of Intent outlining these changes for FNS approval. The contractor shall provide FNS with copies of draft analytic tables of quantitative data for FNS review during this time.

The contractor shall analyze the findings and prepare a draft memo on the findings. The memo outline shall be submitted to FNS for feedback which will be provided within 15 working days.

The draft memo will be due in 15 working days following receipt of FNS feedback on the outline. FNS will provide feedback on the draft memo within 15 working days if the memo is fewer than 25 pages and within 6 weeks if the memo is more than 25 pages. The final memo must incorporate FNS’s review. This memo is separate from the final report and will address key findings from the qualitative data. Key findings from this memo shall be included in the final study report, fully incorporated with the quantitative findings from the other, FNS-provided data sources.

Deliverable 1.7.1: Draft Analytic Tables

Deliverable 1.7.2: Memorandum of Intent (If Necessary)

Deliverable 1.7.3: Outline of Memorandum on Qualitative Findings

Deliverable 1.7.4: Draft Memorandum on Qualitative Findings

Deliverable 1.7.5: Final Memorandum on Qualitative Findings

Task 1.8 Prepare Study Report

The contractor shall prepare a draft, revised and final report that detail the results of analyses conducted in Task 1.8 for each year of the study. A detailed outline and subsequent drafts of the final report shall be submitted to FNS for review and comment. At a minimum, this report shall contain the following:

• an executive summary of the major study findings (which should be included in all drafts);

• an introduction which provides the study background data;

• a delineation of the study issues, objectives and research questions;

• a discussion of the study design and methodology, including respondent recruitment, data collection procedures, outcome measures, and analytic techniques;

• a presentation of the major findings of the study;

• a discussion of the study limitations;

• study conclusions; and

• technical appendices necessary to fully document all analytic procedures used.

The contractor shall submit to FNS a draft, revised and final report for each study year. FNS shall provide comments on the draft and revised final report within 6 weeks of submission. A copy of the final report for each study year, including all tables and appendices, shall also be submitted to FNS in electronic format in a microcomputer word processing format agreeable to

FNS. Rather than simply regurgitate findings, the final report must identify meaningful data patterns in the context of program policy and communicate those findings in clear, succinct, accurate, persuasive prose and illuminating graphics that are immediately useful for multiple audiences, particularly non-researchers.

Deliverable 1.8.1: Draft Final Report and Executive Summary

Deliverable 1.8.2: Revised Final Report and Executive Summary

Deliverable 1.8.3: Final Report and Executive Summary

Task 1.9 Prepare and Submit Data Files

At the time of submission of the draft final report for each study year, the contractor shall provide a copy of all of the restricted use data files (raw/cleaned and analysis files) used for generating the analyses presented in the first draft of these final reports and also provide all documentation that was used by the contractor when running these draft files.

For the final deliverables for this task for each study year, the contractor shall prepare and submit to FNS a set of CDs that include the raw data files and data analytic files used to produce all results that appear in the final reports. These files shall be provided in micro-computer format for use with PC SAS, NVivo or other suitable analytic software selected by FNS. In addition, the contractor shall provide full documentation for these data files including:

File structure (data set name, record format, record length, block size, and number of records);

Code book (record layout including variable names, variable format, variable labels, value labels, and missing values);

Formulation of any calculated variables; and

Method and detailed description of any content analyses for qualitative data;

Computer code (NVivo, SAS/SPSS, etc.) used to clean the data, assemble the analytic data bases, and generate all tables, graphs and flowcharts that appear in the final report.

The files shall be complete, such that FNS can quickly and easily replicate all analyses without further information from the contractor.

A separate set of “Public Use” CDs that eliminates confidential information and is ready for copying and dissemination to the public shall be prepared and submitted to FNS for each study year. These public use data CDs should include a file that provides the codebook and documentation of these data sets. The audience for the Public Use files is the broader research community. Thus, all documentation necessary to understand and use the data shall be included such that a researcher could use the files without further guidance from FNS or the Contractor.

Deliverable 1.9.1: Restricted Use Data Files (CDs) with Documentation

Deliverable 1.9.2: Revised Restricted Use Data Files (CDs) with Documentation

Deliverable 1.9.3: Public-Use Data Files (CDs) with Documentation

Task 1.10 Dissemination of the Study Findings

Concurrent with the submission of the Draft Final Report for each study year, the contractor shall conduct a briefing at FNS Headquarters in Alexandria, Virginia presenting an overview of the study, the study design, and final results for that year, including a summary of the overall findings for all study years conducted to that point. This presentation shall include visual display materials and handouts. Copies of the visual display materials and handouts shall be submitted to FNS for review two weeks prior to the presentation.

Deliverable 1.10.1: Draft Briefing Materials

Deliverable 1.10.2: Revised Briefing Materials

6.0 PERIOD OF PERFORMANCE

The period of performance for this effort will have a base period of twenty-four (24) months with two (2) additional twelve (12) month option periods.

OPTION 1: SCHOOL YEAR 2016-17

Option 1 adds a third year of data collection. Task 2.1 through Task 2.10 and the associated deliverables for Year 3 (SY 2017-18) would be a replication of Task 1.1 through Task 1.10 completed for Year 2 (SY 2016-17). The Agency's decision to exercise Option 1 will be exercised before the Contractor completes Task 1.4 of the Base Contract.

OPTION 2: SCHOOL YEAR 2017-18

Option 2 adds a fourth year of data collection. Task 3.1 through Task 3.10 and the associated deliverables for Year 4 (SY 2018-19) would be a replication of Task 2.1 through Task 2.10 completed for Year 3 (SY 2017-18). The Agency's decision to exercise Option 2 will be exercised before the Contractor completes Task 2.4 of Option 1.

7.0 DELIVERABLE SCHEDULE

DESCRIPTION WEEK

(AFTER

AWARD)

Deliverable 1.1.1 Draft Study Plan 8

Deliverable 1.1.2 Revised Study Plan 14

Deliverable 1.2.1 Draft Data Collection Instruments and Procedures 20

Deliverable 1.2.2 Revised Data Collection Instruments and Procedures 26

Deliverable 1.2.3

Memo and Final Version of Data Collection

Instruments and Procedures

Deliverable 1.3.1 List of State Agency contacts 31

Deliverable 1.3.2 Letter to State Staff 32

Deliverable 1.3.3 Summary Report on Study Sample 36

Deliverable 1.4.1 Draft data collection manual 40

Deliverable 1.4.2 Revised data collection manual 46

Deliverable 1.4.3 Final data collection manual 50

Deliverable 1.5.1

Memo Summarizing Data Collector Training Within 1 week of completing training

Deliverable 1.6.1 Weekly Data Collection Progress Reports Weekly

Deliverable 1.6.2 Log of All Helpdesk Calls

Deliverable 1.6.3 Data Collection Summary Memo 60

Deliverable 1.6.4 Mid-point Coding and Analysis Notes 66

Deliverable 1.6.5 Final Coding and Analysis Notes 72

Deliverable 1.7.1 Draft Analytic Tables 78

Deliverable 1.7.2 Memorandum of Intent (If Necessary) 78

Deliverable 1.7.3 Outline of Memorandum on Qualitative Findings 82

Deliverable 1.7.4 Draft Memorandum on Qualitative Findings 85

Deliverable 1.7.5 Final Memorandum on Qualitative Findings 91

Deliverable 1.8.1 Draft Final Report and Executive Summary 97

Deliverable 1.8.2 Revised Final Report and Executive Summary 103

Deliverable 1.8.3 Final Report and Executive Summary 109

Deliverable 1.9.1 Restricted Use Data Files (CDs)with Documentation 109

Deliverable 1.9.2 Revised Restricted Use Data Files (CDs) with 113

Documentation

Deliverable 1.9.3 Public-Use Data Files (CDs)with Documentation 116

Deliverable 1.10.1 Draft Briefing Materials 117

Deliverable 1.10.2 Revised Briefing Materials 123