ATTACHMENT_I.docx

FOOD AND NUTRITION SERVICES (FNS) ON BEHALF OF

Office of Policy Support

PERFORMANCE WORK STATEMENT

1.0 INTRODUCTION

USDA’s Food and Nutrition Service (FNS) is responsible for authorizing and managing retailer participation in the Supplemental Nutrition Assistance Program (SNAP), and only authorized food retailers can redeem benefits for cash from the Federal Government. FNS uses a variety of resources from state-of-the-art technology to undercover investigations to reduce and prevent retailer-level trafficking of SNAP benefits[footnoteRef:1] [footnoteRef:2]. As part of USDA’s efforts to monitor program integrity, beginning in 1993, FNS has periodically produced estimates of retailer-level benefit trafficking, with the most recent estimates spanning 2009-2011[footnoteRef:3]. This study will produce retailer-level SNAP benefit trafficking estimates for the 2012-2014 and develop a new regression based methodology for producing retailer-level SNAP benefit trafficking estimates. [1: Fraud: What is FNS Doing to Fight Fraud: http://www.fns.usda.gov/fraud/what-fns-doing-fight-snap-fraud] [2: http://www.fns.usda.gov/fraud/what-snap-fraud ] [3: Mantovani, Richard, Eric Sean Williams, and Jacqueline Pflieger. The Extent of Trafficking in the Supplemental Nutrition Assistance Program: 2009–2011. Prepared by ICF International for the U.S. Department of Agriculture, Food and Nutrition Service, August 2013. Available online at: http://www.fns.usda.gov/extent-trafficking-supplemental-nutrition-assistance-program-2009-2011-august-2013. ]

2.0 BACKGROUND

Although SNAP retailer trafficking does not increase costs to the Federal Government, it is a diversion of program benefits from their intended purpose of helping low-income families access a nutritious diet. This study will update the series by calculating the retailer trafficking estimates for the period spanning 2012 to 2014.

Definition of Trafficking The sale or exchange of SNAP benefits for anything other than food sold by an authorized SNAP retailer is illegal. Trafficking means[footnoteRef:4]: [4: Please see: CFR Title 7, Subtitle B, Chapter II, Subchapter C, Part 271, §2712.]

(1) The buying, selling, stealing, or otherwise effecting an exchange of SNAP benefits issued and accessed via Electronic Benefit Transfer (EBT) cards, card numbers and personal identification numbers (PINs), or by manual voucher and signature[footnoteRef:5], for cash or consideration other than eligible food, either directly, indirectly, in complicity or collusion with others, or acting alone; [5: Under the terms of the Agricultural Act of 2014, manual vouchers will be phased out. ]

(2) The exchange of firearms, ammunition, explosives, or controlled substances, as defined in section 802 of title 21, United States Code, for SNAP benefits;

(3) Purchasing a product with SNAP benefits that has a container requiring a return deposit with the intent of obtaining cash by discarding the product and returning the container for the deposit amount, intentionally discarding the product, and intentionally returning the container for the deposit amount;

(4) Purchasing a product with SNAP benefits with the intent of obtaining cash or consideration other than eligible food by reselling the product, and subsequently intentionally reselling the product purchased with SNAP benefits in exchange for cash or consideration other than eligible food; or

(5) Intentionally purchasing products originally purchased with SNAP benefits in exchange for cash or consideration other than eligible food.

(6) Attempting to buy, sell, steal, or otherwise affect an exchange of SNAP benefits issued and accessed via Electronic Benefit Transfer (EBT) cards, card numbers and personal identification numbers (PINs), or by manual voucher and signatures, for cash or consideration other than eligible food, either directly, indirectly, in complicity or collusion with others, or acting alone.

For the purposes of this study, trafficking refers to a retailer buying benefits from a SNAP participant for cash, which is part of Section 4. When individuals sell their benefits for cash, both program intent and the law are violated and public perception of the program’s integrity is undermined.

Estimation Methodologies

a. Development and Application of the “Current” Estimate The available information on all forms of investigation has expanded over time, and the technical approach has evolved to account for improved surveillance techniques and database capacities. While the basic estimation methodology has remained largely unchanged, FNS has made several improvements over the years – largely by expanding the sources of investigative results that feed into the estimates. The basis of the “original”, “revised”, and “current” estimates are described below.

Between the 1993 and 2009-2011, studies of SNAP retailer-level benefit trafficking included estimates based on actual investigations of retailer behaviors. In the 1993 and 1996-1998 retailer-level trafficking studies, the data analyzed were based on only the in-store investigations by FNS employees. To insure comparability across time, subsequent studies also generated these “original” estimates based solely on FNS investigations.

Nationwide implementation of SNAP Electronic Benefit Transfer (EBT) systems in the late 1990’s and early 2000’s has allowed for the automation of retailer redemption analysis, augmenting FNS efforts. With the Anti-fraud Locator using EBT Retailer Transactions (ALERT) system, FNS scans millions of EBT redemptions for anomalous transaction patterns. ALERT has proven to be highly successful at identifying suspicious retailer behaviors, resulting in the levy of fines, temporary suspensions from the program, and permanent disqualifications. Between 1999-2002 and 2006-2008 retailer-level trafficking studies, “revised” trafficking estimates included retailers charged with trafficking based on EBT data analysis in the denominator and retailers permanently disqualified based on EBT data analysis in the numerator.

Starting with the 2002-2005 analysis, the definition of a suspicious retailer was expanded to include all stores on the “Watch List”. The Watch List includes all stores with suspicious transaction patterns, not just those who were ultimately charged with trafficking. It also includes stores with cases opened for reasons beyond FNS investigations and EBT review (such as United States Department of Justice investigations). The inclusion of “Watch List” retailers led to the third or “current” trafficking estimate.

Starting with the 2009-2011 retailer-level trafficking report, only the most up to date method of estimating retailer-level benefit trafficking for each study period is reported[footnoteRef:6]. Thus, the original estimates are reported for the 1993 and 1996-1998 study periods; the revised estimates are reported for 1999-2002; and the current estimates are reported for the subsequent studies. [6: Mantovani, Richard, Eric Sean Williams, and Jacqueline Pflieger. The Extent of Trafficking in the Supplemental Nutrition Assistance Program: 2009–2011. Prepared by ICF International for the U.S. Department of Agriculture, Food and Nutrition Service, August 2013. Available online at: http://www.fns.usda.gov/extent-trafficking-supplemental-nutrition-assistance-program-2009-2011-august-2013.]

Even with the inclusion of Watch List retailers, the use of retailer investigations as the basis for retailer-level trafficking estimates introduces an unmistakable upward bias. To partially address this bias, investigation results have been routinely weight-adjusted by population group marginals with an iterative, post-stratification algorithm. The process is referred to as data raking[footnoteRef:7]. Data raking has been used to produce population and all mean retailer-level trafficking estimates – original, current and revised – with bootstrapped variance estimates. While data raking is a practical and reasonable approach, the results it produces could be dependent on the choice, number, and variable values of the raking dimensions. Just as importantly, the estimates produced by the raking algorithm can be sensitive to the variable values selected for categorical demarcation. In other words, a data rake that uses the percentage of households in poverty in the zip code where the store is located, taking values 0-20%, 21-30%, 31-40%, and more than 40%, may produce substantially different estimates than one that assigns the values of 0-10%, 11-20%, 21-30%, and more than 30%. [7: Cf. Longford, Nicholas T. Missing Data and Small-Area Estimation: Modern Analytical Equipment for the Survey Statistician (New York: Springer Science Business Media, Inc., 2005, pp. 44-47). See also: Deville, Sarndal, and Sautory. Generalized Raking Procedures in Survey Sampling. Journal of the American Statistical Association, vol. 88, 1993, pp. 1013 – 1020.]

b. Regression-Based Approaches The data raking methodology has been critiqued for various reasons, including the upward bias noted above and the sensitivity to subtle changes in the values of the cells created by the intersection of categories within the selected variables. During the 2009-2011 study, FNS convened a technical working group (TWG)[footnoteRef:8] to determine if there were alternative methodologies for estimating trafficking that used the latest statistical approaches, addressed some of the issues associated with the data raking methodology, and did not require additional data collection. The TWG voiced concern about the strong assumptions built into the data raking method. One of the TWG’s suggestions was to use a regression-based approach, specifically to use generalized regression with continuous variables as the logical next step to the data raking methodology which uses categorical variables. The following information is taken from the TWG report: [8: The TWG included Robert Axtell, George Mason University; F. Jay Breidt, Colorado State University; Andrew Gelman, Columbia University; Michael Larsen, The George Washington University; T. E. Raghunathan, University of Michigan; Fritz Scheuren, NORC, University of Chicago; and Mahlet Tadesse, Georgetown University.]

“One approach that was suggested focused on modeling weights, much in the same way the raking approach does, only within a regression-based framework. The emphasis in this approach remains to develop weights, not to predict the probability of trafficking. Using better measures accommodated by this approach, such as continuous variables, is a logical next step from the current raking model, which would allow for the two models to be compared. This approach also may eliminate some of the convergence issues and allow for improvement of the current model.

This approach would use a logistic regression to model a variable called sample. The sample would be given a value of ‘1’ if an investigation was conducted and a value of ‘0’ if no investigation was conducted. The independent variables would be all variables included in the raking approach and possibly others. The resulting output would represent the probability of being selected for the sample. The inverse of this would be the weight of the sampled observation. This would work for retailers, but it is not necessarily in line with redemptions, since this measure is continuous.

For redemptions, the approach for obtaining weights is to specify the outcome variable as total redemptions, and maintain the sample dummy variable on the right side of the equation along with the important covariates. The ratio of the predicted values for each identical vendor, as specified by retailer traits, provides the weights.

Another regression based approach focuses on estimating, not the weight, but rather the probability of trafficking... One of the panel members has published on this topic as a way to tackle complicated post-stratification challenges in surveys.[footnoteRef:9] Using the definition presented in Equation 1, the ‘sk’ component can be formulated as a probability function ‘pk’ using variables that include main effects and all interactions. The ‘k’ in this case is defined as the set of predictors used in a predictive function. Thus, instead of raking to obtain the ‘w’, the effort is to obtain the probability of trafficking for retailers with certain traits. The cell estimate ‘p’ can be applied to each retailer in the population, the weight therefore being represented in the aggregate by the number of times stores with a particular vector of traits are present in the population. Hence, the sample becomes self-weighting.[footnoteRef:10] [9: Gelman, A. (2007). “Struggles with Survey Weighting and Regression Modeling.” Statistical Science, 22(2), 153-164.] [10: The function for predicting ‘y’ takes into account the post-stratification factors, and thus provides for each individual retailer the probability of violation given the post-stratification factors.]

Using this approach, the model-based regression estimate consists of three steps:

· First, a probability of trafficking is estimated from a regression model using investigated retailers.[footnoteRef:11] [11: It is recognized that the convenience property of the sample could lead to biases in the estimated parameters.]

· Second, this equation is applied to each retailer in the population to obtain a probability of trafficking for a specified retailer.

· Third, probabilities over all retailers are summed to yield the total number of retailers that traffic, and when divided by the number of retailers in the population, it provides the trafficking rate.

The first step in estimating a trafficking rate is to generate the probability that a retailer will traffic. Our starting point is the following equation:

Equation 2

Where ‘p’ is the probability of a retailer trafficking and y = is a logistic function; ‘y’ is whether the retailer has trafficked or not, ‘x’ is the predictor of trafficking, ‘β’is the parameter to be estimated, and ‘ε’ is the error term with a mean of zero. The model can also include multilevel or hierarchical effects that control for variation in higher level aggregations such as States.

There are two tenets that underlie the regression approach to dealing with adjusting sample values to the population. All factors included in the model should be important in identifying differences between the population and the investigative sample, and they should be useful in predicting a violation. A goal is also to generate a parsimonious model—one that provides a high level of prediction with the fewest variables.

Currently, the main effect variables found to be important both in adjusting the investigative sample and predicting trafficking include:

· Store ownership, which reflects a hypothesis that stores attached to publicly traded corporations are less likely to traffic due to an expected higher level of public scrutiny within these stores and the concomitant need to maintain systems to deal with this scrutiny.

· Retailer type, which is hypothesized to be important in showing that supermarkets and other similar stores are less likely to traffic than small groceries. This would reflect the added complexity of trafficking in stores with larger workforces than those in which one or two individuals control the transactions.

· The neighborhood in which the retailer is located, which is hypothesized to be associated with trafficking, with greater levels of trafficking found in urban, high-poverty areas. In actuality, these are contextual variables in that they describe potential retailer clusters.

There are other variables that could be considered, but these would have to be characteristics collected for the retailer population as a whole and not just for those retailers being investigated.

Data Sources Required: One necessary of data source is EBT data, which could predict the types and frequency of particular transactions among stores. Another source of data could be State-maintained information on SNAP recipients, which when combined with EBT data, might provide some very powerful predictors as a result of capturing the interaction between individual transactions and State-level investigative practices or distribution of resources within a State or region.

The multilevel approach cited above would incorporate second- and possibly third-level variables representing neighborhood characteristics and State- or regional-level characteristics. As mentioned, neighborhood characteristics such as poverty level and urbanization may be important for predicting trafficking. These and other neighborhood characteristics (such as the clustering of stores that traffic) may be used within a two-level model, one level focusing on retailers and the other on neighborhood.

Feasibility: Regression-based approaches have the advantages of allowing flexibility in the specification of the model in terms of the number and form of the predictor variables. In other words, we do not have to specify every combination of the variables used as predictors. It also allows a more exact representation of the probability of trafficking, avoiding the use of a dichotomous variable to represent trafficking… An issue with the regression approach as with the raking approach is selection. The data and resulting models are mostly focused on high-risk stores. This may create a problem with translating information to lower-risk stores. Overall, this approach is a natural extension from the current raking approach and would not require additional data collection.”

Therefore, in theory, a regression-based method would moderate some of the upward bias introduced from using categorical variables used for the raked estimates by using continuous variables and removing the sensitivity of variables such as the redemption decile[footnoteRef:12]. [12: The final report from the Technical Working Group, Alternative Approaches for Estimating SNAP Trafficking Rates: 2013 Technical Working Group Report, is not available online but will be provided upon award. ]

3.0 SCOPE OF WORK

USDA-FNS seeks a contractor to update the retailer trafficking estimates by generating the “current” estimates for the period spanning 2012 to 2014 and produce the seventh report in the series on retailer trafficking estimates. The study will also require that the contractor develop a regression-based approach for calculating trafficking estimates and apply this regression adjustment to the 2009-2011 and 2012-2014 data to generate trafficking estimates to compare with the “current” estimates.

Study Objectives Therefore, the primary purpose of this study is to produce the “current” retailer-level trafficking estimates for SNAP retailers for 2012-2014 using the methodology described in the 2009-2011 retailer-level trafficking report[footnoteRef:13] to allow for consistent inter-temporal comparisons with estimates for prior years. This study will also develop a regression-based approach and recalculate trafficking estimates using the regression-adjustment. [13: Mantovani, Richard, Eric Sean Williams, and Jacqueline Pflieger. The Extent of Trafficking in the Supplemental Nutrition Assistance Program: 2009–2011. Prepared by ICF International for the U.S. Department of Agriculture, Food and Nutrition Service, August 2013. Available online at: http://www.fns.usda.gov/extent-trafficking-supplemental-nutrition-assistance-program-2009-2011-august-2013. ]

The objectives for the proposed study are:

(1) To produce current retailer-level trafficking estimates for SNAP retailers from 2012 through 2014. Each set of estimates will include the following indicators:

· Total estimated value of SNAP redemptions that were trafficked at the retailer-level;

· Retailer-level trafficking rate, or the estimated proportion of SNAP redemptions that were trafficked at the retailer-level;

· Total number of stores estimated to be trafficking; and

· Store violation rate, or the estimated proportion of authorized stores that engage in trafficking.

(2) To produce 2012-2014 SNAP retailer trafficking estimates by store characteristics including:

· Store size and type (example supermarket, convenience store etc);

· Ownership (public or private);

· Urban/Rural classification;

· Poverty level of store’s neighborhood; and

· SNAP redemption level.

(3) To develop a regression-based estimation method for calculating retailer-level SNAP trafficking and apply this regression adjustment to recalculate the 2009-2011 and 2012-2014 retailer trafficking estimates.

Under this objective, the contractor will apply the regression based estimation method to recalculate, and produce the retailer-level trafficking estimates specified under objectives (1) and (2) above.

Tasks/Subtasks/Deliverables The contractor shall perform all tasks below and deliver all documents and in-person briefings needed to address the study objectives and deliver the major deliverables in final form acceptable to FNS.

CLIN #001: Task #001: Produce Raked Trafficking Estimates The selected contractor shall produce “current” estimates of SNAP benefit trafficking for the 2012-2014 period using the data raking algorithm to address objectives 1 and 2. These estimates shall be generated using the methodology used for the 2009-2011 estimates.

The raking matrix will be defined in the following dimensions:

· Store Type

· Ownership Type

· Redemption amount deciles

· Census Bureau Zip Code Tabulation Area (ZCTA) Degree of Urbanization

· Census Bureau ZCTA Percentage of Households in Poverty To assist with this task, FNS will provide the following information spanning 2012 through 2014:

· Retailer descriptive data from the Store Tracking and Redemption System (STARS), including but not limited to store type, ownership type and address (zip code) information. A codebook will be provided as well.

· Case data associated with:

· FNS Retailer Investigations Branch investigations

· Cases based on EBT data analysis

· Investigations conducted by USDA Office of the Inspector General (OIG), U.S. Department of Justice, State, and other entities

· Other case types

· Watch List outcomes.

The selected contractor shall merge this data with Census bureau files, matching retailer postal zip code to the nearest Census Zip Code Tabulation Area (ZCTA). Those proposing should note that there is not a one-to-one correspondence between U.S. Postal Service Zip Codes and ZCTAs. Rather the ZCTA represents the most frequently occurring USPS Zip Code in an aggregation of census blocks[footnoteRef:14]. The selected contractor shall match the retailer postal zip to the nearest Census ZCTA. [14: For more information regarding the correspondence, please see: https://www.census.gov/geo/reference/zctas.html.]

FNS will also provide the SAS code used to generate the previous estimates and dictionaries for all relevant data sets. Results shall be reported by the same variables and categorizations as in the most recent report. The tables in this report will be reproduced with the updated estimates. FNS will also provide the data used to calculate the 2009-2011 retailer-level SNAP trafficking estimates.

CLIN # 001/Subtasks Subtask #001.1: Revise Work Plan Within two (2) weeks of execution of the contract, the contractor shall review their proposed Work Plan with the FNS project officer and FNS staff at FNS Headquarters, Alexandria VA. Based on this review the contractor shall submit a draft revised work plan within four (4) weeks of contract award. Revisions to the work plan shall not change the scope of the contract.

A final revised work plan incorporating FNS comments shall be delivered within eight (8) weeks of the contract execution in MS Word and pdf formats.

Subtask #001.2: Acquire Data FNS will provide a copy of the 2009-2011 and the 2012-2014 data on authorized food retailers including data on investigations and administrative cases (see (a) and (b) below) along with the data, SAS program files, and documentation used in producing the 2009-2011 retailer trafficking estimates, within 6 weeks of contract award.

(a) Data on Authorized Food Retailers FNS will provide data from the STARS database on the universe of food retailers authorized to participate in SNAP from January 1, 2012 through December 31, 2014. For each retailer in STARS, there are approximately 20 – 30 data fields relevant to this study that record information such as store name, store address, store category type (e.g., supermarket, convenience, etc.), estimated gross sales and annual SNAP redemptions. There may be variations in the data dictionary (and data layout) of STARS over time. Some of these fields (e.g., neighborhood poverty level) will need to be collapsed into broader categories to calculate estimates.

(b) Data on Cases against Stores FNS will provide data on all cases completed from January 1, 2012 through December 31, 2014.

(c) Data on Watch List stores FNS will provide data on all Watch List instances between January 1, 2012 and December 31, 2014 FNS will provide data dictionaries and layouts for STARS and Watch List data to assist with the merging of records and the interpretation of data fields.

There are approximately 7,500 cases opened against stores each year, which includes in-store investigations. Each case record will have about 30 relevant variables, including such elements as store authorization number (the unique FNS identifier), case number, investigation initiation source (e.g., lead, OIG, ALERT, etc.), and multiple measures of outcome (e.g., investigator reports trafficking, compliance officer determination, administrative review upholds/reverses the findings, etc.). Some stores may have more than one type of case, with the potential for duplicate counts in the data.

EBT cases reflect actions taken by field staff because of suspicious redemption activity as determined through the ALERT system. Suspicious activity often results in a charge letter being sent to retailers. To calculate the trafficking estimates, the case outcome (e.g., permanent disqualification, trafficking civil money penalties[footnoteRef:15] etc.) shall be used to form the numerator. The Watch List is the appropriate denominator for “current” estimates. It is a database of retailers who have been selected for inclusion because their SNAP transaction/redemption patterns signal possible trafficking as well as stores that have had cases opened against them for other reasons. [15: For legislative information on civil money penalties go to Food and Nutrition Act of 2008 - Sec.12

(d) Acquire and Merge Census Data to FNS Files The contractor shall obtain 2013 American Community Survey data from the US Census Bureau regarding urbanization and poverty and merge it with the information described above within two (2) weeks of these data being made available by the US Census Bureau. As there is not a one-to-one correspondence between USPS Zip Code and Census ZCTA with the latter being a proper subset of the former. Consequently, the contractor shall map retailer Zip Codes to the closest ZCTA.

Subtask #001.3: Prepare and Calculate Trafficking Rate Estimates The contractor shall prepare a SAS analytic file to analyze the data acquired in the Subtask #1.2. These files shall be prepared in a format for use with PC SAS software. If the contractor wishes to calculate the retailer-level trafficking rates using software other than SAS, the analytic file that the contractor uses shall also be delivered to FNS, in addition to the SAS analytic file.

Using the data acquired in Subtask #001.2, the contractor shall produce the current redemption-based (estimate 1) and store-based (estimate 2) trafficking estimates. The production of these estimates will require Watch List of retailers with suspicious transaction patterns, and investigations conducted by others, like OIG.

Subtask #001.4: Produce Updated Tables on the Extent of Retailer-Level Trafficking Within twenty-six (26) weeks of the execution of the contract but no later than eight (8) weeks of receiving 2013 Census data, the contractor shall compute a set of draft tables comparable to those contained in the those included in the 2009-2011 final report and provide them to FNS for review. This includes all tables found in the appendices and footnotes.

Within thirty-four (34) weeks of execution of the contract but no later than four (4) weeks after receiving FNS comments, the contractor shall submit final data table that incorporate FNS comments.

Subtask #001.5: Prepare Final Report The contractor shall prepare a report based on the data analyses conducted in Subtask #001.3 and the tables produced under Subtask #001.4. This report shall include an Executive Summary of no more than 10 pages that includes a concise description of major findings of the descriptive and comparative analyses. The report shall include a background description of the project as it relates to Task #001, a discussion of objectives #1 and #2 and their research questions, a description of the methodology, the results of the analyses and all relevant limitations and qualifications. Results will be reported by the same variables and categorizations as in the 2009-2011final report.

Within thirty-four (34) weeks of execution of the contract, the contractor shall submit an electronic copy (in MS Word) of the draft final report to FNS for review and comment.

Within forty-two (42) weeks of execution of the contract, the contractor shall submit to FNS a revised final report incorporating FNS comments and requested revisions.

Within forty-two (42) weeks of the execution of the contract, the contractor shall submit a draft PowerPoint presentation to the FNS project officer for review.

Within forty-three (43) weeks of execution of the contract, the contractor shall brief FNS management and staff on the results of the analyses at FNS Headquarters in Alexandria, Virginia and a copy of the final PowerPoint presentation shall be submitted to the FNS project officer electronically.

Within fifty (50) weeks of execution of the contract, the contractor shall make any final revisions and submit two electronic copies (MS-Word and PDF) of the final report to FNS. Upon clearance of the report, the contractor shall submit electronic files in MS-Word and pdf formats, along with fifteen bound hard copies.

Subtask #001.6: Provide Data, Data Dictionary, and All Data Cleaning and Analytic Files The contractor shall supply FNS with DVDs[footnoteRef:16] containing the final data sets (flat and SAS formats) and all SAS files used to clean the data, rake the data, and produce the retailer-level trafficking rates and tables described in subtasks #001.3 and #001.4. Additionally, the contractor shall provide a data dictionary that describes the final SAS data set. The dictionary shall describe the file structure (data set name, record format, record length, block size and number of records); and all variable names, formats, labels, values, missing values, and the frequency of missing values. The formulation of any calculated variables shall also be described. [16: These may be provided in another mutually agreed upon format. ]

Within fifty (50) weeks of execution of the contract, a draft data dictionary and technical description shall be provided to FNS.

Within fifty-eight (58) weeks of execution of the contract. The final data DVD[footnoteRef:17], analytic and cleaning code, and final data dictionary and technical description shall be provided to FNS. [17: These may be provided in another mutually agreed upon format. ]

CLIN #002 (Task #2): Create Regression-based Trafficking Estimation Methodology and Recalculate Retailer-Level Trafficking Estimates For this task, the selected contractor shall create a regression-adjustment for calculating retailer-level SNAP benefit trafficking estimates based upon the findings of the TWG[footnoteRef:18] to replace the raking-based methodology when calculating retailer-level SNAP benefit trafficking estimates using the same administration data that were used for producing the “current” estimates under CLIN #001. The regression-based estimation methodology shall be based upon commonly accepted[footnoteRef:19] and mutually agreed upon statistical techniques, and built by experts in the economics, computational social science, statistics, or a similar field. The methodology shall encompass at least the five variables that have traditionally been used in the trafficking reports, but can also include others recommended by the TWG (e.g., a risk score). Two experts, selected in consultation with FNS, who have not been involved in the design of the methodology and are not employed by the contractor or subcontractor shall peer-review the methodology and provide a written assessment[footnoteRef:20]. Additionally, the methodology shall be tested for accuracy, validity, and reliability, and be calibrated with the estimates from the present and the 2009-2011 study periods[footnoteRef:21] [footnoteRef:22]. This testing shall be based on commonly accepted methods and justified by peer reviewed literature. Furthermore, it shall be clearly described in the proposal, all work plans, and the final report. Finally, after the regression-based trafficking estimate methodology has been developed, the contractor shall recalculate the raked trafficking estimates using the regression-based trafficking methodology for the 2009-2011 and the 2012-2014 study periods. If a new variable is added in the regression-based methodology, it will be measured during the same time period as the variables from the raking methodology (e.g., a variable from the ACS would be taken from the same measurement period as the poverty level and urbanicity variables used in the raking methodology). [18: While the report of the TWG has not been publically released, it will be delivered to the selected contractor. ] [19: Such as those found in published, peer reviewed academic publications ] [20: It is acceptable to compensate the reviewers for their time. However, the reviewers may not be consultants for other tasks/subtasks of this contract. ] [21: Other methods for testing accuracy, reliability, and validity may be acceptable based upon the offeror’s justification in the proposal. However, these methods must be clearly described in the necessary detail in the proposal and justified based upon peer reviewed literature. ] [22: If the offeror believes that testing against data from a greater number of the previous trafficking estimate studies is needed, they should justify the need. ]

CLIN #002/Subtasks Subtask #002.1: Finalize Work Plan Within six (6) weeks of the execution of the contract, the contractor shall submit the draft revised Work Plan specific to task #2 the development of a regression-based methodology for calculating SNAP trafficking estimates. The proposed work plan shall include a detailed description of the contractor’s approach to developing the regression-based methodology for calculating retailer-level SNAP benefit trafficking estimates. The plan shall include the results of a review of data available from FNS in terms of what could be used to build the method. Furthermore, the plan shall include an explanation for any variables beyond those that are currently used that the contractor intends to use in the regression-based methodology and why any of the five currently used variables would be excluded. The finalized work plan shall not change the scope of the study.

Within twelve (12) weeks of the contract award, the contractor shall submit a final Revised Work Plan for this task taking into account FNS comments.

Sub-Task #002.2: Develop Regression Based Methodology for Calculating SNAP Retailer-Level Trafficking Estimates Within eight (8)[footnoteRef:23] weeks of contract award, the contractor shall begin the process of developing a regression-based methodology for calculating SNAP retailer-level trafficking estimates. The methodology should be based upon the findings of the TWG report provided to the contractor and conform to the criteria set forth in Objective #3. FNS envisions this to be a collaborative process with the expectations that new method will be fully developed needing only minor revisions before the first draft of the regression-based methodology memo has been delivered. It shall take into account at least the five concepts used in the data raking methodology: [23: FNS understands the different approaches to developing the regression-based method could take different amounts of time. Therefore, the deliverable schedule should be seen as a guide. Proposals should enumerate what the offeror believes to be the most realistic timetable along with an explanation for what that timetable has been chosen. ]

· Store type,

· SNAP benefit redemption level,

· Store ownership type,

· Poverty level of the area around the store, and

· Urbanicity of the area round the store.[footnoteRef:24] [24: FNS recognizes that the offeror may find some of concepts be redundant or otherwise not to be useful in a regression-based model. If this is the case, there should be an explanation of why a variable was excluded. ]

Within forty-four (44) weeks of the execution of the contact, the contractor shall deliver to FNS a regression based methodology for calculating SNAP retailer-level trafficking estimates and accompanying draft descriptive memo. The methodology shall be designed to achieve the government’s aims set forth in Objective #3. Accompanying the methodology shall be a memo explaining 1) the process the offeror used to create the methodology; 2) a description of what each included variable is intended to measure; 3) an explanation of the usefulness of any interaction between variables that is included in the methodology; 4) the reason that any variables discussed in the work plan that are not included in the methodology have been excluded; 5) an in-depth description of the steps used to validate the methodology for reliability, internal validity, and external validity; and 6) a discussion of the strengths and limitations of the new methodology.

Within fifty-two (52) weeks of execution of the contract, the contractor shall deliver to FNS a finalized regression based methodology for calculating SNAP retailer-level trafficking estimates and accompanying revised descriptive memo, taking into account all FNS comments to the draft methodology.

Within fifty-three (53) weeks of execution of the contract, the contractor shall deliver to FNS draft PowerPoint briefing slides for a briefing about the development of the regression-based methodology for calculating SNAP retailer-level trafficking estimates.

Within fifty-four (54) weeks of execution of the contract, the contractor shall provide to FNS a briefing on the methodology created to calculate SNAP retailer-level trafficking estimates. The briefing shall take place at FNS headquarters in Alexandria, Virginia and will include FNS staff and anyone else FNS deems appropriate. The briefing shall otherwise be closed to the public. The briefing should address all research questions under Objective #3, provide a thorough description of how the methodology was created, and provide a description of the strengths and limitations of the methodology. Final PowerPoint briefing slides should be delivered to FNS two days before the briefing.

Within fifty-eight (58) weeks of execution of the contract, the contractor shall provide to FNS a final regression based methodology for calculating SNAP retailer-level trafficking estimates and accompanying final descriptive memo, taking into account all FNS comments to the revised methodology and from the briefing.

Sub-Task #002.3: Prepare and Calculate Trafficking Regression-Based Estimates Within fifty-four (54) weeks of execution of the contract, the contractor shall prepare a SAS analytic file to reanalyze the data acquired in the Subtask #001.2 (under CLIN#001). These files shall be prepared in a format for use with PC SAS software. If the contractor wishes to calculate the trafficking rates using software other than SAS, the analytic file that the contractor uses shall also be delivered to FNS, in addition to the SAS analytic file.

Using the data acquired in Subtask #001.2, the contractor shall produce the current redemption-based (estimate 1) and store-based (estimate 2) trafficking estimates.

Sub-Task #002.4: Produce Updated Tables on the Extent of Trafficking Within fifty-four (54) weeks of execution of the contract, the contractor shall compute a set of draft tables comparable to those contained in the report subtask #001.4 (under CLIN#001) and submit to FNS for review. This includes all tables found in the appendices and footnotes.

Within fifty-eight (58) weeks of the execution of the contract, the contractor shall present to FNS final tables taking into account FNS comments.

Sub-Task #002.5: Produce Final Report on Regression-Based Trafficking Estimates The contractor shall prepare a report based on the data analyses conducted in Subtask #002.3 and the tables produced under Subtask #002.4. This report shall include an Executive Summary of no more than 10 pages that includes a concise description of major findings of the descriptive and comparative analyses. The report shall include an Executive Summary of no more than 10 pages, a background description of the project as it relates to Task#002, a description of the regression-based methodology contrasting and it with the data raking methodology, a discussion that incorporates the study objectives #1 and #2 and the associated research questions as they are applied to Task #002, the results from recalculating the estimates using the regression adjustment, and all relevant limitations and qualifications. This report will also compare the estimates produces by data raking under Task #1 with those produces under this Task (Task #002).

Within sixty-two (62) weeks of execution of the contract, the contractor shall submit an electronic copy (in MS Word format) of the draft final report regarding the retailer-level SNAP trafficking estimates created from the regression-based methodology to FNS for review and comment.

Within seventy (70) weeks of execution of the contract, the contractor shall submit an electronic copy (in MS Word format) of a revised draft final report to FNS for review and comment. The revised draft shall address any comments FNS has made to the draft final report.

Within seventy-two (72) weeks of execution of the contract, the contractor shall submit to FNS draft briefing slides for a briefing regarding the findings of the regression-based estimation report. The slides should address the major findings of the report and how the new estimates compare to the estimates from the previously used raking methodology.

Within seventy-three (73) weeks of execution of the contract, the contractor shall provide to FNS a briefing on the findings of the regression-based estimation report. The briefing shall take place at FNS headquarters in Alexandria, Virginia and will include FNS staff and anyone else FNS deems appropriate. The briefing should address Objective #3, address the major findings of the report, and discuss how the new estimates compare to the estimates from the previously used raking methodology. Final briefing slides should be delivered to FNS three days before the briefing.

Within seventy-eight (78) weeks of execution of the contract, the contractor shall submit to FNS two electronic copies of the final report (in MS Word and PDF formats). The final draft shall address any comments FNS had made to the draft final report and the revised draft final report. Upon clearance of the report, the contractor shall submit electronic files in MS-Word and pdf formats, along with fifteen bound hard copies.

Sub-Task #002.6: Provide Data, Data Dictionary, and All Data Cleaning and Analytic Files Within seventy-eight (78) weeks of the execution of the contract, the contractor shall supply FNS with DVDs containing the final data sets (flat and SAS formats) and all SAS files used to produce the trafficking rates and tables described in subtasks #002.3 and #002.4. Additionally, the contractor shall provide a data dictionary that describes the final SAS data set. The dictionary shall describe the file structure (data set name, record format, record length, block size and number of records); and all variable names, formats, labels, values, missing values, and the frequency of missing values. The formulation of any calculated variables shall also be described. A draft data dictionary and technical description shall be provided within seventy (70) weeks of award.

The final data DVD, analytic and cleaning code, and final data dictionary and technical description shall be provided within eighty-four (84) weeks of award.

4.0 DELIVERABLE SCHEDULE

CLIN #001: Produce Raked Trafficking Estimates for 2012-2014

PDF via email

MS PowerPoint slides via email

PDF via email

Documents: Microsoft Word Document via email and

CLIN #002: Create Regression-Based Methodology for Calculating Retailer-Level SNAP Benefit Trafficking Estimates

PDF via email

PDF via email

Documents: Microsoft Word Document via email and

5.0 PERIOD OF PERFORMANCE

The period of performance will have a base period of twenty-four (24) months from the date of award.