SOW for Tumult Feasibility Study final v2 - Redacted.pdf

STATEMENT OF WORK: Feasibility Study Follow-up on the Differential Privacy

PURPOSE

The USDA’s National Agricultural Statistics Service (NASS) is exploring innovative methodology to modernize the statistical disclosure limitation techniques applied to the Census of Agriculture and Survey data. Specifically, differential privacy (DP) is be explored as a solution, and NASS wants to evaluate the feasibility of using DP.

EXPECTED OUTCOME

As a result of this contract, NASS will receive a list of services from Tumult Labs.

• Provide progress reports and discuss questions raised by USDA NASS staff at weekly meetings.

• Develop a PowerPoint slide deck of a talk about the NASS disclosure feasibility study that can be presented at NASS and the broader statistical community;

Tumult Labs and NASS will collaborate to present NASS disclosure feasibility studies at the 2024 FCSM research and policy conference.

• Create a slide deck or document to list findings and recommendations on USDA- NASS staff’s pseudo-code implementation.

• Train NASS staff in using Tumult Analytics.

• Assist USDA-NASS staff in their exploration of combining Per-Record differential privacy (PRDP) with the cell suppression method.

• Evaluate at a high level whether the Random Tabular Adjustment-PRDP (RTA-

PRDP) approach might have a formal mathematical grounding.

• Submit a summary report with some recommendations for future work on the continuous different privacy approach.

BACKGROUND

NASS is the Federal Statistical Agency responsible for publishing official statistics on U.S. agriculture. NASS conducts hundreds of surveys each year and the Census of Agriculture every five years and prepares over 400 reports. NASS publications cover a wide range of subjects, from calves born to hogs slaughtered to traditional crops, such as corn and wheat, to specialties, such as mushrooms and flowers. NASS is asked to provide estimates of yield, acreage, and cash rents in support of other USDA programs.

In addition, NASS's various Field Offices (FOs) publish data about many of the same topics for local audiences. Survey estimates continue to be the foundation of published estimates, and increasingly, non-survey data, including administrative, remotely sensed, and weather data, are integrated with the survey data, often through models, to produce official statistics.

Under Title 18, U.S. Code, Section 1905, Disclosure of confidential information, Title 18, U.S. Code, Section 1902, Disclosure of crop information and speculation thereon, Title

7 of U.S. Code, Sec 2276, Confidentiality of Information, and Confidential Information Protection and Statistical Efficiency Act (CIPSEA) of 2018, Title III of the Evidence Act, the National Agricultural Statistics Service (NASS) is required to protect the confidentiality of individual information in the production of official statistics. The confidentiality requirement applies to data released to the general public, and it can also apply to data released to other agencies. The current NASS cell suppression methodology based on a linear program called “Minimum Cost Flow” was developed in the late 1990s. Since then, new disclosure protection methodologies have been developed. The NASS Statistical Disclosure Limitation (SDL) research team is conducting disclosure methodology research, evaluating options for disclosure methodology, and will recommend a methodology for NASS to use as a foundation for its disclosure system, which is to be implemented by 2026.

From October 2021 to March 2022, Research and Development Division (RDD) and Methodology Division (MD) organized the Phase 1 SDL studies through 6 seminars and 16 group presentations and discussions. The methods reviewed were as follows: NASS Cell Suppression System (Minimum Cost Flow), Cell suppression method with General Linear Programming, Controlled Tabular Adjustment (CTA), Random Tabular adjustment (RTA), Noise method, Synthetic Data Approach, Differential Privacy method, and Flexible Table Builder.

On November 1, 2022, Statistical Disclosure Limitation (SDL) Research Team was established. The purpose of the SDL research team is to organize and conduct systematic research on the disclosure methods based on the SDL Phase 1 studies. The initial focus has been on disclosure methods, such as the suppression method with general linear program, controlled tabular adjustment (CTA), and the synthetic data approach. Other methods may be considered as determined by the team. Using the 2017 Census of Agriculture (COA) data, comparative analyses have been conducted between the new methods under consideration and the current NASS disclosure methodology (suppression method with minimum cost flow). Currently, methods prototyped using 2017 data are being applied to the 2022 data and comparative analyses between the proposed methods and the current NASS disclosure methodology are being conducted. Based on the analyses, the team will make recommendations on an SDL approach to the NASS Business Council and Senior Executive Team for implementation on an agreed-upon set of surveys by 2025. The new SDL methodology method and system will be fully operational in 2026 for the 2027

COA.

On March 31, 2023, SDL research team established a separate research sub-team, to specifically study the Differential Privacy method.

On September 8, 2023, USDA-NASS awarded a Feasibility Study contract to Tumult Labs. Tumult Labs performed the contract from 09/11/2023 to 11/10/2023. At the end, Tumult Labs delivered a feasibility report, which included

• A description of the problem and goals of the Census of Agriculture based on Michigan Table 31.

• A list of challenges in applying DP methodology to the data product.

• A proposed high level algorithmic approach that addresses these challenges.

• An enumeration of privacy parameters that impact accuracy in this approach.

• A discussion of privacy-accuracy tradeoffs enabled by the recommended algorithm.

• An illustration of the accuracy vs privacy tradeoffs using a combination of analytical formulae and experiments on simulated data (matching the pilot data product).

• A plan for future work needed to prove the viability of the approach.

TASKS

1. Provide personnel for scientific consulting on PRDP and past engagement:

a. Respond to questions about the PRDP methodology, including general theory or its potential application to Ag Census data products.

b. Respond to additional questions about Tumult Labs 2023 feasibility study.

c. Discuss vulnerabilities of legacy disclosure avoidance techniques.

d. Provide general mathematical consulting on disclosure avoidance related topics.

2. Develop a joint presentation at NASS and FCSM.

3. Examine and validate USDA-NASS pseudocode logic for PRDP.

4. Meet monthly with the Contracting Officer Representative (COR) and twice per month with the Project Manager (PM) to discuss issues and review status throughout contract performance.

5. Provide written summary report on the feasibility of using differential privacy.

KEY PERSONNEL

Contractor shall provide qualified and experienced persons for the completion of the State of Work (SOW) requirements. Key project persons shall have successfully participated in the completion of similar projects. All contractors and subcontractors working on this requirement shall be U.S. Citizens. The Contractors shall provide a key person to perform and/or manage the requirements of the SOW. Replacement personnel shall meet the same qualifications and be approved by an authorized government representative.

Team qualifications must include the following:

• At least 10 years of proven experience in:

Statistical disclosure limitation Differential privacy

Data structure on the Census and survey data for government agencies.

• Evidence of conducting high performance computational statistics.

• Experience in production of DP for government agencies.

• Proven publication record of experience in the field of differential privacy.

• Award-winning research contributions to the field of differential privacy.

• Ph.D. level researcher.

DELIVERABLES

Schedule of Deliverables

Deliverable Description Due Date

1 Provide personnel meeting requirements outlined in the SOW.

At Contract Award

2 Provide a Ph.D. researcher who will mentor junior personnel weekly and be responsible for a high-quality product.

Throughout contract performance

3 Meet monthly with the COR and twice per month with PM to discuss issues and review status throughout contract performance.

Specific date and time will be determined after award.

4 Provide all technical documentation required by the SOW tasks in an organized, usable document to

NASS.

End of contract.

5 Provide a written summary report on the feasibility follow-up on the differential privacy.

End of contract.

PERIOD OF PERFORMANCE

The period of performance will be 2 months and shall be from September 3, 2024 to November 2, 2024.

NASS Project Manager/Contracting Officer’s Representative (COR) Project Manager: Michael Jacobsen USDA/South Building

COR: Denise Abreu USDA/South Building