1_b PWS DOE College Scorecard.pdf

Performance Work Statement Office of Research Analysis and Statistics (RAAS)Internal Revenue Service

(IRS)

Support for Creating a Public Dataset for the Department of Education’s College Scorecard

1. BACKGROUND

The Department of Education’s (ED) College Scorecard calculator is a tool that helps students compare post-graduation earnings across different universities and programs. IRS-Statistics of Income Division (SOI) contributes earnings data to the Scorecard for certain cohorts of students identified by ED’s Federal Student Aid office. Counts and median earnings are computed based on various indicators. Because the data resides on a public-facing website, it is imperative to protect the confidentiality of the Federal Tax Information (FTI) for individuals in the cohorts.

Given the complexity of breakouts requested by ED, we will employ a statistical disclosure limitation methodology based on differentially private algorithms.

1.1. PURPOSE

Production of a public data set for the Department of Education College Scorecard.

1.2 AGENCY MISSION

SOI is one of 13 major Federal statistical agencies. The division’s four branches—Individual & Tax Exempt; Corporation, Partnership, and International; Information Management & Dissemination; and Statistical Services—are staffed primarily by economists, statisticians, computer specialists, and communication specialists. Collectively, about 140 employees are responsible for various SOI projects, including tax data related to individuals, estates, businesses, nonprofit organizations, trusts, and foreign investments. SOI uses both the Internet and a few traditional print publications to provide the public with data on the Federal tax system. It also has a Statistical Information Services staff, who respond to inquiries about SOI data

2. SCOPE

Contractor support to develop a calculator tool through which SOI can release multiple datasets comprising both counts and medians broken down by attributes (e.g., median earnings by college, program, etc.) under a common differentially privacy budget (epsilon) to maintain the confidentiality of FTI such that the output can be released on the Department of Education’s public-facing website.

3. OBJECTIVE

Apply statistical disclosure limitation methodology based on differentially private algorithms (code to run Python) to support the production of a public dataset for the Department of Education College Scorecard.

4. TASKS

The Contractor shall:

• Provide Python code to analyze, aggregate and add noise to aggregates computed from the raw data file comprising information from both ED and IRS.

• Work with IRS to run initial tests on the calculator and to configure and validate the schema on the IRS server.

• Provide IRS with a excel macros to estimate error rates for user-input breakouts at various epsilon levels.

• Provide IRS with Python code to run on a RAAS server that will allow IRS to define public outputs for multiple datasets; visualize the privacy-utility tradeoffs associated with the request; refine the requested tabulations or change the privacy loss budget requested; and to apply a differentially private mechanism customized to the release request and run on the sensitive data to generate the output release. The output will contain the noisy tabulations requested by the analyst as well as error estimates on the noisy values.

• Contractor will develop a customized interactive evaluation tool in the form of complex Excel Macros for IRS-SOI allowing for adjustments in privacy loss budget parameter (epsilon), workload parameters, and suppression thresholds. This tool will compute counts of cell suppression, and relative and absolute error by different quantiles and statistics to support internal discussions on identifying the total and allocation of privacy budget.

• The contractor will develop a customized interactive evaluation tool in the form of complex

Excel Macros for ED allowing for adjustments in workload parameters and suppression thresholds. This tool will compute counts of cell suppression, and relative and absolute error by different quantiles and statistics to support on identifying the workload parameters and/or suppression threshold.

• Contractor will support post-delivery research to further improve privacy-utility balance in future versions of the College Scorecard. Continuing research would include, but not limited to, identifying alternative privatization algorithm that better preserve privacy in a mathematically rigorous sense.

• Provide documentation and/or a presentation to staff from the Dept of Education.

5. QUALITY CONTROL PLAN

The Contractor shall maintain and utilize a quality control plan which correlates to the Performance Requirements Summary and Quality Assurance Surveillance Plan throughout the life of the contract. The quality control plan must contain, at a minimum:

• A description of the inspection system to cover all services in the QASP. The description must include specifics as to the areas to be inspected on both a scheduled and unscheduled basis, frequency of inspection, and the title and organizational placement of the inspectors.

• A description of the methods used for identifying and preventing defects in the quality of service performed.

• A description of the records to be kept to document inspections and corrective or preventive actions taken.

The records must be kept and made available to the contracting officer, when requested, throughout the contract performance period and for the period after contract completion until final settlement of any claims under this contract.

6. PERIOD OF PERFORMANCE

The period of performance for this award will be one year, beginning TOA, and concluding one year later. There are no option years.

7. PLACE OF PERFORMANCE

A. Work will be performed at the Government’s facility located at 77 K Street, NE, Washington, D.C. or at an approved contractor facility. Work may also be performed via telework on an IRS-issued laptop. All work containing FTI will be completed on IRS-issued equipment.

B. There is no requirement for travel.

8. SECURITY REQUIREMENTS

All contractors must pass a standard background investigation and be issued a contractor badge.

Due to the nature of this job, the contractor will need to access Taxpayer Return Information (TRI), Sensitive but Unclassified (SBU), and Personal Identifying Information (PII). All Access to TRI, SBU, and PII will be done on IRS issued laptops. The contractor must complete all assigned mandatory briefings timely.

To maintain the security if IRS data and files, neither the contractor nor the subcontractor may release any information to any individual outside the Statistics of Income Division without explicit written prior approval.

9. DELIVERABLES AND DUE DATES

See item 12. Deliverables Table below for deliverables and due dates.

Statistics of Income shall have the right to reject or require corrections to a deliverable that does not address the requirements set forth in the description of the deliverables.

10. PROVIDE PROJECT MANAGEMENT

The contractor shall provide project management support. This includes the management and oversight of all activities performed by contractor personnel, including subcontractors, to satisfy the requirements identified in this Performance Work Statement (PWS).

11. COORDINATE A PROJECT KICK-OFF MEETING

The contractor shall schedule, coordinate, and host a Project Kick-Off Meeting at the location approved by the Government. The meeting shall provide an introduction between the contractor personnel and Government personnel who will be involved with the TO. The meeting shall provide the opportunity to discuss technical, management, and security issues, and travel authorization and reporting procedures. At a minimum, the attendees shall include the contractor’s Key Personnel, the Program Manager(s), other relevant Government personnel, the CO, and the COR.

At least one business day prior to the Project Kick-Off Meeting, the contractor shall provide a Project Kick-Off Meeting Agenda for review and approval by the COR and the PM prior to finalizing. The agenda shall include, at a minimum, the following topics/deliverables:

a. Points of Contact (POCs) for all parties.

b. Personnel discussion (i.e., roles and responsibilities and lines of communication between contractor and Government).

c. Project Staffing Plan and status.

d. Financial reporting and invoicing requirements.

The Government will provide the contractor with the number of Government participants for the Project Kick-Off Meeting, and the contractor shall provide copies of the presentation for all present.

The contractor shall draft and provide a Project Kick-Off Meeting Minutes Report documenting the Project Kick-Off Meeting discussion and capturing any action items.

12. DELIVERABLES TABLE

Statistics of Income shall have the right to reject or require corrections to a deliverable that does not address the requirements set forth in the description of the deliverables.

Del.

# Milestone - Deliverable Date of Completion or Delivery Gov't

Rights 1 Project Start NLT 1 business day after TOA N/A 2 Project Kick-Off Meeting Agenda NLT 2 business days after TOA N/A 3 Project Kick-Off Meeting NLT 3 business days after TOA N/A 4 Project Kick-Off Meeting Minutes Report NLT 5 business days after TOA N/A 5 Provide IRS with onboarding paperwork for contractors NLT 9 business days after TOA N/A

6 HCO to complete background investigations and provide Interim Staff Like Access Memo for contractors

NLT 19 business days after

TOA N/A

Provide IRS with Python code to run on a RAAS server that will allow IRS to define public outputs for multiple datasets; visualize the privacy-utility tradeoffs associated with the request; refine the requested tabulations or change the privacy loss budget requested; and to apply a differentially private mechanism customized to the release request and run on the sensitive data to generate the output release. The output will contain the noisy tabulations requested by the analyst as well as error estimates on the noisy values. Use Python code to analyze, aggregate and add noise to aggregates computed from the raw data file comprising information from both ED and IRS.

NLT 30 business days after

TOA N/A

8 Work with IRS to run initial tests on the calculator and to configure and validate the schema on the IRS server.

NLT 32-34 business days after

TOA N/A

9 Provide IRS with excel macros to estimate error rates for user-input breakouts at various epsilon levels.

NLT 33 business days after

TOA N/A

The contractor will assist SOI with data analysis and processing to create a public file for the Department of Education.

NLT 41 business days after

TOA N/A

Contractor will develop a customized interactive evaluation tool in the form of complex Excel Macros for IRS-SOI allowing for adjustments in privacy loss budget parameter (epsilon), workload parameters, and suppression thresholds. This tool will compute counts of cell suppression, and relative and absolute error by different quantiles and statistics to support internal discussions on identifying the total and allocation of privacy budget.

NLT 5 business days before the end of the Period of Performance of the TO

N/A

The contractor will develop a customized interactive evaluation tool in the form of complex Excel Macros for ED allowing for adjustments in workload parameters and suppression thresholds. This tool will compute counts of cell suppression, and relative and absolute error by different quantiles and statistics to support on identifying the workload parameters and/or suppression threshold.

NLT 5 business days before the end of the Period of Performance of the TO

N/A

Contractor will support post-delivery research to further improve privacy-utility balance in future versions of the College Scorecard. Continuing research would include, but not limited to, identifying alternative privatization algorithm that better preserve privacy in a mathematically rigorous sense.

NLT 5 business days before the end of the Period of Performance of the TO

N/A