J-8g-DRB_Cover_Sheet_and_De-Identification_Worksheet.docx

MCC Evaluation Microdata Data Package Instructions This template is informed by MCC’s Evaluation Microdata Documentation and De-Identification Guidelines. In addition to reviewing these Guidelines, MCC contractors responsible for preparation and documentation of evaluation-related microdata for public and/or restricted-access use should be familiar with the following US government guidelines for data de-identification and re-identification:

· NIST 2015 - http://nvlpubs.nist.gov/nistpubs/ir/2015/NIST.IR.8053.pdf

· NIST 2016 - http://csrc.nist.gov/publications/drafts/800-188/sp800_188_draft2.pdf

MCC, the evaluator, and stakeholders should consider the following multi-stage process for data review and release:

1. Evaluator and M&E PM should agree on expected DRB review date as early as possible to confirm. This should be scheduled at least one month before Evaluator’s contract expires.

2. Evaluator should submit full package to M&E PM. The package includes:

· One completed Section 1 of the DRB Data Package Worksheet for ALL data components (i.e. individual, household, and community data for one survey round are three data components with different risks)

· One completed Section 2 & 3 for EACH data component

· Datasets and code package(s)

· Informed consent(s)

· Questionnaire(s)

· Most recent Metadata file (for Evaluation Catalog entry)

3. M&E PM should review Metadata and DRB Data Package Worksheet for clarity and completeness. This may require one round of revision based on the M&E PM requests for clarity and completeness.

4. Evaluator should submit full package to M&E PM. M&E PM and the M&E DRB members should establish a first-round review and feedback to the Evaluator on the proposed data de-identification process. This may require a second round of revision to the package.

5. Evaluator should submit full package to M&E PM for the confirmed MCC DRB review date at least 2 weeks prior to confirmed DRB review date.

6. If any feedback/revisions are required following MCC DRB review, Evaluator should revise and resubmit full package to M&E PM with documented responses to MCC DRB feedback to ensure timely virtual review and clearance of the full package. All final de-identification efforts and their impact on verification of analysis should be documented in the evaluator’s Transparency Statement available on the Evaluation Catalog.

All red font text are instructions in the Worksheet and must be replaced with standard black font with the contractor’s response.

Unless otherwise agreed with MCC, the final document will be made public to complement/underlie the contractor’s Transparency Statement to document the data preparation and de-identification process required for the public and/or restricted-access microdata and any impact on the data for verifying evaluation analysis and broader data usability.

Section 1: Cover Sheet Overview of Data Package (Instructions: Include a paragraph summarizing each data package component included in the package. For example, if the package includes household, individual, and community level data sets, please include a paragraph summarizing each of these three components, including information on the content and timing of the data collection.)

This data package includes the following components:

· Data Package Component 1:

· Data Package Component 2:

Complementary Data (Instructions: Complementary data collection efforts are those efforts that complemented the data packages under review for de-identification, but do not necessarily require de-identification. The evaluator should list these data and provide a brief summary on how they connect to any data package components and affect the data package components’ de-identification. For example, if the geospatial data for the project infrastructure is collected and will be publicly released, it should be listed in the complementary data collection efforts.)

This data package considers the following complementary data efforts:

· Complementary Component 1:

· Complementary Component 2:

Data Package Folder Contents (Instructions: Please list the Data Package Component File Name, and then include the File Names of each of the corresponding required documents [Metadata, Worksheet, Informed Consent, Questionnaire, Other docs]. Only one de-identification worksheet per survey is requested unless discussed.)

Data Package

Data Package Component #1 Name

Data Package Component #2 Name

Data Package Component # Name

Section 2: Data Component Preparation Overview

The available data must allow new users to replicate evaluator analysis to the extent allowable by providing the full data set + analysis code. The constructed variables may also be included in a dataset, but if the dataset+code produces those variables, it is not necessary.

To be considered Incomplete: The available data only provides a sub-section of data as produced by the survey and/or the constructed variables only. Incomplete data files are limited in terms of full verification of analysis and/or broad usability of data and must be justified.

Incomplete

If one stage only – please (i) confirm demand and/or contractual justification and (ii) discuss how preparation and release of this data as presented to the DRB may affect future data round releases.

If combination, please discuss if this file replaces any previously published datasets.

Interim only

Endline only

Combination of rounds

Please discuss how the promises of confidentiality in the informed consent informed de-identification efforts. Please include any additional guidance provided by the IRB as applicable.

Medium restriction

Low restriction

Medium risk

Low risk

Publication Type

Restricted-use only

Both

COUNTRY & EVALUATION NAME

CONTRACTOR NAME

Prepared on: DATE

Section 3: Data Component Preparation Details

Describe how to mitigate link to existing data that enables re-identification

List all DIRECT identifiers removed from the dataset.

Describe process for de-identification. For example: introduce random errors into geographic data (GPS, GIS, etc.).

Displace urban points 0-2 km, rural points 0-5 km, and additional 1% of rural points 0-10 km[footnoteRef:3]. [3: ICF International, Demographic & Health Surveys]

Describe top/bottom coding: set upper & lower bounds to remove outliers for continuous. Specify: are values set to the median, or other?

For large categories/datasets, the OMB suggests top coding at least the highest .5%; for smaller categories/datasets, top code the highest 3-5%. The same principles apply to bottom coding.[footnoteRef:4] [4: Office of Management and Budget, Checklist on Disclosure Potential of Proposed Data Releases (current link)]

Describe any variables that require collapse and describe construction of new variable

Describe any global re-coding to group observations into categories (e.g., age 0-5, 5-10, 65+, etc.). Ensure that the categories are neither too broad nor too narrow.

For each identified rare data, describe the local suppression techniques employed to mitigate the identification risk of unique and rare observations. Specify: are values set to missing, the median, or other?[footnoteRef:5] [5: To preserve the analytic value of rare data, MCC generally recommends replacing outlier values of continuous variables with the outlying group’s median value – e.g., outliers in the 99th income percentile are replaced with the median of that quantile. And grouping rare categorical values with analytically similar categories (if meaningful similarities exist) or grouping them with other rare categories.]

(See [Footnote] for MCC’s general guidance; evaluators should either confirm that that this guidance is appropriate and was used, or explain the alternate method(s) used and why.)