CFPB RFI - PII Discovery and Redaction.pdf

CONSUMER FINANCIAL PROTECTION BUREAU

Request for Information Regarding Available Products and Services for PII Discovery and Redaction

Agency: Consumer Financial Protection Bureau

Action: Request for Information (RFI)

Background:

The Consumer Financial Protection Bureau (CFPB or the Bureau) is a 21st century agency that implements and enforces Federal consumer financial law and ensures that markets for consumer financial products are fair, transparent, and competitive. As part of its work, CFPB receives consumer complaints about consumer financial products and services. Complaints are submitted by the public through CFPB’s Salesforce Customer Relationship Management (CRM) system. Complaints that the CFPB sends to companies for response are published in the public-facing Consumer Complaint Database after the company responds, confirming a commercial relationship with the consumer, or after 15 days, whichever comes first. The CFPB sends more than 32,000 complaints to companies each week. With their consent and after taking steps to remove Personally Identifiable Information (PII), the CFPB publishes consumers’ complaint narratives. The narrative of these complaints is a key component because it allows the consumer to describe their experience in their own words, adding details and context that is useful for the public to know.

Before publishing consumer complaint narratives, the CFPB takes steps to remove personal information.

Complaint narratives are reviewed to identify and redact PII, as well as identify other types of sensitive information and other information within the complaint narratives such as content written in foreign languages and profane language that does not meet the publication standard. The current process involves three steps:

1) A custom algorithm outside of the CRM detects and redacts PII and certain other information in complaint narratives using natural language processing (NLP) and Python.

2) A trained human reviewer reads the narrative within the CRM to redact any remaining PII in the data and correct false positives from step 1 and the narrative may also be reviewed by another trained human reviewer.

3) Separate final algorithm outside of the CRM does a last review of the data to ensure PII is detected and redacted.

Since complaints are naturally written narratives submitted by a diverse population of consumers, the text does not adhere to standard formatting or writing conventions. Narratives average several sentences in length but can range from one sentence to several paragraphs, with the writer deciding how much and what kind of information to include. While most complaints are written in normal sentence case with standard capitalization and punctuation, some narratives may not follow those rules, such as being written in all capital letters or containing grammar and spelling mistakes. The accepted character set is Unicode utf-8, and reviewed complaints are limited to the English language.

Below is a hypothetical example of a complaint narrative, presented first as an original submission from the consumer before being redacted per the CFPB’s Narrative Scrubbing Standard1 as shown in the second narrative. All PII in this example is fictitious.

Original Narrative I received nine separate emails from MoneyBank saying my mortgage payment was not received.

I then received an email that my mortgage was current, as I did make my December payment. But after receiving an apology email from MoneyBank at 10:32am today, I then received another email (the 10th email) at 11:45am notifying me my December mortgage payment was not received. Even though I have confirmation from my bank that the payment did post on 01/15/2024. I attempted to call MoneyBank but encountered long hold times so I didn't have time to stay on the phone.

Redacted Narrative I received nine separate emails from MoneyBank saying my mortgage payment was not received.

I then received an email that my mortgage was current, as I did make my XXXX payment. But after receiving an apology email from MoneyBank at XXXX XXXX today, I then received another email (the 10th email) at XXXX XXXX notifying me my XXXX mortgage payment was not received. Even though I have confirmation from my bank that the payment did post on XX/XX/2024. I attempted to call MoneyBank but encountered long hold times so I didn't have time to stay on the phone.

Purpose of RFI:

The CFPB is looking for information on industry’s capabilities to provide a solution for PII detection and redaction that would replace the existing custom algorithm and eliminate or significantly reduce the need for manual reviews. The goal of this RFI is to solicit information that will help CFPB with planning for a future acquisition, primarily by understanding available solutions and the estimated costs that potentially meet the agency’s requirements.

Statement of Objectives:

The future solution would serve as the CFPB’s primary method of detecting and redacting PII, sensitive data, and certain other data in consumer-submitted complaint narratives received by the CFPB. The solution would replace the custom algorithm in the current detection process described above (step 1), and it would ideally limit or even eliminate the need for human review per step 2 and a separate algorithm per step 3.

1 For more information about how redacted consumer complaints are published, visit https://www.consumerfinance.gov/data-research/consumer-complaints/

For purposes of this RFI, the CFPB’s definition of PII includes both “direct identifiers” such as an individual’s name, address, and social security, as well as “indirect identifiers” such as medical conditions, expenditure amounts, and multiple dates. To reduce re-identification risk, CFPB requires the capability to process and accurately discover all direct and indirect identifiers present in complaint narratives.

Furthermore, it is important that CFPB keeps each narrative true to the writer’s experience and ensures the public can understand the narrative after PII is removed. Therefore, in addition to precisely identifying all PII present within the narratives, CFPB also requires that its future solution sufficiently avoids over-redaction of information that can be mistakenly identified as PII. Over-redaction negatively impacts the readability of consumer complaint narratives, requiring human review to undo the redaction of words prior to publishing.

RFI Questions

1. What solution(s) would you recommend to CFPB to meet the objectives stated above? Please also be sure to explain the nature of the solution such as whether it is Software as a Service (SaaS), Commercial Off the Shelf (COTS) software, a custom solution, or other, and indicate if the solution is Federal Risk and Authorization Management Program (FedRAMP) authorized.

2. Describe the contract and cost breakdown structure of the recommended solution(s), identifying types of costs associated with the solutions and variables that contribute to the costs to acquire, operate, and maintain the solution.

For each relevant variable, please insert an estimated cost range into the table below (assuming a volume of 32,000 complaints per week) and include the table in your response. More precise estimates are welcome, if available.

Cost Variables Estimated Cost

Purchase price

Per-transaction

SaaS

Hardware

Licensing fees

Labor costs

Custom development

Operations & maintenance costs

Other (please insert)

Note: For all remaining questions, please be sure to explain whether any claimed capabilities will carry significant cost implications for acquiring, operating, or maintaining the solution overtime.

3. Describe the solution’s approach to accurately identifying direct PII and indirect PII within a dataset. If possible, please provide the precision and recall rates (or confusion matrices) of any algorithmic solution, and other relevant performance indicators (i.e., ROC, RMSE, F1).

4. Describe the solution’s approach to minimize false positive results (aka, over-redaction) while maintaining overall accuracy. If the solution includes out-of-the-box capabilities for algorithmic tuning or incorporating custom PII definitions, please identify these in your response.

5. Describe any post-processing, output, or algorithmic artifacts the solution produces. For example, does the solution output redacted data, track changes in a data log, report on processing results, provide risk scoring, or offer any additional analytics or workflow functions?

6. Describe the capability of the solution to scale as future data volumes increase.

7. Describe the infrastructure and hardware requirements for implementing and operating the solution.

8. What information does the CFPB need to provide to support accurate proposals (e.g., cost, performance, schedule) for a future solicitation?

Submission Instructions

If your organization has the requisite expertise and can provide information, services, or products to address the above questions, please submit a capabilities statement in accordance with the instructions below. The capabilities statement should also address the following:

1. Vendor’s name, address, Unique Entity Identifier (UEI), SAM.gov registration status (e.g. Active / Inactive / Unregistered), point of contact, phone number, and e-mail address;

2. Vendor’s existing federal contract vehicles (e.g. GSA Schedule(s), NASA SEWP, NIH CIO- SP3/4, etc.) and/or relevant information pertaining to reseller partners or alternative methods of solution procurement;

3. A description of your firm’s expertise to perform the functions described herein, including examples of relevant experience addressing similar requirements for other clients (examples of federal clients preferred) to those described herein. If possible, provide details such as a point of contact, contract value, period of performance, and a description of services provided.

4. Responses to the questions listed in the Request for Information section;

5. Vendor’s small-business size status, including whether the vendor is a woman-owned, minority-owned, or Service-Disabled Veteran-Owned Small Business (SDVOSB)

The written response to the RFI questions should not exceed 10 pages, not including any relevant attachments. Please be advised that all submissions become government property and will not be returned.

Interested parties or vendors are asked to respond no later than February 28, 2024, at 5:00 PM EDT.

Please email responses to Nicholas Olson at Nicholas.Olson@cfpb.gov. All information in response to this RFI must be sent via email in Adobe Acrobat (.pdf) or Microsoft Word format.

Please Note: The information provided in the RFI is subject to change and is not binding on part of the Government to purchase services or issue a solicitation(s) for such services. The Government will not pay for any information or administrated costs associated with the preparation of responses, and any costs incurred will be solely at the respondents’ expense. Any information provided to the Government in response to this RFI will become the property of the Government, and, in addition, may be subject to disclosure under a Freedom of Information Act (FOIA) request.

mailto:Nicholas.Olson@cfpb.gov