Attachment J-5 EDMAPS_pMDM_EDWA_FTI DM Business Requirements 113023.pdf

Version 2.0 1 11/30/2023

Controlled Unclassified Information (CUI)

Enterprise Data Management & Analytics Platform Business Requirements

Version 2.0 ● 11/30/2023

EDMAPS Business Requirements Overview

Version 2.0 2 11/30/2023

Controlled Unclassified Information (CUI)

Table of Contents

SECTION 1. OVERVIEW

SECTION 2. EDMAPS CURRENT STATE

2.1. PMDM

2.2. EDWA

2.2.1. Data Acquisition and Integration

2.2.2. Data storage and organization

2.2.3. Data Access, Analysis, and Distribution

2.2.4. Data Processing and Analysis

2.3. DEBT RELIEF, PCA, PHEAA

2.4. DATA LAKE

2.4.1. Enterprise Central Document Repository

2.4.2. EDMAPS Metadata

2.5. FTI DATA MART SUPPORT

2.6. EDMAPS-WIDE GENERAL REQUIREMENTS

2.6.1. Improvements to EDMAPS’ Current State

2.6.2. Data Quality Management

SECTION 3. DEFERRED REQUIREMENTS – DATA MAPPING

SECTION 4. EDMAPS NEW DEVELOPMENT

EDMAPS Business Requirements Overview

Version 2.0 3 11/30/2023

Controlled Unclassified Information (CUI)

Section 1. Overview Federal Student Aid (FSA), an office of the U.S. Department of Education, provides loans and grants (student aid) to college students for higher education. Accepting and processing loan and grant applications, originating, disbursing, and servicing are handled by a set of information systems both internally managed as well outsourced.

However, because these systems were purpose-built for a specific set of business functions, the data within them is also specifically for their needs, with their own formats, meanings, and processing rules. This presents a problem when FSA teams need integrated data, and comprehensive views and reports looking across systems.

To solve this, the Enterprise data Office (EDO) developed a unified data platform called EDMAPS (Enterprise Data Management and Analytics Platform Services), with a grand vision of incrementally centralizing, integrating, and managing student aid, customer, school, and partner data. When this vision is fully realized, FSA will have improved data consistency and quality, the capability to deliver reports and views to even more teams within FSA, the ability to streamline fulfilling data requests, and capability to acquire and make available more datasets to enhance machine learning. The new EDMAPS will also have enhanced security coming from centralizing access, reporting, and logging.

The Contractor shall propose their technical solution to EDMAPS requirements in two volumes:

1. Volume 1: Current-state Operations and Maintenance – the Contractor shall propose their service model for operation and maintenance of EDMAPS and shall include Information Technology Infrastructure Library (ITIL) based processes, staffing model, and integration with FSA’s IT systems’ change, release, Software Development Life Cycle (SDLC), and Lifecycle Management Methodology (LMM) processes. In this volume, the Contractor shall propose industry best practices for cloud-based operations and maintenance.

2. Volume 2: New Capabilities Development – FSA requires new system development and enhancements from time to time. The Contractor shall be required to analyze these requirements for impact on other FSA systems, infrastructure, operations and maintenance, and cost. The Contractor shall also propose a technical solution, budget, a project manager, and schedule to meet those requirements. The Contractor shall also ensure any new development or enhancement aligns with FSA’s Logical Data Architecture (refer to Data Strategy Plan, attachment 7). In this volume, the Contractor shall propose best practices and methodologies for managing and delivering new systems development or enhancement requirements from

FSA.

EDMAPS Business Requirements EDMAPS Current State

Version 2.0 4 11/30/2023

Controlled Unclassified Information (CUI)

Section 2. EDMAPS Current State EDMAPS is FSA’s highly successful unified data platform, with an increasing user base and growth in data. As shown in the figure below, EDMAPS acquires, integrates, and stores data from various sources and makes it available to users and systems.

This overview information is provided below to give the Contractor a sense for the data sources, flows, structures, architecture, tools, and functionality, which, in effect, are our current-state requirements. The Contractor shall use this information to understand and develop an innovative, efficient, and reliably secure technical proposal.

EDMAPS Current-state Overview

EDMAPS houses two mission-critical data systems:

1. Person Master Data Management (PMDM), a database system of master or golden records of persons

(customers) used by FSA’s systems. PMDM is described in more detail in its own section below.

2. Enterprise Data Warehouse and Analytics (EDWA), a comprehensive data warehouse of customers, loan and grants, schools, and partners used for analytics, reporting, fulfilling data requests, statistical and pattern-based predictive modeling, and machine learning. EDWA is also described in more detail in its own section below.

There are two standalone data repositories:

1. Debt Relief, a database of applicants who applied for financial aid debt relief.

2. Private Collection Agency (PCA), a database of private collection agency data intended to be used for analytics and archiving.

EDMAPS’ support components are:

Version 2.0 5 11/30/2023

Controlled Unclassified Information (CUI)

1. Data Lake, currently used to capture incoming data files within its Raw, Archive, and Consumer zones, for storing inactive data, which currently stores historical servicer data. The data lake also hosts an enterprise document repository.

2. EDMAPS Metadata, a cataloging tool storing EDMAPS’ data dictionary, business glossary, and data lineage.

3. Reports Views, this layer has delivered reports, views, dashboards, and visualizations for use by FSA’s departmental and EDO teams.

4. Report Tools, EDMAPS’ users utilize Cognos, SAS, and Tableau for analytics, reports, statistical analysis, and visualizations. Python and R are used for research, modeling, and machine learning. Alation is used for query storage and collaboration.

2.1. PMDM

PMDM is a mission-critical data system, containing a database of “master” records of customer basic information sourced from other FSA systems. The types of customer data mastered and maintained within PMDM include customer identifiers, name, address, phone, mobile phone, email, demographic, preferred contact method, employer information, and attributes. This data is then provided to requesting systems on either a real-time basis for individual records, or batch basis for groups of records. The figure below shows PMDM’s data sources, flows, and components.

PMDM Current-state Overview Data acquisition and integration. PMDM receives person data from Common Origination & Disbursement (COD), Person Authentication System (PAS), and National Student Loan Data System (NSLDS) at various intervals. Batch data enters the PMDM system into the File Inbound Area where it is quality-checked and filtered to allow only new data. Each person record is then sent to the Web Services engine using hierarchy rules to allow only the best possible record into PMDM. The same Web Services engine is used to acquire real-time data from the Digital Customer Care (DCC) application – again using the hierarchy rules.

Version 2.0 6 11/30/2023

Controlled Unclassified Information (CUI)

The Web Services engine is primarily an Application Programming Interface (API) exchanging data between PMDM and other internal systems, but also includes the hierarchy rules to determine which source and fields of a Person's basic data is considers as "master" data.

COD sends their person data to PMDM batched on an hourly basis. PAS sends their person data in real-time.

Student Aid Internet Gateway (SAIG) Mailbox servicer data file, which contains person data, is picked up on a weekly basis.

The data in transit uses the latest Chief Information Security Office (CISO) mandated Secure Socket Layer (SSL) / Transport Layer Security (TLS) for encryption.

Data storage and organization. PMDM is on an Oracle database engine, organized in the 3rd normal form relational structure. The database is approximately 1.5TB with about 500M active customer records. The database also stores inactive records either demoted or maintained for auditing purposes.

PMDM does not use archiving.

Data access and distribution. The most active real-time user of PMDM’s data is Digital Customer Care (DCC) Digital (studentaid.gov). DCC Digital is customer-facing and often the first point of entry for those who want to learn about or apply for student aid. Customers are also encouraged to visit DCC Digital often to check on the status of their application and to update their contact information and contact preferences. Information is entered into PMDM directly and in real time through the Web Services component.

DCC Digital also presents a settings and preferences page to customers when they first log in, so they can verify or update their contact information and preferences. This functionality works by DCC Digital requesting the customer’s record from PMDM, using their login information. When the customer either verifies their information or updates it, DCC Digital sends data to PMDM.

PMDM never deletes data. It only makes the best customer record or field “active” meaning it is the master record.

The non-master records and fields are demoted to “inactive” status but maintained in the database for audits and reviews.

Other systems including Marketing and Communication, Customer Relationship Management (CRM), Marketing Campaign, Partner Connect, Medallia, and Virtual Assistant request batch data on an ad-hoc basis to get the person master data from PMDM. EDWA receives person records daily from PMDM.

There were over 500M real-time service calls to PMDM to create, read, or update it. PMDM has processed approximately 400M customer records and produced over 150M master records. More than 50K master records are added each week and more than 300K such records are added during the peak month of October each year.

There are no end-users who have access to the PMDM database or Web Services.

2.2. EDWA

EDWA is FSA’s data warehouse containing customer, student aid, school, and partner data. It is a mission-critical system.

Version 2.0 7 11/30/2023

Controlled Unclassified Information (CUI)

2.2.1. Data Acquisition and Integration

EDWA receives customer data from PAS and PMDM daily, student aid application data from Central Processing System (CPS) and FAFSA Processing System (FPS) daily, student aid and school data from COD daily, and partner data from Partner Connect and Ez-Audit daily. It receives enrollment data from NSLDS. It also receives loan servicing data from the SAIG Mailbox where servicers place their data files daily. Incoming data is processed based on business rules specific to each data source to integrate them with EDWA. Various methods and tools are used for replication, such as Apache Kafka tool, which is utilized and managed by Confluent for data exchanges between internal systems.

The data in transit is encrypted using the latest CISO-mandated SSL/TLS for encryption.

The system shall provide mechanisms to integrate and ingest data from a variety of sources. This can include support for Extract Transform Load (ETL) processes, as well as external tables to access data stored in the Amazon Web Service (AWS) data lake. The system shall allow for the creation and management of data integration and ingestion processes within the database.

2.2.1.1.Address Cleansing and Standardization

The Contractor shall use reliable and efficient address cleansing tools to clean and standardize addresses in the database.

Version 2.0 8 11/30/2023

Controlled Unclassified Information (CUI)

Smarty Streets is currently being used for address cleansing, but the Contractor is free to use other address cleansing tools if they can provide reliable results.

The Contractor shall also ensure latitude and longitude information and congressional district data is available for each address in the database.

The address cleansing process shall be automated and integrated with other processes using the mailing addresses in the database.

The Contractor shall also provide regular reports on the effectiveness of the address cleansing process and any issues identified and addressed.

2.2.1.2.Phone Number Validation Service The Contractor shall support:

Retrieving information about a phone number in real time internal to EDMAPS system boundary.

Providing phone number validation, which can confirm whether a phone number is valid and whether it's mobile or landline. Determine the carrier associated with the phone number, as well as the phone number's country and geographic region.

Providing real-time phone number insights, which include things like the device type, communication method preferences, and more.

2.2.2. Data storage and organization

EDWA’s database runs on Greenplum, a massive parallel processing database engine. Its data structure is atypical for a data warehouse, it uses a 3rd normal form relational model, instead of a star or a snowflake schema. With some exceptions, this design has worked successfully. The EDWA team wants to keep both Greenplum and the current data structure in place due to code and query dependencies but is open to discussions about better database platforms and schema designs.

In addition to the core database, EDWA also includes a separate user analytics schema where authorized users can create their own tables for queries.

There is also an Elastic Compute Cloud (EC2) instance referred to as the “analytics platform” with views used for advanced analytics, dashboards, general reporting, statistical methods, leveraging Python, R, Jupyter, AWS Sagemaker, and other statistical software. This is also where the FSA team creates statistical and machine-learning models in the Development environment for use in Production. The Contractor shall support the analytics platform for the FSA team by providing data science Subject Matter Experts (SME) to assist in the development of statistical modeling.

The Personally Identifiable Information (PII) in the EDWA database is tokenized using Protegrity.

EDWA contains about 90TB of data increasing by 1TB per quarter from about 124 different file formats from FSA Financial Partners (e.g., Services, Guaranty Agencies), Education Partners (e.g., Schools), Department of Education, Internal Revenue Service (IRS), more than seven FSA internal systems, commercial data (e.g., `Smarty Streets for data cleansing), and other ad hoc data as needed to fulfill data requests. Approximately 32 of the about 124 file formats contain PII.

EDWA does not archive any data, so all data is on the active database.

2.2.3. Data Access, Analysis, and Distribution

There are a variety of reports, dashboards, and views available to FSA’s departmental users. There are two teams doing mission-critical work on EDWA.

Version 2.0 9 11/30/2023

Controlled Unclassified Information (CUI)

2.2.3.1.The Data Request Team This team fulfills data requests from congressional members, government agencies, and internal business leaders.

On an annual basis, the team fulfills over 1,000 requests by querying EDWA and corroborating data with NSLDS.

This team also publishes FSA data for public consumption on the FSA Data Center website.

The Contractor shall propose a total number of service hours to fulfill the 1,000 or so data requests per annum, support for data governance (metadata and business glossary management), and support for publishing open data to the FSA Data Center website. The total number of proposed service hours shall be a “bucket” of hours that can be used by FSA to draw down from during the contract period. The Contractor shall note that these service hours are not for use in standard operations and maintenance activities of EDMAPS, which include infrastructure, data interfaces, application management, user access management, security services, among other services mentioned in this document as well as in the Minimum Information Technology Contract Requirements (MITCR) document.

2.2.3.2.The Research and Modeling Team This team develops statistical modeling and pattern-based machine learning to predict fraud and defaults and prevent overpayments and wrongful payments. The team uses Python for machine learning, statistical analysis, and modeling, and Alation to manage and collaborate on their queries. This team has full access to all aspects of EDWA, including the core database.

2.2.3.3.Advanced Analytics and Machine Learning:

The system shall support the deployment and management of advanced analytics and machine learning models, including the ability to integrate with third-party tools and libraries. The system shall also provide mechanisms for testing, validation, and monitoring of models to ensure accuracy and performance.

a. Virtual Machine Requirements Users shall have access to initiate a virtual machine of appropriate size and specifications for a given task, with the necessary software to accomplish advanced analytics projects.

The virtual machines shall be cloud-based and accessible remotely from anywhere in the United

States with an internet connection.

The virtual machines shall have the ability to establish a connection to the database and submit queries using Python or other programming languages.

The virtual machines shall have the ability to establish a connection to the data lake and retrieve objects using Python and other programming languages.

The Contractor shall ensure virtual machines are secure and follow best practices for data privacy and security.

The Contractor shall provide detailed documentation on how to access and configure the virtual machines.

The virtual machines shall be regularly monitored for usage and performance metrics, with alerts set up for any anomalies. Controls shall be put into place on virtual machine usage to control and monitor costs, ensuring yearly budget for virtual machine usage is efficiently allocated.

The Contractor shall provide support for troubleshooting and resolving any issues with the virtual machines.

The virtual machine shall have the necessary hardware and software to support advanced analytics and machine learning workloads, including Central Processing Unit (CPU), memory, and storage resources appropriate for the task at hand.

The virtual machine shall include a standard Anaconda installation of Python, as well as other software required for audio transcription, image processing, or other advanced modeling projects.

A process for installing software not already approved or available shall be established. It shall take less than a week to complete the process and less than two weeks to get a new software installed.

Version 2.0 10 11/30/2023

Controlled Unclassified Information (CUI)

b. Model Deployment:

The Contractor shall be able to take a model or process developed by FSA using a virtual machine and/or database and deploy it in a production environment.

There shall be a "production" virtual machine available for deploying models and running processes in operations, matching the software specifications used in developing the model and is of appropriate size to complete the task designed for in production.

The production virtual machine or other production IT asset shall have the necessary hardware and software to support the specific requirements of the deployed models, including scheduling of jobs to run in the production environment.

c. Additional Machine Learning Operations (ML Ops):

The Contractor shall have a defined process for deploying, monitoring, and updating machine learning models in production.

The Contractor shall ensure the machine learning models are deployed in a secure and scalable manner.

The Contractor shall provide a version control system for the machine learning models and their associated code.

The Contractor shall conduct performance monitoring of the machine learning models in production, including monitoring of data input/output, computational resources, and model performance metrics.

The Contractor shall ensure all code and data associated with the machine learning models are appropriately documented, versioned, and backed up.

The Contractor shall provide training and documentation to the team on ML Ops processes, tools, and best practices.

The Contractor shall have a plan for handling model failures, including escalation and notification procedures.

The Contractor shall have a process for regularly retraining and updating the machine learning models to ensure they remain accurate and relevant.

The Contractor shall work collaboratively with the team to understand business requirements and ensure the machine learning models align with the business objectives.

d. Ad Hoc Research and Analytics:

The Contractor shall have experience with specialized areas of advanced analytics, such as algorithmic fairness, and be able to provide guidance and support to our research teams as needed.

The Contractor shall know about emerging trends and techniques in specialized areas of advanced analytics and provide updates and recommendations to our teams on how these trends and techniques can be applied to our work.

The Contractor shall be able to provide technical training to our research teams on specialized areas of advanced analytics, including best practices, coding conventions, and other relevant topics.

The Contractor shall have experience with ethical considerations and regulatory compliance related to specialized areas of advanced analytics and provide guidance and support to our teams on these issues.

The Contractor shall be able to collaborate effectively with our research teams and be responsive to requests for assistance and support.

e. Remote Desktop:

Version 2.0 11 11/30/2023

Controlled Unclassified Information (CUI)

The remote desktop shall be accessible to authorized users for specialized access to the data warehouse.

The remote desktop shall have access to all other IT assets, including the database, virtual machines, analytics tools, reporting and visualization tools, and other software used by the organization.

Certain software tools are easier to install on the remote desktop, or require additional security controls, shall be available on it.

The remote desktop shall comply with all security policies and protocols in place.

f. Third party software proof of concept and implementation support:

Ability to integrate the third-party software with the data warehouse and other systems.

Ability to troubleshoot and resolve any issues arising during the proof of concept.

Ability to provide recommendations and guidance on best practices for using the third-party software with EDWA.

Strong communication skills to collaborate with the third-party software provider and internal stakeholders to ensure successful execution of the proof of concept.

g. Reporting, Visualizations, and Dashboards:

The Contractor shall provide a web-based reporting and visualization tool allowing authorized users to access and create reports and visualizations based on the data in the data warehouse.

The tool shall be able to connect to the Greenplum database and other data sources as needed and shall support the creation of reports and visualizations in a variety of formats, including tables, charts, graphs, and maps.

The tool shall support user-defined dashboards customized based on the specific needs of different groups within the organization.

The tool shall allow for the scheduling and automation of report generation and delivery to specific users or groups.

The tool shall support the import and integration of existing reports and visualizations from COGNOS and Tableau.

The Contractor shall provide training and support to users to ensure they are able to effectively use the reporting and visualization tool.

The Contractor shall provide ongoing maintenance and support for the reporting and visualization tool, including upgrades and bug fixes as needed.

The Contractor shall administer the reporting and visualization tool using best practices.

The Contractor shall have the expertise to create advanced dashboards visualizations using the tool meeting user requirements.

h. Data catalog and querying tool: Alation is currently used and acceptable for this requirement.

Hardware/Software requirements:

The tool shall be web-based and accessible from multiple devices and browsers.

It shall be able to integrate with our existing infrastructure and tools, including the database.

It shall be able to handle large volumes of data and metadata.

i. Access requirements:

Users shall be able to easily search for and discover data assets, queries, and other documentation.

Users shall be able to see metadata such as data lineage, data quality, and data owners.

Users shall be able to see and execute Structured Query Language (SQL) queries within the tool.

Version 2.0 12 11/30/2023

Controlled Unclassified Information (CUI)

Other requirements:

• The tool shall have a user-friendly interface easy to use and navigate.

• The tool shall support collaboration and sharing among users.

• The tool shall be able to generate reports and dashboards for data usage and activity.

FSA’s departmental users have access to the data through Cognos and statistical tools such as Statistical Analysis System (SAS). All user access is controlled through an access control process. This process, however, needs improvements.

Functionality. As a data warehouse, EDWA is used for analytics and gaining insight into student aid by a variety of FSA teams. It is also used for fulfilling data requests, and for statistical and predictive modeling and model deployment as indicated above.

2.2.4. Data Processing and Analysis

The Contractor shall provide tools and capabilities for processing and analyzing data, such as data mining, statistical analysis, machine learning, and predictive modeling. The system shall also support data visualization and reporting to facilitate insights and decision-making.

2.2.4.1.Graph Analytics The Contractor shall provide mechanisms to support graph analytics at scale. The Greenplum database with MADlib installed can be used as a graph database, which should be leveraged to support these use cases. The system shall allow for the creation and management of graph data structures within the database, as well as the ability to perform graph analytics queries on this data.

2.2.4.2.Graph Fuzzy String Matching The Contractor shall provide mechanisms to perform fuzzy string matching. This can be accomplished using string similarity algorithms and techniques such as Levenshtein distance. The system shall allow for the creation and management of fuzzy string-matching queries within the database.

2.2.4.3.Geospatial Analytics The Contractor shall provide mechanisms to perform geospatial analytics. This can include support for spatial data types, indexing of spatial data, and spatial queries. The system shall allow for the creation and management of geospatial analytics queries within the database.

2.2.4.4.Advanced Analytics The Contractor shall provide mechanisms to support advanced analytics features offered by Greenplum. This can include support for machine learning algorithms, statistical functions, and data visualization tools. The system shall allow for the creation and management of advanced analytics queries within the database.

2.3. Debt Relief, PCA, PHEAA

Debt Relief is a standalone repository. The application code is on hold, but the application information is available.

Private collection agencies (PCA) were contracted by FSA to recover money owed to loan programs when customers defaulted. PCAs are no longer used, but FSA was required to collect and archive their data for analytics and audit reviews. The PCA repository is standalone, with no users or data outflows to systems.

In addition to PCA data, FSA has collected servicing data from servicers no longer active, such as Pennsylvania Higher Education Assistance Agency (PHEAA).

Version 2.0 13 11/30/2023

Controlled Unclassified Information (CUI)

2.4. Data Lake

The data lake is on AWS’s Simple Storage Service (S3) storage and has two zones: the “Raw” zone, which has a directory structure to place incoming data files from various systems; an “Archive” zone, which currently stores historical data; a “Consumer” zone which holds user-created data files; and the “Document Repository” zone (see below for description), which securely holds digitized versions of paper applications for specialty programs, and loan servicing-related documents. The data lake holds about 144TB of data.

2.4.1. Enterprise Central Document Repository

1. Document Ingestion:

The system shall be able to receive and accept multiple filetypes of unstructured documents, such as Portable Document Format (PDF), Word, Excel, PowerPoint, plain text, image files, and audio files.

The repository shall support bulk document ingestion to facilitate efficient loading of large volumes of documents.

The system shall ensure secure transmission and storage of documents, maintaining data integrity and confidentiality.

2. Document Storage:

The repository shall provide a scalable and reliable storage infrastructure to accommodate many documents.

Documents shall be stored in a structured manner, with appropriate metadata associated with each document, allowing for efficient indexing and retrieval.

The system shall ensure redundancy and backup mechanisms to prevent data loss in case of failures.

3. Document Indexing:

The repository shall support robust indexing capabilities to enable efficient retrieval of documents.

The indexing system shall extract relevant metadata from documents, such as title, author, date, and keywords, to facilitate search and filtering.

Full-text indexing shall be implemented to enable searching within the content of documents, including text extraction from images and scanned documents.

The indexing mechanism shall support both manual and automatic indexing approaches, allowing users to tag documents with additional metadata.

4. Document Search and Retrieval:

The system shall provide a user-friendly search interface to allow individuals and systems to search for documents based on various criteria.

Users shall be able to perform keyword searches, advanced searches (e.g., based on specific metadata fields), and filter results based on document attributes.

The search functionality shall support Boolean operators, wildcards, and fuzzy matching to enhance the accuracy and flexibility of search queries.

The repository shall offer fast and responsive search results, even when dealing with many documents.

5. Access Control and Security:

The repository shall implement robust access control mechanisms to ensure that only authorized individuals or systems can access and modify documents.

Version 2.0 14 11/30/2023

Controlled Unclassified Information (CUI)

Different levels of access privileges shall be supported, allowing for role-based access control and granular permission settings.

The system shall provide encryption of stored documents and during data transmission to maintain confidentiality.

Audit trails and activity logs shall be maintained to track user actions and detect any unauthorized access attempts.

6. Integration and Interoperability:

The repository shall offer APIs and integration options to enable seamless integration with other systems, such as content management systems (CMS) or enterprise resource planning (ERP) systems.

The system shall support common standards for document exchange, such as eXtensible Markup Language (XML) or JavaScript Object Notation (JSON), to facilitate interoperability with external systems.

Integration with external search engines or enterprise search platforms shall be possible to enable unified document search across different repositories.

7. Scalability and Performance:

The repository shall be designed to handle many documents and concurrent user requests efficiently.

The system shall scale horizontally to accommodate increasing storage and processing requirements over time.

Performance optimizations, such as caching, load balancing, and query optimization, shall be implemented to ensure fast response times and minimize latency.

8. Metadata Management:

The repository shall allow flexible management of document metadata, including the ability to define custom metadata fields or attributes.

Metadata shall be easily editable and updatable, both manually and through automated processes.

The system shall provide metadata validation and consistency checks to maintain data integrity and accuracy.

9. Version Control and Document History:

The repository shall support versioning of documents to track changes over time and preserve historical versions.

Document versioning shall allow users to compare different versions and revert to previous versions.

2.4.2. EDMAPS Metadata

EDMAPS’ metadata – data dictionary, business glossary, data lineage, and business rules at the field level are captured with IBM’s Information Governance Catalog (IGC) tool. This information is currently limited to systems within EDMAPS, not all of FSA’s systems. The EDMAPS team has two goals to improve usage of metadata within business teams and our data governance board:

Incrementally capture metadata for all of FSA’s information systems.

Move metadata to a different tool easier for a broader audience to view and update. Our team is open to considering proposals for a better tool.

Version 2.0 15 11/30/2023

Controlled Unclassified Information (CUI)

2.5. FTI Data Mart Support

The Federal Tax Information (FTI) Data Mart is hosted at FSA’s Next Generation Data Center (NGDC), which is managed by FSA’s Technology Directorate. While EDMAPS does not host the Mart, we need support services from the Contractor to operate and maintain it. These include managing the Mart’s database to smoothly acquire data from two primary sources: the FTI Module, which sends FTI and PII data, and EDMAPS, which sends customer, loan, and related historical data. This Mart is used for analytics, visualizations, dashboarding, and reporting by a limited number of analysts.

2.6. EDMAPS-wide General Requirements

EDMAPS is on AWS. Our team is satisfied with its performance. However, Contractors are encouraged to propose any major FedRAMP-compliant cloud platform but must show the benefit of moving away from AWS, as well as an approach not disrupting EDMAPS’ operations, functionality, and users if a different platform is proposed.

EDMAPS full functionality must be implemented, tested, and deployed within the Contractor’s technical environment on the first day of operations, which is expected to follow a brief discovery and requirements confirmation period. The Contractor shall integrate with FSA’s established processes.

2.6.1. Improvements to EDMAPS’ Current State

There are important areas of improvements the EDMAPS team would like to see in the

Contractor’s proposal for improving the current state:

Implement Commercial Off the Shelf (COTS) to replace the custom code in as many areas of

EDMAPS as possible. The EDMAPS team believes with COTS, the time and cost to deliver on changes and new development will be reduced. The Contractor’s proposal shall describe how their proposed COTS tools will reduce or eliminate custom code, be more effective and efficient, and can be implemented without disruption to our user base.

Review EDMAPS’ data structures and their functionality and recommend an approach to optimize their design.

Review EDMAPS’ current components and recommend ways to optimize their utility and usage.

Review and recommend areas for data integration for deriving additional insights and value from

FSA’s data.

2.6.2. Data Quality Management

Data Validation – the Contractor shall validate incoming data for accuracy, completeness, and consistency against predefined rules and business logic.

Data Profiling – the Contractor shall perform regular data profiling to identify data quality issues, such as missing or inconsistent data, and take steps to address these issues.

Data Governance – the Contractor shall implement data governance policies and procedures to ensure data quality is maintained throughout the data lifecycle.

Data Quality Monitoring – the Contractor shall continuously monitor data quality and report any issues or anomalies to the relevant stakeholders.

Data Quality Reporting – the Contractor shall provide regular reports on data quality metrics, such as completeness, accuracy, and consistency, to the stakeholders.

Root Cause Analysis – the Contractor shall conduct root cause analysis to identify the underlying causes of data quality issues, develop a plan to take corrective action and to prevent them from reoccurring. Get approval of plan from stakeholders and implement.

Data Quality Improvement – the Contractor shall implement a continuous improvement process to address data quality issues and improve overall data quality over time.

Version 2.0 16 11/30/2023

Controlled Unclassified Information (CUI)

In Volume 1, the Contractor shall describe their proposed technical solution to support EDMAPS’ operations and functionality without disruption.

The Contractor shall also describe how they will implement ITIL processes and deliverables, and their approach to data security.

EDMAPS Business Requirements Deferred Requirements – Data Mapping

Version 2.0 17 11/30/2023

Controlled Unclassified Information (CUI)

Section 3. Deferred Requirements – Data Mapping FSA is in the process of designing and implementing an Enterprise API Management solution. The contractor’s solution ingress/egress API’s (Exposing APIs for use externally and/or features using APIs provided by other FSA systems) shall integrate/use FSA’s Enterprise API Management solution. The contractor’s solution for APIs shall conform to the API Enablement Guide at go-live of the initial task order.

See attachment: API Gateway Enablement Guide v1.6 and any subsequent versions for more details.

• FSA API Management Solution envisions the following capabilities:

o API Specification Repository capturing full API specifications enabling machine readable API specification that can be used in FSAs DevSecOps tool chain and infrastructure as code capacities.

o API Developers portal allowing developers to discover and leverage FSA APIs o API Governance enabling FSA to manage and coordinate API changes across the organization.

o Gateway standards for all ingress and egress access points. Specific Gateway technologies will be determined later.

o API Management solution standards for securing and monitoring internal APIs.

o Automated deployment and configuration of API’s through DevSecOps tool chain.

o All ingress and egress APIs interacting with FSA systems shall follow FSA’s authentication standards. All access tokens will be managed by one of FSA Enterprise Access Token managers (AIMS, PAS, System to System).

o Security rules and policies for ingress and egress access to APIs shall be externalized and available to the API Management solution for enforcement. Those policies shall be under change management and subject to governance.

o All APIs exposed to Web Sites or Mobile apps will protect those API using FSAs Enterprise API Management standards.

• The contractor shall collaborate with FSA to establish API Security policies externalized and managed in the API Management solution. The contractor can replicate those security policies in their solution but must coordinate with FSA so those rules can be centrally managed, applied, and informed access APIs contractor ingress and egress APIs through FSAs API Management Solution.

• The contractor shall develop and implement FSA-approved protocol specification for data exposure from systems/applications in the vendor IT environment to downstream operational components and data lake in FSA’s IT environment.

o Protocol specification must encompass data governance protocols in addition to integration and interoperability protocols o The contractor shall maintain documented lifecycles and processes for how data flows across its processing platform, define crosswalks for data across multiple systems, and ensure IT solutions are configured to meet FSA’s data integration needs

Version 2.0 18 11/30/2023

Controlled Unclassified Information (CUI) o The contractor shall support data migration and ingestion of historical loan data into FSA-approved destinations, to enable operational reporting and longitudinal analysis involving historical data.

o The contractor shall support data migration and ingestion of relevant third-party data into FSA-approved destinations, to enable the third-party data to be incorporated into operational reporting and analyses.

• DEFERRED REQUIREMENT: The contractor shall enable real-time data services with built-in APIs to provide a universal access method independent of language specific integration, greater flexibility in implementation tools, and capabilities to perform additional business logic on the data, including formatting, standardization, and enrichment.

• DEFERRED REQUIREMENT: The contractor shall ensure proper metadata management to include the following:

o System functionality to capture and store: (a) audit trails of when core, reference, and unmatched data was captured and updated; (b) record of attribute changes; and (c) definitions of data elements, locations, and types o Business capabilities to query, search, and report on metadata

• DEFERRED REQUIREMENT: The contractor shall ensure data-related tools and solutions properly meet all applicable data governance requirements which may include, but are not limited to, the following:

o Data synchronization and availability – Integrate with upstream source systems and downstream consuming systems either in batch ETL or API. (timeliness, frequency, service-level agreements) o Data quality standards – Ensure that data is complete, accurate, and consistent. Profile, cleanse, and monitor data on an on-going basis o Data conformity and interoperability – support common keys, industry standards, common object model to enable longitudinal analysis.

o Data privacy – create processes including any required encryption, data masking and tokenization, etc., with a particular focus on practical PII management without overly conservative access limitations.

o Data change management – maintain a change management process to make changes to data (updates vs initial creation, transactional, exception handling, etc.)

• DEFERRED REQUIREMENT: The contractor shall provide FSA a data dictionary of data stored on the contractor’s systems.

o The contractor shall provide FSA a data dictionary for all data housed on a system’s platform, including ancillary services/sub-contractors accessing data relating for FSA Customers.

o The data dictionary shall include the following:

Version 2.0 19 11/30/2023

Controlled Unclassified Information (CUI)

Field name, field definition (in simple to understand terms), valid values for the content, field format/length for data stored in the system (i.e. databases, access databases, tables, and spreadsheets).

Specialized Data including, but not limited to:

• Imaged data:

o Collateral materials, specifically promissory notes o Old/Legacy data o Other imaged data o Forms and other documents provided either by or to the aid recipient o Call recordings

• Historical transactional data for all actions (financial and non-financial) o The contractor shall provide a catalog of their procedures and training materials.

o The contractor shall provide the deliverables in this requirement to FSA on an annual basis (or upon request) in an excel workbook.

EDMAPS Business Requirements EDMAPS New Development

Version 2.0 20 11/30/2023

Controlled Unclassified Information (CUI)

Section 4. EDMAPS New Development FSA’s EDMAPS team is continuously working on enhancing data quality, availability, access, and distribution with the two principal goals: enhance customer experience and improve data-driven decision and policymaking. The EDMAPS team continues to issue change requests to enhance existing systems and capabilities such as to EDWA and the Data Lake, as well as develop new data systems storing, processing, and distributing data.

Current inventory of key enhancements and development is:

• Implement a document repository within the Data Lake to support United States Diagnostics Standards (USDS) (detailed in Section 2.1 above).

• Participate in FSA’s new Single Sign-on (SSO) initiative including PMDM.

• Participate in enhancements to the FTI Data Mart.

• Convert IGC’s metadata repository to a different tool such as Alation to make it easier for internal users.

• Receive Servicer data from NSLDS to EDWA, instead of the current SAIG mailbox.

The Contractor shall analyze every requirement for its impact on other FSA systems and offer a technical solution not only fulfilling the requirement, but also aligns with FSA’s Logical Data Architecture, LMM, SDLC, and Cloud standards.

The Contractor shall also present their detailed requirements, detailed design, change and release schedule, integrated system testing (IST), and deployment plans, and receive approvals at each of these stages.

The Contractor shall also coordinate with appropriate FSA and Contractor teams to present impacts, perform ISTs, and deploy enhancements and new system.