FRA_EDIMS_LiveRFI2_Presentation_12Nov24.pdf

RAIL

MOVING AMERICA FORWARD

13 November 2024

Federal Railroad Administration Office of Acquisition & Office of Research, Data, and Innovation

Live RFI #2 - FRA Enterprise Data Integration and Management Support (EDIMS) Services

Recording in Progress:

Please note FRA will be recording this session.

NOTE: The Questions and subsequent discussion are for INITAL market research. FRA has not settled on a technical strategy, acquisition strategy or contract structure at this time. Continued market research will inform the composition of the requirement going forward.

Agenda

Agenda

• Welcome & Opening Remarks

• What is the Purpose of Today

• FRA EDIMS Clarification of Requirement

• Break – 10 Min

• Discussion

• RFI Technical Questions – Live review and discussion of market research questions

• Break – 10 Min

• Discussion (Cont.)

• RFI Technical Questions – Live review and discussion of market research questions

• Next Steps & Concluding Remarks

• Questions

Welcome & Opening Remarks

Get a full understanding of our need Open Dialogue / Transparency

Exchange of Ideas / Information Walk away energized and ready to help shape the requirement

What is the Purpose of Today

We want to hear from you!

FRA

Rules of Engagement

• Provide constructive feedback, we’re listening

• Ask Questions, we want your help to improve this requirement

FRA EDIMS Clarification of Requirement

• FRA EDIMS is a support service to provide data management (i.e., cleaning, validating, transforming, and preparing data for use by FRA and its stakeholder data scientists and analysts) in a data lakehouse or similar environment

• What FRA EDIMS is NOT:

• It is not an analysis services contract (e.g.,it is not for statistical analysis, data reporting, or data visualization)

• It is not a contract for building a data management environment (though FRA may ask contractor make recommendations about data architecture of data management infrastructure)

• FRA is not committed to a particular data infrastructure to support a data management solution

FRA EDIMS Requirement Clarification

• Current

• Data Management happening in Railroad Safety

Information System (RSIS) database itself o RSIS Transactional database

• Business Information System (BIS) Contract o BIS (business intelligence system) is currently a data lake o BIS contractor is responsible for implementing the design o Includes management, enhancement, integration, implementation, maintenance, modernization, and infrastructure support for BIS applications (i.e. ,data management environment)

• Future

• FRA-wide implementation of High-Level Objectives 1-9

• FRA EDIMS will include integration & management of data from multiple data bases

• BIS contract will continue to function as a separate contract for Operations & Management

FRA EDIMS Requirement Clarification

Current RSIS Data Management

Current Non-Integrated Data Management

A Potential Architecture for a Future EDIMS Environment

This diagram shows a potential architecture. FRA seeks advice on the best architecture for the EDIMS environment.

10 Minute Break

LIVE RFI Technical Questions Discussion

WE WANT YOUR FEEDBACK!

Discussion Questions:

Questions are dialogue questions, and we need your input.

Process:

Raise your hand to speak or place a comment in the Q&A box.

SPEAK UP: Silence = Compliance

Live RFI Discussion Questions

Technical Related Questions

• How long would a company need to fully staff a complex requirement like EDIMS? Ramp Up time, how much time is needed, are any skill sets going to drive longer ramp up times?

• What are the pros and cons of requiring specialized skill sets like spark SQL on the EDIMS requirement?

• What should the government include in the requirements document to ensure we obtain the right skill sets to support AI/ML in the EDIMS requirement?

• FRA seeks a modern, integrated data management environment such as a delta data lake house or a data lake plus an enterprise relational data warehouse. What are factors that FRA should consider when selecting an architectural approach?

• What should FRA consider when drafting its solicitation to ensure the data management solution remains relevant and up-to-date throughout the period of performance?

• Are there any challenges FRA should consider with interoperability and scalability challenges with data platforms heavily integrated into the Microsoft ecosystem?

Discussion Questions (Cont.)

Technical Related Questions (Cont.)

• What industry trends, best practices, and challenges are important to consider in providing data management services?

• How can FRA develop its enterprise data environment to maximize scalability and interoperability while avoiding vendor lock?

• What supporting information will help a contractor propose on the EDIMS requirement?

Which items (e.g., volume of data, frequency of data updates, velocity, variety, number of database systems, other) will assist with scoping?

• What additional considerations should FRA be thinking about as we define the FRA EDIMS requirement?

Discussion Questions (Cont.)

FRA EDIMS Next Steps

FRA Actions:

• Review the results of Live RFI 1 & 2.

• Determine if FRA will hold one-on-one vendor engagement sessions.

• Finalize market research.

• Prepare the acquisition plan for review and approval.

Notional Timeline to Solicitation:

• Post draft requirements document - anticipated December 2024.

• Host Live Draft Solicitation Review with industry - anticipated January

2025.

Any additional information or questions to help shape the requirement and acquisition strategy can be shared via e-mail.

Next Steps

Thank you!

Contact Us Federal Railroad Administration 1200 New Jersey Avenue, SE Washington, DC 20590

FRA Acquisitions

Email:

FRAAcquisitions@dot.gov

SUBJECT: FRA EDIMS Live RFI#2

Back Up

For Data Engineering, FRA seeks to develop and optimize data analytics pipelines to deliver high quality data that enables data driven decision making through analytics and geospatial analysis.

Build, maintain, and improve FRA data management framework, structure and processes;

Design information architecture for a Delta Data Lake House or comparable system;

Develop and implement Extract Transform and Load (ETL) processes;

Develop and implement scalable data practices;

Maintain change data capture logs;

Following the Medallion model, process (via ETL) data from raw

(bronze) to cleaned and validated (silver) to presentation (gold) levels. Note: Movement of data between environments (ELT) will be predominantly managed via a separate contract.

DRAFT High Level Objective 1 – Data Engineering Support

For Data Quality (Data QA/QC), FRA seeks cleaning and validation of submitted data to review, detect, and correct any inaccurate data.

Review data submissions for accuracy and resolve errors through coordination with submitters;

Develop, implement, and maintain automated data quality rules;

Track submission errors for trend analysis;

Monitor data quality and develop process for elevating data quality errors to FRA;

Clean and validate submitted data to create trusted system of record;

Develop and maintain data transaction log.

DRAFT High Level Objective 2 – Data Quality

For Data Information Architecture, FRA seeks to organize and arrange collected data in accordance with appropriate physical and logical data models to ensure the capability to analyze, interpret, and use data.

Design, implement, and maintain logical and physical data models for FRA and its stakeholders;

Participate in FRA Data Architecture and Planning Review (DAPR) process;

Develop information architecture strategy;

Document database diagrams and provide supporting documentation to enable proper data governance.

DRAFT High Level Objective 3 – Data Information Architecture

For Data Collection Design, FRA seeks to employ user friendly methods for collecting high quality and thorough data that enables effective analysis and decision-making.

Provide design recommendations to FRA and stakeholders for effective data collection, including:

data fields, valid codes, and form design

Align design of data collection instruments with security requirements

Coordinate design implementation with appropriate FRA staff or support contractor;

Validate and verify operability on recommended designs.

DRAFT High Level Objective 4 - Data Collection Design

For Data Governance, FRA seeks support for development and implementation of policies including data standards and other governance initiatives including in AI/ML to ensure data integrity and compliance with federal regulations and standards.

Review and assess existing FRA/DOT principles, standards, tools, methodology and practices;

Recommend updates to improve data management governance and address deficiencies;

Support coordination of recommendations to FRA/DOT stakeholders;

Support designing, developing, implementing, and overseeing a unified data management and governance process spanning the full data lifecycle;

Create and maintain necessary documentation to support new and updated data governance;

Support the use of FRA data governance platform

DRAFT High Level Objective 5 – Data Governance

For Data Strategy Support, FRA seeks to identify and assess current and future capabilities (including AI/ML) in order to make informed decisions to improve data analytics, mining, integration, and governance capabilities.

Assess current FRA capabilities, previous plans, and documentation to forecast future needs;

Develop and maintain data strategy plan that captures the technology, processes, people, and rules required to manage FRAs information assets;

Outline FRA's long-term vision for collecting, storing, sharing, and usage of its data;

Develop business goals, objectives and Key Performance Indicators (KPIs) to measure effectiveness of implemented capabilities.

DRAFT High Level Objective 6 – Data Strategy Support

For Communications Strategy, FRA seeks to develop and implement a communications plan to educate internal and external stakeholders about available data and how to effectively leverage it.

Review current data management processes and communicate them to internal and external stakeholders;

Develop, document, and maintain a plan for bidirectional information flow;

Support implementation of FRA approved communications plan directed to data consumers;

Develop, maintain, and update outreach and educational resources;

Conduct on-going outreach to internal and external stakeholders.

DRAFT High Level Objective 7 – Communications Strategy/Plan

For Coordination Support, FRA seeks to ensure seamless communication, interaction, and collaboration between FRA organizations and contractors to align data management across the agency

Develop, document, and maintain a plan for how the contractor will coordinate activities with FRA organizations and FRA contractors;

Provide support to enable synchronization of data related activities to align with data and communication strategies;

Provide technical guidance and updates to the FRA Data, Architecture, Planning, and Review (DAPR) Board.

DRAFT High Level Objective 8 – Coordination Support

For Data Extraction from BIS, FRA seeks to capture and export curated data sets to enable special studies

Support on-demand dataset extraction requests, as required;

Develop and store (e.g. in DOT GitHub or other repository) necessary code (such as SQL queries) to support data extraction requests;

Document data extraction process including data extraction method, programming, and data sources for each request that documents rationale to support requests in DOT GitHub or other repository to enable reuse;

Advise and support FRA stakeholders on best practices and processes to extract data/information.

DRAFT High Level Objective 9 – Data Extraction from Business Information System (BIS)