PL 007 TRA ENTERPRISE DATA SERVICES.docx

TRA05 – Enterprise Data Services (EDS)

TRA05 – Enterprise Data Services (EDS) Version 0.3 8/1/2017Prepared by:

South Carolina Department of Health and Human Services (SCDHHS) Enterprise Services (ES)

Table of Contents

Table of Figures

Table of Tables

Table of Standards

SCDHHS expressly restricts the distribution of this document to include only those SCDHHS staff, SCDHHS processing environment contractors, or any entity given explicit access to this document with SCDHHS executive or management approval.

Intended audience The SCDHHS Technical Reference Architecture (TRA) audience includes SCDHHS staff, trading partners, and third-party vendors who will be implementing integrating, managing, or operating the SCDHHS Medicaid Enterprise System (MES). The SCDHHS TRA provides the reference model by which technologies and components of the SCDHHS MES will be measured for development and implementation.

Introduction The Enterprise Data Services (EDS), the central database infrastructure of the South Carolina Department of Health and Human Services (SCDHHS) Medicaid Enterprise System (MES), stores data collected from distributed Medicaid enterprise systems. The goal of the EDS seeks to provide MES trading partners with ready access to critical decision-making data elements for operations, data-trading, business process redesign, and analytics activities. The EDS provides a comprehensive view of data entities, such as members, providers, eligibility, and claims, using data collected from disparate sources, including the legacy systems, other state agencies, and federal agencies. The EDS acts as the back end transactional database that serves the data needs of MES and non-MES systems by exposing the data through the integration hub, the preferred channel of communication with the EDS.

The EDS consists of two NoSQL databases—the raw data lake (RDL) and the operational data store (ODS)—and processes that facilitate transport of data to and from the EDS. For more information about the RDL and the ODS, see the TRA05.02 – Raw Data Lake (RDL) and TRA05.03 – Operational Data Store (ODS) supplements.

Architectural Context The MES processes data from disparate sources into the EDS using communication technologies, such as:Figure 1. The technical reference architecture

Web services Extract, transform, and load (ETL) processes File Transfer Protocol (FTP) services The integration hub facilitates these communications and enables systems (such as the member eligibility system and trading partners’ systems) to exchange (send and receive) data with the EDS. Figure 1. The technical reference architecture shows that the integration hub exposes the data in the EDS framework to whichever system component or trading partner requires it.

EDS NoSQL Databases TRA05-S1. The EDS uses NoSQL databases.

The EDS uses NoSQL technology, the key component of the data integration strategy, to store and modify the data from dissimilar data sources. Bringing data to a NoSQL database differs from bringing data into a traditional RDBMS, because NoSQL databases do not require designing a concrete database schema before loading the data. Thus, the NoSQL database offers the SCDHHS the advantages of speed of development.

Data from various sources can be ingested into a single NoSQL database in as-is format, each with its own schema. Although a NoSQL database provides native storage for JavaScript Object Notation (JSON), eXtensible Markup Language (XML), Resource Description Framework (RDF), geospatial, and large binaries (e.g., Portable Document Formats (PDF), images, and videos), at its core, a NoSQL database stores data as JSON or XML documents. The EDS NoSQL databases store data in the JSON format. Figure 2 Ingestion of data from various sources into NoSQL database Shows ingestion of data from multiple sources into a single database.

Figure 2 Ingestion of data from various sources into NoSQL database TRA05-S2. The EDS acts as single sources of information for MES Providing a 360-degree view of data to MES constituents starts with bringing all the data into a single repository. Using the metadata of these data sources, data architects can extract the information they want. Later, they can build a single view of the citizen by adding demographic data, eligibility data, claims data and, encounters data. As additional sources of data become available, data from those sources can also be ingested into the RDL and the appropriate data elements can be extracted to the ODS.

The major advantages of a NoSQL database over traditional relational database management system (RDBMS) databases for EDS include:

Enabling the applications to store both structured and unstructured data.

Schema agnosticism, which allows for changes to the source system schemas, without requiring changes in the routines that extract and load the data into EDS.

Supporting storage of multiple and complex data structures, such as binary values, lists, maps, and highly complex parent-child hierarchical structures.

Purposefully built for high data volume and extreme performance while maintaining an integrated view of Medicaid data entities, such as clients, providers, and claims.

EDS Internal Architecture This section describes the components of the EDS framework. Figure 3 EDS Internal Architecture shows the following internal components and ancillary applications:

Raw data lake Operational data store Metadata repository REST web services Data marts

ETL/ELT

MFT

Figure 3 EDS Internal Architecture Raw Data Lake (RDL) The RDL, a NoSQL database, serves as the integration point for raw data across the MES. The MES loads data in the raw data format of the source system. Table 1 RDL main purposes identifies the main purposes of the RDL.

Table 1 RDL main purposes

Operational Data Store (ODS) The ODS, also a NoSQL database, contains the data materialized from the RDL. Functionally, the ODS provides the most comprehensive view of data within the MES and offers a single view of business-critical entities, such as members, providers, claims, etc. For example, MES consolidates member eligibility data collected from a wide array of distributed information systems into the ODS and displays the data in front-end applications, such as the SCDHHS Portal. The ODS acts as the transactional database to serve the data requests of the portal and all the trading partners in the MES. The portal, web services, and file exchanges all request data from the integration hub, which, in turn, obtains the data from the ODS. When the data consumer updates source records, integration hub web services submit the updated data to the EDS framework. The RDL initially reflects these changes, and then updates the ODS. For more information about the ODS, see the TRA05.03 – Operational Data Store (ODS) supplement.

Metadata Repository The EDS includes a metadata repository that contains descriptive information about the data structure of the sources used to populate the EDS. When processing raw data, the metadata repository appropriately identifies the data collection and data elements. The metadata repository defines the:

Data source owner (trading partner, SCDHHS, etc.)

Data collection format (file, database, web service, etc.)

Purpose of the data collection (member data, claims data, etc.)

Definition of the data elements in the source system (first name, last name, etc.)

REST Data Services The EDS contains an integration layer on top of the NoSQL database. This layer provides representational state transfer (REST) or RESTful interface services to enable the integration hub to create, read, update, and delete (CRUD) content in the RDL. The REST application programming interface (API) supports access to XML, JSON, text, or binary format documents (such as PDF, XLS, etc.). REST API request and response messages can take the form of XML or JSON. Figure 4. REST data services

To implement the REST API, the EDS web application server processes Hypertext Transfer Protocol (HTTP) requests. Shows the flow of data to and from the ODS NoSQL database through the REST API. The TRA04.02 – Web Services supplement describes REST APIs and data services.

Data Marts Data marts provide analytical databases to serve the business intelligence and reporting needs of a particular line of business. I EDS contains a consolidated dataset from systems across the MES, and therefore, provides a consolidated data source for data marts.

Data marts population occurs by first loading the relevant data from the ODS–the initial load. Updates to EDS populate to the data mart on a regular basis (for example, daily or weekly) in delta loads or incremental loads.

The TRA05.04 – Analytical Data Stores supplement provides a more complete discussion on the design and implementation considerations for data marts.

ETL/ ELT Processing Extract, transform, and load processing (ETL), refers to both the process and the technology that move data from a source to a destination while changing the data format according to the destination’s system requirement. ETL differs for another variant of the same process (ELT) in the order in which the transform and load steps take place. For the sake of this document, ETL refers to either of the variants. The TRA04.05 – Extract, Transform, and Load (ETL) supplement describes the design and implementation consideration of ETL in the MES.

The EDS uses the ETL processes as one of the methods to move data into and within the EDS. For example, ETL performs the initial load of data from external sources into the RDL. ETL also moves data from the RDL into the ODS. The TRA05.02 – Raw Data Lake (RDL) and the TRA05.03 – Operational Data Store (ODS) supplements explain the details of these use cases.

MFT

Managed file transfer (MFT) refers to both the process and the system that move files from one location to another by adhering to enterprise best practices, such as security, privacy, control, and auditing. The EDS uses MFT to send and receive file transfers between SCDHHS and its trading partners. The TRA04.06 Managed File Transfer (MFT) supplement details the processes and design considerations of the MFT system.

EDS Key Benefits The EDS framework provides the set of databases and technologies that serve the data needs of the MES. A consolidated EDS framework offers multiple benefits, including:

Data consolidation Application integration Data quality Central reference data Data Consolidation Traditional database applications tend to be repositories of data under the control of one department that does not necessarily share the data with other applications in the enterprise. This leads to departments, often working towards common organization goals, operating with only a subset of available data. As a result, enterprise-level users (users that perform processing across business units) need to interrogate multiple systems, with conflicting data, to perform business processes.

The EDS provides a consolidated view of MES data previously spread across data silos. Caseworkers can see all the relevant data in one place with inconsistencies addressed by enterprise data quality and master data management routines. For example, the EDS consolidates all data related to a member (such as eligibility, claims, and employment history) from the Medicaid Eligibility Determination System (MEDS), Medicaid Management Information System (MMIS), and Access (Cúram), and exposes the data through front end applications, such as the SCDHHS Portal. This consolidated view of data allows caseworkers to spend less time looking for data and more time helping constituents enroll in the state programs for which they are eligible. Figure 5. Applications used for eligibility determination, shows an example of how caseworkers need to review information from a number of systems to determine constituent eligibility for Medicaid.

Figure 5. Applications used for eligibility determination The EDS consolidates the data, as shown in Figure 6. Eligibility data unified in EDS and offers the following benefits:

Streamlines the eligibility determination processFigure 6. Eligibility data unified in EDS

Supports better business decisions through improved data quality Simplifies the training on the operational environment Reduces the time to making new employees productive

Application Integration In a traditional enterprise environment (depicted in Figure 7. Traditional point-to-point integration), each trading partner operates independently, yet needs to coordinate changes to business data (e.g. member, provider, eligibility, claims, etc.) with other systems across the enterprise.

Figure 7. Traditional point-to-point integration

The EDS integration framework (shown in Figure 9 EDS integration framework) provides a modern, cost-effective, and technologically superior alternative to point-to-point integrations between the trading partners. The systems of records send updates to the EDS for consumption and distribution. EDS batch and real-time processes supply the subscribing trading partners and other data sources with the latest versions of the relevant data. For example, the integration hub communicates to the EDS, the changes to a member record. Then, delivery mechanisms, such as web services and file exchanges, propagate the change to the appropriate trading partners and applications. Figure 9 EDS integration framework

Data Quality Several applications maintain data across the MES. Trading partners, for instance, maintain constituent and claims information and pass that information to SCDHHS for processing. Data quality issues potentially encountered in this setting include:

Inconsistent data—Data updates made to more than one system with conflicting information often introduce data inconsistencies. For instance, transitory constituents’ addresses can change frequently, which commonly results in the Dental ASO having a different address for a constituent than the Medical ASO. In another example of data inconsistency, one system processes claims for a constituent that another system identifies as being deceased.

Incorrect data—MES source systems may contain incorrect data, such as incorrect addresses (as validated by an address validation service). Although all systems may contain the same address for a member, the address itself may be incorrect.

Incomplete data—The ODS provides routines to enforce data governance, including completion of mandatory fields.

Each system of record will contain data relevant to the processing needs of the respective system. For instance, one system may require constituents’ Social Security number (SSN) while another may not need SSNs for system processing.

The EDS contains processes and workflows to:

Improve the quality of data Consolidate records across systems of record into a single source of truth (the ODS) Propagate the single version of truth to all applications The TRA05.03 – Operational Data Store (ODS) supplement contains a more comprehensive discussion on data quality.

Central Reference Data Reference data defines the set of permissible values to be used by other data fields across all applications in the enterprise. Examples of centralized reference data include error codes, address codes, date codes, currency codes, etc.

Storing reference data in a central location not only establishes enterprise-wide data standards, but also enables infrastructure services to reuse the standard data. The EDS centrally holds the most current reference data and shares that data with all the applications across the enterprise. For example, the EDS centralizes the enterprise-wide collection of code and description pairs for application-specific and non-application-specific errors.

Enterprise applications can either access the reference data in the EDS directly, or replicate/synchronize the data locally by copying the data from the EDS. If the reference data changes frequently, the applications would be best served accessing the reference data directly from the EDS. On the other hand, applications can avoid network calls by replicating mostly static reference data locally.

Infrastructure services such as error reporting- and process logging services can use this centralized reference data for their operations.

EDS Data Management Pipeline The Technical Reference Architecture document (see TRA – Technical Reference Architecture) introduced the Enterprise Data Management Pipeline (EDMP), the process by which MES processes data. Figure 9. EDS in the EDMP illustrates the sequence of these activities and defines the scope of the EDS activities in the EDMP:

Access - source system data is accessed for processing Ingest – data is ingested into the Raw Data Lake (RDL) Materialize – data is materialized into the Operational Data Store (ODS) Deliver – the EDS delivers data services to the integration hub.

Figure 9. EDS in the EDMP Access As previously stated, EDS data sources include Medicaid information systems, such as mainframe systems, Access, OnBase, and external data partners. Either EDS or the system of record (SOR) can initiate the data exchange. When the SOR initiates the exchange, it can do so by either sending a file for processing or by calling an EDS web service configured for the purpose of the exchange. When the EDS initiates the request, it can either call a web service exposed by the SOR or EDS can access the SOR database directly (if allowed). The integration hub provides the main access to the EDS and provides methods, such as web services, file transport, and custom adapters, to connect and share data with the EDS.

The access methods supported by the EDS and integration hub include those listed in Table 2. Supported methods to access the EDS.

Table 2. Supported methods to access the EDS

Similarly, the source systems expose web services that the integration hub can use to request data updates. For example, a source system exposes a web service ‘GetNewMemberInfo’. The integration hub invokes this service periodically, and passes a time stamp as input parameter. The source system receives the request and responds with all new members created on or after the provided timestamp. The integration hub sends this information to the EDS.

The integration hub supports the following file formats:

Extensible Markup Language (XML) JavaScript Object Notation (JSON) Delimited (comma-delimited, pipe-delimited, tab-delimited, etc.)

Fixed-width text Electronic Data Interchange (EDI) (X12)

Ingest During Ingest, the MES reads and copies data from source systems in an ‘as-is’ format (exactly how it was extracted from the source system) into the RDL NoSQL database. The NoSQL RDL database allows the insertion of data of any format and schema without having to create a new schema when adding a new source. Data sources from which the MES ingests data into the EDS include:

Mainframe systems

MEDS

Access OnBase Other state agencies Federal agencies Each record in the RDL contains metadata that allows subsequent processes in the EDMP to perform processing appropriately. Refer to Section 3.3. Metadata Repository for a more complete discussion on metadata.

Materialize Data ingestion results in a collection of NoSQL data documents of variant layouts. This data requires materialization before delivery to the customer-facing applications. Materialization involves consolidating and transforming the RDL data into the ODS using the following steps:

Design target schema–The EDS application architects design a schema for a materialized set in the ODS (e.g. member data layout).

Map data–The data analysts map the data elements of the source schemas (RDL) to the target schema of the EDS (ODS).

Transform data–Developers implement routines to convert the data residing in RDL in the source’s native format into the JSON target schema of the ODS.

Cleanse data–The data analysts and data stewards employ automated and manual procedures to perform data quality activities, such as the following:

· Augment missing data–Using existing data, cross-referencing algorithms can derive missing data, such as zip codes, city name, and last names.

· De-duplicate the redundant data–Match, merge, and de-duplicate data using data quality tools and procedures.

Deliver Once data has been materialized and data quality routines have been applied, the data can be delivered to data consumers in the appropriate format. Table 3. Methods of delivering data to consumers lists the methods of delivering data to consumers supported by the EDS.

Table 3. Methods of delivering data to consumers

EDS Nonfunctional Features In addition to the core components described in the above sections, the EDS contains ancillary processes and components essential for an enterprise software implementation, including:

Security Exception handling Process and audit logging.

Security TRA05-S3. The EDS adheres to TRA01.02 – Access Control and Identity Management The EDS contains protected health information (PHI) and personally identifiable information (PII) South Carolina Medicaid constituents. To protect the privacy of Medicaid constituents, and to adhere to HIPAA regulations, SCDHHS must keep this data secure and only allow access by properly authorized users.

The MES accomplishes data security by the following means:

Authentication – Ensuring that the user is who they claim to be Authorization – Ensuring that the user has the appropriate access privileges Protecting data at rest – Encrypting data at the operating system level Protecting data in transit – Securing data in transit The sections below provide a brief overview of authentication and authorization. For a more complete security discussion, refer to the TRA01.02 Access Control and Identity Management supplement.

Authentication and Authorization at the Integration Hub The integration hub consumes the data services the EDS exposes. In turn, the integration hub exposes business services to the SCDHHS portal and trading partners within the EDS.

The integration hub has built-in mechanisms for integrating with the identity management component to facilitate user authentication for direct access from trading partners (both inside and outside the SCDHHS domain).

The integration hub will utilize the identity of the logged-in user, as provided by the identity management component, to determine if the user has the appropriate level of system access privileges to perform the requested activity.

The EDS will not perform authentication and authorization of any end-user.

Authentication and Authorization at the EDS The EDS contains its own security features to implement authentication and authorization. Authentication and authorization performed at the EDS will be limited to the system account created for the integration hub and any administrative users, such as database administrators and system administrators.

Exception Handling TRA05-S4. The EDS will contain design considerations for error handling.

All enterprise applications address error handling. All processes and workflows that are part of the EDS follow the standards set forth in the MES error-handling framework.

The MES provides a uniform enterprise-wide exception handling mechanism with a repository of error codes and descriptions. In the event of any failure in any EDS related activities, such as document ingestions, data materialization, etc., the MES handles the exception according to the enterprise error handling guidelines.

TRA05-S5. The EDS will log all defined and undefined errors.

The goal of the exception-handling procedures in the EDS aims to log the cause of the error and to communicate the error details to the systems administrators. When an external system initiates the error-causing routine, the error messages need to be communicated back to that system.

TRA05-S6. The EDS will capture full error context.

Table 4. Details of errors logged in the EDS describes the details of errors logged in the EDS.

Table 4. Details of errors logged in the EDS

Process Logging / Audit Logging Process logging and auditing helps track the functioning of an application by recording events and tasks performed.

TRA05-S7. The EDS logs each event information.

EDS provides a mechanism to audit events, such as document read access, server startup, server shutdown, document permission changes, etc. The EDS logs audit event records in audit tables inside the EDS. Table 5. Events logged in the EDS identifies some of the events logged in the EDS.

Table 5. Events logged in the EDS

Table 6. EDS log entry details describes the details in an EDS log entry.

Table 6. EDS log entry details

Example:

2016-01-11T12:01:00

Example:

event=Document read

Example:

function=http://EDS-domain/xdmp/admin:read-config-file

Example:

expr=cts:element-value-query(xs:QName("info:status"), ("active", "unloading"), ("unstemmed","lang=en"), 1)

Example:

type=node-update

Example:

uri=/queries/5523898374388210414.txt

Example:

database=ODS-Prod

Example:

success=true

Example:

user=SCDHHS-admin

Example:

roles=cpf-restart,infostudio-user

Use Case – Flow of Data through the EDS In this use case, the MES consolidates constituent data from several sources into the EDS and exposes the consolidated data to the SCDHHS portal via the integration hub. The TRA05.02 – Raw Data Lake (RDL) and TRA05.03 – Operational Data Store (ODS) supplements provide more detailed dataflow pattern use cases.

Figure 10. EDS data flow use case illustrates the data flow steps outlined below.

1. The MES ingests constituent data from the SORs into the EDS RDL. As part of the ingestion process, MES also creates metadata inside the EDS. At this point, front-end applications should not consume the RDL data in the RDL because of inconsistent structures and formats.

2. The materialize routines parse the RDL in conjunction with the metadata repository and load the materialized version of constituent data, properly formatted, into the ODS.

3. The integration hub exposes the materialized constituent data in the ODS using web services designed to retrieve constituent data.Figure 10. EDS data flow use case

4. The portal consumes the constituent data services (provided by the integration hub) and displays a constituent record to the user.

5. The user makes changes to the constituent information and the MES sends the changes to the integration hub via a web service designed to update member data.

6. The integration hub writes the modified constituent information into the RDL.

7. The materialize routines pick up these changes from the RDL and update the constituent record in the ODS.

8. The ODS contains a new ‘record of truth’ which, the MES feeds back to the enterprise SOR to ensure the coordination and synchronization of constituent data across all the SORs.

Appendices Acronyms

Referenced documents

8/1/2017 SCDHHS - Confidential Page 4 of 4 image1.png image2.png image3.png image4.png image5.png image6.png image7.png image8.png image9.png image10.png image11.png image12.png image13.png image14.png