Data_Management_Platform_CRD_DRAFT.pdf

DISTRIBUTION STATEMENT A: APPROVED FOR PUBLIC RELEASE DISTRIBUTION IS UNLIMITED

ARMY DATA PLATFORM (ADP) – DRAFT CAPABILITY REQUIREMENTS DOCUMENT

CAPABILITY REQUIREMENTS DOCUMENT (CRD) 3

PROGRAM EXECUTIVE OFFICE 6

ENTERPRISE INFORMATION SYSTEMS (PEO EIS) 7

PROJECT MANAGER 10

ARMY DATA AND ANALYTICS PLATFORMS (PM ARDAP) 11

FORT BELVOIR VA 22060-5526 12

ARMY DATA PLATFORM (ADP) 19

REVISION HISTORY 21

VERSION DATE SUMMARY OF CHANGES

1.0 27 OCT 22 Initial

1.1 02 DEC 22 TMT Staffing Adjudication

DISTRIBUTION STATEMENT A: APPROVED FOR PUBLIC RELEASE DISTRIBUTION IS UNLIMITED

APPROVALS 27

FUNCTIONAL SPONSOR 34

DR DAVID MARKOWITZ 39

Chief Data and Analytics Officer 40

Headquarters, Department of the Army 41

PRE-CERTIFICATION AUTHORITY 45

MR ROBIN P SWAN 50

Director, Office of Business Transformation 51

Headquarters, Department of the Army 52

CHIEF INFORMATION OFFICER 57

DR RAJ IYER 62

Chief Information Officer 63

Headquarters, Department of the Army 64

DISTRIBUTION STATEMENT A: APPROVED FOR PUBLIC RELEASE DISTRIBUTION IS UNLIMITED

TABLE OF CONTENTS 68

EXECUTIVE SUMMARY ................................................................................................ 6 69

SECTION 7. IDENTIFY HIGH-LEVEL BUSINESS PROCESSES .................................. 7 70

7-1. Overview ............................................................................................................................................... 7 71

SECTION 8. CONDUCT BUSINESS PROCESS REENGINEERING (BPR) AND 72

IDENTIFY CHANGES ................................................................................................... 13 73

8-1. BPR Introduction ............................................................................................................................... 13 74

8-2. BPR Methodology .............................................................................................................................. 13 75

8-3. BPR Results ....................................................................................................................................... 14 76

SECTION 9. DOCUMENT PROCESS PERFORMANCE MEASURES AND 77

INFORMATION ASSETS .............................................................................................. 21 78

9- 79

9-2. Objective Requirements .................................................................................................................... 23 80

9-3. Other Mandatory Requirements ....................................................................................................... 24 81

SECTION 10. DEVELOP DOCTRINE, ORGANIZATION, TRAINING, MATERIEL, 82

LEADERSHIP AND EDUCATION, PERSONNEL, FACILITIES AND POLICY 83

(DOTMLPF-P) ACTIONS AND CAPABILITY IMPLEMENTATION PLAN ................... 24 84

10-1. Capability Implementation Plan (CIP) ............................................................................................ 24 85

10-2. DOTMLPF-P ...................................................................................................................................... 24 86

10-3. Army VANTAGE 2.0 Program Schedule ........................................................................................ 25 87

10-4. Army VANTAGE Synchronization and test plan ........................................................................... 25 88

10- 89

DISTRIBUTION STATEMENT A: APPROVED FOR PUBLIC RELEASE DISTRIBUTION IS UNLIMITED

Table 1: High Level Objectives ................................................................................................................... 12 94

Table 4: VANTAGE Data Objects ............................................................................................................... 19 97

Figure 1: Strategic Enablement: Enterprise Data Mesh Services................................................................. 8 100

Figure 2: Data to Decision Components ..................................................................................................... 13 101

Figure 3: VANTAGE Users By MACOM ..................................................................................................... 14 102

Figure 4: Content Creation By MACOM ...................................................................................................... 15 103

Figure 5: Content Creation by Rank ........................................................................................................... 15 104

Figure 6: Content Creation by MOS ............................................................................................................ 15 105

Figure 8: EDSC Roles Life Cycle Crosswalk .............................................................................................. 16 107

Figure 9: EDSC Process Flow .................................................................................................................... 17 108

Figure 10: Data Policy and Guidance Timeline........................................................................................... 18 109

Figure 11: Major Data Platform Decision Support Achievements

ARMY DATA PLATFORM (ADP) VANTAGE 2.0 – DRAFT CAPABILITY REQUIREMENTS DOCUMENT

DISTRIBUTION STATEMENT A: APPROVED FOR PUBLIC RELEASE DISTRIBUTION IS UNLIMITED

EXECUTIVE SUMMARY 114

a. The Army Data Platform (ADP) requirement is a follow-on capability to the116 current Under Secretary of the Army (SAUS)-directed Army VANTAGE program and 117 provides an Enterprise-level data platform capability. It directly supports five of the six 118 components of the Business Enterprise Architecture Data to Decision business process 119 and is the platform component of what the Army is beginning to call a “Data Mesh.” It is 120 a core component of the Army’s efforts to become a Data Centric force and supports 121 the implementation of the Army Digital Transformation Strategy’s goal of using data as a 122 strategic asset. 123

b. The requirement focuses on enterprise level functions, it spans support across125 the Business Mission Area (BMA) and Warfighting Mission Area (WMA) at the 126 operational level of war and above (ie Division and higher). Although many 127 specifications identify the technical requirements of a data platform, such as data ingest 128 and transformation, the critical aspect of this requirement is ease of use for data 129 analysts to generate new data products for the Army. There are several commercially 130 available data platform solutions. Ease of use is a key differentiator. Based on lessons 131 learned from current Army VANTAGE program and Army Data Cloud Executive Order 132

(EXORD), ease of use lowers the barrier of entry to allow ‘citizen’ analysts across the 133

Army to integrate and enrich data for tailored decision support at echelon. It is an 134 accelerator for digital transformation, allows the Army to change at scale, and provides 135 for rapid decision support as required in Multi-Domain Operations (MDO). 136

NOTE: Army Data Platform (ADP), previously identified as ARMY VANTAGE, is only required to submit Part 2 of the CRD which starts at Section 7. The previous SAUS-directed capability requirement (2018) serves as Part 1.

ARMY DATA PLATFORM (ADP) VANTAGE 2.0 – DRAFT CAPABILITY REQUIREMENTS DOCUMENT

SECTION 7. IDENTIFY HIGH-LEVEL BUSINESS PROCESSES 137

As part of modernization, the Army needs to increase the speed and quality of decision 139 at echelon. The right data at the right time is a key enabler for improving decisions. As 140 per the 2018 Undersecretary of the Army (SAUS) signed and Chief of Staff of the Army 141

(CSA) endorsed Statement of Need (SoN), the Army requires a data platform to 142 facilitate this goal. Ease of use at echelon is critical to enable the data democracy and 143 citizen analysts to meet the scale and speed of relevance of our today and future Army. 144

The Army’s current program that provides this capability is Army VANTAGE, previously 145 titled Army Leaders’ Dashboard. This initial capability was required through a Directed 146

Needs Statement signed by the Undersecretary of the Army (USA) and endorsed by the 147

Chief of Staff of the Army (CSA). ARMY DATA PLATFORM (legacy VANTAGE) 148 replaces this initial capability, refining its scope based on lessons from three years of 149 use and maturity of the Army’s data community. 150

7-1. OVERVIEW 152

a. The Army Data Platform is part of a larger data ecosystem or Data Mesh. The 154

Data Mesh follows four main principles: Data Ownership by domain; Data as a Product; 155

Data available everywhere, Self-Serve; and Federated Governance. The goal is to 156 provide the right data to the right echelon at the right time. Data will be simplified, 157 authoritative, and fit the decision required. The data is independent of any source 158 system – it is integrated as a ‘product’ ready for a Soldier or civilian to use. Visible, 159

Accessible, Understandable, Linked, Trusted, Interoperable, and Secure. 160

b. As a component of the Data Mesh, the data platform provides enterprise 162 compute and store to build, run and monitor data products. It is the engine for producing 163 data products to meet mission, giving users the ability to create and use conforming 164 data products without needing detailed understanding of the federated mesh 165 governance policies. The data platform is uniquely suited – it provides the workspace 166 for working with Army decision makers at echelon to determine what data is needed for 167 decision. It extracts, transforms, enriches, and presents data as needed by the decision 168 maker. The platform then can host the data and provide continuous data pipelines, 169 ensuring the data needed by the decision maker meets the decision makers’ latency 170 requirements. The platform is the innovation engine for data products and mission. 171

Figure 1: Strategic Enablement: Enterprise Data Mesh Services 174

7-2. CURRENT CAPABILITY NEED 176

a. The current Army capability provided by Army VANTAGE program is expiring. 178

The basic capability provided by the program is still required. 179

b. The Army requires continued visibility into the vast amount of data contained in 181 numerous databases dispersed across the Army enterprise. The Army requires a big 182 data management platform-like and tailorable solution that integrates, curates, 183 analyzes, and visualizes information from multiple disparate data sources, classified 184 and unclassified. 185

(1) The solution should present information in a way that is immediately 187 understandable and actionable for any end user. Inclusion of a mobile solution (all form 188 factors) is preferred. 189

(2) The solution should be capable of integrating both current and historical 191 structured and unstructured data sets, perform predictive analytics, and forecasting. 192

(3) The solution should be interoperable with the Army’s API, data Catalog, and 194

ICAM capabilities. 195

(4) The product and/or capability is commercially available now. As the Army 197 continues to transform operations in the digital age, leaders have an immediate need for 198 access to data that will meet the demands of information decision-making at all levels of 199 command. The primary scope is for Enterprise level support and JTF capable 200 formations and above. 201

c. The requirement has been refined by lessons learned since its initiation and 203 maturity in the Army Data community. The changes are as follows: 204

(1) The Army Data Platform works with other Data Mesh components including: 206

Army ICAM system, Data Catalog, Enterprise API Management and domain specific 207 data environments. 208

(2) The Army Data Platform does not need to store all Army data as initially 210 envisioned. It provides the ability to ingest, transform, analyze, and export Army data, 211 but focuses on storing simplified, authoritative data used for decisions – that is, data 212 products and the data required to produce them. 213

(3) The Army Data Platform is the innovation engine of the Data Mesh. It is the tool 215

Army analysts in close support to Army decision makers enrich, curate, and integrate 216

Army data into new data products to support changing Army missions at speed and 217 scale. 218

(4) The data platform has three types of users: 220

(a) Decision Makers - The data platform supports decisions at echelon. Decision 222 makers range from General Officers to Contract Resource Managers, each informing 223 their decisions with data bearing on the problem, often with ready or easily built decision 224 aids and visualizations. The platform needs to ensure that the data be readily 225 discoverable and provide intuitive decision aids creation tools. Additionally, decision 226 maker feedback and utilization tracking are important for all users. 227

(b) Data Analysts / Product Generators - Data analysts support decision makers. 229

They generate decision aids and, as or more important, generate new data products to 230 support changing mission needs. They are the innovators. They are at echelon across 231 the force. They see new ways of synthesizing various data into new ways of 232 approaching a problem. Ease of use of the data platform is critical to lower barriers of 233 entry and increase velocity of change in the Army. 234

(c) Platform Enablers - Platform enablers run the technology. They ensure data 236

'pipelines' - the data ingesting, transformation, integration, and dissemination processes 237 are running and meeting mission needs. 238

7-3. HIGH LEVEL OBJECTIVES 240

The following general requirements and capability criteria are threshold capabilities: 242

The Army VANTAGE program’s capability will solve the following two broad high-level 243 problems and two broad objective needs: 244

a. High-Level Problem areas 246

(1) The Army requires the ability to see self (access all its data). 248

(2) The Army has an information technology problem of acquiring, aggregating, 250 analyzing, and integrating data that already exists. 251

b. High-Level Objective needs 253

(1) The Army at echelon needs to be able to operate (with information) at the speed 255 of war. 256

(2) The Army needs to make a fundamental shift in how leaders at echelon access 258 and use information. 259

7-4. HIGH LEVEL PROCESSES 261

Continued and improved use of the Army VANTAGE capability ensures data 263 management and data analytics at echelon. As Army VANTAGE continues to evolve 264 and adapt to the speed and capacity of the Army, it will solve high level objective needs. 265

When implemented to its capacity, Army VANTAGE will provide the serve as the 266 innovation engine that develops and refines data products to meet the Army’s missions. 267

The following (Table 1) highlights high level objective requirements. 268

HLO

PROCESS

DESCRIPTION

TO-BE STATE

User Experience

The solution should present information in a way that is immediately understandable and actionable for any end user. Inclusion of a mobile solution (all form factors) is preferred

The decision maker user experience requires an intuitive, interactive, top-down, large-scale data exploration application to allow nontechnical users to interactively explore massive-scale data in a drill-down, point-and-click application. The analyst requires an intuitive searchable interface for consumers to quickly locate and connect to data sources for use in other applications. Platform enablers require a platform that provides intuitive identification of problems with data extraction, transformation, integration, and export automation.

Data Integration

The solution should be capable of integrating both current and historical structured and unstructured data sets, perform predictive analytics, and forecasting.

Data management solution that acquires data from numerous unclassified and classified authoritative data sources in an automated manner including automated or manually initiated data “pull” jobs through a RESTful web API or direct connections. Data “push” or “subscribe” to other Army systems requiring authoritative, integrated data or to provide other joint platforms authoritative data as needed.

Provisions curated and source datasets to other Data Analytic and Software Development environments and provides high quality metadata on source system and data lineage.

Data-agnostic data integration, including structured, unstructured, and semi-structured data integration capabilities at terabyte scale that provides an intuitive interactive solution to assist with data quality ensuring completeness, accuracy, reliability and timeliness and the ability to automatically generate and curate a searchable catalog of analytic content. Data must be protected in accordance with applicable laws, regulations and policies governing Personally Identifiable Information (PII), Protected Health Information (PHI).

Analytics and Visualization

The solution should be interoperable with the Army’s API, data Catalog, and ICAM capabilities

Integrates, analyzes, and visualizes information in real-time/near real-time from multiple disparate data sources, both private and public with an intuitive, interactive, top-down, large-scale data exploration application. Processes and displays all current data and available historic Army and provides a solution that enables access and/or launches access through all modern Army approved Army devices, enabling the ability to filter, receive/view alert(s) and share information with other users as well as plaintext search capability.

Security and Infrastructure

The product and/or capability is commercially available now. As the Army continues to transform operations in the digital age, leaders have an immediate need for access to data that will meet the demands of information decision-making at all levels of command.

Capable of delivering integrated classified and unclassified content to SIPRNet users and enable the Government to pass data and insights from an unclassified solution to a classified solution. Protect data at rest and in transit using Federal Information Processing Standards and uses Army Identity Management solutions. Employs role and organizational based access controls that meets or can meet the DOD Risk Management Framework (RMF) accreditation requirements for an Authorization to Operate (ATO) on DOD NIPRNet and SIPRNet. Enables dynamic scalability of compute for advanced analytics.

Table 1: High Level Objectives 270

7- 5. REVIEW AND ANALYSIS OF ALTERNATIVES 272

a. The Army has performed two assessments of data platforms. The first, as 274 directed in the Army VANTAGE Acquisition Decision Memorandum, consisted of 275 migration of Army VANTAGE data and data logic to another platform. In 2019, the 276 program office in cooperation with US Army Cyber Command (ARCYBER) exported a 277 data product to Gabriel Nimbus, the ARCYBER data platform. The Army data and logic 278 were successfully migrated, proving the transportability of Army intellectual property. 279

However, data extraction tools in the current VANTAGE program were not available in 280

Gabriel Nimbus and so total replication of the data pipeline was not done. Of particular 281 interest to the Army was data extraction tools associated with System Analysis Program 282

(SAP) databases common in the Army’s Enterprise Resource Programs (ERP) that run 283 the Army financial and logistics processes. Additionally, the effort did not explore 284 automation tools that would facilitate migration at scale. 285

b. DELETED 287

c. DELETED 288

d. DELETED 289

SECTION 8. CONDUCT BUSINESS PROCESS REENGINEERING (BPR) AND 290

IDENTIFY CHANGES 291

8-1. BPR INTRODUCTION 293

a. The Army Business Enterprise Architecture identifies six components for the 295

Data to Decision business process. The Army Data Platform focuses on four: Collect 296

Data, Transform Data, Analyze Data and Execute Decision. The first component, 297

Generate Data, is performed by data governance activities utilizing the Data Catalog 298 portions of the data mesh. Visualize Data is included in the requirement but is a 299 common readily available capability and so is not a focus. 300

b. When the Army VANTAGE program was initiated in 2018, the business process 302 of Data to Decision was immature. The BPR focuses less on engineering changes to 303 the business process and more on capturing lessons learned from employment of the 304 capability. 305

Figure 2: Data to Decision Components 308

8-2. BPR METHODOLOGY 309

a. The Data to Decision Business Re-engineering methodology is a synthesis of 311 three efforts: lessons learned from the current Army VANTAGE program, the Data 312

Cloud EXORD, and the Data Platform requirements review performed under the 2022 313

Capability Portfolio Review (CPR) 4.1 on Software and Data. 314

b. The Army VANTAGE program has provided direct correlations between ease of 316 use of the platform, creation of ‘data objects’ now called ‘data products,’ data analytic 317 decision aids, and improved decisions. The results are shown in the People and 318

Information portions of the BPR results. 319

c. The 2020 Data Cloud EXORD and subsequent 2022 Army Data Plan shaped the 321 data policies currently being used by the Army. It identified the governance structure, 322 priority decision efforts required by the Army to be supported by data, data management 323 processes, and initial insights on how to organize the Amy components of a data mesh. 324

Results are shown in the Processes and Policy portions of the BPR results. 325

d. The CPR 4.1 requirements review expanded the Army Requirements Oversight 327

Council’s requirements analysis of the 2022 XVIII Corps Capability Needs Statement 328 and the Common Operating Environment’s Command Post Computing Environment 329

Capability Development Document. The CPR examined five other platform 330 requirements. Results are shown in the Technology portion of the BPR results. 331

8-3. BPR RESULTS 333

The BPR results are framed in terms of the BPR 5 lenses: People, Process, 335

Information, Policy, and Technology. 336

a. People. The Army VANTAGE program has provided the BPR detailed analysis 338 of who is developing data driven decision aids in the Army and the speed they are 339 created once simplified data products are provided to the community. They are the 340 people that execute the Collect Data, Transform Data, Analyze Data, and Execute 341

Decision components of the Business Enterprise Architecture (BEA) Data to Decision. 342

The content generation report identified the portions of the workforce performing the 343

Collect, Transform and Analyze portions. Note that the highest usage is within 344

FORSCOM but decision aid creation is across the force. Additionally, data analysts and 345 data creators are across the Army and across military MOS’ – a try ‘citizen analyst’ 346 capability. 347

Figure 3: VANTAGE Users By MACOM 350

Figure 4: Content Creation By MACOM 352

Figure 5: Content Creation by Rank 354

Figure 6: Content Creation by MOS 356

b. Process. The Enterprise Data Services Catalog (EDSC) provides the 357 architecture to facilitate the remainder of the data lifecycle. As such, it is critical to and 358 accessed throughout the data production, data governance, data product generation 359 and data consumption activities of Army Data Management and Analytics Strategy. 360

EDSC facilitates metadata tagging and cataloging to capture the totality of available 361 data sources and data products across the mission areas. Additionally, EDSC enables 362 the governance structure to identify and annotate authoritative data, data definitions, 363 data governance responsibilities, and implement access controls. Figure 8 crosswalks 364 specific EDSC tasks against data roles and responsibilities within the lifecycle. Figure 9 365 identifies the processes currently included and in development within EDSC. The Army 366

Data Platform will utilize EDSC data management products, such as appropriate data 367 tagging. This ensures the data used within the Army Data Platform is compliant with the 368 first component of the BEA, Define Information Needs and Goals. 369

Figure 7: EDSC Roles Life Cycle Crosswalk 372

Figure 8: EDSC Process Flow 375

c. Policy. The Army remains committed to executing the guidance and activities 376 that support data centricity. In 2019, the Army established the Chief Data and Analytics 377

Officer (CDAO) position and the Army Data Board (ADB). The Army Data Plan (ADP) 378

Executive Order released in November 2019 positioned the Army to support the May 379

2021 Deputy Secretary of Defense (DSD) memo on “Creating Data Advantage.” The 380

Army published its initial Data Governance Roles and Responsibilities Memo in June 381

2021 (updated 1 JUL 22) which further defined the responsibilities of Mission Area Data 382

Officers (MADOs), Data Stewards (DS), and Functional Data Managers (FDMs) to 383 support the ADP addressed in the ADB charter. Guidance included directing FDMs to 384 execute the data and analytics lifecycle management processes established by their 385

Data Stewards. In February 2022, the Secretary of the Army directed the Army to 386 become “more data centric” to enable success on the future battlefield. The Army 387 published additional guidance to achieve this objective. 388

Figure 9: Data Policy and Guidance Timeline 390

d. Information. As part of the Data Cloud EXORD and Army Data Plan, 59 data 391 products were identified by the Army Data Stewards to capture Title 10 data for use 392 across the Army. They are being registered in the EDSC. They have provided 393 simplified, system independent, data for analytic use across the Army. These data 394 products have been the driver for rapid improved decision making. Table 4 provides a 395 list of notable VANTAGE results and highlights the types of activity the Army Data 396

Platform can support in the Execute Decision portion of the Data to Decision BEA. 397

Table 2: VANTAGE Data Objects 405

Vantage Verified Object Name Primary Domain Primary Purpose

[AVN] Aviation Flying Hour Program Series Segmented Training & Readiness Aviation Readiness LOE

[AVN] Aviation Flying Hours Program Series Training & Readiness Aviation Readiness LOE

[AVN] Aviation Status of Funds Finance Aviation Readiness LOE

[AVN] Aviation Status of Funds Aggregated Finance Aviation Readiness LOE

[AVN] Aviation Status of Funds Monthly Finance Aviation Readiness LOE

[AVN] Aviation Status of Funds Segmented Finance Aviation Readiness LOE

APFT Score Lookup Training & Readiness USASOC LOE

ASC Fund (RAPID) Logistics ASC FVUA Inbox LOE

Asset Logistics General Purpose

BHM Customer Wait Time Logistics General Purpose

BHM NMCS Backorder Logistics General Purpose

BHM Wholesale Supply Availability Logistics General Purpose

Class Enrollment Record Training & Readiness USASOC LOE

Contract Action Reports Acquisition General Purpose

Course Training & Readiness USASOC LOE

Disposed Real Property Asset Installations, Energy & Environment General Purpose

DST EOH Readiness Logistics General Purpose

Environmental Site Installations, Energy & Environment General Purpose

Equipment Directives Logistics General Purpose

FLIPL Form Logistics General Purpose

FVUA Activity Logistics ASC FVUA Inbox LOE

FVUA Alert Logistics ASC FVUA Inbox LOE

Fvua PDF Link Logistics ASC FVUA Inbox LOE

FVUA Request Logistics ASC FVUA Inbox LOE

FVUA Submission Logistics ASC FVUA Inbox LOE

Historical Course Metric Training & Readiness USASOC LOE

Installation (Open Army) Installations, Energy & Environment General Purpose

Item: Material Number Logistics General Purpose

MRC Event Human Resources Management General Purpose

Pipeline Training & Readiness USASOC LOE

Rank Change Event Human Resources Management General Purpose

Real Property Asset Installations, Energy & Environment General Purpose

Registered FVUA User Logistics ASC FVUA Inbox LOE

Relocatable Real Property Asset Installations, Energy & Environment General Purpose

School Training & Readiness USASOC LOE

Schoolhouse Milestone Training & Readiness USASOC LOE

Serials Logistics General Purpose

Shipment (RAPID) Logistics ASC FVUA Inbox LOE

Shipment Content (RAPID) Logistics ASC FVUA Inbox LOE

Shipment Event (RAPID) Logistics ASC FVUA Inbox LOE

Shipment Invoice (RAPID) Logistics ASC FVUA Inbox LOE

Site Installations, Energy & Environment General Purpose

Soldier Human Resources Management General Purpose

Soldier Additional Skill Human Resources Management General Purpose

Soldier Award Human Resources Management General Purpose

Soldier Duty Status Human Resources Management General Purpose

Student Training & Readiness USASOC LOE

Student Admin Event Training & Readiness USASOC LOE

Student Evaluation Training & Readiness USASOC LOE

TAC Logistics ASC FVUA Inbox LOE

Tactical Purchase Requisition Logistics General Purpose

Tactical Work Order Logistics General Purpose

Tours Human Resources Management General Purpose

Transactions (Obligations by Contract) Finance General Purpose

Unit Training & Readiness General Purpose

e. Technology 406

(1) The Army Data Cloud EXORD and the recent assessments of multiple data 408 platform requirements has identified how components of the data mesh need to work 409 together and what attributes of a technology solution need to be prioritized. 410

(2) How Data Mesh components work together: The four components of the Data 412

Mesh work together to generate, manage, and distribute the data products. The data 413

Catalog provides data definitions, data discover, and data management functions to 414 automate data governance both at the system level and at the non-system, data product 415 level. The API layer provides the mechanism to request, share, publish data in 416 accordance with the governance established and managed through the Catalog. The 417

ICAM solution provides credentialling relative to the data access policies. The data 418 platform provides enterprise compute and store for data products too complicated for an 419

API layer to generate, and more importantly, is the engine for developing and refining 420 data products to meet mission. For example, a data steward can identify a cross-system 421 integrated data object as the authoritative data set for Army wide use. The data steward 422 registers the data in the data Catalog and marks it as authoritative data and established 423 the access control restrictions, if any. The data steward then ensures an API is available 424 for the data product with the controls identified. The ICAM solution needs to have the 425 credentialling in sync with the controls the data steward wants to use. The data platform 426 is uniquely suited – it provides the workspace for working with Army decision makers at 427 echelon to determine what data is needed for decision. It extracts, transforms, enriches, 428 and presents data as needed by the decision maker. The platform then can host the 429 data and provide continuous data pipelines, ensuring the data needed by the decision 430 maker meets the decision makers’ latency requirements. If the data product is useful for 431 a broader Army audience, data leadership such as a data steward, can identify the data 432 as authoritative to trigger the rest of the eco-system to provide the data in an easy 433 automated way. The platform is the innovation engine for data products and mission. 434

(3) Note that the platform provides what has been called a ‘business intelligence’ or 436

BI layer, but at an enterprise / cross system level. For example, various Enterprise 437

Resource Programs such as those that run Army finance, personnel, and logistics 438 contain not only large process engines that generate volumes of transactional data but 439

BI layers that help synthesize their data. However, they are local in nature and often not 440 optimized for ease of use for data integration and analysis. The Army Data Platform can 441 subsume this mission. 442

(4) The CPR 4.1 requirements comparison provided two conclusions: first that ease 444 of use was a priority technical capability that distinguished material efforts from success 445 and failure in terms of delivering on the BEA Data to Decision process. Those platforms 446 that were not easy to use had difficulty using the Collect, Transform, Analyze portions of 447 the BEA to support Execute Decision. Second, that incorporating analysis within a 448 platform is critical. It is a driver for innovation. Although the BEA processes for Collect 449 and Transform can be separated from Analyze in different material solutions, the 450 separation runs the risk of failure when in operational use. 451

SECTION 9. DOCUMENT PROCESS PERFORMANCE MEASURES AND 453

INFORMATION ASSETS 454

9-1. THRESHOLD REQUIREMENTS 456

a. User Experience 458

(1) Decision Makers – Intuitive, interactive, top-down, large-scale data exploration 460 application to allow nontechnical users to interactively explore massive-scale data in a 461 drill-down, point-and-click application. Presents a broad and immediate awareness of 462 actual or predicted Army status. Presents information that is immediately 463 understandable and actionable to users. Provides user defined alert(s) and ability to 464 setup and tailor an alert banner. 465

(2) Data Analysts / Product Generators – Provides an intuitive searchable interface 467 for consumers to quickly locate and connect to data sources for use in other 468 applications. Ability to conduct self-service analytics at scale; ingest, integrate, curate, 469 analyze and visualize in individual and group workspaces, providing a robust set of tools 470 that enable collaboration such as version control. Ease of use of the data platform is 471 critical in order to lower barriers of entry and increase velocity of change in the Army. 472

(3) Platform Enablers – Provide status of data within the platform. Provides intuitive 474 identification of problems with data extraction, transformation, integration, and export 475 automation. Provides easy to use platform controls, to include user access, store / 476 compute utilization, optimization tools based on usage rates. 477

b. Data Integration 479

(1) Data “pull” jobs through a Representational State Transfer (RESTful) web API or 481 direct connections that refresh data and can be run manually or on a schedule. Data 482

“push” or “subscribe” to other Army systems requiring authoritative, integrated data or to 483 provide other joint platforms (such as the OSD ADVANA platform) authoritative data as 484 needed. 485

(2) Data management solution that acquires data from numerous unclassified and 487 classified authoritative data sources in an automated manner that minimizes or 488 eliminates the requirement for manual intervention during data acquisition and enables 489 information management. 490

(3) Data must be protected in accordance with applicable laws, regulations and 492 policies governing Personally Identifiable Information (PII), Protected Health Information 493

(PHI). 494

(4) Data-agnostic data integration, including structured, unstructured, and semi-496 structured data integration capabilities at terabyte scale. 497

(5) Organizing and integrating data on NIPRNet and SIPRNet into a SIPRNet-hosted 499 solution (Both NIPRNet and SIPRNet environments with cross-domain solution). 500

(6) Provide an enterprise data platform that can produce and share data products 502 with other Data Analytic and Software Development environments. 503

(7) Provides an intuitive interactive solution to assist with data quality ensuring 505 completeness, accuracy, reliability and timeliness. 506

(8) Provides high quality metadata on source system and data lineage. 508

(9) The ability to automatically generate and curate a searchable catalog of analytic 510 content. 511

c. Analytics and Visualization 513

(1) Integrates, analyzes, and visualizes information in real-time/near real-time (within 515

24 hours of data updates) from multiple disparate data sources, both private and public. 516

(2) Intuitive, interactive, top-down, large-scale data exploration application to allow 518 nontechnical users to interactively explore massive-scale data in a drill-down, point-and-519 click application and filter billions of pieces of data into a digestible set of useful 520 information. 521

(3) Processes and displays all current data and available historic Army data as 523 needed to provide future predictive analytics (5, 10, 20 + years into the future). 524

(4) Provides a solution that enables access and/or launches access through all 526 modern Army approved (Classified and Unclassified) Army devices (desktop, laptop, 527 tablet and mobile). 528

(5) Provides ability to filter, receive/view alert(s) and share information with other 530 users. 531

(6) Provides plaintext search capability across all data. 533

d. Security and Infrastructure 535

(1) Capable of delivering integrated classified and unclassified content to SIPRNet 537 users and enable the Government to pass data and insights from an unclassified 538 solution to a classified solution. 539

(2) Enables dynamic scalability of compute for advanced analytics. 541

(3) Use Army Identity Management solutions. 543

(4) Protect data at rest and in transit using Federal Information Processing 545

Standards (FIPS) 140-2 approved and validated algorithms for encryption, hashing, and 546 signing (e.g., full disk encryption, Kerberos). 547

(5) Role and organizational based access controls that are assignable to individual 549 users or groups of users and are securely stored in lists, which can be updated at any 550 time. 551

(6) That meets or has the ability to meet the DOD Risk Management Framework 553

(RMF) accreditation requirements for an Authorization to Operate (ATO) on DOD 554

NIPRNet and SIPRNet. (DOD Risk Management Framework (RMF) Package 555

Deliverables, DI-MGMT-82001). 556

e. Key System Attributes 558

(1) Ability to export / import / reuse current hosted data along with open source code 560

(e.g., R, Python, SQL) used to develop / build logic for data transformation / curation 561 currently. 562

(2) Ability to have 45,000 named user accounts and scalable to enterprise. Account 564 access includes both NIPR and SIPR instances of the platform. Decision Maker users 565 represent 40k users, Data Analysts / Data Producers 5k, Data Enablers 100. Note. 566

*Base cost will need to include expected user base. 567

(3) Help desk and training support 569

9-2. OBJECTIVE REQUIREMENTS 571

a. Integrate Artificial Intelligence (AI) / Machine Learning (ML), Robotic Process 573

Automation (RPA) and other analytic capabilities directly within the platform. 574

b. Provide an “agility” layer to transactional systems: once a decision is identified in the 576 data analytics platform, provide the ability to transmit transaction requests back to a 577 transactional system based upon that decision utilizing cross platform identify 578 management and permissions and is compliant with IT Audit General Controls. 579

c. Provide the ability to prioritize computational and data exchange activities. In a DIL 581 environment, network and computational abilities will be stressed. Based on 582 operational need, data activities (e.g. ETLs, data export, data ingest) that task 583 limited resources (e.g. compute and network throughput), need to be able to be 584 prioritized by the user to enable priority missions. 585

9-3. OTHER MANDATORY REQUIREMENTS 587

a. Transition Requirements. The data, data workflow, and display logic in the 589 current platform must be transferred to any new platform. This includes: 590

(1) Existing Extract, Transform, Load (ETL) logic and curated data. 592

(2) Linkages to data sources at the current frequency and capacity. 594

(3) Support the current user and analytic workload. 596

(4) Display graphics and dashboard user experience. 598

b. Oversight Threshold Requirements. Government subject matter experts will 600 assess the utility of the solution capability to meet the following four system needs: 601

(1) Provide access to a dashboard and various levels of data with no system time 603 delays (seconds not minutes). 604

(2) Provide semi to fully automated recognition of data errors. 606

(3) Demonstrate scalability from meeting the needs of one senior leader user, then 608 other designated senior Army leader users, and then scale to a broad set of users, 609 potentially at all levels of command. 610

(4) Synchronize with EDSC for data lifecycle management functions. 612

SECTION 10. DEVELOP DOCTRINE, ORGANIZATION, TRAINING, MATERIEL, 613

LEADERSHIP AND EDUCATION, PERSONNEL, FACILITIES AND POLICY 614

(DOTMLPF-P) ACTIONS AND CAPABILITY IMPLEMENTATION PLAN 615

10-1. CAPABILITY IMPLEMENTATION PLAN (CIP) 617

The Capability Implementation Plan (CIP) serves as the program’s information 619 repository. Product Office Army Data Platform (PdM ADP) in support of the CDAO will 620 develop a CIP IAW with DoDI 5000.75. The PMO, with support from the CDAO, will 621 continuously update the CIP during the Acquisition, Testing, and Deployment and 622

Capability Support phases. 623

10-2. DOTMLPF-P 625

a. The primary DOTMLPF-P implications have been provided in BEA results 627 sections containing implications for people, organization, material, leadership and 628 policy. Additional DOTMLPF-P considerations are: 629

b. Organization: In order to adopt the Data ownership by Domain key principle, data 631 domains will become responsible for additional data management activities that were 632 previously provided as a service under the current Army VANTAGE program. 633

c. Training / Leadership: PdM ADP has provided on-line Army VANTAGE training 635 material available on MILSUITE. Depending on the material solution selected, this 636 training material will need to be updated. Additionally, the Army VANTAGE products 637 have been provided to select areas of the training base (e.g., Army Force Management 638

School), these will need to be re-assessed depending on the scope of the solution. 639

10-3. ARMY VANTAGE 2.0 PROGRAM SCHEDULE 641

TBD 642

10-4. DELETED 644

10-5. DELETED 646