Advana_Overview_v1.0.pptx

September 2024 Advana Overview

Introduction

Introduction Who or what is Advana Advana – a mash-up of the words “Advancing Analytics” – is a technology platform, which, in addition to housing a collection of enterprise data, is pushing beyond simply being a data warehouse; we are arming military and business decision makers across the Department with decision support analytics, data management and data science tools, and associated support service

Vision & Mission

Creating Data Advantage Memo, signed by DSD on 5 May 2021, “Senior Leader Decision Support. Common access to authoritative data is essential for providing a "single source of truth" for objective and informed decision-making. The Advancing Analytics (Advana) platform is the single enterprise authoritative data management and analytics platform for the Secretary of Defense, Deputy Secretary of Defense, and Principal Staff Assistants (PSAs), with inputs from all DoD Components.”

OUSDC FMR, DoD 7000.14-R Financial Management Regulation Volume 1, Chapter 10, “2.2.1. Provide authoritative source-level transactional data to Advana from all DoD business systems daily, using the most cost effective and secure methods available. The authoritative source-level transactional data must be obtained and imported by Advana from the business system in which the data originated (i.e., not a separate data warehouse or data lake but the system that originated the transaction).”

Designation of Enterprise Services Supporting Federated Data Cataloguing, singed by DoD Chief Data Officer (CDO) on 2 December 2020, “This memorandum designates the Advanced Analytics (ADVANA) program … as an enterprise service provider for data cataloguing on the Unclassified and Secret fabrics, as well as the DoD’s singular interface with the Federal Data Catalogue.”

Army Business Health Metric Data Memo, signed by Director of Business Transformation Principal Cyber Advisor on 2 July 2022, “The first of five DOD Data Decrees is to maximize data sharing and rights for data use … The Secretary of the Army's establishment of the data-centric Army priority objective extends to our institutional leadership and management responsibilities, as well as to our operating forces. At Headquarters, Department of the Army, as well as with our operating forces, data is leader business. I ask that you implement the following practices within your organizations and with your Office of the Secretary of Defense (OSD) and Joint Staff counterparts to better understand and improve our responsiveness to both their and our data requirements … Understand your data and authoritative data source environment. Collaborate with your counterparts to identify and share the best authoritative data to meet the Army and OSD's BHM needs.”

Army Data Access Guidance, signed by Army Chief Data Officer on 4 August 2022, “2. The Secretary of the Army (SecArmy) provided overall guidance on the need to create a data centric Army and specified guidance on the role of data services that will facilitate this change. This guidance is further reinforced by the Deputy Secretary of Defense (DSD) memorandum on Creating Data Advantage. In particular, the data sharing decree is of critical importance both for Army and DoD initiatives … 2. The Secretary of the Army (SecArmy) provided overall guidance on the need to create a data centric Army and specified guidance on the role of data services that will facilitate this change. This guidance is further reinforced by the Deputy Secretary of Defense (DSD) memorandum on Creating Data Advantage. In particular, the data sharing decree is of critical importance both for Army and DoD initiatives.”

Founded in Doctrine and Policy VISION: Change decision-making behavior across the DoD enterprise using data and analytics.

MISSION: Make data widely accessible, understandable, and actionable across the DoD enterprise by translating common enterprise data into profound yet actionable insights, decisions, and outcomes.

PROBLEM STATEMENT: Historically, it was nearly impossible to meet the demand for the enterprise-wide common operating picture needed to deliver strategic, timely insights based on data because of organizational barriers, siloed data ownership, and antiquated systems. The conflicts and crises of today and tomorrow require modernization of the way we access data and develop analytics for making decisions.

Question: What is the problem you’re trying to solve?

Advana aims to provide decision advantage to the Department’s senior most leaders in every information domain by providing cost effective/free access to enterprise data and data management, analytics, and AI/ML capabilities operating on NIPR, SIPR, JWICS, SUNet, and at the Edge. The capabilities Advana will provide include Industry leading data automation, data ingestion, data science, and AI/ML capabilities operating on NIPR, SIPR, JWICS, Edge, and SUNet. Advana provides the management, oversight, and engineering skillsets to integrate all enterprise systems and meet our mission requirements inclusive of a centralized data repository to perform data and AI/ML operations for the entire DoD.

I can’t easily determine the accuracy of the data I am being asked to evaluate I can’t find the information I am looking for in the analytics I can’t trust AI/ML outputs as they are not transparent Consumers Executives I want to leverage my data assets to make timely and well-informed decisions

I can’t find my organization in the data provided I cannot interpret the analytics provided The tools and applications I use are not integrated

Business/Mission Analyst I want to review models and dashboards

Builders Data Scientist I want to explore data, experiment, and deploy models Business Intelligence Developer I want to query data and build dashboards

IT

Platform Engineer I want to build and maintain scalable data platforms Challenges I can’t find the data or models I need to produce my analytics The tools I am using are not integrated Creating data products are too manual I cannot easily share data I cannot move data from Low to High easily

Common problems experienced by customers: access to quality, trustworthy data and the data literacy and tools to make use of that data at the speed of relevance

Warfighter / Business Operator I want decisions and workflow to be automated

I can’t make decisions fast enough Everything is too manually intensive I cannot connect the dots quick enough in a crisis I don’t have all the tools I need to create models I cannot deploy or monitor the model I created I cannot find high quality data to train high quality models I don’t have an automated way to test and assure models I build Infrastructure and tools are difficult and costly to authorize and maintain in DoD Maintaining a common user experience across tools and networks is difficult Getting access to platforms for data sharing is a challenge Mission Partners

Mission Partners I need a place to collaborate with non-traditional partners I cannot easily get the data I need I need to partner with other nations in times of conflict I cannot perform my research on all the data I need Customer Base

Question: Describe the key Customer(s)/end users.

The customer and end users of Advana products include the 100,000+ Advana users across the Department, 9 community spaces, all CCMDs, all Military Services, JS Directorates, and PSAs, as well as the whole of CDAO providing capabilities to the Department. In the future, the intent is to open this up to anyone with a DoD CAC, whereas today accounts are by request. Users of Advana range from senior leaders and decision makers, data scientists, analysts, AI/ML engineers, platform engineers, Military, PSAs, and the Intelligence Community.

Search data sets across platform Discover trending data Get recommendation based on your searches

Request new Advana project workspace Develop data science and machine learning algorithms against data Run containerized models on scalable infrastructure Visualize data and resulting models with drill-down dashboards Develop reports accessible to your community

Web-based access Horizontally (infinitely) scalable Containerized enclaves Role based access

1. Discover Data

3. Query and Analyze

4. Visualize Results

Advana Approach View data profile pages Understand uses of data Request access to data Advana Principles Advana provides users with a seamless experience and journey across the platform from discovering the data all the way to getting insights from the data

2. Understand Data

Customer Base (cont.)

Executive Decision Support: Advana’s Pulse Platform delivers 372 metrics for DoD executive reporting. Pulse’s tools and dashboards drive organizational performance in several forums, including DMAGs and the Deputy’s Steering Group. HON Austin and HON Hicks’ adoption of Pulse is encouraging data-driven decision-making from headquarters to the field.

Department of Defense (DoD)-wide Impact Advana is changing the way DoD makes decisions. With 100K+ active users, Advana’s impacts are far-reaching, touching a breadth of domains and DoD communities Financial Management: Advana’s Fund Balance with Treasury (FBwT) application was used by Navy (General Fund) and Army (Working Capital Fund) to downgrade their FBwT Material Weaknesses in FY23.

Community Spaces: Advana’s Community Spaces allow DoD agencies and military departments to curate data and build applications with a great degree of autonomy. The top 30 dashboards across Advana’s ten (10) Community Spaces have logged 52,400 hours of user interaction time.

Incident Response: Advana supports the Department’s response to the Israel crisis by delivering data and analytics for operational planning, situational awareness and decision-making. The Combatant Commands have especially benefitted from Advana, using the Platform to understand capability requirements, force flow, logistics footprint, and personnel disposition across the theater.

Advana by the Numbers

100K+ Active Users 450 Data Sources 90K Data Sets 18 Petabytes of Data 100+ Product Lines

By the Numbers

Number of Account Holders:

101,000+ Number of Applications in Production:

500+

Number of Data Sources:

500+ (daily, weekly, monthly feeds)

Number of Accredited Networks:

3 (NIPRNet. SIPRNet, JWICS) Number of DoD Organizations:

55+1

1.) Consists of all Military Departments and Fourth Estate organizations to include OSD Components, DAFAs, and Combatant Commands.

Capabilities

Security Network & Infrastructure Service Layer/ API MLOps/Self Service AI/ML SW Factory Data Governance Data Ops UI / Apps Platform

Service Portal & Home Page Data Catalog Decision Support Tools/Products

Self/Service Data/BI Tools (COTS) User Profile File Store Search Access Control Ontology/ Semantic Data Collaboration Service Desk Knowledge Base Product Management and Document Management Email Notifications Atlassian-as-a-Service Service Abstraction Layer / Engine AI/ML Gateway Data Gateway

Access Control

RBAC/ABAC

PII/PHI Masking and handling Classification Guidance Complex Policy Enforcement Data Quality Accuracy Completeness Timeliness Consistency Data Acquisition Manual Upload/SFTP/API Streaming/Batch

JSON/XML

REST/SOAP

Data Storage S3/Blob (Scalable) Parquet/Delta DW, Delta, Graph Bronze/Silver/Gold Data Processing Elastic and MPP Spark

CPU/GPU

Tagging/Curating Data Orchestration Auto Promotion Cross-Domain Solution Self-Service Promotion Automation Service Bus Messaging (Pub/Sub) Service Registry / Discovery Security

Security

IDAM

Encryption Firewall/ continuous Pen Test 3rd Party Assessments Container Orchestration Monitoring Metadata Repo Security Container Registry

CI/CD

Build Scan Test Deploy Monitoring Logging/Alerting Anomalous Behavior Performance Service Availability

Network

CAP/BCAP/IAP

Cross-Domain Integration Multi-Cloud Integration

Cloud Infrastructure AWS (IL5, IL6, JWICS) + Azure (NIPR/SIPR/JWICS) Scalable and Elastic Regions + Availability Zones (support 24x7 global SLAs) IDS, Firewalls, etc.

Data Catalog Metadata Capture Search Data Lineage Federated Catalog Self/Service AI/ML Dev Tools (COTS) Data Modeling Ontology Reference/Master Data Common Data Models Data Architecture Experiment and Test Package/Lib Mgmt.

Code/Low Code Dev & RPA Automation Feature Store Train/Evaluate/Validate SW Dev and Test Self-Service Publish Model Deployment and Operations Source Code Control CI/CD Publish/Deploy Model Registry Model Serving and Monitoring Model Inference Service Model Performance Data Gateway Automation Models API Gateway

Institutionalize data, analytics, and responsible AI-driven decision advantage Custom dashboards and analytics applications Empower the DoD workforce to use analytics and AI SIPR Home/Search application deployment EDL SIPR deployment Apigee SIPR Deliver a delightful, reliable customer experience Parity across NIPR/SIPR/JWICS Improved software release management processes SIPR Community Space Web Page deployments Data Catalog 2.0 improvements Streamsets sunset/migration Provide secure, scalable, and cost-effective solutions Advana 1.2 Keycloak Containerized SSO upgrade JWICS User Admin application deployment Deliver automated AD entitlement onboarding Provide world-class data management and governance Immuta Data Engineer/Builder NIPR General Availability Immuta SIPR Enable model scanning pipelines Accelerate DoD’s data, analytics, and AI journey with world-class offerings JExNet SIPR and JWICS deployment AoA for Databricks SaaS Edge deployed to SIPR AI/ML model serving via Perceptor MLOps CI/CD tooling on NIPR

Advana Capabilities

Advana Environments

Past, Present, and Future States

Origin & Evolution of an Enterprise Analytics Platform

THE FUTURE

CFO Act of 1990 gives OMB new authority and responsibility for directing federal financial management, modernizing the government's financial management systems, and strengthening financial reporting

The Office of the Under Secretary of Defense, Comptroller and Chief Financial Officer (OUSD(C)) began building a universe of transactions platform to support DoD's financial statement audits

(Jan 2021) OUSD(C) memo ”Disestablishment of the Chief Management Officer of the DoD and Realignment of Functions and Responsibilities” aligns CPIO functions under OUSD(C); Enterprise Data and Business Performance (EDBP) is established Defense Business Board assessment of DoD’s Chief Management Office (OCMO) results in the disestablishment of OCMO and distribution of responsibilities, inclusive of Fourth Estate alignment, to OUSD(C)/CFO Universe of Transactions presence is focused on NIPR and SIPR, hosted in a USMC Data Center. UOT becomes critical to supporting FIAR Governance Board, has a presence of 250 users, and supports 6.6 Billion transactions for Financial Management

Advana migrates NIPR to AWS GovCloud, footprint grows from 250 users to 17,000

Advana expands support to all military services and offers Community Spaces (née "enclaves") to organizations requiring curated analytics support, starting with Department of the Navy (DON) - Jupiter; (6 additional Community Spaces as of 4/2022)

Initial Operating Capability (IOC) for Advana achieved on JWICS

Establishment of the AI and Data Accelerator (ADA) Initiative to support Combatant Command data and AI needs through embedded Operational Data Teams

Establishment of Chief Digital and Artificial Intelligence Office, responsible for strengthening and integrating data, artificial intelligence, and digital solutions in the Department.

Expansion into Artificial Intelligence

COVID-19 Task Force (CVTF) established to support pandemic related executive analytics; later expanded to provide Incident Response support to Afghan NEO and Ukraine crisis.

UOT is rebranded as Advana; Data Management and Analytics Steering Committee (DMASC) approved both the technical architecture and the tool suite employed by Advana to be "the shared service data platform for all common enterprise data."

GAMECHANGER established to transform policy analysis and discovery across DoD (9 clones as of 4/2022)

DoD Data Decree memo is released to “generate transformative proficiency across the DoD data strategy’s focus areas of Joint All-Domain Operations, senior leader decisions support, and business analytics.”

Question: Describe how the program started?

Advana was established to support the OUSD-C financial statement audit and supported the 2018 NDAA sec 911, 912, and 913. The platform then was requested by DSD to scale to support all information domains within DoD. Through continued growth, the addition of business health data, executive analytics data, and eventually operational data grew Advana into the leading DoD enterprise-level analytics and AI platform.

Stakeholders “Focus on Audit and PSAs” Focused on OUSD(C), rooted in audit Some Support to the PSAs under OUSD

person in the Department” Expanded community spaces and edge workspaces Mission “Solve for Audit and Finance Data” Focus on Business Missions / DoD “Back office” Slow expansion into functional communities (logistics, etc) “Expand to Operational Mission Support” Embracing CCMD missions

Business system data (back office data) for DoD Finance, People, health, etc Focus on PII, privacy and security of data Data catalogued MOAs implemented “Catalog and mature data to be analytics ready” Focus on cataloging and exposing analytics Data matured to be analytics ready Data APIs Focus on data lineage to authoritative sources and data quality “Extend standardized data to the edge” "Democratize data" Technical Architecture “Focus on creating capability” Domains: NIPR / SIPR (AWS) Minimal cross domain Expand across capabilities Multi-tenant “Expand to support Mission architecture” Domains: NIPR / SIPR / JWICS (AWS) and NIPR (Azure) Bifrost NIPR-SIPR-JWICS Support multiple tools per capability Multi-tenant “Streamline while empowering down to mission architecture” Multi-domain, multi-cloud Bring your own tool; streamlined set of self-service tools Multi-tenant and disadvantaged user Operating Model “Advana does for you” Customers pay into Advana

Customers subscribe to the Advana Ecosystem Self Service Analytics, Data, and Tools in any environment

Digitized Platform Connected Multi-Domain Platform

(NIPR/SIPR/JWICS)

Optimized Self-Service Platform (work in progress)

FUTURE STATE

CURRENT

PAST

Past, Current, and Future States

Advana provides a set of curated capabilities to empower users across the Department of Defense – including analytics-ready data pushed through our medallion framework, a set of tools to enable analytics and AI, and a set of mission focused analytics to empower decision making. Our users need continuous 24/7 access to enterprise data and enterprise capabilities at the lowest cost possible. All operating within a 100% multi-domain Gov and Mil-owned environment. This includes hosting marquee applications such as Audit, DSD BHM, Pulse, Federated Data Catalog, J4 Op Log, J3 Orion, MRI, and 300+ additional applications, access to an industry-leading data integration layer and enterprise data store, access to an industry-leading AI/ML ops capability that maintains an ATO in all classifications, and seamless data transfer from NIPR to SIPR and SIPR to JWICS.

Past, Current, and Future States (cont.)

CURRENT STATE: Advana is used 24/7 across the globe and includes thousands of data pipelines feeding critical operational analytics assisting senior leadership make data-driven decisions and is wholly a “white glove service” which is unsustainable at scale.

Further, the platform is centrally funded by CDAO, while further customization and Community Spaces are modeled after a “pay to play” business model.

FUTURE STATE: CDAO is moving towards empowerment through the mission space concept that would allow for data sharing across authorization boundaries for customers that wish to utilize their own technical stack.

Advana is continuing to experience viral adoption across the DoD. Today, a large portion of Advana is centrally funded by CDAO, while further customization and Community Spaces are modeled after a “pay to play” business structure, based on an initial Rough Order of Magnitude. In the future, the intent is to move towards both a consumption model for licenses and infrastructure and to empower the mission space concept that would allow for data sharing across authorization boundaries for customers that wish to utilize their own technical stack. The platform is currently used 24/7 across the globe and includes thousands of data pipelines feeding critical operational analytics assisting senior leadership make data-driven decisions.

image2.jpeg image3.png image4.svg image1.png image18.png image19.png image20.png image21.png image22.png image23.png image24.png image25.png image26.png image27.png image28.png image37.png image29.png image30.svg image31.png image32.svg image33.png image34.svg image35.png image36.svg image38.emf image39.emf image40.emf image41.png image42.png image43.png image51.svg image52.png image53.svg image54.png image55.svg image56.png image57.svg image58.png image59.svg image60.png image61.svg image62.png image63.svg image64.png image65.svg image66.png image67.svg image68.png image69.svg image70.png image44.png image71.svg image72.png image73.svg image45.svg image46.png image47.svg image48.png image49.svg image50.png image17.emf image74.png image14.png