GFM_Data_WDP_Gold_Tables_PWS_11Jun2026.docx

UNCLASSIFIED

(DRAFT - PREDECISIONAL)

Part I. Performance Work Statement (PWS) Title: GFM Decision-Ready Data Products (Gold Tables) Services in the SIPRNet War Data Platform Program: Department of War Federated Data Platform – War Data Platform (formerly Advana) Managed By: OSW Chief Data and Artificial Intelligence Office (CDAO) Environment: SIPRNet War Data Platform (WDP)

1.0 Purpose

This PWS defines the requirements for contractor support to engineer, implement, and sustain a modernized, next-generation Global Force Management (GFM) data lifecycle within the SIPRNet War Data Platform (WDP). Transitioning away from legacy methodologies, the contractor shall apply modern Digital Engineering and DataOps best practices to accelerate the movement of GFM data from originating Authoritative Data Systems (ADS). To minimize development time and align with existing ecosystems, the contractor must leverage proven, deployed Government Off-The-Shelf (GOTS) and Commercial Off-The-Shelf (COTS) technologies, using a mix of cloud-native (e.g., serverless functions) and non-native architectures for optimal data pipeline automation.

Operating within the WDP Lakehouse architecture, the contractor shall process GFM data from Bronze (raw, source-aligned ingestion) to Silver (cleaned, validated, and enriched data) layers. Crucially, the contractor will employ Advanced Data Modeling techniques to develop “Single Source of Truth” GFM Gold Tables. These Gold Tables must be architected to strictly align with Joint Fires Network (JFN) and Intelligence Community (IC) "Golden Objects" (Red and Blue Object Management) – highly refined, authoritative, and standardized version of data entities.

To support advanced analytics and autonomy, the contractor shall use a diverse, multi-tool suite of Artificial Intelligence (AI) capabilities for data hygiene, conversion, and interpretation. The ultimate objective is to ensure all final GFM Gold Tables are delivered in a universally "AI-Ready State" – eligible for ingestion by any platform or model without vendor lock-in. Finally, all updates to data, scripts, and models must be executed using a Blue/Green deployment architecture to ensure continuous, zero-downtime availability for production applications supporting the GFM mission across the Department.

2.0 Background

The Department of War Federated Data Platform is the War Data Platform (WDP), managed by OSW CDAO. WDP is a Federated Data Lakehouse environment that supports ingestion, transformation, storage, governance, analytics, BI, applications, APIs, ML models, and LLM-enabled capabilities across Department mission and enterprise domains. WDP organizes data using a Bronze/Silver/Gold architecture, in which Bronze preserves raw ingested source data, Silver provides cleaned and enriched data, and Gold provides highly curated, aggregated, business-ready data optimized for Department use cases.

The Government requires contractor support to operationalize WDP capabilities on SIPRNet to create trusted GFM Gold Tables for Joint and Service communities in support of the GFM mission across the Department.

3.0 Scope

The contractor shall:

Establish repeatable processes for the end-to-end GFM data lifecycle in the WDP.

Ingest GFM data from approved Joint and Service source systems as WDP Bronze Tables.

Clean, validate, enrich, standardize, and align data as WDP Silver Tables.

Curate, version, govern, and publish decision-ready WDP Gold Tables in an accessible location within WDP.

Ensure availability of GFM Data in a common location where all GFM Gold Tables.

All GFM Gold tables shall be connected to the WDP Data Catalog for downstream access, search, and consumption through approved WDP-native access patterns.

Establish and execute a comprehensive Data Intelligence & Active Data Governance framework, including automated data quality monitoring, end-to-end data lineage, active metadata management, and process governance to ensure all GFM data products are trusted, auditable, and decision-ready.

Execute continuous Data Requirements Engineering & Discovery to define target schemas in coordination with GFM stakeholders.

Engineer AI Vectorization & Embedding Generation pipelines to convert GFM Gold Tables into AI-ready formats.

Leverage existing WDP AI capabilities to accelerate mapping, profiling, alignment, anomaly detection, and documentation.

Provide program management, training, sustainment, and operational support using Agile methodologies (e.g., SAFe, Scrum).

Out of scope unless specifically authorized:

· Development of new WDP core platform capabilities.

· Modification of authoritative source systems.

· Delivery outside SIPRNet.

4.0 Definitions

· GFM Data: Global Force Management data supporting force structure, force sourcing, readiness, allocation, assignment, availability, directed readiness related operational decision-making and their associated assessments.

· Bronze Layer/Tables: Raw source data ingested as received, with source fidelity preserved.

· Silver Layer/Tables: Cleaned, validated, standardized, and enriched data derived from Bronze.

· Gold Tables: The most refined data asset in the WDP, representing a curated, consumption-ready, and mission-focused data product. It is designed for high-performance analytics and is the authoritative source for specific business domains or mission areas. Gold Tables must meet the following criteria:

· Schema Standards: Data is explicitly modeled into analytical schemas optimized for read performance, such as Star or Snowflake schemas, and strictly aligned with JFN/IC "Golden Objects" (Red and Blue Object Management).

· Storage & Performance: Stored in optimized Databricks Datalake format to support high-performance, low-latency analytical queries from tools like Power BI, Qlik, and Tableau, as well as AI/ML model training.

· Aggregation & Business Logic: Contains business-level or mission-level aggregated data, tailored for specific reporting requirements and analytical objectives. All transformations and business logic applied must be documented and version controlled.

· Metadata & Naming Conventions: Governed by strict, semantic metadata tagging and standardized naming conventions consistent with the DoD Data Strategy's VAULTIS (Visible, Accessible, Understandable, Linked, Trustworthy, Interoperable, Secure) goals.

· Access Model: Secured via robust Identity and Access Management (IAM) controls and Attribute-Based Access Control (ABAC). Access is provisioned strictly to authorized end-users based on mission alignment and data classification.

· Governance & Promotion: A data product can only be promoted to "Gold" after passing a formal governance gate that validates data quality, schema compliance, comprehensive metadata, auditable lineage back to the Bronze layer, and all security controls.

· Decision-ready: Fit for operational or analytical use, quality-assured, traceable, timely, and aligned to agreed business rules and standards.

· Authoritative source: Government-designated system or provider responsible for a given data element or dataset.

· Conceptual Data Model: A high-level, mission-focused representation of the GFM data domain. It defines the core entities (e.g., Units, Forces, Equipment, Readiness) and the fundamental relationships between them, designed to align stakeholders on business requirements without technical database details.

· Logical Data Model: A detailed, technology-independent blueprint of the GFM data architecture. It defines all specific data fields (attributes), unique identifiers (keys), semantic definitions, business rules, and precise relationships. This serves as the authoritative, vendor-agnostic representation of GFM business logic, independent of physical storage or AI technology.

· Physical Data Model: A technology-specific database design that represents how GFM data is physically stored and structured within the War Data Platform (WDP) environment (e.g., as Databricks Delta tables, SQL schemas, or Vector Stores). It defines actual database data types, partitioning, indexing, and storage parameters required for technical execution.

· AI Vectorization: The computational process of transforming human-readable data (text, documents, or structured tables) into numerical arrays (vectors) that Artificial Intelligence (AI) and machine learning models can read, process, and analyze.

· Embedding Generation (Embeddings): The creation of advanced mathematical representations of data that capture its semantic meaning and context. Unlike traditional keyword matching, embeddings allow AI systems to understand the conceptual relationships and intent behind different GFM data points.

· Vector Store (Vector Database): A specialized data storage system designed specifically to store, index, and efficiently query vector embeddings. It serves as the required infrastructure layer to connect curated Gold Tables to Large Language Models (LLMs) and autonomous agents.

· Retrieval-Augmented Generation (RAG): An AI architectural pattern that queries a Vector Store for authoritative, trusted data to feed into an LLM before it generates a response. This grounds the AI in approved GFM data, ensuring accuracy and preventing fabricated outputs (hallucinations).

· AI-Ready State: Data that has been cleaned, governed (Medallion Gold tier), vectorized, converted into embeddings, and loaded into a Vector Store, making it immediately available for consumption by an AI model without requiring further manual engineering or transformation.

· Schema Standards: Data is explicitly modeled into analytical schemas optimized for read performance, such as Star or Snowflake schemas, and strictly aligned with JFN/IC "Golden Objects" (Red and Blue Object Management).

· Storage & Performance: Stored in optimized Databricks Delta Lake format to support high-performance, low-latency analytical queries from tools like Power BI, Qlik, and Tableau, as well as AI/ML model training.

· Aggregation & Business Logic: Contains business-level or mission-level aggregated data, tailored for specific reporting requirements and analytical objectives. All transformations and business logic applied must be documented and version controlled.

· Metadata & Naming Conventions: Governed by strict, semantic metadata tagging and standardized naming conventions consistent with the DoD Data Strategy's VAULTIS (Visible, Accessible, Understandable, Linked, Trustworthy, Interoperable, Secure) goals.

· Access Model: Secured via robust Identity and Access Management (IAM) controls and Attribute-Based Access Control (ABAC). Access is provisioned strictly to authorized end-users based on mission alignment and data classification.

· Governance & Promotion: A data product can only be promoted to "Gold" after passing a formal governance gate that validates data quality, schema compliance, comprehensive metadata, auditable lineage back to the Bronze layer, and all security controls.

5.0 Applicable Documents

The Government shall provide, as applicable:

· GFM business rules, standards, and canonical models.

· WDP technical guidance, patterns, and governance procedures.

· SIPRNet and WDP security policies.

· Source system access approvals, data sharing agreements, and interface specifications.

· Existing reference data, code sets, and data dictionaries.

· Any previously provided needs, capabilities, and deliverables germane to this effort shall be considered reference material. All contract performance, however, will be measured against the requirements defined within this PWS and any formal GFM business rules, standards, and canonical models provided by the government post-award.

6.0 Government-Furnished Information / Equipment

6.1 Government-Furnished Information (GFI)

The Government shall provide:

· Access to SIPRNet WDP tenant, approved workspaces, and relevant services.

· Available WDP-native ingestion, storage, transformation, governance, and AI capabilities.

· Source system contacts, sample extracts, and Government SMEs.

· Existing GFM schemas, quality rules, and governance artifacts.

· WDP AI capabilities inventory on the SIPRnet.

· The following criteria and metrics established by the Data SWG established under the GFM WG per CJCSI 3256.01B, C2 Governance & Management:

· Named Gold Tables

· Named source systems and source priority

· Quantitative Service Level Agreements (SLA) and Service Level Objectives (SLO) as they relate to capability expectations

· Gold promotion criteria for each or all Gold Tables as appropriate

· Government approval authority and processes for mappings, rules, and release promotion.

6.2 Government-Furnished Equipment (GFE)

The Government will provide access to the approved inventory of WDP-native services on SIPRNet. This includes, but is not limited to, Databricks for data transformation, Unity Catalog for governance (or as appropriate for SIPR security fabric), and specific AI-enabled services for data profiling, schema discovery, anomaly detection, and assisted data mapping. The exact toolchain and service endpoints will be provided post-award.

7.0 Place of Performance

Work shall primarily be performed within the SIPRNet WDP. On-site support may be required for classified coordination, source onboarding, testing, or security-related activities.

8.0 Period of Performance

As specified in the contract. The contractor shall deliver initial operational capability nine months into the base period and incrementally expand Gold Table coverage thereafter.

9.0 Performance Requirements

9.1 Program Management

The contractor shall:

1. Provide a Program Management Plan (PMP) within 15 calendar days of award.

2. Provide and maintain an Integrated Master Plan / Integrated Master Schedule (IMP/IMS) focused on continuous delivery and predictable velocity.

3. Conduct recurring status reviews and provide progress, risk, and issue reporting.

4. Maintain a risk, issue, assumption, and dependency log.

5. Execute continuous Requirements Discovery: Led by the Data Product Manager, the contractor shall proactively engage stakeholders to elicit, validate, and document data requirements and schemas prior to engineering, rather than relying solely on Government-provided requirements.

9.2 Bronze Layer Source Onboarding and Ingestion

The contractor shall:

1. Identify, document, and onboard approved GFM source systems.

2. Implement secure WDP-native ingestion pipelines to Bronze, utilizing a mix of cloud-native (e.g., serverless functions) and non-native architectures to optimize automation.

3. Preserve source fidelity, provenance, timestamps, and source metadata.

4. Implement intake checks for schema, record counts, required fields, and change detection.

5. Use proven, deployed Government Off-The-Shelf (GOTS) and Commercial Off-The-Shelf (COTS) technologies and approved AI capabilities for profiling and schema discovery to minimize custom development time.

9.3 Silver Layer Cleansing, Validation, and Enrichment

The contractor shall:

1. Implement cleansing for duplicates, missing values, invalid records, and data type inconsistencies.

2. Apply validation rules for referential integrity, domain constraints, temporal logic, and business rules.

3. Enrich Silver data using approved reference data and crosswalks.

4. Standardize identifiers, timestamps, codes, and formats.

5. Track validation failures, exceptions, and remediation actions within an automated Data Intelligence framework.

6. Provide a report feature back to the data source system to ID what data has been changed to inform the authoritative system of changes made.

9.4 Canonical Alignment and Standardization

The contractor shall:

1. Align Joint and Service data to the approved canonical GFM model using Advanced Data Modeling techniques.

2. Implement and maintain mapping specifications and crosswalk tables.

3. Version all mapping logic and reference data changes.

4. Document AI-assisted mapping decisions and subject them to Government review.

5. Logical Data Modeling & Business Semantics: The contractor shall not deliver physical Gold Tables without first engineering and documenting the underlying data architecture. Prior to physical implementation, the contractor shall develop and deliver Conceptual and Logical Data Models that define the authoritative business understanding, entity relationships, and semantic definitions of the GFM domain. These models must be designed to be agnostic of the underlying physical storage or AI vendor, ensuring long-term Government portability.

9.5 Entity Resolution and Deconfliction

The contractor shall:

1. Implement approved matching and deconfliction logic for applicable entity domains.

2. Maintain linkage, provenance, confidence, and adjudication records.

3. Support human review of uncertain matches.

4. Use a diverse, multi-tool suite of approved WDP AI/ML capabilities for matching assistance where authorized.

9.6 Gold Table Design and Publication

The contractor shall:

1. Design Gold Tables consistent with WDP Gold-layer purpose, strictly aligning with Joint Fires Network (JFN) and Intelligence Community (IC) "Golden Objects" (Red and Blue Object Management).

2. Apply transformations, calculations, and business logic required for decision-ready outputs.

3. Publish Gold Tables with approved metadata, lineage, access controls, and documentation.

4. Version Gold Tables and maintain release notes, change logs, and deprecation notices.

5. Ensure Gold Tables are consumable through approved WDP-native access methods and delivered in a universally "AI-Ready State."

9.7 Downstream Delivery and Consumption Support

The contractor shall:

1. Support approved access for BI, applications, APIs, analytics, and ML/AI use cases as directed.

2. Configure access controls and support consumer onboarding.

3. Provide consumer-facing documentation and sample usage patterns.

4. Execute AI Vectorization & Embedding Generation: Delivering Gold Tables does not satisfy the requirement for "AI-Ready" data. The contractor shall engineer the automated pipelines necessary to extract data from the GFM Gold Tables, generate appropriate text embeddings, and load this data into optimized Vector Stores (or an equivalent AI-native retrieval layer). This ensures the GFM data is strictly primed for Retrieval-Augmented Generation (RAG) and direct ingestion by authorized AI and machine learning models.

9.8 Data Quality Monitoring and Observability

The contractor shall:

1. Implement automated monitoring for freshness, completeness, validity, consistency, uniqueness, and any other metrics to ensure pipeline health prior to Silver-to-Gold promotion.

2. Provide dashboards, alerts, and recurring quality scorecards.

3. Track and manage Silver and Gold Table corrective actions to closure.

4. Use approved AI capabilities to detect anomalies and drift where beneficial.

9.9 Governance, Metadata, and Compliance

The contractor shall:

1. Maintain metadata for Bronze, Silver, and Gold assets under an Active Data Governance framework.

2. Support cataloging, stewardship, Gold promotion workflows, and auditability.

3. Implement data handling consistent with SIPRNet and WDP policy.

4. Maintain role-based access controls and change traceability.

9.10 AI Enablement and Responsible Use

The contractor shall:

1. Use existing, Government-approved AI capabilities to accelerate lifecycle activities, ensuring final outputs are universally ingestible by any authorized AI platform.

2. Maintain human-in-the-loop review for AI-assisted outputs that affect authoritative mappings, business rules, entity resolution, or Gold release decisions.

3. Document AI usage, validation, and oversight procedures.

4. Ensure AI-assisted processes are auditable and policy-compliant, guided by the contractor's AI Deployment Strategist.

9.11 Training and User Enablement

The contractor shall:

1. Provide training materials, office hours, and user onboarding support.

2. Maintain data dictionaries, glossaries, and sample queries.

3. Provide a mechanism for user feedback and support.

9.12 Sustainment and Operations

The contractor shall:

1. Implement WDP-supported CI/CD or controlled promotion processes using strict DevSecOps pipelines.

2. Maintain SOPs, runbooks, rollback procedures, and continuity documentation.

3. Support incident response and recovery.

4. Reduce manual intervention where possible through repeatable automation.

5. Implement Blue/Green deployment architectures to ensure updates to data, scripts, and models occur without disrupting the Gold Tables feeding production applications, maintaining zero unplanned downtime.

9.13 Contract Transition-Out

The contractor shall:

1. Transition-Out Plan: Develop and deliver a comprehensive Transition-Out Plan for Government approval no later than 90 days prior to the end of the period of performance. This plan will detail the schedule, activities, and resources required for the complete handover of all responsibilities, data, code, and artifacts.

2. Knowledge Transfer & Artifact Delivery: Conduct a phased transition, including the delivery of all technical and project management artifacts with Government Purpose Rights or Unlimited Rights to prevent vendor lock-in. The contractor must ensure these artifacts are organized, documented, and transferred in a manner that enables a successor to assume all responsibilities with minimal disruption. This shall include, at a minimum:

a. Data Architecture Blueprints: The final, as-built Conceptual, Logical, and Physical Data Models in both their native format (e.g., modeling tool files) and a portable format (e.g., PDF).

b. Source-to-Target Mappings (STTMs): The complete and final STTM documentation detailing all data lineage, transformation logic, and business semantics.

c. Code & Logic Repository: The complete, version-controlled code repository containing all ETL/ELT scripts, data pipeline code (e.g., SQL, Python, Databricks notebooks), and AI vectorization logic.

d. Operational Documentation: All runbooks, Standard Operating Procedures (SOPs), and DevSecOps documentation required to operate and sustain the data pipelines and Blue/Green deployment processes.

e. Data & Product Catalogs: The final version of the Data Product Catalog and all associated metadata.

3. Cooperative Knowledge Transfer Sessions: During the transition period (the final 90 days of the contract), conduct a series of joint reviews, "over-the-shoulder" shadowing sessions, and technical workshops with the incoming team to provide hands-on training and answer questions regarding the data architecture, pipeline operations, and sustainment procedures.

10.0 Deliverables

The contractor shall provide, at minimum:

· Program Management Plan (incorporating Agile and DataOps methodologies)

· IMP/IMS

· Status Reports (to include Agile metrics such as Say-Do Ratio and velocity)

· Risk/Issue Log

· Validated Requirements & Data Product Backlog (The tangible output from the continuous Requirements Discovery process, managed by the Data Product Manager)

· Source Onboarding Packages

· Bronze Ingestion Pipelines and Runbooks

· Silver Transformation Specifications

· Validation Rule Catalog

· Quality Exception Reports

· Canonical Mappings and Source-to-Target Mapping (STTM) Documentation (demonstrating Advanced Data Modeling)

· Conceptual and Logical Data Models (delivered in industry-standard, vendor-agnostic formats with Unlimited Rights)

· ETL/ELT Code & Transformation Logic (delivered in raw native formats with Unlimited Rights)

· Entity Resolution Documentation and Linkage Tables, as applicable

· Data Product Catalog / Registry (Evolved from "Gold Table catalog," reflecting a product-centric approach)

· Gold Data Product Releases (confirming strict structural alignment with JFN/IC "Golden Objects")

· Release Notes and Change Logs

· Data Intelligence Dashboards and Scorecards (Automated reports on data quality, lineage, and metadata)

· AI Enablement & Vectorization Plan (Delivered by the AI Deployment Strategist, detailing the architecture for vectorization and RAG)

· AI Vectorization Pipeline & Vector Store Access (The operational pipeline and active access points for AI models to consume GFM data)

· Training Materials and Consumer Guides

· Sustainment and DevSecOps Documentation (to include specific SOPs and runbooks for Blue/Green Deployments)

· Transition-Out Plan

· Final Transition-Out Artifact Package

11.0 Performance Standards

The contractor’s performance will be evaluated based on quantitative, outcome-driven metrics aligned with Agile methodologies (e.g., SAFe, Scrum) and modern DataOps best practices. The contractor is expected to maintain continuous delivery of value, ensuring that GFM data products meet rigorous standards for quality, AI-readiness, and system availability.

The Government will use the following Performance Requirements Summary (PRS) table to conduct surveillance and validate contractor performance.

12.0 Acceptance Criteria

Deliverables shall be acceptable when:

1. Required lifecycle capabilities, including automated Bronze-to-Gold pipelines and Blue/Green deployment architectures, are fully implemented and operational in WDP with zero unplanned mission downtime.

2. Gold Tables are delivered by the validated backlog generated during Requirements Discovery and explicitly align with JFN and IC "Golden Object" standards.

3. Data products meet or exceed the quantitative Acceptable Quality Levels (AQLs) established by the automated Data Intelligence and active governance framework (including automated quality and lineage criteria).

4. AI-assisted processes are documented and reviewable, and all final GFM Gold Tables are delivered in a universally "AI-Ready State" via automated Vectorization pipelines and Vector Stores.

5. Authorized consumers, including both human analysts and AI/Machine Learning models via Retrieval-Augmented Generation (RAG) – can successfully access approved Gold Tables and Vector Stores through designated WDP mechanisms.

6. Required documentation, including Conceptual, Logical, and Physical Data Models, Source-to-Target Mappings, raw ETL/ELT code (delivered with Unlimited or Government Purpose Rights), Agile performance metrics, and training materials are complete and approved by the Government.

7. Final contract acceptance is contingent upon the successful execution of all tasks in the approved Transition-Out Plan and the complete, verified delivery of all required artifacts to the Government

13.0 Staffing

The contractor shall provide qualified personnel, as required, to staff a highly functioning AI-ready data product factory. Key personnel, including the Senior Data Engineer and the AI Deployment Strategist, are required to have demonstrable experience with Databricks, the Medallion Architecture, and modern vector storage pipelines. All contractor personnel must possess and maintain a security clearance level commensurate with the data environment (SIPRNet), as specified for each position.

14.0 Security

The contractor shall:

· Operate within SIPRNet and WDP access and handling requirements.

· Properly mark and protect all work products.

· Follow least-privilege and access auditing requirements.

· Coordinate with Government security officials for provisioning and incident reporting.

The contractor shall operate within the security controls established for the CDAO WDP tenant and its Authority to Operate (ATO). Specific controls, data handling procedures, and incident reporting timelines will be detailed in the Security Classification Guide (SCG) and other GFI provided post-award. The contractor is responsible for implementing role-based access control (RBAC) and least-privilege principles within the scope of their development activities.

Part II. CDRL-Style Deliverables Table

Part III. Requirements Traceability Matrix (RTM)