Amendment 0002 - D2D_IndustryDay Brief.pdf

OSD Data to DecisionsOSD Data-to-Decisions Strategic Initiative

28 February 2011 Randy K Avent

RKA20110228-1

Data-to-Decision Systems

Tactical Operations Operations Intelligence Strategic Intelligence

• Low Latency

• Narrow Field-of-View

• Limited Fusion

• Medium Latency

• Wide Field-of-View

• Hard Sensor Fusion

• Long Latency

• Synoptic Field-of-View

• Hard/Soft Sensor Fusion• Limited Fusion

• Automatic Target Recognition

• Data: ~MB-GB

• Hard Sensor Fusion

• Assisted Target Recognition

• Data: ~GB-TB

• Hard/Soft Sensor Fusion

• Multiple Hypotheses

• Data: ~PB-EB

RKA20110228 - 2

Current System Development Cycle

YR 1 YR 2 YR 3 YR 4 YR 5 YR 6 YR 7 YR 8 YR 9 YR 10

Requirements Compete Develop and Build Deliver

Operational Input

Contract Award

DT, OT

Tests

TECHEVAL

OPEVAL

• There is a continuous, compelling need for new Decision Support Systems

• Current development cycle is protracted hard-wired and does not keep• Current development cycle is protracted, hard-wired and does not keep pace with changing countermeasures and threats

• Development approach does not integrate advances across the

RKA20110228 - 3

defense enterprise and excludes innovative small companies

Historical Precedence for Rapid Development using Modularization

Modular Integration Horizontal Integration “Lego Model”

Vertical Integration “Brick and Mortar”

1950-1960 1960-1970 2000+

Lego ModelBrick and Mortar

• Hard-wired, not adaptable

• Limited to large contractors

• Modular, closed standards

• Includes subcontractors

• Modular, open standards

• Includes all innovators

• Protracted development • Simplified development • Rapid development

Enable Horizontal Integration for new Decision Support Systems by developing extensible module libraries that can be

RKA20110228 - 4

Systems by developing extensible module libraries that can be rapidly configured to solve new Decision Support problems

Outline

• Introduction

• Broad D2D Goals

• BAA Details

• Summaryy

RKA20110228 - 5

Architectural Layers

Human Effectiveness

User Interactions

Operations Management

Image Motion Text

Data Fusion

Analytics N et w or k

Software Infrastructure g Analytics Analytics Analytics y

(CBRNE) N

Sensor (Spatial, Spectral)

Sensor (Temporal)

Sensor (Factual)

Hardware Infrastructure (Compute and Storage)

Sensor

(CBRNE)

RKA20110228 - 6

Data-to-Decisions Adjunct Technology

Hardware Infrastructure

• Grid Cluster • Cloud Computing• Embedded System p gy

• Centralized storage

• Data moved to compute nodes

• Tightly coupled algorithms

• Distributed storage

• Applications moved to compute nodes

• Order-independence

• On-board storage

• Tightly coupled data and algorithms

• Low-latency low-bandwidth • Tightly coupled algorithms

• Parallel file system limits large data use

• Order-independence through map/reduce

• Low-latency, low-bandwidth operations

RKA20110228 - 7

Software Infrastructure

WAMI SIGINT

Sensor Management (User Interface)

Sensor

Sidecar

Template Module

Graph Viewer

Collab Services

Usage Statistics

End-User Program

Workflow Logging

Sensor

Sidecar

Service Bus

Resource Brokering

Cyber SA & Trust

Registry

Msging

Backgrnd Modeling

Probability Assignment

Tracking

Track-Site Association

Data Management

Data/Tag Provenance

Tag Management Indexing

SOA

Mgmt

Start/Stop Detection

Graph Engine

Ontology Module

Interface Module

Uniform Data Model

RKA20110228 - 8

Core Services ata ode

Extended Core Services (Analytics)

Analytic Infrastructure

Challenge Example Sensors Example Analytics

• Spatial analytics

• Reconstruction

• Segmentation

• Classification

• SIGINT

• Range profiles (HRR)

• Imagery (EO,IR, SAR)

• Lidar

• Spectral analytics • Atmospheric distortions

• Spectral mixing

• Hyperspectral

• Multi-spectral

• Temporal analytics

• Robust tracking

• Activity recognition

• Patterns of life

• Motion dynamics

• GMTI

• Wide Area Motion Imagery

• Acoustic UGS

• SIGINT

• Factual analytics y

• OCR

• Speech

• Machine translation

• Language understanding

• Text

• Websites

• News

RKA20110228 - 9

• Language understanding

Data Fusion

{ 1 } {f1 }Feature Extraction Decision

Feature Extraction Decision

{p1 i,j}

{p2 i,j}

{f1i}

{f2i}

D1

D2 Extraction

Feature D i i {p3 i,j} {f3i} D3 Extraction Decision D3

Signature-Level Feature-Level Decision-LevelSignature-Level Fusion

Feature-Level Fusion Fusion

RKA20110228 - 10

User Interface Layer

• Analysts (Single Source) • Aggregators (All-Source)

• Data

• Metadata+

• Data

• Metadata

• Reports Analytics

User Interface

• Tags

• Provenance

• Ontologies

• Reports

• Usage Statistics

• Ontologies

• Workflows

• Provenance

User

Interface†

• Workflows

• Templates

• Ontologies

• Workflows

• Tags

• Templates

. Queries

• Templates

• Data

• Metadata+

• Tags

• Provenance

• Data

• Metadata

Analytics

User Interface

RKA20110228 - 11

†End User Programmable

Provenance

• Ontologies• Workflows

• Templates

D2D Technology Assessment

• Moderately Mature

• Driven by IT Industry

• Immature

• Driven by Defense

• Moderately Mature

• Driven by IT Industry

Data Management Layer Analytics Layer User Interface Layer

• Current assessment is that unstructured data analytics is the most challenging and critical component of D2D

St t fi t f l ti ith i i

RKA20110228 - 12

• Strategy first focuses resources on analytics with increasing focus on data management and user interfaces later

Outline

• Introduction

• Broad D2D Goals

• BAA Details

• Summaryy

RKA20110228 - 13

Implementation Plan

• Pick series of high-priority challenge problems that on s require multi-sensor data

• Create a consortium focused on each challenge m en da tio problem, where success is defined at the consortium level, not at the individual contractor level

Emphasi e data collections ith nclassified so rcesN re co m m

• Emphasize data collections with unclassified sources that can be widely distributed, but test on operational dataJA

SO

N

• Build an open architecture, government-owned testbed, running a Service-Oriented Architecture on distributed nodes

RKA20110228 - 14

nodes

Motion Analytics

Challenges

Preliminary Tracking Requirement

• Robust Trackers

• Statistical Trackers

(s

Grazing angle

Look angle

6° 0° 50° 50° 70° 60°

• Activity-Based Intelligence

• Indications & Warnings at e

In te rv al

( 70 60

• Movement Dynamics

• Map Making

U pd a

With Features

• Production Estimation

-1

Background Road Density (vehicles/km)

Without Features

RKA20110228 - 15

Program Organization

Builds Threats Integrates Solutions

Challenge Problem Build

• Scenario generation

• Data collection

• Data Coordination

Center

• D2D Testbed

• Scalability experiments

• Library maintenance

Spiral Development

Develop Analyze• Algorithms

• Code

• Data Analysis Center

• Architectures/Studies

• Performance testing

• Benchmarks

• Program Leader• Program Leader

Development consortium should be an integrated team

Develops Solutions Analyzes Solutions

RKA20110228 - 16

Development consortium should be an integrated team conducting translational research

Program Organization

Challenge Problem Build

Government Team:

Consortium:

Government Team:

AFRL

ARL

NRL

MIT LL

Consortium:

Industry Academia Labs FFRDCs/UARCs

Develop Analyze

• Primarily 6.3 funding

• Mission critical, infrastructure

• Primarily 6.2 funding

• Opportunistic, innovative

RKA20110228 - 17

• Criterion: speed, access, analyses• Criterion: Innovation, openness

Summary

• The Data-to-Decisions program develops technology for the rapid development of flexible new Decision Supportrapid development of flexible new Decision Support Systems

P i t f i f l t h ll• Program consists of a series of relevant challenge problems that advance the underlying technology in data management, analytics and user interfaces

• Execution is through a consortium that addresses the challenge problems in a coherent and integrated team happroach

• Major research initiatives focus on developing extendable

RKA20110228 - 18

analytic approaches and advanced user-interface modules