Attachment_2_-_Fusion_Architecture.pdf
PDF 3 MB Posted
- Attached to
- Science and Technology for Autonomous Teammates (STAT) Federal contract opportunity
- Solicitation number
- FA8650-17-S-6001
About this file
This document summarizes a solicitation for the Science and Technology for Autonomous Teammates (STAT) program. The solicitation seeks proposals to develop and demonstrate autonomy technologies to enable Air Force mission sets in multi-domain command and control, intelligence processing and dissemination, and manned-unmanned combat teaming. Technologies are needed for mission planning, execution and post-mission analysis, including in multi-domain command and control, manned-unmanned teaming and information analytics. Proposals should detail modular, transferable and open system architectures delivering autonomy technologies applicable across multiple domains. Technologies are required to enable airmen to plan, command, control and execute missions with manageable workloads by ingesting and understanding mission tasks and commander's intent, responding appropriately to human direction and orders, and responding intelligently to dynamic threats and unplanned events. Proposed technologies must be open, reusable, adaptable, platform agnostic, secure, credible, affordable and enduring, and integrate into autonomous systems to support modeling and simulation environments owned by the Air Force Research Laboratory for future capability development.
Attachment 2 - Fusion Architecture
View the file
Other files for this federal contract opportunity
Show all 50
Science and Technology for Autonomous Teammates (STAT) has more files on GovTribe.
On GovTribe
Work with this file on GovTribe
- Download the original file
- Contacts named in this file
- Similar government files
- Ask GovTribe AI about this file
Text version
INFRASTRUCTURE TO SUPPORT AUTONOMY
RESEARCH
FRAMEWORK
DEVELOPMENT
PROCESSES
RESEARCH
DEMO
ALLEN ROWE, SARAH SPRIGGS
SUPERVISORY CONTROL & COGNITION
HUMAN EFFECTIVENESS DIRECTORATE
AIR FORCE RESEARCH LABORATORY
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
PREMISE
• Research is required to identify the issues related to developing a software-based framework to foster the development of intuitive & effective human interaction with advanced adaptive automation tools.
• Having a robust autonomy-based framework enables evaluation of the mechanisms that support cooperation and coordination among widely disparate platforms such as remotely piloted unmanned systems, autonomous unmanned systems, and adversarial components for ground, air, cyber, space, maritime, and submarine entities.
• Tying these interactions into an immersive user interface will help better evaluate user behaviors and confidence in a robust simulation architecture.
The current research is being directed at exploring interfaces that better support human-autonomy coordination and collaboration by introducing immersive dialogs, cooperative planners, intelligent agents, goal-reasoners, and machine learning into a comprehensive autonomy framework.
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
GOVERNMENT OPEN SOURCE
• How
• Why
– Provides a common tool for rapid prototyping and conducting experiments
– Fusion core software can leverage outside project’s work
Project A Project B CORE
CORECORE
Provide source code Manage repositories Track change requests Create software builds
Overview
DISTRIBUTED VIRTUAL LABORATORY
AFRL
FI
LE
SH
AR
IN
G
SO
FT
W
AR
E
RE
PO
SI
TO
RI
ES
CO
N
FI
GU
RA
TI
O N
M
AN
AG
EM
EN
T
SO
FT
W
AR
E
RE
Q
UI
RE
M
EN
TS
M
AN
AG
EM
EN
T
SOFTWARE DEVELOPMENT CONFIGURATION MANAGEMENT
SCRUM AGILE SOFTWARE DEVELOPMENT PROCESS
AN ITERATIVE METHODOLOGY FOR SOFTWARE PROJECTS & PRODUCT DEVELOPMENT
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
Framework
FUSION SYSTEM OVERVIEW
AUTONOMY
COOP
PLANNERS
UX
V
CY
BE
R
SP
AC
E
SYSTEMS
SC
AL
AB
LE
IN
TE
RF
AC
ES
M
UL
TI
M O
D
AL
M O
DU
LA
R
TA
SK
/ G O
AL
O
RI
EN
TE
D
HUMANS
INSTRUMENTATION / TRANSPARENCY
INTELLIGENT
AGENTS
MACHINE
LEARNING
FUSION FRAMEWORK
Fusion* is a Framework that enables Natural Human Interaction with Flexible & Adaptive Automation via Intelligent Agents, Machine Learning, Cooperative Planners & Advanced Simulation thru an Instrumented, Play Oriented UI that empowers Scientific Experimentation & Technology Advancement
Across Multiple Systems *Fusion leverages research accomplished across AFRL building upon aspects of the Vigilant Spirit Software Baseline, a multitude of Simulations, & various 3rd party Open Source Repositories.
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
4-Layer Architecture
FUSION SYSTEM OVERVIEW
FRAMEWORK
CM
AS
I
CM
F
N O
TI
FI
CA
TI
O N
BA
R
VI
DE
O
DA
SH
BO
AR
D
TEST BED, *
TS
D
FE
ED
BA
CK
BA
R
AGENTS, RETROSPECTION, INSTRUMENTATION, *
APP
MODULE /
MESSAGING
FRAMEWORK
DI
S
CO
T
JA
US
DISTRIBUTION A. Approved for public release; distribution unlimited. 88ABW Cleared 08/29/2014; 88ABW-2014-4124
FUSION SYSTEM OVERVIEW
API AND CORE CLASSES OF FUSION
CM
AS
I
CM
F
N O
TI
FI
CA
TI
O N
BA
R
VI
DE
O
DA
SH
BO
AR
D
TEST BED, *
TS
D
FE
ED
BA
CK
BA
R
API AND CORE CLASSES OF A SPECIFIC MODULE & EXTENSIBILITY
APP
MODULE /
MESSAGING
FRAMEWORK
DI
S
CO
T
JA
DISTRIBUTION A. Approved for public release; distribution unlimited. 88ABW Cleared 08/29/2014; 88ABW-2014-4124
FUSION SYSTEM OVERVIEW
FRAMEWORK
TEST BED, *
AGENTS, RETROSPECTION, INSTRUMENTATION, *
APP
MODULE /
MESSAGING
FRAMEWORK
A MODULE FOR TRANSLATING
AND UTILIZING INCOMING AND
OUTGOING MESSAGES OF A
SPECIFIC PROTOCOL
A UNIT OF CODE THAT IS
WRITTEN FOR A SINGLE AND
SPECIFIC PURPOSE
EXAMPLES: TILES AND SERVICES
DISTRIBUTION A. Approved for public release; distribution unlimited. 88ABW Cleared 08/29/2014; 88ABW-2014-4124
FUSION SYSTEM OVERVIEW
FRAMEWORK
CM
AS
I
CM
F
N O
TI
FI
CA
TI
O N
BA
R
VI
DE
O
DA
SH
BO
AR
D
A UNIT OF CODE RESULTING IN AN *.EXE (TEST BED, UTILITY APP)
TS
D
FE
ED
BA
CK
BA
R
AGENTS, RETROSPECTION, INSTRUMENTATION, *
APP
MODULE /
MESSAGING
FRAMEWORK
DI
S
CO
T
JA
DISTRIBUTION A. Approved for public release; distribution unlimited. 88ABW Cleared 08/29/2014; 88ABW-2014-4124
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
Basic Functionality
FUSION
LOGIN
• SECURE SQL
DATABASE
• USER PROFILES
• MISSION SETUP
• LAYOUT SETUP
LAYOUT
• SCALABLE
• MULTI-DISPLAY
• CONFIGURABLE
• TILE MGMT.
NOTIFICATION
• SYSTEM MESSAGES
& ALERTS
• TIME &
STOPWATCH
• MOMU ASSET
MGMT.
TILES
• CUSTOM UI
ELEMENTS
• EDITORS
• PAYLOAD ASSET
• PLAY MGMT.
CANVAS
• WORKSPACE
MGMT.
• CONFIGURABLE
• MAP
OSGEARTH
GIS
WMS
FEEDBACK
• MULTI-MODAL
INTERACTIONS
• SPEECH/
GRAMMAR &
AUTOMATION
FEEDBACK
• ANNOTATION
& SPEECH
MGMT.
• DYNAMIC HELP
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
Example Login Layout
FUSION
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
Example Login Layout
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
Example Layout
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
Example Layout
How can a software system create a framework where every public element, regardless of its role as a model, user-interface element, etc., is customizable, Extendable and override-able by any other software developer in the system?
How can a software system generalize disparate and similar messaging protocols to be protocol-agnostic while allowing a many-to-many relationship between Networked Systems for the generation, distribution and consumption of messages?
How can a software system be Instrumented to gather real-time user/machine interactions & system details for use in experimentation, software agents, & machine learning?
How can a software system record the state of each of its components at a rate near 30hz and make it user-accessible to enable discrete & continuous Retrospection of system in real-time?
Research Focus
FUSION SYSTEM OVERVIEW
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
CLOUD BASED ARCHITECTURE
• One hub acts as the center for all communication
– Uses standard messaging protocols (TCP/IP/UDP)
– Publish/subscribe for messages
– Scalable
– Localized and distributed
• Real-time (real or simulated)
• Retrospection (play back)
• Look-ahead (FTRT, real-time simulation)
• Fusion is fully independent of external systems, and enables the easy exchange of tools (ex. replacing AMASE with AFSIM).
• Multiple Fusion instances respond to same messages.
• Fusion has a “sandbox mode” to enable play back or look ahead at scenario.
FusionFusion
Hub
Fusion
Agents
Simulations
GIS Services
Data Source
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
IMPACT Instance
CLOUD BASED ARCHITECTURE
∅MQ hub communications
Hub
Domain Data
DIS
LMCP
Video Stream
LMCP
Fusion
Machine LearningAMASE/ Simulation
Truth Data
SubrScene
Dialog
CECEP CCA
LMCP
Request GDAL
GIS
Weather
Waves
SQL Server/logger
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
EXTENSIBILITY
• Built from ground-up as an open framework
– Can modify any existing functionality
– Can change how all information is displayed
• Allows for future growth of code base
• Avoids duplication of code
• Adheres to the MVVM design
• Researchers:
• Change how users interact with the system
• Modify visual controls for desired spectrum of control
• Developers:
• Easily exchange one visual control for another
• Quickly add new functionality
FRAMEWORK
CM
AS
I
CM
F
N O
TI
FI
CA
TI
O N
BA
R
VI
DE
O
D
AS
H
BO
AR
D
TEST BED, *
PL
AY
BO
O
K
FE
ED
BA
CK
BA
R
PR
O
JE
CT
A’ S
PL
AY
BO
O
K
AGENTS, RETROSPECTION, INSTRUMENTATION, *
DI
S
CO
T
JA
PLAYBOOK
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
Example
EXTENSIBILITY
Default IMPACT ATACM
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
INSTRUMENTATION
• Collects, distributes and logs:
– Real-time user interaction, including context
– System information
– Simulation data
– Hub traffic
• Supports bi-directional information exchange
– Autonomy feedback
– Modification to user visuals
• Researchers are able to track every action a participant makes.
• Automation aids can know what the user is doing.
• External entities can display & highlight portions of the interface to aid the user.
• Enables real-time and ad-hoc retrospection.
Fusion
Interaction
LogInterpreter
Agent json
∅MQ
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
Example
INSTRUMENTATION
Wpf:AttachedProperties.Key="CenterOn"
CENTER ON BUTTON
KEY SOURCE
SOURCE
ID
SOURCE
OBJECT TYPE TIME
(Testbed Frame Overlay) .(Testbed Frame Overlay
Hud) .(Canvas)
.(SummaryDashboard) .(CenterOn)
Fusion.Vehicles.Air .Impl.AirVehicle Snake-01 Button Click 00:20.2
Standard Vehicle Dashboard Tile
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
RETROSPECTION
• Linked heavily to instrumentation
• Playback
– State of each component
– “World state”
– Hub message traffic
• Enables a play back of the entire system for a recorded scenario
• Researchers:
• Replay through a participant’s trial
• Detailed post hoc analysis of operator performance and task behaviors
• Robustly debrief a participant
• Operators:
• Jump to a specific point in time
• Evaluation of alternatives
• Developers:
• Efficiently test and debug software
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060. 23
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
BACKUPS
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
Excerpts from AFRL, AF, DoD-level Publications
FUSION CONTRIBUTIONS
• AFRL Autonomy Science and Technology Strategy
– Delivering flexible autonomy systems with highly effective human-machine teaming
– Creating actively coordinated teams of multiple machines to achieve mission goals
• 2010 Technology Horizons (Air Force Chief Scientist)
– Highly autonomous decision-making systems
– Fractionated, composable, survivable, autonomous systems
• DoD Defense Science Board
– Planning
– Learning
– Human-robot interaction
– Natural language understanding
– Multi-agent coordination
DISTRIBUTION STATEMENT A: Approved for public release; distribution unlimited. 88ABW Cleared 04/22/2015; 88ABW-2015-2060.
TEAM MEMBERS
• Government
– Allen Rowe – Project Lead
– Sarah Spriggs* – Software Lead
• IST / WSRI
– Sarah Lampke* – PI
– Jerry VanPelt – ATACM integration
– Daylond Hooper, PhD – IMPACT Integration
– Austin Bangert
– Adam Buchanan
– Mike Howard*
– Kenneth Wickline*
– Bruce Clay – Lead for GIS data
– Jeff Page* – Lead database developer
– Jon Steingress
* core Fusion developers
| Slide Number 1 |
| Premise |
| Government Open Source |
| Distributed Virtual Laboratory |
| Fusion System Overview |
| Fusion System Overview |
| Fusion System Overview |
| Fusion System Overview |
| Fusion System Overview |
| Fusion |
| Fusion |
| Fusion |
| Fusion |
| Fusion |
| Slide Number 15 |
| Cloud Based Architecture |
| Cloud Based Architecture |
| Extensibility |
| Extensibility � |
| Instrumentation |
| Instrumentation � |
| Retrospection |
| Slide Number 23 |
| Backups |
| Fusion Contributions |
| Team members |
File details come from the government source that posted it. Updated .