07-OFFSET_Prop_Day_ARL-2.pdf

DISTRIBUTION STATEMENT A: APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED.

Distributed Collaborative Intelligence:

A Tactical Offset Strategy

Brian M. Sadler

Brett Piekarski

Stuart Young

DARPA OFFSET

30 January 2017

Brian.m.sadler6.civ@mail.mil

Mega Cities and Urban Mixed Populations

Triple Canopy Jungle Subterranean

• 3rd Offset : Undersea and extended range air

• Tactical Offset : Unmanned systems to extend the reach and situational awareness for the Soldier across complex worlds

Tactical Offset

Extreme Terrain

Tactical Intelligent Systems for 2025 and Beyond

Underlying Assumptions

• Large #’s of agents – 10s to swarms

• Heterogeneous mix – including Soldiers

• Highly collaborative systems

• Highly distributed systems

• Access to knowledge sources and increased perception and awareness

• Increased cognitive behaviors and real-time op-tempo adaptable operations

• Operation in complex and contested environments

– peer capabilities

Payoff

• Extended reach, situational awareness, and operational effectiveness against dynamic threats in contested environments

• Technical and operational superiority through intelligent, resilient and collaborative behaviors

Vision

• Highly distributed and collaborative heterogeneous teams of intelligent systems

Axes of Complexity

These factors limit the operational capability for a given autonomy technology suite.

1. Complexity of the environment(s)

2. Available infrastructure

3. Operational tempo

4. Number of agents

5. Degree of heterogeneity of the agents

6. Agent behavior complexity and adaptability

7. Degree of interaction and communication among the agents

(both machine and human agents)

The Army challenge: complex unknown environments, little or no infrastructure, and high operational tempo.

ARL Intelligent Systems Center

Decision-Making beyond

Human Op-Tempo

Adaptable to

New Environments

Comprehensive World Model

Common Sense Reasoning

Dynamic Learning

Self-organizing Robot Teams

Distributed Computing &

Autonomous Networking

Robust Physical Agents

Real-time Planning

Socially Cognizant Behaviors &

Adversarial Reasoning

Desired Attributes of Army Intelligent Systems

Mobility & Manipulation | Perception | Adaptive Control

• Cognitive Architectures

• Artificial Intelligence

• Reasoning/Knowledge Engineering

• Natural Language

• Semantics

• Big Data Analytics

• Machine Learning

• Distributed Computing (HPC)

• Game Theory

• Process & Task Allocation

• Information Bandwidth Management

• Cooperation

Provide fundamental science underpinnings of autonomous systems for the Army

From micro-systems to combat vehicles

Soldiers/Unmanned System

Teaming:

• Combat multiplier

• Team member

• Heterogeneous groups

• Following commander’s intent

ARL Intelligent Systems Center

Robotics

CTA

MAST

CTA

Internal ARL

Research

ARO

Micro Autonomous Systems and

Technology (MAST)

Collaborative Technology Alliance (CTA) – 2008-2017

Enhance tactical situational awareness in urban and complex terrain by enabling the autonomous operation of a collaborative ensemble of multifunctional, mobile microsystems – Rapid and Mobile ISR for the Dismounted Soldier

Caves, Strategic

Bunkers, Subterranean

Jungles and Under

Canopies Megacities and

Urban

MAST CTA

Communications, Navigation

& Coordination

Mobility, Control, &

Energetics Sensing, Perception, & Processing

Three Cross-Cutting Research Thrusts and a Joint Experimentation Thrust

• Indoor/Outdoor GPS-denied Navigation

• Dealing with Uncertainty

• Heterogeneous Collaborative Systems

• Aggressive Maneuvers & Collaborative

Behaviors

• Robust Multi-Spectral Comms Solutions

• Novel systems – flapping wing/cyclocopter

• Perching & Grasping

• Gust Mitigation

• Novel Power Source

• Understanding Ambulation over Complex

Terrain

• Hair-like Arrayed Sensors for Gust and

Acceleration Sensing

• Robust State Estimation

• Vision and SWaPP Constrained State-

Estimation

• 5g 220 GHz Radar

• Stereo/Optic Flow Sensor Fusion

Joint Experimentation

• Integrate and extend lower-level capabilities from core research tasks

• Evaluate results on platforms performing elements of mission-like scenarios

• Involve collaborations among multiple institutions, centers, and ARL

MAST CTA & the ARL Enterprise

• Bio-Inspired Sensors and Controls Team

• (SEDD-Materials) - Sensor Integration, State- Estimation, Human In-The-Loop Controls, PNT

• Microsystems Mechanics Team

• (VTD-Sci Mvr) - Aeromechanics, Soft Materials, Fluid Dynamics, Controls, Flight Dynamics

• Manipulation and Mobility Team

• (VTD-Sci Mvr) - Self-righting, grasping

• Asset Control and Behavior Branch

• (CISD-Info Sci) - collaborative behaviors, network aware communications, GPS-Denied nav

• Tactical Network Assurance Branch

• (CISD-Info Sci) - Networked behaviors, VHF and Optical Comms

• Smart Swarm Munitions Team

• (WMRD-Sci for Lethality & Prot) – Vehicles, propulsion, networking, perception

• New infrastructure, labs, and test facilities

The RCTA Vision

Making unmanned systems an integral part of the small unit team

Systems that:

• Understand the environment

• Learn from experience

• Adapt to dynamic situations

• Possess a common world view

• Communicate naturally

• Conduct useful activity

• Can act independently, but within well prescribed bounds

Through research to enable and advance:

• Abstract reasoning

• Learning by example

• Reinforcement learning

• Semantic perception

• Communication through language

• Human behavior modeling

• Agile 3-D mobility at operational tempo

• Human-like manipulation

From tool to teammate http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCKHDhsnj7MYCFUwWPgodYgMLQA&url=http://www.cs.cmu.edu/~pbarnum/&ei=RoWuVaGIKcys-AHihqyABA&bvm=bv.98197061,d.cWw&psig=AFQjCNH9rRswa5rCyNaeXwYb4mOhsH7-Jg&ust=1437587135712599 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCKHDhsnj7MYCFUwWPgodYgMLQA&url=http://www.cs.cmu.edu/~pbarnum/&ei=RoWuVaGIKcys-AHihqyABA&bvm=bv.98197061,d.cWw&psig=AFQjCNH9rRswa5rCyNaeXwYb4mOhsH7-Jg&ust=1437587135712599 https://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCPKz0fHk7MYCFUo9PgodnHcIUA&url=https://catalog.data.gov/dataset/wms-global-mosaic&ei=qIauVfK_Dcr6-AGc76GABQ&psig=AFQjCNEydDipQRI9dmgmSdslK6PE66wkww&ust=1437587471940228 https://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCPKz0fHk7MYCFUo9PgodnHcIUA&url=https://catalog.data.gov/dataset/wms-global-mosaic&ei=qIauVfK_Dcr6-AGc76GABQ&psig=AFQjCNEydDipQRI9dmgmSdslK6PE66wkww&ust=1437587471940228 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCOT316nl7MYCFcoyPgodBi0GQg&url=http://www.ssrn.com/link/florida-state-public-law.html&ei=HYeuVaT1Lsrl-AGG2piQBA&bvm=bv.98197061,d.cWw&psig=AFQjCNFUWZcfRdvv7HJOKzibrIG4x4lbgA&ust=1437587612635885 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCOT316nl7MYCFcoyPgodBi0GQg&url=http://www.ssrn.com/link/florida-state-public-law.html&ei=HYeuVaT1Lsrl-AGG2piQBA&bvm=bv.98197061,d.cWw&psig=AFQjCNFUWZcfRdvv7HJOKzibrIG4x4lbgA&ust=1437587612635885 https://upload.wikimedia.org/wikipedia/en/0/0e/UCF_horizontal_logo.svg https://upload.wikimedia.org/wikipedia/en/0/0e/UCF_horizontal_logo.svg http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCMOWuurm7MYCFcxWPgodBYgNkw&url=http://people.csail.mit.edu/haitham/&ei=soiuVcP1Acyt-QGFkLaYCQ&bvm=bv.98197061,d.cWw&psig=AFQjCNF-D5zteqeZSAn8eaavAp3gR7c7VQ&ust=1437587971070366 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCMOWuurm7MYCFcxWPgodBYgNkw&url=http://people.csail.mit.edu/haitham/&ei=soiuVcP1Acyt-QGFkLaYCQ&bvm=bv.98197061,d.cWw&psig=AFQjCNF-D5zteqeZSAn8eaavAp3gR7c7VQ&ust=1437587971070366 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCIrRha7n7MYCFQFbPgodLGsH5Q&url=http://auvac.org/people-organizations/view/467&ei=P4muVcq1L4G2-QGs1p2oDg&bvm=bv.98197061,d.cWw&psig=AFQjCNGCFE5zdz63NnSCLjiOOcO1duWUcA&ust=1437588157773607 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCIrRha7n7MYCFQFbPgodLGsH5Q&url=http://auvac.org/people-organizations/view/467&ei=P4muVcq1L4G2-QGs1p2oDg&bvm=bv.98197061,d.cWw&psig=AFQjCNGCFE5zdz63NnSCLjiOOcO1duWUcA&ust=1437588157773607

Merging Cognition & Control

From tool to teammate

The unique hybrid architecture combining cognitive upper and geometric lower layers – bridging the gap between artificial intelligence & control

• Introduction of abstractions to facilitate human-robot communication – behavior specification based on structured language

• Algorithms to permit semantic labeling of objects, behaviors and their relationships

• A focus upon learning, to include learning without reliance on large sets of training data

• A broad description of the environment that goes beyond placing objects in four-dimensional space to include concepts such as object compliance and signature

• Natural modes of communication – voice and gesture

• Exploration of unconventional mobility modes

• Human-scale manipulation

Coupled with enhancements to facilitate teaming http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCKHDhsnj7MYCFUwWPgodYgMLQA&url=http://www.cs.cmu.edu/~pbarnum/&ei=RoWuVaGIKcys-AHihqyABA&bvm=bv.98197061,d.cWw&psig=AFQjCNH9rRswa5rCyNaeXwYb4mOhsH7-Jg&ust=1437587135712599 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCKHDhsnj7MYCFUwWPgodYgMLQA&url=http://www.cs.cmu.edu/~pbarnum/&ei=RoWuVaGIKcys-AHihqyABA&bvm=bv.98197061,d.cWw&psig=AFQjCNH9rRswa5rCyNaeXwYb4mOhsH7-Jg&ust=1437587135712599 https://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCPKz0fHk7MYCFUo9PgodnHcIUA&url=https://catalog.data.gov/dataset/wms-global-mosaic&ei=qIauVfK_Dcr6-AGc76GABQ&psig=AFQjCNEydDipQRI9dmgmSdslK6PE66wkww&ust=1437587471940228 https://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCPKz0fHk7MYCFUo9PgodnHcIUA&url=https://catalog.data.gov/dataset/wms-global-mosaic&ei=qIauVfK_Dcr6-AGc76GABQ&psig=AFQjCNEydDipQRI9dmgmSdslK6PE66wkww&ust=1437587471940228 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCOT316nl7MYCFcoyPgodBi0GQg&url=http://www.ssrn.com/link/florida-state-public-law.html&ei=HYeuVaT1Lsrl-AGG2piQBA&bvm=bv.98197061,d.cWw&psig=AFQjCNFUWZcfRdvv7HJOKzibrIG4x4lbgA&ust=1437587612635885 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCOT316nl7MYCFcoyPgodBi0GQg&url=http://www.ssrn.com/link/florida-state-public-law.html&ei=HYeuVaT1Lsrl-AGG2piQBA&bvm=bv.98197061,d.cWw&psig=AFQjCNFUWZcfRdvv7HJOKzibrIG4x4lbgA&ust=1437587612635885 https://upload.wikimedia.org/wikipedia/en/0/0e/UCF_horizontal_logo.svg https://upload.wikimedia.org/wikipedia/en/0/0e/UCF_horizontal_logo.svg http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCMOWuurm7MYCFcxWPgodBYgNkw&url=http://people.csail.mit.edu/haitham/&ei=soiuVcP1Acyt-QGFkLaYCQ&bvm=bv.98197061,d.cWw&psig=AFQjCNF-D5zteqeZSAn8eaavAp3gR7c7VQ&ust=1437587971070366 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCMOWuurm7MYCFcxWPgodBYgNkw&url=http://people.csail.mit.edu/haitham/&ei=soiuVcP1Acyt-QGFkLaYCQ&bvm=bv.98197061,d.cWw&psig=AFQjCNF-D5zteqeZSAn8eaavAp3gR7c7VQ&ust=1437587971070366 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCIrRha7n7MYCFQFbPgodLGsH5Q&url=http://auvac.org/people-organizations/view/467&ei=P4muVcq1L4G2-QGs1p2oDg&bvm=bv.98197061,d.cWw&psig=AFQjCNGCFE5zdz63NnSCLjiOOcO1duWUcA&ust=1437588157773607 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCIrRha7n7MYCFQFbPgodLGsH5Q&url=http://auvac.org/people-organizations/view/467&ei=P4muVcq1L4G2-QGs1p2oDg&bvm=bv.98197061,d.cWw&psig=AFQjCNGCFE5zdz63NnSCLjiOOcO1duWUcA&ust=1437588157773607

Thrust 1 – Op-tempo Maneuvers in Unstructured Environments

Robustness

Op-tempo Collaboration

Complex Environments

Thrust 2 - Human-Robot Execution of Complex Missions

Situational Awareness in Unstructured Environments

Distributed Mission Execution

Trusted Execution of Verified Missions

Thrust 3 – Mobile Manipulation

Complex 3D Environments

RCTA: Key Challenges http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCKHDhsnj7MYCFUwWPgodYgMLQA&url=http://www.cs.cmu.edu/~pbarnum/&ei=RoWuVaGIKcys-AHihqyABA&bvm=bv.98197061,d.cWw&psig=AFQjCNH9rRswa5rCyNaeXwYb4mOhsH7-Jg&ust=1437587135712599 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCKHDhsnj7MYCFUwWPgodYgMLQA&url=http://www.cs.cmu.edu/~pbarnum/&ei=RoWuVaGIKcys-AHihqyABA&bvm=bv.98197061,d.cWw&psig=AFQjCNH9rRswa5rCyNaeXwYb4mOhsH7-Jg&ust=1437587135712599 https://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCPKz0fHk7MYCFUo9PgodnHcIUA&url=https://catalog.data.gov/dataset/wms-global-mosaic&ei=qIauVfK_Dcr6-AGc76GABQ&psig=AFQjCNEydDipQRI9dmgmSdslK6PE66wkww&ust=1437587471940228 https://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCPKz0fHk7MYCFUo9PgodnHcIUA&url=https://catalog.data.gov/dataset/wms-global-mosaic&ei=qIauVfK_Dcr6-AGc76GABQ&psig=AFQjCNEydDipQRI9dmgmSdslK6PE66wkww&ust=1437587471940228 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCOT316nl7MYCFcoyPgodBi0GQg&url=http://www.ssrn.com/link/florida-state-public-law.html&ei=HYeuVaT1Lsrl-AGG2piQBA&bvm=bv.98197061,d.cWw&psig=AFQjCNFUWZcfRdvv7HJOKzibrIG4x4lbgA&ust=1437587612635885 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCOT316nl7MYCFcoyPgodBi0GQg&url=http://www.ssrn.com/link/florida-state-public-law.html&ei=HYeuVaT1Lsrl-AGG2piQBA&bvm=bv.98197061,d.cWw&psig=AFQjCNFUWZcfRdvv7HJOKzibrIG4x4lbgA&ust=1437587612635885 https://upload.wikimedia.org/wikipedia/en/0/0e/UCF_horizontal_logo.svg https://upload.wikimedia.org/wikipedia/en/0/0e/UCF_horizontal_logo.svg http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCMOWuurm7MYCFcxWPgodBYgNkw&url=http://people.csail.mit.edu/haitham/&ei=soiuVcP1Acyt-QGFkLaYCQ&bvm=bv.98197061,d.cWw&psig=AFQjCNF-D5zteqeZSAn8eaavAp3gR7c7VQ&ust=1437587971070366 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCMOWuurm7MYCFcxWPgodBYgNkw&url=http://people.csail.mit.edu/haitham/&ei=soiuVcP1Acyt-QGFkLaYCQ&bvm=bv.98197061,d.cWw&psig=AFQjCNF-D5zteqeZSAn8eaavAp3gR7c7VQ&ust=1437587971070366 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCIrRha7n7MYCFQFbPgodLGsH5Q&url=http://auvac.org/people-organizations/view/467&ei=P4muVcq1L4G2-QGs1p2oDg&bvm=bv.98197061,d.cWw&psig=AFQjCNGCFE5zdz63NnSCLjiOOcO1duWUcA&ust=1437588157773607 http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCIrRha7n7MYCFQFbPgodLGsH5Q&url=http://auvac.org/people-organizations/view/467&ei=P4muVcq1L4G2-QGs1p2oDg&bvm=bv.98197061,d.cWw&psig=AFQjCNGCFE5zdz63NnSCLjiOOcO1duWUcA&ust=1437588157773607

Operational Scenario

Intelligent System Components

Autonomy / Swarms

Network

Experts

Sensors

Knowledge

Bases

Autonomy for Networking : Networking for Autonomy

Autonomous Networking

Autonomous Networking

• Blend networking and autonomy to enhance both and provide resilient seamless services

• Hybrid multi-radio multi-wavelength PHY

• Autonomy as mobile infrastructure

• Cognitive, self-healing radio

• Seamless embedded PNT

• Distributed Beamforming & EW

Research Topics:

Distributed Intelligence: Establish the theoretical foundations of multi-faceted distributed networked intelligent systems combining autonomous agents, sensors, tactical super-computing, knowledge bases in the tactical cloud, and human experts.

Heterogeneous Group Control: Develop the theory and algorithms for control of large autonomous teams with varying levels of heterogeneity and modularity across sensing, computing, platforms, and degree of autonomy.

Adaptive and Resilient Behaviors: Develop theory and experimental methods for heterogeneous multi-agent groups to carry out tasks in the physical world.

Expected Payoff:

• Extended reach, situational awareness, and operational effectiveness against dynamic threats in contested environments

Distributed and Collaborative Intelligent

Systems and Technology (DCIST)

New Collaborative Research Alliance (CRA) Opportunity

• Distributed and Collaborative Intelligent Systems and Technology (DCIST)

• Program Announcement Feb 2017 - See official announcement for programmatic and submission details

• Will be announced through FedBizOpps

• Opportunities for collaboration with ARL researchers, leveraging ARL facilities, and an on-site presence

Vision:

Develop the underpinning science to extend the reach, situational awareness, and operational effectiveness of Intelligent System/Soldier teams against dynamic threats in complex and contested environments and provide technical and operational superiority through fast, intelligent, resilient and collaborative behaviors.

Tactical Offset Summary

• Merge cognition & control for distributed intelligent systems

• Fundamental new science of human – autonomy teaming

• Leap ahead in rapid deployment & autonomy projection

• Focus deep learning on Army intelligent system scenarios

• Autonomy for Networking : Networking for Autonomy

Drive new tactical offset strategy

End

Dr. Brian M. Sadler Senior Research Scientist, Intelligent Systems Army Research Laboratory brian.m.sadler6.civ@mail.mil

(o) 301-394-1239