HR001123S0025 Industry Day Slides.pdf

Distribution Statement A: Approved for public release. Distribution is unlimited.

DARPA FIRE Industry Day

Dr. Lok Yan

March 16, 2023

Distribution Statement A: Approved for public release. Distribution is unlimited. 2

Program Security

Mr. Terry Cook

Distribution Statement A: Approved for public release. Distribution is unlimited. 3

This meeting will be:

UNCLASSIFIED

• Classified discussions are not permitted during this conference!

• Security Topics

• Security POC

• Attachment 6 (Classified BAA Addendum)

• Facility Certification & Accreditation (C&A)

• PRIME & SUB DD-254 Process

• Information Assurance (IA) Requirements

• Documents

• Other Classified Sources & Proposals

Level and Topics

Distribution Statement A: Approved for public release. Distribution is unlimited. 4

BAA Coordinator / General Contact: FIREProgram@darpa.mil

PSR Contact: Terry R. Cook Jr., Terry.Cook.ctr@darpa.mil, 571-218-4514 (vIPer), SAV VoIP 4011218

SCA Contact: Michael (Mike) Corcoran, michael.corcoran.ctr@darpa.mil, 571-218-4272

If proposing, items need to be sent to BAA email ASAP (Sooner is better):

-Attachment 6 Request: Send to FIREProgram@darpa.mil.

-Confirm all performance locations participating in proposal (for approved DD-254). Performers must have an approved DD-254 to begin classified work for each location of their performance.

-Unclassified IT systems must be NIST SP 800-171 compliant for Controlled Unclassified Information.

Note:

Proposer security should contact the PSR and SCA through the BAA email address, FIREProggram@darpa.mil, to initiate discussions soonest company decides to propose to BAA. Coordination items will need to include facility, clearances, sub contracts/teaming, and network or optional laptop decision. Do not fire and forget if you feel we are not responding to your request in a timely fashion call, email, or Cc the QA email group.

Security POCs mailto:FIREProggram@darpa.mil

Distribution Statement A: Approved for public release. Distribution is unlimited. 5

Addendum Request Form (Attachment 6) is the only method of request that will be accepted.

To ensure a swift and organized processing time, please notify us of:

• Your request for the Classified BAA Addendum (will ensure it is processed in time) by April 10th.

• Your intent to submit your classified proposal by April 24th if not sending electronically to allow time to coordinate mailing/hand carrying.

• All information is required for attachment 6. Completed forms will be processed in the order they are received.

Incomplete forms will cause delays in processing.

• Your contracting office address & CAGE Code along with your performance location address and CAGE Code.

• Principal Investigator (PIs) are required to engage and inform their Contractor Program Security Officer (CPSOs) of their intent to propose and it is the CPSO's responsibility to engage and notify DARPA PSR during the proposal.

Attachment 6

Distribution Statement A: Approved for public release. Distribution is unlimited. 6

Ensure you are working any and all Certification & Accreditation (C&A) prior to CONTRACT AWARD (if selected)

Ensure Facilities are Certified and Accredited for storage and processing

Facility identified and listed in NISS (for FCLs) for facility verification

NOTE: A method of contact (Phone#, email or both) at the CLASSIFIED LEVEL for discussion.

Facility Certification & Accreditation (C&A)

Distribution Statement A: Approved for public release. Distribution is unlimited. 7

If teaming, DARPA Security will need to know WHO is teaming with WHOM for tracking purposes. Prime contractors are responsible for submitting DD-254s for subcontractors to DARPA security.

Ensure Facilities are Certified and Accredited for storage and processing.

Ensure personnel performing have the appropriate CLEARANCE level.

Your contracting address CAGE Code and each performance location address and CAGE Code performing classified work on the proposal.

Prime contractor’s need to validate & verify sub-contractors in NISS.

DARPA security cannot edit a subcontractor DD-254. Incorrect DD-254s will cause delays. DARPA Industrial Security will not accept mistakes. The responsibility is with the prime contracting security team to ensure accuracy of subcontractor DD-254.

Proposer’s and Subcontractor DD-254

Distribution Statement A: Approved for public release. Distribution is unlimited. 8

IA REQUIREMENTS (for both PROPOSAL and AWARD) PROPOSAL

• DCSA accredited classified standalone system e.g. Multi-User Standalone (MUSA) or Local Area Network (LAN) etc. Via Classified Networks (i.e. DSWAN, SIPRNET, SAV, JWICS etc.) if available

• DSWAN will need to be activated through DSWAN Coordinator

• Need DSWAN NODE at your site of Performance

• DD-254 / FCL / NODE / CO-SHARE between PMs

WE CAN HELP

INFORMATION ASSURENCE PERSONNEL (8570.01 COMPLIENT)

• ISMM/ISOO need CISSP & SECURITY + APPOINTED and signed off by PSO (AWARD ONLY, not for Proposal)

COMPLIANCE WITH CONTROLLED UNCLASSIFIED INFORMATION (CUI)

• NATIONAL INSTITUTE of STANDARDS & TECHNOLOGY (NIST) Special Publication (SP) 800-171 “Protecting Controlled Unclassified Information in Non-Federal Information Systems and Organizations”

• Forwarding to other personnel and systems that don’t meet the NIST Standards and Non-US Personnel is a violation

Information Assurance (IA) Requirements

Distribution Statement A: Approved for public release. Distribution is unlimited. 9

Performers can expect to receive the following once completing and submitting the Classified Addendum Request (Attachment 6) to FIREProgram@darpa.mil.

• Classified Addendum

This will be sent via:

• High Side Email

• by Classified mailing address upon request

Documents

Distribution Statement A: Approved for public release. Distribution is unlimited. 10

PROPOSALS THAT INCLUDE CLASSIFIED INFORMATION FROM OTHER CLASSIFIED SOURCES

(NOT DARPA)

-Need PERMISSION from Cognizant Gov. Agency.

-Marked IAW the source SCG.

-Source SCG is submitted along with Proposal.

BAA PROPOSALS SUBMISSIONS:

Always refer to the BAA and follow instructions for submitting your proposals. Consult with the BAA coordinator for submission instructions.

If electronic submissions of classified proposals are not available, you must coordinate with the DARPA PSR for mailing or delivering the proposal package via courier.

Other Classified Sources & Proposals

Distribution Statement A: Approved for public release. Distribution is unlimited. 11

Contracting Overview

Mr. Lon Wang

Distribution Statement A: Approved for public release. Distribution is unlimited. 12

• Lots of information is made available to potential proposers to clarify program goals/objectives and proposal preparation instructions – those things that are stipulated in the BAA

However:

• Only information/instructions in the FIRE BAA counts

• Proposals will ONLY be evaluated in accordance with the instructions provided in the BAA

• Any response provided by the Government in the FAQ that’s different than what is provided in the BAA will be made formal by an amendment to the BAA

• Such responses will make note of an impending BAA amendment

Only a duly authorized Contracting Officer may obligate the Government

Proposers Day Disclaimer

Distribution Statement A: Approved for public release. Distribution is unlimited. 13

• BAAs allow for a variety a variety of technical solutions

• The BAA defines the problem set, the proposer defines their unique solution (and SOW which discuss what you will do to attempt to achieve the program goal)

• Allows for multiple award instrument types:

• Procurement Contract, Other Transactions, Cooperative Agreements (CO has sole discretion)

• Grants are NOT a planned award instrument

• Anticipated Funding Type: 6.2 (Applied Research)

• Restricted Research for for-profit team members (prime or subcontractor)

• Fundamental research for some work on universities if work done on campus (prime and subcontractor)

• Attachment 5 to BAA: FIRE CUI Guide

• DARPA Scientific Review Process

• Proposals are evaluated on individual merit and relevance as it relates to the stated research goals/objectives rather than against one another

• Selections will be made to proposers whose proposals are determined to be most advantageous to the Government, all factors considered, including potential contributions to research program and availability of funding

• Government may select for negotiation all, some, one, or none of the proposals received

• Government may accept proposals in their entirety or select portions thereof

• Government may elect to establish portions of proposal as options

FIRE BAA Overview

Distribution Statement A: Approved for public release. Distribution is unlimited. 14

1. BAA Published: 15 March, 2023

2. Proposer’s Day: 16 March, 2023

3. Abstract Submission Deadline: 31 March, 2023

4. FAQ Submission Deadline: 21 April, 2023

5. Proposal Due : 19 May, 2023

6. Proposals are reviewed for BAA Compliance

Noncompliant proposals are not reviewed (and cannot be selected)

7. Government conducts Scientific Review Process

Clarification requests may be sent to various proposers

8. Government sends out notification letters: ~July 2023

9. Contracts negotiated & awarded (performance start): ~October 2023

See BAA for complete date/time information

Monitor SAM.gov for any BAA amendment(s)

BAA Process/Timeline

Distribution Statement A: Approved for public release. Distribution is unlimited. 15

• All interested/qualified sources may respond subject to the parameters outlined in the BAA

• Foreign participants/resources may participate to the extent allowed by applicable Security Regulations, Export Control Laws, Non-Disclosure Agreements, etc.

• FFRDCs and Government entities:

• Are not prohibited by the BAA from proposing

• Are, however, subject to applicable direct competition limitations

• Are, however, required to demonstrate eligibility (sponsor letter)

• The burden to prove eligibility for all such team members rests with the proposer

• All elements of a proposal (tech and cost, prime and subs – even FFRDC team members) must be included in the prime’s submission

• Real and/or perceived conflicts of interest

• Identify any conflict/s

• If any are identified, a mitigation plan must be included

Eligibility Issues

Distribution Statement A: Approved for public release. Distribution is unlimited. 16

• Key Points:

• 2 Phases; address all metrics

• An organization/company/individual can be on multiple proposals, unless proposing to TA5

• If proposing to either of the TA5 tracks, the organization/company/individual must identify and describe a technical firewalling plan in ALL proposals they are a part of

• Submit unclassified proposals ONLY to the DARPA BAA website (https://baa.darpa.mil)

• Click “Finalize Full Proposal” button (otherwise you have not submitted)

• Submit classified proposal addendum to DARPA per “Security Information” section of the BAA

• Do not submit classified proposals/information to the DARPA BAA Website

• If unsure, please ask question (well before the proposal due date)

Propose to the program (goals, objectives, metrics, schedule, deliverables) the BAA has defined not the program you desire

Full Proposal Ground Rules https://baa.darpa.mil/

Distribution Statement A: Approved for public release. Distribution is unlimited. 17

• Volume 1: Technical/Management Proposal

• Be mindful of the page limitations (changes depending on how many Tas)

• Be sure to respond to all of the required “Detailed Proposal Information items”

• Technical Approach – is the centerpiece of the technical proposal

• Statement of Work – Organize by Phase, then by Tasks. Define all tasks (what you are doing, not how you are doing

it) and deliverables (data, software, and material items, as applicable).

• Volume 2: Cost Proposal

• No page limitations

• Fully detailed cost build-ups/estimates (Prime & Subcontractors)(All instrument types)

• Summary Cost Build Up: By Phase and performer fiscal year

• Detailed Cost Build-up: By Phase, technical task, and month

• See Note 4 (this is new to all DARPA BAAs): DARPA Standard Cost Proposal Spreadsheet

• All necessary supporting information (BoE, BoM, vendor quotes, rate agreements, etc.)

• Subcontractor proposals are required (to include SOW)

• No Rough Order of Magnitudes (ROMs)

Keep an eye out for proposal preparation guidance in the “Funding Opportunity Description”

Full Proposal Preparation

Distribution Statement A: Approved for public release. Distribution is unlimited. 18

Volume 2: Cost Proposal

Note 4 – DARPA Standard Cost Proposal Spreadsheet The Government strongly encourages* that proposers use the provided MS ExcelTM DARPA Standard Cost Proposal Spreadsheet in the development of their cost proposals. A customized cost proposal spreadsheet may be an attachment to this solicitation. If not, the spreadsheet can be found on the DARPA website at http://www.darpa.mil/work-with-us/contract-management (under “Resources” on the right-hand side of the webpage). All tabs and tables in the cost proposal spreadsheet should be developed in an editable format with calculation formulas intact to allow traceability of the cost proposal. This cost proposal spreadsheet should be used by the prime organization and all subcontractors. In addition to using the cost proposal spreadsheet, the cost proposal still must include all other items required in this announcement that are not covered by the editable spreadsheet. Subcontractor cost proposal spreadsheets may be submitted directly to the Government by the proposed subcontractor via e-mail to the address in Part I of this solicitation. Using the provided cost proposal spreadsheet will assist the Government in a rapid analysis of your proposed costs and, if your proposal is selected for a potential award, speed up the negotiation and award execution process.

*have a really good reason not to use it (in addition to any other spreadsheet you typically provide or need to provide to be compliant).

• Use the TA spreadsheet applicable for your teaming structure

• Do not make changes that are specifically prohibited by the instructions

• Questions are welcomed. Please direct them to costproposal@darpa.mil

Full Proposal Preparation http://www.darpa.mil/work-with-us/contract-management mailto:costproposal@darpa.mil

Distribution Statement A: Approved for public release. Distribution is unlimited. 19

• Government desires as few restrictions as possible – however….

• If asserting less than Unlimited Rights (e.g., Restrictions):

• Provide and justify basis of assertions using the prescribed format

• Explain how each item will be used to support the proposed research project

• Explain how the Government will be able to reach its program goals (including technology transition)

• The proposer (prime) must submit a Data Rights Cert covering the entire team (prime and subcontractors), as applicable

• Provide a Data Rights Cert no matter the instrument type being proposed

• This information is assessed during evaluations (barriers to transition)

Data Rights

Distribution Statement A: Approved for public release. Distribution is unlimited. 20

• Failure to submit proposal on time – noncompliant!

• Proposal due date and BAA closing date are the same – so, late is late!

• Failure to submit using the correct mechanism – noncompliant!

• Unclassified proposals ONLY to DARPA BAA website

• Classified proposal addendums ONLY per “Security Information” section of the BAA

• Ask questions if you have them (well before the closing date!)

• Failure to submit both proposal volumes – noncompliant!

• OT proposals must also include a full cost volume (Cost realism is an evaluation criterion for all proposals)

• OT proposals must also include a detailed list of payment milestones (Milestone Plan)

• Pages beyond the page limitation (tech prop) – pages will not be reviewed

• ROM/s instead of full subcontract cost proposal/s – noncompliant!

• “I didn’t have time to get the subcontract proposal/s” will not change the outcome

• “My subcontractor/s would not give me a proposal” will not change the outcome

Pitfalls that delay proposal review or result in non-conforming

Distribution Statement A: Approved for public release. Distribution is unlimited. 21

• Prior to Receipt of Proposals (Solicitation phase): No restrictions, however Gov’t (PM/PCO) shall not dictate solutions or transfer technology

• Typically handled through the FAQ

• After Receipt of Proposals/Prior to Selections (Scientific Review Phase): Limited to Contracting Officer or BAA Coordinator (with approval) to address clarifications requested by the review team

• Proposal cannot be changed in response to clarification requests

• After selection/prior to award (Negotiation Phase): Negotiations are conducted by the Contracting Officer

• PM and/or COR typically tasked with finalizing the SOW (with PI)

• PM and/or COR typically involved in any technical discussions (i.e., partial selection discussions)

• Pre-award costs will not be reimbursed unless a pre-award cost agreement is negotiated prior to award

• Information Feedback Sessions (Post Selection): May be requested/provided once the selection(s) are made

• If made on a timely basis (~2 weeks after letter), all requests will be accepted

Communications

Distribution Statement A: Approved for public release. Distribution is unlimited. 22

• http://www.darpa.mil/work-with-us/contract-management#SolicitationContracting

(General DARPA contract management information)

(DARPA Standard Cost Proposal Spreadsheet)

• http://www.darpa.mil/work-with-us/additional-baa

(general BAA info pertaining to all instrument types)

• http://www.darpa.mil/work-with-us/procurementcontracts

(info pertaining to contracts)

• https://acquisitioninnovation.darpa.mil/

(info pertaining to OTs)

• http://www.darpa.mil/work-with-us/reps-certs

(DARPA-specific reps and certs for all instrument types)

(Note August 2020 version of 52.204-24)(Similar is required for OTs as well)

Referenced Links http://www.darpa.mil/work-with-us/contract-management#SolicitationContracting http://www.darpa.mil/work-with-us/additional-baa http://www.darpa.mil/work-with-us/procurementcontracts https://acquisitioninnovation.darpa.mil/ http://www.darpa.mil/work-with-us/reps-certs

Distribution Statement A: Approved for public release. Distribution is unlimited. 23

Contracting Officer’s Representative

Mr. Sergey Panasyuk

Distribution Statement A: Approved for public release. Distribution is unlimited. 24

• BAA Rules

• Before selection

• Selection and awards FDO, Cost(s), Materials, Travel

• Execution of contracts CDRLs, reports, WAWF, FTR

• Contacts at AFRL/RIGB Sergey Panasyuk, COTR, sergey.panasyuk.1@us.af.mil Andrew Gorczyca, SME, andrew.gorczyca@us.af.mil

COR Overview

Distribution Statement A: Approved for public release. Distribution is unlimited. 25

FIRE Program Overview

Dr. Lok Yan

Distribution Statement A: Approved for public release. Distribution is unlimited. 26

Cyber-physical systems (CPS) and its components

Sensors Communications Processing

Applications Firmware BitstreamsSoftware

Hardware

Etc.

Etc.

Cyber vs. Physical CYBER

PHYSICAL

Distribution Statement A: Approved for public release. Distribution is unlimited. 27

CPS are everywhere

Medical devices Industrial control systemsSmart meters

VehiclesDrones

BRAKE

Distribution Statement A: Approved for public release. Distribution is unlimited. 28

CPS vulnerability example: activating automatic braking system

Cyber-physical system Component vulnerability CPS vulnerability

Exploit: inputs and conditions to realize vulnerability

Vulnerability: unexpected behavior

Distribution Statement A: Approved for public release. Distribution is unlimited. 29

FIRE focuses on medium-complexity CPS

Medium-complexity = ~1000 software components ~100 hardware components

Distribution Statement A: Approved for public release. Distribution is unlimited. 30

The Faithful Integrated Reverse-engineering and Exploitation (FIRE) seeks to develop transformative tools to find, exploit and patch vulnerabilities in medium-complexity cyber-physical systems (CPSs) within a month from when an analysis team receives the physical system. FIRE is primarily interested in Cyber-Physical Vulnerabilities (CPV), ones that arise from the composition of hardware, software and physical sub-systems where each component may not be vulnerable in-and-of itself. Component vulnerabilities, ones in individual sub-systems (e.g., software only), are of secondary interest. FIRE tools must not only find vulnerabilities, but also demonstrate impact by generating exploits and predicting the effects to the system.

Patches should also be generated when possible.

FIRE program goals (from BAA)

Distribution Statement A: Approved for public release. Distribution is unlimited. 31

Example: how to crash a quadcopter using acoustics

Source: Son et al. “Rocking Drones with Intentional Sound Noise on Gyroscopic Sensors“ 2015

1. Use speaker to inject false readings in Z-axis

3. Crash 𝑢 𝑡 = 𝐾𝑝𝑒 𝑡 + 𝐾𝑖න 𝑡 𝑒 𝜏 𝑑𝜏 + 𝐾𝑑 𝑑𝑒 𝑡 𝑑𝑡

2. Errant readings lead to wild swings in error 𝑒 𝑡 causing wild swings in output 𝑢 𝑡

0. Wait until quadcopter is in flight

FOR ILLUSTRATION PURPOSES ONLY. NOT COMMENTARY ON APPROACH.

MEMS GyroscopeSpeaker

Distribution Statement A: Approved for public release. Distribution is unlimited. 32

Example: CPS vulnerability requires a path through entire system

Source: Son et al. “Rocking Drones with Intentional Sound Noise on Gyroscopic Sensors“ 2015

1. Recognize component vulnerability

3. N/A 𝑢 𝑡 = 𝐾𝑝𝑒 𝑡 + 𝐾𝑖න 𝑡 𝑒 𝜏 𝑑𝜏 + 𝐾𝑑 𝑑𝑒 𝑡 𝑑𝑡

2. Recognize controller and other sub-systems do not negate component vulnerability

0. N/A

Distribution Statement A: Approved for public release. Distribution is unlimited. 33

Example: exploit uses vulnerabilities to obtain observable effects

Source: Son et al. “Rocking Drones with Intentional Sound Noise on Gyroscopic Sensors“ 2015

1. Identify inputs to activate vulnerabilities at right time

3. Observable effects 𝑢 𝑡 = 𝐾𝑝𝑒 𝑡 + 𝐾𝑖න 𝑡 𝑒 𝜏 𝑑𝜏 + 𝐾𝑑 𝑑𝑒 𝑡 𝑑𝑡

2. Identify paths from inputs to effects

0. Identify dependencies on system and environmental contexts (state)

Distribution Statement A: Approved for public release. Distribution is unlimited. 34

Example: patching nullifies vulnerability without impacting system

Source: Son et al. “Rocking Drones with Intentional Sound Noise on Gyroscopic Sensors“ 2015

1. Change MEMS sensor 3. N/A 𝑢 𝑡 = 𝐾𝑝𝑒 𝑡 + 𝐾𝑖න 𝑡 𝑒 𝜏 𝑑𝜏 + 𝐾𝑑 𝑑𝑒 𝑡 𝑑𝑡

2. Implement low-pass filter

0. N/A

Distribution Statement A: Approved for public release. Distribution is unlimited. 35

CPVA today

• Matlab models

• Software

• Simulators

• Test and evaluation

3. Timely Desired Effects

• Dependency graphs

• Software analysis

• Human reasoning

2. Vulnerability Analysis

CPS

1. Create digital twin

Expert team

Distribution Statement A: Approved for public release. Distribution is unlimited. 36

Technical challenge 1: modeling

• Matlab models

• Software

• Simulators

• Test and evaluation

TC1 Modeling: Scaling Complexity in Models while Maintaining Accuracy

• Models all engineering details

• Created as system is developed

• Limited to sub-systems, does not scale

3. Timely Desired Effects

• Dependency graphs

• Software analysis

• Human reasoning

2. Vulnerability Analysis1. Create digital twin

Distribution Statement A: Approved for public release. Distribution is unlimited. 37

Technical challenge 2: simulators

TC2 Simulation: Maintaining synchronization in simulations as the size of the system grows

• Data, control and timing dependencies

• Exploits must meet system expectations

• Limited to sub-systems, does not scale

• Matlab models

• Software

• Simulators

• Test and evaluation

3. Timely Desired Effects

• Dependency graphs

• Software analysis

• Human reasoning

2. Vulnerability Analysis1. Create digital twin

Distribution Statement A: Approved for public release. Distribution is unlimited. 38

Technical challenge 1: scaling model accuracy

Turing Machines

Linear Bounded Automata

Pushdown Automata

Finite Automata

Partial Differential Equations

Ordinary Differential

Non-linear

Linear

Arbitrary Precision

Double Precision

Fixed Point

Integer

Digital Twins More Accurate

Table Lookups Faster

Complexity scales exponentially with number of components

Distribution Statement A: Approved for public release. Distribution is unlimited. 39

Observation 1

Do you really know what to model and how accurate and precise it needs to be?

Distribution Statement A: Approved for public release. Distribution is unlimited. 40

Potential TC1 Approach: Falsification to identify model errors

Source: Zheng et al. “Falsification-Oriented Signature-Based Evaluation for Guiding the Development of Land Surface Models and the Enhancement of Observations.” 2020.

(a) (b)

AOBO : Range of signature of observations APBP : Range of signature of model predictions R: Runoff P: Precipitation ET: Evapotranspiration

Distribution Statement A: Approved for public release. Distribution is unlimited. 41

Potential TC1 Approach: Partitioning to adjust model accuracy

Adaptive complexity

Source: Qin et al. “Statistical Verification of Cyber-Physical Systems using Surrogate Models and Conformal Inference” (2022)

Uniform complexity

Distribution Statement A: Approved for public release. Distribution is unlimited. 42

Technical Challenge 2: Maintaining synchronization

5Hz

2Hz

Simulation rate of 10Hz

3Hz

Simulation rate of 15Hz

Simulation rate of 30Hz Least Common Multiple

1 2 3 4 5

A B C a b

Missed synchronization events

Distribution Statement A: Approved for public release. Distribution is unlimited. 43

Observation 2

Do you really know what to synchronize in all possible events?

Do you really need to consider all possible orderings?

Distribution Statement A: Approved for public release. Distribution is unlimited. 44

Potential TC2 Approach: Dynamic temporal decoupling to reduce synchronization events

T im in g i n a c c u ra c y μ s

Synchronization period (cycles)

Source: Junger et al. “Optimizing Temporal Decoupling using Event Relevance.” 2021 x x x x x x x x xx x x

Dynamic periods avoid synchronization errors due to temporal decoupling

Distribution Statement A: Approved for public release. Distribution is unlimited. 45

Potential TC2 Approach: Limit time synchronization horizons

0 5 10 15 20 -20

A lt it u d e cm

Time (s)

One second from cyber physical attack to drone crash

Source: Son. “Rocking Drones with Intentional Sound Noise on Gyroscopic Sensors.” 2015

Effects are quickly observable; simulations can end

Distribution Statement A: Approved for public release. Distribution is unlimited. 46

Technical Areas

• TA1 Modeling will focus on developing tools that can model entire systems (to include hardware, software and physical) with enough fidelity to find, exploit, and patch vulnerabilities, and are fast enough to meet the overall one-month program goal.

• TA2 Simulation will develop simulators that have enough precision to model interactions between system components and are fast enough to meet the overall program goals.

• TA3 Preparation will develop tools that reduce the amount of time needed to prepare a system for analysis to include techniques to accurately identify components, connections, and/or board layouts.

• TA4 Integration will create the FIRE tool(s) that meet the overall one-month program metric by integrating TA1, TA2, and TA3 solutions.

• TA5 Engineering Support Task will work with government and Independent Verification and

Validation (IV&V) teams to develop representative medium-complexity CPSs with full data rights for

TA1, TA2, TA3, and TA4 performers to test, evaluate, and demonstrate their solutions.

Distribution Statement A: Approved for public release. Distribution is unlimited. 47

Notional tool suite: hardware-on-the-loop analysis

Programmable laboratory equipment

CPS

Iterative models

Probe physical system only when needed

Vulnerability analysis on imperfect models

Distribution Statement A: Approved for public release. Distribution is unlimited. 48

A note on metrics; evaluating “patches” is hard

Vulnerabilities Exploits Patches

Ground truth IV&V vulnerabilities

IV&V observed effects

IV&V claim?

Evaluation methodology

Existence proof ✓ Existence proof ✓ Exhaustive proof?

Metric % or # found % or # verified % or # test cases passed?

Did the patch introduce new vulnerabilities?

Distribution Statement A: Approved for public release. Distribution is unlimited. 49

Metrics (from BAA)

Phase 1a Phase 1b Phase 2a Phase 2b

TA1

CPVA Accuracy1 90% 90% 90% 90%

TA2

Simulation Time2 10 seconds 10 seconds 1 second 1 second

TA3

Preparation time3 3 days 3 days 1 day 1 day

TA4

CPVA Development Time4 1 month 1 month 1 month 1 month

TA5

Engineering Support Task

10 software 2 hardware

100 software 10 hardware

100 software 50 hardware

1000 software 100 hardware

1CPVA Accuracy: the model’s ability to predict an exploit’s effect 2Simulation Time: the time needed to simulate one second of real-time 3Preparation Time: the time needed to identify components, their inter-dependencies, analysis point locations, and how the analysis points can be used 4CPVA Development Time: the total time from receiving CPS to exploit including preparation

Distribution Statement A: Approved for public release. Distribution is unlimited. 50

NSWCPD Overview

Distribution Statement A: Approved for public release. Distribution is unlimited. 51

• Industrial control systems (ICS) enable control, automation, and monitoring, typically in industry but increasingly found in smaller applications

• Electric plants, manufacturing facilities, refineries, transportation, etc.

• Recent applications – automobiles, home automation

• Consists of local controllers, field devices, sensors, network infrastructure, and supervisory controllers working together to complete a series of tasks and produce an end product or system state

• Typically utilizes embedded, low level controllers

• Requires information technology (IT) and operational technology (OT)

What is ICS?

Distribution Statement A: Approved for public release. Distribution is unlimited. 52

Purdue model for ICS https://dale-peterson.com/2019/02/11/is-the-purdue-model-dead/ https://dale-peterson.com/2019/02/11/is-the-purdue-model-dead/

Distribution Statement A: Approved for public release. Distribution is unlimited. 53

• “Information system (IT), an integrated set of components for collecting, storing, and processing data and for providing information, knowledge, and digital products.

• Information systems are used to run interorganizational supply chains and electronic markets”

• “Operational technology (OT) includes computer systems designed to be deployed in critical infrastructure (power, water, etc.), manufacturing, and similar industries.

• They automate, monitor, and manage the operations of industrial machinery, using custom protocols and software to communicate with legacy and proprietary systems”

Information Technology vs. Operational Technology

Distribution Statement A: Approved for public release. Distribution is unlimited. 54

Information Technology vs. Operational Technology

Distribution Statement A: Approved for public release. Distribution is unlimited. 55

• Maritime ICS is the managing and monitoring of ship systems that enable crews to live onboard and safely control the vessel

• Examples of these maritime control systems include:

• Powering the ship

• Driving the ship

• Navigation

• Feeding the passengers

• Processing waste water

• Balancing the ship

• Communicating with shore

• Etc.

• The onboard ICS systems enable these missions via real time control, extensive sensor data, and communication pathways providing situational awareness of current system states

• Control systems have evolved from early manual and analog systems to relying on digital, interconnected systems operating in harmony on a ship wide network

• There is potential for intelligent control to be built into every level of the Purdue model

What is Maritime ICS?

Distribution Statement A: Approved for public release. Distribution is unlimited. 56

• Ships are mission built

• Cargo shipping, fishing, warfare, hospital, leisure, etc.

• They have universal functionality regardless of mission

• They are designed to be operated by an optimally sized crew and to be self sufficient for some period of time

• This can require refueling and resupplying at sea

• Systems are interconnected which allows crews to maintain situational awareness and control over the entire vessel

What is a ship?

Distribution Statement A: Approved for public release. Distribution is unlimited. 57

• Bridge/Navigation

• Computer systems

• Displays

• Human Machine Interfaces (HMIs)

• Data feeds to physically separate areas on the ship

• Network Equipment

• Sensors

• radar, GPS, weather, AIS, sonar, video, etc.

• All consolidated and feed to appropriate information systems

• Propulsion

• Engines

• Including local engine controllers

• Generators

• Including local generator controllers

• Propulsor (propeller, waterjet, etc.)

• local controllers on these components

• Rudder control

• Water jet control

• Network equipment

• Provide situational awareness to supervisory control

Key Maritime Systems

Distribution Statement A: Approved for public release. Distribution is unlimited. 58

• Electric Plant

• Generators

• Including local control

• Switchboards

• Power converters/inverters

• Including local control

• Network equipment

• To feed situational awareness to supervisory control

• Supervisory Control

• The Machinery Control system that implements overarching control of the various subsystems

• Programmable logic controllers

• Support servers

• Data Historians

• Network Equipment

• Alarms

Distribution Statement A: Approved for public release. Distribution is unlimited. 59

• Water systems

• Hydraulics

• Bilge

• Reverse Osmosis Water maker

• Main Seawater

• Waste water management

• Motors, valves, pumps

• Hotel loads

• Lighting

• Laundry

• Galley

• HVAC

Distribution Statement A: Approved for public release. Distribution is unlimited. 60

• Shipboard systems are interconnected and feed data (status, heartbeat, commands, etc.) to one another in order to facilitate control of the vessel and achieve the ship’s mission

• Controllers are required in all of these systems and at all levels in the form of

• Remote Terminal Units (RTUs)

• Programmable logic controllers (PLCs)

• Intelligent Electronic Devices (IEDs)

• These controllers can interface with a variety of other equipment and support multiple protocols

• As technology advances, these controllers are becoming more intelligent and packed with “convenient” features

Key systems are interconnected

Distribution Statement A: Approved for public release. Distribution is unlimited. 61

Component Breakouts

Smart sensors

Programmable logic controller

Human machine interfaces

Families of smart devices make up the maritime ICS environment

Workstations

Distribution Statement A: Approved for public release. Distribution is unlimited. 62

• The lower level field devices are also essential components in these systems and they are also becoming more intelligent by default

• Network equipment is another essential element of the maritime ICS, also becoming more feature rich and capable

• Traditional IT equipment (workstations, network switches, servers) are necessary to connect the ships systems (OT) and provide crews the situational awareness and control they require

• That means that in addition to potential IT product vulnerability in the cyber arena ships also have to contend with adversaries exploiting weaknesses and bugs in the lower level, embedded OT equipment

Key systems are interconnected

Distribution Statement A: Approved for public release. Distribution is unlimited. 63

Variety of communication protocols

ICS Protocols

Distribution Statement A: Approved for public release. Distribution is unlimited. 64

Variety of communication protocols

Distribution Statement A: Approved for public release. Distribution is unlimited. 65

Ship systems support a variety of communication protocols via Ethernet and serial

• Modbus TCP

• Modbus RTU

• Profibus

• Profinet

• OPC (Open platform communication)

• CANbus

• Proprietary protocols

• Etc.

• Some of these protocols require inherent trust and the equipment that utilizes them must use a defense in depth, layered approach to cybersecurity

• The variety of protocols and the immense degree of interconnectivity between increasingly smarter components makes for a potentially large attack surface

Distribution Statement A: Approved for public release. Distribution is unlimited. 66

• Availabilities are the designated times when a ship can be pulled from its mission and updated or repaired

• This can mean technology remains fielded for years

• Since systems are so interconnected, it may be impossible to upgrade one system without upgrading others at the same time in order to maintain functionality

• This can actually prevent systems from getting upgraded

• Since the ship has to be self sufficient for a time, maintenance must be performed at sea and the tools required for maintenance must live onboard with the crew

• For intelligent systems this means maintenance laptops and troubleshooting software

Ship Maintenance Schedule

Distribution Statement A: Approved for public release. Distribution is unlimited. 67

• Ship design and construction takes several years

• Depending on the length of the design process, installed equipment can become outdated before the ship ever leaves port

• They are fielded for decades

• 25 – 50 years depending on the type of vessel

• Equipment will likely become obsolete during the ship’s life

• Equipment updates must be coordinated between multiple systems and cannot be done in a vacuum

• systems are increasingly interconnected and may require all equipment to “speak the same language”

Ship’s Lifespan

Distribution Statement A: Approved for public release. Distribution is unlimited. 68

• Ship technology typically lags behind the cutting edge

• When modern equipment is used, the industry trend towards feature rich devices increases the potential attack surface

• In most cases, for ship applications, the additional features are not necessary or desired

• There are a complex web of smart devices

• Potentially black box controllers

• Potentially proprietary protocols/software

• Interacting with many other systems on the ship

• Supply chain has to be considered not only from a security standpoint but also from an obsolescence and product reliability perspective since major changes to the ship have to wait for availabilities

Conclusion

Distribution Statement A: Approved for public release. Distribution is unlimited. 69

• Goal: Medium complexity cyber physical system in one month

• Expand idea of vulnerability: use the features of the system against itself

• Skid systems like High Pressure Air Compression

• Example specs:

• 1 HMI

• 1 Switch

• 2 redundant controllers

• 10+ sensors/field devices

• VFD, Valves, Pressure, temperature, flow, moisture

• At least 2 Protocols

• Multiple signal types

Maritime ICS and FIRE

Tank

PLC PLC

HMI

Compressor

S1 S2 S3 S4 S5 SN

Switch www.darpa.mil

Distribution Statement A: Approved for public release. Distribution is unlimited. 70