ScAN_ProposersDay_and_CMO_FinalSlides_DISTRO_A.pdf

Scalable Analog Neural-networks (ScAN)

Proposers’ Day

May 15, 2024

2000 W

8 GPU

system <1 W

Analog ASIC

Moving from Artificial Intelligence in the cloud to Actionable Intelligence in the field

Dr. Bryan C. Jacobs Program Manager, MTO

Unclassified Distribution Statement A: Approved for Public Release, Distribution Unlimited

• Motivation

• High-Level Program Goals

• Technical Challenges

• Program Structure and Schedule

• Metrics and Milestones

• Submission Guidelines

• Evaluation Criteria

• Proposal Timelines and Reminders

Outline

2000x power reduction will transform how and where AI is used

MQ-9 Reaper

Airborne “ground” stations

Satellite-based sensing

Military applications Commercial applicationsDual use applications

AI for triage Search and rescue

Next-gen self-driving car

Unclassified Distribution Statement A: Approved for Public Release, Distribution Unlimited

(U.S. Air Force photo/Tech. Sgt. Ricky Best)

(DARPA) (SeaDronesSee, U.Tuebingen, MaCVi 2023)

(Photo by Dllu, Wikipedia, CC BY 4.0)(DARPA)

After ScAN: Edge sensors recommend actions and transmit intelligenceToday: Edge sensors collect and transmit data

Digital trends are unlikely to move advanced inferencing to the edge

Year

Po w er (W

ScAN will overcome digital technology limits to bring advanced inferencing to the edge

Digital power trend for detection and tracking problems

2000x reduction

NN: Neural network SoA: State-of-the-art TPU: Tensor processing unit IP2: In-Pixel Intelligent Processing program

Inferencing Training

TPU v1

TPU v2

TPU v4i

ScAN will demonstrate:

• Viability of NN subsystems with >200x power efficiency gains

• Scalability of NN architectures with >2000x power efficiency gains

IP2 Projection

ScAN Viability (1M Parameters)

ScAN Scalability (50M parameters)

Hybrid analog architectures: a step in the right direction

Hybrid Processor

Sensor Weight memory

Intermediate result memory

Weights

MAC results

Layer data Digital

Processor (1 GHz)

Analog

MAC

>100 MHz

ADC

(30 Hz)

Analog MAC

MAC: Multiply-accumulate DAC: Digital-to-analog converter ADC: Analog-to-digital converter

NVM

Crossbar Analog Circuit

Analog:

Digital:

Legend

Analog MACs boast >500x power efficiency, but…

• Hybrid architectures limit system advantage to only 20x digital because they are mostly digital

• Digital is used to mitigate analog circuit limitations

New architectures and methods are needed to realize the full potential of analog

Weights

Active and adaptive architectures: re-think everything to maximize analog

Weight memory (e.g., SRAM)

NN cell #1

Activation

Analog

MAC

Sensor- Proximate Adaptive

Algorithms Activation

Analog

MAC

Activation

Analog

MAC

NN cell #2 NN cell #M

Sensor

Active circuits provide robustness to variations while enabling efficiency gains

Adaptive algorithms actively enhance robustness and scalability with increased efficiency

Post-processing

ScAN’s focus is co-design of a new class of robust and scalable analog architectures

MAC: Multiply-accumulate PVT: Process Voltage Temperature NN: Neural network

Architectures implementing adaptive algorithms that use active circuits can overcome analog processing robustness and scalability challenges – to deliver 2000x power efficiency gains

One of many potential system concepts

Analog:

Digital:

Legend

Normalized impedance

Im ag eN et

A cc ur ac y

10-810-9 10-7 10-6 10-5

100%

60%

80%

40%

20%

Scalability and efficiency currently limited by passive techniques

MAC: Multiply Accumulate PVT: Process Voltage and Temperature DAC: Digital-to-analog converter ADC: Analog-to-digital converter

Simulated performance of passive techniques

1152×1152 MAC array

Required level

Manufacturing limit

T. P. Xiao, et al., IEEE Circuits Syst. Mag. 22, pp. 26-48 (2022)

Source: J.-O Seo, et al., ISSCC 2022

Example active circuit

ScAN

Components and circuits less sensitive to PVT variations Output current set by drive circuit – not by parasitic resistance

Passive Active

ScAN challenge is to innovate techniques to overcome complexity of active architectures

Adaptive algorithm strategy

There are many promising research directions to enhance analog efficiency, robustness, and scalability

MAC: Multiply-accumulate PVT: Process Voltage Temperature NN: Neural network

Adaptive algorithms adjust hardware parameters in real-time, or pre-mission

Weight memory (e.g., SRAM)

NN cell #1

Activation

Analog

MAC

Activation

Analog

MAC

Activation

Analog

MAC

NN cell #2 NN cell #M Post-processing

Adaptive Calibration

Pre-mission (off-chip)

Real-time (on-chip)

Adaptive Optimization

Adaptive Tuning

Problem: Accuracy degrades over time Adaptive biasing technique shows promise in restoring and improving lost accuracy

Pre-mission adaptive algorithms enhance robustness Ac cu ra cy

Time (s)

Source: V. Joshi, et al., Nat Commun 11, 2473 (2020)

Accuracy over time for large scale application da y

Large:

ResNet-34 (22M weights)

Required

Simulated

Source: H. Bahng, et al., arXiv:2203.17274 [cs.CV]

Active and adaptive architectures will overcome effects of drift and variations

42.4%

81.4%

Accuracy

Phase targets for neural network sizes

Source: T. P. Xiao, et al., IEEE Circuits Syst. Mag. 22 (2022)

Phase 1:

• Demonstrate viability at the smallest size that makes sense

Risk: Architecture does not mitigate effect of variations

Academic

Mission

Small

100K weights

1M weights

25-50M weights

Phase 2:

• Demonstrate scalability to mission-relevant sizes

Risk: Architectural overhead outweighs efficiency gains

Ph as e

Ph as e

Passive Analog NN sensitivity to size (simulated)

Re la tiv e

Ac cu ra cy

(t o

So A

Di gi ta l)

Analog Circuit Variation

Program structure and schedule

CY 2024 CY 2027CY 2025 CY 2026 CY 2028

FY 2027FY 2024 FY 2025 FY 2026 FY 2028

15 months 12 months 27 months

Phase 1a Robustness

Phase 1b Robustness

Phase 2 Scalability

PDR

CDR

Demo

Intermediate Scale 1000 inputs

1M parameters

Mission Scale 1M inputs, 50M parameters > TBD less power at intermediate scale

> TBD less power at mission scale

Legend:

Critical Milestone

Milestone

PDR: Preliminary design review CDR: Critical design review (tape-out ready)

Metrics and milestones

Metrics and Milestones Phase 1 Phase 2

Accuracy (% of state of the art) > TBD > TBD

Sensitivity (% accuracy degradation) < TBD

Projected power efficiency gain > TBD

Demonstrated power efficiency gain > TBD > TBD

Device-dependent training (core-hours) N/A TBD

Constraints Phase 1 Phase 2 Minimum Model size:1

No. of parameters 1M 50M

Hardware demonstration (Input Output) 103 10 106 103

Latency (ms) < 16 1Equivalent digital model size

Out of scope

• Quantum computing

• All-digital computing

• Non-volatile memory-based computing

• Device processing technology development

• DARPA NaPSAC or OPTIMA program technology development

Requirements

• Proposal must address all program objectives

Comprehensive team with expertise to cover all program areas (hardware and algorithm)

Optional (but encouraged)

• Abstract submission

ScAN Submission Guidelines

A. Innovative Claims

• Concise summary of most innovative claims

B. Technical Approach

• Clearly describe the proposed approach and delineate individual technologies – if applicable

• Succinctly summarize the innovative claims and explain their novelty

• Provide compelling support for how the proposed approach will meet the ScAN program milestones and goals

• Provide a concise description of the interaction between algorithm and hardware tasks

• Especially how the results of each task will inform the development of the other

• Provide estimates of the following resources:

• Computing requirements for performing projections and benchmarking

• Fabrication requirements and cycle-times

C. Cost and Deliverable Schedule

• Rough cost estimate for resources (e.g. labor, materials) and subcontractors

• Broken down by phase

D. Teaming

• High-level teaming arrangements with a brief organization chart

• List of key persons and their roles within individual tasks (with short bio, if space permits)

ScAN Abstract Guidelines – Consult BAA for page limitations

Technical approach

• Clearly describe the proposed approach with no jargon

• Succinctly summarize the innovative claims and explain their novelty

• Demonstrate a clear understanding of the state-of-the-art in analog and mixed-signal neural network technologies, such in-memory computing, and emerging analog solutions for targeted applications

• Provide compelling (ideally, quantitative) support for how the proposed approach will overcome or obviate the technical challenges (listed in the BAA) to meet the ScAN program milestones and goals

• How the proposed architecture approach will overcome scaling limitations and short-term performance fluctuations, not just extrapolation of academic or toy sized networks for, e.g., MNIST digit recognition;

• How the proposed circuit/hardware approach will overcome or mitigate performance (inference accuracy) degradation due to process, voltage, and temperature (PVT)-variations, and long-term drifts, especially in large-scaled networks, not just extrapolation or bit-resolution reduction argument; and

• How the proposed approach will overcome device-dependent variations with minimal calibrations or model fine-tuning for practical applications

ScAN Proposal Guidelines

• Enumerate technical risks of proposed approaches, and explain corresponding mitigation strategies for

• Design and simulation of the proposed architecture; and

• Model training and application performance estimation

• Provide a description of the interaction between hardware and algorithm tasks, especially how the results of each task will inform the development of the other

• Describe the proposer’s expertise and prior experience in large-scale analog circuit design and IP integration in similar efforts, and capabilities of existing design infrastructure suitable for the proposed technology node at the specific foundry

• Provide estimated fabrication cycle-times and expected number of runs (by phase) for the proposed technology, and how the proposed schedule will meet the program specified milestones

ScAN Proposal Guidelines (Cont)

Overall Scientific and Technical Merit

• Demonstrate that the proposed technical approach is innovative, feasible, achievable, and complete

• Describe how the proposed approach will achieve each program metric with sufficient detail and supporting experimental measurements, modeling, calculations, and/or simulations

Potential Contribution and Relevance to the DARPA Mission

• Discuss how the proposed effort addresses the DARPA goals of technology transition and facilitates access to

ScAN technologies for DoD applications

• Impact of proposed IP rights structure (if applicable) to advance U.S. defense capabilities

Cost Realism

• Ensure proposed costs are realistic for the technical and management approach and accurately reflect the goals and objectives of the solicitation

• Verify that proposed costs are sufficiently detailed, complete, and consistent with the Statement of Work

ScAN Evaluation Criteria

Note: Official BAA guidance supersedes everything presented and discussed today

Form integrated teams with comprehensive expertise to cover all program objectives

• Submit a FULL proposal to address all technical requirements

Adhere to page limits for the technical volume (compliance)

ScAN Reminders

Submit additional questions to the BAA Coordinator: HR001124S0022@darpa.mil www.darpa.mil

19Unclassified

Scalable Analog Neural-networks (ScAN)

Proposers Day

15 May 2024 Caroline Allen

Contracting Officer DARPA Contracts Management Office

Distribution Statement A. Approved for Public Release; Distribution Unlimited.

Proposers Day Disclaimer

Lots of information is made available to potential proposers to clarify program goals/objectives and proposal preparation instructions - those things that are stipulated in the BAA

However:

• Only the information/instructions in the BAA counts

• Proposals will only be evaluated in accordance with the instructions provided in the BAA

• Any response provided by the Government in the FAQ that’s different than what is provided in the BAA will be made formal by an amendment to the BAA

• Such responses will make note of an impending BAA amendment

Only a duly authorized Contracting Officer may obligate the Government

Distribution Statement A. Approved for Public Release; Distribution Unlimited. 2

BAA Overview BAAs allow for a variety of technical solutions

– The BAA defines the problem set, the proposer defines the solution (and SOW)

– Allows for multiple award instrument types:

• Procurement Contract, or Other Transaction

• Anticipated Funding Type: 6.2 (Applied Research) Restricted research for for-profit team members (prime or subcontractor) Fundamental research for universities (prime and subcontractor)

DARPA Scientific Review Process

– Proposals are evaluated on individual merit and relevance as it relates to the stated research goals/objectives rather than against one another

– Selections will be made to proposers whose proposals are determined to be most advantageous to the Government, all factors considered, including potential contributions to research program and availability of funding

Government may select for negotiation all, some, one, or none of the proposals received Government may accept proposals in their entirety or select only portions thereof Government may elect to establish portions of proposal as options

22Distribution Statement A. Approved for Public Release; Distribution Unlimited.

Eligibility Issues

Foreign participants/resources may participate to the extent allowed by applicable Security Regulations, Export Control Laws, Non-Disclosure Agreements, etc.

FFRDCs and Government entities:

- Are, however, subject to applicable direct competition limitations

- Are, however, required to demonstrate eligibility (sponsor letter)

The burden to prove eligibility for all such team members rests with the proposer

All elements of a proposal (tech and cost, prime and subs – even FFRDC team members) must be included in the prime’s submission

Real and/or Perceived Conflicts of Interest:

- Identify any conflict/s

- If any are identified, a mitigation plan must be included

23Distribution Statement A. Approved for Public Release; Distribution Unlimited.

Proposal Abstracts (If Applicable in the BAA)

Abstracts are highly encouraged:

1. They minimize unnecessary effort in proposal preparation and review

2. They reduce the potential expense of preparing an out of scope proposal

The abstract provides a synopsis of the proposed project (tech and budget)

Government will reply by letter with one of two possible responses:

1. Encourage full proposal, and may provide feedback

2. Discourage full proposal, and will provide rational

DARPA will not communicate further (verbally or in writing)

Regardless of DARPA’s response to an abstract, proposers may submit a full proposal DARPA will review all full proposals submitted without regard to abstract recommendation/feedback

24Distribution Statement A. Approved for Public Release; Distribution Unlimited.

Data Rights Government desires as few restrictions as possible - however….

If asserting less than Unlimited Rights (e.g., Restrictions):

– Provide and justify basis of assertions using the prescribed format

– Explain how each item will be used to support the proposed research project

– Explain how the Government will be able to reach its program goals (including technology transition)

The proposer (prime) must submit a Data Rights Cert covering the entire team (prime and subcontractors), as applicable Provide a Data Rights Cert no matter the instrument type being proposed This information is assessed during evaluations (barriers to transition)

25Distribution Statement A. Approved for Public Release; Distribution Unlimited.

Pitfalls That Delay Proposal Review or Result in Non-Conforming

Failure to submit proposal on time - noncompliant!

– Proposal due date and BAA closing date are the same – so, late is late!

Failure to submit using the correct mechanism - noncompliant!

– Unclassified proposals (for procurement contract or OT) ONLY to DARPA BAA website

– Grant/Cooperative Agreement proposals ONLY to grants.gov

– Classified proposals/information ONLY per “Security Information” section of the BAA

Failure to submit both proposal volumes - noncompliant!

– OT proposals must also include a full cost volume (Cost Realism is an evaluation criterion for all proposals)

– OT proposals must also include a detailed list of payment milestones (Milestone Plan)

Pages beyond the page limitation (tech prop) - pages will not be reviewed (or could result in being deemed noncompliant!)

ROM/s instead of full subcontract cost proposal/s - noncompliant!

– “I didn’t have time to get the subcontract proposal/s” will not change the outcome

– “My subcontractor/s would not give me a proposal” will not change the outcome

Missing FFRDC or Government Entity cost proposal – noncompliant!

26Distribution Statement A. Approved for Public Release; Distribution Unlimited.

Communications

Prior to Receipt of Proposals (Solicitation Phase): No restrictions, however Gov’t (PM/PCO) shall not dictate solutions or transfer technology Typically handled through the FAQ, but see BAA exceptions

After Receipt of Proposals/Prior to Selections (Scientific Review Phase):

Limited to Contracting Officer or BAA Coordinator (with approval) to address clarifications requested by the review team Proposal cannot be changed in response to clarification requests

After Selection/Prior to Award (Negotiation Phase): Negotiations are conducted by the Contracting Officer PM and/or COR typically tasked with finalizing the SOW (with PI) PM and/or COR typically involved in any technical discussions (i.e., partial selection discussions) Pre-award costs will not be reimbursed unless a pre-award cost agreement is negotiated prior to award

Informal Feedback Sessions (Post Selection): May be requested/provided once the selection(s) are made

– If made on a timely basis (~2 wk after letter), all requests will be accepted

27Distribution Statement A. Approved for Public Release; Distribution Unlimited.

Referenced Links http://www.darpa.mil/work-with-us/contract-management#SolicitationContracting

(General DARPA contract management information) (DARPA Standard Cost Proposal Spreadsheet) http://www.darpa.mil/work-with-us/additional-baa (general BAA info pertaining to all instrument types) http://www.darpa.mil/work-with-us/procurementcontracts (info pertaining to contracts) https://acquisitioninnovation.darpa.mil/ (info pertaining to OTs) http://www.darpa.mil/work-with-us/reps-certs (DARPA-specific reps and certs for all instrument types) (Note August 2020 version of 52.204-24)(Similar is required for OTs as well) https://www.darpaconnect.us/home

28Distribution Statement A. Approved for Public Release; Distribution Unlimited.

http://www.darpa.mil/work-with-us/contract-management#SolicitationContracting http://www.darpa.mil/work-with-us/additional-baa http://www.darpa.mil/work-with-us/procurementcontracts https://acquisitioninnovation.darpa.mil/ http://www.darpa.mil/work-with-us/reps-certs https://www.darpaconnect.us/home