HR001119S0004-Amendment-01.pdf

HR001119S0004

Microsystems Technology Office Broad Agency Announcement

Photonics in the Package for Extreme Scalability (PIPES)

HR001119S0004

October 31, 2018

AMENDMENT 1

As amended on November 16, 2018

Foreword In June 2017, the Defense Advanced Research Projects Agency (DARPA) announced the Electronics Resurgence Initiative (ERI), a five-year, upwards of $1.5B investment in the future of domestic, U.S. Government (USG), and Department of Defense (DoD) electronic systems.

ERI recognizes and addresses long-foreseen obstacles to Moore’s Law, the transistor scaling trend that has allowed for 50 years of rapid progress in electronics. These obstacles include economic, geopolitical, and physics-based complications that strain both the commercial and defense sectors. The cost of integrated circuit fabrication, design, and verification is skyrocketing and limiting innovation; the continued move towards generalization and abstraction is stifling potential gains in hardware; and foreign investments are distorting the market and shifting technology development outside of the United States. To address these issues, ERI kicked off a major investment that draws on the contributions of several ongoing DARPA programs and creates new, long-term technology investments. Through novel research and development (R&D) in semiconductor materials and integration, architectures, and designs, ERI programs will promote circuit specialization as a complement to transistor scaling. The success of these efforts will depend on constructively enmeshing the technology needs and capabilities of the defense enterprise with the commercial and manufacturing realities of the electronics industry.

DARPA publicly announced competitive awards for the first group of six new ERI programs at the first annual ERI Summit, held in July 2018. In addition to introducing the awarded ERI performers, the ERI Summit featured four workshops designed to generate ideas for future ERI programs. Three key issues emerged from the workshop discussions: the need to support domestic manufacturing options and enable them to develop differentiated capabilities for diverse needs; a demand to invest in chip security; and a desire to create new connections between the various ERI programs and to demonstrate the resulting technologies in defense applications. ERI Phase II will build on the existing ERI programs to address all of these challenges, with the goal of supporting a domestic semiconductor manufacturing industry that can implement specialized circuits, demonstrate that those circuits can be trusted through the supply chain and are built with security in mind, and are ultimately available to both DoD and commercial sector users.

To create unique and differentiated domestic manufacturing capabilities, potential areas of exploration in Phase II include the integration of photonics, microelectromechanical systems (MEMS), and radiofrequency (RF) components directly into advanced circuits and semiconductor manufacturing processes. This is important for the DoD because the Department’s electronics manufacturing needs are numerous and diverse and its systems have unique requirements and specific functionality. Although traditional CMOS scaling for digital processing is still an important area of investment, many critical DoD-relevant electronics will likely derive from alternative and complementary vectors. Anticipated investments will seek to ensure that new capabilities support a strategy for the enduring availability of differentiated, high-performance electronics for the DoD and its commercial sector partners.

Over the fall of 2018, DARPA will announce additional investments relevant to issues highlighted at the ERI Summit. For instance, Summit participants discussed the need for traceability of electronics components, from design through to use, and for electronics that can enforce protections for security and privacy. Potential ERI Phase II programs under development will therefore consider the need for assured electronics that incorporate protections from security risks. Summit participants also requested new opportunities for participating in the effort, to include possibilities for increasing the connections between the various ERI efforts – from fundamental research programs to technology application programs. ERI Phase II will therefore explore emerging and future applications of ERI technologies, developed across all sectors, to defense-specific systems. These connections between programs and end users will be key to ERI’s overall success, driving DARPA’s ability to deliver differentiated capabilities to the DoD and its partners. Programs under development should help ensure that the technological advances that derive from ERI will deliver significant impact for national security. Potential areas of exploration could include ERI applications in large-scale physical emulation, cognitive RF systems, next-generation satellites, cybersecurity, and beyond.

Together with the ongoing ERI programs, including the “ERI Page 3 Investments” announced in 2017, ERI Phase II is the next step in creating a more robust, secure, and heavily automated electronics industry that will provide a foundational contribution both to U.S. national security and to the needs and ambitions of the commercial sector, with new capabilities emerging in the 2025 to 2030 timeframe. DARPA is eager to receive proposals from entities that can help to achieve this goal.

Table of Contents

Foreword

PART I: OVERVIEW INFORMATION

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description A. Background B. Program Description and Technical Overview C. Program Structure D. Technical Area Details E. Schedule/Milestones F. Deliverables G. Government Furnished Equipment/Property/Information H. Intellectual Property

II. Award Information A. General Award Information B. Fundamental Research

III. Eligibility Information A. Eligible Applicants

1. Federally Funded Research and Development Centers (FFRDCs) and Government Entities

B. Organizational Conflicts of Interest C. Cost Sharing/Matching D. Associate Contractor Agreement Clause E. Other Eligibility Criteria

1. Collaborative Efforts IV. Application and Submission Information

A. Address to Request Application Package B. Content and Form of Application Submission

1. Abstract Format

2. Full Proposal Format

3. Proprietary Information

4. Security Information

a. Unclassified Submissions

b. Classified Submissions

5. Disclosure of Information and Compliance with Safeguarding Covered Defense

Information Controls

6. Human Research Subjects/Animal Use

7. Approved Cost Accounting System Documentation

8. Section 508 of the Rehabilitation Act (29 U.S.C. § 749d)/FAR 39.2

9. Grant Abstract

10. Small Business Subcontracting Plan

11. Intellectual Property

a. For Procurement Contracts

b. For All Non-Procurement Contracts

12. Patents

13. System for Award Management (SAM) and Universal Identifier Requirements

14. Funding Restrictions

C. Submission Information

1. Submission Dates and Times

a. Abstract Due Date

b. Full Proposal Date

c. Frequently Asked Questions (FAQ)

2. Abstract Submission Information

3. Proposal Submission Information

a. For Proposers Requesting Grants or Cooperative Agreements:

b. For Proposers Requesting Contracts or Other Transaction Agreements

c. Classified Submission Information

4. Other Submission Requirements

V. Application Review Information A. Evaluation Criteria

1. Overall Scientific and Technical Merit

2. Potential Contribution and Relevance to the DARPA Mission

3. Cost Realism

B. Review and Selection Process

1. Review Process

2. Handling of Source Selection Information

3. Federal Awardee Performance and Integrity Information (FAPIIS)

VI. Award Administration Information A. Selection Notices

1. Abstracts

2. Proposals

B. Administrative and National Policy Requirements

1. Meeting and Travel Requirements

2. FAR and DFARS Clauses

3. Controlled Unclassified Information (CUI) on Non-DoD Information Systems

4. Representations and Certifications

5. Terms and Conditions

C. Reporting D. Electronic Systems

1. Wide Area Work Flow (WAWF)

2. i-Edison

VII. Agency Contacts VIII. Other Information

A. Proposers Day B. Protesting

ATTACHMENT 1: Cost Volume Proposer Checklist ATTACHMENT 2: Proposal Templates for Summary, Schedule, and Budget Slide

PART I: OVERVIEW INFORMATION

Federal Agency Name: Defense Advanced Research Projects Agency (DARPA), Microsystems Technology Office (MTO)

Funding Opportunity Title: Photonics in the Package for Extreme Scalability (PIPES) Announcement Type: Initial Announcement Funding Opportunity Number: HR001119S0004 Catalog of Federal Domestic Assistance Numbers (CFDA): 12.910 Research and

Technology Development Dates: (All times listed herein are Eastern Time) o Posting Date: October 31, 2018 o Proposers Day: November 1, 2018 o Abstract Due Date: November 27, 2018 at 1:00 PM o FAQ Submission Deadline: January 3, 2019 at 1:00 PM o Proposal Due Date: January 17, 2019 at 1:00 PM o Estimated period of performance start: July 2019

Concise description of the funding opportunity: The DARPA Microsystems Technology Office is soliciting research proposals for the development of package-level optical signaling technologies for advanced microelectronic systems to enable disruptive performance scaling through parallelism.

Anticipated Funding Available for Award: Approximately $65M of funding is anticipated for awards made against this BAA, with a distribution of:

o $35M for Technical Area 1 (TA1) including Technical Area 1B (TA1B) o $20M for Technical Area 2 (TA2) o $10M for Technical Area 3 (TA3)

Anticipated individual awards:

o Multiple awards are anticipated in each technical area.

o DARPA anticipates individual awards in TA1B not to exceed $600,000.

Anticipated funding type:

o 6.3 for TA1 and TA1B o 6.2 for TA2 and TA3

Types of instruments that may be awarded:

o TA1 and TA1B may award procurement contract or other transaction only.

o TA2 and TA3 may award procurement contract, grant, cooperative agreement or other transaction.

Agency contact:

o Dr. Gordon Keeler, Program Manager BAA Coordinator: HR001119S0004@darpa.mil

DARPA/MTO

ATTN: HR001119S0004

675 North Randolph Street Arlington, VA 22203-2114 mailto:name@darpa.mil

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

The Defense Advanced Research Projects Agency (DARPA) often selects its research efforts through the Broad Agency Announcement (BAA) process. This BAA is being issued, and any resultant selection will be made, using the procedures under Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016 and 2 C.F.R. § 200.203. Any negotiations and/or awards will use procedures under FAR 15.4, Contract Pricing. Proposals received as a result of this BAA shall be evaluated in accordance with evaluation criteria specified herein through a scientific review process.

DARPA BAAs are posted on the Federal Business Opportunities (FedBizOpps) website, http://www.fbo.gov, and, as applicable, the Grants.gov website at http://www.grants.gov. The following information is for those wishing to respond to the BAA.

The Microsystems Technology Office at DARPA seeks innovative research proposals to develop optical technologies for data movement in digital microelectronics to allow disruptive system scaling through parallelism. Proposed research should investigate approaches that enable revolutionary advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.

A. Background

Since the end of clock frequency scaling in the mid-2000s, the microelectronics industry has progressively embraced parallelism to sustain performance growth. Distributed and parallel architectures are now pervasive across all size scales, from multicore processing units to high-performance computing systems, and span application domains from consumer electronics to DoD systems. Increasingly, however, the benefits of parallelism are constrained not by the limits of computation at individual nodes, but by data movement between nodes. While short-reach connectivity between on-chip cores and within multi-chip modules (MCMs) can be achieved using electrical links with high density, bandwidth, and efficiency, interconnect performance rapidly degrades at the length scales of circuit boards and beyond due to unfavorable scaling with frequency and reach. The energy dissipated by Serializer/Deserializer (SerDes) functions used to move data between integrated circuit (IC) packages represents a large and increasing fraction of overall energy use, restricting off-chip I/O capacity, reducing system performance, and limiting scalability.

Conversely, photonic transceiver modules enable optical signaling with high bandwidth and minimal loss over long distances with optical fiber. However, optical I/O to date has taken the form of pluggable modules on circuit boards, connected to MCM packages with electrical links whose power dissipation and density limit overall performance. Improvements must be achieved through a reduction in signaling energy and latency, while overall signaling capacity and component density must be increased. To fully achieve the potential benefits of optical signaling, photonic solutions must move into the microelectronics package.

http://www.fbo.gov/ http://www.grants.gov/

B. Program Description and Technical Overview

The objective of PIPES is to enable disruptive system scalability by developing optical signaling technologies for digital microelectronics. The program will employ intimate integration of photonics with advanced ICs to yield system connectivity with an unprecedented combination of high aggregate bandwidth, power efficiency, channel density, and link reach.

Through the PIPES program, DARPA seeks to: 1) integrate photonic interconnect capabilities within state-of-the-art MCMs for system prototyping; 2) advance embedded optical signaling performance through the development of emerging component technologies, photonic-electronic integration techniques, and scalable architectures and multiplexing concepts; 3) develop and mature low-loss optical packaging and reconfigurable switching technologies; and, 4) establish a domestic ecosystem that facilitates enduring Department of Defense (DoD) access to differentiated capabilities for in-package photonic signaling.

The development of efficient, high bandwidth package-level photonic signaling is anticipated to impact a host of applications that can be categorized in two key areas. The first is the domain of big data, which includes machine learning, large scale emulation, and high performance computing. Recent trends towards massive parallelism and disaggregation have increased the demand for improved interconnect performance. Optical signaling is already common in such systems at the board and rack levels, but has not yet been directly integrated within the component switch chips, central processing units (CPUs), and graphical processing units (GPUs). Second is the area of advanced sensors and wireless interfaces. The DoD makes extensive use of application-specific integrated circuits (ASICs) and field-programmable gate arrays (FPGAs) that push the boundaries of socket power and I/O capacity in such domains.

While big data applications have numerous commercial volume drivers, the market pull for many DoD-relevant specialty ASICs is insufficient to justify the early adoption of optical technologies.

The PIPES program seeks impact in both key areas by aiding commercial advances through technology investment, and by aligning these commercial outcomes with system integrators to establish a path for national security impact.

In order to achieve these goals, PIPES is soliciting innovative research proposals in three main Technical Areas (TAs):

Technical Area 1 (TA1) – Photonically-Enabled MCMs will develop high-performance optical I/O technology for co-packaging with state-of-the-art packaged ICs, including FPGAs, CPUs, GPUs, and ASICs. Specific objectives of this TA include component development leading to 1 pJ/bit energy consumption for end-to-end links, delivery of multiple MCM prototypes containing optical I/O, and a scalability demonstration of 100 Tbps full-duplex I/O bandwidth from the package.

Successful proposals to TA1 must combine the necessary photonic component development, electronic integration, and robust packaging necessary to fulfill technical program metrics, but should also outline a framework within which I/O technology developed under PIPES could be accessed more broadly within a domestic ecosystem for differentiated DoD access to unique capabilities. Such availability may be achieved by: 1) making optical I/O chiplet products available to system integrators, 2) developing open interface standards, 3) providing intellectual property (IP) blocks accessible in foundry design kits, and 4) offering a domestic optoelectronic packaging capability to enable third parties to add photonic I/O functionality to high-performance microelectronic ICs. Establishing a mechanism for DoD access to PIPES technologies is an essential element of the program and of ERI Phase II programs more generally; contributions to realizing this vision are expected from all Technical Areas.

Technical Area 1, Track B (TA1B) – Defense Applications and Demonstration will be a separate effort within TA1 that investigates the application of photonically-enabled MCMs for DoD-specific use cases. This TA will develop performance models with detailed hardware requirements. Pending the results of these studies and availability of funds, exceptional concepts may be advanced to hardware system demonstrations using TA1 prototypes and/or capabilities.

Successful proposals to TA1B will identify the performance constraints imposed by data I/O in current or future DoD systems, and evaluate the improvement provided by TA1 technology.

Ultimately, the goal of TA1B is to demonstrate the national security benefits of photonic I/O technologies, and collaboration with TA1 performers will be strongly encouraged and facilitated during the program.

Technical Area 2 (TA2) – Photonics for Massive Parallelism will develop revolutionary new approaches to in-package optical I/O scalable to 1 Pbps aggregate bandwidth for future microelectronic systems. This TA will investigate novel component technologies and advanced link concepts that target 0.1 pJ/bit energy consumption in end-to-end bidirectional links.

Elements of TA2 are expected to be less mature than those used in TA1, but proposers should identify how the technology can be transitioned to high-volume manufacturing settings.

Successful proposals to TA2 will combine photonic component development, electronic design and integration, and packaging concepts to fulfill technical program metrics, and will describe a path to maturation.

Technical Area 3 (TA3) – Interconnect Fabrics will develop key technologies to facilitate the use of package-level photonic I/O in future systems and amplify its impact. Two technical thrusts will be pursued, both seeking disruptive, innovative technology solutions to support optical connectivity with hundreds of individually routable high-bandwidth channels. The first thrust will develop low-loss optical packaging approaches to enable high channel densities and port counts, while the second thrust will develop reconfigurable, low-power optical switching technologies.

Preferred MCM Form Factors The program vision for all TAs calls for consistency with IC packages that can be easily accepted and employed by the broader microelectronics industry. Therefore, the desired package should not present a significant departure from current IC packaging practice in terms of size, form factor, electrical interfaces, and thermal management. It is recommended that the size does not exceed 2 inches in any horizontal dimension. As in most IC packages, electrical connections should be constrained to the bottom surface, with solder or pin connectors compatible with printed circuit board solutions. Optical I/O may be interfaced through any of the package surfaces, as long as it does not interfere with standard circuit board layout or thermal management requirements. While the use of optical fibers is anticipated, alternative waveguide technologies may be considered, provided they can meet program goals.

C. Program Structure

PIPES is a 42-month program divided into three Phases, summarized below:

o Phase 1 (base) – 18 months: Demonstration of concepts, components, and function o Phase 2 (option) – 12 months: Integration and prototyping o Phase 3 (option) – 12 months: Establishing scalability, complexity, and maturity

DARPA anticipates funding a variety of technical approaches within the PIPES program. It is expected that fewer performers will be funded to participate in Phases 2 and 3 of the program.

Options may be exercised, at the Government’s sole discretion, based on technical progress measured against the metrics and milestones defined in this BAA.

TA1B represents an exception to the structure above. This effort has two phases (base 12 months, option 12 months), with a goal of transitioning the technologies developed under PIPES through demonstrations relevant to DoD needs. Further discussion of the TA1B timeline, structure, and expectations are detailed below.

D. Technical Area Details

PIPES seeks proposals in three main Technical Areas. Entities may submit to more than one Technical Area, and multiple proposals per organization are permitted. Each proposal must address only a single TA. TA1B also requires a standalone proposal. TA3 proposers can address one or both TA3 thrust areas within a single proposal.

Technical Area 1 (TA1) - Photonically-Enabled MCMs

The goal of TA1 is to develop and mature optical I/O for advanced MCMs, including the technology required to enable 100 Tbps aggregate escape bandwidth per IC package with an energy consumption of 1 pJ/bit. These goals meet anticipated I/O requirements for advanced microelectronics of the 2025 era while constraining the power reserved for communication to 100 Watts, thereby enabling the use of conventional cooling techniques. Key outcomes of this Technical Area include the demonstration of in-package photonic signaling technologies meeting prescribed metrics; prototypical, commercially-relevant photonically-enabled MCMs; and the creation of an ecosystem that makes package-level photonic interconnect technology accessible to the DoD.

The technologies, prototypes, and capabilities developed under TA1 funding have clear commercial applications, and performer cost share is highly encouraged.

The photonically-enabled multi-chip modules envisioned in TA1 will include one or more advanced integrated circuit cores that are anticipated to be fully electrical in nature, and may originate from one of many microelectronics foundries. The PIPES program considers signals to originate and terminate on these IC cores in the electrical domain, and seeks to transport these bits with high efficiency in the optical domain. Inter-package optical signaling technology is anticipated to be enabled through in-package integration of additional ICs, photonic dies, chiplets, and/or interposer-like technologies that will be combined with the IC core.

The development of new IC cores lies outside the scope of this program. Proposers are encouraged to use existing IC cores or technology developed outside of the PIPES program for this effort. Proposers must present a test plan to verify all program metrics, including the proposed means to generate, transmit, detect, and validate data streams originating and terminating within the package. This testing is anticipated to be achieved using performer-provided MCM IC cores and proxy IC cores whose development does fall within the scope of

PIPES.

PIPES TA1 proposals should clearly outline all elements of the proposed solution to enable the link metrics posed below. All in-package electrical circuits required for signaling, coding, multiplexing, clock and data recovery, error correction, etc. should be included in the analysis of the proposed link. Laser source(s) can be internal or external to the package, but all power consumption related to laser operation, including light generation and control, must be accounted for in energy calculations.

Phase 1 – In the initial 18-month Phase 1, the TA1 goal is to demonstrate a packaged I/O prototype with intermediate performance of 10 Tbps aggregate bandwidth consuming 2.5 pJ/bit.

Metrics in Table 1 apply to this “10T technology demonstrator,” which can be satisfied by co-packaging optical I/O with a proxy IC core to generate and receive data streams.

Phase 2 (Option) – In the 12-month Phase 2, the TA1 goal is to demonstrate a packaged I/O prototype with intermediate performance of 10 Tbps bandwidth consuming 1 pJ/bit. Again, the “10T technology demonstrator” can be satisfied by co-packaging optical I/O with a proxy IC core to generate and receive data streams. In addition, the same I/O technology at 1 pJ/bit should be packaged in an MCM with a performer-provided, fully-functional IC core. It is envisioned that this MCM demonstrator will be satisfied by an operational state-of-the-art IC (e.g., FPGA, GPU, CPU, ASIC, etc.) with potential to support a DoD-relevant system demonstration. The processing core(s) and one or more optical modules must be fully packaged in a multi-chip package, along with required electrical support components and optical fibers or other waveguides. The data output through the optical I/O may consist of any dataset approximating the intended application. For any MCM proposed, its function and relevance to a DoD application must be clearly specified. A satisfactory demonstration requires a pair of packaged MCMs communicating in full-duplex mode at the specified data rate over a >20 m optical channel. A pair consisting of an optically enabled ASIC and an optically enabled FPGA/GPU/CPU does satisfy the requirement.

Metrics in the Phase 2 column of Table 1 below apply to both the 10T technology demonstrator and the MCM demonstrator. Delivery of ten packaged and operational MCM units and two packaged 10T technology demonstration units is required at the conclusion of Phase 2. A Government agent will perform an independent test and evaluation of the delivered devices.

Deliverable units will not be returned to the performer. Phase 2 MCMs may be made available to

TA1B performers as government-furnished equipment for use in a TA1B demonstration, should DARPA select to award such an activity.

Phase 3 (Option) – In a final 12-month Phase 3, the TA1 goal is a 100 Tbps prototype demonstration with a proxy core, and elements of the technology made available for third party use within a domestic ecosystem for differentiated DoD access to unique capabilities. Metrics in Table 1 below apply to the technology demonstrator. Any core capable of generating 100 Tbps can be used, and the use of a proxy core in lieu of an actual processing core is permitted.

Delivery of two fully packaged operational units is required at the end of Phase 3. A Government agent will perform an independent test and evaluation of the delivered devices. Deliverable units will not be returned to the performer.

Additional TA1 metrics are detailed in Table 1 below.

Table 1. Technical Area 1 (TA1): Photonically-Enabled MCMs – Program Metrics Phase 1 Phase 2 Phase 3

Key outcomes 10T technology demonstration*

Enhanced 10T demo* AND Packaged MCM demo

100T technology demo* AND Differentiating Access

Aggregate bandwidth 10 Tbps 10T demo: 10 Tbps

MCM: proposer defined 100 Tbps

Energy per bit 2.5 pJ/bit 1 pJ/bit 1 pJ/bit Edge bandwidth density 1 Tbps/mm 2 Tbps/mm 2 Tbps/mm Link latency 200 ns + TOF 100 ns + TOF 100 ns + TOF Link reach (between packages) 20 meters 20 meters 20 meters Bit error ratio (BER) 10-9 10-12 10-12

Link margin 0 dB 2 dB 4 dB

Packaged units delivered -- 2x 10T demo units

10x MCM units 2x 100T demo units

Operating temperature range -- Room temperature to 80°C Room temperature to 80°C Solder reflow compatible -- 250°C, 5 min 250°C, 5 min Integration and optical packaging technique manual assembly manual assembly automated

Total port count proposer defined proposer defined proposer defined

* Technology demonstrators are satisfied by co-packaging optical I/O with a proxy IC core to generate and receive data streams.

TA1 Metrics Definitions

Aggregate bandwidth – The full-duplex data rate of optical signaling. Demonstrations must transmit and receive data at this aggregate rate, but the number of independent channels, wavelengths, symbols, fibers, etc. is proposer defined (see ports).

Energy per bit – The energy required to move electrical bits between IC cores of two packages, originating and terminating with digital signaling at standard clock rates and full swing voltage levels. Includes all in-package signaling energy, all required electrical circuits for modulation, SerDes, receiver amplification, coding, error correction, etc., electro-optical components and control overhead (e.g., thermal or other), and laser power consumption. All energy required to drive, control, and cool the laser must be included, and all optical coupling and insertion losses must be accounted for. The energy per bit metric must be supported by a quantitative link power budget including all of the above elements. The link budget should be consistent with the receiver sensitivity and required margin for the modulation format proposed and BER metric given.

Edge bandwidth density – Bi-directional (duplex) optical bit rate divided by the length of package edge required for optical I/O.

Bit error ratio (BER) – Received bit error rate. Any required coding and error correction must be accounted for in signaling power and latency estimates.

Link latency – All added signaling delays must be analyzed and discussed. The evaluation of latency must include delays from both the electrical interfaces and the optical link components, as well as any overhead introduced by signal processing. Time of flight (TOF) for optical signal propagation should not be included.

Operating temperature range – Package temperature range through which the MCM operates and meets all other Table 1 metrics. If the operating temperature inside the MCM package is expected to exceed 80°C, it must be specified and the co-packaged optical I/O must be tested for reliable operation at that higher temperature. If an external laser module is proposed, the operation temperature of that module is proposer-defined and must be specified and evaluated in the context of a realistic usage scenario.

Total port count – An optical port is defined as an indivisible I/O connection between two physically distinct locations. For example, 10 separate optical fibers on an MCM might be connected to 10 separate locations, and would represent 10 ports. As a counterexample, 10 wavelength-division multiplexed (WDM) optical channels routed through a common single fiber represent a single port. The number of ports and the number of wavelengths per port must be analyzed and discussed in the context of the intended application.

TA1 Track B (TA1B) - Defense Applications and Demonstration

The goal of the Defense Applications and Demonstration track is to apply the technology developed in TA1 to a high-value mission of interest to the DoD. The effort consists of two 12-month phases to develop a concept, and to clearly define a hardware demonstration, respectively.

During these phases, TA1B performers are also expected to participate in defining a domestic ecosystem for differentiated DoD access to unique capabilities through discussions and collaboration with technology developers and providers.

At the end of the second phase, DARPA anticipates funding one or more efforts to demonstrate the impact of PIPES technologies in a compelling defense application. DARPA anticipates receiving proposals for such work from PIPES TA1B performers at that time, and may solicit proposals more broadly through a future BAA. A description of this 18-month “Demonstration Phase” is provided below for reference. TA1B proposers should not include Phase 3 activities in response to the current BAA.

Phase 1 – In the initial 12-month Phase 1, the TA1B goal is to develop an integration concept that leverages PIPES technologies to enable a DoD-relevant system application. It is expected that TA1B DoD system integrators will work closely with one or more TA1 performers to co-develop the integration concept, and will identify additional requirements needed to implement the concept. It is anticipated that Phase 2 options will be exercised only for the most promising concepts.

Phase 2 (Option) – In the optional 12-month Phase 2, the TA1B goal is to develop a full demonstration plan for the proposed defense application. The demonstration is expected to use photonically-enabled MCMs produced and delivered at the conclusion of TA1 – Phase 2, and/or custom hardware enabled by access to PIPES capabilities and technologies. At the end of Phase 2, TA1B performers are required to deliver a white paper detailing the integration and demonstration plan for the concept, which could be proposed as a follow-on activity in the Demonstration Phase.

Phase 3 (For reference only. Not to be proposed) – As a follow-on activity, and dependent on the successful development of technologies described above, DARPA anticipates funding one or more performers to participate in an 18-month Demonstration Phase. Technologies developed under TA1, TA2, and TA3 may be provided as government-furnished equipment (GFE) for the purposes of such demonstrations, particularly the photonically-enabled MCMs delivered under TA1 – Phase 2.

Alternatively, performer access to relevant technologies may be facilitated through the collaborations and the mechanisms for differentiated access to unique capabilities established under PIPES.

DARPA reserves the right to solicit and select proposals from performers other than those initially selected to participate in TA1B.

TA1B proposers are required to address the following proposal elements with sufficient detail:

(1) Describe the DoD application targeted, the interconnect requirement in terms of architecture, bandwidth, power, reach, latency and any other relevant performance metrics. Describe how the problem is addressed today, the limitations of the current approach, and why a solution based on PIPES technology will be beneficial.

(2) Clearly show the expected benefit to the DoD mission by projecting the differential advantage provided to the Warfighter. DARPA is interested in providing revolutionary advances in the respective field or enabling new missions not possible with today’s technology. Specifically excluded are applications that primarily result in evolutionary improvements to the existing state of practice.

(3) Outline the specific technical requirements for the photonically-enabled MCMs and/or optical I/O capabilities accessed through an emerging ecosystem and describe how these will be integrated into the proposed effort. Include a description of the required interfaces, operating conditions (e.g., temperature range, acceleration range, radiation environment, etc.), and physical footprint/form-factor requirements. Refinement of the requirements for a successful demonstration will continue through Phase 1 and must be finalized prior to Phase 2. Collaboration with a TA1 proposer at the proposal stage is encouraged but not required. If the proposed concept is selected for an award, it is expected that a TA1 partner will be identified and an Associate Contractor Agreement (ACA) between the two performers will be signed no later than the conclusion of Phase 1.

Technical Area 2 (TA2) - Photonics for Massive Parallelism

The goal of TA2 is to develop revolutionary optical I/O technologies capable of 1 Pbps bandwidth from a microelectronics package at 0.1 pJ/bit. While challenging today, this bandwidth requirement is consistent with projected microelectronic I/O needs in the 2030 era based on historical scaling trends. This technical area will focus on component innovation, intimate electronic-photonic integration, optical packaging, and novel signaling approaches that enable disruptive improvements to bandwidth density, energy consumption, and complexity scaling. A key driver of this effort is to enable system scalability through distributed and massively parallel architectures, yet substantial impact may be realized by achieving only midterm metrics.

While a full hardware demonstration of 1 Pbps I/O is not required within the program, TA2 efforts are expected to develop fully-functional I/O links with sufficient performance and demonstrated scalability such that the ability to reach that goal is clearly established. In a similar spirit to TA1, the energy consumption metric of 0.1 pJ/bit is required to constrain the future 1 Pbps solution to a communication power envelope of 100 Watts, thereby enabling the use of conventional cooling techniques. Note that the TA2 areal bandwidth density metric of 5 Tbps/mm2 corresponds to 100 Tbps I/O occupying an area of less than 5 mm x 5 mm. These efficiency and density targets lie far beyond the state-of-practice for optical interconnect, and success will likely entail the incorporation of novel components and concepts.

While DARPA remains agnostic to the specific technologies proposed as solutions, it is envisioned that some of the following innovations may appear in TA2 links: dense wavelength-division multiplexing (DWDM), comb laser sources, efficient directly-modulated lasers integrated on-chip, coherent signaling, advanced modulation formats, compact/high-efficiency modulators and detectors using plasmonics, nanophotonics, or low-dimensional materials, receiverless photodetection, and the elimination of other circuitry for coding, error correction, clock and data recovery (CDR), SerDes, etc. Further, intimate photonic-electronic integration will be essential to eliminate most in-package electrical signaling power, and integration methods beyond conventional 2.5D approaches may be required.

As described previously, signals are considered to originate and terminate on advanced integrated circuits in the electrical domain. While the extensive development of application-specific ICs lies outside the scope of PIPES, proposers must present a test plan to verify all program metrics. This will require a means to generate, transmit, detect, and validate large data streams originating and terminating within the package. This testing may be performed using proxy IC cores, whose development does fall within the scope of PIPES. Note that TA2 efforts must develop full-duplex bidirectional links and proposals to develop standalone individual components or partial solutions will not be considered for funding.

PIPES TA2 proposals should clearly outline all elements of the proposed solution to enable the link metrics posed below. All in-package electrical circuits required for signaling, coding, multiplexing, clock and data recovery, error correction, etc. should be included in the analysis of the proposed link. Optical source(s) can be internal or external to the package, but all power consumption related to their operation, including light generation and control, must be accounted for in energy calculations.

Phase 1 – In the initial 18-month Phase 1, the TA2 goal is to develop and demonstrate benchtop prototype link concepts consistent with the goals of 5 Tbps/mm2 areal bandwidth density and 0.5 pJ/bit energy consumption. This Phase will be used to demonstrate feasibility of the proposed approach and to validate component performance. Measurements should demonstrate performance aligned with integrated link targets, although photonic-electronic integration processes may require further development before all required integrated circuits are incorporated in Phase 2.

Phase 2 (Option) – In the 12-month Phase 2, the TA2 goal is to demonstrate integrated prototype links with 10 Tbps aggregate bandwidth consuming only 0.2 pJ/bit. Links prototyped and measured should demonstrate performance consistent with a scaled Phase 3 platform to meet the final program goals.

Delivery of two operational demo units satisfying a complete link (i.e., 1 or more ports) is required at the conclusion of Phase 2. A Government agent will perform an independent test and evaluation of the delivered hardware. Deliverable units will not be returned to the performer.

Phase 3 (Option) – In the 12-month Phase 3, the TA2 goal is to demonstrate a fully integrated and packaged system with a 100 Tbps aggregate bandwidth consuming only 0.1 pJ/bit, delivered through 10 or more independent I/O ports, and in a form factor that is scalable to 1 Pbps in the dimensions of typical MCM packages.

Delivery of two operational demo units satisfying a complete link (i.e., 10 or more ports) is required at the conclusion of Phase 3. Deliverable units will not be returned to the performer.

Additional TA2 metrics are detailed in Table 2 below.

Table 2. Technical Area 2 (TA2): Photonics for Massive Parallelism – Program Metrics Phase 1 Phase 2 Phase 3

Key outcomes

Benchtop component demonstration performance traceable to link prototype

Integrated link demonstration performance traceable to scaled system

Scaled multi-port I/O demonstration performance consistent with 1 Pbps capability

Energy per bit 0.5 pJ/bit 0.2 pJ/bit 0.1 pJ/bit Areal bandwidth density 5 Tbps/mm2 5 Tbps/mm2 5 Tbps/mm2

Aggregate bandwidth proposer defined 10 Tbps 100 Tbps Total port count -- ≥ 1 ≥ 10

Link latency -- 100 ns + TOF 50 ns + TOF Link reach (between packages) 1 meters 10 meters 100 meters

Bit error ratio (BER) 10-9 10-12 10-12

Hardware delivered -- 2 demo units 2 demo units

Operating temperature range -- Room temperature to

80°C Room temperature to

80°C

TA2 Metrics Definitions

Energy per bit – The energy required to move electrical bits between IC cores of two packages, assuming digital signaling at standard clock rates and full swing voltage levels.

Includes all in-package signaling energy, all required electrical circuits (modulation, SerDes, receiver amplification, coding, error correction, etc.), electro-optical signal manipulation (thermal or other), and laser/source power consumption. All energy required to drive, control and, cool the laser/source must be included, and all optical coupling and insertion losses must be accounted for. The energy per bit metric must be supported by a quantitative link power budget including all of the above elements.

Areal bandwidth density – Bi-directional (duplex) optical bit rate divided by the area required to implement optical I/O.

Aggregate bandwidth – The full-duplex data rate of optical signaling. Demonstrations must transmit and receive data at this aggregate rate, but the number of independent channels, wavelengths, symbols, fibers, etc. is not prescribed (see total port count).

Total port count – An optical port is defined as an indivisible I/O connection between two physically distinct locations. For example, 10 separate optical fibers on an MCM might be connected to 10 separate locations, and would represent 10 ports. As a counterexample, 10 wavelength-division multiplexed (WDM) optical channels routed through a common single fiber represent a single port.

Link latency – All added signaling delays must be analyzed and discussed. The evaluation of latency must include delays from both the electrical interfaces and the optical link components, as well as any overhead introduced by signal processing. Time of flight (TOF) for signal propagation in fiber should not be included.

Bit error ratio (BER) – Received bit error rate. Any required coding and error correction must be accounted for in signaling power and latency estimates.

Operating temperature range – Package temperature range through which the MCM operates and meets all other Table 1 metrics. If the operating temperature inside the MCM package is expected to exceed 80°C, it must be specified and the co-packaged optical I/O must be tested for reliable operation at that higher temperature. If an external laser module is proposed, the operation temperature of that module is proposer-defined and must be specified and evaluated in the context of a realistic usage scenario.

Technical Area 3 (TA3) - Interconnect Fabrics

The successful development of package-level photonic I/O in TA1 and TA2 will impose new and unprecedented challenges to system architectures employing these technologies. Many applications favor all-to-all network connectivity or dynamic reconfiguration of resources and functions. For massively interconnected networks with distributed parallelism, potentially consisting of hundreds to thousands of nodes with PIPES I/O, management of ports becomes critically important. The goal of TA3 is to develop foundational technologies that will facilitate the use of package-level photonic I/O in such systems. TA3 considers two technical thrusts, each seeking disruptive, innovative solutions to the challenges imposed by successful developments in

TA1 and TA2. These include high-density, high-efficiency optical packaging and signal routing, and the development and maturation of high port count, reconfigurable optical switching technology.

The Packaging Thrust Area looks to further the technologies used for optoelectronic packaging, fiber-to-chip coupling, optical connectorization, and/or system-level signal breakout and routing.

Concepts proposed in this thrust should target the development and demonstration of methods for low-loss fiber attach and high-port-count optical packaging compatible with preferred MCM form factors (as described in Section B. Program Description and Technical Overview) and moderate-scale, automated volume manufacturing. While optical packaging solutions of varying levels of maturity will also be developed within TA1 and TA2 efforts, TA3 seeks high-risk, high-payoff approaches to enable future scaling beyond what will be developed elsewhere within the program. Additionally, in combination with the Reconfigurable Switching Thrust Area described below, the Packaging Thrust will provide the necessary elements of high-bandwidth, high-efficiency port management modules for massively interconnected systems.

While DARPA remains agnostic to the specific technologies proposed as solutions in this Thrust, it is envisioned that approaches related to some of the following innovations may appear in proposals: additive manufacturing of waveguides or micro-optics, polymer waveguides, nanophotonic couplers, high-density optical fanout, multi-core fiber solutions, and package-level connectorization for pigtail-free modules.

Beyond technology innovation, another envisioned outcome of the Packaging Thrust Area is the creation of a domestic photonic-electronic packaging capability consistent with one element of the model, described earlier, for a domestic ecosystem for differentiated access to unique capabilities. The DoD’s ability to leverage state-of-the-art technologies for critical systems is essential, and proposers in this area are encouraged to describe how their capabilities could further this goal. In particular, DARPA seeks a path to perform packaging of advanced ASICs together with the photonic I/O solutions created under PIPES, in a controlled, domestic setting.

Required capabilities include fabrication of electronic interposers, 2.5D assembly, and optical fiber attach consistent with the metrics described above.

The Reconfigurable Switching Thrust Area seeks to develop and mature low power optical switching technologies in chip-scale formats. As optical I/O becomes more pervasive in microelectronic systems, the ability to perform switching and bandwidth steering in the optical domain offers clear advantages to electrical network switching. This thrust looks to enable high port count reconfigurable NxN optical switches (i.e., N > 1,000) with low insertion loss, low power consumption, fast reconfigurability, and wide spectral bandwidth. Solutions that are transparent to signaling rates and formats are preferred. Hardware control through electrical drive signals is anticipated.

While DARPA remains agnostic to the specific approaches proposed as solutions in this Thrust, it is envisioned that some of the following technologies may appear in proposals: micro-electro-mechanical systems (MEMS), optical phase change materials, thermo-optic switching, and electro-optic switches and materials.

Although limited quantitative metrics are provided below as targets for Technical Area 3, additional performance targets may be proposed specific to the technologies under development consistent with the goals outlined above. Clear, quantifiable milestones should be outlined for each Phase to establish technology progress and demonstrate a methodical reduction of risk towards the goals of the effort. Proposers to TA3 are anticipated to target the goals of a single technical thrust with their approach, but may propose to both thrusts.

Phase 1 – In the initial 18-month Phase 1, the TA3 goal is to demonstrate the feasibility of the technical approach through modeling, simulation, and benchtop experimentation of components and concepts.

Phase 2 (Option) – In the 12-month Phase 2, the TA3 goal is to demonstrate, through an integrated prototype, the scalability of the technology towards the goal of achieving high port count functionality with low optical losses and consistent with the final program goals.

Phase 3 (Option) – In the final 12-month Phase 3, the TA3 goal is to demonstrate fully-integrated devices and capabilities to meet the program goals of low loss, high port count optical signal connectivity, routing, and switching.

Delivery of two operational demo units is required at the conclusion of Phase 3. Deliverable units will not be returned to the performer.

Additional metrics are detailed in Table 3 below.

Table 3. Technical Area 3 (TA3): Interconnect Fabrics –…