HR001121S0004-Amendment-04.pdf

DARPA TTO Broad Agency Announcement Robotic Autonomy in Complex Environments with Resiliency (RACER)

HR001121S0004

Questions and Answers 3 December 18, 2020

IN THE EVENT OF ANY INCONSISTENCY BETWEEN THE ANSWERS PROVIDED

HEREIN AND THE SOLICITATION, THE SOLICITATION SHALL TAKE PRECEDENCE

Questions and Answers 3 by Topic Area:

General Q118 Are the page limits in the BAA (in brackets) for each section a recommendation or a strict requirement?

A118 Page limits in the BAA (in brackets) are recommendations.

Q119 Section IV.B.3. Additional Proposal Information (page 30) requests under section b)(1) that proposers include program security requirements unique to this program. Please confirm this information is outside the page limit for Volume 1.

A119 Yes, the information under Section IV.B.3 - Additional Proposal Information is outside the page limit for Volume 1. Note that Section IV.B.2.a.III.G. - Security Management Architecture and/or Approach is included in the page limit for Volume 1.

Q120 Section IV.B.3 Additional Proposal Information (page 34) requests under section (f) that a Small Business Subcontracting Plan be submitted. Please confirm that the SBSP should be submitted as part of the Cost Proposal, and is not subject to page limits.

A120 Yes, the SBSP is part of the cost volume and is not subject to page limits.

Q121 What is the expected TRL for the final deliverable software at Phase 1 and Phase 2?

A121 No TRL level is expected for the final deliverable software.

Q122 For planning and purposes, what location(s) should we assume for the KO and TIM listed as DARPA milestones/meetings?

A122 We anticipate that kick off will be at DARPA in Arlington, VA and that TIMs will be held at performer sites.

Q123 Given the proposal due date has moved to the right by 1 month, should we assume the period of performance to shift by 1 month to May 2021?

A123 Yes, this is a valid assumption for the program start date. Performers may base their cost estimates on a May 2021 start date or continue to base their costs estimates on an April 2021 start date and note this in their cost proposal assumptions.

Q124 Which is a more important metric, speed over terrain or Interventions/km?

A124 There is no weighting of metrics. Achieving all program metrics is equally important.

Q125 Is there a possibility of a proposal deadline extension?

A125 Please refer to RACER BAA Amendments available at beta.sam.gov for the current proposal due date.

Safety Q126 Will DARPA specify additional safety requirements for performer sites (over what was listed in the BAA) upon award?

A126 Safety features implemented will be outlined at kickoff.

Q127 What are the safety systems on the GFX vehicle to engage or disengage the vehicle autonomy, both software and physical systems?

A127 It is anticipated the safety systems will be E-Stop- centric. Safety features implemented will be outlined at kickoff.

Q128 Will the vehicle include the ability for remote teleop for repositioning the vehicle during restarts, or will it only contain e-stop?

A128 Yes, the vehicle is anticipated to have the ability for remote teleop to augment “breakdowns” in autonomy by a performer during runs. A teleop takeover, a field takeover, or an e-stop will generally be considered an intervention.

Security Q129 For GFX Software and Dataset access, can you provide additional guidance on what access foreign nationals can have to these tools? Will they have full access, limited access to portions (can you list which would have limited), or no access?

A129 Please refer to the Security Classification guide for program security requirements.

The complete, assembled autonomy stack software code that is developed under the RACER program and data collected from the complete, assembled autonomy stack software code that is developed under the RACER program will be Controlled Unclassified Information (CUI) and restricted to US-persons.

Q130 Could we please get clarification on whether or not foreign nationals will be allowed to participate in the DARPA-hosted experiments?

A130 Foreign nationals will be allowed to participate in the DARPA-hosted field experiments provided they are able to obtain permission to enter the military installations. DARPA cannot guarantee access will be granted for any person. Please refer to the Security Classification guide for program security requirements.

Q131 Could we please get clarification on what, if any, aspects of the GFX materials will have foreign national restrictions?

A131 Please refer to the Security Classification guide for program security requirements.

Q132 Do the FOUO/US only restrictions on the full software stack code developed under RACER apply to uncompiled source code only, or are executable binaries also covered?

A132 Yes, the executable binaries that represent the complete, assembled autonomy stack software code that is developed under the RACER program and data collected from the complete, assembled autonomy stack software code that is developed under the

RACER program will be Controlled Unclassified Information (CUI) and restricted to US-persons.

GFX - Comms Q133 A number of external test sites we are looking at for additional testing require information about vehicle RF emissions (comms, active sensors, etc.). What frequencies will be used onboard the vehicle?

A133 The GFX vehicle configuration has not yet been determined. It is anticipated that vehicle to control station comms will be via standard Wi-Fi frequency bands, potentially with modest amplification. Please refer to RACER BAA Section I.F. for a basic description of network comms.

Q134 The BAA lists radios as part of the evaluation infrastructure: "A multi-megabit per second wireless network for RACER connectivity across the entire test area, anticipated to be a commercially available field mesh network". Will these radios be part of the GFX vehicles that are delivered to performer sites? If so, will DARPA provide another endpoint to performers for them to use on their own base stations?

If not, can we get details on the radios so we can scope out appropriate equipment to interface with vehicle radios? Can you please provide details on bandwidth, frequency, range, etc.

A134 The GFX vehicle configuration has not yet been determined. It is anticipated these radios will be part of the GFX vehicles. DARPA does not anticipate providing another endpoint. It is anticipated that vehicle to control station comms will be via standard Wi-Fi bands, potentially with modest amplification. Please refer to RACER BAA Section I.F. for a basic description of network comms.

GFX - Configuration - Changes Q135 Can we move sensors on the vehicle – repoint them or actually move to a different mounting location?

A135 No, performers are not permitted to move or repoint the sensors. DARPA will continue to welcome suggestions for alteration of the GFX. Per the RACER BAA Section VIII.C, "Performers are permitted to request modifications or enhancements to the GFX. Should DARPA approve a request, the modifications or enhancements will be applied to the GFX of all performers."

Q136 Can we add our own filters to cameras (e.g. for polarization)?

A136 No, performers are not permitted to add their own filters to the cameras. Thanks for suggesting polarization filters. DARPA will continue to welcome suggestions for alteration of the GFX. Per the RACER BAA Section VIII.C, "Performers are permitted to request modifications or enhancements to the GFX. Should DARPA approve a request, the modifications or enhancements will be applied to the GFX of all performers."

Q137 Can we augment the LTATV’s sensor suite with H3 Luminar LIDARs? (a) We would like to request the provision of 3 Luminar H3 LIDARs. (b) Steerable foveation of high-quality LIDAR allows attention mechanisms to increase 3D resolution at medium to long ranges. This also would allow us to generate more effective point cloud training data for online neural network domain adaptation.

A137 No, performers may not augment the LTATVs sensor suite. Thanks for the specific LIDAR suggestion. DARPA will continue to welcome suggestions for alteration of the GFX. Per the RACER BAA Section VIII.C, "Performers are permitted to request modifications or enhancements to the GFX. Should DARPA approve a request, the modifications or enhancements will be applied to the GFX of all performers."

GFX - Configuration - Controls Q138 Does the vehicle controller command use ROS geometry_msgs/Twist? If so, can we use a different message type that incorporates commanded accelerations/decelerations? This is important for stopping suddenly.

A138 No, the vehicle controller does not anticipate use of ROS geometry_msgs/Twist. The vehicle controller anticipates straightforward accelerator, brake, and steering inputs.

However, the GFX vehicle configuration has not yet been determined.

Q139 Will we have direct control / access to actuator position and actuator feedback (current steering, throttle, brake position)? Do we provide an open-loop set-point, or is it closed loop control on the actuators?

A139 Yes, the vehicle controller anticipates straightforward accelerator, brake, and steering inputs with feedback using an open loop setpoint. However, the GFX vehicle configuration has not yet been determined.

Q140 Will the vehicle come with a stability controller?

A140 Yes, it is anticipated the vehicle will come with a stability controller. It is also anticipated that performers will be able to modify the stability controller to better suit their needs in close coordination with DARPA for consistency and safety.

However, the GFX vehicle configuration has not yet been determined.

Q141 Can all wheels be steered with single command or each wheel can be steered individually?

A141 It is anticipated the vehicle will be steered via a single command. However, the GFX vehicle configuration has not yet been determined.

GFX - Configuration - Information Q142 The BAA states "The planned compute available on the demonstration platforms is

GPUs in the quantity and class of 4-6 Nuvo 7160GCs using <1000W of power consumption." The Nuvo 7160GC is a chassis that supports up to a single 120W GPU, but there is no default GPU or CPU as far as we can tell. Can we get more clarity on onboard GPUs? Would it be possible to add RTX 3080 class GPUs to the DARPA spec, even though they wouldn’t fit in the Nuvo cases?

A142 No additional clarity on the GPUs is available at this time. The GFX vehicle configuration has not yet been determined. Thank you for the suggestion regarding adding GPUs, we will consider it.

Q143 Section VIII.C.1 (page 48) Can you please share a maintenance schedule, manuals, or common spare parts lists for the Polaris MRZR-X vehicle for Phase I? This will aid in bidding the GFX support costs and planning.

A143 DARPA cannot provide a maintenance schedule, manuals or common spare parts lists for the Polaris MRZR-X vehicle. The GFX vehicle configuration has not yet been determined.

Q144 Can DARPA provide the vehicle physical parameters for use in a vehicle model & simulation?

A144 DARPA cannot provide this information at this time. The GFX vehicle configuration has not yet been determined.

Q145 Are you able to provide more information regarding the GPU processing stack (i.e.

quantity) and network architecture that is available?

A145 No additional information is available regarding the GPU processing stack at this time.

Please refer to RACER BAA Section VIII.C.3 for information on the GFX. The GFX vehicle configuration has not yet been determined.

Q146 Will the GPUs be running Linux and ROS? If so, what versions?

A146 We do not know if the GPUs will be running Linux and ROS. The anticipated OS has not been finalized. However the GFX vehicles are anticipated to be provided with an OS and ROS installed. The GFX vehicle configuration has not yet been determined.

Q147 What is the fuel capacity and tank location?

A147 This information is not currently available. The GFX vehicle configuration has not yet been determined.

Q148 What LIDAR make and model will be provided? Can you provide a complete list of GFX provided sensors and compute Hardware including vendor and part numbers? Is there a vehicle electronics architecture specification available?

A148 We cannot provide this information at this time. The GFX vehicle configuration has not yet been determined.

Q149 Assuming the GFX software has been tested in prior programs, will reports on performance, etc. be available to performers on RACER? If so, when can industry expect to receive those reports?

A149 No reports on performance, etc. are available at this time. GFX code configuration has not yet been determined. Overviews of related but not fully inclusive development efforts are available within the document “20201009_RACER_Proposers_Day_.pdf” posted in the RACER contract opportunity on beta.sam.gov

GFX - Configuration - Sensors Q150 What is the output data of the forward-looking radar sensor? Does DARPA expect performers to develop object detection and tracking algorithms from radar returns?

(BAA pg. 50)

A150 Output data from radar sensors is anticipated to be raw data. The radars are anticipated to be automotive grade. However, the GFX vehicle configuration has not yet been determined.

https://beta.sam.gov/api/prod/opps/v3/opportunities/resources/files/e7b91a91ea984744bcbdcbd17430a9a1/download?api_key=null&token=

Q151 Are all the computers and sensors going to be pre-installed on the vehicle? If not, please share the expected installation dates for the "enhanced datasets".

A151 Yes, it is anticipated that the computers and sensors will be pre-installed on the vehicle. The enhanced datasets are anticipated to be available at the start of Phase 1.

However, the GFX vehicle configuration has not yet been determined.

Q152 Are the GFX sensors for ADP (Phase 2) expected to stay the same as Phase 1 GFX sensors?

A152 Yes, while also anticipating that scaling considerations may drive evolutions of GFX sensors for Phase 2.

Q153 Will wheel encoders be included on the LATV? If so, will they be on all the wheels?

A153 Yes, wheel encoders are anticipated to be included. Please refer to RACER BAA

Section VIII.C.3, page 50, for information on the anticipated GFX. However, the GFX vehicle configuration has not yet been determined.

Q154 Will there be suspension travel sensors on the LATV?

A154 Yes, suspension travel sensors are anticipated to be included on all wheels. Please refer to RACER BAA Section VIII.C.3, page 50, for information on the anticipated GFX.

However, the GFX vehicle configuration has not yet been determined.

Q155 Can we have an active front bumper?

A155 Active bumpers are not anticipated. Thank you for the suggestion.

Q156 What is the type of data coming out of the radars – discrete radar tracks or raw radar data?

A156 It is anticipated that raw data will come out of the radars. The radars are anticipated to be automotive grade. However, the GFX vehicle configuration has not yet been determined.

Q157 Will there be any close range sensing around the perimeter of the vehicle?

A157 Please refer to RACER BAA Section VIII.C for details of the GFX for anticipated positioning of the sensors. Stereo cameras and LIDAR are anticipated to provide approximately 360 degree coverage.

Q158 Can a LWIR camera be added to the LTATV baseline sensor package?

A158 A LWIR camera is anticipated to be part of the GFX sensors. Please refer to RACER

BAA Section VIII.C.3 for information on the GFX, page 49.

Q159 What direction will the LIDAR's be facing?

A159 The LIDAR locations, pointing directions, and fields of view have not yet been determined. It is anticipated the LIDAR will provide approximately 360 degree coverage. See RACER BAA Section VIII.C.3 for information on the anticipated GFX.

Q160 Will the sensors have any shielding to keep rain off or any guards or rubberized mounts to help mitigate the effect of minor environmental collisions?

A160 The sensors are anticipated to be appropriately shielded from environmental conditions and appropriately mounted to handle minor environmental collisions.

GFX - Maintenance Q161 What spares (vehicle parts, sensors, computers) will be provided to the performer?

Will the spares be provided with the vehicles or would we need to request spares as needed?

A161 No spares (vehicle parts, sensors, computers) will be provided to the performer. No spares will be provided with the vehicles. The performers will be responsible for purchasing spares from commercial sources. Per the BAA, DARPA anticipates having some spares available to alleviate long lead times. However, availability of spares from DARPA is not guaranteed.

Q162 Who is responsible for maintenance on the LTATVs as they are GFX? Does the Government plan to have SMEs available to assist or should performers be prepared to be fully responsible?

A162 Performers are responsible for maintenance on the LTATVs. Government SMEs are anticipated to be available for advice and consultation.

Q163 The BAA says that "To help mitigate this risk [regarding spare parts], DARPA intends to have a limited inventory of long lead spare parts. The cost of these parts will be tracked." Does this mean that teams will be responsible for replacement part costs?

Are teams also responsible for replacing broken sensors and the like? If so, does DARPA have a preferred way for teams to estimate this cost for proposal purposes, e.g. a flat $100k for replacement parts?

A163 Yes, teams will be responsible for replacement part costs. Yes, teams will also be responsible for replacing broken sensors and the like. No, DARPA does not have a preferred way for teams to estimate this cost for the proposal. A flat amount for replacement would be an acceptable approach.

GFX - Stack Q164 Is a specific OS required? The BAA said ROS is used but that is middleware and not an operating system. Is Ubuntu XX.XX expected?

A164 A specific OS is not required. We do not know the anticipated OS at this time. The GFX vehicle configuration has not yet been determined.

Q165 What is the fidelity of the Unity based simulation? Does it include tire traction and suspension dynamics? Does it include dynamics in various types of terrains?

A165 The Unity based simulation is anticipated to be low fidelity, such that would be appropriate for regression. No, it is not anticipated to include tire traction, suspension dynamics, nor dynamics in various types of terrains.

Q166 What is the fidelity of the sensor emulation in simulation? Does the resolution and update rate match the specifications of the GFX sensors? Does it include radar sensors or are performers expected to add radar models?

A166 The Unity based simulation is anticipated to be low fidelity, such that would be appropriate for regression. No, it is not anticipated to match the resolution and update rate of the GFX sensors. No, it is not anticipated to include radar models.

Q167 Are the GFX computers setup for encryption and authentication for DFARS 252.204- 7012 compliance? Are the performers required to configure them for compliance?

A167 No, it is not anticipated that the GFX computer will be set up for encryption and authentication for DFARS 252.204-7012 compliance. No, it is not anticipated that performers will be required to configure them for compliance.

Q168 Will existing tools for behavior tree mission planning, perception visualization, mission monitoring, or driver interfaces be provided as part of the GFX?

A168 Standard tools are anticipated to be provided as part of the baseline GFX to the extents currently outlined in the RACER BAA Section VIII.C.3 for information on the GFX. However, the GFX vehicle configuration has not yet been determined.

Q169 Will additional computers for the system integration lab be provided as part of the additional Government Furnished Equipment?

A169 No, it is not anticipated that SIL equipment will be provided as part of the GFX.

Q170 What data collection capability will be delivered with the GFX platform?

A170 It is anticipated that a data collection capability will be delivered with the GFX platforms. However, the GFX vehicle configuration has not yet been determined.

Q171 How similar is the GFX drive data to the RACER field experiment environments?

A171 DARPA anticipates the GFX drive data will be gathered from terrain similar to one or more DARPA-hosted field experiments.

Q172 Will DARPA provide a dynamic vehicle model? If so when will this be made available?

A172 DARPA anticipates that a vehicle model, not necessarily a high resolution dynamic or full-physics based, will be provided with the GFX at or around kickoff.

Q173 Are simulation tools included with the GFX package?

A173 Yes, it is anticipated that some simulation tools will be included in the GFX package.

Please refer to RACER BAA Section VIII.C. for details of the GFX.

GFX - Vehicle Configuration Q174 Will this be a Turbo Diesel engine only, or will it be hybrid electric?

A174 The anticipated engine is a turbo gasoline engine. However, the GFX vehicle configuration has not yet been determined.

Q175 In the BAA, paragraph 3(b) on page 50 calls out the use of CAN AUTOSAR to provide syncing and logging of vehicle sensor data. The AUTOSAR consortium doesn't permit the use of its technology in defense applications. Has DARPA received a waiver from the AUTOSAR consortium? (a) In the same section, it indicates provision for logging of incoming sensor data. What additional logging capability will need to be provided by the performer (i.e. data storage and Ethernet data capture)?

A175 Thank you for pointing out the restriction, we will be sure to obtain all necessary waivers should they be needed. CAN AUTOSAR was mentioned in the BAA as one potentially appropriate technology. It is anticipated that there will be a capability for onboard data logging. Performers will need to provide offboard data storage.

Q176 Is the vehicle all-wheel drive or front wheel drive? Can torque be transmitted to wheels individually or one request will go to all wheels?

A176 It is anticipated the vehicle will be mechanical all-wheel drive. However, the GFX vehicle configuration has not yet been determined.

Maps Q177 In what format and resolution will the GFE topographic data be provided?

A177 The GFE topographic data is anticipated to be provided in geo-registered format.

Resolution, format, and content is anticipated to be comparable to existing, publicly available 1:50,000 maps

Q178 What are the specifications of the provided digitized topographical maps? (a) We would like to request information about pixel-to-meters resolution, total scale, data format, and available data fields. (b) Quality, resolution, and format of topographical maps could affect what algorithms are available or effective.

A178 The GFE topographic data is anticipated to be provided in geo-registered format.

Resolution, format, and content is anticipated to be comparable to existing, publicly available 1:50,000 maps

Q179 Section I.E.1. (page 9) states that “Preloaded 1:50,000 topological map (such as those from the National Geospatial-Intelligence Agency (NGA) or the U. S. Geological Survey (USGS)). No additional GPS or pre-loaded information may be used. Nor shall performers be allowed to rely on data or maps from prior runs. This puts the onus on performers to develop algorithms that rely primarily on vehicle sensing and perception to maximize average autonomous speed.” Will the vendor be allowed to pre-process satellite and terrain data, and then load this processed data onto the vehicle?

A179 DARPA will only provide map data as per I.F of the BAA for use in DARPA-hosted field experiment runs. Pre-processed satellite and terrain data may be used for developmental - training purposes. However, this data may not be used on scored runs at DARPA-hosted field experiments. The spirit of RACER is to rely on interpretation of onboard sensor data rather than map-based learning.

Training Q180 Are performers allowed to add additional sensors for use in training? For instance, near IR sensors whose output can help to label data from other sensors, or GPS to enforce geofence constraints. We would remove these for official tests.

A180 Yes, performers are permitted to add additional sensors during training. The vehicles must be returned to the standard configuration for DARPA-hosted field experiments.

Q181 Is it possible to get high resolution mapping data of the experimental areas for use in the development phase to train simulations?

A181 DARPA will only provide map data as per I.F of the BAA for use in DARPA-hosted field experiment runs. Yes, performers may acquire high resolution mapping data of the experiment areas. This will not be provided as GFX. Yes, performers are permitted to use such data in the development phases to train simulators. However, this data may not be used on scored runs at DARPA-hosted field experiments. The spirit of RACER is to rely on interpretation of onboard sensor data rather than map-based learning.

Field Experiments Q182 Will there be dynamic obstacles during testing?

A182 No, DARPA does not anticipate any dynamic obstacles during DARPA-hosted field experiments.

Q183 Will there be dynamic obstacles included during the test events?

A183 No, DARPA does not anticipate any dynamic obstacles during DARPA-hosted field experiments.

Q184 What are the requirements for interaction with other actors? (e.g. dynamic actors:

vehicle, pedestrian or static actors: Parked vehicles, buildings...etc.)

A184 No dynamic actors are planned during DARPA-hosted field experiments. Fixed obstacles may be present but there are no requirements for interaction in pursuit of achieving the program metrics.

Q185 Can we enter the testing area in order to collect data for training our DNN's before the testing begins?

A185 No, DARPA does not plan to provide access to the testing areas or locations prior to the DARPA-hosted field experiments. It is anticipated that performers will have 10 days of access during the DARPA-hosted field experiments.

Q186 Will the Government provide a means for site survey visits to the Field Experiment Locations? If so, how does the Government anticipate doing this?

A186 No, DARPA does not plan to provide access to the testing areas or locations prior to the DARPA-hosted field experiments. It is anticipated that performers will have 10 days of access during the DARPA-hosted field experiments.

Q187 What interactions with water features will be required?

A187 There are no requirements for interactions with water features. It is anticipated that creeks, rain puddles, and other water sources will be a natural part of the environments and that performers’ approaches should account for their presence.

Q188 Will testing at the DARPA Field Experiments will take place on a DARPA platform?

Will performers have the opportunity to calibrate sensors on the DARPA vehicle in that case?

A188 Testing at the DARPA-hosted field experiments will take place on the performer vehicles, obviating the need for calibration on DARPA vehicles.

Q189 Can we clean sensors mid-run if necessary?

A189 Yes, you may clean the sensors mid-run. Since RACER is not a sensor robustness program, DARPA-determined field pauses during DARPA-hosted field experiments may accommodate sensor cleaning. Determinations of an experiment pause versus an intervention will be judged based on conditions; as such, stack approaches that account for sensor degradation during runs should be considered

Transport Q190 Will the vehicles be provided with a means of transportation (trailer)?

A190 No, a means of transporting vehicles will not be provided.

Q191 What is the Government covering from the perspective of shipping and receiving?

A191 The Government does not intend to provide shipping and receiving services as GFX.

Performers should build shipping and receiving costs into their cost proposals.