25RQ000068_ASR IDIQ Draft SOW .pdf

Advanced Safety Research IDIQ

SECTION C - DESCRIPTION, SPECIFICATION, WORK STATEMENT (Draft)

C.1 BACKGROUND

The National Highway Traffic Safety Administration (NHTSA) is an agency of the U.S.

Department of Transportation (DOT). NHTSA’s mission is to save lives, prevent injuries and reduce traffic-related crashes and other economic costs. The agency develops and implements effective educational, engineering and enforcement programs with the goal of ending preventable motor vehicle tragedies and reducing economic costs associated with crashes.

In 2022, motor vehicle-related crashes on U.S. highways claimed 42,514 lives.1 Analysis of crash causation factors indicate that many crashes involve driver-related factors, such as recognition, decision, performance, or non-performance.2 In addition, risky driving behaviors, such as distraction, impairment, and speeding, are often involved in fatal and serious injury crashes. Advancements in fundamental vehicle control and safety systems, including tires, brakes, steering, and lighting systems can help drivers avoid or mitigate crash situations.

Advanced driver assistance systems (ADAS) offer further opportunities to avoid or mitigate crashes by alerting, assisting and/or providing intervention for the driver in crash-imminent situations. In the future, Automated Driving Systems (ADS), which have the potential to perform the entire dynamic driving task, may further reduce crashes caused by driver error.

ADS-equipped vehicles may also increase mobility accessibility to individuals, including those with disabilities and physical challenges. As electronic control systems, software, and connectivity that support ADAS and ADS technologies advance, cybersecurity related vulnerabilities may also increase. These changes will likely have a commensurate increase in software and hardware complexity as well as an increase in functional safety and cybersecurity countermeasures.

The mission of the National Highway Traffic Safety Administration’s Office of Vehicle Crash Avoidance and Electronic Controls Research (NSR-300) is to formulate and conduct research and development programs that build the scientific basis for advancing the safe deployment of advanced safety technologies, including ADAS and ADS components and systems. NSR-300 also supports vehicle cybersecurity and human factors research related to ADAS and ADS systems. In completing such research, NHTSA and its research partners collaborate with and leverage input from a wide range of stakeholders including, but not limited to, the following:

Original Equipment Manufacturers (OEMs), ADAS and ADS technology providers and suppliers, the National Institute of Standards and Technology (NIST), SAE International, other

1 https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/813560 2 Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey, 2015, NHTSA.

countries, other US DOT modes, and other federal agencies.

The purpose of this solicitation action (hereafter referred to as the Advanced Safety Research or ASR IDIQ) is to establish IDIQ contracts between NHTSA and qualified contractors (or contractor teams) to support safety research in the following four broad domains. These four domains are in direct relationship to the seven (7) research areas under C.3, Requirements (below):

• Advanced Safety Technologies (crash avoidance systems, driver assistance systems, other intelligent technologies, and fundamental vehicle systems such as tires, brakes, steering, and lighting)

• Automated Driving Systems (SAE driving automation levels 3 to 5)

• Cybersecurity

• Functional Safety of vehicle electronic components and subsystems.

With such IDIQ contracts in place, NHTSA can efficiently compete and award Task Orders for specific research projects in the areas listed above. Please refer to section L.10.1 (Special Proposal Instructions Overview) for details on issuing and awarding Task Orders. NHTSA research programs related to advanced safety technologies, ADS, cybersecurity, and functional safety are further described below.

ADVANCED SAFETY TECHNOLOGIES. Advanced Safety Technologies research includes crash avoidance systems, driver assistance systems, partial driving automation systems, other intelligent technologies, and fundamental vehicle systems such as tires, brakes, steering, and lighting. Crash avoidance research focuses on technologies that alert, intervene, and assist drivers in avoiding and mitigating crashes, such as: forward collision warning/automatic emergency braking (FCW/AEB), lane departure warning/lane keeping assist (LDW/LKA), and blind spot warning/intervention (BSW/BSI) systems. This research program addresses technologies targeted to improve the safety of vulnerable road users, such as pedestrians, motorcyclists, and bicyclists. It includes the potential role and impacts of vehicle connectivity, such as V2N and V2X. This program area includes safety research on ADAS technologies that assist drivers on a sustained basis (SAE level 1), such as ACC (adaptive cruise control) and lane centering assist (LCA), and technologies that perform partial driving automation (SAE level 2).

Advanced Safety Technologies research also includes other intelligent technologies, such as intelligence speed assistance (ISA), and advancements in fundamental vehicle systems, such as adaptive headlights, non-pneumatic tires, and heavy vehicle electronically controlled braking systems (ECBS).

A primary objective of this research program is to plan and execute research that can enable the safe deployment of Advanced Safety Technologies across U.S. automotive fleets to support NHTSA’s mission of reducing motor vehicle crashes, fatalities, and injuries. Through objective research, NHTSA seeks to build knowledge on state-of-the-art Advanced Safety Technologies and understand their safety performance in terms of capabilities, limitations, effectiveness, and risks across applicable motor vehicles and roadway users.

The Advanced Safety Technologies program seeks research in a number of areas. The program seeks research that focuses on understanding real-world crash scenarios where Advanced Safety Technologies may have the opportunity to avoid or mitigate their outcomes. The program also seeks research that develops and validates test procedures that could be used to assess the safety performance of Advanced Safety Technologies in vehicles. The program also researches the operational performance (i.e., repeatability and reproducibility), effectiveness, consumer acceptance, and potential unintended consequences of these technologies.

AUTOMATED DRIVING SYSTEMS (ADS) (i.e., SAE International Driving Automation Levels 3, 4, and 5) is a major area of crash avoidance research and an emphasis for NHTSA.

These systems, when engaged, have the potential to perform the full driving task without an expectation of an engaged driver. Vehicles equipped with ADS may deliver substantial safety benefits while also advancing enhanced mobility and improved transportation system efficiency.

The Automated Driving System program seeks research in a number of areas. Research could include working with the industry, academic institutions, other government agencies, and standards setting organizations to explore methods and tools for assessing the safety of ADS-equipped vehicles. Research will focus on building tools and pursuing multiple assessment methods including, but not limited to: simulation, closed course testing, and on-road naturalistic testing to understand real-world crash scenarios. The research may include working with other industry standards organizations in developing a common “language” for describing ADS test scenarios, as well as research to identify and parameterize potential test scenarios for ADS vehicles. Additional research will focus on evaluating the application of leading-edge analytical methods that would utilize operational data (or results) from various testing venues to understand safety performance metrics. This program area also seeks research that relates to on-road ADS testing methods, data collection, data reduction, and performance assessment.

Research will also assist NHTSA to evaluate whether remaining regulatory barriers exist to the deployment of ADS-equipped vehicles. For example, this program will seek research on alternative test procedures for crash avoidance standards that would normally require a test driver (or a robotic apparatus affixed to conventional controls) to complete the maneuver. The objective of this research is to gather data and evidence that could support decisions about potential adaptation and/or translation of regulations or test procedures to address compliance issues for vehicles equipped with ADS.

CYBERSECURITY. The evolution of automotive technology has included the expanded use of electronic systems, software, and wireless connectivity. Enhanced wireless connectivity and continued innovations in electronic control systems introduces substantial benefits to highway transportation safety, mobility, and efficiency. However, with the increasing proliferation of computer-based control systems, software, connectivity, and onboard digital data communication networks, modern vehicles need to consider additional failure modes, vulnerabilities, and threats that could jeopardize benefits if the new safety risks are not appropriately addressed. Safety technologies that can intervene to temporarily assist the drivers with the control of their vehicle (e.g. automatic emergency braking, lane keeping assist, etc.) could also raise the cybersecurity stakes, and without proactive measures taken across the vehicle lifecycle, risks could rise accordingly.

The goal of the Cybersecurity research program area is to support enhanced reliability and resiliency of vehicle electronics, software, and related vehicle control systems to both mitigate safety risks associated with failure and/or cyber compromise of such systems, but also so that such concerns do not pose public acceptance barriers for proven safety technologies and driving automation systems.

Vehicle Cybersecurity research concerns safety risk management associated with potential intentional manipulation of software, hardware, sensors, and associated communication networks on-board the vehicle by malicious actors. NHTSA will perform research to advance the automotive industry’s adoption and implementation of the National Institute of Standards and Technology (NIST) Cybersecurity Framework across their organizations and products.

Additionally, NHTSA will conduct research to advance the adoption and awareness of cybersecurity initiatives that affect vehicles as outlined by other federal agencies and offices, that include but not limited to BIS, CISA (Critical Infrastructure Security and Resilience) and NQCO (National Quantum Coordination Office). This will include targeted research on how the auto industry addresses the full life-cycle of cyber risks including identifying, protecting, detecting, responding, and recovering from cyber threats. The cybersecurity research program should be cross-cutting so that research results will be applicable to both conventional vehicles and those equipped with advanced safety systems and technologies, such as ADS.

While no crashes or fatalities have been directly attributed to a vehicle cybersecurity incident, the potential for large scale cyberattacks on vehicles is well-recognized, and as such, this risk warrants pre-emptive and proactive attention. Further, cybersecurity vulnerabilities may also influence public confidence in our Nation’s transportation system and could create a roadblock for the adoption of proven safety technologies as well as future ADS-equipped vehicles. If not appropriately addressed through design and lifecycle risk management processes, a successful cyberattack on automotive computer systems and their associated networks may not only lead to the loss of information and data but may also adversely impact vehicle control systems such as steering, braking, and throttle, potentially resulting in crashes, and injuries.

FUNCTIONAL SAFETY ANALYSIS. Functional safety is an important aspect of overall safety involving risk management associated with potential electrical and electronic failures in components, systems, and software. NHTSA seeks research that focuses on methods for assessing the functional safety and reliability of safety-relevant subsystems including, but not limited to, the following: ADAS and ADS perception, planning, and control; remote assistance;

fundamental vehicle systems such as steering, braking, and tires; and intelligent and connected vehicle technologies. Failures of certain functional components may occur, and ADAS- and ADS-equipped vehicles may be expected to have sufficient redundancy to sustain system operations for a period of time necessary to support fail-safe or fail-operational strategies to avoid or mitigate crash risks. The research will include application of functional safety methodologies to the electronic controls, sensors, software, processors, and electro-mechanical actuation systems employed by both ADAS and ADS-equipped vehicles. Functional safety research scope will extend to new capabilities ADS developers and other technology providers are introducing such as using wireless communications to facilitate remote vehicle operation, including remote assistance.

In executing functional safety analyses, NHTSA will leverage evolving industry process standards (such as International Organization of Standardization (ISO) 26262 and Safety of the Intended Function (SOTIF)) and failure analysis techniques such as failure mode & effects analyses (FMEA), fault tree analysis (FTA), and other applicable methods. NHTSA will also conduct analyses related to the Safety of the Intended Functionality (SOTIF) using industry standards (such as ISO 21448) for assessing reliability, safety and potential unintended consequences associated with advanced electronic control systems, software, and electro-mechanical systems due to misuse and/or operation of the systems beyond their intended functionality and operating domain. Also, software assurance approaches will be used to explore contemporary methods in automated tools and software development. This includes testing and deployment, such as formal methods, and their potential applicability to automotive applications.

System Safety Analyses Methods: System Theoretic Process Analysis (STPA) – is a hazard analysis technique that aims to identify and prevent accidents. SAE has issued J-3187, STPA Recommended Practices for Evaluations of Safety-Critical Systems in Any Industry (Appendix:

STPA and Safety of the Intended Functionality). This standard maps STPA to SOTIF and may be applied to any Safety-Critical System. NHTSA may employ this technique to assess hazards in ADS and ADAS. Related analyses – STAMP (Systems-Theoretic Accident Model and Processes) and CAST (Causal Analysis based on STAMP) may also be explored.

NHTSA may consider conducting ADS evaluation analysis based on such standards as UL 4600.

UL 4600 is a goal-based and technology-agnostic Standard covering ADS-equipped vehicles and other ADS products operating without human supervision. UL 4600 requires a demonstration of how an ADS is developed with safety built in. NHTSA may also conduct analysis on software maturity models in ADAS and ADS, using such models as ASPICE.

C.2 OBJECTIVE and OVERVIEW

The objective of this Indefinite Delivery, Indefinite Quantity (IDIQ) Contract is to provide NHTSA with the capability to compete, and efficiently award, Task Orders to prequalified contractors for research support in multiple areas related to Vehicle Electronic Systems Safety.

Task Order projects often require a wide range of research skills, and will involve planning, conducting, and documenting research to: support the advancement of motor vehicle safety broadly; to provide NHTSA with information that will be useful for the public and industry stakeholders to advance the design and performance of safety-relevant technologies; and to provide NHTSA with objective data and knowledge that may be useful in supporting Agency decisions.

Task orders issued under this IDIQ will address a broad range of research including: assessments of safety technology performance using closed track, virtual, component, and on-road methods.

Additional research areas include crash avoidance, ADAS, ADS, and other intelligent technologies; functional safety, SOTIF, and other safety impact assessments; vehicle cybersecurity; and artificial intelligence, machine learning, and quantum technologies. Further, task orders may include research related to light and heavy vehicles, pedestrians, motorcycles, bicyclists, and other vulnerable road users.

The tasks and experiments performed under the ASR IDIQ Contract will provide the technical support needed by NHTSA in seven Research Areas (RAs) listed below:

1. Controlled-Environment Evaluation (Test Track Research)

2. Virtual Testing (Simulation & Modeling)

3. Vehicle Component Testing and Functional Safety Assessments

4. On-Road Testing and Naturalistic Studies

5. Safety Impact Assessment Support

6. Cybersecurity

7. Artificial Intelligence (AI) / Machine Learning (ML) Methods, Algorithms, and Software.

In its response to this solicitation, an offeror may submit proposals in one or more of the subject Research Areas (up to all seven) listed above. Offerors shall designate in their technical proposal which research areas they are proposing to qualify for and provide supporting details in separate sections for each Research Area (i.e., Understanding of the Research Area, Technical Capabilities, and Organizational Approach).

NOTE: If an Offeror is interested in qualifying for multiple research areas, the offeror must only submit ONE Technical Proposal that covers all the relevant research areas in that proposal. See section L.9.1 for more details.

C.3 REQUIREMENTS

C.3.1 Kick Off Meeting

Within fourteen (14) days after contract award, key members of the contractor’s staff who will be assigned major responsibility for carrying out the tasks of the contract shall schedule a meeting with the COR and other appropriate NHTSA staff, to:

1. Review any contractual matters, including the terms and conditions of the contract;

2. Review and discuss the information furnished in the Statement of Work to ensure a common understanding of contract requirements. The contractor shall review existing materials developed to better understand NHTSA’s crash avoidance efforts; and

3. Review the types of technical assistance activities that the contractor may be called on to perform.

The meeting may be held in person at NHTSA headquarters in Washington, D.C., or as an online meeting (e.g., Teams, Zoom) as determined by the COR at a mutually agreed date and time.

C.3.2 Task Order Requirements

For each Research Area (listed above) that the Offeror intends to qualify quality for, key personnel, including principal investigators, program managers, research engineers, and technical editors for the contractor and applicable subcontractors must have relevant experience, knowledge, and skills and a thorough understanding of the scientific, engineering, and technical issues related to that area. (Labor Categories and Descriptions are listed in Section B.4). Each Task Order to be issued under the ASR IDIQ will follow the NHTSA Style Guide and will include a project statement of objectives (or statement of work), specific requirements, schedule, milestones, deliverables for that Task Order, and a two-page executive summary. The following sections provide additional clarification of NHTSA’s seven research areas.

Research Area 1 - Controlled-Environment Evaluation (Test Track Research)

Areas of focus and desired expertise include, but are not limited to, the following:

• Test-track research to support vehicle-level testing for light vehicles, heavy vehicles, and other vehicle types (e.g., motorcycles, low speed vehicles).

• Advanced test-track methods to support safety assessment of crash avoidance, ADAS, ADS, other intelligent technologies, and fundamental vehicle systems. For example: crash avoidance test surrogates; robotic platforms and equipment, and other necessary software and hardware to support controlled, reproducible, repeatable, and accurate testing.

• Expertise in data acquisition and processing: vehicle instrumentation, ground truth recording, data collection, off-loading, storage, and analytics.

• Testing of vehicle crash avoidance technologies, such as FCW/AEB, LDW/LKA, BSW/BSI, and VRU AEB systems.

• Testing of driver assistance technologies, such as ACC and LCA systems.

• Testing of partial driving automation technologies, such as sustained (e.g., highway pilot) and maneuver-based (e.g., parking, automated lane change) systems.

• Testing of ADS technologies, corresponding to SAE Level 3-5 driving automation, and vehicles equipped with ADS technologies.

• Testing of other intelligent technologies, such as intelligent speed adaptation (ISA), driver monitoring systems (DMS), connected vehicle systems, V2X, and remote assistance systems.

A major part of this research area is the development and validation of test methods to evaluate the performance of crash avoidance, ADAS, ADS, and other intelligent technologies in scenarios to understand how the technologies may avoid or mitigate crash-imminent situations. The focus of this work is to create test methods that are: cost-effective and practical; examine performance in specific crash scenarios; and do not favor one technological approach versus another (i.e., they are technology neutral). Test methods should be thorough, well documented, repeatable and reproducible.

Objective Test Procedures (OTPs) for new technologies can provide important benefits for both industry and consumers. OTPs provide a consistent and reliable means for assessing the safety performance of various technologies. For example, during design stages, OTPs can assist developers with efficiently comparing the relative performance of alternative sensor technologies and/or algorithm designs. The OTPs can also be used by OEMs to compare their systems’ performance with those of competitors. Additionally, the test results can be used by developers as inputs to simulation models to help refine various system calibration settings and improve performance. Such calibrations may determine timing and circumstances of driver warnings and/or vehicle control activations (braking or steering) and thus impact customer acceptance and usage patterns. These tools therefore can assist suppliers and OEMs with tailoring design trade-offs to improve customer acceptance while preserving or enhancing safety benefits.

Controlled-environment evaluation may support relevant work in other research areas. For example, closed track tests may be performed to assist with developing and validating simulation models and in-the-loop methods for ADAS and ADS perception, planning, and control functions (see Research Area 2 - Virtual Testing (Simulation & Modeling)). Specifically, vehicle and/or subsystem performance in response to certain test scenarios would be documented under “real-world” conditions and could then be compared to behavior in a virtual testing environment to refine simulation methods. Controlled-environment evaluation may also enable extensive characterization/sensitivity testing of ADAS and ADS performance in targeted scenarios that were identified in real-world datasets, such as naturalistic driving studies.

Tasks may include, but are not limited to:

• Development of test procedures for evaluation of crash avoidance, ADAS, ADS, other intelligent technologies, and fundamental vehicle systems focusing on specific scenarios and maneuvers.

• Development of advanced test methods, including tools, equipment, instrumentation, software, data collection, data processing, and data analytics.

• Refinement and use of advanced robotic test surrogates representing vehicles and vulnerable road users to enable safe, repeatable, and reproducible evaluation of complex, crash-imminent test scenarios.

• Support for other research areas, such as refinement of virtual testing methods (see Research Area 2 - Virtual Testing (Simulation & Modeling)) and characterization of real-world ADAS and ADS crash scenarios.

• Research into harmonization of test procedures with international standards organizations, regulatory agencies, and consumer testing programs.

Research Area 2 - Virtual Testing (Simulation & Modeling)

Areas of focus and desired expertise include, but are not limited to, the following:

• Experience and expertise in virtual testing methods to evaluate crash avoidance, ADAS, ADS, other intelligent technologies, and fundamental vehicle systems for light vehicles, heavy vehicles, and other vehicle types (e.g., motorcycles):

o Model-in-the-Loop (MIL).

o Software-in-the-Loop (SIL).

o Hardware-in-the-Loop (HIL).

o Vehicle-in-the-Loop (VIL).

o Other applicable simulation and modeling methods/approaches o Nominal operation and critical, crash-imminent scenarios

• Identification of candidate testing scenarios, maneuvers and behavioral competencies that may be used to evaluate vehicle technologies in a virtual environment and are adaptable for test-track and/or on-road testing for validation of virtual methods.

• Application of simulated roadway and traffic environments.

• Co-simulation distributed environment testing and digital twins.

• Driving simulators and human subject testing to evaluate technology acceptance, effectiveness, and potential unintended consequences.

• Virtual testing related to connected vehicle systems, V2X, V2N, and remote operations.

• Mixed or augmented reality testing.

Advancements in safety technologies and driving automation (e.g., ADAS, ADS) can introduce additional challenges and complexity for applied vehicle research and exhaustive testing of real world scenarios. Higher levels of driving automation, in particular, can result in a very large number of real-world, critical scenarios (i.e., edge and corner cases) involving system actions, driver actions if applicable, multiple road users, and complex roadway and environmental conditions. The scale and nature of these scenarios may preclude the practicability of exhaustive real-world testing using test track or on-road methods. This research area will explore the feasibility of employing simulation and other virtual testing methods to augment closed-track and on-road evaluation. Primary objectives of this research area are to explore the technology’s utility and capability limits, define system behavior limits, identify systematic faults and failures, verify corrective actions and safety measures, and evaluate operational performance.

Tasks may include, but are not limited to:

• Development and refinement of virtual test tools, equipment, and instrumentation to support efficient, repeatable and representative-of-real-world evaluations of crash avoidance, ADAS, ADS, other intelligent technologies, and fundamental vehicle systems.

• Development of simulation and virtual environments for testing sub-functions of ADAS and ADS technology, such as perception, planning, and control functions.

• Identifying, developing, and building a virtual test framework and analytic methods required to simulate complex, real-world, test scenarios in a multi-actor (including VRUs) and connected vehicle environment.

• Development and validation of virtual testing methods spanning model/algorithm, software, component, subsystem/system, and vehicle level evaluation of safety performance.

• Utilization of applicable test-track and on-road testing methods to develop, refine, verify, validate, and correlate the virtual testing methods to real-world performance.

Research Area 3 - Vehicle Component Testing and Functional Safety Assessments

• Expertise in conducting STPA (System Theoretic Process Analysis) to identify hazards in

ADS.

• Expertise in J-3187; the application of STPA in conjunction with SOTIF.

• Expertise in integrating STPA in the system and software development process.

• Expertise in applying UL 4600 to evaluate the safety of ADS and to explore tools that can accelerate the audit and assessment of ADS.

• Expertise in functional safety audits and assessments of crash avoidance, ADAS, ADS, other intelligent technologies, and fundamental vehicle systems for light vehicles, heavy vehicles, and other vehicle types (e.g., motorcycles), to include, but not limited to assessment of Safety Management Systems (SMS), Safety Case, survey of process and methods to develop the same.

• Expertise in process, methods and frameworks for Safety Case development, with expertise in frameworks such as UL 4600

• Expertise in process and methods for Safety Management Systems, with expertise in “Designing an Effective Safety Management System (SMS)”*, ISO 26262, and ASPICE.

• Expertise in ASPICE (Automotive Software Process Improvement and Capability Determination) assessments for ADAS and ADS. Expertise in evaluating software capability and maturity, and incorporation of STPA into the software development process.

• Expertise in Functional safety assessment of ADAS and ADS components and subsystems, including perception, planning, and control. Expertise in Safety of the Intended Functionality (SOTIF), hazard analysis and risk assessment, Failure Mode and Effects Analysis (FMEA), FTA, and other relevant methods.

• Exploration of operational limits for ADAS and ADS technologies, and corresponding safety assessments.

• Vehicle component and subsystem testing facilities, equipment, and expertise to support safety-relevant research on crash avoidance, ADAS, ADS, other intelligent technologies, and fundamental vehicle systems for light vehicles, heavy vehicles, and other vehicle types (e.g., motorcycles).

o Vehicle component-level testing (e.g., sensors, tires).

Expertise in quantum sensor technology as applied to ADS, ADAS applications.

o Vehicle subsystem-level testing (e.g., braking, steering).

o Vehicle subsystem diagnostics methods and evaluation.

• Ability to conduct testing at component and subsystem level for ADAS and ADS perception, planning, and control, as well as EMI/EMC, automatic shutoff, rollaway, driver monitoring, and other safety-relevant technologies.

• Performance, durability, and operational testing of electrical and electronic components and subsystems to support virtual test methods, such as MIL, SIL, HIL, and VIL.

The safety and reliability of ADAS- and ADS-equipped vehicles are critical to public acceptance and commercial deployment. As more of the dynamic driving task is transferred from the human driver to the driving automation system, human sensing and cognition functions are essentially being relegated to the system through a collection of integrated hardware and software subsystems. Accordingly, various methods and tools are necessary to assess the functional safety of vehicle subsystems and their building block components. In addition, there may be new hazards and failure modes that surface through the introduction of these subsystems into the automotive architecture. Robust methods are needed by stakeholders to identify causality of failures rooted in these software-intensive subsystems. A primary objective of this research area is to establish the capability to assess the functional safety performance of vehicle components and subsystems, which can augment safety assessments performed in other research areas.

Tasks may include, but are not limited to:

• Analyze ADS and ADAS using J-3187 (STPA and SOTIF) as a means of evaluating the safety and hazards of ADS, ADAS architectures and systems.

• Identify and apply system safety ADS assessment framework such as UL 4600. Explore the benefits of applying this approach as a method to assess safety risk and mitigation.

• Identify and apply functional safety assessment methods such as ISO 26262, and reliability analyses such as system safety analysis, failure mode and effects analysis; fault tree analysis, hazard operations assessments, and other applicable methods to identify and assess safety risks and mitigation options for various vehicle components and systems.

• Identify and apply safety-of-the-intended functionality (SOTIF) analyses to crash avoidance, ADAS, ADS, other intelligent technologies, and fundamental vehicle systems.

• Perform testing to assess the safety performance of ADAS and ADS perception, planning, and control components and sub-systems, other intelligent technologies, and fundamental vehicle systems.

• Perform component and subsystem testing in a broad range of safety-relevant areas to augment closed-track and on-road test methods at the system/vehicle level and to support virtual test methods.

• Document and survey quantum sensor applications in ADS, ADAS. Document quantum sensor fundamentals and considerations in ADS and ADAS applications.

Research Area 4 - On-Road Testing and Naturalistic Studies

• Vehicle instrumentation, ground truth data capture, data acquisitions systems, data collection, data transfer, data processing, and data storage.

• Comprehensive analysis of large volumes of collected vehicle data:

o Scenario identification, extraction (e.g., parsing), and characterization.

o Annotation of recorded video into structured data fields.

o Development and computation of safety metrics and thresholds.

o Data mining of specific behaviors and characteristics.

• Development and validation of on-road testing methods to evaluate performance of crash avoidance, ADAS, ADS, other intelligent technologies, and fundamental vehicle systems.

• Planning, managing, and executing large-scale field operational tests.

• Experience with human-subject recruitment and testing:

o Paperwork Reduction Act (PRA) requirements and approval.

o IRB and applicable safety approval.

o Privacy, confidentiality, and other sensitive information.

Naturalistic driving studies can provide fundamental information on how drivers use crash avoidance, driver assistance and partial driving automation systems. Specifically it can study:

under what circumstances are systems engaged or disengaged; differences in use patterns among driver demographics; driver behaviors when systems are activated/engaged; impacts of various human machine interface (HMI) designs; etc. Naturalistic driving studies can also provide detailed data on real-world scenario conditions and corresponding activations (e.g., alert, intervention) of crash avoidance systems (e.g., FCW/AEB, LDW/LKA, BSW/BSI). Such information may be useful to technology providers and OEMs to improve system performance and customer acceptance. In-use operational data may also be useful to NHTSA to gain insight into the capabilities and limitations of various technologies, which can influence usage and adoption, and support safety assessments in other research areas. Analysis of collected data may also identify potential unintended consequences of technologies, such as over-reliance, misuse, or mistrust of systems by drivers.

Unlike naturalistic driving studies, on-road testing research does not typically involve human subject testing. Trained drivers are utilized by the contractor (or subcontractor) to drive test vehicles and collect real-world data on publicly accessible roads. On-road testing is generally focused on system performance in nominal operation, usually in targeted scenarios, that may be complex and challenging to perform on a closed track. For example, evaluating the overall lane centering performance (e.g., average lateral deviation from lane center) of an SAE Level 2 system on straight roads with varying degradations of lane markings or curved roads with varying curvatures and grades. In contrast, naturalistic driving studies are generally focused on broad, driver usage of technology over an extended time period. Since on-road testing is generally more targeted and involves trained drivers, research studies can be smaller in scale than a naturalistic driving study, with fewer test vehicles, higher vehicle diversity, and lower volume of collected data. On-road testing generally allows for more sophisticated instrumentation (e.g., perception sensors, data acquisition system) and installation (e.g., custom mounting brackets) to improve reliability and accuracy of ground truth data capture for scenario and safety performance analysis.

Tasks may include, but are not limited to:

• Conducting field operational tests to assess the real-world usage, system performance, and potential unintended consequences of advanced safety technologies, including crash avoidance, ADAS, other intelligent technologies, and fundamental vehicle systems.

• Field Operational Test (FOT) planning, management, and execution include PRA approval, safety approval (e.g., IRB), and human subject recruiting.

• Data collection, processing, transfer, storage, management, and analysis of large naturalistic driving datasets.

• Development and validation of on-road testing methods to evaluate the safety performance of ADAS and ADS technologies, including perception, planning, and control functions.

NOTE: Human factors research questions may be examined as part of NHTSA’s field operational and on-road studies. Expertise related to human factors associated with advanced safety technologies and ADS may be necessary to complete some of the proposed studies within this research area.

Research Area 5 - Safety Impact Assessment Support

Areas of focus and desired expertise include, but are not limited to, the following:

• Acquisition, processing, and analysis of relevant datasets for safety impact assessment (e.g., FARS, CRSS, CISS, state crash data, SCI, customer complaints, naturalistic driving data, closed track tests, on-road tests, virtual tests, NITS, etc.)

o Scenario identification, categorization, and characterization Crash causation and modeling o Trend analysis and forecasting o Safety problem definition and quantification o Potential safety benefits estimation (i.e., portion of safety problem that is addressable through motor vehicle safety technology) o Data mining, such as specific scenarios, events, actions, and behaviors relevant to crash avoidance, ADAS, ADS, other intelligent technologies, and fundamental vehicle systems

• Technology assessment of implementation readiness, technical feasibility, and costs.

• Development of safety performance evaluation frameworks, methods, and measures to support safety impact assessments not covered in other research areas.

• Expertise in applying UL 4600, ASPICE, STAMP, STPA, J-3187, System Safety Analyses, Systems Engineering and Architecture.

• Expertise with Federal Motor Vehicle Safety Standards (FMVSS), New Car Assessment Program (NCAP), and NHTSA Rulemaking process.

This research area leverages information gathered through querying of NHTSA crash databases, test-track research, field evaluations, on-road tests, and other relevant datasets to estimate potential safety benefits of crash avoidance, ADAS, ADS, other intelligent technologies, and fundamental vehicle systems. This research area includes the acquisition, processing, and analysis of these datasets to characterize real-world scenarios, understand causation factors, quantify the overall safety problem, and evaluate how vehicle technologies may address the safety problem. This work combines target crash population research, technology effectiveness estimates from test data, market deployment forecasts, and customer usage trends to support safety impact assessments.

Under this research area, NHTSA also seeks support for potential modifications to existing FMVSS test procedures in accommodating advanced safety and ADS technologies. The absence of a human driver in certain SAE Level 4 ADS-equipped vehicles (e.g., driverless robotaxis) creates opportunities for manufacturers to design new vehicle architectures that may remove driving controls, change seating configurations, and establish new interfaces for passengers.

Existing FMVSS may present unintended barriers for vehicles without drivers, in some instances by referring to the driver for reference purposes, and in others by tying test performance to human-performed actions and controls.

Tasks in this research area may include:

• Identifying and acquiring relevant datasets for technology-specific safety impact assessment. Certain research topics may require innovative approaches to finding and acquiring relevant datasets.

• Performing data processing, analysis, and mining to characterize the real-world scenarios and quantify the associated safety problem that may by addressable through advanced safety and ADS technologies. Certain research topics may require advanced and innovative techniques in crash modeling, causation determination, statistics, scenario mining, and text analytics (e.g., large volume of narratives with sensitive and/or redacted data) to estimate the addressable safety problem.

• Assessing the implementation readiness, technical feasibility, and costs of technologies.

• Development of models and methods to assess the safety impact of advanced safety and

ADS technologies.

• Analyzing system maintenance, degradation and reliability issues using data gathered from field operational tests and/or other sources to help estimate changes in system effectiveness over the expected life of the vehicle.

• Assisting with analyzing potential regulatory barriers associated with advanced and novel motor vehicle designs. Developing alternative test procedures and/or regulatory language that ensures equivalent levels of safety.

• Analyzing various industry efforts to develop and apply safety evaluation frameworks for

ADS. Such industry efforts may originate through standards development organizations (SDOs), industry consortia, or related international ADS research and standards development efforts.

Research Area 6 – Cybersecurity

• Vehicle cybersecurity risks and defensive methods.

• Vehicle cybersecurity evaluation approaches.

• External factors impacting cybersecurity.

• Cybersecurity preparedness.

• Emerging considerations in vehicle cybersecurity.

• Evaluation of emerging technologies in vehicle cyber cryptography.

Ever since digital electronics were first introduced into vehicles for engine ignition and injection control, their use has been continually expanded to include many new safety, mobility, and efficiency features commonly included in modern cars today. These software-intensive functions, and wired and wireless data exchange interfaces, may introduce cybersecurity challenges and potential safety concerns.

NHTSA’s vehicle cybersecurity research covers vehicular electronic systems, associated wired and wireless interfaces, and the underlying software that controls a vehicle’s systems. It emphasizes the importance of partnering with industry to broadly implement cybersecurity best practices throughout the vehicle lifecycle, thus addressing potential safety risks to the motoring public.

Tasks may include research in the following areas, but not limited to:

• Perform cybersecurity-related safety risk assessments.

• Develop and evaluate vehicle lifecycle cybersecurity management and organizational approaches to address cybersecurity considerations during design, development, sourcing, manufacturing, operating and servicing of motor vehicles.

• Design-specific approaches for risk mitigation and system protection.

• Cybersecurity data analytics to include monitoring communications and data onboard the vehicle as well as external to the vehicle.

• Develop techniques for cybersecurity incident containment and recovery.

• Develop vehicle cybersecurity test tools, methods, measures, and metrics.

• Examine interactions with consumer devices.

• Explore cloud-based applications and services, including firmware over-the-air (OTA) update security and methods.

• Explore blockchain methods for cryptography.

• Understand and document fundamental principles in applying blockchain in vehicle cryptographic applications.

• Explore advanced technologies for cryptography, to include quantum applications.

• Replacement component security and lifecycle cybersecurity support.

• Vulnerability/incident research and tracking; cybersecurity exercises.

• Cyber-intelligence and information sharing and collaboration, such as: Industry information sharing and collaboration and Government information sharing and collaboration with, but not limited to NIST, BIS, CISA and NQCO.

• Develop strategies for communication and messaging for the motoring public.

• Education, training, technical workforce development.

• ADS cybersecurity considerations.

• Sensor vulnerabilities and countermeasures.

Research Area 7 - Artificial Intelligence (AI) / Machine Learning (ML) Methods, Algorithms, and Software

Areas of focus and desired expertise include, but are not limited to, the following:

• Expertise in emerging artificial intelligence (AI) and machine learning (ML) algorithms, methods, calibration, software, applications and systems as applied to crash avoidance, ADAS, ADS, other intelligent technologies, and fundamental vehicle systems.

o Perception stack/algorithms (e.g., object detection and classification, fusion).

o White/gray box prototypes (e.g., component, subsystem, vehicle).

o Scenario sampling (e.g., TC2 clustering) and parsing (e.g., automated scenario extraction from naturalistic driving data).

o Data analytics (e.g., large language models to mine crash data narratives).

• Understanding and documenting fundamental principles of AI and ML—including initial programming as well as “learning” (i.e., training) processes.

• Understanding potential benefits and risks related to emerging AI and ML applications and usage in motor vehicle technologies.

• Methods and processes for documenting how a particular AI implementation uses inputs, programing constructs, and learned experiences to make decisions (e.g., explainable AI).

• Development of methods and metrics for assessing the performance of AI- and ML enabled systems as applied to motor vehicle technologies.

• AI and ML algorithms, training, models, and applications. Expertise in emerging applications of quantum computing in vehicle safety and crash avoidance.

• Expertise in quantum sensors.

Developers of advanced safety and ADS technologies are increasingly utilizing Artificial intelligence (AI) and machine learning (ML)3 techniques to improve both development processes as well as operational performance. For example, AI techniques (e.g., convolutional neural networks, deep learning) are being used by developers to enhance object recognition software as part of camera-based sensor systems. NHTSA anticipates that ADAS and ADS developers will continue to explore if and how AI may be used to enhance performance of system perception, 3 USDOT defines AI as “AI is "any man-made system that performs tasks under varying and unpredictable circumstances without significant human oversight, or that can learn from experience and improve performance when exposed to data sets." Machine learning (ML) is a subset of AI that enables computers to find patterns in data and improve performance over time without explicit programming.

prediction, planning, and decision-support algorithms. The use of AI however presents unique challenges related to assessing the safety and operational performance both at product launch and over the lifecycle of the product (e.g., over-the-air software updates, system exposure to a real-world object or scenario that differs from training data).

Tasks may include, but are not limited to:

• Analysis of training datasets and development of training data from new sources, potentially employing “big data” methods and synthetic data.

• Development and prototyping of AI/ML models, including stakeholder outreach as necessary to validate algorithms and training. For example, implementation of a white box perception stack to understand timing and confidence of object detection and classification for ADAS and ADS sensor suites and computation of safety performance metrics, which may utilize open-source algorithms, developer toolkits, or proprietary software provided by industry developers.

• Perform AI-related safety risk assessments and assess risk mitigations.

• Verification and validation of trained models and software.

• Evaluate AI software architectures and approaches to integrate into higher level software and hardware systems.

• Development of methods and metrics for performance assessment of AI systems.

• Survey and document advancements in quantum sensors for vehicle safety applications.

• Document fundamental principles of quantum cognition, quantum decision making, quantum information, quantum communications, and quantum computing.

• Survey and document advancements in quantum applications in crash avoidance, ADS, and ADAS.

• Explore quantum decision making in addressing “irrational” (surprise) behavior of pedestrians and VRUs.

C.3.3 Task Order Coordination Meetings

Within sixty (60) days after contract award, the Senior Technical Program Manager shall develop a schedule of regular Task Order Coordination Meetings with the COR at mutually agreed on dates/times as an online meeting (e.g., Teams, Zoom). The purpose of the meetings is to briefly review the status of current task orders, notify the COR of any new issues that have arisen, discuss any open issues that are not yet resolved, and answer questions on the research topics under consideration. The frequency of the meetings will depend on the number and complexity of task orders for the contractor. It is recommended that these meetings be scheduled for 30 minutes and occur initially at a minimum of quarterly. However, once the contractor has received more than one task order, monthly or bi-weekly meetings may be more appropriate in certain circumstances as the potential for more issues and questions arises.

(END OF SECTION C)