693JJ3-20-BAA-0005 Acquiring Connected and Automated Vehicle (CAV) Performance Dataset - Amendment 1.pdf

Office of Acquisitions and

Grants Management

Broad Agency Announcement

693JJ3-20-BAA-0005,

Amendment 1

Acquiring Connected and Automated Vehicle (CAV)

Performance Datasets

693JJ3-20-BAA-0005

I. General Information

Contracting Officer: DeWhyte McPherson, FHWA, (202) 366-0494, Primary

DeWhyte.McPherson@dot.gov

SUMMARY OF IMPORTANT DATES

The Federal Highway Administration anticipates that the proposal, evaluation and award process for this Broad Agency Announcement will proceed generally in accordance with the following schedule:

BAA Opens (Beta.Sam.gov posting) July 23, 2020

Question Period Closes August 5, 2020

Proposals Due 1:00 pm EST September 04, 2020

Anticipated Award Dates: Beginning September 2020 through December

PROPOSAL SUBMISSION: Electronic transmissions of the proposal shall be sent to

HCFA-23SRI@dot.gov and to the Contracting Officer, DeWhyte.McPherson@dot.gov no later than 1:00 p.m., EST, on September 04, 2020. Proposals received after this date and time will not be considered.

The proposal transmission shall be titled in the following sample format:

Topic – New/Existing Datasets, XYZ University – Acquiring CAV Performance Datasets BAA

FUNDING NOTE: Funding is available in the total amount of $900,000 for all awards under this Broad Agency Announcement. FHWA anticipates that a total of three (3) to five (5) contracts may be awarded.

mailto:DeWhyte.McPherson@dot.gov mailto:HCFA-23SRI@dot.gov mailto:DeWhyte.McPherson@dot.gov

II. Overview

INTRODUCTION: This solicitation is being conducted via a Broad Agency

Announcement (BAA), in accordance with Federal Acquisition Regulation (FAR)

35.016 Broad Agency Announcement.

1. Purpose:

The U.S. Department of Transportation (U.S. DOT) / Federal Highway Administration (FHWA) is issuing this Broad Agency Announcement (BAA) to procure datasets that document how

Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS)—with or without connectivity—operate in traffic. The Society of Automotive Engineers (SAE) Levels of

Driving Automation Standard for Self-Driving Vehicles (J3016TM) defines ADAS and ADS features. ADAS features, which include SAE levels 1 and 2, are available in new cars sold today and include features like automatic emergency braking and adaptive cruise control. ADS technology, SAE levels 3–5, is currently under development and is being tested on roadways and test tracks today.

These datasets shall also document how adjacent vehicles in the traffic stream, detectable by vehicle sensors equipped on the ADAS/ADS, behave in the presence of the subject ADAS/ADS.

These adjacent vehicles may be manually driven (SAE L0) or may have some automated capability (SAE L1–L5). These datasets, collected on arterials or freeways, shall include data variables for both the instrumented ADAS/ADS and the adjacent vehicles (manually driven, ADAS, or ADS) in the traffic stream, as described in Section II of this document.

These datasets shall be research-ready and not require additional post-processing, upon delivery to FHWA, to build models of both ADAS/ADS driving behavior and the behavior of vehicles

(manually driven, ADAS, and ADS) in the presence of subject ADAS/ADS. These datasets shall have clear data ownership rights, enabling FHWA to publish the datasets for public access and use, excluding proprietary and personally identifiable information (PII) (as discussed in Section

II-D).

These datasets may be currently in existence, where the supplied funding will pay for the Offeror to clean, integrate, organize, and document the data, or the Offeror may propose a brand-new data collection effort for funding consideration.

2. Scope or Mission

One of the three goals of the Traffic Analysis Tools program within the FHWA Office of

Operations (HOP) is to “operate/analyze the transportation system in a more proactive, integrated, performance-driven manner.” However, robust tools do not presently exist to accurately forecast the impact of ADAS/ADS on the Nation’s roadways.

FHWA will use the data procured as part of this project to develop such models (e.g., car-following, lane changing, and intersection navigation models for both the ADAS/ADS and the vehicles surrounding the subject ADAS/ADS). Thus, there are certain data variables for the instrumented ADAS/ADS and the adjacent vehicles (manually driven, ADAS, and ADS), that must be either directly measured or accurately estimated from directly measured data variables.

This Solicitation explicitly describes these requirements in Section II.

3. Period of Performance

The period of performance will be 12 months from the date of award for existing datasets and 24 months from the date of award for new data collection efforts. FHWA anticipates multiple awards. Dependent on the scope and scale of the proposed datasets and data collection efforts, FHWA anticipates between 3 to 5 awards.

4. Background

Analysis, modeling, and simulation (AMS) tools are critical to infrastructure owners and operators (IOOs) (e.g., State and local Departments of Transportation). In fact, the most recent

Surface Transportation Reauthorization Act, the Fixing America’s Surface Transportation

(FAST) Act, placed increasing emphasis on the application of AMS in the planning process1.

Section 1430 explicitly states:

“The Department should utilize, to the fullest and most economically feasible extent practicable, modeling and simulation technology to analyze highway and public transportation projects authorized by this Act to ensure that these projects—”

(1) will increase transportation capacity and safety, alleviate congestion, and reduce travel time and environmental impacts; and

(2) are as cost-effective as practicable1.” (p. 117)

AMS tools are valuable because they help IOOs understand the impact of various changes to the transportation system (e.g., increases in demand, installation of active traffic management capabilities, new high-occupancy toll or general-purpose lanes, new signal timing plans, new vehicle technology) on system performance (e.g., capacity, speed, travel time). These tools help

IOOs to better operate their existing system and to better plan their transportation system of tomorrow.

Thus, AMS tools are going to be extremely valuable for helping IOOs understand the likely impacts of ADAS and ADS on their transportation system performance, make better investment decisions now to ensure their transportation system is equipped for ADAS/ADS in the future, and to optimally operate their roadways once ADAS/ADS are deployed. Ultimately, having a transportation system that is better designed and equipped to accommodate ADAS/ADS will benefit the general public, automobile original equipment manufacturers (OEMs), and

ADAS/ADS technology companies.

The cornerstone U.S. DOT publication—Development of an Analysis/Modeling/Simulation

Framework for V2I and Connected/Automated Vehicle Environment2—details a comprehensive

1 114th Congress. Fixing America’s Surface Transportation (FAST) Act, Pub. L. No. 114-94 (2015).

Retrieved from https://www.congress.gov/114/crpt/hrpt357/CRPT-114hrpt357.pdf 2 Mahmassani, H. S., A. Elfar, S. Shladover, and Z. Huang. (2019). Development of an

Analysis/Modeling/Simulation Framework for V2I and Connected/Automated Vehicle Environment. Retrieved from: https://rosap.ntl.bts.gov/view/dot/39965 https://rosap.ntl.bts.gov/view/dot/39965 methodological framework for developing connected and unconnected ADAS and ADS AMS tools. As part of this framework, the research team identified existing data sources for developing, calibrating, and validating tools to model ADAS/ADS and human behavior in the vicinity of ADAS/ADS. Through this analysis, it was identified that the availability of naturalistic ADAS/ADS driving data is a significant gap hindering the development of robust models of ADAS/ADS performance in the traffic stream. Very few datasets exist, much less are available freely online, for modeling the performance of a single ADAS/ADS or a string of

ADAS/ADS in naturalistic driving conditions. Moreover, few datasets are available that document how manually driven vehicles react to ADAS/ADS in naturalistic driving conditions.

This project seeks to procure such datasets.

As an additional resource, a forthcoming publication—Modeling Capabilities and Data Needs for

Assessing the Operational Impacts of Connected and Automated Vehicles3—is Attachment 3 of this BAA. This document, recently completed by FHWA, summarizes what models are needed to robustly evaluate the performance of connected and unconnected ADAS/ADS in traffic flow and what data variables are desirable to build these models. This document will help the Offeror better understand the future use cases of the proposed datasets.

5. Performance Objectives, Goals, and Outcomes

The U.S. DOT is seeking datasets that document how a subject ADAS/ADS operates in traffic and how adjacent vehicles (manually driven, ADAS, and ADS) behave in the presence of the subject ADAS/ADS. Adjacent vehicles are defined as the vehicles in the traffic stream that are detectable by vehicle sensors equipped on the ADAS/ADS. These data will be used to develop both predictive and descriptive models of ADAS/ADS performance in traffic flow (i.e., microsimulation models). These datasets will be posted online to the ITS DataHub4 to enable both FHWA and the research community at large to develop these models.

The proposed datasets must be sufficiently significant in scope and duration (i.e., number of trajectories, length of trips) such that robust models of the car-following, lane changing, and intersection navigation behavior of the subject ADAS/ADS and adjacent vehicles (i.e., both manually driven vehicles and other ADAS/ADS) interacting with the subject ADAS/ADS can be developed and estimated.

The primary technical objective of this effort is to procure vehicle trajectory datasets. Depending on the maturity of the dataset (i.e., previously collected vs. proposed to be collected), this could require several different labor tasks for the Offeror including, but not limited to:

• Developing a data management plan (DMP), as described in Section II-C.

• Collecting data (if not previously collected).

• Securing data rights.

• Cleaning data (e.g., smoothing noisy data).

• Post-processing data (e.g., estimating required data variables from directly measured data).

3 Berg, I., A. Berthaume, and R. James. (Forthcoming). Modeling Capabilities and Data Needs for

Assessing the Operational Impacts of Connected and Automated Vehicles.

4 https://www.congress.gov/bill/115th-congress/house-bill/4174/text#toc-

H8E449FBAEFA34E45A6F1F20EFB13ED95

• Scrubbing data (e.g., removing PII and proprietary data that cannot be shared publicly).

• Documenting data (e.g., consistent data format, documenting necessary metadata for the

ITS DataHub).

• Estimating the accuracy of data variables.

6. Materials and Personnel

The materials and personnel required to meet the U.S. DOT objectives are summarized below and outlined in more detail later in this document.

• Materials: Provide all necessary equipment for data collection and processing.

• Personnel: Provide personnel with the technical background to conduct data collection, processing, and documentation tasks.

A. Data Collection Scope

This section describes the scope of the proposed datasets and data collection efforts. FHWA is interested in procuring diverse dataset(s) of connected or unconnected ADAS/ADS driving behavior. This section describes how the proposed dataset(s) may vary between the Offerors.

New vs. Existing Datasets: Previously collected datasets, if they were recently collected (i.e., data collected with vehicles that are representative of current/anticipated vehicle fleet/technology), are eligible for funding consideration. The Offeror may propose to use the available funding to clean, integrate, post-process, organize, and document the data. These previously collected datasets must have clear data ownership, contain the required data variables described in Section II-B, and have the necessary metadata to be eligible for consideration.

Conversely, the Offeror may propose the collection and post-processing of new datasets that meet the needs described in this document. If a new data collection effort is proposed, the

Offeror may propose to use the funding to collect, clean, integrate, post-process, organize, and document the data. These previously collected datasets must have clear data ownership, contain the required data variables described in Section II-B, and have the necessary metadata to be eligible for consideration.

As part of the submission package, Offerors proposing the use of existing datasets must provide a small sample of the existing data and a detailed description of the data variables for evaluation.

Offerors proposing a new data collection effort must submit a detailed description of data variables that will be collected as part of their submission package.

Location of Data Collection: Data may be collected on freeways or arterials. Necessary situational data on the freeways (e.g., number of lanes, road ID, postmile numbers) and arterials

(e.g., intersection access control type, signal phase and timing plan, number of lanes, location) must be provided in the datasets. Preference is given to naturalistic, field test collected data.

However, controlled test track, hardware-in-the-loop, and dynamometer testing data will be considered for funding if insufficient naturalistic datasets are proposed.

Vehicle Technology: It is preferred that the data be collected using ADAS and ADS with higher levels of automation (i.e., SAE L2–L5). This means that the subject ADAS/ADS should have both lateral and longitudinal control/assistance. SAE L1 data will be considered if the Offeror articulates the value added by the dataset. The ADAS/ADS may be connected or unconnected.

Driving Task: The vehicles may be performing a basic driving task (e.g., waypoint following, adaptive cruise control) or more advanced applications (e.g., signalized intersection approach and departure, platooning, lane changes).

B. Required Data Variables

These data are intended to be used in the near future to develop, calibrate, and validate models of

ADAS/ADS driving behavior and the behavior of adjacent drivers in the vicinity of the

ADAS/ADS. Adjacent vehicles are defined as vehicles in the traffic stream that are perceived by vehicle sensors equipped on the ADAS/ADS.

Thus, there are several required data variables that must be collected about the instrumented vehicle (ADAS/ADS), the adjacent vehicles (manually driven, ADS, or ADS), and the driving environment. These data variables may be directly measured by sensors on the instrumented vehicles, estimated from data measured directly by vehicle sensors, collected from corroborative datasets (e.g., Roadway Information Database; infrastructure sensors (e.g., loop detectors); third-party data sources; aerial data (e.g., drones, videos/sensors on tall buildings like the FHWA Next

Generation SIMulation (NGSIM) data collection effort)), or using additional innovative strategies to be described by the Offeror in their proposal. In addition to developing methods for collecting these data variables, the Offeror shall also develop methods for validating the derived data variables and provide estimates of the accuracy of the data variables.

Vehicle Performance Data: The preferred dataset as a deliverable for this project will contain vehicle trajectories. However, datasets reporting aggregate statistics on vehicle performance metrics (e.g., following distance, time gap during AV lane change, time gap accepted during intersection movements, start-up lost time, etc.) under a variety of operational conditions (e.g., varying traffic volumes, day/night, etc.) will be considered.

For trajectory datasets, preferred datasets will contain variables collected for both the instrumented ADAS/ADS and the adjacent vehicles—manually driven, ADAS, and ADS—in the traffic stream. The preference is that processed numerical data should be provided in a comma-separated variable (.CSV) format; other types of data (e.g., video, original sensor data (e.g., LiDAR)) may remain in the original format. The datasets must be consistently formatted. At a minimum, this dataset must contain the following data attributes at a frequency of 10 Hz or higher:

• Subject Vehicle (Instrumented ADAS/ADS):

o Vehicle position.

o Vehicle speed.

o Vehicle acceleration.

o Steering wheel inputs.

o Gas/brake pedal inputs.

o Identified points of the transition of responsibility of the driving task between the

ADAS/ADS and the supervising human driver.

o System settings (e.g., adaptive cruise control time gap setting).

o Vehicle attributes (e.g., SAE automation level, applications installed on vehicle

(e.g., cooperative adaptive cruise control), details of ADAS/ADS equipment, installed sensors and accuracy of sensors, and details about the conspicuity of the vehicle (i.e., could a naïve driver identify the car as an ADAS/ADS)).

• Immediately adjacent vehicles (e.g., vehicles detectable by vehicle sensors equipped on the ADAS/ADS, vehicles detectable by overhead or airborne cameras/sensors):

o Adjacent (front, behind, and in either adjacent lane) vehicle(s) position.

o Adjacent (front, behind, and in either adjacent lane) vehicle(s) speed.

o Adjacent (front, behind, and in either adjacent lane) vehicle(s) acceleration.

o Leading, following, and adjacent vehicle IDs.

o Relative speed (compared to subject/instrumented vehicle).

o Relative spacing (compared to subject/instrumented vehicle).

Adjacent vehicles may be manually driven, ADAS, or ADS.

Driving Environment Data: Additional data about the prevailing traffic conditions and the roadway environment must also be provided. Preferred datasets would contain the driving environment data necessary for microsimulation base model development, as described in the

Traffic Analysis Toolbox Volume 35. However, datasets that provide, at a minimum, higher-level information about the roadway environment (e.g., urban/rural; freeway/arterial, day/night) are eligible for consideration. The additional elements desired include, but are not limited to, the following data variables (as applicable):

o Localized traffic density.

o Number of lanes.

o Current lane number.

o Roadway type and location (e.g., name of roadway, postmile number, sufficient information to link trip to map).

o Signal phase and timing data.

o Weather conditions (if collected during adverse weather conditions).

o Operational condition (e.g., presence of work zone).

Some of this data may be collected with a vehicle performing according to an application or suite of applications (e.g., signalized intersection approach and departure, platooning, lane changing, cooperative merge). If so, data about the specific application (e.g., platoon size, position in the platoon, active lane change, etc.) should also be recorded.

It is important to note that FHWA does not want pristine datasets that are artificially cleaned of noise. Errors in the measured data attributable to the collection process should be removed while natural noise in the data should be preserved.

The final delivered dataset(s) should include both their raw form, final processed form, and documented details on all methods applied to the data to get it into its final form. Additionally, the dataset must contain appropriate metadata to enable its publication on the ITS DataHub; the metadata requirements are described in Section II-I.

5 https://ops.fhwa.dot.gov/publications/fhwahop18036/fhwahop18036.pdf

At the conclusion of this project, the Contractor shall deliver a 508 and Communications

Reference Guide6 compliant draft and final Final Report that details the following (at a minimum):

• How data was collected.

• Data collection and processing assumptions.

• Description of data variables, variable format, and accuracy of variables.

• How data was processed.

• Value of data (i.e., how data can be used to support Traffic Simulation Tools development).

C. Data Management Plan

Offerors must provide a preliminary Data Management Plan (DMP) in their proposal. The U.S.

DOT will use the information in the preliminary DMP as part of its evaluation criteria for award and may ask for clarifications or changes from applicants prior to award to ensure the U.S. DOT and/or public receives appropriate data access. The preliminary DMP must provide a preliminary overview of data that may be collected or created through the project, and include the following:

1.DATA DESCRIPTION: Address the nature, scope, and scale of the data that will be collected or created. Define data variables within a detailed data dictionary.

2.ACCESS POLICIES: Describe any access restrictions that may apply to your data.

3.DATA STORAGE AND RETENTION: Describe where you intend to store your data and why you have chosen that particular option. Different systems may be designated for different types of data (e.g., public, non-public). Also describe how long you intend to retain your data in these systems.

4.DATA RIGHTS: Describe who will hold the intellectual property rights for your data, and any other legal requirements that might need to be addressed.

Awardees must submit an updated DMP within 60 days after effective date of award and receive

FHWA approval for the approach described. The updated DMP must be both 508 and

Communication Reference Guide compliant. The updated DMP must include the following:

1.DATA DESCRIPTION: Address the nature, scope, and scale of the data that will be collected. Define data variables within a detailed data dictionary.

2.ACCESS POLICIES: Describe any access restrictions that may apply to your data.

3.DATA STORAGE AND RETENTION: Describe where you intend to store your data and why you have chosen that particular option. Different systems may be designated for different types of data (e.g., public, non-public). Also describe how long you intend to retain your data in these systems.

4.DATA RIGHTS: Describe who will hold the intellectual property rights for your data, and any other legal requirements that might need to be addressed.

5.TIMING: Describe when your data will be shared, and how frequently it will be updated.

Awardees must then maintain the DMP throughout the period of performance with more detailed, updated information as it becomes available. At a minimum, the Awardee must update

6 https://www.fhwa.dot.gov/publications/research/general/15058/ https://www.fhwa.dot.gov/publications/research/general/15058/ the DMP every six months and any time the Awardee shares a new dataset or changes data standards used. By the end of the project, the DMP must be a complete and accurate record of all data collected or created through the project.

The DMP must demonstrate the project's full compliance with these requirements and all applicable U.S. DOT and federal policies and guidance.

D. Access Requirements

Consistent with federal policy (Open Government Data Act7), awardees must provide public access to data collected or created through the project unless a specific privacy, confidentiality, security, or other valid restriction is identified. This presumption in favor of openness must be reflected in the research project's DMP. Where valid restrictions to public access exist, awardees must provide controlled access to data that is necessary to achieve core research goals. For example, if the purpose of a project is to evaluate effectiveness of a new technology, the awardee must provide U.S. DOT evaluators access to enough data to support a scientifically-sound evaluation. Furthermore, awardees must make final research results and, where possible, sanitized versions of primary data available to the public.

E. Data Storage and Retention

Awardees must store their data in systems that provide an appropriate level of user access, functionality and data life cycle management. For more guidelines on acceptable data storage systems for U.S. DOT, see Guidelines for Evaluating Repositories for Conformance to the DOT

Public Access Plan8.

Awardees may use different systems to store different types of data (e.g., public, non-public, primary, derived, streaming, static). U.S. DOT has designated several public and controlled access data systems that awardees can use to meet their data access and retention requirements, including registering metadata in the ITS DataHub4. Applicants are encouraged to use these shared resources but can propose alternative systems for storing and providing access to data via their draft DMP and supplemental information. If FHWA determines that a proposed system does not provide an appropriate level of user access and functionality and data life cycle management, the awardee must use the systems designated by the U.S. DOT. Wherever possible, these determinations will be made prior to award.

Datasets in ITS DataHub are required to be retained for a minimum of five years in accordance with the ITS DataHub Records Disposition Manual9, NARA schedule number DAA-0406-2014-

000310. Dataset types may include roadway traffic-related data such as vehicle latitude, longitude, heading and speed; travel times on road segments; traffic signal timing plans; rail transit schedules; and weather. This retention period begins when U.S. DOT first receives access to a dataset.

7 https://www.congress.gov/bill/115th-congress/house-bill/4174/text#toc-

H8E449FBAEFA34E45A6F1F20EFB13ED95

8 https://www.congress.gov/bill/115th-congress/house-bill/4174/text#toc-

H8E449FBAEFA34E45A6F1F20EFB13ED95

9 https://www.fhwa.dot.gov/legsregs/directives/orders/hoit-hq_.cfm 10 https://www.archives.gov/files/records-mgmt/rcs/schedules/departments/department-of-transportation/rg-0406/daa-0406-2014-0003_sf115.pdf https://www.congress.gov/bill/115th-congress/house-bill/4174/text#toc-H8E449FBAEFA34E45A6F1F20EFB13ED95 https://www.congress.gov/bill/115th-congress/house-bill/4174/text#toc-H8E449FBAEFA34E45A6F1F20EFB13ED95 https://www.congress.gov/bill/115th-congress/house-bill/4174/text#toc-H8E449FBAEFA34E45A6F1F20EFB13ED95 https://its.dot.gov/data/ https://www.fhwa.dot.gov/legsregs/directives/orders/hoit-hq_.cfm https://urldefense.proofpoint.com/v2/url?u=https-3A__www.archives.gov_files_records-2Dmgmt_rcs_schedules_departments_department-2Dof-2Dtransportation_rg-2D0406_daa-2D0406-2D2014-2D0003-5Fsf115.pdf&d=DwMFAg&c=f4NRRID3zFYDyClb0wZXwA&r=u7RoqcKhr0ZTyUrh9-9vLPNfCjiY1PN51nvbAJRd86I&m=RJHBZQZ681YIadOiU_QhZX08QGI_Sbik_GmJHsmAyOg&s=7h17qK4mdzMLn3SHkGIl4PwNGSvuFmqi2wVm9ZiHQEU&e= https://urldefense.proofpoint.com/v2/url?u=https-3A__www.archives.gov_files_records-2Dmgmt_rcs_schedules_departments_department-2Dof-2Dtransportation_rg-2D0406_daa-2D0406-2D2014-2D0003-5Fsf115.pdf&d=DwMFAg&c=f4NRRID3zFYDyClb0wZXwA&r=u7RoqcKhr0ZTyUrh9-9vLPNfCjiY1PN51nvbAJRd86I&m=RJHBZQZ681YIadOiU_QhZX08QGI_Sbik_GmJHsmAyOg&s=7h17qK4mdzMLn3SHkGIl4PwNGSvuFmqi2wVm9ZiHQEU&e=

F. Data Rights

Consistent with federal policy (M-13-1311), awardees must apply open licenses to data collected or created through the project so there are no restrictions on copying, publishing, distributing, transmitting, adapting, or otherwise using the information. The Open Source Initiative approves open source licenses that are "popular and widely used", a list of which can be found at https://opensource.org/licenses/category12. The ITS JPO uses clauses in FAR 52.227-1413 to acquire unlimited rights to data for the government, though applicants can claim exemptions and propose limited rights data as described in FAR 52.227-1413. Applicants must describe their proposed approach to data rights in their draft DMP, including, where applicable, a detailed rationale for limiting the government's rights to data.

G. Data Standards

Consistent with federal policy (M-13-1311), awardees must use standards to promote data interoperability and openness. Where existing standards are insufficient, awardees must describe in their DMP how they will contribute to the development of new standards.

H. Timing

If new data collection effort is proposed, awardees are encouraged to provide data, via streaming or batch uploads, throughout the period of data collection or creation, as appropriate. The U.S.

DOT encourages awardees to provide sample or test datasets early in a project to refine data dissemination processes and formats, and use automated processes to cut down on the time from data generation to dissemination.

I. Metadata and Documentation

Awardees must provide metadata on all data described in their DMP. Awardees can provide additional metadata beyond the common core fields if desired, and reference more detailed documentation. Consistent with federal policy (M-13-1311), U.S. DOT will provide this metadata to the public via the ITS DataHub4 and other data catalogs to facilitate data discovery, unless a specific privacy, confidentiality, security, or other valid restriction is identified. Awardees can claim such restrictions when they submit their metadata.

The U.S. DOT may request more detailed documentation and data dictionaries where necessary for users to understand and use the data.

J. Project Management Details

The Contractor shall schedule a virtual kickoff meeting with the Contract Officer’s

Representative (COR) within two weeks of the contract award date. Contractor shall present a proposed schedule for the accomplishment of the work activity to the COR for review and comment during the kick-off meeting. The kick-off meeting shall allow for the review team members to participate remotely via webinar format. The Contractor shall prepare all materials and lead the scheduling and set-up of the webinar, with the assistance of the COR. The scope, 11 https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/memoranda/2013/m-13-13.pdf 12 https://opensource.org/licenses/category 13 https://www.acquisition.gov/content/52227-14-rights-data-general https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/memoranda/2013/m-13-13.pdf https://opensource.org/licenses/category https://www.acquisition.gov/content/52227-14-rights-data-general https://www.acquisition.gov/content/52227-14-rights-data-general https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/memoranda/2013/m-13-13.pdf https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/memoranda/2013/m-13-13.pdf https://its.dot.gov/data/ tasks, and deliverables shall be discussed during this meeting. The Contractor shall submit the kickoff meeting minutes one week after the kick-off meeting.

The Contractor shall prepare a Program Management Plan (PMP) that describes the activities required to perform the work per current Project Management Body of Knowledge (PMBOK) guidance14. At a minimum, the PMP shall describe how the Contractor will manage scope, schedule, cost, and risk as well as internal team communications and governance methods, and communications with the COR. The COR and other stakeholders will review the draft PMP and provide comments to the Contractor, who shall incorporate these comments in a revised, final

PMP no more than two weeks after receiving feedback.

The PMP shall be accompanied by a detailed Project Schedule. This is considered to be a logical component of the PMP, although it may be a physically separate electronic file. The

Project Schedule shall list all necessary activities. The Project Schedule shall be updated monthly. The Project Schedule shall describe the following:

• Name of the work activity.

• Expected start and end dates.

• Name of the individual with the primary responsibility for accomplishing the work.

• Dependencies with other work activities in the Project Schedule.

• All deliverables, procurements, or milestones resulting from the work activity.

The Contractor shall also provide monthly progress reports that identify all deliverables and deliverable status (e.g., not initiated, in progress X% complete, draft delivered, in revision X% complete, final delivered, accepted). The monthly report shall contain a narrative of accomplishments by task and projected activities in the next monthly period. The monthly report shall contain the updated Project Schedule. Each monthly report shall contain a schedule risk narrative, a technical risk narrative, and a partnership risk narrative.

III. SPECIFIC RIGHTS RESERVED FOR THE GOVERNMENT UNDER THIS

BAA

The Government reserves specific rights, in addition to rights described elsewhere in this document or by law or regulation, including:

1. The right to award multiple awards, a single award, or no awards.

2. The right to make award without discussions, or to conduct discussions and/or negotiations, whichever is determined to be in the Government’s interest.

3. The right to accept proposals in their entirety or to select only portions of proposals for award.

4. The right to select for award an instrument type that is appropriate to the specific development context, partner relationship, and proposal selected for award.

Instruments types include but are not limited to contracts, grants, or cooperative

14 PMI (2012), A Guide to the Project Management Body of Knowledge, 5th Ed.

agreements.

5. The right to request any additional, necessary documentation upon initial review. Such additional information may include, but is not limited to, a further detailed proposal, budget, and representations and certifications.

6. The right to fund or co-invest in proposals in phases, with options for continued work at the end of one or more of the phases.

7. The right to remove proposers from award consideration should the parties fail to reach agreement on award terms, conditions, and cost/price within a reasonable time, the proposer fails to timely provide requested additional information, or the Government believes it is in its best interest.

IV. INSTRUCTIONS FOR SUBMISSION OF PROPOSALS:

ADMINISTRATIVE INFORMATION: Offerors are required to follow the guidance contained herein. The following sections provide information on proposal format, the submission process, evaluation and funding processes, and other general information. Proposals not meeting the format described in this BAA will not be reviewed.

All administrative correspondence or questions on this BAA should be directed to the

Contracting Officer at the following email address: dewhyte.mcpherson@dot.gov.

FHWA will provide responses to questions received through August 05, 2020 cut-off date.

FHWA may not respond to questions received after the cut-off date.

V. EVALUATION CRITERIA:

The FHWA may select for award approximately three to five proposals.

Evaluations will be performed using the following criteria:

1) Technical Merit

The Government will use the following Technical Subfactors to evaluate the Technical

Merit of the proposal. The following Subfactors are listed in order of descending importance.

• Subfactor 1: Technical Approach

• Subfactor 2: Support for Program Objectives

Subfactor 1: Technical Approach

The proposal demonstrates a sound technical and staffing approach that would successfully deliver cleaned-up research-ready dataset(s) that can be used to develop and calibrate connected/unconnected ADAS/ADS and human driver behavior models. The proposal demonstrates that the datasets and data collection efforts meet the minimum mailto:dewhyte.mcpherson@dot.gov requirements referenced in the Solicitation, namely:

• Discussion of how dataset(s) will be collected (if new data collection effort is proposed).

• Discussion of how data will be cleaned, post-processed, and documented. This includes a discussion of how PII and competitively sensitive data will be scrubbed from the dataset(s) (if applicable).

• Discussion of how data will be delivered to FHWA and made available for public download.

• Clear demonstration of how data variables listed in Solicitation are either contained within or can be derived from the dataset(s).

• Discussion of accuracy of dataset.

• Clear organization and consistent formatting.

• Clear documentation of data contents and formats.

• Clear ownership rights together with rights for public access and use.

Subfactor 2: Support for Program Objectives

• The proposal demonstrates that the dataset(s) and data collection effort(s) proposed for delivery to US DOT will be of value to the goals and objectives of the US DOT and the FHWA Traffic Analysis Tools Program.

• The proposal demonstrates that the dataset(s) and data collection effort(s) will be valuable toward the development of improved models of ADAS/ADS and human driven vehicle driving behavior models (e.g., car-following models, lane changing models, microsimulation models) and outlined in Berg et al. (2020) and

Mahmassani et al. (2018).

2) Fund Availability: Funds availability will be considered in the ultimate award decision.

The Government will evaluate proposals selected for award to determine realistic and reasonableness of proposed prices.

3) Past Performance: Past performance will be evaluated on a “Pass/Fail” basis for the purposes of making a determination of acceptable or unacceptable risk and responsibility.

Experience and Past Performance

• The proposal demonstrates successful and relevant experience in providing well-documented, well-formatted transportation datasets for external use.

• The proposal demonstrates successful and relevant experience in integrating transportation data from diverse sources.

• The proposal demonstrates successful and relevant experience working with transportation datasets, ADAS/ADS data, and vehicle trajectory datasets.

• The proposal demonstrates successful and relevant experience collecting, cleaning, post-processing, and documenting datasets, specifically trajectory datasets.

• The proposal demonstrates successful and relevant experience developing, calibrating, and validating microsimulation models and interpreting their results.

• The proposed personnel have the necessary and relevant qualifications to successfully perform the contract. Note: The term personnel include the

Offeror’s proposed employees, proposed consultants, and proposed subcontractor employees.

• The Government will evaluate the merits of each Offeror’s past performance on the basis of its reputation with its current and/or former customers. The

Government may review all or some of the following areas regarding past performance on projects and/or contracts that are relevant to the work required under this Broad Agency Announcement. The lack of a performance record may result in an unknown performance risk assessment, which will neither be used to the advantage or disadvantage of the Offeror.

o Timeliness;

o Delivery or Performance;

o Problem Responsiveness;

o Quality of Purchased Product or Services;

o Cost Control;

o Technical Support; and o Delivered Quantities.

The Government may evaluate the quality of the Offeror’s past performance based on information obtained from references provided by the Offeror, as well as other relevant past performance information obtained from other sources known to the Government.

Evaluation of past performance will be a subjective assessment based on a consideration of relevant facts and circumstances. It will not be based on absolute standards of acceptable performance. The Government is seeking to determine whether the Offeror consistently has demonstrated a commitment to customer satisfaction and timely delivery of quality services at fair and reasonable prices.

Note: This is used to evaluate how well the Offeror has performed work that is relevant to the work required under the Broad Agency Announcement. The Government will consider the information provided regarding the Offeror, and, as relevant, any predecessor companies, key personnel with relevant experience, and subcontractors proposed to perform major or critical portions of the work. When an Offeror lacks past performance history as an entity, the Government will evaluate the past performance of the Offeror’s principals, predecessor companies, key personnel and major subcontractors in assessing the risk that the Offeror presents in fulfilling the contract requirements.

Offerors who lack any relevant past performance information will be given a neutral rating.

VI. PROPOSAL FORMAT AND CONTENT

The format of the proposal volumes discussed below shall be as follows:

An electronic version of the proposal shall be delivered in PDF format.

Volume I – Technical Proposal

Volume I- Technical shall be 508-compliant.

Text shall be printed using a Times New Roman or Arial font size no less than 12-point.

Page margins (exclusive of headers and footers) shall be a minimum of 1-inch at the top, bottom and on each side. Columns shall not be used as they are difficult to read electronically.

Footnotes, legends, or labels associated with tables or diagrams, and other information that is ancillary to the main text, may be presented in a font size smaller than 12-point font---provided that any such smaller font is fully legible without magnification.

Volume II- Business and Cost/Price Proposal

Offerors shall also deliver an accompanying Microsoft Excel worksheet for any tables that appear in Volume II - Business and Cost/Price. Each table shall appear in a separate worksheet.

All worksheets shall be compiled into a single workbook.

NOTE: If an Offeror chooses to include a Title Page and/or a Table of Contents, these pages will not be counted against the page limit.

VII. PROPOSAL CONTENT AND CONFIGURATION

Each proposal shall be broken out into the volumes and parts described below

Volume Part Page Limit

Volume I – Technical

Part I – Technical Approach 20

Part II – Preliminary Data Management Plan 20

Part III –Team Organization and Staffing

Part IV – Experience and Past Performance 6

Volume II – Business and Cost/Price

Part I – Cost/Price Information

None

Volume I must be no longer than 20 56 pages in length. The contents of any appendices/exhibits shall NOT count against the 20 56-page limit. If items are included in an appendix, which is not explicitly discussed, in the basic proposal, the proposal may not be reviewed. Proposals with

Volume I in excess of 20 56 pages may not be reviewed. Proposals with less than the maximum number of allowed pages will not be penalized. Offerors are encouraged to submit concise, but descriptive, proposals.

Part I: Technical Approach

• Describe in detail the Offeror’s existing dataset(s) and/or new data collection effort being proposed for data clean up and delivery.

o Describe how proposed data set(s), when cleaned up and submitted to the US

DOT, will be of value to the goals and objectives of the US DOT.

o Describe specifically how the data can be used to develop models of connected/unconnected Advanced Driver Assistance Systems (ADAS), Automated Driving System (ADS), and human driver behavior in the presence of

ADAS/ADS. For additional details on what models are needed, please see:

▪ Berg, Ian, Andrew Berthaume, and Rachel James. 2020. Modeling

Capabilities and Data Needs for Assessing the Operational Impacts of

Connected and Automated Vehicles. Federal Highway Administration report. (Attachment 3)

▪ Mahmassani, Hani S., Amr Elfar, Steven E. Shladover, and Zhitong

Huang. 2018. Development of an Analysis/Modeling/Simulation (AMS)

Framework for V2I and Connected/Automated Vehicle Environment.

Available at: https://rosap.ntl.bts.gov/view/dot/39965.

o For each data set the Offeror shall provide a certification of data ownership. This shall include: the name of the organization that owns the data, when and where it was collected, and a brief description.

• For each dataset, deliver documentation that data can be shared openly with the public

(e.g., a signed statement that the Offeror agrees to the Open Data Commons Attribution

License (http://www.opendatacommons.org/licenses/by/) and Terms of Use for this Test

Data Set).

• For Offeror’s proposing existing dataset(s), describe your technical approach to performing the documentation, post-processing, cleaning, and publishing of the dataset planned under this contract. Describe:

o The overall work needed to document the data set(s), including data sources and staffing.

o The necessary tasks that will result in “research-ready” data sets that are fully documented.

o The necessary tasks to post-process datasets to obtain the data variables necessary to build microsimulation modeling components (e.g., car-following and lane-changing models), as described in the Solicitation.

o The necessary tasks to fuse and integrate multiple sources of data (if applicable).

o The necessary tasks to deliver data to FHWA.

o Declare if data is free of (or contains) PII and competitively sensitive data.

o The necessary tasks to ensure that PII and competitively sensitive data are not included in the public dataset(s) (if applicable).

o The necessary tasks to provide FHWA access to PII and competitively sensitive data for internal use only (if applicable).

o The necessary tasks to make the data publicly available through the ITS DataHub or other acceptable public access portal.

o Timeline for completion of each task.

o A discussion on how to ensure and quantify the quality of the data variables within the dataset (e.g., accuracy, reliability). It’s important to note that FHWA does not want pristine datasets that are artificially cleaned of noise. Raw datasets are preferred. If datasets are smoothed or processed, Offeror must provide documentation on what has been done to the data to get it into its delivered state.

https://rosap.ntl.bts.gov/view/dot/39965 http://www.opendatacommons.org/licenses/by/ o Include the data dictionary and/or a data sample (e.g., printed table or figure) with all fields consistently labeled and identified (e.g., format, range, frequency, accuracy, etc.). For each data variable in the data dictionary/sample, supply the source of the data (e.g., which sensor(s) on vehicle were used to collect the data) and how variables were obtained (e.g., directly measured or calculated/estimated).

Discuss where and when data sample was collected.

• For Offeror’s proposing collecting new dataset(s), describe your technical approach to performing the documentation, post-processing, cleaning, and publishing of the dataset planned under this contract. Describe:

o The necessary tasks to collect the dataset(s).

o The overall work needed to document the data set(s), including data sources and staffing.

o The necessary tasks that will result in “research-ready” data sets that are fully documented.

o The necessary tasks to post-process datasets to obtain the data variables necessary to build microsimulation model components (e.g., car-following and lane-changing models), as described in the Solicitation.

o The necessary tasks to fuse and integrate multiple sources of data (if applicable).

o The necessary tasks to deliver data to FHWA.

o Declare whether data is free of (or contains) PII and/or competitively sensitive data.

o The necessary tasks to ensure that PII and competitively sensitive data are not included in the public data set(s) (if applicable).

o The necessary tasks to provide FHWA access to PII and competitively sensitive data for internal use only (if applicable).

o The necessary tasks to make the data publicly available through the ITS Data Hub or other acceptable public access portal.

o Timeline for completion of each task.

o A discussion on how to ensure and quantify the quality of the data variables within the dataset (e.g., accuracy, reliability). It’s important to note that FHWA does not want pristine datasets that are artificially cleaned of noise. Raw datasets are preferred. If datasets are smoothed or processed, Offeror must provide documentation on what has been done to the data to get it into its delivered state.

o Include the data dictionary and/or a data sample (e.g., printed table or figure) with all fields consistently labeled and identified (e.g., format, range, frequency, accuracy, etc.). For each data variable in the data dictionary/sample, supply the source of the variable (e.g., which sensor(s) on vehicle were used to collect the data) and how variables were obtained (e.g., directly measured or calculated/estimated). Discuss where and when data sample will be collected.

Part II – Preliminary Data Management Plan

• Identify the data to be collected, how the data will further the goals of the U.S. DOT, how the data will be made accessible, and how the data will be stored.

Part III – Team Organization and Staffing

• Provide the names of all personnel and the labor category they will occupy as related to this project. Include resumes for proposed personnel. Resumes for Graduate Students need not be included.

Part IV – Experience and Past Performance

• Provide a narrative to describe the proposed team’s experience in performing the activities described in Section II of the solicitation listed above. Offerors must have experience in most, but not all, of the following areas:

o Traffic data collection (e.g., loop detectors, Bluetooth).

o Vehicle data collection (e.g., CAN BUS, radar, LiDAR).

o Vehicle trajectory data collection (e.g., aerial data, instrumented vehicle data, driving simulator data).

o Data cleaning (e.g., identification and correction of errors in the data).

o Data post-processing (e.g., estimating/calculating necessary data variables from data variables measured directly by vehicle sensors).

o Data integration and fusion (e.g., combination and correlation of data from multiple sources).

o Data de-identification (e.g., PII, competitively sensitive data).

o Documentation of the dataset contents and formats.

o Documenting metadata to the data (e.g., quality flags or time/location identifying information).

o Data delivery (e.g., a suitable data storage device or other data delivery method).

o Working with traffic data, ADAS/ADS data, and vehicle trajectory data.

o Working with ADAS/ADS vehicles.

o Microsimulation models (e.g., driver behavior modeling, car-following models, lane-changing models) development, calibration, validation, and interpretation.

• Demonstrate experience in providing well-documented, well-formatted transportation data sets for external use. Demonstrate experience in integrating transportation data from diverse sources.

• Describe past and current activities and projects with…