Task 1212_Report_FHWA-HRT-23-121_CLEAN_I.pdf

Research, Development, and Technology Turner-Fairbank Highway Research Center 6300 Georgetown Pike McLean, VA 22101-2296

PUBLICATION NO. FHWA-HRT-23-121 JANUARY 2024

DREDGE (Disaggregate Realistic Artificial Data Generator)—Design, Development, and Application for Crash Safety Analysis, Volume I

FOREWORD

Data-driven safety analysis models help State and local agencies quantify safety data, identify high-risk roadway features, and predict the effects of proposed safety measures. However, even a model that performs well overall may not accurately represent interactions between variables for a specific location or crash because underlying relationships in the real world are unknown. One proposed solution is to generate realistic artificial data (RAD) with predetermined safety relationships built into them. Since these datasets are known, the RAD can serve as a test bed, indicating how well a model reflects underlying cause-and-effect relationships.

This report details a study performed by researchers working under the Federal Highway Administration’s Exploratory Advanced Research Program. In the study, the researchers created a RAD generation framework on macroscopic and microscopic levels for a diverse selection of roadway facility types and developed a web-based tool. The tool’s framework provides users with the ability to generate RAD for multiple years at macroscopic segment and microscopic trip levels. This report should be of interest to academics and researchers developing crash modification factors or functions and statistical models to determine how the models best represent real-world relationships. This volume is the first in a series. Volume Ⅱ in the series is

FHWA-HRT-23-122.

Brian P. Cronin, P.E.

Director, Office of Safety and Operations

Research and Development

Notice This document is disseminated under the sponsorship of the U.S. Department of Transportation (USDOT) in the interest of information exchange. The U.S. Government assumes no liability for the use of the information contained in this document.

The U.S. Government does not endorse products or manufacturers. Trademarks or manufacturers’ names appear in this report only because they are considered essential to the objective of the document.

Quality Assurance Statement The Federal Highway Administration (FHWA) provides high-quality information to serve Government, industry, and the public in a manner that promotes public understanding. Standards and policies are used to ensure and maximize the quality, objectivity, utility, and integrity of its information. FHWA periodically reviews quality issues and adjusts its programs and processes to ensure continuous quality improvement.

TECHNICAL REPORT DOCUMENTATION PAGE

1. Report No.

FHWA-HRT-23-121

2. Government Accession No.

3. Recipient’s Catalog No.

4. Title and Subtitle DREDGE (Disaggregate Realistic Artificial Data Generator)— Design, Development, and Application for Crash Safety Analysis, Volume I

5. Report Date January 2024

6. Performing Organization Code:

7. Author(s) John Ivan (ORCID: 0000-0002-8517-4354), Shanshan Zhao (ORCID: 0000-0001-5476-6894), Kai Wang (ORCID:0000-0003-1452-4000), Oluwaseun Olufowobi (ORCID: 0009-0001-1339-9095), Naveen Eluru (ORCID: 0000- 0003-1221-4113), Tanmoy Bhowmik (ORCID:0000-0002-0258- 1692), Lauren Hoover (ORCID: 0009-0004-2431-0860), Md I.

Jahan (ORCID: 0000-0002-4056-7816), Sudipta Tirtha (ORCID:

0000-0002-6228-0904), and Mohamad Abdel-Aty (ORCID:

0000-0002-4838-1573)

8. Performing Organization Report No.

9. Performing Organization Name and Address University of Connecticut Connecticut Transportation Institute 270 Middle Turnpike, Unit 5202 Storrs, CT 06269-5202

10. Work Unit No.

11. Contract or Grant No.

693JJ31950017

12. Sponsoring Agency Name and Address Office of Research and Development Federal Highway Administration 6300 Georgetown Pike McLean, VA 22101-2296

13. Type of Report and Period Covered Final report: August 2019–January 2024

14. Sponsoring Agency Code

HRSO-2

15. Supplementary Notes The contracting officer’s representative was Yusuf Mohamedshah (HRSO-2; ORCID: 0000-0003-0105-5559).

16. Abstract Safety analysis primarily focuses on identifying and quantifying the influences of contributing factors to traffic collisions and the consequences of these factors. A practice of relying on observed data only allows relative comparisons between analysis methods and may not lead to determining how well the methods mimic the true underlying crash-generation process, which is often unobserved or known only partially, with varying degrees of certainty. This report describes a study, performed by researchers working under the Federal Highway Administration, to help address this data limitation.

Researchers used two approaches to generate realistic artificial data (RAD): In the macroscopic approach, researchers generated RAD at site level by segment or intersection after generating traffic and roadway characteristics by segment or intersection for various facility types. Crashes were then generated using known model structures, based on these characteristics. The microscopic approach built a high-resolution, disaggregate data-generation process that mimics crash occurrences on road facilities at the trip level and accommodates the influence of a full range of crash-contributing factors to generate crashes. This crash generation employs data describing the generated trips and involves identifying the vehicles involved in the crash, crash location, severity of occupant injuries, and crash type. These generated crashes can be aggregated at any spatial or temporal resolution to estimate and evaluate safety models. The researchers thus developed a RAD generator as a standalone, customizable software application tool that can prepare multiple realizations of RAD. The tool was evaluated using two case studies involving segment crashes and intersection crashes. This volume is the first in a series. Volume Ⅱ in the series is FHWA-HRT-23-122.

17. Key Words Realistic artificial data, crash analysis, RAD generator

18. Distribution Statement No restrictions. This document is available to the public through the National Technical Information Service, Springfield, VA 22161.

https://www.ntis.gov

19. Security Classif. (of this report) Unclassified

20. Security Classif. (of this page) Unclassified

21. No. of Pages

22. Price N/A

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized.

https://orcid.org/0000-0002-0258-1692 https://orcid.org/0000-0002-0258-1692 https://orcid.org/0000-0002-4056-7816 https://orcid.org/0000-0002-6228-0904 https://www.ntis.gov/ ii

SI* (MODERN METRIC) CONVERSION FACTORS

APPROXIMATE CONVERSIONS TO SI UNITS

Symbol When You Know Multiply By To Find Symbol

LENGTH

in inches 25.4 millimeters mm ft feet 0.305 meters m yd yards 0.914 meters m mi miles 1.61 kilometers km

AREA

in2 square inches 645.2 square millimeters mm2 ft2 square feet 0.093 square meters m2 yd2 square yard 0.836 square meters m2 ac acres 0.405 hectares ha mi2 square miles 2.59 square kilometers km2

VOLUME

fl oz fluid ounces 29.57 milliliters mL gal gallons 3.785 liters L ft3 cubic feet 0.028 cubic meters m3 yd3 cubic yards 0.765 cubic meters m3

NOTE: volumes greater than 1,000 L shall be shown in m3

MASS

oz ounces 28.35 grams g lb pounds 0.454 kilograms kg T short tons (2,000 lb) 0.907 megagrams (or “metric ton”) Mg (or “t”)

TEMPERATURE (exact degrees) °F Fahrenheit 5 (F-32)/9 Celsius °C or (F-32)/1.8

ILLUMINATION

fc foot-candles 10.76 lux lx fl foot-Lamberts 3.426 candela/m2 cd/m2

FORCE and PRESSURE or STRESS lbf poundforce 4.45 newtons N lbf/in2 poundforce per square inch 6.89 kilopascals kPa

APPROXIMATE CONVERSIONS FROM SI UNITS

Symbol When You Know Multiply By To Find Symbol

LENGTH

mm millimeters 0.039 inches in m meters 3.28 feet ft m meters 1.09 yards yd km kilometers 0.621 miles mi

AREA

mm2 square millimeters 0.0016 square inches in2 m2 square meters 10.764 square feet ft2 m2 square meters 1.195 square yards yd2 ha hectares 2.47 acres ac km2 square kilometers 0.386 square miles mi2

VOLUME

mL milliliters 0.034 fluid ounces fl oz L liters 0.264 gallons gal m3 cubic meters 35.314 cubic feet ft3 m3 cubic meters 1.307 cubic yards yd3

MASS

g grams 0.035 ounces oz kg kilograms 2.202 pounds lb Mg (or “t”) megagrams (or “metric ton”) 1.103 short tons (2,000 lb) T

TEMPERATURE (exact degrees) °C Celsius 1.8C+32 Fahrenheit °F

ILLUMINATION

lx lux 0.0929 foot-candles fc cd/m2 candela/m2 0.2919 foot-Lamberts fl

FORCE and PRESSURE or STRESS N newtons 2.225 poundforce lbf kPa kilopascals 0.145 poundforce per square inch lbf/in2 *SI is the symbol for International System of Units. Appropriate rounding should be made to comply with Section 4 of ASTM E380.

(Revised March 2003) iii

TABLE OF CONTENTS

CHAPTER 1. INTRODUCTION AND MOTIVATION

CHAPTER 2. LITERATURE REVIEW

CHAPTER 3. MACROSCOPIC APPROACH FRAMEWORK

Roadway Data Generation Crash Data Generation Data Collection Roadway Facility Types Roadway Characteristics Crash Types and Severities Macroscopic Approach Consolidation and Enhancement Roadway Data Generation Inputs Crash Data Generation Inputs

CHAPTER 4. MICROSCOPIC APPROACH FRAMEWORK

Conceptual Framework

Disaggregate Trip Information Generation Module Crash Data Generation Module Crash Aggregation Module

Data Collection SHRP2 NDS Data CRSS Data Chicago Trip-Level Data

Microscopic RAD Module Development Crash Risk Crash Location Crash Type Drivers and Vehicles Crash Severity

Microscopic RAD Module Implementation Crash Risk Crash Location Crash Type Drivers and Vehicles Crash Severity Validation Check

CHAPTER 5. RAD GENERATION TOOL

Macroscopic RAD Datasets Microscopic RAD Datasets

CHAPTER 6. CASE STUDY DOCUMENTATION

RAD Generation and Validation

Generation and Validation Context Data Generation for Case Studies Crash Model Estimation iv

Application To Rural Two-Lane, Two-Way Undivided Segments Application To Urban Four-Leg Signalized Intersections Validation of Crash Count Prediction Empirical Analysis for Evaluating Parameter Stability Discussion of Findings

Driving Simulation Case Study Introduction Literature Review Methodology

CHAPTER 7. SUMMARY AND CONCLUSIONS

REFERENCES

BIBLIOGRAPHY

v

LIST OF FIGURES

Figure 1. Equation. Transition probability matrix for discrete Markov chain Figure 2. Equation. Transition probability matrix with continuous variable Figure 3. Equation. Poisson distribution for crash counts Figure 4. Equation. Crash prediction model given vector of roadway Figure 5. Equation. Crash prediction model with overdispersion accommodated Figure 6. Equation. Crash prediction model with expected crash counts Figure 7. Flowchart. RAD generation using a macroscopic approach Figure 8. Equation. Cramer’s V statistic Figure 9. Equation. SPF adjusted by AFs Figure 10. Equation. Calibration factor formula for crash count generation Figure 11. Flowchart. DREDGE sequential approach I: crash risk to crash type to crash severity to crash location Figure 12. Flowchart. DREDGE sequential approach Ⅱ: crash risk to crash location to crash type to crash severity Figure 13. Flowchart. DREDGE generator development for microscopic approach Figure 14. Chart. Sample variable distribution Figure 15. Equation. Poisson regression model Figure 16. Equation. NB model Figure 17. Equation. MAD Figure 18. Equation. MSPE Figure 19. Equation. Parameter test statistics Figure 20. Photo. UConn driving simulator vehicle Figure 21. Photo. UConn driving simulator control center Figure 22. Photo. Section of urban roadway used in experiment Figure 23. Photo. Section of rural roadway used in experiment Figure 24. Graph. Conflict severity diagram (Laureshyn and Varhelyi 2020) Figure 25. Graph. Speed of participants in scenario 1 Figure 26. Graph. Speed of participants in scenario 2 Figure 27. Graph. Speed of participants in scenario 3 Figure 28. Graph. Speed of participants in scenario 4 vi

LIST OF TABLES

Table 1. Summary of existing literature on RAD generation Table 2. Initial probability table Table 3. Transition probability look-up table Table 4. Crash generation parameters for rural, two-lane, two-way, three-leg unsignalized intersections Table 5. Crash generation parameters for rural, two-lane, two-way, undivided highways Table 6. Summary of developed models Table 7. Descriptive statistics for continuous variables Table 8. Descriptive statistics for categorical variables (datasets 1–10) Table 9. Poisson regression estimates for dataset 10 (1,000 mi) Table 10. NB estimates for dataset 10 (1,000 mi) Table 11. Random parameters—Poisson regression estimates for dataset 10 (1,000 mi) Table 12. Random parameters—NB regression estimates for dataset 10 (1,000 mi) Table 13. Poisson univariate estimates for dataset 10 (1,000 mi) Table 14. Multivariate Poisson lognormal estimates for dataset 10 (1,000 mi) Table 15. Model prediction using dataset 10 (1,000 mi) for estimation Table 16. Cross validation with dataset 5 (1,000 mi) with parameters estimated using dataset 10 Table 17. Model prediction with dataset 5 (1,000 mi) used for estimation Table 18. Cross validation with dataset 10 (1,000 mi) with parameters estimated using dataset 5 Table 19. NB estimates for PDO crashes Table 20. Revised Wald test statistics on NB model parameter estimates (relative to dataset

10) Table 21. Descriptive statistics for continuous variables Table 22. Descriptive statistics for categorical variables (datasets 1–10) Table 23. Summary of developed models Table 24. Poisson regression model estimation (1,000 intersections) Table 25. NB model estimation (1,000 intersections) Table 26. Random parameter Poisson regression (1,000 intersections) Table 27. Random parameter—NB regression (1,000 intersections) Table 28. Model prediction with dataset 10 (5,000 intersections) used for estimation Table 29. Cross validation with dataset 5 (5,000 intersections) using dataset 10 model Table 30. Model prediction with dataset 5 (5,000 intersections) used for estimation Table 31. Cross validation with dataset 10 (5,000 intersections) using dataset 5 model Table 32. NB estimates for PDO crashes Table 33. Revised Wald test statistics on NB model parameter estimates (relative to dataset 10) Table 34. Driving simulator participant statistics by scenario Table 35. Chart. TA values estimated from vehicle speed and distance to collision point

(Laureshyn and Varhelyi 2020) Table 36. Example of extracted spreadsheet data Table 37. FCW ratios by scenario Table 38. CMF values by FCW fleet penetration rate vii

LIST OF ABBREVIATIONS

AADT annual average daily traffic AASHTO American Association of State Highway and Transportation Officials AF adjustment factor ANN artificial neural network CMF crash modification factor CPF cumulative probability function CRSS Crash Report Sampling System CS conflicting speed CV connected vehicle DOCTOR Dutch Traffic Conflict Technique DREDGE disaggregate realistic artificial data generator FCW forward collision warning FHWA Federal Highway Administration GIS geographic information system GOF goodness of fit GOL generalized ordered logit HSIS Highway Safety Information System MAD mean absolute deviation MDC multiple discrete-continuous MMNL mixed multinomial logit MNL multinomial logit MPR market penetration rate MSPE mean-squared prediction error NB negative binomial NDS Naturalistic Driving Study NHTSA National Highway Traffic Safety Administration OL ordered logit POLARIS Polychotomous Choice Agent-Based Risk Model for Integrated Travel Demand and Network and Operations Simulation RAD realistic artificial data SHRP2 Strategic Highway Research Program 2 SPF safety performance function STCT Swedish Traffic Conflict Technique TA time to accident UConn University of Connecticut USTCT U.S. Traffic Conflict Technique

CHAPTER 1. INTRODUCTION AND MOTIVATION

Safety analysis primarily focuses on identifying and quantifying the influences of factors that contribute to traffic collisions and the consequences of these factors. A traditional analysis paradigm that relies on observed data only allows relative comparisons between analysis methods and lacks the ability to show how well the methods mimic the true underlying crash-generation process. This process is often unobserved or known only partially, with varying degrees of uncertainty. At the same time, existing data sources and the availability of data for model calibration and validation pose significant challenges to safety performance and crash modification analysis. Most safety performance analysis employs cross-sectional and time series datasets. Researchers make assumptions about the data, but whether these assumptions truly characterize the safety data that are generated in the real world often remains unknown.

One possible solution to address this issue is for researchers to artificially generate realistic artificial data (RAD) by making assumptions about the underlying crash generation process (Bonneson and Ivan 2013). These generated RAD can then provide a context for investigating various questions and verifying assorted assumptions related to safety performance and crash modification analyses. The idea is that these RAD can potentially be aggregated at any spatial or temporal resolution to mimic data from the real world. Furthermore, because researchers generating RAD have complete knowledge of the data-generation process, they can compare the performance of varied safety analysis methods.

In this study, the research team proposed two approaches to generate RAD: a macroscopic and a microscopic approach. In the macroscopic approach, researchers generated RAD at the site level by segment or intersection. Roadway sites that contained traffic and roadway characteristics— such as annual average daily traffic (AADT), lane width, shoulder width, curvature, and speed limit—were generated first, by facility type. The team then generated crashes using known model structures (e.g., Poisson and negative binomial (NB) models), based on generated site-level characteristics. Meanwhile, the microscopic approach built a high-resolution disaggregate data-generation process that mimicked crash occurrences on road facilities at the trip level and accommodated the influence of a full range of crash-contributing factors.

The microscopic approach employed detailed information for trips, including start and end time, start and end location, characteristics (including solo or group), vehicle used, and precise route.

The crash generation process involved employing gathered data to identify vehicles involved, location, injury severity (including fatal, incapacitating, non-incapacitating, and uninjured), and crash type (including head-on, rear-end, and vehicle-pedestrian). The generated crashes were further aggregated based on location (intersections versus segments) and time (day, week, month, and year).

Next, the research team conducted two types of case studies to supplement RAD generation and validate the proposed RAD frameworks. The first case study estimated different types of statistical models, using datasets generated by the RAD procedure developed in this study, to illustrate how RAD could be used to make comparisons between analysis methods. The second case study used driving simulation to evaluate the effects of an advanced forward collision warning (FCW) system on crash occurrence.

The three objectives of this study were as follows:

1. Develop frameworks for RAD generation that can be used to evaluate methods used for both safety performance and crash modification analysis: The proposed RAD framework operates at both the site and trip level and incorporates the influence of a full range of crash-contributing factors. RAD frameworks are general enough to generate crash data for all roadway facility types, including segments and intersections. The frameworks can also generate data for different combinations of inputs, including modeling methods, model formulation, input specification, and unobserved heterogeneity. Researchers specifically focused on generating RAD to address known knowledge gaps related to aggregation issues, statistical and structural methodologies for safety analysis, and crash data challenges.

2. Develop DREDGE as a stand-alone software application: The application is customizable and can be executed to prepare multiple realizations of RAD. The application embeds features so RAD can be distributed efficiently; this embedding ensures that the integrity of the RAD process for evaluating methods is maintained. The stand-alone RAD software can also be used to study the uncertainty and stochasticity inherent to RAD generation and to evaluate model robustness.1

3. Demonstrate proposed RAD tool feasibility and applicability using two types of case studies: The first case study estimates different types of statistical models using the RAD generated from this study to demonstrate how the RAD can be used to make comparisons between analysis methods. The second case study uses driving simulation to evaluate the effects of an advanced FCW system on crash occurrence.

1FHWA. DREDGE (standalone RAD software).

CHAPTER 2. LITERATURE REVIEW

The research team conducted a comprehensive review of previous research efforts on RAD approaches across various domains: statistics, econometrics, computer science, ecology, medicine, and psychology. In all these disciplines, the primary goal was to assess the ability of analysis methods to draw inferences about the underlying assumptions and assertions that generated the data. Researchers followed criteria to select studies for this review based on a simple core principle of RAD generation. The data generated in the research effort had to be based on a framework that was built on research assumptions—as opposed to real observed data-based simulation efforts.

The criteria eliminated two major sets of transportation studies that generate simulated data.

First, several travel demand modeling forecast systems, such as activity-based models and synthetic population generators, generate individual-level synthetic data—for example, Eluru, Bhat, and Hensher (2008), Kitamura et al.(2000), and Konduri et al.(2016). However, the generation is entirely based on models estimated using observed data. Second, artificial data are generated in microsimulation frameworks for traffic flow modeling. In these studies, the simulated data are generated based on well-calibrated traffic flow models—for example, Asano, Iryo, and Kuwahara (2010); Mamun et al. (2020); Ranade, Sadek, and Ivan (2007); and Yu and Abdel-Aty (2014). Hence, these studies are also not appropriate for this review.

During the review process, the research team identified 30 research studies (based on the RAD generation criteria) that employed artificial data generation in their analyses. These studies included transportation (including transportation safety and travel behavior), medical science, data science, education, ecological modeling, information analytics, and environmetrics. The research team then prepared a summary of the literature review with the objective of developing a comprehensive RAD framework. Thus, as opposed to providing a study-by-study summary of earlier research, this review provides insight on the important elements of RAD frameworks that can be observed from earlier research efforts.

A concise summary of earlier research efforts on RAD generation is presented in table 1. This table provides information on study objectives, dataset adopted and study region (if known), software and procedures followed for generating RAD, conceptual methods and framework employed, contributing field of the study (for example transportation safety), and exact nature of the dependent variable (categorical or continuous). For the ease of presentation, the studies presented in table 1 are categorized into two groups based on the discipline of the study: first, studies related to transportation and second, studies related to other disciplines, including statistics, economics, ecology, and computer science.

Table 1. Summary of existing literature on RAD generation.

Study Study

Objective(s)

Dataset Adopted (Study

Region if Known)

Software/Procedure for RAD Generation Conceptual Methods Field No. of Alt.

Transportation Domain Bhat (2003)

To propose the use of scrambled Halton sequence for simulation estimation.

Simulated dataset

Generated data using unordered discrete choice models using the matrix programming language GAUSS (Aptech Systems 2023).

Mixed probit and multinomial probit models

Travel behavior

Cummings, McKnight, and Weiss (2003)

To review three methods for estimating relative risks in matched-pair crash data.

Simulated and observed dataset

Generated crash data employing Stata® Statistical Software with an assumed probability of fatality as a function of speed and seatbelt use (StataCorp 2023).

Mantel–Haenszel stratified methods, double-pair comparison method, conditional Poisson regression, and Cox proportional hazards regression

Safety 2

Salim et al.

(2007)

To simulate intersection environment and collision and traffic data learning.

Simulated dataset

Vehicles are generated with different speed, position, and trajectory.

Ubiquitous intersection awareness framework Safety —

Paez and Scott (2007)

To develop a discrete choice model that incorporates elements of social influence and more conventional factors.

Simulated dataset

A Monte Carlo simulation was designed to explore the properties of the econometric model proposed.

Logit probability formulation Travel behavior —

Bhat et al.

(2010)

To propose a CML approach to estimate ordered-response, discreet choice models with flexible, copula-based spatial correlation structures.

Simulated and observed dataset (San Francisco Bay area)

Considered three independent variables. Drew values from univariate normal distribution.

Assumed fixed coefficients.

Generated error terms using correlation structure. Generated 25 different datasets with 500 observations.

Copula-based, spatial ordered-response model structure Travel behavior 4

Study Study

Objective(s)

Dataset Adopted (Study

Region if Known)

Software/Procedure for RAD Generation Conceptual Methods Field No. of Alt.

Bhat and Sidharthan (2010)

To investigate the ability of the

MACML

estimator to recover parameters from finite samples.

Simulated dataset

Considered five independent variables. Drew values from univariate normal distribution.

Assumed random coefficients.

Generated error terms from univariate normal distribution with 0.5 variance. Generated 20 datasets with 5,000 observations.

Cross-sectional random coefficients model, panel interindividual random coefficients model, panel intraindividual and interindividual random coefficients

Travel behavior 5

Pinjari and Bhat (2010)

To investigate nonworker out-of-home discretionary activity time-use and activity timing decisions on weekdays.

Simulated and observed dataset (San Francisco Bay area)

Assumed independent variable values were uniformly distributed. Assumed coefficients were nested extreme values. Generated data for 2,500 hypothetical individuals, with an assumption that each individual chose the value to maximize the total random utility.

MDCNEV Travel behavior 3

Ferdous et al.

(2010)

To model interactions in nonwork activity decisions across household and non-household members at the level of activity generation.

Simulated and observed dataset

Drew values for independent variables from univariate normal distribution. Assumed a fixed coefficient and used it to compute the utility for each individual using a linear combination. Generated error terms using predefined correlation structure. Repeated the process at least 50 times.

Multivariate, ordered-response system framework Travel behavior 3, 4, 5

Ye and Lord (2011)

To examine the effects of underreporting crash data.

Simulated and observed dataset (Texas)

Weighted exogenous sample maximum likelihood estimator method was used for appropriately weighting the crash outcomes to address the underreporting issue in crash data.

MNL, OP, and ML models Safety 5

Study

Objective(s)

Dataset Adopted (Study

Region if Known)

Software/Procedure for RAD Generation Conceptual Methods Field No. of Alt.

Geedipally, Lord, and Dhavala (2012)

To apply an NB, generalized linear model with Lindley mixed effects to the analysis of traffic crash data.

Simulated and observed dataset (road segments in Indiana, Michigan)

NB NB-Lindley generalized linear model Safety 0 to α

Lord and Kuo (2012)

To examine the effects of site selection criteria.

Simulated dataset

Used R software to generate sites with crash counts with a predefined overall mean for different dispersion parameters (R Foundation 2021).

Compared four types of before and after studies: Naïve method, control group method, EB method based on the method of moment, and EB method based on a control group.

Safety —

Bhat et al.

(2013)

To apply the

MACML

approach for multiple MDCP models.

Simulated dataset (Michigan)

Considers five independent variables. Draws values from univariate normal distribution.

Generates error terms from positive covariance matrix.

Undertakes data generation process 20 times with different realizations of coefficient and error terms.

MACML Travel behavior 5, 10

Eluru (2013)

To investigate the performance of ordered and unordered injury severity response frameworks.

Simulated dataset

Considered three independent variables. Assumed parameters that provide the same aggregate shares. Generated 5 realizations of the data with 5,000 observations each for each proportional value. Generated a total of six aggregate sample shares.

MNL, OL, and GOL Safety 4

Paleti and Bhat (2013)

To compare the MSL inference and CML approaches.

Simulated dataset

Drew independent variables from univariate normal distribution while coefficients were assumed and drawn from multivariate normal distribution. Considered both independent and correlated realizations. Generated data at least 50 times.

CML and MSL approach Travel behavior 5

Study

Objective(s)

Dataset Adopted (Study

Region if Known)

Software/Procedure for RAD Generation Conceptual Methods Field No. of Alt.

Wu, Lord and Zou (2015)

To generate CMFs using a regression model to estimate the crash counts and compare with the actual crash distribution

Simulated dataset

Assumed CMF values for lane width, curve density, and pavement friction and used them to generate simulated crash counts.

Evaluated the conditions for adopting NB regression models for before after studies. Safety 0 to α

Council et al.

(2017) 2

To use ARD to assess the performance of cross-sectional analysis methods.

RAD (rural two-lane highways, Washington)

Implemented data generation by SAS® programs based on an assumed model structure for AADT and roadway geometry factors.

All available methods of crash count and severity analysis Safety 0 to α

Nontransportation Domain

Gamel and Vogel (1997)

To compare parametric and nonparametric survival methods.

Simulated clinical dataset

Estimated three parametric models using data from breast cancer trials. Assumed the relationships generated survival data in HT Basic (Transera 1991).

Parametric (log-normal) and nonparametric test (logrank test, Gray-Tsiatis and Laska-Meisner methods)

Medical science 0 to α

Scott and Wilkins (1999)

To evaluate data mining procedures.

Artificial dataset

Proposed two alternative ways of generating artificial data for testing data mining approaches.

Self-similarity, classification, and lazy trees Data science 0 to α

Bifulco and Bretschneider (2001)

To compare methods for assessing school performance.

Artificial dataset

Generated datasets from log-linear relationships between three inputs and two outputs and associated parameters.

Generated datasets without bias, with measurement error and endogeneity.

Data envelopment analysis and corrected ordinary least squares Education —

Austin et al.

(2006)

To evaluate statistical methods for predicting plant species distributions.

Artificial dataset

Generated data based on two plant community theoretical models using the computer package COMPAS (Minchin 1987).

Generalized linear models and generalized additive models Ecological modeling 0 to α

2Council, F., E. Hauer, B. Lan, D. Harwood, and R. Srinivasan. 2017. Use of “Artificial Realistic Data” (ARD) To Assess the Performance of

Cross-Sectional Analysis Methods in Capturing Causal Relationships Between Individual Roadway Attributes and Safety. Unpublished Report. Washington, DC:

Federal Highway Administration.

Study

Objective(s)

Dataset Adopted (Study

Region if Known)

Software/Procedure for RAD Generation Conceptual Methods Field No. of Alt.

Bzdusek and Christensen (2006)

To compare a new variant of PMF with other receptor modeling methods.

Artificial dataset

Generated data set using literature source profiles and postulated source contributions.

Eigenvalue-based methods and PMF-based methods Environmetrics —

Whiting, Hack, and Varley (2008)

To describe a method and toolset for creating realistic, synthetic test data.

Realistic simulated dataset

Generate simulated datasets by embedding ground truths. Core software and utility software were written in JAVA® (Arnold, Goslin, and Holmes 2005).

Threat stream generator Information analytics —

Potharst, Ben- David, and Van Wezel (2009)

To generate monotone ordinal datasets.

Observed and artificial dataset

Used a machine-learning algorithm to generate the data.

Algorithms for generating structured and unstructured random monotone datasets Data science —

Zimmermann (2012)

To generate diverse datasets reflecting realistic data characteristics.

Artificial dataset

Implemented data generator in JAVA. Episode mining Data science —

Devroye, Felber, and Kohler (2012)

To estimate a density using real and artificial data.

Observed and artificial dataset

Implemented data generator in R software. Generated artificial data from a regression analysis of observed data.

Classical model, finite information model, and full information model Data science 0 to α

Hazwani et al.(2016)

To develop an automatic artificial data generator for generating artificial datasets based on real data.

Artificial and real dataset

Used random permutation algorithm to generate different sets of artificial data that represent realistic data.

Four-phase framework for data generation Information and communication technologies

Dahmen and Cook (2019)

To introduce a synthetic data generation method.

Simulated and real dataset

SynSys, a machine learning-based synthetic data-generation method.

SynSys, similarity measures, and semisupervised learning Medical science —

Alt. = alternatives; CML = composite marginal likelihood; CMF = crash modification factor; no. = number; EB = empirical Bayes; MACML = maximum approximate CML; MDCNEV = multiple discrete-continuous nested extreme value; MDCP = multiple discrete-continuous probit; ML = mixed logit;

MSL = maximum-simulated likelihood; OP = ordered probit; PMF = Positive matrix factorization.

*After a concept proposed by Hauer, as presented in Harwood et al. (2003).

https://onlinelibrary.wiley.com/journal/1099095x

Researchers made five important observations based on the information seen in table 1. First, earlier research explored RAD applications for wide-ranging topics, including statistical/econometric model performance and comparison, travel-demand forecasting, route-choice behavior, and data mining. Second, RAD applications have typically been developed using several software packages or platforms, including R, GAUSS, and COMPAS (R Foundation 2021; Aptech Systems 2023; Minchin 1987). Third, employing RAD datasets, researchers considered the performance of several model structures, including ordered logit (OL), multinomial logit (MNL), generalized ordered logit (GOL), mixed multinomial logit (MMNL) models, probit models (and their cross-sectional and panel variants), multiple discrete-continuous (MDC) frameworks with probit and extreme value formulations, and artificial neural networks. Fourth—notably—studies within the transportation domain traditionally adopt RAD approaches for econometric models. However, non-transportation domain research typically is more focused on machine-learning and data-mining approaches. Fifth, the number of alternatives in the RAD variables studied is related to the problem context. The number of alternatives for a RAD variable could range from a small number to a very large number. If the RAD variable is a binary outcome variable, such as crash/no crash, then the number of alternatives will be two.

However, if the RAD variable is a continuous value (such as vehicle miles traveled by a household), then the number of possible alternatives is, theoretically, infinite.

The literature review clearly highlights the absence of a single or prevalent software framework for developing RAD across various application domains. Several computer programming languages, matrix programming languages, and statistical software packages have been employed for developing RAD-based frameworks. Thus, RAD framework conceptualization can be platform-agnostic. This fact is particularly beneficial in the current context because no inherent limitations exist in adopting an open-source software platform for RAD implementation.

Thus, the framework developed can be widely deployed by Federal Highway Administration (FHWA) without any constraints.

The authors of the report noted that the most common experimental set-up for RAD generation seen in this review was a statistical or econometric model framework assumed to represent the data-generation process of the variable of interest. Within the model framework, independent variables (either based on observed data or random realizations) and their impact (defined by coefficient values) on the dependent variable are assumed. Using the independent variable distribution and coefficient values, a latent propensity (or utility) is computed based on the model system. For example, if the dependent variable is a count variable and the model system is an NB system, a single propensity is generated with an appropriate overdispersion component.

However, if the model system represents an unordered discrete outcome model, alternative specific utilities are generated. After the model structure is assumed, a random error term is added to (or multiplied by) the propensity or utility generated. Subsequently, a choice scenario is determined for each record based on the properties of the model. This choice scenario results in the formation of the RAD dataset. The process is repeated several times to generate multiple copies of the datasets. The choice variable of interest, such as crash occurrence, may possibly be a result of several layers of decisions, such as trip, route, and time-of-day decisions. The research team’s vision was to use a similar approach to this framework to generate RAD after the conceptual framework for the study was finalized.

Additionally, researchers observed during their literature review that the embedded RAD frameworks were consistently single-level frameworks; in other words, the underlying decision process consisted of only one layer of decisions. Earlier research in modeling crash occurrence related crash occurrence to roadway geometry and traffic volume under prespecified assumptions of what variables would influence crash occurrence (for example, AADT and lane width). This study’s research effort was the first to attempt the development of RAD datasets using a multilayered decision process; as such, researchers anticipated the effort would be challenging.

Hence, the research team was cautious in determining the number of decisions—and thus the number of layers—that could be modeled to determine the crash dependent variables in this study. As the study commenced in full swing, the research team incorporated multilayered complexity within the decision processes to enhance the current state of RAD generation across multiple transportation domains.

CHAPTER 3. MACROSCOPIC APPROACH FRAMEWORK

The macroscopic approach has two steps. The first step is to generate roadway data, such as AADT, truck percentage, lane width, and shoulder width. The second step is to generate crash counts by crash type and severity, based on roadway-level data and statistical and econometric methodologies.

ROADWAY DATA GENERATION

The basic philosophy for generating roadway data follows the principle of the Markov chain (Gagniuc 2017). Given a sequence of random variables X1, X2, X3, … , Xn (where X represents a vector of roadway characteristics), the indices (1, 2, 3, …, n) represent the walking state in time (where the indices represent the contiguous sites for roadway network). The Markov chain assumes that the roadway characteristics in the next state depends only on the previous state. A higher probability may exist that the values for these roadway characteristics on a given site will remain the same as those of the previous site, with lower probabilities that the values will change to other possible combinations.

Overall, a transition probability matrix is required by the Markov chain principle. This matrix presents the probabilities for possible roadway characteristics on one site, given the roadway characteristics on the previous site. Two types of transition probability matrices are included in the Markov chain principle: one matrix is for a discrete Markov chain, and the other matrix is for a continuous Markov chain. The discrete Markov chain transition probability matrix is applied to the categorical variables in this study—such as lane width, shoulder width, and speed limit—and is illustrated in figure 1, which uses numbers that are randomly created to provide a visual example.

Figure 1. Equation. Transition probability matrix for discrete Markov chain.

Where:

I = 1, 2, 3, ….

n = the ith combination of roadway characteristics, such as 1 = 12-ft lane width, 6-ft shoulder width, and 55-mph speed limit while 2 = 11-ft lane width, 4-ft shoulder width, and 45-mph speed limit.

Pi = the ith combination of roadway characteristics for the previous site.

Ci = the ith combination of roadway characteristics for the subject site.

A probability under row Pi and column Cn in the transition probability matrix = the probability of roadway characteristics to be the nth combination for the subject site, given the ith combination of roadway characteristics for the previous site.

Note that the probabilities in each row and column must sum to unity in figure 1.

When a variable is continuous, such as AADT or truck percentage, the transition probability matrix is usually represented by a distribution, which can be written as shown in figure 2.

Figure 2. Equation. Transition probability matrix with continuous variable.

That is, the probability distribution of a roadway characteristic for the subject site follows a normal distribution, with a mean equal to the roadway characteristic for the previous site and a predefined variance σ2.

To illustrate the theoretical process of roadway data generation using the Markov chain, one must first generate an initial probability matrix. This initial probability matrix is used to define the first site in the RAD, since this site cannot be defined from the transition probability matrix in the Markov chain. Table 2 shows a hypothetical probability distribution lookup table, which is used to generate all combinations of roadway characteristics for the first roadway site. For example, suppose four combinations of roadway characteristics are generated in the RAD, such as different combinations of values for lane width, shoulder width, and speed limit. A random number between 0 and 1 can be generated and compared to the cumulative probability in table 2 to define the roadway characteristics for the first site. For example, if the random number generated is equal to 0.8, then the roadway characteristics following under combination C will be set for the first site.

Table 2. Initial probability table.

Roadway Characteristics Initial Probability Cumulative Probability Combination A 0.25 0.25 Combination B 0.30 0.55 Combination C 0.35 0.90 Combination D 0.10 1.00

The next step is to generate all remaining sites based on the first site and the transition probability matrix using the Markov chain principle. First, generate a transition probability matrix from the real data, with the transition probabilities listed in a look-up table (table 3).

Following this first step, the first site has roadway characteristics of combination C. A random number between 0 and 1 can now be generated for the second site, and the random number can be compared to the cumulative probability in table 3, where the roadway characteristics for the previous site follow combination C to determine the roadway characteristics for the subject site.

For example, if the random number generated is equal to 0.6, the roadway characteristics for the subject site will stay the same as they were. If the random number generated is equal to 0.95, then the roadway characteristics for the subject site will change to those values under combination D. This process will be repeated until the last site is generated.

Table 3. Transition probability look-up table.

Roadway Characteristics for

Previous Site Roadway Characteristics for Subject Site Transition Probability

Cumulative Probability

Combination A

Combination A 0.70 0.70 Combination B 0.10 0.80 Combination C 0.10 0.90 Combination D 0.10 1.00

Combination B

Combination A 0.15 0.15 Combination B 0.75 0.90 Combination C 0.05 0.95 Combination D 0.05 1.00

Combination C

Combination A 0.10 0.10 Combination B 0.15 0.25 Combination C 0.65 0.90 Combination D 0.10 1.00

Combination D

Combination A 0.05 0.05 Combination B 0.05 0.10 Combination C 0.05 0.15 Combination D 0.85 1.00

CRASH DATA GENERATION

After generating the roadway data, crash counts by crash type and severity are generated using the known model structures (for example, Poisson and NB models) and “realistic” relationships between crash counts and roadway-level characteristics. Details about how to define the “realistic” relationships between crashes and roadway variables are discussed in section “Crash Data Generation Inputs” below.

Suppose a crash prediction model or safety performance function (SPF) exists for the sites generated in the roadway data-generation process, and this model or SPF contains the coefficients for the relationships between crash counts and all roadway characteristics and other model-related parameters. This model or SPF can then be used to estimate the parameters of the assumed underlying distribution for each site and then simulate the crash count as a random variable. For example, using the two commonly used model frameworks, the Poisson and NB models, to illustrate the process. The Poisson distribution for the crash counts can be written as shown in figure 3.

Figure 3. Equation. Poisson distribution for crash counts.

Where:

Prob[yi|μi] = probability of y crashes occurring at site i.

μi = expected number of crashes at site i.

Given the vector of roadway site characteristics Xi and the vector of coefficients β in the crash prediction model, the expected crashes μi can be predicted as shown in figure 4.

Figure 4. Equation. Crash prediction model given vector of roadway.

If the overdispersion is accommodated by the crash prediction model, then the NB model is used to estimate the vector of coefficients ββ, and the expected crashes 𝜇𝜇𝑖𝑖 can be predicted as shown in figure 5.

Figure 5. Equation. Crash prediction model with overdispersion accommodated.

where:

exp(εi) = error term assumed to follow gamma distribution with mean 1 and variance Error!

Digit expected.1/σ = k.

k denotes the overdispersion parameter in the NB model.

Given the expected crashes 𝜇𝜇𝑖𝑖, the distribution of crash counts in the NB model can be calculated as shown in figure 6.

Figure 6. Equation. Crash prediction model with expected crash counts.

Where Γ is the common gamma function.

Regardless of which statistical and econometric model is used to estimate the expected crash counts μi for each site, its probability distribution function can be used to calculate both the probabilities and cumulative probabilities of observing 0, 1, 2, and up to yi crashes, respectively, for each site. For example, figure 3 and figure 6 illustrate the functions for Poisson and NB, respectively. Then, the actual crash counts can be determined for each site using a random variable generation procedure similar to the roadway data generation. For example, a random number between 0 and 1 can be generated and compared to the cumulative probabilities of different crash counts to define the final crash counts.

DATA COLLECTION

The research team collected data from multiple sources to support generating the transition probability matrix for the Markov chain principle in the macroscopic approach. Specifically, the research team collected the data of all seven States in the current Highway Safety Information System (HSIS), including California, Illinois, Maine, Minnesota, North Carolina, Washington, and Ohio (FHWA n.d.). Additionally, the research team collected extra data from the Ohio, Connecticut, and Florida departments of transportation (DOTs) to supplement the HSIS data (Ivan et al. 2021).

ROADWAY FACILITY TYPES

In this study, the research team attempted to cover RAD generation for as many types of roadway facilities as possible, based on the facility categorizations in the current Highway Safety Manual (HSM) (American Association of State Highway and Transportation Officials (AASHTO) 2010). However, the team decided (based on experiences in other safety-related studies) that freeway ramp and ramp terminal data would be too challenging to attempt to obtain.

Therefore, the team mainly focused on all the HSM roadway facility types except these two ramp types. However, the team also omitted the following facility types from consideration due to too-small sample sizes in their data: urban and suburban three-lane arterials with a center two-way, left-turn lane; urban and suburban five-lane arterials with a center two-way, left-turn lane; and rural eight-lane freeway segments. Specifically, the research team considered the following facility types:

• Rural two-lane, two-way roadways:

o Two-lane,…