Draft SOW_ 308 OlderNoviceNDS_(693JJ923RQ000179).pdf

OLDER NOVICE DRIVER NATURALISTIC DRIVING STUDY

SECTION C – DESCRIPTION/ SPECIFICATIONS/ WORK STATEMENT

C.1 BACKGROUND

The National Highway Traffic Safety Administration (NHTSA) is an agency of the U.S.

Department of Transportation (DOT). NHTSA’s mission is to save lives, prevent injuries, and reduce economic costs due to road traffic crashes, through education, research, safety standards, and enforcement activity.

In 2020, drivers ages 15 to 20—many of whom were novices—represented 8.5% of drivers involved in fatal crashes but only 5.1% of all licensed drivers.1 While young novice drivers’ crash rates have declined since States began implementing Graduated Driver Licensing (GDL) programs in the 1990s,2 motor vehicle crashes remain a leading cause of death among young people.3 As a group, novice drivers’ crash rates are highest during the initial months of licensure and decline over time.4 However, individual novices may not follow this pattern of incrementally declining risk.5 For example, when novices were grouped by longitudinal profiles (“trajectories”) of risky driving over the first 20 weeks of independent driving, some novices drove safely throughout this period while others remained at consistently high risk.6

One factor that may influence heterogeneity of risk among novice drivers is the age at which they receive licensure and begin driving independently. Crash rates are generally highest for novice drivers at first licensure and decline with experience.4 However, the declines among older novices appear to be slower than for those licensed at younger ages,7 and some evidence suggests that drivers first licensed at age 18 may have higher crash rates during their first several months of driving than new drivers first licensed at ages 16 or 17.8,9

1 National Center for Statistics and Analysis. (2022, June). Young drivers: 2020 data (Traffic Safety Facts. Report No. DOT HS 813 313). National Highway Traffic Safety Administration.

https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/813313 2 McCartt, A. T., & Teoh, E. R. (2015). Tracking progress in teenage driver crash risk in the United States since the advent of graduated driver licensing programs. Journal of Safety Research, 53, 1-9.

3 Webb, C. N. (2018, February). Motor vehicle traffic crashes as a leading cause of death in the United States, 2015 (Traffic Safety Facts Crash Stats. Report No. DOT HS 812 499). National Highway Traffic Safety Administration.

https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812499 4 E.g., Mayhew, D. R., Simpson, H. M., & Pak, A. (2003). Changes in collision rates among novice drivers during the first months of driving. Accident Analysis & Prevention, 35, 683-691.

5 Mirman, J.H., Curry, A.E., & Mirman, D. (2019). Learning to drive: A reconceptualization. Transportation Research Part F: Traffic Psychology and Behaviour, 62, 316-326.

6 Missikpode, C., Peek-Asa, C., McGehee, D. V., & Wallace, R. (2019). Classifying and predicting risky driving among novice drivers: A group-based trajectory approach. Journal of Safety Research, 68, 215-222.

7 Curry, A. E., Pfeiffer, M. R., Durbin, D. R., & Elliott, M. R. (2015a). Young driver crash rates by licensing age, driving experience, and license phase. Accident Analysis & Prevention, 80, 243-250.

8 Chapman, E. A., Masten, S. V., & Browning, K. K. (2014). Crash and traffic violation rates before and after licensure for novice California drivers subject to different driver licensing requirements. Journal of Safety Research, 50, 125–138.

9 Walshe, E. A., Romer, D., Wyner, A. J., Cheng, S., Elliott, M. R., Zhang, R., Gonzalez, A. K., Oppenheimer, N., & Winston, F. K. (2022). Licensing examination and crash outcomes post-licensure in young drivers. JAMA Network Open, 5(4), e228780.

Furthermore, while an increasing proportion of young people are delaying licensure until age 18 or older,10 few States currently apply the full GDL program to novices ages 18 to 20.11 Licensure delays are more likely among Latino, Black, and lower socioeconomic status (SES) young people,12 raising the possibility of inequities in which novices receive the benefits of GDL.

However, because little is known about the safety and driving habits of newly-licensed drivers 18 and older, questions remain about whether and how to develop GDL provisions for older novices that reduce exposure to risk while still ensuring mobility.

The heterogeneity of risk among novice drivers has also led to a call to “move beyond” population-level interventions towards interventions tailored for novices with higher levels of risk or who have already exhibited unsafe driving behaviors.13 Therefore, it is critical to understand which novice drivers are likely to begin, and to remain, at greatest risk during the first months of independent driving. Conversely, understanding which novices are likely to have persistently low risk—and whether their early independent driving experiences differ in ways that promote learning but preserve safety—can be useful for those developing recommendations and education for newly-licensed drivers. Yet, few demographic or psychological factors have emerged as significant and consistent predictors of whether novices follow high- or low-risk trajectories of unsafe driving after the transition to independent driving.6,14 Similarly, it is unknown whether the amount, type, or patterns of driving in the first months of independent driving differ, or evolve differently over time, for high- and low-risk novice drivers.

To address these questions, a prior NHTSA project (https://rip.trb.org/View/1747331) involved designing, and determining the potential challenges for conducting, a hypothetical naturalistic driving study (NDS) with younger (age 15.5 – 16.5) and older (age 18 – 20) novice drivers in the first period of independent driving. Under this project, a research team developed materials related to the design and execution of the hypothetical study, including:

• A literature review of existing research relevant to the research questions and potential methodological approaches for the hypothetical study;

• A study design and data analysis plan;

• Questionnaires to be administered at various timepoints during the hypothetical study;

• Materials required for submission to an Institutional Review Board (IRB);

• A data management plan (DMP);

• Participant consent forms;

10 Tefft, B. C., Williams, A. F., & Grabowski, J. G. (2014). Driver licensing and reasons for delaying licensure among young adults ages 18-20, United States, 2012. Injury Epidemiology, 1, 4.

11 Insurance Institute for Highway Safety. (2022, August). Graduated licensing laws by state.

https://www.iihs.org/topics/teenagers/graduated-licensing-laws-table 12 Vaca, F. E., Li, K., Tewahade, S., Fell, J. C., Haynie, D. L., Simons-Morton, B. G., & Romano, E. (2021a).

Factors contributing to delay in driving licensure among U.S. high school students and young adults. Journal of Adolescent Health, 68(1), 191 – 198.

13 Winston, F.K., Puzino, K., Romer, D. (2016). Precision prevention: Time to move beyond universal interventions.

Injury Prevention, 22, 87-91.

14 Roman, G. D., Poulter, D., Barker, E., McKenna, F. P., & Rowe, R. (2015). Novice drivers’ individual trajectories of driver behavior over the first three years of driving. Accident Analysis & Prevention, 82, 61-69.

https://rip.trb.org/View/1747331

• Information Clearance Request (ICR) materials describing the study for submission to the Office of Management and Budget (OMB) to comply with the Paperwork Reduction Act

(PRA);

• A draft Introduction and Methods sections for a Final Report describing the hypothetical study; and

• An assessment of potential challenges to conducting the hypothetical study, along with solutions.

The research team did not develop materials related to assessing how personally identifiable information would be collected, used, shared, and maintained (e.g., a Privacy Threshold Analysis

[PTA]).

The research team for this prior project found that most existing NDS of novice drivers have not included novices 18 and older. Additionally, although older novices are more likely to be Black, Latino, and lower SES,12 most prior NDS have used convenience samples that overrepresented White and higher SES novice drivers. When designing the study, the research team determined that an alternative approach was necessary to obtain more representative samples of novice drivers. Specifically, the research team devised a recruitment approach in which, along with a partnership with a State driver licensing agency, researchers could increase the likelihood of reaching a diverse group of potential participants. Additionally, the research team suggested a ‘hybrid’ approach to data collection, in which most participants would have NDS data collected via an app on their personal smartphones, while a smaller subgroup would additionally be outfitted with the kinds of in-vehicle data acquisition systems (DAS) traditionally used in NDS.

This approach removes barriers to participation associated with the installation of in-vehicle equipment, permits the recruitment of a larger number of participants, and allows researchers to examine the correspondence between NDS data obtained with smartphones versus in-vehicle equipment.

C.2 PURPOSE

The purpose of this contract is to conduct an NDS to address the following research questions:

1) Do older novices exhibit different trajectories of risky driving in the first 12 monthsof independent driving than younger novices?

2) Do older novice drivers differ from younger novices in the amount, type, or patterns of driving they do during the first period of independent driving? Do they differ with respect to demographic, psychological, or other individual characteristics associated with risky driving?

3) After beginning to drive independently, do most novice drivers exhibit the pattern of initially high but decreasing rates of risky driving? Or, do some groups of novice drivers follow different trajectories—e.g., consistently high or low rates of risky driving?

4) Do novice drivers who consistently engage in, or consistently refrain from, risky driving during the first period of independent driving differ in the amount, type, or patterns of driving they do during this period? Do they differ with respect to demographic, psychological, or other characteristics associated with risky driving?

5) How accurate is naturalistic driving data about novice drivers’ performance and exposure collected using a smartphone app compared to data collected with in-vehicle DAS?

C.3 OBJECTIVE

The objective of this contract is to conduct an NDS with younger (age 15.5 – 16.5) and older (age 18 – 20) novice drivers in the first 12 months of independent driving to answer the research questions in Section C.2. The study design and analyses shall be based on materials developed for this study under a prior project (https://rip.trb.org/View/1747331); these will be shared with the Contractor at award.

The study developed under the prior project has the following characteristics:

• Naturalistic driving data will be collected for a period of twelve (12) months from 500 novices age 15.5 – 16.5 (depending on the minimum licensing age of the State in which the study is conducted) and 500 novices age 18 – 20.

• All participants will have naturalistic driving data collected using a smartphone-based DAS (smartphone app) capable of collecting data necessary to answer the research questions in Section C.2. A subset of 84 younger and 84 older novices (168 total) will also have naturalistic driving data collected using vehicle-based DAS (i.e., with camera, accelerometer, gyroscope, GPS, etc.).

• Naturalistic driving data collection will begin when participants receive the first license that allows them to drive unsupervised.

• Potential participants will be quota sampled to ensure that the study sample aligns with what is known about the sociodemographic characteristics of novice drivers in the U.S.

(e.g., 12). Currently, the sampling plan includes considering sex, race and ethnicity, and socioeconomic characteristics (e.g., family affluence) when recruiting novice drivers.

• The study will be conducted in a State that does not currently apply GDL provisions to novices 18 and older; the State should also have a minimum age for unsupervised driving between 15.5 and 16.5 (i.e., the minimum age in the majority of States).

• To recruit participants for the study, the research team will need to partner with a State licensing agency. The State licensing agency should have sufficient numbers of younger and older novices applying for licensure to reach recruitment targets.

• Potential participants will be identified using information available in the State licensing agency records (e.g., age, sex, race and ethnicity, licensure status). Potential participants will additionally be administered screening questions to further determine eligibility (e.g., ownership of an Android or iOS smartphone, information about vehicle access necessary to determine eligibility for the in-vehicle DAS subgroup) at the time they schedule their on-road exams.

• Based in part on the experience of Thomas et al., 2016,15 the current study plan estimates that a high-volume State licensing agency could yield 3 to 18 participants (including 1 to 6 older novices) per day. Assuming this rate, if 3 individual licensing agency offices

15 Thomas, F. D., Rilea, S. L., Blomberg, R. D., Peck. R. C., & Korbelak, K. T. (2016, January). Evaluation of the safety benefits of the risk awareness and perception training program for novice teen drivers (Report No. DOT HS 812 235). National Highway Traffic Safety Administration participated in the study, it would take a maximum of 167 office days (approximately 8 mos.) to complete recruitment. With 6 offices participating, it would take a maximum of 84 office days (approximately 4 mos.) to complete recruitment.

• The current study plan estimates compensation for novices in the “smartphone only” data collection group to be $475 by the end of the study. The plan estimates the “smartphone plus in-vehicle DAS” data collection group will be compensated $675 by the end of the study.

• Participants will complete three web-based, self-administered questionnaires at the beginning, middle, and end of the study that do not take more than 30 minutes to complete. The baseline questionnaire will contain questions about sociodemographic information and other information that prior studies suggest will predict driving performance and/or exposure, including but not limited to: measures of personality, sensation-seeking, attitudes about traffic safety, self-reported driving behavior, parental involvement, experience with driving and driver education, vehicle access. Follow-up questionnaires will be shorter and only include measures that may change over time and/or with driving experience, e.g., attitudes about traffic safety, vehicle access.

• Naturalistic driving and questionnaire responses will be used to derive variables necessary to answer the research questions in Section C.2, including outcome variables like crashes or near-crashes and kinematic risky driving (i.e., elevated g-force) events.

Other variables will include information about the amount and type of exposure (e.g., road type, trip duration, trip time of day, weather, posted speed limit, route familiarity, complexity of driving environment), information about risky behaviors (e.g., speeding, cell phone use), and information from the questionnaires.

• The current plan includes a variety of statistical techniques for data analysis, including approaches that discover groups of participants with similar trajectories of risky driving over the study period (e.g., group-based trajectory models, latent class growth analysis using growth mixture modeling), as well as approaches that allow group-based comparisons between younger and older novices, or higher- and lower-risk novices (e.g., mixed-effects generalized linear models.

• Potential challenges for conducting the study include: finding a State licensing agency willing to participate and maintain its commitment during the approval process, recruiting a sample representative of the younger (15.5 – 16.5) and older (18 – 20) novice driver populations; recruiting the desired number of participants within the specified amount of time; and obtaining accurate measures of performance and exposure from the smartphone app (e.g., distinguishing travel modes; distinguishing passenger from driver).

C.4 GENERAL REQUIREMENTS

The Contractor shall:

• Review and update study design and data analysis plan;

• Identify options for smartphone and in-vehicle DAS;

• Develop a plan for the pilot study;

• Identify potential study sites;

• Review and update ICR package, IRB package, DMP;

• Develop a PTA;

• Obtain IRB approval for pilot and full studies;

• Conduct pilot study;

• Finalize the study site;

• Conduct full study;

• Prepare a letter report describing study results; and

• Prepare the Final Report and analysis dataset.

This contract also includes two optional tasks:

• Collaboratively prepare a draft manuscript intended for submission to a peer-reviewed journal; and

• Prepare a one-page information sheet translating the study’s results for a specific stakeholder audience that includes an infographic or other visual.

C.5 ACRONYMS

The following is a list of key acronyms used throughout the Statement of Work. This list may not be exhaustive.

ACOR Alternate Contracting Officer’s Representative CO Contracting Officer COR Contracting Officer’s Representative CS Contract Specialist DAS Data acquisition system(s) DMP Data Management Plan DOT U. S. Department of Transportation GIF Graphic Interchange Format GPO Government Printing Office ICR Information Collection Request IRB Institutional review board JPEG Joint Photographic Expert Group MOU Memorandum of Understanding NDS Naturalistic driving study NHTSA National Highway Traffic Safety Administration OMB Office of Management and Budget PIA Privacy Impact Assessment PII Personally Identifiable Information PTA Privacy Threshold Analysis

C.6 SPECIFIC REQUIREMENTS

C.6.1 Kick-Off Meeting

Within one (1) month of award, the Contractor shall attend a Kick-Off Meeting, either virtually using teleconferencing software or, if public health conditions permit, at NHTSA Headquarters in Washington, DC, with the COR, CO and/or CS, and any other interested NHTSA personnel.

The purpose of this meeting is to discuss the administration of the contract and the goal, objective, scope, and anticipated challenges for the effort. The meeting will serve to identify any potential problems and their solutions to ensure successful completion of the project.

This shall also be a working meeting for the Contractor and NHTSA, allowing up to two (2) hours for the meeting. The COR will provide the background for the project. The Contractor shall then discuss the firm’s approach for the project, including the Contractor’s understanding of the project objectives, the schedule of project activities, and deliverable requirements and specifications.

Prior to the meeting, the Contractor shall work with the COR to prepare appropriate briefing materials on their proposed approach. Materials for the Kick-Off Meeting shall be completed and submitted to the COR for review and approval at least one (1) week prior to the Kick-Off Meeting. The COR will send feedback to the Contractor on Kick-Off Meeting materials within three (3) days of receipt.

C.6.2 Submit Updated Study Design & Data Analysis Plan

Within one (1) week of award, the COR will send to the Contractor the materials developed for the study under a prior project (https://rip.trb.org/View/1747331). These materials include:

• A literature review of existing research relevant to the study;

• An assessment of potential challenges to conducting the study;

• Study Design & Data Analysis Plan;

• ICR Package (including Supporting Statement Parts A and B, 60-day Federal Register

Notice, consent forms, and study questionnaires);

• IRB Package;

• DMP;

• Draft Introduction and Methods section for the Final Report;

In this task, the Contractor shall review the Study Design & Data Analysis Plan and suggest any updates or changes necessary to ensure the study is feasible and will provide data to answer the research questions in Section C.2. For any updates or changes proposed, the Contractor shall provide a justification, with reference to any documentation that supports the change.

Within two (2) months of award, the Contractor shall submit the Updated Study Design & Data Analysis Plan to the COR. The COR will provide feedback to the Contractor within two (2) weeks of receipt. The Contractor shall submit the Final Updated Study Design & Data Analysis Plan within three (3) months of award.

C.6.3 Submit Data Acquisition Systems Report

Within four (4) months of award, the Contractor shall submit to the COR the Data Acquisition Systems Report. In this report, the Contractor shall identify two (2) to five (5) or fewer options for naturalistic driving data collection via smartphone app and two (2) to five (5) or fewer options for naturalistic driving data collection via in-vehicle DAS.

Smartphone Apps For smartphone apps, the Contractor shall only consider options that can be installed both on smartphones with Android or iOS systems. The Contractor shall consider apps that are currently available on the commercial market, freely available, or apps previously developed by the Contractor. The Contractor shall describe the benefits and drawbacks of each smartphone app option, considering factors such as:

• The kinds of data that can be acquired via the app, and whether these data are sufficient to answer the research questions in Section C.2;

• The costs associated with using, acquiring, and/or storing data via the app for the study period, including any costs arising from the transfer of NDS data via cellular data;

• Whether the app requires any interaction from participants, excluding initial installation and removal of app at the end of the study;

• Whether additional equipment (e.g., cell phone holder, device placed elsewhere in vehicle) can be used to communicate with the smartphone app and provide additional information about travel mode, participant role (driver vs. passenger), etc., as well as the cost of such additional equipment;

• Any available information on the reliability and/or accuracy of the performance and exposure data generated by the app (e.g., from the developer, described in peer-reviewed journal articles, etc.);

• The nature of the data generated by the app and/or received by the Contractor, including the amount of post-processing required to prepare the data for analysis and any required linkages with other data sets (e.g., OpenStreetMaps);

• The way in which the data collected impacts participants’ privacy;

• The ease with which participants will be able to download and install (and use, if necessary) the app; and

• Any other factors the Contractor believes relevant for understanding the app’s benefits and drawbacks.

In-Vehicle DAS For in-vehicle DAS, the Contractor may consider DAS that are currently available on the commercial market or previously developed by the Contractor. The Contractor shall describe the benefits and drawbacks of each DAS option, considering factors such as:

• The kinds of data that can be acquired via the DAS, and whether these data are sufficient to answer the research questions in Section C.2;

• The costs associated with using and/or acquiring the DAS for the study period;

• The ease with which the Contractor can install and remove the DAS, and the ease with which the DAS can be moved to a new vehicle, if necessary;

• Any burden participants are likely to perceive with having the DAS in their vehicles for the study period;

• The way in which the data collected impacts participants’ privacy;

• Any other factors the Contractor believes relevant for understanding the app’s benefits and drawbacks.

Within two (2) weeks of receipt, the COR will submit feedback on the Data Acquisition Systems Report and will select the smartphone app and in-vehicle DAS to be used in the study.

C.6.4 Develop Pilot Study Plan

Within 5 months of award, the Contractor shall submit to the COR the Pilot Study Plan. This document shall describe the plan for conducting a pilot study with nine (9) or fewer participants.

While preferred, participants do not need to be ages 15.5 – 20 or novice drivers. Data collection during the pilot study should last for one (1) month.

The goals of the pilot study are for the Contractor to familiarize themselves with the way in which the smartphone app and in-vehicle DAS are installed, the kinds of data they produce, the way in which the data must be cleaned prior to analysis, and the kinds of analyses that can be performed using the data to answer the research questions in Section C.2. Participants in the pilot study shall also respond to at least one of the study questionnaires, delivered to participants in the same way as will be done in the full study.

The Pilot Study Plan shall describe the way in which the Contractor will recruit participants and the ways in which data from the pilot study will be used by the Contractor to verify the procedures proposed for the full study.

The COR will send feedback to the Contractor within two (2) weeks of receipt of the Pilot Study Plan.

C.6.5 Identify Potential Study Sites

Within 5 months of award, the Contractor shall submit to the COR the Potential Study Sites letter report. When developing this report, the Contractor shall draw upon information about potential States for study sites contained within the Final Updated Study Design & Data Analysis Plan, as well as any other information the Contractor feels is relevant to suggesting potential study sites. In the Potential Study Sites report, the Contractor shall suggest five (5) States, in order of preference, for participation as study sites.

When recommending States as potential study sites, the Contractor shall consider factors such as:

• State GDL laws, both current and pending;

• Sociodemographic characteristics of young people or novice drivers in the State, especially the degree to which those characteristics align with what is known about novice drivers ages 15.5 - 20, nationally;

• The degree to which conclusions made from data collected in that State would generalize to other State contexts;

• The volume of novice driver licensing at State licensing agency offices; and

• Any other factors the Contractor believes are relevant for selecting a study site with the highest likelihood of participating and of yielding data that can answer the research questions in Section C.2.

The COR will provide feedback to the Contractor within two (2) weeks of receipt of the Potential Study Sites, including the COR’s approval of the order in which States should be considered as potential study sites.

C.6.6 Submit Updated Study Materials

In this task, the Contractor shall review study materials developed under the prior project and suggest any updates or changes necessary. The materials to be updated include:

• ICR Package (including Supporting Statement Parts A and B, 60-day Federal Register Notice, consent forms, and study questionnaires);

• IRB Package; and

• DMP.

As described in Section C.6.2, the COR will share with the Contractor the versions of these materials developed under the prior project within one (1) week of award (along with other materials, as described in Section C.6.2).

The Contractor shall ensure that the ICR and IRB Packages reflect the study as described in the Updated Study Design & Data Analysis Plan, including incorporating information about the data acquisition systems selected for use in the study and potential States for study sites.

The Contractor shall also ensure that the DMP is updated and accurately reflects the study design (see https://doi.org/10.21949/1520562 for information on creating DMPs).

Finally, a Privacy Threshold Analysis (PTA) was not developed under the prior project. The Contractor shall create the PTA as part of this task, using the template provided to the Contractor by the COR. The PTA is a questionnaire used to determine if an information technology system contains PII and to answer other questions related to privacy requirements.

Within seven (7) months of award, the Contractor shall submit the following Updated Study Materials:

• ICR Package (including Supporting Statement Parts A and B, 60-day Federal Register Notice, consent forms, and study questionnaires);

• IRB Package;

• DMP; and

• PTA.

The COR will provide feedback to the Contractor within two (2) weeks of receipt. The Contractor shall submit the Final Study Materials within eight (8) months of award.

https://doi.org/10.21949/1520562

C.6.7 Obtain IRB Approval for Pilot Study

The Contractor shall obtain approval for conducting the Pilot Study from an IRB prior to beginning recruitment of participants for the pilot study. The Contractor shall use their own IRB or a certified IRB that is registered and covered by an approved assurance of compliance under the Department of Health and Human Services, Office for Human Research Protections.

Within ten (10) months of award, the Contractor shall submit to the COR a Draft IRB Package for the Pilot Study. The COR will provide feedback within two (2) weeks of receipt. Once the COR approves the Package, the Contractor is responsible for submitting the materials to the selected IRB and responding to any revisions necessary to obtain IRB approval. The COR will be available to assist in answering questions from the IRB.

Within twelve (12) months of award, the Contractor shall have received IRB approval from the selected IRB and submit to the COR electronic copies of the approved IRB Package for the Pilot Study.

C.6.8 Finalize Study Site

Upon receipt of the COR’s approval of the Potential Study Sites letter report (Section C.6.5), the Contractor shall work with the COR to begin reaching out to potential States (and specifically, State licensing agencies) to serve as study sites. Before beginning communication with State officials, however, the Contractor shall wait for approval from the COR. The COR shall be available to assist with identifying and contacting State officials and to meet with State officials to discuss the State and State licensing agency’s involvement in the study.

Within fourteen (14) months of award, the Contractor shall finalize the State (and State licensing agency) to serve as the site for the full study.

C.6.9 Conduct Pilot Study

Upon receipt of IRB approval, the Contractor shall begin conducting the pilot study as described in the Pilot Study Plan (Section C.6.4) with nine (9) or fewer participants.

Within seventeen (17) months of award, the Contractor shall submit to the COR the Pilot Study Results letter report. This report shall describe the results of the pilot study, any challenges encountered during the pilot study, and any changes the Contractor recommends to the full study based on the pilot study results and/or challenges. The COR will provide feedback on the Pilot Study Results letter report within two (2) weeks of receipt.

C.6.10 Obtain IRB Approval for Full Study

Upon receipt of clearance from OMB, the Contractor shall make any changes requested by OMB and/or as a result of findings from the Pilot Study to the Updated Study Design & Data Analysis Plan, the IRB Package, and to any other study materials to which OMB requested a change.

Then, the Contractor shall obtain approval for conducting the full study from an IRB prior to beginning recruitment of participants. The Contractor shall use their own IRB or a certified IRB that is registered and covered by an approved assurance of compliance under the Department of Health and Human Services, Office for Human Research Protections. The Contractor is responsible for submitting the IRB Package for the full study to the selected IRB and responding to any revisions necessary to obtain IRB approval. The COR will be available to assist in answering questions from the IRB.

Within twenty-four (24) months of award, the Contractor shall have received approval from the selected IRB and submit to the COR electronic copies of the approved IRB Package for the full study.

C.6.11 Conduct Full Study

Upon receipt of clearance from OMB and approval from the IRB, the Contractor shall conduct the full study. Within forty-two (42) months of award, the Contractor shall complete data collection for the full study.

For each month in which data is collected, the Contractor shall include in their Monthly Progress Reports information about recruitment efforts, the status of data collection, including the number of participants recruited that month, the retention of previously recruited participants, the number of participants who completed data collection that month, progress towards recruitment for various groups (e.g., older vs. younger novices; novices with different sociodemographic characteristics), and any other issues related to data collection (e.g., technical issues, barriers to recruitment, etc.)

C.6.12 Submit Data Analysis Report

Within forty-seven (47) months of award, the Contractor shall submit the Data Analysis Report. In this letter report, the Contractor shall summarize the results of descriptive and inferential analyses conducted using the data collected in C.6.11. The intent of this report is to provide the COR an opportunity to review the study’s results, offer feedback, and suggest any needed changes to ensure that the analyses address the research questions in Section C.2, prior to the Contractor submitting the Final Report. The COR will provide feedback on the Data Analysis Report within 2 weeks of receipt.

C.6.13 Submit Draft Final Report

Within fifty (50) months of award, the Contractor shall submit the Draft Final Report. In preparing the Draft Final Report, the Contractor shall draw upon material developed in the prior project, namely, the Draft Introduction and Methods sections, making changes as necessary to reflect the current study design.

The Draft Final Report shall include the following sections:

• Executive Summary

• Introduction

• Methods

• Results

• Discussion

• References

• Appendices (if any)

The Draft Final Report shall adhere to standards detailed in the Publication Manual of the American Psychological Association (APA), 7th edition. The Contractor shall ensure that the Draft Final Report is written using the NHTSA report template, which the COR will provide to the Contractor at award. The Contractor shall also ensure that any figures and tables included in the Final Report are compliant with the requirements Section 508 of the Rehabilitation Act (https://www.section508.gov/).

In the Discussion section, the Contractor shall clearly describe how the results inform the research questions listed in Section C.2. The Contractor shall also include in the Discussion any limitations of the study. Finally, the Contractor shall also include in the Discussion information about the way in which the study’s results can be translated to practice—particularly the implications of the results for State GDL programs and older novice drivers, and for driver education and training.

The COR will provide feedback on the Draft Final Report within three (3) weeks of receipt.

Within fifty-two (52) months, the Contractor shall make the required revisions and submit to the COR the Revised Draft Final Report.

C.6.14 Final Briefing Meeting

Within fifty-three (53) months of award, the Contractor shall conduct collaboratively with the COR a Final Briefing Meeting, either virtually using teleconferencing software or, if public health conditions permit, in-person at NHTSA Headquarters in Washington, DC. The Final Briefing shall consist of a 30 – 45-minute presentation of the project given by the Contractor that includes the background and motivation for the project, the study’s design and hypotheses, the study’s results, and a discussion of the study’s results, particularly targeted at implications for novice driver safety research and program development. The Contractor’s presentation shall be followed by a question-and-answer period. During the presentation, the Contractor shall present relevant methodological, statistical, and technical information in an accurate way that is also understandable to those without research backgrounds, as both researchers and non-researchers are likely to be in attendance.

The Contractor shall submit Materials (e.g., a PowerPoint presentation) for the Final Briefing Meeting to the COR for review and approval at least one (1) week prior to the Final Briefing Meeting. The COR (TO) will send feedback to the Contractor on Final Briefing Meeting materials within three (3) days of receipt.

https://www.section508.gov/

C.6.15 Submit Final Report and Other Materials

Within fifty-five (55) months of award, the Contractor shall submit the Final Report and Other Materials. The Final Report shall incorporate any feedback elicited during the Final Briefing Meeting and/or from the COR.

Along with the Final Report, the Contractor shall also submit the following Other Materials related to the study:

• The de-identified dataset(s) that was used to conduct the analyses described in the Final Report in CSV format.

• A manual that details the variable names, meaning, labels, and ranges of responses in the dataset in a way understandable to members of the public who may be unfamiliar with the specifics of the study design and who are seeking to perform additional analyses with the dataset.

• Output from statistical software providing more details about the analyses described in the Final Report, including sufficient information to reproduce the main analyses.

• Sufficient information to generate the tables and figures included in the Final Report.

This information could in the form of an Excel sheet with editable tables and figures, or it could be in the form of syntax for the statistical software that generated the tables/figures.

C.6.16 Optional Task 1: Collaboratively Prepare Draft Journal Manuscript

Within fifty (50) months of award, the Government will notify the Contractor of its intent to exercise Optional Task 1. The Government is not obligated to order this optional task, nor is the Contractor obligated to perform this task unless a unilateral modification is executed by the Government. If the Government elects to proceed with the optional task, then the Contractor will be required to provide the additional support and services. The Contractor shall provide these services after receiving a unilateral modification signed by the CO. Any unilateral modifications will be issued by the CO and received by the Contractor no later than one (1) month after notification of intent to exercise the optional task.

The objective of this task is for the Contractor and the COR to work collaboratively to prepare a Draft Journal Manuscript intended for submission to a peer-reviewed scientific journal. Prior to writing the Draft Journal Manuscript, the Contractor and the COR will determine the content and organization of the manuscript, the journal to which the manuscript will be submitted, whether the COR will be an author and the order of authorship, and the assignment of tasks for preparing and writing the manuscript.

Within six (6) months of execution of Optional Task 1, the Contractor, working collaboratively with the COR, shall submit the Draft Journal Manuscript to the COR. The COR will provide the Contractor with comments within two (2) weeks of receipt.

Within eight (8) months of execution of Optional Task 1, the Contractor shall address the COR’s comments and submit the Revised Draft Journal Manuscript to the COR.

C.6.17 Optional Task 2: Prepare One-Page Information Sheet

Within fifty (50) months of award, the Government will notify the Contractor of its intent to exercise Optional Task C.6.17. The Government is not obligated to order this optional task, nor is the Contractor obligated to perform this task unless a unilateral modification is executed by the Government. If the Government elects to proceed with the optional task, then the Contractor will be required to provide the additional support and services. The Contractor shall provide these services after receiving a unilateral modification signed by the CO. Any unilateral modifications will be issued by the CO and received by the Contractor no later than one (1) month after notification of intent to exercise the optional task.

The objective of this task is to prepare a one-page information sheet describing potential applications of the study’s results to a specific stakeholder audience. The Contractor shall work collaboratively with the COR to determine the specific stakeholder audience that will be targeted. The Contractor shall prepare a one-page information sheet that describes, in brief, the study’s results and the potential applications of the study’s results for the selected stakeholder audience. For example, the information sheet may describe the implications of the study’s findings about the driving behavior and performance of novices aged 18 – 20 for whether it is feasible to extend current GDL policies to older novices or whether these policies require changes for older novices. As another example, the information sheet may describe the way in which the study’s findings on identifying and characterizing high-risk novices can be used to develop novice driver training or education programs tailored for this group. The Information Sheet shall be understandable to a non-scientific audience unfamiliar with the statistical techniques used in the study and shall focus on potential applications of the study’s results within that stakeholder domain. The Information Sheet shall include an infographic or other graphic that helps to convey the study’s results and translate the study’s findings to practice.

Within three (3) months of execution of Optional Task 2, the Contractor shall submit the Draft Information Sheet to the COR. The COR will return feedback on the Draft Information Sheet within two (2) weeks of receipt.

After making the suggested changes, the Contractor shall submit the Final Information Sheet to the COR within five (5) months of execution of Optional Task 2.

(End of Section C)