Performance_Work_Statement_-_April_2018_-_SSDQ.pdf

United States Department of Transportation Federal Motor Carrier Saftety Administration

Statement of Work or Performance Work Statement

For

Data Quality Analysis and Training Support for the FMCSA State Safety Data Quality Improvement Program

1.0 Overview

This is a non-personal services contract or task order to provide Data Quality Analysis and Training Support for the FMCSA State Safety Data Quality Program. The Government will not exercise any supervision or control over the contract or task order service providers performing the services herein. Such service providers shall be accountable solely to the Contractor who, in turn is responsible to the Government as defined in this Statement of Work (SOW) or Performance Work Statement (PWS). The Contractor shall perform to the standards herein.

1.1 Background

State highway safety information systems are a vital source of data for decision making concerning truck and bus safety. At the National level, systems such as FMCSA’s Motor Carrier Management Information System (MCMIS) rely on the data coming from state-level data collection and management systems for 3.5 million roadside inspections and over 170,000 large trucks and buses involved in crashes every year. It is incumbent upon FMCSA to ensure that these data are interpreted and coded consistently among the States and the data collectors.

These data and information are used to support agency operations and priorities, evaluate the success of existing programs and regulations, and to evaluate the safety performance and fitness of commercial motor carriers.

The majority of safety information, such as crashes and inspections, related to large trucks and buses are collected by State and police organizations. This information is collected into a State repository and then transferred to an FMCSA system called SAFETYNET. SAFETYNET also allows for data entry, transfer, and data edit checks before being transmitted into the official FMCSA data repository, MCMIS. This data is then available for Federal and State enforcement personnel, as well as publicly available via the Safety Measurement System (http://ai.fmcsa.dot.gov/sms/).

In 2004, FMCSA started a comprehensive State Safety Data Quality (SDDQ) program. The SSDQ program encompasses a number of efforts including off-site data reviews and analyses, on-site process evaluations for State-provided crash data, the development of a measurement system that evaluates the States’ performance in reporting data in a timely, complete and accurate manner to FMCSA, the development and implementation of a comprehensive training program for State crash data collection and reporting to FMCSA and the development of best practices for reporting quality data to FMCSA. Data quality is an activity that requires vigilant oversight and management. As the challenges of the SSDQ program evolve it is necessary to constantly monitor and evaluate activities associated with the collection of complete, timely and accurate data for FMCSA. The Agency makes this information available to the public via the Analysis & Information (A&I) Online website (http://ai.fmcsa.dot.gov/DataQuality/dataquality.asp).

The SSDQ Team also oversees operation of the DataQs System (http://dataqs.fmcsa.dot.gov), which is the Agency’s administrative mechanism for the public to request FMCSA or the States to review information displayed or published by FMCSA (required under P.L. 106-554). DataQs receives nearly 58,000 Requests for Data Review (RDR) in 2018, most of which go to State agencies for their own review process related to crashes and inspections of large trucks and buses. FMCSA uses the information collected through DataQs to improve existing information and to identify training opportunities at the State and inspector level.

One major challenge over the course of the next 5 years is that FMCSA will be transitioning from its existing crash data standard to the National Highway Traffic Safety Administration’s

(NHTSA) Model Minimum Uniform Crash Criteria (MMUCC). The SSDQ Team has worked with some States to make more of their data MMUCC-compatible, and much work will be done by the team to further this effort.

The SSDQ Team is made up of a Federal program manager at FMCSA, support from the Volpe National Transportation Systems Center (Volpe), and this contract. The Volpe staff administer the day-to-day development and operations of the DataQs and A&I websites, as well as the monthly SSDQ performance measures and map. The Volpe staff also conducts outreach to the States concerning areas identified through the monthly SSDQ analysis, and training of FMCSA’s SAFETYNET system.

The tasks identified below represent on-going efforts necessary to improve the collect of high quality data to support FMCSA’s activities, to target unsafe carriers, vehicles, and drivers, and most importantly, reduce the number of serious and fatal crashes on the Nation’s roadways.

1.2 Objective

FMCSA is seeking an active partner to assist the Data Quality team with providing technical knowledge, analysis and training of State and Federal traffic records systems in helping States provide information that powers the core of the Agency’s safety systems.

The contractor’s focus will be on providing assistance to State agencies responsible for providing consistent data about commercial motor vehicle crashes to FMCSA. The contractor will provided analysis on data quality issues and recommend improvements. The final focus is on providing training to State partners on crash and data quality issues.

1.3 Detailed Requirement

The Contractor shall provide all personnel, equipment, supplies, facilities, transportation, tools, materials, supervision, and other items and non-personal services necessary to perform:

Task 1: Provide Technical and Consultative Crash Data Assistance to FMCSA and the States The contractor shall provide technical assistance and respond to inquiries from FMCSA and the States concerning commercial motor vehicle (CMV) crash data collection, analysis, coding issues, and responses to DataQs submissions. This critical function provides a central point for the uniform identification, resolution and documentation of State and Federal level crash reporting problems, trends and best-practices. The function shall also coordinate with other members of the FMCSA Data Quality support team to best provide timely and accurate responses to our customers.

The contractor shall serve as the subject matter expert in matters related to crash data collection standards, State traffic records systems, decoding information from Vehicle Identification Numbers (VIN), data extraction procedures, and safety data analysis. The contractor will also need law enforcement experience/perspective to highway safety, State judicial procedures, and some knowledge of the Federal Motor Carrier Safety Regulations.

In addition to coding assistance, the contractor shall review State systems (as requested by the States and approved by the COR) and provide technical assistance to State and FMCSA IT developers on the accurate computer processes used to transfer and input data to SAFETYNET and MCMIS. This includes auditing State crash databases and extraction methodologies to ensure all reportable crashes have been properly identified. This task will involve some travel to States to support on-site crash data collection operations (approximately 8-10 trips per year) as well as participation in teleconferences to support State crash data activities, such as modifying a State’s police accident report form or issues that arise in the State traffic records coordinating committees (TRCC).

The contractor shall provide analytical and consultative support services to the FMCSA and its State partners for the commercial motor vehicle (CMV) crash data improvement program including but not limited to the areas of traffic records assessments, coding conventions, training procedures, VIN translation, and programming and translation of codes from State files to MCMIS files.

FMCSA is seeking to expand upon the data States provide to the Agency. The contractor shall provide recommendations to the COR for revisions to the crash data collection program (identify priorities of improving new data to be collected, refining existing policies to States, identifying gaps between information needed versus collected, identifying existing data fields that are not serving a purpose, etc.).

Analytical and consultative services provided in support of the data quality improvement program which may occur as ad hoc or discrete tasks outside of a scheduled training may include but will not be limited to the following:

• Review and evaluate all new and revised State crash report forms; (3-5 per year)

• Review and evaluate all new and revised State electronic crash data collection systems to ensure that quality data is/will be reported to the FMCSA; (2-4 per year)

• Determine the impact of a recent IT change that prevents States from uploading records, and recommend a potential work-around or long-term solution;

• Support State-provided training efforts in large truck and bus crash data. This typically covers some/all of five areas: selection criteria, motor carrier identification, vehicle configuration/cargo body type, (sequence of) crash events, and hazardous materials;

• Recommendations for improving the State Safety Data Quality performance measures (see https://ai.fmcsa.dot.gov/DataQuality/National.aspx);

• Assist States in revising their crash reports, instruction manuals and all other procedures for collecting and inputting data from State level systems to the FMCSA data systems.

The SSDQ Team handles data quality issues and questions from two-thirds of the States in any given year with some States having multiple contacts in a year. Some questions are brief, and some issues will not only require extensive analysis but also navigating a State’s bureaucracy to get State Offices to work towards a common goal (such as State Police or Highway Patrols, State Departments of Transportation, Departments of Motor Vehicles, Public Utility Commissions, etc.). This work will likely require the use of at least one (1) FTE worth of work.

Examples:

Brief questions: (2 business day turnaround, with summary of questions contained in monthly status report)

• A driverless large truck that was in motion and subsequently was involved in a crash.

Did this crash meet the FMCSA threshold for reporting, what fields are required, and how should the driver-related fields be completed?

• A car lost control on a public roadway and hit a large truck parked on private property.

Should this crash be sent to FMCSA?

https://ai.fmcsa.dot.gov/DataQuality/National.aspx

Complex examples: (coordination with COR required with on-going updates contained in monthly status report)

• It was determined that one State was under-reporting non-fatal crashes. A review of the State’s motor vehicle crash data was needed to compare against the information sent to and then received by FMCSA. Also, an analysis of the State’s extraction procedures was needed. The entire process required interfacing with three State Agencies for coordination of information and internal processes.

• A State has implemented changes to its Police Accident Report (PAR) and is seeking input for developing training related to the commercial motor vehicle section of the PAR.

• One State transferred over to a new crash database, which broke the link to FMCSA’s systems. It required a review of the XML schema used by the State to determine if the proper information was being extracted from the State central crash repository and then properly translated to SAFETYNET. Changes were recommended to the State in order to fix the problems that were detected.

Performance Standard: The contractor shall coordinate with appropriate FMCSA headquarters, State division and FMCSA Data Quality support team members to ensure timely and effective support to FMCSA and State agencies. All customary “coding” questions shall be answered within 2 business days of receipt. For more complicated questions, which require further research, the contractor shall contact the requestor to explain that the completed response may take up to 2 weeks. The contractor shall also provide a monthly summary of services conducted to the COR and with their invoices. When possible, the contractor shall estimate the benefit of their contact within a State as it relates to the SSDQ performance measures in monthly contract performance reports (additional records to be uploaded, or reduced days for data to be uploaded).

Task 2: Data Analysis and Quality Control for SAFETYNET/MCMIS Crash Data The contractor shall perform data analysis and quality control on the selected SAFETYNET data fields uploaded into MCMIS. Data will be presented at both the National and State level for summary review and to identify trends in data.

Subtask 2a: Report analysis to improve data quality – tasks may include but will not be limited to:

• MCMIS Field Review – Contractor shall first evaluate MCMIS “snapshots” annually to examine the completeness and validity of individual fields within the crash database;

• SAFETYNET Field Review – On a state-by-state basis when requested by the State and approved by the COR, the contractor shall review the fields that the State supplies through SAFETYNET, and compare this to the State’s crash database (when available).

• SAFETYNET Audit Reports – After conducting a SAFETYNET field review, the contractor shall evaluate State error logs and data-report review logs and develop guidance for the State agencies on a case-by-case basis;

Subtask 2b: DataQs Request for Data Review (RDR) Analysis. The contractor shall conduct analyses of Requests for Data Review from the FMCSA DataQs system, either by a census of certain types of RDRs, a census of all RDRs or a sampling of applicable RDRs in order to identify:

• A statistical review of the RDRs to identify trends or problem areas, by category, State and State Agency;

• Review similar actions across state jurisdictions for consistent DataQs findings;

• Determining how to reduce unnecessary RDRs by targeting areas for training (of law enforcement or DataQs analysts), State Agency follow-up, or systematic flaws at the State or National level;

• Reviewing the carrier and driver responses and approaches to understand trends in RDRs, with a focus on website usability and deterring frivolous RDRs;

• Review of new regulations and the effect through DataQs RDRs; and,

• Developing recommendations and identifying best practices for Federal, State and public

DataQs users to be added to the DataQs User Manual.

Performance Standard: The contractor shall identify specific trends in data quality problems at the Federal, State or State Agency level and propose solutions via corrective action plans to be used in conjunction with state crash collection training and state reviews. These analyses will be at the direction of the COR but may be recommended by the contractor based on prior experience or recent questions raised by the States. Generally, there are eight (8) total analyses of FMCSA crash data and DataQs per year. These analyses are generally of a short duration taking 120-240 hours on average, depending on the complexity of the issue.

Task 3: Documentation Development & Maintenance The contractor shall develop and maintain a crash coding and validation (C&V) manual. For an example of other crash data C&V manuals, see NHTSA’s 2017 Fatality Analysis Recording System (FARS) C&V manual at https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812559. The manual will need to have a table of contents, introduction, an area for changes (empty for the first manual), list of data elements, and then data elements broken into related areas (accident, vehicle, driver, carrier, etc.). Each data element will need the name of the element, data format, definition, remarks (descriptions of the element values), and recommended/required consistency checks (if any).

Over the life of this contract, FMCSA will be changing the crash data received by the States.

This manual will have to reflect the changes from going from the existing data elements to NHTSA’s Model Minimum Uniform Crash Criteria (MMUCC).

The contractor shall also help maintain the DataQs Analyst Guide (Guide) in terms of keeping guidance current and addressing hot topics in a timely manner. The DataQs Analyst Guide contains guidance to Federal and State users of DataQs for consistent responding to requests for data review (RDR) from the public. The contractor will only be responsible for developing content, and providing alt-tags for graphics provided by the contractor. The contractor is not responsible for layout and publication of the Guide.

Performance Standard:

Task 3: Develop an FMCSA crash data coding & validation manual. The first draft shall be developed within 6 months after contract’s start date, with a final version due NTL 8 months after contract’s start date. Annual updates will be held to a similar schedule of first draft due within 6 months of the option year’s start date and final version due within 8 months of the option year’s start date. Develop updates to the DataQs Analyst Guide with first drafts due NTL 10 months after contract’s start date and final version due 12 months after contract’s start date.

https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812559

Annual updates of the Guide will be held to a similar schedule of first draft due within 10 months of the option year’s start date and final version due within 12 months of the option year’s start date.

Optional Task 4: Develop, update as required, and deliver training related to the State Safety Data Quality (SSDQ) program.

Task 4: Develop the curriculum, training materials, and conduct training to educate State data managers, law enforcement, and other traffic records personnel in the standardized coding of data for FMCSA’s Motor Carrier Management Information System (MCMIS) crashes and DataQs system. This task provides, as necessary, support to conduct in-person training as directed by the COR. Past training has typically consisted of 3 or 4 sessions, with each session covering 90 to 120 minutes of crash reporting policies with an emphasis on difficult scenarios taken from DataQs.

Past training has also consisted of 1 or 2 sessions of DataQs best practices for resolving crash data requests, communicating FMCSA’s Adjudicated Citations policy, crash data reporting requirements and definitions, tools and decision-making processes for decoding VINs, and new program elements, including assembling and facilitating panels from experienced State partners on best practices. This training will supplement other data quality training provided by the rest of the SSDQ Team and FMCSA as appropriate.

The training has been given every 18 to 24 months with 50-60 attendees in the room. The current plan is for training in August 2020 (Indianapolis, IN), January 2022, August 2023 (base year, and option years 2 and 3) with the location to be determined. For an example of past training, see the CVSA Data Management, Quality & FMCSA Systems Training at:

https://cvsa.org/eventpage/events/data-quality/.

The COR will provide the contractor with a focus for the course sessions. The contractor shall provide the COR with a draft 8 weeks prior to the training event, and a final copy of materials (slide deck, hand-outs, etc.) 4 weeks prior to the training. Travel will be required for two people to attend the 3-day training event, as well as for two people to travel to either Boston, MA, or Washington, DC, for planning the sessions. The Contractor shall also develop appropriate job aids related to the courses they are providing.

Performance Standard:

Task 4: Develop and facilitate 3 2-hour training modules as described above, as directed by the COR, for 70-125 people. The training locations and dates are subject to change, but are likely to coincide with major Commercial Motor Vehicle Safety Alliance events (January or August).

Receive a majority of positive responses from post-training evaluations by training attendees on all sections of classes taught.

Reporting Requirements:

The contractor shall prepare monthly progress reports due at the same time as the monthly invoice directly to the COR. The reports shall summarize the major activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, changes in personnel and their security designation, and staff use. The contractor shall also participate in weekly conference calls with the COR and other members of the FMCSA Data Quality team that discuss the status of current data quality issues and to plan future work. Separate calls with the COR will be scheduled as needed to ensure effective communication and success of the contract.

https://cvsa.org/eventpage/events/data-quality/

Performance Standard: Monthly progress reports to be submitted to the COR, conference call attendance.

Key Personnel:

The COR requires one key person to be identified by the contractor to participate full time on this contract, who has extensive subject matter expertise with State and/or Federal traffic records experience (10+ years), analytical skills, experience in SQL and XML, and presentation/training experience.

2.0 Period of Performance

The period of performance shall be for one (1) Base period of twelve (12) months and four 12- month option periods. The Period of Performance reads as follows:

Base Period Sep. 30, 2019 – Sep. 29, 2020 Option Year I Sep. 30, 2020 – Sep. 29, 2021 Option Year II Sep. 30, 2021 – Sep. 29, 2022 Option Year III Sep. 30, 2022 – Sep. 29, 2023 Option Year IV Sep. 30, 2023 – Sep. 29, 2024

3.0 General

The Contractor shall develop and maintain an effective Quality Control Plan (QCP to ensure services are performed in accordance with this PWS. The Contractor shall develop and implement procedures to identify, prevent, and ensure non-recurrence of defective services.

The Contractor’s QCP is the means by which he/her assures that his/her work complies with the requirement of the task order, and shall be deliverd to the COR within 30 days after award.

After government acceptance of the QCP, the Contractor shall receive the Contract Officer’s (CO) acceptance in writing of any proposed change to the government accepted QCP.

3.1 Hours of Operation

The Contractor is responsible for conducting business, between the hours of 9 am to 3 pm, EST/EDT (as appropriate) Monday thru Friday except Federal holidays or when the Government facility is closed due to local or national emergencies, administrative closings, or similar Government directed facility closings. The Contractor at all times must maintain an adequate workforce for the uninterrupted performance of all tasks defined within this PWS when the Government facility is not closed for the above reasons. When hiring personnel, the Contractor shall keep in mind that the stability and continuity of the workforce are essential.

3.2 Place of Performance

The work to be performed under this contract shall be performed at the contractor’s place of business, and may include suitable telework arrangements per the contractor’s policy.

3.3 Physical Security

The Contractor shall be responsible for safeguarding all government equipment, information, and property provided for Contractor use.

3.4 Special Qualifications

The contractor is responsible for providing workspace, equipement, software, and supplies for employees. Employees may telework depending on the contractor’s policy.

3.5 Periodic Progress Meetings

The contractor agrees to attend progress meetings. The Contracting Officer or Contracting Officer’s Representative (COR), and other Government personnel, as appropriate, may meet periodically with the Contractor to review the Contractor's performance. At these meetings, the CO will apprise the Contractor of how the government views the Contractor's performance and the Contractor will apprise the Government of problems, if any, being experienced. Appropriate action shall be taken to resolve outstanding issues. These meetings shall be at no additional cost to the government.

3.6 Identification of Contractor Employees

All contractor personnel attending meetings, answering Government telephones, and working in other situations where their Contractor status is not obvious to third parties are required to identify themselves as such to avoid creating an impression in the minds of members of the public that they are Government officials. They must also ensure that all documents or reports produced by Contractors are suitably marked as Contractor products or that Contractor participation is appropriately disclosed. Contractor personnel will not be required to obtain and wear badges in the performance of this service.

3.7 Contractor Travel

The contractor shall be required to travel throughout CONUS during the performance of this task order to attend meetings, conferences, and training. The Contractor may be required to travel to off-site training locations and to ship training aids to these locations in support of this PWS.” All travel requires Government approval/authorization and notification to the COR.

3.8 Materials

Not applicable.

3.9 Organizational Conflict of Interest

Contractor and subcontract personnel performing work under this award may receive, have access to, or participate in the development of proprietary or source selection information (e.g., cost or pricing information, budget information or analyses, specifications or work statements, etc.), or perform evaluation services which may create a current or subsequent Organizational Conflict of Interests (OCI) as defined in FAR Subpart 9.5. The Contractor shall notify the CO immediately whenever he/she becomes aware that such access or participation may result in any actual or potential OCI, and may merit the submittal of a plan to the CO to avoid or mitigate any such OCI. This mitigation plan would be determined to be acceptable solely at the discretion of the CO, and in the event the CO unilaterally determines that any such OCI cannot be satisfactorily avoided or mitigated, the Contracting Officer may effect other remedies as he or she deems necessary, including prohibiting the Contractor from participation in subsequent contracted requirements which may be affected by the OCI.

4.0 Government Furnished Equipment and Services: N/A

4.1 Services

N/A

4.2 Facilities

4.3 Materials