Appendix B_Data Collection Protocols and Database Population.pdf

Appendix B

Data Collection Protocol and Database Population

B-1

The purpose of this appendix is to provide additional details on the data exchange occurring between the contractor and EFLHD, as well as to provide information regarding data should be processed for delivery to EFLHD.

A. DATA FILES PROVIDED TO CONTRACTOR FROM EFLHD

1. EFLHD will supply the following for each site to be collected. Additional items will be communicated on an as-needed basis (more information on the pre-data collection items are available in Appendix C)

a. Geodatabase (containing list of routes to be collected and GIS of those routes)

b. ROUTE_ID table – table is pre-populated and contains a list of routes that exist at each site. This list will be exhaustive and will include routes that will not need to be collected by the contractor (such as unpaved routes and parking areas). Routes to be collected by the contractor will be clearly identified by the cycle collected and collection method.

c. The contractor will be collecting nearly all paved roads at each site, EFLHD will communicate which paved roads will not need to be collected (paved roads not suitable for a data collection vehicle).

2. All pertinent tables listed here will be populated or partially pre-populated on a per site basis – for more information on each table please see Appendix J: Database Data Dictionary.

a. CUMULATIVE_STATUS

b. NOTES

c. TOTALS (partially/incrementally updated)

d. ROAD_CONDITION_ARCHIVE

e. ROUTE_GPS_ARCHIVE

f. ROUTE_ID (partially)

g. RIP_REMOVED (partially)

h. ROUTEID_COMP (partially)

i. Feature Classes (for road/parking GPS referencing only – not to be updated by contractor)

3. “Route ID” .pdf report of routes to be collected (or in a file format agreed upon between

EFLHD and contractor)

4. Route notes for any field assistance and unique situations for collecting roads.

5. On-site contact information for each collection site and EFLHD project manager contact information.

****NOTE - A detailed list of collection sites scheduled for collection, and advice on best collection windows will be provided at time of each task order award.

B. SUBMISSION OF DATA FILES TO EFLHD

1. After completing collection at a site or group of sites, the contractor will compile video and condition data and populate a geodatabase for submittal.

2. Assign the filename to the site geodatabase of SITE_C7.mdb, for example, DEVA_C7.mdb for

Death Valley National Park. The site name will usually be the NPS PARK_ALPHA code. The

B-2 list of NPS Park Alpha’s can be found in Appendix A and the alpha code is also found in the park website address (example: https://www.nps.gov/deva). “C7” is the abbreviation for

“Cycle 7”.

3. Submit the site geodatabase (SITE_C7.mdb) to EFLHD in Microsoft Access format. The data structure is specified in Appendix J: Database Data Dictionary

4. Every site database will consist of the following tables, see Section E below for further information on table population (bolded tables will be populated/updated, at least partially, by the contractor). Tables in SITE_C7.mdb include:

a. CUMULATIVE_STATUS

b. EXCEPTION_NOTES

c. ROAD_CONDITION (in 0.02-mile segment intervals)

d. ROAD_CONDITION_ARCHIVE

e. ROUTE_GPS

f. ROUTE_GPS_C7

g. ROUTE_GPS_ARCHIVE

h. ROUTE_ID

i. WIDTH_MEASURE

5. Contractor shall provide the following data files to EFLHD:

a. All files necessary to perform quality control checks on pavement distress data. A combination of the crack distress viewing software (SOW Task C), pavement images, and distress data files must allow EFLHD to view and edit the results of crack distress analysis on digital pavement images. These files must also allow EFLHD to match data output from the viewing / crack analysis software against the populated geodatabase. All files shall be retrievable from the viewing/analysis software by EFLHD for independent QC verification of contractor’s results.

b. All files necessary to perform quality control checks on the image measurement process to capture roadway measurements (e.g. shoulder width, lane width, pavement width). All files shall be retrievable by EFLHD for independent QC verification of contractor results using contractor-supplied software.

c. Provide images in the Standard Data Format according to AASHTO MP 47 only when requested by the COR.

C. PRIMARY LANES REPRESENT BOTH LANES

The contractor shall collect all roadway data by traveling the primary lane only. The primary lane will be defined in the ROUTE_ID table of the geodatabase (and associated .pdf) by using the Route Description fields (From/To description). The contractor may deviate from primary lane collection in unique situations that will need to be approved prior to collection. Or if deviation from primary lane is necessary in field, but not known prior to collection; the contractor will include the change in any exception list provided to EFLHD immediately after data collection.

It should be noted that for the vast majority of routes, the driving direction is the direction of increasing linear referencing on the route GPS measures. However, there are some instances of roads collected with decreasing mileposts or linear reference measures.

https://www.nps.gov/deva

B-3

D. “5000” ROUTES

When requested by EFLHD, the contractor may collect data on routes designated with Route

Numbers in the 5000 series. Route numbers in the 5000 series indicate the road is a non-NPS owned/maintained route (for example, a state maintained route that is within a National Park boundary). These “5000” Routes will result in collection of ONLY video and GPS. No pavement distress data collection/analysis or width measurements will be performed by the contractor, unless otherwise directed by EFLHD. The contractor shall process roadway video and GPS for delivery to EFLHD. GPS information will be populated in the ROUTE_GPS and

ROUTE_GPS_C7 tables. The contractor shall propose a cost per mile for collection of roads for video/GPS only, and not pavement distress analysis.

E. DATABASE POPULATION ELEMENTS

1. There are several factors that will determine what elements the contractor will populate for database submittal. These factors will be determined after collection has occurred. If route has been noted for pavement distress collection, then ROUTE_ID.DISTRESS_FLAG shall be set to

“YES” and pavement condition fields will then need to be populated in the

ROAD_CONDITION table (see Appendix E: Pavement Distress Identification Manual)

2. Pavement Surface Distresses:

a. Cracking: extents and severity

b. Alligator Cracking

c. Transverse Cracking

d. Longitudinal Cracking

e. Concrete Faulting

f. Patching/Potholes: extents

g. IRI (inches/mile)

h. Macrotexture

i. Rutting (inches)

3. Route “Matching” - GNSS locational data / Length

a. The contractor will be directed when to use new GNSS/length and when to match to a previously collected GNSS/length.

b. While each route cannot be driven exactly the same in each cycle of data collection, it is the intent of EFLHD to establish specified road length data (GNSS and Length) from previous collections as the static, baseline data for this and future collections. Routes collected by the data collection vehicle will be specified to match to a previous collection cycle’s length. If the route’s physical geometry or alignment has changed or the route is new to the inventory, then the route will take new GNSS/length for the current cycle.

c. When “matching” to previous collection GNSS/length; the contractor shall take the as driven length and corresponding distress data and normalize it to the “matching” GNSS and length data, as specified in the ROUTE_ID table.

d. The route length matching methodology is determined by specific database field in the

ROUTE_ID table. This field will indicate when to match lengths to length in previous cycles, and the past cycle number the route shall be matched. EFLHD will populate this database field as part of the pre-collection data provided. However, not all road realignments resulting in length changes may be known in advance of the data collection trip. If collected geometry or field conditions indicate that the route has changed and differs from the EFLHD-provided route shapefiles, then the contractor

B-4 shall notify EFLHD in a route change report immediately after data collection, and treat the route as a new route with new GNSS and length data applied.

e. The following database tables will be affected and/or used depending on how the

ROUTE_ID.MILEAGE_MATCH_CYCLE field is populated or updated:

i. ROUTE_GPS – Will be empty prior to collection, contractor shall populate with data after collection and post-process. Combination of any new/changed route geometry and “matched” routes for a site (contains GNSS data for a route that matches to a previous cycle, only changes if route alignment changes). EFLHD will provide route specific information on whether a route should have the GNSS data brought forward from a previous cycle or if the contractor should provide new GNSS data for this table.

ii. ROUTE_GPS_ARCHIVE – do not populate or change, will contain the correct MATCHING GPS from previous NPS data collection cycles.

iii. ROAD_CONDITION – Contains collected distresses for the current cycle only. The contractor will populate based on fields from the ROUTE_ID table. Use ROAD_CONDITION_ARCHIVE table data if matching to a previous cycle.

iv. ROAD_CONDITION_ARCHIVE – do not populate or change, will contain the following correct GNSS/distress from previous data collection cycles.

f. On a per route basis, if the following is true, populate ROAD_CONDITION table, ROUTE_GPS table, and ROUTE_GPS_C7 table with the current cycle’s GNSS/locational data:

ROUTE_ID.GPS_MILEAGE_MATCH_CYCLE = TOTALS.RIP_CYCLE

g. On a per route basis, if the following is true, populate ROAD_CONDITION table and

ROUTE_GPS table with a previous cycle/iteration’s data using the Table B-1 below as a guide:

ROUTE_ID.GPS_MILEAGE_MATCH_CYCLE <> TOTALS.RIP_CYCLE

Table B-1: Guide for Reference Tables when “Matching” to Previous Cycles

h. Specific fields affected by matching:

i. Route Identification Field (ROUTE_IDENT)

The contractor will be able to refer to the

MILEAGE_GPS_MATCH_CYCLE field in the ROUTE_ID to determine:

If the “MILEAGE_GPS_MATCH_CYCLE” is equal to the current cycle for an individual route the contractor will populate both the

Route ID Field

MILEAGE_GPS_MATCH_CYCLE Archive Tables to refer to

Matching to Cycle 4 4.1 ROAD_CONDITION_ARCHIVE ROUTE_GPS_ARCHIVE

Matching to Cycle 5 5.1 ROAD_CONDITION_ARCHIVE ROUTE_GPS_ARCHIVE

Matching to Cycle 6, Iteration 1 6.1 ROAD_CONDITION_ARCHIVE ROUTE_GPS_ARCHIVE

Matching to Cycle 6, Iteration 2 6.2 ROAD_CONDITION_ARCHIVE ROUTE_GPS_ARCHIVE

Matching to Cycle 6, Iteration 3 6.3 ROAD_CONDITION_ARCHIVE ROUTE_GPS_ARCHIVE

B-5

ROUTE_GPS and ROAD_CONDITION tables with new lengths taken during collection

If “MILEAGE_GPS_MATCH_CYCLE” is set to 4.1, 5.1, 6.1, 6.2, or 6.3 for an individual route, the ROUTE_GPS data will be brought forward from the ROUTE_GPS_ARCHIVE table. Additionally, the

ROUTE_CONDITION lengths will be brought forward from the

ROAD_CONDITION_ARCHIVE table and the process of “distress normalizing” will take place using that length.

ii. The following fields will be updated as a result of the “matching”:

Table: ROUTE_GPS

ROUTE_IDENT

STATE

MP

GPS_LAT

GPS_LONG

GPS_ELEV

GPS_MODE

XFALL

GRADE

HEADING

INSP_DATE

Table: ROAD_CONDITION

ROUTE_IDENT

STATE

BEG_MP

END_MP

INT_LENGTH

NO_LANES

iii. All other fields in these tables should be populated according the data dictionary (Appendix J: Database Data Dictionary)

4. Road Widths: The contractor will be required to take new widths on all roads collected. More information on how to calculate road widths can be found in Appendix D: Video Measurements

F. SITE SUBMITTAL TO EFLHD

As the contractor completes collection and processes the condition data at sites, the contractor shall submit the video, data, and any other pertinent files to EFLHD for review and approval.

B-6

After collection at a site the contractor will have 30 business days to submit the site’ to EFLHD. The site pavement distress analysis shall be complete with populated database to reflect the ratings obtained.

The final video (.jpegs or other collected format) shall also be submitted to EFLHD within 30 business days. Any files not submitted via email or other digital file transfer options shall be mailed to the COR at the following address:

CFLHD-FHWA

12300 W Dakota Ave.

Suite 380

Lakewood, CO 80228

G. QUALITY CONTROL (QC) OF SUBMITTED SITES

EFLHD QC procedures:

Upon receipt of a processed site data set from the contractor, EFLHD will perform a comprehensive QC review. Individual site data sets will be reviewed in the order they are received. After completion the QC review, EFLHD will notify the contractor of any data errors or deficiencies in a timely manner. Notification to the contractor of QC results (pass/fail) will be delivered electronically by email. The contractor will have the following amount of time to correct all identified errors and deficiencies and resubmit the dataset to EFLHD. These timeframes apply to submittals after each failure.

• Sites with less than 10 miles with cracking/patching identification errors – 5 business days

• Sites with 10 or more miles with cracking/patching identification errors – 7 business days

• Site with only non-cracking/patching errors – 5 business days

Subsequent submittals after an initial failure, EFLHD will again perform a full QC review and notify the contractor of its status (pass/fail). EFLHD will also ensure that no new errors were introduced into the subsequent submittals. After the second

EFLHD QC review, the contractor will be provided a detailed report of all identified data errors and deficiencies if the dataset QC review fails a second time. This QC review process will repeat until the site dataset passes its review and is accepted.

B-7

H. DATA PROCESSING FOR INITIAL SITES 1-10:

The first 10 sites (or otherwise decided upon number) shall be agreed upon by the contractor and EFLHD during the kickoff meeting. During the processing of these initial

10 sites, EFLHD intends to assist the contractor with its development of data processing procedures, provide training as needed, or any clarification of EFLHD methods while continually relaying the expectations for processing the data. The goal is to foster a positive working relationship between parties while methods and processes are learned and efficient workflows are achieved. The contractor will be given 90 business days to complete the processing of the initial sites, and to accomplish the mutual training objectives.

During the processing of the initial 10 sites and likely afterwards, updates to the geodatabase template and data dictionary should be expected. EFLHD shall work with the contractor on these updates as determine how this affects the collection and data delivery schedule. EFLHD is committed to working with the contractor to quickly and successfully achieve complete and accurate data submittals with these initial site collections.