Attachment J.7, State Traffic Records Assessment Program (STRAP) Assessment Facilitator System User Manual.pdf

State Traffic Records Assessment Program

(STRAP)

Assessment Facilitator System User Manual

5/28/2019

Maintained by:

Traffic Safety Analysis, Systems & Services, Inc. 1213 Stringtown Road Grove City, OH 43123

STRAP Support:

Kara Mueller: kara.mueller.ctr@dot.gov Phone: (614) 256-1529

Contents Introduction & Access

Other Information

My Profile

Document Library

Assessment Facilitator Tab

Modules Overview

Assessment Report

Rating Distribution

Assessors Summary

Manage Tokens

Documents Linked to Questions

YYYY Assessment Report

My Tools & Reports – Reconciliation Report

Managing Assessors

Edit Assessor Assignment

Assessor Details/Assessment Participation

Manage Assessor Tokens

Final Report

Content on Final Report Tab

Module Narratives (Module Summaries)

Introduction

Considerations

State Highway Safety Office Representatives

Acronyms

Final Report Graphs and Tally

Finding Report

Generating the Final Report

Management

Assessment Information

Assessment Schedule

Respondent Breakdown

Shared, Forwarded and Declined Report

Respondent Progress

Documents Linked to Questions

View Tags

Q Links

Rating Distribution

Role Distribution

Comprehensive Assessment Response Report by Module

Comprehensive Assessment Response Report by Question

YYYY Final Report Download

System Usage

Token Usage Report

Assessment Activity

Assessment Top Users

Active Users

Email Log

Assessment Report

Change History

STRAP System Tally

Raw Findings Ratings Report

Assessor and Module Leader Progress

Ballot Progress – Assessment Level

Ballot Progress – Module Level

Finding Progress

Finding Report

Reconciliation Report

Final Report – Graphs and Charts

Final Report Graphs & Tally

Rating Distribution – Bar

Rating Distribution – Pie

Assessment Results by Question Rank

Assessment Tally

Module Tally

Default Report Settings

Introduction & Access Those identified as Assessment Facilitators are expected to supervise the Assessors and Module Leaders for the assigned assessment. Specific dates for each period will be available once an assessment has been scheduled. For more detailed information on each date and the assessment process, please see the Procedures Manual for State Traffic Records Assessments.

Those people identified as Facilitators will receive a token via email that will allow them to access STRAP. A token is an imbedded hyperlink that provides access to the system. It will look very similar to the image below.

Token

Upon NHTSA TR Team receipt of request Initial pre-assessment conference call 1 month prior to kickoff meeting Facilitator introduction pre-assessment conference call

Between Facilitator conference call and kickoff State Coordinator assigns questions, enters contact information into STRAP, and builds initial document library

As se ss m en t

Monday, Week 1 Kickoff meeting Tuesday, Week 1 – 7pm EST, Friday, Week 3

Round One Data Collection: State answers standardized assessment questions

Friday, Week 3 – Wednesday, Week 5

Round One Analysis: Assessors review State answers and rate the responses and, if needed, request necessary clarifications

Thursday, Week 5 – Tuesday, Week 7

On-Site Preparation: State Coordinator and Facilitator gather questions, issues, clarification to prepare for on-site meeting.

Wednesday, Week 7 On-Site Meeting: Facilitator, State Coordinator and selected respondents meet on-site to discuss assessment results.

Thursday, Week 7 – 7pm EST, Friday, Week 9

Round Two Data Collection: State provides final response to the assessors’ ratings.

Friday, Week 9 – Sunday, Week 12

Round Two Analysis: Assessors review additional information from the State and, if needed, adjust initial ratings.

Monday, Week 12 – Sunday, Week 13 Facilitator prepares final report

Week 13 NHTSA delivers final report to State and Region (After completion of assessment, date set by State) NHTSA hosts webinar to debrief State participants

Tokens cannot be shared as they are specific to each person and role. The email containing the token should be saved for the duration of the assessment process.

Upon accessing STRAP, you will be asked to confirm your participation in the assessment. Users must confirm participation for each new role they take on. (Respondent, Coordinator, Assessor, etc.)

Once you have confirmed your involvement, additional tabs and instructions will be available to you.

The top of the screen will display the progress of the assessment, a welcome message and instructions on how to begin.

The system will send out reminders at the end of each phase for Assessors and Respondents. Phase change notices also go out to Facilitators, State Coordinators and Region Coordinators.

Other Information The tabs in the menu available to all roles are: Help, Feedback, My Profile, Logout, Assessment Advisory and NHTSA Privacy Policy.

Additional help can be found on most screens throughout the system. Clicking on a label or report help will pop up a screen with additional details.

Many reports can be exported to simple spreadsheets by using the Export or Download links.

Please use the feedback to provide comments, requests or report bugs. This will provide the administrator with your exact location in the system and the session state at the time your feedback was submitted.

Automatic Logout – The system will log you out after being left idle for 4 hours or after 8 hours of continuous use.

My Profile Visit the My Profile icon to provide all required contact information.

Document Library Visit the Document Library to view all documents uploaded by the State. If a previous STRAP assessment has been conducted, the past documents will also be available.

All respondents will have access to the document library and will need to reference these documents as part of providing a question response. Please keep in mind that respondents may add new documents throughout the assessment process.

The Library can be searched by keyword and the columns can be sorted and/or filtered by left clicking on the column header.

Documents can be filtered on the left by State, Module, System and Document Type.

To download a single document, tick the checkbox next to the document and then click on the Download Selected button. To download multiple documents, tick the checkbox next to multiple documents and click the Download Selected button.

The Document Tag Count shows a tally of the tags added to the document. To view the tags for a specific document, click on the View Tags hyperlink. This will open the View Tags screen which displays the tags, document tracking and document details.

Clicking on the Q links icon opens the Document to Question Linkage Report. This report displays the questions to which the document has been linked.

Assessment Facilitator Tab Assessment Facilitators may monitor the progress of the assessment via the Assessment Facilitator and Management tabs. The information and reports available will change depending upon the active assessment phase. Please note that these reports are pulling in live data based on the current phase of the assessment. If all Assessor assignments have not been made by the Administrator, not all reports will populate properly.

Modules Overview The modules overview is organized by each module included within the selected assessment, whether or not the module has been assigned to at least one respondent and tallies of assignments, submissions, ballots and findings.

Module Assigned to a respondent? - A checkmark will appear if at least one respondent has been assigned to that module. (Respondents are assigned to modules prior to being assigned to questions.)

Questions Assigned vs Total Questions - The number of questions assigned to at least one respondent out of the total number of questions for that module. (A question can be assigned to more than one person.)

Responses Submitted vs Assigned Questions - The number of responses submitted by State respondents out of the total number of questions assigned for that module.

Ballots Submitted vs Total Ballots - The number of ballots submitted by Assessors out of the total number of ballots expected for that module.

Findings Submitted vs Total Findings – The number of findings submitted by the Assessor out of the total expected for that module.

Assessment Report Display of all questions, submitted responses, submitted ballots and submitted findings. The responses, ballots and findings will not be displayed until they are submitted.

To filter this report, change the module, sub-module and/or number of rows above and click Go.

Perform keyword searches by entering a word into the Search box and clicking Go.

To view all respondent answers along with any attachments, click on “Click to view more” in the State Responses section. Click on the magnifying glass icon to view and download attachments.

Rating Distribution The number of questions that have a specific rating and their rank of very, somewhat or less important.

The rank is predetermined and listed for each question in the assessment advisory.

Assessors Summary Listing of all Assessors, Module Leaders and the Facilitator for the selected assessment. Click on the Go To Manage Assessor Page button to edit Assessor assignments. Assignments should not be changed without consulting a System Administrator.

Manage Tokens View all Assessor and Module Leader tokens, when they were sent and if they have been used. If a token does not have a last used date, the person has not used STRAP since the last time a token was sent. Tokens may also be resent from this page by clicking on the Send Token hyperlink.

Documents Linked to Questions View all documents that have been linked to questions within the selected assessment.

Click on the View Tags hyperlink to view the document tags, tracking (when used) and details.

Clicking on the Q Links icon will display the specific question(s), including page and section references, to which the document has been attached.

YYYY Assessment Report If the State has participated in a previous STRAP assessment, the Assessment Report for that year will be available. The year shown in the title of the report is the year of the last assessment. The report will display the final responses submitted along with the attached documents and the rating. State Coordinators have a similar report that excludes the Finding.

My Tools & Reports – Reconciliation Report The Reconciliation report displays the Assessor Ballots and Module Leader Findings for all questions.

The report can be filtered to only see those questions where there is, or is not, a rating difference. All ratings are color coded. Please keep in mind, this report will display saved and submitted information.

If a record says “no progress” it has not been updated since the round began.

Managing Assessors Normally, the Administrator will be responsible for assigning Assessors within the system. However, TR Team Members and Facilitators can update or view information on this page via the Go To Manage Assessor Page button in the Assessor Summary section of the Facilitator Tab.

The Assessor Assignment Report displays the modules, number of Assessors, and whether or not an Assessor was assigned.

Edit Assessor Assignment To add a person, click the Click to Start Adding New Assignment button.

Then use the select lists to select an Assessor, role and module and click on the Save Assignment button.

You may add the same person more than once to an assessment, presumably under different systems.

To delete a person’s role, click the pencil and paper icon next to their name. This will populate the Assessor, module, and role information. Click on the Delete Assignment button to remove the role from the selected Assessor. Please note that adding and/or removing roles once the analysis round is underway will cause issues.

*Names will not appear in the Assessor drop down list until an Administrator adds them to the Assessor Pool for the selected assessment.

Assessor Details/Assessment Participation To view the Assessor Details, click on the pencil icon next to their name. This will open the Edit/Add Assessor Assignment screen again. Scroll down to view additional information about the Assessor, including the assessment history. For pre-STRAP assessments the module and role information is not available, but for STRAP assessments this information will be present.

Manage Assessor Tokens View all Assessor and Module Leader tokens along with the token expiration dates, when they were sent and if they have been used. If a token does not have a last used date, the person has not used STRAP since the last time a token was sent. Tokens may also be resent from this page by clicking on the Send Token hyperlink.

To exit out of the Assessor management area, click on the Home or Facilitator tab at the top left of the screen.

Final Report The Final Report tab becomes available to Facilitators and Module Leaders when an assessment moves into the 2nd Data Analysis round.

What happens during the 2nd Data Analysis round?

Assessors

• Complete ballots normally. They do not have access to the Final Report Tab.

Module Leaders

• Complete Final Findings on Module Leader Tab.

• Complete Module Summaries and Considerations on the Final Report Tab for assigned module(s).

• Module Leaders MUST be done with all work by the end of the round.

Facilitators

• Review Final Findings, Module Summaries and Considerations as they are completed. This will save revision time later.

• Enter the State Highway Office Representative as well as State-specific acronyms.

• Make sure ALL Module Leader findings have been submitted or the question will be missing from the Final Report printout.

• Create Introduction and enter during the Facilitator round.

What happens after round ends?

Assessors and Module Leaders should no longer access the system and/or update content. During this time the Facilitator will be reviewing and finalizing content. Once all content has been reviewed and deemed final, the appropriate NHTSA Traffic Records Team member should be informed. It is no longer necessary to download and email the final report to the designated Traffic Records Team member. They will run the final report directly from the system, review it and send it to the State.

Content on Final Report Tab Facilitators are able to view and edit all sections below.

• Module Narratives – Section in which Assessors enter their summaries.

• Introduction – Section for Facilitators to enter the report introduction.

• Considerations - Area for Module Leaders to enter considerations for their assigned module(s).

• State Highway Safety Office Rep – The Facilitator should enter the representative’s name. This may be the State GR.

• Acronyms –The Facilitator should use this section to include any that are specific to the State.

National acronyms do not need to be entered as a set list is already part of the final report.

• Final Report Graphs and Tally – This information is generated by the system based on the ratings entered for each module. These cannot be changed and are based on complex formulas weighted at the module level.

• Finding Report – Listing of all Module Leader Findings as well as the ability to edit the findings if you are the Facilitator.

• Final Report – Area where copies of the Final Report can be generated. The resulting document will be an .rtf file. Module Leaders do not have access to this section.

Module Narratives (Module Summaries) Module Leaders are responsible for completing a module summary in addition to providing final findings and considerations.

To create and edit module summaries, go to the Final Report Tab and expand the Module Narratives section. Module Leaders may edit these summaries up until the end of Round 2: Data Analysis. To enter the summary, use Click to Edit.

Enter the summary text as plain text and Save or Submit at the bottom of the screen.

Save – Save the work done so far. Not yet ready for review.

Submit – Submit final summary. Text will be visible and ready for review.

• As text is saved and/or submitted, the status, submit date and current report status will update.

Print Preview – Preview your work. Only works after saving.

After saving a summary, the module should show as In Progress. After submitting a summary, it should show as Draft. A summary may be edited and submitted as many times as needed prior to the end of the phase. The summary will remain at draft status until the assessment is closed out.

Introduction Facilitators are responsible for completing a final report introduction. The introduction should be global remarks (a single paragraph is all that is needed) that addresses all or some of the following:

1. Identifies changes to the specific TR system since the last assessment

2. Focuses on issues that the States raised in their kickoff and midterm meeting and addresses any topics that the States said they were concerned about this assessment covering.

3. Identifies State practices that are exceptional – vis a vis the other states TR systems.

4. Acknowledges the ongoing projects that the states are doing to improve their data system.

To begin entering an introduction, click on the Click to Edit button.

Enter the introduction into the text field. Click the Save button to save a draft. Click the Submit button to submit a final version of the introduction to be included in the final report.

Considerations Module Leaders should enter 3-5 Considerations for each of their assigned modules. To add considerations, click on the Add Consideration button.

Select the Module, Display Order and enter a Consideration. The display order determines the order the considerations will appear under each module in the Final Report.

After clicking save, the Cancel, Delete and Submit buttons will become available. Delete will remove the record, Save will save any further edits and Submit will submit the record through to the Facilitator.

All saved and/or submitted records will appear on the main Final Report tab under Considerations. Each Module Leader will only enter and edit Considerations for their assigned module(s). To edit an existing consideration, click on the pencil icon under the Edit column. All considerations must be submitted to appear in the final report. Saved considerations will not appear in the Final Report.

State Highway Safety Office Representatives This section allows for the entry of the Governor’s Representative. Click on the Edit State Reps button to add a name.

Click on the three horizontal lines in the Name field to search for and select an existing name. To add a new contact, use the link under the Name field. After selecting a name, click the Create button to add the contact as a State Highway Safety Office Representative. To add additional SHSO Reps, repeat the process.

Acronyms The final report template contains a general set of assessment acronyms. The acronym section has been provided to allow for the inclusion of any State-specific acronyms. Click on the Edit Acronyms button to navigate to the screen to edit and add new acronyms.

To add a new acronym, click on the Create button. This will open the Manage STRAP acronym screen.

To edit an existing acronym, click on the pencil icon next to the selected record.

Enter the acronym and definition. Comments are optional and are not included in the final report. Click the Create button to save the new acronym. To remove an acronym, click the Delete button.

Final Report Graphs and Tally This same set of reports can also be found under the Management screen, within the Final Report section. Please see the Management > Final Report section for descriptions of these reports. They are included in the Final Report section for ease of access during final report review.

Finding Report The Finding Report displays the current findings made by each Module Leader. If the Module Leaders are currently providing findings, the most recently saved information will be displayed here. Use the Search and Filter section to change the information being displayed.

Editing Module Leader Findings (Conclusions) – Go to the top right of the Finding Report Section and click on the Edit Findings button.

This section is only accessible to Facilitators and Administrators. To edit a finding and/or rating, click on the pencil icon next to the desired record. This will open the Edit Finding screen.

Use the Rating select list to change the rating (if needed) and edit the Finding Body (if needed). Click on Apply Changes to save your work.

Each time you make a change, the system will return you to the main screen and the first question available. To avoid paging through hundreds of questions, it helps to filter down to a specific module. To do this, click on the Module column header. All other columns can be filtered and sorted if needed.

Generating the Final Report Click the Generate Final Report button to generate a copy of the Final Report. It will come out as an .rtf file that can be converted to word if needed. To update the page numbers in the document’s table of contents, highlight them and hit F9. The final report may be generated as many times as needed. If any updates need to be made to the content, make them in the system and then run the report again. The NHTSA Traffic Records Team members need to be able to access STRAP and create the same final report using the Generate Final Report button.

The final report should not be added back to the system until it is reviewed by NHTSA and delivered to the State. Normally, the Traffic Records Team member assigned to the State will be responsible for uploading a PDF copy of the final report.

Management The management section provides summary reports based on your role for each assessment.

Assessment Information

Assessment Schedule This report displays important assessment dates.

Respondent Breakdown The Respondent Breakdown displays all respondents and their progress in addressing assigned questions.

Shared, Forwarded and Declined Report The Shared, Forwarded and Declined Report displays all questions shared, forwarded and declined by all respondents for the selected assessment. Use the radio buttons at the top of the report to change the display.

Respondent Progress This chart displays the progress of all respondents within the selected assessment in responding to questions. Respondent names will not be displayed, only IDs, which can be used to reference participants.

The Submitted Responses gauge displays the total number of responses submitted compared to the total number of questions assigned less the questions that were declined/forwarded. This display is always for the current/most recent round.

Documents Linked to Questions View all documents that have been linked to questions within the selected assessment.

View Tags Clicking on the View Tags hyperlink to view the document tags, tracking (when used) and details.

Q Links Clicking on the Q Links icon will display the specific question(s), including page and section references, to which the document has been attached.

Rating Distribution Displays the number of each type of rating given for each module by Module Leaders.

Role Distribution This report displays the role distribution for the State’s assessment.

Comprehensive Assessment Response Report by Module This report displays the number of questions within each module that have had at least one response during the life of that assessment. The total number of questions per module is also displayed so that a coordinator may quickly determine whether or not all questions within each module have been answered at least once. Please note that this does not determine which questions have/have not been answered during the current phase, but over the entire assessment.

Comprehensive Assessment Response Report by Question This report is a detailed view of the questions that have not been answered and/or questions that have had at least one answer at some point during the assessment. This lets the coordinator quickly identify the specific questions that have never been answered.

YYYY Final Report Download If the selected State has completed a previous assessment in STRAP, the final report PDF will be available. The Year displayed is the year the assessment was completed.

System Usage Only Administrators, Facilitators and NHTSA TR Team members have access to the System Usage section. Please note that some reports in this section contain information on all States/participants and not just the one selected.

Token Usage Report Displays the date the token was issued, last used, and if the user has confirmed participation. This report is filterable by role. Contact Name is changed to a contact id only when viewed by certain roles.

Please note that the last used date and sent date changes as the assessment moves forward and each time tokens are sent.

Assessment Activity Counts the number of each type of user that visited their section of STRAP. For example, Assessor shows the number of Assessors that visited the Assessor tab that day.

Assessment Top Users Assessment Top Users displays the number of transactions made by each person, within STRAP. Please note that numbers may be slightly inflated for those that are participating in more than one assessment, such as Assessors, etc.

Active Users Displays the most recent users, their location within the system and the time they accessed that area of the system. This report is at the administrative level and shows all active users for all assessments.

The report updates each time you refresh the screen and only displays activity within the last hour.

Email Log The listing of all emails sent by STRAP. These may be tokens or reminders.

Note: This currently does not include the reminder emails sent to Assessors.

Assessment Report The assessment report displays the most recently submitted responses and findings for the current phase. Please note, if the assessment is in a respondent round and a respondent has saved, but not submitted, a response it will not be visible in the report yet.

Change History The full history for each phase, question, respondent and Assessor by question. The download hyperlink at the bottom left may be used to export this table. Each column represents the number of updated Responses, Ballots, and Findings submitted each phase of the assessment.

Clicking on the question number brings up a detailed history report of that question by record type. The reports display all submitted information by round as well as displaying the current text.

STRAP System Tally Summary information for each assessment. The tallies will continue to change until each assessment is completed.

Raw Findings Ratings Report Lists values for the rating received for each question on each completed assessment.

Assessor and Module Leader Progress Ballot Progress – Assessment Level Displays a bar chart of the progress of the Assessors in completing ballots.

Ballot Progress – Module Level Displays a bar chart of the progress of the Assessors in completing ballots by module.

Finding Progress This report displays the progress of each Assessor in completing their findings. As findings are completed, they can be seen in the Finding Report below. Tip: Use this report to make sure all findings are complete prior to the start of a new data collection round. Assessments can’t move forward until all findings are completed each round.

Finding Report The finding report displays findings for each question as they are submitted by Assessors. This report may be filtered and queried.

Reconciliation Report The reconciliation report is also available on the Facilitator tab. Please see the description of the report earlier in this document.

Final Report – Graphs and Charts

Final Report Graphs & Tally

Rating Distribution – Bar Displays the number of meets, does not meet and partially meets ratings each module received.

Rating Distribution – Pie Displays the percentage of meets, does not meet and partially meets ratings received for the entire assessment.

Assessment Results by Question Rank The number of questions from each ranking of importance (as defined in the assessment advisory) and the rating received.

Assessment Tally Summary tallies based on the current assessment information. A description of each type of tally is included below.

Meets, Does Not Meet, Partially Meets – Tally of the rating of how well the State meets the

Advisory ideal, as determined by the Assessors, based on the information provided by the

State respondents.

Respondents Assigned and Participated – Displays the number of respondents assigned to questions and the number that actually provided a response.

Total Questions and Total Answered – The total number of questions asked for the selected type of assessment and the total number that received a response.

Total Assignments and Total Answered of Total Assigned – The sum of all questions asked of all respondents. (For example, if 30 respondents were assigned 10 questions each, total assigned would be 300.) The total answered is the total number of all question assignments that received a response.

Less, Somewhat and Very Important – The tally of assessment questions, by importance ranking, as defined in the assessment advisory.

Module Tally The module tally report displays the same information as the assessment tally above, but is broken out by each module included in the selected assessment.

Default Report Settings Allows the user to determine which reports automatically display upon enter the Management area.