Attachment J.9, State Traffic Records Assessment Program (STRAP) NHTSA Headquarters - System User Manual.pdf

State Traffic Records Assessment Program (STRAP) NHTSA Headquarters User Manual

10/15/2013

Maintained by:

Traffic Safety Analysis, Systems & Services, Inc. 1213 Stringtown Road Grove City, OH 43123

STRAP Support:

Kara Mueller: kmueller@tsass.com Phone: (614) 256-1529

Contents Introduction & Access

STRAP Home Screen

NHTSA Team Member Tab

Edit Assessment Page

Step 1: Assign State and Region Staff

Searching, Filtering & Reports

Step 2: Assign Assessors

Edit Assessor Assignment

Assessor Details

Manage Tokens

Step 3: Respondent Identification

Module Assignment Report by Respondent

Module Assignment Report

Step 4: Question Assignment

Question Assignment – Module Report

To Assign Questions

Management

Assessment Information

Assessment Tally

Assessment Schedule

Respondent Progress

Assessment Documents

Rating Distribution

Role Distribution

Comprehensive Assessment Response Report by Module

Comprehensive Assessment Response Report by Question

System Usage

Token Usage Report

Assessment Activity

Assessor and Module Leader Progress

Ballot Progress – Assessment Level

Ballot Progress – Module Level

Finding Progress

Finding Report

Final Report

Final Report Checklist

Final Report Graphs & Tally

Assessment Results by Question Rank

Rating % distributed by Rank

Assessment Tally

Module Tally

Real-Time Recommendations Reports

Recommendations

Sub-Module Recommendation Score

Recommendations Score by Sub-Module

Default Report Settings

Introduction & Access NHTSA Headquarters are the only accounts (besides Administrators) that use logins. Logins do not expire and can be used throughout the assessment process. All others including facilitators, assessors, and respondents use tokens; tokens are temporary and expire when a particular state assessment is closed. For more detailed information on each date and the assessment process, please see the Traffic Records Assessment Manual.

Kickoff Date The date of the state kick-off meeting.

1st Q&A Start Date Respondents begin answering assigned questions.

1st Q&A Closing Date Respondents finish first phase. Assessors begin reviewing state materials.

2nd Q&A Start Date Respondents begin answering second round of questions.

2nd Q&A Closing Date Respondents finish second phase. Assessors begin reviewing state materials.

3rd Q&A Start Date Respondents begin answering third round of questions.

3rd Q&A Closing Date Respondents finish third phase. Assessors begin reviewing state materials.

Final Findings Close Date Assessors must complete review of state's materials and submit all findings.

Final Report Date Assessment facilitator must have the final report complete.

Report Out Date Facilitator presents final report and summarizes assessment’s findings to the State's TRCC via a webinar.

Log in by going to http://portal.nhtsa-tsis.net/strap/f?p=150

If you do not have, or do not know your user name and/or password, contact Kara Mueller at kmueller@tsass.com or by cell at (614) 256-1529. Please note this phone is on Mountain Time.

http://portal.nhtsa-tsis.net/strap/f?p=150� mailto:kmueller@tsass.com�

STRAP Home Screen The STRAP home screen contains news and system status, as well as information about what a NHTSA Team Member does. Currently, there are three tabs at the top: Home, NHTSA Team Member, and

Management. The tabs available on the home screen will vary depending upon the user’s access rights. At the bottom you will see four buttons which will appear on every screen.

Also at the top you will see the “breadcrumb” navigation, aptly named because it tracks where you are in the site in relation to the home page. In this picture, if you are on the State and Region Staff page and you would like to return to the previous page, click Edit Assessment. Breadcrumb navigation appears on all pages.

At the bottom you will see buttons which will appear on every screen.

From left to right:

House: Takes you back to the home screen.

Chat bubble: Discussion board (in development) Question mark: Help / frequently asked questions.

Bug: Report a bug, request an enhancement, or make a comment.

Person: My profile Letter: Mass email (in development) “X”: Log out

General help text and user manuals are available in the STRAP Help System.

Additional help can be found on most screens throughout the system. Clicking on a label or Report Help will pop up a screen with additional details.

Many reports can be exported to simple spreadsheets by using the Export or Download links.

Please use the feedback to provide comments, requests or report bugs. This will provide the administrator with your exact location in the system and the session state at the time your feedback was submitted.

NHTSA Team Member Tab By clicking the NHTSA Team Member tab, the current assessments screen and instructions will appear.

Current Assessments are those that are active within the system. Future Assessments are those that have not been completed or started. To begin working on an assessment, click on the small paper and pencil icon next to the desired state and date. Editing allows you to assign a NHTSA Team Member, set due dates, select modules, assign State and Region Staff, Assessors, Respondents and Questions.

The cells in the table show green checkmarks indicating whether assessment parameters have been defined, if assessors have been assigned to the assessment, or if at least on respondent has been assigned to all questions. In the Future Assessments section, an arrow is circled in red. When you see this kind of arrow anywhere in the system, you may click on it to expand a selection. To edit a state, click the paper and pencil icon to the left of the state name.

Edit Assessment Page By clicking on the pencil and paper icon, you see the page where the timetable for the assessment is maintained. At the top of this and many other pages in STRAP, is a basic status bar containing information about the assessment you are in, and information about the current assessment phase.

The labels such as state, kickoff date, etc. contain help text. Click on them to learn the definition. Click on the calendar icon or type in the field to enter a date for each phase. The dates must be in chronological order. Click on the Apply Changes button to save the dates. Because this is an automated system, many processes are driven by this screen. Dates can be created now and changed at a later time, as long as the date currently in the box is sometime in the future. For example, if your first Q and A closing date is October 15, 2013 and on the 16th you decided that you weren’t ready for it to close, you will not be able to change it. If the date has passed, processes have run, emails to respondents and assessors have been sent, and there is no going back. So be sure to monitor this screen carefully if you need to adjust the schedule.

At the bottom of this screen is where you define what modules you would like to include. The double arrows move all of the items to the opposite side, a single arrow moves one item to the other side. In the picture below, all modules are selected to be included in the assessment. To reset the modules to what they were before you came to this screen, click the arrows that form a circle.

Step 1: Assign State and Region Staff In this screen, the NHTSA Team Member names the NHTSA Region Assessment Coordinator and the State Assessment Coordinator. You can have more than one entry for each. The Contact ID box uses names from TRIPRS; if you do not see the name of the person you wish to select in this list, they must first be added to TRIPRS, or by using the Name not on the list? Click to add a name function. As a

NHTSA HQ user, you have a login to STRAP and TRIPRS; your login information is the same for both systems.

To add a person, click the arrow to the side of the role box and select NHTSA Region Assessment Coordinator or State Assessment Coordinator. Select a person from the Contact ID box, and then click the Add Staff Member button.

If the person you wish to add is not on the list, add them by clicking on the add a name function next to the name box.

By clicking Send Token in any person’s row, the system will e-mail the individual a token (or access link).

This token will be personalized based on the person’s state and what role or part they play in the assessment.

Searching, Filtering & Reports The following area allows you to search for people in some group. On this page, it is those who are defined as NHTSA Region and State Assessment Coordinators. You will see this search function on other pages as well.

By clicking the Actions button you will see the following menu, where you can filter the table by column, adjust rows, get help, save a report, and download a report to .csv, html, and e-mail.

At the bottom of the screen is the ability to add state staff. These individuals are not given tokens, but the information is used during the generation of the final report.

When you are finished, move onto the next step using the Go to Next Step: Assign Assessors button.

Step 2: Assign Assessors On this page is where the assessment team leader, module team leaders, and assessors are assigned.

The Assessor Assignment Report displays the modules, number of assessors, and whether or not a module leader was assigned.

Edit Assessor Assignment

To add a person, click the assessor drop down box to view and select from a list of eligible assessors.

Then select the same for the module box below. Finally, choose the role that person will have in the process – as an assessment facilitator, module leader or assessor and click Apply Changes. You may add the same person more than once to an assessment, presumably under different systems. Additional help is available just above the listing of assigned assessors. To edit a person’s role or view assessor details, click the pencil and paper icon next to their name. This will populate the assessor, module, and role fields near the top.

Assessor Details Once you have clicked on a person’s paper and pencil icon, click on the Assessor Details tab to view information about the assessor, including the assessment history. For pre-STRAP assessments the module and role information is not available, but for STRAP assessments this information will be present.

Manage Tokens Click on the Manage Tokens tab to view token information. As the screen help under “What is a Token” states, tokens are embedded links sent via e-mail that contain special access information, allowing the assessors of your choosing to provide findings related to this assessment. The assigned NHTSA Team Member has set a schedule for the release of these tokens and the assessment in general. The STRAP online system requires these dates. If you have any questions, please contact your NHTSA Team Member.

Tokens can be revoked by removing all modules from an assessor. Alternatively, tokens can be temporarily turned off at request by emailing strap@nhtsa-tsis.net. Please include the contact's full name, email address and other instructions as needed, such as reactivation date. Tokens cannot be reassigned to another contact, they are created uniquely for a specific contact.

To return to the previous page, click “Edit Assessment” in the breadcrumb navigation near the top of the page.

When you are finished assigning assessors, you may move onto respondent identification using the Go to Next Step: Respondent Identification button.

mailto:strap@nhtsa-tsis.net?subject=Temporary%20Token%20suspension�

Step 3: Respondent Identification At the top of the page are two reports, followed by instructions, followed by the tools necessary to update respondent information. Please keep in mind that the STRAP State Coordinator will normally be responsible for assigning respondents. Respondents are first assigned to modules and then, in Step 4, they are assigned to specific questions within those modules.

Module Assignment Report by Respondent This report displays each contact identified as a respondent. The modules assigned to each respondent are shown with a green checkmark.

Module Assignment Report This report displays the number of respondents assigned to each module. The green check mark will appear when at least one respondent has been assigned to the corresponding module.

The Respondent Identification screen is where a State Assessment Coordinator and/or NHTSA Team Member identify state staff as being part of the question and answer process and assign modules / sub-modules.

Contacts in Database: Displays all contacts currently in the master database for the selected assessment state. Use this report to determine which contacts already exist, if they are missing a valid email address and if they need to be updated.

Search: Using Search Contacts, search for a contact to be assigned as a respondent. Please note, the search tool only looks for existing contacts. If the contacts for this state have not yet been edited for this assessment, they may be updated and/or added below in the Edit Respondent section.

Select and Edit: Select the appropriate contact by clicking on the small paper icon next to their name;

the Edit Respondent form below will then populate with the contact's information. Use the shuttles below to denote which modules and sub-modules the respondent should be assigned to. Adding at least one module to a contact designates them as a respondent. It is a good idea to review the Advisory to get an idea of what questions are in a sub-module. Using this utility can make the task of assigning questions much easier than assigning them individually.

Below the modules, sub-modules may also be assigned to the respondent. Assigning sub-modules will cause all questions within them to be assigned to the respondent. Use the Update Respondent button to save changes.

http://www.nhtsa-tsis.net/stateAssessments/docs/NHTSA_TRProgram_Assessment_Advisory_811644.pdf�

Update: Click on the Update Respondent button to save the selections. This same process may be used to edit assigned modules and sub-modules for contacts that have already been selected as respondents.

Add: If you need to add a new contact record, enter the information into the blank Edit Respondent form and click on the Update Respondent button to save. Review Work: Use the Module Assignment Report above to determine whether or not all modules have been assigned to at least one person. When you are ready to begin assigning specific questions to respondents, click on the Go To Next Step: Assign Questions button, found at the bottom of the page.

Step 4: Question Assignment The State Assessment Coordinator enters this information. In the image below, Citation / Adjudication is selected, as well as the System Description sub-module. There are seven questions in the System Description sub-module. Scroll down the page to see the Question Summary and Edit Assignments. The Edit Assignments section lists the people named as question respondents for that system (this was done in the Respondent Entry section). By clicking on Module or Sub-Module, you can assign that person all questions for either one of those categories. Click the checkboxes to assign questions one by one. To see the detail on what each question is, click the number above the checkbox. In this case, number 1 is selected so we see the detail for this question. If the question number is white, the question has been assigned to someone. If it is red, the question is not assigned to anyone. Repeat this process for every module and sub-module.

Question Assignment – Module Report This report displays total questions in module, whether or not all questions have been assigned to at least one person, and the number of respondents assigned to questions within the module. Click on the number to view a list of the respondents assigned.

After a module is selected in the lower region of the screen, the Question Assignment Sub-Module Report will display. It shows the sub-modules contained in the selected module, questions assigned to at least one person out of all questions in the sub-module, and the number of respondents assigned to that sub-module.

To Assign Questions To assign questions, select a module and sub-module. All respondents assigned to the selected module in the previous step will appear. Use the checkboxes to assign or un-assign specific questions. Click on the Update Assignment button to save changes.

Review Work: Use the Question Assignment summaries at the top of the page to confirm whether or not all assignments have been made.

Need to add another respondent? The Add Respondent button will return you to the Respondent Identification page to add or edit respondents.

Need to add questions to existing respondents? Make sure they are assigned to the module on the Respondent Identification page, then return here (Question Assignments) and begin assigning new questions. Please note that if you change the assigned modules for respondents that have been assigned questions, these assignments will be lost.

Name - Click on a respondent’s name to view their contact details and a report of all questions assigned to them. Only respondents assigned to the currently selected module will appear. To add more names to this module, click on the Add Respondent button, add or edit a respondent to include the desired module.

(Un)Assign All - Click on a module or sub-module under the (Un)Assign All column to assign or un-assign the respondent to all questions within that section.

Once the assessment is underway, you may find that some checkboxes are now inactive for certain respondents and questions. This means that specific question has received a rating of “meets the standard of evidence” and may no longer be changed.

To remove a respondent from the module, click on the red “X” icon next to their name.

If the “X” is unavailable and the (Un)Assign All column now says N/A, this means one of the questions in the module that the respondent has been assigned to has met the standard of evidence. Once that happens, the respondent can no longer be removed from that module.

Question Number – Click on the question number to view the question text, evidence requirement and question rank.

Tip: Save assignments BEFORE moving to other questions or clicking on any links and/or reports.

Otherwise your changes will be lost.

To view individual questions: Click on the question number in the table header to view the Advisory question, evidence requirement, question rank, and question number.

If you have unassigned questions, they will appear in the Unassigned Questions Report.

Management Select the appropriate assessment from the dropdown and give the system a moment to process. Click on each report grouping in the menu below to expand the choice. Please note that these reports are based on the current phase. At the end of each respondent round, all responses of any kind are submitted. This includes non-responses.

Assessment Information

Assessment Tally Use the report pulldown to select the report you wish to view.

1. Primary Report – This report contains a summary of information contained within the other tally reports. Please see the reports below for details on each column.

2. Participation Report – Number of respondents assigned to questions within each module, the number that have responded to at least one question and the number of assessors assigned to each module.

3. Questions Tally

• Unique Questions – Total number of questions in each module as set by the assessment advisory.

• Unique Assigned – Number of questions that have at least one respondent assigned.

• Unique Submitted – Number of questions that have at least one response submitted.

• Unique Completeness – Unique Submitted divided by Unique Assigned.

• Total Assignments – The total number of question assignments for all respondents in the module.

• Total Submissions – Sum of the number of responses submitted out of the number assigned.

• Total Completeness – Total submissions divided by total assignments.

4. Rank Report – Each question has been given a rank in the assessment advisory. This report displays the number of very, somewhat and less important question in each module.

5. Rating Report – The number of question in each module that are currently rated as Meets, Does not Meet or Partially Meets the standard of evidence as well as the number of assessors assigned to each module.

Assessment Schedule This report displays important assessment dates.

Respondent Progress This chart displays the progress of all respondents within the selected assessment in responding to questions. Respondent names will not be displayed, only IDs which can be used to reference participants.

Assessment Documents Search for items provided by the state in the document library.

Rating Distribution Displays the number of each type of rating given for each module by module leaders.

Role Distribution This report displays the role distribution for the state’s assessment.

Comprehensive Assessment Response Report by Module This report displays the number of questions within each module that have had at least one response during the life of that assessment. The total number of questions per module is also displayed so that a coordinator may quickly determine whether or not all questions within each module have been answered at least once. Please note that this does not determine which questions have/have not been answered during the current phase, but over the entire assessment.

Comprehensive Assessment Response Report by Question This report is a detailed view of the questions that have never had an answer and/or questions that have had at least one answer at some point during the assessment. This lets the coordinator quickly identify the specific questions that have never been answered.

System Usage

Token Usage Report This report allows you to view token information about all users for the selected state.

Assessment Activity This report displays the number and type of users over the span of an assessment. Filter the report by role to see only one role or all roles.

Assessment Top Users Assessment Top Users displays the number of transactions made by each person, within STRAP. Please note that numbers may be slightly inflated for those that are participating in more than one assessment, such as assessors, etc.

Email Log The listing of all emails sent by STRAP. These may be tokens or reminders.

Full Response History Displays all responses in all rounds, filterable by question and respondent.

STRAP System Tally

Summary information for each assessment. The tallies will continue to change until each assessment is completed.

Assessor and Module Leader Progress

Ballot Progress – Assessment Level This report allows you to view assessors’ progress by system. You will be able to view only the modules in which you have a role.

Ballot Progress – Module Level

When viewing ballot progress at the module level, the assessor names will be visible to other assessors and module leaders participating in the selected assessment.

Finding Progress Displays the progress of each module leader in submitting findings. The module leader for each will be shown if you are an assessor or module leader for the selected assessment.

Finding Report Search by keyword or select a module and rating from the Search & Filter section and the finding report below will reflect the constraints you have chosen. To view all findings, just use the finding report.

Final Report

Final Report Checklist Displays the Module Leader progress in providing module summaries on the Final Report tab. The Executive and Result summaries are completed by the Facilitator.

Final Report Graphs & Tally

Rating Distribution – Bar Displays the number of meets, does not meet and partially meets ratings each module received.

Rating Distribution – Pie Displays the percentage of meets, does not meet and partially meets ratings received for the entire assessment.

Assessment Results by Question Rank The number of questions from each ranking of importance (as defined in the assessment advisory) and the rating received.

Rating % distributed by Rank The percentage of each rating and rank combination for the entire assessment.

Assessment Tally Displays tally information for the selected assessment.

• Meets, Does Not Meet, Partially Meets – Tally of the rating of how well the state satisfies the standard of evidence, as determined by the module leaders, based on the information provided by the state respondents.

• Respondents Assigned and Participated – Displays the number of respondents assigned to questions and the number that actually provided a response.

• Total Questions and Total Answered – The total number of questions asked for the selected type of assessment and the total number that received a response.

• Total Assignments and Total Answered of Total Assigned – The sum of all questions asked of all respondents. (For example, if 30 respondents were assigned 10 questions each, total assigned would be 300.) The total answered is the total number of all question assignments that received a response.

• Less, Somewhat and Very Important – The tally of assessment questions, by importance ranking, as defined in the assessment advisory.

Module Tally The module tally report displays the same information as the assessment tally above, but is broken out by each module included in the selected assessment. (The example below only shows one module.)

Real-Time Recommendations Reports These reports calculate the recommendation scores and determine which recommendations should be given, based on respondents’ answers.

Recommendations Click on the recommendation to view the Advisory questions related to that recommendation.

Sub-Module Recommendation Score This report displays the score per sub-module.

Max - Calculated by multiplying the question rank by the top possible question rating. Each question has a set rank (see Assessment Advisory) and the top possible rating is “meets the standard of evidence.”

Maximum Values are then tallied to determine the maximum value for each sub-module, module and for the entire assessment.

Example: Q#82 has a rank of very important (3) and the top possible rating is meets the standard of evidence (3), so the Maximum Value for Q#82 is 9. (3x3 = 9)

Earned – Calculated by multiplying the question’s rank by the current rating.

Example: If a question has a rank of less Important (1) and is rated as Meets the Standard of Evidence

(3), then the recommendation score would be 3. (1x3=3)

Recommendations Score by Sub-Module Graphically displays scores by sub-module.

Default Report Settings Allows the user to determine which reports automatically display upon enter the Management area.