Attachment J.6, State Traffic Records Assessment Program (STRAP) Assessor-Module Leader System User Manual.pdf

State Traffic Records Assessment Program (STRAP) Assessor/Module Leader User Manual

10/15/2013

Maintained by:

Traffic Safety Analysis, Systems & Services, Inc. 1213 Stringtown Road Grove City, OH 43123

STRAP Support:

Kara Mueller: kmueller@tsass.com Phone: (614) 256-1529

Contents Introduction & Access

Other Information

My Profile

Assessor Tab

My Ballots

Listing of Ballots

View Responses and Supporting Documents

Save vs. Submit

My Tools & Reports

Assessment Ballot Report

Assessment Documents

Ballot Progress

Rating Distribution

Printouts

My Active Roles

Management

Assessment Information

Assessment Tally

Assessment Schedule

Respondent Progress

Assessment Documents

Rating Distribution

Role Distribution

Comprehensive Assessment Response Report by Module

Comprehensive Assessment Response Report by Question

Assessor and Module Leader Progress

Ballot Progress – Assessment Level

Ballot Progress – Module Level

Finding Progress

Finding Report

Final Report

Final Report Check List

Final Report Graphs & Tally Information

Rating Distribution – Bar

Rating Distribution – Pie

Assessment Results by Question Rank

Rating % distributed by Rank

Assessment Tally

Module Tally

Default Report Settings

Module Leader Tab

My Findings

View Responses and Supporting Documents

Save vs. Submit

My Tools & Reports

Assessment Summary Report

Finding Progress

Rating Distribution

Assessment Documents

My Active Roles

Printouts

Management Tab

Introduction & Access Those identified as Assessors are expected to address the questions assigned to them and will be involved in the assessment process during the highlighted periods. Specific dates for each period will be available once an assessment has been scheduled. For more detailed information on each date and the assessment process, please see the Traffic Records Assessment Manual.

Kickoff Date The date of the state kick-off meeting.

1st Q&A Start Date Respondents begin answering assigned questions.

1st Q&A Closing Date Respondents finish first phase. Assessors begin reviewing state materials.

2nd Q&A Start Date Respondents begin answering second round of questions.

2nd Q&A Closing Date Respondents finish second phase. Assessors begin reviewing state materials.

3rd Q&A Start Date Respondents begin answering third round of questions.

3rd Q&A Closing Date Respondents finish third phase. Assessors begin reviewing state materials.

Final Findings Close Date Assessors must complete review of state's materials and submit all findings.

Final Report Date Assessment facilitator must have the final report complete.

Report Out Date Facilitator presents final report and summarizes assessment’s findings to the State's TRCC via a webinar.

Those people identified as assessors will receive a token via email that will allow them to access STRAP.

This email is first sent out on the date the 1st Q&A closes. A token is an imbedded hyperlink that provides access to the system. It will look very similar to the image below.

Token

Tokens cannot be shared as they are specific to each assessor. The email containing the token should be saved for the duration of the assessor’s involvement in the assessment process.

Upon accessing STRAP, you will be asked to confirm your participation in the assessment.

Once you have confirmed your involvement, additional tabs and instructions will be available to you.

The assessor will see the progress of the assessment, a welcome message and instructions on how to begin. Those identified as Assessors may also be Module Leaders and will see the appropriate tabs for all roles assigned. If you are an assessor but not a module leader, you will not see the module leader tab.

Other Information The bottom of each screen contains icons to return to the home screen, view help or provide feedback and bookmark your token. The Traffic Records Assessment Advisory is also available for download at the bottom of each page.

General help text and user manuals are available in the STRAP Help System.

Additional help can be found on most screens throughout the system. Clicking on a label or Report Help will pop up a screen with additional details.

Many reports can be exported to simple spreadsheets by using the Export or Download links.

Please use the feedback to provide comments, requests or report bugs. This will allow the administrator with your exact location in the system and the session state at the time your feedback was submitted.

My Profile Visit the My Profile icon to provide all required contact information.

Assessor Tab Visit the Assessors tab to provide ratings, reasons and/or clarification requests for the questions within each module assigned to you.

Assessors should review each question, the state response and evidence, to determine a rating, and a reason for the rating. If the assessor believes the state could improve their answer or would like additional information, a clarification request can be sent to those who responded to the question.

Clarification requests can only be used for the first round of questions. The rating, reason and/or clarification request make up a ballot for each question. Upon accessing the Assessor tab, the assessor should see instructions, reports and a listing of ballots to be completed.

My Ballots The ballots visible on the screen may be narrowed down by assessment, module, sub-module and whether or not the ballot has been submitted. Ballots may also be searched using key words.

Listing of Ballots

Information provided with each ballot is defined below.

• Question – The question number and question text.

• Responses – Displays the number of state responses along with the number of supporting documents attached.

• Rating – Displays the assessor’s determination of how well the state response satisfies the standard of evidence.

• Ballot – Reason for the rating selected above as well as any additional comments on the state responses to the question.

• Clarification - If the state responses are not clear or require more information, the assessor should include a clarification request. Available for first round only

• Status - Displays whether or not the ballot has been saved or submitted to the module leader.

• Module – The module and sub-module the question belongs to.

View Responses and Supporting Documents Click on “Click to View” to see the response(s) and/or supporting documents.

Clicking on the small magnifying glass will display all supporting documents attached.

To view each document, click on the download icon.

When you are ready to provide a rating and ballot, click on the “Click to add” hyperlink for the Rating, Ballot and/or Clarification will. All three links will take you to the same page.

From here the assessor may view the question being addressed, each state response and any supporting documentation included.

Click on the small magnifying glass to view the listing of attached documents and download them for review.

Based on the information provided by the state respondents, select a rating and provide a ballot; the reason for the rating. If clarification is needed, enter a clarification request.

Save vs. Submit Ratings, ballots and clarification requests may be edited and saved as many times as needed during, and prior to, the start of the next Q&A cycle. Once you are satisfied, the entire ballot may be submitted to the module leader. Once a ballot has been submitted, it can’t be edited again during the current cycle.

The ballots aren’t seen by anyone else until they are submitted. If you have not submitted your ballots by the end of the Q&A cycle, the system will automatically push your responses through. The submitted ballot can still be seen and will display the information you have submitted.

My Tools & Reports There are several reports available on the Assessor’s tab to help you review and complete your work.

Assessment Ballot Report – This report will display all assessments you are currently involved in, along with the number of ballots you’ve submitted out of the total number expected.

Click on the number of ballots submitted to view a detailed list of the questions.

Assessment Documents - For easy reference, all supporting documents uploaded by respondents for the selected assessment have been listed. Click on the download icon to view a document.

Ballot Progress – Displays a chart of your progress in completing ballots for the selected assessment.

Rating Distribution – Displays the number of each rating you’ve given for each module for the selected assessment.

Printouts The printouts provide downloadable summary reports on the responses and ballots.

Response Printout - This report provides a listing of all responses made by each respondent for each question, in order by module and sub-module.

Ballot Printout - Detailed listing of your assessors ballots, in order by module and sub-module.

My Active Roles – Displays the assessments you are currently assigned to, as well as module(s) you have in each.

Management The management section provides summary reports based on your role for each assessment.

Select an Assessment and role to view available reports and charts.

Assessment Information

Assessment Tally This interactive report displays information about module assignments and the number of questions answered in each module.

There are five reports available in this section under the Reports dropdown box.

1. Primary Report – This report contains a summary of information contained within the other tally reports. Please see the reports below for details on each column.

2. Participation Report – Number of respondents assigned to questions within each module, the number that have responded to at least one question and the number of assessors assigned to each module.

3. Questions Tally

• Unique Questions – Total number of questions in each module as set by the assessment advisory.

• Unique Assigned – Number of questions that have at least one respondent assigned.

• Unique Submitted – Number of questions that have at least one response submitted.

• Unique Completeness – Unique Submitted divided by Unique Assigned.

• Total Assignments – The total number of question assignments for all respondents in the module.

• Total Submissions – Sum of the number of responses submitted out of the number assigned.

• Total Completeness – Total submissions divided by total assignments.

4. Rank Report – Each question has been given a rank in the assessment advisory. This report displays the number of very, somewhat and less important question in each module.

5. Rating Report – The number of question in each module that are currently rated as Meets, Does not Meet or Partially Meets the standard of evidence as well as the number of assessors assigned to each module.

Assessment Schedule This report displays important assessment dates.

Respondent Progress This chart displays the progress of all respondents within the selected assessment in responding to questions. Respondent names will not be displayed, only IDs which can be used to reference participants.

Assessment Documents Search for items provided by the state in the document library.

Rating Distribution Displays the number of each type of rating given for each module by module leaders.

Role Distribution This report displays the role distribution for the state’s assessment.

Comprehensive Assessment Response Report by Module This report displays the number of questions within each module that have had at least one response during the life of that assessment. The total number of questions per module is also displayed so that a coordinator may quickly determine whether or not all questions within each module have been answered at least once. Please note that this does not determine which questions have/have not been answered during the current phase, but over the entire assessment.

Comprehensive Assessment Response Report by Question This report is a detailed view of the questions that have never had an answer and/or questions that have had at least one answer at some point during the assessment. This lets the coordinator quickly identify the specific questions that have never been answered.

Assessor and Module Leader Progress

Ballot Progress – Assessment Level

This report allows you to view assessors’ progress by system. You will be able to view only the modules in which you have a role.

Ballot Progress – Module Level

When viewing ballot progress at the module level, the assessor names will be visible to other assessors and module leaders participating in the selected assessment.

Finding Progress

Displays the progress of each module leader in submitting findings. The module leader for each will be shown if you are an assessor or module leader for the selected assessment.

Finding Report

Search by keyword or select a module and rating from the Search & Filter section and the finding report below will reflect the constraints you have chosen. To view all findings, just use the finding report.

Final Report Final Report Check List Displays the Module Leader progress in providing module summaries on the Final Report tab. The Executive and Result summaries are completed by the Facilitator.

Final Report Graphs & Tally Information Rating Distribution – Bar Displays the number of meets, does not meet and partially meets ratings each module received.

Rating Distribution – Pie Displays the percentage of meets, does not meet and partially meets ratings received for the entire assessment.

Assessment Results by Question Rank The number of questions from each ranking of importance (as defined in the assessment advisory) and the rating received.

Rating % distributed by Rank The percentage of each rating and rank combination for the entire assessment.

Assessment Tally Displays tally information for the selected assessment.

• Meets, Does Not Meet, Partially Meets – Tally of the rating of how well the state satisfies the standard of evidence, as determined by the module leaders, based on the information provided by the state respondents.

• Respondents Assigned and Participated – Displays the number of respondents assigned to questions and the number that actually provided a response.

• Total Questions and Total Answered – The total number of questions asked for the selected type of assessment and the total number that received a response.

• Total Assignments and Total Answered of Total Assigned – The sum of all questions asked of all respondents. (For example, if 30 respondents were assigned 10 questions each, total assigned would be 300.) The total answered is the total number of all question assignments that received a response.

• Less, Somewhat and Very Important – The tally of assessment questions, by importance ranking, as defined in the assessment advisory.

Module Tally The module tally report displays the same information as the assessment tally above, but is broken out by each module included in the selected assessment. (The example below only shows one module.)

Default Report Settings Allows the user to determine which reports automatically display upon enter the Management area.

Module Leader Tab If you are a Module Leader for a specific module, a Module Leader tab will be visible. Visit the tab to provide findings for the modules assigned to you.

If you are also an Assessor, you will also see an Assessor tab. *If you are an assessor and module leader for the same module, you will need to complete your assessor ballots before you can see and complete your module leader findings.

My Findings The findings are found under the Module Leader tab. Module leaders should review each question and the associated assessor ballots, including any requests for clarification. After reviewing these items, the module leader creates a finding based on the ballots and (if necessary), a request for clarification to the state respondents who were assigned that question. Clarification requests can only be used for the first round of questions. Upon accessing the Module tab, the module leader should see instructions, reports and a listing of findings to be completed.

The findings visible on the screen may be narrowed down by assessment, module, sub-module and whether or not the ballot has been submitted. Ballots may also be searched using key words.

Listing of Findings

Information provided with each finding is defined below.

• Question – The question number and question text.

• Responses – Displays the number of state responses along with the number of supporting documents attached.

• Rating – Displays the assessor’s determination of how well the state response satisfies the standard of evidence.

• Finding – Reason for the rating selected above, which will be sent to the state.

• Clarification - If the state responses are not clear or require more information, the assessor should include a clarification request. Available for first round only

• Status - Displays whether or not the finding has been saved or submitted to the state.

• Module – The module and sub-module the question belongs to.

View Responses and Supporting Documents

To view all respondent responses and any supporting documents provided, click on “Click to view” next to Responses.

Clicking on “Click to Edit” next to Rating, Finding and/or Clarification will all open the same screen.

This will allow you to view the assessor’s ballots as well as provide your module leader finding, rating and/or clarification request. The details for the question you’ve clicked on are provided below.

Save vs. Submit Findings and clarification requests may be edited and saved as many times as needed during, and prior to, the start of the next Q&A cycle. Once you are satisfied, the finding may be submitted to the state.

Once a finding has been submitted, it can’t be edited again during the current cycle. If you have not submitted your findings by the end of the Q&A cycle, the system will automatically push them through.

The submitted finding can still be seen and will display the information you have submitted.

My Tools & Reports There are several reports available on the Assessor’s tab to help you review and complete your work.

Assessment Summary Report Tally, by assessment of the number of ballots you’ve submitted as an assessor and the number of findings you have submitted out of the total number expected as a module leader.

Finding Progress The number of your module leader findings in progress, submitted or showing no progress for the selected assessment.

Rating Distribution The number of each rating given for a single module as a module leader.

Assessment Documents View all documents that have been uploaded to the Document Library for this assessment.

My Active Roles Displays the assessments you are currently assigned to, as well as the module(s) you have in each.

Printouts

Response Printout - This report provides a listing of all responses made by each respondent for each question, in order by module and sub-module.

Finding Printout - This report provides a listing of all findings you have submitted, in order by module, sub-module and question.

Management Tab Please see the Management Tab section included earlier in this document for Assessors. Module Leaders and Assessors share the same Management Tab information.

Management Tab