PWS Appendix G Developers Content Performance Reports for CTE and LLC.docx

Appendix G Developer’s Content Performance Reports for CTE and LLC Content Developer’s Performance Report- CTE (ALMS) Developer’s Content Performance Report:

1. CONTENT MENU:

1.1. CTE content Menu, SCO Display: Compare the ALMS menu to the course flow chart. If the course uses simple sequencing, does the menu reflect each SCO within the course? If the course uses complex sequencing and navigation, does the ALMS menu reflect the provided course map?

1.2. CTE content, Menu Functionality: Test each link in the SE menu. Does each menu item respond as expected per the course flow chart?

1.3. CTE content, Display: Monitor the ALMS menu display as you test the course for completion/success status. Does each menu item correctly display current completion/success status and scoring within the ALMS?

2. Bookmarking:

2.1. Normal Exiting: Test bookmarking two methods for exiting:

1. CLOSE button and selection of a different SCO from CTE content Menu.

2. Select content CLOSE button to close course. Does the content set and retrieve bookmarks without error?

2.2. Catastrophic Closure: Test bookmarking under conditions of abnormal termination or software crash.

1. On Windows OS machines: Open Windows Task Manager, select iexplore.exe, and then click End Process. 2. Close content by the selection of the browser’s X on CTE and content window. 3. Use ALT-F4 to close content.

Does the courseware return to the last saved bookmark location or the initial course entry page? The courseware must not crash, hang, or trap the user on the page where the crash occurred.

3. COURSE COMPLETION :

3.1. Course Completion Functionality: Step through the course, following all instructions for course completion, then exit the course. Review the status/completion/score for the user. Does the status/completion list the course completion status as Completed and score?

4. SCORM SEQUENCING AND NAVIGATION:

4.1. Sequencing and Navigation Rules Validity: Compare the intended sequencing and navigation to actual course behavior as you progress through the course. Does the course‘s sequencing operate as described in the course map?

5.1. PLUG-INS AND MEDIA PLAY:

5.1. Version Compatibility: Does the content utilize only plug-ins, browsers and tools on the Army Approved Product List?

5.2. Media Play: Exercise the courseware, particularly the multi- media functions. Do all assets play correctly?

a) If no controls are provided for media elements, do the media play automatically without requiring user intervention?

b) If specific learner action is prompted (either on the screen or as a course- wide convention), does the media respond as expected? If controls are provided, do all media controls work properly and consistently?

5.3. Media File Types: If audio and or video media are used in the course, is the courseware using approved media file size and types (see “TRADOC Pamphlet 350-70-12 for file size standards, ALMS Standards Guidelines and Best Practices and Baseline Home Computer Configurations for file/type standards). Browse through the course directory and note the number and file size range for the various media files types in the comments column to the right.

6. PERFORMANCE:

6.1. Courseware Media Responsiveness: During system testing, be aware of the overall responsiveness of the courseware under test. Does the courseware provide response times as specified in the TRADOC Pamphlet 350-70-12 and ALMS Standards Guidelines and Best Practices?

6.2. Overall Page Responsiveness: During system testing, be aware of the overall responsiveness of the courseware under test. Under normal conditions, do any part(s) of the courseware (multimedia and text) take longer than 13 seconds to fully load?

6.3. Page Navigation Responsiveness: Evaluate the response time of advancing to the next screen (using the next button). Does the courseware allow the user to progress to the next screen within 5 seconds - even if the current page did not finish loading? Note: Courseware navigation menus or frames are not considered in the time a page takes to load.

7. NAVIGATION

7.1. Navigation Functionality: Does the course provide a means for the learner to move forward and backward between screens? If not, is another navigation scheme employed to move the learner through the content? Does the existing navigation function properly?

7.2. Screen Counter Functionality: If applicable, check the screen counter display on several screens in each SCO. Does the screen counter consistently and correctly display the current screen number and the total number of screens within the SCO (or according to an alternative numbering scheme)?

7.3. MENU Contents: Open the content menu. Does the menu appear to reflect the course structure (at least to the lesson level)?

7.4. MENU Functionality: Try several hyperlinks on the Menu in the content. Does each tested hyperlinked item on the screen bring up the appropriate information?

7.5. MENU Functionality: Spot-check the CTE and content menu as you proceed through the course. If intended to do so, does each hyperlinked item display the correct unvisited/visited status or correctly track the learner‘s progress through the course?

7.6. GUI Element Functionality: From various screens within each SCO of the courseware under test, try each available user interface element at least once. Do all elements function correctly?

7.7. Help/Directions Sufficiency: As you move through the course, pay attention to the availability of help and the quality of directions to the learner. Is such learner support sufficient to allow the learner to successfully complete the course without additional explanation or outside assistance? Annotate anything that is likely to prompt a learner to call the ATHD Helpdesk.

7.8. Additional Observations: List any additional content anomalies that were observed.

Content Developer’s Performance Report- LLC (Blackboard) Developer’s Content Performance Report: LLC

1. CONTENT MENU:

1.1. (LLC) content Menu, SCO Display: Compare the (LLC) menu to the course map. If the course uses simple sequencing, does the menu reflect each SCO within the course? If the course uses complex sequencing and navigation, does the (LLC) menu reflect the provided course map?

1.2. (LLC) content, Menu Functionality: Test each link in the menu. Does each menu item respond as expected per the course map?

1.3. (LLC) content, Display: Monitor the My Grades page display as you test the course for the grade status icon. Does each menu item correctly display current grade status and scoring within the (LLC)?

2. BOOKMARKING:

2.1. Normal Exiting: Test bookmarking two methods for exiting: 1. CLOSE button and selection of a different SCO from (LLC) content Menu. 2. Select content CLOSE button to close course. Does the content set and retrieve bookmarks without error?

2.2. Catastrophic Closure: Test bookmarking under conditions of abnormal termination or software crash. 1. On Windows OS machines: Open Windows Task Manager, select iexplore.exe, and then click End Process. 2. Close content by the selection of the browser’s X on (LLC) and content window. 3. Use ALT-F4 to close content.

Does the courseware return to the last saved bookmark location or the initial course entry page? The courseware must not crash, hang, or trap the user on the page where the crash occurred.

3. COURSE COMPLETION :

3.1. Course Completion Functionality: Step through the course, following all instructions for course completion, then exit the course. Review the status/completion/score for the user. Does the status/completion list the course completion status as Completed and score?

4. SCORM SEQUENCING AND NAVIGATION:

4.1. Sequencing and Navigation Rules Validity: Compare the intended sequencing and navigation to actual course behavior as you progress through the course. Does the course‘s sequencing operate as described in the course map?

5. PLUG-INS AND MEDIA PLAY:

5.1. Version Compatibility: Does the content utilize only approved Army Certificate of Noteworthiness approved plug-ins, browsers and tools?

5.2. Media Play: Exercise the courseware, particularly the multi- media functions. Do all assets play correctly?

If no controls are provided for media elements, do the media play automatically without requiring user intervention?

If specific learner action is prompted (either on the screen or as a course- wide convention), does the media respond as expected? If controls are provided, do all media controls work properly and consistently?

5.3. Media File Types: If audio and or video media are used in the course, is the courseware using approved media file size and types (see “TRADOC Pamphlet 350-70-12 for file size standards and Baseline Home Computer Configurations for file/type standards). Browse through the course directory and note the number and file size range for the various media files types in the comments column to the right.

6. PERFORMANCE:

6.1. Courseware Media Responsiveness: During system testing, be aware of the overall responsiveness of the courseware under test. Does the courseware provide response times as specified in the TRADOC Pamphlet 350-70-12?

6.2. Overall Page Responsiveness: During system testing, be aware of the overall responsiveness of the courseware under test. Under normal conditions, do any part(s) of the courseware (multimedia and text) take longer than 13 seconds to fully load?

6.3. Page Navigation Responsiveness: Evaluate the response time of advancing to the next screen (using the next button). Does the courseware allow the user to progress to the next screen within 5 seconds - even if the current page did not finish loading? Note: Courseware navigation menus or frames are not considered in the time a page takes to load.

7. NAVIGATION

7.1. Navigation Functionality: Does the course provide a means for the learner to move forward and backward between screens? If not, is another navigation scheme employed to move the learner through the content? Does the existing navigation function properly?

7.2. Screen Counter Functionality: If applicable, check the screen counter display on several screens in each SCO. Does the screen counter consistently and correctly display the current screen number and the total number of screens within the SCO (or according to an alternative numbering scheme)?

7.3. MENU Contents: Open the content menu. Does the menu appear to reflect the course structure (at least to the lesson level)?

7.4. MENU Functionality: Try several hyperlinks on the Menu in the content. Does each tested hyperlinked item on the screen bring up the appropriate information?

7.5. MENU Functionality: Spot-check the (LLC) and content menu as you proceed through the course. If intended to do so, does each hyperlinked item display the correct unvisited/visited status or correctly track the learner‘s progress through the course?

7.6. GUI Element Functionality: From various screens within each SCO of the courseware under test, try each available user interface element at least once. Do all elements function correctly?

7.7. Help/Directions Sufficiency: As you move through the course, pay attention to the availability of help and the quality of directions to the learner. Is such learner support sufficient to allow the learner to successfully complete the course without additional explanation or outside assistance? Annotate anything that is likely to prompt a learner to call the ATHD Helpdesk.

7.8. Additional Observations: List any additional content anomalies that were observed.