OSOP_Course Design and Change w_Appendixes.pdf

Department of Defense HUMINT Training – Joint Center of Excellence 821 Irwin Street, Matlack Hall Fort Huachuca, AZ 85613

7 March 2016

ORGANIZATIONAL STANDARD OPERATING PROCEDURES

Course Design and Change OSOP

Summary. This Organizational Standard Operating Procedure (OSOP) codifies the HUMINT Training – Joint Center of Excellence (HT-JCOE) approval process for developing and changing HT-JCOE training and educational products. It is based on the analysis, design, development, implementation, and evaluation (ADDIE) instructional system design model (ISD). This OSOP addresses the requirements, associated tasks, procedures, and processes for implementing substantial course changes, development of new lesson plans, and the development of new courses.

Applicability. This OSOP is applicable to all DoD Civilians, contractors, and military personnel of HT-JCOE. All HT-JCOE personnel are expected to adhere to and will be held accountable to the procedures set forth in this OSOP.

Proponent: HT-JCOE, J7, Advanced Concepts

DSN: 879-821, COMM: (520) 533-7899

Table of Contents

1. Purpose 3

2. References 3

3. General 3

4. Roles and Responsibilities 3

5. Procedures 6

a. Analysis Phase 6

b. Design Phase 7

c. Development Phase 7

d. Implementation Phase 9

e. Evaluation Phase 9

f. Evaluation and Maintenance Phase (Ongoing) 10

Appendixes.

A. References

B. Glossary

C. Course Change Guidelines

D. HT-JCOE Course Change Request Form

E. Validation Data Collection Forms

(1) Development Phase: Content Validation Data Collection Form

(2) Development Phase: Observer Data Collection Form

(3) Development Phase: Volunteer Data Collection Form

(4) Implementation Phase: Student Data Collection Form

(5) Implementation Phase: Observer Data Collection Form

F. HT-JCOE Lesson Plan Template

G. Process Work Flow Diagram file:///C:/Users/Jeremy.Welch/AppData/Local/Microsoft/Windows/Temporary%20Internet%20Files/Content.IE5/26QS0EMC/HT-JCOE%20Course%20Change%20OSOP_J1%20Edits_25Nov15.doc

1. Purpose. To provide detailed guidance for producing Human Intelligence (HUMINT) training and educational products that meet current and emerging requirements of the Defense HUMINT Enterprise (DHE).

2. References. Related reference publications are found in Appendix A. Terms and abbreviations used in this OSOP are found in Appendix B.

3. General. Triggering events may require substantive changes to a course or the creation of a new course. Examples of triggering events include but are not limited to:

a. Training Review Board results

b. Doctrine or policy changes

c. External/Internal evaluations results

d. Changes in Tactics, Techniques, and Procedures

e. Changes in DHE Stakeholder’s priorities

The Course Change Guidelines (Appendix C) defines the three levels of change and the associated activities for each level. This OSOP outlines the process for changing or creating courses to include changes to the Program of Instruction (POI), course content and structure, and lesson plans.

4. Roles and Responsibilities.

a. The Commander will provide guidance and priorities in executing the Training and Education Development Program for HT-JCOE.

b. Chief of Operations (COO) will provide operational oversight and implement the

Commander’s guidance and priorities in executing the HT-JCOE Training and Education Development Program. The COO will approve all substantial course change requests.

(1) Senior Advisor.

(a) Reviews all Tier 3 course change requests, resolves content issues as needed, make recommendations, and forwards to the COO for initial approval.

(b) Coordinates changes to the Individual Student Assessment Plans (ISAP) based on approved Course Change Requests as required.

(2) J3.

(a) Reviews Tier 2 and Tier 3 course change requests (see Appendix D) and asks for clarification as needed.

(b) Reviews Tier 2 course change requests and forwards to the COO for initial approval.

(c) Reviews and forwards Tier 3 course change requests to the Senior Advisor.

(d) Ensures a collaborative environment between staff and courses which continues to embrace new educational strategies and instructional program compliance with the requirements set forth by the Defense HUMINT Enterprise, and the Training and Doctrine Command (TRADOC).

(3) J5. Forwards Lessons Learned documentation to the appropriate course, J3, and J7.

(4) J7.

(a) Ensures successful and timely completion of the project.

(b) Monitors the progress of the project and recommends corrective action if necessary.

(c) Ensures all documentation is entered into and maintained in the Training Administration and Assessment Program (TAAP), and other Army training databases as necessary.

(d) Verifies the successful completion of the validation of course, and courseware and the ongoing evaluation plan.

(e) Oversees projects and provides support to the J7 Team:

- Instructional Designer (ID)/Training Specialist - is the educational voice in the Training and Development process. Responsible for conducting the analysis; producing the Analysis/Design milestones and timelines; and, once authorized, leads the development of the Analysis/Design products (content flow chart, Program of Instruction, lesson plans, assessments) in collaboration with assigned course SMEs and Standardization Cell personnel as applicable. Reviews and approves lesson plans for instructional integrity.

- J7 Program Evaluation Specialist - reviews and approves the validation plan.

Provides expertise and assistance during creation and execution of the evaluation and validation plans, provides historical data and analyzes course evaluation data. Supports development of assessment tools, such as rubrics and exams. Coordinates the validation of all assessment tools.

- J7 Training Specialist, Training Development Capability (TDC) Specialist -works with Course Director or designee as required to enter data into TDC (i.e., Instructor Contact Hours (ICH), Academic Hours (AH), Instructor Actions (IA), equipment, materials, etc.)

- J7 Audio and Visual Support Specialist - assists with the design and development of instructional materials such as student handouts, slide presentations, charts, graphs, visual aids, etc.

- J7 Accreditation Liaison Officer - is the accreditation voice in the process.

Reviews for compliance with Council on Occupational Education (COE) accreditation standards.

c. Course and Committee Leadership (MSO Training Directorate, Management

Committee Director, and Course Directors). Serve as the trainer’s voice in the process.

Work closely with J7 staff to oversee the development and implementation phases to include the validation process.

(1) Appoint the Senior Subject Matter Expert (SME). Also, assign instructors who are content SMEs to develop lesson plans and who, ideally, will be available for the entire project. Appointed Course Senior Subject Matter Expert (SME):

(a) Participate in the analysis and design phases as a resource and content decision maker, and serve as the primary point of contact for course content.

(b) Develop and review lesson plans as needed.

(2) Responsible for providing access to course personnel. Provide expertise on current implementation of the POI.

(3) Responsible for ensuring slide presentations accurately reflect the intent of the lesson plan, its content, and presentation methods. This is also part of the annual lesson plan review process.

(4) Responsible for creating/maintaining the master training schedule beginning in the Design Phase.

(5) Provide resources and data throughout the process. Responsible for identifying Academic Hours (AH), and Instructor Actions (IA).

(6) Determine facilities and equipment requirements or changes.

(7) Responsible for coordinating the content validation, individual trials, and operational trials (see Appendix E).

(8) Responsible for developing the evaluation report in coordination with the J7 team.

5. Procedures. HT-JCOE aligns the course change/development process with the steps of the ADDIE process: analysis, design, develop, implement and evaluate. Each phase requires the steps specified below:

a. Analysis Phase.

(1) The Course Director identifies the requirement to:

(a) Restructure the course content flow, POI.

(b) Integrate new content.

(c) Remove obsolete content.

(d) Analyze the course lesson plans and materials for redundancies.

(e) Update lesson plans.

(f) Create new lesson plans.

(g) Incorporate changes recommended as a result of the Annual Trend Profile Report and other J7 course reviews.

(2) The Course Director submits a Tier 2 or Tier 3 Course Change Request Form (Appendix D) to J3. The Course Change Request Form details the needed changes and supporting evidence justifying the change.

(a) If a Tier 2 or Tier 3 Course Change Request is approved, the J3 provides the approved Course Change Request to the Course Director and J7 for coordination.

(b) If the request is disapproved, the request is returned to the Course Director, and the J7 is provided a copy.

(3) The J7 ID / Training Specialist host an initiating meeting with the Course Director, the Senior course SME, and the instructor(s) assigned to lesson plan development who will work with the J7 staff throughout the project. During the meeting the J7:

(a) Reviews the request.

(b) Receives or requests existing materials.

(c) Reviews End of Course Reports for the last three course iterations and the

Annual Trend Profile Report.

(d) Reviews the Critical Task List (CTL) and the Performance Standard Levels determined by the DHE Executors, and documented as part of the Training Review Board (TRB) process.

(e) Sets up a timeline for completion of a macro analysis if applicable.

(4) The J7 Team prepares initial broad recommendations based on:

(a) Initial analysis.

(b) Course observations.

(c) Review of course materials.

(d) Discussions with the senior course SME (and Course Director as necessary).

(e) Additional instructor competencies required by the change.

(5) The J7 ID / Training Developer provides the design recommendations to the J7, Course Director, and Senior course SME; gains unanimous concurrence and conducts additional discussions and revisions as necessary before beginning the project.

(6) J7 Team creates a project timeline with identified project milestones and checkpoints.

b. Design Phase (begin with the end in mind) - Design Team: J7 ID / Training Specialist, Senior course SME, and assigned instructor(s).

(1) Produces a lesson/course concept overview.

(2) If redesigning a course, completes content sequencing flow chart.

(3) Writes or revises objectives (TLO and LSAs).

(4) Determines content to be included or removed.

(5) J7 Evaluation Specialist works with the Design Team to produce blueprints of assessment instruments (rubrics, checklists, tests, ISAP).

(6) J7 Evaluation Specialist develops evaluation and validation plans in collaboration with Course Director.

(7) Identifies media requirements, required materials and resources.

c. Development Phase - Development Team: J7 TDC Specialist, ID, and Evaluation Specialist, Senior course SME, and assigned instructor(s).

(1) Creates or adjusts lesson plans using the HT-JCOE Lesson Plan Template.

Note: All cadre members assigned to develop lesson plans must have attended a J7 Lesson Plan Development workshop.

(2) J7 TDC Specialist enters required fields into TDC.

(3) Validates assessments - test instrument blueprints are produced during the design phase. Course cadre and the course SME work with the J7 Evaluation Specialist to develop, and validate the test/evaluation instruments as soon as the objectives have been written.

Note: Validated test/evaluation instruments are needed prior to conducting any course/courseware validation trials.

(4) Creates a master training schedule, if applicable.

(5) Conducts delivery training for instructors who will deliver the lessons.

(6) Completes validation of all course/courseware content and supporting training materials before conducting individual trials.

(a) Content validation (see Appendix E) is the process used to verify that the information in the course material is technically and doctrinally accurate. Content validation is completed at the lesson plan level. SMEs review all course/courseware materials and supporting documentation to ensure that all materials:

- Are doctrinally correct

- Include sufficient detail

- Are clearly written

- Use language that the target audience understands

- Use current, appropriate, and applicable references

(b) To conduct content validation, complete the following steps:

- Have at least two SMEs review the materials. The lesson plan developer will not be a reviewer. The Senior course SME should be one of the reviewers.

- Ensure each SME has access to a complete set of draft course materials for review.

- As the SMEs review the material, they complete the Content Validation Data Collection Sheet (Appendix E).

- SMEs meet with the Course Director to review results and discuss recommendations. A SME’s comment or recommendation does not automatically drive a change to the materials. The Senior Advisor arbitrates and resolves all content issues.

(7) Under a course redesign - as part of the validation plan, the Development Team conducts individual trials to determine whether or not the materials train or educate the student on the objective so that the student meets the objective standard.

(a) Conducts Individual Trials with performers and non-performers. Have all participants complete the Pre-course Knowledge Assessment.

(b) Use a minimum of four participants; two performers and two non-performers.

Each participant completes a Content Validation Data Collection Form and a Volunteer Data Collection Form (Appendix E). In addition, a Training Specialist, Instructional Designer, and/or Evaluation Specialist observe the training and complete an Observer Data Collection Form (Appendix E) and interview the participants as necessary.

(c) After an analysis of all collected data, the lesson, course, or module is adjusted, as needed, before moving on the Operational Trials.

(8) Adjust courseware as necessary based on analysis of individual trials and conduct a new trial, as necessary. Repeat the cycle until all major problems have been eliminated. All trial documentation and analysis is maintained in the Training Administration and Assessment Program (TAAP), and other Army training databases as necessary.

d. Implementation Phase.

(1) As part of the validation plan, conduct Operational Trials with a class of students to test the instructional materials under actual training conditions. A J7 observer completes an Observer Data Collection Form (Appendix E) after each lesson.

If possible, the same individual observes the entire course or module to identify possible gaps and needed adjustments.

(2) J7 executes program evaluation process IAW Course Evaluation OSOP.

(3) Adjustments are made as necessary.

(4) Students complete post-course surveys at intervals as prescribed in the Course Evaluation OSOP, which are analyzed to determine course efficacy.

e. Evaluation Phase. An evaluation plan provides a process for systematic, continuous assessment with the intent to further develop or improve a program, process, or product. It determines whether objectives have been met and assesses the efficacy of the training provided. A well-designed evaluation plan provides feedback that enables decisions to adjust an instructional program. Following course implementation, the J7 Evaluation Specialist:

(1) Analyzes student performance data including assessments, field exercises, etc.

(2) Analyzes student survey data.

(3) Analyzes student and instructor AAR responses.

(4) Produces an Evaluation Report with recommendations as needed.

f. Maintenance and Continuous Evaluation (ongoing).

(1) The J7 Evaluation Specialist analyzes student post-course surveys at intervals as specified in the Course Evaluation OSOP.

(2) Course Director ensures that all lesson plans are reviewed annually IAW the Course Evaluation OSOP.

(3) J3 ensures the review of course POIs annually to guarantee instructional programs comply with the requirements set forth by the Defense HUMINT Enterprise, and TRADOC.

(4) Course Directors make adjustments, as appropriate, using the processes described above.

3/17/2016

X Travis A. Phillips

Signed by: PHILLIPS.TRAVIS.ALLEN.1156987634

TRAVIS A. PHILLIPS

COL, US Army Commanding

HT-JCOE Course Design and Change OSOP

Appendix A References

Council on Occupational Education, “Handbook of Accreditation for Federal Institutions:

2015 Edition” http://www.council.org/manuals/

Council on Occupational Education, “Self-Study Manual for Federal Institutions: 2015 Edition” http://www.council.org/manuals/

HT-JCOE Strategic Plan, 2014 - 2019

TRADOC Pamphlet (TP) 350-70-4, “Systems Approach to Training: Evaluation,” January 12, 2004

TRADOC Pamphlet (TP) 350-70-10, “Systems Approach to Training: Course and Courseware Validation,” March 29, 2004

TRADOC Regulation (TR) 350-70, "Army Learning Policy and Systems," December 6, TRADOC Pamphlet (TP) 350-70-14, “Training and Education Development in Support of the Institutional Domain,” March 27, 2015

Appendix B Glossary

Term Definition

AAR After Action Review

AH Academic Hours

COE Council on Occupational Education

Content Flow Chart A graphic organizer showing the content in the order which it will be presented in the course.

Content Validation Content Validation is the process used to verify that the information in the course materials is technically and doctrinally accurate. During this process, all course/courseware materials and supporting documentation is reviewed to ensure that all materials—

Are doctrinally and technically correct.

Include sufficient detail.

Are clearly written using approved formats.

Use language the target audience understands.

Developers provide SMEs, who are considered master performers of the tasks being trained, all the materials for review, including supporting resources and references. The SMEs verify that the material is doctrinally and technically accurate. Keep in mind to choose the SMEs wisely and that the best of the best may not belong to your immediate organization.

COO Chief of Operations

DHE Defense HUMINT Enterprise

IA Instructor Actions

ICH Instructor Contact Hours

ID Instructional Designer

Individual Trials Individual Trials begin the process of determining the educational sufficiency of instructional materials to determine whether the materials address the objective being trained and allow students to achieve the training standard. Individual Trials allow training developers to observe and gather information by exposing a sample audience to the instructional materials. After the sample audience has received instruction, members participate in an in-depth interview. The primary purpose of Individual Trials is to allow developers to determine whether the materials effectively and efficiently teach the objective.

Developers also analyze all of the information gathered and correct any identified discrepancies if necessary. Learning Step Activities that appeared sufficient during the Design Phase may not work in reality. After each trial is completed, corrections are applied as applicable, and a new trial is conducted.

The cycle repeats until all major problems have been eliminated, which generally takes three iterations to eliminate the major problems in an objective.

ISAP Instructional Student Assessment Plan

Lesson Plan Validation A lesson plan is validated after it has undergone Individual Trials and Operational Trials to determine that the content is sound and well-grounded, that the lesson works under operational conditions, and that all aspects of the lesson are integrated and flow smoothly.

Macro analysis The first step of a redesign process, the macro analysis provides a broad overview of the problem and proposed solutions. This analysis provides the basis for discussion of implications of change (i.e., budgetary or staffing concerns, etc.)

Non-Performers A participant to whom the content is new.

Operational Trials Operational Trials test instructional materials under actual training conditions using the target audience for which it was developed. The process is used to validate a lesson, course, or the individual objectives of a lesson plan. An Operational Trial determines whether the training system actually works under operational conditions, provides feedback from an actual instructional experience, and ensures the integration and smooth functioning of all aspects of the course/courseware. The trials allow J7 to gather information through observation, interviews, and surveys. Using the data above combined with an analysis of students’ results compared to the training standard provides a method for confirmation of objective/lesson/course validity.

Performers A participant who is well-versed in the content presented.

POI Program of Instruction

SME Subject Matter Expert

TAAP Training Administration and Assessment Program

TDC Training Development Capability

Test Validation Test Validation determines whether the test accurately measures what it purports to measure. Specifically, that the test can accurately measure student abilities with regards to the objective of the task trained. To validate a test, an analysis is performed to ensure that the test:

Aligns with lesson objectives, content, and performance standards

Measures what it is supposed to measure Does not measure irrelevant attributes (For example, a poorly written test may measure a student’s ability to read and infer purpose rather than testing content taught.)

TRB Training Review Board

Appendix C Course Change Guidelines

Tier 1 Changes

1. Tier 1 changes do not affect the content, standards, or resources of a course.

Examples of Tier 1 changes include:

a. Changes to the motivator and/or instructional lead in. Motivators can be personalized by the instructor and as such can be changed at the course level.

Instructors are encouraged to consider the purpose of the motivator and instructional lead in when changing to ensure that they are effective within broader scope of the lesson.

b. Changes to Slides. This does not include the addition or removal of approved content, which would be a Tier 2 or Tier 3 change. Slides are tools used to organize and communicate information. Updates to slides are acceptable. Instructors are encouraged to consider the educational importance of graphics that may have been created specifically for the lesson when making changes to the slides.

c. Checks on Learning. Courses are encouraged to consider the Checks on Learning (CoL) on a regular basis. Revisions to checks on learning are acceptable.

2. In general, changes that do not alter the decisions captured in the Lesson Plan Outline are authorized for course level action. The following are NOT considered Tier 1 changes:

a. Changes to objectives (both TLO and LSA)

b. Changes to assessments both summative (ISAP-captured) and formative (not part of ISAP)

c. Changes to resources including time, instructors or support personnel, locations, and equipment

d. Adding or removing content

Tier 2 Changes

1. Tier 2 changes are generally in response to a perceived training/education gap which may change course content, structure or instructional methodology but do not have major impact on resources. If the J3 determines that it more closely aligns with Tier 3 guidelines, the J3 will inform the course that the request will be processed as a Tier 3 request.

Examples of Tier 2 changes include:

a. Clarifying a learning objective. If a learning objective needs clarification, the J3 will review the proposed change to ensure that the objective’s intent has not changed. Changing the intent of a learning objective would be a Tier 3 change.

b. Augmenting a Lesson Plan. Sometimes a training/education gap is noticed after the implementation of the formal lesson plan necessitating lesson plan augmentation. For example, the addition of an LSA to a TLO. As long as there is not a change to resources (time, personnel, equipment, or location) or to the objective of the TLO, this augmentation is considered a Tier 2 change.

c. Modifying a formative (non-ISAP) assessment (i.e., Practical Exercise, etc.).

While formative assessments are not part of the ISAP, the J3 will review the proposed changes to ensure that it still meets the educational intent and prepares the student for the summative assessment (those captured in the ISAP and are part of graduation criteria).

Tier 3 Changes

1. Tier 3 changes are more substantial and generally affect doctrine, student assessments, resources, and course flow.

Examples of Tier 3 changes include:

a. Updating course material to reflect substantive doctrine changes. The Senior Advisor ensures changes to doctrine are communicated to the course(s) and properly incorporated into training materials such as lesson plans, slides, handouts, and other documentation. Depending upon the extent of the change, resources may also be affected. Note: inconsequential doctrine changes, such as the changing of an acronym or name of an organization, is a Tier 1 change and can be accomplished by the course.

b. Adding or removing content from a course. Course content is ultimately determined in response to Critical Tasks formulated by the Training Review Board (TRB) and approved by the Defense HUMINT Enterprise (DHE). Altering the content of a course can move the course away from the original DHE intent in regards to student task proficiency levels.

c. Changes to course resources. Resources include, but are not limited to, instructors, support personnel, time, equipment, and location of training. Normally, any change to course resources would be as a direct result of an addition or removal of course content, but doesn’t have to be so.

d. Changes to a summative assessment. Summative assessments are tests or evaluations which are captured in the ISAP and are considered as part of graduation criteria for a course.

e. Command Directed Changes. All command directed changes to courses are usually as a result of changes which occurred during the TRB process.

f. Course flow redesign. Redesigning even a portion of a course requires a dedicated effort. HT-JCOE leadership will prioritize these larger scale projects.

Appendix D HT-JCOE Course Change Request Form

HT-JCOE Course Change Request Form

Name of Course: _________________ Date of Request: ____________________

Course Director: ______________________ Tier Change Level: ___ 2 ___ 3

LP No(s). (if applicable) _______________________

1. Is the problem performance or content oriented?

_____ Performance Oriented: The student performance within the course is below the expected level

_____ Content Oriented: Course content has been identified as doctrinally outdated, missing content, or is no longer relevant to course’s learning objectives.

2. If you checked performance, please complete the following.

a. Describe the performance problem. Include supporting evidence that it is a problem.

b. Describe the current assessment tools.

c. What mitigation strategies have you tried to solve this problem?

3. If you checked content, please complete the following.

a. Is the content related to this change request currently taught in the course?

_____ yes _____no

b. If you answered yes, please respond to the following questions.

i) What is the problem associated with this content?

ii) Where did the requirement to change this content originate?

iii) How many lesson plans (number, titles, and academic hours) are related to this problem?

iv) What implications do you see with changing this content?

c. Who have you identified as the Senior SME?

d. List identified instructor(s) who will develop lesson plans or instructional content.

e. Additional Remarks:

Appendix E Validation Data Collection Forms

Development Phase

Content Validation Data Collection Form

Course: _______________________ Date: ________________ Name: ________________________ LP No.:_______________ Presenter Name: ________________

To be completed by Performers and non‐Performers

Discussion

1. Is the doctrine used current? Identify any instances of obsolete doctrine.

2. Could you execute training based on this lesson plan? Focus on the instructor notes, instructions and administrative guidance. How could this lesson plan be improved?

3. Does the content address all the critical points of the objective? Identify where and what content needs to be added or expanded.

4. Is the terminology and nomenclature used in this content the same as is currently used on the job? Identify any differences and provide source of new terminology/nomenclature.

5. Were all graphics, illustrations, tables, figures and other types of supporting information clearly identified and referenced in the text/body of the content? Identify any instances where improvement is needed.

6. Are graphics, illustrations, tables, figures, and other type of supporting information technically accurate and support training the learning objective? Identify any instances that need to be corrected and the source of correction.

7. Is there a logical flow between the content points you were provided? Does the organization of the content make sense to you? Identify any sections where improvement is needed.

8. Are notes, cautions and warnings included wherever they are needed? Identify any instances where one should be added.

9. Additional comments (add additional pages if needed)

Observer Data Collection Form

To be completed by J7

1. Did the instructor communicate the standard(s) clearly so that you understood what the students had to do to meet the standard? Were all standard statements clear, measurable, and objective? Identify any that were not.

2. Did the instructor communicate the importance of the content in a meaningful way? Note any areas that could be improved.

3. Did the instructor deliver the lesson/learning activity according to the approved lesson plan? Note any deviations.

4. Were the volunteers given sufficient opportunities to practice the skills and knowledge, so that you felt comfortable that they could perform the objective being taught? Identify the specific instance(s) that you feel need revising.

5. Were the volunteers given a chance to review/redo activities/steps that they had a hard time mastering? Identify any instances where you feel additional training/support should be available.

6. Were examples and situations presented realistic and reflective of the actual job or battlefield conditions? Identify any that need improvement, and recommend changes.

7. Did the volunteers have any problems using the references? If so, explain. Cite specific examples.

8. Are the illustrations/graphics/visuals clear and uncluttered, free from spelling errors, grammatically correct?

9. Was feedback given to the volunteers on their progress? What kind of feedback was provided?

10. Were the discussions held relevant to the objective being taught? Note any that weren’t for follow up.

11. Were most of the prompting questions for discussions phrased to elicit student thought?

12. Were the students actively engaged (writing, talking or doing something) in the planned learning activities? Note those that were very effective and those that need to be improved.

13. Did the training materials/media support the learning activities? Were the student‐media (handouts, slides…) interactions effective? Note those that were very effective and those that need to be improved.

14. Was the time listed in the lesson plan reflective of what occurred during this observation? If not explain.

15. Did you notice students struggling to understand the content? Where in the LP did they struggle? (follow up with students to determine the cause of the struggle)

Additional Comments:

Volunteer Data Collection Form

To be completed by Performers and non‐Performers

1. Were the standard statements clear on identifying the requirements you must complete to successfully accomplish the objective? Identify any standards that were unclear.

2. Did the lesson/learning activity involve you in your own learning? Were you required to do, say or write something? Are there any places you would change what you were required to do, say or write?

3. Did the discussions held help you understand the lesson materials? Were you asked questions that made you think? Give an example of a question that made you think.

4. Was the length of the lesson/learning activity appropriate? Cite examples of where you think it could be shortened or lengthened.

5. Were the slides, handouts and other media effective in helping you learn? How could we improve these?

6. Other than media, was there any part of the lesson/learning activity that was especially effective in helping you learn?

7. What got in the way of you learning during this lesson/learning activity?

8. Were you given a chance to review/redo activities/steps that you had a hard time mastering? Identify any instances where you feel additional training/support should be available.

9. Did you understand what was expected of you during this instruction? How could we improve this?

10. After this instruction are you comfortable that you understand and could perform the tasks discussed?

11. Is there any content/skills/knowledge that you think the instructor assumed you already knew but you actually DIDN’T? Was there content that you were taught that you already knew? Where did you know it from?

Implementation Phase

Student Data Collection Form

To be completed by the Student

1. Were the standard statements clear on identifying the requirements you must complete to successfully accomplish the objective? Identify any standards that were unclear.

2. Did the lesson/learning activity involve you in your own learning? Were you required to do, say or write something? Are there any places you would change what you were required to do, say or write?

3. Did the discussions held help you understand the lesson materials? Were you asked questions that made you think? Give an example of a question that made you think.

4. Was the length of the lesson/learning activity appropriate? Cite examples of where you think it could be shortened or lengthened.

5. Were the slides, handouts and other media effective in helping you learn? How could we improve these?

6. Other than media, was there any part of the lesson/learning activity that was especially effective in helping you learn?

7. What got in the way of you learning during this lesson/learning activity?

8. Were you given a chance to review/redo activities/steps that you had a hard time mastering? Identify any instances where you feel additional training/support should be available.

9. Did you understand what was expected of you during this instruction? How could we improve this?

10. After this instruction are you comfortable that you understand and could perform the tasks discussed?

11. Are there any content/skills/knowledge that you think the instructor assumed you already knew but you actually DIDN’T? Was there content that you were taught that you already knew? Where did you know it from?

Implementation Phase

Observer Data Collection Form

To be completed by J7

1. Did the instructor communicate the standard(s) clearly so that you understood what the students had to do to meet the standard? Were all standard statements clear, measurable, and objective? Identify any that were not.

2. Did the instructor communicate the importance of the content in a meaningful way? Note any areas that could be improved.

3. Did the instructor deliver the lesson/learning activity according to the approved lesson plan? Note any deviations.

4. Were the volunteers given sufficient opportunities to practice the skills and knowledge, so that you felt comfortable that they could perform the objective being taught? Identify the specific instance(s) that you feel need revising.

5. Were the volunteers given a chance to review/redo activities/steps that they had a hard time mastering? Identify any instances where you feel additional training/support should be available.

6. Were examples and situations presented realistic and reflective of the actual job or battlefield conditions? Identify any that need improvement, and recommend changes.

7. Did the volunteers have any problems using the references? If so, explain. Cite specific examples.

8. Are the illustrations/graphics/visuals clear and uncluttered, free from spelling errors, grammatically correct?

9. Was feedback given to the volunteers on their progress? What kind of feedback was provided?

10. Were the discussions held relevant to the objective being taught? Note any that weren’t for follow up.

11. Were most of the prompting questions for discussions phrased to elicit student thought?

12. Were the students actively engaged (writing, talking or doing something) in the planned learning activities? Note those that were very effective and those that need to be improved.

13. Did the training materials/media support the learning activities? Were the student‐media (handouts, slides…) interactions effective? Note those that were very effective and those that need to be improved.

14. Was the time listed in the lesson plan reflective of what occurred during this observation? If not explain.

15. Did you notice students struggling to understand the content? Where in the LP did they struggle? (follow up with students to determine the cause of the struggle)

Appendix F HT-JCOE Lesson Plan Template

Legend:

Items highlighted in yellow ‐ will be completed/calculated automatically by TAAP

Items highlighted in bright green ‐ will have a drop‐down list (DDL) to choose from in

TAAP

Items highlighted in light gray ‐ depending on responses to certain fields in the form, an entry may or may not be required; if not required, these fields will be unavailable (subdued) in TAAP.

Items highlighted in dark gray ‐ designates a link with which files can be uploaded into TAAP for a specific LP

Item highlighted in fuchsia ‐ describes general actions or “checks and balances” completed by TAAP

Note: From the resourcing information contained in a Lesson Plan, a schedule can be generated which serves as a secondary check.

Until TAAP is operational, all required fields must be completed manually.

LP Developer: J7 Approval: (approver’s name; date)

Part I: Administrative Data (Required for resourcing purposes)

a. Course:

b. LP Number (PFN): Note: TAAP will check for duplicate PFNs

(1) School code (designation for HT‐JCOE): 307

(2) Course code (ASO, SOC, DSD, J2X, JHA, JHO, JIM):

(3) Rank (for all ranks ‐ applies to all HT‐JCOE courses): 6

(4) Module ( letter designation; determined by course):

(5) LP no. (a one‐up three‐digit number within each module beginning with 001):

(6) LP designator; L (Lesson), T (Test‐graded event and graduation criteria), X

(Exercise‐FTX/STX), A (Administraive‐check‐in, introduction, graduation, etc.), D (Distributed Learning) :

Examples: 307SOC6A001A; 307DSD6B003T; 307DAT6E034X; 307DAT6E035T

c. LP Title

(1) LP Title:

(2) LP Title Classification:

(3) LP Title, Unclassified Version: (if LP title is classified):

d. LP Content Classification:

e. Schedule

(1) Is the LP conducted on separate training days and/or non‐contiguous times on same day (Y/N):

(2) If “No,” then (enter data once, immediately below for entire LP):

(a) Training Day (when lesson is taught):

(b) Start Time:

(c) End Time:

(d) Total Actual Minutes:

(e) Total TRADOC Hours (TRADOC hour = 50 minutes):

(3) If “Yes,” then enter the data in each LSA below.

f. Lesson Plan Elements

(1) Instructor Qualification Requirements (In order to teach this lesson, what qualifications must the instructor possess?):

(2) LP contains copyrighted material (Y/N):

(a) Applied for Letter of Authorization (Y/N):

(b) Obtained Letter of Authorization (Y/N):

(c) Link to Upload Authorization memo

Part II: Critical Task List Information Identify the critical tasks addressed in LP ‐ Identify if the tasks are “Taught” (with an *) (Task is taught in its entirety or this is the last time the last step of a task is being taught), “Supported” (only a portion of the task is being taught in this LP), or “Reinforced” (Task was taught previously; this LP reinforces the task):

Section A ‐ Taught (*) or Supported:

Critical Task/Subtask Number Task Title

Section B ‐ Reinforced:

Critical Task/Subtask Number Task Title Note: TAAP will compile the Critical Tasks from the LSAs in this Lesson Plan

Developer considerations:

a. Lesson context

(1) Where does this lesson sit within the course?

(2) What did the students learn just prior to this lesson?

b. Prerequisite LPs:

c. Follow‐on LPs:

Part III: POC Information

a. Instructional Designer:

b. Lesson Plan Developer(s):

c. Primary Instructor:

Part IV: Evaluation strategy: Thoroughly describe how students will be evaluated. What defines success? When (in the course) and where will students be evaluated? What PEs are used to help students achieve hands‐on proficiency prior to the evaluation? This is required for each lesson.

a. Formative Evaluation: (How will the instructor monitor student learning during the lesson.)

b. Summative Evaluation:

(1) (How is student learning evaluated at the end of the module or course. They are

“high stakes” and are identified in the Student Evaluation Plan.)

(2) “Test” Lesson Plan(s) (multiple PFNs are possible): (In which PFN(s) is the material contained in this Lesson Plan tested?)

NOTE: Submit a draft evaluation tool with this LP outline i.e. rubric, evaluation sheet.

(Link to Upload rubric, evaluation sheet, etc.)

Part V: Homework assigned prior to lesson: Describe both the purpose of the homework (i.e.

review of previously learned material, introducing new knowledge, practicing a skill from previous lesson…). Be sure the homework is used / referred to in some way in the body of the lesson plan.

(Link to Upload Homework material)

Part VI: Lesson Plan Design:

Terminal Learning Objective (TLO): (only one per lesson plan)

Action: ([One verb‐present tense‐observable and measureable] Ask yourself “At the end of this training what must the student demonstrate?)

Condition: Given….(Under what conditions [ a cue or purpose for performing the action, and with what specific references i.e. AR, FM title, number and classification identifier, student handouts, technology, etc.] must students complete the standard.)

Standard: ([present tense of the verb used in the Action Statement] Standard must be observable and measureable. Ask yourself “At the end of this training what must the student demonstrate and how well?” Often the answer to this question is the Standard. Make certain there is congruence between the Action and the Standard)

Lesson Introduction with motivator (When you add up the total time for the entire lesson, ensure to include the times for the introduction, LSAs, and lesson summary and record Part I above.):

Total Actual Minutes for Introduction (minimum 5 minutes):

Note: TAAP will add all times in the LP parts to ensure accuracy of start/end times.

Learning Step Activity (LSA) 1:

a. Activity Title

(1) Activity Title:

(2) Activity Title Classification:

(3) Activity Title, Unclassified Version: (if Activity title is classified):

b. Activity Content Classification:

c. Schedule Note: In TAAP, if the LP is conducted in a contiguous block of time, then the fields below (1) (a)‐(e) will be unavailable.

(1) If the LP is conducted on separate training days or non‐contiguous times on same day, complete below:

(a) Training Day (when this LSA is taught):

(b) Start Time:

(c) End Time:

(d) Total Actual Minutes:

(e) Total TRADOC Hours (TRADOC hour = 50 minutes):

d. Critical Task(s) addressed in LSA (multiple Critical Tasks are possible):

e. Objective/Outcome (What do you want the students to learn, perform as a result of the LSA? Focus on what the student is doing and why. Start the LSA statement with:

The student will [this strategy keeps the focus on the student NOT the instructor and creates a student centered learning environment]):

f. Number of Instructor(s) Present/Students Present:

g. Blooms Taxonomy Level (enter only one):

h. Content Type: (process, product, combination)

i. Instructional Strategy (Use only Strategies included on page 80 and 81 of TRADOC

Pamphlet 350‐70‐14 (27 March 2015). It lists the five instructional strategies as Direct Instruction, Independent Study, Indirect Instruction, Collaborative/Interactive Instruction, and Experiential Learning. Suggested MOIs for each strategy are listed as well.) (enter only one):

j. Method of Instruction (MOI) (Approved MOIs on pages 146‐152 of TRADOC Pamphlet

350‐70‐14) (27 March 2015); (enter only one):

k. Lesson Flow (Indicate how the content is chunked and the time spent on each chunk.

Identify the student activities that you will use in each content chunk or lesson segment. These activities should support the instructional strategy identified in item 8 and the method of instruction identified in item 9 above. For example, you identify “collaborative interactive instruction” as the instructional strategy in item 8 and “peer‐partner learning” as the method of instruction in item 9. Then, “Think, Pair, Share” or “Reciprocal Teaching” are two student activities that support the “Collaborative Interactive Instruction” as the strategy and “Peer Partner Learning” as the method of instruction. The activities that you select should provide opportunities for students to actively process new information approximately every 30 ‐40 minutes.

Identify points in the lesson where the instructor will assess or check student learning and comprehension. Indicate times in minutes for each activity. Make sure that the times indicated in the lesson flow add up to the total time indicated in item 5 above. )

l. Media: (List items within the LSA needed for full implementation, i.e. handouts, practical exercises, scenarios, PowerPoint slides, assessment items, etc.) (Link to Upload material)

m. References used to create content (List all references by publication number, title, chapter and paragraph and the overall classification of the reference. Do not include Advanced Sheet or student notes.):

n. List references to which the student must have access IOT participate in this LSA:

o. Resource Requirements (What resources, i.e., equipment/technology/assistants, do you need to conduct the training?):

(1) Classroom

(a) Type and size:

(b) Classroom Setup Time:

(c) Classroom Breakdown Time:

(2) Equipment (Equip Name/Quantity –‐computers (NIPR, SIPR, JWICS), GSA vehicles, phones, etc.):

(3) Support Personnel (J6, J4, etc.; this does NOT other/secondary instructors):

(4) Expendable Materials (flip charts, white boards, markers, pens, paper, etc.):

(5) Other Instructor Action(s) (see list of Instructor actions prepared by J5; include action and time required for completion) (multiple actions are possible):

(a) Instructor Action (IA):

(b) Time required for IA:

(6) Are the resources listed above exactly the same for every LSA in this Lesson Plan (Y/N) (if “Y,” there is no need to complete in subsequent LSAs):

p. Student Production (Y/N):

(1) Student Production Description:

(2) Student Production Graded (Y/N):

(a) Time (in minutes) to grade report(s) (per student; for this LSA):

(3) Graduation Criteria (Y/N):

Add a Learning Step Activity (LSA): (Y/N)

Follow‐on Activities: (homework assignments, reports, reflections, etc.)

Lesson Summary:

Total Actual Minutes for Summary (minimum 5 minutes):

Note: TAAP will add all times in the LP parts to ensure accuracy of start/end times.

The following table summarizes the items within the lesson plan outline that must be developed for full implementation of the lesson plan, i.e. handouts, practical exercises, scenarios, PowerPoint slides, assessment items, etc.

Media Item Location in Lesson Plan Outline

The Tables below containing Instructional Strategies and Methods of Instruction (MOI) are included here for your convenience. Please delete them before submitting your LP.

Excerpt from TP 350‐70‐14 (27 March 2015); pages 80‐81

Instructional strategy uses with suggested MOIs

Instructional Strategy Uses & Suggested Methods of Instruction

Direct instruction Uses: This strategy is most effective at teaching knowledge acquisition involving facts, rules, and action sequences.

Suggested MOIs: compare and contrast, demonstration, drill and practice, guided reading and thinking, lecture, structured overview, tutorial

Independent study Uses: Generally involves research outside of a school setting for an agreed upon amount of credits, this strategy can be self‐paced, and can overcome geographical barriers.

Suggested MOIs: interactive multimedia instruction, tutorial, writing assignments

Indirect instruction Uses: This strategy is best used when presenting concepts, abstractions, or patterns, and when the learning process is inquiry‐based, the result is discovery, and the learning context is a problem.

Instruction helps learners develop content organization, inductive and deductive reasoning, personal experience(s), and group discussion skills.

Suggested MOIs: brainstorming, case study, concept mapping, inquiry, problem solving, reflective discussion, tutorial, writing assignments

Collaborative/interactive instruction Uses: Students learn from peers and instructor/facilitators to develop social skills and abilities, to organize their thoughts, and to develop rational arguments. Allows for a range of groupings and collaborative/interactive methods.

Collaborative/Interactive instruction requires the refinement of observation, listening, interpersonal, and intervention skills and abilities by both instructor/facilitator and learners.

Suggested MOIs: brainstorming, cooperative learning groups, debates, discussion (small or large group), interviewing, laboratory groups, panel, peer partner learning, problem solving, role playing, seminar, tutorial

Experiential learning Uses: The emphasis in experiential learning is on the process of learning and not on the product.

Experiential learning helps to develop inductive reasoning, analysis, personal reflection, formulating plans, speaking and writing, and lifelong learning attitudes.

Suggested MOIs: case study, conducting experiments, field observations, field trip/site visit, gaming, model building, practical exercise (hands‐ on/written), role playing, simulation, storytelling

Excerpt from TP 350‐70‐14 (27 March 2015); pages 146‐152

Methods of instruction (MOI) Method Description Uses

Brainstorming A method of generating…