0009-EC OI 36-100 Education and Training (July 2020) Final.pdf

BY ORDER OF THE COMMANDER USAF EXPEDITIONARY CENTER

OPERATING INSTRUCTION 36-100

20 July 2020

Personnel

EDUCATION AND TRAINING

Accessibility: Publications and forms are available on the United States Air Force (USAF) Expeditionary Center .mil website for viewing and downloading. If you do not have access, please contact the Administration Office via email: usafec.knowledgeOps@.us.af.mil or the Office of Primary Responsibility (OPR) to request a copy.

Releasability: There are no restrictions on this publication OPR: USAF EOS/EOA Certified by: USAF EOS/CD (Mr. Charles Nesemeier) (Mr. Ken Arteaga)

Pages: 64 This Operating Instruction (OI) establishes the operational education and training rules, procedures, and general governance for the USAF Expeditionary Center (EC). This OI applies to all personnel assigned to the EC, including the staff and the Expeditionary Operations School (EOS). All records created as a result of processes prescribed in this publication are to be maintained in accordance with Air Force Manual 33- 363, Management of Records, and disposed of in accordance with Air Force Records Information Management System records disposition schedule located at https://www.my.af.mil/gcssaf61a/afrims/afrims/.

In accordance with the intent of Air Force Instruction (AFI) 36-360, Publications and Form Management, this OI reduces and simplifies the above-referenced OIs it supersedes.

The primary goal of this instruction is to establish a sound foundation to train and educate total force Airmen for a broad spectrum of expeditionary operations. The requirements of this instruction support the expectations of the EC Commander (EC/CC), EOS Commandant (EOS/CC), course functional managers, students, the Southern Association of Colleges and Schools Commission-on-Colleges, Air University, and the Community College of the Air Force.

SUMMARY OF CHANGES

This instruction supersedes:

USAF EOSI 36-100, Academic Standards, Policies and Procedures, 7 February 2020

FA4484-21-R-0003 Attachment 0009

2 USAF EC OI 36-100, 20 JULY 2020

Table of Contents

PART 1—USAF EOS ACADEMIC POLICIES AND STANDARDS 7

Chapter 1—ACADEMIC STANDARDS 7

1.1. Description. 7

1.2. Foundation of EOS Academic Policies and Standards. 7

1.3. EOS Mission and Academic Standards. 7

1.4. Roles and Responsibilities. 7

1.5. Advanced Study of Air Mobility. 7

Chapter 2—ACADEMIC POLICIES 8

2.1. Purpose. 8

2.2. Academic Integrity. 8

2.3. Academic Freedom. 8

2.4. Course Attendance. 9

PART 2—USAF EOS CURRICULUM STANDARDS 10

Chapter 3—GENERAL GUIDELINES 10

3.1. Purpose. 10

3.2. Initial Staff Training. 10

3.3. Maintaining Currency. 10

3.4. Curriculum Defined. 10

3.5. Curriculum Standards. 10

3.6. Roles and Responsibilities. 11

3.7. Release of Curriculum. 16

Chapter 4—INSTRUCTIONAL SYSTEM DEVELOPMENT PROCESS 17

USAF EC OI 36-100, 20 JULY 2020 3

4.1. Description. 17

4.2. Types of Instructional System Development (ISD) Events. 17

4.3. Analysis Phase. 17

4.4. Design Phase. 17

4.5. Development Phase. 19

4.6. Implementation Phase. 21

4.7. Evaluation Purpose. 21

4.8. Evaluation Phase. 22

Chapter 5—HW PROCESS 26

5.1. Level I Feedback Collection. 26

5.2. Generate Hotwash (HW) Reports and Assign Routing. 26

5.3. Assemble and Route HW. 26

5.4. Level III Data Collection. 27

PART 3—STANDARDS OF INSTRUCTION 28

Chapter 6—ROLES AND RESPONSIBILITIES 28

6.1. Purpose. 28

6.2. Standards of Instruction. 28

6.3. Staff Roles and Responsibilities. 28

6.4. Faculty Roles and Responsibilities. 31

Chapter 7—FACULTY QUALIFICATION 34

7.1. Initial Qualification. 34

7.2. Maintaining Faculty Qualifications. 36

7.3. Unqualified Faculty Process. 36

4 USAF EC OI 36-100, 20 JULY 2020

Chapter 8— FACULTY PROFESSIONAL DEVELOPMENT PROGRAM 38

8.1. Faculty Professional Development (FPD).Initial Staff Training. 38

8.2. Minimum Requirement. 38

8.3. New Instructor FPD. 38

8.4. Professional Development Event Defined. 38

8.5. Accomplishing FPD. 38

8.6. Funding Professional Development. 39

8.7. Documenting Professional Development. 39

Chapter 9— FACULTY EVALUATION PROGRAM 40

9.1. Evaluation Requirement. 40

9.2. Evaluator Defined. 40

9.3. Evaluator Training. 40

9.4. Evaluation Scheduling. 40

9.5. Conducting Evaluations. 41

9.6. Processing Evaluations. 41

9.7. Quality Assurance Evaluations. 42

PART 4—STUDENT SUPPORT STANDARDS 43

Chapter 10—STUDENT SUPPORT STANDARDS 43

10.1. Purpose. 43

10.2. Student Support Standards. 43

10.3. Responsibilities. 43

Chapter 11—REGISTRATION AND PROCESSING 46

11.1. Registration. 46

USAF EC OI 36-100, 20 JULY 2020 5

11.2. Enrollment. 46

11.3. Waivers. 46

11.4. Reporting Instructions. 46

11.5. In-processing. 46

11.6. Foreign Students 47

Chapter 12— STUDENT ACHIEVEMENT 48

12.1. Orientation. 48

12.2. Promoting Achievement. 48

12.3. Student Rights. 48

12.4. Student Responsibilities. 49

12.5. Student Elimination. 49

12.6. Early Release. 5

Chapter 13— GRADUATION REQUIREMENTS 51

13.1. Graduation. 51

13.2. Graduation Roster. 51

13.3. Student Recognition. 51

13.4. Student Feedback. 51

PART 5—USAF EOS OPERATIONS 52

Chapter 14— LIBRARY OPERATIONS 52

14.1. Circulation. 52

14.2. Reference. 53

14.3. Acquisitions. 54

14.4. Cataloging. 55

6 USAF EC OI 36-100, 20 JULY 2020

14.5. Administration. 56

Attachment 1—GLOSSARY OF REFERENCES AND SUPPORT 57

INFORMATION

Attachment 2—TRAINING MATRIX 59

Attachment 3—CUSTOMER REGISTRATION FORM 60

Attachment 4—INTERLIBRARY LOAN REQUEST 61

Attachment 5—RESEARCH REQUEST 62

Attachment 6—CIRCULATION CHECK-OUT SHEET 63

Attachment 7—APPOINTMENT OF LIBRARY OFFICE ACCOUNT CUSTODIAN 64

USAF EC OI 36-100, 20 JULY 2020 7

PART 1

USAF EC ACADEMIC POLICIES AND STANDARDS

Chapter 1

ACADEMIC STANDARDS

NOTE: Throughout this document the terms “faculty” and “instructor” are used interchangeably.

Unless stated otherwise, these terms are Air Force terms and are not considered academic rank.

1.1. Description. Academic standards are the benchmark of quality and excellence in education.

The reputation of a school is measured by the performance of its students, and disciplined adherence to rigorous standards is a critical ingredient in producing trained and educated total force Airmen.

1.2. Foundation of EC Academic Policies and Standards. EC academic standards are derived from several sources: EC and EOS missions and Commander’s intent, EOS affiliation with Community College of the Air Force (CCAF) and requirements derived from CCAF’s accreditation by Southern Association of Colleges and Schools Commission-on-Colleges.

1.3. EOS Mission and Academic Standards. The EOS mission is to train and educate total force Airmen for a broad spectrum of expeditionary operations today and tomorrow. To ensure individual course owner requirements are met, the EOS has established general academic policies affecting faculty, staff and students. These are sub-divided into four key areas addressed in succeeding parts of this instruction: curriculum, instruction, student support and EOS operations.

1.4. Roles and Responsibilities. All personnel are expected to pursue academic excellence in order to accomplish the EOS mission and further establish the EOS as the Air Force school of choice for expeditionary training and education. No matter the role, faculty or staff, or level of responsibility, each EOS member is expected to uphold all policies and standards identified in this guidance.

1.5. Advanced Study of Air Mobility (ASAM). ASAM students will adhere to all polices and standards identified in this guidance in addition to the program, policies and expectations identified within ASAM Handbook and Air University.

8 USAF EC OI 36-100, 20 JULY 2020

Chapter 2

ACADEMIC POLICIES

2.1. Purpose. Academic policies form the foundation of academic standards.

2.2. Academic Integrity. Academic Integrity is the uncompromising adherence to a code of ethics, morality, conduct, scholarship and other values related to academic activity. Violations of academic integrity include, but are not limited to, plagiarism, cheating and misrepresentation. Failure on the part of any individual to practice academic integrity reflects discredit both on the individual and their military service. Individuals who violate EC academic policies may be subject to adverse administrative actions including disenrollment from school. Additionally, military members may also be subject to discipline in accordance with the UCMJ.

2.2.1. Plagiarism is the act of appropriating the literary composition of another, or parts or passages of their writings, or the ideas or language of the same and intending to pass them off as the product of one’s own mind. For example, copying verbatim without providing proper citation material constitutes plagiarism.

2.2.2. Cheating is the act of giving or receiving improper assistance such as, but not limited to, copying answers from another student’s examination; using texts, notes, issue materials or other references not authorized for examinations or other assigned work; knowingly permitting another student to copy one’s writing assignments, speech material or answers from an examination paper; collaborating with other persons on individual assignments, except as specifically authorized.

2.2.3. Misrepresentation is the act of making an assertion to intentionally deceive or mislead.

Misrepresentation may be an oral or written statement that is misleading or deceiving and meant to be so; for example, false reporting.

2.3. Academic Freedom. The privilege of debate with discretion on any subject related to curricula within the EOS instructional environment requires application of the principle of academic freedom and the guarantee of non-attribution.

2.3.1. Academic Freedom is the policy whereby visiting lecturers, faculty and students are encouraged to state their opinions and support or criticize any objective or policy while pursuing knowledge and improving the military profession. Responsible classroom exploration and discussion of controversial issues are permitted. However, this policy does not include the freedom to use offensive remarks and irresponsible statements. This means academic freedom must be tempered with good judgment to preclude remarks of an offensive or disparaging nature, such as sexual, ethnic, racial nature and unprofessional behavior.

2.3.2. Non-Attribution is the treating of statements made in an educational forum as privileged information not to be attributed to a specific individual. Statements, disagreements and other comments made by guest speakers, instructors or students will be safeguarded through the practice of non-attribution. It is acceptable to say “a previous speaker” or “it was discussed in

USAF EC OI 36-100, 20 JULY 2020 9

our seminar,” but the names of the speakers will not be divulged. Permission must be received from speakers and the school concerned before remarks may be released or discussed with other individuals outside the academic forum.

2.3.3. Resolution of issues involving academic freedom will be handled through the course Commandant’s chain of command.

2.4. Course Attendance. EOS courses are considered mandatory formations. Students are expected to attend all scheduled classes. Any request to miss class will be coordinated with the chain of command.

10 USAF EC OI 36-100, 20 JULY 2020

PART 2

USAF EOS CURRICULUM STANDARDS

Chapter 3 GENERAL GUIDELINES

3.1. Purpose. World-class expeditionary training and education is vital in ensuring EOS graduates are prepared to carry out their mission. To meet this objective, course curriculum will focus on quality, relevance and responsiveness; training should be designed to mirror the mission environment to the maximum extent possible, whether in the field or a physical classroom. In addition, to preserve the EOS reputation as a school of choice for expeditionary training, and ensure compliance as a CCAF Affiliate, all curriculum must comply with the following standards.

3.2. Initial Staff Training. Attachment 2, Training Matrix, identifies specific training requirements for each staff position in the EOS. The Instructor Qualification Course (IQC) and Instructional System Development (ISD) Implementation Course will be the primary qualification training for staff (refer to paragraph 3.6.) members. Individuals filling both faculty and staff roles require both courses. Required position training will be completed as soon as possible after assignment.

3.3. Maintaining Currency. Staff members will stay abreast of evolving educational policies, procedures and innovations, and are encouraged to attend any faculty development events related to their position and mission.

3.4. Curriculum Defined. Curriculum refers to master lesson plan parts I-IV (refer to paragraph 4.5.2.).

3.5. Curriculum Standards. All curriculum will be:

3.5.1. Initiated IAW AMCI 36-2607, USAF EOS Course Development and Ownership Process, to ensure course owner requirements and required resources are identified prior to implementation.

3.5.2. Designed using the EOS ISD process to ensure course owner training objectives and required graduate knowledge, skills and abilities are documented (see Chapter 4, ISD Process).

3.5.3. Derived from an ISD Final Report, which will be approved by the EOS/CC as a signed contract between the course functional manager (FM) and EOS/CC prior to courseware development. Authority to sign ISD Final Reports may be delegated to the Dean of Academic Affairs.

3.5.4. Developed using EOS approved templates/guidance to ensure standardized delivery of training.

USAF EC OI 36-100, 20 JULY 2020 11

3.5.5. Managed by an assigned course director trained in EOS course management to ensure compliance with EOS and CCAF standard.

3.5.6. Maintained on the EIM Course Hotwash (HW) page at

https://eim2.amc.af.mil/org/afec/Training/hotwash/default.aspx to ensure standardization, transparency and continuity of course material is maintained.

3.5.7. Reviewed by the office of the Academic Affairs Section prior to course implementation to ensure readiness for delivery.

3.5.8. Submitted to CCAF for evaluation IAW CCAF Campus Affiliations Policies, Procedures and Guidelines (PPG) for all courses with an enlisted target audience prior to initial class start date to ensure graduates are awarded appropriate credit.

3.5.9. Evaluated using the EOS HW process to ensure currency, relevance and a culture of continuous improvement.

3.5.10. Updated using the EOS curriculum change process outlined in paragraph 3.6.6.7.2.3.

3.5.11. Reviewed annually by the Dean’s Standardization and Evaluation Section to ensure compliance with EOS and CCAF policies.

3.6. Roles and Responsibilities. EOS and subordinate unit staff manage the educational process and resources to ensure students receive world-class training and education. Staff is defined as EOS personnel who manage or directly support the educational process of the school and may or may not additionally serve as faculty. Positions include but are not limited to: Academic Affairs Section, Unit Commander (Unit/CC), Unit Director of Operation (Unit/DO), Flight Commander/Chief, Section Commander/Chief and Course Director. This section describes the curriculum management roles of the EOS/CC and each staff position. Roles and responsibilities are in addition to and IAW AMCI 36-2607, USAF EOS Course Development and Ownership Process.

3.6.1. Commandant, USAF Expeditionary Operations School (EOS/CC). The Commandant provides leadership and resources to ensure the EOS meets course owner requirements, remains a CCAF-affiliated school and is recognized as a school of choice.

3.6.1.1. Ensures all courses are developed using the USAF ISD process applicable volume of AFH 36-2235, Information for Designers of Instructional Systems.

3.6.1.2. Responds to new course requests.

3.6.1.3. Identifies subordinate unit responsible for course development and implementation.

3.6.1.4. Signs ISD final report or delegates signing authority to the Dean of Academic

Affairs.

12 USAF EC OI 36-100, 20 JULY 2020

3.6.2. Academic Affairs Section (EOA). The Academic Affairs Section is responsible for the overall standardization of course design and ensures compliance with CCAF Campus Affiliations PPG.

3.6.2.1. Implements ISD process in EOS.

3.6.2.2. Facilitates all ISD reviews, manages the ISD design phase and manages the Routing/Request Tracker located on EOS EIM site to identify status of ISD products.

3.6.2.3. Creates requests for Personnel Data System (PDS) codes.

3.6.2.4. Posts ISD final report to course HW Tab 1 after final coordination.

3.6.2.5. Reviews master course documents (course lesson plan parts I/II on HW Tab 1) prior to implementation date in ISD final report.

3.6.2.6. Revises ISD reports upon course owner and Unit/CC request.

3.6.2.7. Submits ISD final reports to CCAF for evaluation.

3.6.2.8. Tracks and updates HW Tab 1 course hour percentage changes to curriculum.

Notifies course director when the course requires an ISD due to a 40% change since the last ISD review.

3.6.2.9. Maintains USAF EOS lesson plans and slide templates.

3.6.2.10. Performs annual course reviews on all courses and creates action items for any deficiencies.

3.6.2.11. Develops individual course documentation for instructor internship and experience hours.

3.6.2.12. Provides training for USAF EOS staff and course directors on HW ISD and EOS processes.

3.6.2.13. Provides end of quarter/year briefing to EOS/CC and subordinate leadership on significant trends in quality and responsiveness that affects the academic health of the

USAF EOS.

3.6.2.14. Publishes EOSLearn course pages as requested by course directors.

3.6.2.15. Conducts ISD workshop out-briefs for EC/MS representatives, EOS leadership and unit leadership.

3.6.3. Unit Commander (Unit/CC). Unit/CCs provide leadership and resources necessary to satisfy course owner training requirements.

USAF EC OI 36-100, 20 JULY 2020 13

3.6.3.1. Designates a point of contact (POC) for coordinating an initial ISD review date with Standardization and Evaluation and coordinates with EOS/CC on new course requests.

3.6.3.2. Reviews ISD workshop final report prior to submission to Academic Affairs Section. ISD final reports have a 60 day routing suspense to USAF EOS/EOA.

3.6.3.3. Receives ISD workshop out-briefs from designated course director or POC

3.6.3.4. Appoints qualified course directors to manage courseware development.

3.6.3.5. Reviews HWs and action items to ensure course effectiveness.

3.6.3.6. Ensures Unit Resource Advisor includes course budgets with annual financial plans that are inclusive of TDY costs, ISD reviews, etc.

3.6.3.7. Appoint an additional duty Unit Training Manager from their Support Staff for units without a 3F2X1 (Unit Training Manager) assigned, and ensure the member completes the Additional Duty Unit Training Manager Computer-Based Training located in ADLS within 60 days of appointment (90 days for ARC). Forward a copy of the appointment letter to the Base Training Office and Base 3F2 Functional Manager.

3.6.3.8. Appoint a unit member to manage the ancillary training program.

3.6.4. Unit Commanders are also responsible for executing daily operations. This responsibility may be delegated (typically to the Operations Officer). This includes managing academic processes within the unit and serving as the conduit between Course Directors and the Academic Affairs Staff on academic and curriculum standards.

3.6.4.1. Executes specific authorities granted by Sq/Det Commanders to facilitate the unique needs of the unit and its mission. These authorities may include but are not limited to, final review of HWs, instructor evaluations and instructor assignments.

Delegated authorities will be captured in a delegation letter and provided to the Academic Affairs Staff who will adjust document routings as appropriate.

3.6.4.2. Manages unit HW action items.

3.6.4.3. Works with Course Directors, section, and flight leadership to de-conflict TDY requirements of assigned personnel to ensure course continuity.

3.6.4.4. Recommends instructor upgrades to Instructor Evaluator and Master Instructor for Sq/Det CC approval.

3.6.4.5. Monitors instructor training and requirements.

14 USAF EC OI 36-100, 20 JULY 2020

3.6.5. Flight Commanders, Flight Chiefs, and Section Chiefs. Flight CCs, Flight Chiefs and Section Chiefs play critical front line leadership roles within the unit.

3.6.5.1. Executes specific authorities granted by unit commanders to facilitate the needs of the unit. Delegated authorities will be captured in a delegation letter and provided to the Academic Affairs Staff who will adjust document routings as appropriate.

3.6.5.2. Recommends Course Director and Instructor Assignments for approval.

3.6.5.3. Monitors instructor training, qualifications, and evaluations for assigned personnel.

3.6.5.4. Manages assigned resources and provides Sq/Det Resource Advisor with course budget information for all assigned courses.

3.6.6. Course Director. The course director is the OPR for the management of an assigned course.

3.6.6.1. Successful completion of the EOS ISD course is a prerequisite for being designated as a course director.

3.6.6.2. Ensure quality of the courseware and qualifications of instructors assigned.

3.6.6.3. Performs recurring informal course reviews with course owner to address future curriculum and budget change requirements. Informal course reviews time requirements will be determined by the Unit/CC. Formal course reviews are conducted IAW Chapter 4, ISD Process.

3.6.6.4. Performs the following ISD roles:

3.6.6.4.1. Coordinates with Standardization and Evaluation prior to scheduling any ISD review. Works closely with FM to address all requirements of AMCI 36-2607 prior to an ISD for all new and revised course requests. Will utilize the ISD Workshop Checklist posted on the EOS Academic Standards and Evaluation EIM site.

3.6.6.4.2. Schedules and prepares classroom for ISD workshop (dry-erase board, easel, markers, etc.).

3.6.6.4.3. Provides ISD workshop attendees any applicable course statistical data (course metrics, feedbacks, etc.) and access to EC facilities (see EC Security Manager).

3.6.6.4.4. Develops course and class schedules during the ISD workshop.

3.6.6.4.5. Identifies known equipment/personnel requirements and costs for course development and operation. Assets belonging to another EOS subordinate unit requires cross-unit coordination prior to making a resource estimate/commitment.

USAF EC OI 36-100, 20 JULY 2020 15

3.6.6.4.6. Ensures course owner addresses all requirements of AMCI 36-2607, USAF EOS Course Development and Ownership Process prior to an ISD workshop for new and revised course.

3.6.6.4.7. Develops and maintains Education and Training Course Announcement (ETCA) and submits to Registrar Division for publication.

3.6.6.5. Develops, posts and maintains course master lesson plans (Parts I and II) in applicable HW Tab 1 for each lesson in the ISD final report.

3.6.6.6. Develops, posts and maintains course syllabus on EOS Student web site. Course syllabus template is located on the Standardization and Evaluation EIM site.

3.6.6.7. Performs the following HW requirements:

3.6.6.7.1. Posts Level I and III reports, Level II results and master course schedule to HW Tab 1 after each class. The course director or commander’s representative will post instructor hours.

3.6.6.7.2. Posts course owner approved curriculum changes to HW Tab 1.

Standardization and Evaluation will update ISD final report.

3.6.6.7.3. Posts master lesson plan changes approved by course owner to HW Tab 1.

Underlines changes followed by date and initials e.g., MP 1. Types of ISD reviews (1 Jul 15, MJC).

3.6.6.7.4. Reviews HWs and initiates/implements course improvement through HW action items to ensure continuity and accountability.

3.6.6.7.5. Performs Level I, II and III analysis.

3.6.6.7.6. Provides quarterly course report to course FM IAW course ISD FR.

3.6.6.8. Corrects deficiencies found during review of master course documents.

3.6.6.9. Ensures use of personalized lesson plans derived from posted master lesson plans.

3.6.6.10. Ensures instructor subject matter qualification (SMQ) is completed prior to course assignment. Documents SMQ for all assigned instructors (See Part 3, Standards of Instruction, for specific requirements).

3.6.6.11. Secures facilities for course delivery.

3.6.6.12. Coordinates with Registrar on course schedules, HWs, level-I feedback, ETCA, etc.

16 USAF EC OI 36-100, 20 JULY 2020

3.7. Release of Curriculum. Master Lesson Plans (refer to para. 4.5.2.) Parts I and II (Part IV as approved by Unit/CC) will only be released to outside parties by the course director and only after in receipt of approval email by the course FM. Upon release, the curriculum must no longer be represented as EOS courseware (remove all reference to USAF EOS in released curriculum).

Part IIIs and any supporting test material will not be released since these are controlled items.

USAF EC OI 36-100, 20 JULY 2020 17

Chapter 4

ISD PROCESS

4.1. Description. The ISD process supports the development, delivery and maintenance of world- class curriculum with a focus on quality, relevance and responsiveness. The ISD is a contract between USAF EOS and the course FM. USAF ISD model is a five-phase approach to training development: Analysis, Design, Development, Implementation and Evaluation (AFH 36-2235 Volumes 1-10). The USAF EOS ISD process (for new and existing courses) begins with a formal course request outlined in AMCI 36-2607, USAF EOS Course Development and Ownership Process.

4.2. Types of ISD Events.

4.2.1. Initial ISD Workshop. Designed to conduct a comprehensive design of course content utilizing onsite feedback from the course owner, subject matter experts (SMEs) and Standardization and Evaluation facilitator to determine training requirements based on current and future trends. Results of the ISD workshop will be documented in an ISD Final Report (approved by course owner and EOS/CC) and submitted to CCAF IAW CCAF Campus Affiliations PPG.

4.2.2. ISD Revision. Course owners may request an ISD review at any time. Additionally, a review is conducted on all active courses after a 40% change in lesson content has occurred since the last ISD. The course director, course owner and Standardization and Evaluation facilitator will conduct a review using course Level I-III data. The course owner will determine field SME representation. All changes will be documented in an ISD final report (approved by course owner and EOS/CC) and submitted to CCAF IAW the Campus Affiliations PPG. Unit CCs will ensure unit ISD reviews are programed for and budgeted annually.

4.3. Analysis Phase. During this phase, the course owner conducts an analysis of mission, occupation, task, learning, resources and target audience to determine instructional requirements. EOS personnel may assist with gathering input from the field experts used in course design.

4.3.1. Approach. Guidance for establishing new courses is covered in AMCI 36-2607.

4.3.2. Process. The creation of a new course typically starts with a dialog between Functional Manager (FM) and the EOS. AMCI 36-2607 provides a sample course request letter to FM’s to assist in their course request to EOS/CC. This request will be assigned to an EOS subordinate unit to manage the development process.

4.4. Design Phase. Curriculum is designed based on objectives articulated in the Functional Manager FM’s course request letter based on the requirements outlined in AMCI 36-2607. An objective is a precise statement of a learned capability--knowledge, skills and abilities (KSA)--graduates must demonstrate.

18 USAF EC OI 36-100, 20 JULY 2020

4.4.1. Approach. The ISD Workshop is required to design all EOS courses. Participants required to accomplish the course design include an EOS/EOAC Standardization and Evaluation facilitator, appointed course director/course POC, FM, FM selected SME’s and assigned instructors (specific roles and responsibilities are addressed in para 4.4.3. below).

During the workshop the participants will populate the Instructional System Development Final Report and Course Control Document (ISD FR & CCD) as explained in para. 4.4.4.

below, ISD workshops are typically five days long. Unless waived by the EOS/CC, ISD workshops will be conducted in the CONUS to contain costs. The FM is the final approval authority on course objectives, samples of behavior and course length, but must provide the resourcing to support those outcomes.

4.4.2. Process. The EOS/EOAC Standardization and Evaluation facilitator will guides and manage the effort of the ISD workshop to meet course design requirements with the ISD Final Report (FR) and Course Control Document (CCD) as the workshop deliverable. Upon completion of the ISD workshop, EOS/EOAC will initiate ISD FR & CCD which is essentially a signed contract between the EOS/CC and FM. Course development begins once the ISD FR & CCD is signed by the EOS/CC (or Dean of Academic Affairs), FM, and recourses are in place.

4.4.3. Roles and Responsibilities of workshop participants.

4.4.3.1. EOS/EOAC Standards and Evaluation facilitator. Initiates scheduling an ISD workshop with unit and FM’s POC. Briefs workshop attendees of their assigned roles and responsibilities. Facilitates the ISD workshop using the ISD FR & CCD populated from the FM’s course request letter required for all new and revised course requests IAW

AMCI 36-2607.

4.4.3.2. Course Functional Manager. The FM is the final approval authority on course content, course length and target audience

4.4.3.3. FM selected SMEs. The SMEs will provide the needed subject matter expertize to design course curriculum and reference material.

4.4.3.4. Assigned course director/course POC and Instructors. The assigned course director/course POC and instructors will provide the needed inputs to recommend course content and identify time required for lesson delivery, resources required, and lesson development.

4.4.4. Workshop Execution

4.4.4.1. Throughout the workshop, the EOS/EOAC Standardization and Evaluation facilitator will guide and manage the ISD workshop. During the workshop, the CD or Course POC appointed scribe will take direction from the EOA/EOAC facilitator and the FM to populate the Instructional System Development Final Report and Course Control Document. The ISD workshop begins with a meeting of the FM, course director/course POC, EOS/EOAC facilitator, and appointed scribe to populate the

USAF EC OI 36-100, 20 JULY 2020 19

portions of the Instructional System Development Final Report and Course Control Document that do not require inputs from the SME’s or assigned instructors. A short orientation will include introductions, explanation of roles and responsibilities, and short description of the workshop schedule and flow.

4.4.4.2. The SME’s and assigned instructors will join the workshop after the initial meeting to begin the curriculum review. The SME’s and assigned instructors role is to support the design of the summary of lessons and length of lessons sections of the ISD FR & CCD. EOS/EOAC will conduct a short orientation to include introductions and re-state the ISD roles and responsibilities followed by short sample lesson review beginning with identifying the title of the lesson, creating lesson objective, sample of behavior’s, main points, sub-points and sub-sub points as required by the EOA/EOAC facilitator identifying reference material, and time needed to deliver and develop the lesson. SME’s and assigned instructors will continue to provide lesson inputs until the time all lessons are completed/approved by the FM at which time the SME’s will be dismissed by the EOA/EOAC facilitator.

4.4.4.3. The FM, course director, EOS/EOAC facilitator, and unit appointed scribe will continue to populate the ISD FR & CCD as required. Lastly, a full review will be conducted.

4.5. Development Phase. The development phase produces master course documents as follows:

4.5.1. Course Syllabus. The course director will develop a course syllabus based on the content of the ISD final report. The course syllabus is a contract between the course director and student. At a minimum, the course syllabus will contain the requirements identified on the syllabus template located on the Standardization and Evaluation EIM page.

4.5.2. Master Lesson Plans. The course director will develop the master lesson plans from the ISD final report posted in the course HW Tab 1, creating a master lesson plan or activity statement for each lesson identified in the final report. Master lesson plans will not be personalized. Use the EOS standardized lesson plan templates for all courses. Master lesson plans include the following: Part I Cover Sheet, Part II Teaching Plan, Part III Evaluation Instrument and Part IV Instructional Aids.

4.5.2.1. Part I. Addresses content /delivery and contains three parts: Part I, Cover Sheet; Part IA, Lesson Objectives; and Part IB, Strategy.

4.5.2.1.1. Part I. Cover Sheet, identifies course title, lesson title, OPR, method and length of instruction, references and required instructional aids. References are derived from SMEs, including the lesson developer and will be available to all applicable instructors.

4.5.2.1.2. Part IA. Lesson Objectives, are derived from the course ISD appendix consisting of at least one cognitive and one affective level of learning objective, or one criterion objective and one affective objective, or an activity statement as appropriate to the lesson. The level of learning objective will have as many samples

20 USAF EC OI 36-100, 20 JULY 2020

of behavior as deemed necessary to adequately sample mastery. Samples of behavior may be at a lower cognitive level to assist instructors with course evaluations. At least one sample of behavior will be written at the overall lesson objective level of learning.

4.5.2.1.3. Part IB. Strategy, is a detailed plan that explains the overall lesson objective and steps to achieve that objective effectively. The strategy can be compared to a set of instructions written by the lesson developer to deliver the lesson. The strategy statement should walk the instructor through the entire lesson, focusing on every lesson element to include teaching techniques and instructional aids. At a minimum, indicate what is to be taught, why it is being taught and how to teach it, including employment of instructional aids/handouts. A well written strategy statement gives an instructor cues to personalize the lesson and develop strong transitions and motivational themes.

4.5.2.2. Part II. The teaching plan contains the lesson outline and support material to achieve lesson objectives. Use the introduction, body, conclusion format.

4.5.2.2.1. The introduction consists of attention step, motivation step and overview.

Attention and motivation steps will be left blank to allow for instructor personalization.

At a minimum, the overview will be populated with the lesson outline found in Part

IB.

4.5.2.2.2. The body contains all main points and support. Main points must match the outline contained in Part IB and contain enough support material to satisfy all samples of behavior. A simple outline is not sufficient.

4.5.2.2.3. The conclusion consists of a summary step, re-motivation and closure. Re-motivation and closure steps will be left blank to allow for instructor personalization.

At a minimum, the summary will be populated with the lesson outline found in Part

IB.

4.5.2.2.4. Include applicable references from Part I. Copying large amounts of information from AFIs or Tech manuals into the body is not an effective method of capturing the required support material. Simply state the reference and section number and add useful information for the instructor to execute/teach the material.

4.5.2.3. Part III. The Part III, evaluation instrument, consists of test questions for cognitive objectives or an evaluation checklist for criterion objectives. Test questions will include the cognitive lesson objective, sample of behavior, test question, correct response, minimum of three alternatives for multiple choice test questions, rationale for the correct answer and all alternatives. Three test questions for each sample of behavior are required (one for pre-test, one for end-of-course test and one for re-test). Criterion objective lesson plans will use an evaluation checklist in place of test questions. Evaluation checklists must contain the standard and method of grading the performance. Courses using a Capstone/FTX evaluation will follow criterion objective evaluation checklist for grading performance.

4.5.2.3.1. Test Control. All lesson plan Part IIIs and SMQ tests will be secured as

USAF EC OI 36-100, 20 JULY 2020 21

controlled items to prevent test compromise (see Standardization and Evaluation EIM “CRS Test Management” page). Document printed test material used for instructor subject matter qualification and student testing on a test control log. Identify test number, log-in date, log-out date and destroy date. Student answer sheets are also a controlled item and will be destroyed when no longer required.

4.5.2.4. Part IV Instructional Aids. Use guide located on Standardization and Evaluation EIM site.

4.5.2.5. Activity Statement. A document composed of: Activity Statement, Purpose of the Activity and Activity Procedures. It serves as the lesson plan for selected hours in a course which are designed to support course execution or enhance knowledge but do not contain tested material.

4.5.3. Personalized Lesson Plans. Instructors will use personalized lesson plans when teaching. Use the current master lesson plan posted in the course HW Tab 1 as a foundation.

4.5.3.1. Part I. The only area in which an instructor may personalize a Part I is by adding additional instructor aids to be used to the original required instructional aids listed on the master lesson plan.

4.5.3.2. Part II. Instructors are required to create an effective attention, motivation, re-motivation and closure statement which directly correlates to the lessons cognitive and affective objectives. Instructors are required to create required transition statements between introduction, identified main points and conclusion. Instructor will not add, delete or modify master lesson plan main points, sub points, etc. Instructors can add questions, references, stories, etc., to reinforce the subject area.

4.6. Implementation Phase. Implements support, maintenance, and evaluation to ensure the effective preparation of graduates to meet mission requirements.

4.6.1. Approach. The course director or unit commander designee is responsible for implementing the training. Course will be delivered IAW the course syllabus and course documents identified in para 4.5. Before the first class is taught, the course director will ensure that faculty, curriculum, facilities, equipment and other resources identified in the ISD final report are in place to deliver an effective course.

4.6.2. Student Performance. Student performance is measured against course objectives to validate the quality of the curriculum and instruction. Refer to the course syllabus for graduation requirements.

4.7. Evaluation Purpose. Provides guidelines and procedures for institutional effectiveness (IE) in support of the educational programs, student administrative and support services within EOS. The school engages in ongoing, integrated and institution-wide planning and evaluation processes that result in continuous improvement of institutional and program quality. EOS’ IE process supports educational programs; develops and implements a systematic solution to

22 USAF EC OI 36-100, 20 JULY 2020

training; provides a comprehensive process for measurement of performance and assessment of student learning. IE is comprised of the strategic planning, operational planning, and multi-faceted analysis to integrate and continuously evaluate EOS’s training programs in an effort to track courses progress in fulfilling its course owner’s vision and mission requirements.

4.8. Evaluation Phase. The EOS uses Kirkpatrick’s evaluation model to assess course effectiveness at three levels: (1) Student Reaction: student feedback on the effectiveness of training; (2) Learning: student knowledge measured at course outset (pre-test/evaluation) and conclusion (post-test/evaluation) to ensure students were trained at the right point in their development and learned intended objectives; and (3) Behavior change in the workplace:

graduate and supervisor/observer assessment as defined in the ISD Final Report after graduation to ensure training is having the desired effect on workplace deliverables. Assessment feedback is analyzed to drive action items. Metrics for each class are maintained on the HW Dashboard.

4.8.1. Evaluation Requirements. EOS course directors will analyze course Level I, II and III data to assess course effectiveness, take corrective actions to maintain required EOS standards, and create action items to improve course performance. Course directors will provide historical course performance to course FM when conducting any course review.

For courses that have low annual student graduation numbers and or low annual course offerings, evaluation requirements (Level I, II, and III as applicable) will be aggregated as determined by the Unit/CC but no less than once a year.

4.8.1.1. Level I Rating. Student feedback administered for in-resident and instructor led online courses at the end of each class measures reaction to course curriculum, instructor and relevant logistical support areas. Metric scores for each area are combined to deliver an overall Level I rating. Non-target audience ratings will be excluded from overall course ratings.

4.8.1.2. Level I Analysis. The analysis of Level I data provides supervisors, EOS leadership and the course owner with student perceptions of course effectiveness and a plan for improvement. Level I performance has a scale of 0.00 – 5.00. Ratings below 4.00, identified deficiencies and negative trends should be targeted for corrective action.

4.8.1.2.1. Overall course rating of 4.00 or above, indicates the course is being delivered effectively.

4.8.1.2.2. Overall course rating of 3.51 through 3.99, indicates deficiencies requiring attention.

4.8.1.2.3. Overall course rating of 3.50 or below, indicates deficiencies requiring immediate attention.

4.8.1.3. Level I Considerations. Key elements contributing to Level I ratings include target audience attendance, instructor delivery, curriculum/supporting infrastructure quality and student readiness. Factors to consider:

USAF EC OI 36-100, 20 JULY 2020 23

4.8.1.3.1. Target Audience. Course directors should monitor demographic data to ensure students meet the target audience identified in the ISD final report. Notify the course owner if graduates are outside the target audience.

4.8.1.3.2. Instructor/Curriculum Delivery. Ensure a qualified instructor is present whenever a student instructor (IAW para. 7.1.3.1.2.) or guest speaker is presenting course material (IAW Part 3, Standards of Instruction). Level I feedback is a good indicator of the preparation and effectiveness of presenters that are not fully qualified faculty.

4.8.1.3.3. Currency and Accuracy of Curriculum. Written Level I negative feedback is frequently an early warning that curriculum is not current or accurate. Examine all student concerns and take prompt action when trends occur.

4.8.1.4. Level II Analysis. Level II analysis is critical to validating the achievement of learning objectives and accuracy of the target audience. To support Level II assessment, all in-resident and instructor led online courses will have a pre- and post-evaluation at the level of learning objectives identified in the ISD final report unless exempted in the report. Develop Level II assessments to match course objective—for example, a course with only cognitive objectives (knowledge/comprehension) is typically assessed with a multiple- choice test, while a course with application/analysis/synthesis objectives would be more appropriately assessed by student performance (criterion evaluation). For courses with blended cognitive and psychomotor objectives, use a blended pre/post assessment. Assess criterion objectives using an evaluation checklist. Non-target audience ratings will be excluded from overall course ratings. Course directors may elect to only post-test criterion objectives when in their judgment pretesting will not yield sufficient value for the resources expended.

4.8.1.5. Level II Cognitive Objective Parameters. Student performance on assessments will be scored from 0-100%.

4.8.1.5.1. Pre-tests/assessments. Course pre-tests scores indicate the knowledge/skill level of students prior to instruction. If the correct target audience is present and the pre- test is properly constructed, average pre-test scores should not exceed 40%.

4.8.1.5.1.1. Class average assessment rating equal to or less than 40% indicates the course is being delivered effectively.

4.8.1.5.1.2. Class average assessment rating greater than 40% indicates deficiencies requiring attention.

4.8.1.5.2. Post-tests/assessments. Course post-tests indicate the knowledge/skill level of students after instruction. Again, if the correct target audience is present and the test is properly constructed, average post-test scores should range between 70- 95%.

24 USAF EC OI 36-100, 20 JULY 2020

4.8.1.5.2.1. Class average assessment rating greater than 70% indicates the course is being delivered effectively.

4.8.1.5.2.2. Class average assessment rating less than 70% indicates deficiencies requiring attention.

4.8.1.6. High Pre-test Scores. High pre-test scores may indicate students already know the course content, limiting the value of training and affecting attitude toward attendance.

The first action to take is a test analysis to ensure the test properly measures the learning objectives of the ISD. If the test is properly constructed, recommend the course owner either adjust the selection of students or amend the curriculum to achieve a higher return on training investment.

4.8.1.7. Low/High Post-test Scores. Low average post-test scores may indicate substandard achievement of desired learning objectives and may require improvement in either faculty or curriculum. Excessively high post-test averages (>95) may indicate that either the test is not rigorous enough, poorly designed or that the instructor is “teaching the test.” In either case, further causal analysis is warranted, beginning with test analysis. Critical training areas involving (e.g., safety, loss of life, hazardous materials handling and etc.) can reach higher test item averages of 95% or higher and should not be considered as an area of concern.

4.8.1.7.1. Ease Index. Ease index is simply the percentage of students taking the test who answered the item correctly--the larger the percentage, the easier the question. To compute the ease index, divide the number of students answering the item correctly by the total number of students taking the test. An item answered correctly by 85% of the students would have an ease index of .85. Questions that receive an ease index of 50% or lower requires a point bi-serial review identified in the paragraph below to determine how well a test question discriminates.

4.8.1.7.2. Point Bi-Serial. A point bi-serial correlation coefficient relates observed item responses to a total test score and rates the observed item from +1.00 through -1.00. A high coefficient means students selecting the correct response achieved higher total scores, and students selecting the incorrect response achieved lower total scores (positive correlation). A low coefficient means students selecting the correct response achieved lower total scores, and students selecting the incorrect response achieved higher total scores (negative correlation). A point bi-serial rating of .21 or less indicates that the test item should be removed from the test.

4.8.1.8. Level II Criterion Objective Evaluations. Criterion evaluations are intended to measure student task performance in a controlled standardized condition set by the instructor. “Go/No-Go” performance evaluations may be used for student elimination or re-evaluated IAW the master lesson plan.

4.8.1.9. Level III Analysis for Resident/Mobile Training Team (MTT) or Instructor Led Online Courses. Level III assessment is key to validating a course is having the intended effect in improving workplace performance or reducing the time necessary to reach

USAF EC OI 36-100, 20 JULY 2020 25

mission qualification. Course directors will gather Level III data through feedbacks sent to target audience individual graduates and their supervisors/observers using the EOS CD Reporting Tool or through an alternative means such as interviews/theater visits.

Feedbacks will be accomplished IAW ISD final report after class graduation to allow time for the training effect to be adequately assessed. The individual identified as a “supervisor/observer” must be in a position to observe the effect of EOS training on job performance. Ratings are reported for both graduate and supervisor/observer assessments.

Feedback requests will include specific guidance along with a web link to the EOS Level III training feedback site. The feedback site is also accessible directly from the EOS public webpage (EOS Student link). Course directors are provided direct login access to pull Level III reports at any time. The report should include a minimum of 12 months of feedback.

26 USAF EC OI 36-100, 20 JULY 2020

Chapter 5

HW PROCESS

5.1. Level I Feedback Collection. On the last day of class, course directors/instructors will brief students on Level I data collection procedures and administer the online feedback process. Paper feedback collection will only be used in the absence of on-line access through any internet connection. Directions for online feedback completion, validation and using paper feedback collection can be found in the HW Training Library.

5.1.1. If paper feedback collection is used due to a lack of student on-line access, course directors will input the feedback on-line and attach a scanned copy of all student feedback in .pdf file format to their HW Level I section.