AFMCMAN_36-301-DRAFT_29JUL2019.docx

DRAFT – NOT FOR IMPLEMENTATION OR COMPLIANCE

AFMCMAN 36-301

BY ORDER OF THE COMMANDER AIR FORCE MATERIEL COMMAND

AIR FORCE MATERIEL COMMAND MANUAL 36-301

[DATE]

Maintenance

AFMC FUNCTIONAL DEPOT MAINTENANCE COURSEWARE DEVELOPMENT AND MANAGEMENT GUIDE

COMPLIANCE WITH THIS PUBLICATION IS MANDATORY

ACCESSIBILITY: Publications and forms are available for downloading or ordering on the e-publishing website at www.e-Publishing.af.mil RELEASABILITY: There are no releasability restrictions on this publication.

OPR: HQ AFMC/A4/10/A4PT

Certified by: HQ AFMC/A4/10/A4P (Mr. Donald Lucht) Supersedes:

Pages: XX

This manual presents Instructional System Development (ISD) methodology and activities as they apply to functional courseware development and management in Air Force Materiel Command (AFMC). It implements guidance contained in AFPD 36-26; Total Force Development and Management, AFI 36-2651, Air Force Training Program; AFI 36-401, Civilian Training, Education, and Professional Development; and AFH 36-2235 VOL. 9, Information for Designers of Instructional Systems-Application to Technical Training. This manual only applies to military, civilian, and contractor personnel responsible for developing and managing AFMC functional depot maintenance training courseware.

Ensure that all records created as a result of processes prescribed in this publication are maintained in accordance with Air Force Manual (AFMAN) 33-363, Management of Records, and disposed of in accordance with Air Force Records Disposition Schedule (RDS) located in the Air Force Records Information Management System (AFRIMS). Refer recommended changes and questions about this publication to the Office of Primary Responsibility (OPR) using AF Form 847, Recommendation for Change of Publication; route AF Form 847 from the field through the appropriate functional chain of command. Submit requests for waivers through the chain of command to the appropriate Tier waiver approval authority, or alternately if a non-Tiered requirement, to the publication OPR for non-tiered compliance items consideration.

SUMMARY OF CHANGES

This is a new manual and requires a complete review.

Chapter 1

INTRODUCTION

1.1. Purpose. In order for participants in AFMC Courseware Development and Management Process to work efficiently and effectively together, procedures must be well documented. Such documentation is particularly valuable when applying a complex and dynamic methodology such as Instructional System Development (ISD). ISD is flexible and systematic. It is a tool to get the right training for the problem, not a lockstep linear process. This manual promotes standardization of courseware development and management activities across AFMC and reduces time needed to train new participants entering the process. It lists typical activities used for the full spectrum of courseware issues and allows a courseware program/project manager to select only those activities appropriate for a particular situation.

1.2. Application. Use of the processes in this manual and minimum evaluation criteria for performed activities is mandatory for development of formal, functional training provided to AFMC Depot Maintenance audiences. It is recommended for development of education and employee development instruction. It applies to military, civilian and contractor personnel performing one or more activities in the AFMC Courseware Development and Management Process for functional training.

1.2.1. Instruction Defined. Instruction is information provided in a structured manner for education, employee development, or training purposes.

1.2.2. Formal Training Defined. Formal training is instruction based on established learning objectives that enables delivery of training to be accurate, complete and consistent information time after time. Formal training is assigned a course number to document student completions in employee records.

1.2.2.1. Informal Training. Informal training does not have defined learning objectives. It provides basic information but not always the same way or at a similar level of detail. It is tailored specifically to the situation and audience. Coaching, tutoring and informal on-the-job training that is based entirely on variables of each situation (e.g., actual workload, prior employee experience, trainer background etc.) are considered forms of informal training.

1.2.3. Functional Training Defined. Functional training is instruction that helps employees learn how to perform specific work processes so they can efficiently and economically accomplish their mission. It differs from education and employee development instruction which focus on learning for growth of an individual rather than immediate support of a mission.

1.2.4. Soft Skills Defined. Soft skills address feelings, emotions and attitudes, which are part of the affective domain of learning. Instruction on teambuilding, change management and customer service are examples of soft skills training. Soft skills courses are considered functional training for purposes of this guide if they are implemented to support performance in the workplace.

1.2.5. Education Defined. Education are opportunities that helps learners acquire new knowledge disciplines, skill sets, and viewpoints that are not based on performance requirements of the current job. It prepares learners for non-specific future requirements and its effectiveness cannot be fully evaluated until the learner is on a job that requires performance related to educational learning objectives.

1.2.6. Employee Development Defined. Employee development is opportunities that focuses on growth of an individual rather than performance of a job. While it may indirectly help the mission by increasing promotion potential of employees or improving their perspective or morale, employee development instruction does not directly relate to specific job tasks.

1.2.7. Instructional System Defined. An instructional system is an organized combination of procedures, techniques, and resources (e.g., instructors, students, courseware, equipment, tools, facilities, etc.) that are used to instruct specified learning objectives.

1.2.8. Courseware Defined. Courseware is a term commonly used in the Department of Defense (DoD) for course control documents, instructional materials, evaluation materials, and supporting documentation for an instructional system. Examples of courseware items are plans of instruction, training simulation programs, instructional presentations, student exercises, written tests, source code and authoring files, correspondence, decision documentation, etc.

1.3. Scope. This manual does not address methods and resources needed to deliver instruction after courseware validation. This manual covers:

1.3.1. Planning of courseware projects;

1.3.2. Analysis of instructional requirements;

1.3.3. Design of instruction;

1.3.4. Development of courseware;

1.3.5. Implementation of instruction;

1.3.6. Evaluation/Finalization of instruction;

1.3.7. Formal Re-Evaluation (Periodic Review) of instruction;

1.3.8. Management of courseware.

1.4. How to Use This Manual. Depending on the situation, users may choose to go directly to a phase and activity that applies, and decide which parts of in this manual will help them in performing their roles.

1.4.1. Content of the Manual. This manual contains a group of activities typically performed during functional courseware development and management.

1.4.1.1. Methods and Formats. When an activity can be accomplished in many effective ways, or when the outcome of an activity is likely to be impacted by rapidly changing technology, this manual does not specify methods or formats for accomplishing the outcome.

1.4.1.2. Reference to Additional Guidance. The AFMC ISD Courseware Resource SharePoint Site provides courseware forms, examples, and valuable working aids. It does not restrict experienced developers from incorporating creative ideas, quality improvements, and emerging technology into their processes and products. Experienced developers are encouraged to add new content to the AFMC ISD Courseware Resource Site to keep it updated. Assistance may also be provided by a Complex Courseware Program Manager (CPM).

1.4.2. Manual Structure.

1.4.2.1. Overview of AFMC Process. Chapter 2 contains an overview of the AFMC ISD model and AFMC Courseware Development and Management Process. Flow charts for each phase and function are located in Appendix 1.

1.4.2.2. Courseware Activity Chapters. Chapters 3-10 describes various activities that typically take place in phases and functions of the courseware process. Each activity includes these categories of information:

1.4.2.2.1. Purpose. Explains the reason an activity is performed.

1.4.2.2.2. Activity Outcomes. Defines typical deliverables and decisions that result from the activity.

1.4.2.2.3. Additional Guidance. Lists working aids (procedural guidance, examples, templates, worksheets, quality checklists, etc.) and references found in the AFMC ISD Courseware Resource Site or other sources.

1.4.2.2.4. Evaluation Criteria. Identifies elements and factors that should be considered when deciding if an activity has been accomplished well enough to move to the next process step.

1.4.2.3. AFMC ISD Release Points Checklist.

1.4.2.3.1. Courseware references.

1.4.2.3.2. Acronyms and terms associated with courseware.

1.4.2.3.3. Guidance on structure and content of courseware master files.

1.5. Roles and Responsibilities. While anyone in any organization may be assigned an activity in the Courseware Development and Management Process, responsibilities of such individuals can be categorized into a few basic roles.

1.5.1. Maintenance Training Manager. An Air Force, or equivalent, ISD Theory course can help prepare a training manager for duties relating to processing of training gaps, courseware/training/non-training issue worksheets, search for existing content, manage courseware documentation, etc. Offical responsibilities are contained in AFI 36-2650 AFMCSUP. As part of his/her responsibility, a training manager:

1.5.1.4. When training gaps are identified, conduct a Training Needs Assessment (TNA) and only recommend courseware be developed when it is determined (1) there is a valid instructional need; (2) the proposed instruction is an effective, cost-effeicent solution; and (3) existing material cannot meet a defined requirement.

Note: Education officers and employee development managers may perform the same role as training managers for education and employee development courseware.

1.5.2. Complex Courseware Program Manager (CPM). A Courseware Program Manager is responsible for applying ISD methodology and AFMC Courseware Development and Management Process for a functional area. The primary role is to serve as a liaison between their area of responsibility and higher Education and Training (E&T) offices on matters related to courseware. Offical responsibilities are contained in AFI 36-2650 AFMCSUP. An Air Force, or equivalent, ISD Theory course will prepare a CPM to effectively address these issues.

1.5.2.1. Conducts or assigns periodic evaluation of instructional courseware for which responsible and eliminates instruction unrelated to the mission/job.

1.5.2.2. Assigns course numbers IAW HQ AFMC/A4PT.

1.5.2.3. Oversees electronic and physical courseware master libraries.

1.5.2.4. Implements and tracks official changes to courseware master files and processes courseware Interim Change Memos (as needed).

1.5.2.5. Distributes courseware to authorized individuals for review or implementation.

1.5.2.6. Activates, archives, deactivates, and reactivates courseware upon request from authorized individuals.

1.5.2.7. Coordinates courseware signature page process as outlined in AFI 36-2650 AFMCSUP Maintenance Training.

1.5.2.8. Coordinates Subject Matter Expert (SME) support for command courseware development and periodic review projects.

1.5.3. Courseware Project Manager. A courseware project manager is responsible for monitoring progress of assigned courseware projects, assisting courseware development team in removing obstacles that threaten a timeline or budget of the project, and reporting project status to appropriate training offices and customer representatives. Courseware project managers oversee tasks in each ISD phase and function, where their qualifications enable them to identify and prioritize instructional requirements, and to intercede as necessary to ensure that they are met. An Air Force or equivalent Instructional System Development (ISD) Theory course will prepare a courseware project manager to perform these and other critical functions.

1.5.3.1. Duties. A courseware project manager demonstrates knowledge of the following:

1.5.3.1.1. The principles of learning, systems engineering, Project Management Body of Knowledge (PMBOK) and quality improvement that are the foundation of ISD.

1.5.3.1.2. Various instructional design models, methods, techniques, and media used in a blended learning approach to instruction.

1.5.3.1.3. Processes within Air Force and AFMC education, training, and employee development programs.

1.5.3.1.4. Reason for each activity in the courseware process.

1.5.3.1.5. Risks of not performing a particular activity in the courseware process.

1.5.3.1.6. Formulas and adjustment factors for computing assignment priority and rough order of magnitude of resources needed to accomplish courseware projects.

1.5.4. Courseware Developer. A courseware developer is responsible for creating relevant, engaging, and effective instruction in an efficient and low-risk manner. A developer performs most tasks in Planning, Analysis, Design, Development, and Implementation Phases. An Air Force or AFMC Instructional System Development Theory course will prepare a courseware developer to perform these tasks.

1.5.4.1. Qualifications. A courseware developer demonstrates knowledge of the following:

1.5.4.1.1. Principles of learning, systems engineering, Project Management Body of Knowledge (PMBOK) and quality improvement are the foundations of ISD.

1.5.4.1.2. Various instructional design models, methods, and techniques are used to develop instruction.

1.5.4.1.2.1. Courseware developer should attend a formal Air Force ISD Theory course, or equivalent, training prior to assignment to a development position.

1.5.4.1.3. Various methods, techniques, and media are used in a blended learning approach to instruction.

1.5.4.1.4. Purpose and value of each activity is part of the AFMC courseware process.

1.5.4.2. Ensures adequate planning activities are accomplished before a project enters a later phase in a Courseware Development and Management Process.

1.5.4.3. Sets reasonable suspense dates for courseware activities, realizing that development of instructional systems can be a relatively slow and complex process.

1.5.5. Instructor or Trainer. The instructor or trainer serves as an expert on effectiveness of instructional and evaluation activities/materials in meeting the intent of the course. An Air Force, or equivalent, ISD Theory course and Principles of Instruction (POI) will help to prepare the instructor. The trainer for these and other duties are part of his/her responsibilities as an instructor or trainer:

1.5.5.1. Assists in course or learning aid planning.

1.5.5.2. Provides input on course flow, design and content of instructional and evaluation materials.

1.5.5.3. Tests and provides feedback on effectiveness of presentation and learning activities.

1.5.5.4. Attends train-the-trainer events (as needed) to become qualified to teach new courses.

1.5.5.5. Assists in formal Re-Evaluation (periodic review) of courses.

1.5.5.6. Initiates Interim Changes and out-of-cycle reviews of courseware (as needed).

1.5.6. Subject Matter Expert (SME). The SME has a high level of knowledge and skill in a particular topic, task, system, or process. The primary role of a SME is to verify accuracy and completeness of course content. A SME:

1.5.6.1. Assists in development of course or learning aid.

1.5.6.2. Advises on specific tasks, performance steps and decision points to be covered, along with conditions and standards for acceptable performance.

1.5.6.3. Contributes to content identification of instructional and evaluation materials.

1.5.6.4. Verifies accuracy and completeness of information in instructional and evaluation materials.

1.5.6.5. Assists in formal Re-Evaluation (periodic review) of courseware related to their area of expertise to verify currency and accuracy of materials.

1.5.6.6. Participates in technical reviews, small group tryouts, and operational tryouts of courseware to advise on content and provide input on ways to add relevance to instruction (real-world examples, lessons learned, etc.).

1.5.6.7. Alerts instructors or appropriate training office about changes to work processes, directives, or systems when such changes may impact instruction.

1.5.7. Maintenance Supervisor. The maintenance supervisor is ultimately responsible for training, qualification, and/or certification of assigned personnel. Offical responsibilities are contained in AFI 36-2650 AFMCSUP. The maintenance supervisor:

1.5.7.1. Works with the appropriate Group Training Manager to identify work center training requirements based on regulatory guidance and work center tasks.

1.5.7.2. Supports Group Commander in providing SMEs and students as requested in support of courseware development and management activities.

1.5.7.3. Provides graduate assessment survey feedback as necessary to evaluate the relevancy and effectiveness of courses and learning aids.

1.5.8. ISD Evaluator. ISD Evaluators at each installation are functional experts on the application of ISD methodology and AFMC Courseware Development and Management Process. Only experienced courseware developers or courseware program/project managers with the ability to interpret courseware policy for their installation should be assigned this role. The ISD Evaluator reviews courseware activities performed by others to verify evaluation criteria in this manual have been met and the project complies with ISD process intent. Attending an approved ISD course will prepare selected personnel to perform this function.

Chapter 2

OVERVIEW OF AFMC INSTRUCTIONAL DEVELOPMENT PROCESS

2.1. Instructional System Development (ISD) Defined. ISD is a practical and flexible approach for developing and delivering instruction that promotes transfer of learning from an instructional setting to the workplace. It adapts a systems engineering process to provide a means for sound decision making and integrates art and technology with scientific principles of psychology, sociology, andragogy, and anthropology to ensure quality of instruction. ISD produces relevant, effective, and economical instructional systems.

2.2. Benefits of ISD. Although the ISD method may look cumbersome and costly, it has proven itself to be both efficient and inexpensive in the long run. ISD requires a systematic collection and careful analysis of data and encourages objectivity. ISD focuses on specific requirements and expectations of the customer to promote relevancy. Use of the ISD process results in learning objectives are based on job performance requirements and instruction is effective and economical to sustain.

2.3. History of ISD. ISD is not new. It has its roots in the 1950s. A partnership between the Army and Florida State University (FSU) to research better ways to develop instruction for adult learners advanced the concept. In the 1970s, the Department of Defense (DoD) commissioned faculty members from FSU to formally define a process for development and delivery of instruction for all DoD services. Those efforts resulted in the original ISD model.

2.4. ISD Model Evolution. Over time, advances in instructional technology added more conditions and variables to the original ISD model. A systems engineering process was adapted to control and manage an increasingly complex process. Next, the ISD process was made flexible and interactive with the introduction of phases that could be entered at any point depending on the specific situation. This Analysis, Design, Development, Implementation, Evaluation (ADDIE) model is the basis of nearly all instructional design methodologies in use today. Additionally, advances in learning theory and educational technology caused major changes in some ISD variables. Continual evaluation and quality improvement are embedded into all phases and functions of ISD process to facilitate sound decision-making based on research and analysis.

2.4.1. Most recently AFSC has adopted Art of the Possible (AoP) and used it in conjunction with ISD. It incorporates the basic AoP tenets of speed, safety, quality and cost effectiveness. AoP is the fundamental basis of how we operate across the entire AFSC. It is not what we do, it is HOW we do everything. AoP enables us to have a better understanding of our processes and identify the constraints that prevent us from operating better, faster, and cheaper. Refer to AFSCH 60-101, Art of the Possible Handbook. Refer to ISD Resource Site for guidance on calibrating the Gated Machine.

2.5. ISD Process Variations. Nearly all businesses, academic institutions, and government agencies that use the ISD process modify it somewhat to meet their specific needs. Some variations go by different names such as: Systems Approach to Training (SAT), Instructional Development (ID), or Instructional Systems Design & Development (ISDD). However, the systems engineering process and basic ADDIE model are still at the core of all of these customized methodologies.

2.6. Current Department of Defense (DoD) ISD. Below is an illustration of a DoD ISD model configured for Air Force and AFMC ISD Art of the Possible (AoP) processes. MIL- HDBK-29612-2A, Instructional Systems Development/Systems Approach to Training and Education, and AFH 36-2235 VOL. 9, Information for Designers of Instructional Systems- Application to Technical Training, provide additional information on the ISD model and methodology. This manual includes a brief overview to provide context for later chapters.

Figure 2.1. AFMC DoD ISD Model Example.

2.7. Quality Improvement (QI) Infrastructure. Phases and functions of the ISD model are embedded in, and held together by, an overarching QI process. This QI infrastructure permeates every activity and outcome of the ISD process. It continually generates refinements to an instructional system based on the evaluation of activity outcomes against defined performance requirements. If a product or service resulting from an activity does not meet specified standards, then quality controls in each phase of the ISD process enable management to quickly redirect the project to an appropriate step or activity so that needed remedies or requested improvements are accomplished with little threat to budget or schedule.

2.7.1. ISD System Functions. Management, support, administration, and delivery are basic functions that sustain QI infrastructure and core phases of the ISD process. Each of these functions involves numerous responsibilities and activities that ensure instructional systems are developed, delivered, and maintained in an effective manner.

2.7.1.1. Management. This function directs, monitors, and controls all activities associated with the instructional system throughout its life cycle. Various participants (e.g., supervisors, instructors, courseware developers, etc.) can perform management roles. Some basic management activities are:

2.7.1.1.1. Planning for instructional system activities.

2.7.1.1.2. Arranging resources such as skills, facilities, equipment, funding, etc. needed to accomplish planned activities in support of the instructional system.

2.7.1.1.3. Reporting on status of instructional system elements and activities.

2.7.1.2. Support. This function involves long-range and day-to-day tasks that are performed in order to implement and maintain an instructional system throughout its life cycle. Some basic support activities are:

2.7.1.2.1. Providing funding, facilities, manpower authorizations, and services in support of instructional activities.

2.7.1.2.2. Supplying equipment, skilled personnel, courseware, and instructional aids.

2.7.1.2.3. Maintaining facilities, equipment, skilled personnel, courseware, and instructional aids.

2.7.1.3. Administration. This function conducts day-to-day operations of the instructional system throughout its life cycle. Some basic administration activities are:

2.7.1.3.1. Preparing documentation such as instructional materials, course control documents, plans, and reports.

2.7.1.3.2. Maintaining records such as courseware master files, rosters, test scores, student completions, budgets, and contracts.

2.7.1.3.3. Providing staff support such as processing personnel actions and maintaining employee development programs.

2.7.1.3.4. Scheduling resources such as personnel, equipment, and facilities.

2.7.1.3.5. Implementing and maintaining automated information management systems and other tools to support instructional system activities.

2.7.1.3.6. Administering funds and contracts.

2.7.1.4. Delivery. This function provides instruction to students throughout its life cycle. Some basic delivery activities are:

2.7.1.4.1. Providing an infrastructure for distance learning including an on-line course management system, equipment, and facilities.

2.7.1.4.2. Supplying skilled personnel to instruct courses.

2.7.1.4.3. Maintaining current, effective, and appealing courseware.

2.7.1.5. Evaluation. This function is the bridge between QI infrastructure and system phases of the ISD process. It is a continual measurement of activity outcomes against defined performance requirements and is part of every step in the ISD process. The evaluation function gathers feedback to assess instructional system and student performance. Some basic evaluation activities include:

2.7.1.5.1. Formative evaluations include internal reviews, individual tryouts, small group tryouts, and ISD validations conducted during Analysis, Design and Development Phases.

2.7.1.5.2. Summative evaluations include operational tryouts during Implementation Phase.

2.7.1.5.3. Operational evaluations include periodic instructor/trainer review, subject matter expert/program manager review, graduate assessment surveys, and other types of reviews in the instructional system after implementation.

2.8. AFMC Functional Courseware Development and Management Process. AFMC employs DoD Instructional System Development (ISD) model as its standardized process for courseware development and management. The AFMC process includes Planning, Analysis, Design, Development, Implementation, Evaluation/Finalization and Periodic Review of course material, as well as Courseware Management, Acquisition and Contract Oversight Functions, as defined below. AFH 36-2235 VOL. 9, Information for Designers of Instructional Systems- Application to Technical Training, provides additional information on the ISD model:

2.8.1. Planning. Analyze performance deficiencies, define training requirements, decide on instructional strategies, identifies activities and deliverables of a proposed courseware project, and estimates resources needed to accomplish it. Note: AFH 36-2235 VOL. 9, Chapter 3, provides additional Air Force related information and guidance for completing the Planning function of the ISD process.

2.8.2. Analysis Phase. Further analyze requirements to determine precisely what should be included in instruction, types of learning involved, proficiency level of learning needed for the target audience, and any prerequisite or follow-on instruction required. Note: AFH 36-2235 VOL. 9, Chapter 4, provides additional Air Force related information and guidance for completing the Analysis Phase of the ISD process.

2.8.3. Design Phase. Create a blueprint of the course or learning aid that shows what instruction, evaluation, and media will be used in each module to meet defined requirements. Note: AFH 36-2235 VOL. 9, Chapter 5, provides additional Air Force related information and guidance for completing the Design Phase of the ISD process.

2.8.4. Development Phase. Develop, test, and revise instructional and evaluation materials and prepare the instructional system for full operational tryout in the field. Note: AFH 36-2235 VOL. 9, Chapter 6, provides additional Air Force related information and guidance for completing the Development Phase of the ISD process.

2.8.5. Implementation Phase. Conduct evaluation of an instructional system under field conditions, make final changes, posts official courseware master file in the appropriate installation functional courseware library, distribute courseware to authorized users, and personalize lesson plans so that a course or learning aid is fully operational and ready for use in the field. Note: AFH 36- 2235 VOL. 9, Chapter 6, provides additional Air Force related information and guidance for completing the Development Phase of the ISD process.

2.8.6. Evaluation/Finalization of course material. Consists of operational, internal and external evaluations conducted throughout the ISD process and is the process for closing out the project and distributing the courseware for instructional delivery.

2.8.7. Periodic Review of course material. Periodically review courseware to determine if courseware is still needed, identify if changes are needed and either revalidate instruction, initiate a revision project, or archive the courseware.

2.8.7.1. Periodic reviews also apply to standard maintenance of course material as a result of minor changes to processes. Examples include data refreshes, and reference updates.

2.9. AFMC Process Flow Charts. Process flow charts can be found in Attachment 2. Note: Keep in mind ISD is by no means a rigid or linear process, and individual projects include only those activities needed to ensure relevant, effective and economical instructional systems.

2.10. Applying AFMC Courseware Process. The AFMC Courseware and Development Process for functional training may at first appear intimidating, restrictive, and too time- consuming to be practical in the real world. Actually, it is quite flexible and serves more as a problem-solving and decision-making model than a mandatory list of activities.

2.10.1. Choosing Activities. The courseware project manager or other instructional design expert decides which activities will likely be needed in each phase, depending on constraints and requirements of the particular project.

2.10.2. Mandatory Steps and Procedures. Mandatory steps and procedures are clearly identified in activity chapters. Note: Courseware developers are free, however, to incorporate emerging technology and their own creative ideas, lessons learned, and quality improvements into the process and its products.

2.10.3. Entering and Re-Entering Process. Issues and projects enter the courseware process in the first step of Planning. After the second step of Planning, (Note: Refer to Chapter 10.1) a courseware project manager and courseware development team enter or re-enter a process after deciding on the next appropriate step for the situation.

2.10.4. Adapting Process. The steps in the AFMC courseware process are intended to remind participants of activities involved in developing and managing instructional systems in a variety of situations and are not supposed to be an obstacle course of rules that must be followed. Issues often arise that require quick and ingenious solutions that a courseware project manager and courseware developer may need to bypass, modify, switch the sequence, or add new steps of their own in response. If constraints of a particular situation prevent them from following the standard courseware process, they determine ways to mitigate risks, document their decisions, and implement the plan.

2.10.5. Flexible Sequence of Activities. While flow charts illustrate a courseware process in a linear manner, steps and phases should not be thought of as consecutive in nature. For example, one activity does not have to be completed before the next one is started and steps do not always have to be accomplished in a recommended sequence. Each project has its own natural flow of activities depending on each situation. As long as required activities are completed, and variation in sequence does not waste resources or add unacceptable risk, flexibility is encouraged.

2.10.6. Making Decisions Based on Data. The ISD approach is successful because decisions are based on careful analysis of data collected from customers and other expert sources rather than on assumptions. Skipping the Planning and Analysis steps out of convenience, or making decisions based on assumptions or incomplete data threatens the success of a project.

2.10.7. Evaluation and Quality Control. Evaluation and quality control measures have been built into the courseware process to ensure that instruction is relevant, economical, and effective. If an activity outcome does not meet evaluation criteria or does not provide enough information to make a sound decision, the ISD Evaluator or courseware project manager should not hesitate to request rework or to re-insert a project into a remedial activity to mitigate risk and ensure quality.

2.10.8. Documenting Decisions. Decisions to skip a step, modify requirements for an activity outcome, or redirect a project are documented for audit. Supporting documents/arguments should be provided in ISD Evaluator review notes.

2.10.9. Experience Counts. The less experience that participants have with developing and managing courseware, the closer they should follow the AFMC process in this manual. Not fully understanding scientific principles and theories underlying ISD and the reason for each activity in the courseware process has led many novices in a wrong direction, resulting in wasted resources and unnecessary, ineffective, or inefficient instruction.

2.11. Adapting AFMC Courseware Process to Education and Employee Development. The AFMC Functional Courseware and Development Process can be adapted easily to development and management of education and employee development instruction. Whenever an activity specifies “training,” simply substitute “education” or “employee development” as appropriate, and guidance will readily apply.

2.11.1. Planning Activities. Because functional training is the most complex type of instruction, some activities in Planning may not be necessary when developing education and employee development courseware.

2.11.2. Academic Accreditation. Accreditation guidance for developing education courseware for formal schools is not covered in this manual, but may be obtained from the academic institution where instruction will be delivered.

Chapter 3

PLANNING

3.1. Planning Defined. This is the foundation of a courseware process. During the Planning process, SMEs, functional area supervisors, subject area program managers, and automated system Officers of Primary Responsibility (OPRs) work together with courseware specialists to establish training concepts, decide on training strategies, analyze performance deficiencies, define training requirements, identify activities and deliverables of the proposed courseware project, and estimate resources needed to accomplish it. For additional information and guidance relating to the Planning process of the Air Force Instructional System Development (ISD) process, refer to AFH 36-2235 VOL. 9, Information for Designers of Instructional Systems-Application to Technical Training.

3.2. Reason for Separate Planning Process. Some courseware development is contracted out to professional instructional designers. Many of the activities which normally fall under the Analysis Phase of the ISD model need to be accomplished before writing contract requirement documents. A second reason for establishing a separate Planning process is to emphasize the importance of these often overlooked activities. Keep in mind that data collected and decisions made during Planning are later validated and adjusted (as needed) during the Analysis Phase, based on changes in the situation and deeper analyses of student demographics and content to be trained.

3.3. Receive Potential Training Issue. A customer or training specialist identifies a perceived training issue and submits it to the appropriate Maintenance Training Office level for action. This is a required step in the AFMC courseware process.

3.3.1. Purpose. This activity represents the start of a courseware process regardless of what other activity (e.g., courseware review, occupational training analysis, performance root cause analysis, etc.) triggers it.

3.3.2. Activity Outcome. Outcome is an issue submitted to the appropriate E&T office for action.

3.3.3. Additional Guidance. Sample tasking worksheets are at the AFMC Depot Maintenance Training Library (DMTL) SharePoint Site.

3.3.4. Evaluation Criteria. Is documentation of a training issue sufficient to determine:

3.3.4.1. Nature of perceived issue?

3.3.4.2. Priority of assignment?

3.3.4.3. Appropriate organization to handle tasking request?

3.3.5. Issues/actions: Are any issues/action items that emerged during this activity documented sufficiently to determine:

3.3.5.1. Nature of perceived issues?

3.3.5.2. Priority of assignment?

3.3.5.3. Appropriate organization to handle tasking request?

3.3.6. Follow-on: Is the decision to send this issue to another organization for action without accomplishing additional activities supported by regulation, policy, or sound logic?

3.4. Training Needs Assessment. In this activity, a training function decides how to act on a perceived training issue submitted by the customer or training specialist. This is a required step in the AFMC courseware process.

3.4.1. Purpose. This activity allows a training function to decide if a training request is best resolved through training and where to enter the courseware process based on how much is known about the training issue. It enables experienced participants to solve a problem efficiently without compromising effectiveness or relevance of the outcome.

3.4.2. Activity Outcomes:

3.4.2.1. Training Needs Assessment documentation.

3.4.2.2. Decision that either an issue is outside the scope of a courseware process, or that the courseware process will be used to address an education, training or employee development issue.

3.4.2.3. Decision on which organization should handle the issue at this point.

3.4.2.4. Decision on where to go next in the courseware process.

3.4.3. Additional Guidance. See AFMC Courseware Resource Site

3.4.4. Evaluation Criteria.

3.4.4.1. Is there enough information in the issue description to justify a determination that courseware is not a direct part of the solution to this problem?

3.4.4.2. Is a decision to send this issue to another organization for action without accomplishing additional activities supported by regulation, policy, or sound logic?

3.4.4.3. Is there existing training that will fulfill this requirement?

3.5. Transfer Issue to Appropriate Organization. In this activity, a training specialist forwards the issue to a more appropriate organization to handle remaining project activities.

3.5.1. Purpose. This activity allows a training function to forward an issue to an organization that is more responsible for subject matter or scope of the work involved in it.

3.5.2. Activity Outcome. Outcome is a message with attached issue description that is submitted to a more appropriate organization.

3.5.3. Additional Guidance. Sample tasking worksheets are at the AFMC Depot Maintenance Training Library (DMTL) SharePoint Site.

3.5.4. Evaluation Criteria. Does justification in the message explain:

3.5.4.1. Why an approached training function is not the most appropriate for tasking?

3.5.4.2. Why this falls into the jurisdiction of another organization?

3.6. Conduct an Occupational Analysis. In this activity, a training office identifies training needed to perform work of an occupation from entry level through advanced journeyman. It is performed in support of Civilian Career Field Education and Training Plan, Civilian Training Plan, Career Development Plan, Occupational Training Template development, etc. Military occupational training analysis is not addressed here because clear guidance already exists in AFI 36-2651, Air Force Training Program.

3.6.1. Purpose. This activity consolidates several other courseware activities to identify functional training needed for an occupation. Occupational training analysis helps supervisors and training specialists develop individual and occupational training plans for employees, and assists courseware developers by identifying training requirements and target populations.

3.6.2. Activity Outcomes:

3.6.2.1. Occupational Training Analysis Report.

3.6.2.2. Possible tasking worksheets on training issues, courseware issues, training gaps, or non-training issues submitted to appropriate organization for action.

3.6.2.3. Decision on where to go next in the courseware process.

3.6.3. Additional Guidance. Sample tasking worksheets are in the AFMC Depot Maintenance Training Library (DMTL) SharePoint Site.

3.6.4. Evaluation Criteria.

3.6.4.1. Does an Occupational Training Analysis Report include, at a minimum:

3.6.4.1.1. Knowledge/Task Analysis Report that meets criteria for that analysis activity?

3.6.4.1.2. Training Requirements Matrix that meets criteria for that activity?

3.6.4.1.3. Instructional System Analysis Report that meets criteria for that activity?

3.6.4.1.4. Training Standardization Assessment that meets criteria for that activity?

3.6.4.1.5. Completed tasking worksheets for any action items defined during an occupational training analysis?

3.6.4.2. Did data come from, or was it approved by, those in authority to define training requirements for the subject area?

3.6.4.3. Are any issues/action items that emerged during this activity documented sufficiently to determine:

3.6.4.3.1. Nature of perceived issues?

3.6.4.3.2. Priority of assignment?

3.6.4.3.3. Appropriate organization to handle tasking request?

3.6.5. Issues/actions: Are any issues/action items that emerged during this activity documented sufficiently to determine:

3.6.5.1. Nature of perceived issues?

3.6.5.2. Priority of assignment?

3.6.5.3. Appropriate organization to handle tasking request?

3.6.6. Is a decision to send this issue to another organization for action without accomplishing additional activities supported by regulation, policy, or sound logic?

3.7. Conduct Performance Root Cause Analysis. In this activity, a training office or customer organization examines a performance deficiency, analyzes underlying causes, and determines a solution. Include occupational and training requirements analysis.

3.7.1. Purpose. This activity identifies causes and effects of a performance deficiency and proposes holistic solutions for improving performance which may or may not include a training intervention. Performance root cause analysis considers not only a lack of skills or knowledge, but other factors that might contribute to the performance deficiency.

3.7.2. Activity Outcomes:

3.7.2.1. Performance Root Cause Analysis.

3.7.2.2. Decision on where to go next in the courseware process.

3.7.2.3. Possible tasking worksheets on training issues, courseware issues, training gaps, or non-training issues submitted to appropriate organization for action.

3.7.3. Additional Guidance. See the AFMC Courseware Resource Site.

3.7.4. Evaluation Criteria.

3.7.4.1. Were appropriate personnel involved in collection and analysis of data?

3.7.4.2. Was adequate data collected to support the analyses and recommendations?

3.7.4.3. Were appropriate analysis methods and statistical process control techniques used to evaluate and illustrate data?

3.7.4.4. Does the problem statement focus on what a deficiency is and not on why it exists? (Watch for “lack of” and “no” statements as they imply solutions.)

3.7.4.5. Does each cause statement clearly define a cause and effect of a performance deficiency so that anyone who becomes involved from this point forward can understand the reason for a proposed corrective actions?

3.7.4.6. Are plans included to evaluate the effectiveness of the intervention(s) after they have been implemented?

3.7.4.7. Is a proposed solution logical, feasible, and cost-effective? (Note: The average cost of one-hour training may exceed 1 million dollars.)

3.7.4.8. For each root cause identified:

3.7.4.8.1. Would a performance deficiency have occurred had the cause not been present?

3.7.4.8.2. Will correction or elimination of the cause prevent recurrence of a like condition?

3.7.4.9. Does a Performance Root Cause Analysis Report clearly:

3.7.4.9.1. Define performance deficiency?

3.7.4.9.2. Describe facts and root causes?

3.7.4.9.3. Identify pros and cons of possible corrective actions?

3.7.4.9.4. Provide recommendations on how to proceed?

3.7.4.9.5. Identify initial tasks for instruction if training is part of a performance deficiency’s root cause.

3.7.5. Issues/actions: Are any issues/action items that emerged during this activity documented sufficiently to determine:

3.7.5.1. Nature of perceived issues?

3.7.5.2. Priority of assignment?

3.7.5.3. Appropriate organization to handle tasking request?

3.8. Conduct Training Mandate Analysis. In this activity, a responsible office or customer organization documents training mandated and recommended (from initial awareness through refresher) for each target population in support of a particular work process, program, system, or subject area.

3.8.1. Purpose. This activity assists developers by identifying training mandates and target populations.

3.8.2. Activity Outcomes:

3.8.2.1. Training Mandate Analysis.

3.8.2.2. Possible tasking worksheets on training issues, courseware issues, training gaps, or non- training issues submitted to appropriate organization for action.

3.8.2.3. Decision on where to go next in the courseware process.

3.8.3. Additional Guidance. See the AFMC Courseware Resource Site.

3.8.4. Evaluation Criteria.

3.8.4.1. Does a Training request include, at a minimum:

3.8.4.1.1. Numbers and titles of directives, training plans, etc. which prescribe training requirement?

3.8.4.1.2. List of all existing or proposed instruction (including training gaps)?

3.8.4.1.3. Priority of assignment?

3.8.4.1.4. Appropriate organization to handle tasking request?

3.8.5. Issues/actions: Are any issues/action items that emerged during this activity documented sufficiently to determine:

3.8.5.1. Nature of perceived issues?

3.8.5.2. Priority of assignment?

3.8.5.3. Appropriate organization to handle tasking request?

3.8.5.4. Is a decision to send this courseware deficiency issue to another organization for action without accomplishing additional activities supported by regulation, policy, or sound logic?

3.9. Conduct Instructional Courseware Analysis. In this activity, a courseware project manager or courseware developer analyzes courses and learning aids within available resources for a particular work process, program, system, or subject area to meet an identified training requirement. It is highly recommended that a Training Mandate Analysis be accomplished before analyzing the full instructional system.

3.9.1. Purpose. This activity determines the availability/compatibility of existing training for a particular work process, program, system, or subject area. It is performed in order to identify gaps and duplication, and to decide if existing courses and learning aids within a system are effective in meeting the defined training requirements (from initial awareness through refresher) for all target populations.

3.9.2. Activity Outcomes:

3.9.2.1. Instructional Courseware Analysis.

3.9.2.2. Possible tasking worksheets on training issues, courseware issues, training gaps, or non-training issues submitted to appropriate organization for action.

3.9.2.3. Decision on where to go next in the courseware process.

3.9.3. Additional Guidance. AFMC Form 96 should be used to document research. See the AFMC Depot Maintenance Training Library (DMTL) SharePoint Site.

3.9.4. Evaluation Criteria.

3.9.4.1. Does the Instructional Courseware Analysis Report adequately describe the following:

3.9.4.1.1. Courses and learning aids that exist or are in work for the instructional system?

3.9.4.1.2. Any gaps in instruction that appear to exist, in terms of tasks that need to be taught and proficiency level of learning required?

3.9.4.1.3. Any duplication of instruction that appears to exist?

3.9.4.1.4. Any unnecessary instruction that appears to exist?

3.9.4.1.5. Any courseware deficiencies that appear to exist, in terms of tasks that need to be taught and proficiency level of learning required?

3.9.4.2. Did data come from an adequate mix of novices, SMEs, training managers, subject area program managers, system OPRs, instructors, trainers, courseware developers, and other customer representatives to ensure accurate and complete information on which to base findings and recommendations?

3.9.5. Issues/actions: Are any issues/action items that emerged during this activity documented sufficiently to determine:

3.9.5.1. Nature of perceived issues?

3.9.5.2. Priority of assignment?

3.9.5.3. Appropriate organization to handle tasking request?

3.10. Conduct Training Gap Analysis. In this activity, a courseware project manager or courseware developer works with a customer to better define and analyze a perceived gap in training and determine the nature and scope of a fix. This is accomplished by using various surveys to determine if an issue stems from a process, experience, attitudes, satisfaction, climate, management practices, equipment, facilities, budget, or a combination.

3.10.1. Purpose. This activity analyzes a perceived training gap or need identified by a customer or training specialist so that a solution can be planned.

3.10.2. Activity Outcomes:

3.10.2.1. Training Gap Analysis Report.

3.10.2.2. Possible tasking worksheets on training issues, courseware issues, training gaps, or non-training issues submitted to appropriate organization for action.

3…