Attachment 4 ATF SOP 6020.1 Update 8-15-11_Watermark.pdf

ATF SOP 6020.1A

Effective

August 2011

Standard Operating Procedure for

Curriculum Development and Management

Office of Human Resources and Professional Development

Assistant Director Human Resources and Professional Development

Date

ATF SOP 6020.1A

August 2011

Standard Operating Procedure for Curriculum Development and Management

FOREWORD

TO: All Office of Human Resources and Professional Development (HRPD)

Directorate Employees

1. PURPOSE. This standard operating procedure (SOP) defines the processes and procedures that will be utilized to create, revise, document, and authorize curriculum that will be implemented by the Office of Human Resources and Professional Development.

2. SCOPE. All employees and contractors of the Office of Human Resources and Professional Development will utilize the processes and protocols contained within this SOP. It will be the responsibility of all managers and supervisors within the directorate to ensure that these operating procedures are followed.

3. CANCELLATION. ATF Office of Training and Professional Development

Standard Operating Procedure for Curriculum Development and Management, November 1, 2008.

4. AUTHORIZATION. ATF training policies and procedures are consistent with the following laws and regulations:

a. The Government Employees Training Act (GETA) of 1958, Pub. L. No.

85-507, as amended, and codified into Chapter 41 of Title 5, United States Code, has been amended many times since 1958, to include the Federal Workforce Restructuring Act of 1994. This Act created the framework for agencies to plan, develop, establish, implement, evaluate, and fund training and development programs designed to improve quality and performance of the workforce.

b. Chapter 41 of Title 5, United States Code (5 U.S.C.), is dedicated to human resource issues and is organized into various chapter headings, with Chapter 41 addressing “Training” in the Federal service.

c. Executive Order 11348, providing for the further training of

Government employees (1967), provides agency heads and the U.S.

Office of Personnel Management with additional information on how GETA is to be carried out. The Order was amended by Executive Order 12107, relating to the Civil Service Commission and labor-management in the Federal service (1978). This order emphasizes the importance of using effective interagency training programs to meet common needs Government-wide and requires that employees be selected equitably for training.

d. Parts 410 and 412 of Title 5, Code of Federal Regulations (5 C.F.R.).

The C.F.R. is a codification of the general and permanent rules published in the Federal Register by the executive departments and agencies of the Federal Government. Part 410 of 5 C.F.R. represents the general and specific policies and requirements for training in Government agencies. Part 412 of 5 C.F.R. addresses developing executives, managers, and supervisors. Both parts were substantially amended in 1996.

e. Executive Order 13111 of January 12, 1999, Using Technology to

Improve Training Opportunities for Federal Government Employees, establishes the President’s Task Force on Federal Training Technology, directs Federal agencies to take certain steps to enhance employees’ training opportunities through the use of technology, and creates an Advisory Committee on Expanding Training Opportunities.

5. RELATED ORDERS.

a. ATF O 1170.1a, Delegation Order-Delegation of Authorities within the Bureau of Alcohol, Tobacco, Firearms and Explosives, dated 11/09/2010.

b. ATF O 6000.2, Training and Professional Development, dated

XX/XX/2011 (update in draft).

c. ATF O 6220.2A, Leadership Competencies and Executive Core

Qualifications (ECQs), dated 6/26/2008.

6. REFERENCES.

Federal Law Enforcement Training Accreditation Standards and Procedures, 2010 Edition

U.S. Department of Defense Handbook, Instructional Systems Development/Systems Approach to Training and Education (Part 2 of 4 Parts), MIL-HDBK-1379-2, 1997.

U.S. Department of Justice, Drug Enforcement Administration, Standing

Orders 401 – 407 [on Instructional Systems Development], 2004.

U.S. Department of State, Office of Training and Performance Support, Standard Operating Procedures, 2005.

U.S. Department of Energy, National Training Center, Standard Operating

Procedures 310, 2005.

Kirkpatrick, Donald L., Evaluating Training Programs; 2nd ed. Barrett- Koehler. San Francisco. 1998.

Bloom, B.S., Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York. 1956.

Anderson, L. W., & Krathwohl, D. R. (eds.), A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Longman, 2001.

Simpson, E. J., The Classification of Educational Objectives in the Psychomotor Domain. Washington, DC. 1972.

Dave, R.H., Developing and Writing Behavioral Objectives. 1975.

7. QUESTIONS.

Any questions regarding this standard operating procedure should be directed to the Chief, Learning Management Branch.

Standard Operating Procedures for Curriculum Development and Management

6020.1A (August 2011)

TABLE OF CONTENTS

I. INSTRUCTIONAL SYSTEMS DEVELOPMENT OVERVIEW

1.0 Authority

2.0 Instructional Systems Design (ISD)

3.0 HRPD ISD Phases Overview

3.1 Assessment Phase

3.2 Analysis Phase

3.3 Design Phase

3.4 Development Phase

3.5 Implementation Phase

3.6 Evaluation Phase

3.7 Revision Phase

4.0 ISD And Quality Improvement

5.0 Exemptions to the HRPD ISD Process

5.1 Bundled E-Learning Libraries

5.2 Commercial Software Training

5.3 Externally Controlled Mandatory Training

5.4 Emergency Training

5.5 Professional Associations and Industry Conferences

6.0 Training Development

II. ROLES AND RESPONSIBLITIES

1.0 Management

1.1 Branch Chief

1.2 Division Chief

1.3 Learning Management Branch (LMB) Chief

1.4 Learning Technologies Branch (LTB) Chief

1.5 HRPD Chief of Staff

1.6 Assistant Director (AD)

2.0 Curriculum Development Team

2.1 CDT Composition

2.2 Instructional Systems Specialist (Learning Management

Branch)

2.3 Training Administrative Staff

2.4 Instructors

2.5 Subject Matter Experts (SME)

2.6 Stakeholder Representatives

3.0 Review and Approval Responsibilities per Deliverable

3.1 Needs Assessment

3.2 Job-Task Analysis (JTA) Report

3.3 Training Specification Report (TSR)

3.4 Pilot Lesson Plans, Examinations, and Materials

3.5 Final/Revised Lesson Plans, Examinations, and Materials. . 35

3.6 E-Learning (on-line) Training Project Consideration Form. .. 36

3.7 E-Learning Design Specification

3.8 E-Learning Scripts

3.9 E-Learning Storyboards

3.10 E-Learning Final Course Content Review

III. GENERAL REQUIREMENTS FOR ATF CURRICULUM DEVELOPMENT

1.0 ATF Training Types

1.1 Training Types Descriptions

1.1.1 Mission Training

1.1.2 Continuing Education

1.1.3 Mandated Training

1.1.4 Program Support

2.0 Curriculum Development Documentation

2.1 Curriculum Development Reports

2.2 Document Management and Archiving

2.3 Curriculum Documentation Ownership

3.0 Use of Vendors/Contractors for Curriculum Development for Mission Training

3.1 Vendor/Contractor Standards

3.2 Copyright and Disclosure Clauses

3.3 Needs Assessment Objectivity

IV. ISD PROCESSES—NEEDS ASSESSMENTS

1.0 Authorization Of Needs Assessment

2.0 Conducting the Assessment

3.0 Needs Assessment Documentation

3.1 Needs Assessment Signature Page

3.2 Needs Assessment Report

3.2.1 Section 1: Description of the Perceived Need

3.2.2 Section 2: Problem and Impact Summary

3.2.3 Section 3: Training Request Recommendations

3.2.4 Section 4: Resource Requirements

3.2.5 Section 5: Supporting Documentation (Appendix) . 52

3.3 Needs Assessments for Mandated Training

4.0 Management Review of the Needs Assessment Report

4.1 Chain of Review

4.2 When Training is Recommended and Authorized

4.3 Review and Revision of Existing

V. ISD PROCESSES—ANALYSIS PHASE

1.0 Developing the Job Task Analysis Report

1.1 Job and Target Audience Analysis:

1.1.1 Defining Duties and Responsibilities

1.1.2 Job Competencies

1.2 Conducting the Job-Task Analysis

1.2.1 Task Statements

1.2.2 Selecting Tasks for Training

1.2.2.1 Criticality or Importance of the Task

1.2.2.2 Difficulty of the Task

1.2.2.3 Risk/Safety Issues Related to the Task

1.2.2.4 Frequency of the Task

1.2.3 Identifying Job Performance Requirements

1.2.4 Student Evaluation Strategies

1.2.5 Instructional Settings

1.2.5 Level III Performance Measurement

1.3 Validating the Analysis

1.4 Job-Task Analysis Report for Mission Training

1.4.1 Management Review for Mission Training

1.5 Job Task Analysis Report for Continuing Education (CEd) .. 69

1.5.1 JTA/Competency Report Contents for the CEd

Training Type

1.5.2 Management Review for Continuing Education

Training

1.6 Job-Task Analysis Report for Mandated Training

VI. ISD PROCESSES—DESIGN PHASE

1.0 Training Specification Report Overview

2.0 Developing Learning Objectives overview

2.1 Learning Objective Composition

3.0 Learning Objective Behaviors

3.1 Hierarchy of Learning – Cognitive Domain

3.1.1 Knowledge

3.1.2 Comprehension

3.1.3 Application

3.1.4 Analysis

3.1.5 Evaluation

3.1.6 Creation

3.2 Hierarchy of Learning Use in Course Development

3.3 Hierarchy of Learning Relationships

3.4 Physical Skill Levels of Learning—Psychomotor Domain

3.4.1 Perception

3.4.2 Imitation or Guided Response

3.4.3 Manipulation

3.4.4 Precision

3.4.5 Naturalization

3.4.6 Adaptation

3.4.7 Origination

3.5 Domain Job Aid

3.6 Behavior statements

4.0 Learning Objective Conditions

5.0 Learning Objective Standards

7.0 Clustering and Sequencing of Learning Objectives

7.1 Clustering LOs

7.2 Sequencing of LOs

7.3 Relationships in sequencing

8.0 Documenting Learning Objectives

9.0 Developing and Documenting Student Evaluations—Level II

Evaluation

9.1 Student Evaluation Requirements

9.2 Developing Performance Tests (Level II Evaluations)

9.2.1 Written Examinations

9.2.2 Practical Exercises

9.2.3 Setting the Pass/Fail Scores:

9.2.4 The Angoff Method of Determining Test Cut-off

Scores for Written Examinations:

9.2.5 Practical Exercise Cut-off Scores

9.3 Documenting Performance Tests

9.3.1 Student Assessment Plan and Objective-to-test Matrix

9.3.2 Student Remediation and Retesting

9.3.2.1 Remediation and Retesting Exceptions ... 102

10.0 Determining Instructional Strategy

11.0 Risk Analysis Specifications

12.0 Identifing Instructional Methodologies and Resources

12.1 Reporting Instructional Strategy, Methodologies, and

Resources

13.0 Budget Estimate

14.0 Training Specification Report Specifications

15.1 Management Review

VII. ISD PROCESSES—DEVELOPMENT PHASE

1.0 Development Phase Overview

2.0 General Development Requirements

2.1 Course Syllabuses

2.2 Lesson Plans and Instructional Materials

2.2.1 Sequencing of Lesson Modules

2.3 Instructional Standard Operating Procedures (ISOPs)

2.4 Development of instruction for a diverse workforce

2.4.1 Equal Opportunity Issues in Training Development

2.4.2 Section 508 Compliance

2.5 Instructional Staff to Student Ratios

2.6 Copyright Requirements

2.6.1 Fair Use of Copyright Protected Materials

2.6.2 Situations that Do Not Require Copyright Permissions

2.7 Equipment, Facility, and Safety Requirements

2.7.1 Training Safety

2.7.2 Equipment and Facilities

2.7.3 Environmental Safeguards

2.8 Student Evaluation Requirements

2.8.1 Minimum Requirements

2.8.2 Examination Remediation

2.8.3 Assessment/Evaluation Frequency

2.8.4 Student Evaluation Review

2.9 Instructor Qualifications and Role Player Documentation ... 122

3.0 ATF Lesson Plan documentation

3.1 Lesson Plan Format

3.1.1 Lesson Plan Overview Pages

3.1.2 Lesson Introduction

3.1.3 Lesson Body

3.1.4 Lesson Summary

3.1.5 Lesson Graded Assessment (if applicable)

3.1.6 Lesson Graded Assessment Debrief (if applicable).

4.0 Instructional Standard Operating Procedures

4.1 ISOP Content and Organization

4.2 ISOPs Approval

5.0 Piloting of New Curriculum

6.0 Lesson Plan Approval Package: Requirements (Pilot and Final) 128

6.1 Management Review

VIII. ISD PROCESS—EVALUATION PHASE

1.0 Evaluation Phase Overview

2.0 Level I Evaluation

2.1 Customer Satisfaction Surveys

2.2 Customer Satisfaction Survey Implementation

2.3 Review and Reporting

3.0 Level II Evaluations

3.1 Minimum Requirements

3.2 Test Banks

3.3 Lesson Plans

3.4 Securing Examinations

3.5 Exam Compromise

3.6 Test Administration

3.7 Student Challenges to Test Items

3.8 Test Item Analysis

3.9 Disposition of Examinations

3.10 Revising Examinations

4.0 Level III Evaluation

4.1 Roles and Responsibilities

4.1.1 Learning Management Branch (LMB)

4.1.2 Branch Chief (Training Office)

5.0 Level IV Evaluations

6.0 After-Action Reports

6.1 After-Action Report Form

7.0 Curriculum Review Conferences

IX. ISD PROCESS—CURRICULUM REVIEW AND REVISION

1.0 Curriculum Revision

2.0 Curriculum Review Conference

X. E-LEARNING PROJECT GUIDELINES

1.0 E-Learning at ATF

2.0 Introduction to E-Learning

2.1 Types of E-learning

2.2 Benefits of E-learning

2.3 Project Considerations

2.3.1 Criteria for Viable Projects

2.3.2 Cost Considerations

3.0 Roles and Responsibilities

3.1 Program Office

3.2 Curriculum Development Team (or applicable SMEs)

3.3 HRPD Training Program Office

3.4 E-Learning Project Manager

3.5 Chief, LTB

4.0 E-Learning Development Process

4.1 Decision Phase

4.1.1 Step 1 – Contact LTB

4.1.2 Step 2 – Assess project viability

4.1.3 Step 3 – Assign e-learning project manager

4.1.4 Step 4 – Research off-the-shelf courseware

availability (if applicable)

4.1.5 Step 5 – Identify SME to review courseware

4.1.6 Step 6 – Acquire or obtain temporary access to

courseware

4.1.7 Step 7 – Review off-the-shelf products identified to

determine scope of customizations needed

4.1.8 Step 8 – Gather or generate content

4.1.9 Step 9 – Determine development responsibility

4.1.10 Step 10 – Develop a Statement of Requirements. . 157

4.1.11 Step 11 – Review Statement of Requirements

4.1.12 Step 12 – Obtain proposal(s) from course

developer(s)

4.1.13 Step 13 – Review proposal(s)

4.1.14 Step 14 – Address funding shortfall (if applicable). 159

4.1.15 Step 15 – Finalize procurement

4.2 Design/Development Phase

4.2.1 Step 1 – Hold project kick-off meeting

4.2.2 Step 2 – Establish project plan

4.2.3 Step 3 – Make design decisions

4.2.4 Step 4 – Create the E-learning Design Specification.

4.2.5 Step 5 – Review and approve the E-learning Design

Specifications

4.2.6 Step 6 – Script and storyboard the course

4.2.7 Step 7 – Review and approve script and storyboards.

4.2.8 Step 8 – Program course

4.3 Testing/Implementation Phase

4.3.1 Step 1 – Create a test plan

4.3.2 Step 2 – Load course

4.3.3 Step 3 – Perform technical testing

4.3.4 Step 4 – Perform end user testing

4.3.5 Step 5 – Correct problems discovered during end

user testing

4.3.6 Step 6 – Final approval of course

4.3.7 Step 7 – Set up course on learnATF

4.3.8 Step 8 – Announce course availability

4.3.9 Step 9 – Conduct post-launch review

5.0 Rapid Development

6.0 Tips for the HRPD Program Office Training Administrative staff and the CDT

APPENDICES

Appendix 1 Curriculum Development Review Form Appendix 2 Pilot Lesson Plan Review Form Appendix 3 Sample Job-Task Analysis Page Appendix 4 Sample Task to Learning Objective Page Appendix 5 PowerPoint Development Guide Appendix 6 Acronym List Appendix 7 After-Action Report Form

I. INSTRUCTIONAL SYSTEMS DEVELOPMENT OVERVIEW

1.0 AUTHORITY

Human Resources and Professional Development (HRPD) holds primary responsibility for policy and administrative functions for Bureau human capital planning and management, staffing and recruitment, employee benefits, and other programs related to human resources. HRPD further supports ATF’s strategic plan and mission objectives through the administration of enterprise level training programs and functions that require centralized control. This includes the administration of the Bureau’s learning management system and the administration of training policies that affect the directorate and other ATF Federated Training Offices (FTOs), offices with designated training responsibilities.

HRPD ensures that enterprise basic, leadership, and mandated training programs are developed and administered in accordance with Department of Justice (DOJ) and ATF priorities, strategic goals, and directives; Executive Order; applicable laws and regulations; and industry standards. This Standard Operating Procedure (SOP) defines Bureau standards related to curriculum development and management.

Enterprise Training Programs. Include leadership development, new supervisor training, compliance training, occupational development training, e-learning programs, new employee training (special agent and industry operations investigator basic training), basic instructor training, and any programs directed by the executive staff or the Director to be managed by HRPD.

Enterprise Level Functions. Include but are not limited to the development and management of ATF training policy;

instructional design and documentation standards; administration of training technology delivery platforms and the DOJ learning management system; e-learning content development, integration, and testing; reporting of Bureau training activities;

conducting Bureau learning needs assessments and gap analyses; facilitating the administration of training resource allocations; instructor management; performance training analysis and development consulting; administration of accreditation through the Federal Law Enforcement Training Accreditation (FLETA) Board; and evaluation and measurement of Bureau training programs.

2.0 INSTRUCTIONAL SYSTEMS DESIGN (ISD)

HRPD utilizes a formal performance-based training system for assessing, analyzing, designing, developing, implementing, evaluating, and revising training. In the training industry, these processes as a whole are referred to as Instructional Systems Design (ISD). ISD increases the effectiveness and cost efficiency of education and training. The products of ISD include:

Instructional systems based on ATF’s missions and strategic goals with job performance requirements based on assessments, analyses, and competencies

Courses focused on instruction of relevant knowledge and skills

Graduates who have the necessary knowledge, skills and abilities (KSAs) to perform the mission or job

Instruction to known and consistently applied standards

Measurements of successful performance

Validated course content

A legal record of training assessment, analysis, development, implementation, evaluation, and revision processes.

3.0 HRPD ISD PHASES OVERVIEW

3.1 Assessment Phase

In this phase, HRPD or FTO program staff is assigned to assess the potential for training development in response to an identified performance issue or requirement and makes recommendations to senior management on whether to proceed. The need to develop training may arise from a formal request or as part of a training initiative to further workforce development. During the assessment, the assessment team will formally identify the problem or perceived need in employee performance in meeting a mission goal or responsibility. The assessment will include a thorough description of the target training audience, scope of the need, environment, liability issues, and other relevant issues necessary to making an informed decision as to the development of training. The needs assessment documentation could include a variety of forms of analyses (such as a skill gap assessment) that support the findings of the assessment. All supporting analyses will be provided along with the assessment documentation and recommendation. The assessment will determine root causes and make recommendations as to how to strategically best address the problem or need. The solutions identified in the assessment can focus on a number of tactics to improve employee performance which may or may not include training.

3.2 Analysis Phase

The analysis phase consists of a Job Task Analysis Report: a detailed examination of a variety of job and performance related specifications that will be documented and utilized to define training requirements. The project staff will identify job duties, responsibilities, and critical job tasks; identify skill gaps that need to be addressed; select appropriate tasks for training; identify job performance requirements and measures; select instructional settings; identify equipment and a number of other factors relevant to the training. Training is based on the tasks and performance requirements identified in the analysis.

3.3 Design Phase

The design phase is based on the results of the previous phases. The development team will further examine and refine the training requirements utilized to create the course. Some products produced by the development team during this phase include measurable training objectives based on the job tasks;

training evaluation instruments and requirements; requirements for instructional methodologies, strategies, and resources; future program performance evaluation instruments; and a review of existing training that might already meet requirements. The development team will document these results in a Training Specification Report that will serve as a blueprint for the actual development of the course. The report will also include an estimated budget for further development efforts.

3.4 Development Phase

Based on the design phase results, the development phase involves the creation of lesson plans, unit exercises, drills, case studies, and any instructional materials for both the student and the instructor. Media are produced or procured. If required, technology based components of training such as on-line or CDROM-based training are produced or procured. Special equipment is purchased, if necessary. Instructional materials are validated through subject matter expert, management, and stakeholder reviews, and courses are piloted to a sample audience and refined as required. The last step in this phase is the finalization, authorization, and archiving of all training materials and all the documentation from the previous phases.

3.5 Implementation Phase

In this phase, the instructional materials are deployed under operational conditions. The deployment of the training is closely evaluated and monitored in order to assure consistent, quality training and the appropriate use of resources and instructional staff. Training activities are documented via a variety of reporting mechanisms such as weekly reports, after-action reports, instructor evaluations, and through the ATF Learning Management System (LMS). Budgetary activities are documented through ATF’s financial management system.

3.6 Evaluation Phase

Evaluation is a continuous process that flows throughout the ISD model. Feedback from evaluation modifies the results of each step. To ensure continuing quality of the deployed training system and materials, customer satisfaction evaluations by students (Level I) and results from program evaluations that determine if skills are being utilized on the job (Level III) provide the periodic feedback required to keep training focused and relevant. Students are evaluated during and immediately following training (Level II) to measure student acquisition of knowledge and skills and identify possible gaps in instruction.

HRPD currently bases its curriculum evaluation efforts on the Kirkpatrick model, first presented by Donald Kirkpatrick in 1975, utilizing four levels of evaluation—the fourth level evaluating the affects of the training on the organization and other stakeholders.

ATF also utilizes other forms of program evaluation and reporting in order to monitor and assure a high standard of everyday operation.

3.7 Revision Phase

The revision phase is similar to the evaluation phase as it is ongoing throughout the life of the training. Customer satisfaction reports, instructor recommendations, and requests from stakeholders might require immediate curriculum updates in order to respond to a dynamic changing environment. However, HRPD also requires formal periodic reviews of mission training in order to ensure the needs of the stakeholders are still being addressed. Typically, the products of the evaluation phase are used, in addition to input from training administrators, instructors, and stakeholders. Curriculum review committees review all analysis, development, and design documentation to make sure they accurately reflect any new training needs. Lesson content and materials are thoroughly reviewed by the committee for accuracy, changes in field processes, and changes in technical content. In order to control course consistency, revision of course materials is monitored, documented, and must be formally authorized before utilization.

4.0 ISD AND QUALITY IMPROVEMENT

Quality Improvement (QI) underlies the ISD process. The objective of QI is to foster continuous improvement in both the ISD process and its products. The relationship between QI and ISD are provided below in the following underlying quality principles:

The customer defines quality: The customer identifies the performance requirements. Training organizations provide the instruction necessary to support the performance requirements. ISD emphasizes criterion-based instruction.

The criteria are directly linked to occupational performance requirements in the work environment. All evaluations are focused on the performance of the graduate and mission and goals of ATF.

Know your mission: All instruction is based on mission, job, or education requirements.

Know your target audience: The ISD process requires that the capabilities, attitudes, and aptitudes of the target audience be considered. By continuing to trace the relationship between the mission/job requirements and the knowledge and skill needed to do the mission or job, a continual focus on the performance requirement is maintained.

Set standards: Curriculum is developed to meet published agency, industry, and academic standards.

Manage by fact: The process requires constant evaluation and collection of both normative (opinions, reactions, etc.)

and quantitative data. Only through documentation of feedback can valid decisions be made regarding the effectiveness of training and any changes needed to improve the instructional system.

Foster teamwork: An instructional program cannot be developed in a vacuum. The design team must establish open lines of communication and be in constant contact with all the stakeholders involved.

Integrate quality in all phases: Quality is achieved through ongoing evaluation and acting on the results. Training content is validated utilizing subject matter experts (SMEs) from both within the training organization and from the work environment. Curriculum is kept current through regular curriculum review, content management efforts, and Kirkpatrick-type evaluations. ISD process is a cyclic process of constant improvement based on evaluation at all steps and phases of the process.

5.0 EXEMPTIONS TO THE HRPD ISD PROCESS

5.1 Bundled E-Learning Libraries

HRPD provides a variety of individual development training opportunities online. The online training library provides courses on a broad variety of topics that are relevant across the spectrum of the workforce. Typically, these libraries are purchased in bulk from commercial venders and placed on our learning management system so they are available to all ATF employees.

The selection of these libraries is based on a combination of cost, the relevance of the courses included, the quality of the courses, number of courses, and their compatibility with ATF’s learning management system.

These online courses are not purchased to address a specific training need but are provided as a general work resource and an opportunity for individuals to further meet their career goals.

As the libraries are bulk purchased, it is not reasonable or cost effective to require ISD documentation on each course within the library.

However, when an ATF specific online training solution is sought to address a specific training request within ATF, then all requirements as defined in this curriculum development SOP apply.

5.2 Commercial Software Training

Commercial Off-the-Shelf Software (COTS) consists of a variety of computer software applications (such as Microsoft Word) used across the Federal government. As vendor based training for COTS products is widely available and fairly standardized, it is not cost effective to mandate more than an initial needs assessment to justify the purchase of the training.

When a new COTS product is being introduced and vendor based training is readily available, it is not necessary to conduct a formal needs assessment as basic training in the new product is implicit in the purchase. In fact, it is often the case that vendor training is offered as part of the purchase of new software programs for the Bureau. Software venders include training as part of the purchase contract.

Management approval for training for COTS products is accounted for through the management of the directorate budget and the development and maintenance of the directorate Training Execution Plan.

5.3 Externally Controlled Mandatory Training

In some cases, mandatory training is defined, developed, and controlled by Federal entities external to ATF. External, mandatory training, which might include the completion of online or instructor-led training, is exempt from the ISD process unless the training is developed and managed by HRPD.

5.4 Emergency Training

Emergency training courses are those courses created quickly to meet the need of a mission critical issue that must be addressed immediately and cannot afford the time spent on the full ISD process. The necessary response time required, the risk and safety factors involved, and the mission criticality of the need are the primary factors in determining course development under emergency conditions. For example, the need to train special agents how to respond to a new, immediate, and direct terrorism threat in their area. Ideally, these courses would be based on other courses in order to provide training that has already been subject to ISD validation. It is recognized, though, that this will not always be possible when responding to truly new and unique situations.

However, poor planning or overzealous enthusiasm in responding to existing or emerging training needs does not qualify a course to be developed under emergency conditions.

Unless a direct immediate threat to employee or public safety can be shown, or a training response is needed to an urgent mission critical issue (such as the DC sniper shootings), then curriculum must be developed as defined in this SOP under normal conditions. The intent of this exemption is to provide ATF with the ability to quickly respond to direct threats. The exemption should be used judiciously and rarely.

Only the Assistant Director (AD) responsible for the program area has the authority to declare an emergency training development situation. Emergency training situations are to be documented in memo form from the applicable AD. Copies will be provided to both the program division responsible for the training and to the Learning Systems Management Division for archiving and the assigning of a course ID.

It is still necessary to develop documentation as to what training was provided (detailed training descriptions, lesson outlines, and syllabuses) and who developed the training. This documentation must be reviewed by the responsible division chief and provided to the Learning Management Branch (LMB) for review, archiving, and placing the default course in the learning management system. Policies and procedures regarding the implementation of training remain in full effect.

After the emergency training need is addressed, if it is determined that the training will still be relevant and continue to be utilized in the future, then the training will be subject to a full curriculum review and the responsible training program will develop the full ISD documentation that was not created due to the emergency. This documentation must be reviewed and authorized as per the normal ISD process for new training.

5.5 Professional Associations and Industry Conferences

As part of ATF’s commitment to provide opportunities for individual professional development and continuing education, the Bureau encourages participation in job related professional organizations and industry conferences. The intent of these opportunities is to provide for an informal means for ATF employees to be exposed to changes in the industry, to meet and network with members of other organizations, and to identify possible best practices that may be brought back to ATF. As such, this is not considered formal training by HRPD and is exempt from the curriculum development requirements of this SOP. However, this does not relieve the burden of justifying the conference expense to the appropriate ATF management.

6.0 TRAINING DEVELOPMENT

Bureau training must be coordinated to create the best overall strategy to address ATF’s mission goals and responsibilities. In addition, development of new curriculum consists of a major commitment of ATF resources. As such, the development of new curriculum is tightly controlled and must be monitored at the supervisory level. Branch chiefs, division chiefs, Deputy Assistant Directors (DADs), and ADs with Bureau training responsibilities are the only positions that may authorize needs assessments. ADs have the sole authority to authorize the development of new curriculum.

II. ROLES AND RESPONSIBLITIES

1.0 MANAGEMENT

HRPD and FTO supervisors and managers have specific review, authorization, and administrative responsibilities at various stages of the development process in order to assure quality, consistency, and accuracy of curriculum documentation and content, as well as to monitor and direct the efficient utilization of Bureau resources.

Review by supervision is essential to ensure that managers stay well informed and provides a formal opportunity to document and communicate concerns both up and down the chain-of-command.

Position specific management responsibilities in regards to curriculum development processes have been defined as follows:

1.1 Branch Chief:

Authorizing development of needs assessments and assigning to the appropriate personnel within their authority.

Providing first line of management review of documentation produced by curriculum development teams under their responsibility.

Reviewing all needs assessments; job-task-analysis and training specification reports; all lesson plans, materials, and Level II evaluation instruments for both pilot and final training, and preliminary Level III evaluation questionnaires developed by curriculum development teams under their supervision to ensure adherence to Bureau standards.

Managing curriculum development personnel, funding, and other resources at the branch level.

Identifying, coordinating use of, and authorizing subject matter experts (SMEs), stakeholders, instructors, and program and other training staff to participate on curriculum development teams.

Ensuring the coordination of efforts and resources with the Learning Technology Branch in the development of online or blended learning solutions (for instance, a blend of instructor-led and online training).

Coordinating program office content review of all lesson plans and materials before a final draft is forwarded to the division chief for review.

Ensuring that curriculum documentation is forwarded to the next appropriate level of review and authorization.

Assisting in the management and development of a Bureau training strategy that meets ATF mission goals and responsibilities.

Ensuring curriculum utilized by the branch is formally reviewed as per schedule or as needed.

Reviewing all documentation for the review and modification of curriculum.

Note: Branch chiefs do not have the authority to independently develop new training. Branch chiefs may authorize the development of need assessments; however, further training development can only be continued after the needs assessment has been reviewed by the chain of command and development authorized by the appropriate AD.

1.2 Division Chief:

Authorizing development of needs assessments and assigning to the appropriate personnel within their authority.

Reviewing needs assessments to determine if training development should be pursued.

Providing a second level of management review of all needs assessments; job-task-analysis and training specification reports; all lesson plans, materials, and Level II evaluation instruments for both pilot and final training, and preliminary Level III evaluation questionnaires developed during curriculum development.

Allocating and managing division resources for curriculum development processes.

Ensuring that curriculum documentation is forwarded to the next appropriate level of review and authorization.

Assisting the branch chief, as necessary, in identifying, coordinating use of, and authorizing subject matter experts (SMEs), stakeholders, instructors, program managers, and training staff to participate on curriculum development teams.

Participating in the management and development of a Bureau training strategy that meets ATF mission goals and responsibilities.

Ensuring curriculum within the division is formally reviewed as per schedule or as needed.

Reviewing documentation for the review and modification of curriculum.

Note: Division chiefs may delegate the above responsibilities to their deputy division chief with the exception of responsibilities associated with needs assessments.

1.3 Learning Management Branch (LMB) Chief:

All responsibilities previously designated to branch chiefs.

Reviewing HRPD needs assessments to determine if directorate standards and documentation requirements have been met.

Tracking curriculum development and curriculum review processes within HRPD.

Reviewing HRPD job-task-analysis and training specification reports; all lesson plans, instructional materials, and Level II evaluation instruments to ensure adherence to Bureau standards, Federal Law Enforcement Training Accreditation (FLETA) standards, or other applicable industry or academic requirements.

The LMB Chief will also review such documents for FTOs acquiring or maintaining FLETA accreditation.

Reviewing course review and modification documents.

Providing instructional design, FLETA standard and HRPD standard consultation services to development projects within HRPD.

Providing instructional design, FLETA standard and HRPD standard consultation services to development projects outside HRPD when staffing levels allow.

Reviewing curriculum for the use of sound instructional methodologies based on training goals, objectives, risk and liability, training parameters, and Bureau policy.

Ensuring curriculum documents address risk management/safety issues as per Bureau policy.

Reviewing all course documentation in regards to E- Learning to ensure adherence to Bureau standards and/or FLETA standards.

Assigning lesson plan identification numbers and archiving official Bureau training curriculum and development documentation.

Assigning course identification numbers and ensuring new courses are setup in the Bureau learning management system.

Entering new course Level III evaluation instruments into the Bureau survey deployment system.

Consolidating and reporting data collected from Level III training program evaluation reporting instruments.

Conducting and reporting on training program evaluations.

Tracking utilization of authorized curriculum within HRPD and maintaining curriculum version control.

Tracking basic and specialized instructor training requirements.

Note: If assigned training assessment, analysis, design, and development responsibilities, the LMB chief shares the same responsibilities as previously defined for all other branch chiefs.

The LSMD chief will assume responsibility for the review of curriculum documentation normally performed by the LMB chief.

1.4 Learning Technologies Branch (LTB) Chief:

All responsibilities previously designated to branch chiefs.

Providing first line of review and authorization of technology based training development projects (such as online courses, podcasts, computer based synchronous training, etc.).

Reviewing, authorizing, and ensuring adherence to Bureau curriculum development and documentation standards for all training technology focused Statements of Work, course project plans, design specifications, course storyboards, and matrices.

Recommending technology based courses for vendor development.

Assigning appropriate LTB personnel to manage and develop technology based training.

Ensuring the coordination of efforts and resources with other Bureau offices in the development of blended learning solutions (for instance, a blend of instructor-led and online training).

Managing curriculum development personnel, funding, and other resources at the branch level for technology based training.

Reviewing and authorizing vendor selection for technology based training projects.

Working with HRPD and FTO management to ensure the coordination of SMEs, stakeholders, instructors, and program managers, and training staff for the review of course content.

Reviewing and authorizing final technology based courseware.

Ensuring that curriculum documentation is forwarded to the next appropriate level of review and authorization.

Note: If assigned training assessment, analysis, design, and development responsibilities, the LTB chief shares the same responsibilities as previously defined for all other branch chiefs.

The LSMD chief will assume responsibility of the review of curriculum documentation normally performed by the LTB chief.

1.5 HRPD Chief of Staff

Coordinating content review and authorization by program offices for Special Agent Basic Training and Industry Operations Investigator Basic Training lesson plans.

1.6 Assistant Director (AD)

Identifying, delegating, and authorizing training needs assessments.

Reviewing needs assessment results.

Authorizing the development of new training.

Authorizing training type designations.

Developing, authorizing, and managing directorate training strategy to meet ATF mission goals and responsibilities.

Managing and authorizing the allocation of all training related resources within the directorate.

Reviewing reports by directorate management regarding curriculum development, implementation, evaluation, and revision activities.

Acting as a liaison to other Assistant Directors within the Bureau for training related activities.

Reporting training activities as per standard reporting procedures to the Director, ATF.

Note: The AD may delegate the above responsibilities to the

DAD.

2.0 CURRICULUM DEVELOPMENT TEAM

Curriculum development teams (CDTs) are responsible for the actual development of courses and all of the associated analysis and design documentation required by ATF policy and procedure. In electronic courses, they are responsible for developing all preliminary documentation and content that will serve as the foundation for courseware development; after which, they serve as subject matter experts to review content of developed courseware products. LTB project managers serve dual roles as both consultants in the early phases of curriculum development and as developers when curriculum is being developed and programmed.

2.1 CDT Composition

The CDT is comprised of individuals whose knowledge and skills are necessary in order to create a validated, instructionally sound, implementable, and well-documented curriculum.

Development teams must make use of subject matter experts in order to provide relevant information for the training analyses, identify and review critical tasks for training, define levels of performance, and provide detailed training content. In addition, the CDT can consist of instructional systems specialists (ISS), training staff, instructors, and other stakeholders to the training.

For Mission Training courses, it is strongly recommended that at least one SME on the development team come from field supervision, academia, or a topically related professional organization and not HRPD or the FTO program office. This action will provide a broader content perspective.

Instructional Systems Specialists (ISS) (GS-1750) are employees with specialized education in developing training programs. LMB maintains a small staff of ISSs that act as consultants and team members to curriculum development projects across the whole of ATF as staffing and priorities allow.

2.2 Instructional Systems Specialist (Learning Management Branch):

The responsibilities an ISS (LMB) include, but are not limited to, the following:

Providing expert advice on the assessment of training requests and the analysis, design, development, implementation, evaluation, and revision of curriculum.

Participating, as available, in developing curriculum documentation as defined by ATF policy. This includes, but is not limited to, needs assessments; job-task-analyses and training specification reports; all lesson plans, materials, Level II evaluation instruments; and preliminary Level III evaluation questionnaires.

Reviewing curriculum development documentation developed by the CDT to ensure that it meets ATF policy.

Participating in the validation of all curriculum development documents.

Determining and reviewing curriculum for appropriate instructional methodologies based on training goals, objectives, risk and liability, training parameters, and ATF policy.

Serving as liaison to the Learning Management Branch (LMB) and the Learning Technologies Branch (LTB).

Identifying curriculum risk management/safety issues.

Ensuring curriculum documents address risk management/safety issues as per ATF policy.

2.3 Training Administrative Staff:

Training administrative staff consists of individuals responsible for the coordination of training classes, instructor selection, and the administration of training content for a course or program.

During curriculum development projects, the responsibilities of the training administrative staff include, but are not limited to, the following:

Leading curriculum development projects as per ATF policy and standards.

Coordinating CDT resource needs.

Facilitating CDT workshops and projects.

Participating as a team SME, if qualified.

Participating as a training instructor, if qualified.

Developing curriculum documentation as defined by ATF policy. This includes, but is not limited to, needs assessments, job-task-analyses, training specification reports, all lesson plans, materials, Level II evaluation instruments, and preliminary Level III evaluation questionnaires. This also includes the documentation of team research activities in conjunction with the development of course documentation.

Reviewing curriculum and determining appropriate instructional methodologies based on training goals, objectives, risk and liability, training parameters, and ATF policy.

Reviewing curriculum documentation to ensure program resources will be able to meet the requirements of the course.

Communicating development team needs and activities to management.

Selecting appropriate development team members.

Identifying curriculum risk management/safety issues.

Developing curriculum solutions to identified risk management/safety issues.

Ensuring curriculum documents addresses risk management/safety issues.

Coordinating and implementing the pilot training.

Evaluating piloted training.

2.4 Instructors:

The responsibilities of instructors include, but are not limited to, the following:

Researching, developing, providing, writing, and revising subject matter content.

Developing documentation as defined by ATF policy.

This includes, but is not limited to, needs assessments, job-task-analyses, training specification reports, all lesson plans, materials, Level II evaluation instruments, and preliminary Level III evaluation questionnaires.

Reviewing curriculum documentation to make sure content is complete, accurate, and current.

Reviewing curriculum documentation to identify potential instructional obstacles based on past experience instructing similar topical areas.

Making recommendations regarding instructional strategies that might be utilized to meet the identified needs of the course.

Identifying target audience prerequisite knowledge, skills, and abilities.

Identifying curriculum risk management/safety issues.

Developing curriculum solutions to identified risk management/safety issues.

Ensuring curriculum documents addresses risk management/safety issues.

Instructing pilot course lesson plans.

Providing mentoring and support to instructors implementing the new course.

2.5 Subject Matter Experts (SME):

The responsibilities of SMEs include, but are not limited to, the following:

Researching, developing, providing, writing and revising subject matter content.

Developing curriculum documentation as defined by ATF policy. This includes, but is not limited to, needs assessments; job-task-analyses and training specification reports; all lesson plans, materials, and Level II evaluation instruments; and preliminary Level III evaluation questionnaires.

Reviewing curriculum documentation to make sure content is complete, accurate, and current.

Identifying target and prerequisite knowledge and skills.

Identifying curriculum risk management/safety issues.

Developing curriculum solutions to identified risk management/safety issues.

Ensuring curriculum documents addresses risk management/safety issues.

2.6 Stakeholder Representatives:

The responsibilities of stakeholder representatives include, but are not limited to, the following:

Identifying problem scope, relationship to ATF mission and strategic plans, measurable performance requirements, target audience, and environmental parameters.

Identifying target audience needed and existing knowledge, skills, and abilities in relation to the training project.

Representing program or field office interests in curriculum development activities.

Communicating curriculum development activities to the represented program or field office.

Reviewing curriculum documentation to make sure content is complete, accurate, current, and meets stakeholder needs. This might include, but is not limited to, review of needs assessments, needs analysis, job-task analyses, training specification reports, all lesson plans and materials, and courseware.

In conjunction with the training administrative staff, identifying and coordinating SMEs for use to develop curriculum documentation and to serve as instructors if no…