Attachment 1 Vendor Presentation 2.pdf

8/30/2011

ISD Training Vendor Proposal BriefingVendor Proposal Briefing

James Scott ChiefChief

Learning Management Branch

Office of Human Resources and Professional Development

ISD Training/Vendor Presentation

Background

Agenda

Student Audience Description

ATF ADDIE Model

Deliverables

Qualifications, Assumptions and ConstraintsConstraints

Review of Submitted Questions

Presentation Questions

ISD Training/Vendor Presentation

Background

Bureau of Alcohol Tobacco Firearms andBureau of Alcohol, Tobacco, Firearms and Explosives

(ATF)

Firearms licensing and sales -- Regulatory

Explosives licensing -- Regulatory

Explosives storage -- Regulatory Explosives storage -- Regulatory

Illegal use and trafficking of firearms -- Enforcement

Illegal use and storage of explosives -- Enforcement

Acts of arson and bombings -- Enforcement

Illegal diversion of alcohol and tobacco products -- Enforcement

ISD Training/Vendor Presentation

Background

Bureau of Alcohol Tobacco Firearms andBureau of Alcohol, Tobacco, Firearms and Explosives

(ATF)

ISD Training/Vendor Presentation

Background

Types of TrainingTypes of Training

Classroom

Laboratories

Firearms Ranges

T ti l RTactical Ranges

Simulated Environments

ISD Training/Vendor Presentation

Background

Types of TrainingTypes of Training

ATF uses the terms high-risk, high liability training to apply to both the occupational environment and the instructional environment when training content targets job duties and tasks that:

If fulfilled incorrectly could result in serious injury or death, Si ifi t li bilit fi i l l t th B Significant liability or financial loss to the Bureau, Or failure of primary mission responsibilities.

ISD Training/Vendor Presentation

Background

Types of TrainingTypes of Training

The training environment for these types of skills are often considered high-risk, high liability in that the instructional methodology creates an environment where serious injuries or death may be incurred during the training if the class is mismanaged (i.e., training that uses firearms and explosives ranges, simulated tactical environments, etc.).

Decisions related to the selection and delivery of content for training may needDecisions related to the selection and delivery of content for training may need to be defended in a court of law. Legally defensible curriculum documents the subject matter experts used, decisions made during training analysis and design, and provides for detailed lesson plans to ensure correct and safe delivery of training content and exercises.

ISD Training/Vendor Presentation

Background

Examples of ATF TrainingExamples of ATF Training

Basic special agent

Basic industry operations investigator

Post-blast Investigation

Explosives and accelerant detection canine handler

Firearms trafficking

Undercover investigations

Tactical operations

New supervisor training

Arson investigation

ISD Training/Vendor Presentation

Background

Change in ATF Training Policy and MethodsChange in ATF Training Policy and Methods

Update of all Office of Training and Professional Development standard operating procedures to meet Federal Law Enforcement Training Accreditation standards

Integration of the ADDIE Instructional Systems Design model into directorate standard operating procedures

Instructional Systems Specialists GS-series utilized in ATF for the fi ifirst time

Established the Learning Management Branch to manage training policy, provide consulting to offices developing curriculum, manage FLETA accreditation, provide instructor training, conduct program evaluations, and assist in updating existing mission critical training programs to comply with new standards.

ISD Training/Vendor Presentation

Background

Reorganization of Training OfficeReorganization of Training Office

Change from a centralized training administration model to a federated training model.

Advanced training for Special Agents and Industry Operations Investigators transferred from the training directorate to ATF field and program offices.

Basic training programs retained in the Office of Training and P f i l D l ll f i iProfessional Development, as well as, management of training policy

Human Resources Division combined with the training office to form the Office of Human Resources and Professional Development

ISD Training/Vendor Presentation

Background

Office of Training and Professional

Development

Basic Training

Advanced Training

LeadershipLeadership Training

Continuing Education

Online Training

ISD Training/Vendor Presentation

Background

Advanced Training

Office of Training and Professional

Development

Basic Training

Advanced Training

LeadershipLeadership Training

Continuing Education

Online Training

ISD Training/Vendor Presentation

Background

Special Operations

Advanced Training

Firearms and Explosives Industry

Division

Arson and Explosives

Enforcement Branch p p Division

Violent Crime Intelligence Division

Division

Other

ISD Training/Vendor Presentation

Student Audience Description

Subject matter experts in a diverse collection of fields.

Many with little or no formal training in the delivery of training.

Most with little or no formal training in the formal development of curriculum.

From a number of offices and different chains-of-commands.

May be a collateral duty. Primary duties dedicated to mission responsibilities.responsibilities.

Many will not understand the necessity or benefit of a formal curriculum development and documentation process.

ISD Training/Vendor Presentation

ATF ADDIE Model

AssessmentAssessment

Training staff identifies a problem or perceived need and determines how it affects ATF’s ability to meet strategic mission, goals, or administrative responsibilities.

For mandatory training, staff identify the Federal law or regulation, Executive Order, or DOJ/ATF directive that must be addressed and the required training parameters.

Training staff develops a Needs Assessment Report that includes:

• Relevant information about the assessment team and methodologies,

• A description of the target training audience,

• Defines the scope of the need,

• Identifies any liability and safety issues, and

• Documents any other relevant items necessary for management to make an informed decision regarding the development or procurement of training.

ISD Training/Vendor Presentation

ATF ADDIE Model

AssessmentAssessment

The assessment documentation can include results from a variety of analyses (such as a gap analysis) that support the findings of the assessment.

When applicable, the assessment may include a root cause analysis of performance gaps and make recommendations as to how to strategically address an issue.

The solutions identified can include a single tactic or a combination of tactics to resolve the identified performance requirement, some of which may not be training-related.

When training is recommended, an initial budget estimate (rough order of magnitude) for the development and delivery of training.

ISD Training/Vendor Presentation

ATF ADDIE Model

AnalysisAnalysis

Curriculum development team (CDT) develops an analysis report that documents job- and performance-related specifications used to define training requirements. The team deliverable is a job-task analysis report that incorporates information necessary for the design of targeted training such as:

A listing of all job duties and responsibilities targeted for training. In some cases, competencies also may be identified.

A listing of critical tasks and subtasks necessary to implement each duty A listing of critical tasks and subtasks necessary to implement each duty area with the tasks. Tasks must be defined in sufficient detail to derive training objectives in the next phase of development.

A listing of any applicable Federal law or regulation, Executive Order, DOJ/ATF directive, standard operating procedure, manual, or other material related to the job function and tasks being addressed.

ISD Training/Vendor Presentation

ATF ADDIE Model

AnalysisAnalysis

Job-task analysis report output continued:

Identification of equipment, on-the-job references materials, forms, specialized software, etc. necessary to complete the tasks.

Identification of any special conditions under which tasks must be performed.

An assessment of task frequency, risk, difficulty, and criticality.

Typically provided in two parts:

1. A cover report, and

2. A spreadsheet with all of the detailed data.

ISD Training/Vendor Presentation

ATF ADDIE Model

DesignDesign

Design documents build off of tasks and performance requirements identified in the job-task analysis and matches the requirements to instructional methodologies, evaluation requirements, and facility and equipment requirements. This phase produces Training Specification Report which includes information necessary for the development of the training, such as:

Measurable training objectives based on actual job tasks.

Training objectives sequenced to facilitate student progress from one level Training objectives sequenced to facilitate student progress from one level of skill and knowledge to another.

Written evaluation or graded practical exercise instruments linked to corresponding lesson objectives.

Identified instructional strategies and methods.

Identified training equipment and resource needs.

Estimated budgetary requirements for the training program.

ISD Training/Vendor Presentation

ATF ADDIE Model

DesignDesign

The Training Specification Report consists of a cover report and a spreadsheet that documents all of the pertinent data. The spreadsheet is a continuation of the one started for the job task analysis.

ISD Training/Vendor Presentation

ATF ADDIE Model

DevelopmentDevelopment

The development phase involves the creation of lesson plans, practical exercises, drills, case studies, and any other instructional materials needed by either the student or the instructor. This phase also may include the procurement of videos, equipment, reference materials, or other items needed for the training. If required, technology-based components of training such as online courseware or electronic job aids are produced or procured. Instructional materials are validated through subject matter expert, stakeholder, and management reviews Courses are piloted using a sample of the targetmanagement reviews. Courses are piloted using a sample of the target population and refined as required. The last step in this phase is the finalization, authorization, and archiving of all training materials and documentation.

ISD Training/Vendor Presentation

ATF ADDIE Model

DevelopmentDevelopment

Lesson Plans have a particular format. Some components of a lesson plan include a:

Lesson information cover page(s) which provide information about the learning objectives, setup requirements, needed equipment and supplies, etc.

Lesson introduction which includes a review of all learning objectives, student assessment requirements, and safety protocols (when li bl )applicable).

Lesson body giving detailed information regarding presentation delivery instructions, core content, scripts for electronic presentations, practical exercise instructions, etc.

Lesson summary that reviews the learning objectives and any items of particular difficulty for the student during the instruction.

Student assessment and debrief, when applicable (which may be formative or summative).

ISD Training/Vendor Presentation

ATF ADDIE Model

Implementation and EvaluationImplementation and Evaluation

The remaining phases of Implementation and Evaluation (to include revision) are included in the ATF ISD model. However with the exception of elements of evaluation, the two phases are not covered in the training to be contracted.

Implementation – Delivery of the training content as per design Evaluation – Using Kirkpatrick’s model for training evaluation, ATF provides for level 1, 2, 3 evaluations.

Development of level 2 evaluation (student assessments) instruments is included in the design phase and to be trained. This includes the development of written exams, graded practical exercises, and the establishing of cut-off (pass/fail) scores based on the Angoff method.

ISD Training/Vendor Presentation

ATF ADDIE Model

Output From PhasesOutput From Phases

The final reports and related documents are part of the official record for the training program and must be maintained. The documents from each ISD phase (Assessment, Analysis, Design, Development, and Revision) require formal review and authorization.

Needs Assessment Reportp Job-Task Analysis Report (and spreadsheet) Training Specification Report (and spreadsheet) Pilot Lesson Plans, Examinations, and Materials Final Lesson Plans, Examinations, and Materials

ISD Training/Vendor Presentation

Deliverables

Action Plan Action Plan Kickoff Meeting to review action plan, delivery schedule, ATF curriculum development model, and ATF templates Draft lesson plans for two courses (3 weeks prior to delivery, at minimum):

1. Assessment and Analysis

2. Design and Development

Delivery of 6 classes, 3 for each course:

ATF Headquarters Washington DCATF Headquarters, Washington, DC ATF National Academy, Glynco, GA

ISD Training/Vendor Presentation

Deliverables

ATF Furnished ResourcesATF Furnished Resources

ATF will provide the standard operating policy manual for curriculum development and current forms and spreadsheets used by ATF during the phases being addressed. Learning Management Branch staff will be available for consultation to clarify how the procedures and other materials are used to develop training documentation.

ATF staff will not be available for the development of work products required by the contractor related to this agreement.

ISD Training/Vendor Presentation

Qualifications, Assumptions and

C t i tConstraints The Contractor will:

Have expertise employing, documenting, and instructing the ADDIE model for use in developing high risk, high liability, legally defensible training.

Have experience developing high risk, high liability, legally p p g g , g y, g y defensible training using the ADDIE model.

Emphasize a technical training model and not an education model.

Integrate ATF processes and templates into each of the specified courses.

ISD Training/Vendor Presentation

Qualifications, Assumptions and

Constraints

The provided curriculum development standard operating procedure, forms, and templates remain government property and is not for public dissemination in whole or in part without the express written consent from ATF managementthe express written consent from ATF management.

ISD Training/Vendor Presentation

Submitted Questions

What do you see the level of effort in terms of teaching these y g courses?

If the vendor already has existing lesson plans, the biggest level of effort will likely involve becoming familiar with ATF curriculum development policy and including those things that may be unique to our organization in existing lessons. The policy follows a fairly basic ADDIE model, but we have some formal reports that we use to document project progress and allow for various levels of supervisory review. For instance, we make some early estimates on budget and resources in both the assessment and analysis phase for planningbudget and resources in both the assessment and analysis phase for planning purposes and during the design phase, and we have a report called the Training Specification Report that links identified job task and field requirements to learning objectives, student examination instruments, and other design related information. If you do not already have existing lesson plans, however, the level of effort will be much greater and will be dependent on the resources the vendor has available for development.

ISD Training/Vendor Presentation

Submitted Questions How often do you envision these courses to be taught, beyond the timeframe stated in the Statement of Workbeyond the timeframe stated in the Statement of Work

(SOW)?

The solicitation should be sufficient to cover our current and most immediate business need. However, the training is targeted to existing individuals who have recently been assigned new responsibilities regarding the development of training and does not account for the reassignment of individuals and having to training new staff. As such, it may be the case that additional classes may be necessary beyond the timeframe stated. However, such additional work cannot y y , be guaranteed.

ISD Training/Vendor Presentation

Submitted Questions Historically, how many of these classes has the Human Resources and Professional Development's (HRPDs) LearningResources and Professional Development s (HRPDs) Learning Management Branch (LMB) taught?

While ATF has historically taught thousands of law enforcement classes, the LMB has not ever provided staff training specifically on the ADDIE model beyond brownbag lunch seminars or briefings. In the distant past, some classes have been provided through a vendor on adult learning principals and curriculum development, but the training was not specific to ADDIE or a law enforcement environment. While LMB staff has the technical competencies to develop and deliver such training, the branch lacks sufficient resources to support the effort given existing mission responsibilities. Use of the ADDIE model is new to the Bureau and has only recently been applied within the last few years. As a result, beyond the small staff in LMB, most training staff is fairly unfamiliar with the model unless they have served in the military and have been trained to develop instruction for that environment.

ISD Training/Vendor Presentation

Submitted Questions There are many types of analyses (learner analysis, context analysis gap analysis and task analysis) Do you envisionanalysis, gap analysis and task analysis). Do you envision for all these to be part of the instructional effort in the first 2 phases of the course Assessment and Analysis course?

The primary focus for the training should focus on conducting task analyses, followed by conducting a gap analyses. The gap analysis relates to our needs assessment phase and the task analysis relates to our analysis phase. It would not hurt to touch on the other two as they can obviously inform the development decisions Due to time limitations however the most time will need to be spentdecisions. Due to time limitations, however, the most time will need to be spent on the task analysis.

ISD Training/Vendor Presentation

Submitted Questions Will you email or send a copy of the ATFs Curriculum Development Standard Operating Procedures (SOPs)?Development Standard Operating Procedures (SOPs)?

Yes, a copy of our curriculum development SOPs will be available. The copies will be made available electronically and can be emailed.

ISD Training/Vendor Presentation

Submitted Questions What other existing documentation is available, in addition to ATFs Curriculum Development SOPs and specificto ATFs Curriculum Development SOPs and specific examples, templates and exercises?

There is a general template (defined content requirements) for our cover reports for Needs Assessments, Job-Task Analyses (JTA), and Training Specification Reports (TSR). ATF has developed a spreadsheet template to capture most of the primary data for the JTA and TSR. Lesson Plans have template cover pages that contain important information about a particular lesson such as: learning objectives, instructional strategies, equipment requirements, materials, safetyobjectives, instructional strategies, equipment requirements, materials, safety requirements, facility requirements, and instructor to student ratios. Lesson plan content follows a particular format. Examples can be provided. With the exception of the spreadsheet, requirements for all of the other described documentation are defined in the curriculum development SOP.

ISD Training/Vendor Presentation

Submitted Questions What do you qualify as "legally defensible… based curriculum and high risk high liability training? Does "highcurriculum and high risk, high liability training? Does high risk, high liability training pertain to the occupational hazards of the Target Audience (TAs)?

High-Risk, High-Liability Training Content: When training content targets job duties and tasks that if fulfilled incorrectly could result in serious injury or death, significant liability or financial loss to the Bureau, or failure of primary mission responsibilities, it is considered high-risk, high liability. Typically, the training environment for these types of skills are often considered high-risk, high liability as well in that the instructional methodology creates an environment where serious injuries or death may be incurred during the training if the class is mismanaged (i e training that uses firearms and explosives ranges simulated tacticalis mismanaged (i.e., training that uses firearms and explosives ranges, simulated tactical environments, etc.).

Due to the nature of high-risk, high-liability training content, decisions related to the selection and delivery of content for training may need to be defended in a court of law.

Legally defensible curriculum documents the subject matter experts used, decisions made during training analysis and design, and provides for detailed lesson plans to ensure correct and safe delivery of training content and exercises.

ISD Training/Vendor Presentation

Submitted Questions Please clarify your comment that the courses "must use ADDIE as technical/performance based training model NOTADDIE as technical/performance based training model NOT topic/subject based education model."

Often in subject based education, the subject matter/topic is the primary driving force behind content delivery decisions. The development of the training is not student centric nor does it necessarily relate directly relate to established job based requirements for an organization. As such, the training may not be sufficiently targeted to organizational needs in regards to the selection of content, required levels of student performance, or the identification of training resource requirements. Performance requirements for students tend to be norm based. This is NOT the type of training model that is being pursued in this solicitationsolicitation.

The desired model is Technical/performance based. This type of training model uses organizational needs and specific job duties and responsibilities to define training content and performance requirements. Job-task analyses, gap analyses, competency analyses, etc., related to specific organizational needs and requirements drive decisions for training content, performance expectations, and resource needs. Student performance requirements are criterion referenced and based on subject matter expert review and on-the-job needs.

ISD Training/Vendor Presentation

Submitted Questions Section 1.3 Audience paragraph reads: "Each course must stand on its own and cannot assume course work in one course can carry over to theown and cannot assume course work in one course can carry over to the other as it cannot be guaranteed that the same students will be able to attend both classes." How do you envision students to become proficient in utilizing the ADDIE model to develop training if they attend only one course that covers part of the ADDIE process?"

The requirements stems from the reorganization of the ATF training office from a centralized unit to a federated model in which different parts of the mission organization now have new training responsibilities. While HRPD maintains centralized responsibility for core basic training programs and training policy and procedure, staff with advanced and field training responsibilities no longer report to this directorate. As a result, control over student selection for classes other than those provided at the ATF National Academy in GA will be limited. It is most likely that many of the same students will attend both classes, but not guaranteed. As a result, training strategies that include the carryover of projects or reading assignments from one class to the next should be avoided. The course on design and development may need to include a very brief review of topics covered in assessment and analysis with handouts or texts made available to the students to review on their own time.

Staff from the Learning Management Branch will provide some limited consulting services after the training is provided to those who miss one or the other class.

ISD Training/Vendor Presentation

Submitted Questions When do you anticipate awarding this contract?

There will be a fairly rapid turnaround on the selection of a vendor for this solicitation. A selection will be made by the review panel prior to September 14th.

ISD Training/Vendor Presentation

Submitted Questions What is the page length of the proposal?

There are no limitations set on the page length for proposals.

ISD Training/Vendor Presentation

Submitted Questions What is the page length of the proposal?

There are no limitations set on the page length for proposals.

What is the page length of past performance?

No more than three pages for past performance descriptions.

What is the year of currency of past performance, i.e., 3 years f d f hi li i i ?from date of this solicitation?

Do not go back farther than Sept 2008.

ISD Training/Vendor Presentation

Submitted Questions What is the submission process: email or UPS?

Submissions are to be submitted electronically to gustav.jakowitsch@atf.gov by the close of business Sept 6, 2011.

Should resumes of key personnel and letters of support be included in the proposal?

Yes, your staff will be evaluated on their qualifications.

Each training course will be delivered 3 times for a total of 6Each training course will be delivered 3 times, for a total of 6 training courses. Will the 6 training courses be offered in one year? What is the expected timeline?

Yes, the training courses will be delivered within a one year timeframe. It is expected that the first course will be delivered within the first 3 months of the contract. The final course delivered by the end of July.

ISD Training/Vendor Presentation

Submitted Questions Is the contractor expected to pay for any fees associated with issuing CEUs to training participants?issuing CEUs to training participants?

Yes. Any fees associated with CEUs should be incorporated into the proposal.

Do you have a template that we can use for the cost/price factor?

No, there is no standardized template for the budget estimate. However, budget estimates breakdown accordingly: travel costs services costs equipment costsestimates breakdown accordingly: travel costs, services costs, equipment costs, and supply costs. Travel costs are based on Federal Travel Regulation restrictions for airfare and per diem. Service costs are those costs associated with using vendors or consultants. Equipment costs include items that are not consumables. Supply costs include consumables such as office supplies, text books, handouts, etc. Depending on the training, it may also include ammunition costs in a separate category.

ISD Training/Vendor Presentation

Questions?