Appendix_G_-_NIMS_Training_Plan.pdf

NIMS Training Plan

Food and Drug Administration

ENTERPRISE PERFORMANCE LIFECYCLE

NIMS TRAINING PLAN

Document Version Number: 1.5

Document Version Date: 5/03/2013

Training Plan

VERSION HISTORY

Version Number

Implemented By

Revision Date

Approved By

Approval Date

Description of and Reason for Change

1.0 Jerry Zaborowski 10/02/2012 Initial draft – partially

complete

1.1 Jerry Zaborowski 10/22/2012 Added content

1.2 Sarah Latimer 11/5/2012 Added new sections and

text based on Paula

Reible’s draft from 2/2012 and 10/31/2012 NIMS

Training Meeting

1.3 Sarah Latimer 1/10/2013 Made grammatical

corrections and added communications and blended learning information

1.4 Paula Reible 2/19/2012 Performed high level

review.

1.5 Paula Reible 5/3/2013 Performed revision

based on review with

Trish Koussis

Table of Contents

1.0 INTRODUCTION

1.1 BACKGROUND AND SCOPE

1.2 POINTS OF CONTACT

1.3 GLOSSARY

2.0 INSTRUCTIONAL ANALYSIS

2.1 NEEDS AND SKILLS ANALYSIS

2.2 DEVELOPMENT APPROACH

2.3 ISSUES AND RISKS

3.0 INSTRUCTIONAL METHODS

3.1 TRAINING METHODOLOGY

3.2 TESTING AND EVALUATION

4.0 TRAINING RESOURCES

4.1 COURSE ADMINISTRATION

4.2 COMMUNICATIONS

5.0 NIMS Communications

5.1 ROLL OUT

5.2 NIMS COMMUNICATIONS MAINTENANCE

6.0 RESOURCES AND FACILITIES

6.1 SCHEDULES

7.0 FUTURE TRAINING/IMPLEMENTATION PLAN

8.0 TRAINING MATERIALS LIST

APPENDIX A: TRAINING PLAN APPROVAL

1.0 INTRODUCTION

The Nonclinical Information Management System (NIMS) is a software/database tool that enables CDER Reviewers (Users) to effectively use electronic study data submissions by providing functionality for accessing, analyzing, and visualizing study data that has been received through electronic submission. The NIMS Training Plan is intended to provide managers of the development of NIMS Training a guide for training tools and services that will ensure Reviewers’ effective and efficient use of NIMS.

1.1 BACKGROUND AND SCOPE

The Nonclinical Information Management System (NIMS) project is designed to provide nonclinical study data management, analysis, search, and visualization functionalities to Reviewers As a result of this tool implementation, FDA Reviewers (pharmacologists and toxicologists) who follow the current process, and new Reviewers who are added to the user community, will need to be trained to use the tool. The training must be targeted to the specific audience by analyzing the audience, the tasks, and learning outcomes expected of the audience.

This training plan provides a roadmap for the project team and stakeholders from a management perspective. It identifies the training design, materials, needs analysis, course administrative tasks and operations, and communications in order to support the training deployment. The training plan is based on the following process.

Any scope identified outside of the scope described in this Plan will be addressed in another contract, project and plan.

Assumptions

The NIMS Training Plan assumes that Users are knowledgeable, and/or have received training in the following areas:

- Review, interpretation, and evaluation of Study Data from IND and NDA submissions

- Formats of Study Data

- Basic computer, network and software operation

The scope of the NIMS Training Plan will be limited to the operation of the functionality within the NIMS application.

1.2 POINTS OF CONTACT

NIMS Business Sponsor – Timothy Kropp, FDA

NIMS Training Project Manager – Paula Reible, DRT Strategies

Computational Science Center (CSC) Coordinator – Patricia Koussis, FDA

NIMS Project Manager – David Epstein, CNI

Training Vendor Project Manager - Jerry Zaborowski, Point Cross

1.3 GLOSSARY

NIMS – The Nonclinical Information Management System (NIMS) project.

Reviewers – the FDA’s Pharmacologists, Toxicologists, Pathologists, and other Specialists who evaluate IND applications, and New Drug Applications (NDA).

Introduction course – 1.5 hour instructor-led course designed to introduce the NIMS application, its value, etc. The NIMS course is essential for all OND pharmacology and toxicology Reviewers.

Online module – An on-demand course that allows a Reviewer easy access to more in-depth explanation and application on how to use the NIMS tool. The online course provides the user step-by-step simulation of using the tool for realistic review tasks.

OND – Office of New Drugs

PHARM/TOX Reviewers – Reviewers categorized as pharmacologists and toxicologists.

2.0 INSTRUCTIONAL ANALYSIS

2.1 NEEDS AND SKILLS ANALYSIS

We found at CDER that the best approach to blended learning was to provide classroom and web-based resources, inclusive of class recordings and user guides and manuals.

ID Course Name

Type of Learning

Duration Description Audience

1 Introduction to NIMS

Instructor-led training

(ILT)

2 hours Pharmacology and toxicology reviewers learn how to apply the data management, analysis, and visualization functionality of NIMS in order to inspect, interpret, and evaluate INDs and NDAs

Medical reviewers and statisticians at CDER. The users in the introduction level have no or limited

MAED

experiences prior to training;

2 Web-based training

(WBT)

Recording of #1 above

2 hours The Online Training Module is an on demand resource housed within the NIMS tool.

The online training module teaches the reviewer how to navigate through NIMS, by choosing lessons from a menu of sample reviewer tasks. The module incorporates a hands-on guided discovery learning approach used to develop skill in applying procedures

Medical reviewers and Statisticians at CDER. The users in the introduction level have no or limited

MAED

experiences prior to training;

5 User Guide, Quick Start Guide, FAQs

Resource Documents

N/A NIMS References and User Support can be found on the CSC NIMS Resource page.

Reviewers;

available on FDA website

Recording of Introduction to NIMS (video)

WBT 1-2

hours

This is a recorded version of the class. The course is recorded through Adobe Connect, close captioned, then hosted in a designated place where users can access the course.

Reviewers that haven’t gone through the class and need to take the course online.

Perhaps for Reviewers who need a refresher.

6 Use Cases Power Point

(PPT)

N/A Use Cases are power point presentations that help demonstrate the use of NIMS in various scenarios

For Reviewers interested viewing scenarios of common activities with the NIMS tool.

Target Audience:

The audience for this training will include approximately 180+ Office of New Drug pharmacology and toxicology Reviewers with some level of engagement with the NIMS tool. The majority of the users, known as Reviewers, will need to learn how to apply the data management, analysis, and visualization functionality of NIMS in order to inspect, interpret, and evaluate IND’s and NDA’s. The audience for the training may not immediately use the NIMS tool after it is introduced; therefore there is a need for a more http://inside.fda.gov:9003/ProgramsInitiatives/Drugs/ComputationalScienceCenter/ucm330339.htm#Use_Case comprehensive, on-demand learning module in addition to the introductory instructor-led course. The pharmacology and toxicology Reviewers will be required to attend the introductory training session.

There is a segment of users called Super Users who will be given the highest level of access and may need the most training. They will also be able to teach the tool to others, having the most experience with the tool. In order to ensure they have expertise/proficiency in the tool, it is beneficial to involve them in the review of the development of the training as well as engage them during the pilot sessions.

2.2 DEVELOPMENT APPROACH

The first stage of training development will be to create Use Cases and General Operational Instructions. Use Cases will be based upon NIMS Requirements gathered during the EPLC Requirements Analysis Phase, as well as from workflows and applications that are identified as common practices by the Reviewers. The General Operational Instructions will provide an understanding of the basic features and functionality of the NIMS tool.

The second stage of development will be the production of training materials. Print formats will include Quick Start Guide, User Guide and a Reference Card. Computer-based format includes the online module.

Subject Matter Expert (SME) Engagement

Development of these training materials can benefit from FDA SME insights and recommendations. As a result, seeking buy-in from office and division leaders to encourage expert staff to volunteer their time with training efforts is helpful.

Subject Matter Experts are recommended to perform the following types of activities:

Support development and refinement of training documentation and resources available on the CSC site

Provide insights to reviewer perspectives, skepticism or hesitancy Assist contracted trainers or act as trainers/faculty for classroom sessions Conduct or support clinics or brown bag sessions Seek out external collaborators to come onsite to speak on a topic or note the availability of special seminars to their peers

Important: Identify which SMEs will act as proctors and support instructors to assist with delivery since they are often more respected within the organization and the exposure to training sessions can groom them to potentially become instructors.

The plan for the specifics for SME engagement will be determined as we implement the Reviewer Support Strategy.

2.3 ISSUES AND RISKS

A staged deployment of the NIMS application to the User community is expected because full deployment of NIMS is dependent upon receiving a larger number of electronic study data submissions than is anticipated during the first 12 months after the launch of NIMS.

Intermittent use of the NIMS application by the User community is expected during the first 12 months of deployment because the continual use of NIMS is dependent upon receiving a larger number of electronic study data submissions than is anticipated during the first 12 months after the launch of NIMS.

Because a staged deployment and intermittent use is expected during the first 12 months after the launch of NIMS, it is recommended that the primary training tool enable self-directed learning whenever it is required by the user.

Pharmacology and toxicology Reviewers are under time constraints and pressure to meet strict deadlines; therefore the training development team will frame expectations so that training needs are met and Reviewers understand the direct value of the course and materials to their job performance.

3.0 INSTRUCTIONAL METHODS

3.1 TRAINING METHODOLOGY

• The NIMS Introduction class (1 to 1.5 hours in length) will be developed with the objective of providing an orientation to the NIMS application, and making Users aware of the training materials and support resources available to them.

To that end, the development approach is to first predict the initial set questions User will have about the NIMS application, and then prepare content and materials that address those questions; which are expected to be high level questions related to the why, what, how, who, and when of the NIMS application.

• The online module will be developed using the procedural learning model; a guided discovery learning approach used to develop skill in applying procedures.

The intent is to allow the User to select a task from a menu and then place the User in a problem-solving environment that employs visual and functional simulation of the NIMS application. Users are expected to complete a specific task, which is an example of a range of similar specific tasks they will need to perform in the use of the NIMS application. The goal is not just to complete the example task, but also to learn the underlying procedure so as to apply it in future situations, where applicable.

Each Use Case that is developed to inform the online module will require a dataset that supports that specific Use Case. These datasets may be supplied by the FDA and/or by the contractor. In some cases, the datasets may be modified to augment the instructional value of the Use Case, and/or to remove proprietary information from the dataset. These datasets will be placed into a Training Database or training section of the NIMS eRoom and used in Training Courses.

3.2 TESTING AND EVALUATION

Pilots:

Two pilot sessions with time in between pilots to make changes will be performed for the classroom training A facilitated focus group will be conducted for the each of the two piloted NIMS classes to determine where there are areas of improvement during development. The focus group will ask similar questions to the course evaluation.

Once this feedback has been incorporated, final reviews will take place until the training materials are considered final. Then the materials are produced and stored in the necessary websites or eRooms for dissemination.

Important: Prior to final production, Training Materials will be reviewed by editors and subject matter experts for accuracy.

Evaluations:

Surveys will be prepared for each training offering and issued to Users for their feedback. That feedback will be used to gauge the quality of the Training Materials and Training Courses.

The survey will evaluate the following aspects, but not limited to - Learning effectiveness Quality of facilitator/instructor (if applicable) Quality of training materials Soundness of instructional design Evaluate online training

4.0 TRAINING RESOURCES

4.1 COURSE ADMINISTRATION

CSC is currently working towards developing a more centralized coordination of training efforts, including support, reporting, enrollment and registration, evaluation and communications. The Training Project Manager and the CSC Coordinator will work with OBI and OND to ensure that facilities, materials, enrollment, technical environment, and resources are ready to support the training roll out. This will require communication and agreement of roles and responsibilities between the offices, and action item tracking by the project manager.

NIMS Introduction Training will encompass 12 sessions with a maximum of 23 Reviewers per class session, and one instructor. NIMS Introduction Training registrations and attendance recording will be the responsibility of OBI.

4.2 COMMUNICATIONS

Communication about the training must occur at least six weeks before training roll out.

The Coordinator is responsible for these communications.

5.0 NIMS Communications

5.1 Roll Out

In the next fiscal year (2013 – 2014), the training course will be available online via video. The project lead will work with the non-clinical data standards leads to identify new pharm/tox reviewers. These new reviewers will be directed to the online class and met with individually to discuss the NIMS and resources available to them. We’ve aligned our training so that it is reflected on the OND website for Pharm/Tox reviwers in two places –

1. Under the webpage titled: OND PharmTox Reviewer Training Courses & Learning Materials. The link is provided here:

http://inside.fda.gov:9003/CDER/OfficeofNewDrugs/ImmediateOffice/ONDLe arningandCareerDevelopment/ucm229210.htm

2. And also in the PharmTox Reviewer Curriculum Schedule. The is provided here:

http://inside.fda.gov:9003/downloads/CDER/OfficeofNewDrugs/ImmediateO ffice/ONDLearningandCareerDevelopment/UCM348467.doc

Vehicle Description Date / Timing Date

Completed Emails

From OND From CSC Training account From CSC General account CDER All-Hands OND bi-weekly training

Immediately following general announcement from the CSC and refresher messages as needed

12/14

CSC Board Meeting Showcase

Open meeting including addition of all Pharm/Tox Reviewers, demonstration of the NIMS tool

12/6/2012 12/6/2012

OND-CSC

Quarterly Meeting

Quarterly meeting between OND and CSC

December 4th 12/4/2012

Pharm / Tox Division Data

Monthly meeting; Tim Kropp will present

January 1/2013 http://inside.fda.gov:9003/CDER/OfficeofNewDrugs/ImmediateOffice/ONDLearningandCareerDevelopment/ucm229210.htm http://inside.fda.gov:9003/CDER/OfficeofNewDrugs/ImmediateOffice/ONDLearningandCareerDevelopment/ucm229210.htm http://inside.fda.gov:9003/downloads/CDER/OfficeofNewDrugs/ImmediateOffice/ONDLearningandCareerDevelopment/UCM348467.doc http://inside.fda.gov:9003/downloads/CDER/OfficeofNewDrugs/ImmediateOffice/ONDLearningandCareerDevelopment/UCM348467.doc

Standards Leads Meeting CSC Website http://inside.fda.gov:900

3/ProgramsInitiatives/Dr ugs/ComputationalScien ceCenter/ucm153283.ht m

Allow one business week for update

12/11/2012

CDER

Homepage:

Banner http://inside.fda.gov:900 3/CDER/default.htm

Before September

Publications CDER Connection Feature (600 words or less) http://inside.fda.gov:900 3/CDER/OfficeofTraining andCommunication/ucm 262613.htm CDER Connection In Brief (250 words or less) http://inside.fda.gov:900 3/CDER/OfficeofTraining andCommunication/ucm 340694.htm

Due the 15th of the month for the next month (Dec. 15th for January issue) - TBD

Email from OND Non clinical Board Member

Paul Brown As needed Feb 24, 2013

OND Bi-Weekly Training Email

Susanne Walker is the contact

Due just before the 1st and 15th of each month. ( they email goes out on these days)

January, 2013

5.2 NIMS Communications Maintenance

Vehicle Description Frequency Date Initiated Emails From OND

From CSC Training account From CSC General account CDER All-Hands OND bi-weekly training

Quarterly / as needed

Refresher messages as needed

CSC Website http://inside.fda.gov:9003/ ProgramsInitiatives/Drugs /ComputationalScienceCe nter/ucm153283.htm

As needed Update with relevant training registration information as needed

OND Senior Leadership Team II Meeting

Monthly Meeting – to discuss OND business and policy issues.

Monthly February

Pharm Tox Coordinating Committee Meeting

Subgroup of OND Monthly Paul Brown will provide monthly updates in this meeting

Communications tasks associated with the training are built into the Training Project Schedule. Each course may have a course summary and description, course ID #, a target audience description, course objectives, course size limit, course length and duration, course facilitator, and other pertinent information for the participant.

Participants will need to know what training is available and when it is offered so a participant can register for a course easily.

6.0 RESOURCES AND FACILITIES

The updated training materials, User’s Guide, Quick Reference Guide, and FAQs will be provided to the project manager by the vendor.. The latest copies can be requested from the project manager. These will be posted in designated spaces determined by the team based on the easiest access for Users.

The ILTs and webinars will be provided by CDER. The training equipment (computers & projector) will be provided by CDER. The instructors are required to be on site or on line 1 hour prior to the beginning of the class.

The NIMS introduction course requires a classroom that has a projector and projection screen for a Power Point presentation and seating for approximately 20-25 people. The introduction course does not need to be held in a computer lab.

For webinars, the instructions for the website and call-in information will be distributed by the project manager 3 days prior to the class.

6.1 SCHEDULES

The NIMS introduction Class will be delivered starting in January 2013 over the course of approximately twelve 2-hour sessions, to allow for questions and answer sessions.

There will be one to two NIMS introduction training per week for approximately twelve weeks. OND supervisors will ensure that the pharmacology and toxicology Reviewers are completing the introduction course. OBI is in charge of reserving classrooms, coordinating registrations, and on-site training support. OBI will need to work directly with the vendor, Point Cross, for training delivery. The classes must be delivered by summer 2013.

7.0 FUTURE TRAINING/IMPLEMENTATION PLAN

Course Updates Courses will be updated and evaluated as needed. We plan to evaluate the online module through the program evaluation that will take place after fall 2013.

Aligning New Pharm/Tox Reviewers to NIMS Training The CSC Training Team met with the NIMS Lead to discuss how to integrate new PharmTox Reviewers to the NIMS training.

We’ve aligned the Introduction to NIMS training so that it is reflected on the Office of New Drugs (OND) website for Pharm/Tox reviewers in two places –

1. Under the webpage titled: OND PharmTox Reviewer Training Courses & Learning Materials. Scroll to the bottom and find the Introduction to NIMS course. The link is provided here:

http://inside.fda.gov:9003/CDER/OfficeofNewDrugs/ImmediateOffice/ONDLearningandCareerDevelo pment/ucm229210.htm

2. In in the PharmTox Reviewer Curriculum Schedule. The is provided here:

http://inside.fda.gov:9003/downloads/CDER/OfficeofNewDrugs/ImmediateOffice/ONDLearn ingandCareerDevelopment/UCM348467.doc

Introduction to NIMS Course Again, in the next fiscal year (2013 – 2014), the Introduction to NIMS course will be available online via video. The project lead will work with the non-clinical data standards leads to identify http://inside.fda.gov:9003/downloads/CDER/OfficeofNewDrugs/ImmediateOffice/ONDLearningandCareerDevelopment/UCM348467.doc http://inside.fda.gov:9003/downloads/CDER/OfficeofNewDrugs/ImmediateOffice/ONDLearningandCareerDevelopment/UCM348467.doc new pharm/tox reviewers. These new reviewers will be directed to the online class and met with individually to discuss the NIMS and resources available to them.

8.0 TRAINING MATERIALS LIST

Classroom Training (1.5 to 2 hours)

- PowerPoint Slides with script

- Live or recorded NIMS demonstration

- Reference card handout

- Projector & projection screen

- Sign-in sheet

- Email with login information and NIMS shortcut to be sent immediately after training session

- Q&A follows the actual class.

NIMS Online Training

- Waterfall approach with titles and subtitles

- Help drop down menu with resources and troubleshooting

NIMS User Guide, Quick Start Guide, FAQ, & Use Cases

APPENDIX A: TRAINING PLAN APPROVAL

The undersigned acknowledge that they have reviewed the NIMS Training Plan and agree with the information presented within this document. Changes to this Training Plan will be coordinated with, and approved by, the undersigned, or their designated representatives.

Signature: Date:

Print Name:

Title:

Role: Project Manager