PWS EMS 1-22-26.pdf

EVALUATION MANAGEMENT SYSTEM (EMS)

PERFORMANCE WORK STATEMENT (PWS)

IRS Human Capitol Office, IRS University

1.0 INTRODUCTION/BACKGROUND

A Training Evaluations Management System (EMS) is needed by the Internal Revenue Service (IRS) to assist the Service wide Learning & Education (SL&E) community in developing and delivering training evaluations, capturing training evaluation data and delivering reports on the effectiveness of the training programs.

• The capabilities and user interface of the existing Learning Management

System (LMS) do not provide sufficient support to measure the impact of training programs.

• Internal Revenue Service has used Training Evaluation and Measurement for Performance Optimization (TEMPO) as its EMS continually since FY14. Since then, the IRS has successfully utilized the EMS’ capabilities to provide Level 1 through Level 3 training evaluation data collection, analysis and reporting to measure the overall health of the organization’s training program. TEMPO also delivers level 4 evaluation capabilities, and the IRS continues to work with the vendor to improve its Level 4 evaluation process. Additional hours for non-routine improvement efforts are expected only during the base year.

• It is necessary to retain an integrated, standalone solution. In the absence of such a solution, there is a significant reliance on paper forms, internal staff and customized solutions. Multiple systems and inconsistent approaches used for training evaluations create a significant manual workload and prevent collection of consistent, reliable data and exchange of information among the SL&E community.

• Recent work has been performed by PTG International under contract number 2032H5-20-C-00052. That current contract is set to expire on 2/28/2026.

2.0 SCOPE OF WORK

The contractor shall provide an Evaluation Management System (EMS) that will effectively and efficiently design, deliver, analyze and report the required Levels 1 – 4 training evaluation data to meet the requirements of 5 CFR 410.202.

2.2 This system shall be capable of integrating and interfacing with other IRS data and financial systems required by the IRS or the Office of Personnel Management (OPM), such as the Integrated Talent Management (ITM) System, without requiring modifications to these systems. Based on defined requirements, the EMS may include data from other Learning Education and Development Services and Human Capital Office systems as required.

2.3 The EMS system identified will be purchased as a commercial off-the-shelf (COTS) product.

2.4 The selected EMS shall be available to all Servicewide Learning and Education employees for developing and delivering training evaluations, capturing training evaluation data and delivering reports on the effectiveness of the training programs.

2.5 The selected EMS shall be available to all IRS employees (approximately 100,000) for completion of evaluations developed and delivered by the SL&E community.

2.6 The selected EMS shall be able to accommodate a variety of access levels (e.g., read only, read and search, and read, search, input and edit).

2.7 The selected EMS shall work with data feeds and/or imports from other IRS training systems and sources.

2.8 The selected EMS shall have a reporting tool that will provide the ability to report out data directly from the EMS in formats (i.e. XML, HTML, CSV) compatible with software presently used by IRS employees such as MS Office Suite/Adobe PDF.

4.0 REQUIREMENTS - PRODUCTS/EQUIPMENT/SERVICES

The Contractor shall be able to deliver a COTS EMS product that supports the following product and services:

4.1 General Functional Requirements

• The system shall be user friendly.

• The system shall have a Help Function.

• The system shall incorporate job aids and prompts.

• The system shall be 508 compliant for authorized users of the system.

• The system shall be 508 compliant for individual users providing evaluation input, (i.e., completing training evaluations created through the system).

• The system shall provide access to authorized users.

• The system shall provide at least 3 level of access:

o Read only, o Read and search, and o Read, search, input and edit.

• The system shall be used for internal and external training events administered through the LMS.

• The system shall have the capability to receive course, class and survey data from the LMS, including data from custom fields.

• The system shall provide accessibility and store the Level 1 through Level 4 training evaluation data in a database for the purpose of generating reports and dashboards.

• The system shall run queries and produce reports with certain fields hidden.

• The contractor shall support ad hoc and customized reports.

• The system shall create and export reports in tools used by the IRS, (e.g., MS Office Suite, Adobe PDF, etc.).

• The system shall report the evaluation data in different formats (e.g., XML, HTML, CSV, etc.).

• The system shall generate graphic and statistical representations of data.

• The system shall provide data in number and percent (%) format.

• The system shall provide data organized in statistical categories such as trends, means, deviations, and cumulative data.

4.2 Specific Functional Requirements

• The system shall allow automatic and manual entry, accessibility, storage, and query of data related to development, analysis and reporting of Level 1 through Level 4 training evaluations.

• The system shall allow manual input of training evaluation data that is not systematically captured (e.g., alternative Level 1, Level 3 paper training evaluation data, etc.).

• The system shall accommodate scanned training evaluation data.

• The system shall store, provide accessibility and query curriculum, course, and class data (e.g., completion date, delivery platform/method, target audience, delivery mode, item completion status report, etc.).

• The system shall store, provide accessibility and query historic training evaluation data by several different types of methods (e.g., by type of training evaluation, course, class, question response, employee category, delivery method, location, time-period, training evaluation question, learning objective, key performance indicator, etc.).

• The system shall have the capability for users to create Level 1 through Level 4 training evaluation instruments.

• The system shall store and provide accessibility to previously designed Level 1 through Level 4 training evaluation instruments.

• The system shall store and provide accessibility to previously designed Level 1 through Level 4 training evaluation data such as questions, objectives, remediations, and key performance indicators by providing searchable pools (e.g., Test / Bank Questions, objective pools, key performance indicator pools, etc.).

• The system shall allow manual input of data related to the curriculum, course, and/or class evaluation (e.g., information about the learning objectives, remediations, job tasks, key performance indicators, etc.).

• The system shall incorporate automatic access to the training evaluation instrument when the user is prompted via landing page, hyperlink and/or Quick Response (QR) code to complete online training evaluations.

• The system shall provide Level 2 evaluation reports, including test scoring.

• The system shall assist in the capture, storage and production of Level 1 through 4 training evaluation reports and dashboards.

o The contractor shall support ad hoc and customized reports and dashboards.

o The system shall generate easy-to-read, customized reports and dashboards from the stored data.

• The system shall identify and flag problem areas (i.e., items above or below one standard deviation).

• The system shall align with IRS record retention policies.

4.3 Optional Functional Requirements

• The system may allow for static links to policies, IRMs, job aids and guidelines pertaining to training evaluation.

• The system may display and filter the scheduled training evaluations in a calendar format by course number; training evaluation level; business unit;

training evaluation Point of Contact (POC); project manager.

• The system may track the expiration date of training evaluation records per course/class to support records disposition per the IRM.

• The system may provide alerts for the deadline of the training evaluation submissions.

• The system may store tips for question development.

• The system may provide randomization of test questions.

4.4 Licensing and Hosting Requirements

• The system shall accommodate the following license quantities:

o 1,000 user reporting licenses o 5 admin licenses.

• The contractor shall host the FedRAMP compliant, web-based system.

• The system shall monitor and support transfer of data from the LMS (i.e., automated and/or manual data feed).

• The contractor shall provide a dedicated Help Desk function to assist users .

• The system shall accommodate Level 1 through 4 evaluations associated with 24,000 courses and/or curricula per year.

• The contractor shall provide processing for 300 paper Level 1 through 4 training evaluation forms per year.

• The contractor shall provide processing for 35,000 electronic Level 1 through 4 training evaluation forms per month.

• The contractor shall provide 80 hours of Level 1 – Level 4 training evaluation consulting support per month.

• The contractor shall provide 80 hours of Level 1 – Level 4 training evaluation administrative (i.e., project management, ad hoc reporting and analysis) support per month.

• The EMS contractor shall provide eight (8) training sessions / webinars for authorized users or designees per year.

• The contractor shall provide 280 hours of development support per year (e.g., modify existing or develop new system functionality such as reports, dashboards, instrument types, etc.).

• In the base year only, the contractor shall provide 500 hours of development support (in addition to the 280 annual hours listed above) for creating custom Level 4 functionality.

The contractor shall meet with designated members of the IRS Servicewide Training Evaluation Program (STEP) team to review the requirements for the evaluations project. After requirements have agreed upon, the contractor and the team shall work together to develop a baseline project plan that shall be used to document and track progress and to monitor deliverables and due dates. The contractor shall use Microsoft Office products when preparing deliverables for the

IRS.

The contractor shall coordinate with the IRS Information Technology (IT) and IRSU Technology organizations (with the assistance of a designated STEP representative) to ensure that the tools are compatible with the IRS Common Operating Environment (COE) and the IRS Enterprise Architecture prior to making recommendations.

The contractor shall, at no cost to the IRS, demonstrate the capabilities of the system(s) recommended for purchase.

Annual Deliverables Table The following services will be delivered annually with the one noted exception delivered only in the base year.

Item Description Unit Type # Units Enterprise Level Service Fee (based on 1501 - 2000 active courses)

# of Months 12

Web Services Managed Hosting # of Months 12

Training Sessions for New Users / Webinars

# of Days 8

Paper Form Fees # of Forms per year

Electronic Form Fees # of Forms per Month

30,000

Monitor and Support Feed from LMS # of Hours per Month

Custom Level 4 Development – Delivered during base year only

# of Hours 500

Revisions / Custom Reports # of Hours 320

Consulting Support - Evaluation SME # of Hours per Month

Ad Hoc Reporting and Analysis # of Hours per Month

Project Management - Program Manager # of Hours per Month

5.0 CONTRACTOR PERSONNEL

5.1 Standards of Behavior

The contractor and contractor employees shall maintain a “professional” standard of employee/subcontractor employee competency, conduct, appearance and integrity and shall be responsible for taking such disciplinary action with respect to its employees as necessary. The contractor shall ensure employees do not disturb papers on desks/surfaces, open desk drawers or cabinets, or use Government telephones/faxes, except as authorized. Employees shall not eat while on duty, only during authorized breaks or lunch periods, then they shall be in an appropriate/designated area. This facility is non-smoking.

5.2 Government Closures

The contractor shall in inclement weather/unforeseen circumstances coordinate with Contracting officer’s representative to determine if contracted service should continue work when the OPM closes the Federal government.

6.0 CONTRACTOR QUALITY CONTROL PROGRAM

The contractor shall establish a complete Quality Control Program to assure the requirements of the contract are provided as specified. Within 7 calendar days prior to the starting date of the contract, the contractor shall submit a copy of its program to the COR for approval. The program shall include, but not be limited to, the following: an inspection system covering all the services stated in Section C. A system for identifying and correcting deficiencies in the quality of services before the level of performance becomes unacceptable or the government inspectors point out the deficiencies or both. An on-site file (this can be electronic) shall be kept of all inspections conducted by the contractor with the corrective action taken. This documentation shall be submitted to the COR at the end of the month during the term of this contract. The contractor shall be held responsible at all times for ensuring that the functions and specifics of their Quality Control Program are fully and accurately implemented.

7.0 PLACE OF PERFORMANCE

Performance will be at the contractor’s facility.

8.0 PERIOD OF PERFORMANCE

The base period is one year: March 1, 2026.- February 28, 2027 Option Year One: March 1, 2027.- February 28, 2028 Option Year Two: March 1, 2028.- February 28, 2029 Option Year Three: March 1, 2029.- February 28, 2030 Option Year Four: March 1, 2030.- February 28, 2031

9.0 GOVERNMENT-FURNISHED PROPERTY

No government property, access card or government ID will be issued.

10.0 GOVERNMENT-FURNISHED INFORMATION

Data from surveys and extracted data form IRS and Treasury systems noted above, will be provided to the contractor. While not containing PII or SBU information the contractor will still be expected to properly return or destroy all information provided to or generated by the EMS system.

11.0 Travel

No travel will be required.

Evaluation Management System (EMS)