Section_J-AttachmentC.11_Sample_IEP_Standard_Operating_Procedures_Outline-20160909-V2.pdf

IRS Integrated Enterprise Portal Program

IRS IEP Standard Operating Procedures Page 1

Internal Revenue Service

Solicitation Number: TIRNO-16-R-00026

IRS Integrated Enterprise Portal 1.5 (IEP 1.5)

IRS Integrated Enterprise Portal Program

Sample Standard Operating Procedures

SECTION J-ATTACHMENT C.11

IRS IEP Standard Operating Procedures Page 2

Table of Contents

1 Introduction

1.1 Purpose

1.2 Document Scope

2 ITSM Tool

2.1 User Roles

2.2 When to Use

2.3 Notifications

2.4 Request Fulfillment Process Flow

3 IEP Service Desk

3.1 Service Desk Contact Information

3.2 Service Desk Support Hours

3.3 IEP Service Desk Procedures and Responsibilities

4 Access Management

4.1 Access Management Governing Rules

4.2 Roles and Responsibilities

4.3 Access Request High Level Process Flows

4.3.1 Standard IRS Access Request Fulfillment Process

4.3.2 Non-Standard IRS Access Request Fulfillment Process

4.4 IRS Access Request Procedures

5 Change Management

5.1 Overview

5.2 Change Management Governing Rules

5.3 MSP-Initiated Patching Changes

5.4 Change Management Procedures

5.4.1 Creating IEP Change Requests

5.4.2 Procedures for all Application Change Requests

5.4.3 Procedures for IEP Patching

5.4.4 Firewall Change Requests and when they should be submitted

IRS IEP Standard Operating Procedures Page 3

5.5 Change Management Back-out Guidelines

5.6 Emergency Change Requests

5.6.1 Creating an Emergency Change Request

5.6.2 Approving an Emergency Change Request

5.6.3 Emergency Change Request Process

6 Release Management

6.1 Overview

6.2 Release Management in the IEP (definition and policies)

6.3 IEP Application Release Management Governing Rules

6.4 Roles and Responsibilities

6.5 IEP Application Release Management High Level Process Flow

6.6 IEP Application Deployment Scheduling

6.7 Deployment Coordination

6.8 Deployment Methods

6.9 Release and Deployment Management Procedures

6.9.1 Release and Deployment Management Procedures for Applications

6.10 Release and Deployment Management in ITSM Tool

7 Configuration Management

7.1 Overview

8 Event Management

8.1 Overview

8.1.1 What is an Event

8.1.2 Event Categories

8.2 Roles and Responsibilities

8.3 Event Management Governing Rules

8.4 Event Management High Level Process Flow

8.5 Event Management Procedures

9 Incident Management

9.1 Overview

9.1.1 What is an Incident?

9.1.2 Determining the Impact of an Incident

IRS IEP Standard Operating Procedures Page 4

9.1.3 Determining the Urgency of an Incident

9.1.4 Production Incident Priority Matrix

9.1.5 IEP non-production environment Incident Priority Matrix

9.2 Incident Management Governing Rules

9.3 Roles and Responsibilities

9.4 Incident Management High Level Process Flow

9.5 Ticketing System Criteria

9.6 Responsibility Matrices

9.7 Incident Management Procedures

9.7.1 End User Impacting P1/P2 Incidents reported by MSP

9.7.2 End User Impacting P3/P4 Production Incidents reported by MSP

9.7.3 P1/P2 Production Incidents reported by the IRS

9.7.4 P3/P4 Production Incidents reported by the IRS

9.7.5 All Incidents in IEP non-production Environments

9.7.6 Upgrading a P3/P4 KISAM Incident

9.7.7 Downgrading a P1/P2 KISAM Incident

9.8 Service RestorationTeam(SRT) Process

9.9 Incident Management in ITSM Tool

9.9.1 Who can submit an Incident in ITSM tool

9.9.2 When to submit an Incident in ITSM tool

9.9.3 Incident States in

10 Problem Management

10.1 Overview

10.1.1 What is a Problem

10.1.2 What is a Known Error

10.2 Roles and Responsibilities

10.3 Problem Management Governing Rules

10.4 Problem Identification

10.4.1 Evaluation and Identification of a Problem

10.4.2 Problem Management Creation Guideline

10.5 Problem Management High Level Process Flow

IRS IEP Standard Operating Procedures Page 5

10.6 Problem Management Procedures

10.6.1 Problem Procedures for end user impacting Incident(s) where a Change is required for resolution

10.6.2 Problem Procedures for end user impacting Incident(s) where no Change is required for resolution 19

11 Knowledge Management

11.1 Overview

11.1.1 What is a Service Knowledge Management System (SKMS)

11.1.2 What is a Knowledge Base Article?

11.2 Knowledge Management Governing Rules

11.3 Roles and Responsibilities

11.4 Knowledge Management High Level Process Flow

11.5 Knowledge Management Procedures

11.5.1 Creating a Knowledge Base Article

11.5.2 The Knowledge Coordinator reviews the content and determines if a SME needs to review the

Knowledge Base Article

11.5.3 Creating a Knowledge Base Article in KISAM if needed

11.6 Knowledge Management in ITSM Tool

11.6.1 Who can submit a Knowledge Base Article

11.6.2 Who can publish a Knowledge Base Article

11.6.3 Types of Knowledge Base Articles that can be created

11.6.4 Knowledge Workflow States

11.6.5 Maintaining Knowledge Base Articles

12 Appendix

12.1 Appendix A – Acronym List

12.2 Appendix B –Process Flows

12.3 Appendix C – Access Management ITSM Tool Instructions

12.3.1 Opening an Access Request (with ITSM tool access)

12.3.2 Viewing and Updating your Access Request

12.4 Appendix D – Incident Management ITSM Tool Instructions

12.4.1 Reporting an Incident

12.4.2 Viewing and Updating your Incident

IRS IEP Standard Operating Procedures Page 6

12.5 Appendix E – Application Change Request Instructions

12.5.1 Opening an Application Change Request

12.6 Appendix F – Application Migration Request Instructions

12.6.1 Opening an Application Migration Request

12.6.2 Viewing and Updating your Application Migration Request

12.7 Appendix G – External Firewall Change Request Instructions

12.7.1 Opening an External Firewall Change Request

12.8 Appendix H – Instructions for OLS Requests

12.8.1 Open an OLS Request

12.9 Appendix I- P1/P2 Emergency Change Request Process

12.9.1 Priority 1 (P1) Emergency Change Process

12.9.2 Priority 2 (P2) Emergency Change Process

IRS IEP Standard Operating Procedures Page 7

Table of Tables

Table 1 ITIL- Based Procedures

Table 2 Access Management Governing Rules

Table of Figures

IRS IEP Standard Operating Procedures Page 8

1 Introduction

1.1 Purpose

The purpose of this document is to describe the daily operational procedures for the IRS Integrated Enterprise

Portals (IEP) Platform. Specifically, the procedures described in this document will help IRS Stakeholders and

O&M teams request changes to the overall service, move key items such as application code patches and infrastructure changes through the IEP, and track the deployment and usage of Configuration Items and their associated product licenses.

Additionally, these procedures will allow users to request access to systems, report incidents, track recurring issues, and manage events and utilization levels.

These procedures are based on ITIL v3 standards. As such, they are interrelated and provide a high level of operational integrity with respect to management activities across both the IRS and . The overarching goal of the implementation of these procedures is to enable continuous work while maintaining stable control of the IEP

Platform.

1.2 Document Scope

This document provides step by step procedures for the different ITIL-based procedures applied to carry out daily operational activities by the IEP team, who at times works in conjunction with the IRS. Additionally, it provides an overview of the IT Service Management Tool that is used to facilitate these procedures.

Table 1 lists all of the ITIL-based procedures that are being implemented by the IRS IEP Program. Note that for

Change and Configuration Management the procedures are documented in the IEP Configuration Management

Plan. The detailed procedures for those areas are documented in this document.

ITIL-Based Procedures

Access Management

Change Management

Release and Deployment Management

Configuration Management

Event Management

Incident Management

Problem Management

IRS IEP Standard Operating Procedures Page 9

Knowledge Management

Table 1 ITIL- Based Procedures

IRS IEP Standard Operating Procedures Page 10

2 ITSM Tool

ITSM is a IT Service Management tool, and will be the main ticketing system used by to manage incidents and requests for the IRS Enterprise Portal Program.

The ITSM Tool is accessible through a web-based interface. Users can login using their username and passwords to submit requests and report Incidents. ITSM Tool credentials can be requested through OL5081.

After creation, requests will then be routed to the appropriate service desk staff and managed appropriately.

provides an easy-access self-service tool in which users can request and receive assistance in a timely, consistent, and accurate manner.

For further information on capabilities, including watch lists, please see the External IRS Developer Training document.

2.1 User Roles

2.2 When to Use

2.3 Notifications

2.4 Request Fulfillment Process Flow

3 IEP Service Desk

The IEP Service Desk will serve as the initial support point of contact for all IEP requests. All ITSM tool tickets will be routed to the IEP Service Desk, who will do the first review, triage, and assignment of the ticket. Once it has been assigned to the appropriate support group, the Service Desk will transfer ownership of the ticket to the support group that is handling it.

3.1 Service Desk Contact Information

3.2 Service Desk Support Hours

IRS IEP Standard Operating Procedures Page 11

3.3 IEP Service Desk Procedures and Responsibilities

4 Access Management

Access Management is the process responsible for allowing end users to make use of IT services, data, or other assets. It helps to protect the confidentiality, integrity, and availability of assets by ensuring that only authorized users are able to access or modify IT assets.

The objectives of the Access Management process are to:

Manage access to services based on policies and actions defined in Information Security Management

Efficiently respond to requests for granting access to services, changing access rights or restricting access, ensuring that the rights being provided or changed are properly granted

Oversee access to services and ensure rights being provided are not improperly used

The Access Request procedures outlined in the sections below apply to any IEP system managed by to which an IRS end user or support resource will need to gain access. Examples include access to dashboards and the

Sandbox and Development environments.

4.1 Access Management Governing Rules

Table 6 lists overarching rules associated with Access Management within the IEP. These rules govern the creation and fulfillment of Access Requests, and define the system of record for the various types of requests.

Rule

Number

Rule Description

1 All users of IEP systems must be authorized by the IRS

2 All IRS resources requesting access to IEP systems (e.g., JBoss, Apache, etc.) must submit their Access Requests through tool and theOL5081 tool

3 All IEP resources requesting access to IEP systems must submit their Access Requests through the ITSM tool

4 All IEP resources requesting access to IRS systems must submit their Access Requests through the OL5081 tool

5 All Access Requests submitted through the OL5081 tool to for IEP systems access must be reflected and fulfilled through an accompanying ticket

In cases where the OL5081 tool is unavailable, IRS resources can request access to IEP systems through the ITSM tool; however, once the tool becomes available again, IRS users must retroactively submit their request through the OL5081 tool

IRS IEP Standard Operating Procedures Page 12

Rule

Number

Rule Description

7 In order to use the Non-Standard process for submitting an Access Request, the user must first receive approval from ETI to follow the Non-Standard process

8 For users requesting access to only , they can all use a guest account to submit their request directly in , foregoing the use of OL5081

Table 2 Access Management Governing Rules

4.2 Roles and Responsibilities

4.3 Access Request High Level Process Flows

4.3.1 Standard IRS Access Request Fulfillment Process

4.3.2 Non-Standard IRS Access Request Fulfillment Process

4.4 IRS Access Request Procedures

5 Change Management

5.1 Overview

The scope of our Change Management effort is to control all IEP changes to workloads running on the IEP platform. This will be accomplished by standardizing methods and procedures that are used for efficient and prompt handling of all changes with minimum disruption to IT services, thereby ensuring quality and continuity of the IT service at all times. IEP Change Control manages, dictates, governs, and oversees changes only in IEP;

changes that are not controlled by the IEP will be out of scope. For further information on the Change Management process please reference the Configuration Management Plan.

5.2 Change Management Governing Rules

IRS IEP Standard Operating Procedures Page 13

5.3 MSP-Initiated Patching Changes

The MSP will carry out patching of the middleware components based on the IEP Security Patch Management

Plan. The MSP will publish this schedule to communicate the timeline to all applications and stakeholders and get concurrence from all applications.

5.4 Change Management Procedures

5.4.1 Creating IEP Change Requests

5.4.2 Procedures for all Application Change Requests

5.4.3 Procedures for IEP Patching

5.4.4 Firewall Change Requests and when they should be submitted

5.5 Change Management Back-out Guidelines

Every Change Request must have back-out instructions provided in the ticket itself. In Production, the back-out instructions must be followed when it has been identified that the change being deployed is unsuccessful.

5.6 Emergency Change Requests

An Emergency Change is a change that has been implemented under a Priority 1 or Priority 2 incident in order to restore service. Emergency Changes are documented and authorized on the management Service

Restoration Team (SRT) call by the initiator.

5.6.1 Creating an Emergency Change Request

5.6.2 Approving an Emergency Change Request

5.6.3 Emergency Change Request Process

6 Release Management

IRS IEP Standard Operating Procedures Page 14

6.1 Overview

Release Management is responsible for implementation and quality control of application codes, Middleware/COTS releases, all hardware upgrades. In this section, there will be detailed information on the different roles and responsibilities within Release Management and the release management process. This section also contains helpful information regarding the requirements and rules around Release Management.

6.2 Release Management in the IEP (definition and policies)

6.3 IEP Application Release Management Governing Rules

6.4 Roles and Responsibilities

6.5 IEP Application Release Management High Level Process Flow

6.6 IEP Application Deployment Scheduling

6.7 Deployment Coordination

6.8 Deployment Methods

6.9 Release and Deployment Management Procedures

6.9.1 Release and Deployment Management Procedures for Applications

6.10 Release and Deployment Management in ITSM Tool

7 Configuration Management

IRS IEP Standard Operating Procedures Page 15

7.1 Overview

For further information regarding the Configuration Managemenbt process, please reference the Configuration Management Plan.

8 Event Management

8.1 Overview

Event Management is the basis for operational monitoring and control. The process is used to manage events for all environments. Event Management will detect all changes of state of a Configuration Item (CI) or IT Service running on the IEP platform that have significance to the CI or the IT Service. It will then be determined what the appropriate control action for those events and ensure its communicated to the appropriate function or group within the IEP.

8.1.1 What is an Event

8.1.2 Event Categories

8.2 Roles and Responsibilities

8.3 Event Management Governing Rules

8.4 Event Management High Level Process Flow

8.5 Event Management Procedures

9 Incident Management

9.1 Overview

The goal of Incident Management is to resolve any incidents that cause an interruption of service in the quickest, most effective way possible.

This section contains the roles and responsibilities associated with all Incident Management activities and will show when and how to create different types of incidents and the Incident Management process flow. It also includes rules and general information regarding Incident Management and the managing of incidents in ITSM

IRS IEP Standard Operating Procedures Page 16 tool and other ticketing systems. To contact the IEP Service Desk regarding Incident Management, please see

Section 3 of this document for Service Desk hours of operation and contact information.

9.1.1 What is an Incident?

9.1.2 Determining the Impact of an Incident

9.1.3 Determining the Urgency of an Incident

9.1.4 Production Incident Priority Matrix

9.1.5 IEP non-production environment Incident Priority Matrix

9.2 Incident Management Governing Rules

9.3 Roles and Responsibilities

9.4 Incident Management High Level Process Flow

9.5 Ticketing System Criteria

9.6 Responsibility Matrices

9.7 Incident Management Procedures

9.7.1 End User Impacting P1/P2 Incidents reported by MSP

IRS IEP Standard Operating Procedures Page 17

9.7.2 End User Impacting P3/P4 Production Incidents reported by MSP

9.7.3 P1/P2 Production Incidents reported by the IRS

9.7.4 P3/P4 Production Incidents reported by the IRS

9.7.5 All Incidents in IEP non-production Environments

9.7.6 Upgrading a P3/P4 KISAM Incident

9.7.7 Downgrading a P1/P2 KISAM Incident

9.8 Service RestorationTeam(SRT) Process

The SRT Process outlines the steps required to restore service of a critical, high-Priority Incident. When an

Incident is created in KISAM, all P1/P2 tickets are assigned to the ITOCC. While reviewing the ticket, the ITOCC will determine if all information is included, if the whole scope is addressed, and if it has been assigned correctly. The ITOCC will then stand up a Management SRT with leadership to determine the focus area of the investigation and resolution activities.

Leadership assigns an IMR to lead the triaging process

Initially, the ITOCC will ask the IMR who needs to be engaged. The ITOCC will then initiate the Technical SRT after the IMR informs the ITOCC who should be involved. The ITOCC is also responsible for reaching out to various external SMEs to engage them in the process. The ITOCC will also help with gathering SRT attendees, coordinating stakeholder engagement, and capturing all notes and tactical outputs from the Technical SRT. A single bridge will be opened for external SMEs to work with SRT members.

9.9 Incident Management in ITSM Tool

9.9.1 Who can submit an Incident in ITSM tool

9.9.2 When to submit an Incident in ITSM tool

9.9.3 Incident States in ITSM tool

10 Problem Management

IRS IEP Standard Operating Procedures Page 18

10.1 Overview

Problem Management is the process responsible for identifying and resolving underlying errors in the technical environment that lead to incidents. Problem Management attempts to eliminate these errors using Root Cause

Analysis (RCA) techniques to determine Root Cause and propose permanent fixes to them.

This section contains the roles and responsibilities associated to all problem management activities and will show when and steps to create problems in the ITSM Tool and the Problem Management Process Flow. It also includes rules and general information regarding Problem Management and the managing of problems in the

ITSM Tool.

10.1.1 What is a Problem

10.1.2 What is a Known Error

10.2 Roles and Responsibilities

10.3 Problem Management Governing Rules

10.4 Problem Identification

10.4.1 Evaluation and Identification of a Problem

10.4.2 Problem Management Creation Guideline

10.5 Problem Management High Level Process Flow

10.6 Problem Management Procedures

This section will provide step by step procedures on how to create a problem until resolution

IRS IEP Standard Operating Procedures Page 19

10.6.1 Problem Procedures for end user impacting Incident(s) where a Change is required for resolution

10.6.2 Problem Procedures for end user impacting Incident(s) where no Change is required for resolution

11 Knowledge Management

11.1 Overview

The purpose of Knowledge Management is to gather, analyze, store and share knowledge and information within the IEP organization. Creating an effective knowledge base can improve efficiency by reducing the need to rediscover knowledge, reduce agent response time and improve service quality.

This section contains the roles and responsibilities associated with all Knowledge Management activities and its process flow. It also includes rules and general information regarding Knowledge Management, how to create

Knowledge Base Articles and the managing of Knowledge Base Articles.

11.1.1 What is a Service Knowledge Management System (SKMS)

11.1.2 What is a Knowledge Base Article?

11.2 Knowledge Management Governing Rules

IRS IEP Standard Operating Procedures Page 20

11.3 Roles and Responsibilities

11.4 Knowledge Management High Level Process Flow

11.5 Knowledge Management Procedures

11.5.1 Creating a Knowledge Base Article

11.5.2 The Knowledge Coordinator reviews the content and determines if a SME needs to review the

Knowledge Base Article

11.5.3 Creating a Knowledge Base Article in KISAM if needed

11.6 Knowledge Management in ITSM Tool

11.6.1 Who can submit a Knowledge Base Article

11.6.2 Who can publish a Knowledge Base Article

11.6.3 Types of Knowledge Base Articles that can be created

11.6.4 Knowledge Workflow States

11.6.5 Maintaining Knowledge Base Articles

IRS IEP Standard Operating Procedures Page 21

12 Appendix

12.1 Appendix A – Acronym List

Acronym Definition

CI Configuration Item

CM Configuration Management

COTS Commercial Off-the-Shelf

CSC Computer Sciences Corporation

KISAM Enterprise Change Management System

FAQ Frequently Asked Question

GFE Government Furnished Equipment

IEP Integrated Enterprise Portal

IMR Incident Management Resource

IRS Internal Revenue Service

IT Information Technology

ITIL Information Technology Infrastructure Library

ITSM Information Technology Service Management

KBA Knowledge Base Article

KEDB Known Error Database

KISAM Knowledge, Incident/Problem, Service, Asset Management System

OL5081 Online 5081 Tool

O&M Operations and Maintenance

POC Point of Contact

IRS IEP Standard Operating Procedures Page 22

PROD Production Environment

PUP Public User Portal

P1 Priority 1

P2 Priority 2

P3 Priority 3

P4 Priority 4

RACI Responsible, Accountable, Consulted, Informed

RCA Root Cause Analysis

RUP Registered User Portal

SBX Sandbox Environment

SD Service Desk

SKMS Service Knowledge Management System

SME Subject Matter Expert

ITOCC Systems Operations Command Center Resource

SRT Service Restoration Team

TBD To Be Determined

12.2 Appendix B –Process Flows

12.3 Appendix C – Access Management ITSM Tool Instructions

12.3.1 Opening an Access Request (with ITSM tool access)

12.3.2 Viewing and Updating your Access Request

IRS IEP Standard Operating Procedures Page 23

IRS IEP Standard Operating Procedures Page 24

12.4 Appendix D – Incident Management ITSM Tool Instructions

12.4.1 Reporting an Incident

12.4.2 Viewing and Updating your Incident

12.5 Appendix E – Application Change Request Instructions

12.5.1 Opening an Application Change Request

12.6 Appendix F – Application Migration Request Instructions

12.6.1 Opening an Application Migration Request

12.6.2 Viewing and Updating your Application Migration Request

12.7 Appendix G – External Firewall Change Request Instructions

12.7.1 Opening an External Firewall Change Request

12.8 Appendix H – Instructions for OLS Requests

12.8.1 Open an OLS Request

IRS IEP Standard Operating Procedures Page 25

12.9 Appendix I- P1/P2 Emergency Change Request Process

12.9.1 Priority 1 (P1) Emergency Change Process

12.9.2 Priority 2 (P2) Emergency Change Process