Reading Library 7 - Legacy SOP_Problem Management_Redacted.pdf

DISA IT Enterprise Problem Management Standard Operating Procedure

Version 10.6 Page | iii

Table of Contents

1 Purpose

2 Scope

2.1 Objective

2.2 Critical Success Factors

3 Trigger

4 Procedures

4.1 Prerequisites

4.2 Define Problem

4.3 Detect Problem

4.4 Log Problem

4.4.1 Log a New Problem Record

4.4.2 Update a Current Problem

4.5 Categorize Problem

4.6 Prioritize Problem

4.7 Investigation and Diagnosis

4.8 Workaround Decision

4.9 Known Error

4.10 Create Known Error Record

4.11 Resolve Known Error

4.12 Resolve Problem

5 PbM Reporting

5.1 PbM SE6 Top Issues Report Template

Appendix A: Roles and Responsibilities

Appendix B: Acronyms

Appendix C: High Level Process Flow

Appendix D: Fields and Utilization Requirements

His al

Refe ren ce D oc um en t

Version 10.6 Page | iv

Appendix E: Related Documents/Links

Appendix F: References

Authorizing Signatures

His al

Refe ren ce D oc um

Version 10.6 Page | v

List of Tables Table 1: Critical Success Factors Table 2: Roles and Responsibilities Table 3: Acronyms Table 4: Problem Record Table 5: Known Error Record

Refe ren ce D oc um

Version 10.6 Page | 2

4 Procedures

4.1 Prerequisites

1. Unresolved incident ticket(s) with no resolution from Known Error Database (KEDB).

4.2 Define Problem

A problem is defined by IT Infrastructure Library (V3) as a cause of one or more incidents:

• Is the service unusable?

• Is there a degradation of the service?

• Is the business process affected greatly?

• Are service levels breached?

• Is it a network device that provides service to multiple users?

If the answer is yes to any of these questions, then it is an incident. As such, the incident is to be handled and resolved in accordance with the Incident Management (IM) process and procedural workflow, which establishes the framework to restore normal function or service operation as quickly as possible. Further investigation and documen ation by IM is required prior to submitting a Problem Request.

PbM is the process of analyzing related incidents identifying patterns of faults, and garnering the appropriate resources to come up with workarounds or solutions. A problem record is opened only when the cause of the multiple in idents is unknown or to document the workaround. If the cause of multiple incidents is documented in the KEDB, no problem record is required and tickets will be assigned to appropria e Tier II/Tier III groups.

4.3 Detect Problem

Potential problems are detected in one of the following ways:

• In analyzing problem requests generated by the DSMC, PbM will accept the problem request for investigation if 50 or more DISANet users are impacted and a known error is not documented in the KEDB.

• In analyzing incidents captured via ITSM, PbM creates problem records for incidents that impact 50 or more DISANet users per the weekly Trend Analysis reporting period for which a known error is not documented in the KEDB.

4 4 Log Problem

PbM Team personnel:

1. Create problem record in ITSM.

His al

Refe ren ce D oc um

Version 10.6 Page | 3

2. Verify that incidents are being related to the problem record.

3. Ensure ticket updates exclude classified information.

4. Update the problem record daily.

5. Assign and track the PbM ticket to the appropriate team working the issue.

6. Correlate relationship to Configuration Items (CIs).

7. Link to related Known Errors and Workarounds, if applicable.

8. Link to Change Request/Change Records.

4.4.1 Log a New Problem Record

When an infrastructure outage occurs that causes a service to become unavailable, or multiple related incidents occur, the Problem Owner creates a problem record in ITSM Remedy and logs the following information:

1. Coordinator Group*+

2. Problem Location

3. Summary*

4. Investigation Driver*

5. Target Date

4.4.2 Update a Current Problem

Adds a Notes entry in the work detail, every time:

• Anyone works on the record.

• Status changes occur.

• Work detail must be updated daily or as needed.

4.5 Categorize Problem

Problem Owner:

1. Fill in the Operational Categorization Tier I*+ field. This shall be for Authorized Scheduled Interruptions or “Failure” for unplanned Infrastructure Outages that have caused or may cause the unavailability of a service.

2 Fill in the Operational Categorization Tier II* field.

3 Fill in the Operational Categorization Tier III* field.

4. Fill in the Product Categorization Tier I* field.

5. Fill in the Product Categorization Tier II* field.

6. Fill in the Product Categorization Tier III* field.

7. Fill in the Product name* field.

His al

Refe ren ce D oc um

Version 10.6 Page | 4

8. Fill in the Model/Version field.

9. Fill in the Manufacturer field.

10. Fill in the Reproducible field.

4.6 Prioritize Problem

The Problem Owner assigns the Urgency and Impact of the problem according to DISA Process Guide 450 Rev-1 guidelines. The severity of problems must be examined, which requires an assessment of the degree of impact to the end-users.

4.7 Investigation and Diagnosis

Problem Owner:

1. Complete the Assigned Group+.

2. Save the Problem Record.

3. Notify the Global Service Desk (GSD) that an open problem record exists and provide the problem ID, so Tier I technicians can immediately begin relating incident records to the problem record.

4. Call the Assigned Group+ manager and advises hem that a problem exists. The Assigned Group+ manager assigns the problem record to a Problem Analyst and works the problem through to resolution.

5. Communicate problem status to SE6 Branch Chiefs, via the SE6 DISANet Top Issues Reports, weekly OPS PbM Status reports, email, and technical working group meetings, as required.

PbM events generally require analysis of a pervasive problem in the environment to determine the root cause. Once the Root Cause Analysis (RCA) is complete, a solution is proposed. If the solution requires a higher level of engineering expertise than what is provided for by SD functions, the PbM ticket is closed and an Engineering Action Request is created for that proposed solution referencing that prior PbM ticket.

Problem Analyst:

1. Perform RCA, seeking a permanent solution or improved workaround to the problem. A workaround is a stopgap action and does not fix the underlying root cause of an incident, but provides a solution for service to be restored to the customer.

2. Update the Notes under the Summary* field to reflect the current status of the problem investigation.

3. Update the Notes under the work detail tab, under the following circumstances:

a. Anytime the technician attempts to resolve the problem (e.g., “Verified SCCM was pushing patches properly”).

His al

Refe ren ce D oc um

Version 10.6 Page | 5

4.8 Workaround Decision

If a workaround exists, the Problem Analyst moves the status to “Update Knowledgebase and Inform GSD” and describes it in the workaround field of the problem investigation record.

If a work around does not exist, the status is moved to “Create a Known Error Record.”

4.9 Known Error

If PbM agrees that a change is the best way to resolve the root cause of the problem, initiate the change by creating a known error and assigning it appropriate team as defined in the Change Management process.

4.10 Create Known Error Record

Problem Owner:

1. Open the relevant problem investigation.

2. Set the “Status” field to Completed and the “Status Reason” field to Known Error.

3. Click Save.

4. Click Yes when prompted by the system. This opens the “Known Error” form and creates a relationship between the Known Error and the problem investigation.

5. In the “Notes” field, enter a brief description of the Known Error and a detailed description of the change requirements

6. Select the impact and urgency; the priority and weight are calculated based on the impact and urgency.

7. Select the client company

8. Select the appropriate operational categorization.

9. Select the appropriate product categorization.

10. Click the Relationship tab.

11. Ensure the Known Error is related to all the affected service infrastructure and to the CI in which the problem resides

12. Ensure the Status is set to Assigned.

13. Click Save.

4.11 Resolve Known Error

Problem Analyst:

1. Work the “Known Error” record using the process described in DISA Process Guide 450 Rev-1.

2. Add a Notes entry under the “Work Detail” tab every time the Known Error is worked.

His al

Refe ren ce D oc um

Version 10.6 Page | 6

4.12 Resolve Problem

Problem Manager:

1. Refer to DISA Process Guide 450 Rev-1 for resolution actions.

2. Add a Notes entry under the “Work Detail” tab every time the problem is worked.

5 PbM Reporting

The PbM Team opens and maintains investigation logs for each active SE6 PbM investigation.

The logs are located on the DoD Enterprise Portal Service (DEPS) website at the following location:

https://disa.deps.mil/org/SE6/OPSCNT/C/Lists/Problem%20Management%20Report/Open.aspx

The investigation logs are updated every Tuesday and Thursday. When updated, an email notice is submitted to the DSMC, and is included in the SE6 Top Issues Reports. The updates reflect the status change of problems categorized as Critical and High over the previous 24 hours. If the PbM report requires an immediate update, contains errors, or an update is missed, an email notification informing the stakeholders of the status change is sent immediately.

5.1 PbM SE6 Top Issues Report Template

The PbM SE6 Top Issues Report template is shown below:

[Insert Problem Ticket Number] Problem Summary: [Insert Summary in Remedy ITSM] Status: [Insert Status* from Remedy ITSM] Priority: [Insert Priority from Remedy ITSM] Assigned Group: [Insert Assigned Group] Work Detail: [Insert Update Date] [Insert Most Recent Work Detail Entry from Remedy ITSM] [Insert Update Date] [Insert Second most recent work detail entry from Remedy

ITSM]

Refe ren ce D oc um

Version 10.6 Page | 9

Appendix C: High Level Process Flow ist ori ca l R efe ren ce D oc um

Version 10.6 Page | 25

Appendix E: Related Documents/Links

1. Problem Management Investigation and Reporting:

https://disa.deps.mil/org/SE6/OPSCNT/C/Lists/Problem%20Management%20Report/Op en.aspx

Refe ren ce D oc um

Version 10.6 Page | 26

Appendix F: References

1. Defense Information Security Agency (DISA). DISA Process Guide 450: Problem Management Process Guide.

2. Department of Defense (DoD). DoD Instruction No. 1253, Committee on National Security Systems (CNSS), Security Categorization and Control Selection for National Security Systems. March 2014.

3. National Institutes of Standards and Technology (NIST). NIST Special Publication (SP) 800-53 Security and Privacy Controls for Federal Information Systems and Organizations, Revision 4. Joint Task Force Initiative. April 2013.

4. National Institutes of Standards and Technology (NIST). NIST Special Publication (SP) 800-37 Guide for Applying the Risk Management Framework to Federal Information Systems: A Security Life Cycle Approach, Revision 1. Joint Task Force Initiative.

February 2010.

Refe ren ce D oc um