Reading Library 4 - Legacy Process_After Hours Outage Notification_Redacted.pdf

After Hours Outage Notification Process The DSMC monitors numerous devices throughout the DISANet Enterprise that include: circuits, switches, and servers on a 24x7x365 global scale. The DSMC tracks latency on the DISANet Network and connectivity status for Internet Control Message Protocol (ICMP, Simple Network Management Protocol (SNMP), and Windows Management Instrumentation (WMI).

1 Reporting

The DSMC is responsible for detecting and reporting outages, potential outage symptoms, and keeping the various tiers and teams updated on the status of these occurrences. They also provide daily reports on incidents and problems, including status and statistics, occurring on the network to senior operations staff. All DSMC activities must be documented within th Master Station Log (MSL) by the DSMC technicians. All outage reporting and notifications must be verified by 2 onsite staff members prior to distribution; the reports must include 2 initials by the staff member that viewed the reporting.

The DSMC will send out outage reports after the initial occurrence and every 4 hours thereafter through resolution for ALL Major Incidents or Priority 1 (P1), and every 24 hours for all Priority 2 (P2) outages, until the FINAL. Unless directed otherwise from DSMC Lead or the Chameleon Program Manager.

Once the outage is reported, the DSMC shall follow the following process:

1. Obtain verification of outage from at least one of the following: Site Admin (SA), Point of Contact (POC), or Tier III Server/Infrastructure, and validate that 10 + end users are being affected.

2. Open Remedy Ticket by logging into the following URL:

https://webitsm.csd disa.mil/arsys/forms/itsmapp1.csd.disa.mil/SHR:LandingConsole/Defa ult+Administrator+View/?cacheid=a9df13ab

3. Login to MSL using the below link:

https://disa.deps.mil/org/SE6/OPSCNT/C/SitePages/MSL.aspx

4. Send the Outage Notification from the DISA Ft Meade SE Mailbox SE6B Systems Monitoring Center mailbox using “Outage” template.

5. Ent r outage information into Reason For Outage (RFO) tracker at the following link: https://disa.deps.mil/org/SE6/OPSCNT/C/SitePages/RFO%20Tracker.aspx

Hist ori ca l R efe ren ce

D oc um en t

Hist ori ca l R efe ren ce D oc um en t

NOTE: All after-hour P1 or widespread outages that interrupt normal operations must be reported to SE6 and Chameleon leadership:

1. DSMC Lead contacted by DSMC Technician

2. Watch Officer contacted by DSMC Technician

3. Tier III Lead contacted by DSMC Lead

4. Program Manager contacted by DSMC Lead or Tier III Lead

Once the outage has been reported to Chameleon leadership, a Government POC must be notified.

A Watch Bill list is provided by the Government to show who is on-call. It is updated quarterly and posted on DEPS website and DSMC One Note Page. During the call to the Watch Bill manager, DSMC/Tier III support are required to provide all details of the outages and any additional vendor support needed.

2 Tier III Server/Infrastructure

When actively working and troubleshooting issues After Hours, the Tier III technician should be updating the Remedy Ticket and actively communicating with DSMC to ensure the appropriate information is being provided for the outage and in a timely manner. After the issue has been worked for an hour or more and no progress or resolution of issue has not been achieved, the Tier III technician is to call the Tier III Lead with an update and next steps plan. This may result in the need for additional resources.

3 Rapid Response

As part of an outage/outbreak related to Network Security affecting any site during HQ (Ft. Meade) core hours of 0700-1600 or After Hours, the DSMC will notify the DISA Ft Meade SE List SE6 Rapid Response Team distribution ist and open the following bridge:

Dial-In Number: 301-909 7350 Dial-In DSN Number: 312-723-7350 Access Code: 34721224

This bridge will allow for uniform communication and coordination of course of action. The DSMC will also call the Rapid Response POC, .

Acknowledgement:

I acknowledge I have read and understand the above After-Hours Process.

Name Date

Hist ori ca l R efe ren ce D oc um en t

Hist ori ca l R efe ren ce D oc um en t