(U) FS2_INC_Notfication_Process.pdf
PDF 310 KB Posted
- Solicitation number
- HM047625R0016_Bidder_Library
About this file
This document is a presentation describing the FS3i Incident Notification and Corrective (INC) Process for the National Geospatial-Intelligence Agency (NGA). The presentation outlines procedural updates for handling Priority 1 (Critical) and Priority 2 (High) incidents, establishing an IPF Notification List, defining notification frequency requirements, and implementing a Root Cause Analysis (RCA) and Lessons Learned (LL) template for incident resolution.
The INC process follows a three-tier escalation structure. Users submit incidents via phone or web-based portal to Tier 1 (ESC), which attempts resolution through standard troubleshooting. Unresolved incidents escalate to Tier 2 (FS3i OPS Team), where immediate notification to GPMO and key stakeholders is required for Priority 1-2 incidents, with notification at the next daily ops meeting for Priority 3-4 incidents. The contract mandates 24-hour resolution time for Priority 1 incidents and 72-hour resolution time for Priority 2 incidents. Unresolved Tier 2 cases proceed to Tier 3 (Adaptive Maintenance/Development Support), where Priority 1-2 incidents trigger E-fix processes for hotfix development and deployment, while Priority 3-4 incidents generate development tickets for backlog prioritization. For enterprise-level issues, incidents are reassigned to applicable enterprise teams with FS3i OPS providing ongoing follow-up support. Priority 1-2 incidents receive minimum twice-daily email updates to stakeholder distribution lists (AM prior to 0900 ET ops meeting and PM prior to 1800 ET), with additional updates sent for major status changes. INC notes are updated minimum twice daily or upon completion of major troubleshooting steps. Following INC downgrade or closure, the FS3i OPS Team conducts RCA/LL meetings within three business days to document root causes, corrective actions, lessons learned, process improvements, and assigned action items, with results shared to originally notified stakeholders and added to a master Priority 1-2 INC list maintained on FS3i SharePoint.
View the file
Other files
Show 40 more
50 files listed here. See the rest on GovTribe.
On GovTribe
Work with this file
- Download the original file
- Contacts named in this file
- Similar government files
- Ask GovTribe AI about this file
Text version
The overall classification of this presentation is:
UNCLASSIFIED
UNCLASSIFIED
FS3i
IPF INC Notification Process
Contents
- Updated INC Flow Chart
- Amended Process for Pri 1 (Critical) and Pri 2 (High) INC process
- Creation of IPF Notification List
- Frequency of INC updates and email notifications
- Root Cause Analysis (RCA) and Lessons Learned (LL) Template
• UNCLASSIFIED
• UNCLASSIFIED
User
Submits INC via phone call to Customer ESC or web-based self-service portal
Tier 1
(ESC)
Resolve?
INC Updated and Closed
Yes
INC Sent to FS3i Tier 2 OPS Team or new INC discovered by OPS Team
No
Tier 2 Resolve
Priority 1-2 Priority 3-4
Troubleshooting via system knowledge, troubleshooting guides, SOPs;
break-fix process (if needed);
Systems Team called (if needed)
Immediate notification to
GPMO and key stakeholders
(see the IPF INC
Notification List for applicable system on SharePoint)
Notification to
GPMO at next daily OPS
Status
Meeting
Troubleshooting via system knowledge, troubleshooting guides, SOPs
Yes No
Tier 3 Adaptive Maintenance/Development
Support Notified
Priority 1-2 Priority 3-4
Dev ticket written; E-fix
Process initiated to develop and deploy hotfix
Dev ticket written; Ref INC # in Dev ticket and Dev # in INC;
Dev ticket goes on backlog for prioritization
INC Updated and Closed
Hotfix tested, deployed to OPS and validated
O&S Corrective Maintenance and
INC Resolution Process
Note: Contract mandates 24 hour resolution time for Pri 1 INCs and 72 hour resolution time for Pri 2 INCs.
INC opened against or reassigned to applicable enterprise team. Work with SEIN and IPF Liaison where necessary to assign to correct team.
User or FS3i OPS Team discovers Enterprise
Issue impacting IPF systems
FS3i OPS Team will continually follow-up with
Enterprise Team and provide troubleshooting support when needed.
INC Updated and Closed
Status given at daily OPS Status Meeting until resolved
Determine if code issue or enterprise issue
Enterprise FS3i Dev/Adaptive Maintenance.
No Bueno
Recommended
Notification & Updates after Pri 1 (Critical) and Pri 2 (High) INC logged
- When a Pri 1 or Pri 2 is logged (or a Pri 3 INC is raised in priority), immediate notification via high side email will be sent to key stakeholders in IPF
- A list of stakeholders for each applicable system is listed on the FS3i SharePoint site (sample below for a subset of systems)
- Responsibility falls to FS3i OPS Manager or delegate
- FS3i PMO will identify any additional stakeholders
- Email updates to the same distro list will continue to be sent out at minimum twice per day (AM update prior to the
0900 ET OPS Status meeting and PM update prior to 1800 ET)
- Emails will also be sent out if there is a major update or change in status (at discretion of the FS3i OPS Manager or delegate)
- INCs will be updated in the “Notes” tab at minimum twice per day or as any major step is completed in the troubleshooting or resolution process
- Stakeholders can receive these additional updates by clicking “follow” within the INC in Service+
- Updates will also be given at the 0900 Daily OPS Briefing
UNCLASSIFIED
UNCLASSIFIED
Maritime DNC/ENC Maritime TOD Boundaries
Joe Minus Joe Minus Joe Minus
Glen Neises Glen Neises Glen Neises
John Grimes John Grimes John Grimes
Len Pedowitz Len Pedowitz Len Pedowitz
Major OB Major OB Captain Meoli
Frank Chamberlain Frank Chamberlain Lloyd Weber
Scott Reeves Scott Reeves #NGA SFGGB GOV
Drusilla Morgan Natalie Martin
Jenna Johnson Kat Furry
Tom Verna Tom Verna Tom Verna
Scott Alcusky Scott Alcusky Scott Alcusky
#NGA FGMOD Support #NGA FGMOD Support #NGA FGMOD Support
Stacey Herring Stacey Herring Stacey Herring
PMO
IPF GOV
FGS-IOS
Root Cause Analysis/Lessons Learned Process and Template
- Once the INC is downgraded or closed, the FS3i OPS Team will hold an internal Root Cause Analysis (RCA) and
Lessons Learned (LL) meeting
- Root Cause Analysis – What happened and why?
- Corrective Actions – What did we do to fix it?
- Lessons Learned – How do we prevent it or resolve quicker in the future?
- Opportunity for Improvement – Process/Documents updates?
- Action Items – Who put the changes into place?
- RCA/LL Meeting will occur within 3 business days of INC downgrade or closure unless prevented by other events
- Notes will be sent to the same stakeholders originally notified of the INC
- Action Items discussed in the RCA/LL will be assigned if needed
- The INC will also be added to a master list of Priority 1 and 2 INCs that will be created as part of this process
- It will be found on the FS3i SharePoint site and will be separate from the full INC kept by the FS3i OPS Team
- It will include INC number, brief description, date opened, date upgraded to Pri 1 or 2, date downgraded, date closed, and link to the RCA/LL Notes on SharePoint
Root Cause Analysis/Lessons Learned Template (Page 1 of 2)
UNCLASSIFIED
UNCLASSIFIED
STEPS NOTES
ROOT CAUSE ANALYSIS In this box, please describe the issue that occurred. If applicable, identify direct cause, but also latent causes as well. When was the issue noticed, by whom, and how long was it present on the IPF system in question?
CORRECTIVE ACTIONS What did we do to resolve the issue? When was the issue resolved?
Root Cause Analysis/Lessons Learned Template (Page 2 of 2)
UNCLASSIFIED
UNCLASSIFIED
Lesson Learned Opportunity For Improvement Action Item and Assignee Example 1:
Not all critical drives on IPF systems are monitored by the OPS Team
Example 1:
We can use our current tools, such as Grafana and Cloud Watch, to set up monitoring on drives that currently have little to no OPS Team insight
Example 1:
• Action to Bill Denbrow to audit all drives and list ones not monitored
• Action to Richie Tozier to add drive monitoring to Grafana and CloudWatch
Example 2:
User profiles for FG MOD Maritime DNC get too big
Example 2:
Document a process for cleaning up user profiles
Example 2:
• Action to Beverly Marsh to write the profile cleanup procedure and add it to the OPS repository on Confluence
| Slide 1 |
| Slide 2: Contents |
| Slide 3 |
| Slide 4: Notification & Updates after Pri 1 (Critical) and Pri 2 (High) INC logged |
| Slide 5: Root Cause Analysis/Lessons Learned Process and Template |
| Slide 6: Root Cause Analysis/Lessons Learned Template (Page 1 of 2) |
| Slide 7: Root Cause Analysis/Lessons Learned Template (Page 2 of 2) |
File details come from the government source that posted it. Updated .