Attachment_3_-_Draft_Task_Order_2_PWS_O&M.pdf

National Environmental Satellite, Data, and Information Service (NESDIS)

Office of Satellite and Product Operations (OSPO) and

Office of Satellite Ground Services (OSGS)

Draft Operations, Maintenance, and Sustainment (OMS) Task Order 2

Performance Work Statement (PWS)

August 5, 2019

DRAFT OPERATIONS, MAINTENANCE, AND SUSTAINMENT

TASK ORDER 2 PERFORMANCE WORK STATEMENT -

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 2

1.0 Overview 4

2.0 Scope 4

2.1 Place of Performance 5

3.0 Requirements 5

3.1 Program Management 6

3.2 Contract Transition 6

3.3 Operations 6

3.3.1 Operations Support 6

3.3.1.1. Operations Training Support 7

3.3.1.2. Meetings & Data Call Support 8

3.3.1.3. Satellite Launch and Special Activity Support 8

3.3.2 Help Desk Service to Operations 8

3.3.2.1 Hours of Operations 9

3.3.2.2 Help Desk Activities 9

3.3.2.3 Operations Electronic Call Tracking and Reporting 13

3.3.3 Systems Analysis and Resolution Support 13

3.3.3.1 Operational and Back-up Sites 13

3.3.4 Workstation/Desktop/Hardware Support for Operational Workstations/Printers

3.3.5 Data Center Infrastructure Management 14

3.3.5.1 Facilities Change Configuration Board (CCB) Report 14

3.3.5.2 Cable Plant Management 15

3.3.6 Antenna Operations Support 15

3.3.7 Network and Security Operations 16

3.3.7.1. Wide Area Network (WAN) and Local Area Network (LAN) Support 16

3.3.7.2. Security Configuration Management 17

3.3.7.3. Security Data Analysis Services 17

3.3.7.4. Communications Security (COMSEC) Key Loading and Technical Support

3.3.8 Continuity of Operations (COOP) 19

3. 4 Maintenance 19

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 3

3.4.1 Network & Systems Administration 19

3.4.1.1 Networks Fault Management 20

3.4.1.2 Networks Account Administration Management 21

3.4.1.3 Networks Performance Management 22

3.4.1.4 Networks Security Management 23

3.4.1.5 System Administration and User Support 24

3.4.1.6. System Administration Fault Management 25

3.4.1.7 System Administration Performance Management 25

3.4.1.8 System Administration Security Performance 25

3.4.1.9 IT System Analysis 26

3.4.1.10 Capacity Planning 26

3.4.1.11 Telecommunications Support (NSOF and OSPO Phone Systems) 27

3.4.2 Preventive Maintenance 27

3.4.2.1 Replacement Parts Modifications and Upgrades 27

3.4.2.2 Operational Equipment 27

3.4.2.3 Obsolescence Prevention and Remediation 28

3.4.3 Installation, De-installation, and Relocation 28

3.4.4 IT Asset Management 28

3.4.4.1 System Hardware and Software Inventory 28

3.4.4.2 Hardware Procurement Support 30

3.4.4.3 Software and Software License Procurement and Maintenance Support 30

3.4.4.4 Software License Management 30

3.4.4.5 Shipping and Receiving Equipment 31

3.4.4.6 Property Contact 31

3.4.5 Antenna Maintenance 32

3.4.6 Information & Cyber Security Support 34

4.0 Performance Requirements Summary 34

Appendix A. Deliverables 35 Appendix B. Acronym Listing 39

Source Selection Information – See FAR 2.101 and 3.104-4 4

1.0 Overview

The Office of Satellite and Product Operations (OSPO) and the Office of Satellite Ground Services (OSGS) have shared responsibilities in managing legacy satellite ground systems and product processing systems. This Task Order 2 Performance Work Statement (PWS) specifies the operations and maintenance requirements needed to support the ground systems.

2.0 Scope

Table 1.1-1 shows the Government ground systems to be supported under Task Order 2 and the type of support required (i.e. operations only or operations and maintenance). These systems are referred to as “assigned systems” herein.

Table 1.1-1 NESDIS Assigned Systems Under Task Order 2 PWS Operations Maintenance

GOES Legacy Geostationary Operational Environmental Satellites NOP Ground System

Y Y

DSCOVR Space Weather Ground System (Deep Space Climate Observatory (DSCOVR)) is an independent system that is a part of the GOES NOP system security boundary. It may become a security boundary that contains only DSCOVR at some future date.)

Y Y

ESPC Environmental Satellite Processing Center Y Y

NJGS NOAA Jason Ground System Y Y

GOES R Series Geostationary Operational Environmental Satellites Ground System

Y N

JPSS Joint Polar Satellite System Ground System Y N

POES Legacy Polar-orbiting Operational Environmental Satellite Ground System

Y Y

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 5

DMSP

Non-NOAA

Defense Meteorological Satellite Program (NOAA Operates this DOD Satellite)

Y Y

MSL Mission Support Local Area Network (LAN) Y Y

A detailed list of NESDIS Assigned Systems is provided in Appendix A of the IDIQ PWS, NESDIS System Boundary General Correspondence to NESDIS Missions/Functions. Individual system inventories, drawings, descriptions and interfaces are available in the OMS Document Library.

2.1 Place of Performance

The primary work locations for this task order are:

● NOAA Satellite Operations Facility (NSOF) located in Suitland, MD

● OSGS Program Offices located in Greenbelt, MD and Silver Spring, MD

● NOAA Center for Weather and Climate Prediction (NCWCP) located in College Park, MD

● The NESDIS Consolidated Backup Unit (CBU) located in Fairmont, WV

● NESDIS Command and Data Acquisition (CDA) Stations located in Wallops, VA;

Fairbanks, AK; and Barrow, AK

Remote equipment locations are:

● USAF Environmental Satellite Operations Center (ESOC) located at Schriever Air Force Base, Colorado Springs, CO

● Remote Ground System (RGS) location Dongara, AUS, and other associated sites in Australia.

● Lockheed Martin Facility, Sunnyvale, CA

● NASA’s Goddard Space Flight Center located in Greenbelt, MD

3.0 Requirements

The Contractor shall:

● Provide the required subject matter expertise, personnel, material, equipment, services, and facilities to perform and deliver operations and maintenance support for the Assigned Systems with no interruption of services. Performance requirements are per Appendix 4.0 of this PWS.

● Perform operations and maintenance activities at all work sites.

● Perform operations and maintenance activities for Assigned Systems listed in Table 1.1-

1, as well as the interfaces to Assigned Systems listed in Tables 1.1-2 of the OMS IDIQ

PWS.

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 6

● Provide qualified personnel with professional and technical expertise commensurate with their responsibilities, and any certifications in accordance with IDIQ Appendix D Labor Categories Description Table, if required.

3.1 Program Management

Program management related requirements for Task Order 2 activities are specified under Task Order 01.

3.2 Contract Transition

Transition related requirements for Task Order 2 activities are specified under Task Order 01.

3.3 Operations

3.3.1 Operations Support

The Contractor shall provide all processes and resources to implement and deliver products and services supporting the efficient and effective operations of NOAA mission critical Assigned Systems.

PWS-TO2-OP01: The Contractor shall provide all required resources for performing operational activities to include:

● Documentation updates of requirements, planning events, system configuration, and operational procedures [Deliverable]

● Provide accurate daily, weekly, and monthly Assigned Systems status reports to the Government [Deliverable]

● Support operational activities for ingest, archival, product generation (PG), and dissemination of datasets

● Detect, identify, and assess system anomalies

● Provide notifications to stakeholders and users (e.g., for outages, etc.)

● Support activities and agreements for system components and services

● Ensure the secure operational environments for the assigned systems

● Establish, monitor, report, and tracking metrics per section 4.0.

PWS-TO2-OP02: The Contractor shall perform operator support for system components. This typical support and services includes:

● Monitor and report status and health of system components daily, weekdays.

● Monitor and report system workflow and resource management daily, weekdays.

● Detect, analyze, and resolve system component or service problems and discrepancies

● Resolve problems in command and control systems as soon as possible, if it affects the ability to control and monitor the spacecraft.

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 7

● Resolve problems for data, product applications, systems software, and networking systems

● Report system problems and discrepancies as soon as detected;

● Report mission critical equipment repairs in no more than 15 minutes during business hours and in 2 hours during non-business hours,

● Report upgrades and related activities and provide a schedule to stakeholders and users for the activity [Deliverable]

● Implement system updates, optimizations, and upgrades to operational components and services, in coordination with the Government, as approved by the Contracting Officer Representative (COR). Operational components are defined as the hardware, software, communication devices, operating systems, documentation, licenses, and agreements necessary to support operational services.

● Coordination of corrective maintenance activities

● Support integration and test (I&T)

● Support software reviews and testing to include set up and configuring test environments, verify configuration for the test, and verify that modifications provide the intended functionality

● Respond to system issues on a 24-hour, year-round basis. The response time for mission critical problems shall not exceed 15 minutes during business hours and within 2 hours during other times.

● Provide technical support to resolve software problems documented in operations incident reports.

● Plan response for critical weather days

● Provide immediate backup capability for operational systems

PWS-TO2-OP03: The Contractor shall coordinate technical issues between the Government and Contractor support teams, and across internal and external stakeholder organizations.

PWS-TO2-OP04: The Contractor shall provide status reports on assigned system performance to the Government as scheduled (daily, weekly, and monthly) and fully coordinate on testing, maintenance and release of upgrades, anomaly investigations, and implementing modifications to operational systems [Deliverable]

PWS-TO2-OP05. The Contractor shall provide support to meet system operational availability requirements. Operational availability and notification requirements are listed in the OSPO Incident Policy.

3.3.1.1. Operations Training Support

PWS-TO2-OP06: The Contractor shall:

● Provide training, as needed, to staff on functionality and execution of operational systems (including anomaly, contingency, and recovery)

● Provide current training material for operations and maintenance for all assigned systems [Deliverable]

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 8

● Maintain a training log by employee, and provide a training status during program review meetings (submitted monthly with Program Review Status Package) [Deliverable]

3.3.1.2. Meetings & Data Call Support

PWS-TO2-OP07: The Contractor shall participate in and facilitate the meetings shown below, and record meeting minutes and action items for designated operational meetings at the Government’s request.

● Operations stand up (Daily: Monday - Friday)

● Helpdesk stand up and ticket review (Daily: Monday - Friday)

● Contract status (Weekly)

● System Support Coordination Meetings (Weekly)

● Plan of Action and Milestones (POA&M) (Weekly)

● Executive Review Board (Monthly)

● Web meeting (Monthly)

● Help Desk Working Group (Bi-weekly)

PWS-TO2-OP08: The Contractor shall document meeting minutes, action items, and lessons learned to the Government for the following meetings: Contract Status, Configuration Control Board (CCB), technical working groups, Executive Review Board, Web, and Help Desk Working Group within 3 days of the event [Deliverable].

PWS-TO2-OP09: The Contractor shall respond to Data Call support requests including organizational and technical queries. The schedule or the completion of these requests is on a case by case basis.

3.3.1.3. Satellite Launch and Special Activity Support

PWS-TO2-OP10: The Contractor shall respond to Government request for satellite launch and special activity support, to include:

● Providing on-site maintenance support 24 hours per day x 7 days per week for specific launch operations and special events (to include subject matter expertise and number of personnel on each shift requested by the Government). Typical coverage begins 1-3 days prior to launch and lasts 1-3 days after launch.

● Loading and maintaining antenna systems tracking controllers with ephemeris data (provided by spacecraft navigators) during and after satellite maneuvers.

PWS-TO2-OP11: The Contractor shall provide and maintain a current staffing list and update at least monthly with changes (including name, company, hire date and where applicable, departure date) [Deliverable].

3.3.2 Help Desk Service to Operations

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 9

The Help Desk provides numerous services including: network workstation configuration support, general desktop troubleshooting of hardware and software problems for approximately 625 mission system users (600 users at NSOF and 25 at NCWCP) and nine (9) very important person (VIP) users. The Help Desk staff coordinates the receipt and dissemination of calls, emails, and help desk tickets and distributes according to priority and expertise.

3.3.2.1 Hours of Operations

PWS-TO2-OP12. The Contractor shall provide help desk support coverage 12 hours per day x 5 days per week (M-F 6am-6pm) to support the daily call, email, and help desk ticket volume estimates shown in Table 3.3.2-1

Table 3.3.2-1. Help Desk Ticket Volume

Daily Average Annual Calls 25 9125 E-mails 20 7300 Help Desk tickets 6 2400

3.3.2.2 Help Desk Activities

PWS-TO2-OP13 - The Contractor shall provide the OSPO Mission Support Services (MSS) at NSOF and NCWCP Offices with an operational help desk team to perform the functions below:

● Receive all initial customer service requests for mission network services and supported desktop users via telephone, email, voicemail, and online.

● Respond to customer service requests via telephone, through desktop consultation with users, and/or by using remote management software.

● Provide direct desktop support, once all information is gathered and telephone consultation proves inadequate to resolve the user’s problem.

● Close service requests with detailed information of the resolution documented in the help desk tracking system.

● Conduct quarterly surveys to determine customer satisfaction.

● Operate and maintain the help desk tracking system, to include collecting and maintaining user profile data, and creation of reports.

● Provide assistance with Mission Support applications, such as Enterprise Configuration

Management Tool (ECMT), SharePoint, Citrix, Remote Desktop, One Way Links (OWL), and all services running on the Mission Support LAN.

● Facilitate issues if users’ contact the help desk for Mission Support Services, such as phone services, cable plant, communication room access, badge access, etc. The Help Desk coordinates these requests with the appropriate staff as needed.

● Provide basic telephone support such as connecting telephones, troubleshooting partially connected phones in the computer, end user support related to voicemail setup, etc.

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 10

● Transfer NESDIS Administrative LAN requests to the NSDesk.

● Coverage for Help Desk Support:

○ MSS Help Desk shall be covered from Monday - Friday, 6:00 AM to 6:00 PM EST, including lunch time hours (normal work hours).

○ All staff shall assist in providing coverage of the work queues.

○ All staff shall monitor and perform help desk tickets.

○ All staff shall perform patching of all Mission Support LAN systems

(approximately 60 servers and 100 workstations) during a weekly after-hours maintenance period and on emergency on-call basis.

PWS-TO2-OP14 - The Contractor shall support Tiered activities and provide Tier 1, Tier 2, and Tier 3 troubleshooting support, as defined below:

● MSS Tier 1 trouble call types will be those calls for assistance that can be resolved over the phone or by email by the Contractor staff.

● Tier 2 trouble calls will be identified as those issues that require a support person to meet with the user and/or additional hours to resolve. This typically requires a physical interaction with the user and desktop workstation/devices needing service.

● Tier 3 trouble calls will be those instances where there is a need for systems and application support expertise to resolve.

PWS-TO2-OP15 - The Contractor shall adjust appropriate Tier level support in accordance with any future NOAA mandates and service provisions.

● The Contractor shall provide support for MSS Tier 1:

○ Unified Messaging Services (UMS) - Provide a day-to-day Point of Contact (POC) between users, IT services, UMS Contractors, NOAA IT services Contractors, and third party support organizations. The POC provides advice and guidance, and may also be involved in providing a rapid restoration of normal services to users following any service disruption.

○ Ticketing - Answer calls, assign and track help tickets/incident reports using Government-approved automated systems.

○ Call Answering/Response - Answer telephone, voice mail, and emailed requests to the Mission Support LAN help desk during normal work hours.

○ General POC for IT Service - Support role as the initial point of service for all IT support calls at NSOF and NCWCP.

○ Active Directory (AD) Services - Provide AD Authentication Services, password resets, etc.

○ Knowledge Base - Maintain comprehensive knowledge base within the OSPO CM tool of information gathered from solving previous trouble calls, vendor-provided trouble-shooting solutions. The knowledge base is utilized in assisting users during support phone calls.

○ Call Escalation - Escalate calls, when appropriate.

○ Call Tracking:

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 11

- Capture, through an automated tracking system, IT support call open date and time, priority, diagnosis, action, possible recommendations, close date and time, final diagnosis, and root cause.

- Coordinate help desk Call Tracking Administration problem ticket distribution and escalation among technicians, system administrators (SA), and other support staff.

- Monitor submitted help desk tickets through the status of “In Progress” within one (1) hour when the help desk is in operation.

- Log all help desk events and generate performance metrics (e.g. number of emails, calls and tickets per day).

- Ensure all help desk tickets are resolved and closed according to the negotiated and government-approved Contractor Service Level Agreement and minimum acceptable response levels.

- Maintain tracking system (central repository) for calls, call prioritization, call management, and call distribution to specialized tier staff (such as network engineer, support for mail client, server software, protocols used, and troubleshooting issues specific to those applications).

- Maintain and track detailed logs, notes, design, or configuration changes initiated from an initial help desk call.

- Facilitate and provide daily logs and tickets prior to the daily trouble ticket review.

● The Contractor shall provide support for MSS Tier 2:

○ Desktop Security Implementation:

- Implement desktop security standards to the equipment on the Mission

Support LAN, provide the applicable documentation on desktop security rules (including anti-virus configuration and local security policy settings).

- Perform patch management on Mission Support LAN desktop systems in accordance with the MSS LAN Patch Management Policy.

○ Desktop Software Support - Implement, maintain, and troubleshoot desktop software on the Administrative network, provide recommendations, and establish compatible replacement software compliant with NSOF SLA requirements for operations on the Administrative network.

○ Desktop Hardware Support (desktop hardware comprises desktop telephones, desktop PCs, monitors, printers, scanners, and other computing devices)

- Implement, maintain, and troubleshoot desktop hardware on the Mission Support LAN and all supported Mission Systems.

- Provide recommendations, identify compatible replacement hardware compliant with NSOF SLA requirements for operations on the NSOF administrative network, maintain desktop equipment database including any warranty information, provide annual recommendations for enhancements to the Common Services organization.

○ Troubleshooting and Call Response Services:

- Provide technical support and troubleshooting of desktop hardware and software.

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 12

- Utilize the applicable call tracking system, perform all levels of desktop technical troubleshooting, including telephone technical support.

○ Trouble Tickets - Monitor and resolve all trouble tickets. Trouble tickets must include symptoms, basic diagnostic data, resolution, outcome, corrective actions, and information about the configuration item and service affected. The Mission Support LAN help desk shall establish a ticket ownership on each trouble ticket.

○ Problem Identification - Document service requests within the ticketing system, route the request per the Service Level Agreement to the appropriate service group. The final (reported) trouble ticket classification may vary from the initially reported classification.

○ Problem Resolution:

- Complete and close service requests per the Service Level Agreement while providing the user with the means to continue their business functions (minimizing the user impact).

- Provide consistent follow up on desk calls and complete requests within the prescribed SLAs. When the trouble has been resolved, the help desk shall verify the accuracy of the completed trouble ticket, and the user agrees with the resolution.

○ Move, Add, Change Processing:

- Coordinate a move, add and change (MAC) Request Process.

- Record all data for MACs, as the single point of contact for MAC requests, and route all MAC requests to the proper NOAA and NESDIS authority for approvals.

○ Configuration Updates - Update all asset management databases for MAC requests.

● The Contractor shall provide support for MSS Tier 3:

○ Trending:

- Research, evaluate, and provide feedback on problematic trends and patterns.

- Provide technical guidance and recommendations for system upgrades.

○ Operations - Verify all systems are reliable, operating, and configured per configuration management standards.

○ Planning - Provide technical recommendations for product user requirements.

○ Special Projects - Provide support for ad-hoc special projects (including network migrations and email administration, large hardware/software installations, and equipment relocations).

○ Training:

- Provide IT support training activities, ad-hoc one-on-one hands-on training sessions, as required, develop training curriculums, conduct classes; monitor and track current training technology, analyze training requirements, and recommend purchase of training materials.

- Training activities shall cover formal and informal training. Formal training shall be conducted in a classroom setting with prepared training materials (e.g. handouts, presentations), scheduled in advance with

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 13 advanced notification, and conducted at a time convenient to the most attendees, or as directed by the Government. Informal training may require less preparation and scheduling (e.g. brown bag seminars, drop-in unstructured training sessions, sessions used to convey information not crucial to mission performance, or training that could not be reasonably performed as formal training).

○ Presentation Support:

- Provide backup support preparing presentations, graphics, automated forms creation and maintenance, PowerPoint presentations, word processing documents, and spreadsheets for presentation.

- Provide product support, including business presentations, and digital image scanning, capture, enhancement, compression, and conversion.

○ Schedule Development and Implementation and Diagnostics Procedures and Training - Develop and implement procedures for scheduling, maintenance, network and hardware diagnostics, and training for any procedures.

○ Customer Surveys - Implement improved processes based on analysis of customer satisfaction surveys.

○ System Usage Statistics - Generate and review statistics of system utilization, and generate an ECMT help ticket in response to any anomalies; report issues for correction to system administration.

3.3.2.3 Operations Electronic Call Tracking and Reporting

PWS-TO2-OP16 The contractor shall perform electronic call tracking and reporting activities in the Government-provided tool, including the accurate population of all required fields in log requests covering maintenance, upgrade, de-installation, installation, relocation, and preventative maintenance. Data format must be compatible with Microsoft Excel. The Data fields include:

● Date and Time of the request for service

● Unique tracking number

● Equipment Type

● Mission (GOES, POES, DMSP, JASON, etc.)

● Equipment serial number and DOC inventory barcode number

● Name of person(s) requesting service

● Description of services requested

● Description of resolution actions taken for maintenance requests

● Required staff to resolve the maintenance request

● Time required to resolve the request (to the nearest 30 minutes)

● Completion date and time

● Spare parts used to complete repair

3.3.3 Systems Analysis and Resolution Support

3.3.3.1 Operational and Back-up Sites

PWS-TO2-OP17. The Contractor shall provide support for systems analysis and problem resolution to operational and back up sites. This includes personnel to address problem

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 14 resolutions in command and control systems, data, product applications, systems software, and networking systems.

3.3.4 Workstation/Desktop/Hardware Support for Operational Workstations/Printers

PWS-TO2-OP18. The Contractor shall perform on-site remedial hardware maintenance on operations systems (e.g., workstation/desktop/hardware supporting SOCC operation area, satellite operator and engineering workstations/printers, and Ground System Equipment and systems). Support activities include:

● Correctly identify, diagnose and repair equipment,

● Troubleshoot and repair of ground systems equipment including hardware components replacement to maintain compliant operabilitions.

PWS-TO2-OP19. The Contractor shall provide support to SOCC operations for satellite operator and engineering workstations. These functions include:

● Respond to maintenance requests within 15 minutes of receipt of request

● Replace hardware used for repair or maintenance with Government provided spare workstations or printers within two (2) hours of receiving the request [Defective workstations shall be repaired and placed in Government Equipment Spares Inventory]

● Complete repairs and replacements of SOCC Operations Area workstations; return workstations to Government inventory no later than 4:30 p.m. local time on the business day following the date of receipt of the maintenance request

● Return repaired workstations and printers to Government Equipment Spares inventory for use as replacement equipment.

3.3.5 Data Center Infrastructure Management

3.3.5.1 Facilities Change Configuration Board (CCB) Report

PWS-TO2-OP20. The Contractor shall provide support for facilities change management activities:

● Complete Configuration Change Requests (CCR) that address any change impacting the Mission Support LAN, NSOF Public Branch Exchange (PBX), cable infrastructure (e.g., Domain Policy changes, add/remove services or servers on the Mission Support LAN, firewall or switch configuration change, PBX modifications, etc.), or modification of the physical plant (i.e., adding an equipment rack).

● Submit CCRs to the Facilities CCB, address questions, provide any clarification requested.

● Perform all required activities to close CCR per the agreed schedule. Report progress in CCR closure, and elevate or expedite any activities required to maintain the CCR closure schedule.

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 15

3.3.5.2 Cable Plant Management

PWS-TO2-OP21. The Contractor shall provide support in cable plan management activities:

● Provide at least one (1) Building Industry Consulting Services International (BICSI) certified person to be on call at all times

● Build /terminate/ test/verify any required IT-related cabling (as directed by the Government). Examples of cabling include CAT6, coaxial, serial, and fiber optic cables.

● Build/terminate low voltage wiring connections (directed by the Government), including serial cables from systems to appliances, or antennas.

● Provide cable management support for all assigned systems, cable connections, and maintenance

● Provide all cable connections from where the operational center plugs into assigned demarcation points (located in one of the six communication rooms in NSOF or at a consolidation point)

● Manage all infrastructure cable connections using a cable database application (iTRACS) which contains a graphical interface and output to facilitate the making and tracking of connections

● Coordinate all repairs for problems found on infrastructure cabling

● Maintain a cable database (in the NSOF) and infrastructure cable plant including reserving patch panel ports in all NSOF communication rooms. Patch panel port reservations shall be approved by the Government.

3.3.6 Antenna Operations Support

The Government assets documented in this section require operations support:

● ViaSat 9.1m Geostationary Satellite Antennas, and

● GOES Variable (GVAR)/GOES ReBroadcast (GRB) Data Distribution and Ingest

System.

The four 9.1m ViaSat (receive-only) geostationary satellite antennas and the accompanying ingest and data distribution system are of vital importance to satellite operations. These four antenna systems (located on the roof of the NSOF in Suitland, MD) are used to capture mission critical image data in the GVAR format for the GOES N/O/P satellites, and in the GRB format for GOES-R satellites. The data distribution system provides the GVAR/GRB data to the ESPC Advanced Front End Processor Systems (AFEPS). The AFEPS ingest patching system receives and distributes GVAR/GRB signals to required AFEPS equipment.

PWS-TO2-OP22. The Contractor shall support the aforementioned antenna systems. Activities include:

● Load and maintain the tracking controller of each system with current, accurate and validated ephemeris data to provide antenna tracking to a [minimum of 99% operational capability].

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 16

● Perform pointing adjustments, and signal peaking of the antennas to maintain data reception.

● Re-point antennas to different spacecraft, when approved by the OMS COR.

● Coordinate 100% of outages associated with or created by adjusting the antennas or directing them onto different satellites, with authorized Government staff.

● Adjust and reposition antennas to different satellites and perform patching on the AFEPS ingest patch system to ensure GVAR/GRB reception and distribution is maintained at the nominal performance standard.

● Perform antenna alignment and pointing for the four (4) LRIT antennas and the two (2) GEONETCAST antennas to meet nominal performance standards.

● Perform problem troubleshooting and maintenance for the four (4) LRIT and two (2) GEONETCAST antennas.

● Respond to LRIT and GEONETCAST issues within 2 hours.

● Return affected out of service antennas to nominal performance standards within 8 hours.

3.3.7 Network and Security Operations

Network and security operations monitoring staff work side-by-side to ensure the reliability, availability, and integrity of the operational networks.

PWS-TO2-OP23. The Contractor shall provide resources supporting network, application, systems engineering, and security engineering tasks. These tasks include:

● Manage enterprise tools to support security operations. This includes: system administration activities, installation, patching, updates, troubleshooting issues, configure tools used in the NESDIS environment, parameter modification, script and middleware code writing. Examples of tools include: ArcSight, Tripwire, Altiris, SolarWinds, and Enterprise Continuous Monitoring Operations (ECMO).

● Provide Enterprise Tools Status Report [Deliverable] supporting security operations

● Support IT security scans

3.3.7.1. Wide Area Network (WAN) and Local Area Network (LAN) Support

PWS-TO2-OP24. The Contractor shall support WAN and LAN support activities. This includes:

● Maintain/ update diagrams of WAN / LAN data communication connections (i.e., external point to point data communication circuits, internet connections, circuit and/or packet switching, mission voice conferencing system, and dedicated dark fiber connections)

● Provide automated alerts from network monitoring system for network (WAN) outages in compliance with Government-approved Service Level Agreement(s) (SLA), or federal mandates

● Coordinate with Government-approved communication vendors, customers, partners, and other mission systems to troubleshoot network problems, outages, or any other issues impacting the SLA. This includes coordination with leased communication providers to initiate trouble tickets.

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 17

● Diagnose and resolve issues given a network equipment failure or connection outage

● Respond to calls from Operations and/or MSS Helpdesk for resolution of an outage or unacceptable mission system network performance

● Provide remote instruction (when not on site) to the Government and onsite support for problem resolutions. This does not absolve the requirement of the Contractor to be physically present when needed.

● Document network outage event reports for submission to the Ground Systems Engineer (GSE), Product Area Lead (PAL), and Government Network Manager; add the resolution to Work Request or Trouble Ticket as part of the CM process [Deliverable].

● Maintain current, documented configuration of OSPO networks (LAN) and network equipment in the CI database.

● Provide network engineering support for approved change requests associated with the operational networks.

● Comply with Government-approved configuration practices for WAN network equipment configurations, port assignments, network diagrams, and IP addresses assignments.

3.3.7.2. Security Configuration Management

PWS-TO2-OP25. The Contractor shall support Security Configuration Monitoring activities of security systems for all relevant devices and applications, and report security configuration status relevant to security configuration baselines. Contractor activities include:

● Monitor systems using OSPO-approved monitoring tools, Security Content Automation Protocol (SCAP), content, and processes

● Evaluate any identified security configuration settings against relevant OSPO and National Aeronautics and Space Administration (NASA) security configuration baseline

● Provide electronic transfer of results of all security configuration monitoring using OSPO standard tools and processes, per the agreed schedule, to the OSPO authoritative security risk management repository

● Coordinate with information systems owners, Information System Security Officers (ISSO), and system administrators to report and clarify any identified deviations from security configuration baselines

3.3.7.3. Security Data Analysis Services

PWS-TO2-OP26. The Contractor shall provide security data analysis services, and perform security data analytics services of the IT Security Incidents and Operational Anomalies. Services

● Configure, fine-tune, and monitor Security Information Management (SIM) or Security Information and Event Management (SIEM) solutions

● Proactively monitor environments using data mining and other advanced data analytic techniques

● Review information in NOAA’s security repositories (such as ArcSight ) to determine patterns, trends, security vulnerabilities

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 18

● Provide security incident summary reports [Deliverable]

● Provide trend analysis

● Provide anomaly detection

● Provide root cause analysis

● Monitor Network Trends

● Monitor system and network device behaviors to establish and maintain a baseline of normal patterns and trends for all networks

● Report network activities that diverge from normal network patterns or trends

● Correlate data from multiple log sources to validate potential intrusions or assess system risk. Examples of relevant log sources include: centralized log system, patch status reporting system, vulnerability scanning system, firewall logs, full packet capture system, and web content filtering system.

3.3.7.4. Communications Security (COMSEC) Key Loading and Technical Support

PWS-TO2-OP27. The Contractor shall provide Communications Security (COMSEC) Key Loading and Technical Support. The government utilizes various Communication Encryption Devices for satellite command and control and payload processing. The devices are vital to ensure accurate, secure communication with satellites and remote ground stations. Encryption keys of these devices must be kept up to date and synchronized with remote encryption devices.

Contractor support to this task is at the NOAA Satellite Operations Facility (NSOF), 4231 Suitland Road, Suitland, Maryland. Government operations at this facility requires personnel have a security clearance at the SECRET level, and be a U.S. citizen. The SECRET clearance is issued by the DOC Office of Security.

The communication security activities include:

● Perform scheduled activities (weekly, monthly and quarterly) loading of COMSEC keying material at the scheduled times per OSPO’s standard COMSEC operating procedures

● Coordinate all performed loads on encryption devices, with Government satellite operations staff and remote COMSEC technicians, to verify loads do not interrupt or interfere with satellite operations

● Ensure COMSEC loads performed to an operational capability of no less than 99% as measured by circuit connectivity and availability

● Troubleshooting and diagnosis requests of COMSEC equipment problems (by the COMSEC Account Manager (CAM))

● Repair/replace/configure/ test/verify operations of any COMSEC equipment failure and respond in 15 minutes during normal business hours and within 4 hours during non-business hours. The Government will provide a replacement unit(s).

● Assist with packaging and shipment of failed unit(s) for depot repair

● Proper safeguard of all COMSEC material in Contractor possession in accordance with

COMSEC requirements

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 19

3.3.8 Continuity of Operations (COOP)

PWS-TO2-OP28. The Contractor shall provide support and services required for Continuity of Operations (COOP) during OSPO COOP exercises or activations. The Government also participates in these activities. The level of support, at NSOF and the backup site(s), is the same regardless of whether the NSOF is untenable or cannot be used for all, or part, of normal operations. Upon a COOP event activation, activities include:

● Deployment to remote COOP sites for the duration of the event

● Support of maintenance COOP and disaster recovery plan/procedures to address the immediate and longer-term needs of the Government.

● Provide any updates of the Continuity of Operations Plan [Deliverable]

● Assess vulnerability to significant service interruptions for NOAA missions and programs

● Identify and analyze budgetary, service, and other impacts of extended service disruptions

3. 4 Maintenance

High level maintenance functions and requirements are provided for in Section 3.3 of the OMS

IDIQ PWS.

Definitions of maintenance activities is provided for in section 1.4 of the OMS IDIQ PWS.

Additional maintenance requirements are provided in the following sections.

3.4.1 Network & Systems Administration

PWS-TO2-M01. The Contractor shall provide support all network task support. Network tasks

● Perform network management of all Government operational networks and components.

This includes the NSOF facility, remote facilities, CDAs, and CBU facility are the primary locations for this required support. The supported networks are per Appendix A of the OMS IDIQ PWS.

● Provide comprehensive on-site support for network administrators and performance management, at NSOF (Monday through Friday with core business hours (7:00am – 6:00pm local time), excluding Federal holidays).

● Provide support and response for network outages, issues, and service degradations that cannot be resolved during normal business hours at the NSOF, or other OSPO network locations, when alerted. Provide on-site network administration support for satellite launches and special events requested by the Government, outside of core business hours.

● Perform on-site network administration support, upgrades, installations, troubleshoot or verify information for documentation, planning and configuration management at remote sites, and other sites the government requests that have communication paths

● Follow established Government processes and procedures when configuring firewall, intrusion detection systems (IDS), intrusion prevention systems (IPS), routers, switches

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 20 and all other network-specific equipment to allow authorized and approved traffic.

Configuration includes providing alerts of possible compromise, unauthorized use, or service interruption.

● Track and report metrics for availability, data usage, error rates, packet types, packet types per time, and queue lengths.

3.4.1.1 Networks Fault Management

PWS-TO2-M02. The Contractor shall provide support for Networks Fault Management. Typical network fault management activities include:

● Diagnose problems; conduct problem resolution given network performance degradation, network equipment failure and connection issues/outage

● Respond to calls for resolution of outage or unacceptable network performance from the Shift Supervisor, ESPC Helpdesk, and Ground System Engineers

● Respond to Contractor automated network monitoring system alerts on the network (WAN/LAN) outages, service degradation, and performance issues according to the established Government-approved SLA(s), or federal mandates

● Document event reports on network issues, network performance degradation, network equipment failures, and connection issues/outages; these will be provided to the GSE and

PAL.

● Participate in mission system anomaly review boards in reporting of problems and outages, and providing any recommended corrective actions

● Coordinate with contract vendors, partners, and other mission systems, and stakeholders in troubleshooting network problems, outages, or any other issues impacting a SLA.

Coordinating with contract leased circuit providers to initiate trouble tickets on leased telecommunications circuits is also required.

● Document/maintain inventory of spare parts as a part of the CI database, throughout the entire equipment life cycle.

● Comply with Government-approved configuration management practices for WAN/LAN network equipment configurations, port assignments, network diagrams, and IP address assignments.

● Maintain or create (document) configuration of networks and location of all components/hardware connected to the mission systems.

● Provide with compliant marking requirements for network documentation as “Controlled Unclassified Information” (CUI), and any appropriate sub-categories.

● Review network diagrams and systems documentation for accuracy and correctness following the completion of any network configuration installation, movement, addition, or change.

○ There are approximately 50 network diagrams maintained for all applicable ground systems covered by this task order.

● Verify that all like-components utilize the same software, firmware, and engineering levels at all sites.

○ The SolarWinds Orion network monitoring tool will be provided to demonstrate that like components across sites are configured the same. The Network

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 21

Administrator shall generate a report using the tool and submit the Components Compliance Report annually [Deliverable].

● Configure network components compliant with approved baseline configurations; and comply with OSPO policy for system hardening.

○ The security settings used by the Contractor when configuring information technology products shall be the most restrictive, consistent with information system operational requirements.

○ The Contractor shall utilize configuration checklists from the United States Government Configuration Baseline (USGCB) checklists that reflect the most restrictive mode consistent with operational requirements CITR 17. If the USGCB does not contain applicable checklists use (in order of priority): other checklists from NIST (https://nvd.nist.gov/ncp/repository); from the Defense Information Systems Agency (DISA) Security Technical Implementation Guides (STIG) program, or National Security Agency secure baseline checklists; vender produced checklists; and third party checklists.

○ Vendor or third party checklists shall be used only when Federal government checklists are not available, or when the approved SSP specifies otherwise.

○ The Contractor shall identify, document, and approve any deviations from established configuration settings for key components of infrastructure, network servers, desktops and databases, etc. based on mission requirements running within the FISMA system boundary.

○ The Contractor shall use NIST SP 800-70 Rev 3 or the latest mandated revision, for guidance on configuration settings (i.e., checklists) for information technology products.

○ The Contractor shall use reports in SolarWinds Orion network monitoring tool and Tripwire Enterprise to verify and demonstrate success with system hardening.

The Network Administrator will generate a report in SolarWinds Orion network monitoring tool and Tripwire Enterprise and submit the System Hardening Report quarterly [Deliverable].

● Provide network engineering review, impact assessment, and support for all approved change requests regarding operational networks.

● Review system description documents for impact assessment to operations per the OSPO CM Plan for all new installations.

3.4.1.2 Networks Account Administration Management

PWS-TO2-M03. The Contractor shall provide Network Account/Administration Management services. Typical tasks include:

● Adherence to Government-mandated account management policies and procedures for adding, maintaining, monitoring and disabling user accounts on network equipment

● Coordinate with the Government to process account request forms, configuration change requests, and any other forms documented in Government network administration procedures

● Complete system account reviews every six (6) months or as scheduled per NIST, DOC and NOAA policy, and per System Security Plan (SSP) [Deliverable]

OPERATIONS & MAINTENANCE

Source Selection Information – See FAR 2.101 and 3.104-4 22

● Administer appropriate access to system and network devices using secured methods approved by the ISSO.

● Coordinate with the Government to create and maintain a Spare Parts Plan, spare parts inventories, system inventories, rack elevations, and equipment room layouts. These documents formats and data will be approved by the Government [Deliverable].

● Implement network and devices supporting both IPv6 and IPv4.

● Verify that networks support automatic failover to redundant connections when the initial connection fails, or is degraded.

● Support use of descriptions to include hostname of connected device, remote port identification for network to network connectivity, and network interface name on workstation/server hardware of connected device into the loaded and enabled device configurations using the network equipment operating system

● Document and manage data center racks and patch cables per ANSI/TIA-942 standard and Building Industry Consulting Service International (BICSI) best practices

● Maintain labeling of network cabling and equipment per ANSI/TIA-606-B labeling standard

● Manage and troubleshoot approved public and private Domain Naming Service (DNS) name spaces owned and operated by the Government. This includes obtaining domain and IP space from the NOAA Network Operations Center (NOC). For public facing networks, manage and assign the proper IP address to correspond to the correct domain names, and update DNS names and entries.

● Manage, track, and communicate to appropriate Government personnel all Dynamic Host Configuration Protocol (DHCP) and static IP address reservations within the network [Deliverable].

● Maintain accurate records of network port connections and IP assignments for all Government boundary network equipment; identify network port connections with remote hostname, location ID, and Interface ID (port number, Eth0, etc.)

● Provide/ maintain an implementation schedule for any network changes, including applying network operating system updates or patches which require government approval.

● Comply with Government-provided limitations regarding downtime and SLAs

● Coordinate scheduled network down times with the Government representative and operations personnel

● Implement/enforce multi-factor authentication methods for remote users and other

Government-identified users.

● Provide remote access (Virtual Private Network (VPN)) services to monitor and manage methods…