DRAFT_STATEMENT_OF_WORK.pdf

SECTION C

STATEMENT OF WORK

C.1 OBJECTIVE

The objective of this acquisition is to acquire systems integration services to support the Department of Commerce (DOC), National Oceanic and Atmospheric Administration (NOAA), Office of the Chief Information Officer (OCIO), High Performance Computing and Communications requirements (HPCC).

C.2 PURPOSE

The purpose of this acquisition is to:

C.2.1 Acquire High Performance Computing (HPC) Support Services and Equipment

The acquisition of HPC support services and equipment includes but is not limited to, system integration, system administration, system operations and support for NOAA’s Environmental Modeling Program (EMP). HPC includes computational platforms, data storage, networking and systems software.

C.2.2 Advance Target Architecture

Implement and enhance HPC system architecture and system design, reduce system obsolescence, and increase interoperability, compatibility, flexibility and reliability for the continued future evolution of NOAA’s high performance computing.

C.2.3 Maximize Acquisition Efficiency and Expediency, Leverage Private

Sector Expertise

This acquisition allows NOAA to acquire HPC services efficiently and expeditiously to allow the Government to concentrate on core missions while exploiting private sector expertise in managing state of the art high performance computing.

C.2.4 Advance Environmental Science

This acquisition allows NOAA to enhance the performance and engineering of NOAA user community application software and HPC systems, thereby enabling the advancement of environmental science.

C.3 BACKGROUND

C.3.1 Agency Mission

C.3.1.1 The mission of NOAA, found at www.noaa.gov, is to understand and predict changes in the Earth’s environment and to conserve and manage coastal and marine resources to meet our Nations’ economic, social, and environmental needs. NOAA's mission is embodied in four strategic goals:

C.3.1.1.1 Climate Adaptation and Mitigation: An informed society anticipating and responding to climate and its impacts:

• Improved scientific understanding of the climate system and its impacts.

• Assessments of current and future states of the climate system that identify potential impacts and inform science, service and stewardship decisions.

• Mitigation and adaptation efforts supported by sustained, reliable and timely services.

• A climate-literate public that understands its vulnerabilities to climate impacts and makes informed decisions.

C.3.1.1.2 Weather-ready Nation: A society that is prepared for and responds to weather-related events

• Reduced loss of life, property, and disruption from high-impact events.

• Improved freshwater resource management.

• Improved transportation efficiency and safety.

• Healthy people and communities due to improved air and water quality services.

• A more productive and efficient economy through environmental information relevant to key sectors of the U.S.

economy.

C.3.1.1.3 Healthy Oceans: Marine fisheries, habitats, and biodiversity sustained within healthy and productive ecosystems:

• Improved understanding of ecosystems to inform resource management decisions.

• Recovered and healthy marine and coastal species.

• Healthy habitats that sustain resilient and thriving marine resources and communities.

• Sustainable fisheries and safe seafood for healthy populations and vibrant communities.

http://www.noaa.gov/

C.3.1.1.4 Resilient Coastal Communities and Economies: Coastal and Great Lakes communities that is environmentally and economically sustainable

• Resilient coastal communities that can adapt to the impacts of hazards and climate.

• Comprehensive Ocean and coastal planning and management.

• Safe, efficient and environmentally sound marine transportation.

• Improved coastal water quality supporting human health and coastal ecosystem services.

• Safe, environmentally sound Arctic access and resource management.

C.3.1.2 In support of NOAA’s mission and goals, NOAA conducts research and gathers data about the global oceans, atmosphere, space, and sun, and applies this knowledge to science and services that touch the lives of all Americans.

C.3.1.3 The ability to predict changes in the Earth’s environment depends primarily on a diverse set of environmental models, requiring considerable computational resources.

These models are developed through research and development (R&D) efforts within

NOAA.

C.3.1.4 Computational modeling continues to emerge throughout NOAA and is germane to the daily operation of many of its laboratories. Organizations such as the Pacific Marine Environmental Laboratory (PMEL), Earth System Research Laboratory (ESRL) and Air Resources Laboratory (ARL) use models for study of atmosphere, ocean, climate, air quality, and ecosystem behavior.

C.3.1.5 NOAA collaborates with multiple external organizations in developing models that can be used and reused for the needs of the individual agency’s missions. NOAA employs the use of community developed models to improve collaboration to accelerate the transition of these enhancements to operations. Recently, NOAA has inaugurated a program to produce the Next Generation Global Prediction System as a multi-organization, community development effort. NOAA’s goals include a new Global Forecast System in Operation by the beginning of Government Fiscal Year 2020.

C.3.1.6 Environmental modeling helps NOAA meet its mission in other ways as well.

NOAA is developing improved air quality models that incorporate the changing chemistry and radiative properties of the atmosphere in severe smog events into weather models. To this end, NOAA is executing on a vision to provide a National Air Quality Forecast System with sufficient accuracy and advance notice to take action to prevent or reduce adverse effects. Exploratory work is also going on in very high resolution hurricane models to lay the foundation for further improvements, not just in track forecasts, but also in the rapid intensification that can occur shortly before landfall increasing the destructive power of the hurricane.

C.3.2 Current Technical Environment

NOAA operates three high performance computing subsystems for research and development in support of NOAA’s mission which in turn supports over 1,000 users.

Together, the following three subsystems comprise NOAA’s R&D High Performance Computing System (R&D HPCS).

C.3.2.1 National Climate-Computing Research Center (NCRC) www.ncrc.gov/

The largest of these subsystems is a partnership between NOAA and DOE and is hosted at the Oak Ridge Leadership Computing Facility (OLCF) located at Oak Ridge National Laboratory. This system is primarily used for climate modeling with scientists at NOAA’s Geophysical Fluid Dynamics Laboratory (GFDL), Princeton NJ being the principal users. On this system the models for climate prediction are developed and runs of various climate scenarios are made for the US Global Change Research Program (USGCRP). The NCRC system is critical to climate prediction and developing a national response. Currently the scientists at the GFDL are developing the next generation of Earth System Models on the Gaea system. These models will provide input for phase 6 of the Coupled Model Intercomparison Project (CMIP6) due to start model runs in 2017.

Previous such efforts have led to a share in the Nobel Peace prize. By some measures, the climate prediction models developed and run at GFDL are among the best in the world.

GFDL also contributes research towards improving hurricane track and intensity prediction by coupling the hurricane prediction model with a full ocean model.

A detailed description of the current R&D architecture is attached in Appendix A.

C.3.2.1.1 Geophysical Fluid Dynamics Laboratory (GFDL) www.gfdl.noaa.gov NOAA Office of Oceanic and Atmospheric Research (OAR), GFDL is located on the campus of Princeton University in Princeton, NJ. On the order of 50-100 TB of model output per day flows from NOAA’s RDHPCS large system computing assets (primarily NCRC at this time) across NOAA’s dedicated WAN back to GFDL for post-processing and analysis. To support this, GFDL maintains a cluster of approximately 100 post-processing nodes and an SGI DMF archiving solution currently storing approximately 100PB of data. Archive growth exceeds 1PB per month.

C.3.2.2 National Environmental Security Computing Center (NESCC) www.noaanews.noaa.gov/stories2010/20101013_fairmont.html

The research and development computer in Fairmont, WV is used for weather model research, exploring advanced methods of using satellite data in weather models, and developing improved climate forecasts on seasonal through inter‐annual scales, http://www.ncrc.gov/ http://www.gfdl.noaa.gov/ http://www.ncep.noaa.gov/ such as El Niño forecasts. This computer primarily supports NOAA National Weather Service (NWS), National Center for Environmental Prediction (NCEP) located in College Park, MD and NOAA’s Earth System Research Laboratory (ESRL) located in Boulder, CO.

Currently the NESCC houses a Cray CS400 with 1208 nodes in the compute partition. Each node is comprised of two Intel Haswell sockets with 12 cores/socket with a clock speed of 2.6GHz. Most of the nodes have 64GBytes of RAM (14 have 256GBytes of RAM). There is a total of 11 PB of disk storage supplied by Panasas (totaling greater than 100GB/s). Attached into the CS400 infiniband fabric are 100 Cray CS Storm nodes. Each Node is comprised of 2 8-Core Haswell nodes, 256GBytes of RAM and 8 NVIDIA Pascal GPUs.

Additionally, the NESCC houses a hierarchical storage system supplied by IBM based upon HPSS. There are two Oracle SL8500 robots as well as 4 PB of disk cache.

C.3.2.3 Earth System Research Laboratory (ESRL) www.esrl.noaa.gov/gsd

The research and development computers at NOAA OAR ESRL in Boulder, Colorado are collectively known as Jet and serve many users. The majority of Jet supports NOAA's Hurricane Forecast Improvement Program (HFIP) model development including real-time R&D forecasts during hurricane season. Jet also supports R&D and real-time R&D for a number of NOAA's global, regional, and advanced modeling projects, and a diverse group of projects from several NOAA Laboratories/Services.

C.3.2.4 Other Federal Agencies

NOAA also partners with other Federal Agencies to leverage the use of their HPC resources. In particular, collaborations with the National Aeronautics and Space Administration (NASA), National Science Foundation (NSF), and Department of Energy (DOE) have been very beneficial to NOAA. NOAA has worked closely with NASA in developing an atmospheric dynamical core, capable of scaling to very high processor counts.

Atmosphere models built on this core have scaled to over 128,000 MPI ranks with over 1,000,000 shared memory threads on one DOE’s largest computers: the IBM BG/Q (Mira) at Argonne National Laboratory.

C.4 SCOPE

C.4.1 High Performance Computing Services

The Contractor shall at a minimum:

C.4.1.1 Provide balanced high performance computing capabilities and support services (e.g. system administration, computational science support, help desk);

http://www.esrl.noaa.gov/gsd

C.4.1.2 Provide the supporting infrastructure, including but not limited to, storage, data management, and data movement; and

C.4.1.3 Provide growth capabilities to augment existing systems and acquire additional large scale high performance computing to support model research;

and

C.4.1.4 Provide advice and insight on existing, near-term and long-term hardware and software technologies and approaches suited to supporting NOAA’s HPC objectives.

C.4.2 NOAA’s HPC Architecture (ESA)

NOAA’s HPC architecture is called the Environmental Security Architecture (ESA). This acquisition is designed to support NOAA’s ongoing efforts to support this architecture. This procurement focuses on a portion of the R&D segment of the ESA. The procurement will support large scale compute, long-term storage, pre/post processing, visualization, and data analysis needs for NCEP, GFDL, ESRL, and other NOAA locations as necessary. The ESA is documented in the Strategic Plan located at:

www.cio.noaa.gov.

C.4.3 R&D HPCS Facility

C.4.3.1 The NOAA R&D HPCS subsystem components are currently located at four facilities. GFDL’s large scale compute is housed at ORNL and is not the subject of this procurement. The facilities at Boulder, CO, Princeton, NJ and Fairmont, WV are Government furnished.

C.4.3.2 The Government facilities at Fairmont, WV, Princeton, NJ and Boulder, CO will be available throughout the life of this contract. GFDL houses a very large tape archive system as well as computing resources used for data handling, analysis and visualization. ESRL houses computational resources used for modeling, data handling, analysis and visualization. Additional facility information is provided in Section 5.16.

C.4.3.3 In addition, during the life of the contract, the Government may offer additional facilities to house HPC resources.

C.4.4 Long Term Storage (Tape)

NOAA possesses significant long term storage systems at GFDL and NESCC. Both the GFDL and NESCC subsystems employ Sun/StorageTek SL8500 tape silos. As of December 2016, there are four silos located at GFDL and three silos located at NESCC. The silos at GFDL are managed by an SGI Data Migration Facility (DMF) system, and the silos at NESCC are part of an IBM High Performance Storage System http://www.cio.noaa.gov/HPCC/pdfs/HPC_Strategic_Plan.pdf

(HPSS). Each of these systems currently stores over 80PB of data. The Contractor shall continue to maintain and support these systems through the life of the contract. Long term storage assets shall be owned by the Government.

C.4.5 Transition Activities

May 7, 2018 - May 6, 2019 is a transition year for the existing R&D HPCS contract. The new Contractor shall be required to work with the Government to manage transition activities. At contract award, the Government will provide the Contractor with a statement of work to transition current R&D operations and support services from the incumbent Contractor.

C.5 REQUIREMENTS

C.5.1 Project Management Services

C.5.1.1 Project Manager (PM)

The Contractor shall appoint a PM and shall identify this project manager as key personnel. The project manager shall have PMP certification or obtain such certification within one year of the appointment. At a minimum, the PM shall perform the following duties:

C.5.1.1.1 Integration

• Ensure integration across all contract elements

• Implement contract strategic and tactical planning C.5.1.1.2 Collaboration, Communication and Coordination

C.5.1.1.2.1 Communicate R&D HPCS and contract priorities to the NOAA Integrated Management Team

(IMT);

C.5.1.1.2.2 Seek innovative approaches and foster collaboration among R&D HPCS centers to improve effectiveness and efficiency; promote teamwork and synergy;

C.5.1.1.2.3 Ensure all Contractor staff actively collaborate, communicate, and coordinate among each other and with Government both within the R&D HPCS centers as well as across the

R&D HPCS program, including users; and

C.5.1.1.2.4 Ensure information exchanges are timely, thorough, and accurate;

C.5.1.1.2.5 The contractor staff shall use NOAA communication tools (email, calendaring, documentation) when working with NOAA or its collaborators. All official communications shall originate with a noaa.gov email address.

C.5.1.1.3 Customer Satisfaction

C.5.1.1.3.1 Work with Government to achieve and maintain high customer satisfaction;

C.5.1.1.3.2 Provide an escalation path for resolution of customer issues;

C.5.1.1.3.3 Identify the customer’s principle needs and interests;

C.5.1.1.3.4 Manage the customer’s expectations to avoid unanticipated requirements; and

C.5.1.1.3.5 Solicit feedback and use it constructively to improve customer service.

C.5.1.1.4 Contract Management

C.5.1.1.4.1 Ensure all activities are within scope and comply with contract requirements;

C.5.1.1.4.2 Develop responses to Statements of Work, Performance Work Statement and/or Statement of Objectives in compliance with NOAA’s mission and in the best interest of the Government;

C.5.1.1.4.3 Compete, award, and administer subcontracts;

C.5.1.1.4.4 Ensure performance of the business and administrative aspects of the contract;

C.5.1.1.4.5 Ensure that resources are efficiently and effectively managed and that contract status (including the use of Government furnished resources) is reported to Government representatives as required; and

C.5.1.1.4.6 Develop, implement and manage formal associate Contractor agreements as required.

Management Best Practices

Performance Objectives

Performance Expectation

Method of Surveillance

Proactive leadership team – effective integration and results management

Delivery on proposal initiatives and proposed results; Government provided workload is executed within cost and schedule commitments

Periodic program reviews

Efficient and effective collaboration, communication, and coordination

Timely, accurate direction, tracking and reporting of all contract activities and resources (no unanticipated requirements)

Periodic status reviews

Quality customer relations

Actionable responses to customer issues are identified and actions assigned in a timely manner; actions are completed by the negotiated deadline

Periodic program reviews

Thorough contract administration

Timely, accurate, and comprehensive contract actions; no deviations of contract terms and conditions

Review of contract deliverables and periodic program reviews

Effective financial data and reports

Accurate, timely and consistent with generally accepted accounting principles

Review of contract deliverables and periodic program reviews

Responsive staffing

Fully staffed with highly qualified and technically competent personnel at all times within the proposed technical solution (vacancies shall only be filled within 90 days; otherwise the Government shall seek consideration)

Review of contract deliverables and periodic program reviews

Effective process management

Tasks are delivered through documented and controlled processes

Periodic process and program reviews

Fig. C.5.1.1: Management Best Practices

C.5.1.2 Meetings and Reports

C.5.1.2.1 Project Kickoff Meeting

The Contractor shall schedule and coordinate a Project Kick-Off Meeting at the location determined by the Government upon issuance of a task order. The meeting will provide an introduction between the Contractor personnel and Government personnel who will be involved with the contract. The meeting will provide the opportunity to discuss technical, management, security issues, and reporting procedures. At a minimum, the attendees shall include key Contractor personnel and key Government personnel.

C.5.1.2.2 Weekly Status Meeting

The Contractor shall hold a brief weekly status meeting with the Government management team each Tuesday from 1:00 PM – 2:00 PM (Eastern Time). Adjustments to this schedule will be made to accommodate Federal holidays.

These meetings will generally be teleconferences hosted by the Government. Government hosted video teleconferences may also be held periodically. The content for these meetings shall be based on the content from the Weekly Status Reports (WSR).

C.5.1.2.2.1 Weekly Status Report (WSR) The Contractor shall provide a weekly report to the COR electronically each Monday by noon Eastern Time. The target audience for this report is the Government Integrated Management Team. Adjustments to this schedule will be made to accommodate Federal holidays. This report will be the focus of the weekly meetings. The Government will work with the Contractor to develop and refine the format for the report after contract award.

C.5.1.2.2.1.1 WSR Content

At a minimum, the WSR shall include the following for each center and for the entire system as a whole:

a. Risks/issues/concerns technical and other programmatic issues;

b. Out of the ordinary events;

c. High level summary of change management;

d. System metrics (as appropriate) includes, but is not limited to, dependability, run-time variability, data integrity, system utilization, job wait times, storage system utilization, storage capacity remaining, and data migration status;

e. System status includes, but is not limited to, HPC system(s), local systems (analysis cluster), storage systems and interconnects;

f. Security Issues;

g. Maintenance Report; and

h. Status of contract management including status of change proposals, payments, and any other contractual issue that may arise.

i. Upcoming plans at each center.

C.5.1.2.2.1.2 WSR Color

Scheme The Contractor shall initially use the following color scheme and definitions to provide status reports to the Government on the major areas of the R&D HPCS. The Government may modify the color scheme, definitions and reporting boundaries at its discretion.

WSR Color Scheme

Green Normal. Contract performance metrics are being met.

Yellow

Contract performance metrics are not being met. A Government approved plan is being implemented to bring the issue back to green.

Red

Contract performance metrics are not being met. A Government approved plan is required to bring the issue back to green.

Fig. C.5.1.2.2.1.2: WSR Color Scheme

The Government will work with the Contractor after contract award to identify those areas of the system that the color codes will be applied to for reporting purposes.

C.5.1.2.3 Quarterly Management Meeting

The Contractor shall hold recurring quarterly management meetings with the Government. The Contractor shall schedule these meetings. These meetings will generally be face- to-face meetings hosted by the Government. Video teleconferences, hosted by the Government, may also be held periodically. The content for this meeting shall be based on the content from the Quarterly Status Reports (QSR).

C.5.1.2.3.1 Quarterly Status Report

(QSR)

The QSR shall be provided to the COR and Integrated Management Team at least 2 business days prior to the quarterly management meeting. Adjustments to this schedule will be made to accommodate Federal holidays. At a minimum, the QSR shall include:

a. A summary of activities which occurred during the reporting period, by task. Include on-going activities, new activities, activities completed, progress to date, and targeted completion dates;

b. Summary of year to date performance;

expectations;

c. Highlight outliers both above and below

d. Discuss significant challenges not been addressed;

e. Explain any long-standing risks that have not been addressed.

f. Identify areas of opportunity where cost savings or efficiencies may be found;

g. Identify problems and corrective actions taken. Also include issues or concerns and proposed resolutions;

h. Contractor personnel additions, losses and status;

Contractor manpower forecast;

i. Government actions required;

j. Schedule. Shows major tasks, milestones, and deliverables;

planned and actual start and completion dates for each; and

k. A summary of key metrics to be identified by the Government.

Some sample metrics may include system utilization by allocation, resource trends (such as storage utilization), high level system description, and summary of availability /downtime.

C.5.1.2.4 Annual Executive Meeting

The Contractor shall hold a high-level annual meeting with the Government during the first quarter of each fiscal year at a mutually agreed upon date, time and location. The Contractor shall provide an assessment of the performance of the contract to date. The Contractor shall provide the Government with an analysis of HPC related technologies and the marketplace. The Government will request personnel to attend on an as-needed basis. Hand-outs or presentations for this meeting shall be provided to the COR and Integrated Management Team three business days before the meeting.

C.5.1.2.5 Special Reports

The Contractor shall promptly respond to requests for special reports regarding management and operations of the HPC systems and on topics related to HPC Support, such as a utilization report. In addition, data calls regarding security are not uncommon and shall be promptly addressed by the Contractor when requested. The Government will work with the Contractor to further define the details and structure of these reports after contract award.

C.5.1.3 Workforce Requirements

The Contractor shall be responsible for maintaining a qualified workforce able to perform the broad spectrum of functions necessary to operate, support, maintain, and improve the R&D HPCS.

C.5.1.3.1 Personnel shall be technically qualified and certified as required to operate and maintain any piece of hardware and software that is either Government furnished or that the vendor provides as a part of this contract;

C.5.1.3.2 The Contractor shall maintain staffing records identifying company organizational designations, a brief description of the functions, and the number and types of personnel assigned;

C.5.1.3.3 The Contractor shall provide an organization chart to the Government at contract award and provide updates within 7 calendar days when changes to staffing are made; and

C.5.1.3.4 All resources used for the performance of work under the contract are identified, their roles clearly defined, and their relationship to the remainder of the project established and identified.

C.5.1.3.5 All positions shall remain filled at all times. In the event, the contractor sustains attrition of a position on this contract, the position(s) shall be filled with technically qualified and certified personnel within 90 days. Otherwise, the Government shall seek consideration after the 90 day period for the vacancy.

C.5.1.4 Organizational Structure

The Contractor shall propose an organizational structure for operation, administration, and maintenance of the services provided under this contract. The organization structure shall include at a minimum, the following key personnel:

C.5.1.4.1 PM;

C.5.1.4.2 Contract Manager;

C.5.1.4.3 Engineering/Architecture Lead;

Personnel may serve in more than one capacity. Additional labor categories may be necessary as task requirements are defined. Within the organization structure, the following functions shall be staffed by the Contractor:

C.5.1.4.7 Customer Service;

C.5.1.4.8 Operations, Administration, and

C.5.1.4.9 Site Management;

C.5.1.4.10 User Liaison;

C.5.1.4.11 User Support;

C.5.1.4.12 Facility Support; and

C.5.1.4.13 Communications tuning expertise for each vendor provided system.

C.5.1.5 Project Management Plan (PMP)

The PMP involves all elements of program management.

The principle objective of this element is to ensure proactive and sustained excellence in providing accurate, safe, secure, timely, and efficient contract support to meet the Government’s requirements as described in the Statement of Work.

C.5.1.5.1 PMP Components

The PMP shall describe the proposed management approach. At a minimum, the PMP shall address the following:

C.5.1.5.1.1 Schedule Management;

C.5.1.5.1.2 Quality Management;

C.5.1.5.1.3 Process Improvement;

(leverage CMMI best practices as appropriate)

C.5.1.5.1.4 Communications Management;

C.5.1.5.1.5 Risk Management;

C.5.1.5.1.6 Risk Register;

C.5.1.5.1.7 Work Breakdown Structure (WBS)(Level II);

C.5.1.5.1.8 Baseline Schedule (including tasks and milestones); and C.5.1.5.1.9 Cost Management.

C.5.1.5.2 The Contractor shall identify and maintain, within this contract and the PMP document, an up-to-date management structure describing the organization and overall project control and authority for the performance of all work under the contract.

C.5.1.5.3 The Contractor shall document support requirements in a PMP within thirty (30) days after contract award. The Government will provide comments within 10 days from receipt of the draft document. The revised PMP shall incorporate Government comments. The Contractor shall deliver the final version of the PMP to the Government within 60 days of contract award.

C.5.1.5.4 Updates to the Project Management Plan

The PMP will be an evolving document. The Contractor shall update the PMP at least annually, or more often as directed by the Government, or when processes, WBS, or as a result of other significant changes. The Government expects insight into the Contractor’s planning process to gain exposure to the project plan activities that may be impacted by the Government. The Contractor shall work from a Government approved PMP and schedule. The Contractor shall work from the new version of the PMP once approved by the Government.

C.5.2 IT Security Services

C.5.2.1 Security Team

The Government plans to establish a security team that will be responsible for all aspects of the security for this system. The team will consist of members from the Government and the Contractor. The Government will be responsible for all decisions and approvals. The Contractor shall provide a security lead with significant experience in security engineering and auditing, pertaining to Federal IT systems with a CISSP certification. The security engineer will need to convey security concepts to technical experts, senior systems administrators, and junior administrators, and operations. Good communication skills and expertise in systems administration, networking, storage, and high performance computing concepts are essential. The security team shall be involved with all potential enhancements. All team members will be required to successfully complete a background investigation as defined by DOC Policy 7.4 for Personnel Screening, which states that NOAA system access shall be limited prior to completion of screening and that user access shall be restricted to the least privilege (e.g., read, write, execute, delete). The Contractor shall also be required to separate critical functions among its staff to ensure system integrity and oversight. The Contractor shall provide additional staff to support Assessment & Authorization (A&A) and other defined security team support requirements.

C.5.2.2 Security Policies

This system will be maintained consistent with Government-wide laws and regulations.

C.5.2.2.1 Office of Management and Budget Policies

The Office of Management and Budget (OMB)

Circular A-130 requires all Federal agencies to:

C.5.2.2.1.1 Plan for the security of all IT systems throughout their life cycle;

C.5.2.2.1.2 Implement and maintain a program to assure that adequate security is provided for all agency information collected, processed, transmitted, stored, or disseminated in general support systems and major application;

C.5.2.2.1.3 Implement policies, standards, and procedures that are consistent with Government-wide policies, standards, and procedures issued by the Office of Management and Budget (OMB) and Department of Commerce; and

C.5.2.2.1.4 Establish a minimum set of controls to be included in Federal automated information security programs and assigns Federal agency responsibilities for the security of automated information.

C.5.2.2.2 FISMA Policies

The Federal Information Security Management Act of 2002 (FISMA), (HR 2548 E-Government Act, TITLE III – Information Security, SEC. 301.

Information Security), addresses the program management and evaluation aspects of IT Security.

C.5.2.2.3 DOC Policies

C.5.2.2.3.1 DOC Manual of Security

Policies and Procedures

C.5.2.2.3.2 DOC IT Security Program Policy and Minimum Implementation Standards

C.5.2.2.4 NOAA OCIO Policies and Directives

C.5.2.2.4.1 NOAA 212-1300 IT Security Policy provides specific guidance on Patch Management and Risk

Management controls to be used for all NOAA systems.

C.5.2.2.5 Additional Policies

Additional policies are identified at the following websites:

C.5.2.2.5.1 www.cio.noaa.gov (NOAA CIO)

C.5.2.2.5.2 www.csp.noaa.gov (NOAA IT Security Office) C.5.2.2.5.3 www.osec.doc.gov/cio/oipr/IT

SEC/ITSECDOC1.HTM (DOC IT Security Program Office)

C.5.2.2.5.4 csrc.nist.gov/policies (NIST Federal Requirements)

C.5.2.2.5.5 csrc.nist.gov/publications/n istpubs/index.html (NIST Guidelines)

C.5.2.3 Compliance

C.5.2.3.1 The Government reserves the right to augment or update the list of applicable documents. The Contractor shall assist the Government in meeting the security requirements for all systems delivered under this contract.

C.5.2.3.2 The Contractor shall comply with all aspects of the

Governance and security documentation defined in Section 5.3, Governance, and provide complete documentation for all systems components provided by the Contractor. In addition, solutions proposed under this contract shall conform with NOAA’s Enterprise Architecture (a controlled document available through the CO or COR). Task order proposals shall clearly indicate the methodology and personnel responsible for performing all security tasks. The Government may delegate aspects of system resource documentation and management to the Contractor, but will retain full authority over system resources and information security. Under no circumstances is a Contractor permitted to make any use of organization computer equipment or supplies for purposes other than performance on this contract.

C.5.2.4 Assessment and Authorization (A&A)

C.5.2.4.1 In order to maintain a consistent security posture across potentially multiple sites, the Contractor shall be required to adhere to the Governance documents list in Section 5.3. All procedures shall be documented and maintained as part of the required NIST SP 800-37 Assessment and Authorization (A&A) package. All IT equipment delivered as part of this procurement is to be considered a Government http://www.cio.noaa.gov/ http://www.csp.noaa.gov/ http://www.osec.doc.gov/cio/oipr/ITSEC/ITSECDOC1.HTM http://www.osec.doc.gov/cio/oipr/ITSEC/ITSECDOC1.HTM http://csrc.nist.gov/policies/ http://csrc.nist.gov/publications/nistpubs/index.html http://csrc.nist.gov/publications/nistpubs/index.html computing resource, regardless of its location or actual owner. NOAA will provide the current A&A documentation for the existing sub components to the Contractor upon request. The Contractor shall participate in the maintenance of the complete security documentation for R&D HPCS from the time of award through the end of the contract.

Key NOAA personnel will also participate in this process.

C.5.2.4.2 The System Owner (SO) will be the Director of the NOAA HPCC Office.

C.5.2.4.3 The NOAA Information System Security Officer (ISSO) will be the IT Security Project Manager.

C.5.2.4.4 The Contractor may elect to employ equipment that is not part of the delivery for use in remotely diagnosing, monitoring, or managing the delivered system. If this Contractor-owned equipment has any special access or trust relationship with the delivered system, then the Contractor shall initiate and maintain any appropriate C&A documentation for this equipment as a separate Government system.

C.5.2.4.5 The Contractor shall support all aspects of the C&A effort. The Contractor shall provide a Concept of Operations detailing its actions in support of all system tasks prior to acceptance. An independent party will conduct the Certification Test and Evaluation (CT&E) testing for the R&D HPCS.

C.5.2.5 System Boundaries and Network

C.5.2.5.1 The Government will provide the network connectivity to all devices outside of the system. The Government will provide dedicated private communications between sites. The Contractor is responsible for establishing, maintaining and supporting communications point of presence for all its systems to enable connectivity to NOAA-provided WAN/LAN resources. The Contractor shall assist in performance tuning to ensure the best possible available communications with all external devices. Additionally, the Contractor shall also assist in troubleshooting and issue diagnostics for wide area and local area network communications. Local area network bandwidth and performance requirements will be defined at the task order level. The Contractor shall monitor all internal network segments. The Government will work with the Contractor to develop specific reporting metrics after contract award.

C.5.2.5.2 The Contractor will provide infrastructure services, including but not limited to, domain name services (DNS), directory services, remote access, authentication, network time protocol (ntp), and web. The Government will work with the Contractor to identify and develop, as necessary, specific services after contract award.

C.5.2.5.3 The system boundary is defined as containing all devices in the system inventory. The Security Team will maintain devices and services necessary to deliver the services identified in C.5.2.5.2.

Network Schematic

C.5.3.1 Documentation

C.5.3.1.1 All documents that the Contractor provides to the Government shall be in electronic format. Electronic copies shall be delivered via e-mail attachment or other media by mutual agreement of the parties. The Contractor shall use appropriately secure transmission and storage methods (e.g., encryption during transmission for sensitive but unclassified materials);

C.5.3.1.2 The electronic document copies shall be compatible with MS Office products or other applications as mutually agreed to by all parties;

C.5.3.1.3 It is the responsibility of the Contractor to ensure that all data transmitted via tape, CD, diskette, or e-mail is virus free;

C.5.3.1.4 The Government requires the Contractor to maintain a current, itemized list of all Contractor-supplied hardware and software items in printable electronic form. The Contractor shall include in this list equipment, not provided by the Contractor, that is part of this overall HPC program. The Contractor-supplied ad non- Contractor supplied equipment shall be marked or categorized in the inventory so that they can be listed separately. The inventory listing shall be updated where a system upgrade or engineering change proposal results in a change to the system configuration or delivery of additional equipment. The Government may also request assistance with Government property management, and equipment with a Government property tag shall be annotated as such in the inventory. The inventory listing shall be published and maintained in the on-line project document repository provided by the Government; and

C.5.3.1.5 The Contractor shall provide adequate documentation for maintenance of the Project Management and the Configuration Management systems. This includes, but is not limited to, baseline configuration, implementation methodology, a formal project plan, and documentation of administrative tasks.

C.5.3.2 Document Repository

All documents pertaining to this project shall be placed in the online repository provided by the Government, likely as a SharePoint site.

C.5.3.3 Policies

The Government will provide initial versions of the governance documents. These documents establish the governance policies. The Contractor shall adhere to these policies. The Contractor may propose revisions to the documents;

however the Government is solely responsible for approval of changes to the documents. The documents will be located in the Government repository for reference.

The documents will cover, at a minimum, the Security Plan, configuration management, inventory management, account management, disaster recovery, remote access, media handling, records management, facilities CONOPs, and incident management.

C.5.4 Transition Services

C.5.4.1 Transition-In Task

At contract award, the Government will provide the Contractor with a statement of work to transition current R&D operations and support services from the incumbent Contractor. The Government will support the Contractor by providing access to pertinent documentation and facilitating in-depth site visits to familiarize the Contractor with existing systems and expectations for transition.

C.5.4.1.1 Proposed Transition-In Solution

Within thirty (30) days from receipt of the

Government’s transition requirements, the Contractor shall propose a transition solution for maintaining and operating the current R&D HPCS. The proposal shall include a description on the disposition of all existing leases held by the incumbent Contractor. At a minimum, the plan shall be consistent with the PMP referenced in Section 5.1.5. The transition must be completed no later than May 6, 2019.

C.5.4.1.2 Continuity of Operations

Of particular concern is maintaining continuity of essential services such as user trouble ticket support, configuration management and security. The Contractor is expected to work with the Government to develop transition/support plans during the third and fourth quarters of FY 2018, to implement straightforward and/or pilot programs during the third and fourth quarters of FY 2018, and to provide comprehensive support by the end of the second quarter FY 2019. The Contractor shall be responsible for supporting the additional computational capabilities received by NOAA from external sources.

C.5.4.1.3 GFDL (Princeton, NJ)

Currently, GFDL houses its pre/post processing, large scale computing and long term storage assets at its Princeton location. GFDL’s large scale processing requirements are provided by ORNL. The Contractor shall be responsible for pre/post processing and long-term storage requirements which will remain local to GFDL. With production computing at ORNL and NESCC, the Contractor shall work with the Government to fully supporting on-going activities. The Contractor shall support all existing leased and Government-owned R&D HPCS subsystem assets at GFDL until the Government no longer requires support. System availability of 97% and scientific data availability of 98% shall be met or exceeded.

C.5.4.1.4 NESCC (Fairmont, WV)

Currently the NESCC supports pre/post processing, large scale computing and long term storage assets at its Fairmont location.

The NESCC is housed in a Government facility managed by the General Services Administration (GSA). Details regarding interaction with GSA shall be provided by the Government if required. The Contractor shall support all existing leased and Government-owned R&D HPCS subsystem assets at NESCC until the Government no longer requires support.

C.5.4.1.5 ESRL (Boulder, CO)

Currently, ESRL supports pre/post processing and large scale computing at its Boulder location. ESRL is housed in a Government facility managed by the General Services Administration (GSA). Details regarding interaction with GSA shall be provided by the Government if required. The Contractor shall support ESRL’s existing leased and Government-owned R&D HPCS subsystem assets until the Government no longer requires support.

C.5.4.2 Transition-Out Task

The Contractor shall fully support the transition from this contract to its successor.

C.5.4.2.1 Proposed Transition-Out Solution

At the end of this contract, the Contractor shall develop and submit, within sixty (60) days of a written request by the Government, a comprehensive Transition Baseline Plan. The goal of the Transition Baseline Plan is to avoid disruption of the day-to-day conduct of HPC computing while achieving a smooth and orderly transfer of responsibility from the incumbent Contractor to a successor Contractor. At a minimum, the Transition Baseline Plan shall contain all information related to current system configuration, performance, support requirements, issues, risks, and other information deemed pertinent by the Government. This plan shall be submitted to the Contracting Officer and COR.

C.5.4.2.2 Continuity of Operations

Up to six (6) calendar months prior to the completion of this contract, an observation period shall occur, during which time personnel of the successor Contractor may observe operations and performance methods of the outgoing Contractor. This will allow for orderly turnover of facilities, equipment, and records and will help to ensure continuity of service. The outgoing Contractor shall not defer any requirements for the purpose of avoiding responsibility for or of transferring such responsibility to the succeeding Contractor. The outgoing Contractor shall fully cooperate with the succeeding Contractor and the Government so as not to interfere with a smooth transition and ongoing accomplishment of the Government's work.

C.5.5 ESA Architecture Task

The Contractor shall provide all labor, equipment, and material necessary to operate, maintain and upgrade NOAA’s Environmental Security Architecture (ESA).

Travel may be required in accordance with the Federal Travel Regulations. On a per task order basis, the Government will provide the Contractor with a set of requirements, benchmark codes, and any desired functional design updates for the

ESA.

C.5.6 System Migration Services

The Contractor shall install, de-install, configure, and maintain acquired high performance computing systems, maintain long term storage systems, and integrate new and existing systems over the life of this contract.

C.5.6.1 Migration Requirements

On a per task order basis, the Contractor shall assist in the migration of workloads from existing systems to target systems with minimal disruption to the user community. These migrations shall be time phased. The Contractor shall develop the migration schedule, which depends on factors that include the new system architecture, hardware acquisition and other NOAA considerations and requirements.

C.5.6.1.1 Transition Plan

The Government’s data is essential to its mission.

Therefore, ninety (90) calendar days prior to all data migrations, system changes and system upgrades, and/or at the Government’s request, the Contractor shall submit a comprehensive Transition Plan to the Contracting Officer and COR. At a minimum, it shall contain an overview, schedule, information related to configuration, performance, support requirements, risks, a cost/benefit analysis and other information deemed pertinent by the Government. This plan shall be submitted with enough lead time for the Government to make required facility modifications if necessary and to coordinate technical constraints with the Government’s respective users. This activity shall be integrated with, updated and tracked in the PMP.

C.5.6.1.2 Transition Support

The Contractor shall provide system migration support and complete all migration activities, including inventory management. Support services includes: planning and analysis of migration activities, development of migration schedules, transport, installation and de-installation of Government-provided hardware (if required, e.g., for removal of excess or obsolete systems or transport to another local or remote site), software, and data. Migration also includes data and software conversion. No Government equipment, even equipment which has been traded in, may be removed without final Government property clearance. The Contractor shall provide assistance as requested with Government property management.

C.5.7 Acquisition Services

The Contractor shall competitively acquire HPC systems for the Government’s use. HPC systems will be defined at the task order level.

C.5.7.1 Acquisition Services Requirements

As part of acquisition support, the Contractor shall

(1) prepare and deliver an acquisition sourcing package and (2) acquire solutions consistent with the acquisition sourcing package. Sourcing high performance computing under this contract is not limited by past acquisition and ownership practices. All methods of acquiring these solutions may be considered. The acquisition sourcing package shall include the following as applicable:

C.5.7.1.1 Requirements analysis and analysis of alternatives;

C.5.7.1.1.1 The Contractor shall provide appropriately detailed and persuasive analyses in support of acquisitions, particularly alternative source acquisitions. The Contractor shall provide a trade-off analysis of proposed alternatives. If the Contractor proposes to meet a requirement with internal resources, e.g., the Contractor’s own time-sharing facility, then that recommendation shall be persuasively and transparently justified to the Government;

C.5.7.1.2 Develop IT system solicitation documentation and supporting documentation, including a source selection plan;

C.5.7.1.3 Design, build, and administer performance validation package;

C.5.7.1.4 Solicit proposals from Offerors;

C.5.7.1.5 Evaluate proposals and negotiate best value;

C.5.7.1.6 Document results of the evaluation and negotiation of received proposals;

C.5.7.1.7 Identify subcontract(s) and alternative source subcontracts;

C.5.7.1.8 Propose supplemental performance validation tests to enhance the Government’s Inspection and Acceptance Criteria;

C.5.7.1.9 Propose required physical plant and facilities and changes/impacts risks and alternatives that require significant lead time;

C.5.7.1.10 Propose required governance documents and changes/impacts risks and alternatives that require significant lead time; and

C.5.7.1.11 Provide any recommendations for improvements of Government acceptance tests.

C.5.7.2 Acquisition Considerations

In each case, the Government will evaluate the solution that best fits current and expected future circumstances, considering budget, finance, technology, information technology market, user requirements, enterprise architecture, compatibility, risk, security, performance, level of automation, functionality, policy, regulation, and other relevant factors.

C.5.7.3 Acquisition Frequency

The Contractor shall be required to acquire new or upgraded IT hardware, system software, and support services when requested.

There is no fixed interval for periodic upgrades; the schedule depends upon user requirements, resource availability, and technical, architectural, energy efficient and management considerations that are not established in advance. These factors shall be considered by the Contractor in developing an optimal implementation schedule.

C.5.8 Acquisition and Implementation of High Performance Computing Systems at the NOAA Environmental Security Computing Center

(NESCC)

See Appendix A.

C.5.9 Storage Management Services

The Contractor shall manage all NOAA mass storage subsystems, including robotic tape libraries, fabrics, data movement, disk subsystems, storage servers, and related hardware, software, and infrastructure. For a sense of scale, it is anticipated that GFDL’s HSMS will grow at a rates around 25-35 PB/year, and faster if more computational capability becomes available.