TI STATEMENT OF WORK- DRAFT.docx

High Performance Computing Modernization Program (HPCMP) Technical Insertion Statement of Work

STATEMENT OF WORK

TECHNOLOGY INSERTION BASIC ORDERING AGREEMENT

Industry Classification System (NAICS) code 334111.

1.0 OBJECTIVE.

The primary objective of this Basic Ordering Agreement (BOA) is to provide the High Performance Computing Modernization Program (HPCMP) with world-class high performance computing capabilities for the United States Department of Defense (DOD). The BOA holders will be responsible for providing the Government with balanced, commercially- available, production-grade High Performance Computing (HPC) systems, which contain an appropriate combination of processor, memory, disk Input/Output (I/O), interconnect, and Operating System (OS) capabilities in order to conduct complex, tightly-coupled, large-scale, scientific calculations.

2.0 DESCRIPTION

2.1 Locations.

Requirements for each order will be located primarily at the following four DSRCs and will encompass the unique requirements of each center:

· U.S. Navy DSRC, John C. Stennis Space Center, Mississippi

· U.S. Army Corps of Engineers, Engineer Research and Development Center (ERDC) DSRC, Vicksburg, Mississippi

· Air Force Research Laboratory (AFRL) DSRC, Wright-Patterson Air Force Base, Ohio

· Army Research Laboratory (ARL) DSRC, Aberdeen Proving Ground, Maryland

2.2 System Manufacturing Capabilities.

To satisfactorily complete the work, the contractor, who must be an Original Equipment Manufacturer (OEM), must have expert knowledge of federal Government standards and industry recognized national standards for advanced computing and communication technologies. Targeted system attributes at the order level include the following:

· A high ratio of peak CPU memory bandwidth to peak CPU floating point capability

· 2.5 GB to 4 GB of memory per compute core (in special cases, 4GB to 8GB of memory per compute core)

· Five distinct file systems with all metadata stored on solid state devices, no single point of failure, and protection from double storage device failure (e.g RAID6): (1) /work1 with at least 125 GB of formatted usable disk storage per core, (2) /work2 with at least 250 TB of formatted usable disk storage, (3) /work 3 with at least 50 GB formatted usable storage per core and the ability to encrypt data consistent with NIST Special Publication 800-131A, (4) /app with at least 1 GB of formatted usable disk storage per core, and (5) /home with at least 8 GB of formatted usable disk storage per core

· Aggregate data transfer rates between the compute nodes and each file system as follows:

· /work1: 4 MB/s per compute core

· /work2: 500 KB/s per compute core

· /work3: 500 KB/s per compute core

· An ability to simultaneously mount /work1, /work2, /work3, /home, and /app on all login nodes and all compute nodes

· A high number of I/O operations per second (IOPs) using 1, 32, and 128 active cores for the following file operations for each scratch file system (i.e. /work, /work2, and /work3): (1) open/create, (2) file stat, (3) #1 and #2 simultaneously, and (4) unlink

· A tightly integrated interconnect with (1) each link having at least 100 Gbps of bandwidth and (2) an end-to-end latency that is no greater than 2µs (two microseconds)

· A well-tuned operating system (OS) with low jitter, configured in accordance with DISA security technical implementation guides (STIGs)

· A 40 gigabit Ethernet interface (or better) for external (i.e. wide-area network [WAN]) connectivity with IPv4 and IPv6 dual-stack functionality

· A 40 gigabit Ethernet interface (or better) for internal (i.e. local-area network [LAN]) connectivity with IPv4 and IPv6 dual-stack functionality

· Use of workload management software with ability to launch jobs using container-based solutions (e.g. Docker, Shifter, Singularity)

· An appropriate number of job scheduling nodes with (a) suitable job scheduling software and (b) adequate memory, processing capability, and redundancy

· Use of HPCMP Kerberos and HPCMP Public Key Infrastructure (PKI) authentication software and HPCMP Secure Shell (SSH) communication software (which will be furnished by the Government) Adherence to the following power quality standards: CBEMA, ITIC, SEMI F47, IEC 61000-4-11/34 A cabinet supply voltage of 480V (when possible/feasible)

· A cooling solution in which at least 98% of the heat generated by the system is removed by liquid- cooling

· Login nodes which contain the same processor type as the compute nodes, but contain more memory per core than compute nodes to facilitate compiling and other interactive functions

· A test and development system (TDS) which is a smaller version of the base system (i.e. configured with similar hardware, software, file system, and I/O attributes) to allow effective system software testing, user code porting, and base system configuration management

· An ability to execute as many software packages listed on the HPCMP’s software web page (https://centers.hpc.mil/software) as possible

· At least two suites of high-level language compilers: (a) GNU Compiler Collection (version 6.3 or higher) and (b) a proprietary suite

· At least two implementations of MPI with at least one being compliant with the MPI-3.1 standard JavaScript and go scripting languages

· Python 2.7 must be provided with NumPy, PyMPI, MPI4Py, SciPy, PyTorch, Keras, theano, and IPython extensions; Python 3.6 (or higher) must be provided with as many of the following extensions as possible: NumPy, PyMPI, MPI4Py, SciPy, PyTorch, Keras, and IPython.

· Multiprocessing APIs (including OpenMP), standard computational libraries (including BLAS, BLACS, FFTW, LAPACK, and ScaLAPACK), accelerator computational libraries/compilers (Including CUDA, OpenCL, and OpenACC), performance libraries (including PAPI), and an integrated development environment

· Application performance instrumentation tools (including at least one code profiler that interfaces with PAPI and at least one tool that provides MPI messaging statistics)

· A high level of system reliability (i.e. a high overall effectiveness level and a low number of user/job interrupts).

· A high level of performance on high performance applications and libraries such as ALEGRA, AVUS, CTH, FFTW, GAMESS, and HYCOM over processor ranges from 1,024 to 65,536 processor cores.

· A high level of performance on TENSORFLOW using 1 and 8 GPGPUs.

· For each file system, a high level of performance for MDTEST using 64, 128, 192, and 256 compute nodes (which are distributed evenly across the standard compute node racks)

2.3 Administration and Maintenance.

The BOA holders shall be responsible for system administration and 24 hours per day, 7 days per week system maintenance (over five one-year options at the individual order level) and (therefore) must ensure that (1) the monthly system effectiveness level does not fall below 97%, (2) the monthly number of system-wide interrupts does not exceed the maximum allowable (which is determined by (a) dividing the number of compute nodes in the system by 1000, (b) truncating any fractional portion of the quotient, and (c) adding 1), (3) the monthly number of user/job interrupts does not exceed the maximum allowable (which is determined by dividing the number of nodes in the base system by one hundred), (4) the response time for remedial maintenance is less than two hours (in all instances), (5) new software is tested and ready for operation prior to implementation on the base system, and (6) software/hardware security vulnerabilities are addressed in a timely manner.

3.0 SECURITY REQUIREMENTS.

Since HPCMP systems will be placed (predominantly) at DOD facilities, BOA holders must have sufficient personnel with SECRET level clearances for the purposes of system installation, capability testing, effectiveness level testing, administration, and maintenance. All personnel requiring administrative access to any system and/or supporting network/security component must have an adjudicated tier 5 background investigation., a clearance commensurate with the classification of the system, Information Assurance Technician (IAT) Level II or III certification (see h ttp://iase.disa.mil/eta/iawip/content_pages/iabaseline.html and h ttp://iase.disa.mil/eta/iawip/content_pages/summary_wf_requirements.html), and IA certifications specific to all relevant operating systems.

4.0 PUBLIC AFFAIRS

The BOA holder shall not publicly disclose nay data generated or reviewed under this contract. The BOA holder shall refer all requests for information concerning projects to the Contracting Officer for comment. Prior to release, any publication articles shall be coordinated and approved by the Contracting Officer.

5.0 SUBMITTALS

All submittal requirements will be identified at the order level. These may include but are not limited to: safety plans, quality control plans, project schedules, testing plans, testing results, and closeout documentation.

6.0 REPORTING REQUIREMENTS

All reporting requirements will be established at the order level. Typical reporting requirements are on a monthly basis however, each order will identify the specific requirements for that procurement.

7.0 POINTS OF CONTACT

The POCs for the BOA level are listed below. Other POCs may be identified at the individual order level. The Contracting Officer shall always be included on all correspondence related to this BOA and all orders.

7.1 Contracting Officer: TBD

7.2 Contract Specialist: TBD