HPC_Final_SOW_V5_0_20May15.pdf

STATEMENT OF WORK

FOR

DTRA J9CXQ HYBRID GRAPH ANALYTICS

HIGH PERFORMANCE COMPUTING SYSTEM

20 May 2015

1.0 Background: The Defense Threat Reduction Agency (DTRA) is the Department of Defense (DoD) lead Combat Support Agency for combating weapons of mass destruction (CWMD).

DTRA’s Special Programs Office (J9CXQ) under the Counter Weapons of Mass Destruction Technologies Department (J9CX) is responsible for providing CWMD planning support and analysis for US Combatant Commands (CCMDs). DTRA J9CXQ requires a high performance computing system in order to conduct intensive research and development analytics to accomplish mission objectives and milestones.

DTRA J9CXQ HPC system will be used to research and test the operational feasibility of integrating advanced graph analytics to support DTRA J9CXQ analytic workflow. This will support finding previously unknown WMD threats, improve forecasting threat development timelines and event horizons, and enhance the explanatory power or visualization of graph analysis to improve DTRA’s support to CCMD planning cells.

1.2 Scope

The contractor shall provide the following:

1.2.1 Delivery of hardware, operating system, and software to meet the requirements in accordance with this statement of work.

1.2.2 Installation and testing of the system at the DTRA Herndon II Facility

1.2.3 Training of DTRA J9CXQ (Government and Contractor) personnel in operation and maintenance of the HPC system.

1.2.4 Advanced parts replacement and software support.

1.2.5 Consulting to support DTRA J9CXQ success in initial use of the system for research project.

2.0 Applicable Documents

2.1 General Description of Products Services and Deliverables, dated 16 March 2015 (U//PROPIN FAR 15.509 Limited Use of Data)

2.2 DoD 8570.01-M Information Assurance Workforce, 24 Jan 2012

3.0 Requirements

3.1 The contractor shall deliver, install and test two (2) Convey Hybrid Core Graph Analytics Scalable Units each comprised of a rack with eight (8) Convey HC-2ex Hybrid Core servers, each with a host system and a Xilinx Virtex-6 coprocessor. The nodes shall provide an aggregate of 6TB of host memory, 512GB of Convey Scatter-Gather memory, and 128TB of HDD storage. The nodes within the Scalable Unit shall be connected via dual 10Gbit Ethernet connections to a top-of-rack switch.

3.2 The contractor shall deliver, install and test, two (2) Convey HC-2ex Hybrid Core Graph Analytics server nodes which shall include an Intel host with two E5-2670v2 processors (for a total of 20 2.5Ghz cores), and 768GB of host memory. The node shall be equipped with a Convey coprocessor providing four (4) Xilinx Virtex-6 LX760 FPGAs for application acceleration, and 64GB of scatter-gather Dual in-Line Memory Module (DIMM). Each node shall run a variant of the Linux operating system (ConveyOS), and include sixteen (16) 1TB Serial Attached Small Computer System Interface (SCSI) (SAS) disk drives and two (2) 10Gbit Small Form-Factor Pluggable (SFP+) Ethernet ports.

3.3 The contractor shall provide Personality and Application Development Environment and Training. The following items are required to stand up a multi-station training and development environment for use on site by DTRA engineers and contractors for the acceleration of applications:

3.3.1 Two (2) Personality Development Support Packages that provides the development environment and twelve (12) months of support for the Verilog libraries for the Convey coprocessor and host interfaces.

3.3.2 Two (2) each Hybrid Threading (HT) development environment (repos) and support for the Open Sourced Hybrid Threading Tools.

3.3.3 Six (6) each Training, Personality Development HT and Personality Development Kit.

3.3.4 Thirty Five (35) hours of Professional Services Principle Investigator to assist DTRA Government and contractor engineers in the architecture development of the first target application for acceleration

3.3.5 Eighty (80) hours of the contractor’s Senior Hardware Engineer(s) to assist DTRA and contractor engineers in the architecture development of the first target application for acceleration.

3.4 In addition to the software modules used in the Development workflow, the contractor shall provide six one and one half (1.5) day training sessions to be executed during three (3) three (3) day periods of training. The formal training sessions shall include a level of effort for architectural and engineering assistance to the DTRA engineers and contractor for architecting the first target application for acceleration.

3.5 The contractor shall provide Engineering Consulting for the Personality/Application Development. The Contractor shall provide Professional Services Engineering to accelerate the Tabular Structured Global Database of Events, Language, and Tone (GDELT), data application as well as engineering support to assist DTRA Government and Contractors to accelerate a Natural Language Processing (NLP) application for deployment of the Graph Analytics System.

3.6 For the base year and each subsequent option year the contractor shall provide one

(1) year Advanced Parts Hardware and ConveyOS Support per Scalable Unit

3.6.1 Advance Parts Replacement for the Convey Scalable Units including parts and repair technician services at the site of system installation. The contractor shall ship the required parts to the site for receipt by DTRA and delivery to the location where the systems are installed.

3.6.2 The contractor shall ship a replacement part, and dispatch a technician to arrive on-site to complete the repair service during normal business hours Monday thru Friday 8:00 AM – 5:00PM EST. The contractor shall ship the repair parts on a next day basis if the service request is received by 11:00 AM EST, and on a two (2) day response if received after 11:00 AM EST. DTRA will keep hard drives and not return them to the contractor.

3.6.3 The Service Technician response time shall be on a next day basis, assuming parts have arrived at the systems location by 11:00 AM EST, and on a 2 day response time if received after 11:00 AM EST. The service technician(s) shall be scheduled to complete the repair during business hours once the part(s) have arrived on-site at the systems location.

3.6.4 Software Support shall consist of (i) Error Correction; (ii) Telephone Support (during normal business hours) regarding the operation and use of Software provided by the contractor to the customer’s Technical Support Contact; (iii) the contractor’s provision of the Updates to the customer as they are made generally available to all other customers; and (iv) access to the contractor’s support website. The Government will provide necessary information and other assistance to expedite any such Software Support.

3.6.5 The contractor shall exercise all commercially reasonable efforts to correct any Error reported by the customer in the unmodified then current version of the Software according to the priority level reasonably assigned to such Error by the contractor after consultation with the customer. For Priority A Errors, the contractor’s support personnel will immediately commence the following procedures: (i) assign support personnel to investigate and commence correction of the Error, (ii) initiate work to provide the customer with a Workaround or Fix, (iii) notify the contractor representative responsible for customer support that such Error has been reported by the customer, and (iv) regularly update the customer’s Technical Support Contact on the status and effectiveness of the Error Correction efforts being taken. For Priority B Errors, the contractor shall include the Fix for the Error in the next major release of the Computer and/or Software.

3.7 For the base year and subsequent option year the contractor shall provide one (1) year Advanced Parts Hardware and ConveyOS and Support for Graph Analytics Server Node for Test and Development Systems (TDS)

3.7.1 Advance Parts Replacement for the Convey Graph Analytics Server Node including parts and repair technician services at the site of system installation. The contractor shall ship the required parts to the site for receipt by DTRA and delivery to the location where the systems are installed.

3.7.2 The contractor shall ship a replacement part, and dispatch a technician to arrive on-site to complete the repair service during normal business hours Monday thru Friday 8:00 AM – 5:00PM EST. The contractor will ship the repair parts on a next day basis if the service request is received by 11:00 AM EST, and on a two (2) day response if received after 11:00 AM EST. DTRA will keep hard drives and not return them to the contractor.

3.7.3 The Service Technician response time shall be on a next day basis, assuming parts have arrived at the systems location by 11:00 AM EST, and on a two (2) day response time if received after 11:00 AM EST. The service technician(s) shall be scheduled to complete the repair during business hours once the part(s) have arrived on-site at the systems location.

3.7.4 Software Support to consist of (i) Error Correction; (ii) Telephone Support (during normal business hours) regarding the operation and use of Software provided by the contractor to the customer’s Technical Support Contact; (iii) the contractor’s provision of the Updates to the customer as they are made generally available to all other customers; and (iv) access to the contractor’s support website. The Government will provide necessary information and other assistance to expedite any such Software Support.

3.7.5 The contractor shall exercise all commercially reasonable efforts to correct any Error reported by the customer in the unmodified then current version of the Software according to the priority level reasonably assigned to such Error by the contractor after consultation with the customer. For Priority A Errors, the contractor’s support personnel shall immediately commence the following procedures: (i) assign support personnel to investigate and commence correction of the Error, (ii) initiate work to provide the customer with a Workaround or Fix, (iii) notify the contractor representative responsible for customer support that such Error has been reported by the customer, and (iv) regularly update the customer’s Technical Support Contact on the status and effectiveness of the Error Correction efforts being taken. For Priority B Errors, the contractor shall include the Fix for the Error in the next major release of the Computer and/or Software.

3.8 The system shall meet the following specifications:

3.8.1 The system shall provide a highly scalable graph traversal capability that can perform real-time, high-throughput transactions and handle graphs with millions of nodes and billions of edges in a timely manner to match the speed of the data streaming in from various sources.

3.8.2 The system shall store and manage the anticipated number of objects and relationship (nodes and edges) that constitute the graph, and support the high number of transaction in a timely way (High Transaction Computing). The estimated number of object vertices will grow to be on the order of 1010 entity vertices. The number of event-type nodes will grow at a constant factor of the number of entities under management. Assume that currently each entity is involved in 100 events resulting in 1011 event vertices. The mean number of edges is estimated at ten. The distribution of the count of these edges is unknown, however at 10 edges/event the total number of objects under management is 1012 objects.

3.8.3 The system shall combine an architecture that allows the HPC system to be recognizable and interact with an x86 Microsoft Windows server network (MS Server 2008 or above) and Field Programmable Gate Arrays (FPGA) personalities for targeted algorithm implementation to extract the best performance of the mission specific application. The HPC system shall have a dual or hybrid processing infrastructure that allows the HPC asset to perform as a standard HPC (process intensive batch processing jobs), real time HPC (able to handle streaming high compute inline analytics), and as a high speed multi-processor x86 server.

3.8.4 The architecture shall be suited for data-intensive computing applications and have the following characteristics: limited memory locality which prohibits effective data partitioning;

dynamic or unpredictable computational requirements; high-dimensional massive data derived from disparate and heterogeneous sources.

3.8.5 The processing efficiency shall be reasonable in terms of a low number of nodes, cores, but a minimum 15 billion edges traversed per second (GTEPS). The processing efficiency will have a direct impact on environmental constraints, which are constrained in DTRA J9CXQ’s data center.

3.8.6 The System shall meet environmental constraints of the data center that consists of space constraint, electrical power load, and A/C load. Further details are discussed in paragraph 4.8.7.

3.8.7 The system shall meet the following design parameters:

less than 2200 lbs.

supports 12 Terabytes of Memory supports 4 Petabytes of storage draw less than 40kw of power produce less than 11 tons of BTU accredited to process classified up to

TS/SCI/SI/TK/G/HCS

able to process a graph with over 30 billion nodes able to support an estimated 100 million graph queries per day architecture to support a mean labeled node size of 5kb.

Memory addressable in 8 byte quantities assessed on the Graph 500 database low latency and high bandwidth remote direct memory access capability system occupies no more than 4 standard racks of space able to input over 500,000 new documents per day integrate into the existing x86 cluster environment

64 GB of scatter-Gather DIMMs support 80GB/s Bandwidth with a 10Gbit Ethernet connection Multi-core Intel Xeon Processors Installed Linux OS Individual processor over 2.5Ghz Field-Programmable Gate Array Capability

3.9 Place of Performance

3.9.1 All classified work for this effort will take place within a Defense Intelligence Agency (DIA) accredited Sensitive Compartmented Information Facility (SCIF) spaces following approved physical, technical, personnel, and Automated Information Systems security policies and guidelines. Any work/development done in a non-DIA accredited SCIF must have the approval of the DTRA Special Security Office (SSO) and a Co-Utilization agreement must be in place. The contractor shall ensure compliance with DTRA security requirements to ensure proper security issues are addressed and security approval is obtained prior to initiation of work.

3.9.2 The locations for contract performance are at the DTRA Herndon II Facility and approved contractor sites that have had a successful Defense Security Service (DSS) inspection.

3.10 Travel

3.10.1 The contractor shall provide a detailed estimate of any required travel in advance to the Contracting Officer’s Representative (COR) in writing via memorandum or email. All travel shall be in accordance with the Joint Travel Regulation and all travel will be approved by the COR in writing via memorandum or email prior to the contractor incurring any travel expenses and prior to any departure.

3.10.2 Contractor personnel with SCI access shall report foreign travel in accordance with the DD254. Foreign travel must be requested and reported with adequate time for DTRA security to process the required paperwork.

3.11 Security

3.11.1 The contractor’s employees performing work under this contract shall be US citizens and have been granted the appropriate security clearance, based on need for access to specific classified information. A minimum of a SECRET clearance based on favorably adjudicated National Agency Check with Law and Credit Check is required for all contractor employees.

3.11.2 A TOP SECRET clearance, based on Single Scope Background Investigations (SSBI) favorably adjudicated by the Defense Security Service, DoD Central Adjudicative Facility, Fort Meade, Maryland with access to Sensitive Compartmented Information (SCI), is required for all contractors sitting onsite at the DTRA Herndon II SCIF. All contractor employees requiring SCI access must be US citizens and eligible for a TOP SECRET clearance and have met the

Intelligence Community Directive (ICD) 704, Personnel Security Standards and Procedures Governing Eligibility for Access to SCI and Other Controlled Access Program Information, 1 October 2008 and Intelligence Community Policy Guidance (ICPG) 704.1, 704.2, 704.3, 704.4, and 704.5 requirements prior to granting of SCI access. Clearance is based on a SSBI by Defense Intelligence Agency (DIA), Bolling, AFB, Washington, D.C. Individuals must be indoctrinated for SCI prior to being given access to such information. It is the Contractor/Facility Security Officer's responsibility to ensure that the SSBI is current and that a Phased Periodic Reinvestigation /Periodic Reinvestigation is submitted in a timely manner for all of their contractor employees performing SCI work. The contractor must provide individuals who are able to achieve and maintain the adjudicative standards set forth in ICD 704

3.11.3 As a further enhancement of security measures, for the protection of classified information, Contractors granted access to National Intelligence (SCI) are subject to periodic Counterintelligence-scope polygraphs (CSP). Contractors who refuse to take, or fail to successfully complete a DIA-administered CSP examination in connection with continued employment, assignment, or detail may be denied assignment, detail, or access to classified information.

3.11.4 The contractor shall ensure that personnel accessing information systems have the proper and current information assurance certification to perform information assurance functions in accordance with DoD 8570.01-M, Information Assurance Workforce Improvement Program.

The contractor shall meet the applicable information assurance certification requirements.

3.12 Contract Data Requirements List (CDRLs)

The contractor shall submit deliverables in accordance with Exhibit A.