A2___SOW_rev02_DRAFT_for_posting_SS_.docx

Boulder Computing Facility (BCF) Intelligent Power Distribution Units (IPDUs) and IPDU Sensors Computer Facilities Services Team (CFST) Statement of Work (SOW)

Requisition Number: NB184000-18-01211

A. Background:

The Computing Facilities Services Team (CFST), part of the Infrastructure Services Division (ISD/184), provides computing support for NIST locations in Gaithersburg and Boulder and to other agency customers of the computer facilities. This request is for the Computer Facility Services Team for a new computer facility called the Boulder Computing Facility (BCF) located at the NIST Boulder location.

The Computing Facilities Services Team is responsible to support and manage NIST OISM computer facilities including colocated computer facilities (not on NIST property). A consistent monitoring and management architecture was developed to provide a standard set of montioring and management tools, dashboards, reports, and error notification in all CFST managed computing facilities. This architecture provides robust and consistent alerting, reporting, meets all mandated IT Security requirements, and provides extensive management capability.

This standard power management, reporting, and power distribution architecture has been implemented in all OISM CFST managed computing facilities: Gaithersburg Computing Facility, the AML Computer Facility, Equinix East Colocation Computer Facility, NOAA Colo Computer Facility, the current Boulder Central Computing Facility. The power management standard specifies the intelligent power distribution unit (IPDU) type, power management appliances (PMAs), and the custom proprietary programmed database that is used for monitoring, reporting, and access control for all the OISM managed computing centers.

The new Boulder Computing Facility will be built in a different, more energy efficent achitecture utilizing the use of containments to prevent the heat from mixing with the cold air outside of the IT units. This design change will require the CFST CF Operations team to very closely monitor this facility for issues, in order to quickly respond to any computing facility issues including selective load shedding, calling and alerting staff as soon as an issue occurs, and being able to effectivly monitor enviormental and power trends in real time. The current PowerIQ management system that they have been trained on and use in Gaithersburg provides this fuctionality. The Raritan IPDUs that are the NIST OISM CFST standard IPDU, are part of this system linked to the PowerIQ server.

B. Purpose:

In preparation for migrating to the BCF, CFST is responsible to define the requirements, order, and install the power distribution infrastructure equipment for the BCF. This hardware (the intelligent power distribution unit, or IPDU) distributes, reports, and controls power in every server rack. The IPDUs in this procurement must be compatible with the OISM CFST computer facility standard architecture, meet the specified IT security requirements, meet the custom programmed database interoperability requirements, have the capability to provide “intelligent” power management with our power management server, and have the ability to collect and analyze power data for each IPDU.

The procurement also includes combination temperature/humidity sensors that are installed in the IPDUs to collect and analyze environment data.

Description of items to be acquired IPDU voltage/plug/outlet and sensor requirements:

· 67 Raritan PX3-5459R-E2K2: 3PH Delta, 208V AC 30A (24A/PH rated) 18 x IEC60320 C-13 locking outlets, 3 x NEMA 5-20R, plug; NEMA L21-30P (4P 5W), 8.6kVA 2U, Yellow

· 67 Raritan PX3-5459R-E2K6: 3PH Delta, 208V AC 30A (24A/PH rated) 18 x IEC60320 C-13 locking outlets, 3 x NEMA 5-20R, plug; NEMA L21-30P (4P 5W), 8.6kVA 2U, Blue

· 11 Raritan PX3-5541R-E2K2: 3PH Delta, 208V AC 60A 24 locking outlets: 6 x IEC60320 C-13 12A, 12 x IEC60320 C19 16A plug: IEC 60309 3P+E (3P 4W), 17.3kVA, 2U, Yellow

· 11 Raritan PX3-5541R-E2K6: 3PH Delta, 208V AC 60A 24 locking outlets: 6 x IEC60320 C-13 12A, 12 x IEC60320 C19 16A plug: IEC 60309 3P+E (3P 4W), 17.3kVA, 2U, Blue

· 80 Raritan DPX2-T1H1: Single combo temperature and humidity sensor, field replaceable sensor modules, 13ft (4m) cable, RJ-12 connector.

C. Technical Requirements:

· Half of the IPDUs will be yellow and the other half blue. This is to allow for clear assignment to either A or B side of the power in the Boulder Computing Facility. This lowers the risk that the customer will plug into the wrong electrical side.

· Aluminum extrusion chassis with a powder coat finish for optimal heat dissipation; no punch out or stamp holes which could allow particles to contaminate electronics.

· Printed circuit board (PCB) and soldered outlets to PCB

· IPDUs must be able to operate in temperature of at least 140 degrees F.

· IPDUs must allow bulk configurations to be loaded onto a large quantity of IPDUs via a DHCP server, TFTP server, or power management appliance server software.

· IPDU must be configurable via a USB-A port without being connected to a network.

· Universal MIB file that can be downloaded off vendor website.

· IPDU scripting using JSON RPC or other scripting application

· LCD/LED metering accuracy of +or-1%

· Billing grade accuracy of +or-1%

· The 208V 30 amp and 60 amp IPDU outlets must have a push button to allow for the plugged in power cord to be securely locked.

· The IPDU must be produced in an ISO 9000 certified manufacturing facility.

· Sophisticated outlet sequencing must be available to power on equipment with single outlet or outlet groups in a set order to minimize circuit breaker trips from inrush currents. Must provide capability to maximize uptime when powering on circuits and racks full of equipment.

· IPDU circuit breakers must be flush mounted to the chassis. Terminal Block Accessible option required to provide quick access to the terminal block and to wire the unit directly to an existing power whip.

· IPDU must be able to be cascaded, and must allow easy cascade (daisy chain) of multiple IPDUs in a rack. This provides efficiencies on IP drops, Ethernet ports, and patch ports. Cascading required so that if network fails, connectivity won’t be lost to cascaded IPDUs

· The following outlet control options must be on the IPDU:

· Power-on Sequencing with Customizable Delays

· Outlet Grouping Across Multiple IPDUs

· IPDU-based Load Shedding capability

· Last Known State Power-on

· Compatible with Raritan KVM switches

· Remote outlet and outlet group on/off

· Power metering options required:

· individual outlet, IPDU, and line power

· branch circuit, circuit breaker status

· current (A)

· voltage (V)

· power (W. VA)

· power factor (PF)

· energy usage (kWh)

· The smart LED display must support LAN, USB-A, USB-B, serial and a feature port.

· The IPDUs must be configurable via a USB port that is not connected to a network.

· IPDU must be a 2U horizontally mounted IPDU that fits in a standard 19” rails of a 24” and 28” IT equipment rack; we do not use vertically mounted IPDUs as they inhibit the ability of our customers to use full rack depth of their assigned IT equipment racks which could prevent them from being able to install their IT equipment.

· The IPDUs must work natively with the existing OISM standard IPDUs, as they may be paired and installed in the same rack supporting BCF hardware. The current OISM standard IPDUs are:

· Raritan PX3-5459R (30A 3-phase 208)

· Raritan PX3-5541R (60A 3-phase 208)

· The CFST intelligent power distribution architecture specifies the horizontal intelligent power distribution units (IPDUs) to be installed in each server rack and the sensors that will be plugged directly into the sensor ports of the IPDUs. These IPDUs must be compatible with NIST ISD Computing Facilities Services Team existing IPDUs and all features of the IPDUs must work with our existing power management appliance server without requiring additional MIBs to be loaded on the power management appliance server.

· The IPDUs must work natively with the existing OISM standard power management appliances. The OISM standard power management appliance is a Sunbird PowerIQ server.

· The IPDU must be fully compatible with the current power management appliance without the need to installed MIBs.

· IPDUs must be able to accept firmware upgrades, password changes, and configuration updates from the PowerIQ server.

· The PowerIQ server is currently implemented in OISM CFST managed computing facilities.

· NIST Operations personnel, who monitor the IPDUs and power management appliance dashboards 24x7x365, are already trained on the OISM standard power management appliances, dashboard, and intelligent power distribution units. The operations staff have documented procedures for load shedding, alert notifications, and other tasks they will be required to implement in the new Boulder Computing Facility. As stated before, the new Boulder Computing Facility will be heavily loaded using a hot aisle containment system designed to maximize the energy efficacy, which will require more intensive monitoring and the ability to selectively load shed in the event there is an issue.

· The power management appliance server must work with all the IPDUs that are used in all of the computer facilities; the Computer Facilities Services Team has been trained on all of the software imports into the power management appliance server. The cost of modifying the databases and retraining staff, including the operations team, would create an undue burden on the government. This means that all IPDUs must be able to be managed, retrieve data from, and be updated by the CFST standardized power management appliance server software using the NIST OISM CFST standard.

Functional Design and Criteria:

· IPDU chassis: An extruded aluminum chassis with a powder coat finish that is lightweight and strong with excellent heat dissipation. The BCF will implement different IPDU colors to easily indicate to customers which IPDU and outlet to use for devices in their IT equipment rack.

· IPDU Required Mounting: Chassis must be mountable to an IT equipment rack using vendor supplied mounting bracket that must be included as part of the IPDU purchase. These horizontal IPDUs must fit the rack provided standardized 19” IT equipment rails in both 24” and 28’ IT equipment racks.

· Smart LED Display / Controller: Smart LED display on front of the IPDU must provide the capability to scroll through data, check individual outlet power consumption, local IPDU alarms / alerts, the IPDU line draw, and for electrical loading balance for 3-phase IPDUs before more IT devices are added to the IPDU.

· IPDU Controller Ports: IPDU controller must include two gigabit LAN, a serial port, two USB-A ports and one USB-B ports, an expansion port, and a feature port.

· IPDU Feature Port: The listed feature port will allow for the addition of specific environmental sensors, which ISD will utilize in the future, as a critical part of our remote shedding of electrical load based on inputs from these sensors reaching defined set points.

· LED Display Accuracy: The smart LED display must be capable of showing three significant digits i.e. 1.42.

· LED Display Color: The LED display must have multi-color status LEDs for each receptacle.

· IPDU Local Alarms: IPDU alarms for the IPDU are displayed on the local IPDU LED display as well as sending the trap to the PowerIQ server.

· LED Display Change: The LED display will flip 180 degrees when the IPDU is installed upside down so that the display is always right side up. This is important since there are many times that the IPDU will have to be installed differently due to IT equipment in the rack causing issues with where and how the power cord for the IPDU is routed.

· Redundant Network Connections: We require two network connections so that we can use the IPDUs for both our monitoring network and also be able to be connected via a separate network for monitoring. The IPDU must be able to support communications with two separate networks at the same time.

· IPDU Controller: Field Replaceable Controller, which ensures that no downtime will be required if we have to replace the IPDU control unit.

· IPDU Outlet Status Indicators: The LED status indicators on each outlet of the IPDU must denote on, off, an over-load condition, and circuit breaker open, IPDU booting, or a firmware upgrade in progress.

· USB WiFi: The IPDU must be able to be USB WiFi capable for networking and cascading of IPDUs.

· IPDU Communication Protocols: The IPDU must support SNMPv3, USB-A and B, IPv4 and 6, JSON-RPC, and MODBUS TCP protocols.

· Xerus Technology Platform: The IPDUs must use the Xerus technology platform to ensure that the IPDU meets the required IT security requirements including but not limited to SNMPv3, secure HTTPs browser for the local IPDU, secure account management, and the ability to be linked directly to the PowerIQ servers (NIST OISM CFST Standard) without the use of MIBs.

· Certifications:

· FCC Part 15 (Protection against unintentional radiators)

· UL 60950 (Specification for Safety of Information Technology Equipment)

· IPDU must have an operating temperature of 140 degrees Fahrenheit (60 degrees centigrade)

· Government Safety Compliance: RoHS (Restriction of Hazardous Substances Directive) / WEEE (Waste Electrical and Electronic Equipment Directive) compliant.

· IPDU Circuit Breakers: The IPDUs must have UL 489 rated circuit breakers – no fuses. This is required to enable CF staff to quickly respond to an alert condition, preventing a possible longer interruption to services provided.

· Flush Mounted Branch Circuit Breakers: These circuit breakers must be low-profile flush mount circuit breakers.

· Branch Circuit Breaker Trip Alarming: In the event that a branch circuit breaker trips, the IPDU must indicate which branch circuit has tripped to allow for CFST Operations to begin responding immediately.

· IPDU Circuit / Line Monitoring and Management:

· Line Metering: Power metering of each line for both current (A) and voltage (V).

· Circuit Break Metering: Power metering of each circuit breaker for current (A).

· Outlet Metering: IPDU must have power metering of each individual outlet / receptacle for current (A), voltage (V), active power (W), apparent power (VA), energy (kWh), and power factor (pf). Outlet level metering with a required +/- 1% accuracy. Dedicated energy management chip on each outlet.

· Data Management: IPDU must include user adjustable power data sampling and buffering. Outlet data collected to include volts, amps, kVA (kilovolt Amps), kW (kilowatts), and kWh (kilowatt hours).

· Outlet Switching: IPDU must include individual outlet and outlet group switching.

· Outlet Naming: Ability to name each outlet or outlet group.

· Outlet Level Delays: The IPDU must have user configurable outlet level delays for power sequencing.

· Latching Relays: The IPDU must include bi-state latching relays, making outlet switching safer while consuming 70% less power than conventional methods.

· Residual Current Monitoring: The IPDU must include the measurement of the current flowing in the ground wire, reducing the risk of electrical shock.

· Load shedding and alerting: Ability to set thresholds and alerts for power and environmental monitoring. This feature is vital to protect all equipment installed in the Boulder Computing Center.

· IPDU Management: The IPDUs must be manageable and configurable through an Ethernet port, a serial port, or a USB port. This will allow for IPDU management / configuration in case one of the methods of communication to the IPDU is not functioning or accessible.

· Firmware updating: Firmware updates should not require a reboot of the IPDU.

· IPDU failure: If the IPDU should have a circuit board failure, the outlets of the IPDU shall fail in the “on” condition.

· Data Center Power Usage Effectiveness (PUE) Measurement: Executive Order 13693 (Section 3) requires that data center energy be monitored and recorded continuously down to the IT device level. Power usage effectiveness (PUE) is a measure of how efficiently a data center uses its power; specifically, how much power is actually used by the computing equipment (vs cooling and other energy overhead). ISD must provide PUE information, which is the basis for this requirement. This requires that the BCF implements energy efficient hardware and monitoring to allow us to track energy usage for data reporting

· IPDU Warranty: Must come with a minimum of a two year warranty that includes 100% parts, labor, and shipping. The warranty period must begin once the final acceptance is completed (as specified in the SOW).

· Temperature + Humidity Sensor Warranty: 2 Year Warranty

· IPDU Environment Temperature Requirements: The IPDUs must be able to work in an operating temperature of 140 F/60 C. Will be installed using hot aisle containment system; racks will be heavily loaded with network and IT server equipment. Some IT equipment, such as clusters, database servers, storage devices, etc. have high power ratings and therefore high heat outputs.

D. Installation & training requirements: N/A

E. Inspection, testing criteria, and final acceptance The IPDUs will be inspected by CFST staff within 30 days of delivery of all IPDUs to NIST. Any defects found during inspection will be corrected by the vendor prior to final acceptance. Once all IPDUs are accepted by NIST this constitutes final acceptance. Any defects encountered after final acceptance will be subject to the manufactures warranty.

F. Manufactures Warranty IPDUs: Must come with a minimum of a two year warranty that includes 100% parts, labor, and shipping. The warranty period must begin once the final acceptance is completed.

IPDU combo temperature and humidity sensors: Must come with 2 year hardware warranty.

G. Maintenance

See warranty section above.

H. Period of performance Vendor will deliver IPDUs and sensors up to 8 weeks after date of award.

I. Place of performance IPDUs will be delivered to NIST Boulder. See Delivery Requirements below.

J. Deliverables:

IPDU voltage/plug/outlet and sensor description and quantities required:

· 67 Raritan PX3-5459R-E2K2: 3PH Delta, 208V AC 30A (24A/PH rated) 18 x IEC320 C-13 locking outlets, 3 x NEMA 5-20R, plug; NEMA L21-30P (4P 5W), 8.6kVA 2U, Yellow

· 67 Raritan PX3-5459R-E2K6: 3PH Delta, 208V AC 30A (24A/PH rated) 18 x IEC320 C-13 locking outlets, 3 x NEMA 5-20R, plug; NEMA L21-30P (4P 5W), 8.6kVA 2U, Blue

· 11 Raritan PX3-5541R-E2K2: 3PH Delta, 208V AC 60A 24 locking outlets: 6 x IEC60320 C-13 12A, 12 x IEC60320 C19 16A plug: IEC 60309 3P+E (3P 4W), 17.3kVA, 2U, Yellow

· 11 Raritan PX3-5541R-E2K6: 3PH Delta, 208V AC 60A 24 locking outlets: 6 x IEC60320 C-13 12A, 12 x IEC60320 C19 16A plug: IEC 60309 3P+E (3P 4W), 17.3kVA, 2U, Blue

· 80 Raritan DPX2-T1H1: Single combo temperature and humidity sensor, field replaceable sensor modules, 13ft (4m) cable, RJ-12 connector.

Delivery requirements:

All deliveries will take place Monday thru Friday, during normal business hours (8AM – 4 PM). Deliveries will be made to the NIST Boulder OISM CF Site Manager:

Dawn Weller

(303) 497-6710; dweller@nist.gov NIST Boulder OISM CF Site Manager 325 Broadway Building 1- 4563 Boulder, CO 80305

Hardware delivery schedule & location:

· Vendor will deliver IPDUs and sensors up to 8 weeks after date of award.

· Prior to factory shipment, vendor will contact CF manager via email to finalize the exact delivery date and delivery location on the NIST Boulder campus. Because of minimal staffing and multiple deliveries, this is required to ensure personnel available to receive the delivery.

· On the formally agreed upon date (via email), vendor to deliver IPDUs and sensors to the Boulder NIST campus.

K. General Info:

· Vendor will employ delivery personnel eligible for NIST visitor badges.

· Vendor will provide names of delivery personnel to CF manager (via email) 5 business days prior to delivery so that the Visitors Center can be notified.

Requisition Number: NB184000-18-01211 9