A21-Technical Exhibit 3-1 _ FacilitySite Power Coordinator Evaluation Re....pdf

1ST THEATER TATICAL SIGNAL BRIGADE

Facility/Site Power Coordinator Evaluation Report for S12409 New Zoeckler ADN Camp Humphreys, South Korea

2 July 2018 – 11 July 2018

Prepared by:

1TTSB/S4 Engineering Brigade Power Coordinator

Unclassified/FOUO

MEMORANDUM FOR THE COMMANDER 16 July 2018

SUBJECT: Facility/Site Power Coordinator Evaluation Report for HMP TCF ADN

Camp Humphreys, South Korea

Reference: (a) NETCOM Regulation 420-1 Facility Engineering Site Power

Coordinator Program, 27 September 2010

1 Overview

The Brigade Power Coordinator Evaluation Team from 1TTSB/S4 Engineering conducted an evaluation of New Zoeckler ADN S12409 Camp Humphreys. This facility is operated and maintained by personnel of the 501st Signal Company, 41st Signal Battalion, 1st Tactical Theater Signal Brigade. This evaluation was conducted in accordance with reference (a) to determine the facility's readiness in sustaining its mission. This is accomplished by identifying deficiencies, items, and comments that affect current and future operations of the facility. The evaluation was conducted at New Zoeckler ADN, 2 July - 11 July 2018.

1.1 Site Overview

Figure 1: Floor Layout

Figure 2: Server Room Layout

Figure 3: UPS Room Layout

1.2 These categories are defined below:

1.2.1 Deficiencies: Problems that have resulted or will result in an operational impact to the mission and require immediate corrective action. May be beyond the capability of the site or facility to correct.

1.2.2 Item: An item that was documented during a subsequent audit and still exists without sufficient action.

1.2.3 Comments: Observations, either favorable or unfavorable, that warrant mentioning. Recommendations to improve the facility.

2 Site Power Coordinator Program

2.1 Deficiencies: (3)

2.1.1 Site Power Coordinator not appointed. The 41st needs to nominate a replacement for SSG Radouane Moukraj, IAW NETCOM 420-1 9. Estimated time to complete: 15 days.

2.1.2 Replacement needs to be trained in Electrical Safety (S4 to offer monthly electrical safety beginning in July) and emergency response (unit responsibility), NETCOM 420-1 11. Estimated time to complete: 60 days.

2.1.3 Site Power Coordinator Program mandates SOP be present onsite to deal with emergencies IAW NETCOM 420-1 18.a. If site power fails and DC batteries run out of energy (concurrent failure), then it would take an emergency work order to DPW to bring the site’s power back online. This could leave the site without power for an indeterminate time. This is not in line with service level agreements and mission. Estimated time to complete: 180 days.

2.2 Items: (0)

2.2.1 None Noted.

2.3 Observations & Comments: (0)

2.3.1 None Noted

3 Generators

General Description: The site has two power feeds supplying the two x GE ATSs.

Each ATS is coupled to two CUMMINS generators, one on each feed capable separately capable of supporting the entire facility. The facility also has a couple of dry type pad mounted transformers including K-Factor transformers. Downstream, there are 4 x 100 kVA/90 kW GE UPS supplying the technical loads on UPA1, UPB1, UPA2 and UPB2 400A panels (including bypass function). The facility also has a DC SAGEON power inverter with batteries which is supplying the DC power to the transport equipment.

Two 2 x 1125 KVA Cummins generators, one on each feed capable separately capable of supporting the entire facility are installed to supply as an emergency power source. These generators supply way more capability than the observed load. The generators are connected to a 3000 gallon fuel tank that had approximately 2200 gallons in the tank. The condition of the generators system are in an okay condition.

But this equipment has 1 deficiency, 1 observation and comment.

3.1 Deficiencies: (1)

3.1.1 There has been regular generator exercising/testing. The site’s generators have not be serviced since November 2017. This is a simple DPW work order to be completed by facility manager. NETCOM 420-1 9.d.(1). The Site NCOIC or Site Power Coordinator to submit FEWR to Building Manager and track DPW Maintenance. DPW should perform the maintenance. Estimated time to complete: 30 days.

3.2 Items: (0)

3.2.1 None Noted.

3.3 Observations & Comments: (1)

3.3.1 The BPC team performed a generator readiness test on both generator A & B, the test should be performed following maintenance.

Time Action 1023 A-Side Main Power Disconnected

1023 A-Generator cranked, providing L1: 99 A, L2: 113 A, L3: 113 A

1024 UPS no issues carrying the load on A-Bus

1026 B-Side Main Power Disconnected

1026 B-Generator cranked, providing L1: 29 A, L2: 36 A, L3: 31 A

1027 UPS no issues carrying the load on B-Bus

1033 A-Side Main Power Connected

1045 A-Generator Shutdown, 5 min transfer time and 5 min cooldown delay on ATS, plus additional cooling required by generator

1046 B-Side Main Power Connected

1058 B-Generator Shutdown, 5 min transfer time and 5 min cooldown delay on ATS, plus additional cooling required by generator

1100 Back to the normal condition.

Table 1: Gen. Test Timeline

4 Power Distribution

This facility has two server rooms access to the server room 2 was restricted for security reasons. So BPC team performed only server room 1 and UPS room audit.

Regarding power distribution system above the racks in the server room 1, the 400A rating panel boards UPA2/UPB2 are supplying the power to the racks via LS Cable & system buses. The power system for DC required racks is configured to power SAGEON DC inverters (requiring 50 Amps x 45/48EA rating) via panel board “BMP” and “BMP-DC” including dry type transformer. Server room 1 has 5 rows of racks consuming less than AC 30 amps. There are 4 deficiencies and 2 comments.

Figure 4: UPA2/UPB2 Panel board

Figure 5: SAGEON DC Inverter

4.1 Deficiencies: (5)

4.1.1 The abandon wires were observed on the cable tray during the audit. The National

Electrical Code is very directive concerning accessible abandon conductors and its impacts to LHS. Recommend the 41st installers secure these cables with proper materials or remove them and mark for future use. Anticipated completion 30 days from report acceptance.

Figure 6: Unsecured wires

4.1.2 Damaged RBB and uninstalled RBB were observed. The RBB shall be installed horizontally or vertically on the rack using insulators IAW TIA 607C 7.4. The site NCOIC or Site Power Coordinator request to 41st installation team. Anticipated completion 90 days from report acceptance.

Figure 7: Damaged and Uninstalled RBBs

4.1.3 Flexible conduit connection is not installed properly. If left unrepaired, there is no effective ground fault current pathway. Metal parts of conduit must be connected to the supply source via an effective ground fault current pathway IAW NEC

250.4. The site NCOIC or Site Power Coordinator request to 41st installation team. Anticipated completion 90 days from report acceptance.

Figure 8: Improper flexible conduit connection

Damaged

RBB

Uninstalled

RBB

4.1.4 The DC PDUs in racks 5.02 & 5.09 are displaying improper current values. If left unrepaired, might be confused to read actual current status. The PDU should be maintained IAW NETCOM 420-1 9. D. The site NCOIC or Site Power Coordinator request to 41st installation team. Anticipated completion 90 days from report acceptance.

Figure 9: Improper Current Value in PDU

4.1.5 3 Sageon alarms were displayed. If left alone on, these alarms might induce

Sageon and/or battery malfunctions. Recommend the site NCOIC or Site Power Coordinator work with 41st MST.

Figure 10: 3 Alarms at Sageon display

4.2 Items: (0)

4.2.1 None noted.

4.3 Observations & Comments: (2)

4.3.1 Measured power consumption of building was documented. The site is using

7% of capacity except DC power. Running generators may issues… as below 25% can create engine issues.

DC PDU

Bus IA IB IC N

UPA2 26 24 19 5

UPB2 30 26 22 3

Table 2: PQM Summary

Figure 11: PQM Readout UPA2

Figure 12: PQM Readout UPB2

4.3.2 Measured power consumption of SAGEON board was documented. As shown on the result of Generators running test in section 3.3.1 and power consumption of UPA2/UPB2 in section 4.3.1, we can see that the SAGEON’s power consumption was incorporated with switchboard “A”. The SAGEON receiving the power from switchboard “A” via the panel board “BMP”. The BMP, which providing power to the SAGEON, consist of two main power circuit breakers which are connected its power source from switchboard “A” and “B”. These two main CBs are interlocked each other to prevent simultaneous turn on.

Figure 13: Ampere Readout SAGEON

5 Figure 41: Rack 5.14, 5.16

Figure 42: Battery Status

Figure 43: SAGEON Status

Figure 44: SAGEON Distribution List – A

Figure45: SAGEON Distribution List – B

6 HVAC

There are 2 x Carrier chillers supply 9 x DATA Aire CRAC units in this facility. 2 x CRAC in server room 2 and 1 x CRAC in ROK server room couldn’t audit due to access issue. These equipment have one deficiency.

6.1 Deficiencies: (1)

6.1.1 The ACU indoor unit in the electrical room was not operating properly. The

HVAC system should be operated and maintained IAW NETCOM 420-1 9. D.

(2). Recommend DPW inspect and maintain this unit to secure required room temperature. Anticipated completion 90 days from report acceptance.

Figure 46: Inoperable ACU in Electrical Room

6.2 Items: (0)

6.2.1 None noted.

6.3 Observations & Comments: (0)

6.3.1 None noted.

7 Lightning Protection System, Earth Grounding System & Bonding

Description: There are no damaged conductors in the ground test wells which are located at the adjacent of facility. The facility has one anticorrosive busbars. There are 1 deficiency and 1 comment.

7.1 Deficiencies: (1)

7.1.1 No anticorrosive on BGB (Building Grounding Busbar), PBB (Primary Bonding

Busbar) and Secondary Bonding Busbar (SBB). TIA-607C 7.2.1, 7.3.1 & 7.4.1, 1st paragraph. The busbar shall be cleaned and an anti-oxidant should be applied prior to fastening connectors to the busbar. Site NCOIC or Site Power Coordinator. Anticipated completion 90 days from report acceptance.

Figure 47: Bldg. SBB & Improper lugs and connector

7.2 Items: (0)

7.2.1 None noted.

7.3 Observations & Comments: (1)

7.3.1 No recent EGS or bonding testing available for viewing. S4 lacks the Megger to inspect resistances between bonds and coupling. NETCOM 420-1 23.a.(4).

Grounding systems installed in accordance with MIL-STD-l88-l24 and MIL- HDBK-4l9 are properly maintained. DPW should preform ground ring testing every 21 Months per NETCOM 420-1 (TM 5-690 Chapter 7 – testing should occur 24 mo.) and provide results to unit. The Site NCOIC or Site Power Coordinator shall submit FEWR to Building Manager, track DPW Maintenance, & maintain copies of tests. Anticipated completion 180 days from report acceptance.

Figure 48: Empty Fire Extinguisher

8 Fire Suppression System (FSS)

Description: The condition of the fire suppression system are in an okay condition, but there is one comment.

Figure 49: Empty Fire Extinguisher

8.1 Deficiencies: (0)

8.1.1 None noted

8.2 Items: (0)

8.2.1 None noted.

8.3 Observations & Comments: (1)

8.3.1 No recent testing available for viewing. NETCOM 420-1 25. Fire protection systems may vary from simple hand held fire extinguishers to sophisticated installed water or chemical extinguishing systems. All systems require periodic testing or inspection, usually on an annual basis or as stated below in table 1.

DPW is to preform FSS testing. Site NCOIC or Site Power Coordinator shall submit FEWR to Building Manager, track DPW maintenance, and maintain copies of tests.

9 Description: Facility/Site Power Coordinator Evaluation Team consisted of the following individuals:

Joe Dubinsky, Project Management Professional, NPFA 70E Qualified, 1TTSB Site Power Coordinator, Electrical/Facility Engineer

Mr. Chi Yong Chon, Project Management Professional, Certified Information Systems Security Professional, 1TTSB/S4 Engineering, Electronics Engineer

Mr. Jin Hur, Professional Engineer, 1TTSB Engineering, Electrical Engineer

The 1TTSB/S4 Engineering Team wish to express their gratitude for the assistance provided by the personnel of the Camp Humphreys.

JOSEPH G. DUBINSKY

Brigade Power Coordinator joseph.g.dubinsky.civ@mail.mil

DSN: 315-755-0882

COM: 0503-355-0882

mailto:joseph.g.dubinsky.civ@mail.mil