Attachment 2 - C108435 SOP Water Treatment KPIs.docx.pdf

Attachment 2

Instructional Document

Abstract This document will serve as a guide to Central Utility Plant water treatment.

NATIONAL INSTITUES OF HEALTH (NIH) – BETHESDA, MD

Division of Technical Resources (DTR) October 23, 2020 EFAM‐ADMIN‐01 Revision 1.0 Page 1

STANDARD OPERATING PROCEDURE

Title: SOP for Water Treatment KPI

Document Number: Department: Division of Technical Resources (DTR)

Version Number: 2.0 Effective Date: 05/21/2021

Revision 1.0 Page 2

Water treatment is a very crucial part of CUP operations. Water treatment providers perform their tasks to maintain water/steam quality according to the prescribed key performance indices which ensure minimum scaling and corrosion in CUP boiler systems and chiller systems, as per KPI. They use online and batch devices that produce many data points. CUP staff also performs rounds and tests to produce data, entering manually collected data to the data historian through the CUP ChemEntry page. The weekly lab data is parsed through the weekly uploader on the website. The CUP operation key performance index (KPI) represents the real time system operation. Most of the CUP operation KPIs are set as hard limits, that NIH is required to achieve. Some CUP operation KPIs are set as targets (see notes under each KPI), which means it is either not entirely under CUP’s control or a known issue to the CUP and requires an on‐going effort to mitigate.

2.1 SOFTENER EFFLUENT

2.1.1 TOTAL HARDNESS PPM AS CACO3 (<1 PPM)

Potential causes: The ion‐exchange resin can be exhausted and could not be regenerated. The regeneration process is not performed physically, but the software shows completed. Online hardness analyzer is reading incorrectly, or not properly maintained or calibrated. Any mechanical issue in the softener equipment. Brine level is too low. Brine flow is clogged.

Action: Check if softener regeneration is working properly, check the quality of resin beads.

Alarm notification: See attachment

Reporting mechanism:

Operator: When noticed, will inform supervisor.

CUP operation: When noticed on boiler log, will inform supervisor

CUP Engineers: When receive email notification and notice on water dashboard during daily review, will inform CUP operation, plant chief and supervisor.

2.2 REVERSE OSMOSIS INFLUENT

2.2.1 PH (6.0 – 9.0)

Potential causes: pH of city water has been changed. Any microbio growth of the line after softeners.

Check caustic pumps for proper operation.

Action: Check if caustic feed line clogged or feeding pump not working.

Revision 1.0 Page 3

Alarm notification: See attachment

Reporting mechanism:

Operators: When noticed, will inform supervisor.

CUP operation: When noticed on RO log/display, will check onsite and inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during data review, will consult with CUP operation.

2.2.2 ORP (< 500 MV)

Potential causes: city water or IWS water contains high level of chlorine. Bisulfite feed for dechlorinators is either clogged, or pump is not working, or feed rate is low.

Action: Check if bisulfite feed line clogged or feeding pump not working.

Alarm notification: See attachment

Reporting mechanism:

Operators: When noticed on RO log, will inform supervisor.

CUP operation: When noticed on RO log/display, will check onsite and inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

2.2.3 SILT DENSITY INDEX (<3)

Potential causes: If SDI is measured after RO cartridge filter, it may happen that the cartridge filter is already exhausted. The city water is dirty. Softener’s resin is contaminated, or silt deposited in the bed.

Debris has been collected anywhere in the line from city water up to RO.

Action: Check if membrane fouling occurred. Use biocides/CIP to clear up the microbio. The SDI needs to be taken after the RO cartridge filter.

Alarm notification: See attachment

Reporting mechanism:

Operators: None

CUP operation: When noticed after test, will check onsite and inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

2.3 REVERSE OSMOSIS EFFLUENT

Revision 1.0 Page 4

2.3.1 SALT REJECTION ( > 97%)

Potential causes: RO membrane has been fouled. May need replacement. MB grows at membrane level. Need cleaning. Too many contaminants in the city water. Softener is not working properly.

Action: For RO1 and RO2: Perform high/low pH CIP. Check if there is degradation on the membrane, or an o‐ring leak. Inlet valve adjustments, recycle valve adjustments.

Note: The conductivity KPI is for RO1 and RO2. E8 effluent typically is in the 30 – 40 uS/cm due to high influent conductivity. For RO E8: Adjust the recovery rate, make sure the recovery rate is around 65% based on flow. If it is lower than 50%, check if a membrane cleaning is needed.

Alarm notification: See attachment

Reporting mechanism:

Operator: When noticed on RO log, will inform supervisor.

CUP operation: When notice on RO log/display, will check onsite and inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

2.4 CONDENSATE

2.4.1 AMINE (< 10 PPM CYCLOHEXYLAMINE, < 15 PPM DEAE)

Potential causes: amine can be overfed if plant/steaming rates change and pump is not adjusted properly Amine pump can be defective. If the amine value is negligible, there may occur campus‐wise steam condensate loss. Recycled amines may be removed through condensate polishers. Some acidity formed in the condensate line either through additional carbon dioxide source, or acidic contamination.

This will reduce amine concentration in the condensate.

Action: Check if the amine pump is working, if there are amine losses through condensate return loss.

Check if there are any vented condensate returns, and deaerator venting.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When notice on PI display, will inform supervisor.

2.4.2 PH (8.0 – 9.3)

Revision 1.0 Page 5

Potential causes: If pH of the condensate is less than 8, some carbonic acid formed in the condensate line, or some acidic contamination is present and is neutralized by amines, If the pH of the condensate is higher than 9.3, there may be boiler carry over that will increase the pH of the condensate.

Contamination of condensate return from campus to CUP can impact pH readings.

Action: Check the amine pump and feed rate, check for boiler carryover, condensate loss and condensate contamination. Check the amine level in analyzers.

Note: This KPI is for standard operation and does not include condensate contamination or boiler carryover.

Alarm notification: See attachment

Reporting mechanism:

Operator: When noticed on boiler log, will inform supervisor.

CUP operation: When notice on boiler log/PI display, will check onsite and inform supervisor.

2.4.3 TOTAL HARDNESS (< 5 PPM)

Potential causes: Boiler carry‐ over can slightly increase the hardness. City water may be contaminating the condensate return. Condensate contaminated due to leaking heat exchangers around the campus.

Some other contamination gets added in the steam distribution line.

Action: Check for boiler carryover and condensate contamination. Check points in tunnels/outlying buildings previously identified. Notify DFOM.

Alarm notification: See attachment

Reporting mechanism:

Operator: When noticed on boiler log, will inform supervisor.

CUP operation: When notice on boiler log/display, will check onsite and inform supervisor.

2.4.4 TOTAL IRON (0.2 PPM)

Potential causes: corrosion in iron piping. Underfed amines. Low pH. Air intrusion in piping.

Revision 1.0 Page 6

Action: Check the possibility of carbonic acid attack and oxygen pitting.

Note: This is a target under normal operations.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive email notification, will inform supervisor.

2.4.5 TOTAL COPPER (0.1 PPM)

Potential causes: corrosion of copper at heat exchanger. Underfed amines. Air intrusion in piping.

Condensate contamination from campus.

Action: Check the pH level, amine dosing and oxygen content. Bring them to KPI levels.

Note: This is a target under normal operations.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive email notification, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

2.4.6 TOTAL ORGANIC CARBON (< 100 PPB)

Potential causes: Condensate contamination. New piping installed in the steam line without prior treatment of the pipes. MB growth in the steam line.

Action: Check for internal deposition and carryover, contamination of oily materials. Need to run tests to establish a baseline before setting up a firm KPI level.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive email notification, will inform supervisor.

Revision 1.0 Page 7

CUP Engineers: When receive email notification and notice on water dashboard or PI during

NIH: When notice on the weekly report, will inform CUP operation and CUP engineers

2.5 CONDENSATE POLISHER EFFLUENT

2.5.1 TOTAL HARDNESS (0.3 PPM)

Potential causes: Polisher resin exhausted. Regeneration not working. Iron contamination at resin bed.

Biofouling.

Action: Check quality of resin beads, biofouling on the resin. Check the polisher regeneration process and make sure it is working.

Note: This is a target. If color is indicated by the wet test, regeneration is needed. If not meeting the KPI level, PM work is needed on the polishers.

Alarm notification: See attachment

Reporting mechanism:

Operator: When noticed on boiler log, will inform supervisor.

CUP operation: When notice on boiler log, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on boiler log, will inform CUP operation and CUP engineers

2.5.2 TOTAL IRON (0.1 PPM)

Potential causes: pH of condensate return is at acidic range. Corrosion issues either in piping or storage tank. Polisher bed may need iron cleaning. City water may have high iron content. Condensate contamination.

Action: Check if a resin cleaning is required. Initiate a regeneration cycle. If resin cleaning occurred regularly, check pH and amine level of influent and effluent. Check resin cleaning feed system as an additional troubleshooting measure.

Note: Iron cleaning is also needed if polishers are discharging water with iron content higher than what is coming in. If not meeting the KPI level, PM work is needed on the polishers.

Alarm notification: See attachment

Reporting mechanism:

Revision 1.0 Page 8

CUP operation: When receive email notification, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

NIH: When notice on weekly report, will inform CUP operation and CUP engineers

2.5.3 TOTAL COPPER (< 0.05 PPM)

Potential causes: condensate at low pH. corrosion of copper at heat exchanger. Underfed amines. Air intrusion in piping. Condensate contamination.

Action: Check if a resin cleaning is required, check the pH and amine level of influent and effluent.

Note: Resin cleaning is also needed if polishers are discharging water with copper content higher than what is coming in. if not meeting the KPI level, PM work is needed on the polishers.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive email notification, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.6 DEAERATOR/BOILER FEEDWATER

2.6.1 DISSOLVED OXYGEN (< 7 PPB)

Potential causes: Sulfite underfed. Air intrusion. Deaerator may have mechanical issue, not efficiently removing oxygen. Sulfite pump may be plugged, or malfunctioning.

Action: Check if there is any mechanical issue on the deaerator, temperature and pressure. Check the sulfite pump status. Check the status of the dissolved oxygen sensor.

Note: If there is an issue, the CUP will not begin to address it chemically, will try to correct it operationally or mechanically. If the oxygen analyzer is not working properly, dissolved oxygen field test is needed.

Alarm notification: See attachment

Operator: When notice on boiler log, will inform supervisor

Revision 1.0 Page 9

CUP operation: When notice on boiler log, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

2.6.2 TOTAL IRON (< 0.1 PPM)

Potential causes: Feed water contamination. Iron corrosion issues in the piping or deaerator. pH meter may not read properly. Oxygen level higher than KPI. Air intrusion or pipe leaking.

Action: Check the pH, amine level, and feed water conductivity. Check if there is any leak by on the emergency fill bypass of the DA.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive email notification, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.6.3 TOTAL COPPER (<0.05 PPM)

Potential causes: contaminated steam condensate return. corrosion of copper at heat exchanger. pH meter may not read properly. Underfed amine. Air intrusion in piping.

Action: Check the pH, amine level and feed water conductivity.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive email notification, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.6.4 PH (8.3 – 10)

Potential causes: Underfed sulfite, pump may be malfunctioning. Mechanical issue at the deaerator or the vent. Air intrusion. Corrosion issue. Line leaking.

Revision 1.0 Page 10

Action: Check the sulfite (KOH is part of the sulfite program) pump, check if there is any mechanical issue with the deaerator and vent. Check treated water tank pH, if it is low check the post RO caustic injection.

Alarm notification: See attachment

Reporting mechanism:

Operator: When notice on boiler log, inform supervisor

CUP operation: When notice on boiler log, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

2.7 BOILER WATER

2.7.1 OH ALKALINITY (> 100 PPM)

Potential causes: Underfed of caustic at RO level. Intrusion of air through leakage. Pump failure for caustic and sulfite.

Action: Check the silica level and pH and boiler water. Check the RO caustic pump status.

Note: Run the calculated OH alkalinity. Also test for the OH alkalinity with titration method.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive email notification, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.7.2 M ALKALINITY (< 700 PPM)

Potential causes: high cycle of concentration, defective blowdown, high pH of feed water than KPI; high feed water alkalinity, excessive caustic fed at RO, high pH or alkalinity of city water, Action: Check if the boiler polymer pump and the RO caustic pump are working. Check if the blowdown works.

Note: The KPI might not be possible to achieve if RO is not operating and the boiler plant is on soft water makeup. If the RO is under bypass, the conductivity setpoint should be increased.

Revision 1.0 Page 11

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive email notification, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.7.3 P ALKALINITY (300 ‐ 700 PPM)

Potential causes: Underfed of caustic at RO level. Intrusion of air through leakage. Pump failure for caustic and sulfite.

Action: Check if the RO caustic pump works. Check the sulfite pump status (KOH is part of the sulfite program).

Note: If this specification is too low at times of heavy condensate contamination, this will become the limiting factor for boiler operations. 300 might not be achievable when on RO water without caustic feed.

Alarm notification: See attachment

Reporting mechanism:

Operator: When notice on boiler log, inform supervisor

CUP operation: When notice on boiler log, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

2.7.4 SULFITE (30‐50 PPM)

Potential causes: underfed or overfed of sulfite, pump malfunctioning, oxygen sensor malfunctioning.

Air intrusion.

Action: Check if the sulfite pump at deaerator works. Check if there is any intrusion or leak or air.

Note: Feed water economizer tube corrosion issue will happen if the sulfite level is too low. Request change to the erythorbate/sulfite program for improved metal passivation with lower influence on the boiler conductivity.

Alarm notification: See attachment

Revision 1.0 Page 12

Operator: When notice on boiler log, inform supervisor

CUP operation: When notice on boiler log, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

2.7.5 CONDUCTIVITY (UNNEUTRALIZED, < 4300 US/CM)

Potential causes: high cycle of concentration, defective blowdown, high pH or alkalinity of city water, conductivity meter defective, high conductivity of city water.

Action: Check if blowdown works as programmed.

Note: This KPI is for standard operation conditions when RO is online. It will likely be the limiting factor when the ROs are bypassed.

Alarm notification: See attachment

Reporting mechanism:

Operator: When notice on PI display, inform the supervisor

CUP operation: When receive email notification, will check onsite, and inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

2.7.6 SILICA (< 35 PPM)

Potential causes: High silica level of city water, RO is not working properly, low OH alkalinity, boiler over‐cycled, requires setpoint adjustment, boiler blowdown is not functioning correctly.

Action: Reduce the cycle of concentration, check the silica level at RO permeate, check the silica level in the condensate

Alarm notification: See attachment

Note: This KPI is for standard operation when RO is online. The KPI might not be possible to achieve when RO is not running.

Reporting mechanism:

Operator: When notice on PI display, inform supervisor

CUP operation: When receive email notification, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

NIH: When receive email notification, will check onsite

Revision 1.0 Page 13

2.7.7 CYCLES OF CONCENTRATION (BASED ON SILICA, 100 WHILE MAINTAINING SILICA <35 PPM)

Potential causes: defective blowdown, high pH or alkalinity of city water, conductivity meter defective, high conductivity of city water, incorrect silica analysis, feed water contamination.

Action: Maintain the cycles of concentration under the condition silica level does not exceed 35 ppm.

Alarm notification: See attachment

Reporting mechanism:

Operator: When notice on PI display, inform supervisor

CUP operation: When receive email notification, will check onsite and inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.8 BOILER

2.8.1 HEAT TRANSFER SURFACE DEPOSIT THICKNESS (< 1/64”)

Potential causes: underfed of boiler polymer, fouling occurred, accumulation of solid particles, biofouling, corrosion in the boiler system, Action: Cut a window at the appropriate locations and analyze for deposit weight density, make sure it is magnetite, and recommend cleaning based on the results.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive the inspection report and notice on water dashboard or PI during

NIH: When receive the inspection report, will consult with CUP operation and CUP engineers.

2.8.2 STEAM & MUD DRUMS, STEAM SEPARATORS (NO DEPOSITS, NO ACCUMULATED SOLIDS, NO

DEPOSITION ON STEAM SEPARATORS, NO PITTING)

Potential causes: underfed of boiler polymer, fouling occurred, accumulation of solid particles, biofouling, corrosion in the boiler system, Revision 1.0 Page 14

Action: Open the mud drum and do a visual inspection. Do a bulk sample if necessary. Check pump status of chemical addition. Determine if chemical cleaning is needed.

Note: Layup must be performed properly for offseason boilers not to suffer from pitting corrosions.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive the inspection report and notice on water dashboard or PI during

NIH: When receive the inspection report, will consult with CUP operation and CUP engineers

2.8.3 NUMBER OF WATER‐RELATED BOILER FAILURES (ZERO FAILURES)

Potential causes: underfed of boiler polymer, fouling occurred, accumulation of solid particles, , corrosion in the boiler system, Action: Cut a window at the appropriate locations and send it to the lab.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive the inspection report and notice on water dashboard or PI during

2.8.4 NUMBER OF WATER‐RELATED STEAM FAILURES (ZERO FAILURES)

Potential causes: Corrosion, underfed of boiler polymer, fouling occurred, accumulation of solid particles, biofouling, Action: Cut a window at the appropriate locations and send it to the lab.

Alarm notification: See attachment

Revision 1.0 Page 15

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive the inspection report and notice on water dashboard or PI during

2.9 COOLING TOWER RECIRCULATING WATER

2.9.1 CHLORINE DIOXIDE GENERATOR EFFICIENCY (>85% BASED ON CHLORITE CONSUMPTION)

Potential causes: The chlorine dioxide generator is not working properly. The CD23 precursor chemical deteriorated.

Action: Check the chlorine dioxide generator, the generator mixer, venturi pump.

Alarm notification: See attachment

Reporting mechanism:

Operator: none

CUP operation: When receive the report, consult with CUP engineers

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When receive email notification, will check onsite

2.9.2 CHLORINE DIOXIDE CONCENTRATION AFTER GENERATOR (>1500 PPM)

Potential causes: The chlorine dioxide generator is not working properly. The CD23 precursor chemical deteriorated.

Action: Check the chlorine dioxide generator, the generator mixer, venturi pump.

Alarm notification: See attachment

Note: The high efficiency chlorine dioxide generator produces the chlorine dioxide concentration at greater than 1500 ppm. If the concentration is less than 500 ppm, that means an emergency and there is no chlorine dioxide residual in the cooling towers.

Reporting mechanism:

Operator: none

CUP operation: When receive the email notification, consult with CUP engineers

CUP Engineers: When receive email notification and notice on water dashboard or PI during

Revision 1.0 Page 16

2.9.3 CHLORINE DIOXIDE CONCENTRATION AT THE CONTROLLER “AS LOW AS REASONABLY ACHIEVABLE

“(ALARA)

Potential causes: The chlorine dioxide generator is not working properly. The CD23 precursor chemical deteriorated. The chlorine dioxide sensors at the controller may require service.

Action: Check the chlorine dioxide generator. Check the chlorine dioxide sensors at the controller of each cooling tower, replace the membrane if necessary.

Note: The ClO2 sensor in the controllers may need to be replaced. Will start taking measurements when the IWS injection and new generator is in place.

Alarm notification: See attachment

Reporting mechanism:

Operator: none

CUP operation: When receive the email notification, consult with CUP engineers

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.9.4 PH (8.3 – 9.0)

Potential causes: Acid overfeeding at the chlorine dioxide generator. Cooling tower over cycling.

Decreased cooling tower makeup water conductivity.

Action: If pH is greater than 9.0, decrease the cooling tower COC. If pH is lower than 8.3, increase the COC, trouble shoot the chlorine dioxide generator to find out if the sulfuric acid is overfeeding.

Alarm notification: See attachment

Reporting mechanism:

Operator: When notice on PI display, inform the supervisor

CUP operation: When receive the email notification, consult with CUP engineers

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.9.5 LSI (2.0 – 2.8)

Revision 1.0 Page 17

Potential causes: High total alkalinity and calcium hardness of cooling tower water will increase the LSI, and vise versa. LSI will increase as the cycles of concentration and pH increase.

Action: Adjust the cooling tower cycle of concentration, monitor the tower pH, calcium hardness and total alkalinity.

Alarm notification: See attachment

Reporting mechanism:

Operator: When notice on PI display, inform the supervisor

CUP operation: When receive the email notification, consult with CUP engineers

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.9.6 TOTAL IRON (< 1.0)

Potential causes: Corrosion in the cooling tower from various causes. High iron concentration in the cooling tower makeup.

Action: Check for microbiological induced corrosion. Measure the operating pH, LSI, chloride, total suspended solids. Check if the side stream filter is operating properly, check makeup water source for potential high iron

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive email notification, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.9.7 TOTAL COPPER (<0.1)

Potential causes: Corrosion in the cooling tower from various causes. High copper concentration in the cooling tower makeup.

Action: Examine the operating pH, LSI, chloride, TDS, sulfate. Check the azole level in the cooling tower.

Check if the side stream filter is operating properly, check makeup water source for potential high iron

Revision 1.0 Page 18

Reporting mechanism:

Operator: None

CUP operation: When receive email notification, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on weekly report, will inform CUP operation and CUP engineers

2.9.8 HALOGEN STABILIZED AZOLE (>1.5PPM FREE AND AVAILABLE)

Potential causes: Chlorine contamination, azole injection problem.

Action: Increase the HST injection rate based on HST probe reading.

Note: This is measurable azole level, which includes free and available. The CUP recently switched to HST, don’t have enough data yet. Propose 1.5 ppm as the baseline.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When notice on weekly report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on weekly report, will inform CUP operation and CUP engineers

2.9.9 ATP (<100 PG/ML)

Potential causes: High microbiological growth from the cooling tower makeup. Low chlorine dioxide residual level. Dead legs. Warm ambient temperature.

Action: Hyperhalogenation (Please refer to the SOP in the attachment).

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on the report, will perform hyperhalogenation, inform CUP operation and CUP engineers

2.9.10 BSI (BIOMASS STRESS INDEX, >45%)

Revision 1.0 Page 19

Potential causes: Low chlorine dioxide residual level in the condenser water system. The ATP analyzer reagent may need to be replaced. The cooling tower is not running when the measurement was taken.

Action: Make sure the ATP analyzer is measuring accurately. Check the chlorine dioxide residual level in the basin.

Note: Requires additional tests to establish a meaningful KPI level.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive email notification, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on weekly report, will inform CUP operation and CUP engineers

2.9.11 HETEROTROPHIC BACTERIA COUNT (TBD BASED ON REVISED COOLING TOWER SHOCK PROTOCOL)

Potential causes: Low chlorine dioxide residual level in the basin. Dead leg. Warm ambient temperature.

Water stagnation in the offline system for an extended period of time, low dose of non‐oxidizing biocide feed.

Action: Hyperhalogenation (Please refer to the SOP in the attachment).

Note: May require adjustment of feed frequency and feed rate of non‐oxidizing biocide.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

NIH: When notice on the report, will check onsite, inform CUP operation and CUP engineers

2.9.12 SESSILE BACTERIA COUNT (105 CFU/ML)

Potential causes: Low chlorine dioxide residual level in the condenser water system. No flow in the biofilm coupon rack Flow recirculation not successfully executed, low dose of non‐oxidizing biocide feed.

Action: Measure the chlorine dioxide residual concentration in the cooling tower basin. Check the generator and ClO2 injection to the IWS tank makeup. Check if the recirculation protocols are executed regularly

Revision 1.0 Page 20

Note: This is a target under normal operations. Put plastic mesh coupon on 23 and 24 to start the testing. Swab a plastic mesh coupon for sessile bacteria measurement.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on the report, will check onsite, inform CUP operation and CUP engineers

2.9.13 IRON‐OXIDIZING BACTERIA (50 CFU/ML)

Potential causes: Low chlorine dioxide residual level in the condenser water system. Downtime microbiological growth. No flow in the biofilm coupon rack, flow recirculation not successfully executed.

low dose of non‐oxidizing biocide feed.

Action: Measure the chlorine dioxide residual concentration in the cooling tower basin. Check the generator and ClO2 injection to the IWS tank makeup. Change the non‐oxidizing biocide dosing and feed schedule. Check for ferrous oxides. Check if the recirculation protocols are executed regularly

Note: This is a target under normal operations. Take water samples at the condenser barrel and tower basin for measurement.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.9.14 SLIME‐FORMING BACTERIA (100 CFU/ML)

Potential cause: Low chlorine dioxide residual level in the condenser water system. Downtime microbiological growth.

Action: Measure the chlorine dioxide residual concentration in the cooling tower basin. Check the generator and ClO2 injection to the IWS tank makeup. Check if the recirculation protocols are executed

Revision 1.0 Page 21

Note: This is a target under normal operations. The KPI is for flowing water, stagnant water cannot be treated.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.9.15 NITRIFYING BACTERIA (100 CFU/ML)

microbiological growth. low dose of non‐oxidizing biocide feed.

Action: Measure the chlorine dioxide residual concentration in the cooling tower basin. Check the generator and ClO2 injection to the IWS tank makeup. Check if the recirculation protocols are executed regularly

Note: This is a target under normal operations.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.9.16 SULFATE REDUCING BACTERIA (10 CFU/ML)

microbiological growth. Low dose of non‐oxidizing biocide feed

Action: Measure the chlorine dioxide residual concentration in the cooling tower basin. Check the generator and ClO2 injection to the IWS tank makeup. Check if the recirculation protocols are executed

Note: This is a target under normal operations.

Revision 1.0 Page 22

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.9.17 ACID PRODUCING BACTERIA (AS LOW AS REASONABLY ACHIEVE CFU/ML)

microbiological growth. low dose of non‐oxidizing biocide feed

Action: Measure the chlorine dioxide residual concentration in the cooling tower basin. Check the generator and ClO2 injection to the IWS tank makeup. Check if the recirculation protocols are executed regularly

Note: This is a target under normal operations.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.9.18 LEGIONELLA (NON‐DETECTABLE)

Potential cause: Low chlorine dioxide residual level in the condenser water system. Downtime growth.

low dose of non‐oxidizing biocide feed, flow recirculation not successfully executed.

Action: Measure the chlorine dioxide residual concentration in the cooling tower basin. Check the generator and ClO2 injection to the IWS tank makeup. Hyperhalogenation as needed. Check if the recirculation protocols are executed regularly

Revision 1.0 Page 23

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on the report, will perform hyperhalogenation, inform CUP operation and

2.9.19 CARBON STEEL CORROSION (COUPON, <1.0 MPY)

Potential cause: Microbiological growth, air intrusion. Corrosion inhibitor excursions.

Action: Check for microbiological induced corrosion. Measure the cooling tower operating pH, LSI, chloride and total suspended solids. Check corrosion inhibitor levels.

Note: If the system is operating normally, and the coupon results are exceeding the KPI limits, additional actions need to be taken.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.9.20 COPPER CORROSION (COUPON, <0.1 MPY)

Potential cause: low azole level, microbiological growth, air intrusion.

Action: Measure the cooling tower operating pH, LSI, chloride, total suspended solids, sulfate levels.

Check mechanical, operational, chemical and automation.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

Revision 1.0 Page 24

2.9.21 CARBON STEEL CORROSION (COUPON, NO PITTING)

Potential cause: microbiological growth, air intrusion.

Action: Check for microbiological induced corrosion. Measure the operating pH, LSI, chloride, total suspended solids. Check mechanical, operational chemical and automation. Look at the chemical residual levels, make sure the corrosion program is operating within the specification. Check for any operation upsets. Initiate root case analysis.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.9.22 HEAT TRANSFER SURFACE DEPOSIT THICKNESS (< 1/100”)

Potential cause: High suspended solids level in water. Microbiological induced corrosion. Overcycling of cooling towers, exceeding the LSI control limits. Heat exchanger effectiveness, heat not removed effectively from the refrigerant.

Action: Conduct a root cause analysis to identify source of deposits (such as baffle plate issues)

Note: Total suspended solids cannot be prevented by chemical treatment.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.9.23 BIOFOULING (NO BIOFOULING ON TOWER DECK OR FILL, NO BIOMASS OR SCALE ON LOUVERS OR

SPRAY SYSTEMS)

Potential cause: Sun light, warm weather.

Revision 1.0 Page 25

Action: Hard brush with 1% hypochlorite, low pressure jet washing.

Note: This is a target under normal operations.

Alarm notification: See attachment

Reporting mechanism:

Operator: When notice any fouling, inform the supervisor

CUP operation: When notice any fouling, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice any fouling, will check onsite, perform actions and inform CUP operation and

2.10 CHILLED WATER CLOSED LOOP

2.10.1 CHLORINE DIOXIDE RESIDUAL LEVEL (ALARA)

Potential cause: large amount of chilled water loss, chemical pump failure, chlorine dioxide generator not working properly.

Action: Check the chlorine dioxide generator efficiency. Measure the free chlorine concentration in the product stream. Troubleshoot the chlorine dioxide pump.

Note: Start testing after the new generator comes online and establish a baseline.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the email notification, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When receive the email notification, will check onsite, inform CUP operation and CUP engineers

2.10.2 PH (8.3 – 10.5)

Potential cause: large amount of chilled water loss, chemical pump failure, chilled water contamination, heat exchanger leaks

Action: Increase the pH booster pump feed rate.

Note: The CUP struggles to maintain the chilled water pH when the daily chilled water loss is high.

Revision 1.0 Page 26

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the email notification, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When receive the email notification, will check onsite, inform CUP operation and CUP engineers

2.10.3 TOTAL IRON (< 1.0 PPM)

Potential cause: chilled water contamination, pipe line corrosion, low chilled water pH

Action: Check for contamination issue. Check the chilled water pH. Check mild steel corrosion coupon corrosion rate.

Note: This is a target. It is currently in the 2 ppm range. Coupon corrosion rate is within range on the chilled water. Iron level in the past year is in the low or mid 2 ppm range. 2 ppm of iron in the chilled water is acceptable based on the corrosion rate at 0.2 mpy.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

NIH: When notice on the report, will inform CUP operation and CUP engineers

2.10.4 TOTAL COPPER (0.1 PPM)

Potential cause: Chilled water contamination, pipeline corrosion, low chilled water pH, low azole level

Action: Check the HST levels. Check the corrosion rate of copper coupons.

Note: This is a target. 0.2 to 0.3 ppm is currently where we stand.

Revision 1.0 Page 27

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

NIH: When notice on the report, will inform CUP operation and CUP engineers

2.10.5 HALOGEN STABILIZED AZOLE (10 PPM)

Potential cause: azole pump problem, large chilled water loss

Action: Increase the HST pump feed rate based on the chilled water loss.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.10.6 ATP (< 50 PG)

Potential cause: Low chlorine dioxide residual level, large chilled water makeup could introduce the microbiological growth

Action: Check the chlorine dioxide residual level. Increase the pump size, feed rate. Dose the system with 240 ppm of isothazolin.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.10.7 BSI (>45%)

Revision 1.0 Page 28

Potential cause: Low chlorine dioxide residual level, the analyzer may draw a bad sample, the analyzer reagent may need to be replaced.

Action: Make sure the ATP analyzer is measuring accurately. Check the chlorine dioxide residual level in the basin.

Note: Requires additional tests to establish a meaningful KPI level.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on the report, will check onsite, inform CUP operation and CUP engineers

2.10.8 HETEROTROPHIC BACTERIA COUNT (< 103 CFU/ML)

Potential cause: Low chlorine dioxide residual level, chilled water loop contamination, large amount of chilled water makeup, the existence of chemical as nutrients for bacteria to grow

Action: Check the chlorine dioxide residual. Increase the pump size, feed rate. Dose the system with 240 ppm of isothiazolin.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during

2.10.9 SESSILE BACTERIA (105 CFU/ML)

Potential cause: Low chlorine dioxide residual level, chilled water loop contamination, large amount of chilled water makeup, the existence of chemical as nutrients for bacteria to grow

Action: Check the chlorine dioxide residual. Increase the pump size, feed rate. Dose the system with 240 ppm of isothiazolin.

Note: This is a target under normal operations. Add an additional plastic mesh coupon to the current chilled water controller. Swab a plastic mesh coupon for measurement.

Revision 1.0 Page 29

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on the report, will check onsite, inform CUP operation and CUP engineers

2.10.10 IRON‐OXIDIZING BACTERIA (50 CFU/ML)

Potential cause: Low chlorine dioxide residual level, chilled water loop contamination, large amount of chilled water makeup, the existence of chemical as nutrients for bacteria to grow

Action: Check the chlorine dioxide residual. Increase the pump size, feed rate. Dose the system with 240 ppm of isothiazolin.

Note: This is a target under normal operations. Take water samples at the inlet of side stream filter for measurement. Consider non‐oxidizing biocide dosing if the KPI is exceeding the limits.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on the report, will check onsite, inform CUP operation and CUP engineers

2.10.11 SLIME‐FORMING BACTERIA (100 CFU/ML)

Potential cause: Low chlorine dioxide residual level, chilled water loop contamination, large amount of chilled water makeup, the existence of chemical as nutrients for bacteria to grow

Action: Check the chlorine dioxide residual. Increase the pump size, feed rate. Dose the system with 240 ppm of isothiazolin.

Note: This is a target under normal operations. Take water samples at the inlet of side stream filter for measurement. Consider non‐oxidizing biocide dosing if the KPI is exceeding the limits.

Alarm notification: See attachment

Reporting mechanism:

CUP operation: When receive the report, will inform supervisor.

Revision 1.0 Page 30

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on the report, will check onsite, inform CUP operation and CUP engineers

2.10.12 NITRIFYING BACTERIA (100 CFU/ML)

Potential cause: Low chlorine dioxide residual level, chilled water loop contamination, large amount of chilled water makeup, the existence of ammonium and nitriate in the chilled water loop

Action: Check the chlorine dioxide residual. Increase the pump size, feed rate. Dose the system with 240 ppm of isothiazolin.

Note: This is a target under normal operations. Take water samples at the inlet of side stream filter for measurement. Consider non‐oxidizing biocide dosing if the KPI is exceeding the limits.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on the report, will check onsite, inform CUP operation and CUP engineers

2.10.13 SULFATE REDUCING BACTERIA (10 CFU/ML)

Potential cause: Low chlorine dioxide residual level, chilled water loop contamination, large amount of chilled water makeup, the existence of sulphate in the chilled water loop

Action: Check the chlorine dioxide residual. Increase the pump size, feed rate. Dose the system with 240 ppm of isothiazolin.

Note: This is a target under normal operations. Take water samples at the inlet of side stream filter for measurement. Consider non‐oxidizing biocide dosing if the KPI is exceeding the limits.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

2.10.14 ACID PRODUCING BACTERIA (AS LOW AS REASONABLY ACHIEVE)

Revision 1.0 Page 31

Potential cause: Low chlorine dioxide residual level, chilled water loop contamination, large amount of chilled water makeup, Action: Check the chlorine dioxide residual. Increase the pump size, feed rate. Does the system with 240 ppm of isothiazolin.

Note: This is a target under normal operations. Take water samples at the inlet of side stream filter for measurement. Consider non‐oxidizing biocide dosing if the KPI is exceeding the limits.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on the report, will check onsite, inform CUP operation and CUP engineers

2.10.15 CARBON STEEL CORROSION (< 0.2 MPY)

Potential cause: high microbiological growth, low polymer level, air intrusion

Action: Check mechanical, operational, chemical and automation. Look at chemical residuals, make sure the corrosion program is operating within the specification. Check for any operation upsets. Initiate root cause analysis.

Note: Take water samples at the condenser barrel for measurement. Consider non‐oxidizing biocide dosing if the KPI is exceeding the limits.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on the report, will check onsite, inform CUP operation and CUP engineers

2.10.16 COPPER CORROSION (< 0.1 MPY)

Potential cause: high microbiological growth, low polymer level, air intrusion

Action: Check mechanical, operational, chemical and automation. Look at chemical residuals, make sure the corrosion program is operating within the specification. Check for any operation upsets. Initiate root cause analysis.

Revision 1.0 Page 32

Note: Take water samples at the condenser barrel for measurement. Consider non‐oxidizing biocide dosing if the KPI is exceeding the limits.

Alarm notification: See attachment

Reporting mechanism:

Operator: None

CUP operation: When receive the report, will inform supervisor.

CUP Engineers: When receive email notification and notice on water dashboard or PI during review. Will consult with CUP operation.

NIH: When notice on the report, will check onsite, inform CUP operation and CUP engineers

2.10.17 CARBON STEEL CORROSION (NO PITTING)

Potential cause: high…