W9127N23R0024 JA BN Digital Governors_Redacted.pdf

NWP-23-024

CUI

availability of spare parts and technical expertise. In 2008 a contract was awarded to retrofit all the plants in the Portland, Walla Walla, and Seattle Districts with new AGC Digital PLC Governors. In 2015, a similar contract was awarded to retrofit all of Omaha District. Although the contract retrofitted all Bonneville Powerhouse 2 unit governors, Bonneville Powerhouse 1 unit governors were not included in the regional supply contract as they were not nearing end of life at that time. This contract is for the programming and training services for the new equipment to upgrade powerhouse 1 governors by Emerson Process Management Power & Water Solutions, Inc. In April 2020, the American Governor Company was purchased by Emerson Process Management Power & Water Solutions, Inc.

(2) Site Background: Bonneville Hydroelectric Dam consists of two powerhouses (PH), with a shared pool of dam operators, electricians, and technicians. PH 1 and PH 2 are both controlled from PH 1 Control Room, with PH 2 governors additionally controlled from PH 2 Control Room. At the moment, the powerhouses do not share controls due to the differences in technology, but integration is planned in the future.

(3) System Integrator Background: System integration is a field which focuses on the creation of compatibility between electro-mechanical equipment and digital control equipment. Electro-mechanical equipment has historically been controlled through fully physical mechanisms. A prime example of electro-mechanical equipment is electricity creation at hydroelectric dams using physical governors and exciters. Digital systems are digital software-based controllers, such as a PLC, which use software to replace much of the physical control elements of the electro-mechanical equipment. When a digital controller is put into an electro-mechanical system, such as that used for electricity creation at a hydroelectric dam, compatibility between the remaining physical equipment, the new or replacement digital equipment, and software is created. After creation, the software is customized, tested in a controlled environment, and then tested on-site with final commissioning occurring directly on live equipment. All elements of the system must be fully understood and compatible with one another in order to reduce inadvertent operations, reduce excessive safety risks, and reduce the risk to life and property.

(4) Project Background: A Phase 1a was conducted on this project. 4 alternatives were reviewed. -

a. Do nothing

b. Buy spare parts

c. Retrofit all units and replace critical components with an alternate governor supplier and programmer

d. Replace existing governors with similar units to powerhouse 2 and rest of the

NPR.

The Government and project cannot pursue Alternative A as the age and condition of the equipment present an unacceptable risk of failure. If an outage occurs, there would no longer be automated control of the turbines in the powerhouse, resulting in loss of electricity generation and negative effects on the electrical grid. Alternative B was also eliminated due to lack of spare parts availability.

(5) Alternative C was eliminated as the cost of the system itself would be similar to sole source alternative but the cost of training would be doubled for plant operations since plant staff would have to go to 2 trainings, one for each system instead of 1 training for both systems. There would also be an increased maintenance cost since plant personal would be less familiar with either system and trouble shootong and maintemce would take longer adding to cost both in added man hours and lost generation. Additionally, powerhouse 1 would not be compatible with PH 2 or the rest of the region making future interoperability within Bonneville Dam or Northwest Division (NWD) extremely difficult.

Technology Background: Although the mechanical / physical portions of the governor equipment are not proprietary to Emerson, the governor software is proprietary.

Emerson will only supply their proprietary software, customization services, commissioning services, and training services if the mechanical / physical portion of the governors are supplied by their company. This is typical within the industry due to the integrated nature of electro-mechanical equipment that is controlled digitally by Programmable Logic Controller (PLC).

(6) Justification: The purchase of powerhouse 1 governors with the same software customization, and training from the same integrator as the existing powerhouse 2 governors is required for a number of reasons:

a. There are long-term plans to integration operations between a number of NWD dams for greater power generation reliability, and consistent interoperability between dams is essentially for this success.

b. Consistency of governor software and integration between powerhouses at Bonneville Dam reduces mis-operations and permits greater reliability of operations within Bonneville Dam, and of the electrical generation reliant upon the dam.

c. Consistency of software and integration between powerhouses at Bonneville dam allows interoperability of software and mechanical support, as well as troubleshooting of issues, which again facilitates greater security and reliability.

d. Consistency of software and integration between powerhouses at Bonneville dam allows risk assessment and cybersecurity to include both powerhouses at one time and allows for greater security through more extensive training and assessment.

e. Consistency of software and integration between powerhouses at Bonneville dam allows training to be utilized at both powerhouses for system maintenance and troubleshooting, which increases reliability and security by allowing more focus on one type of governor.

Pursuing different governor integration and training is not practical given the historical and on-going investment both by the regional stakeholders and by the Corps throughout the Northwestern Division to ensure consistency and interoperability. Purchasing the equipment and proprietary programming from Emerson is necessary to ensure integration and compatibility with the existing governors at the Project, to capitalize on the existing investment made by the Corps and our regional partners, to take advantage of existing Government owned AGC equipment, to reduce the risk of mis operations, and to ensure sensible and efficient maintenance of all the governors at the Project.

If a different integrator were used, the operability of the governors could have differences that would require operator retraining and would increase government risk of mis-operation during routine and emergency control of the generators. Both powerhouses are operated from the same control room. A different governor system would require the operators to interact with the equipment in a manner not consistent with PH 2 units and would require additional equipment in the control room to operate PH 1 units.

If a different type of governor were installed, additional cyber security reviews and modifications would be required, it would require further training, a separate set of spare parts, and have different maintenance requirements than the other units at the Project.

Additionally, given that hydroelectric dams are part of the nation’s critical infrastructure sector with heightened awareness and responsibility for national security, it is critical to obtain supplies that are known to be tested and compatible with the existing systems. Purchase of the same parts, software, and software customization helps maintain security and resilience of the system that is already in place.

Since the Project is already outfitted with AGC governors and AGC governor software programming, Project personnel and staff have received training in their operations and maintenance and have experience installing them. Therefore, it is logical and economically efficient for the Government to continue purchase of replacement or upgraded parts with this company for purposes of upgrading the station service governors. The best value to the Government is to purchase the additional governors from AGC to allow for interoperability with the existing Government owned AGC equipment (i.e., powerhouse 2 governors), to ensure efficiency/compatibility with the existing network infrastructure at the Project, and to ensure efficiencies with respect to equipment servicing, repairs, maintenance, and operation already established through the use existing AGC equipment.

The mechanical / physical portions of the governor equipment are not proprietary to Emerson. However, the controlling software of the governor equipment is proprietary, and Emerson will not supply commissioning and training services on their proprietary programming in the mechanical / physical portion of the governor equipment if that portion of the system were to be supplied by a different firm. This is typical of the industry due to the integrated nature of electro-mechanical equipment controlled by Programmable Logic Controller (PLC).

(7) Impact:

a. The replacement of existing governors must occur. The existing governors are nearing the end of their expected service life of 20-25 years. Without replacement, parts will fail, and units will have to be taken out of service for repairs. As the governors age, spare parts have become hard to find resulting in longer outages. In a blackout situation, the failure of one or both station service governors could result in an inability to restart the powerhouse.

b. The replacement system must match with the Bonneville Powerhouse 2 Digital Governor system.

Therefore, it is very important that Emerson Process Management Power & Water Solutions, Inc. provide the services for this acquisition. It is industry norm to provide a finish product that includes services from the same firm that supplies the hardware.

6. EFFORTS TO OBTAIN COMPETITION: A Notice of Intent (NOI) to Sole Source was posted on SAM.gov by the District on 27 April 2023 which stated the intent to award as a sole source for the services. To date, no responses were received. There are currently no subcontracting opportunities available.

7. ACTIONS TO INCREASE COMPETITION: Currently Emerson is the only company capable of providing the required governor software customization, that meet the requirement and existing equipment. Redesigning the digital governor product to compete it in the future increases plant infrastructure operating risks associated with system security and resiliency, and interoperability around the division. The programming required is proprietary to Emerson.

This J&A supports a discrete requirement that will end when the services are complete;

therefore, there are no planned actions to increase competition for the requirement covered by this J&A.

The Government will consider alternative sources for future procurements, however the intent of NWD is to ensure compatibility and interoperability between powerhouses so no future competition is anticipated.

8. MARKET RESEARCH: Emerson Process Management Power & Water Solutions, Inc is the only manufacturer and only supplier of the compatible parts, equipment, software, software customization, commissioning, and training, and equipment to integrate with the existing system. Market research was conducted to determine if there are other manufacturers available to meet the requirement with a Notice of Intent. Industry norm for control retrofits is for the control applications not to be separated from hardware.

9. INTERESTED SOURCES: A Notice of Intent to Sole Source and request for information was posted online at SAM.gov on 27 April 2023 for 15 days. No responses were received. As discussed in paragraph 5, it is critical to have the same governors at the two Bonneville powerhouses and the rest of the Pacific Northwest region. The government is not only requesting the same physical parts, but also the same software and programming code to control the digital governors which neither firm demonstrated they can supply. To date, no other sources have expressed an interest in writing and all offers received shall be considered. The J&A will be posted within 14 days after contract award in accordance with FAR 6.305(a).

10. Other Facts: There are no other facts supporting the use of other than full and open competition. The Seattle District (NWS) procured identical supplies/services as a sole source contract (W912DW21P0041) to the same contractor for the Chief Joseph Dam Station to provide constancy within all the powerhouses in the Pacific Northwest.