Attachment_A_PWS__Data_Analytics.docx

Data Analytic System Performance Work Statement (PWS) 10/26/2026

1. Summary

1.1. Implement a data analytic system (software/platforms/models) that can reduce and prevent failures and guide maintenance events in. With the many data streams that the Bureau of Reclamation (Reclamation) powerplants already collect from various power generation equipment, Reclamation is seeking to identify and monitor useful relationships between multiple data streams to identify signals of deterioration, impending failures, or other operational issues, as well as to identify likely preventative actions when anomalies occur. To do this, Reclamation is seeking an Artificial Intelligence (AI)/Machine Learning (ML) data analysis software/platform to implement. The system will be evaluated as a pilot project for the first year using data from 4 units at a hydro powerplant. Upon successful conclusion, and at Reclamation discretion, the software/platform will be fully deployed within Reclamation’s on-premise virtualized infrastructure and integrated with Reclamation’s historian located within this same environment. Deployment will initially start with 4 units and then be expanded in steps in successive years to cover additional Reclamation powerplants.

2. Background & Introduction

2.1. Reclamation is a water management agency in the 17 western states. We are known best for construction of dams, powerplants, and canals, including Hoover Dam on the Colorado River, Grand Coulee Dam on the Columbia River, and Folsom Dam on the American River. Reclamation is the largest wholesaler of water in the country and second largest producer of hydroelectric power in the western United States. Reclamation is organized into five regions (Missouri Basin and Arkansas-Rio Grande-Texas Gulf [MB], Lower Colorado Basin [LCB], California-Great Basin Region [CGB], Columbia-Pacific Northwest [CPN], and Upper Colorado Basin [UCB]).

2.2. As equipment continues to age in the Reclamation infrastructure, a means to predict and prevent failure is at the forefront of operations and maintenance. One of the best economical means to accomplish this is through data anomaly detection and predictive maintenance for power generation applications. Reclamation would like to initially pilot a data analysis solution (software/platforms/models) that can predict and prevent failures and guide maintenance activities in hydroelectric power generation plants. With the many data streams already being collect from various power generation equipment and stored in a centralized historian, Reclamation is seeking to identify and monitor useful relationships between multiple data streams to identify signals of deterioration, impending failures, or other operational issues, as well as to identify likely preventative actions when anomalies occur. To do this, Reclamation intends to implement an Artificial Intelligence (AI)/Machine Learning (ML) data analysis system and platforms via a pilot project. Reclamation is seeking a proven, commercial-off-the-shelf (COTS) solution that includes software licenses, training, technical, system administration, end-users, and patch/updates support.

2.3. The desired features and outcomes of the software/platform/tools being sought include:

2.3.1. Identification of indeterminate and anomalous data patterns that signal potential deterioration, impending failures, and other operational issues.

2.3.2. Unsupervised learning of normal behavior from historical data streams in order to detect when anomalies occur.

2.3.3. Ability to learn from previous events to enable prediction, early-detection, and prevention of impending failure events and dangerous deterioration.

2.3.4. Supervised learning from anomalies to help automate identification of likely causes and potential maintenance solutions.

2.3.5. Provide automated avoidance/prevention measures based on anomalous conditions identified and detected. Provide equipment condition assessments and for anomalies provide potential causes and predict time to failure.

2.3.6. Enablement of predictive maintenance to avoid failures and permanent equipment deterioration, as well as unnecessary outages for inspections.

2.3.7. Efficient system design via consideration of the normal way the quantities react to changes in machine operation, machine operating state, and/or user changes; therefore, optimizing the number of specific data streams and data relationships to monitor, allowing the software to process large amounts of data without utilizing extensive computer resources.

2.3.8. A Pilot Project approach will be utilized the first year to gain hands-on experience with system package and evaluate the system to determine the value to Reclamation.

2.3.9. Reclamation-wide deployment software/platform/tools will be pursued during subsequent years at Reclamation’s discretion to realize the long-term benefits.

2.4. The purpose of this system is to detect anomalies in real-time through comparison of historical process conditions and real-time information; and also provide the means for users to access this data; to display high-quality graphics; plot and trend this data; and provide notification of equipment performance anomalies. The anomaly detection system must be designed to accommodate very large real-time and historical databases so that multiple anomaly detection models can be run simultaneously. The models must be scalable and efficient so that they can run at any interval up to once every 1 minute, as required by the end user. The results of all calculations must be stored on-line for years, in its original resolution preferably within Reclamation’s historian. The archive results must not in any way affect the calculated data from its original resolution and sampled state. With this high-resolution data available, users can view both the plant conditions and analytic results while providing a very clear and accurate picture of past operations. The system must also provide a method for managing and dispositioning alarms. Any dispositioned alarm should be easily accessible; users should be able to bring up a full analytic copy of all alarm information associated with a dispositioned alarm.

2.5. The system must implement cybersecurity controls in accordance with Department of Interior (DOI), Reclamation, Federal Information Security Modernization Act (FISMA) and National institute of Standards and Technology (NIST) guidance.

2.5.1. Complete all required System Security and Privacy Plan documentation.

2.5.2. Undergo an independent security assessment.

2.5.3. Obtain and maintain an Authority to Operate (ATO) prior to commencing operations.

2.6. Reclamation plans to host this system on-premise, utilizing virtualized infrastructure within a centralized Reclamation data center. The data center consists of multiple clusters that can be expanded as the system is deployed at additional powerplant sites. It is planned to implement the Reclamation’s historian on this same infrastructure.

3. System Requirements: Reclamation is seeking a proven, commercial-off-the-shelf (COTS) system and not a system custom designed just to meet the requirements in this section. The following requirement describe the data analytic system that is desired by Reclamation and will be used to evaluate proposed system. Proposed COTS data analytic system should provide a majority of the functions and meet a majority of the minimum requirements listed in this section to be favorably evaluated. (“Must” indicate a higher desired features compared to “should.”)

Please use the attached table to address quote information. Please clearly indicate any exceptions or non-conformance to these requirements. Please clearly indicate any features that go above or beyond these listed requirements.

3.1. Models

3.1.1. The system must allow users to build custom descriptions for normal operation for a variety of plant components (models) based on previous historical operation.

3.1.2. Users must be able to import data for models from AVEVA PI (formally OSISoft) or GE Proficy Historian which is Reclamation’s current historian. Additional support of Real-Time data services, text files, standard commercial data historians such as IBM DB2, Wonderware Historian, and Wonderware eDNA, or standard online data sources (Oracle, ODBC, OPC, etc.) is beneficial. Importing data should only require a click of a few buttons to configure.

3.1.3. Users must be able to export data from models to AVEVA PI or GE Proficy Historian. Additional support of Real-Time data services, text files, standard commercial data historians such as IBM DB2, Wonderware Historian, and Wonderware eDNA, or standard online data sources (Oracle, ODBC, OPC, etc.) is beneficial. Exporting data should only require a click of a few buttons to configure.

3.1.4. Users must be able to easily import, export and move data across different data sources.

3.1.5. The system must provide the ability to develop a model with up to 50 datapoints.

3.1.6. Users must be able to import multiple blocks of historical data and support multiple independent sets of training data for a single model.

3.1.7. The system should display input data statistics for correlation, maximum / minimum, range, and standard deviation. The users should have the option of comparing multiple data blocks and comparing differing status to see the effect of excluding data records from a model based on differing data cohesion.

3.1.8. The system should allow multiple models to be built from the same set of historical training data, but with different parameters.

3.1.9. The system must provide for the ability to define alarm thresholds on independent model signals or an overall model deviation.

3.1.10. The system supplier shall describe in their submission how alarms are managed and prioritized.

3.1.11. Users must be able to define multiple alarm thresholds for any model. The alarms should include high and low values for alarms, warnings, and out-of-range trips. The alarming package shall support the ability to filter alarms. Please describe in the submission how this is currently accomplished in the system.

3.1.12. The system should be able to detect multiple power cycling sequences / transient conditions (e.g., starting, shutdown, rough zone, normal) that will control how the models operate under those specific conditions. The conditions should be driven by the measurement of real-time values.

3.1.13. Users should be able to define one or more transient conditions (e.g., starting, shutdown, rough zone, trips) that will control how the model operates under specific conditions. The conditions should allow the user to temporarily turn off all or part of the model. The conditions should be driven by the measurement of real-time values.

3.1.14. The system must support the ability to analyze power sequences / transient conditions. Transient analysis should support a resolution of 1 second and support specifying the logical starting and ending conditions for the transient. Users may also specify a maximum amount of time to run the transient analysis.

3.1.15. The system must support the ability to automatically search the historian and identify periods where the power sequence / transient conditions are met. The user should be able to select one or more of these transient periods to be imported for use in model creation or testing.

3.1.16. The system must provide the ability to compare multiple instances of the transient condition by overlaying each of the transients based on logical trigger conditions.

3.1.17. The system must provide the ability to quickly and easily update models when new conditions are detected that are valid but not part of the original model description. Please describe in the submission how this is currently accomplished in the system.

3.1.18. It must be possible to easily turn off one or more signals in a model while the model continues to execute without use of these deactivated signals and without the need to rebuild or retrain the model (e.g., eliminate a sensor that has failed from the model but allow the remainder of the model to continue to execute). It must be possible to perform this function while the model continues to operate (not acceptable to disable the model).

3.1.19. The users should be able to define how often each model runs. It should be possible for each model to run at an independent rate.

3.1.20. Users should be able to define preferences for alarms, archiving options, and model frequency. If these preferences are defined, they should automatically be assigned to the model.

3.1.21. The system should provide the options to archive (write to the historian) derived values, calculations, predicted values, and deviations for each real-time value used in or produced by a model. Changing archive points should be transparent to the end-user.

3.1.22. It is necessary to be able to test and validate models prior to deployment. The system shall provide a mechanism for testing models for anomaly detection so they can be optimally configured before being made active. This mechanism should provide a visual graph of what the model results would have been under different circumstances by running the model against the live system, historic data, or flat files for one or more time periods of the users choosing. The testing system should allow the user to toggle model inputs on and off so their impact can be seen. It should also provide a visual indication of where individual alarm thresholds are located and mark alarm events as they occur. Please provide a detailed response in the submission on how this is currently accomplished in the system.

3.1.23. The system shall provide an integrated asset hierarchy for organizing models. This shall provide for the ability to create customized hierarchical organizations of the equipment and provide for alarm and status roll up within the hierarchy. Users should be able to drag and drop models for easy organization.

3.1.24. The system should be able to import and export training data and models to flat files.

3.1.25. The system should allow users to easily make copies of the dataset and the model through the use of a “Save As…” option.

3.1.26. The system must have the capability to retrain models with a minimal amount of intervention from the user. The vendor must outline their process in the submittal for retraining models.

3.1.27. The system supplier shall provide details in the submittal on how a test environment of the system can be configured separate from the production environment.

3.1.28. The system must provide the ability to create a base model and then apply that template to several different sub models at once. Users should be able to specify what information from the base model is to be supplemented or changed and pushed to other sub models. Please describe how this is currently accomplished by the system in the submittal.

3.1.29. The system must provide the ability to easily export all model information to a text file (example CSV or XML) or only portions of the data so that it may be shared with non-system users or transferred to other systems as needed.

3.1.30. The system must provide the ability to use customized model algorithms in place of the default algorithms and to be able to test those algorithms with operational data.

3.1.31. The system must provide the ability to automatically track all changes made to a model and view all those changes at a summary and detail level.

3.1.32. The system must provide the ability to create one or more “Restore points” for a model, which can serve as backups for a design that can be “rolled back to” a future date or imported as a new model. Please describe how this is currently accomplished by the system in the submittal.

3.1.33. The system must provide the ability to assign ownership and privacy settings to specific models so that users may individually choose how their models are shared through the system.

3.1.34. The system must provide the ability to mark models that have known issues, for example sensor issues that have been detected but which will not be repaired until the next maintenance window, and to have the system remove those known alarms from the user’s active alarm list. The user should be able to specify the default conditions under which these known issues are displayed. The users should have the option of specifying time-based, deviation, or signal conditions under which the known issue should be elevated to a new alarm level so that additional anomalies may be identified even if the known condition persists. Please describe how this is currently accomplished by the system in the submittal.

3.1.35. The system supplier shall describe in detail the process used to develop and verify a new model.

3.1.36. Machine learning and artificial intelligent model development shall not require the use of software coding. Software coding can be an optional feature.

3.1.37. Model templates for hydroelectric powerplants shall be provided within the system to build, maintain, and scale models. Please provide a list of model templates that pertain to hydroelectric powerplants that are currently available.

3.1.38. Developed models shall be made available as templates to help build additional models.

3.1.39. Model processing shall include real time operational data (high frequency sensors, SCADA, and condition monitoring), along with historical test result data (records and observations), records of prior maintenance activities and equipment failures and costs.

3.1.40. Multiple system algorithms shall be available including automated anomaly detection models to identify unusual behavior and machine learning risk models to predict system condition and time to failure.

3.1.41. It must be possible to train models using only historic normal operational data.

3.1.42. Model training shall include using unsupervised machine learning techniques.

3.1.43. Model testing and tuning shall utilize data playback. This includes determining and displaying predictive data values and ranges, determining the impact of sensor failure, and simulation of model deviations. Data playback shall incorporate both historical data and/or externally imported data.

3.1.44. The system supplier is encouraged to list additional model features not listed in this section that are currently supported and why they may be beneficial to Reclamation.

3.1.45. Models must be capable of incorporating physics-based machine learning (PIML). This would include: 1) Models that obey the physic based limitation of the physical system. 2) Models that incorporate physics information during their development and training. 3) Health calculations, predictive analysis and fail modeling that incorporate PIML.

3.2. System

3.2.1. The system must include a real-time monitoring component. This component must be able to simultaneously run all models at the frequency specified by the user. Users should have the option of specifying different frequencies for different models.

3.2.2. The monitoring component should have the option of sending e-mail based notification messages to users when an alarm threshold is tripped. Users should have the option of signing up for notification based on specific models or assets. Notification messages can be a custom format including variables that can replaced with specific model information for model name, alarm type, signal contributions, and a URL to the model analysis.

3.2.3. The system should provide an option for automatically integrating with active directory for security policy and access management. The system should detect the account of the current user, and use that information to enforce multifactor authentication and single sign on to manage access to the system, if the user has appropriate access rights. If not, it should present the user with the option of supplying a different set of login credentials. The system should support both individual user accounts and custom user groups, which should be defined in active directory.

3.2.4. The system shall be able to monitor a minimum of 800 deployed models with an average of at least 10 points per model and all models updating once per minute on a single server. Please give an example in the submittal and reference where this configuration is currently deployed in a production environment.

3.2.5. The system should support these common scripting languages: VB, Net, C#, Python, R, etc.

3.2.6. The system should allow for interfacing with other systems/applications including third party AI applications. This should include shareware analytical AI/ML applications along with license software. Please describe in the quote how this interface could be accomplished.

3.2.7. The system should provide a mechanism for preventing multiple users from editing a common resource (e.g., a model or calculation library) at the same time by locking access so that additional users after the first may only open the resource in “read-only” mode. System administrators should be given the option of over-riding any resource locks. Please describe how this is currently accomplished by the system in the submittal.

3.2.8. The system supplier shall provide a system diagram and define the system architecture and design of the system solutions.

3.2.9. The system must provide both a web-based and a user desktop client.

3.2.10. The system supplier must provide details on the hardware and software infrastructure required for the software solution in the submittal. It is highly desirable for the system to work with existing Reclamation IT infrastructure: 1) Solution must integrate with AVEVA PI Historian or GE Proficy Historian located on Reclamation’s on-premise virtualized infrastructure. 2) It is anticipated that the preferred solution will be hosted on the system on Reclamation’s on-premise virtualized infrastructure. 3) Describe alternative deployment hardware and software infrastructure and how the alternative would be beneficial to Reclamation.

3.2.11. The system supplier must describe the security provided by the system for user multifactor authentication login and privileged access management for different user or admin roles in the submittal.

3.2.12. The system must detect and train models based on observed power cycling sequences / transients (startup, ramping, load levels, etc.) to enhance model sensitivity and reduce false alarms.

3.2.13. System shall utilize fast model processing with minimum computer resource usage to allow system to efficiently run on a server and be highly scalable to meet future expansion.

3.2.14. System shall provide sensor fault detection. System model shall automatically adapt to loss of input signals and produce usable results.

3.2.15. System shall allow customized Python and R scripts to be integrated into the system. System shall work with Power BI to develop customized displays.

3.2.16. System shall be capable to interface with maintenance management systems including IBM Maximo as Reclamation currently uses Maximo.

3.2.17. System software updates shall be provided as needed. Please provide information related to software update development process/lifecycle. How often is software updated? Are the updates required, or will the vendor continue to support the previous version of software?

3.2.18. System will need to comply with federal cybersecurity requirements, such as Multifactor Authentication (MFA), Data at Rest (DAR), and Data in Transit (DIT) encryption requirements. Please describe how the system will meets these requirements.

3.3. Database Integration

3.3.1. The system should support a commonly available commercial database, such as Microsoft SQL Server, for the model database.

3.3.2. It is understood that fleet monitoring of assets can create large volumes of data to be stored when considering the number of points per model, sample frequency and number of points to be stored with the model. Please detail how much the connected historian will increase in size each year for deployment of 1,000 models including the requirement needed for retaining storage of the calculated model signals for further display and analysis.

3.3.3. The connected historian is anticipated to store at least 10 years of the continuous measured data and calculated data from all system models for further analysis. The system trending/analysis, model training, etc., must not degrade in performance as the connected historian continues to grow. Please describe how this is currently accomplished in the system.

3.3.4. The system shall provide the ability to define engineering units for all numerical points.

3.3.5. The system must provide the ability to attach one or more notes or annotations to a project and to associate those notes with specific time ranges or signals so that the notes may be automatically displayed on a trend of data when the associated signal and/or time range is displayed.

3.3.6. Adding new points to the historian system shall not adversely impact the system. It must be possible to add points to the system without the need to stop and restart the system.

3.3.7. The system must provide the ability to easily define one or more custom file formats that can be used to import data into the system. The system should provide a test utility for importing data. Please describe how this is currently accomplished in the system.

3.3.8. The system should provide the ability to simultaneously connect to multiple databases from different vendors. Reclamation has deployed AVEVA PI or GE Proficy as its current historian but other historians may be used in the future. List the interfaces that are currently supported.

3.4. Web component

3.4.1. The system should include a web-based component which will provide access to information from the monitoring component.

3.4.2. The web component should provide the same security options used by the client component.

3.4.3. The web-based component should allow users to view the current status of their monitored equipment and any alarm events. The web component should provide rollup information for nested assets, so that users may easily see the overall status of an asset, then drill into the asset for information about specific sub-components.

3.4.4. The web component should provide trending of both analysis results and raw data signals from the monitored equipment.

3.4.5. The system must provide for a Web Services interface for easily interfacing the system with external systems and applications.

3.4.6. The system should provide e-mail and web-based notification when there is loss of connectivity to network resources (e.g., database, plant historian, OPC server, etc.).

3.4.7. The system should provide web-based notification when there has been a configuration change in a network resource.

3.4.8. The system should provide a method by which users can easily define and publish their own custom report through the web.

3.4.9. The system should provide a method by which users may create web based customized annunciator-panel-style displays for use in presenting system-wide alarm information. Users should be allowed to define their own layout, colors, font sizes, and flashing / notification rules. Please provide an example on how this is currently accomplished in the system.

3.4.10. The system must provide a web-based client for trending and exports to both CSV and PDF.

3.5. Alarms

3.5.1. Alarms are alerts, faults, failures, etc., that provide notification to users.

3.5.2. Both the client and web component should allow users to view alarm details and disposition alarm events.

3.5.3. Users should be able to define their own custom classifications for dispositioning alarm events.

3.5.4. Alarm capability shall incorporate built in workflow so that users can classify alarms as mechanical, sensor issues, etc. The alarming shall include color coding to indicate the severity of the alarm and status of resolution should be incorporated.

3.5.5. User shall be able to configure alerts and link alerts to diagnostic and prescriptive asset information that can be used to plan maintenance activities.

3.6. HMI / Trend Displays

3.6.1. Data trends must be fully adjustable. Users must be able to rescale both axes, re-label both axes, zoom in on data preferably by selecting (drawing a box around) the data.

3.6.2. Users should be able to display data using either the defined time range, or sequentially, so that data gaps can be visually eliminated. Users should be able to plot any number of points of their choosing simultaneously. Users should be able to toggle between normalized and actual values, for easy data comparison.

3.6.3. The system shall provide the ability to plot two independent signals on same trend and x-y plot. When segments of the data are selected on the trend it shall be possible to automatically highlight the corresponding times series data segments on a standard x-y plot of the data trend.

3.6.4. The system must provide a single model output such as overall model deviation that gives an indication of the overall health of the model (not only signal by signal).

3.6.5. Users must be able to visually compare and edit historical model learning data, including having the ability to trend and identify irregular data. The users must be able to delete or exclude data from the model. Preferably this would occur by selecting the data in a trend to be excluded by drawing a box around the data. The user must be able to easily make changes in the data status, such as later including the previously excluded data. Please describe in detail how this is currently accomplished in the system.

3.6.6. The system shall provide the ability to trend overall model deviation, and trend on the same display raw signal data, signal deviation, and percentage of contribution to overall deviation for each signal in the model.

3.6.7. User shall be able to scroll across in time an overall model deviation plot and see automatic updates for the percentage of contribution of the highest contributing values to the model deviation.

3.6.8. It shall be possible to hover over any signal to see the corresponding tag name or description.

3.6.9. Trends shall allow users to zoom in on data by preferably drawing a box around the data.

3.6.10. The system shall provide the ability to overlay alarm thresholds on the trend display for any selected period in time.

3.6.11. The system shall provide for customization of trend properties such as foreground and background color, scaling, line color, font size, plot style, type of data marker, etc.

3.6.12. The system shall be able to plot the actual signal with the predicted signals.

3.6.13. Analytic trends should provide the ability to show both alarm thresholds and actual alarm occurrences.

3.6.14. For a given alarm, the user should be able to open the full set of analytic trends, including overall model deviations, actual and predicted values, and individual deviation contributions, around an alarm time frame through a single click.

3.6.15. The system should include an easy method for trending and displaying the individual contribution of each signal to the overall model deviation from expected.

3.6.16. The system must provide the ability to easily compare models and equipment of the same type by displaying an overlay of similar signals for separate systems.

3.6.17. The system must provide the ability to specify the desired order in which sensor names are displayed, either alphabetically or in a specific user defined order. Each individual user should be able to set this as a preference and to update this preference as needed.

3.6.18. Visualization tools shall allow for customizable dashboards.

3.6.19. System shall support export of data (model results) to PI historian in order to be displayed within Pi Asset Framework, PI Event Frames, and PI Vision Tools.

3.7. Tags/Data

3.7.1. The system must provide an internal option for creating composite or calculated signals.

3.7.2. User shall be able to program simple and complex calculations for use as model inputs.

3.7.3. The system must provide an option for creating reusable collections of common composite or analytic calculations.

3.7.4. The system must be able to dynamically back calculate values for calculated signals for any relevant time range. Please describe how this is currently accomplished within the system.

3.7.5. The system must provide the ability to store test cases for any calculated signals to assist users in creating and validating accurate custom calculations.

3.7.6. The system must provide an aliasing method by which point names can be altered without impacting any calculations that reference those points. Please describe how this is accomplished in the system.

3.7.7. The system should allow users to define their own return error codes for any calculation.

3.7.8. The system must provide the ability for calculated values to be derived from a group of signals (for example the average bearing temperature of a list of sensors) or an individual signal. For example, the system should support the average, maximum and minimum calculation for a group of signals.

3.7.9. The system must provide the ability to add in external calculations such as the ability to call functions from external libraries (win32 DLLs) if desired from any user defined calculation.

3.7.10. The system must support the ability to manually recreate calculations if needed, such as when an external DLL library has been modified or update and the calculated history for a model should be re-created using the new version of the external or linked library. The system should alert the user when calculation libraries have been updated and provide them with the option to automatically update the history.

3.7.11. The system must provide the ability to use the output of one calculation as the input of another calculation at the same time interval. The system should automatically prevent or resolve circular references and automatically properly sequence through the calculations based on dependencies. Please describe how this is currently accomplished in the system.

3.7.12. The system must provide the ability to create calculation libraries which can be used in multiple models.

3.7.13. Data cleansing must be provided and include the following: 1) It must be possible to easily turn off one or more data signals if data errors are encountered. 2) Users must be able to visually compare and edit historical data, including having the ability to trend then outline irregular data. The users must be able to delete or exclude data from the model. Preferably this would occur by selecting the data in a trend to be excluded by drawing a box around the data. The user must be able to easily make changes in the data status, such as later easily including the previously excluded data. 3) The system must be able to detect failed sensor data and automatically remove it. This includes data outside of anticipated limits and data that is no longer changing in value. 4) The system must be able to interpolate data points as needed for signals that are not time synchronized. 5) Please describe any advanced algorithms that are deployed to help find bad data and remove or replace it. Please describe in detail how this is currently accomplished in the system.

3.8. Cause Analysis

3.8.1. The system must provide the ability to characterize a fault when an anomaly alarm is generated. At a minimum the system should provide the ability to list data inputs that caused the alarm, percent contribution of each input data to the anomaly, probable fault matches for the observed conditions and a description of how well the fault signature matches a known fault signature. Please describe how this is currently accomplished by the system.

3.8.2. The system should provide the ability to add additional information for each fault, which can supplement signal-based information, such as what IR image or visual inspection would typically reveal. Please describe how this is currently accomplished by the system.

3.8.3. The system must provide the ability to define fault diagnostics for each asset and to easily compare current issues to the fault diagnostics for support in diagnosing issues.

3.8.4. The system must provide the ability for the fault diagnostics for each asset to be easily linked to alarm thresholds as defined in the model or model template.

3.8.5. The system shall incorporate some type of Failure Mode and Effect Analysis (FMEA) to identify potential failures and resultant effects.

3.8.6. The system must provide the ability to define prescriptive actions for each assets fault diagnostic.

3.8.7. The system must provide the ability to trend remaining useful life for each asset and the remaining useful life trend shall incorporate both an upper and lower bound.

3.8.8. System shall provide a case library that provides the ability to document early warning detection cases for comparison to future asset problems. The case library shall allow multiple people to easily collaborate on an issue. This shall include the ability to track the estimated cost savings achieved through the early warning identification.

3.8.9. The case library shall provide the users the ability to upload attachments such as reports, visual inspections, etc.

3.8.10. The case library shall allow users to close issues with a defined resolution following a fault.

3.8.11. The provided case library shall be searchable and filterable to allow users to find common causes or cases of similar type.

3.8.12. Fault diagnostics shall be provided that allow user to clearly understand asset performance and related asset fault conditions (including sensor analysis) to assist user in asset diagnostics.

3.8.13. Case management tools shall be utilized that will help user to isolate problems, track cases and prioritize maintenance activities. Results of maintenance activities shall also be used to further enhance models.

3.8.14. Output information shall include probable failure modes and forecast time to failure.

3.9. Cybersecurity Requirement/508 Accessibility Requirement

The following requirements provides a comprehensive overview of the general key requirements for operating a new IT system in a federal environment. This is a comprehensive list and not all items may apply to the system provided. Consult the latest versions of relevant standards and guidelines, as regulations may evolve over time. Reference DOI IT Baseline Compliance Contract Guidelines for detailed information concerning these requirements. For each item listed indicate if the cybersecurity requirement is applicable to the system being provided, who you propose is responsible for the requirement (Government, Vendor, Both) and if applicable describe how the requirement is implemented by the system or services to be provided.

3.9.1. Secure Configuration Management: 1) All assets (i.e., hardware, software, firmware, appliances, etc.) must be fully supported by the vendor and not near end of life. 2) Implement secure configurations based on guidelines from DOI, NIST SP 800-171, and CIS Benchmarks for all hardware, software, and virtual assets to include Operational Technology (OT) components. 3) Configuration items must be identified and managed following a standardized change management process.

3.9.2. Access Control: 1) Enforce strong, phishing resistant authentication mechanisms (e.g., multi-factor authentication) via Security Assertion Markup Language (SAML), Open Authorization (OAuth 2.0) or OpenID Connect (OIDC). 2) Implement authentication via Login.gov for any sites requiring external users (Non-DOI) to access the system as a user.

3.9.3. Encryption and Data Protection: 1) Encrypt data at rest and in transit using FIPS 140-2 (minimum) or FIPS 140-3 (preferred) validated cryptographic modules.

3.9.4. System and Network Security: 1) Implement network segmentation and firewalls. 2) Regularly update and patch systems within 30 days of patch release. 3) Implement security controls required by Trusted Internet Connection 3.0 (TIC 3.0) use cases. 4) Resolve DNS queries using encrypted DNS wherever technically supported. 5) Enforce authenticated HTTPS for all production HTTP traffic, including traffic that does not cross the public internet.

3.9.5. Physical Security: 1) Secure physical access to servers, data centers, and other critical infrastructure. 2) Maintain and review access logs. 3) Maintain inventories of physical access mechanisms (i.e., keys, cards, etc.).

3.9.6. Privacy Compliance: 1) Ensure compliance with DOI Privacy requirements.

3.9.7. Vendor Risk Management: 1) Participate in DOI C-SCRM Program and submit vendors to the DOI SCRM Risk Analysis Tool. 2) Conduct thorough security assessments of third-party vendors and service providers. 3) Request assurance letters and software bill of materials (SBOMs) from vendors. 4) Complete C-SCRM and familiarize with the Counterfeit Detection practices.

3.9.8. Audit and Compliance Reporting: 1) Systems and applications must be capable of generating and audit logs for all system activities. 2) Audit logs must be maintained and reviewed, and appropriate storage and access control must be implemented. 3) Audit logs must be forwarded to Reclamation Splunk for aggregation and storage.

3.9.9. Continuous Monitoring and Incident Response: 1) Establish continuous monitoring processes to detect and respond to security incidents in accordance with BOR Continuous Monitoring requirements. 2) Report suspicious events or confirmed incidents following the BOR Incident Response Plan. 3) Deploy Endpoint Detection and Response (EDR) via Microsoft Defender for Endpoint in alignment with DOI. 4) Consistently perform vulnerability scans, analyze the results, and remediate findings within minimum required timeframes based on categorization. 5) Consistently check assets for available security updates or patches, and test/implement within minimum require timeframe.

3.9.10. Secure Software Development Lifecycle (SDLC): 1) Integrate security into the software development process from design to deployment in alignment with the Secure Software Development Framework. 2) Ensure any commercial of the shelf (COTS) or custom applications are fully compatible with existing BOR infrastructure and security solutions. 3) Acquire and maintain well-secured software components (i.e., software libraries, modules, middleware, frameworks). 5) Perform secure code analysis and vulnerability scans, analyze results, and remediate findings.

3.9.11. Secure Mobile Computing: 1) Implement policies and controls for secure mobile device usage. 2) Mobile applications must be submitted to the DOI GFE Mobile Device App Catalog for review and approval.

3.9.12. Secure Remote Access: 1) Ensure compatibility with BOR remote access solution and enforce usage for remote employees.

3.9.13. System Development Lifecycle (SDLC): 1) Acquire, develop, and manage the system using an acquisition development life cycle that incorporates information security and privacy considerations. 2) Identify all data types and assets necessary to support system, including explicit identification of critical assets. 3) Procure and maintain current licensing and support for all assets. 4) Monitor usage of licensing. 5) Ensure adequate budgetary considerations are made for maintaining and refreshing assets for the duration of the system’s lifecycle.

3.9.14. Federal Information Security Modernization Act (FISMA) Compliance: 1) Implement the Risk Management Framework (RMF) as per NIST SP 800-37 Rev.2 in accordance with DOI Security and Privacy Security Control Standards. 2) Conduct regular security assessments and authorization processes. 3) For administrators and cybersecurity personnel, provide role-based security training. 4) Develop and test disaster recovery and business continuity plans to ensure timely recovery from disruptions. 5) Ensure adequate cybersecurity resources and personnel (e.g. ISSO) are assigned to the system. 6) Obtain FedRAMP certification prior to commencing operation of the system. 7) Obtain an Authorization to Operate prior to commencing operation of the system.

3.9.15. Section 508 Accessibility compliance. Supply voluntary product accessibility templates (VPAT) or Accessibility Conformance Report (ACR) if available. Accessibility Conformance Report (ACR) | Section508.gov

4. Scope

4.1. Pilot Project (Base Year)

4.1.1. A pilot project for the base year (12-month period) will be used to demonstrate the system.

4.1.2. Pilot will utilize data from 4 units from 1 powerplant. Each unit has roughly 110 input tags. Four models (such as Generator, Bearings, Turbine) for each unit will be built, trained, deployed, and tested. Total of 440 tags and 16 models.

4.1.3. Plant data will be hosted on AVEVA PI historian or GE Proficy historian located on Reclamation’s on-premise virtualized infrastructure. For pilot only, transfer of data between the historian and the Data Analytic System can be performed manually. Goal is to minimize the cost of the pilot and to expedite system setup time. For example, software can be implemented on: a) Reclamation owned appliance, b) Reclamation virtual environment, c) Department of Interior cloud environment, d) on a secured commercial or private cloud, or e) a vendor owned appliance loaned to Reclamation.

4.1.4. All hardware, software, and network connections will need to be vetted by Reclamation IT staff before they can be deployed for the demo. Proposed pilot architecture may need to be modified as requested by Reclamation.

4.1.5. Historical Data will be made available by Reclamation: 1) Reclamation will make available 1 plus years of historical data for the pilot. 2) Data will include Supervisory Control and Data Acquisition (SCADA) Data, Machine Condition Monitoring (MCM) data, and industrial monitoring system data. This data is collected at different intervals. SCADA data will be about 1 data point every second. MCM data is about 1 data point every 1.5 seconds. 3) Data will not be continuous (data holes will exist) and data will need to be cleansed (abnormal data from failed sensors or other issues will exist).

4.1.6. Reclamation data responsibilities (anticipated): 1) Provide proper permission to data source (PI historian). 2) Provide PI historian data streams of historic and live data (if available). 3) Make data ready for manual transfer, if needed. 4) Receive and store results in PI historian.

4.1.7. Vendor data responsibility (anticipated) includes but not limited to: 1) Configure system for connection to data source. 2) Import data implementation. 3) Export data implementation. 4) Manual import and export of data, if needed. 5) Please provide comprehensive list of additional data that is needed for the pilot. Include items such as: maintenance history, event lists, etc. Note: Remote access to Reclamation applications is preferred but may not be available due to IT security concerns.

4.1.8. Data Analytics System Installation and Configuration

4.1.8.1. Please provide a detailed description of your proposed equipment and its configuration to implement the hardware and software required for the pilot system. Include what software platforms will be utilized, details on how data will be kept secured, how the software will be installed and maintained, location of software and hardware, list any hardware/software to be supplied by Reclamation, list any hardware/software that will be supplied by the vendor.

4.1.8.2. Please provide a detailed description of the services that are required to implement the pilot system. Include what services are expected to be performed by Reclamation, services that will be provided by the vendor, if service can be performed remotely or on-site, services required to install and maintain system.

4.1.9. Reclamation IT.

4.1.9.1. Reclamation has very limited IT resources and time to dedicate to this project beyond decision making, review of proposed equipment and software, and approval. Reclamation is depending upon the selected vendor to perform as much of the work as possible including but not limited to: proposing pilot data analytic system hardware components, software installation and configuration, installation and maintenance of the system, interfacing system to historian, addressing system execution errors, and system testing.

4.1.9.2. All IT work involving interfacing with Reclamation historian or with Reclamation owned equipment to be conducted on this project is at advisement and recommendation of the vendor and to be implemented and directed by Reclamation.

4.1.10. Configure, train, and deploy up to 16 predictive analytics models for 4 hydrogenerators.

4.1.10.1. Reclamation responsibility (anticipated): 1) Compile and organize list of historian tags for the target assets. 2) Compile data description and location of sensors as needed. 3) Compile control system alarm and alert thresholds. 4) Provide the top fault incidents for the asset(s) (e.g., symptoms of tags involved in the fault incidents) for the assets in scope. 5) Designate knowledgeable personnel resources to assist with model design, build, testing, review, and approval.

4.1.10.2. Vendor responsibility (anticipated) includes but not limited to: 1) Provide workshop on overview of Data Analytic System (virtual or on-site). 2) Provide workshop on Data System Model Building (virtual or on-site). 3) Lead model design, build, testing and implementation effort.

4.1.10.3. Joint responsibility - Model designs and build using Data Analytics System Tools.

4.1.11. Model Deployment, Co-monitoring, and tuning.

4.1.11.1. Reclamation responsibility (anticipated): 1) Provide in person and/or remote access to relevant systems. 2) Designate knowledgeable personnel resources to support deployment, monitoring and tuning. 3) Provide additional data to model as new data becomes available. 4)Perform monitoring of deployed models and provide feedback to vendor as necessary. 5)Take prompt action on cases and recommendations issued by the vendor analyst. 6) Provide feedback for created cases in a timely manner.

4.1.11.2. Vendor responsibility (anticipated) includes but not…