B08.03 Answers to Questions - Solicitation W912HQ25R0007.docx

W912HQ25R0007 CWMS Database Authorization

1. Factor 3 - Past Performance states: “A minimum of one and a maximum of three examples of past performance may be submitted.” And Basis for Evaluation states: “Offerors shall submit a minimum of two (2) and a maximum of four (4) past performance examples to demonstrate their performance history on contracts similar to the effort required in the solicitation.” Please clarify this discrepancy.

The Combined Synopsis-Solicitation is amended. It is a minimum of one (1) and a maximum of three (3).

1. Is there an incumbent? If yes, what is the incumbent's name, contract number, award amount and duration of the last contract/project?

This is a new requirement.

1. Are incumbents eligible to re – bid?

See answer to number 2 above.

1. Please provide information on the previous contract number for this effort, incumbent, and a scope change from the previous contract if any.

See the answer to number 2 above.

1. What is the ceiling Value?

The Government cannot provide the ceiling valuet due to the need to maintain price competition.

1. The NTE for Task 3 is listed as $184,775, but no budget is given for Tasks 1 & 2. Can USACE confirm an overall budget ceiling or expected range for the entire contract?

See the answer to number 5 above.

1. The solicitation provides a budget for task 3, is there one for task 1 and 2?

See the answer to number 5 above.

1. Can the Govt. share the details of current spending?

See the answer to number 5 above.

1. Can you please confirm type of contract?

Firm fixed price commercial services.

1. What issues, if any, were encountered with the prior contract?

See the answer to number 2 above.

1. Can the Govt. Please confirm level of effort (No. of key personnel’s & FTEs) Also, can the Govt. Please share the Qualifications of the key Personnel including experience?

The level of effort is to be proposed by the offeror. This will be evaluated with the technical evaluation to determine the offeror’s understanding of the requirement. Level of effort per each labor category per each task in the PWS is evaluated within Factor 1.

The key personnel are also to be proposed by the offeror. Typically, this would be a Project Manager and any significant technical personnel. The key personnel proposed will be evaluated within Factor 2.

1. Do we need to submit Live resumes of all key personnel’s?

Yes. Resumes do not count against the page limits of the technical proposal, however resumes should be no more than 2 pages.

1. Can we replace the candidates after the award?

Awardees may replace personnel with similarly qualified personnel. The Government reserves the right to approve these changes.

1. Will the contractor be required to work on-site full-time or will remote work be the primary mode of performance?

Contractor personnel are not required to be present at HEC full-time or other Gov’t/DoD facility.

1. Will the Govt. accept state and commercial Past Performances?

Yes. These past performance examples should be submitted on the PPQs.

1. Do we need to provide responses of all the Factors in single file?

No, it is preferred to split the files into one volume for technical, one volume for past performance, one volume for price and one volume for administrative.

1. Can the Govt. Please Confirm the Annual Hours? 1920 or 2080?

Annual hours are not applicable. This is a deliverable based Performance Work Statement. See the period of performance for each task.

1. Can the Govt. Please Confirm estimated date of the Project?

Estimated start date is approximately 10 – 15 days after expiration of the solicitation.

1. Can the Govt. Please Confirm award date?

See the answer to number 18 above.

1. Amendment 001 states, “The correct date proposals are due is 20 March 2024” Please clarify if the proposal due date 20 March 2024 or 20 March 2025?

This is clearly a typo. Proposals are due 20 March 2025.

1. Solicitation states: “While CWMS DB is currently using Oracle, it is likely the CWMS DB will migrate the data to various forms of storage” Are there active plans to migrate from Oracle during the 270-day POP inclusive of the storage changes pertaining to the authorization system?

There are no such plans at this time, nor do we foresee any developing before the POP is finished. If something does come up, HEC will take responsibility for or may allow the awardee to develop against a specific “system versions” to meet deadlines and goals.

1. Task 3 has a “Period of Performance - 180 Days from completion of Task 2” but the overall POP is 270 days, however, if Task 2 takes 90 days, Task 3 would exceed the total 270-day POP. Is the 270-day POP inclusive of all tasks, or is Task 3 exempt from this constraint?

The overall POP is 450 days including task 3. The Combined Synopsis Solicitation has been amended accordingly.

1. What is the expected volume of concurrent users and data transactions the authorization system must support?

Read-Only (e.g. general public) we’ve seen 100,000+/hour during various flood events Regularly Usage varies and we have no good estimates at this time. Between each of the districts there are about an average of 20,000 timeseries each. As data goes through the system and various computations are done this data gets aggregated into smaller amounts, generally per “Project” (usually Dam or Lock) where 5-10 timeseries are regularly accessed to render on various pages with a similar amount of supporting data around the project used to support communications of what’s happening.

1. What are the acceptable latency thresholds for authorization checks during peak operational periods?

Unfortunately, HEC does not have a good understanding of this. The faster the better, of course. Anything more than 200ms will likely cause issues.

1. Does the existing Oracle VPD system impose limitations on schema modifications or API interactions?

No schema modifications are allowed except by authorized accounts. The API currently does make use of this system to affect the authorization it performs now. However, the API will be allowed to bypass the VPD system if it is deemed to be an impediment to performance or sensible operation. It can also be used. We do not have a preference at this time.

1. How are user roles (e.g., Data Manager vs. Water Manager) currently defined, and are hierarchical permissions required?

They are not currently defined with such separation. We would not say hierarchical permissions are required; however, HEC and all of USACE are organized by Division -> District -> Staff, and each division and district should retrain control of their data.

1. What are the RTO/RPO metrics for the CWMS authorization system during outages?

This is still being determined. For guidance our current goal is a multi-region system (duplicate, at least one other region) doing a round-robin. So active-active.

1. Are there existing design frameworks (e.g., React components) that the web-based admin UI must align with?

https://github.com/USACE/groundwork/ https://github.com/USACE-WaterManagement/groundwork-water/

1. Will the government provide test instances of the CWMS database?

HEC can arrange to provide test instances. HEC can also provide data extracts or sample data to load locally. The current docker-compose.yaml file in the data-api project can be run on any system that can run docker. No data is present though.

1. Is there an expectation for the contractor to provide training or documentation beyond the POP?

No. All work is expected within the PoP.

1. How many environments are required for the initial phase?

There will be “one” system. Expectation for the contract is to work in local development environments to prove the concepts. We are still in the process of getting our centralized infrastructure setup and manageable, so we think requiring this effort to work in-situ is unreasonable. That said, we will likely try it and provide results.

1. Are there any high-availability or disaster recovery requirements?

Not specifically, but designs should assume HA will be used.

1. What is the O/S preference? We recommend Linux 64-bit for performance reasons.

The data-api is run using AWS ECS. Anything that cannot run within the data-api should be at least containerizable. Targeting Linux 64-bit is acceptable. Operating sans OS (e.g. FROM SCRATCH) is preferred when practical. If an OS is required, Alpine is our most common target.

1. Will we be able to develop against "production-like" data in the DEV environment?

Copies of real data or access to systems with live data, can be provided. HEC may be able to provide access to a test environment with live data; however, that would require a user with CAC access which we are not planning to acquire during this effort due to the short timeline.

1. Does this project require a US-based team?

Yes.

1. What are the expected performance benchmarks for authorization checks in terms of latency and throughput?

See the answer to 24 above.

1. What are the expected performance benchmarks and data volume expectations for the new centralized system? Are there service level requirements for response time, concurrent users, or peak load scenarios?

Hard numbers are unknown at this time. For reference there are 30 districts with an average of 6 users that would consistently interact, a few hundred automated processes, and some percentage of the general population of a given region whenever a weather event turns into a flood control event. It is currently assumed that we will need to horizontally scale the data-api and have planned most designs around that assumed need.

1. What are the specific limitations of the current Role-Based Access Control that the new authorization model needs to address?

Our current roles are “You can modify data for a given office”, and “You can administer users in a given office” (office = district). Additionally, to prevent certain information, like required time delays from third party data, from being an issue, such data is not published to the public national systems. As we are moving towards a single shared database, we need to be able to store that data while only sharing it within the agreements, and to avoid otherwise trusted and trustworthy users from accidentally, or intentionally, altering data for which they have no reasonability.

1. If external partners will require authentication and authorization in the future, what are the plans and timelines for implementing this?

Unknown, and currently this is not a critical piece. However, if done it would utilize login.gov which is another OpenID Connect provider. External users were mentioned in case it would help inform the initial design.

1. Should the authorization model account for future external users, or will that be handled in a separate contract?

This is not a requirement. This would be nice to have.

1. Are there any SSO or IAM systems in place that we will be expected to utilize, or is 100% of the current authentication done directly at the database user and role level?

Authentication is going to be handled through an OpenID Connect provider hosted within our providers environment. A rough mimic of this is setup in the https://github.com/usace/cwms-data-api docker-compose.yaml file.

1. Will there be a transition period where direct database access is phased out, and what is the timeline for this transition?

Not specifically, but as we move data operations to the central system, we are going to be drastically reducing such usage. It is reasonable for this effort to ignore the direct database access and focus on access through the API.

1. What is the target hosting environment for CWMS? Will it be in AWS GovCloud or another platform? Are there specific FedRAMP or IL4/IL5 compliance requirements?

AWS GovCloud. IL2. No specific requirements: however, the way we are allotted services is not through root account or delegated access. We are allowed to write our own infrastructure as code to be used. One of our primary design considerations is to only use those AWS services for which there is a direct functional equivalent on other cloud providers.

1. Are there any requirements to the future-state time zone for the "32 geographically independent" source databases, given AWS's GMT time zone standard?

Where a time zone matters, users should be able to specify the time-zone a provided date is in. For the actual data, *ALL* dates stored as dates, timestamps, or other time type within the database are UTC.

1. When planning database migrations into the cloud, are there defined outage windows/acceptable downtime requirements for source databases, or are they expected to be online 24x7?

Not determined at this time. HEC will attempt to avoid outages but that will be determined as migrations are executed. Currently all districts are streaming minimum required data to the central system, and it is thus already “up-to-date” for certain situations.

1. The PWS is vague when stating that the customer will eventually have to "migrate the data to various forms of storage appropriate to the type of data and its usage." Can we get more details on the "type of data and its usage"?

Two examples are provided below:

45. The data-api supports the storage of large text and binary data. This currently uses CLOB and BLOB. We will likely change this to S3 in the future; however, actual access to the data would still be through the same endpoint of the data-api

45. Rating Tables. These can be large sets of paired data (lookup tables elevation to storage, stage to flow, gate setting to flow) which are currently taken from XML and normalized to relation tables. We are considering, after our full migration, moving this data to a separate microservice and not using an RDBMS (or just using an XML column) to store the data. This may lead to an additional API, or we may have the current data-api handle things.

Beyond that it is simply a statement that we wish to avoid any unnecessary lock-in to our current design for future flexibility.

1. Since we will be consolidating all 32 regional DBs into one central spot, is the assumption that there will no longer be a need for GoldenGate replication?

Correct, or at least likely correct. HEC is still working out requirements but one intention it to run an active-active system in two regions. That said, given the purpose it’s arguable that Goldengate itself should be able to bypass any if not all authorization constraints as they would’ve been applied by the time it’s doing anything with data. However, it is reasonable to assume some form of streaming or data replication is going to happen.

1. How will the new authorization model affect or require changes in other CWMS modules (e.g., data ingestion tools, modeling software, or the public data portal)?

Yes, however, our expectation at that time is that all authorization operations will happen at point of operation, or at least after client has made request, so any given user or machine process would still just be responsible for providing appropriate credential for authentication.

1. Are there external interfaces or partner agencies (e.g., NOAA, USGS) that consume CWMS data and will need to support the new access control model? Will updates be required for public-facing applications like RiverGages and CWMS Data Dissemination?

The current design establishes a “guest” principal internally if no authentication information is provided. Access by external agencies is through the API. RiverGages is an independent system that may eventually use the data-api.

1. Does USACE require a CMMC certification level for this contract, or are NIST SP 800-171 controls sufficient?

NIST SP 800-171 is sufficient.

1. Will the contractor be responsible for updating CWMS Authority to Operate (ATO) or supporting a Security Control Assessment (SCA) as part of this contract?

HEC will have responsibility for completing updates. The Contractor only needs to provide information related to the design so HEC can make appropriate documentation updates. HEC does not expect an SCA during this effort.

1. Will the contractor be responsible for live training sessions, or will documentation and recorded materials suffice?

HEC has no preference; however, at least one short live session would likely be helpful.

1. Will the contractor have direct access to USACE database administrators and developers, or will all interactions be through formal interviews?

The USACE water management staff to be ready to answer questions related to this contract. HEC is accustomed to responding directly to contractors for feedback. Contractors should feel free to use whatever combination of formal interviews or direct email is necessary to complete the tasks in the PWS.

1. Are there any restrictions on personnel eligibility for this contract, such as U.S. citizenship requirements, export-controlled information handling, or security clearance needs?

HEC needs to answer this.

1. Will any in-person meetings, site visits, or on-premises testing be required, or is this expected to be fully remote?

In-person is not required.

1. The solicitation references a Past Performance Questionnaire (PPQ), but we do not see an attachment. Can USACE provide the PPQ form required for submission?

The attachment is uploaded concurrent with the amendment to solicitation. Further, the due date for proposals is extended to 28 March 2025 to account for time needed to receive completed PPQs from evaluators.

1. Can subcontractor experience be fully considered for past performance evaluation, or does the prime contractor need to demonstrate direct experience in similar projects?

The prime contractor needs to demonstrate their experience. Note: The prime contractor can use their experience performing as a subcontractor to demonstrate their experience.

1. Are there any formatting requirements for submission (e.g., page limits, font size, margins) that we need to adhere to?

Please limit proposals to no more than 25 pages for Factors 1 and 2. Font size should be 12 for body text, no smaller than 10 for charts and graphs. Times New Roman is preferred for body text. Arial is preferred for Headers and Subheaders. There is no page limit for Factors 3 and 4.

1. Will past performance for a prime/subcontractor team be evaluated the same as that of a prime contractor performing independently?

Yes, assuming the subcontractor is a major subcontractor performing more than 15 percent of the requirement.

1. As a small business set aside, what is the limit on the amount or percentage of work under this contract that can be subcontracted to one or more organizations that are not small businesses?

FAR 52.219-14 “Limitation on Subcontracting” will be included in the contract. See FAR 52.219-14 for additional information.

1. The PWS for this project states that "We plan to move the database source to a public repository at github.com/HydrologicEngineering Center". Since CUI is not permitted on publicly available sites like github, does this mean there will not be CUI involved with this contract and, if so, is it still required to have an SPRS score? If so, can that be completed after submitting our RFP proposal response?

The github project does not contain any operational data. And the “bootstrap” data there in is public. Additionally, MOST water data is public by law (though various Water Resource Development Acts). There may be some data that is restricted by time, as called out in the PWS, or through other agreement (paid data sets).

1. The PWS includes under Task 3 - "NTE - $184,775". Is this meant as an upper limit for this task and how does it bear on the total proposed pricing?

The NTE amount is an upper limit as to Task 3. See also the answers to 5 – 8 above.

1. “Previous to this decision, much of the data authorization was done within each individual database so that appropriate data constraints could be implemented while allowing District applications to access Oracle directly.” Need to know if those rules in 32 databases are documents or you expect consultant to review and document? Please provide those, if documented.

This is should not be required for this effort. However, this package https://github.com/HydrologicEngineeringCenter/cwms-database/blob/main/schema/src/cwms/cwms_env_pkg.sql is used by users or applications on behalf of users with a database account to set context variables such as the “USER” and a permissions such as “WRITE”. Which some procedures will use to determine if an operation can be done.

1. What is the difference expected between write and update permissions? Is write permission the same as extract/report data?

Some data should be write-once except under exceptional circumstance. Example: Raw data from a data logger should be stored by automated systems copied to a new timeseries for any editing requirements. This allows a record of what was received while the 2nd timeseries can be updated based on new information. For example, if validation has flagged the data as questionable and a human reviewing it agrees and adjusts the data based on information – such as someone on site informing them of a tree in the river altering the stage and flow - that human can update the 2nd timeseries, but they would not be allowed to alter the first time series.

The second question is not clear to us. Though one would expect users with permission to write a given set of data to also read it, at least by default. Reports are generated from the data by both automated and manual processes. If someone, or something, has write permissions, they should be able to then read that data back; at least for verification.

1. In reference to “restrict/grant ability to create a new data object”, by default it will have restrictions unless granted. Does that suffice the requirement? Also create new object means new record and it does not mean create new table/view, correct?

That interpretation is correct.

1. In reference to “Code changes to CDA to perform the appropriate authorization checks as defined by the users.”, if we provide API to check for authorization, can existing team take care of making changes to CDA?

Yes, however, HEC’s intent is for the Contractor to completed as many such changes as practical and for the existing team to take over.

1. There is nothing mentioned about the number of APIs and what exactly the functionalities of each API, so it would be hard to estimate the effort. Can we get more information?

The number of “end-points” listed at https://cwms.sec.usace.army.mil/cwms-data/swagger-ui.html are what is active and what operations may be performed against each.

The API of primary concern is:

· Timeseries

The APIs of secondary concern, in order of concern:

· Properties

· Location Levels

· Ratings

· All remaining end points

1. The data model provided in the document is not clear. Can you please send us a legible copy of the data models?

Better pictures of the data models are attached to this amendment to the solicitation.

1. Is the project manager going to be assigned to this project?

HEC will monitor progress but will not otherwise manage or direct the Contractor beyond making sure the contract performance requirements are met.

1. Our solution is completely Oracle Database based, with Front-end web interface built using Oracle APEX. Is this, Okay or we need to use React Framework?

HEC prefers to avoid the use of APEX apps.

1. Do we need to have any security clearance for this project? If yes, what is the process?

No security clearances are required. Source code is already publicly published, and any data provided would be suitable for public release or approved by the owner for release to the Contractor.

1. Do we need to travel to any site for this work or this can be completed remotely?

No travel to our sites is required or expected for this effort.

1. What if the scope changes for any reason during the project?

If the Contractor believes the scope changed, they shall inform the Government Contracting Officer. The Contracting Officer will then determine if the changes are within the general scope of the contract and may seek a modification to the contract if necessary. If the Government needs to make changes that are within the general scope of the contract, the Contracting Officer may execute a modification to the contract if necessary.

1. What if the Tasks are completed before or after given estimated days?

For deliverables-based performance requirements (Tasks 1 and 2), if all work is completed, delivered and accepted prior to the end of the period of performance, the Contractor may invoice in full for that task. If there is a delay in performance completion, the Contracting Officer will determine if the delay is excusable in accordance with FAR 52.212-4(f). If the delay is excusable, the Contracting Officer may execute a modification to extend the period of performance.

For Task 3, the CLIN type will be FFP-Level of Effort. The price for that CLIN will be set at $184,775.00. Offerors shall propose the mix and level of effort to complete the work described in the PWS Task 3 and shall not exceed $184,775.00. If the proposed price has not been completed exhausted upon completion of the minimum requirement specified in the PWS, additional refinement may be required, i.e. improved GUIs. The price for this CLIN shall not exceed $184,775.00.

1. Is it acceptable to use existing opensource software for the access management web application, or is this expected to be fully custom?

Contractor shall avoid unnecessary work. Use of existing open-source software is acceptable in general; however, HEC reserves the right to approve the particular software prior to use.