Weather_hydro_forecasting_SOW_QA.pdf

ITEM 0001: Calibration of the Hydrological Model used for Streamflow Forecasts

1) The solicitation implies that the calibration should be performed using hourly model-generated stream-flow compared against hourly observed data at two locations: (1) inflow to Howard Hanson reservoir; and (2) the USGS gauging station at Soos Creek. Should the model calibration be performed at hourly time-increments?

While an hourly time-step is not explicitly stated, calibration should be at an hourly time interval for streamflow, as flood peaks occur at these intervals. This is standard practice for model calibration used for flood forecasting in small basins.

3) The solicitation also implies that model snow-water-equivalent should be calibrated against hourly snow-water equivalent data at three SNOTEL locations in the basin. However, it appears that only daily-average SWE data is available (see attached example download from Cougar Mountain for beginning of water year 2013). How can an SWE calibration be performed at hourly time-scales if the SWE data is at best daily averages?

The solicitation does not imply calibration to hourly SNOTEL data. Calibration would be based on the interval of observed, available data.

4) No list of hourly station-based temperature and precipitation is given. It is well known that having just a few meteorological stations in a given basin under-represents the variability in that basin. Please provide a list of stations that are both publically available and will be standardized as required inputs for the model calibration. See also the companion question under ITEM-003 below.

The scope of work states that the model “will be calibrated using historical hydrological and meteorological observations publicly available and provided by NWS”. It is the responsibility of the contractor to determine what stations they can obtain publicly. The list of stations with data to be provided by the Corps is provided in the scope of work.

5) The solicitation states that the calibration should be accurate, yet no objective metric for accuracy is provided. What objective statistical measures will be used in order to "accept" the calibration as "accurate"? (It is well known, for example, that it is far easier to obtain relatively good statistical calibration metrics for 24-hour average flows, but much more difficult for hourly flows). Given that this is a firm-fixed price contract, an acceptable objective metric or metrics that can be generally agreed upon as "reasonable as reported in the standard literature" needs to be clearly articulated. A subjective assessment of "accurate" is not scientifically defensible for model calibration. What values of those chosen statistical measures should be met or exceeded in order to accept the calibration as "accurate"?

Calibration should be performed using an hourly time interval to be able to capture hydrologic peaks as a result of rainfall and snowmelt. The calibrated model should provide an accurate depiction of basin hydrology, including accurate depiction of the magnitude of hydrologic peaks, an accurate depiction of the rate of rise and fall of the hydrograph, and an accurate depiction of baseflow conditions.

6a) How long a historical record should be chosen in order to calculate the objective metrics that you will provide? For example, one could meet or exceed a given metric for a 5-year calibration period, but attempting the same using 10-years of historical data could reveal an undiscovered model weakness (or data input weakness) that would lower the said metric to below "acceptable" levels. We request clarification and a specific date-range be provided for the calibration so as to be as objective as possible about how the metrics are to be calculated.

There is not a defined duration for this, the vendor should use professional judgment to determine a reasonable duration.

6b) However, how can similar objective metrics be calculated against SNOTEL data if that data is unavailable at hourly time-scales?

The solicitation does not imply calibration to hourly SNOTEL data. Calibration would be based on the interval of observed, available data.

6c) If the SNOTEL data is only available for daily averages, then it is inconsistent to attempt to calibrate the hydrological model streamflow data at the hourly timescale. Please clarify, revisiting questions 1 and 2.

The solicitation does not imply calibration to hourly SNOTEL data. Calibration would be based on the interval of observed, available data.

ITEM-002:

7) It is impossible to deliver WRF-ECMWF with 4x daily updates for the 60-hour product. WRF-ECMWF is only updated 2x daily. Please clarify.

You are correct, for the 60-hour product, the WRF-ECMWF will be delivered two times per day. Delivery of the WRF-GFS will be four times per day for the 60-hour product.

ITEM-003.

8) While an accurate and objective calibration based on station data is desirable and important, DHSVM forecasts will be based on meteorological model fields, not station data. Meteorological forecast model precipitation (quantitative, spatial location, timing, and type) all differ from point-based station data, and that means the meteorological forecast data used to drive the DHSVM model will not be statistically similar to the station data used to calibrate the model. There is no provision to reconcile this deep inconsistency; that is, DHSVM forecasts could even be "unusable" because WRF QPF, precipitation type (snow versus rain) and temperature forecasts are biased or have significant errors (MAE, RMSE, etc.)

against the point meteorological station data used to calibrate the model, including the SNOTEL data.

Please clarify how you can expect a model calibrated against point meteorological station data (NWS, SNOTEL) to provide "accurate" forecasts when driven with NWP model forecast data? In fact, it could be strongly argued that this approach is scientifically indefensible. How will the accuracy and usability of the DHSVM model forecasts be judged? Please clarify.

The calibrated model should provide an accurate depiction of basin hydrology, including items such as the magnitude of hydrologic peaks and the rate of rise and fall of the hydrograph. Including other observed hydrological variables (e.g., SWE) as ancillary objectives in the calibration procedure confirms that the physical processes that influence runoff generation are accurately represented and the model predictions are robust under a range of conditions.