04._CVPIA_Fish_Data_and_Analytic_Support_Draft_PWS.pdf

PR #18MPRO0000

Attachment 0001

Performance Work Statement Data and Analytic Support for the Central Valley Project Improvement Act (CVPIA)

Fish Resource Area and the Bay Delta Office(BDO) Structured Decision Making Central Valley Project Improvement Act

Dated: March 29, 2018

1.0 Background:

The purpose of this Performance Work Statement (Work Statement) is to describe the methodology that will provide support the next five years in the ongoing process of Structured Decision Making (SDM) within the Fish Resource Area of the Central Valley Project Improvement Act (CVPIA), Title 34 of Public Law 102-575 and the Structured Decision Making (SDM). Support will take the form of data compiling, making data available, and using numerical tools to facilitate analysis of data related to fisheries performance, mitigation actions, and habitat restoration. This Project will refine and further develop the quantitative modeling component of the SDM through a collaborative process with agencies (State, Federal, and Local) and stakeholders (non-government organizations, Federal and State Water Contractors, etc). The resulting products will assist to prioritize funding actions to achieve the anadromous fish doubling goal of the CVPIA and management objectives.

This Work Statement also works to continue efforts that were initiated as part of Task Order Rl4PD00958 (2014) - Data and Analytic Support for Fish Resource Area Structured Decision Making - Central Valley Project Improvement Act (CVPIA)

In 2008, Listen to the River (Service, 2008), an independent review of the anadromous fish related provisions of the CVPIA, recommended improving the science based framework for decision making. The agencies jointly implementing the CVPIA, the U.S.

Bureau of Reclamation (Reclamation) and the U.S. Fish and Wildlife Service (Service), selected the SDM process to address the recommendation and led a “Core Team” consisting of representatives from the Service, Reclamation, California Departments of Fish and Wildlife (DFW) and Water Resources (DWR), and the U.S. National Marine Fisheries Service (NMFS) in developing an SDM framework as part of the Implementation Plan for the Fish Resource Area of the CVPIA (Implementation Plan, Service, 2014 Draft).

SDM provides a formal, documented, and open source process to develop quantifiable and measurable objectives and determine the best decision alternatives to meet those objectives using quantitative models. In the development of the Implementation Plan and the initial SDM framework, the agencies focused efforts on a limited number of decision models to represent potential hypotheses for how actions addressing limiting factors would improve natural production for Chinook salmon (of all races), steelhead, and sturgeon. The models provided reasonable results and a logical method for prioritizing limited resources. Sensitivity studies identified uncertainty in both the parameters and models where refinement could potentially improve the ability to determine the best decisions to maximize natural production.

Initial model development relied extensively on expert opinion for the parameters and relationships. Through this Performance Work Statement, the incorporation of measured data not available at the time of the Implementation Plan has helped reduce some of the parameter uncertainty.

Recognizing that the state of the science continually develops, the Implementation Plan establishes an adaptive management process to update and reevaluate priorities through SDM. This Project supports the adaptive management effort through refinement of the modeling components already developed by agency staff. Refinements to the SDM models facilitated under this Performance Work Statement may seek to develop alternative models and sub-models.

The initial models in the Implementation Plan represent an initial agency effort.

Reclamation and the Service seek support and buy-in for resource decisions by facilitating stakeholder participation in the process of formulating models. This Work Statement expands SDM to include stakeholder input by facilitating stakeholder participation in the models. The strategy for using SDM in the Implementation Plan does not seek consensus on any single decision model to represent potential costs and benefits.

SDM allows for competing hypotheses represented by different decision models and different modules within the models.

The Implementation Plan describes the update and revision process using SDM.

Products for this Work Statement would update the priorities determined by SDM and be reflected in the process for developing CVPIA Annual Work Plans. The first step is to create a dynamic Ontology that standardizes data naming conventions and linkages and makes it available to support and evaluate SDM models for the San Francisco Bay (Bay) and Sacramento and San Joaquin Delta (Delta), collectively known as the Bay-Delta.

Ultimately, the data will help support and refine Delta regional-level management actions. Establishing linkages between the data and SDM actions will also foster knowledge transfer between projects and regions across the regions of the Delta. The fishery and water management systems of interest to Reclamation are dynamic and our understanding of the model parameters we use to describe and be predictive about these systems is constantly developing. Thus, there is a possibility to continually iterate and always adapt for more accurate understanding by describing opportunities for more monitoring and more data.

2.0 Scope:

The scope of this Work Statement includes identifying potential data, compiling and making the data available, and the development of analytical models consistent with the Implementation Plan and the SDM Framework in coordination with the Core Team agencies and stakeholders. This Work Statement anticipates that agency and stakeholder representatives will provide the majority of the biological expertise and institutional knowledge. The Contractor would have an understanding of anadromous fish biology, experience in analytic tools, the ability to make data and information publicly available, and facilitation skills for the use of the analytic tools in a transparent public process.

3.0 Type of Contract:

The government will award a Firm Fixed Price Task Order from the 2012 IDIQ Contracts. Pricing will be per contract year.

4.0 Period of Performance:

The period of performance shall be for one (1) Base Year and four (4) option years. The Period of Performance reads as follows:

• Base Year – date of award through September 30, 2019

• Option Year I – October 1, 2019 through September 30, 2020

• Option Year II – October 1, 2020 through September 30, 2021

• Option Year III – October 1, 2021 through September 30, 2022

• Option Year IV – October 1, 2022 through September 30, 2023

5.0 Administrative Information:

5.1 Recognized Holidays:

The contractor is not required to perform services on federal holidays including:

New Year’s Day Labor Day Martin Luther King Jr.’s Birthday Columbus Day President’s Day Veteran’s Day Memorial Day Thanksgiving Day Independence Day Christmas Day

5.2 Hours of Operations:

The contractor is responsible for conducting business, between the hours of 8:00 AM and 6:00 PM Monday thru Friday except Federal holidays or when the Government facility is closed due to local or national emergencies, administrative closings, or similar Government directed facility closings. The Contractor shall at all times maintain an adequate workforce for the uninterrupted performance of all tasks defined within this PWS when the Government facility is not closed for the above reasons. When hiring personnel, the Contractor shall keep in mind that the stability and continuity of the workforce are essential.

5.3 Place of Performance:

Coordination with stakeholders and agencies will occur within the Central Valley of California, primarily in Sacramento and the surrounding areas, but could include Areas from Bakersfield, CA to Redding, CA. The contractor shall coordinate with the COR on meeting locations that may include government and stakeholder facilities. Technical work that does not require coordination may be performed at the contractor’s facilities. It is anticipated that there will be nine (9) in person meetings with other meetings being help by teleconference supplied by the contractor.

5.4 Security:

No specific level of security is required. The contractor shall be responsible for safeguarding all government equipment, information and property provided for contractor use.

5.5 Delivery:

The contractor shall be responsible for developing interim products and materials necessary to accomplish this Work Statement. Deliverables shall be developed during the base year and updated in each of the option years. Specific required deliverables include:

1. Project Management Plan: The contractor will maintain and update the Project Management Plan (PMP) as necessary and at least monthly. The PMP will include a once monthly report on contract activities including a project contact log.

2. Ontology Technical Memo (TM): Documentation of the datasets uploaded and made accessible for exploration and analysis as well as the datasets that were desired, but incompatible or unavailable. This TM shall be updated with each option year.

3. Data Exploration TM: Documentation of initial efforts to manipulate the data and the identification of potential avenues to develop in more detail. This TM shall be updated with each option year.

4. SDM Priorities TM: documentation of the final custom metrics and conceptual models developed in full and the resulting priorities for the fish resource area.

This TM shall be updated with each option year.

All products shall be provided electronically in native file formats and adobe acrobat portable document format (PDF) using appropriate media. Products shall include the supporting data in a broadly accessible format, e.g. Microsoft Excel, Microsoft Access, text files, or other agreed upon format.

Up to 20 hard copies of each TM shall be provided upon request when finalized for each task.

6.0 Government-Furnished Property:

The contractor is responsible for all equipment, materials, and facilities to accomplish this Work Statement. The Government may make facilities available for meetings and coordination.

7.0 Applicable Publications (Current Editions):

1. Documents directly relevant to this Work Statement include:

2. Central Valley Project Improvement Act anadromous fish related provisions, section 3406(b) of Public Law 102-575. – www.usbr.gov/mp/cvpia

3. Listen to the River: An independent review of the Anadromous Fish Restoration

Program – www.usbr.gov/mp/cvpia

4. Implementation Plan for the Fish Resource Area of the Central Valley Project Improvement Act – www.usbr.gov/mp/cvpia

5. Structured Decision Making Fact Sheet -http://www.fws.gov/science/doc/structured_decision_making_factsheet.pdf

6. Conroy MJ, Peterson JT. 2009. Integrating management, research, and monitoring: balancing the 3-legged stool. Pages 2 - 10 in Cederbaum SB, Faircloth BC, Terhune TM, Thompson JJ, Carroll JP, eds. Gamebird 2006: Quail VI and Perdix XII. 31 May - 4 June 2006. Warnell School of Forestry and Natural Resources, Athens, GA, USA.

7. Conroy, MJ, and Peterson JT (2013). Decision Making in Natural Resource

Management: A Structured, Adaptive Approach. Wiley-Blackwell, April p474.

Additional information on the decisions supported by SDM can be found in the Final Plan for the Anadromous Fish Restoration Program (Service, 2001) and related “Working Papers” – http://www.fws.gov/stockton/afrp/

8.0 Objectives:

The overall objective is, for the next 5 years (federal fiscal years 2018, 2019, and 2020, 2021, and 2022), to inform funding decisions and projects to address the factors limiting natural production consistent with the Implementation Plan, in order to achieve the doubling goal under section 3406(b)(1) of the CVPIA, “develop within three years of http://www.usbr.gov/mp/cvpia http://www.usbr.gov/mp/cvpia http://www.usbr.gov/mp/cvpia http://www.fws.gov/science/doc/structured_decision_making_factsheet.pdf http://www.fws.gov/stockton/afrp/ enactment and implement a program which makes all reasonable efforts to ensure that, by the year 2002, natural production of anadromous fish in Central Valley rivers and streams will be sustainable, on a long-term basis, at levels not less than twice the average levels attained during the period of 1967-1991…”. The specific goals and objectives under this Work Statement include:

1. Identify data relevant to natural production of anadromous fish and the availability and quality of that data;

2. Compile available data and make the data accessible for analysis by agencies and stakeholders;

3. Update the expert opinion in the Implementation Plan SDM models with measured data where possible;

4. Develop web based tools in coordination with agency representatives and stakeholders for transmitting model data and translating model output; and

5. Update the SDM priorities using the refined models and updated data.

The agencies have developed and will support one or more SDM models. This Performance Work Statement represents an experimental effort to test collaborative ways of developing decision support both within, between, and outside of the agencies.

Updates and revision is a part of the adaptive management process recommended by the AFRP Plan and proposed under the Implementation Plan.

Tasks shall occur for the base year and then repeat for updates in each of the option years.

9.0 Tasks:

Task 1 – Project Management

1. This Performance Work Statement anticipates coordination with multiple agencies and stakeholders. A project management plan (PMP) provides the formal documentation for approaches to accomplish the goals and objectives. The PMP shall address at minimum:

2. Technical Approach: The methods and procedures to accomplish the deliverables under this Work Statement including interim deliverables and milestones proposed to track progress and seek feedback.

3. Schedule: Proposed dates for interim deliverables and milestones.

4. Coordination: Practices and procedures for working with the Core Team agencies and stakeholders including:

a. Water Users: Water users make payments into the Central Valley Project Restoration Fund (CVPRF) and, upon completion of reasonable measures, would find payments decreased by half. Actions that require additional flows may affect water supply.

b. Power Users: Power users make payments into the CVPRF and, upon completion of reasonable measures, would find payments decreased by half.

c. Commercial and Sport Fishing Interests: Fishing interests would desire increases in overall production of adults for ocean and in-river harvest including measures that also support increased hatchery survival.

d. Environmental Groups: Environmental groups would be anticipated to support ecological benefits provide by many of the restoration actions.

e. Local Watershed Groups: Local groups would be anticipated to support geographic based priorities that align with the individual watersheds.

5. Roles and Responsibilities: Anticipated functions to be provided by the various agencies, stakeholders, and proposed members of the consultant project team.

6. Risks and Mitigation: Factors that may increase the scope, schedule, and/or budget under this Work Statement and the proposed methods to mitigate adverse consequences to the accomplishment of this Work Statement. Anticipated risks include, but are not limited to:

a. Proprietary Information: Agencies and entities may be unable or unwilling to release data and information that would benefit the project, e.g. draft data, hard copies, ongoing research, etc.

b. Insufficient Data (quantity or quality): Adequate reliable information at a spatial or temporal scale may not be available for a sufficient continuous time span to perform the desired analyses.

c. Competing Data: More than one set of information may be available to represent a specific process or metric leading to different conclusions for the same analysis.

d. Insufficient Capacity: The number of metrics and analyses may exceed the ability to manage within the available schedule and budget.

e. Scientific Disagreement: One or more parties may disagree on the appropriate metrics or data models to explore for evaluating priorities.

f. Policy Positions: Participants may favor specific outcomes and attempt to bias analysis towards specific interests.

g. Inadequate Participation: Low levels of involvement by key offices, agencies, and stakeholders may undermine support for the priorities developed by the revised SDM models.

7. Review and Approval Process: How the contractor intends to achieve acceptance of final deliverables.

Task 2 – Data Discovery:

Data discovery shall result in an Ontology Technical Memorandum (Ontology TM) to document the identification and organization of available datasets and analysis methods into a logical structure that supports access, and recall. The Ontology TM shall include, at minimum:

• A listing of physical and biological data relevant to anadromous fish based on measurements, numerical models, and expert opinion;

• Current status and future availability of information relevant to anadromous fish;

and

• Organizational strategy for retrieving and maintaining information over time.

This task is primarily composed of the development of an Ontology TM to document the identification and organization of available data sets for use in SDM Tool Development.

Currently, Reclamation has developed a data ontology that captures hundreds of unique data objects relevant to water resources and aquatic ecosystem restoration and management. This task will expand that based on Technical Team input. It is proposed that the Ontology developed for the CVPIA Science Integration Team (SIT) Ontology be considered as a reference for use in the initial development of the Ontology for this project. Currently, the data objects in the Ontology include individual records for CDEC and US Geological Survey stream gages and groundwater monitoring stations, fisheries data, water quality records, hydrologic and hydraulic model output, and other data, all of which are of potential value to a Bay-Delta decision support tool. As currently configured, these data objects are available for rapid visualization, exploration, and analysis.

The ontology should use a hierarchical data structure where data objects are sorted into individual “models”, with subset “metrics”. Clearly-stated terms and quick access to data is critical for facilitating real-time visualization of relationships between data in the wide range of engagements required to support the Bay-Delta decision support tool. Use of the data and relationships during discussions by the various groups allows for a deeper consideration of the complex technical relationships being addressed by the structured decision model.

This enhanced platform with added data should automatically scale to the size of the data when displaying it for analysis. The enhanced platform should also automatically integrate disparate data that is related spatially, temporally, or otherwise. Accordingly, an Ontology TM will be maintained and updated regularly throughout the project for use by analysts. As models and metrics change, or are edited for clarity, the analyses that rely on the data objects will be automatically updated. The Ontology should be updated annually or more frequently as necessary. Further, by using the Ontology to foster technical discussions in real time, the Core Team will improve knowledge management because all the meaningful findings from past and ongoing studies will be captured quickly and clearly on the data platform. For example, if technical representatives are considering the effects of temperature and flow in a particular region of the Delta, the real-time data and analyses supporting that discussion can be saved and immediately recalled at a subsequent meeting, or used at future briefings.

Deliverable:

Draft Ontology TM Final Ontology TM

Task 3 – Data Integration:

Data integration shall result in a Data Exploration TM that documents the backend infrastructure that allows for the upload of different data types, visualization across different data sets, and transparent workflow processes for performing calculations and identifying the relationships between data. This Work Statement anticipates that agencies and stakeholders shall provide data in a wide range of formats. The Data Exploration TM shall include the uploading of information and the identification of preliminary lines of investigation in preparation for model development as an intermediate step in collaboration to guide future efforts. Data integration shall include, at minimum:

• The ability to readily incorporate and update different types of information including, but not limited to tables, spatial data, time series, and unstructured information;

• Workflow tools to query, visualize, combine, and run analyses on data in manner that transparently documents logic processes for knowledge transfer;

• Accessibility for agencies and stakeholders to operate and experiment with workflow tools outside of formal workshops and independent of the facilitators, ideally via the internet;

• Security to allow for controlling access and modification to workflow products so that small groups of specific individuals can experiment prior to the larger agency and stakeholder community.

• Scalability to allow for increasing to storage to account for additional data.

• Potential data patterns and pathways to explore in the development of detailed decision models.

• Large group workshops held in person, within the Central Valley, for facilitated exploration of data and development of workflow processes held no more frequently than every other month.

This task outlines the majority of effort required to support a Bay-Delta decision support tool with a data analytics platform. Expertise in integrating disparate data across multiple disciplines is required for developing visualizations of the integrated data for interdisciplinary teams for analysts. Draft data visualization will serve for reviewing and iterating towards functional, meaningful products that will transfer knowledge regarding complex concepts. By maintaining the dynamic project Ontology (Task 2), visualization is easily repeatable for all regions of the Bay-Delta. Analytics within the Data platform also allow for various mathematical and statistical analyses, as may be appropriate.

Preliminary visualizations would be developed for each component of the SDM models that were developed by the Technical Team. It is anticipated that some Delta regions will be more data rich than others. A data integration platform is immediately scalable to petabyte level data sets. In terms of our data platform, this means that it is possible to integrate very large volumes of data all onto the same analytical space. While it’s difficult to imagine immediate use of a petabyte of monitoring data in the Delta, it is much easier to imagine petabyte-scale modeling output.

Maintaining the analysis and visualization elements on a platform that scales to petabyte-level input insures that data exploration and discovery can continue in a seamless fashion.

The data platform should be available to facilitate workshop-scale presentations and discussions that can fundamentally change the nature of decision-making. By using the same data platform to present different hypotheses, it becomes easier to drill down to the core differences in an issue. It becomes a more straightforward comparison of the various weighting of factors affecting the resource. Once the core elements of difference are highlighted, it often becomes easier to develop monitoring programs that can build knowledge about a particular limiting factor, or evaluate the range of possible outcomes and the degree of effect one outcome might have at the expense of the other.

Deliverable:

Draft Data Exploration TM Final Data Exploration TM

Task 4 – Metrics and Model Development:

Metrics and model development shall result in a Decision Model TM documenting the custom metrics and analysis process for manipulating the data and updating the SDM models. This Work Statement anticipates that models shall be developed through a combination of cooperative large group sessions facilitated by this Work Statement and independent efforts by one or more agency and stakeholder groups working largely independently with support by this Work Statement on a case by case basis. Metrics and Model Development shall include, at minimum:

• Large group all day workshops held in person, within the Central Valley, for facilitated exploration of data and development of workflow processes held no more frequently than every other month.

• Small group coordination held on an as needed basis primarily via conference calls and webinars as follow-ups from the large group workshops.

• Documentation of multiple conceptual models representing different hypotheses for linking SDM decision alternatives to SDM objectives and the Utility function described in the SDM framework.

This task would connect the Data analytics described and developed in Tasks 2 and 3 to a set of continuously updated SDM models for all regions of the Delta. As necessary, these can easily be expanded to include the scenarios developed by the Technical Team. As with prior tasks, much of this effort will be iterative, in part due to the evolving nature of the knowledge base, and in part because of the range of uses for the product. However, the data platform should be available immediately to begin supporting technical team meetings, even while new versions are under development and additional alternatives are considered.

As noted in Task 3, a data platform should be easily scalable between small conference-call subject groups and large multi-interest work groups. The model development staff should be available to support Reclamation activities as early as the annual planning effort and with minimal preparation time. The model development team should be available to support conference calls and should be able to use the data platform to provide real-time data interaction via on-line meeting (GoToMeeting, WebEx, or similar). It is anticipated that monthly 1-day meetings and half-day conference call participation by the model development staff will support the Interagency Technical Teams preparation so that the model’s analysis would be available to support discussions as they occur. Some preparation and coordination would occur with the Reclamation project manager to assure that important data interactions are pre-loaded and ready;

however, the data platform would accommodate a degree of free-flowing dialogue and information discovery. This “on-the-fly” usage often helps refine the immediate questions so that more detailed analysis can become more focused. This task would also produce multiple conceptual models representing different approaches to linking the technical team’s scenario alternatives with Reclamation Biological Opinion and DWR’s D-1641 San Joaquin Delta Estuary Program objectives related to fish populations and water quality requirements. Ultimately, the weighing of scenarios results in a utility score that is meant to inform Bay-Delta decisions and capture knowledge gained from past decisions to build an increasing store of knowledge for future decisions.

Deliverables:

Draft Decision Model TM

Final Decision Model TM Participation in Monthly Technical Team Meetings and Phone Calls

10. Performance Assessment and Disincentives:

1. Monitoring Performance. During the course of the evaluation period, the COR will track Contractor performance. Interim (mid-term) evaluations may be provided to identify strengths and weaknesses in the Contractor's performance during the period being evaluated. At the end of the period, the COR will assess the Contractor's performance in accordance with the Quality Assurance Surveillance Plan (QASP) and report to the Contracting Officer.

2. Contractor Self-Assessment. Following each evaluation period, the Contractor may provide a written self-assessment of its performance to the COR to be considered in its report to the Contracting Officer. The self-assessment shall be submitted not later than 5 working days after the end of each evaluation period. The self-evaluation shall not exceed 1 page per PRS element. The self-assessment shall address both the strengths and weaknesses of the Contractor's performance during the evaluation period. If deficiencies in performance are noted, the Contractor shall describe the actions planned or taken to correct such deficiencies and avoid their recurrence.

3. COR Recommendation. The COR will consider all evaluations and any other pertinent information, including Contractor self-assessment, and will prepare a report to the Contracting Officer with findings and recommendations. The Contractor will be provided a copy of the draft findings and recommendations of the COR and will be afforded the opportunity to identify factual errors. The COR's draft recommendation is not subject to negotiation and the COR will not engage in discussions with the Contractor. Any errors identified by the Contractor will be addressed by the COR in its final report. The Contractor will be provided a copy of the final COR report at the same time the report is submitted to the Contracting Officer.

4. The Quality Assurance Surveillance Plan is an evaluation method the government uses to surveillance performance to determine whether the Contractor meets the standards of performance as defined in the PWS. The absence of a QASP for any contract requirement, however, shall not detract from its enforceability or limit the rights or remedies of the government under any other provision of the contract in determining the quality of the Contractor performance.

PERFORMANCE REQUIREMENTS SUMMARY AND

QUALITY ASSURANCE SURVIELLANCE PLAN (QASP)

Task 1 – Project Management Product Performance Indicator Performance Standard Project Management Plan Updates

Prompt receipt of updates including scope, schedule, budget, and risks.

Modifications received by the first business day of each month unless otherwise agreed to by the COR. For each update, schedule changes should be less than 1 month and budget changes less than 10% without an agreed upon change in scope. Scope changes should be tied to previously identified risk and mitigation measures.

Schedule Deliverables provided within the appropriate CVPIA or BDO phase as described by the Implementation Plan.

Task 2 – Data Discovery

Ontology TM Follow-through of dataset identification by participants.

All agencies and stakeholders have been provided an opportunity to identify datasets and each identified dataset is reflected in the TM.

Agreement on organization structure.

The majority of agencies and stakeholders agree that the proposed structure provides a reasonable means to organize data.

Task 3 – Data Integration

Data Integration Availability of the data integration tool for agency and stakeholder use.

Information provided to the contractor is made publicly available within one month or as otherwise agreed to by the COR. When identified as available by the contractor, and unless notified in advance of a planned outage, 90% of information is available within one business day.

Task 4 – Metrics and Decision Model Development

Decision Model TM Action item follow-through from Stakeholder workshop.

Action items will be recorded within each workshop or stakeholder meetings and agreed to upon completion of the meeting within one day. All action items identified during stakeholder workshops are associated with a plan that addresses the comment within 1 week unless otherwise agreed to by the COR. Plans include a schedule and specific deliverable.