Attachment 1 - EPIC 2.0 IDIQ PWS.docx.pdf

EPIC 2.0 IDIQ PWS EARTH PREDICTION INNOVATION CENTER (EPIC)

Department of Commerce (DOC)

National Oceanic and Atmospheric Administration

(NOAA)

Office of Oceanic and Atmospheric Research (OAR)

Weather Program Office (WPO)

Earth Prediction Innovation Center (EPIC)

5-Year IDIQ

PERFORMANCE WORK STATEMENT (PWS)

1. EXECUTIVE SUMMARY 3

2. BACKGROUND 3

3. OBJECTIVES AND SCOPE 4

4. TASKS 5

5. DELIVERABLES 6

7. PLACE OF PERFORMANCE 9

8. CONTRACT TYPE 9

9. SECURITY REQUIREMENTS 9

10. PERFORMANCE REQUIREMENT SUMMARY (PRS) 9

1. EXECUTIVE SUMMARY

This Performance Work Statement (PWS) outlines the Earth Prediction Innovation Center (EPIC) requirements for advancing weather modeling skill, reclaiming and maintaining international leadership in the area of numerical weather prediction, and improving the transition of research into operations. The general objective and scope of this PWS is to obtain state-of-the-art software architecture, software engineering and scientific and technical support services to maintain and continuously expand the community modeling infrastructure that facilitates robust user support and easy access for advanced Earth system models, applications and components in the framework of the Unified Forecast System (UFS), a community-based, coupled, comprehensive Earth modeling system spanning spatial scales from local to global, and temporal scales from hourly to seasonal forecast scales.

The contractor shall facilitate innovation and accelerate research to operations including through an extramural, virtual center that will:

● Leverage the Weather Enterprise to provide expertise on removing barriers to improving numerical weather prediction;

● Enable scientists and engineers to effectively collaborate in areas important for improving operational numerical weather prediction skill, including model development, data assimilation techniques, systems architecture integration, and computational efficiencies;

● Strengthen the National Oceanic and Atmospheric Administration (NOAA)’s ability to undertake research projects in pursuit of substantial advancements in numerical weather prediction skill;

● Utilize and leverage existing resources across the NOAA enterprise; and

● Advance and create community weather research modeling systems that:

○ Are accessible by the public;

○ Meet end-user requirements for running on computers and networks apart from and not associated with secure NOAA information and technology systems; and

○ Utilize, whenever appropriate and cost-effective, innovative strategies and methods, including cloud-based computing capabilities, for hosting and management of part or all of the forecast systems.

EPIC will continuously integrate and develop the community-modeling environment with all partners in the Weather Enterprise through community-based, coupled, comprehensive Earth system models, to improve Numerical Weather Prediction (NWP) and accelerate the Research to Operations and Operations to Research (R2O2R) process. The NWP approach shall span local to global domains and predictive time scales from sub-hourly analyses to seasonal predictions, and will include both physics-based and data-driven (Machine Learning) methods. EPIC’s approach shall leverage the Weather Enterprise in alignment with NOAA Modeling Strategy 2024-20331. It is envisioned that EPIC will lead NOAA in providing strategies and infrastructure for data-driven / ML-based Earth system modeling.

The Contractor shall provide all Scientific and Technical Support Services supporting EPIC, as outlined in this PWS. All activities in this contract will be managed through the Weather Program Office (WPO) within NOAA’s Oceanic and Atmospheric Research (OAR) Line Office.

2. BACKGROUND

The Department of Commerce (DOC), NOAA, and OAR’s WPO support world-class weather research, leading to advancements in NOAA’s forecasting capabilities that save lives, reduce property damage and enhance the national economy.

In pursuit of NOAA’s vision and mission, WPO works in close partnership with OAR Laboratories, the National Weather Service (NWS), and the weather enterprise to improve extreme weather forecasts across a variety of phenomena and timescales. including severe weather, extreme rainfall, winter weather, subseasonal to seasonal forecasts, tropical weather, and fire weather and atmospheric composition. Through EPIC, WPO enables a community of practice including OAR Laboratories, the NWS, industry, academia, and other national and international organizations to improve the UFS and to help democratize operational numerical weather prediction. In short, WPO facilitates the improvement of the modeling suite leading to more seamless weather-to-climate numerical weather prediction on timescales from minutes to months

1 https://repository.library.noaa.gov/view/noaa/56332 and on spatial scales from individual clouds to the globe (WPO Strategic Plan, 2022-20262).

The Weather Research and Forecasting Innovation Act of 2017 (WRFIA, Public Law 115-25) directs NOAA to prioritize improving weather data, modeling, computing, forecasting, and warnings to protect life and property and enhance the enhancement of the national economy. The authorizing language for EPIC, in the National Integrated Drought Information System Reauthorization Act of 2018 (NIDISRA, Public Law 115- 423), calls for NOAA to accelerate community-developed scientific and technological enhancements into the operational applications for NWP, which requires “advancing weather modeling skills, reclaiming and maintaining international leadership in the area of numerical weather prediction, and improving the transition of research into operations.” EPIC’s vision is to enable the most accurate and reliable operational numerical forecast model in the world. EPIC’s mission is to be the catalyst for community research and modeling advances that continually inform and accelerate advances in our nation’s operational forecast modeling systems (EPIC Strategic Plan3). The EPIC-supported community modeling framework also supports innovation in modeling by facilitating the public’s access to NOAA model codes, as mandated in the LEGEND Act of 20214.

3. OBJECTIVES AND SCOPE

The objective of this contract is to make continuous progress building upon the strategic outcomes achieved in past performance from activities associated with the EPIC Program in years 1-5, and to support and participate in developing an updated EPIC program-level strategic plan and an associated strategic implementation plan for defining success of the EPIC contract in years 6-10, including:

1. Maintenance and enhancement of the Earth Prediction Innovation Center;

2. Enabling strong collaboration with the Earth and Atmospheric Sciences community to nurture a highly inclusive and diverse modeling community of practice across the Weather Enterprise;

3. Maintenance and enhancement of a publicly accessible end-to-end testing and development environment, from pre- to post-processing, for core Earth system model applications;

4. Demonstrate EPIC success by bringing in innovations to improve Earth System model performance.

The contract shall commit to maintain and continuously update key elements of the Earth Prediction Innovation Center using state-of-the-art technology and project management principles, including but is not limited to the following scope:

● Project Management. Provide a consistent project management approach that prioritizes strategic goals, integrating priorities across technical teams, program office, and stakeholders, ensuring a high-rate of objectives completion and achieving high-levels of business value.

● Infrastructure. Support and expand a cross-cutting community platform as the public-facing interface of the UFS, including the EPIC Community Portal (ECP, epic.noaa.gov), integrated with continuous integration and continuous delivery (CI/CD) pipelines to enable a public-facing testing and development environment that leverages on-premise and cloud high-performance computing resources provided by NOAA and partners, hosted in an EPIC cloud account as part of the NOAA enterprise cloud system

● Code Management. Support and expand platform-agnostic containerized versions of the UFS code, including configurations, applications and selected components, to ensure portability and usability of on-premise and cloud High Performance Computing (HPC) platforms. Support and expand a code management framework for UFS code that follows an agreed-upon system architecture and infrastructure design and best practice following an open-source and open-development paradigm. This system architecture should facilitate the transition of research to open source code and operations within NOAA, taking into account the needs of research and operational modeling communities.

● Systems Integration. Support end-to-end prototypes and continuous integration of UFS configurations, applications and selected components, a unified build-test package management system for the UFS including but not limited to the data assimilation framework (e.g., Joint Effort for Data Assimilation Integration, JEDI), and continuously update the ECP to ensure web artifacts are up-to-date with the status of UFS and EPIC.

● Data Management. Support data management and ingestion requirements for unit, regression and scientific testing of UFS configurations, applications and selected components.

● Tracking performance and skill improvement. Support and expand a hierarchical testing and development

4 https://www.congress.gov/bill/117th-congress/senate-bill/1127 3 https://wpo.noaa.gov/wp-content/uploads/2022/09/Signed_EpicStrategicPlan2021-0903.pdf 2 https://wpo.noaa.gov/wp-content/uploads/2022/11/WPO_2022_2026_Strategic_Plan_Final.pdf framework that continuously tracks improvements in UFS computational performance and forecast skill.

● Community Releases. Support community releases for UFS applications and components, including release plans and a framework for continuous coordination and support.

● Advanced User Support. Consolidate and expand an advanced user support framework.

● Community Engagement. Support and update a UFS community engagement strategy in collaboration with UFS application and cross-cutting teams. Facilitates engagement between the UFS modeling community and other modeling communities in the United States.

● Continuous Stakeholder Engagement. In collaboration with the Government, support and participate in a community governance framework to enhance and support community access to the UFS, define and implement best practices for community development, outline evaluation criteria for community contributions, and document the processes to guide those contributions back into the UFS and where applicable, operations. Supports various teams within the community governance framework, including but not limited to functions/teams for architecture, infrastructure, education, outreach, performance, evaluation, and releases.

● Machine Learning (ML) for NWP. Support and expand a public-facing community modeling infrastructure to allow for easy initialization, training, and visualization of data-driven NWP models based on AI and ML and hybrid NWP models that combine the strength of data-driven and physically-based models using UFS/NOAA analyses and data products. Previous elements in this list (e.g. project management, infrastructure, code management, data management, community releases, stakeholder engagement, etc.) also apply to the AI/ML for NWP scope.

In addition to sustaining the foundational functions of EPIC, the contractor is also expected to actively facilitate and provide innovative solutions to improve Earth system model performance in collaboration with UFS Application and cross-cutting teams, with an ultimate goal toward their potential integration in next-generation UFS models and applications.

4. TASKS

All tasks are associated with deliverables described in Section 5 Deliverables. Requirements for deliverables are provided in Section 9 Performance Requirement Summary (PRS). Tasks, deliverables and requirements listed in this PWS are subject to modifications, extensions and new scope of work to be determined through PWS specific to contract task-orders.

4.1. Project Management:

4.1.1. Kickoff Meeting: The Contractor shall participate in an initial kickoff meeting to be held virtually within ten

(10) business days after issuance of the IDIQ award. The Contractor shall assume that the initial project kickoff meeting will require all contractor key personnel in attendance. The Kickoff Meeting is not expected to be more than two (2) hours long. The Contracting Officer will coordinate logistics for and chair the Kickoff Meeting.

4.1.2. Project Management Plan: The contractor shall develop a Project Management Plan (PMP) to fulfill the requirements of the contract. The PMP shall include:

4.1.2.1. The management approach, organizational structure, organizational resources, and management controls to be used to meet the IDIQ requirements throughout the contract’s POP;

4.1.2.2. The process to oversee and manage work activities at the IDIQ level and at the TO level ;

4.1.2.3. The process to recruit, train, and retain personnel at the IDIQ level;

4.1.2.4. A communication plan detailing all lines of communication between the Government and the

Contractor.

4.1.2.5. A Project schedule that details activities, dependencies, and timeframes for meeting NOAA objectives and deliverables;

4.1.2.6. For each awarded task order, a staffing plan and organizational chart identifying relationships, authorities, and responsibilities;

4.1.2.7. Regular updates to reflect any applicable programmatic changes. Updates to the PMP shall be submitted to the COR and PM for review and approval. Comments from the COR and/or PM shall be addressed within 10 business days.

4.1.3. Quality Control Plan (QCP): The contractor shall perform in accordance with the Quality Control Plan incorporated into the contract. The QCP shall be updated on an as needed basis, subject to review and approval of the COR and PM.

4.1.4. Transition Plan: The contractor shall perform in accordance with the Transition Plan incorporated into the contract.

4.1.5. Biweekly Progress Meetings: The contractor shall participate in biweekly (every two weeks) meetings, during which the Contractor shall identify and report for resolution to the Government any issues that prevent the Contractor from performing the work activities that the Contractor is unable to resolve independently. The contractor shall develop a Biweekly Progress Meetings agenda, and resulting minutes, to track and maintain an IDIQ-level risk registry. The COR may authorize progress meetings to reduce in frequency (e.g., monthly) at the discretion of the COR. The risk registry should contain, at a minimum:

4.1.5.1. List of relevant issues and short description of the issues.

4.1.5.2. Severity: assessment of issue severity i.e., likelihood and consequences (High/Medium/Low);

4.1.5.3. Date Found: – date that issue was identified by the Contractor;

4.1.5.4. Duration: number of days that issue has been open;

4.1.5.5. Status: representing issue status as open or closed.

4.1.5.6. Updated: date for updated status;

4.1.5.7. Details: narrative description of activities that have been completed to resolve issue;

4.1.5.8. Assigned To: identify Government POC or Contractor that is assigned to resolve the issue;

4.1.5.9. Solution or Workaround: description of the actions and activities taken to resolve or mitigate the issue

4.2. The contractor shall sustain foundational functions of EPIC as stated in section 3, and propose innovative solutions to improve Earth system model performance.

4.3. The contractor shall continuously support and enhance a cross-cutting community platform as the public-facing interface of the UFS.

4.4. The contractor shall continuously support and enhance a code management framework for the UFS.

4.5. The contractor shall support the development of end-to-end UFS prototypes and continuous integration of UFS configurations, applications and selected components that facilitate and continuously improve the R2O2R process .

4.6. The contractor shall plan and support UFS releases in coordination with application and cross-cutting teams.

4.7. The contractor shall continuously coordinate, support and enhance an advanced user support framework, including the development and maintenance of user support products and services.

4.8. The contractor shall continuously support and enhance a community engagement strategy, including community engagement products and services.

4.9. The contractor shall support and actively participate in a community governance framework to enhance and support community access to the UFS and to engage key stakeholders to co-develop the UFS.

4.10. The contractor shall support and enhance a public-facing community modeling infrastructure to allow for easy initialization, training, and visualization of data-driven NWP models based on ML and hybrid NWP models.

4.11. IDIQ level Key Personnel, as designated in the Key Personnel clause CAR 1352.237-75, are required to support the EPIC program on a full-time basis, provided their positions are fully funded under a task order. The COR may grant waivers from this requirement on a case by case basis.

5. DELIVERABLES

All deliverables shall follow requirements provided in Section 9, Performance Requirement Statement (PRS), which also provides performance standards, methods of surveillance and incentives for deliverables. Tasks, deliverables and requirements listed in this PWS are subject to modifications, extensions and new scope of work to be determined through additional PWS specific to contract task-orders. The following is a schedule of all deliverables, including administrative deliverables, required during the period of performance of this contract. Note that reporting requirements with conditional applicability are not included in the below table, but are still required if applicable (e.g., FAR 52.219-28, FAR 52.222-50, etc.).

Deliverable Time or Frequency Medium/Format/# of Copies Submit To

Per 4.1.1, attendance at the IDIQ Kickoff Meeting

Within 10 business days of award Video-teleconference Contracting Officer

Per 4.1.2, A contract IDIQ

PMP.

One month after the start of the contract period of performance;

updates as needed.

Microsoft (MS) Word Report via email.

EPIC Program Manager (PM) and Contracting Officer Representative

(COR)

Per 4.2, A plan for sustaining the foundational functions of EPIC as stated in section 3, and innovative solutions to improve Earth system model performance in collaboration with UFS application and cross-cutting teams, with an ultimate goal toward their potential integration in next-generation UFS models and applications that can help execute EPIC’s vision and mission.

As part of the plan, include a transition plan to sustain EPIC foundational functions with a clear timeline no later than 30 days from the beginning of the period of performance (PoP), and a development and integration plan for innovative solutions no later than 90 days from the beginning of PoP.

Microsoft (MS) Word Report via email.

PIC Program Manager (PM) and Contracting Officer Representative

(COR)

Per 4.3, A cross-cutting community platform as the public-facing interface of the UFS, including the ECP, CI/CD pipelines to enable a public-facing testing and development environment.

An architecture design and operating plan following the proposal.

Documentation is Continuously updated as new architecture components become available.

Products and services are delivered and reported according to timelines determined on a task-order basis. A comprehensive report annually.

Online5 artifacts as part of the NOAA enterprise information technology (IT) systems, MS Word Reports, and Power-Point presentations or additional documentation artifacts, where applicable, via email.

EPIC PM and

COR

Per 4.4, A code management framework that is continuously supported, A code management plan following the proposal. Documentation is continuously updated as new repos, procedures, and policies become

Online artifacts as part of the NOAA enterprise information technology (IT) system, MS Word

EPIC PM and

COR

5 Online refers to websites, wiki pages, GitHub, and any other EPIC-related and community web-based resources. All online deliverables require a snapshot (word doc or PDF) reflecting their content and state at delivery time.

Copies Submit To coordinated and enhanced. available. Products and services are delivered and reported according to timelines determined on a task-order basis. A comprehensive report annually.

Reports, and Power-Point presentations or additional documentation artifacts, where applicable, via email.

Per 4.5, The development of end-to-end UFS prototypes and continuous integration of UFS configurations, applications and selected components that facilitate and continuously improve the R2O2R process.

An enhanced UFS system architecture and infrastructure plan following the proposal. Documentation as new components, system integration procedures, and policies become available. Products and services are delivered and reported according to timelines determined on a task-order basis. A comprehensive report annually.

Online artifacts as part of the NOAA enterprise information technology (IT) system, MS Word Reports, and PowerPoint presentations or additional documentation artifacts, where applicable, via email.

EPIC PM and

COR

Per 4.6, Releases and capability announcements of UFS Applications and core components/models, including release plans and a framework for continuous coordination and support.

An updated EPIC Release/capability announcement Process for UFS applications and core components/models following the proposal. Documentation is updated as new procedures and policies become available. Planned releases delivered and reported according to timelines determined on a task-order basis. A comprehensive report annually.

Online artifacts as part of the NOAA enterprise information technology (IT) system, MS Word Reports, and Power-Point presentations or additional documentation artifacts, where applicable, via email.

EPIC PM and

COR

Per 4.7, An advanced user support framework that includes the development and maintenance of user support products and services, and is continuously coordinated, supported and updated.

A standard operating plan (SOP) following the proposal.

Documentation is updated as new procedures and products become available. Products and services delivered and reported according to timelines determined on a task-order basis. A comprehensive activity report annually.

Online artifacts as part of the NOAA enterprise information technology (IT) system, MS Word Reports, and Power-Point presentations or additional documentation artifacts, where applicable, via email.

EPIC PM and

COR

Copies Submit To

Per 4.8, A community engagement strategy that includes community engagement products and services, and is continuously supported, maintained and updated.

An EPIC community engagement and outreach plan following the proposal.

Documentation is updated as new procedures and types of services become available. Products and services are delivered and reported according to timelines determined on a task-order basis. A comprehensive report annually. Interim public-facing reports after specific engagements such as community modeling workshops that represent the activities and outcomes of the engagement.

Online artifacts as part of the NOAA enterprise information technology (IT) system, MS Word Reports, and Power-Point presentations or additional documentation artifacts, where applicable, via email.

EPIC PM and

COR

Per 4.9, A plan to support and participate in a community Governance Framework to enhance and support community access to the UFS and to engage key stakeholders to co-develop the

UFS. .

Documentation is updated as UFS governance or EPIC organization structure change. Evidence of participation is delivered and reported according to timelines determined on a task-order basis. A comprehensive report annually.

Online artifacts as part of the NOAA enterprise information technology (IT) system, MS Word Reports, and Power-Point presentations or additional documentation artifacts, where applicable, via email.

EPIC PM and

COR

Per 4.10, A public-facing community modeling infrastructure to allow for easy initialization, training, and visualization of data-driven NWP models based on ML and hybrid NWP models that combine the strengths of data-driven and physically based models using UFS/NOAA analyses and data products.

An architecture design and implementation plan following the proposal. Documentation is updated when new architecture components become available. Products and services are released and reported according to timelines determined on a task-order basis. A comprehensive report annually.

Online artifacts as part of the NOAA enterprise information technology (IT) system, MS Word Reports, and Power-Point presentations or additional documentation artifacts, where applicable, via email.

EPIC PM and

COR

52.204-10 – Reporting Executive Compensation and First-Tier Subcontract Awards

Executive compensation of the prime contractor: as a part of its annual registration requirement in SAM.gov First Tier Subcontract Awards: by the end of the month following the month of award of a first-tier subcontract valued at or above the threshold specified in FAR 4.1403(a) on the date of subcontract award

SAM.gov data entry SAM.gov data entry https://www.acquisition.gov/far/4.1403#FAR_4_1403

Copies Submit To

52.222-37 - Employment Reports on Veterans (Jun 2020)

No later than September 30 of each year.

VETS-4212 "Federal Contractor Veterans’ Employment Report" (see "VETS-4212 Federal Contractor Reporting" and "Filing Your VETS-4212 Report" at http://www.dol.gov/vets/v ets4212.htm

Department of Labor

NAM 1330-52.222-70 NOAA

Sexual Assault and Sexual Harassment Prevention and Response Policy

Evidence of annual training by name and date completed for each contractor employee shall be submitted no later than March 1st of each calendar year of contract performance.

Electronic/Contractor format acceptable

Contracting Officer and

COR

6. PERIOD OF PERFORMANCE

The projected contract period of performance spans five (5) years following the date of award. An option to extend services up to six months will be included in accordance with Federal Acquisition Regulation (FAR) 52.217-8 and evaluated as part of the award determination.

7. PLACE OF PERFORMANCE

The place of performance is to be at the direction and discretion of the contractor to fulfill the requirements of this contract in an acceptable manner.

7.1. Travel.

The Government anticipates that both local and non-local domestic travel will be required for certain positions during the term of this IDIQ to attend workshops and/or other meetings as authorized by the COR. The Contractor shall identify, forecast, and manage travel requests. All travel must be approved by the COR at least 10 business days in advance of the travel requirement. The Contractor shall maintain a travel log of all approved trips and include all estimated/vouchered costs, which may be reviewed by the COR as needed. During performance, only actual travel costs are reimbursed in accordance with Federal Acquisition Regulation (FAR) 31.205-46 and Federal Travel Regulations, subject to the COR’s approval. All travel requests being sent for COR approval shall include the following items:

● Purpose of the Travel;

● Tentative Travel Dates;

● Method of Travel (e.g., Private vehicle, Government Owned Vehicle, airplane, etc.) & cost for that method, and baggage expense, if applicable;

● Duration of Travel;

● Departure Location and Arrival Location;

● Per diem expenses per day;

● Lodging Expenses per day, if applicable;

● Lodging Taxes per day, if applicable;

● Distance from Office or Home to Airport or Other Transportation Facility, if applicable;

● Parking costs per day at the Airport, if applicable;

● Method of Transportation at Arrival Destination (e.g. – Subway/Metro, Bus, Taxi, Rental Car} with projected costs, if applicable); and

● Quantity and Cost of Fuel for a Rental Car, if applicable.

8. CONTRACT TYPE

This is a firm-fixed price IDIQ contract.

9. SECURITY REQUIREMENTS

The contractors shall take the following security considerations into account when performing the required services: there may be circumstances that require contractors to access NOAA facilities and computer resources, which would be specified in the PWS of each individual task order. A Federal sponsor and security clearance are required for Foreign National guests to access NOAA computer resources. Foreign Nationals accessing NOAA facilities under contracts, grants, and cooperative agreements require at least 30 days advance notice.

Low Risk - The suitability or risk level for this work has been determined Low Risk; performance under this IDIQ does not require administrator or elevated levels of access. As such, when recruiting Contractor personnel for performance under this IDIQ, the Contractor must prescreen all candidates to eliminate anyone who does not meet the eligibility requirements defined in contract clause CAR 1352.237-71 Security processing requirements-low risk contracts.

10. PERFORMANCE REQUIREMENT SUMMARY (PRS)

This PRS provides performance standards, methods of surveillance and incentives for deliverables listed in Section 6.

Tasks, deliverables and requirements listed in this PWS are subject to modifications, extensions and new scope of work to be determined through additional PWS’s specific to contract task-orders

Required Service

PWS

Section

Performance Standard Method(s) of Surveillance

Incentive

Develop an IDIQ PMP to fulfill the requirements of this contract.

4.1.2 ● In accordance with the Project Management Book of Knowledge (PMBOK), Seventh Edition,

● Includes the following component plans:

○ Project scope and management,

○ Schedule management,

○ Cost management,

○ Risk and opportunity management,

○ Configuration management,

○ Release management,

○ Quality management,

○ Communication plan for coordinating and reporting project management activities,

○ Staff management and training,

○ Innovations management,

○ Transition-In and -Out plan,

○ Stakeholder management,

○ Subcontract management,

○ OCI avoidance,

○ Procurement management.

● Features a consistent project management approach that prioritizes strategic goals,

● Addresses the need of agile software architecture and engineering project management principles,

● Addresses the needs of consistent community and

100% inspection by the PM, COR and EPIC Program Team

(EPT).

Past Performance Review

Required Service

PWS

Section

Performance Standard Method(s) of Surveillance

Incentive stakeholder engagement activities,

● Integrates priorities across development teams, business owners and stakeholders,

● Ensures a high-rate of objectives completion

(80% or above)

● Achieves high-levels of business value (80% or above)

● Document is reviewed for quality prior to submission,

● Follows format approved by the program office team,

● Outlines the approach for scheduling the submission of deliverables,

● Provides a consistent definition of done,

● Outlines how features will be prioritized.

Sustain foundational functions of EPIC as stated in section 3, and propose innovative solutions to improve Earth system model performance in collaboration with UFS Application and cross-cutting teams, with an ultimate goal toward their potential integration in next-generation UFS models and applications that can help execute EPIC’s vision and mission.

4.2 ● In collaboration with UFS Application and cross-cutting teams, with the ultimate goal toward their potential integration in next-generation UFS models and applications that can help achieve EPIC’s vision and mission.

● All foundational functions of EPIC are maintained or to be enhanced/replaced by innovative solutions as part of the proposal.

● Past performance of relevant services is clearly stated.

● The timeline of transition and development/integration is clearly identified.

● Innovative solutions shall add value to the UFS model computational performance and/or forecast skill in the context of potential integration to next generation models and application, or EPIC community modeling infrastructure,

● Provides a rationale as to why proposed approaches, services and products would contribute to EPIC’s vision and mission,

● Provides a timeline and level of effort associated with the development of approaches, products and services,

● Provides a vision of strategic outcomes associated with proposed additional approaches, products and services.

100% inspection by the PM, COR and EPIC Program Team

(EPT).

Past Performance Review

Best Value Offered to Government

Service

PWS

Section

Performance Standard Method(s) of Surveillance

Incentive

A cross-cutting community platform as the public-facing interface of the UFS, including the ECP, CI/CD pipelines to enable a public-facing testing and development environment.

4.3 ● Develop, deploy and continuously update the following products and services:

○ Include the front-end of the EPIC web presence known as ECP (epic.noaa.gov)

○ Include automated CI/CD pipelines hosted in NOAA Enterprise IT System with linkage to end-to-end public-facing testing and development environment that leverages on-premise and/or cloud high-performance computing and data dissemination resources provided by NOAA and partners,

○ A platform-agnostic unified community model containerized solution for Earth system modeling applications.

● Supports and enhances, improving accessibility and quality of:

○ A hierarchical testing framework (HTF) for Earth system modeling applications,

○ A hierarchical system development (HSD) framework for Earth system modeling applications.

● All infrastructure components are cloud-ready, platform-agnostic, and use infrastructure as code (IaC) where possible.

● All infrastructure components are compatible with, built and tested automatically on relevant platforms,

● Supports the development and operability of community Earth system models, applications and components,

● All systems use a unified software stack (e.g, spack-stack) as the package management system for the relevant model configurations, applications and components,

● Software stack management system is automatically built with specified and dependent libraries,

● Products and services are portable, inter-operable, and consistent with the containerization approach,

● Provides a flexible architecture to facilitate testing new system architectural innovation, 100% inspection by the PM, COR and EPT.

Past

Service

PWS

Section

Performance Standard Method(s) of Surveillance

Incentive

● Provides interfaces for integrating scientific innovations provided by the community towards improving Earth system models,

● Development activities are coordinated with community stakeholders, where applicable,

● Consistent with best practices developed in coordination/collaboration with key stakeholders and partner organizations,

● Activities are performed in coordination with and, wherever applicable, approved by relevant UFS working groups as part of the UFS community Governance,

● Services and products are aligned with NOAA best practices and Information Technology (IT) requirements,

● Services and products developed within the NOAA system boundaries are documented and aligned with the NIST SP800-53-r5 risk management framework.

● Consistent with NOAA R2O2R processes and procedures,

● Operational prototypes are consistent with NOAA Central Operations (NCO) High Performance Computing Implementation Standards,

● Infrastructure leverages NOAA enterprise IT systems, where applicable,

● All services and products are properly documented,

● Documentation includes full technical description, benchmarking results and relevant metrics across supported HPC resources.

Continuously support and enhance a code management framework for the

UFS.

4.4 ● Continuously assess and implement best software architecture and engineering practices to improve UFS software integration process, including but not limited to, incorporating new programming languages, enabling and enhancing an hierarchical testing framework, and enforcing code and architecture standards in coordination with UFS System Architecture and Infrastructure cross-cutting team.

● Coordinate and support code and repository management for UFS Weather Model and Applications, including integration of alternative options of dynamical cores, physics packages, 100% inspection by the PM, COR and EPT.

Past

Service

PWS

Section

Performance Standard Method(s) of Surveillance

Incentive and Earth system components.

● Continuously supports and enhances the seamless integration of UFS code and repository management environments,

● Continuously supports and enhances a framework for coordinating continuous integration teams,

● Continuously supports and enhances the design, development and deployment platform agnostic end-to-end workflows, leveraging the Unified Workflow (UWF) tools for Earth system modeling applications,

● Continuously supports and enhances the development, deployment and maintenance of data management frameworks for Earth system modeling applications,

● Continuously supports and enhances development of relevant UFS systems, applications, selected components, and cross-cutting infrastructure code repositories,

● Supports code management meetings, with proper documentation, reporting and stakeholder communication processes,

● Includes a ticketing system, developer’s resources (e.g., wiki pages) and user support to the code management tool,

● Code integration and porting is coordinated across supported platforms,

● Code integration is consistent with all relevant systems, applications and components,

● Continuously coordinates testing for software releases including system, dependent libraries and subcomponents,

● Ensures that applications connect to the individual component models after changes are made,

● Ensures code updates are made when/where necessary,

● Ensures computational performance is maintained,

● Benchmarking to ensure that new developments do not degrading accepted operational forecast skill and computational performance,

● Ensure requirements of the integrated EPIC code and repository management environment are met,

● Continuously maintains, updates and expands

Service

PWS

Section

Performance Standard Method(s) of Surveillance

Incentive unit, regression, cross-platform and integration tests as part of the HTF, investigating and correcting failures,

● Integrates new model functional tests into the regression test framework, verifying and validating test results and creating new baselines whenever needed,

● Optimizes the UFS regression test framework to improve the efficiency of regression testing,

● Ensures that as systems, applications and components get upgraded, changes do not break system across relevant platforms,

● New features introduced in systems, applications and component repositories meet implementation standards,

● New features are thoroughly tested using available research and operational workflows,

● Ensures latest results of regression tests are captured within relevant code repositories,

● Activities are coordinated with community stakeholders,

● Services and products are aligned with NOAA best practices and information Technology (IT) requirements,

● Provides the approach for developing plans for:

○ Development, coordination and support activities,

○ Definition of supported systems, applications, selected components, and cross-cutting infrastructure code repositories,

○ Supporting selected components, and cross-cutting infrastructure code repositories,

● All services and products are properly documented,

● Documentation incorporates both operational and research standards,

● Document dependencies of application repos on individual components on confluence as well as on GitHub repo wikis.

Service

PWS

Section

Performance Standard Method(s) of Surveillance

Incentive

Support the development of end-to-end UFS prototypes and continuous integration of UFS configurations, applications and selected components that facilitate and continuously improve the R2O2R process.

4.5 ● Supports the development and publication of system architecture and infrastructure documents,

● Supports the development and publication of draft software standards, policies, and processes to be approved by applicable UFS governance decision-making bodies

● Coordinates code reviews and integrates pull requests,

● Develops and deploys a minimal set of workflows end-to-end (WE2E) tests,

● Supports the integration of operational and research-oriented verification and validation packages as part of the HSD framework on EPIC supported platforms, including containerized environments if appropriate,

● Improves the workflow of selected components,

● Benchmarks the computational performance of end-to-end configurations of selected applications and components,

● Delivers optimized prototypes for the supported application as appropriate,

● Supports and updates UFS documentation that includes and integrates relevant systems, applications and components of relevant Earth system models, applications and components,

● Activities and documentation updates serve both operational and research communities,

● The prototypes are portable across all supported HPC platforms,

● Activities are integrated with the EPIC CI/CD pipeline framework,

● Supported applications and components are determined in coordination with UFS application and cross-cutting teams,

● Integrated systems are ready for operational hardening whenever possible,

● Accommodates O2R wherever possible,

● Facilitates R2O starting from NOAA Readiness

Level (RL) 2 needs,

● Coordinates with NOAA and WPO grantees to incorporate new innovations into UFS code and applications

● Updates relevant systems, applications and components documentation,

● Includes all components such as dynamical cores, pre-processing, physics, post processing, 100% inspection by the PM, COR and EPT.

Past

Service

PWS

Section

Performance Standard Method(s) of Surveillance

Incentive verification and validation, and workflow,

● Includes the necessary initial conditions (ICs), lateral boundary conditions (LBCs),

● Includes necessary dependent data,

● Includes an evaluation of workflow tools for its readiness to facilitate R2O2R,

● Includes an evaluation of the readiness of containers as a means to facilitate academic and industrial community distribution and engagement,

● Provides the approach for developing plans for:

○ Development, coordination, documentation and support activities,

○ Leveraging collaboration with community and key stakeholders,

● All services and products are made available and published for community and stakeholder access,

● All services and products are reviewed by key stakeholders.

Plan and support UFS releases and capability announcements for relevant UFS models, applications and components.

4.6 ● Addresses need of developing and publishing updated EPIC release process and charter documents for selected UFS WM configurations, applications and components.

● Supports release coordination efforts of selected systems, applications and components.

● Provides a framework for coordination and support of releases of platform agnostic prototypes for selected configuration, applications and components.

● Provides end-to-end support for release activities, including preparation, testing and deployment of relevant systems, applications and components,

● Consistent with relevant community release approaches and plans,,

● Release plans indicates clearly key stakeholders for each release,

● Release plans are made available and easily accessible to relevant key stakeholders,

● Code is on open-source code repositories and runs consistently in on-premise, cloud, and limited testing environments, where applicable,

● Containers for full community releases have all necessary infrastructure for ensuring systems, applications and components are properly built, 100% inspection by the PM, COR and EPT.

Past

Service

PWS

Section

Performance Standard Method(s) of Surveillance

Incentive

● Includes compilers that support community development work and transitions to operational applications,

● Address requirements for data dependencies,

● Provide user-friendly evaluation tools, work datasets and documentation.

● Activities are coordinated with relevant communities and key stakeholders,

● Provides the approach for developing activity plans and SOPs,

● All services and products are properly documented,

● All documents are reviewed by key stakeholders, release teams, and representative cross-cutting teams,

● Release plans take into account guidance and direction from UFS Governance decision-making bodies, Continuously coordinate, support and enhance an advanced user support framework, including the development and maintenance of user support products and services.

4.7 ● Ensures continuous development, coordination and management of user support forums,

● Ensures development and provision of technical and logistic support for community events,

● Ensures development and provision of technical support toward the development of tutorials and instructional materials.

● Supports the development of documentation for community Earth system models,

● Ensures the development, deployment and support of interfaces that allow users seamless access to code repositories,

● Ensures the development, deployment and support of an interface providing selected users access to public-facing computing environments for training, tutorials, and hackathons.

● Supports all relevant model configurations, applications and components,

● Ensure consistency with relevant partners, communities, and key stakeholders,

● Activities are planned in collaboration with relevant communities and key stakeholders,

● Provides a list of subject matter experts (SME) for the various systems, components and applications,

● Creates a database of support teams within

100% inspection by the PM, COR and EPT.

Past

Service

PWS

Section

Performance Standard Method(s) of Surveillance

Incentive contract staff with subject matter expertise to provide answers reducing dependency on SMEs,

● All elements of the framework are integrated to and available through ECP and resources, including websites, wiki pages, and social media platforms,

● All elements of the framework are accessible to users of all backgrounds and levels of expertise,

● All elements of the framework are reviewed by technical and non-technical teams to ensure quality and accessibility,

● All elements of the framework follow format and standards developed in collaboration with relevant communities and key stakeholders,

● All elements of the framework address needs of full functionality and readiness for supported cloud and/or on-premise platforms,

● Provides the approach for developing activity plans and SOPs,

● All services and products are properly documented; to include all configuration management plan documentation.

Continuously support and enhance a community engagement strategy, including community engagement products and services.

4.8 ● Supports web-based artifacts, including portals, websites, wiki pages, other online resources and user interfaces, and social media platforms,

● Continuously rolls out and supports schedules for maintenance, design and content updates of web-based artifacts,

● Supports all the capabilities of web-based artifacts,

● Provides a standard operating procedure for planning and hosting meetings and workshops,

● Supports content development for web-based artifacts including social media needs,

● Provides a continuously updated communications strategy document.

● Continuously supports, updates and expands web-based artifacts to address community and key stakeholder queries, such as frequently-asked questions pages,

● Provides a framework for showcasing program outcomes at relevant conferences and workshops,

● Provides a plan and framework for developing and publishing community engagement artifacts, such as newsletters, to support and promote community and stakeholder engagement, 100% inspection by the PM, COR and EPT.

Past

Service

PWS

Section

Performance Standard Method(s) of Surveillance

Incentive

● Engagement strategies, products and services are integrated to the ECP and to general community and user support framework,

● Expands the number of active UFS community members through engagement with other modeling communities across the United States.

● All products and services, where applicable:

○ Follow responsive design and 508 compliance,

○ Follow human-centered design best practices,

○ Are Search-Engine Optimized (SEO),

○ Follow approved style guidelines,

● Provides the approach for developing activity plans and SOPs,

● All services and products are properly documented.

● Reports should include documentation of requested capabilities from community partners.

● Documents should include strategies for how to foster collaboration with modeling communities across the United States.

Support and participate in a community governance framework to enhance and support community access to the UFS and to engage key stakeholders to co-develop the

UFS.

4.9 ● Participates and leads teams as part of the UFS governance, and provides recommendations for UFS community standards, policies, and processes to applicable UFS governance decision-making bodies.

● Ensures the development and deployment of a continuous key stakeholder engagement framework,

● Strategy is consistent with the WPO and EPIC program Strategic Plans, and NOAA Modeling Strategy,

● Creates opportunities for incorporating stakeholder feedback into relevant work items,

● Continuously identifies key stakeholder groups and individuals,

● Continuously assesses and integrates key stakeholder needs and priorities to EPIC features and use story backlogs,

● Includes a key stakeholder database that can be accessed by program members,

● Conducts quarterly stakeholder engagement analysis activity to identify key stakeholders and

100% inspection by the PM, COR and EPT.

Past

Service

PWS

Section

Performance Standard Method(s) of…