Draft_PWS_-_Sources_Sought.docx

DRAFT Performance-Based Work Statement (PWS) for the

OPERATION OF A CENTER FOR COMMUNITY AIR QUALITY MODELING AND ANALYSIS (CMAS)

INTRODUCTION

The United States Environmental Protection Agency (EPA) has collaborated in the establishment of a non-profit center for the air quality modeling community, the Community Modeling and Analysis System (CMAS). The CMAS focuses on support of regional air quality modeling products used by EPA, primarily the Community Multi-scale Air Quality (CMAQ) model and the Sparse Matrix Operator Kernel Emission (SMOKE) model. The center was funded initially for four years (November 1, 2001 - October 31, 2005) under a cooperative research agreement between EPA and the Microelectronics Center of North Carolina (MCNC) North Carolina Supercomputing Center, later novated to the University of North Carolina, Carolina Environmental Program (Agreement CR82950902). Since June 1, 2005, EPA has participated in supporting the continuation of CMAS through two competitive contracts awarded to the University of North Carolina, Institute of the Environment (previously the Carolina Environmental Program). The current contract is anticipated to expire on May 31, 2016 (Contract EP-W-09-023). The EPA, through the Office of Research and Development (ORD) plans to continue support of key functions of CMAS through this procurement action. The CMAS center is a functional entity providing expertise in support of open code, public-domain air quality model products, rather than a physical property.[footnoteRef:1] This award will support key aspects of the CMAS to provide user assistance, periodic public model code releases and maintenance, model updates or extensions (enhancements), model documentation, user guides and key reference data sets, and technical workshops focused on specific developments or review of specific aspects of air quality model science support. Because CMAS supports regulatory users of the specified air quality modeling products in addition to EPA (e.g., states and regional entities), OAR is cooperating with ORD in sponsoring the technical support to be provided under this contract. Comment by Benjey, William: “MCNC” is no longer used as an acronym – it is used as is without standing for anything. [1: http://www.cmascenter.org]

EPA'S MISSION AND RESEARCH AND DEVELOPMENT STRATEGY

EPA's National Exposure Research Laboratory (NERL) of ORD has divisions located in Research Triangle Park, NC; Cincinnati, OH; Athens, GA; and Las Vegas, NV which provide scientific understanding, information and assessment tools to reduce and quantify the uncertainty in the Agency's exposure and risk assessments. Mathematical simulation models are used for examining strategies for clean air policies, for evaluating scientific findings and their impact on policy and for specific, defined regulatory requirements. This research, especially the modeling aspects, is endorsed and used by the regulatory offices in EPA, as well as by states, by industry and consultants, other Federal agencies, and the international research community.

BACKGROUND

Environmental problems continue to grow in complexity and scope. Local management solutions alone can no longer address many of today's problems. Thus, regional efforts are needed in the context of global scenarios. Accordingly, the models to assess these problems and evaluate alternative solutions are increasing in complexity. Consequently, the concept of an integrated air quality modeling and analysis system was formulated, and implementation was begun using the current structure of CMAS. At this time, the EPA regional air quality modeling software system consists principally of CMAQ and SMOKE for urban to regional to hemispherical scale modeling, with related important ancillary tools including the Meteorology Chemistry Interface Processor (MCIP), Visualization Environment for Rich Data Interpretation (VERDI) for visual analysis of gridded data, the Spatial Allocator for geographic gridding of input data, the Atmospheric Model Evaluation Tool (AMET) for statistical analysis of meteorological and air quality model outputs, and the Environmental Benefits Mapping and Analysis Program (BenMAP) for examination of the benefits of potential regulatory requirements. It is anticipated that the CMAQ system will continue to evolve during the term of the proposed contract, and at some point may no longer be called CMAQ although the development of the science and modeling simulation software is a continuous process. EPA also develops local-scale dispersion models released through CMAS, currently the R-Line model. The dispersion models are also supported through CMAS with periodic releases of updates and documentation. This contract specifies particular functions to be accomplished in support of the modeling system; however, separately funded efforts by the center are encouraged to leverage the scientific and technology advancements of other Federal agencies, industry and nonprofit institutions, thereby evolving toward a more unified comprehensive approach to multi-discipline environmental modeling.

The primary underlying goals for CMAS are to assist EPA in improving 1) the environmental management community's ability to evaluate the impact of air quality management practices for multiple pollutants at multiple spatial and temporal scales in the atmosphere, and interactions with other components of the ecosystem and human health; and 2) the scientist's ability to better probe, understand, and simulate chemical and physical interactions in the atmosphere and other ecosystem components. These two goals represent the needs of two different modeling groups, the air quality management user and the science user and developer; i.e., those involved in air quality management and those involved in air quality research. Their needs are different, but there are significant advantages in using the same problem-solving environment for both groups. Thus, CMAS is intended to serve as a community foundation resource for the widespread application of air quality models and for their continued scientific advancement.

To make these complex and advanced models more available to everyday users, especially those in state and local governments, industry, and academia, a continued concentrated effort is needed to transfer the technical capabilities in advanced modeling components. The magnitude of this need is borne out in the steadily increasing use of the current CMAS center. For example, model-related software downloads from the CMAS web site increased from over 3,700 in 2005 to about 9,800 during 2014. Participants at the annual CMAS workshop increased from 205 in 2005 to 250 in 2013 representing 18 countries. In addition to model support, it is acknowledged that new research support is needed. However, research is not the central element of this contract, although improvement of the science-based model code is a component. This contract is directed primarily towards maintaining and improving the air quality modeling community’s ability to use and improve EPA air quality modeling products by supporting and improving both new and existing products.

CMAS PROGRAM OBJECTIVES AND TASKS

The primary goal of CMAS is to support the air quality modeling community in their mutual efforts to develop and use sound regional, state, and local policies to protect the health of humans and ecosystems, and to advance the science leading to more reliable modeling tools for community environmental problem solving. This SOW scope supports the CMAS goal by providing modeling support and information sharing to the user community and air quality modeling science support to EPA, as specified by the following tasks. The contractor shall perform work under the seven tasks of CMAS, as defined by performance-based work assignments issued by the Contracting Officer. The general performance criteria are specified at the end of this scope of work. Additional criteria may be specified with each work assignment (WA), as appropriate depending upon the specific nature of the work to be accomplished. Failure to meet performance standards may result in reduction of WA invoice amounts by as much as ten percent. Comment by Jacob, Sini: Comment by Jacob, Sini: Are you anticipating a work assignment contract type? Those are generally atypical for IT contracts……..esp if issued under Schedules, GWACs. I will need a written description from you (separate from this SOW) in order to substantiate a WA-contract type – so what you mentioned on the phone put into words. Comment by Benjey, William: Not an IT support contract in the sense of maintaining equipment or a call center, which would use tracking tickets. Only one sub-task responds to user queries. The balance of the work is involved research support of computer simulations requiring response to a scope of work more specific than can be defined in the contract scope. I’ll elaborate separately.

Task 1. User Support and Training - The contractor shall provide the following training and support to all users:

a. Web-site based internet support for users of CMAQ, SMOKE, MCIP, Spatial Allocator, VERDI, AMET, BenMAP and R-Line. This does not preclude support of air-quality-related modified or additional models or modeling tools specified by work assignment. The web site shall be available at least 90 percent of the time, 24 hours each day on a monthly basis as documented by summarized operation logs in the monthly reports, and provide available model documentation (user guides and science), and downloads of model codes and documentation. The contractor shall track downloads by registration for purposes of providing future assistance to users. Comment by Jacob, Sini: Do you mean just website operations/maintenance? Comment by Benjey, William: No, the web-site provides support to the users of the models. I added a word to clarify this. Of course, the contractor also has to maintain the website.

b. Software support shall maintain the above specified software with respect to operating system and bug fixes. This means the contractor shall be capable of diagnosing operational problems of the models, whether the problems are based on limited understanding by the user, or malfunctions or limitations in the model codes. The typical level of effort range for this work may be from 1000 to 1400 hours per year. Computer operating system-related activities such as porting to additional computer operating systems other than supported by EPA and system enhancements, are not included in the basic effort of the contract. Comment by Jacob, Sini: Any numbers from current contract available as to how many enhancements, how frequent? Comment by Benjey, William: We have averaged 7 work assignments per option year. I’ve inserted a range of labor hours for this subtasks based on our experience. The low end of the ranges reflects current conditions. The high end reflects anticipated greater need in the next few years.

c. An e-mail based query system for operating questions and program bugs. The system shall archive queries and responses for the on-line reference of all users. In order to successfully address queries and program issues, the contractor shall be able to describe the science basis and structure of the model codes. The contractor shall make an initial response to and attempt to resolve user queries and bug reports within three days of the report posting, whether the issues are based in user understanding or with problems in the model code. The typical level of effort range for this work may range from 700 to 1100 hours per year. Current e-mail query volume via the list-server system is approximately 250 queries per month. The query system shall be available at least 90 percent of the time, 24 hours each day on a monthly basis.

d. Help desk operation (back up for web based support or follow-on for more complex problems). Again, the contractor must address problems based either in limits of user understanding or in the model codes. Typically only one or two queries per month require intensive support at this level. Generally, issues can be addressed by e-mail.

e. Promote and facilitate collaboration and information sharing among air quality model developers and users in a variety of organizational settings (universities, states, regions, Federal agencies, national laboratories, industry, consulting). This shall include, at a minimum, using Internet listservers to communicate with the user community and an on-line change request tracking system. The contractor may develop and conduct training with regard to the EPA modeling tools for non-EPA entities at their discretion, separate and apart from this contract.

f. Sharing of selected model output data sets and other auxiliary information relevant to users of CMAQ and related software tools. The contractor shall, as specified by work assignment, provide selected air quality modeling output data sets and auxiliary information that are of common interest to the community of air quality modelers. These data sets shall be documented and made available by means of an anonymous ftp. The contractor shall analyze the data sets as may be required by specific work assignments.

Task 2. Public Release of Models and Updates

a. The contractor shall maintain an on-line archive for model users. The archive shall maintain model code through a suitable software control system, such as the Concurrent Version System (CVS) or GIT (not an acronym). The contractor shall coordinate with EPA model developers in documenting model changes for the public releases of EPA developed models. Updated model code for CMAQ and other EPA air quality modeling products, and associated release notes provided by EPA, shall be released once each year on the average.

b. As directed by work assignment, the contractor shall also plan and release up to two other public releases each year. These releases may be additional CMAQ updates or be code updates for SMOKE, MCIP, PAVE, Spatial Allocator, VERDI, or AMET or other modeling tools. All releases shall be by internet.

Task 3. Model Extensions

In order to foster community participation in model improvements, the contractor shall receive, evaluate, and release improvements contributed to the supported models by other collaborators in accordance with unit and system testing protocol guidelines established by the contractor and approved by EPA. Consequently, the contractor shall have specified procedures, knowledge and ability to develop and update software unit and system testing protocols, knowledge of the development and air quality model evaluation procedures, and demonstrated specific knowledge of the underlying science and code of the above reference suite of air quality models and modeling tools. Extension and application of CMAS supported air quality modeling tools to multi-media environmental problems (for example, fluxes to and from land and water surfaces, effects of land cover on fluxes, and the role of nitrogen compounds in the environment) may be reflected in model extensions.

All model source code released through CMAS is to be public domain, open-source code. Intellectual rights and contributions of all contributors shall be acknowledged in the code and documentation. EPA source code for science and non-proprietary components of the framework code shall be available to all users with no charge if internet delivery is used. EPA intends that new code provided by EPA or other members of the air quality modeling community to CMAS be open to all. The contractor shall specify their protocol for respecting the intellectual property rights of contributors.

Task 4. Model Testing, Review, and Release Procedures

The contractor shall establish procedures to accomplish the objectives above including: model testing, review and distribution policies and procedures for official software releases; review criteria and procedures for incorporation of new science or new models into the system from any contributor; acknowledgment of significant intellectual contributions, and exchange and resource linkages among the community of air quality modelers to facilitate technology transfer and training. This work requires the contractor to have knowledge and experience in establishing and maintaining air quality model testing criteria (meteorology and emission scenarios and statistical evaluation), reflecting knowledge of the science and coding basis of the models. Prior CMAS protocols shall be drawn upon in establishing these procedures by EPA. These versions shall be maintained in a public archive available through the Internet, and documented separately from the model versions released by CMAS for EPA.

Task 5. Model Code Development and Modification

The contractor shall develop or modify software code of the above-referenced EPA air quality modeling products as specified by a statement of needs requirements from EPA. Work shall include development or modification of model code to implement new or revised air quality modeling-related concept(s), testing of the code in accordance with CMAS procedures, code documentation, and unit and system tests of the model code. In order to accomplish this work, the contractor shall have detailed knowledge and prior experience with the scientific basis and software code of the above specified set of air quality modeling tools. It is anticipated that approximately 6 work assignments per year will require this type of work.

Code development and modification may be required in specific chemical or physical processes of the CMAQ model or in its numerical solution routines, or data processors including MCIP. The contractor shall use EPA model coding standards as specified by the COR for all model code and development. Development and testing of refined features for the Weather and Research Forecasting (WRF) or other meteorological models used for producing input meteorological fields to CMAQ may be required. The contractor may be required to develop specific code to assist in the application of CMAQ or related modeling code to multi-media applications, for example interaction with land and water surfaces and resulting issues. The contractor shall have the demonstrated existing capabilities in the disciplines needed to accomplish assigned work in this task for the separate models, as well as potential work in integrated meteorological-air quality models.

Task 6. Model Documentation

The contractor shall prepare new or revised documentation of EPA air quality modeling products. Changes in air quality modeling components may require new or updated procedural (User Manual) documentation and documentation of code and file structure. All model releases shall be accompanied by revised documentation and read-me files. In order to accomplish this task, the contractor shall have specific experience and knowledge with the scientific basis and software code of the above specified set of air quality modeling tools.

Task 7. Specialty Air Quality Modeling Workshops

The contractor shall conduct small specialty workshops related to the EPA air quality modeling tools or issues impacting them. For planning purposes, assume one specialty workshop each year, to be held in the Research Triangle Park, NC, area with no more than 40 participants. The contractor’s scientific staff shall participate in the workshops, as appropriate to the subject of the specialty conferences. As directed by work assignment, the contractor shall be responsible for making logistical arrangements for the workshops, including making arrangements for conference space and presentation equipment, assisting invited participants with preparation of presentation information, preparation of workshop materials, agenda, and handouts; registration; and facilitating the workshop.

Deliverables

Deliverables documenting the user support function of the contract are specified in Table 1.

Because of the research nature of some of the work to be performed, specific deliverables and due dates for that work cannot be specified far in advance in this scope of work. Specific deliverables will be specified in detailed work assignments issued to the contractor by the CO. In general, deliverables will consist of model codes and tests, associated documentation, and related products, such as QC products, analyses and graphic outputs. Other deliverables associated with specialized model applications as noted above in the statement of work will be listed in written work assignments issued by the CO. The contractor shall develop or modify specified model code, prepare associated data sets, evaluate the new or modified model code, perform standard quality control (QC) checks, execute air quality simulation models as part of model code evaluation, and perform requested analysis of model results for all applications required during the period of performance. The contractor shall begin work only after the issuance of a work assignment by the CO. Work may begin upon receipt of the work assignment, subject to submission of a work plan within 15 calendar days, including the technical approach, schedule, staffing plan, and hour and cost estimate by the Contractor. Work will proceed beyond the initial 15 calendar day period upon written approval of the work plan by the CO. Unless otherwise specified, any deliverables that are produced for the work described above shall use the following software and conventions or the hardware and software specified in Appendix A:

Acceptance Criteria

Deliverables shall be submitted to the COR with a copy to the CO for review. The COR or WACOR will use the quality and timeliness performance criteria described in Appendix B to judge the deliverables. Evaluation of the quality of the deliverables will include consideration of: clarity and successful operation of model code, correctness of input and output data, completeness of documentation including in-line code documentation, new or updated user manuals and readme files, adherence to approved QA procedures in the contract Quality Assurance Project Plan (QAPP), clearness and quality of graphics, adequacy of model version and analytical procedures used. The COR or WACOR has the authority to accept or reject the deliverables or request modifications by technical direction. Upon completion of the requested modifications and within 30 days the COR or WACOR will review the deliverables and provide notification of acceptance. Upon receipt of approval the contractor shall provide the number of copies of the final version as specified in the work request to the COR.

All deliverables must meet other requirements set forth in the contract and work assignments.

Deliverables shall be submitted in accordance with the contractor's approved estimate of completion dates provided to the COR in the approved work plan.

Documentation and Deliverable Management

Documentation includes written material and computer model code and model run output that specifies the contents of new or modified model code, results of model test runs, summaries, analyses, quality control documentation, periodic reports such as monthly reports, and non-recurring reports required by work requests.

All written documentation shall be submitted as scheduled in Word format (for written reports, documentation and analyses) unless otherwise specified by work request.

A. Deliverables:

1. Monthly reports, by work assignment, including technical progress, problems encountered and solution proposed, and a breakout of labor hours and costs, as required by the contract.

2. A contract Quality Assurance Project Plan (QAPP) applicable for the duration of the contract, to be submitted within 30 days of contract initiation. The QAPP shall reflect EPA guidance for contractors and modeling. The QAPP shall include the provisions of EPA QA guidance publications QA G-5 (Guidance tor Quality Assurance Project Plans) and QA G-5M (Guidance for Quality Assurance Project Plans for Modeling). The QAPP shall define Standard Operating Procedures (SOPs) to be applied to model development and evaluation, as well as execution of emission, meteorology and air quality modeling. A QA Coordinator and minimum Standard Operating Procedures (SOPs) shall be specified by the Contractor in coordination with EPA during preparation of the QAPP. Work under the contract shall not begin prior to the approval of the contract QAPP.

3. Any revisions to the approved contract Quality Assurance Project Plan (QAPP) required by the CO, with the deliverable date specified in writing.

4. Non recurring draft and final report(s) and/or computer model code and documentation required by work assignments. Each report shall include a specific section addressing adherence to approved QAPP procedures. The COR or WACOR will provide comments (if any) within 30 calendar days of receipt of the draft material. The contractor shall provide a final version of the report(s) and/or computer model code within 10 days of receipt of comments, unless otherwise specified in writing.

5. Work Plans to be submitted within 15 days of initiation of each work assignment.

1 Alt COR

1 WACOR

1 CO

1Alt COR

1 CO

1 Alt. COR

1 CO

1ALT COR

1 WACOR

1CO

1 ALT COR

1WACOR

1 Alt. COR

1 WACOR

1 CO

Acceptance Criteria:

1. Monthly Report: The timeliness of monthly reports will be judged on the basis of the performance criteria set forth in Appendix B, Paragraphs B and C. Beyond the information required in the contract and work assignment the contractor shall include: descriptions of: major activities completed, major activities in progress, problems, delays, actions required by EPA, future activities, and list of server hardware and software configurations. The report shall separate Windows related information from Unix related information. The monthly report shall include, in addition to normal task order status and activities, all activities of the Program Manager, and other key personnel in support of this contract The cost portion of the Monthly Report shall include a detailed breakdown of the number and type of labor employed at a work request level.

2. QAPP: All SOPs shall demonstrate quality assurance and control aspects of each task in accordance with the applicable approved QAPP.

3. Work Assignment Plans: Work Plans for each work assignment shall be prepared and submitted to the CO and COR within 15 days of work assignment initiation. The Work Plans will be judged on the basis of the completeness of technical approach, adherence to the scope of work provided to the contractor at work assignment initiation, and thoroughness and accuracy of work plan schedules, and hour and cost estimates.

4. Work Assignment products: Completeness, clarity, organization, consistency, technical accuracy, adherence to QAPP procedures, timeliness, and grammatical correctness of all written materials and reports.

Training and Travel:

The contractor is expected to provide staff whom are already fully trained in the subject matter needed to successfully perform work under the contract. However, the contractor may be asked to send one or more persons to specialized courses, workshops, and/or meetings that either are unique to EPA's work or involve specialized modeling applications described in the scope. All travel must be within the scope of an approved work assignment in advance of travel, consistent with current U.S. Government travel rules and cost limitations and approved in advance in writing by the CO. Occasional technical support for preparation of technical products for conferences or papers, or travel to conferences may be required. For the purposes of planning, the contractor shall assume that, during each option year, one person each shall travel to two conferences of four days duration, one in Los Angeles and one in Washington, DC.

Travel reports shall be prepared that include the purpose and location of the travel, attendees, meeting agenda, significant findings, and any plans or events resulting from the meeting. Travel reports shall be submitted with the next monthly report attached to the relevant work assignment section.

Travel reports will be judged in accordance with the performance criteria described in Appendix B, paragraphs B and C, with respect to completeness, clarity, organization, technical accuracy, timeliness, and grammatical correctness.

Financial Reporting/Cost Tracking

Financial reporting and cost tracking shall include a mechanism for providing costs and estimates at the work assignment level. This will require establishing separate project codes as specified in the individual work request or technical direction document. All costs associated with specific project codes shall be reported in the monthly reports. Monthly work assignment time and cost reports shall be organized to show labor hours and costs by work request for the month and cumulatively for the option year.

Monthly financial reporting shall be judged in accordance with the performance criteria described in Appendix B, paragraph C, with respect to completeness, clarity, organization, consistency, computational accuracy, timeliness, technical accuracy, and grammatical correctness.

Please indicate which Application Life Cycle stages will be covered based on the systems/tools that will be supported under this contract:

___Definition/Initiation (New Project) _X_ Development/Acquisition _X_ Implementation (Deployment) _X Operation/Maintenance ___Termination (Disposal)

Appendix A

Required Software Capabilities

Contractor personnel performing the above air quality-related work shall have knowledge and operating experience with the following software models, tools, and software development languages:

NCAR Graphics – Graphics analysis package support by NCAR

Appendix B

Performance Criteria

The following metrics will be used in evaluating the Contractor’s performance for the work described in the paragraphs above. These metrics are designed to be applicable in the uncertainty inherent in supporting research and development science modeling.

A. Quality Comment by Jacob, Sini: I may convert this into a table format for easier use.

1. Documentation of adherence to the approved contract QAPP and associated SOPs in deliverable products. (Deduction for failure to timely submit an approvable QAPP – 10% of invoice; deduction for failure to document adherence to QA procedures in the approved QAPP – 5% of invoice).

2. The QAPP provisions shall be followed and documented for each work assignment by the Contractor’s QA coordinator. A QA report (contents defined in the QAPP) reflecting the requirements of the QAPP and SOPs shall be submitted for each work request within 30 work days after submission of the final work assignment deliverable. (Deduction for each late work request QA report – 2 % of invoice).

3. Quality in research modeling support may sometimes reflect events or items that are not readily foreseeable. Diligence by the Contractor in detecting problems outside of prescribed QA procedures is important. Consequently, Contractor-initiated and documented solutions to significant QA problems outside of the required QA procedures will be rewarded by reducing any percentage penalties on an invoice by up to 2% per documented event for the relevant invoice period. EPA retains the right to determine what constitutes a significant QA problem.

B. Timeliness

1. The initial Project QAPP shall be submitted within 45 calendar days of contract award. (Deduction for late QAPP submission – 5% of invoice).

2. Work request QA reports shall be submitted within 15 work days after completion of submission of the final work request product (Deduction for late submission – 2% of invoice for each late work request QA report).

3. Products for each work assignment shall be submitted according to the schedule in the work assignment (allowing for due date modifications agreed to and/or initiated in writing by the CL). (Deduction for late deliverable without prior agreement on changed due dates – 3 % of invoice for each late work request final submission).

C. Project Cost Management

1. Each work request shall be completed within estimates prepared by Contractor and approved by the CO. Estimates are subject to revision for cause if agreed to in advance by the CO. (Deduction for each unapproved work request cost overrun – 2 % of invoice).