Draft_PWS.pdf

Performance Based Work Statement (PWS)

Draft

OPERATION OF A CENTER FOR COMMUNITY AIR QUALITY MODELING AND

ANALYSIS (CMAS)

INTRODUCTION

The United States Environmental Protection Agency (EPA) has collaborated in the establishment of a non-profit center for the air quality modeling community, the Community

Modeling and Analysis System (CMAS). The CMAS focuses on support of regional air quality modeling products used by EPA, primarily the Community Multi-scale Air Quality (CMAQ) model and the Sparse Matrix Operator Kernel Emission (SMOKE) model. The center was funded initially for four years (November 1, 2001 - October 31, 2005) under a cooperative research agreement between EPA and the Microelectronics Center of North Carolina (MCNC)

North Carolina Supercomputing Center, later novated to the University of North Carolina, Carolina Environmental Program (Agreement CR82950902). Since June 1, 2005, EPA has participated in supporting the continuation of CMAS through two competitive contracts awarded to the University of North Carolina, Institute of the Environment (previously the Carolina

Environmental Program). The current contract is anticipated to expire on May 31, 2016

(Contract EP-W-09-023). The EPA, through the Office of Research and Development (ORD) plans to continue support of key functions of CMAS through this procurement action. The

CMAS center is a functional entity providing expertise in support of open code, public-domain air quality model products, rather than a physical property.1 This award will support key aspects of the CMAS to provide user assistance, periodic public model code releases and maintenance, model updates or extensions (enhancements), model documentation, user guides and key reference data sets, and technical workshops focused on specific developments or review of specific aspects of air quality model science support. Because CMAS supports regulatory users of the specified air quality modeling products in addition to EPA (e.g., states and regional entities), OAR is cooperating with ORD in sponsoring the technical support to be provided under this contract. This work has been determined to be Information Technology (IT) related by EPA.

Consequently, it must address IT security and track resources expended by modeling life cycle component (the components for this proposed contract are: Planning and Development for tasks

3-7; and operation and maintenance for tasks 1 and 2).

EPA'S MISSION AND RESEARCH AND DEVELOPMENT STRATEGY

EPA's National Exposure Research Laboratory (NERL) of ORD has divisions located in

Research Triangle Park, NC; Cincinnati, OH; Athens, GA; and Las Vegas, NV which provide scientific understanding, information and assessment tools to reduce and quantify the uncertainty in the Agency's exposure and risk assessments. Mathematical simulation models are used for examining strategies for clean air policies, for evaluating scientific findings and their impact on policy and for specific, defined regulatory requirements. This research, especially the modeling aspects, is endorsed and used by the regulatory offices in EPA, as well as by states, by industry

1 http://www.cmascenter.org and consultants, other Federal agencies, and the international research community.

The EPA's air quality modeling and analysis research is centered in Research Triangle

Park, NC. Beginning in the early 1990's, the Atmospheric Modeling and Analysis Division

(AMAD) designed and developed a third generation air quality modeling system, including

CMAQ and SMOKE as principal components1. The modeling system components have progressively been improved with updates released to the model users on a nearly annual basis.

New local-scale dispersion models, such as R-Line, have also been developed and released through CMAS in recent years. SMOKE is now principally supported through the OAQPS, although it remains a key part of the CMAQ modeling system. The OAQPS has also been working with AMAD to test and use these models, which are identified in this Statement of

Work as the basis for community modeling and analysis.

BACKGROUND

Environmental problems continue to grow in complexity and scope. Local management solutions alone can no longer address many of today's problems. Thus, regional efforts are needed in the context of global scenarios. Accordingly, the models to assess these problems and evaluate alternative solutions are increasing in complexity. Consequently, the concept of an integrated air quality modeling and analysis system was formulated, and implementation was begun using the current structure of CMAS. At this time, the EPA regional air quality modeling software system consists principally of CMAQ and SMOKE for urban to regional to hemispherical scale modeling, with related important ancillary tools including the Meteorology

Chemistry Interface Processor (MCIP), Visualization Environment for Rich Data Interpretation

(VERDI) for visual analysis of gridded data, the Spatial Allocator for geographic gridding of input data, the Atmospheric Model Evaluation Tool (AMET) for statistical analysis of meteorological and air quality model outputs, and the Environmental Benefits Mapping and

Analysis Program (BenMAP) for examination of the benefits of potential regulatory requirements. It is anticipated that the CMAQ system will continue to evolve during the term of the proposed contract, and at some point may no longer be called CMAQ although the development of the science and modeling simulation software is a continuous process. EPA also develops local-scale dispersion models released through CMAS, currently the R-Line model.

The dispersion models are also supported through CMAS with periodic releases of updates and documentation. This contract specifies particular functions to be accomplished in support of the modeling system; however, separately funded efforts by the center are encouraged to leverage the scientific and technology advancements of other Federal agencies, industry and nonprofit institutions, thereby evolving toward a more unified comprehensive approach to multi-discipline environmental modeling.

The primary underlying goals for CMAS are to assist EPA in improving 1) the environmental management community's ability to evaluate the impact of air quality management practices for multiple pollutants at multiple spatial and temporal scales in the

1 http://www.epa.gov/asmdnerl/ atmosphere, and interactions with other components of the ecosystem and human health; and 2) the scientist's ability to better probe, understand, and simulate chemical and physical interactions in the atmosphere and other ecosystem components. These two goals represent the needs of two different modeling groups, the air quality management user and the science user and developer;

i.e., those involved in air quality management and those involved in air quality research. Their needs are different, but there are significant advantages in using the same problem-solving environment for both groups. Thus, CMAS is intended to serve as a community foundation resource for the widespread application of air quality models and for their continued scientific advancement.

To make these complex and advanced models more available to everyday users, especially those in state and local governments, industry, and academia, a continued concentrated effort is needed to transfer the technical capabilities in advanced modeling components. The magnitude of this need is borne out in the steadily increasing use of the current

CMAS center. For example, model-related software downloads from the CMAS web site increased from over 3,700 in 2005 to about 9,800 during 2014. Participants at the annual CMAS workshop increased from 205 in 2005 to 250 in 2013 representing 18 countries. In addition to model support, it is acknowledged that new research support is needed. However, research is not the central element of this contract, although improvement of the science-based model code is a component. This contract is directed primarily towards maintaining and improving the air quality modeling community’s ability to use and improve EPA air quality modeling products by supporting and improving both new and existing products.

CMAS PROGRAM OBJECTIVES AND TASKS

The primary goal of CMAS is to support the air quality modeling community in their mutual efforts to develop and use sound regional, state, and local policies to protect the health of humans and ecosystems, and to advance the science leading to more reliable modeling tools for community environmental problem solving. This PWS scope supports the CMAS goal by providing modeling support and information sharing to the user community and air quality modeling science support to EPA, as specified by the following tasks. The contractor shall perform work under the seven tasks of CMAS, as defined by performance-based task orders issued by the Contracting Officer. The general performance criteria are specified at the end of this scope of work. Additional criteria may be specified with each task order (TO), as appropriate depending upon the specific nature of the work to be accomplished. Failure to meet performance standards may result in reduction of TO invoice amounts by as much as ten percent.

Task 1. User Support and Training - The contractor shall provide the following training and support to all users:

a. Web-site based internet support for users of CMAQ, SMOKE, MCIP, Spatial

Allocator, VERDI, AMET, BenMAP and R-Line. This does not preclude support of air-quality-related modified or additional models or modeling tools specified by task order. The web site shall be available at least 90 percent of the time, 24 hours each day on a monthly basis as documented by summarized operation logs in the monthly reports, and provide available model documentation (user guides and science), and downloads of model codes and documentation. The contractor shall track downloads by registration for purposes of providing future assistance to users.

b. Software support shall maintain the above specified software with respect to operating system and bug fixes. This means the contractor shall be capable of diagnosing operational problems of the models, whether the problems are based on limited understanding by the user, or malfunctions or limitations in the model codes. The typical level of effort range for this work may be from 1000 to 1400 hours per year.

Computer operating system-related activities such as porting to additional computer operating systems other than supported by EPA and system enhancements, are not included in the basic effort of the contract.

c. An e-mail based query system for operating questions and program bugs. The system shall archive queries and responses for the on-line reference of all users. In order to successfully address queries and program issues, the contractor shall be able to describe the science basis and structure of the model codes. The contractor shall make an initial response to and attempt to resolve user queries and bug reports within three days of the report posting, whether the issues are based in user understanding or with problems in the model code. The typical level of effort range for this work may range from 700 to

1100 hours per year. Current e-mail query volume via the list-server system is approximately 250 queries per month. The query system shall be available at least 90 percent of the time, 24 hours each day on a monthly basis.

d. Help desk operation (back up for web based support or follow-on for more complex problems). Again, the contractor must address problems based either in limits of user understanding or in the model codes. Typically only one or two queries per month require intensive support at this level. Generally, issues can be addressed by e-mail.

e. Promote and facilitate collaboration and information sharing among air quality model developers and users in a variety of organizational settings (universities, states, regions, Federal agencies, national laboratories, industry, consulting). This shall include, at a minimum, using Internet listservers to communicate with the user community and an on-line change request tracking system. The contractor may develop and conduct training with regard to the EPA modeling tools for non-EPA entities at their discretion, separate and apart from this contract.

f. Sharing of selected model output data sets and other auxiliary information relevant to users of CMAQ and related software tools. The contractor shall, as specified by task order, provide selected air quality modeling output data sets and auxiliary information that are of common interest to the community of air quality modelers. These data sets shall be documented and made available by means of an anonymous ftp. The contractor shall analyze the data sets as may be required by specific task orders.

Task 2. Public Release of Models and Updates

a. The contractor shall maintain an on-line archive for model users. The archive shall maintain model code through a suitable software control system, such as the Concurrent

Version System (CVS) or GIT (not an acronym). The contractor shall coordinate with

EPA model developers in documenting model changes for the public releases of EPA developed models. Updated model code for CMAQ and other EPA air quality modeling products, and associated release notes provided by EPA, shall be released once each year on the average.

b. As directed by task order, the contractor shall also plan and release up to two other public releases each year. These releases may be additional CMAQ updates or be code updates for SMOKE, MCIP, PAVE, Spatial Allocator, VERDI, or AMET or other modeling tools. All releases shall be by internet.

Task 3. Model Extensions

In order to foster community participation in model improvements, the contractor shall receive, evaluate, and release improvements contributed to the supported models by other collaborators in accordance with unit and system testing protocol guidelines established by the contractor and approved by EPA. Consequently, the contractor shall have specified procedures, knowledge and ability to develop and update software unit and system testing protocols, knowledge of the development and air quality model evaluation procedures, and demonstrated specific knowledge of the underlying science and code of the above reference suite of air quality models and modeling tools. Extension and application of CMAS supported air quality modeling tools to multi-media environmental problems (for example, fluxes to and from land and water surfaces, effects of land cover on fluxes, and the role of nitrogen compounds in the environment) may be reflected in model extensions.

All model source code released through CMAS is to be public domain, open-source code.

Intellectual rights and contributions of all contributors shall be acknowledged in the code and documentation. EPA source code for science and non-proprietary components of the framework code shall be available to all users with no charge if internet delivery is used.

EPA intends that new code provided by EPA or other members of the air quality modeling community to CMAS be open to all. The contractor shall specify their protocol for respecting the intellectual property rights of contributors.

Task 4. Model Testing, Review, and Release Procedures

The contractor shall establish procedures to accomplish the objectives above including:

model testing, review and distribution policies and procedures for official software releases; review criteria and procedures for incorporation of new science or new models into the system from any contributor; acknowledgment of significant intellectual contributions, and exchange and resource linkages among the community of air quality modelers to facilitate technology transfer and training. This work requires the contractor to have knowledge and experience in establishing and maintaining air quality model testing criteria (meteorology and emission scenarios and statistical evaluation), reflecting knowledge of the science and coding basis of the models. Prior CMAS protocols shall be drawn upon in establishing these procedures by EPA. These versions shall be maintained in a public archive available through the Internet, and documented separately from the model versions released by CMAS for EPA.

Task 5. Model Code Development and Modification

The contractor shall develop or modify software code of the above-referenced EPA air quality modeling products as specified by a statement of needs requirements from EPA.

Work shall include development or modification of model code to implement new or revised air quality modeling-related concept(s), testing of the code in accordance with

CMAS procedures, code documentation, and unit and system tests of the model code. In order to accomplish this work, the contractor shall have detailed knowledge and prior experience with the scientific basis and software code of the above specified set of air quality modeling tools. It is anticipated that approximately 6 task orders per year will require this type of work.

Code development and modification may be required in specific chemical or physical processes of the CMAQ model or in its numerical solution routines, or data processors including MCIP. The contractor shall use EPA model coding standards as specified by the COR for all model code and development. Development and testing of refined features for the Weather and Research Forecasting (WRF) or other meteorological models used for producing input meteorological fields to CMAQ may be required. The contractor may be required to develop specific code to assist in the application of CMAQ or related modeling code to multi-media applications, for example interaction with land and water surfaces and resulting issues. The contractor shall have the demonstrated existing capabilities in the disciplines needed to accomplish assigned work in this task for the separate models, as well as potential work in integrated meteorological-air quality models.

Task 6. Model Documentation

The contractor shall prepare new or revised documentation of EPA air quality modeling products. Changes in air quality modeling components may require new or updated procedural (User Manual) documentation and documentation of code and file structure.

All model releases shall be accompanied by revised documentation and read-me files. In order to accomplish this task, the contractor shall have specific experience and knowledge with the scientific basis and software code of the above specified set of air quality modeling tools.

Task 7. Specialty Air Quality Modeling Workshops

The contractor shall conduct small specialty workshops related to the EPA air quality modeling tools or issues impacting them. For planning purposes, assume one specialty workshop each year, to be held in the Research Triangle Park, NC, area with no more than 40 participants. The contractor’s scientific staff shall participate in the workshops, as appropriate to the subject of the specialty conferences. As directed by task order, the contractor shall be responsible for making logistical arrangements for the workshops, including making arrangements for conference space and presentation equipment, assisting invited participants with preparation of presentation information, preparation of workshop materials, agenda, and handouts; registration; and facilitating the workshop.

Deliverables

Deliverables documenting the user support function of the contract are specified in Table 1.

Because of the research nature of some of the work to be performed, specific deliverables and due dates for that work cannot be specified far in advance in this scope of work. Specific deliverables will be specified in detailed task orders issued to the contractor by the CO. In general, deliverables will consist of model codes and tests, associated documentation, and related products, such as QC products, analyses and graphic outputs. Other deliverables associated with specialized model applications as noted above in the statement of work will be listed in written task orders issued by the CO. The contractor shall develop or modify specified model code, prepare associated data sets, evaluate the new or modified model code, perform standard quality control (QC) checks, execute air quality simulation models as part of model code evaluation, and perform requested analysis of model results for all applications required during the period of performance. The contractor shall begin work only after the issuance of a task order by the CO.

Work may begin upon receipt of the task order, subject to submission of a work plan within 15 calendar days, including the technical approach, schedule, staffing plan, and hour and cost estimate by the Contractor. Work will proceed beyond the initial 15 calendar day period upon written approval of the work plan by the CO. Unless otherwise specified, any deliverables that are produced for the work described above shall use the following software and conventions or the hardware and software specified in Appendix A:

Word Processor: Microsoft Word 2013 or higher

PC Data Base: Microsoft Excel 2013 or higher, Email: Microsoft Outlook

Operating System: Windows 7

Data Base: Oracle 7.1 or higher, ASCII

Acceptance Criteria

Deliverables shall be submitted to the COR with a copy to the CO for review. The COR or

TOCOR will use the quality and timeliness performance criteria described in Appendix B to judge the deliverables. Evaluation of the quality of the deliverables will include consideration of: clarity and successful operation of model code, correctness of input and output data, completeness of documentation including in-line code documentation, new or updated user manuals and readme files, adherence to approved QA procedures in the contract Quality

Assurance Project Plan (QAPP), clearness and quality of graphics, adequacy of model version and analytical procedures used. The COR or TOCOR has the authority to accept or reject the deliverables or request modifications by technical direction. Upon completion of the requested modifications and within 30 days the COR or TOCOR will review the deliverables and provide notification of acceptance. Upon receipt of approval the contractor shall provide the number of copies of the final version as specified in the work request to the COR.

All deliverables must meet other requirements set forth in the contract and task orders.

Deliverables shall be submitted in accordance with the contractor's approved estimate of completion dates provided to the COR in the approved work plan.

Documentation and Deliverable Management

Documentation includes written material and computer model code and model run output that specifies the contents of new or modified model code, results of model test runs, summaries, analyses, quality control documentation, periodic reports such as monthly reports, and non-recurring reports required by work requests.

All written documentation shall be submitted as scheduled in Word format (for written reports, documentation and analyses) unless otherwise specified by work request.

A. Deliverables:

1. Monthly reports, by task order, including technical progress, problems encountered and solution proposed, and a breakout of labor hours and costs, as required by the contract.

2. A contract Quality Assurance Project Plan (QAPP) applicable for the duration of the contract, to be submitted within 30 days of contract initiation. The QAPP shall reflect EPA guidance for contractors and modeling. The QAPP shall include the provisions of EPA QA guidance publications QA G-5 (Guidance tor Quality Assurance Project Plans) and QA G-

5M (Guidance for Quality Assurance Project Plans for Modeling). The QAPP shall define

Standard Operating Procedures (SOPs) to be applied to model development and evaluation, as well as execution of emission, meteorology and air quality modeling. A QA Coordinator and minimum Standard Operating Procedures (SOPs) shall be specified by the Contractor in coordination with EPA during preparation of the QAPP. Work under the contract shall not begin prior to the approval of the contract QAPP.

3. An Information Technology Security Plan (ITSP) applicable for the duration of the contract, to be submitted within 30 days of contract initiation. The ITSP shall include: (1) computer security and file backup procedures; (2) work activities to accomplish their set procedures in a way that is consistent with EPA remote access and computer cloud security policies; and

(3) all activities staff accomplished to safeguard products, the model software codes provided by EPA, and the code modules prepared by the contractor. Work under the contract shall not begin prior to the approval of the ITSP.

4. Any revisions to the approved contract Quality Assurance Project Plan (QAPP) required by the CO, with the deliverable date specified in writing.

5. Non recurring draft and final report(s) and/or computer model code and documentation required by task orders. Each report shall include a specific section addressing adherence to approved QAPP procedures. The COR or TOCOR will provide comments (if any) within 30 calendar days of receipt of the draft material. The contractor shall provide a final version of the report(s) and/or computer model code within 10 days of receipt of comments, unless otherwise specified in writing.

6. Work Plan estimates shall be submitted within 15 days of each task order request for review and approval.

# DELIVERABLE TASKING Distribution Acceptance Criteria DATE DUE

1 Monthly Report of Technical

Progress by Task order, including labor hour and cost breakout.

Contract requirement

1 COR

1 Alt COR

1 TOCOR

1 CO

Completeness, clarity, organization, consistency, timeliness

& technical accuracy

Monthly by 15th

Calendar day

2 QAPP Contract requirement

1COR

1Alt COR

1 CO

Completeness, clarity, technical accuracy, timeliness, and organization

No later than 30 days after contract award

3 QAPP updates Contract and task order requirement

1 COR

1 Alt. COR

1 CO

Completeness, clarity, accuracy, consistency with EPA QA/QC requirements

Within calendar

30 days of receipt of directive from

CO

4 Draft and Final

Task order

Written Products

Task order requirement

1 COR

1ALT COR

1 TOCOR

1CO

Completeness, clarity, technical accuracy, timeliness, and organization

As required by written task order contract modification

5 Draft and Final

Task order computer code and documentation

Task order requirement

1COR

1 ALT

COR

1TOCOR

Completeness of code, adherence to in-line documentation and

EPA coding standards, timeliness, successful operation, completeness of documented testing.

As required by written task order contract modification

6 Work Plans including hour and cost estimate for each task order

Written task order scopes and technical directives

1 COR

1 Alt. COR

1 TOCOR

1 CO

In response to task order contract modifications.

Completeness, clarity, technical accuracy and timeliness

Within 15 calendar days of receipt of task order directive.

Acceptance Criteria:

1. Monthly Report: The timeliness of monthly reports will be judged on the basis of the performance criteria set forth in Appendix B, Paragraphs B and C. Beyond the information required in the contract and task order the contractor shall include: descriptions of: major activities completed, major activities in progress, problems, delays, actions required by EPA, future activities, and list of server hardware and software configurations. The report shall separate Windows related information from Unix related information. The monthly report shall include, in addition to normal task order status and activities, all activities of the Program

Manager, and other key personnel in support of this contract The cost portion of the Monthly

Report shall include a detailed breakdown of the number and type of labor employed at a work request level.

2. QAPP: All SOPs shall demonstrate quality assurance and control aspects of each task in accordance with the applicable approved QAPP.

3. Task order Plans: Work Plans for each task order shall be prepared and submitted to the CO and COR within 15 days of task order initiation. The Work Plans will be judged on the basis of the completeness of technical approach, adherence to the scope of work provided to the contractor at task order initiation, and thoroughness and accuracy of work plan schedules, and hour and cost estimates.

4. Task order products: Completeness, clarity, organization, consistency, technical accuracy, adherence to QAPP procedures, timeliness, and grammatical correctness of all written materials and reports.

Training and Travel:

The contractor is expected to provide staff whom are already fully trained in the subject matter needed to successfully perform work under the contract. However, the contractor may be asked to send one or more persons to specialized courses, workshops, and/or meetings that either are unique to EPA's work or involve specialized modeling applications described in the scope. All travel must be within the scope of an approved task order in advance of travel, consistent with current U.S. Government travel rules and cost limitations and approved in advance in writing by the CO. Occasional technical support for preparation of technical products for conferences or papers, or travel to conferences may be required. For the purposes of planning, the contractor shall assume that, during each option year, one person each shall travel to two conferences of four days duration, one in Los Angeles and one in Washington, DC.

Travel reports shall be prepared that include the purpose and location of the travel, attendees, meeting agenda, significant findings, and any plans or events resulting from the meeting. Travel reports shall be submitted with the next monthly report attached to the relevant task order section.

Travel reports will be judged in accordance with the performance criteria described in Appendix

B, paragraphs B and C, with respect to completeness, clarity, organization, technical accuracy, timeliness, and grammatical correctness.

Financial Reporting/Cost Tracking

Financial reporting and cost tracking shall include a mechanism for providing costs and estimates at the task order level, including costs associated with information technology security. This will require establishing separate project codes as specified in the individual work request or technical direction document. A minimum of five percent of costs by lifecycle shall be reflected in the codes. Tasks 1 and 2 are deemed to be included in the operation and maintenance life cycle of modeling; and tasks 3 through 7 are included in the planning and development portion of the life cycle. All costs associated with specific project codes shall be reported in the monthly reports. Monthly task order time and cost reports shall be organized to show labor hours and costs by work request for the month and cumulatively for the option year.

Monthly financial reporting shall be judged in accordance with the performance criteria described in Appendix B, paragraph C, with respect to completeness, clarity, organization, consistency, computational accuracy, timeliness, technical accuracy, and grammatical correctness.

Please indicate which Application Life Cycle stages will be covered based on the systems/tools that will be supported under this contract:

___Definition/Initiation (New Project)

_X_ Development/Acquisition

_X_ Implementation (Deployment)

_X Operation/Maintenance

___Termination (Disposal)

Appendix A

Required Software Capabilities

Contractor personnel performing the above air quality-related work shall have knowledge and operating experience with the following software models, tools, and software development languages:

AERMOD - American Meteorological Society/Environmental Protection Agency Regulatory

Model

AMET - Atmospheric Model Evaluation Tool

ArcINFO - GIS software system

ARCSDE - Server software accompanying ArcINFO

AVS - Advanced Visualization System

BEIS3.x - Biogenic Emission Inventory System

BENMAP - Benefits Mapping and Analysis Program

C/C++ - Programming language

CMAQ - Community Multiscale Air Quality Model

DDM-3D - Direct Decoupled Method – 3 dimensional

EMF - Emission Modeling Framework

EMS-HAP - Emission Modeling System for Hazardous Air Pollutants

FEST-C - Fertilizer Emission Scenario Tool for CMAQ

Fortran - Programming language

GMISS - Gridded Model Information Support System

IDV - Integrated Data Viewer tool

JAVA - Programming language

ACCESS - Microsoft database program

MCIP - Meteorology-Chemistry Interface Processor (CMAQ tool)

MEGAN - Model of Emissions of Gases and Aerosols from Nature

MOVES - New EPA GIS-based model for Mobile and Offroad Vehicle Emissions

MPAS - Model for Prediction Across Scales

NCAR - National Center for Atmospheric Research

NCAR Graphics – Graphics analysis package support by NCAR netCDF - net Common Data Format convention

NCL - NCAR Command Language

OLAM - Ocean-Land-Atmosphere Model

Oracle - Database program

Python - A scripting language

QAPP - Quality Assurance Project Plan

R - R statistical analysis software

R-Line Research Line source dispersion model for near surface releases

RAGG - (R statistical aggregation routine)

SA - Spatial Allocator

SAS - Statistical Analysis System

SOP - Standard Operating Procedure

SMOKE - Sparse Matrix Operator Kernel Emission model\

SPARROW - SPAtially Referenced Regressions On Watershed attributes model

Unix (Linux) - Computer operating system language

VERDI - Visualization Environment for Rich Data Interpretation tool

VIC - Variable Infiltration Capacity model

VIS5D - Visualization 5 dimension tool

Appendix B

Performance Criteria

The following metrics will be used in evaluating the Contractor’s performance for the work described in the paragraphs above. These metrics are designed to be applicable in the uncertainty inherent in supporting research and development science modeling.

A. Quality

1. Documentation of adherence to the approved contract QAPP and associated SOPs in deliverable products. (Deduction for failure to timely submit an approvable

QAPP – 10% of invoice; deduction for failure to document adherence to QA procedures in the approved QAPP – 5% of invoice).

2. The QAPP provisions shall be followed and documented for each task order by the Contractor’s QA coordinator. A QA report (contents defined in the QAPP) reflecting the requirements of the QAPP and SOPs shall be submitted for each work request within 30 work days after submission of the final task order deliverable. (Deduction for each late work request QA report – 2 % of invoice).

3. Quality in research modeling support may sometimes reflect events or items that are not readily foreseeable. Diligence by the Contractor in detecting problems outside of prescribed QA procedures is important. Consequently, Contractor-initiated and documented solutions to significant QA problems outside of the required QA procedures will be rewarded by reducing any percentage penalties on an invoice by up to 2% per documented event for the relevant invoice period.

EPA retains the right to determine what constitutes a significant QA problem.

B. Timeliness

1. The initial Project QAPP shall be submitted within 45 calendar days of contract award. (Deduction for late QAPP submission – 5% of invoice).

2. Work request QA reports shall be submitted within 15 work days after completion of submission of the final work request product (Deduction for late submission –

2% of invoice for each late work request QA report).

3. Products for each task order shall be submitted according to the schedule in the task order (allowing for due date modifications agreed to and/or initiated in writing by the CL). (Deduction for late deliverable without prior agreement on changed due dates – 3 % of invoice for each late work request final submission).

C. Project Cost Management

1. Each work request shall be completed within estimates prepared by Contractor and approved by the CO. Estimates are subject to revision for cause if agreed to in advance by the CO. (Deduction for each unapproved work request cost overrun

– 2 % of invoice).