Sol_SOL-CI-15-00011_Amd_000001.pdf

(x)

SOL-CI-15-00011

x x

1 copies of the amendment; (b) By acknowledging receipt of this amendment on each copy of the offer submitted; or (c) By separate letter or telegram which includes a reference to the solicitation and amendment numbers. FAILURE OF YOUR ACKNOWLEDGEMENT TO BE RECEIVED AT

THE PLACE DESIGNATED FOR THE RECEIPT OF OFFERS PRIOR TO THE HOUR AND DATE SPECIFIED MAY RESULT IN REJECTION OF YOUR OFFER. If by virtue of this amendment you desire to change an offer already submitted, such change may be made by telegram or letter, provided each telegram or letter makes reference to the solicitation and this amendment, and is received prior to the opening hour and date specified.

x

Cincinnati OH 45268

CPOD

Mail Code: NWD 26 West Martin Luther King Drive US Environmental Protection Agency

CPOD

08/12/2015000001

13. THIS ITEM ONLY APPLIES TO MODIFICATION OF CONTRACTS/ORDERS. IT MODIFIES THE CONTRACT/ORDER NO. AS DESCRIBED IN ITEM 14.

12. ACCOUNTING AND APPROPRIATION DATA (If required) is not extended.is extended, Items 8 and 15, and returning

Offers must acknowledge receipt of this amendment prior to the hour and date specified in the solicitation or as amended, by one of the following methods: (a) By completing

The above numbered solicitation is amended as set forth in Item 14. The hour and date specified for receipt of Offers

11. THIS ITEM ONLY APPLIES TO AMENDMENTS OF SOLICITATIONS

FACILITY CODE CODE

10B. DATED (SEE ITEM 13)

10A. MODIFICATION OF CONTRACT/ORDER NO.

9B. DATED (SEE ITEM 11)

9A. AMENDMENT OF SOLICITATION NO.

CODE

8. NAME AND ADDRESS OF CONTRACTOR (No., street, county, State and ZIP Code)

7. ADMINISTERED BY (If other than Item 6)CODE 6. ISSUED BY

PAGE OF PAGES

4. REQUISITION/PURCHASE REQ. NO.3. EFFECTIVE DATE2. AMENDMENT/MODIFICATION NO. 5. PROJECT NO. (If applicable)

1. CONTRACT ID CODE

AMENDMENT OF SOLICITATION/MODIFICATION OF CONTRACT

08/12/2015

CHECK ONE A. THIS CHANGE ORDER IS ISSUED PURSUANT TO: (Specify authority) THE CHANGES SET FORTH IN ITEM 14 ARE MADE IN THE CONTRACT

B. THE ABOVE NUMBERED CONTRACT/ORDER IS MODIFIED TO REFLECT THE ADMINISTRATIVE CHANGES (such as changes in paying office, C. THIS SUPPLEMENTAL AGREEMENT IS ENTERED INTO PURSUANT TO AUTHORITY OF:

D. OTHER (Specify type of modification and authority) appropriation date, etc.) SET FORTH IN ITEM 14, PURSUANT TO THE AUTHORITY OF FAR 43.103(b).

E. IMPORTANT: Contractor is not, is required to sign this document and return __________________ copies to the issuing office.

ORDER NO. IN ITEM 10A.

14. DESCRIPTION OF AMENDMENT/MODIFICATION (Organized by UCF section headings, including solicitation/contract subject matter where feasible.)

Amendment 1 to SOL-CI-15-00011 STREAMS 3

16A. NAME AND TITLE OF CONTRACTING OFFICER (Type or print)15A. NAME AND TITLE OF SIGNER (Type or print)

15C. DATE SIGNED 16B. UNITED STATES OF AMERICA 15B. CONTRACTOR/OFFEROR 16C. DATE SIGNED

(Signature of person authorized to sign) (Signature of Contracting Officer)

Matthew Growney

STANDARD FORM 30 (REV. 10-83)

Prescribed by GSA

FAR (48 CFR) 53.243

NSN 7540-01-152-8070

Previous edition unusable

Except as provided herein, all terms and conditions of the document referenced in Item 9A or 10A, as heretofore changed, remains unchanged and in full force and effect.

08/25/2015

Technical Question Responses

STREAMS 3 SOL¿CI¿15¿00011

Question 1

Please confirm that the proposal due date is Sept 22, 2015 at 1300 EDT.

Response: Confirmed. The proposal due date is Sept 22, 2015 at 1300 EDT. Note, an early draft

Confirmed. The proposal due date is Sept 22, 2015 at 1300 EDT. Note, an early draft of the cover memo was automatically posted to the Federal Business Opportunity (FBO) site by our contract writing system with the body of the solicitation. This early draft cover page has the following placeholder date July ##, 2015. It also includes a reference to a September 15 due date, and some early local clause references that are updated in the final cover memo which is dated August 12, 2015 and is posted on FBO as a separate document. Please ignore the information in the early draft cover memo dated July ##, 2015.

Question 2

Can EPA please clarify what percentage of the total evaluation of the technical proposal each sub section of the proposal will constitute, i.e., what is the maximum number of points that could be awarded to each section?

Response: To address what was viewed as an undue focus on the quantitative elements

To address what was viewed as an undue focus on the quantitative elements introduced through point use, EPA has revised its source selection and evaluation process to abandon the use of point weighting and is using instead a listing of technical criteria by relative importance. The relative importance of the technical criterion and subcriterion are provided under section (c) of the M¿3 local clause EPA¿M¿15¿101 `Evaluation Factors for Contract Award¿.

Question 3

Will the QMP (or any other portion of the technical proposal) be evaluated on a pass/fail basis or will it be scored against a maximum number of points?

Response: Points will not be used, and the QMP and other criteria will be evaluated based on the

Points will not be used, and the QMP and other criteria will be evaluated based on the technical merit the proposal demonstrates in the areas encompassed by the technical factors listed under section (b) of the of the M¿3 local clause EPA¿M¿15¿101 `Evaluation Factors for Contract Award¿.

Note, there are some aspects of this solicitation, that are assessed on a pass/fail basis but these aspects are applicable to the `apparently successful offeror(s)¿, thus, they are considered to be matters of responsibility and outside of our technical merit evaluation and comparative assessment activities. (See

FAR 9.504(e) and 19.702(a)(1))

Question(s) 4

Please confirm that a CBI plan is not required since it is not listed as part of the Technical or Cost

Proposal Instructions. Solicitation ¿ Section H¿9¿H¿15, PP. 35¿39

A CBI plan is referenced as excluded from the page limitation of the technical proposal, but is not described in the RFP. We recognize that an approved CBI Plan will need to be in place for contractor to accept CBI ¿ however since the nature of the CBI (FIFRA, TSCA, etc.) could vary on the contract, is it EPA¿s intent that the applicable CBI Plan be provided and approved during performance of the contract as needed so that the plan be specifically aligned with the information to be received?

Response: There is no requirement to submit a CBI plan in response to this solicitation.

There is no requirement to submit a CBI plan in response to this solicitation.

Question(s) 5

Please confirm that the Small Business Plan should be included with the Cost Proposal.

Solicitation ¿ Section L¿27, P. 93 & Attachment 4 ¿ P. 2, #4.

Section L¿27, Page 90 states: ¿. . . provide a subcontracting plan with their initial proposal.¿

Where do you want us to provide this plan?

Response: Confirmed. Those items listed as excluded from the page count of the technical

Confirmed. Those items listed as excluded from the page count of the technical proposal should be provided in the Cost Proposal volume. The page limitation associated with the technical proposal volume supports the goals of attaining well¿honed demonstrations of technical merit for our technical evaluation team. Those elements, such as, the Small Business Plan, that are other than comparative evaluative elements can and should be placed in the cost proposal so they do not use up page space available for offerors to fully demonstrate their technical merit.

Question 6

Where should we include our Representations and Certifications?

Response: They should be provided in the Cost Proposal.

They should be provided in the Cost Proposal.

Question(s) 7

On page 2 of Attachment 4, it states that ¿past performance information¿ is not included in the page limitation. Please clarify what portion of our past performance information is included in the page limitation?

Please confirm that the information required in Criterion II, Past Performance, is not included in the page count of the technical proposal, as per the instructions in #4 on page 2 of Attachment

4, Technical Proposal Instructions.

Response: Information necessary for our technical evaluators to assess the technical scope/relevance

Information necessary for our technical evaluators to assess the technical scope/relevance of the referenced contact should be provided in the technical proposal and is subject to the page limitation of that document. Past Performance criterion are complex criterion involving an assessment of technical scope/relevance along with customer satisfaction and other elements. The members of our evaluation team assess and provide input on the technical scope/relevance of the referenced contracts so information necessary for our technical evaluators to assess the technical scope/relevance of the referenced contract should be provided in the technical proposal. Any additional information may be provided in the cost proposal volume.

Question 8

No Past Performance Questionnaire (PPQ) or Client Authorization Letter template were included in the RFP. Is it correct to assume EPA will contact the Contracting Office of the respective contract for past performance information?

Response: Correct, when EPA recently revised its internal guidance documents it elected not to retain

Correct, when EPA recently revised its internal guidance documents it elected not to retain these documents. EPA will search the Past Performance Information Retrieval System (PPIRS) and contact the provided references as its primary means of collecting past performance information.

Question 9

Are resumes required as a tabbed attachment in the technical proposal submission, and if so, for which personnel?

Response: Resumes are not required but would likely be appropriate to substantiate the

Resumes are not required but would likely be appropriate to substantiate the qualifications of the individuals proposed to perform in the evaluated P4 and P3 positions. Submitting them under a tabbed attachment to the technical proposal is an acceptable approach.

Question 10

Attachment 4 (Technical Proposal Instructions) pg. 2 Shall the hardcopy be single or doublesided?

Response: EPA¿s preference is for double¿sided.

EPA¿s preference is for double¿sided.

Question 11

Attachment 6, Cost Proposal Instructions, Page 4, paragraph (g) states ¿THIS CHART SHOULD BE

INCLUDED IN BOTH THE TECHNICAL PROPOSAL AND THE COST PROPOSAL. HOWEVER, ONLY IN

THE COST PROPOSAL SHOULD THE CHART ALSO REFLECT THE ALLOCATION OF THE SPECIFIED

ODC.¿ Where does this chart belong in the Technical Proposal?

Response: The allocation of level of effort between team members as provided in this section

The allocation of level of effort between team members as provided in this section establishes a crosswalk between the technical and cost proposals and allows the technical evaluators to ensure that the labor break down is not inconsistent with the technical approach. Where it is placed is a matter of offeror discretion.

Question 12

Is it acceptable for large tables to be printed or formatted as fold¿out pages (i.e., on paper or electronic pages that exceed the 8.5 x 11 size)?

Response: Consistent with the instructions under item 4 in Attachment 4 `Technical Proposal

Consistent with the instructions under item 4 in Attachment 4 `Technical Proposal

Instructions¿ all information in the technical proposal shall be standard 8.5 X 11 size. It is understood that a number of tables in the cost proposal may be better presented on pages formatted to exceed the

8.5 X 11 size and it is left to offeror discretion how to most clearly present the information required in its cost proposal submission.

Question 13

Attachment 6, Cost Proposal Instructions, refers to a `Base Fee¿. Can EPA clarify whether this contract will be awarded as a cost plus fixed fee or a cost plus award fee contract?

Response: The reference to `base fee¿ in the format chart in section (b) of Attachment 6 `Cost

The reference to `base fee¿ in the format chart in section (b) of Attachment 6 `Cost

Proposal Instructions¿ would have been better stated as `fixed fee¿ as there is no award fee component in the proposed contract. As provided under the CO added clause at B¿6 item 2 task orders may be issued as firm¿fixed¿price type or cost¿plus¿fixed¿fee type.

Question 14

In Attachment 6, Cost Proposal Instructions, Other Direct Costs (ODCs) have been condensed into two components, Travel and Other Direct Costs. As separate ODC elements were not provided for additional items such as IT/telecommunications, how should ODC elements normally treated as indirect be handled in the cost proposal?

Response: As with the labor mix, the ODC categories and levels are estimates of what ODCs will be

As with the labor mix, the ODC categories and levels are estimates of what ODCs will be required and are used to provide a sound basis upon which to compare competing proposals. Whether such costs are captured as direct or indirect may vary based on an offeror¿s established accounting practice, and information on how to handle ODC elements that may be captured by an offeror indirectly is provided in section (h) of the cost proposal instructions.

Question(s) 15

The level of effort distribution table includes 5,200 hours per year for the Contract Manager. As this is more than the number of hours available for one single individual, we are interpreting this to mean that EPA would like for this to include the hours associated with the prime contractor¿s

Contract Manager as well as the hours associated with the subcontractor¿s Contract Manager.

Please confirm that this interpretation is correct. Attachment 6¿Cost Proposal Instructions, P.

Given the 5,200 hour per year allocation for the Contract Manager in the cost proposal instructions, does EPA require that offerors feature more than one individual in Section I.A of the technical proposal, Qualifications of the P4 Contract Manager?

Response: The number of individuals, their position title (Co, Alternate, etc.), and employment

The number of individuals, their position title (Co, Alternate, etc.), and employment affiliation are all matters of offeror discretion and business judgement.

Question(s) 16

In most EPA RFPs, experience/qualification equivalencies are provided, however, Attachment 5 (Labor Classification Standards) of the RFP does not provide any Equivalent

Experience/Qualification Substitutions for the identified Professional and Technical levels (e.g.

Bachelor¿s degree plus how many years of experience is equivalent to a Ph.D.). Can the government please provide the desired equivalencies?

In attachment 5 Labor Classification Standards, it shows that all Level 4 staff must have a Ph.D.

and 10 years of experience and that all Level 3 staff must have a Master¿s degree and 6 years of experience. Is the following substation language allowable or must all P4¿s have Ph.D. and all

Level 3 have Masters?

Substitution Language: Experience/Qualification Substitutions

(1) Any combination of additional years of experience in the proposed field of expertise plus full time college level study in the particular field totaling four years will be an acceptable substitute for a Bachelor Degree.

(2) A Bachelor Degree plus any combination of additional years of experience and graduate level study in the proposed field of expertise totaling two years will be an acceptable substitute for a

Master¿s Degree.

(3) A Bachelor Degree plus any combination of additional years of experience and graduate level study in the proposed field of expertise totaling four years or a Master¿s Degree plus two years of either additional experience or graduate level study in the proposed field of expertise will be an acceptable substitute for a Ph.D. Degree.

(4) Additional years of graduate level study in an appropriate field will be considered equal to years of experience on a one¿to¿one basis.

Response: The labor classifications expressed in this solicitation are mere `standards¿ as opposed to

The labor classifications expressed in this solicitation are mere `standards¿ as opposed to

`minimum requirements¿ thus there is no need for substitutions as there is no need for an individual to meet the `normal¿ qualifications or `typical¿ experience (note use of the terms normal and typical vs.

minimum) expected for individuals proposed at that level. This approach is more open and flexible than a minimum standard approach and allows offerors greater freedom to propose the individuals they believe to be the best fit for each position and not potentially have their candidate options limited by rigid cut offs. As always the burden is on the offeror to demonstrate that the individuals proposed to fill the professional level positions are well qualified to perform the position and the provided standards give offerors notice of the standards that we generally anticipate an individual proposed in the various levels would have.

Question 17

Under L.28 (3) ¿ If we have subcontractors that have proprietary information and they are providing the prime a sealed envelope for the cost proposal, how is that information to be submitted electronically to EPA? Does the subcontractor also submit the sealed package information via mail, through FedConnect or do they e¿mail it to the procuring contracting officer and identify in the e¿mail the team that the pricing information is for?

Response: It is my understanding that vendors may identify and submit information as `cost

It is my understanding that vendors may identify and submit information as `cost senstitive¿ in FedConnect. To do so they would upload the attachment under ¿Cost Sensitive

Attachments¿ instead of ¿Attachments¿. Once the attachment is submitted to EAS, it will be available under Vendor¿Responses¿ Attachments to the CO and Buyer that are listed on the document. These are the only users who will be able to see the cost sensitive attachments. Because FedConnect is hosted by Compusearch, vendor/contractor questions relating to FedConnect security and detailed information on how a subcontractor can confidentially submit in the above circumstance can best be answered by Compusearch and not EPA. Contact information for information and questions on FedConnect can be found at the following web site https://www.fedconnect.net/FedConnect/.

Question 18

L.28, (a)(3) on page 90 states: ¿Notwithstanding any other provision or clause in this solicitation to the contrary, the original technical and cost proposal shall be submitted via the FedConnect® web portal (www.fedconnect.net) as the official submission.¿ The RFP also requires offerors to submit a hard copy of both technical and cost proposals (L.28 (a)(4)). Given that EPA is considering the electronic copy of the proposals submitted through FedConnect as the official submission, would EPA allow an additional day beyond the due date of September 22, 2105 for submittal of hard copy proposals?

Response: As the `official submission¿ the timeliness rules shall apply only to the electronic

As the `official submission¿ the timeliness rules shall apply only to the electronic submission. Therefore, in order to avoid rejection of a proposal for being untimely, a complete electronic version of an offeror¿s proposal must be received by EPA via FedConnect by the Sept 22, 2015 at 1300 EDT closing time of the solicitation. Hard copy versions may arrive later but should be arraigned for delivery at EPA by close of business on September 23. (See provision at L¿28 for appropriate addresses) The burden of submitting a complete and timely proposal is on the offerors, agency personnel have no duty to review submissions for completeness prior to the solicitation closing time.

Question 19

Attachment 1 (Performance Work Statement) pg.13 "I. The Contractor shall comply with the EPA policy entitled "EPA QA Field Activities Procedure". See PWS Attachment D for a copy of the procedure." Should it read "...See PWS Attachment 4..."?

Response: In an effort to reduce potential confusion, the attachments associated with the PWS were

In an effort to reduce potential confusion, the attachments associated with the PWS were to be labeled A¿D so that they could be distinguished from the SOL attachments which were labeled 1¿ 6.

It appears that one reference to a PWS attachment 4 instead of D failed to be scrubbed. The ¿EPA QA

Field Activities Procedure¿ is the document dated 7/7/05 issued by the EPA Chief Information Officer.

Question(s) 20

Page 7 of Attachment 4 of the RFP refers to the following web address for more information on

ORD/NRMRL¿specific Quality Management Plan requirements:

http://www.epa.gov/ORD/NRMRL/qa/.

This link is currently not functioning (returns an `error404.html¿ message). Are there other locations at which we can access ORD/NRMRL¿specific QA documents? If not, can EPA please clarify which documents are to be referenced in the QMP, or provide electronic copies of these documents?

Attachment 4 Criterion IV, P. 7. Following the link provided in the last sentence, ¿For more information addressing ORD/NRMRL requirements specifically, the following web address can be accessed: http://www.epa.gov/ORD/NRMRL/qa/¿ results in ¿The webpage cannot be found¿

Please provide an updated link to these ORD/NRMRL requirements for the Quality Management

Plan.

Response: see following document ¿ see following document ¿

QAPP REQUIREMENTS AND GUIDANCE

For QA Categories 1 and 2 QAPPs (ORD Category A), QAPPs must be prepared to meet the

EPA requirements specified in EPA Requirements for QA Project Plans, QA/R-5 (PDF) (40 pp, 121 K) (EPA/240/B-01/003) March 2001.

For QA Category 3 (ORD Category B), QAPPs can be prepared to meet a subset of the requirements above, depending on the type of project, as presented in the following example QAPP templates.

Measurement Projects

A measurement project involves the performance of environmental measurements to:

¿ Demonstrate the performance of technologies under defined conditions

¿ Monitor environmental parameters in order to characterize a particular population

¿ Evaluate unproven theories or processes

The project may be bench scale or field scale.

NRMRL QAPP REQUIREMENTS FOR MEASUREMENT PROJECTS

GENERAL REQUIREMENTS: Include cover page, distribution list, approvals, and page

Include cover page, distribution list, approvals, and page numbers.

0. COVER PAGE

Include the Division/Branch, project title, revision number, EPA technical lead, QA category, organization responsible for QAPP preparation, and date.

1. PROJECT DESCRIPTION AND OBJECTIVES

1.1 Describe the process and/or environmental system to be evaluated.

1.2 State the purpose of the project and list specific project objective(s).

2. ORGANIZATION AND RESPONSIBILITIES

2.1 Identify all project personnel, including QA, and related responsibilities for each participating organization, as well as their relationship to other project participants.

2.2 Include a project schedule that includes key milestones.

3. SCIENTIFIC APPROACH

3.1 Describe the sampling and/or experimental design that will be used to generate the data needed to evaluate the projective objective(s). A description of the design should include the types and numbers of samples (including QC and reserve samples), the design of the sampling network, sample locations and frequencies, and the rationale for the design.

3.2 Identify the process measurements (e.g., flow rate, temperature) and specific target analyte(s) for each sample type.

3.3 Describe the general approach and the test conditions for each experimental phase.

4. SAMPLING PROCEDURES

4.1 Describe any known site-specific factors that may affect sampling procedures as well as all site preparation (e.g., sampling device installation, sampling port modifications, achievement of steadystate) needed prior to sampling.

4.2 Describe or reference each sampling procedure (including a list of equipment needed and the calibration of this equipment as appropriate) to be used. Include procedures for homogenizing, compositing, or splitting of samples, as applicable.

4.3 Provide a list of sample containers, sample quantities to be collected, and the sample amount required for each analysis, including QC sample analysis.

4.4 Specify sample preservation requirements (e.g., refrigeration, acidification, etc.) and holding times.

4.5 Describe the method for uniquely numbering each sample.

Page 1 of 2 Revision 0 10/2008

4.6 Describe procedures for packing and shipping samples, including procedures to avoid crosscontamination, and provisions for maintaining chain-of-custody (e.g., custody seals and records), as applicable.

5. MEASUREMENT PROCEDURES

5.1. Describe in detail or reference each process measurement or analytical method to be used. If applicable, identify modifications to EPA-approved or similarly validated methods.

5.2. If not provided in Section 5.1 or the referenced method, include specific calibration procedures, including linearity checks and initial and continuing calibration checks.

6. QUALITY METRICS (QA/QC CHECKS)

6.1. For each process measurement and analytical method, identify the required QC checks (e.g., blanks, control samples, duplicates, matrix spikes, surrogates), the frequencies for performing these checks, associated acceptance criteria, and corrective actions to be performed if acceptance criteria are not met.

6.2. Any additional project-specific QA objectives (e.g., completeness, mass balance) shall be presented, including acceptance criteria.

7. DATA ANALYSIS, INTERPRETATION, AND MANAGEMENT

7.1 Identify the data reporting requirements, including data reduction procedures specific to the project and applicable calculations and equations.

7.2 Describe data validation procedures used to ensure the reporting of accurate project data.

7.3 Describe how the data will be summarized or analyzed (e.g., qualitative analysis, descriptive or inferential statistics) to meet the project objective(s).

7.3.1 If descriptive statistics are proposed, state what tables, plots, and/or statistics (e.g., mean, median, standard error, minimum and maximum values) will be used to summarize the data.

7.3.2 If an inferential method is proposed, indicate whether the method will be a hypothesis test, confidence interval, or confidence limit and describe how the method will be performed.

7.4 Describe data storage requirements for both hard copy and electronic data.

8. REPORTING

8.1 List and describe the deliverables expected from each project participant responsible for field and/or analytical activities.

8.2 Specify the expected final product(s) that will be prepared for the project (e.g., journal article, final report).

9.REFERENCES

Provide references either in the body of the text as footnotes or in a separate section.

Secondary Data Projects

A secondary data project involves the gathering and/or use of existing environmental data for purposes other than those for which they were originally collected.

NRMRL QAPP REQUIREMENTS FOR SECONDARY DATA PROJECTS

GENERAL REQUIREMENTS: Include cover page, distribution list, approvals, and page numbers.

Include cover page, distribution list, approvals, and page numbers.

0. COVER PAGE

Include the Division/Branch, project title, revision number, EPA technical lead, QA category, organization responsible for QAPP preparation, and date.

1. PROJECT DESCRIPTION AND OBJECTIVES

1.1 Describe the process and/or environmental system to be evaluated.

1.2 State the purpose of the project and list specific project objective(s).

2. ORGANIZATION AND RESPONSIBILITIES

2.1 Identify all project personnel, including QA, and related responsibilities for each participating organization, as well as their relationship to other project participants.

2.2 Include a project schedule that includes key milestones.

3. SCIENTIFIC APPROACH

3.1 Identify the secondary data needed to meet the project objective(s). Specify requirements relating to the type of data, the age of data, geographical representation, temporal representation, and technological representation, as applicable.

3.2 Identify the source(s) for the secondary data. Discuss the rationale for selecting the source(s) identified. If a hierarchy of sources exists for the gathering of secondary data, specify that hierarchy.

4. QUALITY METRICS

4.1 Specify the quality requirements of the secondary data. These requirements must be appropriate for the intended use of the data. Address accuracy, precision, representativeness, completeness, and comparability, if applicable.

4.2 Describe the procedures for determining the quality of the secondary data.

4.3 If no project-specific data quality requirements exist, state this in the QAPP. If the quality of the secondary data will not be evaluated by EPA, require that a disclaimer be added to any project deliverable to indicate that the quality of the secondary data has not been evaluated by EPA for this specific application. Provide the wording for the disclaimer.

5. DATA ANALYSIS, INTERPRETATION, AND MANAGEMENT

5.1 Identify the data reporting requirements, including data reduction procedures specific to the project and applicable calculations and equations.

5.2 Describe data validation procedures used to ensure the reporting of accurate project data.

5.3 Describe how the data will be summarized or analyzed (e.g., qualitative analysis, descriptive or inferential statistics) to meet the project objective(s).

5.3.1 If descriptive statistics are proposed, state what tables, plots, and/or statistics (e.g., mean, median, standard error, minimum and maximum values) will be used to summarize the data.

5.3.2 If an inferential method is proposed, indicate whether the method will be a hypothesis test, confidence interval, or confidence limit and describe how the method will be performed.

5.4 Describe data storage requirements for both hard copy and electronic data.

6. REPORTING

6.1 List and describe the deliverables expected from each project participant.

6.2 Specify the expected final product(s) that will be prepared for the project (e.g., journal article, final report, etc.). Specify the source(s) of the secondary data in any deliverable.

7. REFERENCES

Provide references either in the body of the text as footnotes or in a separate section.

Method Development Projects

A method development project is typically needed in situations for which there exists no standard or known method, or when an existing method needs to be modified to meet a project-specific need.

NRMRL QAPP REQUIREMENTS FOR METHOD DEVELOPMENT

PROJECTS

GENERAL REQUIREMENTS: Include cover page, distribution list, approvals, and page

Include cover page, distribution list, approvals, and page numbers.

0. COVER PAGE

Include the Division/Branch, project title, revision number, EPA technical lead, QA category, organization responsible for QAPP preparation, and date.

1. PROJECT DESCRIPTION AND OBJECTIVES

1.1 Provide a description of the situation that requires the generation of a new or modified method.

1.2 State the purpose of the project and list specific project objective(s).

2. ORGANIZATION AND RESPONSIBILITIES

2.1 Identify all project personnel, including QA, and related responsibilities for each participating organization, as well as their relationship to other project participants.

2.2 Include a project schedule that includes key milestones.

3. SCIENTIFIC APPROACH

3.1 Identify the specific analyte(s) of interest and the matrix/matrices under study.

3.2 Identify the analytical approach that will be used and how it will be optimized for this study. Also describe any tests of interference and analyte stability.

3.2 Identify the method performance metrics (QA/QC checks) that will be used to evaluate the method, including the procedures used. These metrics could include (but are not limited to) positive and negative controls, sensitivity, precision, accuracy, recovery, linearity, specificity, robustness, and range.

4. SAMPLING PROCEDURES

4.1 Provide the requirements for samples that will be used to test the method (including matrix and presence/concentration of analytes).

4.2 If synthetic (i.e., laboratory-prepared) samples are used, describe the preparation of these samples.

4.3 If non-synthetic (i.e., real-world sample) samples are used, address the following:

¿ describe the sampling design that will be used and the steps taken to assure that representative samples are collected

¿ discuss or reference each sampling procedure

¿ provide a list of sample containers, sample quantities to be collected, and the sample amount required for each analysis, including QC sample analysis

Page1 of 2

Revision 0 10/2008

¿ describe procedures for packing and shipping samples, and provisions for maintaining chain-of-custody, as applicable

4.4 Specify sample preservation requirements (e.g., refrigeration, acidification, etc.) and holding times.

4.5 Describe the method for uniquely numbering each sample.

5. MEASUREMENT PROCEDURES

5.1 Describe in detail or reference each preparation or analytical procedure to be used, if known.

Include steps for preparation, calibration, measurement, quality control, and reporting.

5.2 If not provided in Section 5.1 or the referenced method, include specific calibration procedures, including linearity checks and initial and continuing calibration checks.

6. METHOD PERFORMANCE METRICS

For each method performance metric (QA/QC check) identified in Section 3.2, specify the frequencies for performing these checks, associated acceptance criteria, and corrective actions to be performed if acceptance criteria are not met.

7. DATA ANALYSIS, INTERPRETATION, AND MANAGEMENT

7.1 Identify the data reporting requirements, including data reduction procedures specific to the project and applicable calculations and equations.

7.2 Describe data validation procedures used to ensure the reporting of accurate project data.

7.3 Describe how the data will be summarized or analyzed (e.g., qualitative analysis, descriptive or inferential statistics) to meet the project objective(s).

7.3.1 If descriptive statistics are proposed, state what tables, plots, and/or statistics (e.g., mean, median, standard error, minimum and maximum values) will be used to summarize the data.

7.3.2 If an inferential method is proposed, indicate whether the method will be a hypothesis test, confidence interval, or confidence limit and describe how the method will be performed.

7.4 Describe data storage requirements for both hard copy and electronic data.

8. REPORTING

8.1 List and describe the deliverables expected from each project participant.

8.2 Specify the expected final product(s) that will be prepared for the project (e.g., journal article, final report, etc.). If a method/SOP will be developed, specify the required format.

9. REFERENCES

Provide references either in the body of the text as footnotes or in a separate section.

Software Development Projects

A software development project typically involves software (and often hardware) systems development and/or database design, including maintenance.

NRMRL QAPP REQUIREMENTS FOR SOFTWARE DEVELOPMENT

PROJECTS

GENERAL REQUIREMENTS: Include cover page, distribution list, approvals, and page

Include cover page, distribution list, approvals, and page numbers.

0. COVER PAGE

Include the Division/Branch, project title, EPA technical lead, QA category, organization responsible for QAPP preparation, and date.

1. PROJECT DESCRIPTION AND OBJECTIVES

1.1 Describe the software and its intended application.

1.2 State the purpose of the project and list specific project objective(s).

2. ORGANIZATION AND RESPONSIBILITIES

2.1 Identify all project personnel, including QA, and related responsibilities for each participating organization, as well as their relationship to other project participants.

3. FUNCTIONAL REQUIREMENTS

3.1 Provide a list of the most important functions that the software system must address.

3.2 Identify requirements for functionality, external interfaces (includes graphical user interfaces and interfaces which are needed for other programs to call subroutines from the software, as applicable), performance, and design constraints. Each requirement should be uniquely identified and defined such that its achievement is capable of being objectively verified and validated.

3.3 Specify computer hardware and operating system requirements.

4. SYSTEM DESIGN

4.1 Provide an overview of the system design (e.g., block diagrams showing relationships between major program modules, hardware devices, and data input/output).

4.2 Describe the components and subcomponents of the software design, including databases and internal interfaces. The description should link the software structure to the functional requirements.

4.3 Provide the rationale for selecting the proposed hardware and software tools.

5. IMPLEMENTATION

5.1 Describe how a working software system is developed from the design specifications.

5.2 Describe how the requirements for functionality, external interfaces, performance, and design constraints will be verified and validated.

5.3 Describe how release and delivery of the product is managed, including versions for alpha and beta testing.

5.4 Describe the procedures for controlling, documenting, and archiving all significant changes to software and hardware.

5.5 Identify the archiving software used for controlling, documenting, saving, and recovering changes made to the source code.

6. VALIDATION, VERIFICATION, AND TESTING

6.1 Describe the testing strategy that will be used along with the procedures for each planned test.

Testing may include individual module tests, integration tests, system testing, acceptance testing, and alpha and beta testing.

6.2 Describe the procedure for checking the correctness of outputs.

6.3 Describe how it will be determined if the developed software product conforms to requirements, and whether the software product fulfills the intended use and user expectations. This includes analysis, evaluation, review, inspection, assessment, and testing of the software product and the processes that produced the product.

7. DOCUMENTATION, MAINTENANCE, AND USER SUPPORT

7.1 Specify the requirements for project documentation (e.g., requirements and design document, configuration maintenance plan, operations manual, source code, user¿s guide).

7.2 Describe the procedures for maintenance and user support when software or data generated by the project will be distributed outside NRMRL.

7.3 Define the methods and facilities used to maintain, store, secure, and document controlled versions and related artifacts of the identified software during all phases of the software life cycle.

8. REPORTING

8.1 List and describe the deliverables expected from each project participant.

8.2 Specify the expected final product(s) that will be prepared for the project (e.g., software, user documentation, user interface).

9. REFERENCES

Provide references either in the body of the text as footnotes or in a separate section.

Design, Construction, and Operation of Technology Projects

A technology design, construction, and operation project involves the engineering activities performed in support of the development of an environmental technology. ¿Environmental technology¿ is an all-inclusive term used to describe pollution control devices and systems, waste treatment processes and storage facilities, and site remediation technologies and their components that may be used to remove pollutants or contaminants from or prevent them from entering the environment. Usually, this term will apply to hardware-based systems; however, it can also apply to general methods or techniques used for pollution prevention, source reduction, or containment of contamination to prevent further movement of the contaminants.

NRMRL QAPP REQUIREMENTS FOR THE DESIGN, CONSTRUCTION,

AND OPERATION OF ENVIRONMENTAL TECHNOLOGY PROJECTS

GENERAL REQUIREMENTS: Include cover page, distribution list, approvals, and page

Include cover page, distribution list, approvals, and page numbers.

0. COVER PAGE

Include the Division/Branch, project title, revision number, EPA technical lead, QA category, organization responsible for QAPP preparation, and date.

1. PROJECT DESCRIPTION AND OBJECTIVES

1.1 State the purpose of the technology.

1.2 Describe the specific environmental technology components or system that are(is) to be designed, fabricated, constructed, and operated.

1.3 State the purpose of the project and list specific project objective(s).

2. ORGANIZATION AND RESPONSIBILITIES

2.1 Identify all project personnel, including QA, and related responsibilities for each participating organization, as well as their relationship to other project participants.

3. DESIGN OF SYSTEMS

3.1 Provide key design inputs, including initial design drawings, with clear identification of flow paths, process control/monitoring devices, etc., using standard engineering protocols.

3.2 Identify technical and quality specifications required for each component or for the complete system, both in materials of construction and in performance requirements.

3.3 Describe the process for documentation, review, and approval of design outputs.

3.4 Describe the system for documenting and controlling design changes.

4. CONSTRUCTION/FABRICATION OF SYSTEMS AND COMPONENTS

4.1 Describe calibration procedures for all measuring and test equipment used in construction/fabrication, including acceptance criteria.

4.2 Document the process for management, acceptance, and control of all construction materials.

4.3 Describe periodic reviews performed to verify conformity of the constructed system and components to design specifications.

4.4 Describe procedures for final inspection, acceptance, and documentation of completed systems and components.

5. OPERATION OF SYSTEMS

Specify documentation requirements for system operating instructions/guides, including:

¿ System startup procedures and requirements

¿ Appropriate controls for materials (including consumables)

¿ Calibration and controls for measurement and testing equipment

¿ Configuration management

¿ Operating procedures and parameters for specific components and systems configuration, including specified safety limits

¿ Routine inspection and testing requirements

¿ Spill, fire, and other hazard safety procedures

¿ Process equipment control and maintenance, including specifications during abnormal conditions for inspection and test situations and fault and emergency conditions;

¿ Special environments, time, temperature, or other factors affecting the quality of operation

¿ The skill, capability, and knowledge of operators needed to meet operational and quality requirements

6. VERIFICATION AND ACCEPTANCE OF SYSTEMS

Specify requirements for the review of design, construction, or operational outputs (e.g., other organizations, peer review), including acceptance criteria.

7. REPORTING

7.1 Describe the process for controlling and maintaining key design, construction, and operational records.

7.2 List and describe the deliverables expected from each organization responsible for design, construction, and operational outputs.

7.3 Specify the expected final product(s) that will be prepared for the project (e.g., journal article, final report, etc.).

8. REFERENCES

Provide references either in the body of the text as footnotes or in a separate section.

Research Model Development or Application Projects

A research model project is a study performed to develop a new model or apply an existing model to provide information to support non-regulatory environmental research or decision making.

NRMRL QAPP REQUIREMENTS FOR RESEARCH MODEL

DEVELOPMENT AND APPLICATION PROJECTS

GENERAL REQUIREMENTS: Include cover page, distribution list, approvals, and page numbers.

Include cover page, distribution list, approvals, and page numbers.

COVER PAGE (MODEL DEVELOPMENT AND MODEL APPLICATION)

0. Include the Division/Branch, project title, revision number, EPA technical lead, QA category, organization responsible for QAPP preparation, and date.

1. PROJECT DESCRIPTION AND OBJECTIVES (MODEL DEVELOPMENT AND

MODEL APPLICATION)

In this document, ¿project¿ can mean (a) development or substantial modification of a model for application to address a general problem; (b) application of an existing model (including minor modification to the existing model) to address a specific problem; or (c) a development or substantial modification and application of a model to address a specific problem.

1.1 State the purpose of the project and list the project objective(s). Indicate whether a new model will be developed or an existing model will be used.

1.2 Describe the problem, the data to be generated by the model, how the data will be used to address the problem, and the intended users of the data. Describe the environmental system/setting to be modeled, where the model will be applied, and the circumstances and scenarios to be considered for the modeled system.

2. ORGANIZATION AND RESPONSIBILITIES (MODEL DEVELOPMENT AND MODEL

APPLICATION)

2.1 Identify all project personnel, including QA, and related responsibilities for each participating organization, as well as their relationship to other project participants.

2.2 Include a project schedule that includes key milestones.

3. MODEL SELECTION (MODEL APPLICATION ONLY)

3.1 Discuss model selection with respect to how it will be used and how it is consistent with the project objectives. Include fundamental details such as whether the model will be used to predict the world beyond the model or in scenario analysis of the model itself. Describe the limits to where the model is applicable.

3.2 Provide a description of the model attributes/capabilities required for the project. This description should include hardware requirements and restrictions. Provide an overview of the candidate model attributes, including:

¿ model origin and its original purpose, if applicable

¿ model structure (e.g., stochastic vs. deterministic, structural framework)

¿ parameters and variables

¿ the algorithms and equations that have been developed to support the model theory, along with the sources of the algorithms

¿ spatial extent (individual, group, population)

¿ spatial resolution (location independent/dependent, dimensionality)

¿ temporal extent (length of modeling period)

¿ temporal resolution (time step)

¿ 3.3 Identify the model to be used or, if the model has not yet been selected, describe the process to be used for the selection of an existing model.

¿ 3.4 Identify specific requirements for application of the selected model for this specific purpose (e.g., current and appropriate data, parameter values, assumptions).

4. MODEL DESIGN (MODEL DEVELOPMENT ONLY)

4.1 Describe the conceptual model(s) for the system, including model parameters.

4.2 Identify algorithms and equations that have been developed to support the model theory, or if such equations are not already available, describe the process used to develop these equations.

4.3 Specify required sources for model databases and any requirements for these data (e.g., quality, quantity, spatial, and temporal applicability). If data sources are not currently known, describe the criteria used to identify sources. Describe how any data gaps will be filled.

5. MODEL CODING (MODEL DEVELOPMENT ONLY)

5.1 Discuss the requirements for model code development, where applicable.

5.2 Identify computer hardware and software requirements.

5.3 Discuss requirements for code verification.

6. MODEL CALIBRATION (MODEL DEVELOPMENT AND MODEL APPLICATION)

Calibration is the process of adjusting model parameters within physically defensible ranges until the resulting predictions give the best possible or desired degree of fit to the observed data.

Calibration should be applied each time the model is modified.

6.1 Discuss how the model will be calibrated.

6.2 Identify the type and source of data (e.g., new data, existing data, professional judgment, expert opinion elicitation) that will be used to calibrate the model, including any requirements for the data (quality, quantity, and spatial and temporal applicability). If data sources are not currently known, describe the criteria used to identify sources.

6.3 Specify acceptance criteria which need to be met for the difference between predicted and observed data during model calibration, where applicable. The statistical methods (e.g., goodness-offit, regression analyses) or expert judgment to be used should also be discussed.

7. MODEL VERIFICATION (MODEL DEVELOPMENT AND MODEL

APPLICATION)

Verification consists of comparing the predictions of a calibrated model with available data that were not used in the model development and calibration.

7.1 Discuss the approach to be used for model verification. Describe how the verification is appropriate based on the model¿s purpose. Identify the type and source of data (e.g., new data, existing data, synthetic test data sets, professional judgment, expert opinion elicitation) that will be used to verify the model. If data sources are not currently known, describe the criteria used to identify sources.

7.2 Discuss the characterization of model uncertainty (model framework, model input, and model applicability) and sensitivity (model application only).

7.3 Describe any requirements (quality, quantity, and spatial and temporal applicability) for the data that will be used to verify the model.

7.4 Describe the approach used to determine if the independent data verify the model predictions.

Specify the criteria which need to be met for the difference between predicted and observed data for the model to be considered to be verified. Discuss any statistical methods to be used (e.g., goodnessof-fit, regression analyses).

8. MODEL EVALUATION (MODEL DEVELOPMENT AND MODEL APPLICATION)

8.1 List and describe the qualitative or quantitative assessment process to be used to generate information to determine whether a model and its analytical results are of a quality sufficient for the intended use.

8.2 List and describe any independent/external evaluation and review of the model and model design, such as scientific peer review.

9. MODEL DOCUMENTATION (MODEL DEVELOPMENT AND MODEL

APPLICATION)

Specify the requirements for model documentation. Good documentation includes:

¿ final model description, final model…