9_Document_Quality_Guidance_V1_20190830.docx

Document Quality Guidance

30 August 2019

Introduction

The purpose of this guidance is to provide a framework for establishing a consistent and agreed upon process for preparing and reviewing documents. This approach should be tailored to meet the specific needs of the Optimized Remediation Contact (ORC) as defined by the Air Force (AF) and United States Army Corps of Engineers (USACE), regulators, and the Contractor; as well as the number and complexity of documents as defined by the Contractors approach to meet the performance objectives (POs) in the Performance Work Statement (PWS). This guidance does not replace the Contractor’s Quality Control/Quality process (QC/QP) or the Quality Assurance Surveillance Plan (QASP); rather, its purpose is to ensure that there is agreement among all parties on the approach to document review, the number of reviews and the response required (which may vary according to document type), and the method of the responses required.

The Contractor is responsible for the quality of all work performed under the contract regardless of whether the work is performed by Contractor employees or by subcontractors. This will ensure smooth flow of reviews, acceptance, and payment. The Contractor measures quality through their QC/QP program, as documented in the quality management section of the Project Management Plan (PMP). The Contractor’s QC/QP sets forth the staffing and procedures for self-inspecting the quality, timeliness, responsiveness, customer satisfaction, and other performance standards in the PWS and QASP while reporting the inspection results to the designated government representative.

Establishing and following a QP throughout the document development and review process, from scoping to document finalization, must be a high priority for Contractors. Up front communication and submittal of high quality documents has the ability to build trust with the AF, USACE, and regulators resulting in less burdensome and costly rounds of response to comments (RTC), document revision, re-review, etc. Submittal of poor quality documents leads to increased resistance to innovative approaches by the AF and regulators, invites increased scrutiny by all reviewers, and contributes to a breakdown in communication. This could stop the progress of the work, resulting in delayed payments to the Contractor, delayed AF achievement of performance goals, and reductions in life-cycle cost (LCC).

Applicability

This approach applies to all technical plans and documents unless approved by the AF and USACE. The document preparation and review process described in this guidance is an example; however, the specific process for each installation may vary. The process, number of document versions, and specific names of the documents (i.e., Draft, Draft Final, Version 1, Version 2, etc.) should be discussed at the contract kick-off meeting. Additionally, certain documents may follow a more streamlined process, for example, documents that do not require regulatory review and approval.

Note: All documents submitted shall be complete; substantively correct and consistent with applicable laws, regulations, Department of Defense (DoD)/AF policies; in proper format; and free of grammatical and typographical errors.

PWS Quality Requirements

The following sections of the PWS are related to the contractor’s responsibility for preparing and delivering quality documents. This is not inclusive of all applicable sections, and should not substitute for review of the contract.

PWS Section 6.2 Technical Plans and Reports The Contractor shall provide technical plans, explosive safety documents, and reports, as well as various other deliverables. The Contractor shall complete these documents according to the most appropriate industry standard, or where applicable, any federal or state guidelines.

Technical plans and reports are required to demonstrate that POs and milestones are being met and that payment is appropriate.

PWS Section 8.1.1 Scope Discussion and Limits The Contractor shall be responsible for establishing a QA/QP to ensure documents are of an acceptable quality and do not contain typos, formatting issues, contradictory information in tables and text, etc.

PWS Section 9.0 Quality Assurance Surveillance Plan The AF and USACE will utilize the QASP to define the procedures and guidelines to evaluate the technical and quality performance of the Contractor in accordance with this PWS.

Document Preparation and Review Process

The following describes the document preparation and review process.

Note: The version names used in this guidance document may vary based on the installation and regulatory structure. Any deviation to the process needs to be discussed in advance and agreed to by the AF, USACE, and the Contractor.

DOCUMENT SCOPING

Prior to preparing the document, the Contractor coordinates a scoping meeting with the AF and USACE. The scoping meeting is a systematic planning meeting in accordance with the Uniform Federal Policy for Quality Assurance Project Plans (UFP-QAPP) process[footnoteRef:2], and should integrate technical and quality control aspects of a project including planning, implementation, assessment, and corrective actions. The Contractor shall take meeting minutes and ensure delivery of the final minutes to the entire team. The meeting should establish: [2: Reference: https://www.epa.gov/fedfac/uniform-federal-policy-quality-assurance-project-plans-training-materials]

1) Agreement on site history and problem definition;

2) Purpose and scope of the document;

3) Document format;

4) Data required to make decisions;

5) Decision pathways (if-then statements);

6) Identification of any data needs; and

7) Resolution for issues.

If needed, the AF coordinates a scoping meeting with the regulators attended by the AF, USACE, and the Contractor.

If Legal review is required based on Air Force Instruction (AFI) 32-7020, the Contractor shall keep the AF and USACE notified of the schedule for submitting the Draft document so the AF and USACE can coordinate with AF Legal’s schedule to ensure timely reviews.

DRAFT DOCUMENTS

Draft Documents are internal documents that are submitted for AF and USACE review and approval. In all respects, Draft documents shall be complete; substantively correct and consistent with applicable laws, regulations, DoD/AF policies; in proper format; and free of grammatical and typographical errors. Draft documents shall not be submitted to regulatory agencies for review. Below is the preparation and review process for Draft Documents:

The Contractor prepares Draft document and completes QC/QP prior to submitting the document to the AF and USACE.

If there is an extended time between the scoping meeting(s) in Section 2.1 and the completion of the Draft document, the Contractor may coordinate a meeting or provide Meeting Minutes to the AF and USACE to recap the purpose of the document and the agreements made in the scoping meeting(s).

The Contractor submits the Draft to the AF and USACE for review through the USACE Contracting Officer’s Representative (COR).

Ensure that the timing of the submission of the Draft is coordinated with AF Legal, as needed.

AF and USACE conduct a quick quality screening of the Draft to determine if the document is ready for technical and legal review. If the Draft does not substantively meet the following criteria, the Draft will be rejected: (1) overall free of grammatical, typographical, and figure/graphical errors; (2) in the agreed upon format; and (3) adheres to the agreements made during the scoping meeting. Based on the quality screening, the AF and USACE will either proceed immediately to the technical document review or reject the Draft.

· If the Draft passes the quick quality screening, proceed to Step 2.2.4.

· If the Draft is rejected, the USACE COR sends a rejection notice to the Contractor. The AF and USACE stops all technical review while the Contractor revises the Draft and resubmits. When the Draft is resubmitted, the document review starts again at Step 2.2.2.

AF and USACE conducts a thorough technical and legal review (if applicable) including determining if the document is:

1) Substantively correct with sufficient lines of evidence to support the conclusions;

2) Consistent with applicable laws, regulations, DoD/AF policies, etc.

The below tables are two different examples of comment categorization which have proven useful to facilitate review of comments. The AF, USACE, and Contractor should discuss and agree to the comment categories that will be utilized during the kick-off meeting.

Table 1 – Comment Categories

Table II – Comment Categories Alternate

USACE COR consolidates all AF and USACE comments and sends the Contractor the consolidated comments.

RESPONSES TO COMMENTS – DRAFT DOCUMENTS

The Contractor reviews the AF and USACE comments.

If needed, the Contractor coordinates a meeting with applicable AF and USACE reviewer(s) to clarify comments and discuss the Contractor’s initial responses in the Response to Comments (RTC) matrix.

In the event the AF and USACE reviewers fundamentally disagree with key elements of the document, for example, lines of evidence toward a specific conclusion, it is recommended that the Contractor plan an on board review of the document where the issues can be discussed in person.

The Contractor develops Draft RTCs based on the comments, and submits the RTCs to the AF and USACE through the USACE COR.

AF and USACE review the Draft RTCs. AF and USACE either respond to the RTCs or accept the RTCs.

· If the AF and USACE accepts RTCs, skip to step 2.3.5.

· If the AF and USACE responds to the RTCs, USACE consolidates all responses and COR sends the responses to the Contractor.

The Contractor reviews AF and USACE responses to RTCs, and revises RTCs as needed. The Contractor resubmits the RTCs in step 2.3.2. If there is continued disagreement, the Contractor coordinates a meeting with applicable AF and USACE reviewer(s) to resolve issues.

USACE COR sends the Contractor RTC acceptance and approval to submit the subsequent version of the document. Based on the quantity and content of comments and responses, the USACE COR’s email will specify if the next submission of the document should be a revised Draft rather than proceeding to the Draft Final (this is either the revised Draft or Draft based on the email in step 2.2.6).

If the next version is a revised Draft, resubmit the document in section 2.2.

Note: Redline versions will be utilized as needed as indicated by the COR.

APPROVAL OF DRAFT DOCUMENTS

Once comments have been resolved, revisions incorporated, and the AF and USACE are confident that the quality and content of the document is ready to be reviewed by the regulators, the COR sends the Contractor a notice of the AF and USACE Approval of the Draft.

The Contractor may submit an invoice for the Draft if there is an applicable milestone in the approved milestone payment schedule (MPS).

DRAFT FINAL DOCUMENTS

A Draft Final Document is submitted for regulatory agency. Below is the preparation and review process for Draft Final Documents:

The Contractor relabels the Approved Draft as the Draft Final in preparation of submittal to the regulators.

The Contractor submits the Draft Final to the AF and USACE through the COR.

The AF Restoration Program Manager (RPM) signs the transmission memo and submits the Draft Final to the regulators.

Regulators review the Draft Final and submit comments to AF RPM, and the RPM provides regulators’ comments to USACE. The USACE COR sends Draft Final comments to the Contractor.

RESPONSES TO COMMENTS – DRAFT FINAL DOCUMENTS

The Contractor reviews the regulators comments.

If needed, the Contractor coordinates a meeting with AF and USACE to clarify regulators comments and discuss the Contractor’s initial responses. If additional clarification or discussion with the regulators is required, the RPM coordinates a meeting with the regulators. The AF, USACE, and the Contractor attend the meeting.

The Contractor develops Draft Final RTCs, and submits the RTCs to the AF and USACE.

AF and USACE review the Draft Final RTCs. AF and USACE either respond to the RTCs or accepts the RTCs.

· If the AF and USACE accept RTCs, skip to step 2.6.5

· If the AF and USACE respond to the RTCs, USACE consolidates all responses, and COR sends the responses to the Contractor.

The Contractor reviews AF and USACE responses to RTCs, and revises RTCs as needed. The Contractor resubmits the RTCs in step 2.6.3. If there is continued disagreement, the Contractor coordinates a meeting with AF and USACE to resolve the issues.

COR sends the Contractor RTC acceptance, and RPM sends RTCs to regulators.

Regulators review RTCs. Regulators either respond to the RTCs or accept the RTCs.

· If the regulators accept RTCs, skip to step 2.6.7.

· If the Regulators respond to the RTCs, the RPM provides regulators’ comments to USACE. The COR sends comments to the Contractor. The process returns to step 2.6.1.

The Contractor revises the Draft Final and submits a revised Draft Final for AF and USACE review.

AF and USACE review revised Draft Final to ensure regulators comments are resolved. Once AF and USACE confirm revised Draft Final incorporates regulators comments, the RPM resubmits the document to regulators (step 2.5.3).

APPROVAL OF FINAL DELIVERABES

Final Documents are the version of the document that has received AF, USACE, and regulatory concurrence, where applicable. Below is the process for Final Documents:

Regulators send an approval letter to RPM. The document becomes the final version and the Contractor relabels the Approved Draft Final as the Final.

If the document requires signatures, the document goes through the signature process.

Once the document is signed (if applicable), or is final with AF, USACE, and regulatory approval (as applicable), the Contractor may submit an invoice for the Final if there is an applicable milestone in the approved MPS.

Conflict Resolution

The conflict resolution process should adhere to the following business rules.

1. Conflicts should be resolved at the lowest level possible, but must be made by persons with authority to make the decision.

2. Decisions should be made in the meeting whenever possible and meeting minutes should be taken in order to document decisions, changes, and action items. Meeting Minutes should be distributed to the entire AF, USACE, and Contractor team.

3. Small groups of decision-makers should be used where possible.

4. If the conflict cannot be resolved at the current level, it should be elevated to the next level.

Note: The Contractor shall take no direction from any Government employee or any person other than the Contracting Officer on anything that may change the terms and conditions of the contract action, the scope, or any change that impacts the price or schedule.

Version 1: 23 August 2019 1

Version 1: 30 August 2019 2