ST PMO QASP_28 May 2020.pdf

QUALITY ASSURANCE SURVEILLANCE PLAN

FOR

Contractor Name

Contract # Date

1. Purpose of Contract:

The contractor, (insert name of contractor) has a requirement to provide Science & Technical (S&T) Program Management Office (PMO) Assistance & Advisory Support (A&AS). This support includes the required S&T PMO A&AS expertise for the Defense Threat Reduction Agency (DTRA) in support of the Research and Development (RD) Directorate which requires programmatic, financial and policy expertise across the chemical, biological, radiological, nuclear and high-yield explosives (CBRNE) spectrum and in the countering weapons of mass destruction (CWMD), countering improvised threats (CIT) and countering improvised threat networks (CTN) mission domains that RD supports. RD’s activities include all possible RDT&E funding and management activities and tasks, from 6.1 to 6.5 funding.

RD’s RDT&E activities, however, mainly focus on basic research through advanced component development and prototypes. RD’s activities beyond the advanced component development and prototype activities include system development and demonstration activities such as the Nuclear Arms Control Technologies (NACT) and other programs. In addition, RD directs and manages the Joint Science and Technology Office (JSTO) component of DoD’s Chemical and Biological Defense Program (CBDP), along with engaging in operational activities that are complementary to its R&D activities in order to support CCMD and customer requirements.

2. The requirement is to provide these services in compliance with applicable regulatory guidance. The effective date of the contract is from ________ (insert date) for a base period of one year with 5 consecutive one-year renewable options, thus possibly extending the period of performance to 6 April 2026.

3. Review Responsibilities: The RD-Nuclear Technologies Division will be designated as the Contracting Officer's Representative (COR) for this contract. As COR, Tracy Wentworth, RD-NTI is responsible for interfacing with the contractor in the performance of duties such as those set forth below.

4. Methods and Frequency:

The following process will be employed to collect, validate, consolidate and analyze the within the contractor team, across the government team and engagement with the S&T SME A&AS contractor and government team.

4.1 Surveillance will be accomplished in several methods: random basis; weekly, monthly, quarterly and bi-annual reviews. The method used for evaluation will be based upon document reviews, reports, observation, and received customer feedback.

4.2 Timely feedback to the contractor on unacceptable performance will be provided in writing by the COR.

4.3 The COR will be responsible for collecting and validating all Contractor Discrepancy Reports (CDR) and for each COR validated instance where the contractor’s performance was deemed inadequate and/or detrimental to the overall execution of the DTRA RD mission, and the contractor has not resolved the performance inadequacy. The COR shall also solicit and collect positive and negative comments relating to the contractor’s performance of the requirements in the PWS.

4.4 The contractor shall report to the COR in the QASR if it is notified of any validated CDRs and how they were addressed/resolved. The contractor shall also provide the COR with a report of the contractor’s performance relating to the requirements of the PWS in the monthly Program Status & Management Report.

4.5 The COR will be responsible for collecting and validating all Contractor Innovation & Continuous Improvement Reports (CICIRs) for each recommended innovation & continuous improvement recommendation for the DTRA RD mission. The COR shall review, validate and coordinate the approval and, where required, the implementation of the innovation & continuous improvement recommendation.

4.6 The contractor shall also provide the COR with a report of the contractor’s innovation and continuous improvement performance relating to the requirements and activities of the PWS in the monthly Program Status & Management Report.

4.7 A summary of any performances that did not meet the terms and conditions of the contract shall be prepared by the COR and forwarded to the Contracting Officer (KO).

4.8 The surveillance report and a summary of the surveillance logs will be prepared by the COR and submitted to the KO annually. The COR may submit out-of-cycle reports to the KO to support claims of unacceptable performance or when there is a significant change in performance that alters the assessment in one or more evaluation areas.

4.9 The Surveillance Delivery Summary/Performance Surveillance Checklist provided in enclosure (1) will be used by the COR in monitoring this contract.

5. Performance Areas:

The following performance areas will be evaluated as described. However, they are not all inclusive, and any service on this contract may be evaluated at any time; all services provided on this contract are subject to inspection and evaluation at any time.

5.1 Performance Objective (PO) #1, Quality of Program Management, Financial Management and Policy Support: The contractor will be evaluated on the Quality of Program Management, Financial Management and Policy Support. It is a measurement of how the contractor performs requirements as an integrated team member, ensuring quality, consistent, relevant, timely, accurate, comprehensive products.

Contractor’s effectiveness will be rated on 1) accuracy and completion requirements as an integrated team member; 2) accuracy, consistency and comprehensiveness of products developed as an integrated team member; 3) frequency and timeliness of products developed as an integrated team member. This performance area will be reviewed on a quarterly basis.

5.2 PO#2, Integration of Program Management, Financial Management and Policy Support:

The contractor will be evaluated on the Integration of Program Management, Financial Management and Policy Support. This performance metric is a measure of performance of contract requirements and activities as specified in the PWS or surge statements of work. It is also a measurement of how the contractor performs requirements and activities as an integrated team member, in collaboration and engagement with the S&T SME A&AS contractor and the government team, ensuring unconstrained flow of program data and information to effectively and efficiently complete tasks and foster a collaborative environment to ensure requirements are completed with a high standard of quality.

Contractor’s effectiveness will be rated on 1) timely submission of tasks and requirements as an integrated team member; 2) appropriate collaboration and engagement with S&T SME A&AS and government team; 3) appropriate sharing of program data and information to effectively and efficiently complete tasks as an integrated team member; 4) fostering of collaborative environment to ensure requirements completed with high standard of quality, as an integrated team member. This performance area will be reviewed on a quarterly basis.

5.3 PO #3, Innovation and Continuous Improvement of Program Management, Financial Management and Policy Management: The contractor will be evaluated on the Innovation and Continuous Improvement of Program Management, Financial Management and Policy Management. This performance metric is a measure of performance of contract requirements and activities as specified in the PWS or surge statements of work. It is also a measurement of how the contractor performs active innovative and continuous improvement of existing processes, procedures and policies impacting the quality and timeliness of fulfilling requirement, along with ensuring the latest programmatic support capabilities, processes and procedures are investigated to determine effectiveness and value-added to the RD program management program.

Contractor’s effectiveness will be rated on 1) timeliness, quantity and quality of innovation and continuous improvement recommendations of existing processing, procedures and policies timeliness of draft plan submission for coordination and 2) effectiveness, value-added, quality and quantity of recommendations for programmatic support capabilities, processes and procedures. This performance area will be reviewed on a quarterly basis.

5.4 PO #4, Provide, Anticipate, Recruit and Retain Workforce Expertise: The contractor will be evaluated on its ability to Provide, Anticipate, Recruit and Retain Workforce Expertise. This is a measure of the contractor’s ability to fill existing, new (additional) or restructured positions and promptly fill urgent vacancies due to departures for existing positions (not restructured).

Contractor’s effectiveness will be rated on 1) timeliness of filling validated urgent fills; 2) timeliness of filling currently existing positions and 3) retention of contractor personnel. This performance area will be evaluated on a monthly basis.

5.5 PO #5, Provide a Trained Workforce: The contractor will be evaluated on Provide a Trained Workforce. This is a measure of the contractor’s ability to conduct workforce training required by the Government.

Contractor’s effectiveness will be rated on 1) number of contractor personnel that have completed all required workforce training. This performance area will be reviewed on a quarterly basis.

5.6 PO #6, Management: The contractor will be evaluated on Management. This is a measure of the contractor’s ability to submit CDRL requirements as required.

Contractor’s effectiveness will be rated on 1) products delivered by the deadline, with all suspense’s met; and 2) accuracy of products and correctness of content. This performance area will be reviewed on a quarterly basis.

6. Evaluation Criteria:

The evaluation criteria for the performance metrics are listed in the below tables:

PO #1: Quality of Scientific, Technical, Managerial, and Logistical Support Metrics

1. Qualitative Assessment:

CPAR Assessing Official Comments Quality Section of CPARS

2. Quantitative Assessment:

Exceptional 0 Validated Complaints Satisfactory 1-2 Validated Complaints Unsatisfactory 3 or more Validated Complaints

PO #2: Integration of Program Management, Financial Management and Policy Support

Metrics Exceptional 0 Validated Complaints Satisfactory 1-2 Validated Complaints Unsatisfactory 3 or more Validated Complaints

PO #3: Innovation and Continuous Improvement of Program Management, Financial

Management and Policy Management Metrics Exceptional 2-I&CO Suggestion per quarter OR 2 I&CO Suggestions Implemented per year Satisfactory 1 I&CO Suggestion per quarter OR 1 I&CO Suggestions Implemented per year Unsatisfactory 0- I&CO Suggestion per quarter OR 0 I&CO Suggestions Implemented per year

PO #4: Provide, Anticipate, Recruit and Retain Workforce Expertise Metrics

Exceptional No more than 2 validated urgent positions left unfilled after 30 days following the urgent fill designation (with the exception of COR validated extenuating circumstances).

Satisfactory 3 or 4 validated urgent positions left unfilled after 30 days following the urgent fill designation (with the exception of COR validated extenuating circumstances).

Unsatisfactory 5 or more validated urgent positions left unfilled after 30 days following the urgent fill designation (with the exception of COR validated extenuating circumstances).

Exceptional No more than 10% of all funded currently existing positions left unfilled more than 90 days following date of vacancy approval (with the exception of COR validated extenuating circumstances).

Satisfactory No more than 20% of all funded currently existing positions left unfilled more than 90 days following date of vacancy (with the exception of COR validated extenuating circumstances).

Unsatisfactory No more than 30% of all funded currently existing positions left unfilled more than 90 days following date of vacancy (with the exception of COR validated extenuating circumstances).

Exceptional 90% or greater retention for the quarter. Contract personnel that left because of government level of effort reductions or funding reductions, or for internal moves inside the contract will not count against this metric.

Satisfactory 80% or greater retention for the quarter. Contract personnel that left because of government level of effort reductions or funding reductions, or for internal moves inside the contract will not count against this metric.

Unsatisfactory 70% or less retention for the quarter. Contract personnel that left because of government level of effort reductions or funding reductions, or for internal moves inside the contract will not count against this metric.

PO #5: Provide a Trained Workforce

Satisfactory 95% or greater of workforce completes required work training in DTRC and other performance locations as specific in the DD254

Unsatisfactory Less than 95% of workforce completes required work training in DTRC and other performance locations as specified in the DD254

PO #6: Management Metric Satisfactory

100% CDRL (including monthly Program Status & Management Report, QASR, monthly financial progress report, Scientific and Technical Reports as required, Presentation Material as required, and a final contract summary report) delivered on time as specified in contract

Unsatisfactory Less than 100% COR required CDRLs(including monthly Program Status & Management Report, QASR, monthly financial progress report, Scientific and Technical Reports as required, Presentation Material as required, and a final contract summary report) delivered on time as specified in contract

7. Surveillance Documents: Documentation used and referenced to perform surveillances will consist of Contract Discrepancy and Suggestion Reports listed below for the Quality of Program Management, Financial Management and Policy Support metric, Integration of Program Management, Financial Management and Policy Support), the Contract Innovation & Continuous Improvement Suggestion Report for the Innovation & Continuous Improvement Support metric, the Performance Work Statement, Customer Feedback, the quarterly Contract Quality Assurance Surveillance Reports and the Surveillance Delivery Summary/Performance Surveillance Checklist. Documentation shall include the date, time, and location of the surveillance; applicable remarks to the surveillance; and signature of the COR. Unsatisfactory ratings must be signed by the contractor. The following metric reports will be used:

For reporting PO #1: Quality of Program Management, Financial Management and Policy Support metric and PO #2: Integration of Program Management, Financial Management and

Policy Support issues, the below Contract Discrepancy Report will be used:

CONTRACT DISCREPANCY REPORT DESCRIPTION OF SERVICE or STANDARD:

SURVEY PERIOD:

DATE/TIME COMPLAINT RECEIVED:

SOURCE OF COMPLAINT: (NAME)

(ORGANIZATION)

(PHONE NUMBER)

(EMAIL ADDRESS)

NATURE OF COMPLAINT:

RESULTS OF COMPLAINT INVESTIGATION:

DATE/TIME CONTRACTOR INFORMED OF COMPLAINT:

CORRECTIVE ACTION TAKEN BY CONTRACTOR:

RECEIVED AND VALIDATED BY:

PREPARED BY:

DATE:

The PO #3: Innovation and Continuous Improvement of Program Management, Financial Management and Policy Support metric uses the below Contractor Innovation & Continuous Improvement Report to track and manage recommended innovation improvement recommendations for the DTRA RD mission.

CONTRACT INNOVATION & CONTINIOUS IMPROVEMENT SUGGESTION REPORT

DESCRIPTION OF SUGGESTION:

SURVEY PERIOD:

DATE/TIME SUGGESTION RECEIVED:

SOURCE OF SUGGESTION: (NAME)

(ORGANIZATION)

(PHONE NUMBER)

_ (EMAIL ADDRESS)

INNOVATIVE & CONTINIOUS IMPROVEMENT SUGGESTION AND

RATIONALE:

RESULTS OF SUGGESTION ANALYSIS (VALUE-ADDED) PERFORMED BY GOVERNMENT:

DATE/TIME CONTRACTOR INFORMED OF ANALYSIS/IMPLEMENTAITON DECISION:

EXECUTION ACTION TAKEN BY CONTRACTOR:

RECEIVED AND APPROVED FOR IMPLEMENTATION BY:

PREPARED BY:

DATE:

The following process will be employed to collect, validate, consolidate, and analyze the PO #4:

Provide, Anticipate, Recruit and Retain Workforce Expertise metric.

The contractor shall collect workforce data quarterly for each RD Department. The contractor shall report this performance data to the COR in the QASR as an EXCEL spreadsheet attachment (format and data may be modified as directed by the COR). The contractor is required to report the following information for each position requirement: RD Department, Position, If Surge/Urgent Fill is Required include Days to Fill, Name, Status of Fill, Estimated Start Date, Note if Consultant or Part-time Employee, Note if Existing/New/Restructured/Deleted, and Note any Mitigating or Extenuating Circumstances.

The following process will be employed to collect, validate, consolidate and analyze PO #5: Provide a Trained Workforce metric.

The contractor shall track and report to the COR in the first QASR provided after the anniversary of contract award the status of annual security training, annual counterintelligence training, annual TS/SCI awareness training (for those individuals with TS/SCI clearances) per the DD254.

The following process will be employed to collect, validate, consolidate, and analyze PO #6:

Management metric.

The COR shall note if the CDRL deliveries in this contract (including monthly Program Status & Management Report, QASR, monthly financial progress report, Scientific and Technical Reports as required, Presentation Material as required, and a final contract summary report) are submitted on time.

8. Rating: All contractor tasks will be rated per the Evaluation Criteria in Section in the Surveillance Delivery Summary/Performance Surveillance Checklist. At the end of the contract or contract order, the KO will collect the Surveillance Delivery Summary/Performance Surveillance Checklist and all supporting documentation (Contractor Discrepancy Reports and Contractor Innovation & Continuous Improvement Reports) and provide an overall rating for each of the performance areas using the evaluation criteria in Paragraph 5 of this plan.

9. Recording Observations: The Surveillance Delivery Summary/Performance Surveillance Checklist will be used to document observations and problems and rate the individual contractor tasks. The Performance Assessment Report (PAR) Form will be used to record each observed performance issue noted, for performance areas identified in Paragraph 4 or any other tasks identified in the PWS. The Surveillance Checklists and PARs become the formal record for reference. The contractor is told each time a performance issue is found during the observations.

10. Potential Unacceptable Performance: The COR must identify the specific service that is unacceptable and the possible causes and ask a number of questions that will likely identify the source of the problem. The COR shall attempt to resolve the problem with the contractor.

10.1 The COR shall forward the original Surveillance Delivery Summary/Performance Surveillance Checklist and the PAR documenting the observation and attempted solution (along with an evaluation of the results) to the KO, when a specific service becomes unsatisfactory during a surveillance period. The surveillance reports will be used to provide an overall rating for the performance areas identified in Paragraph 4 of this plan.

10.2 The contractor shall complete a response to the unacceptable evaluation if requested by the KO or by its own choice.

11. Taking Action:

11.1 The COR shall evaluate the contractor's performance and document any non-compliance, but only the KO may take action against the contractor for an unacceptable rating.

11.2 When the contractor's performance is unacceptable and a formal action is indicated, the COR and KO will determine what action is appropriate for the specific circumstances.

The KO will determine the final action.

12. Records: During the course of the contract, the COR retains a copy of all Surveillance Delivery Summary/Performance Surveillance Checklists and Performance Assessment Report (PAR) Forms, with performance issues noted. The Surveillance Delivery Summary/Performance Surveillance Checklist and PARs become the formal record for reference. At the end of the surveillance period, the COR will forward all inspection records to the KO for inclusion in the contract file.

Performance Objective Standard Performance

Threshold Method of

Surveillance PO #1, Quality of Program Management, Financial Management and Policy Support (5.1

– 5.2)

Contractor’s effectiveness will be rated on 1) accuracy and completion requirements as an integrated team member; 2) accuracy, consistency and comprehensiveness of products developed as an integrated team member; 3) frequency and timeliness of products developed as an integrated team member.

Exceptional: 0 Validated Complaints

Customer Feedback, COR periodic inspection

Satisfactory: 1 – 2 Validated Complaints

Customer Feedback, COR periodic inspection

Unsatisfactory: 3 or more Validated Complaints

Customer Feedback, COR periodic inspection

PO#2, Integration of Program Management, Financial Management and Policy Support (5.1 – 5.2)

Contractor’s effectiveness will be rated on 1) timely submission of tasks and requirements as an integrated team member; 2) appropriate collaboration and engagement with S&T SME A&AS and government team; 3) appropriate sharing of program data and information to effectively and efficiently complete tasks as an integrated team member; 4) fostering of collaborative environment to ensure requirements completed with high standard of quality, as an integrated team member.

Exceptional: 0 Validated Complaints

Customer Feedback, COR periodic inspection

Satisfactory: 1 – 2 Validated Complaints

Customer Feedback, COR periodic inspection

Unsatisfactory: 3 or more Validated Complaints

Customer Feedback, COR periodic inspection

PO #3, Innovation and Continuous Improvement of Program Management, Financial Management and Policy

Contractor’s effectiveness will be rated on

1) timeliness, quantity and quality of innovation and continuous improvement recommendations of existing processing, procedures and policies timeliness of draft plan submission for coordination and 2) effectiveness, value-added, quality and quantity of recommendations for

Exceptional: 2-I&CO Suggestion per quarter OR 2 I&CO Suggestions Implemented per year

COR periodic inspection

Satisfactory: 1- I&CO Suggestion per quarter OR 1 I&CO Suggestions

Management (5.1 – 5.2) programmatic support capabilities, processes and procedures.

Implemented per year

Unsatisfactory: 0 I&CO Suggestion per quarter OR o I&CO Suggestions Implemented per year

PO #4, Provide, Anticipate, Recruit and Retain Workforce Expertise (5.1 – 5.2)

Contractor’s effectiveness will be rated on timeliness of filling validated urgent fill.

Exceptional: No more than 2 validated urgent positions left unfilled after 30 days following the urgent fill designation (with the exception of COR validated extenuating circumstances).

COR periodic inspection

Satisfactory: No more than 3 -4 validated urgent positions left unfilled after 30 days following the urgent fill designation (with the exception of COR validated extenuating circumstances).

Unsatisfactory: 5 or more validated urgent positions left unfilled after 30 days following the urgent fill designation (with the exception of COR validated extenuating circumstances).

Contractor’s effectiveness will be rated on timeliness of filling currently existing positions.

Exceptional: No more than 10% of all funded currently existing positions left unfilled more than 90 days following date of vacancy approval (with the exception of COR validated extenuating circumstances).

Satisfactory: No more than 20% of all funded currently existing positions left unfilled more than 90 days following date of vacancy approval (with the exception of COR validated extenuating circumstances).

Unsatisfactory: No more than 30% of all funded currently existing positions left unfilled more than 90 days following date of vacancy approval (with the exception of COR validated extenuating circumstances).

Contractor’s effectiveness will be rated on retention of contractor personnel.

Exceptional: 90% or greater retention for the quarter. Contract personnel that left because of government level of effort reductions or funding reductions, or for internal moves inside the contract will not count against this metric.

Satisfactory: 80 – 89% retention for the quarter.

Contract personnel that left because of government level of effort reductions or funding reductions, or for internal moves inside the contract will not count against this metric.

Unsatisfactory: 70% and below retention for the quarter. Contract personnel that left because of government level of effort reductions or funding reductions, or for internal moves inside the contract will not count against this metric.

PO #5, Provide a Trained Workforce (5.1

– 5.2, 7.0)

Contractor’s effectiveness will be rated on 1) number of contractor personnel that have completed all required workforce training.

Satisfactory: 95% or greater of workforce completes required work training in DTRC and other performance locations as specific in the

DD254

COR periodic inspection

Unsatisfactory: Less than 95% of workforce completes required work training in DTRC and other performance locations as specific in the

DD254

PO #6, Management (5.1 – 5.2)

Contractor’s effectiveness will be rated on 1) products delivered by the deadline, with all suspense’s met; and 2) accuracy of products and correctness of content.

Satisfactory: 100% CDRL (including monthly Program Status & Management Report, QASR, monthly financial progress report, Scientific and Technical Reports as required, Presentation Material as required, and a final contract summary report) delivered on time as specified in contract

COR periodic inspection

Unsatisfactory: Less than 100% CDRL (including monthly Program Status & Management Report, QASR, monthly financial progress report, Scientific and Technical Reports as required, Presentation Material as required, and a final contract summary report) delivered on time as specified in contract