QASP 24 Jan 11.doc

“Source Selection Information - - See FAR 2.101 and 3.104.”

Defense Information Systems Agency

Test & Evaluation

Mission Support Service

Quality Assurance Surveillance Plan 24 January 2011

TABLE OF CONTENTS

11.0 PURPOSE

12.0 AUTHORITY

13.0 SCOPE

14.0 GOVERNMENT RESOURCES AND RESPONSIBILITIES

14.1 Contracting Officer

24.2 Contracting Officer’s Representative (COR)

24.3 Task Monitor (TM)

35.0 QUALITY SURVEILLANCE AND ASSESSMENTS

35.1 DISA T&E MSS Quality Performance Standards

55.2 Measures of Contractor Performance

65.2.1 Performance Measure #1: Management Effectiveness

6Performance in the area of Management Effectiveness will consider the following:

75.2.2 Performance Measure #2: Technical Performance

75.2.3 Performance Measure #3: Quality of Work

7Performance in the area of Quality of Work will consider the following:

85.2.4 Performance Measure #4: Timeliness

8Performance in the area of Timeliness will consider the following:

85.2.5 Performance Measure #5: Cost Control

8Performance in the area of Cost Control will consider the following:

96.0 MONTHLY PERFORMANCE ASSESSMENT REPORT

107.0 PERFORMANCE STANDARDS

1.0 PURPOSE

This Quality Assurance Surveillance Plan (QASP) is a Government developed and applied guideline to ensure systematic quality assurance methods are used in performance-based services contracts and any subsequent tasks issued for the Defense Information Systems Agency (DISA) Test and Evaluation (T&E) Mission Support Services (MSS). The DISA T&E MSS will be centrally managed by the Business Division. This QASP establishes procedures to utilize in ensuring the contractor performs in accordance with the basic contract Performance Work Statement (PWS), provides quality products in response to requirements, and the Government only pays for acceptable levels of service.

2.0 AUTHORITY

Authority for issuance of this QASP is provided under Federal Acquisition Regulation (FAR) 46.4, which provides for inspections and acceptance of articles, services, and documentation specified by the Contracting Officer (CO) or duly authorized representative by appointment.

3.0 SCOPE

The contractor is responsible for management of support services and the quality control requirements necessary to meet the standards set forth by the basic contract and individual task proposals. The QASP outlines the Government’s plan to monitor and inspect the Contractor’s work in fulfillment of the DISA T&E MSS contracts. Use of the QASP will provide the Government with reasonable assurance that efficient methods and effective cost control measures are used. The Government reserves the right to modify the QASP at any time during the contract period of performance, through modification to the contract.

4.0 GOVERNMENT RESOURCES AND RESPONSIBILITIES

The Government is responsible for implementing this QASP. The DISA T&E MSS Service Manager (SM) manages all activities for the DISA T&E MSS program. The DISA T&E MSS CO is the only person authorized to enter into, administer, or terminate contract actions for this program. The Government resources under this QASP and associated responsibilities are identified below:

4.1 Contracting Officer

A person duly appointed with the authority to enter into, administer, or terminate contract actions such as basic awards or task orders and make related determinations and findings on behalf of the Government.

Responsibilities: Ensure performance of all necessary actions for effective contracting, ensure compliance with the terms of the basic contract, and safeguards the interest of the United States in the contractual relationship. Assure the contractor receives impartial, fair, and equitable treatment under the basic contract. Determine the final adequacy of contractor performance.

4.2 Contracting Officer’s Representative (COR)

An individual designated in writing by the CO to act as the authorized representative to assist in the administration of specific contracts, as specified by appointment. The source of authority for the COR is the CO.

Contract COR Assessment Responsibilities:

· Review Monthly Performance Assessment reports submitted for individual Task Orders and validate supportability of any identified performance issues

· Consolidation of monthly contract quality assessment data

· Ensure monthly reports completed and available for contractor review

· Coordinate and conduct quarterly review of contractor performance

· Input Contractor Performance Assessment Reporting Systems (CPARS) data based on consolidated Monthly Performance Assessment reports for each task order with a total projected value of greater than $1 million dollars, and a consolidated contract level report annually

· Work any recommended program change of contractual scopes, such as those that may effect contract price, terms, or conditions with the DISA T&E MSS SM and the CO

4.3 Task Monitor (TM)

An individual identified by the CO and designated in writing by the CO to act as the authorized functional representative on individual task orders and to assist with the administration, execution and management of task orders, as specified by appointment letter from the CO. The TM shall be a Government employee in the functional organization responsible for supporting the COR in the management of all day-to-day tasks associated with the execution of the task order. The TM shall not perform any contractual functions but shall act as the subject matter expert for all changes in requirements and the assessment of daily performance throughout the execution of the contract. The TM detailed assessment responsibilities are defined below.

TM Assessment Responsibilities:

· Complete Monthly Performance Assessment report for each active task order

· Provide supporting data for all identified performance issues as directed by the COR

· Appropriately address performance issues, elevating for action as needed

· Participate in COR/Contract Program Management (PM) Review of Monthly report as required.

· Participate in Quarterly Review of Contractor Performance as required.

· Assist the COR with technical administration of individual task orders and assure proper Government surveillance of the contractor performance.

· Assist COR in identification of recommended contractual or task order change of scopes.

5.0 QUALITY SURVEILLANCE AND ASSESSMENTS

The DISA T&E MSS performance-based service contracts and task orders are addressed by this surveillance plan. The Government team described above is responsible for the execution of this assessment process. Monthly Status Reports (MSRs) and deliverables presented by the contractor will provide data that the Government may use in conjunction with other data in order to assess performance. This surveillance plan will focus on the definition of each quality standard, the methodologies, process and forms to be utilized in the gathering and assessment of contractor performance data and the communication process necessary to ensure effective and timely identification and resolution of contractor performance issues.

5.1 DISA T&E MSS Quality Performance Standards

a. Assessments will be based on objective facts and supported by program and contract management data, such as functional and program performance evaluations, customer comments, quality of products, customer feedback, and general technical interchange and engagement with team members.

b. The principle goal of the DISA T&E MSS assessment process is to ensure that the contractor performs requirements at acceptable quality levels (AQL), and that the Government receives services that satisfy the requirements of the contract. The DISA T&E MSS assessment must be:

· Accurate: All data collected must be reflective of actual contractor performance

· Fair: The assessment must be based on known baseline requirements

· Relevant: Data must be reflective of those quality assessment areas identified in the PWS or in the task order

· Comprehensive: Data shall be comprehensive enough to facilitate identification of shortfall root causes not just the fact that a shortfall exist

· A Repeatable Process: The processes, forms, analysis methodology and reporting system must remain consist across each contract and each task order throughout the life of the effort

· Timely: Assessment reporting cycle must ensure timely notification of issues so as to enable timely resolution.

· A Joint Agreement. The processes, quality standards and frequency of assessment must be clearly understood by all parties.

· Consistent: The processes used to conduct the DISA T&E MSS assessment must be consistent across tasks.

c. The DISA T&E MSS Program Office has established five (5) core measures of contractor performance which are defined in paragraph 5.2. Data on each of the core measures will be consolidated monthly by the program office and documented in DISA T&E MSS Performance Assessment Report. The DISA T&E MSS Monthly Performance Assessment report will be made available for contractor review monthly and will be formally reviewed with the contractor on a quarterly basis. Information consolidated from the Monthly Performance Assessment Report will be utilized as the principle input to the annual CPARS reporting.

d. The core measures of contractor performance will be measured through the following methods of surveillance:

i. Customer Feedback: Customer feedback may be obtained either from the results of formal customer satisfaction surveys or from random customer complaints. The TM or COR will keep copies of customer satisfaction surveys and customer complaints in a documentation file.

ii. 100% Inspection: This surveillance type is preferred for those tasks that occur infrequently, as well as for tasks than cannot be randomly sampled because the sample size for a small lot may exceed the lot size. When, this type of surveillance is used, the COR must inspect and evaluate the contractor's performance each time it performs. The result is used to determine acceptability of the contractor's performance.

iii. Periodic Inspection: Periodic inspections shall be conducted by the TM or COR as called for in this QASP. This method involved a thorough evaluation of selected outputs, whether scheduled, or tied to a key event, such as a phase of the system life cycle.

iv. Random Monitoring: Random monitoring shall be conducted by the TM or COR as called for in this QASP.

e. Task orders issued under the DISA T&E MSS will either be Firm Fixed Price (FFP) task orders or Cost Plus Fixed Fee (CPFF) task orders. Surveillance schedules will vary according to task order type. FFP orders will be assessed twice per month to coincide with the acceptance schedule in Wide Area Work Flow (WAWF). CPFF orders will be assessed one time per month, after Monthly Status Report (MSR) actuals have been submitted in the DISA T&E MSS provided electronic system.

f. Performance will be monitored by the COR at the contract level, and by the TM at the task order level. Data archived electronically will be consolidated once a year for all tasks with an expected contract life value of $1 million or more, and for the overall contracts, and entered into CPARS to document performance assessments. The COR will ensure that the TMs review all Monthly Status Reports submitted for the task and will ensure TMs complete the Monthly Performance Assessment report. Statistics on compliance will be collected from the electronic repository system that stores the information, and used in assessing the COR/TM Critical Element #10 on yearly Performance Work Plan and Appraisal, DISA Form 208A.

g. A final, more detailed assessment will be made at the end of CPFF tasks for acceptance of final deliverables, or completion of task. This final assessment will be used as the basis for the close out evaluation in CPARS, and for release of the 15% of the fixed fee that has been held back in accordance with Section G of the contract until satisfactory completion of the task. The COR will collect the final assessment and provide final Government acceptance in conjunction with the procedure set up by the contracting office.

h. In the event that performance is not meeting Government standards, every effort will be made to work collaboratively with the contractor to remediate the issue. In the event that the quality issue can not be surmounted, remedies including disallowing of costs, holding back 15% of the fixed fee, notation of deficiencies in CPARS, and/or Termination for Default will be considered.

5.2 Measures of Contractor Performance

The contractor shall employ the necessary resources to ensure contract objectives are accomplished in a timely manner, and in accordance with basic contract and task requirement. The 5 core measures of contractor performance will be reviewed on a routine/daily basis by the Program Office(s), and other stakeholders are:

Performance Measure #1: Management Effectiveness Performance Measure #2: Technical Performance

Performance Measure #3: Quality of Work

Performance Measure #4: Timeliness

Performance Measure #5: Cost Control

Each area will be rated one per month by task (twice per month for FFP tasks), and supporting statements will be annotated as needed. The ratings are:

-Exceptional: Contract requirements have been exceeded in most cases, with no more than a few minor problems. Corrective actions have been highly effective.

-Very Good: Contract requirements have been exceeded in many cases, with no more than some minor problems. Corrective actions have been effective.

-Satisfactory: All contract requirements have been met, with no more than some minor problems. Corrective actions have been satisfactory.

-Marginal: Some contract requirements have not been met, and there have been serious problems. However, recovery from those problems is possible. Corrective actions have been marginally effective or not fully implemented.

-Unsatisfactory: Most contract requirements have not been met, and there have been serious problems which have adversely affected the DISA MSS T&E ability to meet customer or internal mission requirements. Recovery from those problems is not likely. Corrective actions have been ineffective.

Expectations should be tailored to the difficulty, complexity, and time limitations for the task. The following areas should be considered when evaluating the contractor under the performance measure criteria.

5.2.1 Performance Measure #1: Management Effectiveness

Performance in the area of Management Effectiveness will consider the following:

-Personnel quality: selecting, assigning, utilizing, developing, and motivating personnel; providing qualified and well trained personnel in accordance with task proposals; developing, maintaining and enhancing employee currency in relevant disciplines; retaining qualified employees -Liaison Effectiveness: contractor provides updates on the task progress, pro-active identification and avoidance of potential problems, proposed solutions, cost or schedule overruns, and other pertinent information that could impact cost, schedule, deliveries, technical performance, and quality of work inclusive of cost control

-Planning and Execution: items to consider include the ability of the contractor to have the right personnel at the right place at the right time, anticipation of problems, ability to execute a plan, the ability of the contractor to react to unexpected events in a timely manner without adverse impact on the mission

-Management Control: contractor’s directing of team members, applying quality assurance and control processes, partnering with the Government to resolve problems that impact effective mission accomplishment, safeguarding, processing, handling and accounting for Government Furnished Property (GFP), complying with contract provisions and Small Business Utilization

-Business Management: contractor's business and performance are complementary and supportive of the Government’s; cooperating with the Government in working out task and business problems that impede mission accomplishment or compliance with DoD Major Range and Test Facility Base (MRTFB) and DISA T&E MSS directives; cooperating with Government and other contractor personnel in presenting a favorable image, submitting timely invoices for services

5.2.2 Performance Measure #2: Technical Performance

Performance in the area of Technical Performance will consider the following:

-Technical competence: qualifications of personnel assigned to the task and the degree to which contractor work products reflect high quality and technical expertise, understanding of technical requirements, performance in meeting Government expectations; addressing complex issues; providing quality studies, analyses, reports or services; developing sound approaches reflective of state-of-the-art technology -Problem Solving: applying established and creative problem solving techniques to activities such as anticipating problems, generating workarounds, developing realistic and workable technical solutions to problems; recovering from setbacks; and applying lessons learned

5.2.3 Performance Measure #3: Quality of Work

Performance in the area of Quality of Work will consider the following:

-Adherence to task specified requirements -Professional products delivered on due date in useable form -Technical accuracy and completeness -Clear and understandable products

-Compliance with DISA T&E MSS guidance documents

-Punctuation, grammar, spelling, syntax, and publication error free

5.2.4 Performance Measure #4: Timeliness

Performance in the area of Timeliness will consider the following:

-Completion of work in accordance with (IAW) Government approved schedules, timelines, or established milestones -Performance in demonstrating ingenuity to short-fused deadlines

-Notifying the Government of potential technical or resource problems on a task before they adversely impact task accomplishment

-Seeking Government concurrence on significant task issues and implementing Government feedback and guidance -Accommodating Government-directed rearranged priorities or revised program direction emphasis

-Adhering to timeliness requirements considering difficulties associated with accomplishing the schedule such as frequently changing scope of effort, unexpected loss or gain of personnel, test item/test equipment difficulties or failures, technical challenges, software anomalies or changes, test scheduling windows, or unexpected issues -Developing realistic time estimates

5.2.5 Performance Measure #5: Cost Control

Performance in the area of Cost Control will consider the following:

-Effectively attending to cost control issues.

-Completing tasks in accordance with task proposal estimates -Developing realistic cost estimates -Providing timely notification to Government personnel of task changes and resource readjustments -Recommending and implementing cost-saving initiatives -Developing the proper mix of labor categories and labor levels for various tasks -Taking an active role in minimizing contract costs to include establishment of policies and practices intended to minimize overtime and travel costs

6.0 MONTHLY PERFORMANCE ASSESSMENT REPORT

The Government will assess the contractor’s performance one time per month at the contract level by the COR, and one time per month at the task order level by the TM (two times per month at the task order level by the TM for FFP tasks only). The contractor will be assessed in the five performance measurement areas. Assessment information will be stored in an electronic repository and will be available for review by contractor management.

Key to effective contract performance is the ability to establish a repeatable assessment process which enables both the Government and contractor team to identify and resolve performance issues in a timely manner. The DISA T&E MSS assessment process as described in this QASP will facilitate that goal.

One time per year, information stored in the repository will be consolidated by the COR for the contract, and by the TM for task orders with anticipated total values of greater than one million dollars and entered into CPARS. The contractor will be able to review the information in that system.

Information on contractor performance to be used in assessments will be collected using the surveillance methods previously described. Specific measures will be taken as summarized in Section 7.0. In addition, information submitted by the contractor in Monthly Status Reports (MSRs), information collected in quarterly meetings, and information collected from deliverable tracking will all be considered in the preparation of these reports.

7.0 PERFORMANCE STANDARDS

Phase-in / Phase-out – Measured by TM

(TEP review)

Performance Measure #1 – Management Effectiveness – Measured by TM

5.1

5.2

5.3

(proposal acceptance with low incidence of returns)

5.2

5.3

5.2

5.3

5.2

5.3

5.2

5.3

5.2

5.3

5.2

5.3

Performance Measure #2 – Technical Performance – Measured by TM

5.2

5.3

5.2

5.3

5.2

5.3

5.2

5.3

Performance Measure #3 – Quality of Work – Measured by TM

5.2

5.3

5.2

5.3

5.2

5.3

5.2

5.3

5.2

5.3

5.2

5.3

Performance Measure #4 – Timeliness – Measured by TM

5.2

5.3

5.2

5.3

5.2

5.3

Performance Measure #5 – Cost Control – Measured by TM

5.2

5.3

-Not applicable to Firm-Fixed-Price tasks-

5.2

5.3

-Not applicable to Firm-Fixed-Price tasks-

5.2

5.3

-Not applicable to Firm-Fixed-Price tasks-

Performance Measure #1 – Management Effectiveness – Measured by COR

4.2

5.2

5.3

Performance Measure #2 – Technical Performance– Measured by COR

5.2

5.3

13.2

H.24

H.20

H.24

Performance Measure #3 – Quality of Work – Measured by COR

5.2

5.3

(compliance with requirements in contract)

Performance Measure #4 – Timeliness – Measured by COR

Performance Measure #5 – Cost Control – Measured by COR

-Not applicable for Firm-Fixed-Price tasks-

-Not applicable for Firm-Fixed-Price tasks-

(report run after due date with submission percentage)

� EMBED Unknown ���

“Source Selection Information - - See FAR 2.101 and 3.104.”

PAGE

_1332762336.psd