Attachment_4_Quality_Assurance_Surveillance_Plan_(QASP).rtf

QASP

Page 11 of 13 Pages Ver. 10/20/2015

ATTACHMENT 4

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

ESS Operations, E3, and Strategic Planning file_0.wmf

QASP

Page 11 of 13 Pages Ver. 10/20/2015

Contract or Task Order Title : Electromagnetic Spectrum Services (ESS) for Operations, Electromagnetic Environmental Effects (E3) Control, and Strategic Spectrum Planning

Work Requirements:

Task 1 – Operational Spectrum Support Task 2 – DoD E3 Program Support Task 3 – Strategic Spectrum Planning Task 4 – Research Emerging Technology

Primary Methods of Surveillance: Acceptable surveillance methods for this contract include the following:

100 Percent Inspection. This is usually only the most appropriate method for infrequent tasks or tasks with stringent performance requirements, e.g., where safety or health is a concern. With this method, performance is inspected/evaluated at each occurrence. One hundred percent inspection is too expensive to be used in most cases.

Random Sampling. This is usually the most appropriate method for recurring tasks. With random sampling, services are sampled to determine if the level of performance is acceptable. Random sampling works best when the number of instances of the services being performed is very large and a statistically valid sample can be obtained. Computer programs may be available to assist in establishing sampling procedures.

Periodic Inspection: These services are monitored weekly, monthly, quarterly, semiannually, annually, etc. Periodic types of activities are perfect for periodic inspection because not only are they infrequent, but there is normally a predetermined, specified time frame within which the tasks must be accomplished. Therefore, you know exactly when to conduct the evaluations. Periodic inspections automatically become 100 percent evaluations or "100 percent checks." Inspections should be divided and scheduled by frequency: annual, semiannual, quarterly, monthly, weekly and as required. Sometimes services are required for which the time or frequency cannot be predicted, such as accident investigations, one-time special tasking by higher headquarters, etc. These would be labeled "as required inspections." Others are known and predictable such as the quarterly status report or the monthly travel report currently included in some DISA service contracts.

Customer Input. Although usually not a primary method, this is a valuable supplement to more systematic methods. For example, in a case where random sampling indicates unsatisfactory service, customer complaints can be used as substantiating evidence. In certain situations where customers can be relied upon to complain consistently when the quality of performance is poor, e.g., dining facilities, building services, customer surveys and customer complaints may be a primary surveillance method, and customer satisfaction an appropriate performance standard. In all cases, complaints should be documented, preferably on a standard form.

Scope of Performance: The contract shall provide the operational (embedded and technical reach back), E3 and Spectrum Engineering, Strategic Spectrum Planning, and Research and Analysis of Emerging Spectrum Technologies (EST) to support comprehensive and integrated spectrum operations, engineering, analysis, planning, and long term strategies for the DoD. This contract will provide technical support to DSO for the following tasks:

Operational Spectrum Support – Deploying qualified spectrum managers in support of Combatant Commands (CCMDs) or JTF operations, contingencies, and exercises; deploying technical experts to take field measurements and analyze incidents of EMI, determine causes, and recommend methods of resolution; and operating and maintaining standard SM systems. Providing reach back and deployable engineering support for operations, contingencies, and exercises. The PWS reference is Paragraph 5.a.

E3 Engineering Support – Developing electromagnetic compatibility (EMC) standards; analyzing equipment and systems for potential E3 problems and recommending solutions; developing and conducting E3 training; assisting the acquisition community by assessing E3; developing joint policy and procedures for Hazards of Electromagnetic Radiation to Ordnance (HERO); and developing and maintaining joint ordnance database tool. The PWS reference is Paragraph 5.b.

Strategic Spectrum Planning – Analyzing national and international EMS issues and advising executive decision-makers on the technical viability of policy and implementation options. Develop and execute DoD strategic spectrum plans. Identifying and defending DoD’s spectrum needs in national and international forums. Overseeing DoD’s the preparation for the World Radio communication Conference (WRC). The PWS reference is Paragraph 5.c.

Research and Evaluate Emerging Spectrum Technologies – Identifying and assessing emerging spectrum technologies, such as DSA, LTE, and spectrum sharing technologies, to determine EMS implications, risks, and benefits to DoD and formulate strategies and technical approaches to maximize the benefit and/or minimize the threat posed by emerging spectrum technologies. The PWS reference is Paragraph 5.d.

Performance Standards: The QASP will be used to ensure that the contractor performance is effectively monitored and documented. The Contracting Officer Representative (COR) uses the methods contained in this QASP to ensure the contractor is in compliance with contractual requirements. The COR will be responsible for a wide range of surveillance requirements that effectively measure and evaluate the contractor's performance. Additionally, this QASP is based on the premise that the contractor, not the Government, is responsible for management and quality control/quality assurance actions to successfully meet the terms of the contract.

Table 1 provides the performance standards, acceptable quality levels, and surveillance methods that will be employed by the COR inspect the contractor’s work performance. Inspection of each element will be documented in the COR file.

Table 1. Technical Performance Surveillance Matrix

Technical Performance Standard Acceptable Quality Level (AQL) Methods of Surveillance All Tasks – Reports Technical Requirements: Percentage of technical requirements contained in the PWS that were met by the contractor. In addition, the deliverable must be complete and free from major technical errors before being accepted by the government. Meeting 95% of the requirements and being complete and free of major technical errors is an AQL.

At most, reports should contain only minor technical errors that, although they are minor, require fixing prior to delivering the final product to a sponsor. These technical errors should have no, or a very minor impact on the final conclusions drawn in the report.

Material in the report must be presented in a logical easy to read, and unambiguous manner. All relevant data must be included in the report. Conclusions must be fully sup- ported by data and information in the report. The re- port must be accurate. Re- ports should reflect innovative solutions or unique in- sights to resolve problems.

Information and graphics must be presented in a pleasing format.

100 Percent Inspection, Random Sampling.

Periodic Inspection, Customer Input of delivered products.

Technical Performance Standard Acceptable Quality Level (AQL) Methods of Surveillance

Reports must not require multiple returns to the contractor for editorial rework prior to being accepted by the DSO, user, or sponsor. Reports must be organized so that a typical reader can follow the flow of information in the report with figures and tables properly marked and easily read. The DSO approved re- port format should be used unless other provisions are stipulated at the beginning of the task. Minimal report re- work post-delivery is an AQL.

Schedule Requirement: AQL is report delivered on-time.

The delivery date is the date the contractor provides the deliverable to the government.

Cost Requirement: AQL is achieved when the cost charged to the TO for the product is equal to the target cost.

All Tasks – Technical Support Technical Support AQL: The contractor must adequately prepare for, and participate, in meetings and conferences. If material is presented, it must be accurate and professional. If documents are prepared, they must be accurate, free of errors, and require little or no editorial rework. If database entries/outputs are required, they must be accurate and in the proper format. If ad hoc analyses are performed, they must be accurate and meet the 100 Percent Inspection, Random Sampling.

Customer Input of delivered products and services.

Methods of Surveillance sponsor's requirements. The contractor must provide the appropriate response (in- house analytical and deployed staff) to resolve interference events.

Subtask 6.1.1 – General Operational Support Performances occur with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.1.2 –C/S/As Support Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.1.3 – Operational Training Support Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.1.4 – EMC Analyses Support Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.1.5– C-E Equipment Signal Coverage Plots and Analyses Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.1.6 – Spectrum Plans, Manuals, and Doctrine Input Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.1.7 – IO/STO Performance occurs with no 100 Percent Inspection, Methods of Surveillance Support required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.1.8 – JSIR Support Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.1.9 – Satellite Research Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.1.10 – Operational Business Processes and Spectrum Analytics Support Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.1.11 – SCIF Operations Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.1.12 – ACSSOs and ISSOs Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.1.13 – Con- tractor ISSOs Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.2.1 – DoD E3 Program Management Performance occurs with no required re-performance or 100 Percent Inspection, Periodic Inspection, Methods of Surveillance Support re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

Customer Input of delivered products and services.

Subtask 6.2.2 – Joint Capabilities Acquisition Support Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.2.3 – EMC Standards Support Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.2.4 – E3 and SS Awareness and Training Support Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.2.5– Joint Ordnance E3 Support Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.2.6 – JOERAD Software Support Data Management and Maintenance Support Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.2.7 – JSC Ordnance E3 Risk Assessment Definition and Development Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Random Sampling.

Periodic Inspection, Customer Input of delivered products.

Subtask 6.3.1 – Regula- tory and Legislative Support Performance occurs with no required re-performance or re-work at least 95% of the

Customer Input of delivered

Methods of Surveillance time. Problems that occur are minor and are resolved in a satisfactory manner.

products and services.

Subtask 6.3.2 – Spectrum Reallocation Sup- port Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.3.3 – World Radiocommunication Conference Support Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.3.4 – DoD EMS Strategy Implementation Support Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.3.5 – DSO Program Support Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.4.1 – Stake- holder Collaboration and Coordination Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.4.2 – Emerging Spectrum Technology Surveillance and Assessment Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Subtask 6.4.3 – Advanced Spectrum Management Concepts Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are

Customer Input of delivered products and services.

Methods of Surveillance minor and are resolved in a satisfactory manner.

Subtask 6.4.4 – Support to Advanced SM Regulatory Processes Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

100 Percent Inspection, Periodic Inspection, Customer Input of delivered products and services.

Mandatory or regulatory compliance items: The contractor will provide initial proof of compliance with regulatory and compliance items contained in the contract. The COR and the Contracting Officer (KO) will evaluate the initial submission. The COR and KO will conduct ad hoc reviews of the contractor’s compliance throughout the remainder of the contract. Table 2 provides mandatory/regulatory compliance standards for Combating Trafficking in Persons. Table 2 is not intended to address all mandatory and/or regulatory compliance items in the contract. Table 2 provides mandatory compliance standards, acceptable quality levels, and surveillance methods that will be employed by the COR to inspect the contractor’s work performance. Inspection of each element will be documented in the COR file.

Table 2. Mandatory or Regulatory Performance Surveillance Matrix

Mandatory/Regulatory Compliance Standard Acceptable Quality Level (AQL) Methods of Surveillance Combating Trafficking in Persons Contractor requirements. The Contractor shall--

Notify and train its employees on the regulations FAR 52.222-50, Combating Trafficking in Persons.

Zero tolerance policy de- scribed in paragraph (b) of this clause; and (ii) The actions that will be taken against employees for violations of this policy. Such actions may include, but are not limited to, removal from the contract, reduction in benefits, or termination of 100 Percent Inspection, Random Sampling.

Mandatory/Regulatory Compliance Standard

Methods of Surveillance employment; and

(2) Take appropriate action, up to and including termination, against employees or subcontractors that violate the policy in paragraph (b) of this clause

Acceptable Quality Level (AQL) Performance Rating/Evaluation: Reference Section 5 for AQLs for each performance standard.

Evaluation Method: The COR is responsible for providing continuous technical oversight of the contractor's performance. The COR uses the QASP to conduct the oversight/surveillance process. The COR shall keep a Quality Assurance file that accurately documents the contractor's actual performance. The purpose is to ensure that the contractor meets the performance standards contained in the contract. The COR is responsible for reporting early identification of performance problems to the KO. The COR is required to provide an annual performance assessment to the KO which will be used in documenting past performance. The QASP is the primary tool for documenting contractor performance. The COR is not empowered to make any contractual commitments or to authorize any contractual change on the Government's behalf.

The performance standards shown in Table 3 will be used to evaluate the contractor’s performance with respect to technical quality, timeliness, and schedule.

Table 3: Performance Ratings

Performance Rating Criteria Excellent Performance meets contractual requirements and exceeds many to the government’s benefits. The contractual performance of the element or sub-element being assessed was accomplished with no problems and contractor actions were highly effective.

Very Good Performance meets contractual requirements and exceeds some to the government benefits. The contractual performance was accomplished with few minor problems for which corrective actions taken by the contractor were effective.

Satisfactory Performance meets contractual requirements. The contractual performance contains some minor problems for which corrective actions taken by the contractor were satisfactory.

Marginal Performance does not meet some contractual requirements. The contractual performance reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear marginally effective or were not fully implemented.

Unsatisfactory Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance contains serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

Incentives: The TO incentive fee will be composed of technical, schedule, and cost performance elements. Technical and schedule performance will each be worth 40% of the total fee. Cost performance will make up the remaining 20%.

Technical. Technical performance will be evaluated on a scale of 1 to 5, according to how well the contractor meets the desired performance thresholds. An overall technical quality score of “Unsatisfactory” from 1.00 to 1.49 percent equates to no technical performance fee being earned. An overall technical quality score of “Marginal” from 1.50 to

2.49 percent equates to the technical performance earning the target fee from 3.00 to 4.99 percent. An overall technical quality score of “Satisfactory” from 2.50 to 3.49 equates to the technical performance earning the target fee from 5.00 to 6.99 percent. An overall technical quality score of “Very Good” from 3.50 to 4.49 equates to the technical performance earning the target fee from 7.00 to 8.99 percent. Lastly, an overall technical quality score of “Exceptional” from 4.50 to 5.00 equates to earning the maximum technical performance fee from 9.00 to 10.00 percent. The range of performance scores and the associated percentage fee earned is presented in Table 4.

Table 4. Performance Surveillance Matrix

Score Performance Percent Fee Min Max

Min Max 1.00 1.49 Unsatisfactory 0.00 0.00 1.50 2.49 Marginal 3.00 4.99 2.50 3.49 Satisfactory 5.00 6.99 3.50 4.49 Very Good 7.00 8.99 4.50 5.00 Exceptional 9.00 10.00

Schedule. TO deliverables provided to the government within three days of the delivery date will be rated satisfactory and earn the target fee for schedule performance. Deliverables provided more than 10 days late will be rated unsatisfactory and earn the minimum schedule performance fee. Deliverables provided more than 10 days early will be rated exceptional and earn the maximum schedule fee.

Cost. Cost at target amount earns the target cost performance fee. Cost above, or below, the target amount will cause the cost performance fee to be proportionally increased or decreased by $0.35 for each dollar of variance. Cost performance fee cannot vary below the minimum or above the maximum amounts.