Attachment_3_-_SAC_Final.pdf

Surveillance Activity Checklist (SAC) For the Air Vehicle Modification and Instrumentation (AVMI) Solicitation

1. Purpose This Surveillance Activity Checklist (SAC) is a Government developed and applied document used to make sure systematic quality assurance methods are used in the administration of the non-performance based contract. The intent is to ensure that the contractor performs in accordance with the performance metrics and the Government receives the quality of services required in the contract.

This contract requirement is for technical services to support the mission of the Air Vehicle Modification and Instrumentation (AVMI) Facility of the NAVAIR Range Department (5.2). This contract provides a broad range of services to support designing, developing, procuring, building, installing, testing and evaluating, calibrating, modifying, operating and maintaining, and documenting instrumentation on aircraft, weapons, and engines for the Navy and other government, commercial and foreign government customers.

The resulting contract vehicle will be a C-type contract. The contract will contain cost-plus-fixed-fee (CPFF) labor CLINs, and cost-reimbursable CLINs for travel, material, and miscellaneous ODCs.

The contract will be for a one-year base period with four (4) one-year options. A properly executed SAC will assist the Government in achieving the objectives of this procurement.

2. Authority Authority for issuance of this SAC is provided under Contract Section E – Inspection and Acceptance, which provides for inspections and acceptance of the services and documentation called for in the contract, to be executed by the Contracting Officer or a duly authorized representative.

3. Scope To fully understand the roles and responsibilities of the parties, it is important to first define the distinction in terminology between Quality Control Plan and the Surveillance Activity Checklist (SAC). The contractor, and not the Government, is responsible for management and quality control actions necessary to meet the quality standards set forth by the contract. The SAC, on the other hand, is put in place to provide Government surveillance oversight of the contractor’s efforts to assure that they are timely, effective and are delivering the results specified in the contract.

4. Government Resources The following definitions for Government resources are applicable to this plan:

Contracting Officer - A person duly appointed with the authority to enter into (Procuring Contracting Officer (PCO)) or administer (Administrative Contracting Officer (ACO)) contracts and make related determination and findings on behalf of the Government. The PCO for this contract is John Tomechko, AIR-2.5.1.11. The ACO will be designated in the resulting contract. Contracting Officers are designated via a written warrant, which sets forth limitations of authority.

Contracting Officer Representative (COR) - An individual appointed in writing by the PCO to act as their authorized representative to assist in administering the contract. The COR will be appointed in the resulting contract. The limitations of authority are contained in a written letter of appointment.

5. Responsibilities The following Government resources shall have responsibility for the implementation of this SAC:

Contracting Officer – The Contracting Officer ensures performance of all necessary actions for effective contracting, ensures compliance with the terms of the contract and safeguards the interests of the United States in the contractual relationship. It is the Contracting Officer that assures the contractor receives impartial, fair and equitable treatment under the contract. The Contracting Officer is ultimately responsible for the final determination of the adequacy of the contractor’s performance.

Contracting Officer Representative – The COR is responsible for technical administration of the contract and assures proper Government surveillance of the contractor’s performance.

The COR is not empowered to make any contractual commitments or to authorize any changes on the Government’s behalf. Any changes that the contractor deems may affect contract price, terms, or conditions shall be referred to the Contracting Officer for action.

6. Methods of QA Surveillance Surveillance shall be used by the COR in the technical administration of this contract and will be accomplished through 3 principal avenues. The first will be through the surveillance checklist shown in Table 1. Within this table are a variety of items to check on the quality/level of support. The data to support these items are collected throughout the year. A midyear ‘tracking’ rating will be calculated and shared with the contractor with the final rating determined at the end of the performance period.

The ratings for these items feed directly into the final CPARS rating at the end of each period of performance. The second is through Government Functional Leads that have contractor support.

These leads are expected to provide a monthly assessment of the quality of the support they receive.

Required topics are identified in Table 2, however, they are encouraged to share all positive and negative feedback in a timely manner. The third method will be through general awareness via regular COR meetings, Department-level staff meetings, personal conversations, and first-hand observations.

7. Contractor Performance Assessment Reporting System (CPARS) The contractor will be rated at the end of each performance period using CPARS. The basis of the ratings will primarily be the surveillance performed above. However, all available performance data will be used – including requesting information from the contractor for their perspective prior to assigning ratings. The rating is not about punishing or rewarding the contractor – it is about documenting their performance. The ratings and their definitions are taken from the FAR as shown in Table 3. The CPARS ratings may be determinant in exercising an option if ratings are less than

Satisfactory. For this procurement the Government will address the quality of product or service, schedule, cost control, business relations, management, and other important areas. As this information may affect future source selections throughout DoD, and the continuation of the contract, the annual Government assessment will be used appropriately as an additional performance oversight and communication tool along with the SAC.

8. Documentation In addition to providing timely reports to the Contracting Officer, the COR will maintain a complete Quality Assurance file. The file will contain copies of all reports, evaluations, recommendations, and any actions related to the Government’s performance of the surveillance activity function. All records will be maintained for the life of the contract. The COR shall forward these records to the Contracting Officer at termination or completion of the contract.

N00421-17-R-0071 Attachment 1

9. AVMI Surveillance Elements

9.1 Surveillance Checklist

This checklist will be utilized to rate the contractor’s performance in critical areas during execution of the contract and will assist the Government in calculating final CPARS rating(s) for each contract year.

Table 1 Surveillance Checklist

# CPARS

Element Surveillance Element Evaluation Measurement Evaluation Method

1 Cost Control

Management of Labor rates with respect to proposal

Exceptional: No group more than 2% over; total of all groups <= 2% over

Very Good: No group more than 3% over; total of all groups <= 4% over

Satisfactory: No group more than 5% over; total of all groups <= 7% over

Marginal: At least one group >5% or total of all groups <= 15% over

Unsatisfactory: Total of all groups is > 15% over

Based on Labor Category Cost Report CDRL A003. Compare average burdened labor rates against the contractor's proposal for that year (e.g. during the Base period, use Base year proposal data). If there is no proposal data for a labor category at a site, an analysis will be performed based on similar labor categories at that site to determine a reasonable baseline cost for that labor category. See further explanation in the calculation approach following the table.

2 Cost Control

Provide ideas, suggestions, or actions that could reduce cost on the contract and/or to the Government overall;

Conversely, don't cost the Govt/Contract more due to inaction or inattention.

Exceptional: Overall cost savings and/or avoidances totaling $100k or more. No significant cost incursions

Very Good: Overall cost savings and/or avoidances totaling $50k or more.

Satisfactory: No cost savings/avoidances or minor cost savings, avoidances, and incursions in the range of +/- $50k when added together.

Marginal: Overall cost incursions totaling $50k to $100k

Unsatisfactory: Overall cost incursions totaling $100k or more.

COR/GGFR/Govt observation; CDRL A022 Accident/Incident Report; Customer feedback. Use all data. Cost savings and cost avoidances can offset contractor cost incursions and vice versa.

3 Manage ment

Employee retention to maintain workforce capability

Exceptional: >= 98% Very Good: >=95% Satisfactory: >= 90% Marginal: >= 80% Unsatisfactory: <80%

Using Locator CDRL, COR to calculate retention by counting number of personnel that have left the contract (other than reduced or insourced by AVMI) since the beginning of the Base/Option period and divide by the highest count of contractors onboard during that period.

# CPARS

Element Surveillance Element Evaluation Measurement Evaluation Method

4 Manage ment

Contractor addresses poor performers (skill, attitude, hygiene, performance)

Exceptional: 100% addressed within 1 week or according to set plan/process

Very Good: 95% addressed within 1 week or according to set plan/process

Satisfactory: 90% addressed within 1 week or according to set plan/process

Marginal: 75% addressed within 1 week or according to set plan/process

Unsatisfactory: Less than 75% addressed within 1 week or according to set plan/process

COR/GGFR/Govt observation; Customer feedback. Documented start of problems in surveillance reports and COR meetings/conversations. PM expected to address problems and/or determine plan of action quickly. Plan should address future actions in case problem(s) continues. PM expected to follow plan. Plan may be updated as needed with COR concurrence.

5 Quality Accurate CDRLs Exceptional: No more than 10 CDRLs rejected, returned, or requires fixing

Very Good: No more than 15 CDRLs rejected, returned, or requires fixing

Satisfactory: No more than 25 CDRLs rejected, returned, or requires fixing

Marginal: No more than 40 CDRLs rejected, returned, or requires fixing

Unsatisfactory: 41 or more CDRLs rejected, returned, or requires fixing

COR/Govt review of CDRL deliveries.

Between the 4 bi-weekly funding CDRLs, 3 monthly status reports, and a weekly absence report, there are approx. 225 deliverables in a year. 10 returned CDRLs would equate to a 4.4% rework rate. 15 = 6.7%; 25 = 11.1%;

40 = 17.8%

6 Quality Produces quality products (e.g. machined part, sheet metal part, cable, instrumentation installation, system, engineering or administrative document)

Exceptional: Nearly everything correct initially (approx.

98%); all correct on rework first time

Very Good: Minimal flaws and/or rework required on initials (approx. 95% correct); all correct on rework first time

Satisfactory: Predominately correct on initials (approx.

90%) and predominately correct on rework

Marginal: Mostly correct on initials (approx. 80%) and/or rework

Unsatisfactory: Somewhat correct on initials and/or rework.

Govt observation, Documentation review, Customer feedback. Since the government can't and doesn't inspect every item that is produced at the time of production, definitive percentages don't apply. Approximate percentages are provided only to help frame expectations. The COR must decide based on evidence whether quality work is being performed, then assign a relevant rating. The trend from Exception to Satisfactory is allowing a little bit more error in fabrication.

However, the quality of rework should remain high.

7 Quality Improves situation when presented with feedback, both formally and informally.

For example, when an individual, team lead, or PM is provided feedback or other actionable information does the situation get resolved and communicated back to the originator?

Exceptional: 100% addressed within 1 week or according to set plan/process provided within 1 week

Very Good: 95% addressed within 1 week or according to set plan/process provided within 1 week

Satisfactory: 90% addressed within 1 week or according to set plan/process provided within 1 week

Marginal: 75% addressed within 1 week or according to set plan/process provided within 1 week

Unsatisfactory: Less than 75% addressed within 1 week or according to set plan/process provided within 1 week

COR/GGFR/Govt observation; Customer feedback. Documented start of problems in surveillance reports, COR meetings, or email.

PM expected to address problems quickly and determine plan in case problem continues. PM expected to follow plan. Plan may be updated as needed.

8 Quality Follow Standard Operating Procedures (SOPs), Ground Operations Procedures (GOPs), and other required processes.

Exceptional: No documented violations Very Good: 3 or fewer documented violations Satisfactory: 4 to 5 documented violations Marginal: 6 to 10 documented violations Unsatisfactory: More than 10 documented violations

COR/GGFR/Govt observation; Customer feedback. GOPs are an important part of Contractor being allowed to work on Government aircraft. Contractors that are not following GOPs (or other required procedures) may cause serious problems.

Expect most of these to be documented through Safety office and the GGFR. There are processes requiring COR approval that also must be followed (e.g. Contractor travel, Material Purchase on the Contract, and shipping on the contract).

9 Quality Use proper safety equipment as required

Exceptional: No documented violations Very Good: 1 documented violation Satisfactory: 2 documented violations Marginal: 3 to 5 documented violations Unsatisfactory: More than 5 documented violations

COR/GGFR/Govt observation; Accident Report CDRL; Customer feedback

10 Schedule Hiring to meet personnel requests (not including internal promotions)

Exceptional: 95% of hires on board within proposed timeline

Very Good: 90% of hires on board within proposed timeline

Satisfactory: 80% of hires on board within proposed timeline

Marginal: 60% of hires on board within proposed timeline

Unsatisfactory: Less than 60% of hires on board within proposed timeline

COR to track support requests date and personnel onboard dates. Include only closed out requests from the beginning of the Base/Option period through the current reporting period. If required, security clearance or interim clearance shall be obtained prior to start.

11 Schedule CDRL Delivery Exceptional: All CDRLs delivered on time Very Good: 10 or fewer CDRLs delivered late and none overdue Satisfactory: 11 to 20 CDRLs delivered late and none overdue Marginal: 21 to 39 CDRLs delivered late or 1 overdue Unsatisfactory: 40 or more CDRLs delivered late or 2 overdue

COR to track CDRL deliveries. A CDRL is late if it is delivered after the due date listed on the DD1423. A CDRL is overdue if it is more than 1 calendar month late or not delivered before the next due date for that

CDRL.

Management of Labor rates with respect to proposal – Calculation Approach Each invoice period, the contractor shall submit CDRL A003 which provides the average fully burdened labor rate for each labor category (including sub pass-through) for that 2 week invoice period. The labor categories will then be averaged across each A003 submission to come up with an Average Invoiced Rate per labor category for that year. Based on the proposal rate for that contract period, each labor category will have a calculated over/under percent.

To keep low dollar, high percent changes from directly offsetting high dollar, low percent changes (see LC1 vs LC9 in the example below where $1.50 for LC1 would offset $4.63 for LC9), the labor categories shall be grouped into 3 groups. The groups will contain the low, middle, and high proposed rates and will be named as such. Each group will contain 1/3 of the labor categories. The individual labor category over/under percentages will be added to obtain the group over/under percentage. The group over/under percent calculations will be added to get the total over/under.

a) Labor categories with lowest proposed rates.

b) Labor categories with the middle proposed rates

c) Labor categories with the highest proposed rates

Example of a Very Good rating (no group >3% and total <=4%):

Labor Cat A003 Calc Avg Rate Proposal Rate Delta Over/Under Group

Group Over/Under

Total Over/Under

LC1 $30.80 $32.30 ‐$1.50 ‐4.6%

LC2 $35.26 $35.00 $0.26 0.7%

LC3 $41.52 $40.92 $0.60 1.5%

LC4 $39.77 $41.00 ‐$1.23 ‐3.0%

LC5 $44.65 $43.00 $1.65 3.8%

LC6 $45.87 $45.00 $0.87 1.9%

LC7 $59.11 $60.58 ‐$1.47 ‐2.4%

LC8 $64.58 $64.22 $0.36 0.6%

LC9 $109.63 $105.00 $4.63 4.4%

2.9%

Low ‐2.4%

Middle 2.8%

High 2.5%

Labor Cat – List of labor categories defined in the contract

A003 Calc Avg Rate – Calculated by averaging each labor category from each A003 submitted throughout the year.

Proposal Rate – Contractor proposed labor rate for current year (e.g. Base, Opt1, etc). Option to extend will use the same delta increases from Opt4 as proposed from Opt3 to Opt4.

Delta – A003 Calc Avg Rate minus the Proposal Rate

Over/Under – Percentage of Avg Rate to Proposal Rate (=Avg Rate/Proposal Rate – 1; negative values are under, positive values are over)

Group – Groupings based on the Offeror’s Proposal Rate. Groups allow some trade within the group for lower rates to offset higher ones. For example, LC5 at 3.8% over would trigger a marginal rating. As part of the Middle Group, LC4 helped offset it to keep the group < 3%.

Group Over/Under – Add the Over/Under for each labor category within the group

Total Over/Under – Add the Over/Under for each Group

Note:

- A table will be created for each of the 3 sites.

- When adding a labor category with no proposal costs at a site, an analysis will be performed using similar labor categories at that site to determine a baseline cost.

- The new labor category will be added to the table in the appropriate group based on cost.

- Labor categories that fall between groups will be added to the group with the fewer number of categories or to the closest group.

9.2 Surveillance Items

The following items in Table 2 will be used by the COR to collect monthly information about the contractor’s performance from the Government Leads that have contractors supporting them.

Throughout the contract, items on this list may be refined, added, or removed to ensure a thorough picture of the contractor’s efforts and activity are captured. These items will be evaluated based on the FAR Table 42-1 – Evaluation Rating Definitions, which is presented in Table 3.

Table 2 Government Functional Lead Surveillance Items

Monthly Government Lead Surveillance Items

Provides customer oriented service.

Provides good communication to COR and Government team leads concerning upcoming issues and issues in work

Provides adequate subcontracts and subcontractor management

Individuals take ownership of assigned tasks and fully executes them

Works well within group or team

Completes the administrative, contractual, design, parts creation, and engineering services activities in a timely manner. And/or does not adversely impact project schedules due to task completion time, workmanship, and/or absences.

Work completed is of a quality nature, requiring minimal re-work, if any.

PM and Team Leads schedule the work within the team adequately

Provides plan to meet schedule when absences may interfere with milestones or other due dates.

9.3 Evaluations Approach

All SOW/CDRL tasks, including SOW/CDRL sub-tasks, will be assessed focusing on the following.

Quality of Product or Service – Assess the contractor’s effort to transform operational needs and requirements into an integrated solution. Areas of focus may include the planning and management of program tasks, the quality of support provided throughout all phases of contract execution, the integration of program management specialties, management of interfaces, and the management of a totally integrated effort of all program management concerns to meet cost, performance, and schedule objectives. Assess how successfully the contractor meets program quality.

Schedule – Assess the contractor’s adherence to the required delivery schedule by assessing the contractor’s efforts during the assessment period that contribute to or effect the schedule variance. Also address significance of scheduled events (i.e., design reviews), discuss causes, and assess the effectiveness of contractor corrective actions.

Cost Control – Assess the contractor’s effectiveness in forecasting, managing, and controlling contract cost. Is the contractor experiencing cost growth or under run? If so, discuss the causes and contractor-proposed solutions for the cost overruns. For contracts where task or contract sizing is based upon contractor provided person-hour estimates, the relationship of these estimates to ultimate cost should be assessed. In addition, the extent to which the contractor demonstrates a sense of cost responsibility, through the efficient use of resources in each work effort should be assessed.

Management – Assess the contractor’s success with timely award and management of subcontracts, including whether the contractor met small/small disadvantage and women-owned business participation goals. Discuss the extent to which the contractor discharges its responsibility for integration and coordination of all activity needed to execute the contract;

identifies and applies resources required to meet schedule requirements; assigns responsibility for tasks/actions required by contract; communicates appropriate information to affected program elements in a timely manner. Assess the contractor’s risk mitigation plans. If applicable, identify any other management areas that are unique to the contract.

Other areas – Assess additional evaluation areas unique to the contract or that cannot be captured elsewhere.

The evaluation ratings used are defined in FAR Table 42-1 ‘Evaluation Rating Definitions’

Table 3 FAR TABLE 42-1—Evaluation Rating Definitions

Rating Definition Note Exceptional Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the element or sub‐ element being evaluated was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

To justify an Exceptional rating, identify multiple significant events and state how they were of benefit to the Government. A singular benefit, however, could be of such magnitude that it alone constitutes an Exceptional rating. Also, there should have been NO significant weaknesses identified.

Very Good Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub‐ element being evaluated was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

To justify a Very Good rating, identify a significant event and state how it was a benefit to the Government. There should have been no significant weaknesses identified.

Satisfactory Performance meets contractual requirements. The contractual performance of the element or sub‐ element contains some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

To justify a Satisfactory rating, there should have been only minor problems, or major problems the contractor recovered from without impact to the contract/order. There should have been NO significant weaknesses identified. A fundamental principle of assigning ratings is that contractors will not be evaluated with a rating lower than Satisfactory solely for not performing beyond the requirements of the contract/order.

Marginal Performance does not meet some contractual requirements. The contractual performance of the element or sub‐element being evaluated reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented.

To justify Marginal performance, identify a significant event in each category that the contractor had trouble overcoming and state how it impacted the Government. A Marginal rating should be supported by referencing the management tool that notified the contractor of the contractual deficiency (e.g., management, quality, safety, or environmental deficiency report or letter).

Unsatisfactory Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub‐element contains a serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

To justify an Unsatisfactory rating, identify multiple significant events in each category that the contractor had trouble overcoming and state how it impacted the Government.

A singular problem, however, could be of such serious magnitude that it alone constitutes an unsatisfactory rating. An Unsatisfactory rating should be supported by referencing the management tools used to notify the contractor of the contractual deficiencies (e.g., management, quality, safety, or environmental deficiency reports, or letters).