Attachment_D_-_QASP.pdf

Effective Date: 09/14/2015

Expiration Date: 09/14/2020

Check https://ipdtdms.gsfc.nasa.gov to verify that this is the correct version prior to use

400-FORM-0002 (4/16/2014)

National Aeronautics and Space Administration

Goddard Space Flight Center

Greenbelt, Maryland

PACE-SMA-PLAN-0002, Revision -

Pre-Aerosol, Clouds, and ocean Ecosystem (PACE), Code 427.0

PACE Government Surveillance Plan

GSFC PACE/OCI CMO

09/14/2015

Released https://ipdtdms.gsfc.nasa.gov/

PACE Government Surveillance Plan PACE-SMA-PLAN-0002, Revision -ii

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

PACE Government Surveillance Plan

Signature/Approval Page

Prepared by:

John C Rauscher

Reviewed by:

Robert Estep Bill Sluder Kenn Yienger Beth Weinstein Bob Schweiss Jeremy Werdell Eric Gorman Noosha Haghani David Sohl Mike Hill

Approved by:

Andre’ Dress

Electronic Signatures available online at: https://ipdtdms.gsfc.nasa.gov/ iii

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

Preface

This document is under Pre-Aerosol, Clouds, and ocean Ecosystem (PACE) Project configuration control. Changes to this document require prior approval of the PACE

Configuration Control Board (CCB) Chairperson or designee. Submit proposed changes to the

PACE Configuration Management Office (CMO), along with supportive material justifying the proposed change. Changes to this document will be made by complete revision.

iv

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

Change History Log

Revision Effective Date Description of Changes

(Reference the CCR & CCB/ERB Approval Date) Revision - 09/14/2015 Baseline Release following the Approval of PACE-CCR-0001 v

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

Table of Contents

1.0 INTRODUCTION

1.1 PURPOSE

1.2 SCOPE

1.3 PROGRAM DEFINITION AND CONTRACT DESCRIPTION

1.4 APPLICABLE DOCUMENTS

1.5 REFERENCE DOCUMENTS

2.0 SURVEILLANCE STRATEGY DEFINITIONS

2.1 INSIGHT

2.2 OVERSIGHT

2.3 HYBRID SURVEILLANCE APPROACH

3.0 RESOURCES

3.1 GENERAL

3.2 GSFC SURVEILLANCE TEAM

3.3 DEFENSE CONTRACT MANAGEMENT AGENCY AND/OR NASA

CONTRACT ASSURANCE SUPPLIER

4.0 MANAGEMENT REVIEW AND CONTROL PROCESSES

5.0 SURVEILLANCE STRATEGY AND APPROACH

5.1 GENERAL

5.2 FORMS OF SURVEILLANCE

5.2.1 Communications and Data Collecting

5.2.2 Evaluation and Reporting

5.3 TYPICAL SURVEILLANCE METHODS

5.3.1 Management Information Systems

5.3.2 In-Depth Observation

5.3.3 Inspection

6.0 SELECTED SURVEILLANCE ACTIVITIES

6.1 SURVEILLANCE TEAM MEMBER ACTIVITIES

6.1.1 Project Planning and Management

6.1.2 Reviews

6.1.3 Configuration Control

6.1.4 Audits

6.1.5 Work Area Inspections

6.1.6 Work Review and Performance

6.1.7 Property Administration

7.0 METRICS

7.1 PROBLEMS/CONCERNS/ISSUES

7.2 PROGRAM COSTS (AS APPLICABLE)

7.3 FORECAST DELIVERABLE MILESTONE SCHEDULE

7.4 SOFTWARE AND HARDWARE DEVELOPMENT SCHEDULES

7.5 CONTRACT DELIVERABLES

7.6 RESOURCES

vi

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

7.7 SUBCONTRACTORS, VENDORS, AND SUPPLIERS

7.8 AUDIT RESULTS

7.9 REVIEW RESULTS

7.10 FAILURE/ANOMALY REPORT RESULTS

7.11 NON-CONFORMANCE RESULTS

7.12 GOVERNMENT INDUSTRY DATA EXCHANGE PROGRAM (GIDEP)/NASA

ALERT/ADVISORY/PROBLEM REPORTS RESULTS (Note: All responses must include subcontractor, vendor, and supplier responses/results)

8.0 SUMMARY

APPENDIX A: ABBREVIATIONS AND ACRONYMS

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

1.0 INTRODUCTION

1.1 PURPOSE

The purpose of this Surveillance Plan is to define the overall approach that the National

Aeronautics and Space Administration (NASA) Goddard Space Flight Center (GSFC) intends to use to monitor and survey contractor performance under all Pre-Aerosol, Clouds, and ocean

Ecosystem (PACE) Mission contracts and subcontracts. This plan defines the process the

Government expects to follow to obtain data and information, evaluate the contractors, and determine if contract performance is acceptable. The Government intends to apply the amount, degree, and depth of surveillance in proportion to the level of risk in any areas.

1.2 SCOPE

The term “PACE Mission” encompasses the PACE Spacecraft, Instruments, and Ground System.

Throughout this Surveillance Plan, the term “PACE contractor” is also used. Unless explicitly stated otherwise, this term is applicable to the PACE Spacecraft or Instrument prime contractor and any and all of their subcontractor(s), suppliers, and vendors. This plan is also applicable to any contractor(s), suppliers, and vendors used in the fabrication of PACE Mission hardware.

This plan identifies program requirements, strategies, resources, review and control processes, surveillance activities, and metrics for the continuous measurement of contractor performance under PACE Mission contracts and subcontracts. It is intended to be a living document over the life cycles of the PACE contract. It will be reviewed on a regular basis (at a minimum annually) and will be updated as required. Project specific deviations shall be identified herein.

The surveillance program addresses all elements and aspects of the contract including (but not limited to) the management and project/program office, systems management and engineering, flight hardware and software, data systems and ground interface, systems verification, safety and mission assurance, emergency services and studies, and ground hardware and software.

1.3 PROGRAM DEFINITION AND CONTRACT DESCRIPTION

This plan will also be implemented on various contracts initiated through GSFC for PACE

Spacecraft, Instrument, and Ground System hardware. The purpose of this Plan is to obtain flight and ground hardware, software, and support for NASA GSFC's PACE Program throughout all phases of the project from the date that contracts are signed through on-orbit services as detailed in contract documents such as the Statement of Work.

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

1.4 APPLICABLE DOCUMENTS

See Federal Acquisition Regulations (FAR) Parts 46.103, 46.104, 46.202-2, 46.4, and 46.5 for government quality assurance requirements at contractor facilities. See FAR Part 52.246 for inspection clauses by contract type.

The documents listed in this section apply directly to the performance of PACE Mission contracts.

These documents establish detailed requirements information necessary for the performance of the

Contract. Unless otherwise specified, the document version in effect at the time the contract is executed shall apply.

NPR 8735.2B Management of Government Quality Assurance Functions for NASA

Contracts

1.5 REFERENCE DOCUMENTS

The documents listed in this section may contain information pertinent to the performance of PACE

Mission Contracts and should be referred to for clarification purposes only. Government-produced contract documentation shall take precedence over the reference documentation.

NPD 7120.4D Program/Project Management

NPR 7120.5E NASA Program and Project Management Processes and Requirements

2.0 SURVEILLANCE STRATEGY DEFINITIONS

2.1 INSIGHT

Insight is an assurance process that uses performance requirements and performance metrics to ensure process capability, product quality and end-item effectiveness. Insight relies on gathering a minimum set of product or process data that provides adequate visibility into the integrity of the product or process. The data may be acquired from contractor records, usually in a non-intrusive parallel method.

Insight, as applied to the PACE contract and the work to be performed therein, involves lower levels of Government surveillance to allow the contractor to assume increased responsibility and accountability for the integrity of processes and/or products. For this type of surveillance, the government will retain rights of approval, and Mandatory Inspection Points (MIPs) will be reduced, or possibly replaced by a limited number of non-mandatory, inspection points. The government will have full insight, on a non-interference basis, into design, technical issues, development and production processes, quality functions, and schedules at all levels. Insight will rely heavily on evaluating planned contract deliverables and existing contractor procedures and working documents, if available.

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

The Government will use an oversight or hybrid approach (see below) to monitor contractor performance. Insight will be used in those areas where risk is low and the contractor clearly demonstrates compliance with mandated requirements.

2.2 OVERSIGHT

Oversight is an assurance process that uses Government-imposed product specification and process controls, such as military (MIL) specifications, MIL standards, and mandatory inspections, to direct the development and production of the product. Oversight is intrusive in that it requires gathering contractor product or process data through on-site, in-series involvement in the process. Oversight entails very detailed monitoring of the process itself. Oversight also entails a measure of in-line involvement in an activity, principally through inspection including Mandatory Inspection Points , with review and approval authority implicit to the degree necessary to assure that a process’s or product’s key characteristics are stable and under control.

As applied to the PACE Project, areas where the Government will use oversight include, but are not limited to, those processes for which one or more of the following apply:

The Government has determined that oversight is an effective method to mitigate risk;

The contractor has limited experience;

The contractor has not demonstrated acceptable performance;

Instances where the Government assumes the liability;

Full access to contractor’s or subcontractor’s data and documentation is lacking.

2.3 HYBRID SURVEILLANCE APPROACH

A hybrid surveillance approach combining elements of insight and oversight will be instituted at a contractor’s facility to ensure a high level of confidence exists regarding the contractor’s ability to identify, manage and control programmatic risks, including when new technology is acquired or unproven processes are employed by a contractor or their subcontractors, suppliers, or vendors. In this situation, oversight surveillance is used until the Government is satisfied that the contractor, subcontractor or supplier has all critical processes under control. The oversight activities usually impose Government MIP’s in-series with the contractor’s development and manufacturing processes.

Only after the contractor’s demonstration of sufficient risk mitigation capabilities will NASA consider transitioning to insight activities that rely predominantly on internal contractor data. The transition period from oversight to insight activities would be accomplished gradually, depending on contractor performance.

3.0 RESOURCES

3.1 GENERAL

All surveillance activities will be implemented using Government personnel and/or Government support contractor personnel and resources from/at GSFC or at the contractor’s (or their

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

subcontractor’s, supplier’s, or vendor’s) facility including personnel who are resident in the facility in either a Resident Office (including NASA Contractor Assurance Services [NCAS] representatives) or a Defense Contract Management Agency (DCMA) Office. In the case of the contractor, subcontractors, suppliers, and vendors, the use of Government personnel or support personnel will most likely be temporary, based on the activity being performed. The surveillance team may be composed of resident or visiting personnel, and civil servant or contractor personnel, which may include:

a. The Contracting Officer’s Representative (COR), the Contracting Officer (CO), and/or the Contract Specialist

b. PACE Project Office personnel, PACE Resident Office personnel, or PACE support service contractor personnel

c. PACE management personnel or other GSFC or NASA Headquarters management personnel and/or their designated representative(s)

d. GSFC Safety and Mission Assurance Directorate technical and support personnel including the GSFC Supply Chain Representatives, the PACE Chief Safety & Mission

Assurance Officer (CSO), the PACE Project Safety Manager (PSM), hardware and software quality engineers, system safety engineers, reliability engineers, risk management engineers, and DCMA and/or NCAS representatives

e. GSFC Applied Engineering and Technology Directorate (AETD) technical and support personnel

f. GSFC technical and support personnel other than from AETD; i.e., Health and Safety support personnel and/or IT security personnel

3.2 GSFC SURVEILLANCE TEAM

The GSFC Surveillance Team will be composed of key GSFC personnel. Their primary purpose, under the cognizance of the Contracting Officer's Representative (COR) will be to provide direction for contract surveillance activities and to serve as the Government’s focal point in reviewing and evaluating overall contractor performance. They will obtain information from various sources, including deliverable contractor documents, communications with the contractor, and reports by other personnel or representatives (e.g., the Project Manager, Work Breakdown Structure (WBS)

Area Managers, the CSO, and the PSM) who interact with the contractor.

3.3 DEFENSE CONTRACT MANAGEMENT AGENCY AND/OR NASA CONTRACT

ASSURANCE SUPPLIER

Defense Contract Management Agency (DCMA) and/or NASA Contract Assurance Supplier

(NCAS) representatives may be collocated with the contractor in addition to other CSO representatives including GSFC quality engineers. The DCMA and/or NCAS representatives will

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

support the CSO with the surveillance effort in accordance with the provisions of the GSFC Letter of

Delegation or Service Order/Task and this plan. If DCMA and/or NCAS support is required at a subcontractor, supplier, or vendor facility, every effort will be employed to use their local (collocated or itinerant) DCMA and/or NCAS support if available.

4.0 MANAGEMENT REVIEW AND CONTROL PROCESSES

Government participation in the following management review processes will be used to guide and direct surveillance activities:

Contractor’s daily status meetings

Contractor’s ad-hoc meetings

Contractor’s planning meetings

Weekly Project Management Telecons

Monthly Financial Reports

Monthly Business Meetings

Monthly Technical and Management Reviews

Preliminary and Critical Design Reviews and other System and Integrated

Independent Reviews

Fabrication/Manufacturing Readiness Reviews

Flight Readiness Reviews

Audits, Surveys, and Assessments (Contractor-led or Government-Led)

Peer Reviews and Technical Working Group Meetings

Parts Control Board, Anomaly/Failure Review Board, Materials Review Boards, Materials and Processes Control Board, Risk Management Reviews, Safety Working

Group, and Reliability Working Group Meetings and Related Activities and

Documentation

Meetings, Teleconferences, and Discussions

5.0 SURVEILLANCE STRATEGY AND APPROACH

5.1 GENERAL

The level of risk and the impact of failure are major determinants in helping define the type of surveillance to be conducted. If the impact of failure is minor and the level of risk is low, only a small amount of insight-driven surveillance would normally be needed. Conversely, if the impact of failure could be significant and the level of risk is high, more extensive surveillance (including oversight surveillance) is warranted.

The overall surveillance plan is to obtain objective evidence and data that enable the Government to determine whether the contractor's program and processes are functioning as intended in accordance with the terms of the contract. The focus will be on prevention rather than detection, i.e., emphasizing controlled processes and methods of operation, as opposed to relying solely upon inspection and test to identify problems.

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

The Contractor shall provide PACE Surveillance team members with open access to all areas in which PACE activities are being performed and will interface directly with their contractor counterparts. Surveillance team members will document problems, concerns and issues, and take note of contractor accomplishments. They will collect performance metric data, where applicable, and will participate in contractor review meetings. Information will flow from individual team members through the COR to the Project Managers. Information gained from these formal and informal exchanges of ideas and collection of data will be compiled and evaluated as a continuous measure of contract performance.

The COR and Contracting Officer will annually complete a NASA Form 1680, “Evaluation of

Performance,” which will also be reviewed by the contractor, and become a part of the active GSFC database of Contractor Performance Assessments.

5.2 FORMS OF SURVEILLANCE

Support of the PACE contract surveillance will take the following forms:

5.2.1 Communications and Data Collecting

The communications and data collecting methods used for surveillance will include (but not be limited to):

Both formal and informal teleconferences/discussions

Electronic mail

Contractor web-based work monitoring/reporting system (if applicable)

Progress Reviews (e.g., Monthly Status Reviews)

Technical Officer meetings/teleconferences

Business meetings

Daily reviews

Other communication methods, as needed

5.2.2 Evaluation and Reporting

The evaluation and reporting methods used for surveillance may include (but not be limited to):

Documentation of problems, issues, and concerns including problem/failure/anomaly reports and their resolution

Daily activity reports

Review of deliverables from milestone achievements

Review of Contract Data Requirements List (CDRL), if included with the Statement of Work (SOW)

Data collection reporting

Goals/metrics evaluations

General and required reporting

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

5.3 TYPICAL SURVEILLANCE METHODS

Each surveillance method is provided below including a description of how it may be used and specific examples.

5.3.1 Management Information Systems

This is the surveillance tool where insight into contractor performance is achieved through assessment of contractor or Government-generated data. In most cases, the contractor will be generating data in order to manage his processes. This data can be in electronic, web-based, or hard copy media. Use of this contractor-generated data by the Government represents an efficient means of gaining insight.

5.3.2 In-Depth Observation

This entails Government personnel directly observing the contractor during performance of work.

This tool may be used when Government witnessing of each occurrence of a specific activity is necessary due to the Government’s need to ensure that contractor performance is demonstrated or product performance is consistent with contractual requirements. It is often used when the work involves tasks which present high risk to Government assets; however, use of the tool is not limited to such critical activities.

An example of application of the in-depth observation tool includes the contractor’s Monthly

Technical and Management Reviews, which are status meetings to discuss current activities such as systems engineering, mission operations support, flight and ground software development, flight hardware integration and test, PACE programmatic and technical risks, and system safety and mission assurance.

Tool selection factors to consider when selecting the in-depth observation tool(s) include the following conditions:

Does it represent a constraint to the contractor’s authority to proceed? Will the lack of Government attendance/approval prevent the contractor from continuing work?

It allows the Government to have real time insight into contractor performance and provides stop work authority for unacceptable work.

It can be a risk mitigation/control tool for highly critical tasks.

It can be a qualitative tool.

It can be a significant resource driver for NASA.

Data cannot be used in a sampling or statistically-based assessment of the contractor’s performance, since it is not truly random.

It is best used by personnel who possess an in-depth knowledge and experience on the systems, tasks or disciplines involved.

It is a mechanism for maintaining civil service experience base and expertise.

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

5.3.3 Inspection

This is an in-line function in which the Government reviews and approves a specific contractor product or service. The Government observes a test or process to verify use of correct procedures and processes for specific activities. This in-line involvement represents a constraint to the contractor’s authority to proceed further with the work. Inspection indicates approval and acceptance of a contractor requirement by the Government and is performed for each occurrence of the requirement.

The Government may choose to use this surveillance technique due to high risk to its assets and a need to ensure performance is demonstrated.

Tool selection factors to consider when selecting the inspection tool(s) include the following conditions:

It represents a constraint to the contractor’s authority to proceed.

It allows the Government to have real time insight into contractor’s performance.

It is a risk mitigation/control tool for highly critical tasks.

It is a qualitative tool.

It can be a significant resource driver for NASA.

6.0 SELECTED SURVEILLANCE ACTIVITIES

6.1 SURVEILLANCE TEAM MEMBER ACTIVITIES

The following selected activities will be performed by various Surveillance Team members during applicable stages of contract performance:

6.1.1 Project Planning and Management

The contractor is responsible for managing the technical work specified in the SOW, and if included, the CDRL. The Surveillance Team will provide assistance, as necessary, with the development and approval of technical requirements, and with design, development, production and test activities. The

Surveillance Team can also assist by verifying that requirements are being flowed-down correctly.

Surveillance Team technical and business members will review schedules, resource plans, and risk management plans to obtain insight into the planning for and execution of the SOW. Additionally, the Safety and Mission Assurance (S&MA) Surveillance Team members will also review S&MA-related documentation and activities. If the contractor consistently fails to meet performance metrics, either over a period of time or due to severe technical issues, the Surveillance Team may adjust the surveillance approach towards greater oversight until the problems are effectively addressed.

6.1.2 Reviews

Active participation by Surveillance Team members in the various unit subsystem and system reviews will be an integral part of the surveillance process. These reviews provide an opportunity to

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

assess program progress. Cognizant unit and subsystem managers and engineers will interface with designated contractor managers and technical personnel and review closure actions for acceptability.

Team members will be kept apprised of the tracking status, open action items and formal closures.

6.1.3 Configuration Control

The contractor is responsible for managing configuration control in accordance with their

Configuration Management Plan (CMP). The plan applies to all hardware and software related to the

PACE project and hardware development activities as well as any ground support or test equipment used. Changes affecting contract requirements must be dispositioned for the GSFC PACE project’s approval.

6.1.4 Audits

The contractor will conduct internal audits using its own Quality Assurance (or other independent) organization in accordance with contractor standard practices and policies. Surveillance Team members may concurrently participate in contractor-led audits involving any aspect of PACE mission operations and hardware development and/or related use of contractor or Government resources.

When the Government has concerns regarding contractor performance, Surveillance Team members may conduct independent audits of the contractor's activities, processes, products, documentation and data in order to provide assurance that the program is being implemented according to all requirements and specifications. These audits will normally be conducted with advance notification and coordinated with the contractor. However, the Government reserves the right to conduct unscheduled audits when evidence indicates that contractor performance is deficient.

6.1.5 Work Area Inspections

The surveillance team members will make physical inspections of the contractor's work areas. These inspections are made to assure compliance with PACE technical requirements. The inspections may include verifying that:

All contractor employees have the proper credentials/certifications, as appropriate for performing their duties; e.g., soldering certification or Electrostatic Discharge (ESD) training certification.

The contractor employees are aware that work area inspections may be conducted at any time.

The contractor employees are familiar with the contractor and NASA requirements, processes, and/or procedures (as appropriate) that govern their duties/performance.

When appropriate, the contractor employees are aware of the appropriate protection procedures for handling Government planning data and other contractor confidential or financial data.

The contractor employees are aware of the GSFC Security compliance requirements that affect their duties/performance.

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

The COR or other surveillance personnel may make periodic checks of the contractor's workspace to assess adequacy of facilities, equipment, and materials.

6.1.6 Work Review and Performance

The Surveillance Team will perform the following functions to evaluate the contractor's performance:

Review the contractor's Monthly Progress Report for accuracy and completeness.

Assess the fidelity of reports.

Meet/teleconference at least monthly with the contractor's program manager to discuss overall contract management and performance, schedule issues, and review cost related issues.

Review contractor work plans to ensure that all milestones and deliverables have been identified.

Certify the contractor's invoices for payment in accordance with GSFC procedures and contract requirements.

Assess the contractor’s overall contract performance against the metrics (technical, schedule, etc.) established in the contract.

In the event of a discrepancy in the contractor's performance, the COR will promptly notify both the Contracting Officer and the contractor's program manager and arrange a meeting to rectify the situation.

Perform Quality Assurance (QA) inspections and QA witnessing/monitoring of tests.

6.1.7 Property Administration

The COR will make informal inspections, as required, to ensure Installation-Accountable

Government Property (IAGP) is being properly utilized and protected in accordance with the contract terms and conditions.

7.0 METRICS

Surveillance Team members will be responsible for collecting production, integration, test, and product assurance data, and for facilitating overall surveillance activities. For example, this may involve reviewing Work Order Authorizations (WOAs)/Travelers, deviations, waivers, etc. They will ensure that acceptable limits, data ranges, pass/failure criteria, and goals have been established.

Sources of the data include the contractor, GSFC, DCMA (if participating), NCAS (if participating), and subcontractors, suppliers, and vendors. The majority of data will be readily available through established data documentation processes including contract deliverables, contractor self-assessment programs, and Government monitoring. Direct access to contractor data, through electronic means, may also be required. Federal Acquisition Regulation (FAR) 42.1104 states the Government shall make maximum use of any reliable contractor data; therefore, prior to using contractor data, the

Government must determine that the data is reliable by some validation process. A typical validation includes the following:

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

A periodic audit to verify that the data collection and reporting systems have adequate processes to reliably produce accurate data and metrics.

Data sampling to validate that stored data accurately reflects reality.

Verification that the metric formulation (i.e., transformation of raw data into graphical elements) captures all data, uses appropriate transformations, and displays it accurately.

The COR will screen the data, as appropriate, and in consultation with the Surveillance Team, select a limited scope of key activities for continuous surveillance, development of metrics, and presentation to management. Metrics/goals will be developed to provide a continuous measure of contractor performance. Utilization of metric data requires an understanding of the performance as well as the sources and types of errors. Metrics provide maximum understanding if the information is interpreted by the subject matter expert. The data will be tailored to the phase, risks, issues, and problems encountered.

Below are potential performance metrics which may be monitored by the Surveillance Team during various phases of the contract.

7.1 PROBLEMS/CONCERNS/ISSUES

Statement of problem/concern/issue

Programmatic impact (cost, schedule, scope changes/impacts)

Root cause identified

Action taken

Date established

Current status

Date resolved/closed

7.2 PROGRAM COSTS (AS APPLICABLE)

Actual versus planned

Percent completed under budget

Cost to complete

Earned Value Management Metrics

7.3 FORECAST DELIVERABLE MILESTONE SCHEDULE

Milestone elements

Actual versus planned completion dates

Percent completed early/on-time/late

Quality

Completeness and (if applicable) risks addressed

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

7.4 SOFTWARE AND HARDWARE DEVELOPMENT SCHEDULES

Schedule margin

Milestone completion

Date start/finish (actual versus planned)

Percent completed early/on-time/late

Major delays/problems/concerns

7.5 CONTRACT DELIVERABLES

Percent completed early/on-time/late

Percent approved/rejected

Percent requirements satisfied

7.6 RESOURCES

Manpower utilized (actual versus planned)

Variances in Full Time Equivalents (FTE) over/under plan

Skill mix (all disciplines covered/not covered)

Workforce health indicator (percent of leave used by type and month)

7.7 SUBCONTRACTORS, VENDORS, AND SUPPLIERS

Source inspection results

Audit results

Delivery schedule (late/early)

Other assessments

Concerns/issues/problems

7.8 AUDIT RESULTS

Number of findings by category

Trending data - time from finding to closure

Trending data - open versus closed audit reports

Trending data - corrective action closeouts

Trending data on open action items - 30, 60, 90, 180, etc. days

7.9 REVIEW RESULTS

Number of Requests for Action (RFAs) or Requests for Information (RFIs) by category

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

Trending data - time from issuing RFA/RFI to closure

Trending data - open versus closed RFAs/RFIs

Trending data – RFA/RFI closeouts

Trending data on open RFAs/RFI’s - 30, 60, 90, 180, etc. days

7.10 FAILURE/ANOMALY REPORT RESULTS

Number of Failure Reports (FRs) or Anomaly Reports (ARs) by category

Trending data - time from issuing FR/AR to closure

Trending data - open versus closed FRs/ARs

Trending data – FR/AR closeouts by category (e.g., test equipment problem, operator error, reworked, etc.)

Trending data on open FRs/AR - 30, 60, 90, 180, etc. days

7.11 NON-CONFORMANCE RESULTS

Number of Non-Conformance Reports (NCRs) by category

Trending data - time from issuing NCRs to closure

Trending data - open versus closed NCRs

Trending data – NCR closeouts by category (e.g., “use as is,” etc.)

Trending data on open NCR’s - 30, 60, 90, 180, etc. days

7.12 GOVERNMENT INDUSTRY DATA EXCHANGE PROGRAM (GIDEP)/NASA

ALERT/ADVISORY/PROBLEM REPORTS RESULTS (Note: All responses must include subcontractor, vendor, and supplier responses/results)

Number of requests for GIDEP/NASA Alert/Advisories/Problem status

Trending data - open versus closed requests

8.0 SUMMARY

This Surveillance Plan describes the approach NASA GSFC intends to use to monitor the PACE contract efforts and assure that contractors perform in accordance with terms and conditions of contracts. GSFC anticipates using a hybrid surveillance approach consisting of insight and oversight.

The goal is to apply the level of Government surveillance proportional to the perceived impacts and risks of mission failure.

GSFC plans to utilize a Surveillance Team to evaluate the contractor’s performance and direct surveillance activities. The Team will obtain data and provide information on contractor activities.

The Team will establish and rely on specific performance metrics and assess contractor performance against metric requirements/expectations.

Use or disclosure of data contained on this page is subject to the restriction(s) on the title page of this document.

As experience continues to be gained with the contractor, the Government reserves the right to change this Surveillance Plan and the metrics used to evaluate the contractor’s performance.

APPENDIX A: ABBREVIATIONS AND ACRONYMS

AETD Applied Engineering and Technology Directorate

AR Anomaly Report

CDRL Contract Data Requirements List

CM Configuration Management

CMO Configuration Management Office(r)

CMP Configuration Management Plan

CO Contracting Officer

COR Contracting Officer Representative

CSO Chief Safety & Mission Assurance Officer

DCMA Defense Contract Management Agency

DID Data Item Description

ESD Electrostatic Discharge

FAR Federal Acquisition Regulation

FR Failure Report

GFP Government Furnished Property

GIDEP Government Industry Data Exchange Program

GSFC Goddard Space Flight Center

IAGP Installation-Accountable Government Property

MIL Military

MIP Mandatory Inspection Point

MIS Management Information Systems

NASA National Aeronautics and Space Administration

NCAS NASA Contractor Assurance Services

NCR Non-Conformance Report

NPR NASA Procedural Requirement

PACE Pre-Aerosol, Clouds, and ocean Ecosystem (PACE)

PSM Project Safety Manager

QA Quality Assurance

RFA Request for Action

RFI Request for Information

S&MA Safety and Mission Assurance

SMAP Safety and Mission Assurance Plan

SOW Statement of Work

SSPP System Safety Program Plan