Enclosure 2 - NBOC Surveillance Plan September 2019.pdf

820-PLAN-2019-05

QUALITY ASSURANCE SURVEILLANCE PLAN

FOR IMPLEMENTATION OF THE

NASA BALLOON OPERATIONS CONTRACT

(NBOC)

Effective Date (Interim)

September 2019 Revision -

820/NASA Balloon Program Office

Goddard Space Flight Center Wallops Flight Facility

Wallops Island, Virginia 23337

National Aeronautics and Space Administration

This is an interim NBOC Surveillance Plan to address immediate requirements for Training & Certification and monitoring of the same. It replaces the previous plan dated January 2014, which was not in accordance with current BPO documentation numbering and signature approval. While this plan dated September 2019 is yet to be placed under the current BPO document control processes, it nonetheless addresses the same elements as the former plan with the addition of addressing Training and Certification to insure consistency with other Program Procedures and Guidelines. Other elements in this Surveillance Plan are currently under review and consideration of potential need for updating. The document number 820-PLAN-2019-05 is a placeholder that will remain consistent with first release under the current BPO document control processes, and for any subsequent revisions.

Interim Prepared and Released by:

David D. Gregory Date

820/NBOC COR

CHANGE HISTORY LOG

REVISION EFFECTIVE DATE DESCRIPTION OF CHANGES

- September 2019

Interim revised and issued to replace plan dated January 2014, which previously was not under current BPO document control processes. Preparations for initial release under current BPO document control processes underway; however, this interim release addresses immediate needs specific to Training & Certification while those preparations are underway.

FOREWORD

Under performance-based contracts such as this one, the Contractor assumes more responsibility and greater risk in exchange for more flexibility and less direct Government involvement in contract activities. However, the Government still has a responsibility to conduct surveillance. Surveillance spans a spectrum of Government involvement.

Surveillance may be as simple as inspecting a delivered support or service at acceptance or as complex as continually monitoring Contractor performance. To meet this responsibility, the Government needs to understand the risks involved in the Contractor's activity and how the Contractor is managing those risks.

This Government Quality Assurance Surveillance Plan has been prepared to describe the Government’s surveillance of this contract. It is a “living” document that will be tailored to the contractor selected and perhaps during a contract period, pending the need of the Government. The Government welcomes suggestions for improving this Plan. Of particular interest are ideas on what information the Government should monitor (i.e., metrics) and how the Government can most cost-effectively obtain the relevant performance data it needs.

NASA BALLOON OPERATIONS CONTRACT (NBOC)

GOVERNMENT QUALITY ASSURANCE SURVEILLANCE PLAN

1.0 INTRODUCTION

1.1 Purpose

This Quality Assurance Surveillance Plan (QASP) will define the process the National Aeronautics and Space Administration’s (NASA’s), Goddard Space Flight Center (GSFC) Wallops Flight Facility (WFF) Balloon Program Office (BPO) Code 820 expects to utilize to obtain data, evaluate the NBOC Contractor, and determine if contract performance is acceptable.

This QASP defines the process the Government expects to follow to obtain data, evaluate the Contractor, and determine if contract performance conforms to contract requirements. The goal is to balance the level of Government surveillance with perceived impacts and risks associated with performance hereunder. The QASP can be changed unilaterally by the Government at any time during the contract.

GSFC/WFF plans to utilize a surveillance team to evaluate NBOC Contractor performance and direct surveillance activities. The team will establish and rely on objective and subjective performance metrics based on the NBOC Statement of Work (SOW) and task orders issued thereunder to evaluate Contractor performance against requirements.

The QASP is a Government-developed surveillance tool prepared in accordance with FAR

46.601 and NFS 1846.401. It is not part of the contract, per NFS 1846.401, but provided to the Contractor for informational purposes only.

1.2 Scope

This QASP identifies the program requirements, strategies, resources, review and control processes, surveillance activities, and metrics for continuous measurement of Contractor performance. This plan provides effective and systematic surveillance methods for evaluating the Contractor’s services, processes, and products provided under this contract. The Government may evaluate work at any time during the Contractor’s work performance.

The intent of the QASP is to ensure that the Contractor performs in accordance with acceptable quality levels and the Government receives the quality of services and products called for in the contract. This QASP does not detail how the Contractor accomplishes the work. Rather, the QASP is based on the premise that the Contractor, not the Government, is responsible for managing its quality controls and ensuring that performance meets the terms of the contract. The role of the Government is quality assurance to ensure contract standards are achieved.

The QASP is intended to be a “living” document from which resources and activities will evolve from one phase to another during the life of the contract, and will be updated as required and defined in this document.

This plan is applicable to any service or product provided, as well as all areas in which work is being performed by the NBOC Contractor. Throughout this QASP, the term NBOC Contractor is used. Unless explicitly stated, this term is applicable to NBOC Contractor and any and all subcontractors, if applicable.

The surveillance program shall be a collaborative and integrated effort that includes all areas of contract management, including the following:

a. Engineering & Technology;

b. Quality Assurance;

c. Training, Certification and Quality of Workforce

d. Procurement/Subcontracting/Purchasing;

e. Finance;

f. Property;

g. Environmental;

h. Export Control;

i. Safety and Health; and

j. Security.

1.3 Program Definition and Contract Description

1.3.1 Program Background and Definition

The NASA BPO provides management of NASA’s high altitude scientific balloon activities funded by the Astrophysics Division within the NASA Science Mission Directorate (SMD).

The BPO provides this support in accordance with the Program Commitment Agreement (PCA) for the Research Carriers Program. BPO is managed within GSFC Sub-Orbital and Special Orbital Project Directorate (SSOPD), Code 800, located at WFF. BPO oversees management of the contract for operation of the Columbia Scientific Balloon Facility (CSBF), located in Palestine, Texas.

1.3.2 Contract Goals and Objectives

The purpose of this Cost-Plus-Fixed-Fee (CPFF) Core with, CPFF Indefinite Delivery Indefinite Quantity (IDIQ) Portion contract is to provide the labor, material, and equipment to support the management, operations, engineering, and technical support services for the NASA Balloon Program at the CSBF and its remote sites.

The NBOC Contractor is responsible for managing and providing all services and supplies (except those provided as Government or customer provided property) necessary for implementation of the NASA Balloon Program. As such, the NBOC Contractor plans, coordinates, develops, validates, inspects, reviews, controls and integrates customer supplied products, and performs flight qualification testing of balloon payloads. The NBOC Contractor controls nonconforming products and implements corrective and preventive actions; provides launch vehicles and associated standard systems and hardware; performs launch operations; and real-time mission support operations. The NBOC Contractor is responsible for operating and maintaining fabrication, testing, and operational facilities located at the Columbia Scientific Balloon Facility in Palestine, TX and remote facilities including Fort Sumner, NM, among others. The NBOC Contractor is also responsible for post flight operations which include:

recovery; refurbishment; post flight studies; and supporting investigations of anomalies, failures, and systemic problems associated with balloon vehicle systems, payload systems, ground support equipment, and operational methods. The NBOC Contractor performs handling, storage, packaging, and delivery functions associated with all these activities.

Engineering and Technical Support Services

The NBOC Contractor is responsible for providing general engineering and technical support for the Balloon Program and any associated projects. This effort typically requires expertise similar to that necessary for implementation of the NASA Balloon Program and includes the following:

general project management support; general mechanical, structural, electrical, and electronic fabrication and assembly support; general environmental testing and flight qualification support;

and general engineering support, engineering and environmental studies, system and subsystem development, general drafting and technical support, and education outreach activities.

1.4 Guiding Directives

The guiding documents for this surveillance effort include the Contract SOW, performance standards, deliverable requirements, and Task Order requirements as specified in issued Task Orders. The contract identifies general requirements and the Task Orders identify specific objectives or results desired for each issued Task Order requirement.

1.5 References and Applicable Documents

Applicable standard and policies documents are included in Attachment O of the NBOC SOW.

Among others they include:

a. American National Standard Quality Systems - Model for Quality Assurance (QA) in Design, Development, Production, Installation and Servicing, American National Standards Institute (ANSI)/International Organization for Standardization (ISO)/American Society for Quality Control (ASQC) Q9001: 2000

b. NASA Procedural Requirements (NPR) 7120.4, NASA Engineering and Program/Project Management Policy

c. NPR 7120.8, NASA Research and Technology Program and Project Management Requirements

d. NPR 8000.4, Agency Risk Management Procedural Requirements

e. NPR 8735.2, Management of Government Quality Assurance Functions for NASA

Contracts

f. NPD 8730.5, NASA Quality Assurance Program Policy

2.0 SURVEILLANCE STRATEGY AND APPROACH

2.1 General

There exists a wide-ranging spectrum associated with surveillance, ranging from oversight to insight. The strategy and approach to surveillance by GSFC/WFF for the NBOC contract, as detailed in this plan, is one that concentrates primarily on insight as opposed to oversight.

However, some limited areas do exist where oversight is conducted either via GSFC/WFF exercising approval authority on contract-deliverable documentation in critical areas of performance or participation through on-site involvement and/or inspection. Regardless, the Government reserves the right to initiate additional surveillance activities (insight or oversight) on an ‘as-needed’ basis, based upon circumstances and data collected (adverse trends, negative data points, lack of corrective action, etc.) via the surveillance activities defined in this plan. As applicable, any and all oversight activities would be communicated and coordinated with the Contractor and subsequently documented within this QASP.

The level of risk and the impact of failure are major determinants in helping define the type of surveillance to be conducted. Clearly, if the impact of failure is minor and the level of risk is low, only a small amount of insight-driven surveillance would normally be needed. Conversely, if the impact of failure could be significant and the level of risk is high, more extensive surveillance (including possible oversight surveillance) is warranted.

This insight-based approach to surveillance will utilize and leverage the NBOC Contractor's Quality Manual and Quality Management System (QMS). Definitions, requirements, and specifications contained in the contract, SOW, and referenced documents will establish a baseline for the surveillance activities. This insight-based approach will seek objective evidence and data that the NBOC Contractor's program and processes are functioning as intended in accordance with the terms of the contract. The focus will be on trusting NBOC Contractor's QMS, and verifying that the NBOC Contractor is performing according to the policies, procedures, plans, and processes defined by their QMS.

GSFC will strive to use an insight-driven surveillance approach throughout the performance of this contract. The overall surveillance goal will be to obtain objective evidence and data that enable the Government to determine whether the Contractor's program and processes are functioning as intended in accordance with the terms of the contract. The focus will be on prevention rather than detection, i.e., emphasizing controlled processes and methods of operation, as opposed to relying solely upon inspection and test to identify problems.

This insight-based approach to surveillance as applied to the contract will result in lower levels of Government intervention, thus allowing the NBOC Contractor to assume full accountability and responsibility for integrity of processes. Although less obtrusive than oversight, this insight-based approach to surveillance continues to provide the Government with visibility into the NBOC Contractor's programmatic processes, technical processes, progress, and issues at all levels.

As required by FAR 42.1502 and GPR 5100.2, Supplier Performance Evaluations, the Contracting Officer (CO), in collaboration with the Contracting Officer’s Representative (COR), will annually complete a Contractor Performance Assessment Reporting System (CPARS) evaluation, which will also be reviewed by the Contractor, and become a part of the Past Performance Information Retrieval System (PPIRS).

2.2 Surveillance Activity Limitations and Guidance

2.2.1 General

Surveillance of NBOC contract, will be conducted on a non-interference basis and in a manner that will not unduly delay work being performed by the NBOC Contractor.

2.2.2 Insight

Insight is an assurance process that uses performance requirements and, if definable, performance metrics to ensure process capability, product quality and end-item effectiveness.

Insight relies on gathering a minimum set of product or process data that provides adequate visibility into the integrity of the product or process. The data may be acquired from Contractor records, usually in a non-intrusive parallel method.

Insight as applied to this contract will result in lower levels of Government surveillance and allow the Contractor to assume increased responsibility and accountability for the integrity of processes. Insight will rely heavily on evaluating planned contract deliverables, performance standards, and existing Contractor procedures and working documents, if available.

2.2.3 Oversight

Oversight as applied to this contract will result in higher levels of Government surveillance. The Government will gather information pertaining to the Contractor’s process through on-site involvement and/or inspection in the process and will monitor the process itself. The Government’s involvement in the Contractor’s performance, through oversight, will be determined necessary by the COR.

2.3 Surveillance Organization and Resources

2.3.1 General

The activities detailed in this plan will be supported and performed by a group of individuals, many with differing levels of responsibilities, but all maintaining a level of consistency in terms of the surveillance strategy, approach, and activities in general. Each of these entities and their associated responsibilities/input to the surveillance activities on the NBOC are described in the following paragraphs.

2.3.2 Surveillance Team

2.3.2.1 General Organization and Responsibilities

General organization and responsibilities of the Surveillance Team are as follows:

a. The surveillance team will be composed of key NBOC Government personnel. All surveillance activities will be implemented using NASA and contractor support personnel. The surveillance team may be composed of:

1. GSFC/WFF Procurement Personnel (i.e., CO, Contract Specialist)

2. GSFC/WFF BPO support personnel (i.e., COR, Task Monitors and Technical

Monitors, and Resource/Financial Analysts);

b. The team’s primary purpose will be to provide direction for contract surveillance activities and to serve as the Government’s focal point in reviewing and evaluating overall Contractor performance under the NBOC contract. The team will obtain information from various sources, including deliverable Contractor documents, communications with the Contractor, and reports by other personnel or representatives (e.g., both Task Monitors and Technical Monitors) who interact with the Contractor.

c. NASA/GSFC/WFF has the responsibility for independently assuring that the NBOC Contractor's operations meet NASA's contract performance requirements and enable success. As such, surveillance team members will have open access to all areas in which this contract is being performed and will interface directly with their NBOC Contractor counterparts. Government expertise with regards to the NBOC contract may be applied in the form of technical consultants and/or providing assistance at working group meetings, design/development and specification reviews, review board meetings, surveys, audits, program reviews, and as in-plant representatives. The team will document problems, concerns and issues, and take note of Contractor accomplishments. They will collect performance metric data, where applicable, and will participate in Contractor review meetings, such as those described herein. Information will flow from individual team members through the COR to surveillance team representatives, who will present issues and achievements at surveillance team meetings. Information gained from these formal and informal exchanges of ideas and collection of data will be compiled and evaluated as a continuous measure of contract performance.

d. All available information will be evaluated, and any action by GSFC/WFF will be determined based upon the scope and magnitude of any particular issue or problem. The surveillance team chairperson, the COR, will formally notify the CO of situations where it is perceived that the Contractor has failed to take prudent corrective or preventive action, of situations that increase risk, or of findings of continued contractual non-compliance.

2.3.2.2 NBOC Contracting Officer

NBOC CO responsibilities are as follows:

a. The CO is responsible for ensuring performance of all necessary actions for effective contracting, ensuring compliance with the terms of the contract, issuing task orders, and safeguarding the interests of the United States in its contractual relationships. Within the surveillance area the CO takes inputs from the Program/Project managers, COR, and others to establish the detailed surveillance requirements to be performed by NASA personnel, delegated to another Federal agency via a GSFC/WFF Letter of Delegation, or to be performed under contract by a surveillance support Contractor. The CO will also assure that the Contractor receives impartial, fair, and equitable treatment under this contract. The CO is ultimately responsible for the final determination of the adequacy of the contractor’s performance.

b. The CO will complete an annual Contractor performance assessment report using the

CPARS that will also be reviewed by the Contractor and become a part of the PPIRS.

2.3.2.3 NBOC Contracting Officer’s Representative

NBOC COR responsibilities are as follows:

a. The COR is designated in writing by the CO to act as his or her authorized technical representative to assist in administering the contract. The COR monitors the technical work performed under the contract, evaluates Contractor performance, serves as the primary interface for the Contractor and the CO for all technical matters, reports on contract status to Program/Project Management, and recommends corrective action when necessary. The COR is not empowered to make any contractual commitments, authorize any contractual changes on the Government’s behalf, or in any way direct the Contractor to operate in conflict with the contract terms and conditions. Any changes that the Contractor deems may affect the contract value, terms, or conditions shall be referred to the CO for action. The COR’s limitations of authority are contained in the NASA Form 1634, COR Delegation.

b. The COR assumes full responsibility for directing the surveillance activities identified in this plan. The COR also trains both Task Monitors and Technical Monitors on evaluation procedures for evaluating contractor performance.

c. The COR will assist the CO in the completion of the contract’s annual performance assessment report using CPARS.

2.3.2.4 Task Monitors

GSFC/WFF Task Monitors are individuals appointed by the COR for developing Task Orders, reviewing the Contractor’s Task Plans and Task Order reports, and monitoring Task Order performance. Task Orders will include quantitative metrics, as appropriate. Task Monitors provide detailed technical oversight of the Contractor’s performance and report findings to the COR in a timely, complete and impartial fashion. While the Task Monitors may serve as a direct conduit to provide Government guidance and feedback to the Contractor on technical matters, the Task Monitors are not empowered to make any contractual commitments or to authorize any contractual changes on the Government’s behalf.

2.3.2.5 Technical Monitors

GSFC/WFF Technical Monitors are individuals appointed by the COR for the oversight of specific technical work on the contract. Technical Monitors provide detailed technical oversight of the Contractor’s performance and report findings to the COR in a timely, complete and impartial fashion. While the Technical Monitors may serve as a direct conduit to provide Government guidance and feedback to the Contractor on technical matters, the Technical Monitors are not empowered to make any contractual commitments or to authorize any contractual changes on the Government’s behalf.

2.3.2.6 NBOC Contractor Quality Assurance

It is expected that the selected NBOC Contractor will maintain a QA lead as part of its QMS. It is expected that the QA lead will perform QA-related activities for the NBOC efforts. The NBOC Contractor's QA lead will serve a vital role in the success of the surveillance efforts detailed in this plan. In particular, it is expected that the NBOC Contractor will task its QA lead to serve as a focal point for the Government in several areas including but not limited to provision of and access to all requested insight data/lifecycle-related assets and artifacts as they pertain to the insight areas described in this plan, and all QA-related activities conducted by this group.

The Government expects that as necessary and applicable, the QA lead may direct the Government to other groups/individuals supporting the NBOC effort in order to obtain requested insight data. These groups/individuals may include the NBOC Contractor's Program/Business Management office and/or representatives, discipline engineers, Configuration Management representatives, etc.

3.0 SURVEILLANCE STRATEGY AND APPROACH

3.1 General

The level of risk and the impact of failure are major determinants in defining the type of surveillance to be conducted. Insight-driven surveillance levels may be implemented if the risk and consequent failure are low. Similarly, insight-driven, and in some instances, oversight-driven surveillance levels may be implemented if the risk and consequent failure are high.

3.2 Surveillance Techniques

In an effort to minimize the performance management burden, simplified surveillance methods shall be used by the Government to evaluate Contractor performance when appropriate. The primary methods of surveillance are:

Ø Random monitoring - Performed by the COR or COR-designated inspector.

Ø Periodic Inspection – COR typically performs the periodic inspection on a monthly basis.

3.3 Surveillance Approach

GSFC will strive to use an insight-driven surveillance approach through the period of performance/effective ordering period of the contract. The overall surveillance goal will be to obtain objective evidence and data that enable the Government to determine whether the Contractor's program and processes are functioning as intended in accordance with the terms of the contract. Definitions, requirements, and specifications contained in the contract, SOW, and referenced documents will establish a baseline for the surveillance activities. The focus will be on prevention rather than detection, i.e., emphasizing controlled processes and methods of operation, as opposed to relying solely upon inspection and test to identify problems.

There are some limited areas where oversight will be conducted including onsite mission management, reviews, investigations, or exercising approval authority on contract deliverable documentation in critical areas. The Government reserves the right to expand existing and initiate additional surveillance activities (insight or oversight) on an “as-needed” basis, based on circumstances and data collected (adverse trends, negative data points, lack of corrective action) via the surveillance activities defined in this plan. Any and all oversight activities will be communicated and coordinated with the NBOC Contractor and subsequently documented in this plan.

The Government has the responsibility for independently assuring that the NBOC Contractor’s operations meet NASA’s performance requirements. As such, surveillance team members have open access, on a non-interference basis, to all areas in which NBOC work is being performed and will interface directly with their NBOC Contractor counterparts. Government expertise with regards to the NBOC effort may be applied in the form of technical consultants and/or providing assistance at working group meetings, design development and specification reviews, review board meetings, surveys, audits and program reviews.

3.4 Forms of Surveillance

Contract surveillance will consist of the following methods:

Communications – The Contractor is expected to establish and maintain both informal and formal communications processes with the Government, its personnel, and the customer. The surveillance team will monitor all communications including: teleconferences; informal discussions; electronic mail; Contractor-provided progress reports; campaign situation reports;

technical information meetings; and other communication methods, as needed.

Hardware and Software Product and Process Verifications – The Contractor is expected to establish and maintain robust processes and procedures in development of critical products in support of the NBOC. The surveillance team will monitor process reviews as provided in the task; Government Mandatory Inspection Point (GMIP); Random inspections, as necessary; and Onsite monitoring of Contractor activities. The Contractor shall identify GMIP’s with concurrence by the COR in the fabrication and refurbishment processes of critical flight systems and track quality progress.

Formal Reviews – The Contractor is expected to support or lead formal technical or non-technical reviews in performance of the contract. The surveillance team will monitor design reviews; mission readiness reviews; project initiation conferences; and flight readiness reviews

Evaluation and Reporting – The Contractor is expected to fulfill all obligations with regard to evaluating and reporting on their performance with regard to the contract, SOW, and individual IDIQ task orders. This includes review of Contract Deliverables from Contract Data Requirements List; documentation of problems, issues and concerns data collection; metrics evaluations; reporting; and customer feedback. Customer feedback may be obtained either from the results of formal customer satisfaction surveys provided by the Contractor, government, or from valid informal customer complaints.

4. SURVEILLANCE ACTIVITIES

4.1 General

There exist specific insight and oversight areas that the Government and the NBOC Contractor shall concentrate on during applicable stages of contract performance. Each of the insight areas and the Government’s expectations for these areas are described in Table 1. Each of the oversight areas and the Government’s expectations for these areas are described in Table 2.

Table 1. Surveillance Insight Areas

Area of Risk Identified Impact to Government Surveillance Team Activity System Maintenance and Availability

System downtime or loss of functionality could result in loss of service to the user community

Review Contractor-developed maintenance plan for improvements.

Review data and trouble data.

Review corrective action performance.

Information Technology (IT) Security

Computer Security: Potential corruption and loss of data;

disruption of schedule

Annual review of IT security plans and contingency test results and controls. Review compliance with policies, firewalls, protection software, vulnerability scans and external systems.

Configuration Management (CM) Documentation

Uncontrolled models, hardware or, software processes, or documents could lead to erroneous results, incompatible interfaces, wasted resources, and/or mission failure

Periodically sample current documentation through GMIP’s, and active management documents to verify compliance with the Contractor’s CM System and CM Plan.

Area of Risk Identified Impact to Government Surveillance Team Activity Property Management, Control, and Maintenance

Loss of or damage to equipment; potential schedule impact

Review Contractor property management techniques, compliance with policies, and record-keeping.

Safety Loss of work-time or equipment, with schedule or cost impact

Evaluate compliance with the Contractor's Safety and Health Plan and safety requirements.

Technical Documentation and Control

Loss of knowledge of processes and results

Periodically sample documents (review for accuracy) and ensure they are under CM control.

Process Controls Degradation of work products;

increase in safety risk;

potential schedule impact

Periodically monitor the Contractor’s adherence to key processes and their internal audit schedules/results.

Continuous Risk Management

Technical, cost, schedule, safety, and program success

Periodically ensure that the Contractor is performing a Continuous Risk Management program that identifies, analyzes, tracks, mitigates, controls and reports on related risks.

Quality Management Technical, cost, schedule, safety, and program success

Monitor the Contractor’s internal and external audits for compliance with the Contractor’s established Quality Management Systems, including ANSI/ISO/ASQ Q9001:2000 or AS 9100 compliance.

Quality of Work Force a. Inability to fill positions and meet commitments on scheduled deliverables or science results, including NASA Performance Metrics

b. Additional cost resulting from decreased productivity of other staff reliant on unfilled positions

c. Lack of expertise or inadequate experience in key areas

d. Delayed data delivery and/or poor data quality

a. Monitor time required to fill positions, and evaluate Contractor efforts and approaches used to fill vacancies.

b. Assess Contractor efforts to train staff in areas of required expertise.

c. Evaluate Contractor technical performance

d. Monitor progress and timeliness and evaluate the quality of data received.

Quality of Workmanship (End-Items)

a. Inability to meet commitments of scheduled deliverables

a. Monitor and track schedules and delivery due dates.

Area of Risk Identified Impact to Government Surveillance Team Activity

b. Additional cost and time resulting from rework, nonconforming, latent defects

b. Conduct/witness testing and inspections, when necessary. Ensure end-item deliverables conform prior to acceptance.

Schedule Services or products not provided in a timely manner can impact project schedule and cost

Monitor progress via management reviews and reporting.

Cost and Funding

Cost Overrun:

a. Inability to implement contract requirements within negotiated costs may lead to erosion of technical performance, delay, or deletion of work

b. Reduction of work due to funding limitations/fluctuations

Monitor and track costs incurred through the NASA Form 533, NASA Contractor Financial Management Report submitted on a monthly and quarterly basis.

Organizational Conflicts of Interest (OCI) Avoidance

Potential restrictions, ineligible to perform, and/or unfair competitive advantage on future work

Monitor submittal, enforcement and compliance with Contractor OCI Avoidance Plan.

Environmental Environmental damage to local and remote sites

Conduct periodic inspections to ensure compliance with environmental requirements.

Export Control Violation of International Traffic in Arms Regulations

(ITAR)

Ensure the Contractor has Technical Assistance Agreements as required by the NASA Export Control Program.

Training and Certification

Degradation of products, potential restrictions or ineligible to perform, increase in safety risk

Assess adequacy of Training & Certification plan in accordance with procedures and guidelines. Monitor for compliance with Training and Certification Plan and program requirements.

Facilities Degradation of program and project support to increase technical, safety, cost risk

Monitor ongoing operation and maintenance and the Construction of Facilities Plan

Task Orders TBD (as designated on individual Task Orders)

TBD (as designated on individual Task Orders)

Table 2. Surveillance Oversight Areas

Area of Risk Identified Impact to Government Surveillance Team Activity Mission Management Lapse in critical processes, increase in safety risk, loss or damage to mission or mishap, Provide onsite Mission Manager oversight for all missions (Safety and Mission Assurance).

Reviews Inadequate design, planning, scheduling, support, and risk mitigation

Provide panel support and monitor all formal reviews and consequent action items and closeout

Failure or Anomaly Investigations

Inadequate lessons learned, product or process changes, recurrence of failure

Provide Investigation chair and panel, conduct investigation, and ensure consequent action items and closeout addressed

4.2 Surveillance Team Activities

The surveillance team members will participate in review meetings, if applicable. They will provide support, as necessary, with the development and approval of technical requirements;

flow-down of requirements; and with design, development, production and test activities. They will also maintain insight into the NBOC Contractor’s compliance with relevant deliverables submitted under the contract and services performed. When the Government has concerns regarding Contractor performance, surveillance team members may conduct independent audits of the Contractor’s activities, processes, products, documentation and data, in order to provide assurance that the program is being implemented according to all requirements and specifications. These audits will normally be conducted with advance notification and coordinated with the Contractor. However, the Government reserves the right to conduct unscheduled audits when evidence indicates that Contractor performance is deficient.

The following selected surveillance team activities will be performed by various surveillance team members during applicable stages of contract performance:

4.2.1 Work Review and Performance Monitoring

The COR, with the assistance of the Task Monitors and Technical Monitors, will perform the following functions to evaluate the Contractor's performance:

a. Reviews specific SOW areas with the Technical Monitor(s) to assure that work being performed and deliverables are in accordance with the technical requirements of the SOW and timely.

Reviews individual Task Orders with the Task Monitors to assure that each Task Order is technically within the scope of the contract and its personnel requirements and schedule are within the Contractor's capabilities. Reviews Contractor Task Plans to ensure that performance estimates are acceptable and that all milestones and deliverables have been identified.

b. Reviews the Contractor's Monthly Progress Report for accuracy and completeness.

Consult with both Task Monitors and Technical Monitors, as necessary, to assess the fidelity of reports.

c. Meets monthly, or more often if required, with the Contractor's Program Manager to discuss overall contract management and performance, review staffing and schedule issues, and review cost related issues.

d. Certifies the Contractor’s invoices for payment in accordance with GSFC and WFF procedures.

e. Perform QA inspections and QA witnessing/monitoring of tests.

f. In the event of a discrepancy in the Contractor's performance, the COR promptly notifies both the CO and the Contractor's Program Manager and arranges a meeting to rectify the situation.

g. Assesses the Contractor’s overall contract performance.

4.2.2 Government Property Administration

The COR will carry out the following functions.

a. Review Contractor's request to purchase controlled property and facilities and make recommendations to the CO.

b. Validate that no property or facilities are being acquired without the CO's prior approval.

c. Conduct periodic inspection of equipment and its location, compared to the data on the property records.

d. Validate that all existing and new equipment is properly tagged.

e. Validate that Government property is made available in accordance with the terms of the contract.

f. Validate that the Contractor does not modify or provide additional facilities, plant equipment, or real property at GSFC, except as specifically required by the contract, or as directed or approved in writing by the CO.

g. Validate that the Contractor is maintaining records for all offsite Government-Furnished

Property (GFP).

4.2.3 Performance Monitoring

The COR will ensure that employer–employee relationships do not occur between Government and Contractor personnel. This is achieved if the following is adhered to:

a. Only the Contractor interviews prospective employees.

b. Only the Contractor’s Program Manager assigns work directly to the employees.

c. Only the Contractor approves timecards and absences.

d. Government personnel do not interfere with the Contractor regarding personnel and administrative prerogatives.

4.2.4 Safety

The responsibility for meeting all safety requirements rests with the Contractor. Surveillance team safety engineers and technical personnel will review Contractor-generated hazard analyses, safety compliance data packages or other safety-related documentation, as appropriate, to help ensure all safety requirements have been satisfied. Surveillance team personnel will also maintain insight into the Contractor’s safety activities through the review of the Contractor’s submitted Safety and Health Plan, and updates, as required by this contract.