Enclosure 7 (Quality Assurance Surveillance Plan).pdf

SOLICITATION NUMBER 80GSFC21R0037

NASA SOUNDING ROCKET OPERATIONS CONTRACT IV

(NSROC IV)

ENCLOSURE 7

QUALITY ASSURANCE SURVEILLANCE PLAN

JULY 2021

Solicitation No. 80GSFC21R0037 NASA Sounding Rocket Operations Contract IV ENCLOSURE 7 – Quality Assurance Surveillance Plan

FOREWARD

Under performance-based acquisitions such as this, the Contractor assumes more responsibility and greater risk in exchange for more flexibility and less direct Government involvement in contract activities. However, the Government still has a responsibility to conduct surveillance.

Surveillance spans a spectrum of Government involvement. Surveillance may be as simple as inspecting a delivered support or service at acceptance or as complex as continually monitoring contractor performance. To meet this responsibility, the Government needs to understand the risks involved in the Contractor's activity and how the Contractor is managing those risks.

This Government Quality Assurance Surveillance Plan has been prepared to describe the

Government’s surveillance of this contract. It is a “living” document that will be tailored to the contractor selected. The Government welcomes suggestions for improving this Plan. Of particular interest are ideas on what information the Government should monitor (i.e., metrics) and how the Government can most cost-effectively obtain the relevant performance data it needs.

(06/2014)

NASA SOUNDING ROCKET OPERATIONS CONTRACT IV (NSROC-IV)

GOVERNMENT QUALITY ASSURANCE SURVEILLANCE PLAN

1. INTRODUCTION

1.1 Purpose

The purpose of this Government Quality Assurance Surveillance Plan (QASP) is to define the overall approach the NASA Goddard Space Flight Center’s (GSFC) Wallops Flight Facility

(WFF) intends to use to monitor and survey Contractor performance under the NSROC-IV

Contract No. TBD. This QASP defines the process the Government expects to follow to obtain data, evaluate the Contractor, and determine if contract performance conforms to contract requirements. The goal is to balance the level of Government surveillance with perceived impacts and risks associated with performance hereunder. The QASP can be changed unilaterally by the Government at any time during the contract.

GSFC plans to utilize a surveillance team to evaluate Contractor performance and direct surveillance activities. The team will establish and rely on objective and subjective performance metrics based on the contract Statement of Work (SOW) to evaluate Contractor performance against requirements.

The QASP is a Government-developed surveillance tool prepared in accordance with FAR

46.601 and NFS 1846.401. It is not part of the contract, per NFS 1846.401, but provided to the

Contractor for informational purposes only.

1.2 Scope

This QASP identifies the program requirements, strategies, resources, review and control processes, surveillance activities, and metrics for continuous measurement of Contractor performance. This plan provides effective and systematic surveillance methods for evaluating the Contractor services, processes, and products provided under this contract. The Government may evaluate work at any time during the Contractor’s work performance.

The intent of the QASP is to ensure that the Contractor performs in accordance with acceptable quality levels and the Government receives the quality of services and products called for in the contract. This QASP does not detail how the Contractor accomplishes the work. Rather, the

QASP is based on the premise that the contractor, not the Government, is responsible for managing its quality controls and ensuring that performance meets the terms of the contract. The role of the Government is quality assurance to ensure contract standards are achieved.

The QASP is intended to be a “living” document from which resources and activities will evolve from one phase to another during the life of the contract, and will be updated as required and defined in this document.

This plan is applicable to any service or product provided, as well as all areas in which work is being performed by the NSROC-IV Contractor(s). Throughout this QASP, the term NSROC-IV

Contractor is used. In terms of this plan, it should be known that unless explicitly stated, this term is applicable to both the NSROC-IV Contractor and any and all subcontractors.

The surveillance program shall be a collaborative and integrated effort that includes all areas of contract management, including the following:

a. Engineering & Technology

b. Quality Assurance

c. Procurement/Subcontracting/Purchasing

d. Finance

e. Property

f. Environmental

g. Export Control

h. Safety and Health

i. Security

1.3 Program Definition and Contract Description

1.3.1 Program Background and Definition

The Sounding Rockets Program at GSFC/WFF Code 810 utilizes expendable sub-orbital rockets to conduct a host of scientific missions for the study of near earth and space environments and to advance new technologies. The time required to conduct a specific mission from payload design and development through launch will vary substantially and may range from a few months to two years or longer. Most of these missions are conducted from established launch ranges, while some are conducted as mobile launch campaigns from ranges that have been temporarily established. The Sounding Rockets Program risk posture is consistent with NASA’s goal to provide Low Cost Access to Space (LCAS). Consequently, the Program is governed by NASA

NPR 7120.8, NASA Research and Technology Program and Project Management Requirements.

Furthermore, the Program operates under an 85% mission success metric established by NASA

Headquarters, thus allowing the Program to be tolerant of elevated technical risks on each mission.

1.3.2 Contract Goals and Objectives:

The GSFC goal for this NSROC-IV contract is to implement the NASA Sounding Rockets

Program, as well as provide engineering and technical support services for other programs and projects.

The purpose of this Cost-Plus-Award Fee contract is to manage and provide all services and supplies (except those provided as Government or customer provided property) necessary for implementation of the NASA Sounding Rockets Program. As such, the NSROC-IV Contractor plans, develops, validates, and reviews designs; fabricates, inspects (including control of inspection, test, and measurement equipment and maintenance of inspection status), controls and integrates customer supplied product into, and performs flight qualification testing of sounding rocket payloads; controls nonconforming products and implements corrective and preventive actions; provides launch vehicles and associated standard systems and hardware; performs activities associated with mission launch operations; and applies statistical techniques to these processes as appropriate. The NSROC-IV Contractor is responsible for operating and maintaining fabrication, testing, and operational facilities located at Wallops Flight Facility and

White Sands Missile Range. The NSROC-IV Contractor is also responsible for post flight operations which include: the reduction of scientific data; post flight studies; and the investigation of anomalies, failures, and systemic problems associated with flight vehicle systems, payload systems, ground support equipment, and analytical methods. The NSROC-IV

Contractor performs handling, storage, packaging, and delivery functions associated with all these activities.

1.4 Guiding Directives

The guiding documents for this surveillance effort include the Contract SOW, performance standards, and deliverable requirements.

1.5 References and Applicable Documents

[List any documents external to the contract that affect the surveillance approach like NPRs, NPDs, GPRs, GPDs, PGs, NASA Standards, GSFC Standards, other standards. Examples of documents that could be listed are:

a. 810-PG-1310.1.1 - Sounding Rockets Program Office Process For Establishing Customer

Requirements

b. 810-PG-1410.2.1 – Sounding Rockets Program Office System for Identification of

Sounding Rocket Missions

c. 810-PG-5100.1.1 - Management Of The NASA Sounding Rocket Operations Contract

(NSROC)

d. 810-PG-5100.1.2 - SRPO Supplier Performance Evaluation

e. 810-PG-5100.1.3 – NSROC Development and Routine Project Assignments Process

f. 810-PG-7120.1.1 - Review Of NSROC Sounding Rocket Mission Work Orders

g. 810-PG-7120.1.2 - Review Requirements for NASA Sounding Rockets Program Office

Developmental Projects

h. NPR 7120.8, NASA Research and Technology Program and Project Management

Requirements

i. NPR 8735.2, Management of Government Quality Assurance Functions for NASA

Contracts

2. SURVEILLANCE STRATEGY AND APPROACH

2.1 General

There exists a wide-ranging spectrum associated with surveillance, ranging from oversight to insight. The strategy and approach to surveillance by GSFC/WFF for NSROC-IV contract, as detailed in this plan, is one that concentrates primarily on insight as opposed to oversight.

However, some limited areas do exist where oversight is conducted either via GSFC/WFF exercising approval authority on contract-deliverable documentation in critical areas of performance or participation in the Contractor's configuration management process. Regardless, the Government reserves the right to initiate additional surveillance activities (insight or oversight) on an ‘as-needed’ basis, based upon circumstances and data collected (adverse trends, negative data points, lack of corrective action, etc.) via the surveillance activities defined in this plan. As applicable, any and all oversight activities would be communicated and coordinated with the Contractor and subsequently documented within this QASP.

The level of risk and the impact of failure are major determinants in helping define the type of surveillance to be conducted. Clearly, if the impact of failure is minor and the level of risk is low, only a small amount of insight-driven surveillance would normally be needed. Conversely, if the impact of failure could be significant and the level of risk is high, more extensive surveillance (including possible oversight surveillance) is warranted.

This insight-based approach to surveillance will utilize and leverage the NSROC-IV Contractor's

Quality Assurance Plan (QAP) and Quality Management System (QMS). Definitions, requirements, and specifications contained in the contract, SOW, and referenced documents will establish a baseline for the surveillance activities. This insight-based approach will seek objective evidence and data that the NSROC-IV Contractor's program and processes are functioning as intended in accordance with the terms of the contract. The focus will be on trusting the NSROC-IV Contractor's QMS, and verifying that the NSROC-IV Contractor is performing according to the policies, procedures, plans, and processes defined by their QMS.

GSFC/WFF will strive to use an insight-driven surveillance approach throughout the performance of this contract. The overall surveillance goal will be to obtain objective evidence and data that enable the Government to determine whether the Contractor's program and processes are functioning as intended in accordance with the terms of the contract. The focus will be on prevention rather than detection, i.e., emphasizing controlled processes and methods of operation, as opposed to relying solely upon inspection and test to identify problems.

This insight-based approach to surveillance as applied to the contract will result in lower levels of Government intervention, thus allowing the NSROC-IV Contractor to assume full accountability and responsibility for integrity of processes. Although less obtrusive than oversight, this insight-based approach to surveillance continues to provide the Government with visibility into the NSROC-IV Contractor's programmatic processes, technical processes, progress, and issues at all levels.

As required by FAR 42.1502 and GPR 5100.2, Supplier Performance Evaluations, the

Contracting Officer (CO), in collaboration with the Contracting Officer’s Representative (COR), will annually complete a Contractor Performance Assessment Reporting System (CPARS) evaluation, which will also be reviewed by the Contractor

2.2 Surveillance Activity Limitations and Guidance

2.2.1 General

Surveillance of NSROC-IV contract, will be conducted on a non-interference basis and in a manner that will not unduly delay work being performed by the NSROC-IV Contractor.

2.2.2 Insight

Insight is an assurance process that uses performance requirements and, if definable, performance metrics to ensure process capability, product quality and end-item effectiveness.

Insight relies on gathering a minimum set of product or process data that provides adequate visibility into the integrity of the product or process. The data may be acquired from Contractor records, usually in a non-intrusive parallel method.

Insight as applied to this contract will result in lower levels of Government surveillance and allow the Contractor to assume increased responsibility and accountability for the integrity of processes. Insight will rely heavily on evaluating planned contract deliverables, performance standards, and existing Contractor procedures and working documents, if available.

2.2.3 Oversight

Oversight as applied to this contract will result in higher levels of Government surveillance. The

Government will gather information pertaining to the Contractor’s process through on-site involvement and/or inspection in the process and will monitor the process itself. The

Government’s involvement in the Contractor’s performance, through oversight, will be determined necessary by the COR.

2.3 Surveillance Organization and Resources

2.3.1 General

The activities detailed in this plan will be supported and performed by a group of individuals, many with differing levels of responsibilities, but all maintaining a level of consistency in terms of the surveillance strategy, approach, and activities in general. Specific entities supporting the

NSROC-IV contract surveillance activities include the identified NASA personnel; NSROC-IV

Contractor QA Department personnel (including their subcontractors); and contractor support services and delegated agency personnel, if applicable. Each of these entities and their associated responsibilities/input to the surveillance activities on NSROC-IV contract are described in the following paragraphs.

2.3.2 Surveillance Team

2.3.2.1 General Organization and Responsibilities

General organization and responsibilities of the Surveillance Team are as follows:

a. The surveillance team will be composed of key NSROC-IV Government personnel. All surveillance activities will be implemented using NASA personnel. The surveillance team may be composed of:

1. GSFC/WFF Procurement Personnel (i.e., CO, Contract Specialist)

2. GSFC/WFF’s Sounding Rockets Program Office support personnel (i.e., COR, and

Task Monitor(s), and Resource/Financial Analyst(s));

3. GSFC/WFF Safety & Health and Security personnel (both physical and Information

Technology (IT) Security);

4. GSFC/WFF Property Administrator personnel.

b. The team’s primary purpose will be to provide direction for contract surveillance activities and to serve as the Government’s focal point in reviewing and evaluating overall Contractor performance under the NSROC-IV contract. The team will obtain information from various sources, including deliverable Contractor documents, communications with the Contractor, and reports by other personnel or representatives

(e.g., Task Monitor(s), GSFC Health & Safety personnel) who interact with the

Contractor.

c. NASA/GSFC/WFF has the responsibility for independently assuring that the NSROC-IV

Contractor's operations meet NASA's contract performance requirements and enable success. As such, surveillance team members will have open access to all areas in which this contract is being performed and will interface directly with their NSROC-IV

Contractor counterparts. Government expertise with regards to the NSROC-IV contract may be applied in the form of technical consultants and/or providing assistance at working group meetings, design/development and specification reviews, review board meetings, surveys, audits, program reviews, and as in-plant representatives. The team will document problems, concerns and issues, and take note of Contractor accomplishments. They will collect performance metric data, where applicable, and will participate in Contractor review meetings, such as those described herein. Information will flow from individual team members and task monitor(s) to the COR at quarterly surveillance team meetings. Information gained from these formal and informal exchanges of ideas and collection of data will be compiled and evaluated as a continuous measure of contract performance.

d. All available information will be evaluated, and any action by GSFC/WFF will be determined based upon the scope and magnitude of any particular issue or problem. The surveillance team chairperson, the COR, will formally notify the CO of situations where it is perceived that the Contractor has failed to take prudent corrective or preventive action, of situations that increase risk, or of findings of continued contractual non-compliance.

2.3.2.2 NSROC-IV Contracting Officer

NSROC-IV CO responsibilities are as follows:

a. The CO is responsible for ensuring performance of all necessary actions for effective contracting, ensuring compliance with the terms of the contract, and safeguarding the interests of the United States in its contractual relationships. Within the surveillance area the CO takes inputs from the Program/Project managers, COR, and others to establish the detailed surveillance requirements to be performed by NASA personnel. The CO will also assure that the Contractor receives impartial, fair, and equitable treatment under this contract. The CO is ultimately responsible for the final determination of the adequacy of the contractor’s performance.

b. The CO will complete an annual Contractor performance assessment report using the

CPARS that will also be reviewed by the Contractor.

2.3.2.3 NSROC-IV Contracting Officer’s Representative

NSROC-IV COR responsibilities are as follows:

a. The COR is designated in writing by the CO to act as his or her authorized technical representative to assist in administering the contract. The COR monitors the technical work performed under the contract, evaluates Contractor performance, serves as the primary interface for the Contractor and the CO for all technical matters, reports on contract status to Program/Project Management, and recommends corrective action when necessary. The COR is not empowered to make any contractual commitments, authorize any contractual changes on the Government’s behalf, or in any way direct the Contractor to operate in conflict with the contract terms and conditions. Any changes that the

Contractor deems may affect the contract value, terms, or conditions shall be referred to the CO for action. The COR’s limitations of authority are contained in the NASA Form

1634, COR Delegation.

b. The COR assumes full responsibility for directing the surveillance activities identified in this plan. The COR also trains Task Monitor(s) on evaluation procedures for evaluating contractor performance.

c. The COR will assist the CO in the completion of the contract’s annual performance assessment report using CPARS.

2.3.2.4 Task Monitor(s)

GSFC task monitor(s) are individuals appointed by the COR for the oversight of specific technical work on the contract. Task monitors provide detailed technical oversight of the

Contractor’s performance and report findings to the COR in a timely, complete and impartial fashion. While the task monitors may serve as a direct conduit to provide Government guidance and feedback to the Contractor on technical matters, the task monitors are not empowered to make any contractual commitments or to authorize any contractual changes on the Government’s behalf.

2.3.2.5 NSROC-IV Contractor Quality Assurance

It is expected that the selected NSROC-IV Contractor will maintain a QA lead as part of its

QMS. It is expected that the QA lead will perform QA-related activities for the NSROC-IV efforts. The NSROC-IV Contractor's QA lead will serve a vital role in the success of the surveillance efforts detailed in this plan. In particular, it is expected that the NSROC-IV

Contractor will task its QA lead to serve as a focal point for the Government in several areas including but not limited to provision of and access to all requested insight data/lifecycle-related assets and artifacts as they pertain to the insight areas described in this plan, and all QA-related activities conducted by this group.

The Government expects that as necessary and applicable, the QA lead may direct the

Government to other groups/individuals supporting the NSROC-IV effort in order to obtain requested insight data. These groups/individuals may include the NSROC-IV Contractor's

Program/Business Management office and/or representatives, discipline engineers, Configuration

Management representatives, etc.

2.4 Forms of Surveillance

2.4.1 General

Surveillance on NSROC-IV contract will be performed using any of the primary surveillance forms applied to the insight areas described in Section 3 of this document, during applicable stages of the NSROC-IV contract. These primary forms of surveillance are described below.

2.4.2 Communications

Communications is a general surveillance activity. Communications is a two-way process and includes both written and oral communication. Examples of written communications activities that may be used in conducting surveillance include:

a. Exchanges from the NSROC-IV Contractor to the Government of plans, procedures, quality records, reports, etc., and/or provision of read-only access to repositories which retain these items.

b. Exchanges from the Government to NSROC-IV Contractor of letters, reports, review results, etc.

c. Ad hoc information submitted by COR and/or Task Monitor(s) to the CO related to the

NSROC-IV Contractor’s electronic mail.

Examples of oral communications activities that may be used in conducting include:

a. Informal telephone calls, teleconferences.

b. Informal verbal inquiries, discussions, engineering consultations.

c. Working group meetings, IPT participation, technical/status briefings, progress reviews, technical information meetings, and formal and informal reviews.

d. Informal discussions.

2.4.3 Management Reviews and Reporting

Examples of management review and reporting activities that may be used in conducting surveillance include:

a. Formal, process, and progress reviews

b. Review of contract deliverables

c. Documentation of problems, issues and concerns

d. Data collection reporting

e. Review of deliverables, products, and documentation

2.4. Participation in NSROC-IV Contractor Configuration Management Processes

The NSROC-IV Contractor is required to facilitate NASA insight into the contractor configuration management process. This process will be accomplished through NASA participation in the contractor configuration management process, and insight into NSROC-IV

Contractor configuration controlled documentation.

3. SURVEILLANCE ACTIVITIES

3.1 General

There exist specific insight areas that the Government and the NSROC-IV Contractor shall concentrate on during applicable stages of contract performance. Each of these insight areas and the Government’s expectations for these areas are described in Table 1.

Table 1. Surveillance Insight Areas

Area of Risk Identified Impact to Government Surveillance Team Activity

Information Technology (IT)

Security

Computer Security: Potential corruption and loss of data;

disruption of schedule

Review of IT security plan upon delivery and when revised. Review compliance with policies, firewalls, protection software, vulnerability scans and external systems.

Configuration Management

(CM) Documentation

Uncontrolled models, hardware, software, or documents could lead to erroneous results, incompatible interfaces, wasted resources, and/or mission failure

Periodically sample current documentation, and active management documents to verify compliance with the Contractor’s CM

System and CM Plan.

Property Management, Control, and Maintenance

Loss of or damage to equipment;

potential schedule impact

Review Contractor property management techniques, compliance with policies, and record-keeping.

Safety Loss of work-time or equipment, with schedule or cost impact

Evaluate compliance with the Contractor's

Safety and Health Plan and safety requirements.

Technical Documentation and

Control

Loss of knowledge of processes and results

Periodically sample documents (review for accuracy) and ensure they are under CM control.

Process Controls Degradation of work products;

increase in safety risk; potential schedule impact

Periodically monitor the Contractor’s adherence to key processes and audit schedules/results.

Risk Management Technical, cost, schedule, safety, and program success

Periodically ensure that the Contractor is conducting a Risk Management program that identifies, analyzes, tracks, mitigates, controls and reports on related risks.

Quality Management Technical, cost, schedule, safety, and program success

Monitor the Contractor’s audits for compliance with the Contractor’s established

Quality Management Systems, ISO9001 or

AS9100.

Quality of Work Force a. Inability to fill positions and meet commitments on scheduled deliverables or science results, including NASA Performance

Metrics

b. Additional cost resulting from decreased productivity of other staff reliant on unfilled positions

c. Lack of expertise or inadequate experience in key areas

d. Delayed data delivery and/or poor data quality

a. Monitor time required to fill positions, and evaluate Contractor efforts and approaches used to fill vacancies.

b. Assess Contractor efforts to train staff in areas of required expertise.

c. Evaluate Contractor technical performance

d. Monitor progress and timeliness and evaluate the quality of data received.

Schedule Services or products not provided in a timely manner can impact project schedule and cost

Monitor progress via management reviews and reporting.

Cost and Funding

Cost Overrun:

a. Inability to implement contract requirements within negotiated costs may lead to erosion of technical

Monitor and track costs incurred through the

NASA Form 533, NASA Contractor

Financial Management Report submitted on a monthly and quarterly basis.

performance, delay, or deletion of work

b. Reduction of work due to funding limitations/fluctuations

Organizational Conflicts of

Interest (OCI) Avoidance

Potential restrictions, ineligible to perform, and/or unfair competitive advantage on future work

Monitor submittal, enforcement and compliance with Contractor OCI Avoidance

Plan.

Environmental Environmental damage to local and remote sites

Conduct periodic inspections to ensure compliance with environmental requirements.

Export Control Violation of International Traffic in

Arms Regulations (ITAR)

Ensure the Contractor has Technical

Assistance Agreements as required by the

NASA Export Control Program.

Overall Mission Technical

Performance

Potential violation of Program success metric: 85% mission success

Evaluation of all mission milestone meetings; monitoring of design, fabrication, testing, and launch operations; pre-launch vehicle walk-down.

Overall Development and

Routine Project Assignments

(DRPA) and Performance

Task Order (PTO) Technical

Performance

Technical, cost, schedule, safety, and program success

Regularly monitor Contractor’s progress via required DRPA and PTO status meetings and daily interaction with Contractor.

3.2 Surveillance Team Activities

The surveillance team members will participate in quarterly review meetings, if applicable.

They will provide support, as necessary, with the development and approval of technical requirements; flow-down of requirements; and with design, development, production and test activities. They will also maintain insight into the Contractor’s compliance with relevant deliverables submitted under the contract and services performed. When the Government has concerns regarding Contractor performance, surveillance team members may conduct independent audits of the Contractor’s activities, processes, products, documentation and data, in order to provide assurance that the program is being implemented according to all requirements and specifications. These audits will normally be conducted with advance notification and coordinated with the Contractor. However, the Government reserves the right to conduct unscheduled audits when evidence indicates that Contractor performance is deficient.

3.2.1 Product Assurance

The following selected activities will be performed during applicable stages of contract performance:

3.2.1.1 Government Mandatory Inspection Points for Safety Critical Processes

For circumstances where noncompliance can result in loss of life, Government Mandatory

Inspection Points (GMIP) shall be performed to ensure 100 percent compliance with all safety critical processes. These safety critical Government Mandatory Inspections (process witnessing) shall be performed by qualified Operational Safety Supervisors (contractor or civil servant) and documented by their signatures on the corresponding safety critical procedures. All Operational

Safety Supervisors shall be certified and licensed by the NASA Wallops Safety Office.

3.2.1.2 Government Mandatory Inspection Points for Mission Critical Reviews

Government Mandatory Inspection Points shall be conducted for 100 percent of all Mission

Critical Reviews. These reviews shall include all Requirements Definition Meetings, Design

Review Meetings, and Mission Readiness Review Meetings. A representative of the Sounding

Rockets Program Office shall witness each review meeting and complete a formal evaluation of the review in accordance with the Sounding Rockets Program Office’s Procedure and Guideline

810-PG-5100.1.2.

3.2.1.3 Government Mandatory Inspection Points for Prelaunch Operations

Representatives of the Sounding Rockets Program shall conduct a review (Government

Mandatory Inspection) in accordance with the Sounding Rockets Program Office’s Procedure and Guideline 810-PG-7120.1.1 for 100 percent of all Missions prior to the initiation of launch operations. This Government Mandatory Inspection shall verify that all Corrective Actions

(Action Items), Non-conformances, and Engineering Departures (i.e. engineering changes after baseline design) have been approved by the NSROC IV Contractor and resolved to the satisfaction of the Sounding Rockets Program Office. Successful completion of this mandatory inspection shall be documented by the Sounding Rockets Program Office’s issuance of a formal

“Notification of Approval to Proceed with Launch Operations” in accordance with the aforementioned Procedure and Guideline.

3.2.1.4 Government Mandatory Inspection Points for Launch Operations

Representatives of the Sounding Rockets Program Office shall conduct a Government

Mandatory Inspection to formally witness Launch Operations for 100 percent of all Sounding

Rockets Missions. This shall include (but not be limited to) the launch vehicle and payload

Walk-Down Inspection, Payload Range Horizontal Test, Payload Range Vertical Test, and the final Mission Countdown and Launch Operation.

3.2.1.5 Government Mandatory Inspection Points for Mission Closeout

Government Mandatory Inspection Points shall be conducted for 100 percent of all Mission

Closeout Reports. A representative of the Sounding Rockets Program Office shall review each

Mission Closeout Report and complete a formal evaluation of the Report in accordance with the

Sounding Rockets Program Office’s Procedure and Guideline 810-PG-5100.1.2.

3.2.1.6 Monthly Program Reviews

The Contractor shall conduct Program Reviews on a monthly basis with active participation by the COR, Contracting Officer, and other Government representatives from the Sounding Rockets

Program Office. These reviews provide an opportunity to assess segments of the entire program and provide inputs on current status, accomplishments, and issues to date. Cognizant

Government managers and engineers will interface directly with designated Contractor managers and review the current status of various topics such as Safety and Mission Assurance, Mission/DRPA/PTO Performance, Programmatic Issues (i.e. ITAR, staffing, subcontract status, outreach activities, training, anomaly investigations, discipline specific issues (engineering, manufacturing, etc.), facilities and equipment status, etc.), and Financial Status.

The Government shall also conduct bi-weekly Mission Scheduling and Prioritization Reviews with Contractor representatives where the status and prioritization of each individual Mission is reviewed and any issues brought to light for discussion.

The Contractor shall also conduct a monthly DRPA and PTO Review with representatives of the

Sounding Rockets Program Office where the status and prioritization of DRPA work orders and

PTO task orders are reviewed and any issues brought to light for discussion.

3.2.2 Nonconformance Reporting and Corrective/Preventive Action

The NSROC IV Contractor is responsible for managing configuration control in accordance with

SOW requirements and their Quality Manual. Configuration Control applies to all mission critical processes, procedures, documentation, hardware, and software. Changes affecting contract requirements must be identified to the CO and COR for disposition.

Non-conformances and associated dispositions for individual components or subsystems that transpire during the implementation of a Mission or DRPA or during the acquisition process

(fabrication, procurement, etc.) associated with development/maintenance of the Sounding

Rockets Program stock inventory system are documented by the NSROC IV Contractor’s Non-

Conformance Reporting (NCR) System and tracked to closure. Relevant non-conformances will be reported out at all Design Review and Mission Readiness Meetings associated with individual

Missions, DRPAs, or PTOs.

Systemic non-conformances or non-conformances that result from formal audits or as the result of anomaly investigations are documented by the NSROC IV Contractor’s Corrective Action

Reporting (CAR) System and tracked to closure.

The NSROC IV Contractor documents engineering changes that occur after a baseline design has been established through their Change Request (CR) System.

The NSROC IV Contractor conducts a weekly meeting of the Material Review Board (MRB) to review, discuss, edit (if necessary), and close NCR’s, CAR’s, and CR’s. The NSROC Safety and

Mission Assurance Manager chairs the MRB with participation by individual NSROC

Engineering Discipline Leads and Senior Program Management who provide approval authority for NCR, CAR, and CR closure. The Sounding Rockets Program Office’s Payload Systems

Manager and/or the Safety and Mission Assurance Manager also participates in this meeting and monitors the process.

The NSROC IV Contractor conducts a dedicated post-launch review for individual Missions to determine any Lessons Learned (LL) during their implementation. A resulting memo and database entries into the NSROC Lessons Learned Database capture these lessons for propagation to subsequent missions. The database is utilized by Mission Managers of newly initiated missions to search for lessons captured on previous similar missions.

3.2.3 Quality Assurance Metrics

The NSROC IV Contractor’s overall performance in meeting Statement of Work (SOW) contractual requirements will be assessed on an annual basis by the Government. The SOW itself will be utilized as a guideline/checklist to assess the Contractor’s performance to determine if systemic failures of performance exist in meeting any SOW requirement(s). The Contracting

Officer’s Representative shall make the final determination of the severity of systemic failure based on input from and discussions with the associated Performance Monitor(s). These

Performance Monitors will have broad background and experience in developing and implementing sounding rocket missions and will maintain ongoing insight into the Contractor’s compliance with contract requirements. Systemic failures of performance in meeting specific

SOW requirements will be documented by the COR and promptly reported back to the

Contractor in a written annual evaluation of performance. Systemic nonperformance by the

Contractor may result in documented entry into the Contractor Performance Assessment Report

System (CPARS). This may also result in enhanced oversight surveillance approach until the problems are effectively addressed.

Award Fee will be calculated and awarded based on Contract Clause J.1, Attachment S, Award

Fee Plan.

3.2.4 Quality System Evaluation

The NSROC IV Contractor shall develop, implement, and maintain a Quality Management

System that complies with all requirements of ISO 9001 or AS9100 and obtain certification from an independent Accredited Registrar within eight (8) months of contract effective date. This shall include development of a Quality Manual in accordance with ISO 9001 or AS9100 and the effective implementation of the requirements, policies, practices, and procedures contained and referenced therein. This Quality Manual shall be provided to and approved by the Sounding

Rockets Program Office’s Contracting Officer’s Representative (COR) for the NSROC IV contract.

The NSROC IV Contractor shall maintain ISO 9001 or AS9100 certification by an independent

Accredited Registrar throughout the life of the NSROC IV contract and successfully complete independent Accredited Registrar audits on a semiannual basis. The results of all such audits shall be provided to the Sounding Rockets Program Office’s COR for the NSROC IV contract for review and confirmation of the NSROC IV Contractor’s compliance with the requirements of this section.

When the Government has concerns regarding Contractor performance, the Government may conduct independent audits of the Contractor's Quality Management System, activities, processes, products, documentation, and data in order to provide assurance that the program is being implemented according to all contractual requirements. These audits will normally be conducted with advance notification and coordinated with the Contractor. However, the

Government reserves the right to conduct unscheduled audits when evidence indicates that

Contractor performance is deficient.

3.2.5 Risk Management

The Sounding Rockets Program Office recognizes that risk management is an inherent part of any quality assurance system and has therefore incorporated risk management as a component of its overall quality assurance system. This includes conducting an ongoing assessment of what could go wrong, determining what risks are important, and implementing risk management strategies that are reasonable and commensurate with the probable adverse effects should a mishap occur. In order to establish the strategy for managing technical risks for all SRPO missions and projects, the Sounding Rockets Program Office has developed the Sounding

Rockets Program Office Risk Management Plan (controlled document 810-RMP-0001). At the core of this approach is the assignment of risk management responsibilities to the appropriate management level where there is direct professional involvement and awareness of the impact of risks, and where identification, mitigation, and reporting activities become an integral component of program mission and project management planning, budgeting, and execution. A significant portion of these responsibilities has been delegated to the Risk Management Board (RMB).

Roles and responsibilities of the RMB for managing technical risks are defined in this NSROC

IV SOW and the Sounding Rockets Program Office Risk Management Plan. Representatives of the Sounding Rockets Program Office will monitor the performance of those responsibilities assigned to the NSROC-IV Contractor on an ongoing basis.

The NASA Wallops Safety Office manages safety risks for Sounding Rocket missions in conjunction with other range safety organizations at launch ranges where SRPO missions are conducted. These organizations have independent, long-standing, documented processes addressing how safety risks are managed and dispositioned that are independent of the Sounding

Rockets Program Office Risk Management Plan for technical risks and beyond the scope of this plan.

3.2.6 Final Acceptance

Prior to initiating any scheduled launch operation for Missions, the NSROC IV Contractor conducts a Flight Readiness Review to ensure all processes have been properly followed and all assigned actions (NCR, CAR, CR, Design & Mission Readiness Review Action Item, etc.) have been closed. Completion of this process is documented in a memorandum from the NSROC IV

Program Manager to the Chief of the Sounding Rockets Program Office. This memorandum also provides a formal recommendation regarding the flight readiness of the mission to proceed with launch operation. The Chief of the Sounding Rockets Program Office then meets with

NASA Code 800 Senior Management in an Authorization to Proceed meeting to review the launch readiness of the mission. Concurrence from the Director of Code 800; Chief, Range and

Mission Management Office; Chief, Wallops Safety Office, and Chief, Sounding Rockets

Program Office (or their designees) is required in order for the mission to be approved to proceed with launch operations. The mission is formally authorized to proceed with launch operations once consensus is achieved.

Prior to final acceptance and formal closeout of any Mission, the Contracting Officer’s

Representative (COR) for the NSROC IV contract verifies that all contractual requirements have been satisfied and that a Mission Principal Investigator Assessment of Mission Results has been received for the Mission. The COR then determines the NSROC IV Contractor’s overall performance for the subject Mission in accordance with the process described in the NSROC IV

Contract Clause J.1 Attachment S Award Fee Plan.

3.2.7 Quality Data Analysis

On an annual basis the Sounding Rockets Program Office analyzes the results of the previous year’s completed missions and determine the overall success rate as compared to the goal of 85% as provided in the Sounding Rockets Program Commitment Agreement with NASA

Headquarters. In addition to this overall mission success rate, individual success rates for three general categories to include launch vehicles, experiments, and payload support systems are also determined. Categories and causes of failure and anomaly data are analyzed to look for trending and compared to historical data. The results of this analysis are then reported out at an annual

Anomaly Investigations and Review with representatives of NASA Headquarters Office of

Safety and Mission Assurance and Science Mission Directorate; Goddard Space Flight Center

Code 300; Wallops Code 800 Senior Management; and the Wallops Safety Office. Negative findings inconsistent with the overall goals of the Program are documented and corrective actions assigned as appropriate.

4. SUMMARY

This Quality Assurance Surveillance Plan describes the approach NASA/GSFC/WFF utilizes to monitor the NSROC-IV and assure the Contractor performs in accordance with the terms and conditions of the contract and that the Program is meeting its goals and objectives.

NASA/GSFC/WFF uses an insight-driven surveillance approach with the goal of balancing the level of Government surveillance with the perceived impacts and risks of mission failure. This process is supported by the NSROC-IV COR and by other organizations and personnel as necessary to obtain insight into Contractor activities and overall Program performance. This plan is a fully functioning living document and will be modified as necessary to meet the needs and requirements of the Sounding Rockets Program as they change over time.