GSFC-STD-1001A.pdf

GODDARD TECHNICAL

STANDARD

Goddard Space Flight Center

Greenbelt, MD 20771

GSFC-STD-1001A

Approved: 10-01-2009

Revalidation Date: 04-01-2020

Superseding GSFC-STD-1001

Criteria for Flight and Flight Support Systems

Lifecycle Reviews

October 1, 2009

THIS STANDARD HAS BEEN REVIEWED FOR EXPORT CONTROL RESTRICTIONS; APPROVED FOR PUBLIC RELEASE

DISTRIBUTION IS UNLIMITED

i

FOREWORD

This standard is published by the Goddard Space Flight center (GSFC) to provide a consistent set of guidelines that detail the purpose, timing, success criteria and evaluation factors to be considered in the preparation and conduct of both Agency and Center-level mission and element reviews starting at Pre-Phase A, Concept Studies, through Phase F, Close Out. These guidelines have been derived from best practices in use at GSFC as compiled both internally and at the NASA Agency level.

Approved by:

Original Signed by:

George Alcorn Date

Goddard Technical Standard Coordinator

NASA/GSFC Code 300

Original Signed by:

Steven Scott Date

Chief Engineer

NASA/GSFC Code 500 ii

For any questions concerning this standard, please contact:

Code 300, Associate Director for Independent

Reviews

Goddard Space Flight Center

Greenbelt, Maryland 20771 iii

Original signed by:

Orlando Figueroa Date

Director

Applied Engineering and Technology Directorate

NASA/GSFC Code 500

George W. Morrow Date

Director

Flight Programs and Projects Directorate

NASA/GSFC Code 400

Marcus Watkins Date

Director

Safety and Mission Assurance Directorate

NASA/GSFC Code 300

Arthur Obenschain Date

Deputy Center Director

NASA/GSFC Code 100 iv

CHANGE HISTORY LOG

Revision Effective Date Description of Changes

02/19/2005

Baseline Release

A

10/01/2009 Modified to reflect changes to NPR 7120.5 Rev. D, and

NPR 7123.1 Rev. A.

A 09/17/2015 Updated responsible office and extended for 18 months from original expiration date.

A 04/22/2016 Updated responsible office and extended for 12 months from expiration date.

A 4/1/2017

Updated responsible office, removed references to IPAO and IPAO Review Manager and extended for 12 months from expiration date

A 3/28/2018 Extended for 12 months from current expiration date

A 03/01/2019

Removed references to SRO. Removed references to NPR

7120.5D and NPR 7123.1A. (NPR 7120.5E and NPR

7123.1B are the current versions.) Replaced IIRT with

GSRT.

v

TABLE OF CONTENTS

SECTION PAGE

CHANGE HISTORY LOG ............................................................................................................ iv

TABLE OF CONTENTS................................................................................................................ v

LIST OF TABLES ......................................................................................................................... vi

1.0 SCOPE

2.0 MISSION AND ELEMENT-LEVEL REVIEWS

3.0 MISSION CONCEPT REVIEW (MCR)

4.0 SYSTEM REQUIREMENTS AND SYSTEM DEFINITION REVIEWS (SRR/SDR)

5.0 PRELIMINARY DESIGN REVIEW (PDR)

6.0 CRITICAL DESIGN REVIEW (CDR)

7.0 MISSION OPERATIONS REVIEW (MOR)

8.0 SYSTEMS INTEGRATION REVIEW (SIR)

9.0 PRE-ENVIRONMENTAL REVIEW (PER)

10.0 FLIGHT OPERATIONS REVIEW (FOR)

11.0 PRE-SHIPMENT REVIEW (PSR)

12.0 OPERATIONAL READINESS REVIEWS (ORR)

13.0 POST-LAUNCH ASSESSMENT REVIEW (PLAR)

14.0 CRITICAL EVENT READINESS REVIEW (CERR)

15.0 DECOMMISSIONING REVIEW (DR)

ACRONYM LIST

vi

LIST OF TABLES

TABLE PAGE

2-1 Chairing Organizations of Key Mission and Element-Level Reviews

3-2 MCR Success Criteria

4-2 SRR/SDR or MSRR/MDR Success Criteria

5-2 PDR Success Criteria

6-2 CDR Success Criteria

7-2 MOR Success Criteria

8-2 SIR Success Criteria

9-2 PER Success Criteria

10-2 FOR Success Criteria

11-2 PSR Success Criteria

12-2 ORR Success Criteria

13-2 PLAR Success Criteria

14-2 CERR Success Criteria

15-2 DR Success Criteria

1.0 SCOPE

1.1 Purpose

The guidelines and criteria contained in this document are intended for use by the project team, the GSFC System Review Managers, the Chair of the Goddard System Review Team (GSRT) and the Chair of the NASA appointed Standing Review Board (SRB) in support of planning and implementing the reviews conducted under the authority of each organization. Such reviews range from subsystem and functional reviews to the mission-level reviews, many of which serve as Key Decision Point (KDP) gateways. The criteria defined in this document have been developed to encompass those specified by NPR 7123.1.

As a supplement to this document, the GSFC STD-1001- Appendix, “Lifecycle Review Success

Criteria Key Evaluation Factors”, provides sample evaluation factors intended to be used in assessing the project’s achievements toward meeting the success criteria in the development of associated system. The sample lists are organized by review type (i.e., SRR, PDR, CDR, PER, etc.) and are a compilation of evaluation factors acquired from lesson's learned and NASA best practices. Key evaluation factors may be tailored to suit the needs of the individual project. They are typically provided as reference material to the project in support of review preparation and to the independent review panels to support the conduct of the review (agenda definition, line of questioning, etc.). The Key Evaluation Factors are not success criteria for the associated review.

1.2 Applicability

This document describes the mission and lower level element reviews (e.g., spacecraft, instrument, ground system, operations, etc.) conducted during the development and operations lifecycle for Goddard Space Flight Center (GSFC) managed projects, and incorporates the requirements for:

(a) Agency-level mission reviews as prescribed by National Aeronautics and Space

Administration (NASA) Procedural Requirement (NPR) 7120.5, “NASA Space Flight

Program and Project Management Requirements” and NPR 7123.1, “NASA Systems

Engineering Processes and Requirements”,

(b) Center unique mission, flight, and flight support system reviews as prescribed by GSFC

Procedural Requirement (GPR) 8700.4, “Integrated Independent Reviews”, the results of which are flowed up to the Agency level reviews that are conducted in support of the

Agency level mission reviews at a lower level.

The Project/Product Manager should use this guide when preparing the Systems Review Plan

(SRP) for conducting a comprehensive set of mission, spacecraft, instrument and ground system reviews as required by Agency and GSFC review process documents. In collaboration with the

Systems Review Manager (SRM) and SRB Chair should use this document to assess compliance with unique Agency requirements and to prepare the Terms of Reference (ToR) that documents the charter of the SRB.

2.0 MISSION AND ELEMENT-LEVEL REVIEWS

The Safety and Mission Assurance Directorate (Code 300) is responsible for the implementation of the GSFC independent review requirements as approved by the Center Management Council

(CMC) for all flight projects, including mission and element reviews. This document should be used by the Project Manager, and the Chairs of the GSRT and SRB, to determine the project’s readiness to proceed with a review and to finalize the specific objectives, agenda, and success criteria prior to each review. It provides specific information for the reviews identified in GPR

8700.4 including descriptions of review objectives, typical timing, and success criteria.

The mission-level reviews described in this document include those conducted to meet the unique requirements of the Center in addition to the reviews required by the Agency to be conducted by the SRB at specific progress points along the development lifecycle for NASA missions. These reviews are supported by element reviews that are conducted by a GSFC-convened GSRT and include spacecraft, instrument, operational and ground systems. Furthering the continuity of the Center’s review process, the mission and element reviews are supported by project implemented Engineering Peer Reviews (EPRs) with a principal focus on discipline or subsystem related technical considerations. These reviews are addressed in a project Engineering

Peer Review Plan (EPRP) as required by GPR 8700.6, “Engineering Peer Reviews”.

It is recognized that the full set of mission and element-level reviews described herein will not be appropriate for every project. Therefore, not all projects will conform to the complete lifecycle review process described in NPR 7120.5 and GPR 8700.4, and may require a waiver and/or tailoring of the requirements and criteria to match the specific needs of the project. To accommodate this, project-unique review requirements may be negotiated with the responsible review team chair and System Review Manager to tailor the review requirements and success criteria as appropriate. The details of the agreed upon tailoring are documented in the SRP and/or the ToR and shall be supported by a waiver to the requirements of GPR 8700.4 or NPR 7120.5 as may be required.

It is also recognized that the final complement of reviews, individual review content, review titles, and the timing for the conduct of the individual reviews may vary with each project.

However, unless otherwise agreed upon and documented in the SRP, the complete set of success criteria provided herein relative to the product being developed (end-to-end mission, flight or ground element, etc.) shall be addressed within the total set of reviews being proposed by the project. The distribution of criteria amongst the specific reviews may vary from what is specified in this document to best meet the needs of the specific project.

2.1 Results of Review

Some projects may not fully satisfy all of the criteria at the time of the milestone/gateway review. In making a judgment as to whether the review has accomplished its objectives and has been successfully completed, each member of the review team will assess the degree to which the above success criteria have been met based on the key evaluation factors. Each member should also take into account (a) the criticality of the areas where there are shortfalls, (b) how

Straightforward the path forward is and the likelihood of success, as well as (c) any other relevant factors. Individual findings from each review team member are conveyed to the convening authority in the panel’s final report, including RFA's. An RFA is a formal written request sponsored by the review panel asking for additional information or action by the project team.

They are generally developed as a result of insufficient safety, technical, or programmatic information being available at the time of the review.

2.2 General Criteria

The sections that follow provide the criteria to be used by the independent review panel members during their assessment of a flight or flight support system. The criteria have been divided into five categories: Review Process, Technical Management, System Design and Demonstration, Safety and Mission Assurance (SMA), and Project Management. In section 3.0, a table is provided within the description of each review defining the success criteria within each of these categories. The criteria evolve as the project progresses through its lifecycle to reflect expected changes in the maturity of the system. The following provides a general description of the criteria associated with each category.

Review Process: (a) the identification and reporting of peer reviews conducted since the last element/system review, (b) the status of all critical issues (e.g. critical Requests for Action) surfaced at prior reviews, and (c) Request for Action (RFA) disposition from all preceding reviews and associated risk status.

Technical Management: (a) trade-studies and alternative solutions, (b) system effectiveness, life cycle resources, risk, and customer requirements, (c) requirements traceability, (d) schedules for development and delivery are mutually supportive, (e) integration of technical disciplines, (f) validity, consistency, desirability, and attainability of functional and performance requirements, and (g) traceability of design requirements to the functional and performance requirements and vice-versa, (h) identification and control of critical interfaces, and (i) trending plans/analyses.

System Design and Demonstration: (a) baseline designs and documentation, (b) system level performance specifications, (c) fabrication of engineering demonstration models, (d) processes associated with system/product integration including end-item traceability and product quality,

(e) verification of requirements grounded by sound engineering analysis and test practices, and

(f) production and manufacturing.

Safety and Mission Assurance: (a) quality engineering, (b) quality assurance, (c) safety assurance processes associated with flight, ground, and operational systems/subsystems, and d) reliability engineering (including EEE parts program).

Project Management: (a) cost estimates, (b) control processes, and (c) schedules that indicate the mission will be ready to launch on time and within budget. Examples of control processes being evaluated include a Project Plan, Systems Engineering Management Plan, Configuration

Management Plan, and a Risk Management Plan, etc.

Table 2-1 identifies the typical mission and element-level reviews recognized by the GSFC, in adherence to the Center's requirements and consistent with those specified by NPR 7120.5.

Table 2-1 Chairing Organizations of Key Mission and Element-Level Reviews

Review Title

Mission Element

Observatory

S/C Grnd

Sys

Payloads

Instr 1 Instr 2 Instr n

Mission Concept Review (MCR) SRB

Systems Requirements Review /

System Definition Review (SRR/SDR)

SRB

GSRT

Project 3 Project

Preliminary Design Review (PDR)

Critical Design Review (CDR)

Mission Operations Review (MOR)

System Integration Review (SIR)

Pre-Environmental Review (PER), or Test

Readiness Review (TRR)

Flight Operations Review (FOR)

Pre-Shipment Review (PSR)

Operational Readiness Review (ORR)

Mission Readiness Review

CMC

Flight Readiness Review (FRR)

KSC

Launch Readiness Review (LRR)

KSC

Post-Launch Assessment Review (PLAR)

Project 3

Critical Event Readiness Review (CERR)

Decommissioning Review (DR)

Project 3

1 Observatory is assessed at the Mission-level review which encompasses both the ground and flight segments.

2 Conducted by the GSRT in instances where an SRB has not yet been established or an SRB is not required.

3 Chairing organization is negotiable with the project and can be GSRT.

4 Conducted concurrently with the ORR by the SRB as the operational status warrants.

Conducted by the GSFC CMC prior to KDP-E and supported by GSRT and SRB Chairs.

Chaired by Kennedy Space Center (KSC) and supported by GSRT and SRB Chairs as needed;

requirements/criteria not included in this document.

Supported by GSRT as needed.

3.0 MISSION CONCEPT REVIEW (MCR)

The MCR affirms the mission need and examines the proposed mission's objectives and the concept for meeting those objectives. Key technologies are identified and assessed. It is an internal review that is usually conducted by the system development organization. ROM budget and schedules are presented. At the MCR, the project demonstrates to the review panel that the:

Proposed mission meets the science.

Objectives proposed mission is feasible.

Proposed mission and operations design concepts are viable.

Preliminary plan for lifecycle activities suitably illustrates reasonable execution of the mission within resource budgets and other foreseen constraints.

3.1 Timing

The MCR is normally held upon completion of mission feasibility studies and represents the conclusion of project pre-formulation activities. In advance of the review, the project should highlight and discuss with the review chairperson any areas that may warrant consideration in establishing the composition of the review team (e.g., problematic mission requirements, critical technology dependencies, critical trade studies, or anticipated resource constraints). Depending upon the intended acquisition approach for the mission, GSFC management may decide that an

MCR need not be conducted or that it will be replaced by a management review as permitted within the guidance of NPR 7120.5. Such determination shall be made early in the lifecycle and in conjunction with the development of the project Systems Review Plan (SRP) and consequently the Terms of Reference (ToR) for the Standing Review Board (SRB) as an applicable document incorporated by reference.

3.2 Success Criteria

The review agenda, success criteria, and charge to the independent review board shall be discussed for concurrence by the Principal Investigator (PI-mode only), Program Executive

(SRB reviews only), Project Manager, Review Board Chairperson, Systems Review Manager, and distributed to all parties prior to the review. The Systems Review Manager is responsible for initiating this discussion. Excluding any required tailoring, projects must at a minimum meet the following criteria as part of the MCR or demonstrate an adequate path to completion. Table 3-2 depicts the criteria for a successful MCR.

Table 3-2: MCR Success Criteria

Category MCR Criteria

Review Process A preliminary Systems Review Plan (SRP) including an Engineering Peer

Review Plan (EPRP) is available and deemed compliant with all applicable requirements.

Technical Management Mission objectives are clearly defined and unambiguous.

Potential technology needs are identified and the gaps between such needs and the current and/or planned technology readiness levels have been assessed with acceptable results.

The evaluation criteria and trade space for candidate systems that fulfill the conceptual design requirements have been identified and prioritized.

Technical planning is sufficient to proceed to the next phase.

System Design and

Demonstration

An operations concept and system architecture is provided that meets these requirements, demonstrating the feasibility of the mission and technical solution.

A search was conducted to identify existing assets or products that have a potential to be implemented to satisfy the mission or parts of the mission.

The preliminary set of requirements meeting the objectives is provided and is consistently stated within the project.

Safety & Mission Assurance Safety and mission assurance activities (i.e., safety, reliability, maintainability, quality, and Electrical, Electronic and Electromechanical [EEE] parts) related to the mission and conceptual design have been adequately addressed.

Project Management Initial risk identification and mitigation strategies have been provided and are acceptable.

A rough order of magnitude cost estimate is provided and is both credible and within an acceptable cost range.

The schedule estimates are credible.

3.3 Key Evaluation Factors for the Assessment of Success Criteria

The MCR should contain a complete description of the conceptual mission design. The project team presents the design using block diagrams, flowcharts, schematics, etc., depicting system interfaces with external supporting systems, as well as interfaces between independent system elements. Preliminary modeling and analysis results should be presented in order to illustrate feasibility of achieving science objectives. Programmatic planning and resource estimates shall also be discussed in sufficient detail to permit assessment of relevant review objectives.

The GSFC STD-1001-Appendix provides sample key evaluation factors often applied by individual review team members when assessing the satisfactory achievement of the established criteria.

4.0 SYSTEM REQUIREMENTS AND SYSTEM DEFINITION REVIEWS (SRR/SDR)

The purpose of this review is to verify that the functional and performance requirements are defined for the system under review and to ensure the requirements are satisfied by the selected concept. To justify proceeding with detailed definition and the flow-down of requirements to the major elements of the system, the project must convey to the review panel that the:

Baseline mission requirements are clearly understood.

Top-level requirements for each system element have been determined.

Proposed mission design and operations concept satisfies baseline mission requirements plans for future activities justify expectations the mission design will accommodate imposed constraints and accomplish the mission within allocated resources.

4.1 Timing

An SRR can be conducted for selected elements (e.g., spacecraft, instruments, ground systems, operations, etc.) as well as the mission. An SRR is typically conducted once a feasible system definition is available and while changes to the particular element under review can be accommodated with minimal impact. The SRR for any particular element is typically conducted concurrently with the associated System Definition Review (SDR).

Similarly, as part of the GSFC process, the Mission-level SRR (MSRR) is conducted concurrently with the Mission Definition Review (MDR) toward the end of Phase A and provides the relevant data for the Key Decision Point “B” (KDP-B) gateway at which the decision to proceed with the preliminary design is made. When scheduling the review, the project should highlight and discuss with the review Chair any significant risk areas (e.g., problematic requirements, critical technology dependencies, outstanding trade studies, or significant resource constraints) that may warrant consideration in the timing of the review or the composition of the review team. The determination of the readiness of a project to proceed with the review will be based on these discussions and at the discretion of the

Chair and in consultation with the Project, Systems Review Manager, Program Office and

Convening Authority as applicable.

4.2 Success Criteria

The SRR/SDR and MSRR/MDR criteria provided below should be consulted early enough in the project schedule to select a suitable date for the review and to properly plan an agenda that fulfills the objectives of a System Requirements Review (SRR) simultaneously with those of a

System Definition Review (SDR). The system under review may include elements such as ground systems, instruments, and spacecraft; or the observatory which includes the flight and ground segments. The review agenda, success criteria, and charge to the independent review board shall be discussed for concurrence with the Principal Investigator (PI-mode only), Program

Executive (SRB reviews only), Project Manager, Review Board Chairperson, and Systems

Review Manager; and distributed to all parties prior to the review. The Systems Review Manager is responsible for initiating this discussion. Excluding any required tailoring, projects must at a minimum meet the following criteria as part of the SRR/SDR or MSRR/MDR, or demonstrate an adequate path to completion. Table 4-2 depicts the criteria for a successful SRR/SDR or

MSRR/MDR.

Table 4-2: SRR/SDR or MSRR/MDR Success Criteria

Category

SRR/SDR or MSRR/MDR Criteria Review Process The Mission Concept Review (MCR), if applicable, was successfully completed with every

Request for Action (RFA) reaching an acceptable disposition.

Planned peer reviews associated with the system under review have been successfully completed.

MSRR/MDR Only- A baseline Systems Review Plan (SRP) and Engineering Peer Review

Plan (EPRP) have been approved and implemented.

Technical

Management

The system design builds upon the initial concept by providing a complete definition of the interfaces and key higher-level performance and technical requirements; which have been appropriately flowed to the systems and/or subsystems.

The major risks have been identified and viable mitigation strategies are defined.

The project utilizes a sound process for the allocation and control of requirements at all levels.

Requirements are clearly understood and the project is ready to fully flow requirements to the lower levels.

MSRR/MDR Only- Top level requirements exhibit alignment with NASA needs, goals, and objectives, and has been adequately flowed down to the appropriate mission elements.

System Design and

Demonstration

The system and subsystem design approaches and operational concepts are reasonable, feasible, complete, responsive to the performance requirements, and are consistent with system requirements.

Preliminary approaches have been determined for verifying and validating subsystem

Safety & Mission

Assurance

A system safety plan is completed that details the safety management and engineering requirements for identifying, evaluating, and eliminating or controlling hazards.

A baseline of the mission assurance requirements is available.

The level of quality assurance to be implemented is appropriately defined.

Management A plan has been defined to complete the definition activity within schedule constraints.

The overall concept is consistent with available resources. MSRR/MDR Only- A cost estimate is provided along with a clearly defined basis of estimate and is both credible and within an acceptable cost range.

4.3 Key Evaluation Factors for the Assessment of Success Criteria

The SRR/SDR and MSRR/MDR should contain a complete and comprehensive description of the element and mission design, respectively, with relevant conceptual systems designs. It should present the design by means of block diagrams that depict system interfaces with external supporting systems, as well as depicting internal interfaces between independent system elements. Completed modeling and analyses results should be presented as required to illustrate that science objectives will be achieved.

Programmatic considerations shall also be discussed in sufficient detail to permit assessment of relevant review criteria.

5.0 PRELIMINARY DESIGN REVIEW (PDR)

At the PDR, the project discloses the complete system or subsystem design to the review panel.

The ability of the preliminary design to meet all requirements with acceptable risk is presented.

The projections for completing the project within the identified cost and schedule constraints are also provided. The readiness to proceed with the detailed design is demonstrated by:

Completing a credible and acceptable preliminary design that meets performance

Selecting a suitable design solution, making necessary resource allocations, and identifying critical interfaces and requirements verification methods.

Confirming requirements compliance with supporting design analyses.

Presenting acceptable plans for the completion of system or subsystem development and the subsequent operations (if applicable) within the identified cost and schedule constraints.

5.1 Timing

The PDR is the first major review of the overall system design and is normally held prior to the preparation of detailed design drawings and the initiation of any full-scale flight hardware or software development. A PDR is held when the design is advanced sufficiently to begin some breadboard testing and/or the fabrication of design models. A

PDR is required for all mission elements (i.e., spacecraft, instruments, ground systems, and operations) as well as the mission itself. The Mission Preliminary Design Review

(MPDR) is the last mission-level review in Phase B prior to the Key Decision Point “C”

(KDP-C) gateway identified in NPR 7120.5.

When scheduling a design review of this type, the project should highlight and discuss with the chairperson any significant development areas that may warrant attention when establishing the timing of the review or composition of the review team due to the magnitude of the effort, technical difficulty, complexity, or criticality of success.

5.2 Success Criteria

(SRB reviews only), Project Manager, Review Board Chairperson, and Systems Review

Manager,; and distributed to all parties prior to the review. The Systems Review Manager is responsible for initiating this discussion. Excluding any required tailoring, projects must at a minimum meet the following criteria as part of the PDR or demonstrate an adequate path to completion. Table 5-2 depicts the criteria for a successful PDR.

Table 5-2: PDR Success Criteria

Category PDR Criteria

Review Process The SRR or MSRR/MDR was successfully completed, if applicable, and responses made to each Request for Action (RFA).

All subsystem PDRs and the associated peer reviews have been successfully completed.

All assigned actions have an acceptable disposition including plans to complete for work in process.

Technical

Management

All processes (design, implementation, interface controls, risk management, safety, test & verification, operations, etc.) used to develop and operate the system are at expected maturity level.

The preliminary design is expected to meet the requirements within the resource allocation.

Demonstration

The preliminary design is consistent with the top-level requirements. The operations concept, if applicable, is technically sound.

The defined technical interfaces are consistent with the overall technical maturity.

Adequate margins exist with respect to technical performance.

Any required new technology has been developed to an adequate state of readiness or viable options exist.

Safety & Mission

Assurance

Safety, reliability, maintainability, quality, and Electrical, Electronic and Electromechanical

(EEE) parts have been adequately addressed in preliminary designs and any applicable

S&MA products (i.e., hazard analysis and failure modes and effects analysis) have been identified.

Project

Design definition is sufficient to support initial parametric and bottoms-up cost estimating.

Cost estimates, control processes, and schedule indicate the system will be ready on time

(i.e., integration, delivery, launch, etc.) and within budget.

5.3 Key Evaluation Factors for the Assessment of Success Criteria

The PDR should contain a complete and comprehensive presentation of the entire design. It should present the design and interfaces by means of block diagrams, power flow diagrams, signal flow diagrams, interface circuits, software logic flow and timing diagrams. Appropriate modeling results should be presented. The traceability of all deliverable items discussed at previous reviews shall be updated and presented. Programmatic considerations shall also be discussed in sufficient detail to permit assessment of relevant review objectives.

6.0 CRITICAL DESIGN REVIEW (CDR)

At the CDR, the project discloses the complete system design to the review panel. The project demonstrates that the maturity of the design and development effort:

Justifies proceeding with full scale fabrication activities, assembly, integration and test.

Is on track to complete flight system, ground system, and mission operations development.

Meets mission performance requirements within the identified cost and schedule constraints.

6.1 Timing

The CDR is held near the completion of the final design stage, including the completion of engineering model evaluations, as applicable, and breadboard development and test.

Although substantial completion of drawings is expected, the review should be held prior to any design freeze and before any significant flight fabrication activity begins. A CDR is required for all elements (i.e., spacecraft, instruments, and ground systems) as well as the mission (MCDR).

When scheduling the review, the project should highlight and discuss with the review

Chair any significant development areas (significant due to the amount, the criticality, the technical difficulty/complexity, etc.) which may not be sufficiently mature and may warrant consideration regarding either timing of the review or composition of the review team. The start of limited fabrication (typically long lead items, off-the-shelf hardware or common buy items) before CDR is common and generally acceptable. The project should, however, consult with the review Chair to obtain concurrence with respect to any significant flight hardware fabrication that will take place before CDR.

6.2 Success Criteria

A review agenda, success criteria, and charge to the independent review board shall be discussed for concurrence by the Principal Investigator (PI-mode only), Program Executive (SRB reviews only), Project Manager, Review Board Chair, and Systems Review Manager; and distributed to all parties prior to the review. The Systems Review Manager is responsible for initiating this discussion. Excluding any required tailoring, projects must at a minimum meet the following criteria as part of the CDR or demonstrate an adequate path towards completion. Table 6-2 depicts the criteria for a successful CDR.

Table 6-2: CDR Success Criteria

Category CDR Criteria

Review Process The Preliminary Design Review (PDR) has been successfully completed and responses made to each Request for Actions (RFA).

All subsystem CDRs and the associated peer reviews have been successfully completed.

Technical

Interface Control Documents are appropriately mature to proceed with implementation.

Plans are in place to manage any open items.

System design (including key element interfaces and performance specifications) is complete and the processes used to develop and operate the system are sufficiently defined to start procurement, fabrication, manufacture, integration, and testing.

System Design and

Demonstration

The maturity of the applicable systems and operations designs warrant proceeding to implementation.

High confidence exists in the product baseline, and adequate documentation exists and/or will exist in a timely manner to enter the next phase including fabrication, assembly, integration, and test activities.

The detailed design is expected to meet the requirements with adequate margins.

Adequate resources exist to complete system development within accepted risk constraints.

Safety & Mission

Assurance

Safety, reliability, maintainability, quality, and Electrical, Electronic and Electromechanical

(EEE) parts have been adequately addressed in system and operational designs and any applicable S&MA products (i.e., hazard analysis and failure modes and effects analysis) have been completed and approved.

Project

Management

Adequate technical and programmatic margins and resources exist to complete the development within budget and on schedule.

6.3 Key Evaluation Factors for the Assessment of Success Criteria

The CDR should represent a complete and comprehensive presentation of the entire final design.

It should present the final design and interfaces by means of completed drawings, block diagrams, power flow diagrams, signal flow diagrams, interface circuits, software logic flow and timing diagrams, modeling results, and breadboard and engineering model test results.

Traceability for all items specified for previous reviews, updated to the present stage of the development process, shall be presented. Programmatic considerations shall also be discussed in sufficient detail to permit assessment of relevant review objectives. The GSFC STD-1001-

Appendix provides sample key evaluation factors often applied by individual review team members when assessing the satisfactory achievement of the established criteria.

7.0 MISSION OPERATIONS REVIEW (MOR)

At the MOR, the project demonstrates to the review panel that:

Requirements for all phases and modes of mission operations, data processing, and analysis are thoroughly understood.

All operations will be adequately staffed and executed.

Planned implementation of the ground system satisfies all operational requirements preliminary plans for the execution of a comprehensive end-to-end verification and validation program are complete.

7.1 Timing

The MOR is the first of two ground system reviews designed to focus on mission operations. It is typically held upon completion of detailed design and in all cases should be held prior to initiation of major integration activities of flight subsystem elements.

Chair any extenuating circumstances or problem areas that may deserve consideration regarding timing of the review or composition of the review team.

7.2 Success Criteria

discussed for concurrence by the Principal Investigator (PI-mode only), Program

Executive (SRB reviews only), Project Manager, Review Board Chairperson, and Systems

Review Manager,; and distributed to all parties prior to the review. The Systems Review

Manager is responsible for initiating this discussion. Excluding any required tailoring, projects must at a minimum meet the following criteria as part of the MOR or demonstrate an adequate path towards completion. Table 7-2 depicts the criteria for a successful MOR.

Table 7-2: MOR Success Criteria

Category MOR Criteria

Review Process The MPDR was successfully completed with an acceptable disposition of each mission operations specific Request for Action (RFA).

The associated peer reviews have been successfully completed.

All assigned actions have an acceptable disposition including plans to complete work in

Ground system and operations requirements have been adequately defined.

Technical

Management

Linkage of mission requirements to the ground system support requirements and subsequent flow-down to the operations personnel and technical elements within the ground system is complete, traceable, and verifiable.

The operations approach is expected to meet the mission requirements with an acceptable level of risk.

System Design and

Demonstration

Planning for all phases and modes of mission operations (including observatory operations, data processing, and analyses) adequately addresses all ground system requirements.

An acceptable level of maturity has been demonstrated for the overall operations design including the definition of mission operations requirements, ground system requirements, logistics, training, Information Technology (IT) security, verification tests, and operator certification.

Safety & Mission

Assurance

Safety related requirements for real-time operations, including safety monitoring and safe-mode operation, have been suitably defined and allocated for the operations design.

Project

Management

Identified risk mitigation plans are supported by suitable procedures and resources for the effective management of the risks.

System design meets mission performance requirements within identified cost, schedule, and resource constraints.

7.3 Key Evaluation Factors for the Assessment of Success Criteria

The MOR should focus predominately upon the planning in areas driven by operational considerations. To that end, it is not an in-depth review of the design. Project peer review activity and other mission-level reviews address those considerations. Consequently, information on development tasks should focus on current status and plans for interacting with verification and operations related activities in a coordinated fashion.

relevant review objectives.

8.0 SYSTEMS INTEGRATION REVIEW (SIR)

The SIR evaluates the readiness of the overall system (all elements working together) to commence Integration and Test (I&T). At the SIR, the project demonstrates that:

Required validation & verification plans, integration plans and procedures, and test plans are available and approved to begin integration.

Required system components, support personnel, integration facilities, and test procedures are available and ready to begin system testing and data acquisition, reduction, and control required Ground Support Equipment (GSE) is ready to support I&T

8.1 Timing

The SIR is a mission-level review normally held after completion of the integration and test of the associated flight and ground elements and subsystems that make up the final system. This review marks the beginning of Phase D (system assembly, integration, test, and launch) and precedes the Key Decision Point “D” (KDP-D) gateway review conducted by the Agency.

Successful element/subsystems functional and performance testing are required for baseline performance and trending information prior to the initiation of system integration.

8.2 Success Criteria

(SRB reviews only), Project Manager, and Review Board Chairperson, and Systems Review

Manager; and distributed to all parties prior to the review. The Systems Review Manager is responsible for initiating this discussion. Excluding any required tailoring, projects must at a minimum meet the following criteria as part of the SIR or demonstrate an adequate path towards completion. Table 8-2 depicts the criteria for a successful SIR.

Table 8-2: SIR Success Criteria

Category SIR Criteria

Review Process Successful completion of the MCDR with responses made to each Request for Action

(RFA).

All subsystem PERs and PSRs, as applicable, have been successfully concluded with acceptable disposition of every RFA.

Associated peer reviews have been successfully completed.

Technical

Management

Element and component designs have been satisfactorily qualified and the engineering and performance requirements verified.

All integration plans and procedures are approved and ready to begin system integration.

Demonstration

All support personnel and facilities are available and system components are ready to be integrated into the system.

System elements and components have satisfactorily completed verification testing as required at the lower levels.

Safety & Mission

Assurance

The identification of safety hazards for ground hardware and operations is complete and the required controls are implemented.

Project

Management

Adequate programmatic margins and resources exist to complete the integration effort within accepted risk constraints (i.e., probability versus impact).

8.3 Key Evaluation Factors for the Assessment of Success Criteria

The SIR focuses on assessing the integration plans and procedures for the system. Confirmation that all required elements and/or components are available for integration and that applicable functional, unit-level, subsystem, and qualification testing have been conducted successfully.

Finally, the readiness and availability of integration facilities, including clean rooms, ground support equipment, handling fixtures, overhead cranes, and electrical test equipment, are assessed.

Programmatic considerations shall also be discussed in sufficient detail at the SIR to permit assessment of relevant review objectives.

9.0 PRE-ENVIRONMENTAL REVIEW (PER)

At the PER, the project discloses the complete project status to the review panel. The project demonstrates that the flight system is:

Ready to proceed with environmental testing as an integrated system.

On track to complete development and conduct operations, if required, within allocated cost and schedule resources.

9.1 Timing

The PER is held after completion of the initial successful comprehensive systems test of the fully-integrated flight system and prior to initiation of the system level environmental test sequence. A PER is required for elements such as the spacecraft, instruments, and ground systems as well as for the all-up observatory as a mission-level review. Spacecraft bus testing is often conducted at the observatory level, in which case the Spacecraft PER is performed in conjunction with the Mission PER.

chairperson any extenuating circumstances or problem areas that may deserve consideration regarding timing of the review or composition of the review team.

9.2 Success Criteria

(SRB reviews only), Project Manager, Review Board Chairperson, and Systems Review

Manager; and distributed to all parties prior to the review. The Systems Review Manager is responsible for initiating this discussion. Excluding any required tailoring, projects must at a minimum meet the following criteria as part of the PER or demonstrate an adequate path towards completion. Table 9-2 depicts the criteria for a successful PER.

Table 9-2: PER Success Criteria

Category PER Criteria

Review Process The CDR or SIR, whichever is applicable as the latest review, has been successfully completed and responses made to each Request for Action (RFA).

Test Readiness Reviews (TRRs) have been defined and scheduled as required.

Technical

Management

A review of the test plans, procedures, environments, and the configuration of the test item, provide a reasonable expectation that the objectives of the testing will be met.

Planning for subsequent flight system activities, satisfactory progress on development of other system elements, and adequacy of available resources to complete remaining project activities was demonstrated.

System Design and

Demonstration A requirements-compliant flight system design has been integrated and subjected to a successful comprehensive systems test establishing a baseline for future tests.

Support system components have been successfully integrated into the system and required personnel and facilities are available for system testing.

The objectives of the testing have been clearly defined and documented.

Assurance The status of safety data submissions, procedures, and verification activities indicate a proper maturity level at this point in the life-cycle.

The identification of safety hazards for flight, range, ground hardware and operations is complete.

The disposition and status of previous anomalies, deviations, and waivers have been assessed in their entirety and the identified risks are acceptable to proceeding.

Project

Management

Programmatic risk levels are appropriately identified and have been accepted by the project as required.

9.3 Key Evaluation Factors for the Assessment of Success Criteria

The PER should present a complete and comprehensive status of the final system with emphasis on changes to the requirements and the design since CDR and/or System Integration Review

(SIR) (if applicable). It should trace all fabrication and lower level verification activities with emphasis on discrepancies and their resolution. It should detail the composition and results of the comprehensive system test. It should detail all remaining project activities and detail status of all other mission system elements.

Programmatic considerations shall be discussed in sufficient detail to permit assessment of relevant review objectives. The GSFC STD-1001-Appendix provides sample key evaluation factors often applied by individual review team members when assessing the satisfactory achievement of the established criteria.

10.0 FLIGHT OPERATIONS REVIEW (FOR)

The FOR is the second of two ground system reviews held to examine mission operations status during which the project demonstrates or verifies:

Compliance with all mission operations requirements and the ability to execute all phases and modes of mission operations, data processing, and analysis.

Adequate planning and resources are in place for any remaining activities associated with interactive flight and ground testing, network compatibility testing, and other remaining pre-launch testing.

Acceptable staffing, training and certification of the flight team.

10.1 Timing

The FOR is held during the test flow of the fully integrated flight system, after completion of the initial successful comprehensive systems test but prior to the last major interactive test between the flight and ground system elements that is conducted before shipment of flight system elements to the launch site.

When scheduling the review, the project should highlight and discuss with the review Chair any extenuating circumstances or problem areas that may deserve consideration regarding timing of the review or composition of the review team.

10.2 Success Criteria

only), Project Manager, and Review Board Chairperson, and Systems Review Manager; and distributed to all parties prior to the review. The Systems Review Manager is responsible for initiating this discussion. Excluding any required tailoring, projects must at a minimum meet the following criteria as part of the FOR or demonstrate an adequate path towards completion. Table

10-2 depicts the criteria for a successful FOR.

Table 10-2: FOR Success Criteria

Category FOR Criteria

Review Process The MOR has been successfully completed and responses made to every operations related

Request for Actions (RFA).

The associated peer reviews have been successfully completed.

Technical

Management

Planning for all phases, modes, and aspects (development, Verification and Validation

(V&V), sustaining engineering, staffing, Information Technology (IT) and physical security) of mission operations, data processing, and analysis adequately addresses all

Ground System requirements at the proper level of maturity.

System Design and

Demonstration

Completion of all phases and modes of mission operations, data processing, and analysis has been verified.

Ground System mission elements are ready to proceed with final integrated flight and ground system testing as well as remaining support to pre-launch, launch, mission operations, data processing, and analysis activities.

Results of activities since the MOR as well as plans for all remaining work prior to launch were presented.

Safety & Mission

Assurance

All safety processes related to the operation of the ground system are at expected maturity level.

Project

Management

The flight operations approach is expected to meet the mission requirements with an acceptable level of risk.

Identified risk mitigation plans are supported by suitable procedures and resources for the effective management of the risks.

The system design meets mission performance requirements within identified cost, schedule, and resource constraints.

10.3 Key Evaluation Factors for the Assessment of Success Criteria

The FOR should highlight any changes to requirements or design since the MOR. It should provide details of verification and checkout of ground system elements with emphasis on discrepancies and their resolution. It should detail all remaining activities and emphasize adequacy of operations planning and planned testing to demonstrate that all operations scenarios can be handled successfully.

relevant review objectives.

11.0 PRE-SHIPMENT REVIEW (PSR)

At the PSR the project demonstrates to the review panel that:

All system verification activities have been successfully completed and the system meets its requirements.

The system and support (flight and ground) hardware, software, personnel, procedures, and user documentation accurately reflect the final operational state of the system under review.

The system is ready for shipment and/or final processing prior to integration or launch, whichever is applicable.

11.1 Timing

The PSR is conducted prior to shipment of flight system elements to the site for the next level of integration, or to the launch site.