80AFRC18Q003_RFQ_Portfolio.pdf

For the best experience, open this PDF portfolio in

Acrobat X or Adobe Reader X, or later.

Get Adobe Reader Now!

http://www.adobe.com/go/reader

NASA Armstrong Flight Research Center

RFQ Number: 80AFRC18Q0003

WHEN FILLED IN THIS DOCUMENT IS SOURCE SELECTION SENSITIVE IAW FAR 2.101 & 3.104

SOURCE SELECTION SENSITIVE

PAST/PRESENT PERFORMANCE QUESTIONNAIRE

Installation of Two Zenith Ports

SECTION 1: CONTRACT IDENTIFICATION

A. Contractor: _______________________________________________________________________ B. Cage Code of contractor contract was awarded to: ________________________________________ C. Contract number: __________________________________________________________________ D. Contract type: _____________________________________________________________________ E. Was this a competitive contract? Yes _____ No _____ F. Period of performance: ______________________________________________________________ G. Initial contract cost: $____________________________ H. Current/final contract cost: $_______________________________ I. Reasons for differences between initial contract cost and final contract costs: ___________________ J. Description of service provided: ______________________________________________________

SECTION 2: CUSTOMER OR AGENCY IDENTIFICATION

A. Customer or agency name:__________________________________________________________ B. Customer or agency description (if applicable):

C. Geographic description of services under this contract, i.e. local, nationwide, worldwide, other

Commands:

SECTION 3: EVALUATOR IDENTIFICATION

A. Evaluator's name:

B. Evaluator's title:

C. Evaluator's phone/fax number: _______________________________________________________

SECTION 4: EVALUATION

Please indicate your satisfaction with the contractor’s performance by placing an “X” in the appropriate block using the scale provided to the right of each question. This scale is defined as follows:

CODE PERFORMANCE LEVEL

5 EXCEPTIONAL - The contractor’s performance meets contractual requirements and exceeds many (requirements) to the Government’s benefit. The contractual performance was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

4 VERY GOOD- The contractor’s performance meets contractual requirements and exceeds some (requirements) to the Government’s benefit. The contractual performance was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

3 SATISFACTORY – The contractor’s performance meets contractual requirements. The contractual performance contained some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

2 MARGINAL – Performance does not meet some contractual requirements. The contractual performance reflects a serious problem for which the contractor has not yet identified corrective actions or the contractor’s proposed actions appear only marginally effective or were not fully implemented.

1 UNSATISFACTORY – Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance contains serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

N/A NOT APPLICABLE - Unable to provide a score.

A. TECHNICAL PERFORMANCE

N/A

T1. Contractor’s ability to understand technical requirements

T2. Contractor’s quality of reports

T3. Contractor’s ability to adhere to schedules

T4. Contractor’s cooperation and flexibility

T5. Contractor’s technical expertise to produce aircraft structural design

T6. Contractor’s technical expertise to produce design substantiation

T7. Contractor’s technical expertise with aircraft part fabrication

T8. Contractor’s technical expertise to perform aircraft modification

T9. Contractor’s ability to identify, troubleshoot and correct problems.

T10. Contractor’s communication and cooperation with DER and efficient resolution of resolving any identified issues

Please discuss response for which you scored with a 1 or 2 in response to the questions above (use additional sheets, if necessary).

B. PROGRAM MANAGEMENT

P1. Contract problems are effectively resolved in a timely manner without extensive guidance

P2. Contractor is reasonable and cooperative in dealing with your staff (including the ability to successfully resolve disagreements/disputes) and is responsive to contract changes.

P3. Overall compliance with contract terms and conditions

P4. Contractor’s ability to adhere to schedules.

P5. Commitment to customer service.

Please discuss response for which you scored 2 or 1in response to the questions above (use additional sheets, if necessary).

C. SAFETY / QUALITY ASSURANCE

S1. Adherence to all safety regulations

S2. Contractor’s overall quality regarding safety

S3. Effectiveness in quality control and/or inspection procedures to meet contract requirements

S4. Have there been any health or safety violations? (If YES, please explain) YES NO

Please discuss response for which you scored 2 or 1in response to the questions above (use additional sheets, if necessary).

ADDITIONAL QUESTIONS:

1. Government Contracts Only: Has/was this contract been partially or completely terminated for default or convenience or are there any pending terminations?

Yes___ Default___ Convenience___ Pending Terminations___ No ___

If yes, based on default, please explain (e.g., inability to meet cost, performance, or delivery schedules, etc).

2. Would you have any reservations about soliciting this contractor in the future or having them perform one of your critical and demanding programs?

3. List the type(s) of aircraft subject to the contract.

Evaluator’s Signature Date

Thank you for your prompt response and assistance!

Please return this completed questionnaire no later than 25 May 2018 to:

E-Mail: ryan.c.lyau@nasa.gov

Before use, check the Master List to verify that this is the current version. For reference only when printed. This document contains export-controlled information.

Center distribution only. Contact the OICMS regarding external distribution.

Armstrong Flight Research Center AFG-7120.5-001, Baseline-2 Edwards, California 93523 Expires April 1, 2021

SUBJECT:

Project Chief Engineer's Handbook

RESPONSIBLE OFFICE:

Research and Engineering Directorate

EAR ECCN EAR99

- Export Administration Regulations (EAR) Notice -

This document contains information within the purview of the Export Administration Regulations (EAR), 15 CFR §730-774, and is export-controlled. It may not be transferred to foreign persons in the U.S. or abroad without specific approval of a knowledgeable export control official, and/or unless an export license or license exception is obtained/available from the Bureau of Industry and Security, United States Department of Commerce. Violations of these regulations are punishable by fine, imprisonment, or both.

Project Chief Engineer’s Handbook AFG-7120.5-001, Baseline-2 Expires April 1, 2021

Before use, check the Master List to verify that this is the current version. For reference only when

CONTENTS

1.0 SCOPE & APPLICABILITY

1.1 Scope

1.2 Applicability

2.0 CHIEF ENGINEERS, SYSTEMS ENGINEERS, AND PRINCIPLE

INVESTIGATORS

3.0 USING THIS DOCUMENT

4.0 PROJECT CHIEF ENGINEER RESPONSIBILITIES

4.1 Concept Development

4.1.1 Identifying Stakeholders and Defining Objectives

4.1.2 Developing the System Concept

4.1.3 Developing Technical Plans

4.1.4 Developing the Project Resource Estimates

4.1.5 Documenting and Executing the Project’s Management Processes

4.1.6 Suggested Tailoring and Minimums

4.2 Requirements Definition

4.2.1 Developing Requirements

4.2.2 Developing and Executing Technical Plans

4.2.3 Suggested Tailoring and Minimums

4.3 Preliminary Design

4.3.1 Suggested Minimums and Tailoring

4.4 Detailed Design

4.4.1 Suggested Minimums and Tailoring

4.5 Fabrication, Assembly and Integration, and V&V

4.6 Data Collection and Analysis

4.7 Project Closeout

4.7.1 Suggested Minimums and Tailoring

5.0 RELEVANT DOCUMENTS

5.1 Authority Documents

5.2 Referenced Documents

APPENDIX A, Definitions

APPENDIX B, Acronyms

APPENDIX C, Systems Engineering Management Plans (SEMP)

Before use, check the Master List to verify that this is the current version. For reference only when

1.0 SCOPE & APPLICABILITY

1.1 Scope

This document is intended to address all aspects of the project chief engineer (PCE) function.

1.2 Applicability

This procedure applies to all project chief engineers, as well as all systems engineers, Deputy Project Chief Engineers, and principal investigators who are doing all or some of the PCE duties on a project.

2.0 CHIEF ENGINEERS, SYSTEMS ENGINEERS, AND PRINCIPLE

INVESTIGATORS

Typically, a chief engineer is assigned to projects that involve maintaining an aeronautics research platform, developing systems across multiple disciplines, invasive modifications, research instrumentation systems, or significant envelope expansion after modification.

A systems engineer is often assigned to large, multi-disciplinary, multi-partner projects with multiple layers of product requirements. Projects with more than 50 system-level requirements to be verified and tracked are sometimes assigned a systems engineering support person to assist the chief engineer. Systems engineers may be assigned to smaller projects part-time, particularly during concept definition and requirements development phases, to support a project chief engineer during those systems engineering intensive activities. The Chief of the Systems Engineering & Integration Branch uses these and other guidelines when deciding whether or not a project requires a systems engineer.

Research projects that entail only one or two people, or just a single discipline, do not typically require the skills of a systems engineer or a chief engineer. Those projects may be run by a principal investigator who is responsible for the design, conduct and reporting of the project.

In some cases, a principal investigator may be responsible for development of an experimental system and the conduct of the experiment and the data analysis, while a chief engineer will be responsible for developing the airplane systems required to conduct the experiment, integrating the experiment on the aircraft, and conducting the flights necessary to gather the data required by the principal investigator.

Before use, check the Master List to verify that this is the current version. For reference only when

3.0 USING THIS DOCUMENT

The PCE responsibilities section is organized by project phase. For each project phase, the specific responsibilities of the PCE will be discussed, the systems engineering processes and other Armstrong Flight Research Center (AFRC) (hereinafter referred to as Center) processes that are typically exercised during the phase are listed, and a list of documents and products typically produced or updated during the phase is provided.

More information about the systems engineering processes may be found in the NASA Systems Engineering Handbook (NASA/SP-2007-6105).

Through partnerships with industry, academia, other NASA Centers, and other government agencies, the Center enters into projects at various stages of their lifecycles, often just prior to the data collection phase. For this reason, along with the fact that Center project needs differ significantly depending on the project size and complexity, it is important for the PCE to fit the process to the project needs. Suggested tailoring and a suggestion of the minimum required is included in each section.

Project chief engineers are encouraged to review the lifecycle sections previous to the one at which the center entered the project in order to understand any gaps (and hence technical risks) that might exist.

4.0 PROJECT CHIEF ENGINEER RESPONSIBILITIES

The PCE has three main areas of responsibility: technical lead, system safety lead, and systems engineering lead. On some projects, the PCE may also have some responsibility for doing the work of a technical discipline on the project. On large projects, the PCE may delegate some of the technical lead responsibilities to a deputy, or some of the systems engineering lead responsibilities to a systems engineer or a deputy. Regardless of who does the actual work, the PCE is ultimately responsible.

As technical lead, the PCE is accountable for developing technical objectives, giving technical briefings/presentations, leading experiment and system design, managing technical risk, supporting test planning and preparation, directing project flight operations, and ensuring appropriate documentation and reporting. The PCE is also responsible for developing the engineering team organization, providing direction for project engineering activities, supporting schedule maintenance, working with Research & Engineering Directorate Branch Chiefs to ensure that the project is adequately staffed, and executing the technical work of the project.

As system safety lead, the PCE is accountable for ensuring the appropriate hazard analysis efforts and serving as the focal point for safety concerns. The safety office typically provides a system safety representative to the project, and while that person often facilitates system safety working group meetings and prepares the hazard reports and associated materials, the PCE is ultimately accountable and responsible to ensure that hazards are identified, documented, and mitigated appropriately.

Before use, check the Master List to verify that this is the current version. For reference only when

As Systems Engineering Lead, the PCE is accountable for ensuring the implementation of applicable SE policies, practices, processes, and procedures.

4.1 Concept Development

During the concept development phase, the PCE works with stakeholders and the project team to define and prioritize project and test objectives, develop the system concept, develop project resource estimates, develop preliminary technical plans for the requirements definition phase, and document the project’s management processes. A Mission Concept Review (MCR) is held at the end of this phase to evaluate the project objectives and the ability of the concept to fulfill those objectives.

Process used, plans and solutions developed, and plans and solutions updated are presented in the table below.

Processes Used During this Phase

Plans and Solutions Developed in this Phase

Updated in this Phase

Stakeholder identification and expectations definition

Technical Planning

Technical Risk Management

Technical Data Management

Technical Assessment

Decision Analysis

SEMP or equivalent. See Appendix C for SEMP preparation guidance

Objectives and Requirements Document (ORD). See G-7123.1- 002 for guidance on writing the ORD

Staffing Estimates Organization Structure

Milestone Schedule

None

4.1.1 Identifying Stakeholders and Defining Objectives

Stakeholders are simply the people and organizations that have an interest in the project. They may be providing funding, participating in the project, or have an organizational role in accepting risk for the Center or the agency. The PCE works with the project manager to define the stakeholders and each stakeholder’s role in the project. Those roles are documented in the SEMP. At the Center, stakeholder expectations are typically managed by including customer stakeholders in engineering reviews and by providing regular status. Some stakeholders, particularly partners participating in the system development, require additional efforts to ensure that the stakeholder expectations remain aligned with the project objectives.

Often, top-level project objectives are not well defined and the chief engineer will work with stakeholders to define the project

Before use, check the Master List to verify that this is the current version. For reference only when objectives, as well as the test objectives. Working with the stakeholders to develop objectives, success criteria for each objective, and minimum success criteria for the project can be one of the most challenging tasks in this phase. Chief engineers and systems engineers are encouraged to use success criteria and measures of performance to both bound and clarify the project and flight objectives. As difficult as it is to get agreement on success criteria and priorities in this phase, it is far easier than doing so during or after the test phase.

For many projects, project objectives and test objectives will be one and the same. Other projects will benefit from defining specific test objectives that trace to the project objectives.

Project objectives and test objectives with success criteria and appropriate measures of performance are defined and baselined before MCR closure. Items to be determined (TBDs) or to be refined (TBRs) are numbered and plans in place to resolve them before MCR. The default document for project objectives is the project ORD. PCEs are encouraged to prioritize project objectives prior to MCR closure. If no MCR was required, then the objectives will be baselined and prioritized before system requirements review

(SRR).

In addition to stakeholder project objectives, the PCE identifies areas of research opportunity and work with the project manager, researchers, and discipline branch chiefs to include objectives that will further center research interests.

4.1.2 Developing the System Concept

During concept development, the chief engineer works with a small group of discipline engineers to develop and document the system concept. Often starting from top-level project and test objectives, the team develops test concepts, top-level system descriptions, and top-level external interfaces. The development of flight test objectives and the system concept is often an iterative process where the objectives are clarified as details of the system concept become understood and vice versa. Chief engineers are responsible for ensuring that the project team keep the test objectives foremost as the system concept develops. The system concept and the requirements that follow from it will be constantly checked and rechecked to ensure that the resulting system truly meets the intent of the test objectives.

Before use, check the Master List to verify that this is the current version. For reference only when

The system concept consists of plain language and pictorial descriptions of user-product/system interactions throughout the life of the system from the perspective of all key stakeholders. The system concept includes how the system will be manufactured, integrated, tested, used, maintained, stored, and decommissioned as applicable. The default document for the system concept is the ORD. PCEs are encouraged to consider user interfaces needed to complete integration, test, and ground ops phases. The system concept includes all phases and all external interfaces before MCR.

The system concept is used by reviewers to evaluate whether the project is achievable, thus, the system concept will be clearly aligned with the project objectives.

4.1.3 Developing Technical Plans

The focus of technical planning in the concept development phase is centered on scoping the effort. The PCE leads the project team in identifying the broad set of tasks necessary to complete the project. Finding the right level of detail for technical plans is a challenge and PCEs are encouraged to guide the project team in thinking through the details, but capturing the broader set of tasks.

For example, it is less important to get the order of verification and validation (V&V) tests right at this phase, and more important to account for all of the risk reduction tasks. PCEs will also encourage the project team to think in terms of tasks, rather than documents.

Documents aren’t the real work – the concepts that go into the documents are where the focus will be.

Some projects will require trade studies on things like platforms, tail numbers, launch or range facilities, or other elements of a system concept or test approach. Trade studies necessary to complete the system concept and the system-level requirements will be identified and included in the technical plans by MCR.

Most projects at the Center make use of some heritage or off-the-shelf hardware and software. The chief engineer will ensure that the use of heritage and off-the-shelf items is either within the intended purpose and operating envelope of the item, or will ensure that tests and analyses are planned to ascertain any risk the item may introduce to the project requirements and objectives.

Project technical plans will also take into account the time and effort necessary to build up and execute hardware in the loop simulations, proof-loads tests, instrument calibrations, hangar radiation tests, combined systems tests, and other analyses and tests that will ultimately be used in the system verification and

Before use, check the Master List to verify that this is the current version. For reference only when validation process. These activities will also dictate much of the infrastructure and facilities requirements that are typically documented in the ORD.

Software development is a subject to not only the systems engineering requirements, but also to the center’s implementation of the agency’s software engineering requirements. When software is part of the project solution, the chief engineer works with the RT Branch Chief to ensure that a software manager is assigned to the project. The chief engineer works with the software manager to ensure that software development and management is considered in the SEMP, the risk management process, staffing and facilities requirements, cost estimates, and in preliminary hazard analysis.

4.1.4 Developing the Project Resource Estimates

Once the system concept and the test objectives are well-understood, they are used by the chief engineer, the project manager, and the ops engineer to develop the project organizational structure. That structure, along with the tasks in the technical plan, are used to develop the first project staffing and budget estimates. While the project manager is the lead for resource estimates, the PCE makes a significant contribution to the effort.

For projects that include a significant system development effort, it is often advantageous to align the project organization structure with the system architecture to the greatest extent possible.

The PCE will discuss staffing estimates with the affected Research & Engineering Directorate branch chiefs before those estimates are finalized for MCR. If there is no MCR, then the staffing estimates will be discussed with the affected branch chiefs before SRR.

4.1.5 Documenting and Executing the Project’s Management Processes

The principal project management processes developed during the mission concept phase are the systems engineering management processes defined in the SEMP, the technical data management process, and the risk management process.

The PCE works with the systems engineer, as appropriate, to draft the SEMP. For most projects at the Center, filling the compliance matrix is sufficient. For larger, more complex projects, a standalone document will be required to thoroughly document the project’s processes. See Appendix C for guidance on developing the SEMP.

Before use, check the Master List to verify that this is the current version. For reference only when

As part of the SEMP development, the PCE works with the Center Director for Research and Engineering to define and document the project’s engineering review process. Engineering review plans will be developed based on the project’s needs. Items to consider in developing review plans include at what phase the Center entered the project, the depth and breadth of the development effort, and the risk posture of the project.

The PCE works with the project manager to establish collaborative workspaces and project document storage spaces that are accessible to all project members, and that provide some measure of version control. Popular choices include MARS, NSCKN, and the AFRC P: drive. Contact your IT POC for or the local service desk to initiate project storage space on the P: drive. The organization of the document storage space will lay foundation for the project’s technical data management process and consideration will be given to how the project will use the storage space as part of the configuration management process. It is often advantageous to organize the collaborative workspace in alignment with the organizational structure and with the system architecture. In addition to storage space, the PCE and the project manager will work on preliminary planning for data rights, data sharing among partners, and security and export control considerations for data release. Those items are typically documented in the ORD.

While projects do not typically start formally tracking technical risks during the concept development phase, the technical risks that are considered during this phase are among the first that will be entered into the risk management database. The project manager is responsible for establishing the risk management process, but the PCE typically leads the project team in identifying risks and mitigations; engineering discipline leads and the PCE are typically POCs for individual risks. See AFG-8000.4-001, Armstrong Risk Management Plan Instructions, for guidance on risk management.

4.1.6 Suggested Tailoring and Minimums

At minimum, every project will fill out the SEMP worksheet in Appendix C; document project objectives and success criteria;

document staffing and milestones; practice version control on project documents; and perform preliminary hazard analysis.

See the SEMP guidance in Appendix C for advice on tailoring the

SEMP.

Before use, check the Master List to verify that this is the current version. For reference only when

Most projects will write an ORD, document staffing and milestones in a project schedule, and hold an MCR.

4.2 Requirements Definition

During the requirements definition phase, the chief engineer and the system engineer lead the discipline team in deriving system-level requirements from the system descriptions, interface descriptions, operations concepts; and flight-test objectives that were developed during the concept development phase. The chief engineer is also responsible for leading the team in the preliminary hazard analysis and ensuring that any requirements necessary to implement design mitigations are developed and tracked as safety requirements. A System Requirements Review (SRR) is held at the end of this phase to evaluate whether the system requirements are responsive to the project’s objectives and achievable with project resources.

Technical Requirements Definition

Logical Decomposition

Stakeholder identification and

Technical Planning

Technical Risk Management

Technical Data Management

Technical Assessment

Configuration Management

System Safety Working Group

Baselined System-Level Requirements

Document Tree

Engineering Schedule

Organization Structure

NF1733 Information and Technology Classification and/or Sensitivity Level Determination Checklist

Objectives and Requirements Document (ORD). See G-7123.1- 002 for guidance on writing the ORD

Staffing Estimates

Milestone Schedule

4.2.1 Developing Requirements

PCEs are encouraged to use the details from the system concept, including the interface diagrams, to write system level requirements for functions, physical attributes, and interfaces. Performance requirements will be expressed as a quantitative part of functional requirements – that is, how well a function is to be accomplished.

Environmental requirements are typically expressed as physical attributes, but may also be expressed as functional requirements.

Before use, check the Master List to verify that this is the current version. For reference only when

Requirements will be simple, verifiable, and implementable.

Assumptions and rationale used to arrive at requirements will be documented. See Appendix C of the NASA Systems Engineering Handbook, NASA/SP-2007-6105, for guidance on writing good requirements. See AFG-7123.1-002, ORD Handbook, for suggested requirements format.

System-level requirements are traced to project objectives and allocated to the next level typically based on the system architecture developed during the concept phase. PCEs will use the objectives and the system concept to evaluate whether the requirements set describe everything needed to achieve the objectives and fully describe the system concept without also describing or prescribing an engineering design solution.

Most Center projects choose to allocate system level requirements to subsystems. However, requirements may be decomposed and analyzed by functions, time, behaviors, data flow, objects, states and modes, and failure modes and effects to define sets of logical decomposition models. The models may include function flow block diagrams, timelines, data control flow, states and modes, behavior diagrams, operator tasks, or function failure modes and will be based on performance cost, schedule, health and safety, and risk analyses. Models will be examined to identify possible conflicts.

Tradeoff analyses may be used for conflict resolution.

Those requirements allocated to systems or interfaces not under design control will be considered acceptance criteria and negotiated with the organization that controls that system or interface, or otherwise evaluated to ensure that the requirement is achievable.

System-level requirements will include a preliminary V&V method as well as a preliminary V&V activity. See AFG-7123.1-002, ORD Handbook, for suggested requirements format

Projects are encouraged to use a database to track requirements from definition through verification. The requirements management process will be documented. Larger projects with a standalone SEMP usually document the requirements management process in that SEMP. Projects that use the SEMP worksheet may find it more convenient to document the requirements management process in the document in which the system-level requirements appear – typically the ORD. The verification/validation part of the requirements management process may be documented in the

Before use, check the Master List to verify that this is the current version. For reference only when

V&V plan. Requirements will be version-controlled and, once baselined, requirements changes will be managed by the project’s configuration management plan.

Driving requirements will be identified and technical performance measures (TPM’s) and TPM review plans will be developed and briefed at SRR. Projects developing hardware include mass budgets in the TPMS per agency requirements. Projects developing powered systems include power budgets in the TPMs per agency requirements.

The system concept will be updated to reflect gaps identified and assumptions corrected during the requirements definition phase.

The updated system concept will include system boundaries and parts of the system not under design control (things provided by a partner, legacy systems, etc.) will be delineated.

4.2.2 Developing and Executing Technical Plans

In addition to design requirements, the PCE and the Lead operations engineer (OE) consider which design standards to impose and whether to tailor them. Standards and associated tailoring are typically documented in the SEMP. On multi-center or partnered projects, care will be taken to understand the partner’s design standards and review processes, and steps will be taken to communicate and interpret those Center standards, processes, and expectations that will be levied on partner-supplied equipment.

Trade studies and simulations needed to complete the system concept will be completed before SRR. Trade studies needed to complete the system-level requirements will be identified and included in the project’s technical plans by SRR.

The PCE and the system safety representative will develop the PHA, including a system-level fault tree analysis, develop the SMA plan, and the preliminary mishap plan.

The PCE will develop the flight test plan to the level necessary to complete the system-level requirements. Typically, the flight envelope, a description of the maneuvers, and some preliminary test conditions are sufficient.

Particularly for research objectives, it is common that some technology development has to occur in order for some of the system-level requirements and project objectives to be achieved. In this case, the chief engineer will assess the likelihood that the

Before use, check the Master List to verify that this is the current version. For reference only when technology under development will be ready as needed for the project, and will make contingency plans to achieve the requirements and objectives in the case that the technology development does not go as planned. The assessment and contingencies will be documented in the project risk management process. In some cases it may be prudent to also document the technology assessment in a standalone document.

The chief engineer works with the software manager to ensure that software development is considered in preliminary hazard analysis, logistics plans, technical development plans, and the document tree, as well as that software classification activities are appropriate to the project, System-level software functionality can be described in the system concept or may be described in a separate document. System-level interfaces for software will be included in the system concept and may also be included in the interface requirements.

The chief engineer works with the project manager, lead operations engineer, and discipline engineers to make preliminary plans for spares, ground equipment, deployment kits, and other items needed to support the project logistically

Documenting and Executing the Project’s Management Processes

The PCE works with the project team to document the project’s data management configuration management process, risk management process, system safety process, and decision-making process. When existing center or partner processes are adopted, the SEMP can be used to document that choice along with any project-specific tailoring. Some projects will require project-specific processes and documents. In that case, care will be taken to meet the requirements of the center processes and to use the center waiver process when the intent of those requirements are not met.

Developing the Project’s Resource Estimates and Tracking the Project’s Resource Usage

The PCE works with the project manager and the project team to develop a detailed schedule used to manage engineering tasks. It will be detailed enough to provide insight into how the design is progressing but high level enough that maintaining it doesn’t bog the team down. It aligns with the project’s milestone schedule.

While the project manager is responsible for building and maintaining the schedule, the PCE works with the project manager, Before use, check the Master List to verify that this is the current version. For reference only when the lead operations engineer, and the project team to ensure that schedules are inclusive of all the work, aggressive, and achievable.

4.2.3 Suggested Tailoring and Minimums

At minimum, projects will informally document project requirements in a version-controlled instrument. Most projects will document the system-level requirements in the ORD. Large or complex projects will document system-level requirements in a separate Systems Requirements Document. Most projects will produce a document tree showing where the lower level requirements and interfaces will be documented, and how those documents relate to the higher level requirements. Large or complex projects may require a separate environmental requirements document as well as a document describing which standards are to be levied on which parts of the system.

4.3 Preliminary Design

During the preliminary design phase, the chief engineer and the system engineer assist the discipline leads in decomposing system level requirements to the subsystems and developing designs for meeting the assigned requirements. The chief engineer ensures that subsystem designs coalesce, the design will meet the system-level requirements, that the test objectives will be met on time and on budget, and that trades that cross subsystem or discipline boundaries are conducted and decisions based on those trades are documented and disseminated to the team. A preliminary design review (PDR) is held at the end of this phase to evaluate whether the preliminary design meets all system requirements with acceptable risk and within the project resources. Peer reviews of individual subsystem designs are often held before the system-level PDR to ensure a thorough review of the subsystem before integrating into the system-level.

Stakeholder identification and

Requirements Definition

Logical Decomposition

Design Solution Definition

Interface Management

Subsystem Requirements

System and Subsystem Preliminary Designs;

Includes drawings, specifications, Interface Control Documents (ICDs), Fabrication Plans

Acquisition Plans, Acceptance Plans

Objectives and Requirements Document (ORD). See G-7123.1- 002 for guidance on writing the ORD

Engineering Schedule

Before use, check the Master List to verify that this is the current version. For reference only when

Configuration Management

Technical Data Management

Technical Assessment

Preliminary V&V Plans

Preliminary Acceptance Plans

Preliminary Integration Plans

Preliminary Flight Test Plan/Operations Plan

System and Subsystem Designs

The first step in the system/subsystem design process is to develop subsystem requirements. Starting from the system-level requirements allocated to each subsystem, the subsystem leads derive requirements to fully describe the subsystem.

When a system level requirement is allocated to several subsystems or other decomposition modes, care will be taken at the system level to ensure that the requirement is consistently interpreted and that the sum of the parts is appropriate. When a system level requirement sufficiently describes the requirement at the lower level there is no need to rewrite the requirement at that lower level. When a system-level requirement is not sufficiently described for an allocated level, derived requirements will be written and traced back to the system level requirement.

Starting with the subsystem requirements and the system concept, the subsystem lead works through analysis and design activities to define a subsystem that meets the system level requirements. During this time, the PCE focuses on ensuring that the subsystem requirements are aligned with the system level requirements, and that the subsystem designs are affordable, manufacturable, and will coalesce into a system-level design.

Complex projects may require multiple design-analysis cycles wherein subsystem and system models undergo various analyses to evaluate whether or not the design meets the requirements and the project objectives. The results of each analysis are used to adjust the design until a design emerges that achieves as many of the system-level requirements, TPMS, and project objectives as possible. The PCE will need to have a thorough understanding of objective and requirements priorities and will also need to exercise engineering judgment across disciplines in order to make design decisions. This can be a challenging task and PCEs are encouraged to engage subject matter experts outside the project whenever difficult decisions outside their own expertise are required.

Before use, check the Master List to verify that this is the current version. For reference only when

Preliminary Interface Control Documents are developed to manage the details across the interfaces. The PCE ensures that both sides of each interface are accounted for in the subsystem designs, and that the subsystem designs are consistent with the ICDs.

The chief engineer works with the OE and discipline engineering leads to develop an engineering drawing tree and update the project document tree as appropriate. Drawing numbering schemes and number blocks are allocated to configuration items as defined in the configuration management plan.

The chief engineer works with the lead instrumentation engineer to develop the preliminary master measurement list. Negotiation with discipline engineers is often necessary to whittle desired instrumentation down to that which is absolutely necessary to meet the project objectives efficiently, and within project resource allocations. PCEs are encouraged to lead that negotiation.

Trending information for the system level TPMs, including mass margins and power margins, is aggregated from the subsystem designs. The targets and current actuals are presented at PDR.

Developing and Executing Technical Plans

The chief engineer is responsible for system-level V&V planning and for ensuring that each subsystem has a V&V plan that integrates, to the extent appropriate, with the system-level plan. Starting with the system level requirements, the PCE finalizes the verification methods and the proposed verification activities in the requirements matrix. From there, a V&V plan accounting for all of the verification activities (system-level tests, analyses, demonstrations, and acceptance activities that verify that the requirements are met) and the validation activities (system-level tests, demonstrations, analyses, and acceptance activities that show that the system functions as intended). While the PCE is responsible for developing and executing the V&V plan, the development of the plan is often delegated to a systems engineer or the lead verification and test engineer from Flight Instrumentation and Systems Integration Branch.

In parallel with the V&V plan, the PCE works with the OE and the lead verification and test engineer to develop integration and checkout plans.

Many Center projects are able to integrate using only engineering drawings and an engineering schedule. More complex projects may require significant planning around the order of integrating subsystems and ensuring that subsystem checkouts flow into system-level checkouts in an orderly manner.

Before use, check the Master List to verify that this is the current version. For reference only when

The chief engineer works with the software manager to ensure that the software development and management plans are baselined; software metrics are established; software configuration control processes are either included in the project configuration management plan, or documented in a separate plan; software classification is appropriate;

software assurance plan is complete and that tools and facilities required for development are ready; any required software requirements specifications are baselined; security, supportability, and user interfaces are accounted for in the design; and that partitioning analysis, executive control, and start/recovery analysis and control and data flow analysis are complete, as are software design documents.

The chief engineer works with the ops engineer and the discipline leads to further develop the test plan. Activities in this phase often center on prioritizing test points and measurements, and defining data analysis plans.

The chief engineer works with the project manager, the lead operations engineer, and the discipline engineers to develop preliminary disposition plans for systems and facilities developed or modified for the project.

Plans for technical papers to be written during the closeout phase will also be developed.

Executing the Project’s Management Processes

During the preliminary design phase, most of the project’s management processes are up and running. The PCE ensures that processes are executed as documented, and either adjusts the process or updates the appropriate document when discrepancies are noted. The identification of hazards, risks and mitigations continues. Design mitigations may become the basis for subsystem requirements.

The PCE leads the project team in a review of the engineering tasks and a detailed plan for the detailed design phase is developed. The PCE works with the project manager to reconcile the engineering tasks with the milestone schedule. Staffing plans are updated as needed based on the updated engineering tasks. The PCE will coordinate changes in staffing plans with the appropriate Research and Engineering Directorate Branch Chiefs.

4.3.1 Suggested Minimums and Tailoring

Before use, check the Master List to verify that this is the current version. For reference only when

Projects at low risk for development may be able to combine preliminary design with detailed design or substitute peer reviews for formal reviews. Projects that are not acquiring major subsystems or that are acquiring COTS hardware can forego acceptance testing when the purchased items are to be used within the item’s documented envelope. All projects will develop TPMs in accordance with agency requirements. All projects will develop V&V matrices and plans but smaller, less complex projects might use a simple spreadsheet or something simpler while larger, more complex projects are encouraged to use a data base. Integration planning can take many forms. Smaller, simpler projects may be able to work solely from engineering drawings and regression test plans. Larger, more complex projects may require detailed integration plans and a number of subsystem and system checkout plans to cover all functionality. Flight tests completely inside the design flight manual envelope may require a very simple test plan or simple mission plans. Flight tests that involve significant modifications, envelope expansion, or elevated risk testing require formal test plans and analyses to back the plans up.

4.4 Detailed Design

During the detailed design phase, the discipline engineers complete the design to the point that fabrication, acquisition, and integration can begin.

As in the preliminary design phase, the chief engineer continues to focus on ensuring that subsystem designs coalesce and that the system level requirements will be met on budget and schedule. A critical design review is held at the end of this phase to evaluate whether the maturity of the design is appropriate to support proceeding with full-scale fabrication, assembly, integration, and test, and whether the technical effort is on track to complete the system development and meet project objectives within the project resources. Peer reviews of individual subsystem designs are often held before the system-level PDR to ensure a thorough review of the subsystem.

Design Solution Definition

Interface Management

Configuration Control

System and Subsystem Baselined Designs

Includes drawings, specifications, ICDs, Fabrication Plans, Acquisition Plans, Acceptance Plans

Baselined V&V Plans and procedures

Baselined Acceptance Plans and procedures

Engineering Schedule

Before use, check the Master List to verify that this is the current version. For reference only when

Baselined Integration Plans and

Baselined Flight Test Plan/Operations Plan and preliminary procedures

System and Subsystem Designs

In this phase, the design is completed, drawings and schematics are finalized, and technical data packages necessary for fabrication and assembly are finalized.

As subsystem designs are finalized, trending information for mass margins, power margins, and other TPMs are updated. The PCE will give special attention to any TPMs that are significantly off target at PDR, and efforts to bring the margins in may entail making priority decisions between subsystems, adjusting mass or power budgets, redesign, or waiving some requirements.

Developing and Executing Technical Plans

The PCE ensures that acceptance criteria are finalized and that the plan for verifying the criteria is documented and understood by both the developer and the receiving party.

The PCE ensures that plans and procedures for environmental tests, acceptance tests, and fabrication and assembly, integration, and checkout are ready for baseline at CDR.

The PCE works with the instrumentation engineer and the discipline team to finalize the Master Measurement List (MML).

Acquisition of long lead items sometimes needs to occur before the detailed design is complete. In this case, the PCE will develop and document a risk-informed decision making process to document the risk of proceeding with the acquisition.

The chief engineer works with the ops engineer and the engineering team to draft mission rules, control room staffing plans, and control room displays. The PCE works closely with the Test Instrumentation Engineer from the Range Engineering Branch to ensure that real time displays and any specialized procedures or post-flight data products can be developed in time to support the test.

The chief engineer works with the system safety representative to ensure that safety mitigations are included in test plans and other V&V plans.

Before use, check the Master List to verify that this is the current version. For reference only when

V&V plans are finalized and plans to finalize the V&V procedures are in place. The chief engineer ensures that the V&V tracking process is ready to start executing.

The PCE works with the ops engineer and the discipline team to finalize the test plan, to draft flight-test cards, pre-, post- and through-flight procedures, and periodic inspection plans for research systems.

As during the preliminary design phase, most of the project’s management processes are up and running in the detailed design phase. The PCE ensures that processes are executed as documented and either adjusts the process or updates the appropriate document when discrepancies are noted. The identification of hazards, risks and mitigations continues.

As the project nears the end of the detailed design phase, one last review of the engineering tasks to finish the project is completed. The PCE works with the ops engineer and the project manager to align the detailed schedule with the milestone schedule. Staffing estimates are reviewed and revised as appropriate.