Applicable_Document_-_ATR_SEP_v1_07_.docx

System Engineering Plan (SEP)

Document Number: ATRPE00001 Revision 1.07 14 September 2011

Prepared by:

Systems Engineering Focus Group Atlantic Test Range 23013 Cedar Point Road Patuxent River, MD 20670

ATRPE00001 ATR Systems Engineering Plan Rev 1.07

Distribution Statement D: Distribution authorized to Department of Defense and U.S. DoD contractors for Administrative or Operational Use, (12 June, 2005). Other requests for this document shall be referred to NAWCAD Code 522000A, Range Operations Division, 23013 Cedar Point Road, Patuxent River, MD 20670, DSN 342-9609.

ii

NOTICE TO REVIEWERS

This document was submitted for review and approval to the persons and organizations listed below. The date identifies when Approvers gave approval of the document as submitted.

Rob Vargo, Code 5200 ATR Associate Director Date

Yvonne Davis, Code 5220 Range Operations Division Head Date

Range Instrumentation Division Head Date

Ken Clarke, Code 5240 ATR Range Architect Date

Bill Darden, Code 5200 ATR Chief Engineer Date

Date

REVISION RECORD

Table of Contents

Introduction The purpose of this ATR Systems Engineering Plan (SEP) is to define and describe the steps in a Systems Engineering (SE) process, and provide overarching guidance for the development and sustainment of ATR systems.

This SEP defines a process focused on documenting important project plans, goals, requirements, and decisions, which are reviewed at each phase of development.

This document defines:

· SE activities for system development / sustainment

· Project milestone reviews

· SE process checklists to assist with the project reviews

The complexity of SE documentation and the level of review/oversight shall be adjusted appropriately for each project. It is the responsibility of the individual project teams to define, and document their specific application of ATR’s SE processes, as needed, based on the overarching guidance set forth in this document.

References Government Standards

a) MIL-STD 499, Systems Engineering

b) DI-MGMT-81024, Data Item Description for Systems Engineering Management Plan (SEMP)

c) NAVAIR-4355-19D, System Engineering Technical Review Process

ATR Guidance

a) Atlantic Test Ranges Systems Engineering and Project Management Policy

b) Atlantic Test Ranges Configuration Management Plan

ATR Templates

a) Atlantic Test Range Project Overview Template

b) Atlantic Test Range Project Quad Chart Template

c) Atlantic Test Range Project Plan Template

Systems Engineering Application The ATR Systems Engineering process described in Section 4.0 is intended to fit mid-to-large scale development projects with multiple developers working simultaneously. Smaller scale development efforts are encouraged to adapt the process below to fit within time, resource and schedule constraints.

The specific application of the ATR SE processes, along with any project-specific tailoring must be documented in each project’s Project Plan. See Project Plan template for guidance.

ATR Systems Engineering Process The ATR Systems Engineering Process outlines a structured engineering approach that facilitates the delivery of quality products and systems that are reliable, documented and supportable. The ATR SE Process consists of four phases: Planning, Development, Test, and Deployment.

Figure 1 - ATR SE Phases

The completion of each SE phase increases the level of project maturity. In many cases, discoveries natural to the development process might require temporary backtracking into an earlier phase for reevaluation or correction. This iterative and cyclic process is inherent to many development methodologies, and is compatible with ATR’s SE Process.

Each SE phase contains multiple SE activities, and concludes with a project review. Project reviews will be conducted to ensure sufficient progress has been made before entering the next phase. Project reviews can consist of a peer review, or may be scaled up to the ATR level. Projects reviews should be set at the lowest, most effective level possible, while ensuring all relevant stakeholders are present. An independent reviewer will lead all project reviews, examine project artifacts, and ensure the project has achieved all applicable review criteria. See Appendices A-F for review checklists. Project reviewer selection should take into account experience, knowledge, and independence of the work under review. A peer developer, team lead, section head, branch head, or ATR Range Architect could all be acceptable reviewers, and should be selected to appropriately match the project scope.

ATRPE00001 ATR Systems Engineering Plan Rev 1.07

The ATR SE Process listed in Figure 2 expands each SE phase to show the individual activities and reviews within.

XX

Figure 2 - ATR SE Process ATRPE00001 ATR Systems Engineering Plan Rev 1.07

The SE Process feedback loops, allowing processes to be revisited, have been omitted for diagram clarity. Each block in the above diagram is further described in the following sections.

Planning The Planning Phase is the initial project phase where project objectives are defined, resources are gathered, and plans are made.

The two SE activities within the Planning phase are Project Planning and Requirements Analysis. The Planning phase culminates with the Planning Phase Review (PPR) Project Planning Project Planning is the initial activity in which the project team gathers resources, and works with the customer to define the basic need/scope of the project. Project execution details shall be documented using the ATR Project Plan template.

The Project Plan contains:

· Project goals

· Project deliverables

· Project stakeholders

· Project team members

· Required resources

· Estimated schedule

· Project-specific tailoring decisions Requirements Analysis Requirements Analysis is one of the most important activities in the entire SE Process. In this stage, requirements are identified, documented and customer-validated. The requirements definition process should include meetings between project personnel, customer(s), users and stakeholders to ensure a complete set of requirements is captured. Requirements definition should start with capturing the high-level need, then deriving supporting requirements to include greater levels of detail when necessary.

Requirements should be documented and validated prior to working on the technical solution.

Requirements must take into account:

· Functionality

· External Interfaces

· User Interfaces

· Performance

· Reliability and Availability

· Maintenance/Support Requirements must be expressed in quantitative performance values, which can be verified through test, analysis, inspection, or demonstration. For example, phrases such as “user friendly” or “quickly” are not acceptable.

Requirements will be organized and numbered hierarchically, where top-level requirements are described in more detail by lower-level requirements.

Planning Phase Review The primary purpose of the Planning Phase Review (PPR) is to ensure the project has enough information, resources, and a sufficient plan to begin development. A person with programmatic oversight, such as a Branch Head or Team Lead, should lead the PPR. It is highly suggested that a technical person be included in the review process to evaluate the technical feasibility and/or merits of the requirements.

The PPR formalizes the following work products:

· Project Plan

· Requirements Specification(s) This is a mandatory review for all projects, and cannot be tailored out.

The PPR Checklist can be found in APPENDIX A

Development The Development Phase is the project phase where the project plans are executed, requirements are translated to a technical design, and the design is transformed into a functioning work product that can be tested.

The Development phase activities are System Design and System Development, with a readiness review after each activity (Critical Design Review, and Test-Readiness Review).

System Design System Design is the activity where the requirements are examined, and system functions are extracted and translated into a design architecture. The system architecture shall define the system components, sub-components, as well as external interfaces and the data that flows across it.

System Design Documentation must address:

· System Components

· System Functions

· External Interfaces

· Data Flow Prototyping of some system capabilities may be performed during the design phase to validate concepts or to explore alternatives to reduce program risks.

System design elements should be defined that address each of the requirements for the system. Design elements must be traced back to a system requirement to ensure complete coverage and to minimize unnecessary functionality, or “gold-plating.”

At the completion of the System Design activity, a Critical Design Review (CDR) will be held.

Preliminary Design Review For complex, or high risk development projects, the System Design activity may be split in half by adding a Preliminary Design activity and, pairing it with a Preliminary Design Review(PDR). The introduction of a PDR will give projects the opportunity to review key design decisions early in the development stage, and provide course corrections before they become too costly.

The preliminary design shall define the system boundary, delineating what is internal to system, and its external interfaces. The preliminary design shall also include the primary sub-systems / components, as well as the data that it expected flow between system elements.

The PDR is an optional review, and can be tailored out.

The PDR Checklist can be found in APPENDIX B

Critical Design Review The primary purpose of the Critical Design Review (CDR) is to review the system’s documented design, and ensure design details have been sufficiently captured. The CDR will examine all design artifacts, including (but not limited to) architecture diagrams, drawings, and interface specifications.

Project design documentation must demonstrate the completeness of product specifications to a level that supports system fabrication, and/or software development. A technical assessment will be conducted during CDR to ensure requirements and functions have been properly allocated to sub-systems and components.

The CDR brings closure to technical risk mitigation steps and alternate design paths. Once the design baseline is established, changes to requirements, or design elements can only be accomplished through a formal change request process.

The CDR is a mandatory review for all projects and cannot be tailored out.

The CDR Checklist can be found in APPENDIX C

System Development System Development is the realization of the system design into a functional product. An evolutionary development approach is recommended, delivering minimal functionality as soon as possible. After initial product delivery, additional capabilities and functions will be continuously implemented and incrementally delivered. This development process promotes early detection of integration and performance issues, and should be considered.

When selecting components to fulfill system requirements, an analysis (formal or informal) shall be performed to identify the optimal solution, balancing total lifecycle factors, to include performance, technical maturity, cost, obsolescence, etc. For significant decisions, the analysis of alternatives shall be formalized and documented.

Technical interface meetings should be held with the users and stakeholders during the development phase of the system to ensure the user interface meets the ease-of-use and the system interaction needs of the user.

Unit testing, and developer testing shall be performed as part of the development process, and unit procedures must evolve to match the current system capabilities.

At the successful completion of unit testing, all system components will be integrated to form the complete system, and all unit test procedures will be compiled to form complete system-level test procedures, in preparation for the Test Readiness Review (TRR).

Test Readiness Review The primary purpose of the Test Readiness Review (TRR) is to determine the stability of the current system configuration, and ensure it is ready to proceed into formal test. This review serves as a gateway prior to the start of formal testing, and is independent of any informal testing that might have occurred during the development phase.

The TRR will examine test plans and/or procedures and will verify that sufficient resources are available to begin testing.

The TRR is optional, and can be tailored out.

The TRR Checklist can be found in APPENDIX D

Test The Test Phase is where the developed work product is tested for its ability to meet requirements. Success or failure during this phase will determine if a product is ready to be deployed, or if it needs to return to the development phase for further work.

The Test phase consists of a System Test activity, followed by a Post-Test Review (PTR) to evaluate system performance.

System Test System Test is the activity where the as-built solution is formally tested to ensure the system is stable, robust and meets all of the user needs. The system shall be tested under conditions that are as close as possible to the end state operating environment.

Testing consists of two steps, verification and validation. Verification is the process in which a system is measured against the written specifications and requirements, and validation is the measurement of a system’s ability to satisfy the user’s operational need.

Verification Verification is conducted to ensure the technical solution complies with the approved, base-lined system requirements.

· In simple terms, the verification process answers the question “Was the system built right?”

Formal verification will include the development of test procedures that are traced back to requirements. As each test procedure is systematically exercised, system performance is judged to pass or fail. A test report will be written after each formal verification event, documenting the system’s as-built capability with respect to the requirements.

Validation Validation is conducted to evaluate the system on its ability to perform in its intended environment. Validation is performed with a representative system user, prior to delivery, and serves as the crucial step in evaluating the practicality and usability of a system.

· In simple terms, validation answers the question “Was the right system built?”

Formal validation will include documented proof of user acceptance.

Post-Test Review The primary purpose of the Post-Test Review (PTR) is to examine the test results and determine the degree to which the system meets requirements. The PTR will examine test reports, identified defects, and other test artifacts to determine the system’s technical performance. Weighing technical performance against, cost, and schedule, the PTR will determine if a system will proceed to deployment, or return to the development phase.

The PTR is optional, and can be tailored out.

The PTR Checklist can be found in APPENDIX E Deployment The Deployment Phase is the final project phase where the completed work product is formally released for operational use. Deployment officially marks the end of the development process and transitions the system into maintenance mode.

The Deployment phase consists of the Deployment Planning activity, followed by a Deployment Readiness Review (DRR).

Deployment Planning Deployment Planning is the project activity where the system is prepared for the transition into operational status.

· Deployment planning must include: Development of maintenance manuals

· Development of user’s guides

· Plan to successfully transition the system to operational status

· Plan to ensure deployment does not affect other connected systems

Deployment Readiness Review The primary purpose of the Deployment Readiness Review (DRR) is to evaluate the developed system’s ability to transition to operational status. This review occurs after all formal verification and validation is completed, and is the final step before deployment.

The DRR will examine the final product, and will determine the project has an acceptable level of risk to proceed. Project leadership will be expected to provide test reports, defect reports, and other test artifacts to demonstrate an acceptable level of performance consistent with a production-ready system.

Supportability documentation such as user guides and maintenance manuals, if applicable, will be required for review.

The DRR is a mandatory review for all projects, and cannot be tailored out.

The DRR Checklist can be found in APPENDIX F Systems Engineering Management The following sections detail how ATR Systems Engineering efforts will be managed throughout the development lifecycle.

Requirements Management All requirements will be documented and managed under formal configuration control. Requirements will be base-lined at the end of every project phase, and changes to the approved requirements will be formally controlled via a Configuration Control Board (CCB) in accordance with the ATR Configuration Management (CM) Plan.

Requirements shall be traced to test procedures, and to the elements within the functional architecture. As the system architecture evolves, the requirements repository shall evolve as well, maintaining traceability.

Technical Program Planning and Control ATR Projects will be managed in accordance with their individual Project Plans.

Configuration Management Project artifacts, including hardware, software and documentation will be placed under configuration management in accordance with the ATR Configuration Management Plan Project Documentation All projects will be required to maintain a set of engineering documentation describing the technical details of their project. The minimum set of documentation required is described in the following sections. Any tailoring of the described documentation set must be approved at PPR. Refer to the WIKI Systems Engineering page for the Project Plan template.

Requirements Specifications Documentation must exist that describes the capabilities that the system is expected to perform. Base-lined system requirements documentation forms the foundation of a solid systems engineering process, and shall be used to guide the development of test procedures, and any follow-on capability improvements. Refer to the WIKI Systems Engineering page for System Requirements Document (SRD) examples.

Design Documentation System Design must be documented to the level appropriate for any new, appropriately skilled project team member to understand how to maintain existing capabilities and add new ones. Design documentation at a minimum must identify the major system and sub system components, and their interfaces, both internal and external. Refer to the WIKI Systems Engineering page for System/Subsystem Design Document (SSDD) and Software Design Document (SDD) examples.

Interface Specifications An interface specification must be written to fully describe every external system interface. Each interface specification must describe at minimum the data that is being transmitted by the system, and the data that the system must receive as input to function. All constraints with respect to acceptable physical interfaces, data types, data frequencies, and data boundaries must be documented, and kept up to date as the system evolves. Refer to the WIKI Systems Engineering page for Interface Control Document (ICD) examples.

Test Documentation At minimum, a project shall have a test plan, describing the testing philosophy, methodology, and resources used to verify the completed system will work as required. It is highly recommended that systems use the test plan and the requirements documentation to generate step-by-step test procedures. Each developed test procedure must trace back to a requirement, to ensure that all required functionality is verified, and that each tested procedure is rooted in a requirement.

System Engineering Project Classification The Systems Engineering processes described within this document can be tailored to fit a wide range of development work at the Atlantic Test Range, regardless of system lifecycle phase or development methodology. Each project will be responsible for tailoring ATR’s SE processes to appropriately fit the scope and complexity of the work to be performed. The following sections serve as a guide for project classification and expected level of SE tailoring Development Projects A project is classified a development project when a new capability or system enhancement is being implemented and the project involves any of the following:

· Software Development

· Hardware Development A project is not considered a developmental project when the project consists of:

· Bug fixes

· Maintenance actions

· Duplication of an existing system

· Redeployment of an existing system

· Extension of an existing system, not requiring development effort

· Form, fit, function, replacement of an existing component

· COTS upgrades to an existing system

· Stand alone COTS capability

Projects classified as development projects are required to follow ATR’s SE process at project onset, with the least amount of tailoring possible.

System Upgrade and Sustainment Projects approved to reduce obsolescence or enhance system capability through significant redesign and/or enhancement shall follow the ATR SE processes, but tailoring is expected to match the scope and complexity of the upgrade effort.

Projects that upgrade an existing system shall generate systems engineering documentation describing the development work associated with the upgraded effort only. Unless specifically directed, documentation of existing system functionality that is unchanged by the upgrade effort is not required.

ATRPE00001 ATR Systems Engineering Plan Rev 1.07

Planning Phase Review Checklist

Purpose

· To assess the scope, complexity and execution approach of the project for technical and programmatic risk.

· To ensure system performance requirements are defined, and consistent within programmatic constraints (cost, schedule, risk)

· To ensure the project team is ready to proceed to the development phase Entrance Criteria

1. Completed Project Plan Refer to the WIKI Systems Engineering page for the Project Plan template

2. System Requirements Document (SRD) has been finalized and baselined

3. Initial Risk Assessment Exit Criteria

1. Project scope defined and validated.

2. Project approved, placed on-hold, or rejected.

3. Stakeholder concurrence with project plan

4. Definition of required SE reviews

5. Definition of required SE documents and deliverables

6. Requirements are sufficiently detailed to a level that enables initiation of preliminary design

7. Requirements are approved and validated by key stakeholders i.e. customers and users

Preliminary Design Review Checklist Purpose

· To ensure the initial design concept is sound

· To ensure the project team is ready to proceed to the System Design activity

Entrance Criteria

1. Project Plan has been updated, and accurately describes project status

2. Preliminary system design or architecture is documented

3. Design elements are identified and linked to system requirements

4. External interface data elements are defined

5. User interface design prototype / mock-up, if applicable

Exit Criteria

1. Preliminary system design, as defined, satisfies the operational need.

2. Preliminary design addresses all system requirements for the defined baseline.

3. Deficiencies in preliminary design are identified indicating areas needing additional design details to progress to CDR.

Critical Design Review Checklist

· To ensure a detailed system design is documented and complete

· To ensure the project is ready to proceed with development and/or fabrication

· To ensure the system design implements functional capabilities that will meet all requirements.

· To ensure project execution is achievable within programmatic constraints (cost, schedule, risk)

Entrance Criteria

1. Project Plan has been updated, and accurately describes project status

2. System Requirements are complete, baselined and approved

3. System design architecture is accurately documented, and have been baselined

4. System/sub-system and component constraints have been identified

5. System Interface Specifications have been documented and baselined

6. User interface design is finalized, if applicable

Exit Criteria

1. An acceptable level of technical and programmatic risk, suitable for transitioning to the system development, has been ascertained

2. Detailed system design provides an operationally suitable and effective solution

3. Detailed system design traces completely to the requirements

4. Product baseline has been established, baselined, and placed under formal configuration control to enable hardware fabrication and software development

Test Readiness Review Checklist Purpose

· To ensure the developed system is ready to proceed to test

· To ensure test objectives, methods, and procedures have been properly identified and test resources have been coordinated Entrance Criteria

1. Configuration of the “system under test” has been defined, documented, and agreed upon by all stakeholders

2. All interface specifications are updated, complete, and baselined

3. Version Description Documentation for the “system under test” is updated, and baselined

4. Informal test events (functional tests, unit tests, sub system tests, etc) have been conducted successfully

5. All known system discrepancies have been identified and documented

6. All required test resources (manpower, facilities, test articles, test instrumentation, etc) have been identified and are available to support the test events.

Exit Criteria

1. Test plans have been completed, baselined, and approved for system test

2. Test procedures effectively test all system requirements.

3. Adequate identification and coordination of required test resources has been completed

Post-Test Review Checklist

· To ensure the developed system meets requirements and specifications to a degree suitable for operational use Entrance Criteria

1. All known system discrepancies have been identified and documented Exit Criteria

1. Project approval to proceed, or return to development

Deployment Readiness Review Checklist

· To ensure system is ready for deployment and/or to proceed to production Entrance Criteria

1. System performance/qualification testing has been completed successfully

2. System support documentation (users manuals, drawings, maintenance guides) have been completed and placed under configuration management

3. Deployment Plan documented Exit Criteria

1. Project approval to deploy image1.png image2.emf oleObject1.bin

Deployment

Test

Development image3.emf oleObject2.bin Project

Requirements Analysis

System Development

System

Deployment Planning

System Design

Preliminary Design

Critical Design Review

Readiness Review

Deploy

Post-Test Review*

Readiness Review*

Preliminary Design Review*

Phase Review

Development

- Project Plan / SEMP

- Cost

- Schedule

- Requirements Spec

- Req Spec

- Project Plan

- Market Studies

- Block Diagram

- Data Flow

- Interfaces

- Design Specification

- Design Specification

- Preliminary Design

- System

- Informal Testing

- Test Plan

- Final Product

- Test Plan

- Test Report

- Defects

- Acceptance

- Tech Manual

- User Guide

- Deployment Plan

- Change Review Board

(CRB)

- Final System

- XXX

- YYY

Key

SE Activities

Project Reviews

Products Generated

Products Finalized

- XXX

- YYY

Optional Review