Atch 3 - Entrance and Exit Criteria for SRR PDR CDR 20120223.pdf

DRAFT

Page 1 of 16 FA8307-12-R-0003

Attachment 3, Revision 1

Attachment 3

Entrance and Exit Criteria for System

Requirements Review (SRR), Preliminary

Design Review (PDR) and Critical Design

Review (CDR)

For

Family of Advanced Beyond-Line-of-Sight

Terminals

Page 2 of 16 FA8307-12-R-0003

System Requirements Review (SRR)

Representative items to be reviewed include the results of the following:

Mission and Requirements Analysis

Contractor shall provide a product that explicitly interprets or states no interpretation required for each shall in the Government Technical Requirements Document.

Functional Flow Analysis

Contractor shall provide a system architecture document.

Preliminary Requirements (functional and performance) Allocation to Hardware

Configuration Items (HWCIs) and Software Configuration Items (SWCIs).

Identifying resources of the system and partitioning the system into HWCIs and CSCIs.

Identify the pedigree of each HWCI and SWCI and the planned scope of modification for

CIs that incorporate reused artifacts or products.

Trade studies (e.g., addressing system functions in mission and support hardware/firmware/software).

Program Risk Analysis

Detailed and proactive mitigation plans defined.

Identify immature technologies and mitigation steps to achieve the required maturity.

Identify any potential certification issues.

Integrated Test Planning

Identify test requirements that must be incorporated into the CIs to support system-level integration and test.

Technical Performance Measurement Planning

To include traceability to the Key Performance Parameters (KPPs).

Identify the set of Technical Performance Measures (TPMs) that will be used to monitor the technical status of the ability of the design to meet the requirements.

Provide TPM maturation plans.

Affordability

Report current unit price estimates including a description of the maturity of the estimates

Discuss affordability initiatives, cost drivers and potential risk areas

Entrance Criteria

SRR will be conducted when:

The SRR agenda has been agreed to by the Government 30 days prior to the SRR.

Expectations established for the technical product content, quality, and level of completion have been agreed to by the Government.

A Readiness Review has been conducted 20 days prior to the SRR to review draft briefing material and CDRLs http://sparc.airtime.co.uk/users/wysywig/semp41.htm http://sparc.airtime.co.uk/users/wysywig/semp42.htm

Page 3 of 16 FA8307-12-R-0003

All agreed to artifacts in support of SRR including the briefing is posted 10 calendar days before the meeting.

The Integrated Master Schedule has been updated highlighting all detailed design, development, build, and verification tasks.

Exit Criteria

SRR will be complete when:

▪ The GTRD requirements baseline and verification methods are agreed to by the

Contractor and the Government. Agreement is defined as either no changes to the TRD being required (documented in meeting minutes) or an ECP submitted against an agreed to schedule.

▪ The GTRD requirements interpretation database is updated, agreed to, and flowed down to all subcontractors. Agreement is captured in a configuration controlled product made available to the program.

▪ Full bidirectional traceability of all GTRD requirements agreed to. Agreement is captured in a configuration controlled product made available to the program.

▪ All system functions and non-functional requirements are assessed and determined to be achievable. Agreement is captured in a configuration controlled product made available to the program.

▪ Risk and mitigation strategies updated to reflect any new information noted during the

SRR. Agreement is captured in a configuration controlled product made available to the program.

▪ All Critical action items have been closed by the Government. A Critical action item is defined as a request for information or material that is fundamental to the accomplishment of the milestone.

▪ All non-critical SRR action items and issues have been closed or action plans with closure dates have been developed to the satisfaction of the Government.

All SRR data accomplished per the CDRL.

Page 4 of 16 FA8307-12-R-0003

Preliminary Design Review (PDR)

System Engineering

System design/architecture (for each terminal configuration).

Platform Interface Design (for ground fixed, ground transportable, E-4, and E-6 platforms).

TPMs for each quantitative requirement to include the maturation plan.

Provide detailed definition of the Busy Minute Scenario to include how it will be used to measure the processor utilization requirements.

Trade Studies

Risks and mitigation plans.

Modeling and Simulation efforts, to include but not be limited to ensuring data throughput and processing margins and how they will report regularly with TPMs.

Proprietary and Data Rights.

System integration and test plan, and verification plan.

Maintainability to include identification of any periodic maintenance items or special support equipment, special training and/or special security clearance needs.

Affordability o Report current unit price estimates including a description of the maturity of the estimates o Discuss affordability initiatives, cost drivers and potential risk areas

Compatibility between the Government Technical Requirements Document (GTRD) requirements, the preliminary requirements and design to include:

For HWCIs

Synthesis of the hardware Development Specification for the item being reviewed

Functional flow, requirements allocation data, and schematic diagrams

Equipment layout drawings and preliminary drawings, including any proprietary or restricted design/process/ components and information.

Environmental design aspects

Electromagnetic compatibility

Power distribution and grounding design aspects

Mechanical and packaging design of consoles, racks, drawers, printed circuit boards, connectors, etc.

Safety engineering considerations

Security (IA) engineering and certification considerations

Survivability/Vulnerability (including nuclear) considerations

Pertinent reliability/maintainability/availability data

Page 5 of 16 FA8307-12-R-0003

Weight data

Interface requirements contained in configuration item Development Specifications and interface control data (e.g., interface control drawings) derived from these requirements.

Configuration item development schedule

Mock-ups, models, breadboards, or prototype hardware (need to define specifics)

Transportability, packaging, and handling considerations

Human Engineering including Nuclear, Biological and Chemical Contamination considerations

Life Cycle Cost Analysis

Armament compatibility

Corrosion prevention/control considerations

Support equipment requirements

For CSCIs

Functional flow. The computer software functional flow embodying all of the requirements allocated from the Software Requirements Specification and Interface

Requirements Specification(s) to the individual configuration item.

Storage allocation data. Description (for each Computer Software Configuration Item

(CSCI)) of the manner in which available storage is allocated to individual Top Level

(TL) SW Units. Timing, sequencing requirements, and relevant equipment constraints used in determining the allocation are to be included.

Control functions description. Description of the executive control and start/recovery features for the CSCI shall be available, including method of initiating system operation and features enabling recovery from system malfunction.

CSCI structure. Description of the top-level structure of the CSCI, the reasons for choosing the components, the development methodology which will be used within the constraints of the available computer resources, and any support programs which will be required in order to develop/maintain the CSCI structure and allocation of data storage.

Security. Identification of unique security requirements and a description of the techniques to be used for implementing and maintaining security within the CSCI shall be provided.

Computer software development facility versus the operational system. Information relative to unique design features which may exist in a TL Computer Software

Component (TLCSC) in order to allow use within the computer software development facility, but which will not exist in the TLCSC installed in the operational system and on the design of support programs not explicitly required for the operational system but which will be generated to assist in the development of the CSCI(s). The contractor shall also provide details of the Software Development Library controls.

Development tools. Description of any special simulation, data reduction, or utility tools that are not delivered under the terms of the contract, but which are planned for use during software development.

Page 6 of 16 FA8307-12-R-0003

Test tools. Description of any special test systems, test data, data reduction tools, test computer software, or calibration and diagnostic software that are not deliverable under terms of the contract, but which are planned for use during product development.

Operation and support documents. Review of the preliminary versions of the Computer

Systems Operators Manual (CSOM), Software User’s Manual (SUM), Computer System

Diagnostic Manual (CSDM), and Computer Resources Integrated Support Document for technical content and compatibility with the top-level design documentation.

Software maintenance and upgrades. Description of the method and any constraints associated with the upgrade of software throughout the development and operational lifecycle.

Entrance Criteria

PDR will be conducted when:

The PDR agenda has been agreed to by the Government 30 days prior to the PDR.

A Readiness Review has been conducted 20 days prior to the PDR to review draft briefing material and CDRLs.

Expectations established for the technical product content, quality, and level of completion have been agreed to by the Government.

All agreed to artifacts in support of the PDR including the briefing is posted 10 calendar days before the meeting.

The risk assessment has been updated to include status of all mitigation activities.

The Integrated Master Schedule has been updated highlighting all detailed design, development, build, and verification tasks.

TPMs posted that illustrates product maturity and functional performance gains expected across the development activities.

Documentation delivered per the CDRL

In addition to the CDRL, the following documentation has been made available to the

Government 45 days prior to the event:

System/Subsystem/Software Design Description material supplemented by

Interface Design Descriptions (IDDs)

Engineering Drawings

System/Subsystem/Software Specifications

System Safety Program Plan

The PDR will be complete when:

▪ Government comments on Contractor documentation listed under Entrance Criteria are dispositioned. Agreement is captured in a configuration controlled product made available to the program.

Page 7 of 16 FA8307-12-R-0003

▪ The system preliminary derived requirements have been established, agreed-to, and put under formal configuration control. Agreement is captured in a configuration controlled product made available to the program.

▪ The system preliminary design baseline has been established and agreed-to. Agreement is captured in a configuration controlled product made available to the program.

▪ The risk assessment and mitigation strategies have been updated and posted on a digital environment.

All technical and management processes in accordance with CMMI Level 3 (or higher) have been established and fully implemented (with the exception of production, manufacturing and logistics support processes).

All Critical action items have been closed by the Government. A Critical action item is defined as a request for information or material that is fundamental to the accomplishment of the milestone.

All non-critical PDR action items and issues have been closed or action plans with

All PDR data accomplished per the CDRL.

Page 8 of 16 FA8307-12-R-0003

Critical Design Review (CDR)

System Engineering

System design/architecture (for each terminal configuration).

Platform Interface Design (for ground fixed, ground transportable, E-4, and E-6 platforms).

TPMs for each quantitative requirement to include the maturation plan.

Provide updates to the detailed definition of the Busy Minute Scenario to include its application to measurement of the processor utilization requirements.

Trade Studies

Risks and mitigation plans.

Modeling and Simulation efforts, to include but not be limited to ensuring data throughput and processing margins and how they will report regularly with TPMs.

Proprietary and Data Rights.

System integration and test plan, and verification plan.

Maintainability to include identification of any periodic maintenance items or special support equipment, special training and/or special security clearance needs.

Affordability o Report current unit price estimates including a description of the maturity of the estimates o Discuss affordability initiatives, cost drivers and potential risk areas

Compatibility between the GTRD requirements and the baseline requirements and final design to include:

For HWCIs

Adequacy of the detail design reflected in the draft hardware Product Specification in satisfying the requirements of the HWCI Development Specification for the item being reviewed.

Detail engineering drawings for the HWCI including schematic diagrams.

Adequacy of the detailed design in the following areas:

o Electrical design o Mechanical design o Environmental design o Electromagnetic compatibility o Power generation and grounding o Electrical and mechanical interface compatibility o Mass properties o Reliability/Maintainability/Availability o System Safety Engineering o Security Engineering o Survivability/Vulnerability (including nuclear)

Page 9 of 16 FA8307-12-R-0003 o Producibility and Manufacturing o Transportability, Packaging and handling o Human Engineering and Biomedical Requirements (including Life Support and Crew

Station Requirements) o Standardization o Design versus Logistics Trade-offs o Support equipment requirements

Interface control drawings.

Mock-ups, breadboards, and/or prototype hardware.

Design analysis and test data.

System Allocation Document for HWCI inclusion at each scheduled location.

Initial Manufacturing Readiness (for example, manufacturing engineering, tooling demonstrations, development and proofing of new materials, processes, methods, tooling, test equipment, procedures, reduction of manufacturing risks to acceptable levels).

Life cycle costs (to include per-unit projected costs).

Detail design information on all firmware to be provided with the system.

Verify corrosion prevention/control considerations to insure materials have been chosen that will be compatible with operating environment.

Findings/Status of Quality Assurance Program.

For CSCIs

Software Detailed Design, Data Base Design, and Interface Design Document(s). In cases where the CDR is conducted in increments, complete documents to support that increment shall be available.

Supporting documentation describing results of analyses, testing, etc., as mutually agreed by the contracting agency and the contractor.

System Allocation Document for CSCI inclusion at each scheduled location.

Computer Resources Integrated Support Document.

Software Programmer's Manual

Firmware Support Manual.

Updated operation and support documents (CSOM, SUM, CSDM).

Schedules for remaining milestones.

Updates since the last review to all previously delivered software related CDRL items.

Software maintenance and upgrades. Description of the method and any constraints associated with the upgrade of software throughout the development and operational lifecycle

Entrance Criteria

CDR shall be conducted when:

The CDR agenda has been agreed to by the Government 30 days prior to the CDR.

A Readiness Review has been conducted 20 days prior to the CDR to review draft briefing material and CDRLs.

Page 10 of 16 FA8307-12-R-0003

All agreed to artifacts in support of the CDR including the briefing is posted 10 calendar days before the meeting.

All specifications and design artifacts have been peer reviewed and assessed with quality assessments available for review prior to the CDR.

The risk assessment has been updated to include status of all mitigation activities.

Mapping of requirements and functionality to builds complete per the CDRL.

TPMs and up to date metrics posted that illustrates product maturity and functional performance gains expected across the development activities.

Documentation delivered per the CDRL

In addition to the CDRL, the following documentation has been made available to the

Government 45 days prior to the event:

System/Subsystem/Software Design Description material supplemented by

Interface Design Descriptions (IDDs)

Engineering Drawings

System/Subsystem/Software Specifications

Exit Criteria

CDR shall be complete when:

▪ Government comments on Contractor documentation listed under Entrance Criteria are dispositioned. Agreement is captured in a configuration controlled product made available to the program.

▪ All CDR documentation requiring a final delivery has been complete and has been accepted by the Government per CDRL.

▪ The system final “build to” design and fabrication baseline has been established and under full configuration control. Agreement is captured in a configuration controlled product made available to the program.

▪ The risk assessment and mitigation strategies have been updated and posted.

All technical and management processes in accordance with CMMI Level 3 (or higher) have been established and fully implemented (with the exception of production, manufacturing and logistics support processes).

All Critical action items have been closed by the Government. A Critical action item is defined as a request for information or material that is fundamental to the accomplishment of the milestone.

All non-critical CDR action items and issues have been closed or action plans with closure dates have been developed to the satisfaction of the Government.

All CDR data accomplished per the CDRL.

CDR Capability Demonstration

Entrance Criteria

Two terminals available, one being an airborne configuration and the other a ground configuration

Demonstration Readiness Review conducted (contractor content and format)

Page 11 of 16 FA8307-12-R-0003

Dry runs conducted with results available for review

Government agreed to procedures for the demonstration

Contractor selected matrix of UL and DL modes, data rates, and simultaneous services

Schedule for conduct of the demonstration

Satellite access requests submitted and approved

Crypto keys available having accomplished a timely request and delivery

SATSIM configurations defined and available for review

Test equipment available and calibration records available for review

CDR Capability Demonstration

The following are the capabilities to be demonstrated using the current hardware and software that form the basis of the end state design:

***Note: Demonstration capability criteria to furnished later.

Outbrief of the demonstration results

Government agreed-to report documenting demonstration results to include:

o Detailed demonstration results o Identification of terminal failures, issues, and anomalies o Identification of test environment failures, issues, and anomalies o Completed demonstration procedures and data sheets

Page 12 of 16 FA8307-12-R-0003

IMP Milestone Capability Demonstration

Entrance Criteria

Two terminals available, one being an airborne configuration and the other a ground configuration

Demonstration Readiness Review conducted (contractor content and format)

Dry runs conducted with results available for review

Government agreed to procedures for the demonstration

Contractor selected matrix of UL and DL modes, data rates, and simultaneous services

Schedule for conduct of the demonstration

Satellite access requests submitted and approved

Crypto keys available having accomplished a timely request and delivery

SATSIM configurations defined and available for review

Test equipment available and calibration records available for review

IMP Milestone Capability Demonstration

The following are the suggested capabilities to be demonstrated using the current hardware and software that form the basis of the end state design:

***Note: Demonstration capability criteria to furnished later.

Outbrief of the demonstration results

Government agreed-to report documenting demonstration results to include:

o Detailed demonstration results o Identification of terminal failures, issues, and anomalies o Identification of test environment failures, issues, and anomalies o Completed demonstration procedures and data sheets

Capabilities to be demonstrated during the IMP Milestone Capability Demonstration are to be defined in the contractor’s proposal. The IMP Milestone Capability Demonstration section of Attachment 3 to the contract will be updated prior to award to reflect demonstration criteria defined in the awarded contractor’s proposal.

Page 13 of 16 FA8307-12-R-0003

System Verification Readiness Review (SVRR)

Verification events related to TRD requirements shall be preceded by a SVRR with the following entrance criteria

System Verification shall be conducted when:

Government has approved the test procedure

Configuration of the required system hardware/software under test has been defined, approved by the Government, and is under configuration control

Government has approved all waivers and deviations for configuration under test

All Code Software Anomaly Reports (SARs) are closed or exceptions have been approved by the Government

Hardware configuration approved by the Government to include:

Closure of all Hardware Failure Reports (FRs) that affect the test execution and verification with the exception of those that have been dispositioned and approved by the Government.

All lower level testing is complete (EQT, FQT, EMI/EMC, etc).

Hardware under configuration control.

All software qualification tests (SQTs) complete and software under configuration control. (SQT complete defined as all steps successfully executed against Government approved procedures, and a software baseline where all Code Software Anomaly Reports

(SARs) are closed or exceptions have been approved by the Government going into the system verification event. Successful regression testing will be conducted if Code SARs have been opened since the SQT was initially executed.)

Agreed to schedule for execution of system verification procedures

A “dry run” of the test has been conducted and the Government has approved of any changes (redlines) to test procedures resulting from dry runs

All required test resources (people, facilities, test articles, test instrumentation) have been identified and are available to support the test event

All test support tools/simulators have been disclosed and under configuration control

(Tools used in the verification of a GTRD requirement are required to be verified and validated by the contractor and approved by the Government.)

Successful execution of a Test Readiness Review with all action items closed by

Government approval.

Page 14 of 16 FA8307-12-R-0003

System Functional/Physical Configuration Audits (FCA/PCA)

System FCA/PCA shall be conducted when:

A Critical Design Review (CDR) milestone event has been successfully completed, if applicable

All CDR Requests for Action (RFAs) have been closed, if applicable

All CDR exit criteria key issues have been satisfied, if applicable

All system performance specification qualification test requirements have been successfully completed, if applicable

All configuration items’ test/analysis data, including software unit test results are successfully completed and available.

Complete all requirements for entrance into NSA Technical Review Board

Documentation delivered per the CDRL

In addition to the CDRL, the following documentation has been made available to the

Government 45 days prior to the event:

System/Subsystem Specification (SSS) with Requirements Traceability Matrix

Interface Control Document

Software Requirements Specification with Requirement Traceability Matrix

Specification Requirement Verification Matrix

Engineering Change Proposals to CDR Baseline

Product Drawings

Software Version Description

Software Test Description

Software Test Plan

Software Test Report

Hardware Test Procedures

Hardware Test Reports

System Qual Test Procedures

System Qual Test Reports

Exit Criteria

FCA/PCA shall be complete when:

▪ Government comments on Contractor documentation listed under Entrance Criteria are dispositioned. Agreement is captured in a configuration controlled product made available to the program.

▪ All FCA/PCA documentation requiring a final delivery has been complete and has been accepted by the Government per CDRL.

▪ The system final “as built” design satisfies the GTRD requirements.

▪ Successful NSA Certification of Terminal

▪ The system is producible within the production budget.

▪ The risk assessment and mitigation strategies have been updated and posted.

Page 15 of 16 FA8307-12-R-0003

All Critical action items have been closed by the Government. A Critical action item is defined as a request for information or material that is fundamental to the accomplishment of the milestone.

All non-critical FCA/PCA action items and issues have been closed or action plans with

All FCA/PCA data accomplished per the CDRL.

Page 16 of 16 FA8307-12-R-0003

Low Rate Initial Production (LRIP)

Contractor testing has been completed including environmental qualification of the FAB-

T hardware and successful developmental functional testing.

FAB-T initial interoperability has been demonstrated with the legacy Milstar AF

Command Post Terminal and one non-FAB-T XDR terminal over the air.

FAB-T performance in flight has been accomplished including verification of in-flight

LDR communications over Milstar,.

Contractor has successfully completed verification of all GTRD requirements to be evaluated during the AFOTEC operational assessment of the FAB-T.

Contractor is on schedule to deliver a comprehensive logistics management infrastructure that will support government IOT&E activities This is to include training, technical manuals, support equipment, provisioning technical data, supply support and delivery of spares, maintenance planning and management, and packaging, handling, shipping and transportation (PHS&T).

Successfully complete a production readiness assessment as a part of the Independent

Program Assessment (IPA) to support the LRIP decision.

Successfully complete an Independent Security Review.

Milestone Decision Authority (MDA) approval memo