Atch 16 - Critical Design Review

21 February 2012

Critical Design Review (CDR) Checklist

I. Timing/Readiness:

1. A Preliminary Design Review has been successfully completed, and all associated action items have been resolved and are closed with Government concurrence.

2. All PDR exit criteria key issues have been satisfied, if applicable.

3. Identify System Program Office (SPO) and developer chairpersons for the CDR.

4. Contractor identifies the SMEs to participate in the CDR and identified their roles. Provide list to the SPO.

5. Has the location of the CDR been decided? Are there any facility issues that would hamper the

CDR proceedings?

6. Has a CDR recorder(s) been assigned?

7. Are there any special needs that have been identified that need to be accommodated during the

CDR?

8. If all or a portion of the CDR is classified, have appropriate security management requirements

(e.g. clearances, facilities, documentation, etc.) been addressed?

9. Is the Contractor using an effective Integrated Data Environment (IDE) to disseminate ADPs and store other program data so that it is available to the Government?

II. Planning and Conduct Assessment:

1. What plans are in place for hardware, software, and system integration and how will integration progress be measured?

2. What plans are in place for developing and demonstrating manufacturing including: processes, tooling, CAD/CAM, test, rework, flow, inspection, and acceptance?

3. What quality assurance/control participation, reporting systems, and metrics are planned for integration, manufacturing demonstration and the test program?

4. Is the integrated test team adequately staffed by PMO, other Government support, and contractor personnel to manage the development, operational, interoperability, system of systems

(SoS)/family of systems (FoS) testing, certification and accreditation?

5. Does the test plan and IMS contain appropriate duration events for testing and regression testing?

6. Have detailed DT execution plans and OT support plans (including Government and contractor facilities and test resources) been submitted for PMO review?

7. Are test plans, procedures, scripts, results traceable to the PIDS, Rev G, interface control documents, architecture, and product specification verification requirements?

8. What deficiency tracking programs are planned for DT (e.g. hardware, software, test results etc.)?

9. Have metrics (e.g. regression testing, defect density, schedule, deficiency closure rate, test resource availability etc.) been established to track the test program?

10. What are the plans to provide trained personnel to support DT and OT?

11. Does the developer have a detailed integrated master schedule (IMS) for fabrication, coding, integration, and test?

12. Does the IMS identify the overall program critical path?

13. Are critical path tasks defined well enough to determine whether a task is complete or incomplete?

14. Are critical path task durations reasonable (on the order of several weeks duration, or less)?

15. Does the IMS show what slack exists for those elements off the critical path?

16. Are predecessor and successor links included in the IMS for all program activities from contract award to the conclusion of the contract?

17. Is the periodically submitted IMS being used to show actual progress vs. planned activity?

18. Conduct walkthrough of the IMS. Provide information on current status of ongoing tasks.

Emphasize all critical path tasks, those tasks associated with installation, testing and training, and those tasks requiring coordination with other Government contracts and contractors.

19. Present summary information that shows how the contractor’s IMS integrates with the JMS.

20. What does the current EVM data show with respect to schedule variance?

21. Does the program IMS show GFP dependencies, subcontractor/vendor dependencies, inter-program dependencies, and FoS/SoS impacts?

22. What provisions are included in the IMS for DT and OT deficiency corrections?

23. Is there any schedule slack for integration and test activities?

24. What is the current status of each identified Technical Performance Measurement (TPM)?

25. Do these TPMs monitor the highest technical risks in the program?

26. As the CDR begins, what is the state of the developer's cost estimate (i.e. reductions, additions, overruns, and associated rationale for each incident)?

27. Identify metrics other than TPM or EVM proposed to track performance during fabrication, coding, integration, and test.

28. What is the current status of the logistics metrics that reflect the flow down of the supportability and Key Performance Parameters (KPP) to the CDR design? Do these logistics metrics monitor the highest logistics risks in the program?

29. Have the PMO requested a review of the developers Earned Value Management (EVM) system from the Defense Contract Management Agency (DCMA) to include: data currency, baseline management and use, work package integrity, consistency with program risks, and how resources are allocated to address risks?

30. Is the WBS adequately structured to monitor the fabrication, coding, integration, and test of the

CDR design and is this structuring sufficient to monitor the technical risks and the high cost areas of the program?

31. Can the proposed set of software metrics be used to measure software development progress?

32. Has the Contractor provided a staffing plan for the post-CDR efforts of fabrication, coding, integration, testing, and quality management?

33. Does the staffing plan include the following: organizational structure, size, geographical location of staff, organizational affiliation, task & skills analysis, security clearance status, personnel availability (full-time, part-time, and start date), and key personnel?

34. What provisions are in place for monitoring and configuration managing internal and external interface requirements during the fabrication, coding, integration, and testing?

35. Does each of the systems engineering processes include adequate provisions for system engineering during the fabrication, coding, integration, testing, and production demonstration?

36. Which of the systems engineering processes have shared (Government and Contractor) responsibility during the fabrication, coding, integration, testing, and production demonstration

(e.g. Test, Risk Management, Logistics, Planning and Configuration Management)?

37. What Contractor coding and integration risk mitigation procedures are included in the software processes for coding, integration, and testing (e.g. spirals, peer reviews, incremental integration, and collaborative development environment)?

38. Are all of the software development organizations using the same set of software development processes? If not, how will the developer integrate the software development efforts?

39. How have the warfighter hardware and software supportability requirements in the SOW been reflected in the CDR design?

40. Have all supportability KPPs/Technical Performance Measures (TPMs) been met by the CDR

Design?

41. Does your planned maintenance strategy integrate all the elements of logistics (e.g. training, spares, support equipment, tech. manuals diagnostics etc.) for each level of maintenance?

42. Are there design provisions to support the current maintenance and support concepts at all levels of maintenance?

43. Is there a prescribed parts selection criterion (i.e., derating, environmental, preferred parts, standard parts, shelf life, Diminishing Manufacturing Sources and Material Shortages (DMSMS), obsolescence, critical items) for the CDR design?

44. Has the maintenance plan been developed addressing both Organizational and Depot levels of maintenance including fielding, safety, Human System Interface (HSI) and disposal?

45. What logistics support analysis studies were completed to support the approach in the Integrated

Logistics Support (ILS) Plan?

46. Describe the use of BIT, diagnostics, and prognostics in the maintenance concept for all levels of maintenance.

47. Describe how the planned common and peculiar support equipment (including test program sets) complements the BIT diagnostics/prognostics and technical manuals addressed in the ILS plan.

48. Will technical manuals, training devices, and simulators be available to support DT, OT, and operator & maintenance training?

49. Does the CDR design support on-board training capability in deployed equipment used?

50. What was the selection criteria used for the Packaging, Handling, Storage, and Transportation

(PHS&T) of the components, assemblies, subsystems, and systems in the CDR design?

51. Has the developer identified Environmental Safety and Occupational Health hazards in the CDR design and all associated mitigation strategies?

52. Describe DMSMS implementation with respect to technology road mapping, parts selection, spares, supply chain management, technical data package, production, and technology refresh.

53. What are the plans for supportability demonstration during DT and OT?

54. What plans are in place for the specific logistics tasks (e.g. training, tech manuals, support equipment, spares, PSH&T, RAM) to be completed during the fabrication, coding, integration, testing, and production demonstration?

55. Have the results of any System Design and Operational Effectiveness trade studies been incorporated in the CDR design?

56. How will the risk management process be applied to the evaluation of DT and OT test deficiencies?

57. Is there full traceability from PIDS, Rev G, to developmental or COTS/Non-Developmental Items

(NDI) and Software Items (SIs) to design specifications, interfaces, design documentation, source code/drawings, and test documentation?

58. Are the requirements tracked, traced, and modeled using an automated tool?

59. Describe how requirements traceability to the PIDS, Rev G, is managed for DT and OT testing.

60. For the overall system and each CI, how do you plan to test Reliability, Availability, and

Maintainability (RAM) requirements during DT and OT?

61. For the overall system and each CI, have you scheduled special test facilities to accommodate the environmental effects testing?

62. For the overall system and each CI, are key supplies in place for fabrication, coding, integration, test, and production demonstration?

63. For the overall system and each CI, have there been any changes to the CDR design since the initial hazard analysis has been completed?

64. Does the DT plan cover all the testing requirements needed to verify the CDR design (e.g. test, analysis, simulation, inspection, demonstration, and does the does the Integrated Test Team concur?

65. Are system life cycle cost estimates current and consistent with the CDR design, production, and support approaches?

66. How are adaptability, extensibility, flexibly, open systems design etc. incorporated in the CDR design?

67. Does the Contractor’s task and skills analysis reflect the current state of the CDR design?

68. Does the Contractor’s task and skills analysis conform to the sponsor's manpower constraints?

69. How are information assurance (IA) and system vulnerability addressed in the CDR design?

70. Are software development and life cycle support documentation on the IDE?

71. Has a software configuration management plan been updated to address changes in design, coding, integration, and test?

72. How are the deficiency reporting/correction and software configuration management systems integrated.

73. How will Configuration Management (CM) and Quality Assurance (QA) work together to insure correct versioning during coding, integration, and test.

74. Have facility requirements been established for the sustainment equipment destined for the

Centralized Integration Support Facility (CISF)? Have facility requirements been established for training courses?

75. Describe how COTS, GOTS, and NDI software sustainment will be conducted if the original developer support is withdrawn due to version releases or abandonment of the product (Software

DSMS)?

76. Is a current program risk assessment available?

77. Has the current risk assessment addressed future risks to developmental test, operational test, training, and production / fielding of the system?

78. Have supportability and logistics risk items been defined, analyzed, and included in the Program

Risk Assessment?

79. Present Contractor’s review of the obsolete parts list that was provided by the Government, and present contractor’s updates to this listing.

80. Identify all the makes/models and quantities of parts that will be installed as part of the upgrade

81. Present contractor efforts to ensure that selected parts will be sustainable for 5 years; present lifecycle data from manufacturers and any documentation concerning sustainability.

82. Present information confirming that subcomponents replaced for obsolescence are form, fit, function replaceable so they can be used at all UEWR locations.

83. Present weight and thermal analysis of all upgrade installation parts – show that in-plant lab spaces, CISF and Clear/Cape Cod can provide adequate Heating, Ventilation, and Air

Conditioning (HVAC), power, etc., for equipment to be installed. Show that all racked equipment includes sufficient ventilation to ensure HW operating temperatures are not exceeded.

84. Present all software design information updates associated with the SP/DP rehosting and the

Clear and VCN (synchronization) software updates..

85. Present all facilities changes that will be required to be implemented by the Government at the

CISF.

86. Present all facilities changes that will be implemented at Beale/Clear/Cod.

87. Present updates to all key performance parameters. Present updated status for any PIDS (Rev G), SOW, ICD and IA requirements that could be characterized as at risk, or with minimal margin.

Present summary information for any ECPs, changes, deviations or waivers that have been processed.

88. Present an overview of contractor tracked risks and associated mitigation plans/actions.

89. Present an overview of program hazards and associated mitigation plans/actions.

90. Present a summary of the testing that is planned in-plant, at CISF and at sites.

91. Describe the specific design decisions that were driven by the need to be backwards compatible to Fylingdales.

92. Record all CDR Action Items and submit the AIs with CDR minutes.

93. CDR AIs should be closed via contractual letter documentation. CDR is considered complete with Government concurrence of closure of the CDR AIs.

94. Present reliability, maintainability, testability information showing how the contractor’s design will meet system requirements.

III. CDR Documentation:

Numerous CDRL items should be submitted in either initial or Government-approved final form prior to a successful CDR. A number of ADPs should be available to the Government prior to a successful CDR.

The following documents are associated with conducting and completing a successful CDR. Most due dates are taken directly from the CDRL and ADP listing. Document formats and content should be per the SOW, CDRL listing and 1423-1 forms.

CDRL Title Status of document

A006 Integrated Master Schedule (IMS) The IMS is Government-approved for the CDRL submission received just prior to CDR

A007 Program Protection Management Plan (PPMP) The PPMP is government-approved

(only applicable if the Government’s

PPP is final more than 3 months prior to CDR)

A008 UEWR Logistics Support Plan (LSP) and

Supportability Analysis Plan (SAP)

The LSP and SAP are Government approved.

A010 Engineering Change Proposals (ECPs) There should be no pending ECPs at the start of CDR. Any ECPs shall have been processed to closure prior to start of CDR.

A011 Request for Deviations and Waivers There should be no pending

Deviations and Waivers at the start of CDR.

A020 Technical Publication Plan The Technical Publication Plan is

Government-approved.

A027 System Requirements/ Development Review

(SRDR)

The SR/DR shall have been completed with Government approval, and all associated Action

Items closed with Government concurrence.

A028 Preliminary Design Review (PDR) The PDR shall have been completed with Government approval, and all associated Action Items closed with

Government concurrence.

A029 Critical Design Review (CDR) Agenda is due 14 days prior to the

CDR. The CDR materials are due 5 days prior to the CDR.

A030 Logistics Data Product Government-approved Logistics

Data Product 30 days prior to CDR.

A031 Mission Assurance Implementation Plan A Government-approved MAIP exists.

A032 Safety Assessment Report A Government-approved Safety

Assessment Report exists.

A034 Information Assurance Implementation Plan Government-approved Information

Assurance Implementation Plan 30 days prior to CDR.

A036 Risk Management Plan A Government-approved Risk

Management Plan exists.

A038 Small Business Utilization and Participation Data Government-approved Small

Business Utilization and

Participation Data exists for the prior reporting period.

A039 DIACAP Executive Package The contractor shall submit an update ahead of the CDR.

A040 Status of Government Property Report A Government-approved

Government Property Report exists.

A041 Government Property Physical Inventory A Government-approved

Government Property Physical

Inventory exists.

A043 Software Design Description (SDD) for MAS, STS, SPS & REX

Initial submission due 60 days prior to CDR.

A044 Software Product Specification (SPS) for MAS, STS, SPS & REX

Government-approved SPSs 30 days prior to CDR.

A045 Software Requirements Specification (SRS) for

MAS, STS, SPS & REX

Initial submission of SRSs 60 days prior to CDR.

The following ADPs should have been produced and should be available to the Government prior to the start of CDR:

ADP 1 Identification of Required Support Equipment List

ADP 3 System Engineering Management Plan (SEMP)

ADP 9 UEWR Training Plan

ADP 16 Software Development Plan (SDP)

ADP 22 Quality Assurance Plan

ADP 23 Reliability, Availability, Maintainability And Testability (RAM&T) Program Plan

ADP 25 Foreign Object Elimination (FOE) Program Plan

ADP 28 Human Systems Integration Program Plan

ADP29

ICD UEWR Integrated Tactical Warning / Attack Assessment and Space Surveillance

Network Users

ADP 30 Reliability Design Criteria

ADP 31 Maintainability Design Criteria

ADP 32 Electromagnetic Compatibility (EMC) Control Plan

ADP 33 Dynamic Object Oriented Requirements System (DOORS) database for delivery

ADP 34 Human System Integration Design Assessment Report

ADP 35 Health Hazard Assessment Report

ADP 37 Product Parts Plan

ADP 38 Failure Reporting, Analysis and Corrective Action System (FRACAS) data summaries

ADP 39 Maintainability Math Models, Allocations and Predictions

ADP 40 Reliability Math Models, Allocations and Predictions

ADP 41 Joint Master Schedule (JMS)

ADP42 Specialty Engineering Management Plan (SpEMP)

ADP 46 Technical Performance Measures (TPM)

ADP 47 Systems Safety Program Plan

ADP 51 Contractor Furnished Equipment

ADP 54 Integrated Support Plan

ADP 56 Hazardous Material Management Program (HMMP)

ADP 57 Contractor Quality Control (CQC) Plan

ADP 60 Requirements Verification Traceability Matrix

ADP 61 STIG List

ADP 62 Contractor Configuration Management Plan

ADP 63 Parts Management Process (PMP) Plan