AFFTC-TIH-93-01 _Air_Force_Flight_Test_Center_Test_Plan_Preparation_Guide.pdf

A F F T C

AIR FORCE FLIGHT TEST CENTER

EDWARDS AIR FORCE BASE, CALIFORNIA

AIR FORCE MATERIEL COMMAND

UNITED STATES AIR FORCE

Approved for public release; distribution is unlimited.

MAY 1994

REVISED FEBRUARY 1999

TECHNICAL INFORMATION HANDBOOK

AIR FORCE FLIGHT TEST CENTER

TEST PLAN

PREPARATION GUIDE

AFFTC-TIH-93-01

This handbook (AFFTC-TIH-93-01, Air Force Flight Test Center Test Plan Preparation Guide) was prepared by personnel of the 412th Test Wing, Edwards AFB, California 93524-6843.

This handbook has been reviewed and is approved for publication:

ROGER C. CRANE

Senior Technical Advisor 412th Test Wing

GARALD K. ROBINSON

Colonel, USAF Commander, 412th Test Wing

RICHARD V. REYNOLDS

Brigadier General, USAF Commander, AFFTC

REPORT DOCUMENTATION PAGE Form Approved OMB No. 0704-0188

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188), Washington, DC 20503.

1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE

February 1999

3. REPORT TYPE AND DATES COVERED

Final

TITLE AND SUBTITLE

Air Force Flight Test Center Test Plan Preparation Guide

5. FUNDING NUMBERS

JON:

6. AUTHOR(S) PEC:

PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES

AFFTC

412 TW/CA

Edwards AFB CA 93524-6843

8. PERFORMING ORGANIZATION

REPORT NUMBER

AFFTC-TIH-93-01

SPONSORING / MONITORING AGENCY NAME(S) AND ADDRESS(ES)

N/A

10. SPONSORING / MONITORING

AGENCY REPORT NUMBER

N/A

11. SUPPLEMENTARY NOTES

12a. DISTRIBUTION / AVAILABILITY STATEMENT

Approved for public release; distribution is unlimited.

12b. DISTRIBUTION CODE

A

13. ABSTRACT (Maximum 200 words)

The Air Force Flight Test Center (AFFTC) Test Plan Preparation Guide provides the established guidelines to assist test plan authors in preparing accurate and clearly written AFFTC test plans. The objectives of this handbook are to: provide background information that assists a test planner with the test planning process; serve as a checklist for generally applicable test plan requirements; provide a standard, generalized format that is easily tailored to specific program needs; and to facilitate the development of completed test plans that readily carryover into AFFTC test reporting. The AFFTC test plan, resulting from using this guide, is a document containing all the information necessary for testing. The purpose of this guide is to provide guidelines for test plan preparation by anticipating and answering questions most frequently asked by authors regarding content, format, and style.

The format of this guide allows the test plan author the flexibility to more efficiently tailor the test plan to various test scopes (in terms of size) as well as to particular types of systems. It is written with avionics test plans in mind, but can be adapted to test plans of any discipline.

14. SUBJECT TERMS

technical writing writers guide system acquisition test plan test and evaluation test procedures

15. NUMBER OF PAGES

writing methods test planning test information sheets writing style

16. PRICE CODE

17. SECURITY CLASSIFICATION

OF REPORT

UNCLASSIFIED

18. SECURITY CLASSIFICATION

OF THIS PAGE

UNCLASSIFIED

19. SECURITY CLASSIFICATION

OF ABSTRACT

UNCLASSIFIED

20. LIMITATION OF ABSTRACT

SAR

NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89)

Prescribed by ANSI Std. Z39-18 298-102

This page intentionally left blank.

iii

PREFACE

This handbook provides the established guidelines to assist test plan authors in preparing accurate and clearly written test plans. The Air Force Flight Test Center Test Plan Preparation Guide format allows the test plan author the flexibility to more efficiently tailor the test plan to various test scopes (in terms of size) as well as to particular types of systems. It is written with avionics test plans in mind but can be adapted to test plans of any discipline.

The objectives of this handbook are to:

1. Provide background information that assists a test planner with the test planning process.

2. Provide a checklist for generally applicable test plan requirements.

3. Provide a standard, generalized format that is easily tailored to specific test program needs.

4. Facilitate the transfer of text from the test plan into the final test report.

The January 1999 revision is a product of various contributors from the 412 Test Wing and the Computer Sciences Corporation technical editors.

iv v

EXECUTIVE SUMMARY

The Air Force Flight Test Center Test Plan Preparation Guide provides established guidelines to assist test plan authors in preparing technically accurate and clearly written test plans. The guide format allows the test plan author the flexibility to tailor the test plan for various reasons such as project size and type of system under test.

This document, although intended as a guide for preparing detailed test plans, is applicable to both Integrated Test Plans and Detailed Test Plans, as defined by AFI 99-101, Developmental Test and Evaluation (Reference 1).

The primary goal of any test project is to evaluate selected aspects of system performance for a customer such as a Major Command, System Program Office, or other Government agency. The customer requires these evaluations to support decisions concerning system development, acquisition, and operational use. The end product of a test project is a technical report that is provided to the customer to support the decision making process.

The objectives of this handbook are to:

1. Provide background information which assists a test planner with the test planning process.

2. Provide a checklist for generally applicable test plan requirements.

3. Provide a standard, generalized format that is easily tailored to specific test program needs.

4. Facilitate the transfer of text from the test plan into the final test report.

The Test Plan Preparation Guide is organized into six sections; Introduction, Elements of the Test Plan, References, Bibliography, Master List of Abbreviations, Acronyms, and Symbols, and an Index. There are 14 appendices. Appendix A details the requirements for marking the various elements of a classified test plan.

Appendix B describes how to plan, write, revise, get approval, and prepare the test plan. Appendix C is a checklist to ensure that key information is not left out of the test plan. Appendix D is a checklist to ensure environmental issues have been addressed. Appendices E through N provide references with specific guidance for various test disciplines. A Sample Test Plan is provided so the reader can see how all the elements fit together.

vi vii

TABLE OF CONTENTS

Page No.

PREFACE............................................................................................................................................ iii

EXECUTIVE SUMMARY................................................................................................................... v

LIST OF ILLUSTRATIONS ................................................................................................................ ix

LIST OF TABLES ............................................................................................................................... ix

INTRODUCTION

General Use of the Test Plan Preparation Guide Background Integrated Test Plan versus Detailed Test Plan Key Concepts for Successful Testing

ELEMENTS OF THE TEST PLAN

Front Matter

Outside Front Cover Inside Front Cover Qualified Requesters Statement Preface Executive Summary Table of Contents List of Illustrations and List of Tables

Body of the Test Plan

1.0 INTRODUCTION

1.1 General

1.2 Background

1.3 Test Item Description

1.4 Overall Test Objective

1.5 Limitations

1.6 Test Resources

1.7 Safety Requirements

1.8 Security Requirements

1.9 Test Project Management

1.10 Test Environment

1.11 Environmental Impact Assessment

2.0 TEST AND EVALUATION

2.1 General

2.2 General Test Objectives

2.3 Specific Test Objectives

3.0 TEST PROCEDURES

3.1 Pretest Briefing/Test Readiness Review

3.2 Test Execution

3.3 Post-Test Briefing

4.0 TEST REPORTING

4.1 Deficiency Report

4.2 Progress Report

4.3 Test and Evaluation Results Sheet

4.4 Preliminary Report of Results

4.5 Technical Letter Report

4.6 Technical Report

viii

TABLE OF CONTENTS (Concluded)

Page No.

5.0 LOGISTICS

5.1 General

BACKUP MATERIAL

References Appendices

Test Conditions Matrix Appendix Requirements Traceability Appendix Parameter List Appendix Data Analysis Plan Appendix Instrumentation Plan Appendix Logistics Support Plan Appendix List of Abbreviations, Acronyms, and Symbols Distribution List

REFERENCES

BIBLIOGRAPHY

APPENDIX A - CLASSIFICATION REQUIREMENTS FOR TEST PLANS

APPENDIX B - PROCEDURES

APPENDIX C - TEST PLAN CHECKLIST

APPENDIX D - ENVIRONMENTAL CHECKLIST

APPENDIX E - AIRCRAFT PERFORMANCE INCLUDING AIR DATA

APPENDIX F - STABILITY AND CONTROL, FLIGHT CONTROLS,

AND FLYING QUALITIES

APPENDIX G - HUMAN FACTORS

APPENDIX H - RELIABILITY AND MAINTAINABILITY

APPENDIX I - LOGISTICS

APPENDIX J - ARMAMENT

APPENDIX K - AVIONICS

APPENDIX L - ELECTRONIC WARFARE

APPENDIX M - SUBSYTEMS

APPENDIX N - AERIAL DELIVERY

MASTER LIST OF ABBREVIATIONS, ACRONYMS, AND SYMBOLS

INDEX

ATTACHMENT 1 - SAMPLE TEST PLAN

TEST PLAN PREPARATION GUIDE RECOMMENDED CHANGE FORM

ix

LIST OF ILLUSTRATIONS

Figure No. Title Page No.

1 Test Process Elements

2 Test Objective and Measure of Performance (MOP) Input

LIST OF TABLES

Table No. Title Page No.

1 Verb Definitions

2 Parameter List

3 Example Measure of Performance Cross-Reference Table x

INTRODUCTION

GENERAL

The purpose of this document is to provide guidelines for preparing detailed test plans by anticipating and answering the questions most frequently asked by authors regarding content, format, and style. The format allows the test plan author the flexibility to more efficiently tailor the test plan to various test scopes (in terms of size) as well as to particular types of systems. This document, although intended as a guide for preparing detailed test plans, is applicable to both Integrated Test Plans (formerly System Test Plan) and Detailed Test Plans, as defined by AFI 99-101, Developmental Test and Evaluation (Reference 1).

The primary goal of any test project is to evaluate selected aspects of system performance for a customer such as a Major Command, System Program Office, or other Government agency. The customer requires these evaluations to support decisions concerning system development, acquisition, and operational use. The end product of a test project is some form of technical report that is provided to the customer to support the decision making process.

See Air Force Flight Test Center Instruction (AFFTCI) 99-1, Test Plans (Reference 2), and AFFTCI 91-5, Test and Safety Review Process (Reference 3), for the details of the test plan review and approval process.

USE OF THE TEST PLAN PREPARATION GUIDE

This handbook is organized into 6 sections; Introduction, Elements of the Test Plan, References, Bibliography, Master List of Abbreviations, Acronyms, and Symbols, and an Index; 14 appendices; Appendix A - Classification Requirements for Test Plans, Appendix B - Procedures, Appendix C - Test Plan Checklist, Appendix D - Environmental Checklist, a set of ‘open ended’ appendices (Appendices E through N) for specific guidance for various test disciplines to be added at a later date when and if appropriate; and 1 attachment: a Sample Test Plan (Attachment 1). The Introduction section provides an overview of the Government acquisition and development process, test process elements, and key requirements for successful testing. The Elements of the Test Plan section describes each test plan element in detail including which elements should be transferred, for the most part, to the final test report. The References section contains technical writing references and pertinent regulations. The Bibliography section provides useful test-related handbooks. The Master List of Abbreviations, Acronyms, and Symbols contains the abbreviations used throughout the handbook and most abbreviations used at the Air Force Flight Test Center. It also lists common abbreviations that do not have to be defined when first used. The Index section eases the search for desired information in the handbook. Appendix A details the requirements for marking the various elements of a classified test plan. Appendix B describes how to plan, write, revise, get approval, and prepare the test plan.

Appendix C is a checklist to ensure that key information is not left out of the test plan. Appendix D is a checklist to ensure environmental issues have been addressed. Appendices E through N provide references for various test disciplines. A Sample Test Plan is provided so the reader can see how all the elements fit together to provide the link from the customer requirements to measurable system performance parameters, to how and when the parameters are measured, to final data analysis and data product production.

BACKGROUND

Government military acquisition and development programs are usually based on a top-down process designed to ensure that final products meet customer needs. The process begins with an approved operational need followed by the application of various analytical methods to determine initial system performance and design goals. The analytical methods used, such as modeling and simulation, are based on estimates and statistics. As a result, analytical methods alone cannot adequately characterize system performance.

Developmental testing is required to reduce technical risk and to ensure that performance and design goals are being met.

The developmental test process is used to efficiently measure system performance parameters to determine if performance and design goals are being met. Developmental test and evaluation (DT&E) programs provide an initial assessment of military utility and effectiveness by measuring system performance against performance and design goals. Each of the elements shown in Figure 1 are required for a successful test and evaluation program but the type of test being conducted can have a significant effect on how the test elements are applied. For example, the reportable issue for the demonstration/validation test of an experimental radar system may be to show a capability to track more than three targets simultaneously. During full scale engineering development of the same radar system, a reportable issue may be to measure tracking accuracy while the radar tracks three targets simultaneously.

* Determine Reportable Issues (Critical Questions)

- Define Test Report Elements

* Define Specific Test Objectives

- Relate Performance and Design Goals to Reportable Issues

* Establish Associated:

- Measures of Performance

- Evaluation and Success Criteria

- Data Requirements

- Test Resource Requirements

* Prepare a Test Plan to Include:

- Data Analysis and Logistics Support Plans

- Instrumentation Plan

- Parameters List

- Test Matrix and Schedule

* Conduct Test

* Produce Data Products, Analyze Data, and Evaluate Results

* Produce and Publish Final Technical Report

Figure 1 Test Process Elements

Some of the common terms used within the acquisition community have different meanings depending on how the term is used. Acquisition terms are defined in many official documents but one source of information for the test planner is the Defense Acquisition Deskbook (Reference 4). The deskbook identifies acquisition management core processes, the associated terms, and includes many DoD and Air Force documents for reference. The list of terms below were selected because they are important to test planners. All of the definitions came from official documents.

1. Measure of Effectiveness (MOE): A qualitative or quantitative measure of a system's performance or a characteristic that indicates the degree to which it performs the task or meets a requirement under specified conditions. Where possible, MOEs should be defined to measure operational capabilities in terms of engagement or battle outcomes.

2. Critical Operational Issue (COI): A key question that must be examined in operational test and evaluation to determine the system's capability to perform its mission. Testers normally phrase a COI as a question to be answered in evaluating a system's operational effectiveness or suitability.

3. Measures of Performance (MOPs): A quantitative measure of the lowest level of physical performance (e.g., range, velocity, payload).

4. Participating Test Organization (PTO): A test organization required to afford specific resources during DT&E.

5. Requirement: The validated need of an operational user. Initially expressed in broad operational capability terms in the format of a mission need statement (MNS). It progressively evolves to system-specific performance requirements in the operational requirements document (ORD).

6. Responsible Test Organization (RTO): The lead Government entity that is qualified and responsible for DT&E.

7. Test Information Sheet (TIS): (From AFFTCI 99-1 [Reference 2]) Also known as a detailed test plan (DTP), a TIS will be used to define and amplify specific tests identified in a system test plan and will either stand alone or be part of an overall plan that is general in nature. For AFFTC TISs, an AFFTC Form 5232b, AFFTC Test Information Sheet (with signature blocks), will be used as the cover sheet for stand-alone plans and AFFTC Form 5232a, AFFTC Test Information Sheet, for TISs that are included in overall plans.

The AFFTC Form 5232b will also be used as a cover sheet for contractor-prepared DTPs.

8. Test Objective: The specific performance or technical parameters to be measured during the test to evaluate system performance, system operational effectiveness, or system suitability.

9. Test Requirements: Test requirements are formulated from, but not limited to: MNS, ORD, system specification(s), previous test results, Test and Evaluation Master Plan (TEMP), overall objective and general test objectives, Single Face to the Customer (SFTC) inputs, and previous test experience.

INTEGRATED TEST PLAN VERSUS DETAILED TEST PLAN

The integrated test plan (ITP) is a program management tool that ensures all appropriate test activity is included, duplicative testing is not being done, risks are identified, and all participants understand the total test effort. The ITP is sometimes referred to as a system test plan.

The ITP is written early in the program’s schedule and changes as more detailed planning is accomplished. The ITP is normally written by the program office with a lot of help from the RTO, PTOs, and support groups. Since the ITP is written early in the program’s schedule, prior to the accomplishment of the detailed planning for each test, some sections will necessarily be general in nature. However, all sections should specify the general approach that will be followed in the test effort. The DTPs will contain all the detailed information that the ITP could not contain.

Per AFI 99-101 (Reference 1), the ITP:

1. Describes the total test management effort,

2. Describes the most efficient use of all test resources, and

3. Records all DTPs for all project participants (contractors and government).

An ITP may or may not be required depending on the scope of the test program but all test programs must be documented in a DTP. The DTP(s) contain(s) all the information necessary for the test team to conduct the actual tests. The DTPs go by a variety of names depending on what organization wrote the plan. Most common names are: test information sheet (TIS), detailed test information sheet (DTIS), and flight test work order (FTWO). Most DTPs also have a particular cover sheet unique to the organization that contains basic administrative information such as authors names, proposed test dates, RTO, financial identification numbers, type of aircraft, etc.

A DTP brings together common tests, objectives, and/or measures of performance (MOP) that can be accomplished during a single test condition, test flight, or some subset of a test project. It is used at the working level on a daily basis for such things as writing tests cards, briefing missions, and ensuring all test resources are in place on the actual day of the test. The DTPs are the key documents that describe each unique test setup and the detailed, step-by-step, test procedures. The DTP should contain sufficient information for use by a test engineer to develop test cards and for management to discern the overall technical approach being taken. How detailed each DTP section is depends on the complexity of the system, and the complexity of the tests. The level of detail depends on what the test team and senior managers need to safely accomplish the test. For example a tower-fly-by airspeed calibration on a T-38 aircraft requires relatively little detail since it is a simple test, on a well-known aircraft, using a well-known test method. On the other hand an engine water-ingestion test on a one-of-a-kind prototype aircraft with a new engine will require relatively more detail. The DTPs are also used to track test point completion. The DTP is usually prepared by the test engineer and used by the mission planner and test conductor.

All test plans are management tools. As such, they should be tailored to the specific program. For example: major DT&E programs like the C-17 or B-2 will have a large ITP and numerous DTPs. For major DT&E programs, putting all the detailed information that is in the DTPs into the ITP would create an encyclopedic document that would be unwieldy to use. Therefore, the ITP sets out all the general philosophies and approaches; the DTPs, of which there may be several hundred, set out all the details for each test point.

The DTPs may be attached to the ITP as appendices. At the other extreme, a one flight program to confirm the structural integrity of a new, small radome on a RC-135 aircraft will not have an ITP; however, it will have a single DTP. The difference between the two extremes is in the level of detail and volume of the documents.

Each program should decide for itself what combination of test plans provides the best management tools for its purposes. However, all projects will use a DTP as the primary test plan document for the day-to-day planning, setup, and conduct of the test project.

This guide and the sample test plan (Attachment 1) suggests the same format for both the ITP and DTPs.

The difference between the two will be in the level of detail contained in any section. Parallel formats will make it easy for the test team to correlate the two documents.

KEY CONCEPTS FOR SUCCESSFUL TESTING

There are several important concepts that can help lead to a successful test and all are equally important.

They are the following:

1. Early Involvement of Test Personnel: Test engineers, including all test support disciplines, should become involved in the system development program during the earliest stages. Early involvement can save time, money, and resources. A test planning working group (TPWG) should be setup with members representing the program office, the user, the contractor, the RTO, PTOs, and AFOTEC. Having the first TPWG meeting during the contract proposal review is not too early. Early involvement is recommended to ensure that test-unique requirements can be addressed and long-lead test resources can be acquired.

Test unique requirements may include modifications to the system for test support. The cost to incorporate test-unique data ports and sensors during design is usually insignificant. Modifying systems after they are built is always more expensive and in some cases may be out of scope for the program. Users and contractors almost always, and rightly so, propose and design systems to satisfy operational requirements. Significant lead times may be required to provide test resources such as aircraft modifications, special instrumentation, and data reduction and analysis software. Early involvement of test engineers can help ensure that items such as digital data ports, strain gauges, temperature sensors, and so on are incorporated into the basic system design so testing will be economical, efficient, and productive.

2. Test the System: All system functions defined in the system performance and design goals should be accounted for equally in the test objectives and associated measures of performance. Those functions or subsystems which have already been successfully demonstrated should undergo some level of regression testing to verify that they still operate properly after the new hardware or software is integrated into the system. During test design, the test engineer should perform a 'cold blooded' assessment of system performance and design goals while determining the primary test objectives without taking into account what has and has not been done before. Those functions dependent on advances in technology will, naturally, require more indepth testing than that required to verify functions which have been previously demonstrated.

3. Test Engineer Knowledge: The test engineer should understand the technical details of how and why the system really works. This understanding is essential so that the test engineer can correctly design the test and specify the system instrumentation requirements, such as digital data ports, strain gauges, temperature sensors, and so on. Planning and coordination with the customer and contractor(s) may be necessary to provide the time and resources to satisfy this requirement.

4. Traceability: The physical measurements made during a test must track directly to, and provide answers for, the customer's questions concerning system performance and design goals. Often this is the most difficult challenge the test engineer must face during the design of a test. The customer usually wants answers to questions that relate to operational utility or military worth. Operational utility and military worth are quantities that are elusive and difficult to measure directly. An example of a customer test goal might be, will the system under test out perform enemy systems? The test engineer must first determine what measurable parameters determine the performance of a system of the type under test, then test the system to quantify these performance parameters, and finally compare the test results to the accepted intelligence estimates for the enemy system parameters. Modeling may be required to relate physical measurements to operational utility or military worth.

5. Early Report Definition: The author should begin writing the test plan only after defining the elements of the technical report. Having a clear definition of what the end product will be helps the planner define the questions to be answered during the test, define the data acquisition effort, characterize the data analysis process, and enhance the probability of success. A good test plan, with clearly-stated and well-defined elements, will provide the foundation for the data acquisition and analysis required to generate the customer's technical report. In addition, some elements of the test plan and technical report, often referred to as boiler plate, are almost identical and should be written with the technical report in mind.

ELEMENTS OF THE TEST PLAN

Test plans are made up of three major components: Front Matter, Body of the Test Plan, and Backup Material. Each of these components contains certain elements. Elements can be added or deleted to more efficiently tailor the test plan to your specific test program scope (in terms of size) as well as your particular type of system. The following is the recommended order. Those elements with an asterisk (*) can be transferred, with minor changes, to the technical report. Examples of most elements can be found in the sample test plan.

FRONT MATTER

Outside Front Cover (*) Inside Front Cover (*) Qualified Requesters Statement (*) Preface (*) Executive Summary (*) Table of Contents (required if the test plan contains more than eight pages) List of Illustrations (required if the test plan contains more than five figures) List of Tables (required if the test plan contains more than five tables)

BODY OF THE TEST PLAN

1.0 Introduction

1.1 General (*)

1.2 Background (*)

1.3 Test Item Description (*)

1.4 Overall Test Objective (*)

1.5 Limitations

1.6 Test Resources (*)

1.6.1 Modeling and Simulation (*)

1.6.2 Test Facilities (*)

1.6.3 Frequency Authorization

1.6.4 Test Aircraft (*)

1.6.5 Support Vehicle (*)

1.6.6 Test Range (*)

1.6.7 Instrumentation Requirements (*)

1.6.8 Test Support Facilities and Equipment (*)

1.7 Safety Requirements

1.8 Security Requirements

1.8.1 General Security

1.8.2 Operations Security

1.8.3 Communications Security

1.8.4 Competition Sensitivity

1.9 Test Project Management

1.10 Test Environment

1.11 Environmental Impact Assessment

2.0 Test and Evaluation

2.1 General (*)

2.2 General Test Objectives (*)

2.3 Specific Test Objectives (*)

2.3.1 Test Objective 1 (*)

2.3.1.1 Measure of Performance 1 (*)

2.3.1.1.1 Success Criteria

2.3.1.1.2 Evaluation Criteria

2.3.1.1.3 Exceptional Case to MOP

2.3.1.1.4 Evaluation Terminology

2.3.1.1.5 Final Data Products

2.3.1.1.6 Data Requirements

2.3.1.1.7 Algorithms Processes

2.3.1.1.8 Test Methodology (*)

2.3.1.1.9 Expected Test Results

3.0 Test Procedures

3.1 Pretest Briefing/Test Readiness Review

3.2 Test Execution

3.3 Post-Test Briefing

4.0 Test Reporting

4.1 Deficiency Report

4.2 Progress Report

4.3 Test and Evaluation Results Sheet

4.4 Preliminary Report of Results

4.5 Technical Letter Report

4.6 Technical Report

5.0 Logistics

5.1 General

BACKUP MATERIAL

References Appendices

Test Condition Matrix Requirements Traceability Parameter List Data Analysis Plan Instrumentation Plan Logistics Support Plan

List of Abbreviations, Acronyms, and Symbols (*) Distribution List (*)

FRONT MATTER

OUTSIDE FRONT COVER

The outside front cover is the interface between the test plan and the reader. The test plan is an official U.S. Government publication and should reflect the professionalism of the AFFTC and the USAF. Cartoons or unofficial logos are inappropriate.

The outside front cover contains the following elements:

1. Title: The title should be brief and clearly describe the contents of the test plan. In most cases, fill in the program name. Use acronyms only if they are more commonly known or used.

2. Names: The author should be listed here, where applicable. No more than two names should be listed unless specific circumstances warrant it. Other contributors to the test plan should be acknowledged in the Preface.

3. Test Plan Date: The test plan date consists of the month and year in which the test plan received final review and approval by the Test Wing Commander.

4. Distribution Statement: Almost all test plans use Distribution Statement C (Test and Evaluation).

The date of the Distribution Statement is the date the Controlling Authority approved the distribution list and the distribution statement. The Controlling Authority will vary, but is normally the System Program Office.

INSIDE FRONT COVER

The main purpose of the inside front cover is to document who wrote the test plan and who approved its publication and release. The list of authors should be limited to those individuals that wrote a significant portion of the test plan. Editorial comments do not constitute authorship. Usually, only one or two authors should be listed.

The following guidelines should be used when writing this test plan element:

1. Identify the originating office and its affiliation with the AFFTC.

2. The signature blocks required are those of the authors and the approval authorities. See AFFTCI 99-1 (Reference 2), and AFFTCI 91-5 (Reference 3), for the details of the test plan review and approval process.

3. Include program authorization(s) and date(s), e.g., program element code (PEC) and job order number (JON).

QUALIFIED REQUESTERS STATEMENT

This page is required in all test plans which are not cleared for public release. The qualified requester's page tells the reader where to obtain additional copies of the test plan and destruction instructions for the document. The format and wording for this statement is governed by DoD Regulation 5230.25-PH, Control of Unclassified Technical Data with Military or Space Application (Reference 5).

PREFACE

The Preface is where the author explains how this particular test plan fits into the overall test program, how it relates to previous test efforts, and its relationship to other test plans. The program authority should be recognized here; usually by stating the Program Management Directive number. It is also the section where the author can acknowledge the contributions of people other than the principal authors. The Preface should be very short and definitely not exceed one page.

EXECUTIVE SUMMARY

Because it is designed to be read by upper management, the Executive Summary is very important. The Executive Summary should be a stand-alone document. It may be the only section that executive-level personnel will read. It should be easy to read and relate only essential information. A narrative style is best, but should be brief and to the point. THE EXECUTIVE SUMMARY SHOULD NOT EXCEED ONE PAGE. This section is a summary of the test plan and will have condensed information from the Preface, Introduction, Test and Evaluation (T&E), Test Procedures, Test Reporting, and Logistics sections. Minimize the use of numbers and redefine units and acronyms. Individual paragraph headings are inappropriate. Discuss only the most important facts; these need not be in the same order or format as found in the T&E section. DO NOT INTRODUCE ANY NEW MATERIAL. Impact is a key concern in this section, so give the facts to the reader quickly and concisely. If the reader wants details, he must read on.

For an ITP the Executive Summary should very briefly address the following items, but in a narrative style with no titles or headings. For a DTP some of this material is on the front cover sheet and does not have to be repeated. For a DTP some of this information can be omitted. For example; the technical and safety reviews will necessarily have been conducted before the DTP is submitted for approval. Other documents in the 'safety package' cover the technical and safety review issues.

1. Overall Test Objective (one or two sentences)

a. This section should state the overall objective of the test.

2. Background (one short paragraph)

a. The document purpose (one statement).

b. The project name.

c. Who requested the test.

d. The program authority - Program Management Number and precedence rating.

e. Job order number.

f. Time frame for testing.

g. Number of tests.

h. Test locations, facilities and other test participants.

3. Test Item Description (one paragraph)

a. A description of the test article items unique to this test.

b. Program history.

c. State whether the test article is or is not production-representative for test purposes.

d. Reference detailed test item documents.

4. Test Methodology

a. State how the test(s) will be conducted.

b. State the environment the test(s) will be conducted in.

c. State how the data will be collected.

5. Technical Risk

a. Address any items that may effect the executibility of the test plan.

6. Technical Review

a. State what organizations you expect to participate.

b. When will it take place?

7. Safety Review

a. When will it take place?

b. State whether the test itself or specific aspects of the test are expected to be low, medium, or high risk from a safety standpoint.

8. Environmental Impact Issues

a. State if there are any expected environmental impacts.

b. If an environmental impact study was prepared in the past, state whether an additional one is required.

c. State the source and date of any environmental studies.

TABLE OF CONTENTS

The Table of Contents is a listing of the headings in the order in which they appear within the test plan. It is not required in test plans of less than eight pages. The order and headings must exactly match those appearing in the test plan. It is generally not necessary to list headings beyond the fourth order in the Table of Contents.

LIST OF ILLUSTRATIONS AND LIST OF TABLES

The List of Illustrations and List of Tables contains all the figures and tables, respectively, in the test plan by figure or table number, exact title, and page number. Titles should be descriptive, e.g., Angle of Arrival Summary, instead of AOA. The goal is to have a unique title for each figure or table and have it listed here exactly as it appears in the test plan. Unique titles may, in some cases, be hard to do. All test plans having more than five figures or tables must have a List of Illustrations or List of Tables, respectively.

The following guidelines should be used when writing this test plan element:

1. Ensure titles are identical to those in the test plan.

2. Use unique titles. Avoid duplicate titles except for groups of similar plots or tables.

BODY OF THE TEST PLAN

Here is where you define why and how you are going to do a test and what you expect to find out. You are attempting to explain complex issues, techniques, and procedures to a technically astute reader, not a layman.

The reader will not be an expert in your engineering discipline, but will be reasonably versed in your weapon system. REMEMBER YOUR TEST PLAN IS NOT A NOVEL, SO DON'T COMPOSE IT LIKE ONE.

Clarity is the goal. Keep sentence structure simple. Keep terminology as simple as possible and consistent with the technical aspects of your test. It never hurts to say the same thing a couple of different ways to be sure the point is made. Use terms standard within your discipline. Do not make up your own.

The body of the test plan contains all the elements which logically answer the following questions:

WHY is the test being conducted?

WHO will conduct the test?

WHO are the customers?

WHAT will be tested?

WHAT are the objectives of the test?

WHAT are the measures of performance (MOP) for each objective?

WHAT are the evaluation criteria for each MOP?

WHAT are the success criteria for each MOP?

WHAT final data products will be produced to answer each MOP?

WHAT tests will be conducted?

WHERE will the tests be conducted?

WHEN will the tests be conducted?

HOW will the tests be conducted?

WHAT data need to be acquired, reduced, and analyzed?

HOW will the data be acquired, reduced, and analyzed?

WHAT are the expected results of the tests?

WHAT test reporting will be accomplished?

WHAT logistics support is required for the test?

The following pages outline the common elements of the body of the test plan and the information that is contained in each.

1.0 INTRODUCTION

The Introduction section provides the foundation for and establishes the tone of the test plan. It should bring the reader up to the necessary level of understanding so he/she can understand the rest of the document.

One objective of the test plan introduction is to be able to place it in the technical report with minor changes.

For example, change verb tenses from future to past. All of the following elements that are applicable to the test should be included in the Introduction section. If there are unique elements that are specific to your test and not listed below, add them in the appropriate place. In addition, if some elements do not apply to your test, delete them. The reader must be able to understand your test plan without referring to the reference material.

1.1 General

The General section should include the purpose of the test plan and the following information:

a. System Program Office

b. The program authority - Program Management Directive number

c. Precedence rating

d. Job Order Number

e. Responsible test organization and responsibilities

f. Participating test organization(s) and responsibilities

g. Time frame for testing

h. Number of tests

i. Test location(s) (include ground and flight tests, and simulation tests if appropriate)

j. Test hours (flight and ground)

1.2 Background

The Background section should include at least the following information:

a. A brief description of what the system under test is (maximum of two sentences).

b. A summary of significant historical data leading up to the test.

c. A discussion of previous related tests, problems found, and significant results. How did we get here (approximately one paragraph)?

d. A list of significant phases of the test (approximately one paragraph).

1.3 Test Item Description

Describe what is going to be tested. Include a short overall description and a simplified block diagram, if appropriate, of the item to be tested. More indepth descriptions (over one page) should be included in an appendix. Include reference(s) to documents that contain detailed descriptions (e.g., flight manual, specifications, etc.). Make certain the test article is identified as production representative, proof-of-concept, etc., for purposes of the test. Identify any peculiar configurations required.

1.4 Overall Test Objective

Use one or two clear sentences to describe the overall test objective.

1.5 Limitations

Discuss limitations that apply to the test. Do not include flight manual or other published limitations that can be referenced without losing the continuity of the test plan. Items that might be considered are:

a. The System Under Test (e.g., F-16 T/N 123 has never flown straight since it came out of the factory, but it is the only F-16 available for the test. Although the trim required is small and the effect on test results is judged to be small, there is no way to quantify the differences between this aircraft and a straight jet for the purposes of these tests.)

b. Test Instrumentation (e.g., The instrumentation is only capable of recording 10 samples per second.

The frequency of measurand X is 15 Hz; therefore we will not be able to measure with great confidence the peak values of measurand X. Depending upon the level of importance of peak values in measurand X this may be a factor in the evaluation, but initial judgment indicates that stabilized values will be sufficient to perform the evaluation.)

c. Test Facilities (e.g., In order to fully evaluate the targeting system, over 500 simultaneous threats would be required. However, only 250 are available in the chamber. This is a known risk and the program office is willing to accept the test results for this limited test as indicative of the true performance of the targeting system.)

d. Associated Test Vehicles (e.g., The T-38 will be used as a target. Even though the T-38 RCS is not identical to the actual intended target, no actual targets are available. Test results must be carefully considered before making decisions as to design improvements or production decisions.)

e. Weather (e.g., Every attempt will be made to test the article in three successive MIL-STD desert diurnal cycles; however, current weather patterns indicate that the probability of achieving this is low and the test article is not available at any other time for these tests. Careful consideration must be made relative to the test results and the associated risk of fielding the system without testing in the proscribed MIL-STD conditions.)

1.6 Test Resources

List all resources required to transmit, record, or display test data. List all other resources required to conduct the mission (e.g., aircraft, ranges, hardware, software, facilities, personnel, etc.). The Test Resources section includes, but is not limited to, the following subelements:

1.6.1 Modeling and Simulation

Computer simulation consists of digital modeling of the avionic system, the host platform, other friendly players, the scenario, the combat environment, and threat systems. These models are then executed interactively in simulated time and space domains. Specific computer simulations are constructed at levels of detail which roughly correspond to the level of technical complexity they support (i.e., engineering, platform, or mission). Computer simulation and analysis is used prior to each phase of testing to help design the tests and predict test results and, after each phase of testing, to extrapolate test results to other conditions. In this way, computer simulation serves to tie the five other test facility resources together.

Briefly describe resources for accomplishing any computer and mathematical modeling required for the test. Identify responsibilities for modeling. Briefly identify responsibilities for simulator operation, maintenance, and programming. Reference documents containing detailed simulation and laboratory requirements, if appropriate. However, the reader must be able to understand your test plan without referring to the reference material. Some examples of modeling and simulations are: Digital Simulation Takeoff and Landing, TAC BRAWLER, TAC SUPPRESSOR, and Enhanced SAMS.

1.6.2 Test Facilities

Test facilities can be contractor or Government owned and are brought together under five categories:

measurement facilities, system integration laboratories (SILs), hardware-in-the-loop (HITL) facilities, installed system test facilities (ISTFs), and open-air facilities. Measurement facilities provide capabilities to explore and evaluate advanced technologies, antenna patterns, and electromagnetic (e.g., radio frequency, infrared, laser) signatures. The SILs are facilities designed to integrate aggregations of avionic hardware and software, up to an entire aircraft avionic suite, in a laboratory spread-bench configuration. The SILs are used to evaluate the operation/performance of individual components and subsystems in the context of their interactions with other avionics. The SILs often employ a variety of real-time/near real-time digital models and computer simulations to generate scenarios and electromagnetic backgrounds. These models are interfaced with brassboard, prototype, or actual production hardware of the systems under test. The SIL testing focuses on identifying hardware and software problems, maturing system performance, and evaluating projected reliability and maintainability levels. When SIL testing has been completed, engineering prototype components should be transferred to HITL facilities. The HITL test facilities are indoor laboratories which provide a secure environment to test avionic system hardware against manned, closed-loop simulations. The HITL facilities allow production systems to be tested under controlled and repeatable test conditions. The ISTFs provide a capability to evaluate avionic systems which are installed on, or integrated with, host platforms. These test facilities consist of anechoic chambers in which free-space radiation measurements are made during the simultaneous operations of all host platform avionics. The ISTFs primary purpose is to evaluate integrated avionic systems (e.g., radar, infrared, communications, navigation, identification, electronic combat or subsystems, integrated controls and displays) in installed configurations to test specific functions of complete, full-scale weapons systems. Such testing is conducted to determine if any electromagnetic interference or electromagnetic compatibility problems exist; to determine reaction to electromagnetic environments of hostile and/or friendly systems whose signals cannot be radiated in free space on open-air test ranges for security reasons; and to support flight testing by providing preflight and postflight checkout capabilities. Open-air facilities are used to evaluate the system under natural environment operating conditions. They are used to determine the effects of real-world phenomena on the system under test. Real-world phenomena encountered during open-air testing include terrain effects, multi-path propagation, and commercial electromagnetic interference (television and radio broadcasts, microwave communications, etc.). Some examples of facilities that fall under each of these categories are listed below. See the appropriate SFTC office, AFMAN 99-110, Airframe-Propulsion-Avionics Test and Evaluation Process Manual (Reference 6), and AFMAN 99-112, Electronic Warfare Test and Evaluation Process (Reference 7) for a more complete list.

a. Modeling and Simulation Facilities

1. Test and Evaluation Mission Simulator (TEMS)

b. Measurement Facilities

1. Rome Laboratory (RL)

2. Stores Weight and Inertial System Facility

c. System Integration Laboratories

1. Electronic Warfare Avionic Integration Support Facility (EWAISF)

2. Integrated Defense Avionic Lab (IDAL)

3. Integration Facility for Avionic System Testing (IFAST)

4. Missile/Munition Integration Facility

d. Hardware-in-the-Loop Facilities

1. Air Force Electronic Warfare Evaluation Simulator (AFEWES)

2. Aircraft Gun Harmonization Facility (Gun Butt)

e. Installed System Test Facilities

1. Air Combat Environment Test and Evaluation Facility (ACETEF)

2. Benefield Anechoic Facility (BAF)

3. Preflight Integration of Munitions and Electronic Systems (PRIMES)

f. Open-Air Facilities

1. Air Armament Center (AAC)

2. Air Force Flight Test Center (AFFTC)

3. Kwajalein Missile Range (KMR)

4. Multispectral Open-Air Test Environment (MORTE)

5. Naval Air Warfare Center, Weapons Division (NAWCWPNS)

6. Nellis Range Complex (NRC)

7. Utah Test and Training Range (UTTR)

8. White Sands Missile Range (WSMR)

Briefly describe each facility to be used during each phase of the test. Reference documents containing detailed facilities descriptions, if appropriate. Again, the reader must be able to understand your test plan without referring to the reference material.

1.6.3 Frequency Authorization

To use or operate any Spectrum Dependent System (i.e., equipment that radiates or receives RF energy), you must obtain frequency authorization from the Installation Spectrum Manager (ISM). To obtain frequency authorization contact the Spectrum Management Office (SMO) (95 CS/SCML) as soon as possible in the planning cycle.