Attachment_1_-_RDOI61_103.pdf

BY ORDER OF THE COMMANDER RD OPERATING INSTRUCTION 61-103

AIR FORCE RESEARCH LABORATORY

8 November 2010

COMPLIANCE WITH THIS PUBLICATION IS MANDATORY

ACCESSIBILITY: Publications and forms are available for downloading on the https://org2.eis.afmc.af.mil/sites/RDRV/IM/Forms%20and%20Publications/Forms/AllItems.asp x .

RELEASABILITY: There are no releasability restrictions on this publication.

OPR: AFRL/RDO Certified by: AFRL/RDO (Dr. Thomas Bretz)

Supersedes RDOI 61-103, 12 June 2008 Pages: 24

This Air Force Research Laboratory Directed Energy Directorate (AFRL/RD) Operating

Instruction implements AFRLI 61-103, AFRL Research Test Management. This instruction utilizes the research and development technical and safety review requirements of AFI 99-103Test and Evaluation; meets the requirements of Test Planning and System Safety programs in accordance with (IAW) AFMCI 99-103 Test Management, Test and Evaluation, and AFI 91-202

AFMC Sup1 The US Air Force Mishap Prevention Program. This instruction should be used in conjunction with AFI 91-202, AFMC Sup1, AFRLSup1 Safety. This instruction establishes procedures and responsibilities for managing test programs in AFRL/RD and defines the associated review and approval process. Ensure that all records created as a result of processes prescribed in this publication are maintained IAW AFMAN 33-363, Management of Records), and disposed of IAW the Air Force Records Disposition Schedule (RDS) located at Primary

Responsibility (OPR) using the AF IMT 847, Recommendation for Change of Publication; route

AF IMT 847 through the appropriate functional chain of command.

SUMMARY OF CHANGES

This instruction is substantially revised and must be completely reviewed.

Scientific, Research, and Development

AFRL RESEARCH TEST MANAGEMENT

https://org2.eis.afmc.af.mil/sites/RDRV/IM/Forms%20and%20Publications/Forms/AllItems.aspx https://org2.eis.afmc.af.mil/sites/RDRV/IM/Forms%20and%20Publications/Forms/AllItems.aspx

2 RDOI61-103 8 November 2010

TABLE OF CONTENTS

1. Background and Purpose

1.1. Purpose

1.2. Scope

1.3. Waivers

2. Roles and Responsibilities

2.1. AFRL/RD Director and Deputy Director

2.2. RD Chief Scientist

2.3. RD Chief Engineer

2.4. Division Chiefs

2.5. RD Test Office

2.5.1. Test Lead/Deputy Test Lead will

2.5.2. RD Range Management

2.5.3. RD Test and Environmental

2.6. Division Test Lead

2.7. Project/Program Managers, Scientists, and Engineers

2.8 Test Approval Authority (TAA)

3. Test Process

3.1. Test Process

3.2. Test Documentation

3.3. Technical Review Process

3.4. Safety Review Process

3.5. Review of Classified Tests

3.6. Technology Transition, Demonstrations or Deployments of Research Systems

4. Test Execution

4.1. Test Execution

4.2. Test Plan Changes/Revisions

4.3. Project Completion or Termination

5. Forms

5.1. Prescribed Forms

5.2. Adopted Forms

RDOI61-103 8 November 2010 3

Attachment 1 - GLOSSARY OF REFERENCES AND SUPPORTING INFORMATIO 24

4 RDOI61-103 8 November 2010

1. Overview

1.1. Background and Purpose. The purpose of this instruction is to establish roles and responsibilities and to standardize the basic process for the conduct of research test activities. It is the Director’s goal to establish a clear test process that defines test objectives and measures to provide technical sufficiency and to maintain a safe working environment for all personnel during the conduct of tests, experiments, and demonstrations. To meet this goal, all projects in the

Directorate require adequate planning and review prior to test execution, consistent with Air Force guidance. This OI addresses the steps to formalize the test planning process within AFRL/RD.

This test planning process applies to other organizations and contractors that perform tests in AF facilities or test sites as required.

1.2. Scope. This instruction applies to all AFRL/RD programs, projects, and personnel involving test, experiment, or demonstration activities at any location including those activities performed at geographically separate units such as Maui and North Oscura Peak. Those experiments, tests, or demonstrations that use RD technology and/or devices (e.g. laser, RF), high voltage sources or other sources that use one or more toxic chemical or substance to create a directed energy effect must complete the full test and safety review process. It does not apply to activities such as calibration, maintenance, simple component verification, component machining, and well-established measurement procedures documented in local Standard Operating

Procedures (SOP) approved by the Division Chief, Branch Chief, division test lead, and/or safety office. Other measurement procedures based on established, reviewed, and documented processes may be accepted at the discretion of the appropriate Test Approval Authority (TAA).

Test plans and SOP(s) are valid as long as the scope, objectives and data measures do not change.

Annual evaluation of these efforts will occur by the RD Test office. New signatures are not required unless there are changes to the scope; however, the safety permit will require new signatures IAW AFRLI 91-202. This instruction also applies to AFRL/RD activities performed in collaboration with other organizations at any installation, leased government properties, or remote locations.

1.3. Waivers. Submit waiver requests to local RD test lead (RDOT) for coordination/staffing to DET 8 Safety Office for Kirtland activities, to DET 15/RDOT for RDSM activities and AFRL Center Test Authority as necessary.

2. Roles and Responsibilities

2.1. AFRL/RD Director and Deputy Director

2.1.1. Implement AFMC and AFRL test policies within the Directorate.

2.1.2. Authorize the use of any RD system for demonstration, extended user evaluation, operational assessment, or deployment prior to system delivery or action.

2.1.3. Approve the Responsible Test Organization (RTO) for programs based on the recommendation of the Division Test Lead or Project/Program Manager (PM).

RDOI61-103 8 November 2010 5

The RTO is the lead government developmental test organization that is qualified to conduct and be responsible for overseeing developmental RD Test activities.

2.1.4. Designate the Lead Test Organization (LTO) for programs in their Directorate when working with other organizations. The LTO is the lead government entity that is qualified and responsible for the planning, safety, execution, and reporting on research and development testing. Normal routine efforts performed in-house do not require the formal designation of LTO.

2.1.5. Act as the TAA for unique or complex medium level risk activities as determined by the System Safety Manager (SSM) and the Directorate’s Test

Lead.

2.1.6. Appoint in writing a test lead for RD.

2.1.7. Recommend participants and chairperson for Technical and Safety Reviews.

2.2. RD Chief Scientist

2.2.1. Responsible for technical review process of RD test activities.

2.2.2 Act as chairman or designate in writing the Technical Review Board (TRB) chairman for those RD test activities that require formal technical review.

2.2.3. Responsible for establishing the membership of the TRB.

2.3. RD Chief Engineer

2.3.1. Participate on all safety review boards for RD activities.

2.3.2. Responsible for Readiness Reviews for all unique system/prototype developments that pursue transition or demonstration; extended user evaluation; operational assessment; Joint Concept Technology Demonstration

(JCTD); or Advanced Technology Demonstration (ATD).

2.3.3. Serve as an advisor to the test and safety process.

2.4. Division Chiefs

2.4.1. Responsible for the implementation of test process for division activities.

Ensure all PMs conducting test activities follow the procedures published in this instruction and associated AFRLI 61-103 test volumes. Flight related test activities can be found in AFRLMAN 99-103 and procedures for human use related test activities must be obtained from AFRLI 40-402 and systems engineering processes defined in AFRLI 61-104 and AFMCI 63-1201.

6 RDOI61-103 8 November 2010

2.4.2. Responsible for the approval of any test activity with medium residual risk that is not managed at the directorate level.

2.4.3. Appoint a division test lead to manage division test activities. Submit division representative’s name to AFRL/RD Test Office

2.4.4. Attend or designate representative to participate in technical reviews for their division activities.

2.5. RD Test Office

2.5.1. Test Lead/Deputy Test Lead will

2.5.1.1. Test Lead will act as the Director’s focal point for all test processes/procedures and provide guidance and training to PMs.

2.5.1.2. Provide independent oversight, review, and guidance for transition, demonstration, extended user evaluation, or deployment (see pgh

3.6 below) test strategies, test scope, documentation, suitability, and effectiveness.

2.5.1.3. Conduct “kick-off” meetings to coordinate on overall test strategy for each project.

2.5.1.4. Coordinate on all test plans and consult with Chief Scientist to determine if a formal Technical Review Board (TRB) is required based on the scope, complexity, similarity to previous tests, and anticipated risk level.

2.5.1.5 Conducts technical review for projects/programs not needing a formal TRB, the AFRL/RD Test Lead will determine the technical risk for the test based upon input from PM and Division Test Lead.

2.5.1.6. Designate or recommend the TAA for test events based on the technical and safety risk of the planned test activities.

2.5.1.7. Serve as the AFRL focal point for teaming and collaboration agreements with the T&E community.

2.5.1.8. Participate on all safety review boards for RD activities.

2.5.1.9. Conduct directorate test meetings with division test leads on an as needed basis.

2.5.1.10. Develop Outdoor Test Requirements IAW this OI.

RDOI61-103 8 November 2010 7

2.5.2. RD Range Management

2.5.2.1. RD Range Management will be responsible for the Directorate’s munitions account, frequency assignments, range safety officer, and range resource management. The Range Manager will provide guidance to all PMs on proper field test procedures and assist in field test execution, when necessary.

2.5.2.2 Act as liaison between AFRL/RD and proposed managers of range(s) or test site(s) when AFRL/RD uses directed energy devices outdoors or in the field. Assist project personnel in obtaining appropriate permission for access to property, range assets, facilities and targets.

2.5.2.3. Performs Directed Energy Weapon (DEW) range evaluation for all

RD activities that would be performed on any DoD range or leased site.

2.5.2.4 Perform DEW range evaluation for all programs of record DEW devices and certify the operational use of that device on major AF ranges in accordance with AFI 13-212 Test and Training Range

Policy.

2.5.2.5. Coordinate with headquarters and external agencies, i.e. FAA, AFFMA to obtain necessary permits, equipment registration, frequency assignments, and flight authorizations needed to accomplish RD testing.

2.5.2.6. Manage the Directorate’s Outdoor Test and Experiment Activity

Calendar and provide input to AFRL Center Test Authority in accordance with AFRLI 61-103. Coordinate with the Kirtland

Combined Firing Area Committee (CFAC) for all outdoor testing on Kirtland.

2.5.2.7. Participate on all RD technical and safety review boards involving outdoor testing.

2.5.3. RD Test and Environmental

2.5.3.1. Review all contracts to ensure adequacy of test and environmental requirements and sign the AFRL Form 4, Initial Safety Assessment

(AFI 91-202 AFRLSUP1). The AFRL Form 4 must be submitted to contracting office to ensure incorporation of specific language prior to award.

2.5.3.2. Coordinate local, state, and federal environmental compliance

8 RDOI61-103 8 November 2010 issues associated with RD test activities.

2.5.3.3. Inspect test set-up prior to test execution.

2.5.3.4. Conduct annual review of all test plans for test objective currency and technical risk mitigation.

2.5.3.5. Act as directorate Human Use Resource Coordinator.

2.5.3.6. Sign all RD AF Form 813s, Environmental Impact Analysis and/or coordinate environmental documentation with the appropriate organization. Provide PM guidance on more detailed environmental assessments and environmental impact statements.

2.5.3.7 Manage the environmental program for RD. Provide guidance, inspection and assistance to ensure compliance with environmental requirements.

2.6. Division Test Lead. The Division Test Lead is responsible for the Division’s overall compliance with this OI and must be appointed in writing by the Division Chief.

2.6.1. Responsible for Division test activities. Ensures activities are within the scope of this OI, have completed required documentation (test and safety plans), assists RD Test determining appropriate technical review requirements, and ensures approvals have been obtained prior to test conduct.

2.6.2. Assist PMs in the identification of scope and content required for division test plans.

2.6.3. Manage division SOP(s) and archive them in AFRL/RD/EBS link folders or subsequent automated management system.

2.6.4. Ensure results of the executed test, test plans, technical reviews, Test Readiness

Reviews (TRR), and hot wash meetings, are centrally archived by the PM.

2.6.5. Update the Division’s short- and long-term test and experiments schedule to the

Directorate Test and Experiments Activity Calendar.

2.6.6. Attend directorate test meetings.

2.6.7. Submit flight authorization request packages for Division personnel to the RD

Test Lead for coordination and staffing.

2.6.8. Determines when the TAA and RD Test Office must be re-engaged for changes in scope of PM test activities.

2.6.9. Conducts annual review of division test activities with RD test office.

RDOI61-103 8 November 2010 9

2.7. Project/Program Managers, Scientists, and Engineers

2.7.1. Implements this instruction and AFRLI 61-103, AFRL Research Test

Management. Comply with AFRLMAN 99-103, AFRL Flight Test and

Evaluation; and AFMCI 99-103, Test and Evaluation, Test Management; when activities use flight platforms.

2.7.2. Develop long range test strategy and present early planning events for outdoor tests at Technical Management Reviews (TMRs).

2.7.3. Develop test plans that include annexes for safety, security, human use, flight, and environment as appropriate.

2.7.4. Convene and lead an integrated test plan working group to define test objectives, methodology and exit criteria for proposed tests when appropriate.

2.7.5. Host and schedule test team “kick-off meeting” with RD Test Lead, division test lead, and Safety Office early in the test planning process to obtain specific test guidance for new programs/projects.

2.7.6. Submit and update test schedule via the Directorate Test and Experiments

Activity Calendar for all outdoor tests.

2.7.7. Contact RD test lead for any program(s) that are going to pursue transition, deployment or demonstration offsite early in the program planning phase.

2.7.8. Submit request for munitions, frequency assignments, laser certifications, and

Federal Aviation Administration (FAA) letters of non-objection to RD Test for coordination and external agency processing if required.

2.7.9. Submit all test plan documentation for review, comment and approval to the

RD Test mail box (AFRL/RDO/Test Safety Org Mailbox) or the automated management system, EBS Link https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=12661410&o bjAction=browse&sort=name&viewType=1

2.7.10. Present test scope, objectives, and methodology at technical reviews and conduct Test Readiness Reviews (TRR) as required. See guidance for technical review boards and test readiness reviews on the RD Test EBS Link noted above.

2.7.11. Conduct all test activities IAW approved test plan to include cleanup of test areas.

2.7.12. Ensure mishap accountability, liability, mishap response procedures and https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=12661410&objAction=browse&sort=name&viewType=1 https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=12661410&objAction=browse&sort=name&viewType=1

10 RDOI61-103 8 November 2010 operational reporting (OPREP, AFI 10-206 AFMC Sup1) are determined and documented in the test plan. Report all mishaps to the appropriate safety office IAW locally developed procedures.

2.7.13. Submit all requests for RTO, aeronautical orders and authorization to perform in-flight duties to Division Test Lead and RD Test for processing.

2.7.14. Submit all contract Statements of Work or Objectives, AFRL Form 4, Initial

Safety Assessment, to RD test for review and incorporation of specific test/environmental language and deliverables. Provide coordinated document to contracting office for statement of work update.

2.7.15. Comply with Outdoor Test Requirements IAW this OI.

2.8. Test Approval Authority (TAA). The TAA is the final approval required to begin the execution of the test. The TAA assumes the risk associated with test conduct. The

TAA is determined based on the technical and safety risks associated with the test.

The PM will identify mitigation strategies to minimize all risks to the lowest level, consistent with achieving test requirements. The remaining risk is considered the residual risk, see Figure 2-1 for the TAA levels. Safety risk will be identified by the safety office using the risk assessment matrix in Figure 2-2. Technical risk will be assessed by the PM, Division Test Lead and RD Test Lead based upon the probability and consequences associated with the stated test objectives as shown in Figures 2-3, 2-4 and 2-5. When a formal TRB is conducted the technical risk will be determined by the review board. AFRL/CC or designee will be the TAA for any activity with a high residual safety risk only. AFRL/RD Division Chiefs or designee will be the

TAA for activities with medium residual technical or safety risk that is not managed at the directorate level. Branch Chiefs will be the TAA for activities with low residual technical or safety risk. AFRL/RD director or deputy director will be the TAA for any activity with medium residual technical or safety risk that is of a complex nature, or has high political sensitivity. AFRL RD Test Lead in coordination with the system safety manager will manage the process to identify the TAA level. TAA can be delegated to a lower level as long as it is designated in writing.

RDOI61-103 8 November 2010 11

Figure 2-1 Test Approval Authority

Low Safety or Technical

Medium

Safety or

Technical

Medium

Complex

Safety or

Technical

High

Technical

High Safety

Branch Chief

Division

Chief

Director or

Deputy

AFRL/CC

12 RDOI61-103 8 November 2010

2.8.1. SAFETY RISK. Safety risk is assigned by the safety office as defined in

Figure 2-2.

Figure 2-2 Safety Risk Matrix

2.8.2 TECHNICAL RISK.

2.8.2.1 Low Technical Risk. The test process is clearly understood and well known by the program team. It is very likely that the test objectives can be achieved or the consequences of not achieving the

HAZARD SEVERITY CATEGORY

HAZARD

PROBABILITY

Catastrophic-I

Could result in death, permanent total disability or system/ facility loss>$2M

Critical-II

Could result in permanent partial disability, injuries or illness that may result in hospitalization of >3 personnel, or system/ facility loss >$500k but

<$2M.

Marginal-III

Could result in injury or illness resulting in >1 day of lost work, system/facility loss >$50k but

<$500k.

Negligible- IV

Could result in injury or illness not resulting in lost work time, system /facility loss >$2k but

<$50k.

FREQUENT-A

Likely to occur often in the life of an item or during an event

1 3 7 13

PROBABLE-B

Will occur several times during the life of an item or during an event

2 5 9 16

OCCASIONAL-C

Likely to occur sometime in the life of an item or during an event

4 6 11 18

REMOTE-D

Unlikely, but possible to occur in the life of an item or during an event

8 10 14 19

IMPROBABLE-E

Highly unlikely to occur in the life of an item or during an event

12 15 17 20

RDOI61-103 8 November 2010 13 objectives are negligible.

2.8.2.2 Medium Technical Risk. Portions of the test activities are new or have not been successfully demonstrated in the past. A moderate investment is required to execute the plan. There is a low to medium probability that some test objectives may not be achievable, but the consequences of not achieving them are minimal to moderate. An example would be a test that has a narrow window of opportunity for data collection, but the test could be repeated in a timely manner at a reasonable expense.

2.8.2.4 High Technical Risk. A significant portion of the planned test has not been successfully demonstrated by the test team. A significant investment is required to execute the test. Careful planning is required to ensure all preparation activities come together to ensure successful execution of the test. One or more organizations have committed significant resources to the test activity. There is medium to high probability that test objectives cannot be achieved or the consequences of not achieving the objectives are moderate to severe. An example of an activity with high technical risk is a test with a narrow window of opportunity for data collection with no reasonable means to repeat the test in the event of a failure. This category of test includes test activities requiring significant coordination with members outside the AFRL/RD community.

Technical risk needs to be evaluated for each test series. Reference

DoD Acquisition Guide for Risk Management at AFRL/RDO/Test

Safety Resources EBS site; see Figures 2-3, 2-4 and 2-5. The

Risk Matrix must be uniquely defined for each test effort. Typical risks are related to; threat or sensitivity of a test, requirements, ability of system configuration to achieve engineering objectives, adequacy of test, predictions or modeling & simulation, logistics, cost, and schedule.

The level and types of consequences of each risk are established using criteria as in Figure 2-3.

14 RDOI61-103 8 November 2010

Figure 2-3. Levels and Types of Consequence Criteria

Level Technical Performance Schedule Cost

Minimal or no consequence to technical performance

Minimal or no impact Minimal or no impact

Minor reduction in technical performance, can be tolerated with little or no impact

Able to meet key dates, slip<_* month(s)

Budget increase , <**

(1% of Budget)

Moderate reduction in technical performance, limited impact on program

Minor schedule slip.

Slip<* month(s)

Sub-system slip > * month(s)

Budget increase, < ** (5% of budget)

Significant degradation in technical performance; may jeopardize program success

Program critical path affected

Slip<* months

Budget increase

<** (10% of budget)

5 Severe degradation in technical performance; will jeopardize program success

Cannot meet key program objectives

Slip> * months

Exceeds budget authority

>** (10% of Budget)

The likelihood of a risk item must be defined, using the following chart. If the root cause has an estimated 50 % probability of occurring, the corresponding likelihood is level 3.

RDOI61-103 8 November 2010 15

Figure 2-4. Levels of Likelihood

Level Likelihood Probability of

Occurrence

1 Not Likely 10%

2 Low Likelihood 30%

3 Likely 50%

4 Highly Likely 70%

5 Near Certainty 90%

Using information from Figures 2-3 and 2-4 the resulting matrix would look somewhat like that shown in Figure 2-5.

Figure 2-5. Risk Matrix

1 2 3 4 5

3. Test Process

3.1. Test Process. All AFRL research test activities will follow the top-level process of planning, technical review, safety review, test approval, execution, lessons learned, and documentation. Each of these steps can be tailored to the level appropriate for the activity, i.e., a basic research experiment may be conducted under an AFRL Form 5, AFRL Safety Permit and associated safety document with a one-page test Standard

Operating Procedure, whereas a flight test may require a technical review board and

Flight Safety Review Board (FSRB) coordinated with HQ AFRL.

Typically this effort

Consequence

Li ke lih o o d

16 RDOI61-103 8 November 2010 requires significant interaction with proposed ranges; and results in documentation to address test objectives; measures and instrumentation; transportation; security;

spectrum; and FAA. The process, Figure 3-1, is designed to identify the long lead items and potential pitfalls early on in the test planning process, when corrective actions can be taken and test resources can be identified with minimal impact to the test schedule.

Figure 3-1. Research Test Management Process

Note: This process may be modified as needed for those efforts that i.e. have an RTO or follow a readiness review

RDOI61-103 8 November 2010 17

3.1.1. Each division or branch will identify the critical issues and objectives, taking input from the customer requirements and determining if the test is supportable.

Key issues to consider will be cost, schedule, resources, and if science/test objectives can be obtained. Once the division or branch decides to perform the test, a “kick-off meeting” will be hosted by the PM and held with the Division

Test Lead, RD Test Lead, and the Safety Office to discuss the detailed requirements to accomplish the test.

3.1.2. The RD Test Lead will work with the Chief Scientist to determine if a formal

TRB is required and the Safety Office will determine the need for a formal

SRB. When the technical risk appears to be substantial, a formal TRB will be held. Formal TRBs will be chaired by the Chief Scientist or designee. All test activities involving flight (unmanned air vehicle, aircraft, balloons, or rockets) will convene a TRB and Flight SRB prior to operation. The RD Test

Lead in coordination with the PM and division test lead will discuss the requirement for a formal TRB at the “kick-off” meeting. Typically, TRBs will be held on those efforts with extensive scope, complexity, non-similarity to previous tests, and anticipated high risk levels. First time ever tests with medium safety risk; high value assets, explosives; significant environmental impacts; or high complexity/visibility may require SRB as determined by the safety office and TRBs. When it is determined that formal boards are not required, an informal technical review will occur and the RD Test Lead will assign the risk level. Technical reviews are performed during the test plan evaluation and by branches and divisions specifically for the project/program.

The RD Test Lead will assign residual technical risk to the test plan based upon these technical reviews and analyses provided.

3.2 Test Documentation. To begin the technical review process the PM must prepare test documentation to adequately cover technical, safety, security and any other task necessary to accomplish the test. However when required, the formal technical review board must be completed prior to test plan submittal through the RD test process. The PM must establish and present test objectives, measures of performance and data collection information at the TRB for the board to evaluate the technical risk.

If a formal review board is not required the PM should submit the test plan with their technical risk assessment to RDOT for technical risk evaluation and TAA determination.

3.2.1. The test plan is the umbrella that establishes the type and amount of testing and resources required to satisfactorily accomplish the stated objective(s). The standardized test plan template is designed to help the test planners build a concise test plan with clear objectives and details on how to achieve those objectives. All paragraphs must be addressed except where indicated as optional. Additional paragraphs can be added to this test plan outline to address unique test requirements. The format of the standard test plan and additional guidance is provided in the Test Guidance Folder, available on the

RDO EBS Live Link

18 RDOI61-103 8 November 2010 bjAction=browse&sort=name

3.2.1.1. The Safety Package. The typical safety package includes the

Appendix S template that discusses test hazards and an AFRL Form

5. Both documents are attached to the test plan. AFRL Form 12’s would be included for any flight test activity. The Safety Office provides a template to minimize duplication and focuses on the information required by the safety office.

3.2.1.2. The Environmental Package. All test activities must have an AF

Form 813, Environmental Impact Analysis. If an action can be excluded from further environmental analysis, the AF Form 813 will receive a categorical exclusion (CATEX) in accordance with 32

CFR 989 and will be signed by the RD Environmental Manager

(EM) or appropriate organization. If the action requires further documentation, RD will work with the PM to accomplish an

Environmental Assessment or Environmental Impact Statement.

These efforts will most likely require program funds; therefore, it is critical that the PM identify their actions to the RD EM early in the planning phase or at the “kick-off” meeting. The EM will assist

PMs with hazardous material authorization and creation of an initial accumulation point for any generated hazardous waste. Templates for the environmental packages are available on the automated RD test EBS link under test guidance as noted above.

3.2.1.3. Other packages as required. Test activities that require separate flight, human use, security, transportation or any other applicable attachments shall be appended.

3.2.2. Submission and Review. Submit completed test plan including any environmental or safety documentation to the EBS/RDO Collaborative Website at bjAction=browse&sort=name

Notify AFRL/RDO/Test Safety Org mailbox administrator that your documents have been submitted for review. Any other approvals for operation must be included in the package (i.e., FAA coordination, Frequency

Assignment, Medical/Chemical Hygiene Plans, Laser Certifications, etc…)

The safety office, RDOT, Branch Chief and division test lead for the effort are tasked to review documents for the initial review. A suspense of 7 working days is given for this review. RD Test, Branch Chief, Division Test and the

Safety Office will comment on the test/safety plan and return the marked document to the customer in the EBS RDO Collaborative folder for PM https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=12661410&objAction=browse&sort=name https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=12661410&objAction=browse&sort=name https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=12661410&objAction=browse&sort=name https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=12661410&objAction=browse&sort=name

RDOI61-103 8 November 2010 19 clarification. The PM, Branch Chief and Division Test Lead will be notified via e-mail to let them know the document requires their action and a suspense of 5 working days will be established. All comments and questions must be addressed by the PM. Following the final reconciliation of the comments, the test plan will be assigned residual technical/safety risks by the RD Test Lead and the Safety Office respectively. The residual technical and safety risks will determine the appropriate management level to approve the AFRL Form 5

(System Safety Permit) and Test Plan by the TAA. The EBS can be audited to reveal a complete staff summary for each effort. Once the documents are moved to the TAA folder for signature RD Test will submit the AFRL Form 5 and Test Plan cover page to the appropriate individuals for signature via e-mail, no staff summary is required.

Contact the RD Test Lead and Safety Office for the review of any classified documents. These documents will not be submitted to the EBS website and must be maintained by the PM and/or Division Test Lead. These efforts will meet the requirements of this instruction, but will not be processed electronically through the EBS website.

3.3. Technical Review Process. The technical review process is a thorough assessment of the test plan for technical soundness and adequacy. It will verify if the overall method of test is adequate to evaluate the test objectives and verify that the objectives can be met with acceptable technical risk. The technical review must take place before the safety review. The determined risk from the technical and safety reviews will determine the appropriate management level for TAA. The test cannot begin until the

TAA has signed the test plan which will include the signed system safety permit, AFRL

Form 5.

3.3.1. The technical review can range from the independent review of a test plan by

RD Test Lead or deputy to a full Technical Review Board depending on the complexity of the test activity. The chairman of the technical review board signs the AFRL Form 19b indicating the review has taken place, that the plan is technically sound and that a technical risk level has been assigned. Technical review findings are maintained with the test documents on the AFRL/RDO

EBS link https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=12661410&o bjAction=browse&sort=name&viewType=1.

https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=12661410&objAction=browse&sort=name&viewType=1 https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=12661410&objAction=browse&sort=name&viewType=1

20 RDOI61-103 8 November 2010

The requirements for technical review will be determined on a test-by-test basis by the RD Test Lead, with input from the Division Test Lead and PM.

3.3.2. Technical Review Board (TRB). The intent of the TRB is to independently assess test plans for technical soundness. The TRB evaluates test requirements and objectives, ensures the overall approach relates to the objectives, evaluates the technical adequacy, and verifies that the tester can meet the objectives with acceptable technical risks. The TRB assesses test objectives, test methods and procedures, and scope of tests for completeness and confirms that required test resources have been properly identified, that the data collection will be appropriate and sufficient in depth, and that test strategy will meet laboratory science requirements.

3.3.2.1. All test activities involving Human Use or flight (unmanned air vehicle, aircraft, balloons or rockets) will include a TRB prior to operation. All TRBs are chaired by the AFRL/RD Chief Scientist or designee. Flight test TRBs will follow the guidance given in

AFRLMAN 99-103. The TRB will assign action items to the PM for any areas that warrant further attention prior to the SRB. AFRL

Form 19B, Technical Review Board, will be used to document the risk and technical review process. Personnel from outside organizations may also participate on an ad hoc basis according to their particular expertise.

3.3.3. External or contractor test activities. If the program designates a Responsible

Test Organization or an external Lead Test Organization or if mishap accountability and liability belong to another organization participating in the test, then their respective technical review processes can be used. AFRL/RD test personnel will participate in the TRB or Technical Review as a test advisor for AFRL technology to be utilized during the test.

3.3.3.2. All flight test (i.e., remote control and unmanned air vehicle, balloons, rockets, aircraft) programs must convene a TRB. The technical review or TRB should be convened prior to the safety review process so the Board can introduce changes to the test approach prior to test plan preparation and safety review assessment.

3.4. Safety Review Process. The safety review process prescribed in AFI 91-202, AFMC and AFRL Supplements, Chapter 13 and local instructions will be used for all research, test and technology development/demonstration activities. The purpose of the safety review process is to prevent injury to personnel and damage to property. The review process will identify hazards and associated risks and provide a means to determine the residual risk after the proper selection of control measures.

3.4.1. Safety Review Board (SRB). The SRB is an independent group convened to mailto:

RDOI61-103 8 November 2010 21 review the safety plan, ensure hazards are identified, eliminated, or minimized, and recommend the overall safety risk level to the TAA. The Safety Office will determine the need for an SRB and will designate a chairman. The organization with mishap accountability conducts the SRB. Any FSRB will require the AFRL Form 29 to document test safety mishap for the project/program and the AFRL Form 19A to document the recommendation of the SRB for safety risk of the activity.

3.5. Review of Classified Tests. Classified and Special Access programs are subject to this OI. Equivalent test and safety reviews must be implemented within the security confines of the program. The RD Test Lead or designee will be read into these AFRL programs unless a waiver is granted by the AFRL/RD Director.

3.6. Technology Transition, Demonstrations or Deployments of Research Systems.

When technology development reaches an appropriate level of readiness to initiate transfer or integration in a configuration for the purposes of demonstration, assessment or deployment the PM must implement disciplined systems engineering (SE) process.

At a minimum ATDs and JCTDs will meet the definition of demonstration.

3.6.1. If there is a reasonable expectation of providing a system or end-item as an experimental leave behind capability for operations by the user the systems engineering process must include documentation pertaining to safety, suitability, effectiveness characteristics and required certifications of the system or end-item.

3.6.2. PM must contact the test office and RD Chief Engineer to discuss the proposed integration and test strategy. Topics of interest would focus on customer requirements, integration approach, component verification, configuration management, full system test, and delivery of system documentation. The review team will advise and assist the PM to accomplish the appropriate level of safety, suitability and effectiveness and ensure adequate training and documentation is transferred to the receiving agency.

3.6.3. The PM is responsible for identifying and providing the using command a list of all identified hazards, mitigation measures, training, documentation, risk assessments and risk acceptances, reference AF 91-202 AFMC Sup1.

3.6.4. A Readiness Review will be convened for RD systems deemed ready for demonstration, transition or assessment. The RD Chief Engineer or designee will chair the review. Membership will consist of AFRL/RD Test, the appropriate safety office; other members will be designated by the chairman.

The review team will make a recommendation to the RD Director that the system is ready for transition, operational assessment, extended user evaluation, demonstration, or deployment.

22 RDOI61-103 8 November 2010

4. TEST EXECUTION

4.1. Test Execution. The program manager is responsible for reviewing the test plan and ensuring the following.

4.1.1. All action items identified during the technical and safety review are incorporated into the final version of the test plan.

4.1.2. Test Readiness Review (TRR). The TRR is normally conducted by the PM and is typically presented to the branch chief and division test lead before the commencement of testing for new-starts and system upgrades, test milestones

(e.g., first flights, etc), or after an extended break in test activity. Depending upon the complexity of the effort additional participants (Division Chief

Engineer, RD Test Lead, environmental, safety…) would be invited to the

TRR. The TRR is needed to ensure all test assets are in place; pretest activities are complete; permits and certifications are approved; safety and environmental concerns are addressed; and that the system is ready for testing. The review is to ensure all preparations for initiating a test have been completed and known anomalies will not compromised the execution of the test. The TRR will ensure all key participants are familiar with the hazards, associated risks, minimizing procedures or controls, emergency or corrective actions procedures, and go/no-go criteria before executing any test event. Results of the TRR will be documented by the program manager in meeting minutes or research notebook. The Program Manager will proceed with the test activity based on TAA approval and the outcome of the TRR. If significant issues (i.e., technical or safety) come to light during the TRR the PM will inform the TAA, RD Test Lead and Division Test Lead of those issues to determine if a new review is required.

https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=15807887&o bjAction=browse&viewType=1

4.1.3. In the event of a mishap, testing will be suspended and mishap procedures will be initiated. Any personnel associated with the test may suspend testing if they encounter an unforeseen hazard or unsafe event. Once suspended for safety, only the TAA (or higher) can authorize resumption of test activities.

The TAA will determine if a supplemental SRB is required to resolve the unsafe condition.

4.2. Test Plan Changes/Revisions. The PM will ensure that all testing is conducted according to the approved test plan. Test plan or approved scope changes/revisions may be required due to unexpected test results, adequacy of test controls, test program initiated changes, or hazards not previously identified may require additional reviews

4.2.1. PMs will contact the Division Test Lead, if changes to the test plan are required after approval has been granted. Changes will be evaluated to determine their impacts. Actions may vary from no action, to test and safety plan https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=15807887&objAction=browse&viewType=1 https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=15807887&objAction=browse&viewType=1

RDOI61-103 8 November 2010 23 amendments, to re-accomplishing the technical and safety reviews. If the change is significant the Division Test lead will contact the RD Test Lead and

TAA to determine appropriate approval procedures.

4.3. Project Completion or Termination. The PM will conduct “hot wash” meeting with test team at the end of each test day. The Division Test Lead will be invited to the

“hot wash” when the test director has issues or concerns. The PM will notify the

Division Test Lead and the RD Test Lead for those medium or high risk efforts performed outdoors when the test is complete or test activity has been terminated.

5. Forms.

5.1. Prescribed Forms.

None

5.2. Adopted Forms.

AF Form 813, Environmental Impact Analysis

AFRL Form 4, Air Force Laboratory Risk Management Checklist

AFRL Form 5, AFRL Safety Permit

AFRL Form 12, Test Hazard Analysis

AFRL Form 19A, Safety Review Board

AFRL Form 19B, Technical Review Board

AFRL Form 20, Lessons Learned

AFRL Form 29, AFRL Test Safety Mishap Worksheet

SUSAN J. THORNTON, SES

Director, Directed Energy Directorate

24 RDOI61-103 8 November 2010

ATTACHMENT 1

GLOSSARY OF REFERENCES AND SUPPORTING INFORMATION

References

AFI 99-103 Test and Evaluation

AFMCI 99-103 Test Management, Test and Evaluation

AFI 91-202 AFMC Sup1, AFRLSup1 The US Air Force Mishap Prevention Program Safety

AFRLI 61-103, AFRL Research Test Management

Abbreviations and Acronyms

AFRL/RD - Air Force Research Laboratory Directed Energy Directorate

ATD - Advanced Technology Demonstration

CATEX - Categorical Exclusion

CFAC - Combined Firing Area Committee

CFR - Code of Federal Regulations

DEW - Directed Energy Weapon

EBS - Enterprise Business System

EM - Environmental Manager

FAA - Federal Aviation Administration

FSRB - Flight Safety Review Board

IAW - In Accordance With

JCTD - Joint Concept Technology Demonstration

LTO - Lead Test Organization

PM - Project/Program Manager

RTO - Responsible Test Organization

SE - Systems Engineering

SOP - Standard Operating Procedures

SRB - Safety Review Board

SSM - System Safety Manager

T&E - Test and Evaluation

TAA - Test Approval Authority

TMR - Technical Management Reviews

TRB - Technical Review Board

TRR - Test Readiness Reviews