Specs.docx

Risk-Based Analysis Manual Requirements

1.1 Introduction - The Government is in the process of preparing a competitive solicitation for the acquisition of a risk-based analysis manual at US Army Yuma Proving Ground. The acquisition will include an instruction manual for users to conduct risk-based assessments for artillery and aircraft tests in areas near the US Army Yuma Proving Ground.

1.2 BACKGROUND – The risk-based analysis manual is necessary to allow testers to develop a quantitative assessment for the amount of risk associated with a particular test. The risk assessment will evaluate the risk posed to the public, aircraft, and test support personnel/equipment by conducting military tests.

2.1 SCOPE - This scope involves the acquisition of materials and labor to provide a risk-based analysis manual per the Government’s requirements. The manual is intended to provide methods and practices to quantify risk for certain hazardous operations that will enable test personnel to make informed decisions regarding the conduct of military testing at Yuma Test Center.

2.2 General Specifications- The specifications provided to ensure that the final products meet customer requirements and comply with necessary operational and security criteria. Failure to comply will render the risk-based analysis manual unusable.

Standard 321-16: Common Risk Criteria Standards for National Test Ranges Standard 321-10 (Supplement): Common Risk Criteria Standards for National Test Ranges (Supplement)

· Test item kinematics (artillery and aircraft)

· Population density of areas at risk

· Vehicle traffic within areas of risk

· Aircraft traffic within areas of risk

· Temporal effects

· Monte Carlo simulation of possible impact locations

· Probability of impact

· Probability of fatality

· Expected casualty

2.5 SUMMARY - This draft requirements document is provided to inform vendors of Governments requirements for a risk-based analysis manual. Industry input is welcome but not required. Government Point of Contact (POC) for requirements: Omar Silva; Range Operations and Training Division, YPG; (928) 328-4805; email: Omar.C.Silva.civ@mail.mil image1.emf

321-16_Common_Risk_Criteria_Test_Ranges.pdf

Range Safety Group

STANDARD 321-16

COMMON RISK CRITERIA STANDARDS FOR NATIONAL TEST

RANGES

DISTRIBUTION A: APPROVED FOR PUBLIC RELEASE

DISTRIBUTION IS UNLIMITED

ABERDEEN TEST CENTER

DUGWAY PROVING GROUND

REAGAN TEST SITE

YUMA PROVING GROUND

WHITE SANDS MISSILE RANGE

NAVAL AIR WARFARE CENTER AIRCRAFT DIVISION

NAVAL AIR WARFARE CENTER WEAPONS DIVISION

NAVAL UNDERSEA WARFARE CENTER DIVISION, KEYPORT

NAVAL UNDERSEA WARFARE CENTER DIVISION, NEWPORT

PACIFIC MISSILE RANGE FACILITY

30TH SPACE WING

45TH SPACE WING

96TH TEST WING

412TH TEST WING

ARNOLD ENGINEERING DEVELOPMENT CENTER

NATIONAL AERONAUTICS AND SPACE ADMINISTRATION

This page intentionally left blank.

DOCUMENT 321-16

COMMON RISK CRITERIA STANDARDS FOR NATIONAL TEST

RANGES

August 2016

Prepared by

Range Safety Group Risk Committee

Published by

Secretariat Range Commanders Council

US Army White Sands Missile Range, New Mexico 88002-5110

Common Risk Criteria Standards for National Test Ranges, RCC 321-16, August 2016 iii

Table of Contents Changes to this Edition ..................................................................................................................v

Preface .......................................................................................................................................... vii Acronyms ...................................................................................................................................... ix

Chapter 1. Introduction ........................................................................................................ 1-1

1.1 Purpose ............................................................................................................................. 1-1

1.2 Scope ................................................................................................................................ 1-1

1.3 Implementation ................................................................................................................ 1-1

1.4 Range Responsibilities ..................................................................................................... 1-2

Chapter 2. Policies and Procedures ..................................................................................... 2-1

2.1 General Policy and Goals................................................................................................. 2-1

2.2 Policy Objectives ............................................................................................................. 2-2

2.2.1 General Public ...................................................................................................... 2-2

2.2.2 Mission-Essential Personnel ................................................................................ 2-2

2.2.3 Critical Operations Personnel .............................................................................. 2-2

2.2.4 Catastrophe Potential and Transportation Systems.............................................. 2-2

2.2.5 Spacecraft ............................................................................................................. 2-3

2.2.6 Critical Assets ...................................................................................................... 2-4

2.2.7 Environment ......................................................................................................... 2-4

2.3 Risk Management Process ............................................................................................... 2-4

2.3.1 Phase I: Mission Definition and Hazard Identification....................................... 2-5

2.3.2 Phase II: Risk Assessment .................................................................................. 2-6

2.3.3 Phase III: Criteria Comparison and Risk Reduction ........................................... 2-7

2.3.4 Phase IV: Risk Acceptance ................................................................................. 2-8

2.3.5 Conditional Risk Management ............................................................................ 2-8

2.4 Uncertainty in the Computed Risk ................................................................................... 2-9

Chapter 3. Acceptable Risk Criteria ................................................................................... 3-1

3.1 Performance Standards .................................................................................................... 3-1

3.2 Personnel Protection ........................................................................................................ 3-2

3.2.1 General Public ...................................................................................................... 3-2

3.2.2 Mission-Essential and Critical Operations Personnel .......................................... 3-2

3.3 Aircraft Protection ........................................................................................................... 3-3

3.3.1 Non-Mission Aircraft Criteria.............................................................................. 3-3

3.3.2 Mission-Essential Aircraft Criteria ...................................................................... 3-3

3.3.3 Aircraft Hazard Volumes for Planned Debris Releases ....................................... 3-3

3.3.4 Mishap Response ................................................................................................. 3-4

3.4 Ship Protection ................................................................................................................. 3-4 iv

3.4.1 Non-Mission Ship Criteria ................................................................................... 3-4

3.4.2 Mission-Essential Ship Criteria ........................................................................... 3-5

3.4.3 Ship Hazard Areas for Debris Releases ............................................................... 3-5

3.4.4 Mishap Response ................................................................................................. 3-5

3.5 Spacecraft Protection ....................................................................................................... 3-5

3.6 Catastrophic Risk Protection............................................................................................ 3-6

3.6.1 General ................................................................................................................. 3-6

3.6.2 Ship and Aircraft Hazard Areas ........................................................................... 3-6

3.6.3 General Public Criteria ........................................................................................ 3-6

3.6.4 Mission-Essential and Critical Operations Personnel Criteria............................. 3-8

3.7 Criteria Summary ............................................................................................................. 3-8

Appendix A. Glossary ............................................................................................................. A-1

Appendix B. Citations ............................................................................................................. B-1

Appendix C. References .......................................................................................................... C-1 v

Changes to this Edition This document is an updated version of RCC Document 321-10 (Common Risk Criteria for National Test Ranges). The following is a list of changes.

Note: The use of the word “supplement” used herein refers to the companion (supplemental) document to RCC Document 321.

a. Modified aircraft vulnerability criteria for business class jets.

b. Modified the aircraft vulnerability criteria for large commercial transport aircraft.

c. Modified ship vulnerability criteria.

d. Modified ship protection guidance.

e. Updated collision avoidance standards and guidelines.

f. Corrected substantive and administrative errors throughout the standard and supplement.

vi vii

Preface This is the foundational document that defines consensus standards for the range risk management process and risk criteria. The companion document1 provides additional detailed information to assist in implementation of the standards. The standards in this document are not considered absolute; rather, this document and its supplement are intended to provide a baseline for defining acceptable risks for hazardous range operations and to assist the user in the development and implementation of robust risk assessments.

This document presents the results of an extensive cooperative effort by the Range Safety Group of the Range Commanders Council. Planned and unplanned hazardous events generated by flight operations present a safety concern for all test ranges. Each range has established its own set of criteria and analytical methods for protecting personnel, facilities, aircraft, and other assets from hazardous operations. Although these separate efforts have been very successful, the logical relationships among criteria used at the test ranges and across different hazards are often difficult to comprehend. The consensus standards presented in this document are intended to:

a. Promote a common set of safety standards and process among the ranges;

b. Promote valid, repeatable risk assessments;

c. Facilitate innovation to support challenging missions;

d. Nurture openness and trustworthiness among the ranges, range users, and the public;

e. Simplify the scheduling process;

f. Present common risk criteria that can reduce cost for users of multiple test ranges.

The RCC gives special acknowledgement for production of this document to:

Dr. Paul Wilde, Chairman, RSG Risk Committee Deputy Chief Engineer, Federal Aviation Administration, AST-4 800 Independence Ave, SW. Rm 331 Washington, D.C. 20591 Phone (202) 267-5727 E-mail Paul.Wilde@faa.gov

Please direct any questions to:

Secretariat, Range Commanders Council

ATTN: CSTE-WS-RCC

1510 Headquarters Avenue White Sands Missile Range, New Mexico 88002-5110 Phone (575) 678-1107, DSN 258-1107 E-mail usarmy.wsmr.atec.list.rcc@mail.mil

1 Range Commanders Council. Common Risk Criteria for National Test Ranges (Supplement). 321-10 (Supplement). December 2010. May be superseded by update. Retrieved 28 July 2016. Available to RCC members with Private Portal access at https://wsdmext.wsmr.army.mil/site/rccpri/Publications/321- 10_Supplement_Common_Risk_Criteria_Standards_For_National_Test_Ranges/.

mailto:Paul.Wilde@faa.gov mailto:usarmy.wsmr.atec.list.rcc@mail.mil https://wsdmext.wsmr.army.mil/site/rccpri/Publications/321-10_Supplement_Common_Risk_Criteria_Standards_For_National_Test_Ranges/ https://wsdmext.wsmr.army.mil/site/rccpri/Publications/321-10_Supplement_Common_Risk_Criteria_Standards_For_National_Test_Ranges/ viii ix

Acronyms AIS abbreviated injury scale COLA collision avoidance COP critical operations personnel DoD Department of Defense DoDD Department of Defense Directive DoDI Department of Defense Instruction EC expected casualties EF expected fatalities FAA Federal Aviation Administration FTCA Federal Tort Claims Act FTS flight termination system GP general public JSpOC Joint Space Operations Squadron km kilometer MEP mission-essential personnel MRTFB Major Range and Test Facility Base nm nautical mile NOTMAR Notice to Mariner RC Risk Committee RCC Range Commanders Council RSG Range Safety Group UAV unmanned aerial vehicle x

1-1

CHAPTER 1

Introduction

1.1 Purpose

This document provides a common set of range safety policies, risk criteria, and guidelines for managing risk to people and assets during manned and unmanned flight operations, excluding aviation operations. It establishes the following:

a. Acceptable risk criteria for both the general public (GP) (involuntary acceptance) and mission-essential personnel (MEP) (voluntary acceptance), excluding people in the launch or reentry vehicle;

b. Debris injury thresholds for unprotected people;

c. Debris hazard thresholds for aircraft and ships;

d. Vulnerability models for large commercial transport aircraft and business-class jets;

e. Approach for evaluating flight hazards to critical assets.

1.2 Scope

The policies and criteria in this document are intended for use by members of the

Department of Defense (DoD) national ranges and Major Range and Test Facility Bases (MRTFBs). These policies and criteria apply to launch and reentry hazards generated by endoatmospheric and exoatmospheric range activities, including both guided and unguided missiles and missile intercepts, space launches, and reentry vehicles. This does not include aviation operations or unmanned aerial vehicle (UAV) operations. The Range Commanders Council (RCC) document 323-992 provides criteria for UAVs.

1.3 Implementation

This document is an advisory document. Its content is based on the consensus positions held by the Risk Committee (RC) within the Range Safety Group (RSG), which is made up of a broad cross-section of the US range safety community. Therefore, the content of this document represents consensus standards. The main body of this document provides consensus standards with the highest levels of priorities while the supplement contains lower levels of priority requirements, guidelines, and example methods; however, precise language is used in both this document and the supplement in an attempt to capture the intent of the RC as follows.

• The words “must,” “shall,” and “will” indicate a requirement that is strongly recommended. Legitimate alternatives may exist, but each alternative shall demonstrate an equivalent level of safety or be granted a waiver.

2 Range Commanders Council. Range Safety Criteria for Unmanned Air Vehicles. RCC 323-99. December 1999.

May be superseded by update. Retrieved 6 April 2016. Available at http://www.wsmr.army.mil/RCCsite/Documents/323- 99_Range_Safety_Criteria_Unmanned_Air_Vehicles_(Basic)/323-99.pdf.

http://www.wsmr.army.mil/RCCsite/Documents/323-99_Range_Safety_Criteria_Unmanned_Air_Vehicles_(Basic)/323-99.pdf http://www.wsmr.army.mil/RCCsite/Documents/323-99_Range_Safety_Criteria_Unmanned_Air_Vehicles_(Basic)/323-99.pdf

1-2

• “Should” indicates an advisory requirement or a highly desirable procedure. When this standard uses “should,” the committee intends that a range will achieve compliance to the maximum extent practical, but no waiver or equivalent level of safety will be required.

• “Can” and “may” permit a choice and express a guideline.

In order for a range to effectively implement this document, the range should evaluate the contents of this document and incorporate it accordingly into the local regulations and requirements.

1.4 Range Responsibilities

High-level authorizing documents, such as the Department of Defense Instruction (DoDI)

3200.18,3 assign responsibility to each range commander4 (or other launch/reentry decision authority for non-DoD facilities) for ensuring that all missions are conducted safely, consistent with operational requirements. Range flight operations typically involve some level of risk.

Therefore, an important aspect of the range safety responsibility is to ensure that the risk is properly managed within prescribed limits. To accomplish this, each range commander (or designee) must perform the following.

a. Establish risk management procedures (including hazard containment) to implement the risk management process described herein.

b. Establish acceptable risk criteria appropriate to each type of mission flown in consideration of the guidance provided herein.

c. Accept any risks, including those that exceed the established risk criteria when warranted for a mission in consideration of the operational requirements and national need.

(1) Make such decisions based on a thorough understanding of any additional risk that exceeds the risk criteria and the benefits to be derived from taking the additional risk.

(2) Ensure such decisions are documented in a formal waiver process (or equivalent), preferably in advance of the mission.

d. Maintain related range policy and requirements documents.

e. Maintain records of risk assessments and waivers to established risk criteria.

f. For a mission involving more than one range:

(1) coordinate with the other range(s) to clearly document safety responsibility for each phase of the mission;

(2) develop and implement joint plans for controlling the mission risk due to all planned and unplanned events.

3 Department of Defense. “Subject: Management and Operation of the Major Range and Test Facility Base (MRTFB).” 1 February 2010. May be superseded by update. Retrieved 6 April 2016. Available at http://www.dtic.mil/whs/directives/corres/pdf/320018p.pdf.

4 This document uses “range commander” to refer to the duly authorized launch or reentry decision authority.

http://www.dtic.mil/whs/directives/corres/pdf/320018p.pdf

2-1

CHAPTER 2

Policies and Procedures

2.1 General Policy and Goals

In planning any operation, risk must be reduced to the extent that is practical in keeping with operational objectives. Safety should be balanced with operational objectives by cooperative interaction between the range and the range user. To maximize achievement of mission objectives within safety constraints, the range user should consider overall risk along with other factors that affect mission acceptability. These factors include criticality of mission objectives, protection of life and property, the potential for high-consequence mishaps, local political factors, and governing range or programmatic environmental requirements.

All ranges should strive to achieve complete containment of hazards resulting from both normal and malfunctioning flights. If a planned mission cannot be accomplished using a containment approach, a risk management approach may be authorized by the range commander or the designated representative. The risk management approach should conform to the guidelines presented in this document or otherwise demonstrate compliance with the objectives presented.

Range commanders should never regard events (such as injuries) as being routine or permissible. No adverse consequences are routinely acceptable; however, the probability is finite that range mishaps producing adverse consequences may occur. The term “acceptable risks” used in this document can be properly interpreted as “tolerable risks.” These are risks the range commander may tolerate to secure certain benefits from a range activity with the confidence that the risk is properly managed within prescribed limits.

Compliance with this document leads to defensible launch support and launch commit decisions. Employing a sound basis for accuracy and repeatability in risk assessments leads to consistent risk acceptance decisions, thereby fostering public confidence that the ranges are operated with appropriate regard for safety. Thus, individuals living or working at or near a range may go about their daily lives without concern for their proximity to range activities.

Moreover, compliance with these guidelines provides assurance that flights near or over communities by space boosters or weapon systems does not significantly increase the risk to these communities. These goals have led to the policy objectives provided here.

In defining objectives for risk assessment and risk management, the RCC goals are to:

a. Create a common set of safety standards and processes among the ranges that will achieve the stated risk management goals;

b. Promote accurate, repeatable risk assessments by minimizing errors in estimating and ensuring their scientific validity;

c. Facilitate innovation to support challenging missions;

d. Nurture openness and trustworthiness among the ranges, range users, and the public.

2-2

2.2 Policy Objectives

2.2.1 General Public

The GP includes all people located on- and off-base that are not essential to a specific mission or nearby critical operation. This definition applies to all people regardless of whether they are in some mode of transportation (such as airplanes, ships, and busses), are within a structure, or are unsheltered. The GP should not be exposed, individually or collectively, to a risk level greater than the background risk in comparable involuntary activities, and the risk of a catastrophic mishap should be mitigated.

In the above context, the RCC considers “comparable involuntary activities” as those where the risk arises from manmade activities that:

a. Are subject to government regulations or are otherwise controlled by a government agency;

b. Are of vital interest to the US; and

c. Impose involuntary risk of serious injury or worse on the public.

2.2.2 Mission-Essential Personnel

A certain degree of risk is inherent in hazardous operations. The MEP may include persons in training to perform specific tasks that are part of the current operation. The range commander or mission director (or their designees) should identify the MEP in training and justify their designation as MEP. The MEP should not be exposed, individually or collectively, to a risk level greater than that found in comparable high-risk occupations, and the risk of a catastrophic mishap should be mitigated.

2.2.3 Critical Operations Personnel

Critical operations personnel (COP) include persons not essential to the specific operation or launch currently being conducted, but who are required to perform safety, security, or other critical tasks at the range. The critical operations range user (or manager) provides the number and justification of personnel required to conduct the critical operations. The range safety personnel will approve or determine the number and location of COP with the concurrence of the appropriate decision authority. The COP should be included in the same risk category as MEP.

2.2.4 Catastrophe Potential and Transportation Systems

People on aircraft, ships, and other modes of transportation and people on oilrigs and offshore platforms should be protected to a level commensurate with the background risk associated with those activities. The risk assessment should account for potential catastrophic consequences to all exposed people, and mitigations should be implemented to ensure that the risk from catastrophic events is consistent with the allowable risk given in Subsection 3.6.3 and Subsection 3.6.4.

Scenario-specific information should be considered in providing protection against catastrophic consequences. Combinations of factors that should be considered include the number of people who may be simultaneously injured, the risk of damage to high-value assets, 2-3 the risk of a casualty, factors that may significantly impair the range’s ability to perform its mission, and factors that may have national or international consequences.

Transportation systems include all modes of transportation such as airplanes, ships, trains, busses, and automobiles. People in transportation systems must be categorized following the same rules that apply to unsheltered people and people in fixed shelters (i.e., MEP, COP, or GP). Each individual in a transportation system must be protected to the level for his or her population category (MEP, COP, or GP). Collective risk must be assessed to include people in transportation systems. The collective risk to people in transportation systems must be added to the collective risk for unsheltered people and the collective risk for sheltered people. The numerical value defining the acceptability of the total collective risk must be based on the population category. Additional protection, such as defined in Section 3.6, must be applied to people in transportation systems in order to minimize the potential for catastrophic risk.

2.2.5 Spacecraft

Orbiting manned spacecraft will be protected to a level equivalent to that provided to mission-essential aircraft. When the planned missions involve vehicles or propagated hazards with altitude capability greater than 150 kilometers (km), ranges should coordinate with the Joint Space Operations Squadron (JSpOC) for conjunction assessment if needed. In addition, ranges should establish collision avoidance (COLA) periods in the launch window if there are any manned spacecraft within 50 km of, or lower than, the 3-sigma altitude capability of the launch vehicle, payloads, jettisoned objects, or debris cloud boundary.

The RC recognizes that protection for critical unmanned space systems should also be provided because orbital debris mitigation is a national policy goal. The most recent National Space Policy of the United States (dated June 28, 2010) states that “the United States will pursue the following goals in its national space programs …strengthening measures to mitigate orbital debris.”5 While DoD Directive (DoDD) 3100.106 states that DoD activities are to be conducted “…in a safe and responsible manner that protects space systems…,” it does not specify which DoD activity, the launching agency (range user) or the range, has the responsibility for assuring this protection.7 In the absence of clear direction, the RC recommends that the launching agencies and the ranges adopt the provisional standards specified in Section 3.5. In the case of the Air Force, the roles and responsibilities for COLA are documented in AFI 91-217.8

5 Office of the President, United States of America. National Space Policy of the United States of America. 28 June, 2010. May be superseded by update. Retrieved 7 April 2016. Available at https://www.whitehouse.gov/sites/default/files/national_space_policy_6-28-10.pdf.

6 Department of Defense. “Subject: Space Policy.” DoDD 3100.10. 18 October 2012. May be superseded by update. Retrieved 6 April 2016. Available at http://www.dtic.mil/whs/directives/corres/pdf/310010p.pdf.

7 Historically, protection of unmanned systems has been addressed as part of mission assurance by the launching agency but only for DoD missions or highly valued NASA missions. In the commercial space industry, the launch operator must obtain liability insurance to cover such potential mishaps and has historically not required conjunction assessments for mission assurance or unmanned asset protection purposes.

8 Department of the Air Force. “Space Safety and Mishap Prevention Program.” AFI 91-217. 11 April 2014. May be superseded by update. Retrieved 11 May 2016. Available at http://static.e-publishing.af.mil/production/1/af_se/publication/afi91-217/afi91-217.pdf.

https://www.whitehouse.gov/sites/default/files/national_space_policy_6-28-10.pdf http://www.dtic.mil/whs/directives/corres/pdf/310010p.pdf http://static.e-publishing.af.mil/production/1/af_se/publication/afi91-217/afi91-217.pdf

2-4

2.2.6 Critical Assets

Damage to range resources can have public safety and mission assurance implications.

As a minimum, the decision authority should protect resources that could increase the risk to surrounding population centers if they were damaged or not available in the event of an emergency. Additionally, since national security is dependent on range resources, the decision authority should consider the protection of selected facilities and equipment necessary to conduct the range’s mission.

In this standard, critical assets include property that is essential to protect the public health and safety, maintain the minimum operations of the range, or protect the national security or foreign policy interests of the United States. It includes property/infrastructure that must remain operational following a mishap, such as range facilities and equipment, as well as hospitals, fire stations, and power plants/substations.

The criteria for protecting critical assets depend on the consequences of an impact. It can be influenced by the importance of the critical asset in terms of emergency response or continued range operations, the down time and cost of repairing the critical asset, and other considerations.

Therefore, the individual ranges need to establish the protection criteria to meet the policy objective for their particular situation. Sample criteria are included in Chapter 4 of the supplement. The ranges may need to modify the criteria to fit the conditions at their particular range.

2.2.7 Environment

As part of environmental documentation preparation in compliance with federal and local regulations, launch and flight test hazards should be taken into account and mitigated as necessary. While safety is a factor in environmental compliance, environmental protection and regulation are beyond the scope of this standard.

2.3 Risk Management Process

Risk management is a systematic and logical process to identify hazards and control the risk they pose. This process should include the following elements (phases) described in Figure 2-1. Table 2-1 lists the phases and the subsections of this document that provide more detail for each.

2-5

Figure 2-1. Risk Management Process

Table 2-1. Four Phases of Risk Management Process Phase Title Subsection I Mission Definition and Hazard Identification 2.3.1 II Risk Assessment 2.3.2 III Criteria Comparison and Risk Reduction 2.3.3 IV Risk Acceptance 2.3.4

The initial goals of the risk management approach are to contain the hazards and isolate them from populated areas wherever practical. An alternative to hazard isolation is to define hazard containment areas so as to minimize the population exposed or be able to evacuate persons not associated with the hazard-generating event. This is in accordance with the primary policy that no hazardous condition is acceptable if mission objectives can be attained from a safer approach, methodology, or position (i.e., minimizing the hazards and conducting the mission as safely as reasonably possible). When hazards cannot be contained or minimized to an insignificant level, refined assessments are performed to determine if the remaining risks are acceptable. An additional benefit of hazard containment is that this process is typically less costly than risk assessments and can be evaluated relatively quickly with straightforward assumptions and with less required data.

2.3.1 Phase I: Mission Definition and Hazard Identification

Phase I is the “problem definition” step of the process. Information is assembled to identify mission characteristics, objectives, and constraints. Potential hazard sources must be identified by evaluating the system to be flown and the range safety constraints. Information sources typically include:

a. range safety data packages;

2-6

b. system description documents;

c. MEP and COP locations;

d. surrounding population data to include public and commercial facilities and transportation assets (including aircraft corridors and shipping lanes);

e. seasonal meteorological data;

f. the range safety system used;

g. lessons learned on similar missions.

Further details of information sources are in Chapter 2 and Chapter 7 of the supplement to this document. The output of this step provides a basis for hazard analysis and risk assessment and for use in evaluating options for mitigating risks in ways that will minimize adverse mission impact.

2.3.2 Phase II: Risk Assessment

This step provides information needed to determine whether further risk reduction measures are necessary. Risk levels for identified hazards are expressed using qualitative and quantitative methods. This step produces basic measures of the risks posed by hazards. These hazards include inert, explosive, and flammable debris dispersions, explosive overpressure fields, exposure to toxic substances, and exposure to ionizing and non-ionizing radiation. In some cases, this step will provide sufficient information to support the decision making without further analysis.

A valid risk assessment must account for all potential hazards posed by the range activity to personnel, facilities, and other assets. The assessment must be based on accurate data, scientific principles, and an application of appropriate mathematics. The assessment must be consistent with the range safety control that is planned for the mission. Valid calculations to assess risk can be made using the methods presented in the supplement. These typically produce conservative estimates (i.e., they produce a scientifically plausible result that characteristically overestimates risk given existing uncertainties). In all cases, the analyst is responsible for ensuring that the application of the methods in the supplement produces reasonable results. This assessment leads to mitigation measures needed to protect individuals and groups of people.

This topic is discussed more fully in Chapter 3.

In general, risk is expressed as the product of the probability of occurrence of an event and the consequences of that event. Total risk is the combination of the products, over all possible events, of the probability of each event and its associated consequence. The probability of an event is always between zero and one; however, the consequences of that event can be any value. Risk can be relatively high if the probability is high, or the consequence is great, or a combination of the two.

Simple risk models are often employed to make an initial determination of risk. They are also used when the identified hazards are known to result in low risks and the analyst is assured that the estimated risk is conservative. For example, simple models can be used when only inert debris occurs and the debris is fairly limited in size and weight with relatively low values of kinetic energy or ballistic coefficients and shelters would provide protection from debris. These

2-7 models are generally less costly, minimize schedule impacts, and have the following characteristics.

a. Simplified application of input parameters and assumptions.

b. Simplified measures of population estimation utilized.

c. A basic injury model and associated casualty areas.

d. Conservative assumptions of debris fragmentation and survivability.

If the resulting risk estimate is conservative and well within acceptable limits, then models that are more costly and time-consuming, more complex, or of higher fidelity will not be necessary.

When the identified hazards are significant or the initial risk estimate shows that acceptance criteria are, or may be, exceeded, then more complex risk models are typically used.

Use of these models may be more costly, be time-consuming to execute, and require a higher fidelity and more sophisticated application of input data and assumptions. The assessment may require detailed population and sheltering models, more complex human vulnerability models, and more realistic debris fragmentation and survivability models. This may require input parameters and assumptions to be supported by empirical evidence or expert elicitation and quantification of the impact of uncertainty in the risk model and model parameters. Complex risk assessment models are typically used when significant size debris or explosive debris impacts are present that could compromise shelters and the associated population.

2.3.3 Phase III: Criteria Comparison and Risk Reduction

Risk measures are compared with criteria to determine the need or desirability for risk reduction. If the risk is initially unacceptable, measures should be considered to eliminate or mitigate it. Elimination is achieved by design or system changes that remove the hazard source, such as replacing a hazardous material with a non-hazardous one or moving a trajectory to achieve containment. Mitigation is achieved by reducing the consequences of an event or the probability of an event happening. For example, increasing system reliability of a launch vehicle or test article will increase the probability of success, thereby lowering risk. Alternatively, designing a mission to avoid flight over densely populated areas will decrease consequences of casualties and thereby reduce the risk. Mitigation measures may include elements in the operation plan that reduce risk and are consistent with operational objectives, flight termination systems (FTSs), containment policies, evacuation, sheltering, and other measures to protect assets from the hazards. Flight termination criteria and mission rules should be reviewed to assure that the risks induced, should they be exercised, are tolerable. To evaluate the effectiveness of mitigation measures, risk must be reassessed assuming they have been implemented. These risk reduction procedures should be followed until risk levels are as low as reasonably practical.9 See, for example, the waiver of debris containment requirements for

9 Federal Aviation Administration. “Waiver of Debris Containment Requirements for Launch.” Notice of Waiver.

Federal Register, Vol. 81, No. 7. 12 January 2016. Available at https://www.federalregister.gov/articles/2016/01/12/2016-00444/waiver-of-debris-containment-requirements-for-launch.

https://www.federalregister.gov/articles/2016/01/12/2016-00444/waiver-of-debris-containment-requirements-for-launch https://www.federalregister.gov/articles/2016/01/12/2016-00444/waiver-of-debris-containment-requirements-for-launch

2-8 launch granted by the Federal Aviation Administration (FAA) as documented in the Federal Register.

2.3.4 Phase IV: Risk Acceptance

Presentations to the decision authority must be sufficient to support an informed decision.

The presentations should include all range-mandated risk control measures, residual risks, measures of catastrophic loss potential (such as collective risk given a flight termination action, collective risk given failure of an FTS, and risk profiles), key analysis assumptions, the size and impact of uncertainties in the analysis, and the protective measures that have been considered and implemented and their effectiveness. The decision authority must approve proposed mission rules and should compare the operational risk to the criteria defined in this document and to other applicable mission requirements. When local agreements are in place and the range has adequately communicated the content and rationale of this document to the representatives of local government, local agreements should govern. This shall not be interpreted as overriding any federal or state laws or regulations. The three-tiered hierarchy of requirements is:

a. Federal and state laws and regulations;

b. Local agreements;

c. This document.

In general, higher-risk operations require a higher level of approval. The range commander may tolerate risk levels within criteria given herein to secure certain benefits from a range activity with the confidence that the risk is properly managed and consistent with best practices. The outcome of these presentations to the decision authority is the acceptance of operational risks by a properly informed decision authority. This acceptance includes a determination that the residual operational risk is within tolerable limits. By doing so, it avers/ justifies that the proposed conditions for allowing the operation to be initiated and the rules to allow the mission to continue to completion comply with best practices for ensuring that the risk is less than accepted/tolerable levels.

The terms of this acceptance and required implementation conditions must be documented. The responsible safety office should document a risk assessment to demonstrate compliance with the risk management policy applied.

2.3.5 Conditional Risk Management

Flight termination action is a good example of a risk-mitigating safety intervention that induces a conditional risk that should be managed. A conditional risk management process should be implemented to assure that mission rules and flight termination criteria do not induce unacceptable levels of risk when they are implemented.10 This review process is outlined in the supplement in Chapter 2. Figure 2-2 shows the relationship between the risk management process just outlined and the conditional risk management process.

10 The FAA referenced this in the waiver of debris containment requirements for launch it as documented in the Federal Register Vol. 81, No. 7, 12 January 2016, p.1470

2-9

Figure 2-2. Relationship Between Conditional Risk Management and

Basic Risk Management

2.4 Uncertainty in the Computed Risk

The RSG RC recognizes that there is significant uncertainty in the computed risks of rocket launches. Confidence bounds of 90% describing the uncertainty in the computed risk can have a range of several orders of magnitude. For this reason, uncertainty cannot be ignored;

however, it is not necessary to deal with uncertainty if the estimate of the expected casualties (EC) or expected fatalities (EF) is so small that even with a large uncertainty the true risk is unlikely to challenge the acceptability criterion. The RC has introduced a process whereby the uncertainty does not have to be considered if the computed risk is less than one-third of the primary aggregated collective risk criterion.11 On the other hand, if the risk level does not pass this test, the range should compute the uncertainty to assure that a launch is not allowed that would violate the criterion based on best estimates that account for uncertainty. The decision process regarding risk acceptability under uncertainty is shown in the flowchart in Figure 2-3.

11 The choice of 1/3 is consistent with the recommendation in AFSPCMAN 91-710VI, 1 July 2004, which uses 30×10-6 as the acceptability criterion for casualty expectation. In the range between 10 and 30×10−6 (equivalent to one third of the risk criterion), AFSPCMAN 91-710V1, Attachment 4, page 11 states “A4.4.2.2. Hazard Risks >10 through 30 in 1,000,000 (EC > 10×10−6 through 30×10−6). This level of risk may require the Range User to take additional measures to protect personnel and resources. Examples include fix/ correct/improve existing noncompliances, improve risk analyses to reduce the level of uncertainty, require day-of-launch risk analyze, establish disaster aversion criteria. Range Safety is the approval authority for risks >10 through 30 in 1,000,000 (EC > 10×10−6 through 30×10−6).”

2-10

Figure 2-3. Flow Diagram for the Launch Risk Acceptability Process

Considering Uncertainty

3-1

CHAPTER 3

Acceptable Risk Criteria

This chapter defines acceptable risk criteria for people, aircraft, ships, and spacecraft that may be exposed to hazards associated with range flight operations. Hazard thresholds for people, aircraft, and ships are provided in Chapter 6 of the supplement.

There are two major components of the risk acceptability criteria: a set of performance standards for establishing and implementing appropriate risk criteria at a range, followed by a set of quantitative standards. The quantitative risk criteria contained in this chapter prescribe limits on a per-mission and an annual basis. The per-mission requirements are intended to apply separately to launch and reentry missions as defined in the glossary. Chapter 4 of the supplement provides guidelines for establishing a risk budget for complex missions, such as those that involve multiple launches or distinct phases of flight. Chapter 4 also provides guidelines for implementation of these criteria, including annual risk management, catastrophe aversion, and protection of ships, aircraft, and manned spacecraft.

3.1 Performance Standards

Each range must perform the following.

a. Assess the risk to all people from launch and reentry activities in terms of hazard severity and mishap probability. Note: Hazardous operations that can be contained within a controlled area may not require a risk assessment.

b. Estimate12 the EC associated with each activity that falls within the scope of this document. Additional risk measurements may be useful for range operations that are dominated by fatality to ensure fatality risks do not exceed acceptable limits.

c. Document its measure(s) of risk and risk acceptability policy in local requirements and policy documentation.

d. Maintain documentation to demonstrate that its risk measures provide a complete and accurate assessment of the risks, to include documentation needed to demonstrate that its risk measures:

(1) clearly convey the risk for decision makers;

(2) are consistent with the measures used by other scientific or regulatory communities involved in “comparable involuntary activities” (as described in Subsection 2.2.1).

e. Estimate the risk on a per-mission basis, except under special conditions where risk management on an annual basis is justified as described below.

f. Periodically conduct a formal review to ensure that its activities and its mission risk acceptability policy are consistent with the annual risk acceptability criteria.

12 The overall process is a risk assessment, but a particular value (i.e., a point estimate) is referred to as an estimate.

3-2

3.2 Personnel Protection

3.2.1 General Public

a. Individual Risk Criteria. Individuals must not be exposed to a probability of casualty greater than 1E−6 for any single mission.13 If fatality risks are also incorporated into the risk management process, then individuals must not be exposed to a probability of fatality greater than 0.1E−6 (1E−7) for any single mission.

b. Collective Risk Criteria. Collective risk for the GP must not exceed a casualty expectation of 100E−6 (1E−4) for any single mission.13 If annual risk is measured, collective risk for the GP should not exceed a casualty expectation of 3000E−6 (3E−3) on an annual basis.14 Risk management using only an annual measure of collective risk is only justified for range operations that occur frequently and pose low15 risk on a per-mission basis. If fatality risks are also incorporated into the risk management process, then the collective risk for the GP must not exceed 30E−6 (3E−5) EF for any single mission. If risk management using only annual risks is justified, and fatality risks are also incorporated into the risk management process, then the collective risk for the GP must not exceed 1000E−6 (1E−3) EF on an annual basis.

c. Catastrophic Risk Criteria. Catastrophic risk for the GP16 should not exceed the provisional17 criteria outlined in Subsection 3.6.3.

3.2.2 Mission-Essential and Critical Operations Personnel

a. Individual Risk Criteria. Individual MEP and individual COP must not be exposed to a probability of casualty greater than 10E−6 (1E−5) for any single mission. If fatality risks are also incorporated into the risk management process, then individual MEP and COP must not be exposed to a probability of fatality greater than 1E−6 for any single mission.

b. Collective Risk Criteria. Collective risk for MEP and COP must not exceed a casualty expectation of 300E−6 (3E−4) for any single mission. If annual risk is measured, collective risk for MEP and COP must not exceed a casualty expectation of 30000E−6 (3E−2) on an annual basis.14 Risk management using only an annual measure of collective risk is only justified for range operations that occur frequently and pose low15 risk on a per-mission basis. If fatality risks are also incorporated into the risk management process, then collective risk for MEP and COP must not exceed an expected number of fatalities of 300E−6 (3E−4) for any single mission. If risk management using only annual risks is justified, and fatality risks are also incorporated into the risk management process, then the collective risk for MEP and COP should not exceed 10000E−6 (1E−2) EF on an annual basis.

13 If a flight operation creates a toxic risk, then the range must separately ensure the allowable level of risk enforced by them does not exceed other standards for toxic exposure limits for the public when appropriate mitigations are in place.