2017_DHS_outage.pdf

Tracking Network Outages

Henning Schulzrinne

FCC

The views and opinions expressed in this presentation are those of the author and do not necessarily reflect the official policy or position of any agency of the U.S. government. Any resemblance to actual policies, living or dead, or actual events is purely coincidental.

The FCC tool kit

Disclosure Industry best practices

Rules Outreach

CSRIC

47 CFR Part 4 communications 9-1-1

MBA

9-1-1 post-incident reports

FCC measurement history FCC has acquired and analyzed data on legacy PSTN More recent and evolving broadband interest

Section 706 of 1996 Telecommunications Act annual report on availability of advanced telecommunications services to all Americans Resulted in information on deployment of broadband technology

(“Form 477”) but not its performance

FCC’s National Broadband Plan – March 2010 Proposed performance measurements of broadband services delivered to consumer households Work plan evolved from recommendations of National Broadband

Plan

The role of network measurements

Measurement infrastructure

ISP

diagnostics

“my Interwebs are just beach balls”

User diagnostics & validation hard failures soft failures

Public policy

• BB evolution?

• Informed consumer choice

• Universal service

Measuring Broadband America:

Measurement architecture

Lucid broadband Internet access provider (ISP) backbone

ISP

Measuring Broadband America 2011 through today

Measuring Broadband America future?

MBA: What is measured Download speed Voice over IP (loss, jitter, RTT) Upload speed DNS resolution

Web browsing ICMP latency UDP latency ICMP packet loss UDP packet loss Latency under load Video streaming Consumption (up, down) data gathered continuously annual report for one month (September/October) https://www.fcc.gov/reports-research/reports/measuring-broadband-america

The MBA project - logistics Cooperative:

13 largest ISPs covering 86% of US subscribers DSL, HFC, FTTH, satellite vendors, trade groups, universities and consumer groups

Initially, enrolled roughly 9,000 consumers as participants 6,241 participants for 2016 report (measured Sep./Oct. 2015) 4,165 reporting ”white boxes”

Measurements weighted by tier population

MBA example: packet loss

0.0%

0.1%

0.2%

0.3%

0.4%

0.5%

0.6%

0.7%

0.8%

AT

&T

D

SL

AT

&T

IP

BB

C en tu ry

Li nk

Fr on tie r D

SL

Ve riz on D

SL

W in ds tre am

O pt im um

C ha rte r

C om ca st

C ox

M ed ia co m

TW

C

Fr on tie r F ib er

Ve riz on F ib er

H ug he s

Vi aS at /E xe de

DSL Cable Fiber Satellite

Pe rc en ta ge P ac ke t L os s

MBA: from wireline to mobile Smartphone app runs in background on volunteers phone Runs automatically at random times performing measurements Only gathers network related data and anonymizes results to protect privacy Client software provided as open source

4 major carriers (AT&T, Sprint, T-Mobile, Verizon)

Gathering data since August 2013, but data not yet public 9

Using MBA for reliability assessment?

Data currently not available in real time During disasters, need to distinguish power from broadband outages Limited geographic resolution

Example: AT&T 9-1-1 outage on March 8, 2017

Overview of NORS and DIRS

Cybersecurity and Communications Reliability Division

Public Safety & Homeland Security Bureau

Jeffery Goldthorp

Purpose of Outage Reporting Improve response to a crisis provide real-time information on major communications outages during a crisis Improve communications networks reliability & security (patterns) Improve disaster recovery (lessons learned) Individual reports are confidential NORS: “normal” outages, extensive reports DIRS: large-scale disasters, brief reports

What’s Reportable* >= 30 minutes & 900,000 user minutes Mobile Switching

Center outage e.g., 30,000 subscribers for 30 minutes

Voice & data failures

SS7

Special facilities airports military, nuclear, etc.

Satellite providers 9-1-1 service providers Submarine cables

14* simplified

Reporting Process

Within 2 hours or 30 minutes (9-1-1 service provider)

For VoIP: 4 Hours (911) and 24 hours (all others)

Within 3 Days

Within 30 Days

File Notification

Outage Occurs

Submit Initial Report

Submit Final

Report

DHS Notified of Outage

Step 1

Step 2

Step 3

File Notification

Outage Occurs

Submit Initial Report

Submit Final

Report

DHS Notified of Outage

Step 1

Step 2

Step 3

NORS for Situational Awareness:

Example Notification

Disaster Information Reporting System

Information input to DIRS

Contact information Information on network status

Switches Cell Site by County AM, FM, TV Stations

CATV

PSAPS

DIRS: Web-based system which collects Information on status of communications assets during disasters Daily updates Consistent data

Output

Maps

Charts

Cell Sites Operational

0%

20%

40%

60%

80%

100%

6/2 3/2

6/2 4/2

6/2 5/2

6/2 6/2

6/2 7/2

Disaster Recovery & Reporting Department of Homeland Secusity:

Overall Coordination: Coordinate the planning for and provision of national security and emergency preparedness communications for the Federal government during disasters and emergencies.

Federal Communications Commission (FCC):

Situation Awareness Data Collection: Collect network status and restoration data. Develop charts, tables and maps summarizing the information.

Disaster Reporting Process

Company Activity

FCC Activity

Disaster Data Collection Activated

Companies Access DIRS

Companies Input Data

FCC Generates Tables & Input for SitRep

Generate Charts, Maps

DHS Writes SitRep

Send inputs to White House/JTRB

Disaster Data Collection Activated

FCC Notifies CompaniesFCC Notifies Companies

DHS Activity

Done Daily

Chart from Sitrep - Sandy

Number of PSAPs Affected by Sandy

0 0 0 0 0 0

4 4

3 3

15 14

23 23

16 16

10/30 10/31 11/1 11/2 11/3 11/4 11/5

911 Calls Lost

Rerouted W/O ALI

Rerouted With ALI

Operational - No ALI

DIRS – Mapping Examples (Sandy)

Conclusion FCC key federal agency for network reliability in cooperation with state utility commissions

MBA: currently, for “science” analysis not real time, limited geographic coverage public data

NORS & DIRS situational awareness + FCC long-term analysis mostly confidential