B08_140G0218R0016_Attachment_5_ANSS_System_Overview.docx

Department of Interior

Technical Description of Operations and Infrastructure

140G0218R0016 Attachment 5 ANSS System Overview140G0218R0016 Attachment 5 ANSS System Overview

GENERAL DESCRIPTION/PURPOSE

The mission of the ANSS is to provide accurate and timely data and information products for seismic events, including their effects on buildings and structures, employing modern monitoring methods and technologies. This mission serves a basic function of the National Earthquake Hazards Reduction Program (NEHRP) and drives the four basic goals of the planned system:

· Establish and maintain an advanced infrastructure for seismic monitoring throughout the United States that operates with high performance standards, gathers critical technical data, and effectively provides information products and services to meet the Nation's needs. An Advanced National Seismic System should consist of modern seismographs, communication networks, data processing centers, and well-trained personnel; such an integrated system would constantly record and analyze seismic data and provide timely and reliable information on earthquakes and other seismic disturbances.

· Continuously monitor earthquakes and other seismic disturbances throughout the United States, including earthquakes that may cause a tsunami or precede a volcanic eruption, with special focus on regions of moderate to high hazard and risk.

· Thoroughly measure strong earthquake shaking at ground sites and in buildings and critical structures. Focus should be in urban areas and near major active fault zones to gather greatly needed data and information for reducing earthquake impacts on buildings and structures.

· Automatically broadcast information when a significant earthquake occurs, for immediate assessment of its impact. Where feasible, for sites at distance from the epicenter, broadcast an early warning seconds before strong shaking arrives. Provide similar capabilities for automated warning and alert for tsunamis and volcanic eruptions.

To achieve these goals, ANSS will establish nationwide network of over 7000 earthquake sensor systems, serving all areas of the country subject to earthquake hazards and providing dense coverage in 26 at-risk urban areas (Figure 1). Sensors will be located both in the ground and in buildings and other structures.

FIGURE 1 AT-RISK URBAN AREAS OF THE UNITED STATES.

The system will provide real-time earthquake information for emergency response personnel, provide engineers with information about building and site response to strong shaking, and provide scientists with high-quality data needed to understand earthquake processes and structure and dynamics of the solid earth.

As part of its monitoring activities, the ANSS includes a national Backbone network, the National Earthquake Information Center (NEIC), the National Strong Motion Project, and 15 regional seismic networks operated by USGS and its partners. When earthquakes strike, ANSS delivers real-time information, providing situational awareness for emergency-response personnel. In regions with sufficient seismic stations, that information includes, within minutes, a seismic event notification sent out over the Earthquake Notification Service (ENS), a ShakeMap showing the distribution of potentially damaging ground shaking, a PAGER Map showing population exposure and damage estimates, and a suite of informational pages on the World Wide Web. All this information is used to target post-earthquake response efforts, as well as to inform the public. When fully implemented, ANSS will provide such dense station coverage for all at-risk urban areas. Information from ANSS are key inputs to the USGS National Seismic Hazard Maps, which help communities in earthquake-prone regions develop safer building practices.

The ANSS Information System supports the mission of the ANSS.

At the core of the ANSS are two legacy operations, the U.S. National Seismographic Network (USNSN) and the NEIC. Together, these operations are responsible for detecting and reporting on earthquakes anywhere in the continental United States. With only small seed funding to date, the ANSS effort has focused on upgrading the legacy components underlying these operations and positioning them as the national center for earthquake monitoring. Accordingly, the USNSN has been renamed the ANSS Backbone and is currently being expanded from approximately 55 stations to 85 stations, with an ultimate goal of 100 stations. Similarly, modernization efforts are underway to evolve the NEIC into the ANSS National Operations Center (NOC). As additional ANSS funding becomes available, regional and international monitoring networks will be brought in under the ANSS umbrella. Figure 1, ANSS Components, depicts the major components and overall architecture.

FIGURE 2 ANSS COMPONENTS

INFORMATION SYSTEM ENVIRONMENT

PHYSICAL INFRASTRUCTURE

The system is housed at the following facilities:

· USGS National Earthquake Information Center, Golden, Colorado (primary location)

· Albuquerque Seismological Laboratory, Albuquerque, New Mexico

· USGS Menlo Park, California

· USGS Pasadena, California

· Caltech Seismological Lab, Pasadena, California

· Denver Federal Center, Building 25, Room 1846; Building 53 BLM Data Center

· EROS Datacenter, Sioux Falls, South Dakota

The National Earthquake Information Center (NEIC) in Golden houses the primary acquisition and processing infrastructure as well as servers that handle data distribution to select cooperating organizations. The USGS occupies the entire building, and access into the building is restricted by a federally controlled card key lock system, and unrestricted public access is prohibited. The ANSS servers, along with telecommunications equipment and networking gear are kept in a secure, state-of-the art data center. Access to the room is granted through a separately maintained card-key lock system (Lenel On- Guard). All critical equipment are on enterprise UPS’s housed within the computer room and an automatically triggered backup generator that is tested weekly.

The Albuquerque Seismological Laboratory (ASL) is the USGS’s installation and maintenance depot for the Global Seismic Network (GSN) and for the ANSS backbone. The facility is accessible only through the Kirtland Air Force Base with no unmonitored public access. As in Golden, the equipment at ASL is stored in a locked room with restricted access. System components are connected to a UPS and backup generator.

The Pasadena facilities house some of the computing infrastructure of the ENS, and for Legacy EIDS system. These provide backup/contingency roles to servers at the other sites. As with the Golden and Albuquerque facilities, the equipment is located in locked rooms with restricted access. Generators provide backup power at both facilities and critical computing systems are on small UPS units to provide continuity between power failure and generator startup.

The Menlo Park facility houses components of the Did You Feel It?, ENS, Legacy EIDS, and PDL systems, including backup and contingency. All ANSS equipment is located in a locked computer room with restricted access.

The USGS Data Center in Building 25 at the Denver Federal Center houses the servers and network connections comprising the ANSS contingency system. The building is located on the restricted Denver Federal Center Campus, and access to the facility is restricted by a federally controlled card key lock system.

The BLM Data Center in Building 53 at the Denver Federal Center houses the infrastructure that supports the public website, PDL system, and public data distribution servers. The building is located on the restricted Denver Federal Center Campus, and access to the facility is restricted by a federally controlled card key lock system.

The EROS Data Center in Sioux Falls hosts the backup/contingency public website infrastructure. The facility is located on a federally managed campus, and access to the facility is restricted by a federally controlled card key lock system.

COMPUTER NETWORKING ARCHITECTURE

ANSS systems at the various geographical sites are mutually interconnected across the Internet. Public- facing servers are within isolated security zones (DMZ’s).

At the NEIC, where most of the system resides, each server or workstation is located within one of several security zones behind the Internet firewall. The zones are organized in a logical ring structure, ranging from the innermost rings containing the most sensitive systems to the outermost rings containing the most exposed services.

Some of the internal (non-DMZ) servers are dual or multi-homed onto other networks. Some of these are totally internal and private to the NEIC. Others are on private dedicated telecommunications circuits with drop points in the NEIC telecommunications room. A dedicated lines firewall protects ANSS systems from all such circuits. Data from the ANSS Backbone Seismic Network are received by way of satellite feeds to two network operations centers (NOCs) of a satellite provider. Ordinarily, the data are sent from the NOCs to the NEIC by way of a land-based VPN entering through the Internet firewall. In the event of loss of Internet service, these tail segments can be re-routed by way of secondary satellite hops.

Two VPNs over Internet, one at the NEIC, the other at ASL, are used to exchange data with one of the cooperating organizations (CTBTO).

HARDWARE AND SOFTWARE SECURITY INFRASTRUCTURE

Presently, the entire legacy system is protected by one or more of the enterprise security systems outlined below. The new system consists of many interconnected computers operating Agency-developed seismic acquisition, monitoring, processing, and analytical software. The security controls protecting system resources are implemented in a hierarchical manner. Software and hardware in use at the Department or Bureau level to help create a secure operating environment includes, but is not limited to:

· DOI Enterprise Services Network (ESN) “Gauntlet”, which includes:

· Hardware firewall and DOI policies

· ActiveScout Intrusion Detection and Prevention (IDP)

· Juniper IDP (See “Development Phase” below for more information)

· DOI Enterprise Remote Access (eRAS)

· USGS Enterprise Vulnerability Management System eVMS.

· Symantec Anti-virus

· USGS IT Security Operations Team (ITSOT) Firewall Change Request website

Additional security software implemented on site:

· Deny-all-allow-permit by exception irewall policies and rule at all sites

· Secure Socket Layer (SSL) protocol for remote access into the NEIC

· Secure shell (SSH) protocol. Two factor authentication using SSH DSA key technology is enforced for alternate remote access into the NEIC

· TCPwrappers (IP access restriction)

· Enterprise Security Manager (ESM)

· Microsoft Active Directory for an internal domain

· Windows Server Update Services (WSUS)

· DOI Federal Desktop Core Configuration (FDCC) STIGs

· Audit trail logging using UNIX/Linux syslog and Windows Snare Modems are not used for access into ANSS security zones.

The USGS regional email servers, along with their security measures, are used for email services.

The ANSS has setup a secondary slave Domain Name Service (DNS) server, named ”gldsurvey”, to help alleviate DNS related problems for critical ANSS systems. During DOI upgrades, maintenance, or outages of their DNS servers, the VMS systems experienced many problems and outages due to slow or non- existence name resolution. Having a slave name server on the local ANSS network allows for quicker name resolution and guarantees stability during DOI upgrades maintenance or outages to their DNS servers. All DNS zone changes are still controlled by the DOI because the server is setup in a slave configuration.

COMPUTING PLATFORMS

The ANSS has adopted a “platformity” standard, based on using Red Hat Enterprise Linux (RHEL) and Microsoft Windows on Intel hardware. A small number of servers running legacy operating systems VMS, Solaris, and FreeBSD, are in process of being converted to either RHEL or Windows. Some servers are clustered in mutually interdependent configurations.

APPLICATIONS COMPRISING THE INFORMATION SYSTEM

The ANSS system, as designed, consists of eight application areas:

1. Real-time data acquisition system (DAS)

2. Database Management Systems (DBMS) and DBMS-associated middleware

3. Real-time processing

4. Analysis

5. Rapid response production

6. Web services

7. Dedicated real-time and by-request exports

8. System monitoring and response.

The real-time DAS receives real-time global seismic sensor data and disseminates automatic estimates of earthquake locations and magnitudes. The modules within this application are designed to perform the following general functions

· Managing the IP connections to the various data sources

· Automatically verify link integrity and data flows

· Perform re-connection attempts as required.

· Writing the incoming sensor data to continuous waveform buffers for the dedicated export applications

· Staging the incoming sensor data to the DBMS for real-time processing

The DBMS system receives

· Seismic sensor data from the real-time DAS for temporary storage during processing

· Parametric data concerning automatic earthquake estimates from real-time processing

· Analytical earthquake results from analysis

The modules of the real-time processing system:

· Retrieve sensor data from the DBMS

· Perform seismic phase-arrival determination on those data. These algorithms attempt to determine the time of arrival of seismic waves at the sensors

· Perform event association-- Given the above arrival times, compute the location and origin time of the seismic source that produced the observed arrivals

· Compute magnitude estimates-- Execute various algorithms that provide estimates of the magnitude of the observed seismic event

· Insert these results into the DBMS.

· Issue automatic earthquake alarms to specified users

From the available data, the processing system will produce human-reviewed earthquake notifications and processed seismic events for insertion into an archiving system. Its functions include:

· Running redundant Oracle DBMS instances that stage sensor data for the real-time processing

· State-driven processing of earthquakes, wherein the processing steps executed on an earthquake are determined by the nature of the earthquake

· Supporting the DBMS Application Programming Interface (API) between the application modules and the DBMS tables.

· Altering the processing of an earthquake as directed by the human's interactions with an evolving seismic event

· Running redundant Oracle DBMS instances for temporary storage of automated and analyzed earthquake parameters

· Running various State of Health (SOH) monitors to detect software and hardware malfunctions and exception conditions

The analysis system receives earthquake parameters from the DBMS system and commands from human analysts. It inserts into the DBMS human-directed earthquake parameters and human-specified processing directives. Its functions include:

· Supporting various displays of automatically computed earthquake data

· Accepting user interactions with such displays

· Executing the processing steps implied by such interactions

· Inserting re-processed earthquake data into the DBMS

· Inserting modified processing directives into the DBMS to provide adaptive behavior of the state- driven processing.

Rapid response production involves software developed through the USGS to aid emergency responders, decision makers, and researchers in assessing the extent of ground shaking, fatalities and casualties, and infrastructure damage likely to have resulted from a major earthquake. This system runs and disseminates products as soon as initial estimates on the location and magnitude of the earthquake come out of the real-time processing system. Its results are repeatedly revised and updated, both automatically and through review by experienced seismologists, as more data and improved parameters on the earthquake become available. Components of the rapid response system include:

· “ShakeMaps” of estimated ground shaking (seismic intensity) levels surrounding the epicenter

· Customizable reports of estimated damage and losses in specific geographic areas, based on shaking intensity and known information about infrastructure and population densities

· CIIM, also called DYFI. These applications solicit information about an earthquake through the Web from the general public and use it to help improve the other rapid response products.

The Web interfaces use in-house developed sites running on off-the-shelf software to disseminate information about significant earthquakes to the general public. This information includes the earthquake solutions and rapid response products described above as well as other more research-oriented products. The system provides robust availability in times of high user response by taking advantage of commercially owned (Level 3) server farms feeding off of a base of geographically redundant USGS servers.

Several servers at the NEIC use USGS developed applications to provide continuous or by request exports of seismic waveform (sensor) data and “metadata” (sensor parameters) to a number of cooperating organizations. Most of those organizations reciprocate by contributing data from their own seismic networks to the real-time processing system at the NEIC. Connections are firewall- and/or software- restricted to specific IP addresses on specific non-standard ports. Applications include:

· “Earthworm” for continuous exports to the National Oceanic and Atmospheric Administration (NOAA) Tsunami Warning Centers and regional seismic networks and for by-request exports to more general users

· “Ring Replicator Protocol” (RRP) for continuous exports over private circuit and Internet to multiple cooperators

· “Continuous Waveform Buffers” for by-request exports to various organizations, particularly researchers

The monitoring and response system runs at the NEIC to monitor ANSS state of health and alert IT support staff to system failures on a 24x7 basis. It uses both open source and custom software developed at the facility. The open source products consist of:

· Icinga - an automated monitoring and alerting application (http://www.icinga.org/)

· Trac - a bug/issue tracking system, used for trouble-ticket management (http://trac.edgewall.org/)

The in-house software is referred to as the Alarm Sub-system. It provides a front end to Icinga for the real- time DAS servers.

National Earthquake Information Center Systems Overview and Integration Overview The primary mission of the U.S. Geological Survey (USGS) National Earthquake Information Center (NEIC) is comprehensive global earthquake monitoring (M4.5 or larger) and complete seismic monitoring of the United States for all significant earthquakes (M3.0 or larger or felt). In recent years, the NEIC has assumed a more prominent role in local and regional seismic monitoring, backup capabilities, and special studies. The NEIC uses numerous scientific algorithms to produce information products for local, national, and international disaster responders; infrastructure corporations (for example, utility providers and insurers); U.S. embassies worldwide; news agencies; national and international earth science research communities; and millions of public users.

To support this mission, the NEIC operates a constant service to derive numerous information products as rapidly and accurately as possible. Derived information products and data sets provide a wealth of related earthquake information and include products such as location, magnitude, maps of shaking intensity, moment tensors, phase arrivals, and loss and damage reports. These information products are automatically derived in near real-time and refined over time. Earthquake information products vary from well formatted text files, to binary files, to directories full of many files. A team of analysts, who provide constant coverage, 24 hours a day, seven days a week, 365 days a year, review the acquired data and automatically derived products (1) in near real-time for rapid release and (2) in the weeks following events to produce historical catalogs of global earthquakes in support of scientific research. Data and products are reviewed both in near real-time for rapid release and in the weeks following events to produce historical catalogs of global earthquakes in support of scientific research. At the present time, the NEIC publishes data for approximately 25,000 earthquakes annually.

The NEIC software system that supports this service and mission comprises many subsystems. The internals of each subsystem are often complex, but the integration between all of the subsystems is well defined, standardized, and straightforward. This Open-File Report describes the systems’ integration infrastructure and how all the subsystems fit together. It provides summary views of the system as a whole, along with detailed views of the inputs, outputs, and dependencies for each subsystem. It also outlines the many shared data services that support these subsystems and applications as well as external customers. The NEIC systems- integration effort is ongoing. Additional integration steps, needed to reach our goals, are discussed.

It is important to note that this document provides a brief introduction to the work of dozens of software developers and IT specialists, spanning in many cases more than a decade. References to significant amounts of supporting documentation, code, and information are supplied within.

Motivation Well-executed systems integration is critical in making communications between applications or subsystems as robust, transparent, and easy as possible. However, systems integration is difficult and is often the most likely point of failure in operational systems. Systems integration is about adding value to the whole system by supporting capabilities that are possible because of interactions and collaboration across subsystems.

Efficient and flexible systems integration helps our Science Center implement and operate more maintainable and scalable systems. Clearly defined integration supports independent development between numerous developers, as well as collaboration between internal and external teams.

Operation of the system is improved, as the failure of one piece of the system does not cause a failure of the system as a whole. Additionally, by using standardized formats and distribution mechanisms, our Science Center reduces the coupling between the subsystems, the number of points of failure, and the number of technologies supported. All of this leads to reduced complexity and improved efficiencies.

Consistency is also an important part of integrated systems. To achieve consistency, several common or shared data services are centralized and shared across the subsystems and collaborators. Examples of shared data services include waveforms via Continuous Waveform Buffers (CWB), geographic data via GeoServers, geographic information systems (GIS), station metadata via Metadata Server (MDS), aggregated and indexed catalog of all products via Comprehensive Catalog (ComCat), and real-time, well-formatted data feeds for all products. The ease of maintenance is also improved by making these common services shared, rather than having multiple and different implementations to maintain.

Systems Integration—How It All Fits Together There are three main functional components of NEIC’s system: data acquisition, processing, and presentation (fig. 1). Each functional component is composed of multiple specialized applications or subsystems, all of which communicate with, or depend on, one or many other subsystems. These dependencies are confined to shared services and shared data or products. Each component contributes to and(or) uses shared services to achieve consistency across the entire system. Figure 1 depicts a summary of NEIC inputs, outputs, and data flows that span the functional components. In this diagram and throughout the document, data acquisition subsystems are purple, data processing subsystems are blue, data presentation subsystems are orange, shared services are red, and data integration paths are green.

The numerous subsystems across the functional components work to produce products or detailed, supporting information about earthquakes. Each product uniquely tells a piece of the story for any given earthquake. The richness of the data set is significantly increased when all of the products are brought together to tell the full story. This subtle, simple, yet powerful goal is accomplished by our systems integration approach, which is described in this document.

Figure 1. National Earthquake Information Center functional components.

The details behind the summarized functional components are quite extensive. An overview will be provided for each functional component followed by summaries of the applications in that component. Application or subsystem summaries will describe the application function, inputs, outputs, dependencies, and provide references for further details. Additionally, these subsystems create numerous valuable products. In this document, the products are described relative to the role they play in integration, references and links to product details are provided for further reading.

Data Acquisition Data acquisition is composed primarily of the Edge and the Product Distribution Layer (PDL). The Edge earned its name because it is at the edge of the entire system and it interfaces with the rest of the world. It focuses on continuous waveform (or time-series) data and station metadata. The Edge handles gaps, data loss and recovery, and homogenizes all the data to make it available in standard formats for the rest of the system. The focus of PDL is acquisition and distribution of messages or discrete data, such as earthquake parametric data, and other event centric products. Also provided by PDL are data-recovery features and data-format homogenization. The essential difference between the focuses of these systems is that the

Edge works on continuous data and supporting metadata, where PDL is message oriented, for discrete data that is derived from specific events or earthquakes.

For earthquake parametric data, both the Edge and PDL subsystems can handle multiple input formats, but provide conversion to QuakeML, an international XML (eXtensible Markup Language) standard format for earthquake data (https://quake.ethz.ch/quakeml). QuakeML is the standard format for earthquake parametric data for the entire system and the majority of NEIC’s collaborators (Advance National Seismic System, 2013). Using an international, extensible, and standard format significantly increases the ease of integration and shared code for data parsing across the subsystems and with collaborators.

Edge The Edge subsystem handles vast amounts of data from many organizations in many formats and homogenizes it for simplified and efficient consumption, both within the NEIC and externally (fig. 2).

Figure 2. Edge data acquisition subsystem. (ANSS, Advanced National Seismic System; RSNs, Regional Seismic Networks; GSN, Global Seismic Network; MDS, Metadata Server; CWB, Continuous Waveform Buffer; RWS, Random Access Memory (RAM) Waveserver; TCP, Transmission Control Protocol; IRIS, Incorporated Research Institutions for Seismology; NEIC, National Earthquake Information Center; PDL, Product Distribution Layer; PGA XML, Peak Ground Acceleration in extensible markup language.)

This subsystem handles four basic kinds of input information: raw trace (continuous time-series data), station metadata (for example, coordinates and responses), unassociated pick data, and late-arriving contributed event parametric data. Parametric data refers to earthquake parameters including location, magnitude, phases, and moment tensors. For the raw time-series inputs, which come from dozens of regional and international networks, the Edge provides storage and access via multiple waveform interfaces including: Continuous Waveform Buffer (CWB), Winston Waveserver, and Earthworm Waveserver. Station metadata is stored in the Metadata Server (MDS) and available via a data service with a defined query interface. Detailed supporting documentation is available online (ftp://hazards.cr.usgs.gov/CWBQuery). The contributed parametric data comes in via email in different formats that the Edge converts to QuakeML. The Edge then distributes the data via the NEIC Private PDL hub, as these data are generally intended for NEIC consumption only. The Edge subsystem is written predominately in Java and is compatible with MySQL and Postgres.

The Edge subsystem generates peak ground acceleration products for triggered strong- motion data that is transmitted when sensor movement exceeds a pre-defined threshold.

Currently, Edge takes triggered strong-motion data and applies a simple peak ground acceleration (PGA) algorithm to produce XML-formatted data that are sent to ShakeMap via SendFile. To reduce the number of technologies supported and to simplify the overall maintenance of the NEIC system, this PGA data could be made into a product and distributed via PDL. This needs further evaluation and thus is depicted as a red arrow in the Edge Subsystem diagram (fig. 2).

Product Distribution Layer The PDL is a message-based distribution platform. It provides a standard distribution mechanism for both sending and receiving messages. For the NEIC, these messages are well- formatted data packages of derived seismic event information, also known as “products.” Example products include origins, phases, moment tensors, shaking intensity maps, and loss and damage reports. Definitions for all products can be found online (http://ehppdl1.cr.usgs.gov/userguide/products/index.html). Products are well-formatted text- file(s) (for example, XML), binary file(s), a set of directories with many files, or any combination of these. Regardless of the format, products have a set of properties, organized as key value pairs, which describe the product and make it identifiable and searchable, independent of the underlying product format.

As products are distributed via PDL, they are associated and prioritized by the Indexer.

The Indexer applies a set of business rules to associate and prioritize products, where NEIC applies network authoritative logic to assign preference, or what we call “preferredness”, to products. Preferredness is an ordered priority for data such that one product from one source is ultimately the most preferred. The network-based business rules operate on data such as source and event location. Both of these features, aggregation and preferredness, are essential to integrate and accommodate multiple products contributed from different organizations and algorithms.

Products are simultaneously distributed internally between all subsystems and externally with collaborators. As a result, collaboration is necessary on all product definitions and in the presentation of those products. This collaboration requires NEIC developers to coordinate across multiple groups, internal and external to the USGS, and with several regional, national, and international organizations. Consequently, significant documentation and testing are necessary to achieve effective integration. Detailed supporting documentation on PDL and products can be found online (http://ehppdl1.cr.usgs.gov).

A strength of PDL is that it supports loose coupling between data/product producers (senders) and consumers (receivers). All product producers and consumers only need to communicate with the PDL hubs, via PDL Product Client software. The hubs take care of the broader event association, via the Indexer, and wider data distribution via PDL. Because producers and consumers initiate connections to the hubs, most users do not need to reconfigure firewalls if their data needs or collaborators change. Producers use a private key to sign products when they are sent to the PDL Hubs; the key verifies this signature using a public key before distributing products, to ensure only that authorized producers are able to send products.

Receiving clients may optionally verify these signatures as well. Receiving clients can be configured to trigger processing when new products arrive, either using command line arguments or a Java application programming interface (API), eliminating polling latency. The PDL Product Client software supports configuration options to filter the products a receiver downloads. Conveniently, and quite significantly from a maintenance perspective, the PDL hubs, the sender PDL Product Client, and the receiver PDL Product Client software are the same code baselines, with different configurations. The PDL software is publically available for download and documented online (http://ehppdl1.cr.usgs.gov). Additional supporting libraries for parsing formatted products and programmatically searching for and downloading products are available also available online (https://github.com/USGS/devcorner).

The NEIC PDL hubs are configured to host a private/internal and a public/external data distribution stream (fig. 3). The NEIC additionally uses a relay PDL hub to download products from the public/external distribution stream once, rather than many times, for every subsystem/application that runs at the NEIC. This is an efficiency step to reduce network bandwidth consumption, which is a valuable resource at the NEIC most significantly due to raw seismic time series data acquisition and redistribution. The private PDL hub supports acquisition and internal distribution of data that is intended for and limited to the NEIC only.

Figure 3. Product distribution layer architecture. (NEIC, National Earthquake Information Center; TWC, Tsunami Warning Center; ANSS RSN’s, Advance National Seismic System Regional Seismic Networks; EIDS, Earthquake Information Distribution System; PDL, Product Distribution Layer; ComCat, Comprehensive Catalog; PAGER, Prompt Assessment of Global Earthquakes for Response; TED, Tweet Earthquake Dispatch; DYFI, “Did You Feel It?”; NEST, NEIC Executive Summary Tool; EMSC, European-Mediterranean Seismological Centre; CISN, California Integrated Seismic Network.)

In the “Product Distribution Layer Hub Architecture” diagram (fig. 3), the legacy Earthquake Information Data Exchange (EIDS) hubs are almost completely phased out, as they supported only limited text-based products; thus, the corresponding integration arrows are drawn in red. For backwards compatibility with EIDS there is support in PDL, via the EIDSInputWedge, as documented online (http://ehppdl1.cr.usgs.gov/userguide/eidsinputwedge.html).

Data Processing Component All of the NEIC data-processing subsystems, or applications, interact via products distributed over PDL (fig. 4). This infrastructure readily supports decoupling of the subsystems, concurrent processing, triggered processing and highly iterative processing. All of these capabilities contribute to a more robust and efficient system as a whole. This loose coupling allows none of the processing subsystems to depend on understanding or being directly connected to any other processing subsystem, each subsystem only needs to understand the data products for which it either consumes and(or) produces.

Figure 4. Data processing overview. (ANSS, Advanced National Seismic System; IDC, define here; ISC, define here; PAGER, Prompt Assessment of Global Earthquakes for Response; NEST, NEIC Executive Summary Tool.)

Data processing flow over time (fig. 5) demonstrates an example product flow for an event over time, showing how PDL supports triggered, highly iterative, concurrent, and loosely coupled processing. The key stages in an event’s lifecycle include:

1. Event detection and corresponding production of preliminary location and magnitude,

2. Concurrent presentation of preliminary event information and production of initial ShakeMap product,

3. The existence of a ShakeMap product triggers PAGER (Prompt Assessment of Global Earthquakes for Response) to process. Concurrently, more data are coming into Hydra and refined scientific solutions are computed (additional magnitudes, moment tensors, etc.),

4. Steps two and three are repeated for weeks as additional data continues to become available,

5. Event is reviewed, verified, and archived in the Comprehensive Catalog (ComCat) to support long term research and scientific efforts, It is relevant to note that at every step through the data flow, the products are distributed for concurrent processing and presentation to ensure product consumers (other subsystems, end users, etc.) are receiving the information as rapidly as it is being produced.

Figure 5. Data processing flow over time. (NEIC, National Earthquake Information Center; ANSS RSN, Advanced National Seismic System Regional Seismic Network; NEST, NEIC Executive Summary Tool; PAGER, Prompt Assessment of Global Earthquakes for Response; MT, Moment Tensor.)

NEIC Data Flow Over Time Web

Loss

PAGER

Shakemap Shakemap

PDL

PDL

Origins earthquake.usgs.gov

PAGER,

MT’s Origins Mags Shakemap Origins Mags earthquake.usgs.gov

PDL

Origins Mags ComCat Comprehensive Catalog Shakemap

PAGER,

MTs earthquake.usgs.gov Origins Mags Origins Mags Moment Tensors (MTs) PAGER MTs ANSS RSN’s & Intl Contributors

DYFI

Product Origins Mags ComCat Comprehensive Catalog Origins Mags Epicentral Time 0 Initial Location & Magnitude Initial PAGER & Other Scientific Products Time Initial Shakemap Product

ShakeMap

PAGER

Loss Modeling & Analysis Hydra

NEIC

Earthquake Monitoring & Analysis

ComCat Comprehensive Catalog

TED

Tweet Earthquake Dispatch Detector Hydra NEIC Earthquake Monitoring & Analysis

DYFI?

Did You Feel It?

NEST

Executive Summary

ENS

Earthquake Notification System

NEST

Executive Summary

Hydra Subsystem The Hydra subsystem provides continual real time global seismic monitoring and analysis tools, as well as late arriving data aggregation and NEIC catalog (also known as Preliminary Determination of Epicenter [PDE]) production tools. As a seismic monitoring and analysis system, its unique design supports global and regional seismology, and it aggregates solutions across many different contributors, both nationally and internationally.

The inputs into Hydra include raw waveform time series data, station metadata, region and city information, and parametric solutions (fig. 6). As with other NEIC subsystems, Hydra gets waveform time series data from Edge nodes/CWBs, station metadata from centralized NEIC station metadata servers (MDS), geographic information from NEIC geographic data services (GeoServer), and parametric solutions via PDL. Parametric solutions include origins, magnitudes, moment tensors, and phase data from both public/external and private/internal data sources. Internal data inputs include global phase arrival time picks and event detections produced by the continuous earthquake detection software, which processes waveforms in real- time. Hydra uses a single-frequency band teleseismic picking algorithm for automatic arrival time picking; this will soon be supplemented by a multiple frequency band regional / local picking algorithm. These picking algorithms can be tuned as needed to improve the detection capabilities in relevant regions. All picks are delivered in real-time to both (1) Hydra and (2) the automated nucleation and association process for automated and rapid global and regional detections. This nucleation and association back-projection algorithm processes the picked phase arrival times to automatically detect seismic events including location, depth, and corresponding associated and named phase set. Multiple instances of the automated detection process can be run, each with a configuration specific to the region of interest.

Hydra runs numerous scientific algorithms in near real-time and for catalog production. Algorithms include location, event validation, over a dozen magnitude calculations, three types of moment tensor computations, and output generation. Hydra’s outputs are parametric solutions distributed as origin, phase, and moment tensor products via PDL (fig. 6). In September 2013, Hydra replaced the seismic monitoring, analysis, and catalog production system on the legacy VAX/VMS systems, which had been in operations for over twenty years. Hydra provides modern and sustainable technologies, a modular and expandable architecture, powerful graphical user interfaces, and seamless NEIC-wide systems integration, which support the NEIC in taking on additional tasks (for example, after-hours monitoring for Volcano Hazards Program) and an expanding mission.

Figure 6. Data processing—Hydra Subsystem. (NEIC, National Earthquake Information Center; ANSS RSN, Advanced National Seismic System Regional Seismic Network; PDL, Product Distribution Layer; TCP, Transmission Control Protocol; SCP, Secure Copy Protocol; PDE, Preliminary Determination of Epicenter; ComCat, Comprehensive Catalog; ENS, Earthquake Notification Service; NEST, NEIC Executive Summary Tool; EMSC, European-Mediterranean Seismological Centre.)

Hydra currently receives composite focal mechanisms from a decentralized database.

There are plans to make a new product generator to perform the computation of composite focal mechanisms from centralized ComCat data. Then Hydra, and other consumers, could acquire these products via PDL, eliminating the need to maintain a separate database and client licenses for connecting to it. There is also a need to integrate “Did You Feel It?” (DYFI?) reports into Hydra, as felt information is a part of NEIC catalog production. The DYFI? felt information can easily be picked up via PDL and integrated into Hydra data for presentation to analysts and inclusion in NEIC catalog outputs.

“Did You Feel It?” Subsystem The “Did You Feel It?” (DYFI?) subsystem is intended to tap the abundant information available about earthquakes from the people who actually experience them. By taking advantage of the vast numbers of Internet users, the DYFI? subsystem derives a more complete description of what people experienced, the effects of the earthquake, and the extent of damage, than could be derived through traditional ways of gathering felt information, which were often manual and time and labor intensive (Atkinson and Wald, 2007).

The DYFI? subsystem inputs include origin products, via PDL, and firsthand accounts entered via webpages, DYFI? questionnaires, and place names (fig. 7). The questionnaires, or felt reports, are used to estimate a shaking intensity, and are grouped into communities, defined by ZIP code regions, cities, or geocoded areas. An intensity value is then assigned to each community for which a questionnaire was filled out; each intensity value reflects the effects of earthquake shaking on the people and structures in the community. The subsystem is triggered by an event origin, to create a respective map of the intensities for each ZIP code, which summarizes the questionnaire responses provided by Internet users. A DYFI? map is made and updated every 5 minutes following a significant earthquake and then updated less frequently as additional data are received. More specific system and product information is available online (http://earthquake.usgs.gov/research/dyfi/).

Figure 7. Data processing—Did You Feel It? (DYFI) Subsystem. (FE, Flinn Engdahl; NEIC, National Earthquake Information System; ANSS RSN, Advanced National Seismic System Regional Seismic Network; PDL, Product Distribution Layer; SMTP, Simple Mail Transfer Protocol; DYFI, Did You Feel It?; ComCat, Comprehensive Catalog; NEST, NEIC Executive Summary Tool.)

DYFI?-generated products are distributed via PDL. The subsystem does need further integration to get Flinn Engdahl regions, as documented online (http://earthquake.usgs.gov/learn/topics/flinn_engdahl.php), via the GeoServer web service. Additional recommendations on points of integration include considering relying on PDL for distribution of PGA XML data, replacing the DYFI? data link to ShakeMap with PDL, and integrating with Earthquake Notification Service (ENS) for email distribution.

ShakeMap Subsystem A ShakeMap is a representation of ground shaking produced by an earthquake. The information a ShakeMap presents is different from data describing the earthquake magnitude and epicenter that are released after an earthquake because ShakeMap focuses on the geographic distribution of the ground shaking produced by an earthquake, rather than the parameters describing the earthquake source (Worden and others, 2010). So, while an earthquake has one magnitude and one epicenter, ShakeMaps represent the range of ground shaking levels at sites throughout the region depending on distance from the earthquake, the rock and soil conditions at sites, and variations in the propagation of seismic waves from the earthquake due to complexities in the earthquake source and the structure of the Earth’s crust.

ShakeMap inputs include origins/magnitudes and moment tensors via PDL, Finite Fault models, geographic and places information, intensity measurements (MMIs as documented online [http://earthquake.usgs.gov/learn/topics/mercalli.php]), and Peak Ground Motions as XML files distributed via SendFile (fig. 8). PGMs are parametric strong motion data, for which there are no national, international, nor industry standards for archiving and distributing, so much work goes into the acquisition of this kind of data.

ShakeMap produces a variety of outputs. The primary outputs are XML files containing the grids of the parametric ground motions and their uncertainties. It also produces a full suite of shape files, suitable for use in GIS systems and the Hazus loss-modeling software, Environmental Systems Research Institute (ESRI) binary raster grids of the parametric data, maps of the various parameters in several formats, contours of the ground motion data in multiple formats, and its input data and processing parameters. These outputs are distributed via PDL as a directory of many files that contain and represent the ShakeMap product. More system details and product information can be found online (http://earthquake.usgs.gov/research/shakemap/formats.php).

Figure 8. Data processing—ShakeMap subsystem. (MT, Moment Tensor; FE, Flinn Engdahl; FM, Focal Mechanism; PDL, Product Distribution Layer; PGA, Peak Ground Acceleration; XML, extensible markup language; HTTP, hypertext transfer protocol; MMI, Modified Mercalli Intensity; DYFI, Did You Feel It?; ANSS RSN, Advanced National Seismic System Regional Seismic Network; NEIC, National Earthquake Information Center; PAGER, Prompt Assessment of Global Earthquakes for Response; ComCat, Comprehensive Catalog; ENS, Earthquake Notification Service; NEST, NEIC Executive Summary Tool.)

Areas for consideration relative to system integration include:

· Distribute and receive Finite Fault products via PDL.

· Move composite focal mechanism (FM) data into a centralized web service built from moment tensor data on ComCat.

· Acquire Flinn Engdahl regions and place information via the GeoServer web service.

· Using PDL for distribution of PGA XML data rather than SendFile.

PAGER Subsystem PAGER is an automated system that produces content concerning the impact of significant earthquakes around the world (Wald and others, 2010). The system informs emergency responders, government agencies, aid agencies, and the media of the scope of the potential disaster. PAGER rapidly assesses earthquake impacts by comparing the population exposed to each level of shaking intensity with models of economic and fatality losses based on past earthquakes in each country or region of the world. Earthquake alerts—which were formerly sent based only on event magnitude and location, or population exposure to shaking—now are also generated based on the estimated range of fatalities and economic losses.

System inputs include ShakeMap products, and city and population information (fig. 9). The system also maintains a historical catalog of shaking exposure for use in assessments. Such catalogs have been maintained in isolated systems for use by PAGER only. Currently developers are working to format and integrate these catalogs into the Comprehensive Catalog in order to further the effort of supporting fewer applications and data repositories as well as to support more data sharing. More system and product specific information can be found online ( http://earthquake.usgs.gov/research/pager/).

Figure 9. Data processing—PAGER Subsystem. (NEIC, National Earthquake Information Center; FE, Flinn Engdahl; ANSS RSN, Advanced National Seismic System Regional Seismic Network; PDL, Product Distribution Layer; ENS, Earthquake Notification Service; ComCat, Comprehensive Catalog; NEST, NEIC Executive Summary Tool; PAGER, Prompt Assessment of Global Earthquakes for Response; FEMA,Federal Emergency Management Agency; USGS, U.S. Geological Survey; USAID/OFDA, U.S. Agency for International Development/Office of Foreign Disaster Assistance; UN, United Nations) Tweet Earthquake Dispatch The NEIC runs a real-time system that detects earthquakes worldwide using data solely mined from Twitter messages. This system improves the earthquake response of NEIC and its collaborators by providing rapid alerts prior to seismic-based detections in regions sparsely covered with seismic instruments (Earle and others, 2010; Guy and others, 2010). This system capitalizes on the expeditious, free and ubiquitous nature of social-media data to produce rapid and dynamic earthquake effect products based on information shared via social-media networks, from people at the source of the hazard, as the effects unfold. Amassed detection, damage, and effects data are made available to collaborators who participate in hazards response efforts. The input data streams and distributed products are persisted to support analysis and long term scientific research efforts.

System inputs include tweets from Twitter, geocoding places…