Appendix_B.pdf

SIGMA Edge API Release 8.0.1

Two Six Labs & Others

May 09, 2018

Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)

Table of Contents

1 Introduction 1

1.1 Language Definitions

1.2 Edge Protocols

1.3 Background

1.4 Data Requirements

1.5 Which API Should I Use?

2 Streaming API 5

2.1 Data Requirements

2.2 Establishing Communications

2.3 Message Formats

2.4 Device Provisioning

2.5 Device Registration

2.6 Sensor Registration

2.7 Data Reporting

3 Reporting API 13

3.1 Overview

3.2 Data Requirements

3.3 Establishing Communications

3.4 Device Registration

3.5 Sensor Registration

3.6 N42 XML Support

4 Appendix and Glossary 21

4.1 Glossary

4.2 Thrift Messages

4.3 Edge REST API

4.4 References

Index 65 i

CHAPTER 1

Introduction

In its current incarnation, the SIGMA platform enables real-time acquisition, communication, processing, and storage of data from distributed radiation sensors and associated secondary sensors such as cameras;

with integrated detection and ID capability for spectroscopic gamma sensors, as well as the capability to fold in neutron signatures to augment threat detection, the platform provides an end-to- end, real-time threat detection and awareness capability. At its heart, the SIGMA platform is a secure cloud-based archi-tecture for sensor data acquisition and monitoring, intended to keep national security operations one step ahead of the threat. SIGMA was designed as a geographically distributed network capable of scaling to receive data from thousands of radiation sensors with spectroscopic gamma and neutron sensing capabil-ities. In many cases, continuously streaming radiation readings are transmitted from distributed sensor systems, and analysed using advanced analysis algorithms to provide automated anomaly detection and isotope identification; in other cases, existing systems transmit packaged alerts to the SIGMA platform for display. SIGMA delivers analytical outcomes to the users through a variety of user interfaces, including a web-based situational awareness display, and a smartphone application for sensor operators on the ground.

The SIGMA platform is architected to be flexible, providing an approach that can easily be modified to support interconnetion with other systems and integration of new sensors. This document describes the APIs, protocols, and message definitions used to report data to the system; these are collectively referred to as the Edge Protocols.

1.1 Language Definitions

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be inter-preted as described in RFC 2119 (see “Key words for use in RFCs to Indicate Requirement Levels”).

1.2 Edge Protocols

The Edge Protocol is the mechanism through which distributed sensors report data to SIGMA’s cloud-hosted backend. This protocol is designed to be flexible, reliable, and lightweight enough to operate in a variety of real-world operating conditions. The Edge protocol encapsulates two separate APIs that can be used depending on an integrator’s use case, the Streaming API and the Reporting API. To make a decision about which is right for you, please refer to Which API Should I Use?.

Figure 1.1. High Level Diagram of how sensors and devices communicate with the SIGMA Edge

1.3 Background

The Edge protocol details the interaction between fielded units and the cloud backend. There are two aspects of this communication that are detailed in the next two subsections: Sensor/Device and Clien-t/Server Relationships

1.3.1 Sensors and Devices

Communications with the SIGMA cloud service require a Device to manage the interaction between one or more Sensors and the network. In this taxonomy, Sensors are only responsible for collecting data, while the Device is the entity that reads and packages data from the sensor and transmits it to the network.

Though this documentation distinguishes between the Device (or communications unit) and the Sensor (or detector), in some cases, these are packaged together and indistinguishable to the end user while in other cases they are completely distinct entities. As an example, if an Android smartphone communicates with an external sensor over Bluetooth, then the smartphone is a distinct Device. Alternately, a radiation portal monitor is usually seen as a single entity, which encloses both Sensors and a Device.

The SIGMA system supports a one to many relationship between a Device and its connected Sensors. This allows multiple sensors to report through a single communications unit, requiring only one internet connection for a set of detectors; however, a single Sensor cannot be connected to more than one device at once.

This documentation only covers the communication between Devices and the SIGMA network; it does not cover communications between the Device and Sensor.

SIGMA Edge API, Release 8.0.1

2 Chapter 1. Introduction

1.3.2 Clients and Servers

As in all networked communication, there are Clients and Servers in the SIGMA Edge protocols. Clients always initiate the communication. In the SIGMA Edge protocol, Devices, as described above, are always the Client, and the SIGMA Edge is the Server.

The SIGMA Edge handles communications from multiple Clients simultaneously.

1.4 Data Requirements

Regardless of which protocol Sensors utilize for reporting to the system, the following section details general data requirements for integration.

At present the SIGMA system incorporates low-medium resolution spectroscopic gamma detectors and count based neutron detectors as the primary sensors. Spectroscopic gamma sensors deployed within the network generally are crystal-based scintillator technology with an energy range of 30 – 3000 keV and a resolution of 512, 1024 or 4096 channels over the entire energy range.

While the system can accommodate any of these three detector resolutions, for current algorithms, health status monitoring, and graphical presentation layers, the data is down-binned to the 512 channels.

Additionally, all sensors require certain metadata for association with sensor readings; the specific formats and frequencies are detailed in association with each API, but generally include:

• All spectral sensors require calibration data in order to visualize spectral conversions from reported energies as well as for algorithm processing in the case of streaming sensors.

• Location data must be reported for all sensors whether statically deployed or mobile. This data is used for advanced analysis as well as visualization.

• Health and Status of devices and sensors must be reported at intervals as detailed below in the corresponding protocol sections.

Secondary sensors such as camera sensors are also in use in the system, but those sensors use a different API to make their data available to SIGMA.

1.5 Which API Should I Use?

The Streaming API is the original SIGMA sensor interface, and it implements a reliable mechanism for Sensor data to be reported to the SIGMA backbone, covering registration, data delivery, reliability, and caching.

The goals of the Streaming API are to:

• Allow sensors to securely deliver sensor status and spectral data payloads to the SIGMA Edge.

• Allow sensors to reliably cache data and reconnect in the event of interrupted connectivity.

• Allow messages to be passed back to Sensors or Devices in the field.

The Reporting API implements a simplified mechanism for devices that do not support continuous spec-troscopic data to communicate with the SIGMA system. The goal of this API is to :

• Allow devices that do not wish to continuously report data, health information, and/or alerts to the SIGMA system.

• Provide an easier mechanism for integration into the SIGMA system via an authenticated HTTPS

SIGMA Edge API, Release 8.0.1

1.4. Data Requirements 3

channel

When choosing which API is appropriate for your use, the primary differentiator is whether you wish to report high frequency spectral data. At this point, only the Streaming API supports that functionality. For all other use cases, the Reporting API is recommended.

SIGMA Edge API, Release 8.0.1

4 Chapter 1. Introduction

CHAPTER 2

Streaming API

The Streaming API is message-oriented, meaning that it operates over a stream of (strongly-typed) messages. It is not a remote procedure call mechanism.

This section describes the overall flow of the conversation that occurs between a single Client and the Server in the Streaming API. When operating, a Server MAY maintain multiple separate conversations with multiple Clients (or in some special cases with a single Client), but only one such conversation is described here. Detailed descriptions of each part of the conversation described in this diagram is found in the following sections.

Figure 2.1. Overall conversation flow between a Client and a Server, includes an initialization phase, followed by inde-pendent reporting loops for each sensor.

The following describes this conversation (as depicted in Figure 2.1):

• The Client initiates the conversation with the Server by establishing a connection as described in

Transport Layer

• The Client MUST first send a Device Registration (RegisterDevice) request to the server. This request is responsible for authenticating the Device to the server.

• The Client MUST register each Sensor with the Server before sending data for that sensor. The Client

MAY register multiple sensors.

• After Sensor registration, the Client SHOULD send SensorReport messages for each registered sensor at a regular interval.

• If the connection to the Server is lost, the Client MUST tear down their socket and begin the conver-sation again from the beginning, including Device Registration.

2.1 Data Requirements

The Streaming API is the main API through which streaming data is submitted to the SIGMA system.

This streaming data is used for processing by cloud-based detection and identification algorithms.

Currently, the main purpose for the streaming of data is to provide near real-time updates from a Spectral Sensor to be processed by a cloud hosted algorithm to produce Alerts with Isotope IDs.

Note For spectral sensors, it is important that the channels are ordered in accordance with the magnitude of the physical property and that the channel is “stable” such that the paired value to a specific channel maintains a consistent meaning in every sensor report.

Devices are required to provide sensor readings at 1Hz with 10% tolerance, location data at 1Hz, and health/status messages at 1/30 Hz.

Additionally all previously stated requirements from Data Requirements apply.

2.2 Establishing Communications

2.2.1 Underlying Technologies

The Edge API is built on two standard open-source technologies:

• ZeroMQ

• Provides a message-oriented reliable communication layer built on the ZeroMQ networking library, which provides sockets that carry atomic messages across various transports, with support for N-N communication.

• Apache Thrift

• Provides a language-agnostic mechanism for marshalling message objects into binary represen-tation suitable for transport over the network.

The remainder of this document assumes familiarity with ZeroMQ and Apache Thrift.

2.2.2 Transport Layer

The Edge Server binds a ZeroMQ ROUTER socket listening on TCP port 5569. Connections are estab-lished from the remote Device by creating a REQ socket and connecting to the SIGMA Edge. No authenti-cation or encryption is established at the transport layer.

The production Edge Server is located at edge.dtect.net:5569.

Data sent over the transport layer is packed in Thrift-encoded messages as described in Message Formats.

When the Server receives valid messages, it will respond with Thrift encoded messages. However, if the message received is invalid for any reason, the Server will respond with a simple error code (see below).

2.2.3 Basic Error Codes

Basic error codes are sent as raw 4 byte strings, representing a big-endian signed integer. This integer MUST always be one of the following error codes:

SIGMA Edge API, Release 8.0.1

6 Chapter 2. Streaming API Distribution Statement “A” (Approved for Public Release, Distribution Unlimited) http://zguide.zeromq.org/page:all https://thrift.apache.org/

* Errors that happen when messaging fails enum MessageErrorCode {

* General error.

UNKNOWN_ERROR = -1;

* Invalid protocol version

INVALID_PROTOCOL_VERSION = -2;

* Failed to parse the message

MESSAGE_PARSE_FAILURE = -3;

* Failed to parse the message payload

PAYLOAD_PARSE_FAILURE = -4;

* Failed to decrypt due to key mismatch

KEY_MISMATCH = -5;

* Failed to decrypt due to mac comparison failure

MAC_FAILURE = -6;

* Failed to decrypt due to key mismatch

INVALID_SESSION = -7;

2.2.4 Example

The first thing to do is install ZeroMQ

$ pip install zmq

Then you want to establish a socket connection. The Edge protocol is a binary protocol, so all you can do at this point is verify connection. In this example, we receive a binary -3, which indicates a message parsing failure based on the Basic Error Codes. Since we didn’t send a valid message, this is to be expected because -3 indicates a failure to parse the message.

Implementors are expected to handle error responses appropriately. If the response is UNKNOWN_ERROR a simple retry is appropriate. All other responses indicate that some action must be taken by the implemen-tor.

>>> import zmq >>> ctx = zmq.Context.instance() >>> sock = ctx.socket(zmq.REQ) >>> sock.connect("tcp://edge.dtect.net:5569") >>> sock.send("Hello"?)

>>> result = sock.recv() >>> import struct >>> struct.unpack("!i", result) (-3,)

SIGMA Edge API, Release 8.0.1

2.2. Establishing Communications 7

2.3 Message Formats

This section describes the formatting of messages that are transmitted over the ZeroMQ transport layer.

The Edge API uses the Apache Thrift schema and associated generation tools to provide an efficient plat-form and language agnostic serialization scheme.

Each message sent from a Device to the Edge is sent as a base Base Message. This Base Message contains a unique identifier, a sequence number, authorization information, the “topic” (which is used to decode a payload), and a (binary) payload. The payload is generally another Thrift message encoded using the Thrift Compact Protocol encoding. This allows all message types to be generically delivered to the Edge without replicating the data fields needed to track messages between the Edge and the Device. Upon receipt of a message, each part of the SIGMA system reads the base Base Message to determine its type and whether it needs to be processed further.

When beginning a communication session with the Server the Client MUST initially set the sequence number to 1. Each subsequent message sent to the Server MUST increment this value by one. However, any messages re-sent due to failure to receive MUST re-use their existing sequence number.

2.3.1 Authentication and Security

Figure 2.2. A depiction of how messages must be encapsulated and authenticated prior to submission.

The payload field of each base Message sent to Edge is encrypted and authenticated using AES256 in CBC mode with a SHA(1) HMAC message authentication code. The 32-bit AES256 ephemeral session key is selected randomly by the Device at the establishment of a Session. The Device sends the session key (encrypted by the asymmetric key obtained earlier via Device Provisioning) to Edge as part of the RegisterDevice message which initially establishes a Session.

The token field of the Base Message is used to store the following cryptographic values:

• When topic == RegisterDevice:

• token is a 284-byte blob formed by the concatenation of:

• The first 8 bytes of the AES256 Initialization Vector used for this payload.

• The (20-byte) Message Authentication Code produced by HMAC-SHA1.

• The (256-byte) RSA-encrypted ephemeral session key.

SIGMA Edge API, Release 8.0.1

8 Chapter 2. Streaming API Distribution Statement “A” (Approved for Public Release, Distribution Unlimited) https://thrift.apache.org/ https://github.com/apache/thrift/blob/master/doc/specs/thrift-compact-protocol.md

• When topic has any other value:

• token is a 28-byte blob formed by the concatenation of:

• The first 8 bytes of the AES256 Initialization Vector used for this payload.

• The (20-byte) Message Authentication Code produced by HMAC-SHA1.

To be clear:

• Bytes 0:7 of the AES256-CBC Initialization Vector are taken from bytes 0:7 of token.

• Bytes 8:15 of the AES256-CBC Initialization Vector are the sequence field of the Base Message.

2.4 Device Provisioning

In order to initiate a Session with Edge, Clients (Devices) must obtain their 2048-bit RSA key by contacting the “Provision Service” via its (https-secured) REST API. This key is expected to be “long-lived” (e.g. persisted locally on the Device and used to initiate future Sessions indefinitely).

Devices should issue an HTTP GET request to the Provision Service REST API Endpoint with the following URL parameters:

• device = the UUID of the device (encoded in canonical “hex-with-dashes” format as per RFC4122).

• make = the “make” (manufacturer name) of the device

• model = the device’s model number

• name = an identifying device name (need not be unique)

• serial = the serial number of the device as specified in the DTECT Inventory UI.

Note that the meaning or required parameters may vary somewhat between devices.

The production Provision Service endpoint is: https://provision.dtect.net/provision.

The returned document will be the 2048-bit RSA key encoded in PEM format (MIME type:

application/x-pem-file). This key should be persisted on the Device for use in future Sessions and reasonable safeguards should be taken against disclosure of this key. Keys can only be retrieved if the device has been added to the system already.

2.5 Device Registration

Once a Client has established a connection with the Server, the Client must register itself. This registration identifies and authenticates the Client to the Server: information is provided to the Server and an ephemeral symmetric key to use for the remainder of the session is exchanged.

2.5.1 Message Flow

Device registration has the following flow.

SIGMA Edge API, Release 8.0.1

2.4. Device Provisioning 9

https://tools.ietf.org/html/rfc4122

Figure 2.3. Edge Sequence

• The Client sends a RegisterDevice message to the Server.

• If the request was parseable, then the Server MUST send a RegisterDeviceResponse message back.

Note that there can still be an error.

• If the request was not parseable, then the Server MUST send a basic error code as the response.

• The Client MUST continue to attempt to register by sending RegisterDevice messages until it receives a valid response.

2.5.2 Payload Formats

The RegisterDevice message provides information about the device attempting to register with the edge.

RegisterDevice messages MUST include a populated device property, providing details about the device itself. This property is an instance of the Device struct shown below.

The RegisterDeviceResponse contains a simple response code and information about the version of the protocol being used.

The possible set of response codes are:

enum RegisterResponseCode {

OK,

PROTCOL_VERSION_MISMATCH,

UNKNOWN_DEVICE,

UPDATE_AVAILABLE

A device receiving a RegisterResponseCode other than OK will NOT be able to communicate with the network. UPDATE_AVAILABLE is a deprecated response code.

2.6 Sensor Registration

The sensor registration process is similar to the device registration flow, with two key differences: it does not include the cryptographic handshake, and it is repeated once for each sensor.

To register a Sensor, the Client MUST send a RegisterSensor message to the Server. This message MUST

SIGMA Edge API, Release 8.0.1

10 Chapter 2. Streaming API contain a populated Sensor struct, which MUST at a minimum define the make, model, serial number and type of that Sensor.

In response to a valid RegisterSensor message, the Server will respond with a RegisterSensorResponse message. The RegisterSensorResponse can optionally contain sensor settings, characterization infor-mation, or calibration passed back from the server. If any of these fields are present, the Client SHOULD apply them as appropriate.

If the RegisterSensor is invalid for any reason, the Server will respond with one of the Basic Error Codes encoded in the base Message.

2.7 Data Reporting

Once the Client has registered itself and its Sensors with the Server, the Client can begin reporting actual data to the Server. The Client is responsible for maintaining an appropriate data submission rate, and caching recorded data until it has been acknowledged. SensorReport messages SHOULD be submitted to the Server every 1 second.

Once in the data reporting loop, the Client SHOULD only send SensorReport messages to the Server. These messages combine radiation readings, location, and device and sensor status. A SensorReport MUST include AT LEAST ONE of radiation, location, deviceStatus, or sensorStatus, or it will be rejected by the Server.

Each unique message sent to the Server MUST increment the sequence number in the Base Message. The Client MUST cache messages that they send to the Server, until they are acknowledged. The Server MUST indicate successful receipt of each message by responding with a Base Message that has a matching sequence number to the message that is sent. If the message cannot be decoded or is otherwise invalid, the Server MUST indicate that fact by responding with a basic error code.

2.7.1 Message Retry

If the Server does not indicate successful message receipt, the Client MUST resend the exact same message.

2.7.2 Connection Failure

If the Server does not respond to a message within thirty (30) seconds, the Client MUST tear down the ZeroMQ socket and begin the communication flow again (including registration).

SIGMA Edge API, Release 8.0.1

2.7. Data Reporting 11

SIGMA Edge API, Release 8.0.1

12 Chapter 2. Streaming API

CHAPTER 3

Reporting API

3.1 Overview

The Reporting API is a secondary interface through which distributed sensors can report data to SIGMA’s cloud-hosted backend.

This protocol is primarily intended for non-streaming data reporting, such as reports from alert-only systems.

This section describes the overall flow of the conversation that occurs between a single Client and the Server in the Reporting API. When operating, a Server MAY maintain multiple separate conversations with multiple Clients (or in some special cases with a single Client), but only one such conversation is described here.

The Reporting API expects a similar conversation flow to the Streaming protocol, wherein a Client first registers a Device, then registers one or more sensors prior to submitting data from those sensors. The sequence of expected method calls is as follows:

• The Client MUST first register the Device it represents by sending a POST request to the /device/register endpoint.

• The Client MUST submit location and status information at least once every 30 seconds by submit-ting a request to the /device/{deviceId}/location and /device/{deviceId}/status endpoints.

• The Client MUST submit a request to register each connected sensor via the /device/{deviceId}/sensor/register endpoint.

• The Client SHOULD submit a request with status for each registered sensor at least once every 30 seconds to /device/{deviceId}/sensor/{sensorId}/status

• The Client MAY submit a packaged N42 XML file to the /device/{deviceId}/alert/n42 endpoint.

The details for these HTTP endpoints can be found in the Edge REST API section of the appendix.

Note Registrations for devices will time out if no data is received for an hour; if this happens, the Client must re-initiate the flow described above.

3.2 Data Requirements

The Reporting API is the main API through which packaged alerts from legacy or non-streaming system can be reported to the SIGMA system.

There are certain requirements for alerts submitted using the N42.42 XML standard; these are detailed in N42 XML Support below.

In addition, the Reporting API requires that sensors regularly report health and status information in addition to alerts. Specifically, location data should be provided at 1Hz for mobile sensors, and health/s-tatus information at 1/30hz.

All previously stated requirements from Data Requirements apply.

3.3 Establishing Communications

3.3.1 Underlying Technologies

The Reporting API operates over standard HTTPS protocols. The API endpoints are defined using the Swagger API schema; the schema itself is included in Edge REST API.

The remainder of this document assumes familiarity with HTTPS requests and responses.

3.3.2 Transport Layer

The Reporting API uses standard HTTPS to provide an encrypted communications channel. The produc-tion endpoint is located at https://api.dtect.net/sigma/edge/v1/.

3.3.3 Basic Error Codes

Error codes are returned as HTTP errors (non 200 codes).

3.3.4 Authentication and Security

The Reporting API is secured via the use of HTTPS endpoints. In addition, each request must be authenti-cated on a per-device basis, by including the X-Sigma-API-Key header in each request.

To obtain an API key for a device, contact help@dtect.net with information about the device you would like to register. When registering, the device information MUST match what was provided when the device was registered.

3.4 Device Registration

Use endpoint /device/register to register a device. On success will return 200 and a deviceId that is used in all future calls. Bad parameters will result in error code of 400 and a invalid API key will result in a code of 401.

3.5 Sensor Registration

Use endpoint /device/{deviceId}/sensor/register to register each connected sensor. This is a

SIGMA Edge API, Release 8.0.1

14 Chapter 3. Reporting API Distribution Statement “A” (Approved for Public Release, Distribution Unlimited) https://api.dtect.net/sigma/edge/v1/ mailto:help@dtect.net

POST endpoint with a Sensor object (see the Edge REST API section for details on all possible endpoints, and a Swagger schema that can be used to generate client code. reference below). Returns 200 and the sensor UUID on success. Returns code 400 on error.

3.6 N42 XML Support

Use the /device/{deviceId}/alert/n42 endpoint, see below, to submit an alert as an N42.42 XML file.

Submitted N42.42 XML files will be converted to an internal SIGMA structure for processing and visual-ization purposes, but the original data is retained. The N42 xml file will be saved as an annotation as will any images in MultimediaData.

Warning Before submitting an N42 the device MUST be registered and all referenced sensors MUST also be registered.

Warning At present, this API only supports the submission of alerts. That is, measurements where a specific isotope was identified. Future versions of this API may support measurements without identi-fied sources.

See https://www.nist.gov/programs-projects/ansiieee-n4242-standard for the full N42 standard refer-ence.

The following subset is supported by the Reporting API. The part of the document that concerns the reporting api are the contents of the RadInstrumentData tag. Other tags enclosing or in addition to this tag will be ignored.

• RadInstrumentData:

The top element of an instance of a radiation measurement instrument’s N42 XML document.

This element contains all the reported measurement and analysis data, and all the information on the instrument, its radiation detector(s), and the item(s) it measured.

At least one RadDetectorInformation sub tag is required. Other tags not listed below will be ignored.

• RadInstrumentInformation: Required.

Describes the radiation measurement instrument that collected the data contained in the N42 XML document.

• RadInstrumentManufacturerName: Required, must match the ‘make’ of a registered Sensor. Name of the manufacturer of the radiation measurement instrument.

• RadInstrumentModelName: Required, must match the ‘model’ of a regis-tered Sensor. The radiation measurement instrument manufacturer’s model name, number, or other description of the radiation measurement instru-ment.

• RadInstrumentIdentifier: Required, must match the ‘serialNumber’ of a registered Sensor. Identification information for the specific radiation measurement instrument; such as serial number or asset tag number.

• RadDetectorInformation: Contains information describing a radiation detector.

The id attribute is required and there must be at least one of these tags. Other attributes and sub tags are ignored.

SIGMA Edge API, Release 8.0.1

3.6. N42 XML Support 15

https://www.nist.gov/programs-projects/ansiieee-n4242-standard

• RadMeasurement: Must include Spectrum and/or GrossCounts.

This element records a measurement at a particular StartDateTime, for a Real- TimeDuration, of a particular MeasurementClassCode that consists of readings from any number of one or more of the following:

a radiation detector; an occupancy sensor; a positioning sensor that captures the location of a radiation measurement instrument, radiation detector, or measured item; or the state of a radiation measurement instrument, radiation detector, or measured item.

The following attributes can be provided for these sensor readings

• MeasurementClassCode: Required, must provide ‘Foreground’. Indicates whether the data are a measurement of an item (Foreground), an environ-mental background (Background), a calibration source (Calibration), the intrinsic activity of the radiation measurement instrument (IntrinsicActivity), or not specified (NotSpecified).

• StartDateTime: Required, example 2016-05-26T14:51:24 can have +tzoffset or Z for UTC on end. Time corresponding to the start of the collection of the data contained in a particular measurement.

• RadDetectorState: Attribute radDetectorInformationReference is required (must match the id of a RadDetectorInformation tag), other attributes will be ignored. The current state of a radiation detector in terms of its location (absolute or relative), orientation, altitude, and speed. Location (lat/lon) is all the reporting API will consume.

• StateVector State values for a radiation measurement instru-ment, a radiation detector, or a measured item.

• GeographicPoint Geographical coordinates providing latitude, longitude, and elevation (at the point of measurement and at the point on the earths surface), and uncertainty of the coordinates.

• LatitudeValue The latitude of a point on the surface of the earth expressed as geographic coordinates in decimal degrees. Points in the northern hemi-sphere range from 0.0 to +90.0 degrees.

Points in the southern hemisphere range from 0.0 to -90.0.

• LongitudeValue The longitude of a point on the surface of the earth expressed as geographic coordinates in decimal degrees. Points east of the prime meridian range from 0.0 to +180.0 degrees.

Points west of the prime meridian range from 0.0 to -180.0.

• Spectrum: Attribute radDetectorInformationReference is required (must match the id of a RadDetectorInformation tag), other attributes will be ignored. Contains a single spectrum measurement with references to other pertinent information about the measurement.

SIGMA Edge API, Release 8.0.1

16 Chapter 3. Reporting API

• LiveTimeDuration: Required. The duration during which a detection assembly is sensitive to the input signal. The value of LiveTimeDuration is always less than or equal to the value of RealTimeDuration, because it does not include the time that the radiation detector was unable to respond due to the processing of events.

• ChannelData: Attribute compressionCode is required. A list of values, one for each of a spectrum’s channels. The values represent the number of counts per channel.

• GrossCounts: Attribute radDetectorInformationReference is required. A data type providing gross count radiation data.

• LiveTimeDuration: Required. The duration during which a detection assembly is sensitive to the input signal. The value of LiveTimeDuration is always less than or equal to the value of RealTimeDuration, because it does not include the time that the radiation detector was unable to respond due to the processing of events.

• CountData: Required. The number of counts accumulated during a measurement period over the entire energy range measured by the radiation detector or within pre-defined energy windows.

• MultimediaData: Multimedia data (only support images) to annotate to investigation.

Providing unlisted attributes or elements will be ignored, an unsupported mime type or data object will be an error. Multimedia data - e.g., images, sound clips, movies, -regarding a measured item or a measurement environment.

• MultimediaCaptureStartDateTime: Required. Date-time at which capture of the multimedia data was started.

• MultimediaDataMimekind: Required and MUST be image/jpeg or image/png. Media types are listed in http://www.iana.org/assignments/media-types/index.html. If the media type is not listed, then describe the media type using free-form text.

• BinaryBase64object: This or BinaryHexObject required, image data in base 64 encoding. Base 64 binary encoding of data.

• BinaryHexObject: This or BinaryBase64Object required, image data in hex encoding. Hex binary encoding of data.

• AnalysisResults Required, the collection of information resulting from the analysis of the radiation measurements or derived data.

• AnalysisAlgorithmName: Required, can come from this tag or AnalysisAlgorithmComponentName. A unique name of the analysis algorithm.

• AnalysisAlgorithmVersion: Required. Information describing the version of a particular analysis algorithm component.

• AnalysisAlgorithmComponentName: Required. Name of an algorithm component.

SIGMA Edge API, Release 8.0.1

3.6. N42 XML Support 17

http://www.iana.org/assignments/media-types/index.html

• AnalysisAlgorithmComponentVersion: Required, should be in SEMVER format or the version will default to 0.0.0. Version information for the algorithm component.

• NuclideAnalysisResults: Required, list each ‘Nuclide’ observed. The results of radionuclide analysis.

• Nuclide The analysis results for a single radionuclide.

• NuclideName: Required, must be one of the names defined in Isotopes.

• NuclideIDConfidenceValue: Required. Indica-tion of confidence ranging from 0.0 to 100.0 percent, in the identification status of a nuclide, where increasing values indicate more certainty that the nuclide is present. The interpretation of this value is dependent on the characteristics of the nuclide identi-fication algorithm.

XML Example

<RadInstrumentData xmlns="http://physics.nist.gov/N42/2011/N42"> <RadInstrumentInformation id="Portal-1"> <RadInstrumentManufacturerName>Two Six Labs</RadInstrumentManufacturerName> <RadInstrumentIdentifier>SN_0001</RadInstrumentIdentifier> <RadInstrumentModelName>NaIL</RadInstrumentModelName> </RadInstrumentInformation> <RadDetectorInformation id="NAIL-SN1001"/> <RadDetectorInformation id="NAIL-SN1002"/> <RadDetectorInformation id="NAIL-SN1003"/> <RadDetectorInformation id="NAIL-SN1004"/> <RadDetectorInformation id="NEUTRON-1"/> <RadDetectorInformation id="NEUTRON-2"/> <RadMeasurement id="Measurement-2516"> <MeasurementClassCode>Foreground</MeasurementClassCode> <StartDateTime>2016-06-08T08:35:13</StartDateTime> <Spectrum radDetectorInformationReference="NAIL-SN1001"> <LiveTimeDuration>PT0.54S</LiveTimeDuration> <ChannelData compressionCode="CountedZeroes">0 20 8 10 7 12 13 11 5 11 9 6 12 17 11 12 9 9 10 18 7 13 5 8 5 7 11 6 9 11 11 12 10 11 4 4 10 8 10 10 5 7 8 3 5 9 5 6 6 0 1 4 5 5 3 3 5 3 3 4 11 1 4 7 3 3 2 4 7 4 2 4 0 1 1 1 1 2 0 1 2 1 1 1 1 4 0 1 1 2 2 0 1 2 3 2 0 1 1 1 1 0 2 1 4 2 2 2 0 1 1 2 0 1 2 0 1 1 1 3 0 1 2 0 1 1 1 0 1 1 1 1 1 0 2 1 0 4 2 0 2 1 0 1 2 1 1 1 0 2 1 1 1 1 0 2 2 0 2 1 0 1 1 0 4 3 0 2 1 1 0 2 1 1 0 3 1 0 6 3 0 1 3 1 1 1 0 4 1 0 5 2 1 0 1 1 0 1 1 0 1 1 1 3 1 0 6 1 0 1 1 2 0 2 2 0 1 1 1 0 1 1 1 0 8 1 0 1 2 0 1 1 1 1 0 1 1 1 0 1 1 1 0 1 2 0 3 1 0 1 1 0 22 1 0 3 1 1 0 2 1 0 6 1 0 5 1 1 0 5 2 0 5 1 0 2 1 0 2 1 1 0 6 1 0 3 1 0 13 1 1 0 33 1 0 14 1 0 1 1 0 3 1 0 3 1 1 0 8 1 0 12 1 0 4 1 0 22 1 0 3 1 0 10 1 0 16 1 0 17 1 0 7 1 0 11 1 0 2 1 0 21 1 0 17 1 0 12 1 0 12 1 0 19 1 0 9 1 0 11 1 1 0 15 1 0 10 1 0 151 1 0 29 1 0 67 1 0 6 1 0 10 1 0 7 1 0 72</ChannelData> </Spectrum> <Spectrum radDetectorInformationReference="NAIL-SN1002"> <LiveTimeDuration>PT0.54S</LiveTimeDuration> <ChannelData compressionCode="CountedZeroes">0 20 7 7 9 6 11

SIGMA Edge API, Release 8.0.1

18 Chapter 3. Reporting API

7 16 12 15 17 8 11 15 9 10 13 12 11 9 11 11 12 9 8 7 9 4 16 5 7 11 5 16 7 10 4 5 9 14 12 9 5 9 3 5 5 7 4 6 7 6 5 7 5 6 9 3 4 5 4 4 6 5 5 5 0 1 2 3 1 6 4 1 6 2 1 2 1 0 1 4 2 4 3 2 6 1 4 0 1 2 0 1 1 1 1 1 0 1 3 4 1 1 1 1 1 1 1 0 2 4 0 1 5 2 2 1 2 2 0 2 3 3 0 1 3 1 3 0 1 3 1 0 1 2 2 2 0 1 2 1 2 1 0 1 3 0 1 1 1 0 2 1 1 1 0 5 1 1 1 0 3 1 2 0 1 1 2 0 2 1 0 1 2 0 1 2 0 4 1 0 1 2 0 4 1 1 1 0 2 3 0 6 1 1 0 1 1 0 3 1 0 1 1 1 1 1 0 1 1 0 1 1 0 5 2 0 1 2 0 5 1 0 1 1 0 5 1 0 3 2 0 1 1 0 1 2 0 1 1 0 2 1 0 8 1 0 1 1 0 5 1 0 11 2 0 4 1 0 1 1 0 3 1 0 3 1 0 3 1 0 1 1 1 0 2 1 0 15 2 1 0 1 1 0 7 1 1 0 7 1 0 2 1 0 2 1 0 1 1 1 0 5 1 0 11 1 0 3 1 0 11 1 0 27 1 1 0 10 1 0 7 1 0 6 1 1 0 12 1 1 0 11 1 0 10 1 0 9 1 1 1 0 19 1 0 11 1 0 15 1 0 1 1 0 7 1 0 29 1 0 4 1 0 18 1 0 42 1 0 10 1 0 115 1 0 12 1 0 29 1 0 29 1 0 47 1 0 73 1 0 48</ChannelData> </Spectrum> <Spectrum radDetectorInformationReference="NAIL-SN1003"> <LiveTimeDuration>PT0.54S</LiveTimeDuration> <ChannelData compressionCode="CountedZeroes">0 6 2 2 1 3 0 1 1 5 8 2 2 8 3 8 6 7 6 6 8 13 9 15 13 17 13 7 13 10 13 11 13 5 5 15 11 10 12 10 9 14 13 3 9 9 5 7 5 7 5 6 7 2 2 5 8 7 2 5 5 4 9 4 8 4 7 2 6 7 3 1 3 4 1 1 2 2 3 4 1 3 4 3 3 1 5 5 3 2 0 1 4 1 1 1 3 0 1 2 2 2 2 1 2 0 1 1 2 1 2 1 2 5 0 1 1 2 0 1 1 1 0 1 5 0 2 1 3 1 2 0 2 2 1 2 2 0 1 2 3 1 1 2 0 3 1 0 1 1 1 1 0 4 1 0 3 2 1 0 2 1 0 2 1 1 1 0 3 2 0 3 1 0 1 1 0 2 3 0 1 1 0 2 1 0 2 1 2 0 9 1 0 2 1 1 0 4 1 0 3 1 1 0 1 2 0 2 1 2 1 0 3 1 0 2 2 0 1 1 1 0 26 1 0 7 1 0 3 1 0 1 1 0 14 1 1 1 0 6 1 0 12 1 1 0 3 1 0 18 2 0 21 1 0 6 1 2 0 15 1 0 3 1 0 6 1 0 10 1 0 13 1 0 1 1 0 2 2 0 21 1 0 2 1 0 4 1 0 2 1 0 14 1 0 12 3 0 2 1 0 3 1 1 0 11 1 1 0 2 1 0 9 1 0 54 1 0 12 1 0 25 1 0 12 1 0 5 1 0 17 1 0 114 1 0 5 1 0 52 1 0 22 1 0 183</ChannelData> </Spectrum> <Spectrum radDetectorInformationReference="NAIL-SN1004"> <LiveTimeDuration>PT0.54S</LiveTimeDuration> <ChannelData compressionCode="CountedZeroes">0 20 6 12 9 9 13 15 10 15 12 17 17 8 15 14 8 10 16 10 13 8 10 15 10 11 6 11 10 13 13 8 10 5 8 10 9 11 5 8 4 4 9 5 7 7 10 8 6 7 7 4 6 3 4 2 6 3 4 3 4 6 3 4 4 4 2 5 6 2 0 1 6 3 3 3 3 3 2 2 2 3 3 2 4 2 1 2 2 2 5 2 1 1 2 0 1 3 2 0 1 1 3 1 1 2 1 0 1 3 1 4 2 0 1 3 3 0 1 1 1 0 1 1 3 3 1 0 1 4 1 0 1 3 2 1 2 0 1 1 0 1 2 1 0 1 1 1 3 1 1 3 0 1 1 0 1 1 1 0 2 2 0 2 1 2 2 0 1 1 0 2 1 0 1 1 0 7 2 0 1 2 0 2 2 1 0 1 1 1 0 3 1 1 0 3 1 3 1 0 3 1 1 0 1 1 0 1 1 0 2 1 0 5 1 1 1 0 2 1 0 1 1 0 4 3 1 0 3 1 0 3 1 0 2 2 1 0 2 1 0 5 2 0 4 1 0 5 1 1 1 1 0 10 2 0 2 2 0 5 1 0 13 1 0 9 2 0 8 1 0 10 1 1 0 11 1 0 10 1 0 2 1 0 6 1 0 1 1 0 28 1 0 3 1 1 0 4 1 2 1 1 0 34 1 0 9 1 0 18 1 0 4 1 0 19 1 1 0 4 1 0 4 1 1 0 2 1 0 4 1 0 7 1 0 9 1 0 7 1 0 2 2 0 1 1 0 14 1 0 40 1 0 11 1 0 2 1 0 3 1 0 2 1 0 89 1 0 4 1 0 154 1 0 76 1 0 9 1 0 55</ChannelData> </Spectrum> <GrossCounts radDetectorInformationReference="NEUTRON-1"> <LiveTimeDuration>PT0.50S</LiveTimeDuration> <CountData>4</CountData> </GrossCounts> <GrossCounts radDetectorInformationReference="NEUTRON-2"> <LiveTimeDuration>PT0.50S</LiveTimeDuration> <CountData>1</CountData> </GrossCounts> </RadMeasurement> <AnalysisResults> <AnalysisAlgorithmName>GADRAS</AnalysisAlgorithmName> <AnalysisAlgorithmVersion> <AnalysisAlgorithmComponentName>GADRAS</AnalysisAlgorithmComponentName> <AnalysisAlgorithmComponentVersion>12/09/2016</AnalysisAlgorithmComponentVersion> </AnalysisAlgorithmVersion> <NuclideAnalysisResults> <Nuclide> <NuclideName>Neutron</NuclideName> <NuclideIDConfidenceValue>1.00</NuclideIDConfidenceValue> </Nuclide> <Nuclide>

SIGMA Edge API, Release 8.0.1

3.6. N42 XML Support 19

<NuclideName>Cs-137</NuclideName> <NuclideIDConfidenceValue>1.00</NuclideIDConfidenceValue> </Nuclide> </NuclideAnalysisResults> </AnalysisResults> </RadInstrumentData>

SIGMA Edge API, Release 8.0.1

20 Chapter 3. Reporting API

CHAPTER 4

Appendix and Glossary

4.1 Glossary

Message An instance of SIGMA’s core.Message Thrift object. All communications in the Streaming API are encapsulated in a Base Message.

Device A representation of the physical hardware or software used to facilitate communication between a Sensor and Edge. Devices implement the Client side of the Edge Protocol.

SIGMA Edge The Server through which Sensor Reports are delivered to the SIGMA backbone. The Edge imple-ments the Server side of the Edge Protocol.

clients The entity connecting to the SIGMA Edge server; in practice this is a Device

Streaming API The network protocol by which Devices report data collected from Sensors to the SIGMA backbone;

the subject of this document.

Provision Service A REST API endpoint which allows Devices to obtain their long-lived 2048-bit RSA key (sometimes referred to as the “provisioning key”).

Report A discrete message containing data collected from a Sensor. Reports are sent to the SIGMA backbone via Edge.

Server An entity that handles requests from clients over the network. The SIGMA Edge is an example of a Server.

Spectral Sensor A sensor that provides, in every report, a set of channel, value pairs where the channels represent a common physical property’s magnitude and the value represents the intensity of that physical prop-erty at that magnitude.

Sensor Any instrument that data can be collected from – traditionally this has meant Gamma Ray Spectrom-eters or Neutron Radiation Detectors.

Swagger A suite of tooling designed to enable API development. Can be used to generate API clients and documentation. https://swagger.io/

4.2 Thrift Messages

This section includes a subset of the Thrift messages that comprise the SIGMA protocol, focusing on those used in the Streaming API. All messages are wrapped in a Base Message.

Other 4.1. Base Message

* Basic message struct. Specific message types are

* serialized into the payload section of this struct.

struct Message {

* Identifier for this message 1: optional UUID id,

* identifier for the origin device 2: UUID origin,

* The sequence from the origin 3: optional i32 sequence,

* Message auth 4: optional binary token,

* Bit-wise or'd set of options 5: optional i32 opts,

* type/topic. This field will generally be in the form

* <thrift type name>/<subtopic>. Example:

* "sigma.messages.Chat/groupAll" would be the

* value of this field for a chat message sent to the

* "All" group.

6: string topic,

* Rather than define all possible message payloads

* in a union-ish thing here, we leave it open to

* "extension". This and the type should be sufficient

* to deserialize this payload 7: optional binary payload, SIGMA Edge API, Release 8.0.1

22 Chapter 4. Appendix and Glossary https://swagger.io/

* An optional list of timing info that may be populated

* for latency testing within the network 8: optional list<TimingEntry> timing,

* An optional id for the destination device/sensor/algorithm/etc.

9: optional UUID destination,

* An optional session name. This may be used to support multiple registrations

* (and keys and sockets) per device id. If you don't need multiple registrations

* (or you don't understand what that means) don't use this.

10: optional string session,

4.2.1 Device Registration

Other 4.2. UUID

* An RFC 4122 compliant uuid struct UUID { 1: i64 mostSignificantBits, 2: i64 leastSignificantBits

Other 4.3. Device

* A message describing a device. This should be sent when

* a connection to the server is established.

struct Device {

* This is only required if this device isn't wrapped in a Message (with origin == this) 1: optional core.UUID id,

* The make 2: string make (swagger = "true", swagger.required = "true"),

* The model 3: string model (swagger = "true", swagger.required = "true"),

* Firmware info 4: optional string firmwareVersion (swagger = "true"), SIGMA Edge API, Release 8.0.1

4.2. Thrift Messages 23

* A user-intelligible name for the device; may be used for display purposes.

5: string name,

* Which version of the software?

6: optional string appVersion (swagger = "true", swagger.name = "softwareVersion"),

* Optional serial number 7: optional string serialNumber (swagger = "true", swagger.required = "true"),

* Optional imei number 8: optional string imei,

* SIM Card serial number, if available.

9: optional string simSerialNumber,

* Cell provider name, if available.

10: optional string simOperator,

* An additional device identifier, such as a barcode.

11: optional string barcode,

* Phone number from the SIM, if available.

12: optional string simPhoneNumber,

* A bag of attributes that are specific to this device or kind of device

* EXPERIMENTAL: this field may be removed or further defined in future.

13: optional map<string,string> attributes,

* Defines devices that have been filtered out by an admin.

* INTERNAL: this data field should not be populated by clients.

14: optional bool disabledByUser,

* The version number of the hardware ).

15: optional string hardwareVersion (swagger = "true"), } (swagger = "true")

SIGMA Edge API, Release 8.0.1

24 Chapter 4. Appendix and Glossary

Other 4.4. RegisterDevice

* When a device begins to communicate with the server

* this should be sent first.

struct RegisterDevice {

* The protocol version we know about 1: i32 protocolVersion = core.PROTOCOL_VERSION,

* The session key to be used for this connection.

* This should be a securely generated random number *at least*

* 128 bits long 2: optional binary sessionId (swagger.required = "true"),

* The device info being registered 3: optional Device device (swagger.required="true"),

* The time that this request was sent.

4: optional core.timestamp time (swagger.required="true"), Other 4.5. RegisterDeviceResponse

* Device registration response struct RegisterDeviceResponse {

* Code for the response 1: RegisterResponseCode code,

* User intelligible information about the response.

2: string details,

* The protocol version on the server 3: optional i32 protocolVersion;

* DEPRECATED: If an update is available, this is where you'll find it.

4: optional string updateUrl, Other 4.6. RegisterResponseCodes enum RegisterResponseCode {

SIGMA Edge API, Release 8.0.1

4.2. Thrift Messages 25

OK,

PROTCOL_VERSION_MISMATCH,

UNKNOWN_DEVICE,

UPDATE_AVAILABLE

4.2.2 Sensor Registration

Other 4.7. Sensor

* A message describing a sensor. This will be included in a RegisterSensor

* message notifying the server that a new sensor has connected.

struct Sensor {

* An identifier 1: core.UUID id,

* The make 2: string make (swagger = "true"),

* The model 3: string model (swagger = "true"),

* Firmware info 4: optional string firmwareVersion (swagger = "true"),

* The serial number 5: optional string serialNumber (swagger = "true"),

* Type of sensor 6: SensorType type (swagger = "true"),

* The settings for the sensor 7: optional SensorSettings settings (swagger = "true"),

* Enum value that identifies the detector model 9: optional DetectorModel modelType,

* The sensor characterization 10: optional Characterization characterization, SIGMA Edge API, Release 8.0.1

26 Chapter 4. Appendix and Glossary

* The sensor calibration 11: optional Calibration calibration (swagger = "true"),

* If this sensor belongs to a group of colocated sensors,

* this is the unique identifier for that group.

* EXPERIMENTAL: The function of this field may be modified in future.

12: optional core.UUID group,

* Defines sensors that have been filtered out by an admin.

* INTERNAL: this data field should not be populated by clients.

13: optional bool disabledByUser, } (swagger = "true")

Other 4.8. RegisterSensor

* When a device connects to a sensor, it should register that sensor

* with the SIGMA network by sending this message.

struct RegisterSensor {

* The device info being registered 1: optional Sensor sensor (swagger.required="true"),

* The time of the register 2: optional core.timestamp time, Other 4.9. RegisterSensorResponse

* The server should respond to a register with this data struct RegisterSensorResponse {

* Any settings that the sensor should apply

* EXPERIMENTAL: This field may be modified or removed in future iterations.

1: optional SensorSettings sensorSettings,

* The sensor characterization

SIGMA Edge API, Release 8.0.1

4.2. Thrift Messages 27

2: optional Characterization sensorCharacterization,

* The sensor characterization 3: optional Calibration sensorCalibration,

4.2.3 Data Reporting

Other 4.10. SensorReport

* This struct wraps a set of sensor readings. Each

* reading may or may not exist. This is basically a

* way to bundle readings into a single message and

* maintain the ability to cross reference between

* readings.

* A SensorReport should always include at least one of:

* - radiation

* - location

* - sensorStatus struct SensorReport {

1: optional sensor.RadiationReading radiation, 2: optional sensor.LocationReading location, 3: optional device.DeviceStatus deviceStatus, 4: optional sensor.SensorStatus sensorStatus, 5: optional sensor.Sensor sensor, 6: optional core.UUID device

/** Exploratory structures for reporting data from other sensor modalities

* EXPERIMENTAL: these fields may be removed or modified without warning.

100: optional sensor.ChemBioReading chemBioReading, 101: optional sensor.WindReading windReading, Other 4.11. RadiationReading

* A radiation reading. Either count or spectrum are required.

struct RadiationReading {

* Sensor id 1: core.UUID sensor,

* time the reading was taken

SIGMA Edge API, Release 8.0.1

28 Chapter 4. Appendix and Glossary

2: core.timestamp time,

* duration for the reading 3: core.duration sampleDuration,

* A scalar representing neutron counts.

20: optional Count count,

* A gamma spectrum 21: optional Spectrum spectrum,

* Raw spectrum data

* DEPRECATED: This field is deprecated and should not be used.

22: optional list<i16> rawData

Other 4.12. Spectrum Format and Resolution

* Spectrum resolution enum SpectrumResolution {

CH_512 = 512,

CH_1024 = 1024,

CH_4096 = 4096

* Spectrum formats. Sensors may serialize data using one

* of the following data formats.

enum SpectrumFormat {

* Channel data is an array of tuples. Tuples are arranged in

* the array such that the first 16 bit value is the channel number

* and the second 16 bits is the channel count. Channels with

* no counts should be…