SED-DISCoE-ABCD_IRS_001E.pdf

ABCD Interface Requirements Specification SED-DISCoE-ABCD-IRS-001E

INTERFACE REQUIREMENTS

SPECIFICATION FOR

GENERATION OF ARMY BULK

CBM DATA (ABCD) FILES

Updated by the ABCD Practitioners Working Group of The AMC CBM+ Data Management IPT, Files & Messages Workshop

Dept. of Army Civilians Contractors John Porter, AMRDEC Dr. Anthony M. Orme, OASYS Incorporated (DISCoE, SED) Andrew Dunn, TARDEC Beth Allen, OASYS Incorporated (DISCoE, SED)

Gene Belford, CALIBRE Systems, Inc. (LOGSA) Marea Bissonnette, Booz Allen Hamilton (CECOM) David Boylan, Penn State ARL (TACOM/CERDEC)

Gary Hellenga, Booz Allen Hamilton (CECOM) Mike Landgraf, nCode (AMSAA)

Ralph Loser, LogTech (CECOM) Glenn Loveless, Control Point Corporation (TACOM)

Todd Moody, Impact Technologies (CECOM) Xinxin Wang, Booz Allen Hamilton (CECOM)

Version 1.1 Revision 3

Prepared by:

Data and Information Standards Center of Excellence (DISCoE) Aviation and Missile Research, Development and Engineering Center (AMRDEC)

Software Engineering Directorate (SED) Redstone Arsenal, AL 35898-5260

DISTRIBUTION STATEMENT C. Distribution authorized to U.S. Government Agencies and their contractors; (09 Aug 12); other requests for this document shall be referred to:

US Army Aviation and Missile Command

ATTN: AMSRD-AMR-BA

Redstone Arsenal, AL 35898

WARNING - This document contains technical data whose export is restricted by the Arms Export Control Act (Title 22, U.S.C., Sec 2751, et seq.) or the Export Administration Act of 1979 (Title 50, U.S.C., App. 2401 et seq), as amended. Violations of these export laws are subject to severe criminal penalties. Disseminate in accordance with provisions of DoD Directive 5230.25

RECORD OF REVISIONS

REVISION

LETTER

DATE

PAGES INVOLVED

CHANGE NUMBER

Initial Release 3 March 2011

A

B

30 May 2012

06 Dec 2012

Cover Sheet, Distribution statement updated

Reworked section 3. Added new sections and appendix. Added Executive Summary. Changed version to 1.1.

C 25 June 2013 Revision 1 updates Section 4 to list additional recommended practices.

D 28 Sept 2013 All; Revision 2 removes footer containing distribution stmt

E 01 Mar 2014 Page 16; clarify expression for gmt format

Page 20; updates Section 4 to list additional recommended practice.

DOCUMENT NO.

REV LTR

PAGE

NO.

- 3 -

EXECUTIVE SUMMARY

Army Bulk CBM Data (ABCD) is an Army-developed specification on how to leverage NASA’s

Common Data Format (CDF) to produce CBM+ bulk data files. An ABCD file is simply a CDF file that not only contains collected data but also contains enough metadata to uniquely identify the data contained within. Metadata is a simple description mechanism that provides context information about the platform from which the CBM+ data was produced and stored. It identifies the specific vehicle, aircraft or other asset from which data is collected, as well as the specific measurement parameters within the data file. A CBM Data Warehouse receiving

ABCD files would then be able to match the files to assets within its system.

This document describes the ABCD file format for bulk data and defines the structure and form of the required metadata used to identify the information contained within the file. The NASA

CDF library provides programming APIs for creating and reading data files. An overview of the

CDF storage features used to support reading and writing of CBM+ data into ABCD files is provided.

The required ABCD metadata is summarized in a table in Section 3.4 of this document. In order to supplement this summary, examples are provided in the appendix, along with use-cases illustrating how to gather the identifying information and assemble it into an ABCD file along with the bulk CBM+ data.

Since the initial publication of version 1.0 of this document and the subsequent adoption of

ABCD file format for CBM+ bulk data, an active practitioner’s group began working through common case studies to determine how best to write some forms of data into ABCD files. By developing common practices, we can avoid recreating solutions for similar data patterns across the Army. From this working group, a set of recommended practices was produced and published as an addendum to the original ABCD IRS. The current recommended practices suite is listed in Section 4.

- 4 -

TABLE OF CONTENTS

SECTION PAGE

Executive Summary 1 ABCD Introduction 2 Referenced Documents 3 ABCD File Specification

3.1 ABCD FILE SPECIFICATION – USER DATA

3.2 ABCD FILE SPECIFICATION - METADATA

3.2.1 ABCD Attribute Name Part 1: Level

3.2.2 ABCD Attribute Name Part 2: Category

3.2.3 ABCD Attribute Name Part 3: MIMOSA table name

3.2.4 ABCD Attribute Name Part 4: MIMOSA column name

3.3 SUPPORTED DATA TYPES

3.3.1 Integer Data Types

3.3.2 Floating Point Data Types

3.3.3 Character Data Types

3.4 ABCD REQUIRED METADATA

3.4.1 Global Parameters

3.4.2 Zvariable Header Levels & Attributes

4 Recommended Practices 5 CDF Toolkit 6 Data Compression 7 Acronyms 8 CDF Copyright Appendix A

A.1. MIMOSA Site Identifiers A.2. The LVL00.PLAT.Segment.user_tag_ident Attribute A.3. Steps for Specifying Metadata in an ABCD File A.4. Examples

Simple ABCD Example Complex ABCD Example

Appendix B: ABCD Machine Readable Definitions

- 5 -

1 ABCD INTRODUCTION

This document describes both the Army Bulk CBM Data (ABCD) file specification and the application programming interfaces (API) for the ABCD file specification; herein referred to as the ABCD API. The ABCD file specification enables Army platform sensor vendors and others to produce ABCD files.

The ABCD files are based upon the National Aeronautics and Space Administration (NASA)

Common Data Format (CDF). CDF is a self-describing data format for the storage and manipulation of scalar and multidimensional data in a platform- and discipline-independent fashion. Although CDF has its own internal self-describing format, it consists of more than just a data format. CDF is a scientific data management package (known as the "CDF Library") that allows programmers and application developers to manage and manipulate scalar, vector, and multi-dimensional data arrays.

CDF, in its most basic terms, is a conceptual data abstraction for storing, manipulating, and accessing multidimensional data sets. We refer to CDF as a data abstraction because we never discuss the actual physical format in which data sets are stored. Instead, we describe the form of the data sets and the means (interface) by which they may be manipulated.

This important difference from traditional physical file formats is reflected in the orientation of the document toward defining form and function as opposed to a specification of the bits and bytes in an actual physical format. It is important to state here that the use of a data abstraction in no way inhibits access to physical data or necessarily makes such access inefficient. It merely provides a way of generalizing the data model and makes possible the specification of a uniform interface for manipulation of a data set.

The ABCD file specification integrates specific Machinery Information Management Open

Standards Alliance (MIMOSA) metadata into the CDF scalar and/or multidimensional data. The

ABCD file specification allows for future extensibility in terms of the metadata needed for any specific application.

- 6 -

2 REFERENCED DOCUMENTS

CDF V3.4.1 User's Guide (February 2012)

CDF V3.4.1 Reference Manuals (February 2012)

CDF Download Site: http://cdf.gsfc.nasa.gov/

ISO 8601: Data Elements and Interchange Formats – Information Interchange – Representation of Dates and Times

ISO 13374: Condition Monitoring and Diagnostics of Machines – Data Processing, Communication and Presentation

MIL STD-3008B: Department of Defense Standard Practice: Interactive Electronic Technical Manual (IETM) Technical Data Requirements to Support the Global Combat Support System – Army (December 2008)

MIMOSA OSA-EAI Version 3.2.1: Open System Architecture for Enterprise Application Integration

- 7 -

3 ABCD FILE SPECIFICATION

The contents of a CDF fall into two categories: 1) a series of records comprising a collection of variables consisting of scalars, vectors, and n-dimensional arrays, and 2) a set of attribute entries

(metadata) describing the CDF in global terms or specifically for a single variable. This dual function of CDF is what provides its "data set independence." Both the metadata (attributes) and the data objects (variables) are combined into an integrated data set.

The ABCD file specification requires specific global and variable level MIMOSA metadata be written into a CDF file. The producer of the file will then structure their data (zVariable(s)) in the manner that makes most sense for their application. Figure 1 below shows the overall conceptual file structure of an ABCD file.

Figure 1: ABCD File Conceptual Structure

- 8 -

3.1 ABCD FILE SPECIFICATION – USER DATA

An important feature of CDF is that it can handle data sets that are inherently multidimensional in addition to data sets that are scalar. To do this, CDF groups data by "variables" whose values are conceptually organized into arrays.

CDF's "variable" is a generic name for an object that represents data, and it does not have any scientific context associated it. For example, a variable can be data representing an independent variable, a dependent variable, time and date value, or whatever data might be (e.g. image, XML file, etc.). In other words, the variable doesn't contain any hidden meanings other than the data itself. One may describe a variable or one variable's relationship with other variable(s) through

"attributes" (see the last paragraph of this section for more details). The dimensionality of a variable depends upon how the data is specified by the user. For scalar data, as an example, the array of values would be 0-dimensional (i.e., a single value), whereas for image data the array would be 2-dimensional. Similarly, the array for volume data would be 3-dimensional. CDF allows users to specify arrays of up to ten dimensions. The array for a particular variable is called a "variable record." A collection of variable records, one for each variable, is referred to as a

CDF record. A CDF can, and usually does, contain multiple CDF records.

Two types of variables may exist in a CDF: rVariables and zVariables. ABCD compliance requires the use of zVariables as they are more efficient in terms of storage and offer more functionality than an rVariable. An ABCD file will normally contain many zVariables, each of which can contain scalar values or multi-dimensional arrays.

Figures 2 and 3 illustrate 0-dimensional (scalar) and 2-dimensional zVariables, respectively.

Figure 4 illustrates several zVariables each with different dimensionality.

It is important to note that there is no single "correct" way to store data in a CDF. The user has complete control over how the data values are stored in the CDF depending on how the user views the data. This is the advantage of CDF. Data values can be organized in whatever way makes sense to the user.

- 9 -

Figure 2: All zVariables 0 dimensional (scalar) data

- 10 -

Figure 3: All zVariables 2 dimensional data

- 11 -

Figure 4: zVariables with different dimensions of data

- 12 -

3.2 ABCD FILE SPECIFICATION - METADATA

While CDF’s variable is a mechanism for storing/representing data, CDF’s “attribute” is a mechanism for describing the CDF file and the individual CDF variables in the file. There are two types of attributes in CDF: global attributes and variable attributes. Global attributes are used for describing the CDF file and variable attributes are used for describing individual variables. Examples of global attributes would include such things as file creation date, file author, source of data, and data set documentation. Examples of variable attributes would include such things as a field name for the variable, the valid minimum and maximum, the units in which the variable data values are stored, the format in which the data values are to be displayed, and a fill value for errant or missing data.

Both global and variable attribute names specified by ABCD consist of four parts, separated by periods. An example of an attribute name is: LVL00.FILE.Model.model_db_site. The parts of the names explain where and how the metadata is used.

3.2.1 ABCD Attribute Name Part 1: Level

Levels are of form “LVL##” and represent logical groupings of ABCD metadata. Existing levels are listed in Table 1.

Table 1: ABCD Attribute Levels

Level Description

LVL00 Required ABCD metadata described in section 3.4

LVL01 Required ABCD metadata described in section 3.4

LVL02 Optional metadata specified by the ABCD Time Series recommended practices

LVL03 Optional metadata specified by the ABCD Sequencing recommended practices

LVL04 Optional metadata specified by the ABCD Multicolumn recommended practices

LVL05 Optional metadata specified by the ABCD Classification recommended practices

LVL06 Optional metadata specified by the ABCD Asset and Segment recommended practices

LVL07 Optional metadata specified by the ABCD Encoding recommended practices

LVL08 Optional metadata specified by the ABCD Histogram recommended practices

LVL09 Optional metadata specified by the ABCD Geographical Position recommended

- 13 -practices LVL10 Optional metadata specified by the ABCD Missing Value recommended practices

LVL11 Optional metadata specified by the ABCD Engineering Unit recommended practices

LVL12 Optional metadata specified by the ABCD Digital Source Collector recommended practices

LVL13-

LVL89

Reserved for future recommended practices

LVL90-

LVL99

User-defined global and variable attributes. User-defined parameters in ABCD files are expected to follow the four-part structure.

3.2.2 ABCD Attribute Name Part 2: Category

ABCD attributes are separated into categories, describing their general areas of application.

Existing categories are listed in Table 2.

Table 2: ABCD Attribute Categories

Category Description

FILE Attribute applies to the ABCD file itself

PLAT Attribute applies to the platform being measured

ACQ Attribute applies to the data acquisition event

SENS Attribute applies to the sensor being used

DATA Attribute applies to the data being generated

3.2.3 ABCD Attribute Name Part 3: MIMOSA table name

The MIMOSA table name references the table in the MIMOSA Common Relational Information

Specification in which this metadata would be stored. If MIMOSA does not provide a specific table and column for the metadata, this value will represent the document where this metadata is defined. In the previous example of LVL00.FILE.Model.model_db_site, Model is the MIMOSA table name being referenced.

3.2.4 ABCD Attribute Name Part 4: MIMOSA column name

Similar to the MIMOSA table name in section 3.2.3, the MIMOSA column name references the column in the MIMOSA table where the metadata would be stored. If MIMOSA does not provide a specific table and column for the metadata, this value will represent a description of the parameter. In the previous example of LVL00.FILE.Model.model_db_site, model_db_site is the MIMOSA table column being referenced.

- 14 -

3.3 SUPPORTED DATA TYPES

CDF supports a variety of data types consistent with the types available in C, FORTRAN and

JAVA compilers on most computers. All data types are based on an 8-bit byte.

3.3.1 Integer Data Types

CDF_BYTE 1-byte, signed integer.

CDF_INT1 1-byte, signed integer.

CDF_UINT1 1-byte, unsigned integer.

CDF_INT2 2-byte, signed integer.

CDF_UINT2 2-byte, unsigned integer.

CDF_INT4 4-byte, signed integer.

CDF_UINT4 4-byte, unsigned integer.

CDF_INT8 8-byte, signed integer.

3.3.2 Floating Point Data Types

CDF_REAL4 & CDF_FLOAT 4-byte, single-precision floating-point.

CDF_REAL8 & CDF_DOUBLE 8-byte, double-precision floating-point.

3.3.3 Character Data Types

CDF_CHAR 1-byte, character.

CDF_UCHAR 1-byte, unsigned character.

- 15 -

3.4 ABCD REQUIRED METADATA

3.4.1 Global Parameters

Table 3: Level 0, Required for Data Warehouse Processing

Attribute Description CDF Data Type Example Value

Version of the ABCD File Identifies the version of ABCD this file was produced under. For ABCD 1.1, use the example values

The values in this example reference ABCD version 1.*.

Future major versions of ABCD will be assigned new model_ids.

LVL00.FILE.Model.model_db_site Site code for ABCD file version (see App. A.1)

CDF_CHAR,

Length = 16

000003F900000000

LVL00.FILE.Model.model_db_id Database id for ABCD file version

CDF_UINT4 1

LVL00.FILE.Model.model_id id for ABCD file version CDF_UINT4 2

Platform Identification Uniquely identifies the vehicle platform or platform part (e.g. engine or electrical system) covered by this ABCD file.

This segment has a TACOM Enterprise tag (00000426) and a site code assigned by the TACOM enterprise to the M1A2 tank with serial number LA23020U. Within the enterprise, the value 0101000A was assigned to this particular vehicle.

LVL00.PLAT.Segment.segment_site Site code for the segment (see Appendix A.1)

CDF_CHAR,

Length = 16

000004260101000A

LVL00.PLAT.Segment.segment_id Id of segment CDF_UINT4 0 (If the Segment represents the entirety of the segment_site above, the Segment ID should be 0)

LVL00.PLAT.Segment.user_tag_ident String description of the segment being monitored.

(see Appendix A.2).

CDF_CHAR,

Length ≤ 254

NIIN:013285964~SN:LA23020U~DESC:TANK CBT

120MM M1A2

Data Set Group Reference / Measurement Data Acquisition Start Time in GMT

The measurement event fields describe the measurement location set contained in the file. This categorizes an ABCD file based on the measurement locations it contains.

The meas_loc_id of 0 represents the set of measurement locations in this ABCD file. If the set changes, a sensor added, removed, or reconfigured, the id should also change.

- 16 -

LVL00.ACQ.Meas_event.meas_loc_site Site of measurement location set

CDF_CHAR,

Length = 16

000004260101000A

LVL00.ACQ.Meas_event.meas_loc_id Id of measurement location set

CDF_UINT4 0

LVL00.ACQ.Meas_event.gmt_event UTC/GMT/Zulu Timestamp (Date/time in the following ISO 8601 variable length character form:

YYYY-MM-

DDThh:mm:ss{.fffffffff}{Z} where: YYYY = four-digit year, MM = two-digit month, DD = two-digit day of month

T = LITERAL "T"

CHARACTER;

HH = TWO DIGITS OF

HOUR; MM = TWO

DIGITS OF MINUTE;

SS = TWO DIGITS OF

SECOND;

FFFFFFFFF =

REPRESENTS AN

OPTIONAL DECIMAL

FRACTION OF A SECOND

TO THE BILLIONTH OF A

SECOND; AND

Z = LITERAL Z

CHARACTER

REPRESENTING ZULU

TIME ZONE (OPTIONAL)

CDF_CHAR,

Length >= 19 and Length <= 29

2012-09-08T13:08:22Z

- 17 -

Table 4: Level 1, Required for Data Warehouse Processing

Attribute Description CDF Data Type Example Value

Platform Site Identification Uniquely identifies the platform.

Will often match segment_site

This is the same tank used above.

LVL01.PLAT.Site.site_code Site code (see Appendix A.1) CDF_CHAR, Length = 16

000004260101000A

Classification (Type) of ABCD File Information to classify the type of platform data captured in this ABCD file. Also called “blob” data type. If the file has been converted from another standard format, that format would be represented here. Defined by MIMOSA or the Enterprise that owns the original file format.

This is the MIMOSA set of ids for ABCD, which should be used if ABCD is the original file format for the data.

LVL01.FILE.Blob_Data_Type.bd_db_site MIMOSA Site ID (see Appendix A.1)

CDF_CHAR,

Length = 16

0000000000000000

LVL01.FILE.Blob_Data_Type.bd_db_id MIMOSA Database ID for

ABCD

CDF_UINT4 4

LVL01.FILE.Blob_Data_Type.bd_type_code MIMOSA Type Code for ABCD CDF_UINT4 6

- 18 -

3.4.2 Zvariable Header Levels & Attributes

Table 5: Level 0, Required for Each zVariable

Attribute Description CDF Data Type Example Value

Measurement Location (Sensor) Identification

For the data contained in this zVariable, uniquely identify from which sensor it was captured.

88 refers to a sensor output (e.g. engine oil temperature) on the tank referenced above.

LVL00.SENS.Meas_Location.meas_loc_site Site id for the sensor (see Appendix A.1)

CDF_CHAR,

Length = 16

000004260101000A

LVL00.SENS.Meas_Location.meas_loc_id Sensor id, must be unique within site

CDF_UINT4 88

MIMOSA OSA-CBM

DA/DM/SD/HA/PA/AG Output Data Type

For the data contained in this zVariable, classify its output type: DA (Data Acquisition), DM (Data Manipulation), SD (State Detection), HA (Health Assessment), PA (Prognostics Assessment), AG (Advisory Generation) as defined in ISO 13374

This example is data acquisition data, so it will be given id 1 for DA. The ml_db_site and ml_db_id remain constant.

LVL00.SENS.Meas_Loc_Type.ml_db_site MIMOSA site assumed CDF_CHAR, Length = 16

000003F700000001

LVL00.SENS.Meas_Loc_Type.ml_db_id MIMOSA db containing these types

CDF_UINT4 1

LVL00.SENS.Meas_Loc_Type.ml_type_code DA for raw data, DM for outputs of algorithms, SD for indicators of platform state, HA for health assessments, PA for prognostics assessments, AG for advisories

CDF_UINT4 1 = DA,

2 = DM,

3 = SD,

4 = HA,

5 = PA,

6 = AG

- 19 -

4 RECOMMENDED PRACTICES

Several recommended practices have been developed by the user community and are maintained by DISCoE. Each of these recommended practices cover a special situation that benefits from a common, agreed-upon approach across the enterprises that may need to implement them within an ABCD file. By using recommended practices, it also enables the development of common services that utilize the information contained within the metadata of an ABCD file that may be optional, but not unique enough to apply to the user-defined levels (level 90 and above). A list of recommended practices that currently exist is shown below in Table 6.

It is important to stress that these recommended practices cover optional metadata that apply to situations that may not be encountered by all projects. Therefore, they are supplied as additional documents that supplement this IRS.

Table 6: Recommended Practices

Title

ABCD

LEVEL

Reference File Name

Description

Time Series Recommended Practices

LVL02 SED-DISCoE-ABCD-RP- LVL02-Time-Series_001.pdf

Methods for storing time-series data and guidelines for selecting between different time series methods.

Sequencing ABCD Files Recommended Practices

LVL03 SED-DISCoE-ABCD-RP- LVL03-Sequencing_001.pdf

Method for sequencing and linking multiple CBM data files together from a single CBM session.

Multicolumn Recommended Practices

LVL04 SED-DISCoE-ABCD-RP- LVL04-Multicolumn_001.pdf

Method for linking zVariables together to create structured data.

Classification Recommended Practices

LVL05 SED-DISCoE-ABCD-RP- LVL05-Classification_001.pdf

Method for grouping CBM data into discrete sets. Also referred to as Engineering Study.

Asset and Segment Recommended Practices

LVL06 SED-DISCoE-ABCD-RP- LVL06-Asset-Segment_001.pdf

Description of parameters for describing segments and assets.

Data Type Encoding Recommended Practices

LVL07 SED-DISCoE-ABCD-RP- LVL07-Encoding_001.pdf

Method for describing encoding applied to data values stored in an ABCD file for the purposes of reducing data size.

Histogram Recommended Practices

LVL08 SED-DISCoE-ABCD-RP- LVL08-Histogram_001.pdf

Description of metadata used to describe and represent the most common aspects of histogram data.

- 20 -

Title

ABCD

LEVEL

Reference File Name

Description

Geographical Position Recommended Practices

LVL09 SED-DISCoE-ABCD-RP- LVL09-Geo_Position_001.pdf

Method for providing geographical position metadata to specify the region a platform is in without exposing actual coordinates.

Missing Value Recommended Practices

LVL10 SED-DISCoE-ABCD-RP- LVL10-Missing-Value_001.pdf

Method for defining placeholder values to represent missing values in a series of data.

Engineering Unit Recommended Practices

LVL11 SED-DISCoE-ABCD-RP- LVL11-Engineering- Unit_001.pdf

Method for including engineering unit descriptions in a file for analysts’ convenience.

Digital Source Collector Recommended Practices

LVL12 SED-DISCOE-ABCD-RP-

LVL12-DSC_001.pdf

Method for including metadata that indicates which version of a digital source collector produced data in the ABCD file.

- 21 -

5 CDF TOOLKIT

A set of utility programs is provided with the CDF distribution that allows a user to perform a variety of operations on CDFs without having to write an application program. Each toolkit program is described in detail in Chapter 3 of the CDF V3.2 User’s Guide.

The available toolkit programs are as follows:

CDFedit - Allows the display, creation, and modification of attribute and variable data in a CDF.

CDFexport - Allows the contents of a CDF to be exported to the terminal screen, a text file, or another CDF. The CDF may be filtered in order to export a subset of its contents.

CDFconvert - Allows the format, encoding, majority, compression, and sparseness of a CDF to be changed. It can also reorganize a fragmented CDF file to make file access more efficient. In all cases a new CDF is created. The original CDF is not modified.

SkeletonCDF - Reads a specially formatted text file (called a skeleton table) and creates a skeleton CDF. A skeleton CDF is complete except for record-variant data.

SkeletonTable - Reads a CDF and produces a specially formatted text file called a skeleton table.

The skeleton table may be modified and then input to SkeletonCDF to create a skeleton CDF.

CDFinquire - Displays the version of your CDF distribution, many of the configurable parameters, and the default CDF toolkit qualifiers.

CDFstats - Produces a report containing various statistics about the variables in a CDF.

CDFcompare - Reports the differences between two CDFs.

CDFdir - Produces a directory listing of a CDF's files. For a multi-file CDF the variable files are listed in ascending numerical order.

- 22 -

6 DATA COMPRESSION

The CDF interface supports several forms of compression for CDF files. One can compress an entire file, or one can compress individual zVariables. The advantage of compressing at the zVariable level, rather than at the entire file level, is that only the zVariable being read needs to be decompressed upon retrieval. When a CDF file is compressed, the library decompresses the entire file before working with it. The CDF library handles the compression and decompression of files and variables, making the process transparent to the user.

CDF supports four types of compression at both the file and variable levels. Selecting the proper algorithm to use will depend on the characteristics of the data being compressed. CDF supports the following compression algorithms: Run-Length Encoding, Huffman, Adaptive Huffman, and

GZIP. GZIP compression can be tailored to levels 1 through 9, where level 9 is the greatest amount of compression but has the slowest execution speed.

- 23 -

7 ACRONYMS

ABCD Army Bulk CBM Data AMRDEC Aviation and Missile Research, Development and Engineering Center API Application Programming Interface CBM Condition-Based Maintenance CCBMDW Common CBM Data Warehouse CDF Common Data Format CECOM U.S. Army Communications-Electronics Command DISCoE Data and Information Standards Center of Excellence IRS Interface Requirements Specification ISO International Organization for Standardization GMT Greenwich Mean Time LOGSA Logistics Support Activity MIMOSA Machinery Information Management Open Standards Alliance OEM Original Equipment Manufacturer OSA-CBM Open Systems Architecture for Condition-Based Maintenance NASA National Aeronautics and Space Administration SED Software Engineering Directorate TACOM U.S. Army Tank-automotive and Armaments Command USAMC United States Army Materiel Command UTC Coordinated Universal Time XML Extensible Markup Language

- 24 -

8 CDF COPYRIGHT

Copyright © 2012 NASA/GSFC

Space Physics Data Facility

NASA/Goddard Space Flight Center

Greenbelt, Maryland 20771 (U.S.A)

This software may be copied or redistributed as long as it is not sold for profit, but it can be incorporated into any other substantive product with or without modifications for profit or non-profit. If the software is modified, it must include the following notices:

The software is not the original (for protection of the original author's reputations from any problems introduced by others)

Change history (e.g. date, functionality, etc.)

This copyright notice must be reproduced on each copy made. This software is provided as is without any express or implied warranties whatsoever.

- 25 -

APPENDIX A

This Appendix covers more information to help implementers gather the necessary information to write ABCD files. It includes sections about MIMOSA Site codes and

LVL00.PLAT.Segment.user_tag_ident, and provides some examples of building an ABCD file using the required information covered in this document.

A.1. MIMOSA Site Identifiers

In MIMOSA, sites represent entities that own assets, segments, agents, and measurement locations. For a fleet management scenario, mobile platforms such as trucks, aircraft, or ships will all be sites. In some scenarios, independently managed systems, such as radios or generators, can also be considered sites. Once assigned, a site code is never changed, even if the platform is transferred to another enterprise.

Sites are CDF_CHAR length 16, indicating a hexadecimal string with values 0-9A-F. Although hexadecimal string letters can be upper- or lower-case, upper-case letters are expected for site ids. Sites are comprised of two 8 character hexadecimal parts, the enterprise id and the site id.

Enterprise ids are assigned by the MIMOSA standards body to organizations that produce site ids. Site ids are created and managed by an Enterprise and must be unique within an Enterprise.

Under this condition, a 16 character site is universally unique.

The MIMOSA body is enterprise id 0 and maintains the site 0000000000000000. It is recommended practice to make site zero in an enterprise the site used for common types used by many sites across the enterprise.

Example:

Penn State Applied Research Laboratory’s enterprise id is 1063. In hexadecimal, this value is

427. Penn State ARL’s base types site is 0000042700000000. An example platform being monitored by ARL uses site 000004270101000A.

In the required ABCD metadata alone, 7 site codes must be identified. Of these, three of them are constant (LVL00.FILE.Model.model_db_site, LVL01.FILE.Blob_Data_Type.bd_db_site,

- 26 -and LVL00.SENS.Meas_Loc_Type.ml_db_site,) requiring no additional decisions. This leaves

4 site codes that must be assigned by the implementer:

LVL01.PLAT.Site.site_code – the platform being monitored

LVL00.PLAT.Segment.segment_site – the definer of the part being monitored (can be same as the platform)

LVL00.ACQ.Meas_event.meas_loc_site – the definer of the measurement location set.

LVL00.SENS.Meas_Location.meas_loc_site (zVariable attribute) – the definer of each individual sensor location

In most basic scenarios, these four sites will be the same: the site of that platform being monitored. The extra flexibility allowed by the four site codes is required by mature and advanced MIMOSA implementations.

A.2. The LVL00.PLAT.Segment.user_tag_ident Attribute

The LVL00.PLAT.Segment.user_tag_ident attribute is a text field used for describing the platform or platform part that produced the data in an ABCD file. ABCD requires no structure to this field beyond being a CDF_CHAR data type, allowing systems to use their own identifiers.

However, LOGSA recommends using the user_tag_ident field to hold common Army identifiers like NIIN, ECC, serial number, and nomenclature. When creating ABCD files to be stored in

LOGSA’s Common CBM Data Warehouse, a specific format should be used in this field.

Identifiers are described in the format “IDTYPE:VALUE”, for example, “NIIN:01234568”.

Multiple identifiers can be included separated by the tilde (~) character as shown in the examples below:

Example:

Good "NIIN:013558250~SN:0005159~DESC:HELIC ATTACK AH-64D"

Better "NIIN:013558250~SN:0005159~NOMEN:HELIC ATTACK AH-64D~MDS:AH-64D"

As such, LOGSA uses the LVL00.PLAT.Segment. user_tag_ident field to provide an alternate method for identifying the Model and Serial Number if the MIMOSA tags are not available to

- 27 -the application generating the ABCD file. The tags ('CAGE', 'DODAAC', 'ECC', 'FSC', 'LIN', 'MACOM', 'MDS', 'NHA', 'NHA_WUC', 'NIIN', 'NOMEN', 'NSN', 'SN', 'SN_WO_DASH', 'SN_W_DASH', 'TN', 'UIC', 'WUC') in the AS_CHR_DAT_TYPE table would be used the

LOGSA Bulk File Handler to research the MIMOSA tags in the Army master tables. See MIL

STD 3008B for further description of these tags.

A.3. Steps for Specifying Metadata in an ABCD File

This section contains a how-to guide for determining what values go in each required section of

ABCD metadata. Several steps involve obtaining or creating identifiers within a data warehouse, such as LOGSA’s CCBMDW or DISCoE’s data warehouse. If neither of these storage locations is accessible for your application, then the identifiers must be created and maintained in some fashion by the enterprise generating them. However, cooperation with one of the data warehouse organizations is highly recommended.

Step 1: Assign the LVL00.FILE.Model parameters. For version 1.* of ABCD, this will always be (‘000003F900000000’,1,2).

Step 2: Assign the LVL01.PLAT.Site.site_code parameter. In a parts breakdown structure, this is the highest level platform to which all parts belong. The id for this should have already been created through negotiations with the data warehouse.

Step 3: Assign the LVL00.PLAT.Segment parameters. In a parts breakdown structure, this is the lowest level part to which all measurements within the file pertain. For instance, if an ABCD file contains nothing but engine data from a HEMTT, the Site in Step 2 is the HEMTT, but the

Segment is the engine. The id for this should have already been created through negotiations with the data warehouse.

Step 4: (optional) Build the LVL00.PLAT.Segment user_tag_ident. See section A.2.

Step 5: Assign the LVL00.SENS.Meas_location parameters for each zVariable included within the file. The list of these should have been created before reaching the stage of writing ABCD

- 28 -files; ordinarily it consists of the site id of the platform for meas_loc_site and an auto incrementing number for the meas_loc_id.

Step 6: Assign the LVL00.SENS.Meas_Loc_Type parameters for each zVariable included within the file. This results from a judgment of whether the data is raw (1), converted in some way to information (2), a state indicator (3), a health or diagnostic indicator (4), a prognostic indicator (5), or an advisory (6).

Step 7: For the list of Meas_location parameters created in Step 5, create a new measurement location that represents that exact set. It is given an arbitrary meas_loc_id, perhaps the next incremented number, and put in LVL00.ACQ.Meas_event. If the list of sensors changes in any way for other files or new configurations of the software, a new number should be generated to represent the new set. The gmt_event time will be created programmatically at the time of writing each file.

Step 8: Assign the LVL01.File.Blob_Data_Type parameters. If the original format of your file is ABCD, then the values (‘0000000000000000’,4,6) should be used. If you are translating the file from another format to ABCD, a set of values representing the original format should be used. First, ask the data warehouse if they have a record of the original file format. If they do not, they will create identifiers for you to use.

- 29 -

A.4. Examples

Simple ABCD Example

The diagram below illustrates the contents of a representative ABCD File containing measurement data for multiple locations on a single asset. The ABCD File contains a single set of global attributes. Each global attribute name / value pair represents a MIMOSA metadata item and its value. The sample shows examples of the required LVL00 and LVL01 items.

Each of the zVariables contains some type of measurement data. There is a set of named Variable Attributes that describes the zVariables using required LVL00 MIMOSA metadata. For each of the required variable attributes, every zVariable must supply an entry value.

Table 7: Simple ABCD Example

Global Attributes

LVL00.FILE.Model.model_db_site "000003F900000000" LVL00.FILE.Model.model_db_id 1 LVL00.FILE.Model.model_id 2 LVL00.PLAT.Segment.segment_site "000004300001E396" LVL00.PLAT.Segment.segment_id 123798 LVL00.PLAT.Segment.user_tag_ident "NIIN:013558250~SN:0005197" LVL00.ACQ.Meas_Event.gmt_event "2007-06-12T01:44:07" LVL00.ACQ.Meas_Event.meas_loc_site "000004300001E396" LVL00.ACQ.Meas_Event.meas_loc_id 1 LVL01.PLAT.Site.site_code "000004300001E396" LVL01.FILE.Blob_Data_Type.bd_db_site “0000041D00000001" LVL01.FILE.Blob_Data_Type.bd_db_id 1 LVL01.FILE.Blob_Data_Type.bd_type_code 103

- 30 -

ZVariables

Variable 1 (Variable Attributes)

LVL00.SENS.Meas_Location.meas_loc_site "000004300001E396" LVL00.SENS.Meas_Location.meas_loc_id 74 LVL00.SENS.Meas_Loc_Type.ml_db_site "000003F700000001" LVL00.SENS.Meas_Loc_Type.ml_db_id 1 LVL00.SENS.Meas_Loc_Type.ml_type_code 1

VARIABLE 1 MEASUREMENT DATA …

Variable N (Variable Attributes)

LVL00.SENS.Meas_Location.meas_loc_site "000004300001E396" LVL00.SENS.Meas_Location.meas_loc_id 75 LVL00.SENS.Meas_Loc_Type.ml_db_site "000003F700000001" LVL00.SENS.Meas_Loc_Type.ml_db_id 1 LVL00.SENS.Meas_Loc_Type.ml_type_code 1

VARIABLE N MEASUREMENT DATA …

- 31 -

Complex ABCD Example

This example implements metadata for a Command Post Platform (Asset #1) with two primary subcomponents, a Power Supply (PS, Asset #2) and an Environmental Control Unit (ECU, Asset #3.) Each component is assigned identifiers as shown in Figure 5.

Figure 5: Command Post Platform with associated metadata

Each component is a serialized asset given a MIMOSA site id, using the enterprise 0000041E (CECOM). Each asset packages and creates its own ABCD file. One or two sensors from each component were chosen to represent the data in the file and given measurement location ids.

The metadata for each file, organized by the steps described in Section 4, above, is shown in the following tables:

Table 8: Complex ABCD Example, Global Attributes

Step ABCD Tag Asset #1 Asset #2 Asset #3 Version of the ABCD File

1 LVL00.FILE.Model.model_db_site 000003F900000000 000003F900000000 000003F900000000

LVL00.FILE.Model.model_db_id 1 1 1

LVL00.FILE.Model.model_id 2 2 2

Platform Site Identification

2 LVL01.PLAT.Site.site_code 0000041E00000001 0000041E00000001 0000041E00000001

Platform Identification

3 LVL00.PLAT.Segment.segment_site 0000041E00000001 0000041E00000003 0000041E00000002

LVL00.PLAT.Segment.segment_id 0 0 0

4 LVL00.PLAT.Segment.user_tag_ident See Section 3 above (See Section 3) (See Section 3)

Data Acquisition Start Time in GMT

7 LVL00.ACQ.Meas_event.meas_loc_site 0000041E00000001 0000041E00000003 0000041E00000002

LVL00.ACQ.Meas_event.meas_loc_id 3 3 2

LVL00.ACQ.Meas_event.gmt_event 2012‐02‐01T08:00:00Z 2012‐02‐01T08:00:00Z 2012‐02‐01T08:00:00Z

Classification (Type) of ABCD File

8 LVL01.FILE.Blob_Data_Type.bd_db_site 0000000000000000 0000000000000000 0000000000000000

LVL01.FILE.Blob_Data_Type.bd_db_id 4 4 4

LVL01.FILE.Blob_Data_Type.bd_type_code 6 6 6

- 32 -

Table 9: Complex ABCD Example, zVariable Attributes z_batt_curr Asset #1 z_batt_volt Asset #1 z_chip_temp Asset #2 z_chip_rul Asset #2 z_cool_pres Asset #3

Measurement location (sensor) Identification LVL00.SENS.Meas_Location.meas_loc _site 0000041E0000001 0000041E0000001 0000041E0000003 0000041E0000003 0000041E00000021

LVL00.SENS.Meas_Location.meas_loc_id 1 2 1 2 1

MIMOSA OSA‐CBM Output Data Type LVL00.SENS.Meas_Loc_Type.ml_db_site 000003F700000001 000003F700000001 000003F700000001 000003F700000001 000003F700000001

LVL00.SENS.Meas_Loc_Type.ml_db_id 1 1 1 1 1

LVL00.SENS.Meas_Loc_Type.ml_type_code 1 1 1 5 1

Notes:

Each asset has different Segment.segment_site ids, but the same Site.site_code. This indicates they all belong to the same platform, but the data within only pertains to their individual assets.

The LVL00.ACQ.Meas_event values were simply assigned the next measurement location available within each asset. These values represent the set of measurement locations within the asset (or Data Event Set.) The gmt_event time must be determined programmatically.

The Meas_Loc_Type.ml_type_code for z_chip_rul in Asset#5 is 5, because it is a remaining useful life calculation, and thus belongs in the Prognostics Layer (PA.) The rest are all 1, signifying that they are raw data in the Data Acquisition Layer (DA.)

- 33 -

APPENDIX B: ABCD MACHINE READABLE DEFINITIONS

The ABCD Machine Readable Definitions, located in SED-DISCoE-ABCD- Machine_Readable_Definitions.zip, contain formal rules detailing the usage of ABCD attributes.

The definitions have been created to enable the consistent development of ABCD validators.

The definitions unambiguously define the types, formats, and relationships of every attribute defined in this document and the recommended practices.

The ABCD Machine Readable Definitions describe only syntactical requirements of ABCD attributes. Semantic requirements are best understood through the descriptions and examples contained within this document and the accompanying recommended practices. The ABCD Machine Readable Definitions are intended to resolve ambiguities in the documents and serve as inputs to ABCD file validation.

Further information about the organization and meaning of the ABCD Machine Readable Definitions can be found in SED-DISCoE-ABCD-Machine_Readable_Def_Overview.pdf contained within the definitions package.