Task_1_Reference_Document__LSDS_749.pdf

LSDS-749

Version 8.0

Department of the Interior U.S. Geological Survey

LANDSAT 8 (L8)

MISSION DATA

DATA FORMAT CONTROL BOOK (DFCB)

January 2014

- ii - LSDS-749

Contents

Document History ............................................................. Error! Bookmark not defined.

Contents ......................................................................................................................... ii

List of Figures .............................................................................................................. iii

List of Tables ................................................................................................................ iii

Section 1 Introduction

1.1 Document Organization

Section 2 Mission Data Content Overview

2.1 File Structure

2.2 File Name Format

2.2.1 GNE-Collected Files

2.2.2 GNE-Created Files

Section 3 Instrument Characteristics

3.1 OLI

3.1.1 OLI Bands

3.1.2 Organization of Detectors

3.1.3 Blind Band

3.1.4 14-bit Detectors / 12-bit Samples

3.1.5 Pixel Readout

3.1.6 OLI Interval Structure

3.2 TIRS

3.2.1 TIRS Bands

3.2.2 Organization of Detectors

Section 4 Data Definition

4.1 Interval Definition File (IDF)

4.1.1 IDF Record

4.1.2 Header Record

4.1.3 Root File Record

4.1.4 File Record

4.1.5 Scene Record

4.2 OLI and TIRS Mission Data Files

4.2.1 Mission Data File Format

4.2.2 Ancillary Data Packets

4.2.3 OLI Data Packet Types

4.2.4 TIRS Data Packet Types

4.3 Checksum File

4.3.1 Checksum Column

4.3.2 Filename Column

Appendix A Example Interval Definition Files

Appendix B Example Checksum File

Appendix C Glossary

References

- iii - LSDS-749

List of Figures

Figure 3-1. SCA Placement in the Focal Plane Figure 3-2. Detector Placement on Adjacent SCAs Figure 3-3. VRP Detector Placement

Figure 3-4. Blind Band Pixel Arrangement Figure 3-5. OLI Response to Stimulation Lamp Source Figure 3-6. OLI Band Output Order Figure 3-7. OLI SCA Readout Directions Figure 3-8. OLI Interval Overview

Figure 3-9. TIRS Band Output Order Figure 3-10. TIRS Focal Plane Figure 4-1. OLI Mission Data File Structure

Figure 4-2. TIRS Mission Data File Structure Figure 4-3. TIRS Image Frame Packet Organization

List of Tables

Table 2-1. Interval Types and Abbreviations Table 2-2. Mission Data File Name Convention

Table 2-3. File Contents and Extension Fields in File Name Format Table 2-4. Landsat Interval ID Convention for Earth Image Intervals Table 2-5. Landsat Interval ID Convention for Calibration Intervals

Table 4-1. Mission Data Packet Format

Table 4-2. Mission Data IDs for Mission Data Packets Table 4-3. Ancillary Data Packet Organization Table 4-4. Processed Attitude Message

Table 4-5. Attitude Filter States Message Table 4-6. IMU Gyro

Table 4-7. IMU Latency Table 4-8. Ephemeris Message

Table 4-9. GPS Receiver Position Channel Status Message Table 4-10. GPS Range Data Table 4-11. Star Tracker Quaternion Message Table 4-12. Star Tracker Centroid Message Table 4-13. OLI Telemetry Group 3

Table 4-14. OLI Telemetry Group 4 Table 4-15. OLI Telemetry Group 5

Table 4-16. OLI and TIRS Spacecraft Temperatures Table 4-17. Gyro Temperatures Table 4-18. TIRS Telemetry Block1 Table 4-19. TIRS Telemetry Block2 Table 4-20. TIRS Telemetry Block3 Table 4-21. TIRS Telemetry Block4 Table 4-22. Frame Header Structure

- iv - LSDS-749

Table 4-23. OLI CRC Definition

Table 4-24. Image Header Structure Table 4-25. TIRS Frame Header Structure

Table 4-26. TIRS CRC Definition

- 1 - LSDS-749

Section 1 Introduction

This Data Format Control Book (DFCB) provides a detailed description of the mission data that the Ground Network Element (GNE) generates for Landsat 8 (L8). Mission data consists of GNE collected and/or created data files. This DFCB describes the format and data content of each file for all collections received from the GNE. This information includes all data collected from the Operational Land Imager (OLI) and the Thermal Infrared Sensor (TIRS). This document does not describe or define the format for any derived OLI and TIRS data products.

1.1 Document Organization

This document contains the following sections:

Section 1 provides an introduction

Section 2 describes the general contents of the files and how the data are logically arranged

Section 3 describes characteristics of the OLI and TIRS instruments

Section 4 describes each file in detail

Appendix A lists example Interval Definition Files (IDFs)

Appendix B lists example Message-Digest algorithm 5 (MD5) checksum files

Appendix C lists glossary terms

The References section contains a list of reference documents

- 2 - LSDS-749

Section 2 Mission Data Content Overview

The Collection Activity Planning Element (CAPE) and the Mission Operations Element (MOE) schedule collection intervals. The CAPE schedules earth imaging intervals and the MOE schedules all other types of intervals. The term “interval” refers to the period of time scheduled and any data the spacecraft recorded during that time. The spacecraft is capable of recording OLI, TIRS, or both during an interval collection. Each mission data interval contains the following data:

OLI instrument data collected during the interval

TIRS instrument data collected during the interval

Ancillary data that the spacecraft collected during the interval

Metadata that GNE collected and/or generated in the course of planning, receiving, and packaging the mission data for the interval

Mission data consists of data from any type of interval. Each interval consists of only one type. For example, a single interval does not contain both earth image data and shutter calibration data. Table 2-1 lists the types of intervals, along with a single-letter abbreviation used to refer to these interval types. These abbreviations are referenced at locations within this document.

- 3 - LSDS-749

Category Interval Type ID Frequency

Primary Earth Imaging I As scheduled by CAPE, typically every orbit

Calibration Stellar T One to two times during commissioning

Lunar U One complete set every 28 days

Slew Imaging R Before and after every U interval

Side Slither Y Once per month

OLI Lamp L Once per day

OLI Solar O Once every eight days

OLI Shutter S Before and after other intervals as required

OLI Shutter Integration Time Sweep H Once before and after every Z interval

OLI Extended Shutter (36 minute) M Once every 6 months

OLI Solar Integration Time Sweep Z Once every 16 days

TIRS Blackbody B Before and after other intervals as required

TIRS Deep Space D Before and after other intervals as required

TIRS Integration Time Sweep G Once every 16 days

TIRS Transmit All F Once per quarter

Test Engineering E Observatory Integration and Test (I&T) only

Solid State Recorder (SSR) Pseudo- Noise (PN) Test Sequence

P Observatory I&T only

OLI or TIRS Test Patterns Q Observatory I&T only

Table 2-1. Interval Types and Abbreviations

2.1 File Structure

Mission data consists of GNE-collected files and GNE-created files. International Cooperators (ICs) may also collect the mission data files or create the IDF. Only GNE generates the checksum file.

GNE- or IC-collected files:

Mission Data Files. All instrument and ancillary data are stored in one or more mission data files. A mission data file may contain either OLI or TIRS data, but never both. The files are referred to as OLI mission data files and TIRS mission data files. These files contain all instrument data and ancillary data interleaved in a time-ordered fashion. Each mission data file is less than 1.0 Gigabyte (GB) in size.

The TIRS Transmit All collects are comprised of "special" mission files that do not conform to this DFCB.

- 4 - LSDS-749

GNE- or IC-created file:

Interval Definition File (IDF). The IDF provides information about the interval, including the list of files and their respective sizes. The IDF includes the list of scenes and a priority flag for each interval. (See Appendix A for example IDFs.)

GNE-created file:

Checksum File. The checksum file lists each file in the mission data interval directory (excluding the checksum file) and the calculated MD5 checksum. The purpose of the checksum file is to ensure that downstream processing systems can verify that all files are present and received without corruption.

The following sections discuss each file in more detail.

2.2 File Name Format

GNE renames the GNE-collected mission data files using the Solid State Recorder (SSR) root file directory / sub-file name that the data was stored to and the date / time the data was downlinked. The GNE-collected data files are named in accordance to section 2.2.1. The GNE-generated IDF and checksum file names are based on the Landsat Interval Identifier (ID). The form of the Landsat Interval ID depends on the type of collection. Earth viewing intervals have a Landsat Interval ID that includes the World Reference System-2 (WRS-2) path and row information for the interval. Calibration intervals, which do not collect data corresponding to the WRS-2 grid, have a Landsat Interval ID that indicates the collection type and time of day for the collection. The detail for mission data files, both types of Landsat Interval IDs, and example file names are given in the following sections.

2.2.1 GNE-Collected Files

GNE-collected mission data files are named according to the following format:

RRR.ZZZ.YYYYdoyHHMMSSsss.XXX

Parameter Description Filename Position

Values

The root file directory number on the SSR where the data was stored.

RRR 001–511

The sequence (or sub-file) of this file within the root file. ZZZ 000–127

The year the data was received. YYYY 2012–2999

The number of the day within the year the data was received. doy 001–366 (366 only valid for leap years)

Hour of the day the data was received. HH 00–23

Minute of the hour the data was received. MM 00–59

Second of the minute the data was received. SS 00–60 (60 only valid when a leap second is added)

Fraction of the second the data was received. sss 000–999

Ground Station Identifier (GSI) where the data was received. XXX Any valid three letter

- 5 - LSDS-749

Parameter Description Filename Position

Values

GSI. Example: LGS

Table 2-2. Mission Data File Name Convention

2.2.2 GNE-Created Files

The IDF and checksum file names are based on the Landsat Interval ID and their contents:

<Landsat Interval ID>_<File Contents>.<File Extension>

Table 2-3 lists the file contents and file name extensions.

File Type File Contents File Extension

Interval Definition File IDF xml

Checksum File MD5 txt

Table 2-3. File Contents and Extension Fields in File Name Format

The Landsat Interval ID differs slightly for earth image and calibration collections. The file name formats and examples are described in the following sections.

2.2.2.1 Earth Image Intervals

The Landsat Interval ID naming convention for earth image L8 collections is VINpppRRRrrrYYYYdddGSIvv. Table 2-4 describes the components of the filename format. The grayed parameters highlight intervals that are not used in calibration collections.

Parameter Description Filename Position Values

<Vehicle> V L = Landsat

<Instrument> I O = Operational Land Imager T = Thermal Infrared Sensor C = Combined (both OLI and TIRS)

<Vehicle Number> N 8 = Landsat 8

<WRS-2 Path> ppp 1–233 (starting path)

<WRS-2 Start Row> RRR 1–248

<WRS-2 End Row> rrr 1–248

<Year> YYYY Four digit acquisition starting year

<Day> ddd Three digit acquisition starting day of year

<Ground Station> GSI Ground Station Identifier

<version> vv Version (vv = 00–99 )

Table 2-4. Landsat Interval ID Convention for Earth Image Intervals

2.2.2.1.1 Example Earth Image File Names

The following example displays five OLI mission data files and two TIRS mission data files. The files are for an OLI and TIRS L8 image, which collects over path 222, rows

- 6 - LSDS-749

001–012 on day 286 of 2014, and are received at Ground Stations within the Landsat Ground Network (LGN).

267.000.2014286134235476.LGS

267.001.2014286134452345.LGS

267.002.2014286134723213.GLC

267.003.2014286134854745.GLC

267.004.2014286134928645.LGS

442.000.2014286135234165.LGS

442.001.2014286135332675.LGS

LC82220010122014286LGN00_IDF.xml

LC82220010122014286LGN00_MD5.txt

The following example displays three OLI mission data files. The files are for an OLI-only L8 image, which collects over path 187, rows 001–005 on day 341 of 2014, and are received at the Alice Springs, Australia (ASA) Ground Station.

124.000.2014341185434398.ASA

124.001.2014341185512511.ASA

124.002.2014341185558031.ASA

LO81870010052014341ASA00_IDF.xml

LO81870010052014341ASA00_MD5.txt

2.2.2.2 Calibration and Test Intervals

For calibration intervals, the path and row information (pppRRRrrr) is replaced with collection type and collection Universal Time Code (UTC) start time (00DHHMMSS).

Table 2-5 describes these components.

Parameter Description Filename Position Values

<Vehicle> V L = Landsat

<Instrument> I O = Operational Land Imager

T = Thermal Infrared Sensor

C = Combined (both OLI and TIRS)

<Vehicle Number> N 8 = Landsat 8

<filler> 00 00

<Collect Type> D Any of the one-letter abbreviations in Table 2-1 except I

<Hour> HH 0–23 (UTC)

<Minute> MM 0–59 (UTC)

<Second> SS 0–60 (allows leap second) (UTC)

<Year> YYYY Four digit acquisition starting year

<Day> ddd Three digit acquisition starting day-of-year

<Ground Station> GSI Ground Station Identifier

<version> vv Version (vv = 00–99 )

Table 2-5. Landsat Interval ID Convention for Calibration Intervals

- 7 - LSDS-749

2.2.2.2.1 Example Calibration File Names

The files created for an L8 OLI shutter calibration collect data on day 265 of 2014 at 12:34:56 UTC and are received at the Landsat Ground System (LGS) Ground Station, which is part of the LGN. The number of mission data files depends on the data volume. The files are named as follows:

267.000.2014265152532476.LGS

LO800S1234562014265LGN00_IDF.xml

LO800S1234562014265LGN00_MD5.txt

- 8 - LSDS-749

Section 3 Instrument Characteristics

Several characteristics of the OLI and TIRS instruments affect the mission data format.

3.1 OLI

3.1.1 OLI Bands

The OLI instrument outputs thirteen band packets covering nine spectral ranges, eight Multispectral (MS) bands, a Panchromatic (PAN) band made up of four individual band packets, and a single Blind band. The MS and Blind bands occupy one band packet each. Because the PAN band has twice as many detectors and is sampled twice as often as the MS bands, OLI outputs it as four band packets with a resolution that is two times that of the other bands. Each OLI band has a spatial resolution of 30 meters (m) with the exception of the PAN band, which has a 15m resolution. The Blind band contains detectors from three of the spectral ranges. The Blind band is not used to image the Earth, but is used for calibration and its detectors are mechanically covered to keep any light from entering.

3.1.2 Organization of Detectors

Positioned within the focal plane are 14 Sensor Chip Assemblies (SCAs). Each SCA contains an array of detectors for every band. The 14 SCAs are identical with respect to electronic configuration and detector layout. As shown in Figure 3-1, the 14 SCAs are arranged in a staggered fashion to ensure complete viewing coverage of the field.

As depicted in Figure 3-2, adjacent SCAs are rotated 180 degrees with respect to each other. Therefore, the PAN band detectors are closest to the axis of the focal plane for all SCAs.

Figure 3-1. SCA Placement in the Focal Plane

Along Track Direction

SCA1 SCA5 SCA3 SCA7 SCA9 SCA11 SCA13

SCA2 SCA6 SCA4 SCA8 SCA10 SCA12 SCA14

Focal Plane axis

- 9 - LSDS-749

Figure 3-2. Detector Placement on Adjacent SCAs

Each detector array consists of imaging detectors and Video Reference Pixels (VRPs).

The VRPs are located on either side of the imaging detectors. The detector arrays for the 8 MS bands have 6 VRPs, followed by 494 imaging detectors, followed by 6 VRPs.

The PAN band has 12 VRPs, followed by 988 imaging detectors, followed by 12 VRPs.

The instrument and the spacecraft treat the VRPs the same way as pixels are treated from the imaging detectors. The pixels for a given band (both VRPs and imaged pixels) are read and transmitted.

VRPs VRPsImaging Detectors

Multispectral bands – 6 pixels Multispectral bands – 6 pixels Multispectral bands – 494 pixels

Pan band – 12 pixels Pan band – 12 pixels Pan band – 988 pixels

Figure 3-3. VRP Detector Placement

3.1.3 Blind Band

The Blind band is the same width (506 pixels per SCA) as the MS bands. However, the Blind band consists of 13 sets of Short Wavelength Infrared 2 (SWIR2) / SWIR1 / Cirrus pixel groups containing 5 VRPs and 8 image pixels in each band per group. With a total of 39 pixels per group (15 VRPs and 24 images), the total number of pixels is 507; the last Cirrus set has only seven image pixels instead of eight, which makes the total pixel

- 10 - LSDS-749

count match the other bands. This calculation yields 65 VRPs for each band, 104 image pixels for both SWIR1 and SWIR2, and 103 image pixels for Cirrus.

Figure 3-4. Blind Band Pixel Arrangement

3.1.4 14-bit Detectors / 12-bit Samples

The detectors used in OLI yield a 14-bit Digital Number (DN). However, only 12 bits are recorded to the SSR and transmitted to ground by the satellite. Specifically, the 12 Most Significant Bits (MSBs) of data are recorded, while the satellite bus ignores the lower two Least Significant Bits (LSBs). For certain collection types, the OLI automatically shifts image pixel values up two bits before outputting those values to the satellite bus. Pixels shifted in this manner have 12 LSBs recorded to the SSR rather than 12 MSBs. The top bits are truncated and possible data loss may occur. Shifting pixels in this manner preserves the resolution of the pixel DNs that require a less dynamic range. For pixels that require the full dynamic range of the detectors, the resolution must be sacrificed.

To accommodate this operation, the OLI employs two operating modes: Standard Imaging mode and Dim / Dark Scene Imaging mode. In either operating mode, the VRPs, the Blind band pixels, and the pixels from shutter data are always shifted up two bits. The pixels from all other bands are shifted only when operating in the Dim / Dark Scene Imaging mode. See References for more detailed information on the Dim / Dark Scene Imaging mode.

Stimulation lamp calibration intervals are sometimes collected in Standard Imaging mode and sometimes in Dim / Dark Scene Imaging mode. The detectors for some bands have a very low response and require full resolution for accurate calibration, while other detectors have a very high response and require the full dynamic range of detectors. Figure 3-5 illustrates the detector response to the stimulation lamp source as reported in the OLI Critical Design Review (CDR). When operating in Dim / Dark Scene Imaging mode, the SWIR1, SWIR2, and Cirrus bands are truncated. The loss of resolution when operating in Standard Imaging mode reduces the precision of the measurements for the remaining bands.

5 8 5 8 5 8 5 8 5 8 5 8 5 8 5 8 5 8 5 8 5 8 5 7

39 39 39 38

1 2 12 13

SWIR2 SWIR1 Cirrus SWIR2 SWIR1 Cirrus SWIR2 SWIR1 Cirrus SWIR2 SWIR1 Cirrus

VRPs

Blind Image Pixels Pixel Count Group Number

Band Pixels per Group

- 11 - LSDS-749

The imaging mode is contained in the Image Header in the Image Data Truncation Setting.

C /A

B lu e

G re e n

R e d

N

IR

S W

IR

S W

IR

P a n

C ir ru s

Band Number

P re d ic te d R e s p o n s e

D N

Figure 3-5. OLI Response to Stimulation Lamp Source

3.1.5 Pixel Readout

The PAN band is transmitted in the same format as the other OLI bands. Due to its increased resolution, the PAN band is treated as four separate bands at this point. This method makes the samples per band the same as the other bands. A detector order divides the PAN samples into separate bands; the even detectors and the odd detectors are split into separate bands. One band contains data from the even-numbered detectors. A second band contains data for the odd-numbered detectors. These bands are called Pan 1 Even, Pan 1 Odd, Pan 2 Even, and Pan 2 Odd. The method of splitting the detectors into individual bands yield bands with 506 pixels, and yields 6 VRPs, followed by 494 imaged pixels, followed by 6 VRPs. The first detector is numbered 1, which means the detector is placed into one of the odd bands.

OLI outputs all image data for a single frame at one time. OLI outputs the data for each band in the order shown in Figure 3-6, with Pan 1 Odd as the first output. Every band contains 14 SCAs * 506 samples per SCA = 7084 samples * 1.5 = 10,626 bytes of output data. The 7084 samples equal 92092 12-bit DNs, resulting in 138138 bytes of image data. An entire frame of OLI data is 138,264 bytes which includes the frame header at the beginning of the frame, padding at the end of every band, and the Cyclic Redundancy Check (CRC) at the end of the frame.

- 12 - LSDS-749

Figure 3-6. OLI Band Output Order

The data are interleaved by SCA within each band. The first 14 sample outputs come from the first detector in each of the SCAs – SCA 1 first, followed by SCA 2, and so on.

The next 14 samples come from the second detector in each SCA. This pattern repeats through detector 506.

Each of the 14 SCAs are identical with respect to electronic configuration. Because adjacent SCAs are rotated 180 degrees with respect to each other, the detectors for every other SCA are read in the opposite direction. Figure 3-7 shows the detector order for each SCA. The arrows in Figure 3-7 indicate the detector order.

Figure 3-7. OLI SCA Readout Directions

3.1.6 OLI Interval Structure

The OLI generates an image header each time it receives an “Image Acquire” command. Only one image header is present for each mission data collection. Except in the case of data loss, a mission data collection never contains partial frames.

SCA1 SCA5 SCA3 SCA7 SCA9 SCA11 SCA13

SCA6 SCA4 SCA8 SCA10 SCA12 SCA14 SCA2

- 13 - LSDS-749

Figure 3-8. OLI Interval Overview

Figure 3-8 displays the interval structure. The beginning of the interval is called the image header and is treated as a frame, specifically frame zero. All frames that follow are image frames. These image frames are numbered 1 through N. At the beginning of every frame, including the image header frame, is a frame header. At the end of every frame is a CRC. The CRC validates that the data within the frame has not been corrupted. The contents and structure of the image header, frame header, CRC, and image data are described in more detail in the following sections.

3.2 TIRS

3.2.1 TIRS Bands

TIRS has three bands: two clear Aperture bands and one Blind band. The imaging bands cover two different spectral ranges, one centered on 10.8 μm and one centered on 12.0 μm.

3.2.2 Organization of Detectors

Each TIRS SCA is a two-dimensional array of detectors. All detectors in the array are the same type and have the same spectral response. To achieve the collected spectral bands, incident light passes through filters en route to the detectors. Because the detectors are all of the same type and the two imaging bands only differ by the filter covering them, only one set of reference blind detectors is used for both bands.

To address issues with bad or poor-performing detectors, each of the clear Aperture bands has 35 rows of redundancy and the Blind band has 40 rows of redundancy. On any given collection, two rows are transmitted to ground for each band on an SCA.

- 14 - LSDS-749

TIRS outputs all image data for a single frame at one time. TIRS outputs the data for each band in the order shown in Figure 3-9. Every band contains 3 SCAs * 647 samples per SCA = 1941 * 2 Rows (for redundancy) = 3882 samples * 1.5 = 5823 bytes. The total number of TIRS generated samples for each band is 3888 * 1.5 = 5832 bytes due to the six DHeader bytes at the beginning of each band and three bytes of padding at the end of each band. An entire frame of TIRS data is 17,534 bytes, which includes the frame header at the beginning of the frame and the CRC at the end of the frame. The contents and structure of the TIRS frame header, DHeader, CRC, and image data are described in more detail in section 4.2.4.1.

Figure 3-9. TIRS Band Output Order

Figure 3-10 depicts the arrangement of the three TIRS SCAs on the focal plane. Each SCA has 640 columns of detectors. A 35 pixel overlap between adjacent SCAs yields a total of 1,850 unique pixel columns.

- 15 - LSDS-749

Figure 3-10. TIRS Focal Plane

- 16 - LSDS-749

Section 4 Data Definition

This section provides detailed information regarding each file type format.

4.1 Interval Definition File (IDF)

The IDF provides information about the interval including the list of file names, sizes, and checksums, and the WRS-2 scenes covered. To create the file, the Ground Station collects the data based on information in the Scene-Interval-File Mapping Table (SIFMT). The Ground Station adds information upon receipt of the data. LSDS-613 Landsat 8 (L8) Mission Operations Element (MOE) to Ground Network Element (GNE) Interface Control Document (ICD) describes the SIFMT file.

The IDF is an Extensible Markup Language (XML) formatted file. The information is divided into a list of records. The IDF has five record types: IDF, Header, Root File, File, and Scene.

The root element in an IDF file is <idf>. Each IDF contains a single IDF record. The IDF record contains standalone variables, in addition to four complex record types that are each encapsulated in an XML element. The names of these elements are “header”, “scene_record”, “rootfile_record”, and “file_record”. Each of the variables defined within a record is a simple element, the text of which is the value for that variable. None of the elements within an IDF file allow attributes.

The number of Scene Records in the file reflects the number of complete WRS-2 scenes that the interval covers. One File Record exists for each file. The IDF contains either one or two Root File Records. Each of the File Records is encapsulated inside a Root File Record. The following sections define the format and content of each record type.

The applicable XML Schema Definition (XSD) file provides a more explicit definition of the IDF. This file is found on the Landsat Mission Web Site (LMWS).

4.1.1 IDF Record

The following elements appear within the IDF Record.

header - Complex data type described in section 4.1.2.

moe_interval_id - A string indicating the ID of the interval that the MOE assigns. This field is an 11-character string, which has the following format: OOOOOO_SS_T, where OOOOOO is the orbit number for the first scene in the interval, SS is the segment number, and T is the type code. The segment number is a counter that resets at the beginning of each orbit. See the list of type codes in Table 2-2.

landsat_interval_id - The Landsat Interval ID as described in section 2.2. Either this field or the Landsat Calibration Interval ID, but not both, must be included.

- 17 - LSDS-749

landsat_cal_interval_id - The Landsat Calibration Interval ID as described in section

2.2. Either this field or the Landsat Interval ID, but not both, must be included.

moe_interval_complete_flag - A one-character string value indicating if a complete interval has been delivered. Valid values are ‘Y’ and ‘N’. This field may be omitted when an IC generates the IDF.

priority_flag - A one-character string value indicating whether or not the interval contains any priority scenes. Valid values are ‘Y’ and ‘N’.

sensor_id - A three- to eight-character string indicating which instrument recorded the data. For OLI data, the string is “OLI”. For TIRS data, the string is “TIRS”. For combined data, the string is “OLI_TIRS”.

offnadir_flag - A one-character string value indicating whether or not the interval was collected nadir or off-nadir. This field is only used for type “I” (earth imaging) intervals.

Valid values are ‘Y’ and ‘N’.

collection_type - Type of data imaged during the interval. This value must be one of the following values: EARTH_IMAGING, LUNAR, OLI_LAMP, OLI_SHUTTER, OLI_SHUTTER_EXTENDED, OLI_SHUTTER_INTEGRATION_TIME_SWEEP, OLI_SOLAR, OLI_SOLAR_INTEGRATION_TIME_SWEEP, SSR_PN_TEST_SEQUENCE, SIDE_SLITHER, SLEW_IMAGING, STELLAR, TEST_PATTERNS, TIRS_BLACKBODY, TIRS_DEEPSPACE, TIRS_INTEGRATION_TIME_SWEEP, or TIRS_TRANSMIT_ALL.

data_category - Indicates the category of the interval. Legacy use of this field is as follows:

VALIDATION – Indicates data are from an IC requesting that EROS evaluate their data. Once ten data sets from the IC are successfully validated within a given timeframe, the future data from that IC are labeled EXCHANGE.

EXCHANGE – Indicates the data are from a validated IC outside the LGN.

DIAGNOSTIC – Indicates the data contains test patterns from the instrument.

NOMINAL – Indicates the LGN Ground Station nominally collected data.

ENGINEERING – Indicates data that should be processed normally, but should not be provided to outside customers. This setting may be used for an interval suspect for one reason or another (e.g., following a maneuver).

wrs_path - An integer indicating the WRS-2 path number of the interval. If the starting path differs from the ending path, the starting path is used. This field is only used for type “I” (earth imaging) intervals. Valid values are from 1 to 233.

wrs_starting_row - An integer indicating the first full WRS-2 row in the interval. This field is only used for type “I” (earth imaging) intervals. Valid values are from 1 to 248.

- 18 - LSDS-749

wrs_ending_row - An integer indicating the last full WRS-2 row in the interval. This field is only used for type “I” (earth imaging) intervals. Valid values are from 1 to 248.

gne_interval_complete_flag - A one-character string value indicating if GNE successfully processed the complete interval. Valid values are ‘Y’ and ‘N’. This field may be omitted when an IC generates the IDF.

rootfile_record - One or two root file records may be included in the IDF. This complex data type is described in section 4.1.3.

scene_record - Zero or more scene records may be included in the IDF. This complex data type is described in section 4.1.5.

4.1.2 Header Record

The following elements appear within the Header Record.

scid - The three-digit spacecraft identifier that the Consultative Committee for Space Data Systems (CCSDS) assigns (i.e., the 10-bit code). For Landsat 8, the code is 506.

product_type - The type of file. This field should always be set to “IDF”.

gen_time - UTC time string holding the time that the file was generated. This field is a 21-character string with the following format:

YYYY:DOY:HH:MM:SS.SSS

Source - Originator of the file. This field is either a six- or eight-character string. For LGN Ground Stations, this value is set to “GNE-DCRS”. For ICs, this value is “IC-XXX”, where XXX identifies the originating Ground Station.

mode - Indicates if the system is in production mode or test mode. This field is either a four- or ten-character string. Valid values are “TEST” and “PRODUCTION”.

4.1.3 Root File Record

For all intervals, the IDF contains one or two root files. Each root file corresponds to a directory on the spacecraft SSR. OLI and TIRS intervals are stored in separate directories, creating the possibility for two root files. Each root file record contains a number of file records. The root file record contains the following elements:

root_file_id - An integer indicating the SSR root file directory number, where the interval is stored. Valid values are from 0 to 511.

root_file_complete_flag - A one-character string value indicating if the collection for the root file has been completed. Valid values are ‘Y’ and ‘N’.

- 19 - LSDS-749

sensor id - A three- or four-character string indicating the association between the instrument and the root file. For OLI data, the string is “OLI”. For TIRS data, the string is “TIRS”. A value of “OLI_TIRS” is not valid because a root file can only contain OLI or TIRS data.

landsat_interval_id - The Landsat Interval ID as described in section 2.2. Either this field or the Landsat Calibration Interval ID, but not both, must be included.

landsat_cal_interval_id - The Landsat Calibration Interval ID as described in section

2.2. Either this field or the Landsat Interval ID, but not both, must be included.

priority_flag - A one-character string value indicating whether or not the interval contains any priority scenes. Valid values are ‘Y’ and ‘N’.

file_record - Zero or more file records may be included in each root file record. Need to have at least one file record for each root file. This complex data type is described in section 4.1.4.

4.1.4 File Record

For every mission data file in the interval, the IDF provides a File Record containing the following elements:

file_name - The name of the file, conforming to the file name format specified in section 2.2.

station_id - A three-character string identifying the Ground Station receiving the file.

file_checksum - A character string identifying the file’s MD5 checksum value.

file_size - An integer indicating the file’s size in bytes.

4.1.5 Scene Record

For earth imaging intervals, the IDF contains a Scene Record of every scene scheduled for imaging. This record is only present in the IDF for earth imaging intervals. Each record includes the following elements:

wrs_path - An integer indicating the WRS-2 path number of the scene. Valid values are from 1 to 233.

wrs_row - An integer indicating the WRS-2 row number of the scene. Valid values are from 1 to 248.

sensor_id - A three- to eight-character string indicating which instrument recorded the data. For OLI data, the string is “OLI”. For TIRS data, the string is “TIRS”. For combined OLI and TIRS data, the string is “OLI_TIRS”.

- 20 - LSDS-749

priority_flag - A one-character string value indicating whether or not the scene is a priority. Valid values are ‘Y’ and ‘N’.

4.2 OLI and TIRS Mission Data Files

Each mission data interval contains one or more mission data files. Because of constraints in the underlying transfer protocol, these files are never larger than 1 GB.

The data are transmitted from the spacecraft to Ground Stations as 1 GB sub-files.

Each of these sub-files contains CCSDS File Delivery Protocol (CFDP) header information. This header information, used in the GNE transfer, is stripped before saving the file to disk. These sub-files, with the header information removed, are referred to as OLI and TIRS mission data files. The amount of header information removed from each file is minimal. With the exception of the final file, each mission data file is near 1 GB in size.

Each mission data file contains data from only one of the instruments (OLI or TIRS).

The mission data files are interleaved with ancillary data that the spacecraft collected during the record operation. No other information exists in the files. No special header information (i.e., file order or file size) exists in the individual files. The concatenation of two or more mission data files yields a valid mission data file, presuming the data are concatenated in the original order.

OLI image data may be compressed. The satellite performs this compression after the OLI instrument outputs the data. The CRC value that the OLI calculates is computed from the uncompressed image data. OLI header information is transmitted in its original, uncompressed format. Ancillary data and TIRS data are always uncompressed.

4.2.1 Mission Data File Format

The mission data files are organized as a packet series. Each packet contains exactly one type of data. At the beginning of each packet is a Mission Data Header (MDH), which indicates the type of data in the packet and the length of the data field within the packet. Table 4-1, derived from LSDS-613 Landsat 8 (L8) Mission Operations Element (MOE) to Ground Network Element (GNE) Interface Control Document (ICD), summarizes the structure of a mission data packet.

MDH Data Field

Mission Data ID Mission Data Length

2 Bytes 2 Bytes Variable Length

Table 4-1. Mission Data Packet Format

Six types of data are stored in mission data packets and include spacecraft ancillary data, five types of OLI data, and four types of TIRS data. The OLI packet types are Frame Header, CRC, Image Header, Ancillary Data, Uncompressed Image Data, and Compressed Image Data. The TIRS packet types are Frame Header, CRC, Ancillary Data, and Uncompressed Image Data. The Mission Data ID indicates which types of data are contained within the packet. For compressed and uncompressed image data,

- 21 - LSDS-749

the Mission Data ID also indicates which bands of data are contained in the packet.

Table 4-2 lists the valid Mission Data IDs and their meanings.

Mission Data ID

(Uncompressed)

Mission Data ID

(Compressed)

Description

2 N/A OLI Frame Header

3 N/A OLI CRC

4 N/A OLI Image Header

5 N/A Ancillary Data

768 256 OLI Pan 1 Odd

769 257 OLI Pan 1 Even

770 258 OLI Blue

771 259 OLI C/A

772 260 OLI NIR

773 261 OLI Red

774 262 OLI Green

775 263 OLI Pan 2 Odd

776 264 OLI Pan 2 Even

777 265 OLI SWIR2

778 266 OLI SWIR1

779 267 OLI Cirrus

780 268 OLI Blind

1026 N/A TIRS Frame Header

1027 N/A TIRS CRC

1792 N/A TIRS Blind

1793 N/A TIRS 10.8

1794 N/A TIRS 12.0

Table 4-2. Mission Data IDs for Mission Data Packets

Both OLI and TIRS mission data files consist of packets that form frames along with Ancillary Data Packets at a fixed 1 Hertz (Hz) rate. Uncorrupted mission data files contain only full frames. In breaking mission data files at or near the 1 GB boundary, a frame is never split across the break. (e.g., all frames in the mission data files are complete).

4.2.2 Ancillary Data Packets

The spacecraft collects measurements for the Ancillary Data Packets independently of instrument recordings. The ancillary data precedes and follows the instrument data by several seconds. Thus, the planned configuration is for ancillary data to record in conjunction with instrument data with the appropriate margins before and after the instrument recording is performed.

- 22 - LSDS-749

4.2.2.1 Ancillary Data Packet Structure

Ancillary Data Packets are fixed sized and always contain information in the same format. The Ancillary Data Packet Data field is divided into sections called messages.

Table 4-3 identifies the organization of messages with the Ancillary Data Packet.

Table 4-3 describes the overall data portion of the Ancillary Data Packet organization, and provides figures to clarify the organization.

Data Type Start Byte Length

Attitude Quaternion 0 1200

Attitude Filter States 1200 80

Inertial Measurement Unit (IMU) Data 1280 708

IMU Latency 1988 120

Spare 2108 80

Ephemeris Data 2188 80

Raw GPS Position Channel 2267 137

Raw GPS Range Data 2405 249

ST Quaternion Output Message 2654 72

ST Centroid Output Message 2726 250

Spare 2976 29

OLI Telemetry 3005 320

OLI & TIRS S/C Temperatures 3325 134

Gyro Temperatures 3459 256

TIRS Telemetry 3715 256

Spare 3971 111

Padding 4082 14

Total Bytes 4096

Table 4-3. Ancillary Data Packet Organization

4.2.2.2 Ancillary Data Group Details

This section defines the detailed organization of each message type found in the Ancillary Data Packet. All multi-byte data fields are big-endian unless otherwise specified.

4.2.2.2.1 Attitude

The spacecraft estimates the spacecraft attitude using a Kalman filter applied to Inertial Reference Units (IRUs) and Star Tracker measurements. The attitude estimates are included in the Ancillary Data. The Kalman filter states are included in the Ancillary Data. This information is separated into two messages: the Processed Attitude message and the Attitude Filter States message. The following sections describe each of these messages in detail.

- 23 - LSDS-749

4.2.2.2.1.1 Processed Attitude Message

The Processed Attitude message is sampled at 50 Hz and generated at 1 Hz. The Processed Attitude message contains the following information:

Attitude quaternion

Spacecraft body rates

Time tag for the moment at which the information applies

Table 4-4 identifies the structure of the Processed Attitude message.

Byte Description Bit Length Type Units

0-7 Time stamp of the Ancillary Data Quaternion solution, sample 0

64 F64 s

8-11 Ancillary Data Fine Attitude Quaternion Estimate Earth Centered Inertial (ECI) to body [q1], Sample

32 I Unitless

12-15 Ancillary Data Fine Attitude Quaternion Estimate ECI to body [q2], Sample 0

32 I Unitless

16-19 Ancillary Data Fine Attitude Quaternion Estimate ECI to body [q3], Sample 0

32 I Unitless

20-23 Ancillary Data Fine Attitude Quaternion Estimate ECI to body [q4], Sample 0

32 I Unitless

24-1199 Repeat above data 49 more times

Table 4-4. Processed Attitude Message

4.2.2.2.1.2 Attitude Filter States Message

The Attitude Filter States message is generated at 1 Hz. The Attitude Filter States message contains the following information:

Time tag for the moment at which the information applies

Gyro biases

Gyro misalignments

Gyro scale factors

Filter attitude error

Covariance matrix of the new quaternion to the prior quaternion

Table 4-5 identifies the structure of the Attitude Filter States message.

Byte Description Bit Length

Type Units

1200-1203 Seconds since the Spacecraft Time Epoch (Jan 1st, 2000 International Atomic Time (TAI)) - UDL Seconds Register

32 UI s

1204-1207 UDL Sub-Seconds Register 32 UI 10E-7 s

1208-1211 FAD body X estimate of gyro drift bias wrt ADF 32 F32 rad/s

- 24 - LSDS-749

Byte Description Bit Length

Type Units

1212-1215 FAD body Y estimate of gyro drift bias wrt ADF 32 F32 rad/s

1216-1219 FAD body Z estimate of gyro drift bias wrt ADF 32 F32 rad/s

1220-1223 FAD estimate of body (X,X) element of gyro correction matrix (scale factor)

32 F32 Unitless

1224-1227 FAD estimate of body (Y,Y) element of gyro correction matrix (scale factor)

32 F32 Unitless

1228-1231 FAD estimate of body (Z,Z) element of gyro correction matrix (scale factor)

32 F32 Unitless

1232-1235 FAD estimate of body (X,Y) element of gyro correction matrix (gyro alignment wrt ADF)

32 F32 rad

1236-1239 FAD estimate of body (X,Z) element of gyro correction matrix (gyro alignment wrt ADF)

32 F32 rad

1240-1243 FAD estimate of body (Y,X) element of gyro correction matrix(gyro alignment wrt ADF)

32 F32 rad

1244-1247 FAD estimate of body (Y,Z) element of gyro correction matrix(gyro alignment wrt ADF)

32 F32 rad

1248-1251 FAD estimate of body (Z,X) element of gyro correction matrix(gyro alignment wrt ADF)

32 F32 rad

1252-1255 FAD estimate of body (Z,Y) element of gyro correction matrix(gyro alignment wrt ADF)

32 F32 rad

1256-1259 Estimated attitude control error about the spacecraft x body axis

32 F32 rad

1260-1263 Estimated attitude control error about the spacecraft y body axis

32 F32 rad

1264-1267 Estimated attitude control error about the spacecraft z body axis

32 F32 rad

1268-1271 FAD state covariance diagonal associated with body X attitude error estimate

32 F32 rad^2

1272-1275 FAD state covariance diagonal associated with body Y attitude error estimate

32 F32 rad^2

1276-1279 FAD state covariance diagonal associated with body Z attitude error estimate

32 F32 rad^2

Table 4-5. Attitude Filter States Message

4.2.2.2.2 Inertial Measurement Unit (IMU) Gyro

The IMU Gyro data are generated at 50 Hz. This data contains the following information:

Time of the last IMU time sync pulse

Fifty gyro samples (each sample includes a sync event time tag)

Table 4-6 identifies the structure of the IMU Gyro.

- 25 - LSDS-749

Byte Description Bit Length

Type Units

1280-1283 UDL seconds register value at the time of the last IMU time sync pulse

32 UI s

1284-1287 UDL sub-seconds register value at the time of the last IMU time sync pulse; 10 MHz counter

32 UI 10E-7 s

1288-1289 IMU Sync Event Time Tag Sample 1 - Oldest 16 I Unitless

1290-1291 IMU Time Tag Sample 1 - Oldest 16 UI Unitless

IMU Gyro A WA or FTR state Sample 1 - Oldest

1 D 0 - FTR_MODE

1 - WAS_MODE

IMU Gyro B WA or FTR state Sample 1 - Oldest

1 D 0 - FTR_MODE

1 - WAS_MODE

IMU Gyro C WA or FTR state Sample 1 - Oldest

1 D 0 - FTR_MODE

1 - WAS_MODE

IMU Gyro D WA or FTR state Sample 1 - Oldest

1 D 0 - FTR_MODE

1 - WAS_MODE

IMU Gyro A Mode Status Sample 1 - Oldest

1 D 0 - LOW_RATE

1 - HIGH_RATE

IMU Gyro B Mode Status Sample 1 - Oldest

1 D 0 - LOW_RATE

1 - HIGH_RATE

IMU Gyro C Mode Status Sample 1 - Oldest

1 D 0 - LOW_RATE

1 - HIGH_RATE

IMU Gyro D Mode Status Sample 1 - Oldest

1 D 0 - LOW_RATE

1 - HIGH_RATE

IMU Gyro A Valid Status Sample 1 - Oldest

1 D 0 – INVALID

1 - VALID

IMU Gyro B Valid Status Sample 1 - Oldest

1 D 0 – INVALID

1 – VALID

IMU Gyro C Valid Status Sample 1 - Oldest

1 D 0 – INVALID

1 – VALID

IMU Gyro D Valid Status Sample 1 - Oldest

1 D 0 – INVALID

1 - VALID

1293 Data validity unused Bits 4

1294-1295 IMU Gyro A integrated angle counter Sample 1 - Oldest

16 UI Unitless

1296-1297 IMU Gyro B integrated angle counter Sample 1 - Oldest

16 UI Unitless

1298-1299 IMU Gyro C integrated angle counter Sample 1 - Oldest

16 UI Unitless

1300-1301 IMU Gyro D integrated angle counter Sample 1 - Oldest

16 UI Unitless

1302-1987 Repeat above data 49 more times

Table 4-6. IMU Gyro

The Integrated Angles given in the IMU message offer a measure on the amount of rotation the gyroscope undergoes between samples. The gyroscopes can work in either a Low Rate Range mode or a High Rate Range mode. When in Low Rate Range mode, the Integrated Angles unit is 0.05 arc seconds. When rotating quickly, the IMU automatically switches to High Rate Range mode. In this mode, the Integrated Angles unit is 1.6 arc seconds. This rate allows the IMU to correctly represent the rotational rate when the values become large at the expense of reduced resolution.

- 26 - LSDS-749

The greatest change in angle represented when in Low Rate Range mode is ~0.455° (32768 * 0.05 / 3600). At a sampling rate of 50 Hz, a rotational rate of ~22.7°/sec is allowed in either the positive or negative direction. The Landsat 8 spacecraft should never rotate at or above this rate during nominal operations. The High Rate Range mode is not expected to be entered. None of the gyroscopes should experience saturation, which would require an angular rate of 726°/sec.

The time given for each set of gyro samples is a delta from the epoch given at the beginning of the message. The resolution of the time tag delta fields is 4 μs. The maximum value represented by this field is 262.14 ms. The IRU message is generated once every 100 ms. The epoch is reset frequently, if not once every message.

4.2.2.2.3 IMU Latency

IMU Latency data are generated at 10 Hz. Table 4-7 shows the contents of IMU Latency.

Type Units

1988-1995 Time stamp of the Fine AD solution 64 F64 s

1996-1999 Measured IMU data latency 32 F32 s

2000-2107 Repeat above data nine more times

Table 4-7. IMU Latency

4.2.2.2.4 Ephemeris

The spacecraft calculates the ephemeris based on output from the onboard Global Positioning System (GPS) unit. The message generates at 1 Hz. The spacecraft outputs its estimate of the ephemeris as part of each Ancillary Data Packet. The ephemeris information consists of the time for the estimate, the location and velocity of the spacecraft at that time, and GPS position residual error values for both the position and velocity. The spacecraft position and velocity are given in Earth-Centered, Earth- Fixed (ECEF) coordinates. The residuals are given in the Earth Centered Inertial (ECI) J2000 coordinate frame. Table 4-8 shows the contents of the ephemeris message.

Type Units

2188-

Orbit Propagator Solution Time Tag 64 F64 seconds

2196-

Spacecraft Position Estimate x-axis component in ECEF reference frame

64 F64 meters

2204-

Spacecraft Position Estimate y-axis component in ECEF reference frame

64 F64 meters

2212-

Spacecraft Position Estimate z-axis component in ECEF reference frame

64 F64 meters

2220- Spacecraft Velocity Estimate x-axis component in 64 F64 meters/second

- 27 - LSDS-749

Byte Description Bit Length

Type Units

2227 ECEF reference frame

2228-

Spacecraft Velocity Estimate y-axis component in ECEF reference frame

64 F64 meters/second

2236-

Spacecraft Velocity Estimate z-axis component in ECEF reference frame

64 F64 meters/second

2244-

Orbit Determination Filter x position orb frame residual

32 F32 meters

2248-

Orbit Determination Filter y position orb frame residual

32 F32 meters

2252-

Orbit Determination Filter z position orb frame residual

32 F32 meters

2256-

Orbit Determination Filter x velocity orb frame residual

32 F32 meters/second

2260-

Orbit Determination Filter y velocity orb frame residual

32 F32 meters/second

2264-

Orbit Determination Filter z velocity orb frame residual

32 F32 meters/second

Table 4-8. Ephemeris Message

4.2.2.2.5 GPS

The raw GPS output is transmitted to ground for definitive ephemeris processing in the event that processing is necessary. The GPS output consists of two messages: the Position Channel Status message and the Satellite Range Data message. Each of these messages is output at 1 Hz. One copy of each message is found in each Ancillary Data Packet.

With the exception of the actual position and velocity at the end of the Position Channel Status message, each of the values in these messages is an unsigned integer.

4.2.2.2.5.1 Position Channel Status Message

The Position Channel Status message gives the position of the spacecraft in polar coordinates and Cartesian ECEF coordinates. The velocity is also multiply-defined, both as a vector in ECEF coordinates and as a magnitude and heading. The ECEF values are of primary interest to the ground system. The Position Channel Status message also provides information about the satellites tracked and the overall status of the receiver. Up to 12 satellites may be tracked simultaneously. Table 4-9 identifies the details for the message.

Type Units

2268 Selected GPS Position Channel Status Message - Function

8 UI Unitless

2269 Selected GPS Position Channel Status Message - Sub Function

8 UI Unitless

- 28 - LSDS-749

Byte Description Bit Length

Type Units

2270 Selected GPS Position Channel Status Message - Month

8 UI Unitless

2271 Selected GPS Position Channel Status Message - Day

8 UI Unitless

2272-2273 Selected GPS Position Channel Status Message - Year

16 UI Unitless

2274 Selected GPS Position Channel Status Message - Hours

8 UI hr

2275 Selected GPS Position Channel Status Message - Minutes

8 UI min

2276 Selected GPS Position Channel Status Message - Seconds

8 UI s

2277-2280 Selected GPS Position Channel Status Message - Fractional Seconds

32 UI Unitless

2281-2284 Selected GPS Position Channel Status Message - Latitude

32 I deg

2285-2288 Selected GPS Position Channel Status Message - Longitude

32 I deg

2289-2292 Selected GPS Position Channel Status Message - Height - GPS Ellipsoid (Uncorrected)

32 I m

2293-2296 Selected GPS Position Channel Status Message - Height - MSL (Corrected)

32…