Attachment 13 -- SDSFIE Quality (SDSFIE-Q) (12 September 2018).pdf

Spatial Data Standards for Facilities, Infrastructure, and Environment (SDSFIE)

SDSFIE Quality (SDSFIE-Q)

Revision 1

12 September 2018

Prepared By:

The Installation Geospatial Information and Services Governance Group (IGG)

For:

The Assistant Secretary of Defense (Energy, Installations & Environment)

© 2018

12 September 2018 SDSFIE Quality Rev1 i

Executive Summary

The Assistant Secretary of Defense for Energy, Installations, and Environment (ASD(EI&E)), has issued this Spatial Data Standards for Facilities, Infrastructure, and Environment (SDSFIE) Quality (SDSFIE-Q) standard as part of the governance of installation geospatial information and services (IGI&S) under the authority granted in DoDI 8130.01. The IGI&S Governance Group (IGG) developed this standard in order to foster coordinated and integrated approaches for IGI&S across the Department.

SDSFIE-Q specifies over-arching guidance for how DoD will implement a tiered approach to quality, including data quality processes, measures, and metrics for vector, raster, and geospatial services developed and used by the installation geospatial information and services (IGI&S) community. This standard does not include a required schema or data model, and therefore is not registered in the DoD IT Standards Registry (DISR) as a traditional IT standard would be. Instead, SDSFIE-Q defines a framework for IGI&S data quality, with enterprise-level and Component-level elements. SDSFIE-Q is based on the ISO 19157 conceptual model of quality for geographic data (ISO 19157 is a normative reference for several DISR–mandated geospatial standards which are related to the SDSFIE family of standards ). When implemented, the intent of SDSFIE-Q is to ensure that IGI&S data and services align with IGI&S mission requirements, conform to DISR mandates, and are understandable, trusted and interoperable in accordance with DoDI 8130.01.

SDSFIE-Q describes processes needed to ensure quality data management or data collection and maintenance at a Department-wide level. It identifies the important elements that should be covered in IGI&S Program level quality management plans (or guidance). These elements can also be used as measures to evaluate the completeness of the Components’ guidance and conformance to the framework prescribed herein. The document includes a formal process for defining measures to assess the quality and completeness of data, as well as the reporting of metrics associated with these data quality measures using the SDSFIE Metadata (SDSFIE-M) standard and other applicable means.

Adherence to the data quality guidance and processes in this document are required any time IGI&S data or services are created, updated, maintained, or shared. SDSFIE-Q also includes certain specific data quality processes, measures, and metrics which apply individually to IGI&S vector data (SDSFIE-V), raster data (SDSFIE-R), or services (SDSFIE-S), as the case may be.

ii

Revision History

Description Date Version

SDSFIE-Q Final Approved 12 Dec 2016

SDSFIE-Q Rev 1 Draft

Major changes include:

Revised SDSFIE-R section

Raster Metrics

Aggregates and Repositories Section

OV-5 and BEA operational activities

1 May 2018 Rev1

SDSFIE-Q Rev 1 WG Final Draft

Major changes includes:

Revised metrics creation and approval process

Addition of “application” scope

Addition of RBJ Guidance

Revised DCS Example

3 Jul 2018 Rev1

SDSFIE-Q Rev 1 Final Draft 13 Aug 2018 Rev1

SDSFIE-Q Rev 1 Final Approved 12 Sep 2018 Rev1 iii

Table of Contents

Executive Summary .................................................................................................................................... i

Part 1: Overview of SDSFIE-Q

1 Introduction

Purpose

Authority

Scope

References

Acronyms

Document Maintenance

Terms and Definitions

2 ISO 19157 Components of a Quality Framework

ISO 19157 Data Quality Evaluation Process

2.1.1 Data Quality Units

2.1.2 Measures

2.1.3 Evaluation Methods

2.1.4 Results

2.1.5 Metaquality

ISO 19157 Data Quality Reporting

3 SDSFIE-Q Framework

Defined Roles

Quality Management Workflow

Quality Management Activity: Define Geospatial Requirements

3.3.1 Quality Management Plans

3.3.2 Data Content Specification (DCS)

3.3.3 SDSFIE Quality Contract Language

3.3.4 Define Quality Metrics

Quality Management Activity: Create Geospatial Information

3.4.1 Create and Acquire Data

3.4.2 Create Metadata

Quality Management Activity: Validate Geospatial Information

3.5.1 Evaluate Data

3.5.2 Report Quality Results - Metadata

Quality Management Activity: Maintain Geospatial Information

3.6.1 Cleanse Data

Quality Management Activity: Provide Geospatial Information

3.7.1 Releasability

3.7.2 Accessibility

3.7.3 Feedback

iv

Part 2: SDSFIE-V Data Quality Guidance

4 SDSFIE-V Introduction

IGI&S Vector Datasets

4.1.1 Common Installation Picture (CIP)

SDSFIE-V Quality Metrics

SDSFIE-V Data Content Specifications (DCS)

4.3.1 DCS General Guidance

4.3.2 DCS Validation

SDSFIE-V Data Quality Documentation: SDSFIE Metadata

4.4.1 CIP Metadata Requirements

Part 3: SDSFIE-R Data Quality Guidance

5 SDSFIE-R Introduction

Raster Acquisition

SDSFIE-R Quality Metrics

SDSFIE-R Data Quality Documentation: SDSFIE Metadata

Part 4: Aggregated Datasets Quality Guidance

6 Aggregated Datasets Introduction

Repository

6.1.1 Authoritative Data Source (ADS) Repository

Aggregate

6.2.1 Common Installation Picture (CIP) Aggregate

Metrics for Aggregated Datasets

Aggregated Datasets Data Quality Documentation

6.4.1 Aggregating Data Quality Evaluation Results

Part 5: SDSFIE-S Data Quality Guidance

7 SDSFIE-S Introduction

Appendix A: ISO 19157 Data Quality Standard Measures

Appendix B: IGG Data Quality Metrics

Appendix C: Metadata Examples

Annex A: Data Content Specification (DCS) General Guidance ....................................................... A-1

Annex B: Data Content Specification (DCS) Example ....................................................................... B-1

Annex C: SDSFIE Quality Contract Language Examples .................................................................. C-1 v

Table of Figures

Figure 1: ISO 19157 Components of a Quality Framework

Figure 2: ISO 19157 Data Evaluation Process

Figure 3: Data Quality Units

Figure 4: IGI&S Quality Management Operational Activity Diagram

Figure 5: Quality Measures vs. Quality Metrics vi

Table of Tables

Table 1: Metadata Codes for Data Quality Scope (from ISO 19115)

Table 2: Data Quality RACI Chart

Table 3: Data Quality Scope Codes for IGI&S Data

Table 4: ISO 19157 Data Quality Standard Measures

Table 5: IGG Data Quality Metrics

Table 6: Metadata Example - Aggregate Scope

Table 7: Metadata Example: Repository Scope

Part 1: Overview of SDSFIE-Q

1 Introduction

The Spatial Data Standards for Facilities, Infrastructure, and Environment (SDSFIE) are a family of IT standards (models, specifications), which define a Department of Defense (DoD)-wide set of semantics intended to maximize interoperability of geospatial information and services for installation, environment and civil works missions. The SDSFIE family consists of seven parts, defined in the SDSFIE Governance Plan Revision 2, September 13, 2017. The SDSFIE Quality standard (SDSFIE-Q) is one of the seven parts of SDSFIE. It specifies over-arching guidance for how DoD will implement a tiered approach to quality, including data quality processes, measures, and metrics for vector, raster, and geospatial services developed and used by the installation geospatial information and services (IGI&S) community as defined in DoDI 8130.01.

SDSFIE-Q does not include a required schema or data model, and therefore is not registered in the DoD IT Standards Registry (DISR) as a traditional IT standard would be. Instead, SDSFIE-Q defines a framework for IGI&S data quality, with enterprise-level and Component-level elements. SDSFIE-Q is based on the ISO 19157 conceptual model of quality for geographic data (ISO 19157 is a normative reference for several DISR–mandated geospatial standards which are related to the SDSFIE family of standards1). When implemented, the intent of SDSFIE-Q is to ensure that IGI&S data and services align with IGI&S mission requirements, conform to DISR mandates, and are understandable, trusted and interoperable in accordance with DoDI 8130.01.

SDSFIE-Q includes a formal process for defining measures to assess the quality and completeness of data, as well as the reporting of metrics associated with these data quality measures using the SDSFIE Metadata (SDSFIE-M) standard and other applicable means. Adherence to the data quality guidance and processes in this document are required any time IGI&S data or services are created, updated, maintained, or shared. SDSFIE-Q also includes data quality processes, measures, and metrics which specifically apply to IGI&S vector data (SDSFIE-V), raster data (SDSFIE-R), and services (SDSFIE-S).

SDSFIE-Q is designed to align with all data standards established or accepted for use by the IGI&S community.

This document is organized into four parts:

Part 1: Quality guidance for all IGI&S data and services. Part 1 consists of two main concepts:

An overview of ISO 19157 foundational concepts that shape SDSFIE-Q. (section 2)

The SDSFIE-Q Framework. (section 3)

Part 2: Quality guidance specific to vector datasets - SDSFIE-V. (section 4)

Part 3: Quality guidance specific to raster datasets - SDSFIE-R. (section 5)

Part 4: Quality guidance specific to aggregates (e.g. CIP) and repositories (e.g. ADS). (section 6)

Part 5: Quality guidance specific to services - SDSFIE-S. (section 7)

Purpose

The purpose of this document is to establish over-arching guidance for how DoD will implement a tiered approach to quality, including data quality processes, measures, and metrics for vector, raster, and

1 The NMF/NMIS 2.x form the basis of SDSFIE-M and the quality model used by these standards are an implementation of the concepts in ISO 19157. Thus the ISO 19157 conceptual model is used as a basis for SDSFIE-Q in order to align with these concepts and schemas in NMF/NMIS and SDSFIE-M.

geospatial services. This guidance is designed to ensure the IGI&S community provides trusted, authoritative, and accurate geospatial data and services to the DoD for decision making. Implementing SDSFIE-Q is necessary to meet the requirements of DoDI 8320.02, which states that data will be made visible, accessible, understandable, trusted, and interoperable throughout their lifecycle for all authorized users.

Authority

In accordance with DoDI 8130.01, this standard applies to the management of DoD installations and environment to support military readiness in the Active, Guard, and Reserve Components with regard to facility construction, sustainment, and modernization including the operation and sustainment of military test and training ranges as well as the US Army Corps of Engineers Civil Works community. Its applicability for other interested organizations is suggested but not mandatory.

DoDI 8130.01 grants authority to the ASD(EI&E) to develop, manage, and publish IGI&S standards and requires that these standards be coordinated through the Geospatial Intelligence Standards Working Group (GWG). SDSFIE-Q is conformal with ISO 19157, the current spatial data quality standard referenced by other geospatial data standards mandated for DoD use by the GWG and the Joint Enterprise Standards Committee (JESC). In accordance with the procedures in DoDI 8130.01, the ASD(EI&E) develops new or revised standards based upon input from the IGI&S Governance Group (IGG), the official standards consensus body for IGI&S. While the IGG is the defacto author of SDSFIE- Q, it becomes mandatory for the IGI&S community once it is formally issued by the ASD(EI&E).

Scope

This standard is intended to be enterprise-level guidance. It consists of a quality management framework of elements to be implemented at the Department (enterprise) level, the DoD Components’ level, and organizational levels below these (as directed by the next highest level). This framework includes a set of processes, quality measures, and corresponding metrics (e.g. minimum standards) for spatial data quality. These are intended to guide and constrain the spatial data quality management policy, guidance and plans issued by the Components’ IGI&S Programs, but not necessarily specify the means by which to achieve them. This plan describes compliance with documented standards, database referential integrity, and data quality reporting elements. It outlines management processes to define and document clear metrics and guidance for data quality.

This document describes processes needed to ensure quality data management or data collection and maintenance at a Department-wide level. It identifies the important elements that should be covered in IGI&S Program level quality management plans (or guidance). These elements can also be used as measures to evaluate the completeness of the Components’ guidance and conformance to the framework prescribed herein.

References

ASD(EI&E), Spatial Data Standards for Facilities, Infrastructure, and Environment (SDSFIE) Governance Plan, Revision 2 13 September 2017

ASD(EI&E), Spatial Data Standards for Facilities, Infrastructure, and Environment (SDSFIE) Metadata (SDSFIE-M): Implementation Guidance, Version 1.0., 8 September 2015

DoD Chief Information Officer Memorandum, Department of Defense Information Enterprise Architecture, Version 2.0, August 10, 2012

DoD Instruction 8130.01, Installation Geospatial Information and Services (IGI&S), 9 April 2015, as amended

DoD Instruction 8320.02, Sharing Data, Information, and Information Technology (IT) Services in the Department of Defense, 5 August 2013

DoD Instruction 8320.03, Unique Identification (UID) Standards for Supporting the DoD Information Enterprise, 4 November 2015, as amended

DoD Directive 5105.60, National Geospatial-Intelligence Agency (NGA), 29 July 2009

ISO 19115-1, Geographic information – Metadata -- Part 1: Fundamentals, Apr 2014

ISO 19157:2013, Geographic information -- Data quality, Dec 2013

DUSD(I&E), Spatial Data Standards for Facilities, Infrastructure, and Environment (SDSFIE) Metadata (SDSFIE-M), Version 2.0, 12 Sep 2018

Acronyms

ADS Authoritative Data Source

APSR Accountable Property System of Record

ASD(EI&E) Assistant Secretary of Defense for Energy, Installations, and Environment

CIP Common Installation Picture

CIO Chief Information Officer

DCG Data Content Guidance

DCS Data Content Specification

DISDI Defense Installations Spatial Data Infrastructure

DISR DoD IT Standards Registry

DoD Department of Defense

DoDM DoD Manual

DLS Data Layer Specification

DoDD Department of Defense Directive

DoDI Department of Defense Instruction

DUSD(I&E) Deputy Under Secretary of Defense for Installations and Environment

EI&E

GEOINT

Energy, Installations, and Environment

Geospatial Intelligence

GIO Geospatial Information Officer

GWG Geospatial Intelligence Standards Working Group

HQ Headquarters

I&E Installations and Environment

IGG IGI&S Governance Group

IGI&S Installation Geospatial Information and Services

ISO International Standards Organization

JESC Joint Enterprise Standards Committee

NGA National Geospatial-Intelligence Agency

NMF National System for Geospatial-Intelligence (NSG) Metadata Foundation

NMIS

NSG

NSG Metadata Implementation Specification

National System for Geospatial-Intelligence

OSD Office of the Secretary of Defense

QAP Quality Assurance Plan

RACI Responsible, Approval, Consulted, Informed

RAM Responsibility Assignment Matrix

RPIR Real Property Inventory Requirements

SDSFIE Spatial Data Standards for Facilities, Infrastructure, and Environment

SDSFIE-M Spatial Data Standards for Facilities, Infrastructure, and Environment –

Metadata

SDSFIE-Q Spatial Data Standards for Facilities, Infrastructure, and Environment -

Quality

SDSFIE-R Spatial Data Standards for Facilities, Infrastructure, and Environment -

Raster

SDSFIE-S Spatial Data Standards for Facilities, Infrastructure, and Environment -

Services

SDSFIE-V Spatial Data Standards for Facilities, Infrastructure, and Environment -

Vector

SMIS SDSFIE Metadata Implementation Specification

Document Maintenance

This document will be reviewed and updated as needed, with each action documented in a revision history log. When changes occur, the version number will be updated to the next increment and the date, owner making the change, and change description will be recorded in the revision history log of the document.

Terms and Definitions

Accessible

Data and services can be accessed via the global information grid (GIG) by users and applications in the enterprise. Data and services are made available to any user or application except where limited by law, policy, security classification, or operational necessity. [DoD CIO Memorandum]

Accuracy

Closeness of agreement between a test result or measurement result and the true value. [ISO 3534- 2:2006]

Aggregate

A data quality scope used to evaluate metrics applied to a dataset that is aggregated from other datasets.

Authoritative Data Source

A recognized or official data source with a designated mission statement, source, or product to publish reliable and accurate data for subsequent use by customers. An authoritative data source may be the functional combination of multiple separate data sources. [DoDD 8320.03]

Common Installation Picture

The distinct minimum set of geospatial features and imagery necessary to provide a foundational map depicting DoD installations and sites. The purpose of the CIP is to provide a readily available, standardized map background to serve as the basis for planning and execution of EI&E responsibilities and functions. [DoDI 8130.01]

Conformance

Fulfillment of specified requirements. [ISO 19105]

Conformance Quality Level

Threshold value or set of threshold values for data quality results used to determine how well a dataset meets the criteria set forth in its DCS or user requirements. [ISO 19157]

Correctness

Correspondence with the universe of discourse. [ISO 19157]

Data Content Specification

Detailed description of a dataset or dataset series together with additional information that will enable it to be created, supplied to and used by another party. [ISO 19131]

Note: The above definition is for a data product specification. In SDSFIE-Q data product specifications are called data content specifications.

Data Quality Basic Measure

Generic data quality measure used as a basis for the creation of specific data quality measures. [ISO 19157]

Note: Data quality basic measures are abstract data types. They cannot be used directly when reporting data quality.

Data Quality Element

A component describing a certain aspect of the quality of geographic data.

Data Quality Result

The output of a data quality evaluation.

Data Quality Scope

Specifies the extent, spatial and/or temporal, and/or common characteristic(s) that identify the data on which data quality is to be evaluated.

Data Quality Unit

The combination of a scope and one or more data quality elements.

Dataset

Identifiable collection of data. [ISO 19115]

Data Steward

An organization within an authoritative source that is charged with the collection and maintenance of authoritative data.

Direct Evaluation Method

Method of evaluating the quality of a dataset based on inspection of the items within the dataset. [ISO 19157]

Feature

Abstraction of real world phenomena. [ISO 19101]

Feature Attribute

Characteristic of a feature. [ISO 19101]

Feature type

Class of features having common characteristics. [ISO 19156]

IGI&S

The subset of GI&S activities that apply to the management of DoD installations and environment to support military readiness in the Active, Guard, and Reserve Components with regard to facility construction, sustainment, and modernization, including the operation and sustainment of military test and training ranges, and which support DoD business enterprise priorities as defined in the DoD business enterprise architecture (BEA). IGI&S supports and is enabled by geospatial engineering and general engineering as defined in Joint Publication 3-34. [DoDI 8130.01]

Interoperability

The ability of systems, units, or forces to provide data, information, materiel, and services to and accept the same from other systems, units, or forces and to use the data, information, materiel, and services so exchanged to enable them to operate effectively together. IT and National Security Systems interoperability includes both the technical exchange of information and the end-to-end operational effectiveness of that exchange of information as required for mission accomplishment. More than just information exchange, it includes systems, processes, procedures, organizations, and missions over the lifecycle and must be balanced with information assurance. [DoDI 8130.01]

Measure

A description of the type of evaluation being assessed to determine a data quality result.

Metric

A specific, measurable indicator that addresses data quality compliance.

Note: metrics are created when measures are assigned a scope.

Metadata

Information describing the characteristics of data, data or information about data, or descriptive information about an entity’s data, data activities, systems and holdings. For example, discovery metadata allows data assets to be found using enterprise search capabilities. Metadata can be structural (specifying the format structure), semantic (specifying the meaning), or descriptive (providing amplifying or interpretive information) for data, information, or IT services. [DoDI 8130.01]

Metaquality

Information describing the quality of data quality. [ISO 19157]

Quality

Degree to which a set of inherent characteristics fulfills requirements. [ISO 19157]

Repository

A data quality scope used to evaluate metrics applied to a data repository.

Standalone Quality Report

Free text document providing fully detailed information about data quality evaluations, results and measures used. [ISO 19157]

Trusted

Users and applications can determine and assess the suitability of the source because the pedigree, security level and access control level or each data asset or service is known and available. [DoD CIO Memorandum]

Understandable

Users and applications can comprehend the data, both structurally and semantically, and readily determine how the data may be used for their specific needs. [DoD CIO Memorandum]

Visible

The property of being discoverable. All data assets (intelligence, non-intelligence, raw, and processed) are advertised or “made visible” by providing metadata, which describes the asset. [DoD CIO Memorandum]

2 ISO 19157 Components of a Quality Framework

The data quality concepts defined in ISO 19157: 2013 Geographic Information: Data Quality provide the foundation for the structure of SDSFIE-Q. SDSFIE-Q conforms to the ISO 19157 conceptual model of quality for geographic data. This model defines components for measuring, evaluating and reporting data quality based on the ISO 19157 data quality elements. Figure 1 (below), from ISO 19157 (Figure 1), provides an overview of these components. This section will describe each ISO quality framework component, providing the background necessary to understand the IGI&S Quality Management Framework introduced in section 3.

Figure 1: ISO 19157 Components of a Quality Framework

ISO 19157 Data Quality Evaluation Process

As shown in Figure 2 (ISO 19157 Figure 12) below, the data quality evaluation process consists of establishing data quality units, defining measures, determining the proper evaluation method and reporting the results of the evaluation in metadata. Data quality should be evaluated when data is created or updated, to determine compliance with a data content specification or to determine compliance with user requirements.

Figure 2: ISO 19157 Data Evaluation Process

(ISO 19157 Figure 12)

2.1.1 Data Quality Units

Per ISO 19157, there are two aspects of data quality; a scope and a defined data quality element. The scope and data quality element pair is referred to as the data quality unit. A data quality unit must be defined before determining measures and evaluation methods (see Figure 2, Step 1). Figure 3 below provides examples of data quality units.

Figure 3: Data Quality Units

2.1.1.1 Data Quality Scope

The data quality scope identifies the extent (spatial, temporal, subset with defined characteristics, etc.) of the data that will be evaluated. Scope is reported in the DQ_Scope element. It is constrained by the MD_ScopeCode2. Table 1 below contains the list of scope codes applicable to SDSFIE-Q and SDSFIE- M.

Table 1: Metadata Codes for Data Quality Scope (from ISO 19115)

MD_ScopeCode Definition Applicable to SDSFIE-Q and SDSFIE-M dataset information applies to the dataset X service information applies to a capability which a service provider entity makes available to a service user entity through a set of interfaces that define a behaviour, such as a use case

X metadata information applies to metadata X document information applies to a document X repository information applies to a repository X aggregate information applies to an aggregate resource X application Information resource hosted on a specific set of hardware and accessible over a network

X

2.1.1.2 ISO 19157 Data Quality Elements

Once a scope is defined, a data quality element must be specified. The data quality element describes the specific aspect of data quality that is being evaluated. ISO 19157 divides data quality elements into six categories: completeness, logical consistency, positional accuracy, thematic accuracy, temporal

2 Scope is constrained in metadata by the MD_ScopeCode element in ISO 19115:2003, which is the ISO standard for metadata.

accuracy, and usability. The following sections define these data quality elements in accordance with ISO 19157.

2.1.1.2.1 Completeness

Completeness is defined as the presence and absence of features, their attributes and relationships.

Completeness is comprised of the following elements:

Commission – data that is excess or additional data beyond what is required.

Omission – data that is absent or missing from the data layer.

2.1.1.2.2 Logical Consistency

Logical Consistency is defined as the adherence to the ruleset for the data structure, attribution and relationships. Logical Consistency consists of:

Conceptual consistency – adherence to rules of the conceptual schema.

Domain consistency – adherence of values to the value domains.

Format consistency – degree to which data is stored in accordance with the physical structure of the dataset.

Topological consistency – correctness of the explicitly encoded topological characteristics of a dataset.

2.1.1.2.3 Positional Accuracy

Positional Accuracy is defined as the accuracy of the position of features within a designated spatial reference system. Positional accuracy includes both absolute and relative accuracy. Positional accuracy consists of the following data quality elements:

Absolute or external accuracy - the closeness of reported coordinate values to values accepted as true.

Relative or internal accuracy - the closeness of the relative positions of features in a dataset to their respective relative positions accepted as true.

2.1.1.2.4 Temporal Quality

Temporal Quality is defined as the quality of the temporal aspect of the data layer. Temporal Quality addresses the following elements:

Accuracy of a time measurement – closeness of reported time measurements to true values.

Temporal consistency – correctness of the order of events.

Temporal validity – validity of data with respect to time.

2.1.1.2.5 Thematic Accuracy

Thematic Accuracy is defined as the accuracy or correctness of attributes. Thematic Accuracy consists of:

Classification correctness – correctness of the classes of features or attributes.

Non-quantitative attribute correctness – correctness of an attribute.

Quantitative attribute accuracy – accuracy of an attribute compared to a known value.

2.1.1.2.6 Usability

The usability element is based on user requirements and is therefore less structured than the other elements. Usability can describe quality information about a dataset’s suitability for a particular application, or conformance to a set of requirements. For SDSFIE-Q, usability is defined as the adherence to a data content specification.

2.1.2 Measures

A measure is a description of the type of evaluation being assessed to determine a data quality result. A measure will be evaluated for a specific data quality unit. Data quality measures can be derived from a basic measure (see Step 2 in Figure 2). A basic measure is a generic data quality measure used as a basis for the creation of specific data quality measures. Basic measures are used for count-related and general statistical measures that share commonalities. Appendix A contains a summarized list of measures and their corresponding basic measures, where applicable, as identified in ISO 19157 Annex G. Examples of measures include:

Rate of missing items

Number of duplicate feature instances

Conceptual schema compliance

Number of invalid self-intersect errors

Number of incorrectly classified features

When creating measures that are not included in ISO 19157 Annex G, the user should structure the new measures using the standard basic measures.

When measures are associated with a specific data quality unit (scope and element), they are referred to as metrics within this document (see section 3.3.4)

2.1.3 Evaluation Methods

An evaluation method is the procedure used to evaluate the defined measure for a data quality unit (see Step 3 in Figure 2). As described in ISO 19157, data may be evaluated using direct, indirect or aggregation-based methods.

1. Direct evaluation requires a full inspection or sampling of the data, either with or without external reference data. The results of a direct evaluation will normally be quantitative. External direct evaluation indicates that external reference data is used for the evaluation. Internal direct evaluation indicates no external data was referenced.

2. Indirect evaluation involves using inherent knowledge along with existing information, such as the source and date, to determine the quality of the data. Indirect evaluation is not recommended for SDSFIE-Q since the results may be subjective, inconsistent and not repeatable.

3. Data quality results may be aggregated or derived from the results of existing evaluations. These results can be aggregated from different scopes and data quality elements to provide an aggregated result.

2.1.4 Results

The output of a data quality evaluation is a result (see Step 5 in Figure 2). Each data quality element evaluated can have multiple results for a dataset. ISO 19157 identifies the following types of results:

Quantitative Result – A single value or multiple values based on the measure being evaluated.

Conformance Result – Used when the results of a data quality evaluation are compared with a known conformance quality level. The conformance quality level should be specified in a data content specification or user requirements.

Descriptive Result – A textual statement describing the subjective data quality evaluation result. Descriptive results are not recommended for SDSFIE-Q since the results are based on a subjective, inconsistent and not repeatable evaluation method.

Coverage Result – The result of a data quality evaluation, organized as a coverage.

2.1.5 Metaquality

Metaquality is used to evaluate the results of a data quality evaluation, or to compare results from multiple data quality evaluations. Metaquality can evaluate confidence, representativity or homogeneity of a dataset or multiple datasets.

ISO 19157 Data Quality Reporting

Reporting the results of a data quality evaluation provides users with information to determine the overall quality of a dataset, including compliance with data content specifications or satisfaction of other user requirements. The results of a data quality evaluation will be reported in metadata using the DQ_DataQuality element. A stand-alone data quality report may also be required if the data quality results are aggregated or derived. The stand-alone report should be used to provide additional details that cannot be expressed in the metadata.

3 The SDSFIE-Q Framework

Quality management for IGI&S data and services is built on the ISO 19157 components of data quality management discussed in the previous section. The SDSFIE-Q framework consists of: roles and responsibilities (including specific data quality management activities); and a workflow of operational activities. These roles, responsibilities, and activities are described in the following sections.

Defined Roles

Most SDSFIE-Q roles and responsibilities flow from the authorities described in section 1.2, specifically from DoDI 8130.01. The roles identified in data quality management are as follows:

IGI&S Governance Group (IGG)

The IGG is responsible for establishing data quality guidance to promote the adoption and refinement of SDSFIE-Q. The IGG will maintain the metadata content standard and provide guidance for using the metadata standard to describe data quality. The IGG will create guidance for and review Component quality managment plans in accordance with SDSFIE- Q.

OASD(EI&E) Geospatial Information Officer (GIO)

The role of the GIO is established in DoDI 8130.01. Specific responsibilities involve leadership and support of the IGG regarding data quality processes and quality management plans, leadership of the data quality working group, and coordination with the DoD GEOINT Manager. The GIO is also responsible for publishing data content specifications at the OSD level in coordination with the Data Quality Working Group.

Component IGI&S Programs

The DoD Component IGI&S Programs are responsible for coordinating and directing the creation, maintenance, and distribution of IGI&S data at the Component level. Component IGI&S Programs will develop Component level data quality metrics and guidance through quality management plans and data content specifications.

SDSFIE Quality Working Group

The IGG created the SDSFIE Quality Working Group to establish enterprise processes and management structures for evaluating data quality. Data Quality Working Group responsibilities include:

Formulate an enterprise approach to IGI&S data quality.

Develop a quality management framework that can be integrated with IGI&S Program management processes.

Identify IGI&S requirements for data quality and establish measures and processes to evaluate data maturity and compliance.

Authoritative Data Source (ADS) Manager

A person, office, or organization responsible for maintaining the IGI&S ADS for each installation. The ADS Manager role is Component-specific. Refer to Component guidance for more information about how the role is implemented.

Policy Proponents

A policy proponent is a DoD organization that has responsibility for enterprise data requirements and guidance. A policy proponent is typically at the Component headquarters or OSD level. Policy proponents are not normally responsible for creating and maintaining spatial data.

Data Requirements Proponents

An agency, department, activity or organization that has primary responsibility for material or subject matter expertise in its area of interest and has lead responsibility for coordinating the collection, coverage and stewardship, including maintenance and update, of a specific spatial data theme or mission dataset. Data proponents are not typically directly in the line of authority for the IGI&S Programs, but create and maintain spatial data. Relations must be developed with data proponents so that the data they create is compliant with enterprise spatial data requirements. Data requirement proponents are responsible for providing clear data requirements and specifications for data.

DoD Contract Office (Project-Based)

The government official responsible for generating and approving contract arrangements for creation and maintenance of project-based IGI&S data.

The contract officer shall ensure that contracts for the generation of IGI&S data include contract language requiring adherence to DoD standards and DCS requirements and shall also ensure that all deliverables are reviewed to confirm that standards and specifications are met.

IGI&S Program Analysts

IGI&S Program Analysts are DoD contracted or government personnel that are responsible for creating and maintaining IGI&S data. IGI&S Program Analysts may be under the direct control of an IGI&S program, but they may also be controlled by other business lines or mission areas (e.g. public works, environmental, etc.). These analysts must ensure that products adhere to standards and specifications (e.g. DCS).

Users

Data users are those persons who need to discover spatial data resources, develop content, or make information release decisions.

Spatial data end-use can be constrained by certain quality considerations, and end users bear responsibilities associated with these considerations in their end use such as:

Consulting metadata to know specific constraints on data being used.

Understanding spatial data constraint implications on decision making.

Reviewing data content products before they are released.

These roles and associated responsibilities, which correspond to the operational activities described in Figure 4, are outlined in Table 2 through the use of a responsibility assignment matrix (RAM), also known as a RACI matrix. A RACI matrix describes the participation by various roles in completing tasks or deliverables for a project or business process3.

3 Margaria, Tiziana (2010). Leveraging Applications of Formal Methods, Verification, and Validation: 4th International Symposium on Leveraging Applications, Isola 2010, Heraklion, Crete, Greece, October 18–21, 2010, Proceedings, Part 1. Springer. p. 492. ISBN 3- 642-16557-5.

RACI is an acronym derived from the four key responsibilities used in this document4:

Responsible (R) - Roles that have a significant responsibility in performing the objective as well as government roles that have responsibility over an objective.

Approval (A) – Roles that have review and approval authority. There may be several levels of review in the same objective.

Consulted (C) – Roles that collaborate and consult at some point during the process but do not have approval authority.

Informed (I) – Roles that are socialized concerning the activities or artifacts but do not have approval, review nor consulting responsibilities.

4 Note that the set selected for this document is most commonly used for decision-making. The “A” is often used to mean “Accountable”, but, in our case “Approval” is a better choice.

Table 2: Data Quality RACI Chart

Quality Management Activities

A p p lic ab ili ty

SD

SF

IE

P ar t

(V , R

S)

IG

I&

S G o ve rn an ce G ro u p

IG

G

O A

SD

(E

I& E)

G eo sp at ia l I n fo rm at io n

O ff ic er

G

IO

C o m p o n en t

IG

I&

S P ro gr am s

SD

SF

IE

Q u al it y

W G

A D

S M an ag er

P o lic y

P ro p o n en ts at a R eq u ir em e n ts P ro p o n en o

D C o n tr ac t O ff ic e (P ro je ct -B as ed

IG

I&

S P ro gr am A n al ys

U se rs

Define Geospatial Requirements

Develop Quality Management Plans V,R,S A A R,A C C C C I I I

*Develop Data Content Specifications

V,S A R R C C C C I I,C I

Specify SDSFIE Quality Contract Language

V,R,S I I C I C C C A,R I I

*Define Quality Metrics V,R,S A R R,A R,C R C C I I I

Create Geospatial Information

*Create and Acquire Data V,R,S A R R C R I R R R I

*Create Metadata V,R,S A R R C I C R R R I

Validate Geospatial Information

Evaluate Data Quality V,R,S I I R,A I R I I R R I

*Report Quality Results V,R,S A R R C R R I

Maintain Geospatial Information (Cleansing)

*Cleanse Data V,R,S I R R C I I R R R I

Obtain Resources for Cleansing V,R,S I R R C I I R R C

Provide Geospatial Information

*Determine Releasability V,R,S I R R I I R C R R I

*Make Data Accessible and Discoverable (includes providing geospatial data, services, visualization services and analysis)

V,R,S I R R I R I R R R I

Provide Feedback V,R,S C C R,A C C R R R R R

Note: Responsibilities with an asterisk (*) denote objectives that will happen at OSD level and at Component level, and thus may have different responsibilities for a given role at each level

Quality Management Workflow

Figure 4 is an OV-5 Operational Activity diagram. It details the workflow of quality management activities for the IGI&S data lifecycle, from requirements gathering to completeness. These activities correspond to Quality Management Activities on the RACI matrix (Table 2). Each operational activity is discussed in further detail in the following sections:

Define Geospatial Requirements (section 3.3)

Create Geospatial Information (section 3.3.4.3)

Validate Geospatial Information (section 3.5)

Maintain Geospatial Information (Cleansing) (section 3.6) o Obtain Resources (section 3.6.1.1)

Provide Geospatial Information (section 3.7) o Provide Geospatial Data and Geospatial Data Services (section 3.7.2.1) o Provide Geospatial Visualization Services (section 3.7.2.2) o Provide Geospatial Analytical Services (section 3.7.2.3)

Figure 4: IGI&S Quality Management Operational Activity Diagram

Quality Management Activity: Define Geospatial Requirements

Quality Management tools are methods used to define geospatial requirements and execute quality control and assurance through guidance, contract language, data content specifications (DCS) and metrics. The following quality management actions are identified in SDSFIE-Q:

Develop Quality Management Plans (section 3.3.1)

Develop data content specification (DCS) (section 3.3.2)

Specify SDSFIE Quality Contract Language (section 3.3.3)

Define Quality Metrics (section 3.3.4)

3.3.1 Quality Management Plans

Each DoD Component shall develop a quality management plan to ensure consistency and quality of IGI&S data and services. The quality management plan shall guide all levels of the Component with respect to implementation of SDSFIE-Q, providing mission-specific guidance, metrics or other direction as needed.

The quality management plan shall include the following information:

Roles and responsibilities for data quality management at the Component level.

Data quality units in addition to those established in SDSFIE-Q.

Metrics and associated measures reported to the Component headquarters and the process for identifying and establishing new metrics. Components should refer to the standardized ISO 19157 measures listed in Appendix A to aid in developing new metrics requirements.

Identification of IGI&S data repositories, and their structure.

Data content specifications defined at the HQ level (may be incorporated by reference).

A projected timeline for completion/update of DCSs with priority given to CIP feature types.

The process for collecting and maintaining layer-level metadata.

Component quality management plans will be reviewed and validated by the IGG through the document approval process defined in the SDSFIE Governance Plan (section 3.4.1) to ensure they are consistent with DoD policy and IGG guidance or processes.

3.3.1.1 Quality Management Plan Implementation Scorecard

Each Component’s progress toward implementation of SDSFIE-Q will be evaluated using the following Implementation Scorecard categories, as established in the SDSFIE Governance Plan, Revision 2 (section 3.3.5.5):

Green: A quality management plan exists and a high level of implementation progress has been made.

Yellow: A quality management plan exists and a significant level of implementation progress has been made.

Red: No quality management plan exists or little or no implementation progress has been made.

The timing of scorecard implementation will be determined by future IGG guidance and procedures.

3.3.2 Data Content Specification (DCS)

OSD and Component IGI&S Programs shall create DCSs for IGI&S vector data. A DCS (alternatively known as a Quality Assurance Plan (QAP), Data Layer Specification (DLS), or Data Content Guidance (DCG)) documents the format and quality expectations for a geospatial data layer (or set of layers). The quality criteria in the DCS shall be used to direct and control quality during the generation, collection or maintenance of geospatial features. The DCS shall also specify metrics and data quality evaluation procedures for each layer. See section 4.3 for DCS format, content and quality reporting guidance.

3.3.3 SDSFIE Quality Contract Language

Data quality is inextricably linked with data production or procurement. Whenever IGI&S data or services are procured, DoD activities should use contract language which specifies the data collection and maintenance expectations in acquisition-binding terms. The contract provisions/requirements should ensure deliverables conform to all standardization and quality requirements specified in the SDSFIE family of standards, including SDSFIE-Q. For example, DoD contracts for installation geospatial data, installation master plans or integrated natural resources management plans (INRMPs) should include a section which requires that GIS data used by the contractor and submitted to the government in any final deliverables must be formatted in accordance with the Component’s current, registered Adaptation of SDSFIE-V and applicable DCSs. The deliverables should include metadata formatted in accordance with the current version of SDSFIE-M, and should also be populated to conform to SDSFIE-Q requirements as well as any applicable Component quality management plans. Annex C of this document (to be published) contains more specific guidelines for how to cite SDSFIE standards in DoD contracts pertaining to IGI&S.

3.3.4 Define Quality Metrics

The IGG, Component IGI&S Programs and ADS Managers are responsible for identifying and establishing data quality metrics based on policy drivers and business requirements. Metrics are specific, measureable indicators that address data quality compliance. Metrics are created when measures (as defined by ISO 19157) are assigned a corresponding scope, as shown in the example in Figure 5 below.

In this example, the number of buildings reported in the Component’s Accountable Property System of Record (APSR) are compared to the number of buildings included in the feature type for building. The data unit consists of the dataset scope and the omission data quality element. The rate of missing items is being measured, which is based on the error rate basic measure. Since the measure is associated with a scope, a metric is created: the percentage of buildings reported to the APSR that do not have a corresponding geospatial representation.

Figure 5: Quality Measures vs. Quality Metrics

Data quality metrics shall be defined for each data quality unit established in SDSFIE-Q, DCSs or Component quality management plans. A list of identified OSD and Component level IGI&S data quality metrics, including their scope, measures and evaluation methods is included in Appendix B. Components may include additional metrics in their Component-level quality management plan, such as those required for Component headquarters reporting. A summarized list of standard measures established in ISO 19157, including the associated basic measure (where applicable), is included in Appendix A. This list should be used for reference when creating new metrics.

3.3.4.1 IGI&S Data Quality Scope

When creating metrics and reporting data quality results, the data quality scope must be identified. The type of metrics defined for IGI&S data are dependent on these identified scope levels. The data quality scope of the data quality unit defines the extent of the data being evaluated. This is normally equivalent to the scope of the metadata record, however, the data quality scope must be at the same or lower hierarchy as the scope of the metadata record. In SDSFIE-Q, the data quality result scope will always be identical to the data quality scope, due to the limitations of the standard procedures and tools for collecting metadata. Table 3 lists the data quality scopes and corresponding administration levels established in SDSFIE-Q. Components may establish additional scopes in their quality management plans and DCSs, such as “feature” and “attribute” that are not included in this list.

The following scopes, listed in hierarchical order, are defined for IGI&S data:

Repository: The repository scope is used to measure quality of any IGI&S data store (e.g.

ADS) as a whole. See Part 4 for reporting requirements for repositories.

Aggregate: The aggregate scope is used to measure quality of an aggregated dataset (e.g.

CIP, Component CIP) that may be extracted from a data repository. See Part 4 for reporting requirements for aggregates.

Dataset (Vector): The dataset scope is used for vector datasets comprised of a single or multiple feature types. The dataset scope will not be used when measuring the overall quality of CIP datasets and IGI&S data repositories. See Part 2 for reporting requirements for vector data.

Dataset (Raster): The dataset scope may also be used to measure data quality for raster data. See Part 3 for raster data reporting requirements.

Service: Data quality measured for web services shall be reported as a “service” in DQ_Scope.

Application: Data quality measured for applications shall be reported as an “application” in DQ_Scope

Document: Data quality measured for supporting documentation, such as a standalone data quality report or a Component quality management plan will be reported as a “document” in DQ_Scope.

Metadata: When reporting the quality of metadata associated with IGI&S data and services, the “metadata” scope should be used.

Table 3: Data Quality Scope Codes for IGI&S Data

Administration Level

Data Quality Scope SDSFIE Part

Use Description

Component HQ repository V,R,S To create metrics and report data quality for the ADS as a whole

Component HQ dataset (raster) R To create metrics and report data quality for raster datasets created by the Component

Component HQ dataset (vector) V To create metrics and report data quality for individual feature types (CIP, non-CIP, ADS layers, and ad hoc datasets).

To create metrics and report data quality for non-CIP or

ADS vector datasets comprised of more than one feature type.

Component HQ service S To create metrics and report data quality for Component services

Component HQ application S To create metrics and report data quality for Component developed applications

Component…