PWS_Attachment_8_-_Installation_Data_Health_Tool_Instructional_Document.pdf

Installation Data Health Tool Instructional Document 25 Jan 2017

Table of Contents Tool Requirements:

Items Included in the Package:

Installation Data Health Tool Summary:

Installation Data Health Tab Outputs:

Installation Data Health Tool Instructions:

Understanding the Output:

Tool Requirements

• Software required to open and use the supplied Arc Toolbox associated with this guidance package is ArcGIS 10.x.

• The Installation Data Health Tool accepts a file geodatabase or a personal geodatabase.

• The Installation Data Health Tool is meant to be ran on a 3.1.0.1 SDSFIE AF Adaptation compliant geodatabase to provide a most useful output.

Items Included in the Package

• Toolbox

• Output folder

• Input GDB folder

• 3.1.0.1 AF Adaptation Data Dictionary

• Installation Data Health Tool Instructional Document

Installation Data Health Tool Summary The Installation Data Health Tool parses a geodatabase by comparing data to the 3.1.0.1 AF Adaptation Data Dictionary and creates an Excel workbook summary report. Please note that this tool is meant to provide an overview of the geodatabases attribute data and does not edit any data. For a more in-depth and automated way to updated errors found in a geodatabase please use the Data Reviewer Batch files located at GeoBase Standards & Documentation website. The metrics that the Installation Data Health Tool assess are the following:

Populated Feature Classes: All populated feature classes Empty Feature Classes: All feature classes that are empty

Standard Feature Classes: Feature class name matches a feature class name in data dictionary

Non-Standard Feature Classes: Feature class name doesn’t match a feature class name in data dictionary

Standard Named Attribute: Attribute name matches an attribute name in data dictionary

Non-Standard Named Attribute: Attribute name doesn’t match an attribute name in data dictionary

Determinate Values: Values that are not indeterminate

Indeterminate Values: Values that are null, TBD, other, n/a, etc.

Valid Values: Determinate values that are valid according to the current data dictionary

Invalid Values: Determinate values that are NOT in the current data dictionary

The Installation data Health Tool also creates a blank justification template tab so that if the tool flags something that there is a justification for, then an explanation is able to be written.

Installation Data Health Tab Outputs Statistics Tab Explanation: This provides an overview of the above mentioned metrics Installation Benefits: Identifies installation level deficiencies within database to assist in improving overall data health and quality AFCEC Benefits: Identifies installations that require additional support improving data health and data quality

Justification Template Tab Explanation: If the tool flags something that there is a justification for, then an explanation is able to be written Installation Benefits: Able to justify why things are flagged—Note, if a Schema Change Request Form (SCRF) has been created write “Justification in SCRF” AFCEC Benefits: Provides an explanation of data inconsistencies and discrepancies that will aid in better understanding installation level issues https://cs1.eis.af.mil/sites/ceportal/ProgramGroups/Resources/GeoBase%20Documents/Data_Migration_Assistance.html

PopulatedFC Tab Explanation: Feature count of populated feature classes Installation Benefits: Able to determine how well populated each feature class is AFCEC Benefits: Able to determine which feature classes have data

EmptyFC Tab Explanation: List of empty feature classes Installation Benefits: Able to determine which feature classes are empty AFCEC Benefits: Able to determine which feature classes are empty

NonStandFC Tab Explanation: Feature count of non-standard feature classes Installation Benefits: Able to target which feature classes are non-standard, and how many features are in the feature class AFCEC Benefits: Able to target which feature classes are non-standard, and how many features are in the feature class

NonStandAttribute Tab Explanation: Feature count of non-standard attributes Installation Benefits: Able to target which attributes are non-standard, and how many features are in each attribute AFCEC Benefits: Able to target which attributes are non-standard, and how many features are in each attribute

InvalidValue Tab Explanation: List of attributes with invalid values and the Worksheet row number in the associated feature dataset tab Installation Benefits: Able to locate which attributes have invalid values AFCEC Benefits: Able to locate which attributes have invalid values

Feature Dataset Tabs (all other tabs) Explanation: After the “NonStandAttribute” tab, a detailed tab for each feature dataset is created listing the counts for indeterminate values, determinate values, valid values, and invalid values. The tool also reports the invalid values in the last column, “InvalidValueList.” The “InvalidValueList” column lists a unique invalid value, followed by a pipe, followed by its frequency. The process continues on that way in the same cell for however many unique invalid values there are for that particular attribute.

Installation Benefits: Able to look at detailed counts of each attribute AFCEC Benefits: Able to look at detailed counts of each attribute

Installation Data Health Tool Instructions Extract contents of InstallationDataHealthPackage.zip Open ArcCatalog Navigate to the “InstallationDataHealthPackage” folder Save a copy of your 3.1.0.1 file or personal geodatabase (GDB) in the “InputGDB” folder Open “DataHealthToolbox” and double click the “GDB Statistics Installation” tool

In the first drop down menu select the installation

In the second drop down menu navigate to the “InputGDB” folder and select the GDB

In the third drop down menu browse for the location of the “DataHealthToolOutput” folder

In the fourth drop down menu browse to and select the “3.1.0.1 - AF Adaptation Data Dictionary.xls”

Click “OK”

A dialog box should pop up, meaning the tool is running

If there are errors while running the tool, red text will appear in the dialog box, and the dialog box should state where in the script/GDB it encountered some errors. Should this occur please email a screen shot of the error to the AFCEC GIO Workflow at: AFCEC.GIO.Workflow@us.af.mil, and we will do our best to assist you.

Once the tool is done running the dialog box will close or remain open with a message stating successful completion, depending on your ArcGIS set-up. Now navigate to the “DataHealthToolOutput” folder outside of ArcCatalog in Windows Explorer and open the Excel Workbook mailto:AFCEC.GIO.Workflow@us.af.mil

The first tab is the “Statistics Tab”

Standard vs. Non-Standard:

A standard attribute/feature class is anything that matches the spelling of the name of the associated attribute/feature class in data dictionary. Non-standard means that the name of the attribute/feature class does not match the spelling in the data dictionary. If a feature class has no attributes, it is considered non-standard.

Determinate vs. Indeterminate:

A determinate value directly defines the attribute of a feature. An indeterminate value is a value that does not directly define an attribute of a feature. Indeterminate values are as follows: TBD, null, blank, NA, unknown, and other. Note: Although indeterminate values in some feature classes are also values found in the associated domain table, in most cases these values have been used to easily mass populate an attribute. For this reason these values have been included in the indeterminate count.

Valid vs. Invalid:

A valid value is an attribute value that matches an assigned value within its domain table in the 3.1.0.1 data dictionary. An invalid value is an attribute value that does not match an assigned value within its domain table 3.1.0.1 data dictionary. This evaluation takes into account a difference in spelling and capitalization of letters/words. Note: These counts are excluding the indeterminate values, due to the reasoning stated in the note above.

The following is the “JustificationTemplate” tab, where if the tool flags something that there is a justification for, then an explanation is able to be written.

The next four tabs after the “JustificationTemplate” are “PopulatedFC”, “EmptyFC”, “NonStandFC”, and “NonStandAttribute”

These tabs show the feature count by the associated variable (populated feature classes, non-standard feature classes, and non-standard attributes). The “EmptyFC” tab simply lists all the feature classes without any features.

After the “NonStandAttribute” tab the tool creates the “InvalidValue” tab.

The “InvalidValue” tab shows where in the Excel Workbook there are invalid values by writing the row number of the associated feature dataset tab containing the invalid value.

After the “NonStandAttribute” tab the tool creates a tab for each feature dataset found in the GDB.

In each feature dataset tab the tool lists the counts for indeterminate values, determinate values, valid values, and invalid values. The tool also reports the invalid values in the last column, “InvalidValueList.” The “InvalidValueList” column lists a unique invalid value, followed by a pipe, followed by its frequency. The process continues on that way in the same cell for however many unique invalid values there are for that particular attribute.

Understanding the Output:

This section shows a visual example of what the tool is counting.

Standard Named Attribute Cells vs. Non-Standard Named Attribute Cells

Determinate vs. Indeterminate

Valid vs. Invalid

All questions regarding the Installation Data Health Tool & Guidance Package should be directed to the AFCEC GIO Workflow at: AFCEC.GIO.Workflow@us.af.mil , and we will do our best to assist you.

mailto:AFCEC.GIO.Workflow@us.af.mil