Working with Raster Attribute Tables
Unlike vector data, a feature with a raster geometry will seldom have attributes attached to the feature itself. However, some raster datasets have machine-readable metadata in the form of a raster attribute table that associates raster cell values with attributes.
In Carmenta Engine, a raster attribute table can be used for thematic data rasters, which contain integers that represent values in a discrete enumeration type. For example, in a thematic raster of land cover types, we could have 11 representing water, 21 representing residential areas, 31 representing barren land, etc. The raster attribute table could then contain a String attribute named LAND_COVER, so that it can associate the value 11 with "Water", 21 with "Residential", 31 with "Barren Land", etc.
VALUE | LAND_COVER | ESTIMATED_HEIGHT |
|---|---|---|
11 | "Water" | 0.0 |
21 | "Residential" | 15.0 |
31 | "Barren Land" | 0.0 |
41 | "Deciduous Forest" | 20.0 |
42 | "Evergreen Forest" | 30.0 |
43 | "Mixed Forest" | 25.0 |
51 | "Shrubland" | 5.0 |
71 | "Grasslands" | 0.0 |
In a schematic layout as above, each row (except the header with attribute names) represents an integer and its attributes. One of the columns is a special value column, and each of the other columns represents the name of an attribute and its values for various raster cell values.
When a Carmenta Engine dataset reads a raster attribute table, each row is represented by a data structure of the type RasterValueAttributeSet , which contains the integer from the special value column and an attribute set representing the rest of the row. Note that Carmenta does not include the special value integer in the attribute set: we regard each value as something that has attributes but is not itself an attribute. The entire table is represented as a collection of such data structures; the collection will be wrapped in an instance of the class RasterAttributeTable, which will be available for programmers as the DataSetInfo.RasterAttributeTable in the DataSetInfo for the dataset (although if it is a multi-file dataset, the root DataSetInfo only gets a raster attribute table if all its children have identical tables).
Note that the design of the row data structure is intended only for thematic raster data. Carmenta Engine will not read a different kind of raster attribute table, where an attribute set is associated with a numeric interval (for example, associating the attribute Beaufort_Number = 3 with the wind speed interval 12—19 km/h).
When Carmenta Explorer displays a map configuration, you can select a dataset in the DataSets panel to see the raster attribute table, if any, in the Properties panel. However, the attribute values for each raster cell value will not appear horizontally as in the schematic layout above.
 |
If the raster attribute table has at least one string attribute, the table and the DataSetInfo.ColorTable will be used by Carmenta Explorer to generate an automatic legend that is also displayed in the Properties panel. This legend is not an explicit part of the DataSetInfo for the dataset, nor an explicit part of the map configuration - it just gives a more compact and readable display of information from the raster attribute table and the ColorTable.
If you want to generate legends from raster attribute tables in a similar way in your own application, you can use a static method of the Legend class, Legend.FromRasterDataSetInfo. The method is versatile and has some overloaded variants that can be used, depending on whether or not you want to specify one or a few columns to use for the item descriptions, and on whether you want to include the raster cell value in each item description, and on the color source.
If you use Carmenta Explorer to open a data file or directory directly (using File → Open → Open Data File... or File → Open → Open Data Directory...), and a dataset with a raster attribute table is found, and there is at least one string attribute, then Carmenta Explorer will also generate an explicit Legend object and attach it to the OrdinaryLayer for the dataset. This legend will be displayed in the Properties panel when the layer is selected in the Layers panel. If you then save the generated configuration as a .px file via File → Save Configuration..., you can open it in Carmenta Studio and edit the legend object for the layer.
Using a Raster Attribute Table from a ReadOperator
Note that a raster attribute table in Carmenta Engine is associated with a dataset, not with each raster feature that is read from the dataset. This is mainly because some Carmenta Engine operators could manipulate the cell values in a raster feature in a way that would invalidate an associated table. So, most built-in operators of Carmenta Engine cannot access the raster attribute table. However, a ReadOperator can do it since it has direct access to its DataSet, and you can modify its behavior by using special properties on a query that you attach to the operator:
Reclassification by attribute table
The query property reclassificationByAttributeTable can be used to make the ReadOperator replace each raster cell value by a corresponding numeric attribute. For example, with a table like the one above, you could replace each integer value by its ESTIMATED_HEIGHT number.
Colors from attribute table
In a perfect world, one should never need to use the query property colorFromAttributeTable that gets the colors from the attribute table, because the correct color information should already be present in the main data file, for example in a .tif file. After all, storing colors for images is what formats like TIFF are designed to do. However, in the real world, one can encounter thematic raster datasets where the main data file has no color table or just a trivial grayscale, while the raster attribute table has numeric columns RED, GREEN and BLUE, intended to define the colors.
By default, Carmenta Engine expects the colors to be defined in the main data file. If you are designing a configuration manually in Carmenta Studio and want to get the colors from the raster attribute table instead, you must define an explicit ImageQuery.ColorFromAttributeTable or GdalQuery.ColorFromAttributeTable.
Carmenta Explorer can construct a colorFromAttributeTable
Defining a colorFromAttributeTable manually can be cumbersome. However, if you use Carmenta Explorer to open a data file or directory directly (using File → Open → Open Data File... or File → Open → Open Data Directory...), it uses a rule of thumb to determine whether it should insert a query with a colorFromAttributeTable property in the generated configuration. The rule of thumb may not work for all data sources, which is a reason not to make it the default behavior for the Carmenta Engine SDK. But since Carmenta Explorer is a separate application for developers that is built on top of the SDK, we can allow it to make some mistakes when opening data files, as long as the rule of thumb works more often than not. If the generated configuration is saved via File → Save Configuration..., you can open it in Carmenta Studio and see that the read operator for the dataset has gotten a query with an explicit colorFromAttributeTable.
 |
There is also one situation where Carmenta Explorer can use its rule of thumb when displaying an already existing configuration file. As mentioned above, if a dataset has been selected in the DataSets panel of Carmenta Explorer, and has a raster attribute table with at least one string attribute, the Properties panel will display a legend that the Explorer has auto-generated from DataSetInfo.ColorTable and DataSetInfo.RasterAttributeTable. The colors in the generated legend are usually taken from the explicit ColorTable, but if the rule of thumb is applicable, they will be taken from the raster attribute table via a generated colorFromAttributeTable. In the latter case, the legend will end with an extra row whose property name is "Assumed colorFromAttributeTable", and whose value is the color expression (see the Alabama screenshot above). If you want to copy this color expression to Carmenta Studio manually, you can right-click on the extra row and choose "Copy Value", then you open Carmenta Studio, create an ImageQuery or a GdalQuery, open the custom editor for colorFromAttributeTable, choose "Value from attribute expression", and paste the color expression into the Expression editor.
You can of course decide to override both the basic ColorTable and the colors in the attribute table by inserting a VisualizationOperator with an explicit RasterVisualizer in your operator chain. But since that would not change the dataset properties, they would appear as before in the Properties panel. Instead, you can use your raster visualizer in an explicit Legend that you attach to your OrdinaryLayer; this legend will then appear in the Properties panel when the layer is selected in the Layers panel.
Formats for Storing Raster Attribute Tables
Some raster data formats can store a raster attribute table internally. For example, the ERDAS IMAGINE format HFA can store a table in the main .img file.
For other raster data formats like TIFF, there are several conventions for storing the table in an auxiliary file. If the main file is called Filename.tif, then the auxiliary file could be:
Filename.tif.aux.xml This kind of file was introduced by GDAL Library, but nowadays there are many geodata vendors that produce their own .aux.xml files, and some are not fully GDAL-compliant.
Filename.aux This is a binary format introduced by ERDAS IMAGINE for use with non-ERDAS raster formats. It is also supported by GDAL Library.
Filename.tif.vat.dbf Using a dBASE file for raster attribute tables is an Esri convention that other GIS systems may not support; for example, GDAL Library currently does not, nor does QGIS.
The first two kinds of auxiliary files can contain other metadata as well, and will not necessarily contain any raster attribute table.
There are two Carmenta Engine datasets that can read raster attribute tables, ImageDataSet and GdalDataSet. Both can read all three kinds of auxiliary files, and GdalDataSet can also read some internal tables like those in the ERDAS IMAGINE .img files.
Some .vat.dbf issues
For a Filename.tif.vat.dbf file, Carmenta Engine will not read the first column named OID (Object Identifier), which is visible in Esri software but not in Excel. The contents of this implicit column is just the row numbers 0, 1, 2, 3, ..., which do not seem very useful.
As for the code page that defines the character encoding, Carmenta Engine will read it from a Filename.tif.vat.cpg file, if any, and otherwise deduce it from the internal Language Driver ID in the Filename.tif.vat.dbf file, and failing that, assume UTF-8. There is no way to specify an explicit code page for a .vat.dbf file in a .px configuration. So, for example, if a geodata vendor has used code page 1252 but neglected to set the Language Driver ID or provide a CPG file, Carmenta Engine will assume UTF-8 which means that diacritics will be garbled. The simplest workaround in this case would be to write 1252 in a text file and save it as Filename.tif.vat.cpg in the same folder.
Viewing raster attribute tables without Carmenta Engine
For troubleshooting, it may be useful to view the original raster attribute tables without using Carmenta Engine.
A .vat.dbf file can be viewed in Excel or in many other dBASE viewers.
An .aux.xml file can of course be viewed in any text editor, and the XML for the raster attribute table is fairly readable. There will be somewhat mysterious integers that represent Type and Usage in the Field Definitions; these integers are zero-based indexes into enumeration types, GDALRATFieldType and GDALRATFieldUsage, that are documented here. If the table begins with <GDALRasterAttributeTable Row0Min="m" BinSize="1"> for some number m, it means that linear binning is used: the special value column is then not represented explicitly, instead the attributes in the first row should be associated with the value m, the attributes in the second row should be associated with the value m + 1, etc. Carmenta Engine supports only a bin size of 1 and an integer m.
If you have installed GDAL Library, you can use gdalinfo to generate information that can include a raster attribute table from some source that GDAL Library can read; an ERDAS .aux file would be readable but an Esri .vat.dbf file would not (as of 2018). The table syntax in the gdalinfo output is the same as the table syntax used in an .aux.xml file.
The gdalinfo utility would of course use GDAL Library to read an .aux.xml file, while a GdalDataSet will read an .aux.xml file using a special parser developed by Carmenta, so it is possible that gdalinfo and Carmenta Engine will disagree about the contents. The special parser is used to handle some kinds of commonly occurring .aux.xml files that are not fully GDAL-compliant.