Virtual paths
In most places in Carmenta Engine where you specify a path and/or file name, you can use a so called virtual path instead of a physical path to a file or folder in the file system.
Two different types of virtual paths are supported:
A path or file name starting with zip: points to a file or folder inside an existing zip archive. Carmenta Engine will read the data directly from the zip, without unpacking it to the file system.
A path or file name starting with data: includes the actual data as part of the path string. This can be used to embed small files, for instance raster symbols, directly into a configuration file, so they don't need to be deployed separately as additional files.
Reading data directly from zip archives
Carmenta Engine can read most types of files directly from inside a zip archive file, see the Limitations section below, without unpacking them to the local file system. This makes it possible to pack entire map configurations with configuration files, all geodata and needed resource files into a single zip archive, that can be easily deployed to multiple target systems. No installation or similar is required on the target machine, all that is needed is that the configuration file is specified as a virtual path pointing into the configuration file in the zip archive.
As a special case, local catalogs (see the Catalog Mechanism Overview) can also be zipped and deployed as single files.
Syntax
The syntax to specify a file or folder inside a zip file is:
zip:<physical path to zip archive>#<local path to file or folder inside zip>
The <physical path to zip archive> can be an absolute or relative path. Relative paths are interpreted depending on context: If used in a configuration file, relative the configuration file itself; if specified through the API, relative the current directory of the process.
The <local path to file or folder inside zip> specifies the path inside the zip archive. Only forward slashes must be used, even on Windows. No initial slash should be specified for the local part. Note that the local path is case-sensitive. If the local part is empty, the path refers to the root folder of the zip archive.
If you have a map configuration that works on Windows on a case-insensitive file system, and want to deploy it as a zip archive, make sure to test that reading from the zip still works. If you used paths with the wrong casing, reading it from a zip archive may not work.
Typical use case
The preferred way to create zip archives like this is to first prepare a local folder where you put all you configuration files, geodata and additional resource files. Make sure to use only relative paths in your configuration files when referring to geodata and resource files. Test and verify that the map configuration works as expected when loaded from the local folder.
Then use a zip program of your choice to create a zip archive of the local folder. A few restrictions apply when you create the zip, see the Limitations section below. The Tested zip software lists some common zip programs known to work for this purpose.
You should then verify that running from the zip file works just like running from the local folder, by loading the map using the virtual path to the configuration file.
If you want to include index files, raster cache files and similar in the zip you can do so, by specifying the index paths with relative paths inside the folder tree that will be included in the zip. Load the configuration to generate the index files, before creating the zip archive, otherwise you will get errors trying to create the index files inside an existing zip archive.
If you don't want to include the index files, you can specify index paths as absolute paths (like C:\temp or /tmp), or specify relative paths starting with a number of '..' components. If you use '..' from the root folder of the zip, this is treated like you "step out" from the zip archive and into the real, physical file system.
The Catalog mechanism supports zip archives
Using zip archives to deploy map configurations and data also works well together with the Catalog mechanism:
Any complete local catalog can be zipped to an archive and deployed as a single file. The receiving application only needs to specify the virtual path to the catalog inside the zip archive.
When a local catalog is loaded, the Catalog mechanism will not only look for .px files defining the service, but also for files with extension .carmentacatalog in the root of the catalog. Each such file is loaded as a separate catalog, and then merged into the parent catalog. Using CatalogLoadLayerSet and CatalogLoadDataSetSet, services in the parent folder can reference services from the "sub-catalogs" in the zip archives, and vice versa.
You can not zip catalogs containing other zip catalogs. Only the zips in the unzipped folder root folder will be used.
Limitations
General limitations
Carmenta Engine can only read from zip archives, not write to them.
Reading from zip files inside other zip files is not supported.
Components not supporting virtual paths
Almost all components in Carmenta Engine supports reading from virtual paths. The only exceptions are:
EcwDataSet supports virtual paths when reading ECW data, but not when reading JPEG 2000 data. If you need to put JPEG 2000 data inside a zip archive, use Jpeg2000DataSet to read it.
Restriction on the zip archives
Not all variants of zip files can be read. The following restrictions apply:
Compression is not supported. Zip archives must use the Store method when writing the files. The Deflate compression method (which is the default in many zip programs) is not supported, not even with compression level 0.
Encryption is not supported.
File names should be UTF-8 encoded to be portable. Some zip programs store file names using the active code page of the system, these may not be possible to interpret correctly on another system.
There are no restrictions on file sizes, not for the included files or the archives themselves. Large zip files may be split into multiple smaller pieces. The following schemes are supported:
Standard zip split/spanned archives. The last part is usually called <name>.zip, this is the file you should specify in a virtual path. The other files must be named <name>.z01, <name>.z02 etc. Carmenta Engine will automatically recognize this and open and read the other parts as needed.
The 7-zip program can also split archives but uses a different scheme. It creates the archive as if it were a single zip file, and then simply chops it up into pieces. The parts are called <name>.zip.001, <name>.zip.002 etc. If the zip file name specified in a virtual path has a .001 extension, Carmenta Engine assumes this scheme is used and automatically loads remaining files.
Tested zip software
The following zip programs have been tested and are known generate zip files that Carmenta Engine can read:
Name | Notes |
|---|---|
WinZip | Specify compression methods Store. |
7-Zip | Specify compression methods Store. By default, 7-Zip stores files names in the system code page; specify parameter cu to make it write UTF-8 file names. |
InfoZIP | Only command line interface tested. Pass -0 to store files uncompressed. Use -s to create split archives. On Linux, UTF-8 is default, but on Windows you may need to specify additional parameters to use UTF-8. |
Python | The 'zipfile' Python module can be used to create zip archives that Carmenta Engine can read. Specify the zipfile.ZIPSTORED flag when you create the archive. |
C#/PowerShell + SharpZipLib | The SharpZipLib module can be used to create zip archives from .NET applications and PowerShell scripts. |
Note that Carmenta Engine will not be able to read zip archives created with the PowerShell Compress-Archive cmdlet.
Embedding file contents into the path or property string
Another type of virtual paths can be used to embed the actual contents of a file directly into a path or filename property, so you don't need to deploy a separate file.
Note that this only works for "standalone" files, like a single image file. If the file references some other file, for instance a 3D model with a texture store in a separate file, the data can not be embedded in a string like this.
Syntax
The standard 'data' URL scheme is used to embed the data. The syntax is:
data:<mediatype>[;base64],<data>
The <mediatype> can be used to specify the type of the data, for instance image/png. This is used by Carmenta Engine whenever the extension of a file is needed, for instance when specifying the image file used by a raster symbol.
But only a limited set of media types are recognized. In some cases you may need to explicitly specify a "file extension" to be used, by appending an ext parameter to the mediatype, like in data:application/octet-stream;ext=ttf;,<TrueType font data>. In this case the actual media type is not used.
If the ;base64 parameter is specified, the data that follows should be encoded as a Base-64 string. If it is not specified, the data should use standard URL encoding according to RFC 2396.
For details on the data URL scheme, see RFC 2397.