MapGuide RFC 2 - Unmanaged Data APIs

This page contains an change request (RFC) for MapGuide Open Source.  
More MapGuide RFCs can be found on the MapGuide RFCs page.

Status

RFC Template Version	(1.0)
Submission Date	1 November 2006
Last Modified	Tonyfang 23:50, 6 December 2006 (CET)
Author	Tony Fang
RFC Status	draft
Implementation Status	pending
Proposed Release Target	1.2
Assigned PSC guide(s)	(when determined)
Voting History Date +1 +0 -0 -1 Nov 17, 2006

Overview

Managed data is data contained inside the server's data repository. Unmanaged data is any data outside of the server's data repository.

We want to add server/web functionality and APIs which will allow Studio to create feature sources from unmanaged data. To accomplish this, directories accessible by the server machine can be specified as unmanaged data directories. APIs will be added which will enumerate the available unmanaged data files from the specified unmanaged data directories.

WebStudio can also make use of these APIs to create unmanaged feature sources.

Motivation

When Studio is used to create a feature source (e.g. using Load Procedure), the data files (e.g. sdf, shp, raster, dwf, odbc) are uploaded to the server's repository (managed data). This is not an issue when the data files are small. If Studio and Server are on the same machine, this uploading is an unnecessary duplication of the data file. If the data file is really large, the copying is slow and it consumes unnecessary hard-drive space.

Currently, a feature source which references unmanaged data can be created. But it must be done manually. The exact path to the data file on the server machine must be known. Then the Feature Source XML must be manually edited and uploaded to the server via the web tier. This is a cumbersome process.

We want to automate this workaround in Studio. To do this, additional APIs and functionality will be added to the server/web. Directories accessible by the server machine can be specified as unmanaged data directories. APIs will be added which will enumerate the available unmanaged data files. Thus direct access to the server machine's file system will no longer be required. Creation of unmanaged data feature sources can now be created remotely.

The API will only enumerate data files contained in the directories and subdirectories specified by the server. For security reasons we do not want to allow the API to return data files in directories which are not specified. For windows, specified directories can be physical drives or network paths. For linux, it can be any accessible path.

Proposed Solution

The proposed changes are open for discussion.

The server's unmanaged data directories will be specified using the serverconfig.ini. A new section defining directory mappings will be used to specify the unmanaged data directories.

For windows, it may look like this:

[Unmanaged Data Mappings]
SomeSdfFiles = c:\mydata\sdf
Some Shp Files = d:\otherdata\shp
BigArseImages = \\some_other_machine\data\images
Some 很大中文 DwfFiles = c:\mydata\很大中文Dwf
大きい　ｓｄｆ = c:\mydata\bigsdf

For linux, it may look like this:

[Unmanaged Data Mappings]
SomeSdfFiles = /usr/mydata/sdf
Some Shp Files = /usr/otherdata/shp
BigArseImages = /mnt/some_other_machine/data/images
Some 很大中文 DwfFiles = /usr/mydata/很大中文Dwf
大きい　ｓｄｆ = /usr/mydata/bigsdf

Unicode characters are supported in the mapping names. Spaces in the mappings are also supported. Square brackets [ and ] are not allowed. They are reserved characters for the serverconfig.ini section titles.

We will add API methods to return the unmanaged data mappings, set the mappings, and also verify that the mappings are valid and accessible.

We will add an API to enumerate the unmanaged data files available on the server machine. The files will be prefaced with their mapping names. You may enumerate all files from all the available drive mappings, or you may select a single drive mapping.

A returned list from all available drive mappings may look like this:

[SomeSdfFiles]world.sdf
[Some Shp Files]ecuador.shp
[Some 很大中文 DwfFiles]large.dwf
[Some 很大中文 DwfFiles]subdir/Big.dwf
[大きい　ｓｄｆ]reallybig.sdf

The server will not be locked down such that it is restricted to those files in the Unmanaged Data Mappings. In other words, this means that a feature source can still reference a file that is not in one of the subdirectories specified by the mappings. The reason we don't have to restrict to those subdirectories is because we can use the operating system's or network's built-in security to limit access to files. It will be dependent upon the user to allow access to the files in the Unmanaged Data Mappings, otherwise they won't be very useful. Thus manually created unmanaged features sources and those created by webstudio will continue to work.

Technical

The folllowing API functions will be added to the Resource Service:

virtual MgStringCollection * MgResourceService::EnumerateUnmanagedData(STRING dataTypeFilter, INT32 depth);

Enumerates the available unmanaged data from the server.

Parameters:

dataTypeFilter is a filter on the returned data. E.g. SHP, SDF, DWF, RASTER, FOLDER. Empty string will return all data types. FOLDER returns a list of subfolders.

depth is how many levels of subfolders it should search for data. 0 to search all subfolders, 1 to search only the top-level, 2 to search down one subfolder, etc.

Returns:

Returns an MgStringCollection containing the available unmanaged data files.

virtual MgStringCollection * MgResourceService::EnumerateUnmanagedDataMappings();

Enumerates the unmanaged data mappings.

Parameters:

None.

Returns:

Returns an MgStringCollection containing the mappings.

virtual void MgResourceService::SetUnmanagedDataMapping(STRING mapping, STRING path);

Sets the unmanaged data mapping. If the mapping does not exist, it adds it. If it does exist, it will set it to the new path.

Parameters:

mapping is the mapping name.

path is the mapped path.

Returns:

Returns nothing.

Implications

TBD

Test Plan

TBD

Funding/Resources

Autodesk to supply the resources to make the required changes.