Difference between revisions of "Edu Data Package North Carolina"

From OSGeo
Jump to navigation Jump to search
(+metadata and original data)
 
(36 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This page is an ideas collection for a new educational/training data set.
+
This page includes information about a new sample data set provided for software testing and development of educational materials (tutorials, workshops, courses).
We want to create a new data set similar to the [http://grass.itc.it/download/data6.php GRASS Spearfish dataset]. This page is being worked on by [[User:Helena|Helena Mitasova]] and [[User:Neteler|Markus Neteler]].
+
The [http://www.grassbook.org/data_menu3rd.php modern North Carolina based data set] aims to replace the [http://grass.itc.it/download/data6.php GRASS Spearfish dataset]. This page is being worked on by [[User:Helena|Helena Mitasova]] and [[User:Neteler|Markus Neteler]].
  
 +
    '''New Image Gallery: http://www.grassbook.org/gallery/'''
  
== NC related Data Sources ==
+
== North Carolina related Data Sources ==
  
 
* Check all possible data layers and status of mapping here: http://nc.gisinventory.net/
 
* Check all possible data layers and status of mapping here: http://nc.gisinventory.net/
Line 9: Line 10:
 
* The above viewer links to USGS seamless data server: http://seamless.usgs.gov/
 
* The above viewer links to USGS seamless data server: http://seamless.usgs.gov/
 
* as well as to the NC One map http://gisdata.usgs.net/website/NC_OneMap/viewer.asp
 
* as well as to the NC One map http://gisdata.usgs.net/website/NC_OneMap/viewer.asp
* NC One map data (in SP meter) can also be dowloaded from ftp site from here http://www.nconemap.com/data.html
+
* Most of NC One map state gov data files (in SP meter) can be directly downloaded from here http://www.nconemap.com/
*Wake county data (in SP feet!) are here http://lnweb02.co.wake.nc.us/gis/gismaps.nsf/WebGlobals%21OpenView&ViewCat=shp
+
*Wake county data files (in SP feet!) are here http://www.wakegov.com/gis/gisdigitaldata.htm
 
* General list: [[Geodata Packaging Working_Group#Public_Datasets_for_Possible_Mirroring_or_Distribution]]
 
* General list: [[Geodata Packaging Working_Group#Public_Datasets_for_Possible_Mirroring_or_Distribution]]
  
Line 17: Line 18:
 
** Imagery:http://glcfapp.umiacs.umd.edu:8080/esdi/index.jsp
 
** Imagery:http://glcfapp.umiacs.umd.edu:8080/esdi/index.jsp
 
** Coastal lidar: http://www.csc.noaa.gov/crs/tcm/about_ldart.html
 
** Coastal lidar: http://www.csc.noaa.gov/crs/tcm/about_ldart.html
** Flood maps:http://www.ncfloodmaps.com/default_swf.asp
+
** Flood maps:http://www.ncfloodmaps.com/default_htm.asp
 
** Real-time streamflow data (these should be available through the national map too): http://water.usgs.gov/waterwatch/
 
** Real-time streamflow data (these should be available through the national map too): http://water.usgs.gov/waterwatch/
 
** Climate data: http://www.nc-climate.ncsu.edu/cronos/map.php
 
** Climate data: http://www.nc-climate.ncsu.edu/cronos/map.php
** USDA should have the soil data or http://www.ncgc.nrcs.usda.gov/products/datasets/ssurgo/
+
** USDA soil attribute data or http://www.ncgc.nrcs.usda.gov/products/datasets/ssurgo
** County data (parcels, building footprints, etc.) can be viewed here: http://imaps.co.wake.nc.us/imaps/main.htm?msize=525 (no downloading - that should be handled by the National map?)
 
  
* Additional links are here: http://skagit.meas.ncsu.edu/~helena/classwork/hon297webgis.html (sorry some are obsolete)
+
== New sample GIS data set from North Carolina ==
  
== Proposed new data set North Carolina ==
+
Area in and around Wake county (eastern section of the Triangle) has public data for all basic data layers that were in Spearfish and there is an easy access to USGS data, local county data, EPA, State Climate Office, NCFlood maps and others through the above listed web sites. There is a big enough city, but also some rural areas that still have some agriculture with rolling topography. SE of it is Johnston county that has FOSS-based county GIS, running Mapserver and PostGIS on-line.
  
Area in and around Wake county (eastern section of the Triangle) has public data for all basic data layers that were in Spearfish and there is an easy access to USGS data, local county data, EPA, State Climate Office, NCFlood maps and others through the above listed web sites. There is a big enough city, but also some rural areas that still have some agriculture with rolling topography . SE of it is Johnston county that has FOSS-based county GIS, running mapserver and PostGIS on-line.
+
Core region (15km x 13.5km, city + rural) in stp[m]:
 
 
Suggested region (15km x 13.5km city + rural) in stp[m]:
 
 
*north: 228500 south: 215000 west:  630000 east: 645000
 
*north: 228500 south: 215000 west:  630000 east: 645000
 
*nsres: 30 ewres: 30 rows: 450 cols: 500
 
*nsres: 30 ewres: 30 rows: 450 cols: 500
Line 37: Line 35:
 
*long: -78.60889171 lat: 35.68750730 (south/east corner)
 
*long: -78.60889171 lat: 35.68750730 (south/east corner)
 
*long: -78.77462049 lat: 35.68792712 (south/west corner)
 
*long: -78.77462049 lat: 35.68792712 (south/west corner)
 
previous suggestion (larger region - that would be too much data)
 
* LL(SW) corner: -78.7854 (78:47:06), 35.5897 (35:35:23)
 
* UR(NE) corner: -78.4466 (78:26:46), 35.8689 (35:52:08)
 
  
 
For DEFAULT region we use entire NC (slightly beyond) to accomodate various subregions for thematic data sets (e.g. coast, mountains, urban)
 
For DEFAULT region we use entire NC (slightly beyond) to accomodate various subregions for thematic data sets (e.g. coast, mountains, urban)
Line 49: Line 43:
 
Coordinate systems:
 
Coordinate systems:
 
*Geographic Coordinate System in decimal degrees WGS84 horizontal datum (close to NAD83 but there are submeter differences). This is EPSG:4326 SRS/CRS that all the OGC compliant software tends to use
 
*Geographic Coordinate System in decimal degrees WGS84 horizontal datum (close to NAD83 but there are submeter differences). This is EPSG:4326 SRS/CRS that all the OGC compliant software tends to use
*NC State Plane Coordinate System, NAD 83 datum, units of meters
+
*NC State Plane Coordinate System, NAD 83 datum, units of meters: EPSG 3358
 
*lot of local data and NCFlood mapping lidar come in US Survey feet (new EPSG 3404, the old wrong 3359 is deprecated)
 
*lot of local data and NCFlood mapping lidar come in US Survey feet (new EPSG 3404, the old wrong 3359 is deprecated)
 
*xy for scanned historical maps and other non-georeferenced data
 
*xy for scanned historical maps and other non-georeferenced data
Line 55: Line 49:
 
In addition to GRASS databases, the data will be provided in their original formats (shapes files, Arcgrid, tiff/tfw, etc.) and links to sites where the data can be downloaded will be provided too.
 
In addition to GRASS databases, the data will be provided in their original formats (shapes files, Arcgrid, tiff/tfw, etc.) and links to sites where the data can be downloaded will be provided too.
  
 +
The data set is packaged as a baseline dataset and thematic sets (aka GRASS Mapsets concept).
  
We suggest to package the data set in thematic sets (aka GRASS Mapsets concept).
+
The below list needs update, Ramona GIS inventory terminology http://nc.gisinventory.net/
 
+
is used in some sections.
The below list needs to be further organized into thematic sets, Ramona GIS inventory terminology http://nc.gisinventory.net/
 
is used in the updated sections.
 
  
 
=== RASTER ===
 
=== RASTER ===
====Status: Framework data layers (minimum needed to replace Spearfish) - put into PERMANENT====
+
====Baseline raster maps in PERMANENT====
  
*elevation: done for 30m NED, lidar for rural subarea (Lake Wheeler farm LW)
+
*elevation: 30m, 10m lidar-based NED, 30m SRTM-V1, 6m NCFlood DEM, 1m lidar raster for rural subarea (Lake Wheeler farm LW)
*landcover: LU 1992, 1996, needs labels (see category.txt)
+
*landcover: LULC 1996 (1992, 1998 and 2001 will go into thematic mapset), landuse 1m for rural area
*orthoimagery: 2 tiles for LW
+
*orthoimagery: 2001, 1m resolution tile for rural area (additional years and higher resolution will go into imagery mapset)
 +
*landsat7
  
all other rasters can be derived from elevation and vector data: put them into dedicated mapsets (this makes packaging in pieces easier - like data plugins)
+
rasters derived from elevation and vector data:  
  
 
*derived from DEM:
 
*derived from DEM:
**slope, aspect
+
**slope, aspect, contributing area, stream network
**contributing area, stream network
 
**erosion
 
  
 
*derived from vector data
 
*derived from vector data
**geodetic points (to replace bugsites)
+
**counties, zipcodes, soils, geology (needs labels), major roads, hydrography - lakes
**fields (replace by parcels?), soils, geology (done, needs labels), landuse
 
**rairoads, roads
 
**hydrography (note Ramona does not use terms streams, lakes, rivers, should we?)
 
  
====Additional data that can go into the basic/general data set====
+
====Additional data that can go into the baseline, imagery, terrain/hydrology, or history mapsets====
 
*Original data:
 
*Original data:
** elevation (NC One map USGS NED, lidar based)
+
** landuse/landcover (NC One map EPA 1998, USGS 2001)
** landuse/landcover (NC One map EPA 1998, check for newer)
+
** aerial images 6inch resolution orthos at http://www.lib.ncsu.edu/gis/raleigh.html, get also (raw stereo pair + orthophotos from DOT?)
** aerial images (raw stereo pair + orthophotos)
+
** [http://mpa.itc.it/markus/grassbook3/ raw geocoded LANDSAT7 satellite data], derived vegetation indices and land surface temperatures
** raw geocoded satellite data, derived vegetation indices and land surface temperatures
 
 
** raw SRTM V2
 
** raw SRTM V2
** DRG topo sheets
+
** DRG topo sheets (available in ncexternal - Mrsid format)
** Historical maps, e.g. [http://memory.loc.gov/cgi-bin/map_item.pl?data=/home/www/data/gmd/gmd388/g3881/g3881e/ct001099.jp2&style=gnrlmap&itemLink=r?ammem/gmd:@filreq(@field(NUMBER+@band(g3881e+ct001099))+@field(COLLID+gnrlmap))&title=Map%20of%20Virginia%20and%20West%20Va.,%20North%20%26%20part%20of%20South%20Carolina,%20Maryland%20%26%20Delaware,%20showing%20the%20situation%20in%20these%20states%20of%20schools%20taught%20by%20graduates%20of%20the%20Hampton%20Normal%20%26%20Agricultural%20Institute%20from%201871%20to%201876. Schools situation North Carolina 1871 to 1876]
+
** Historical maps, http://www.lib.ncsu.edu/gis/historictopos.html, http://www.ncsu.edu/facilities/campus_maps/historic_maps/index.htm, or [http://memory.loc.gov/cgi-bin/map_item.pl?data=/home/www/data/gmd/gmd388/g3881/g3881e/ct001099.jp2&style=gnrlmap&itemLink=r?ammem/gmd:@filreq(@field(NUMBER+@band(g3881e+ct001099))+@field(COLLID+gnrlmap))&title=Map%20of%20Virginia%20and%20West%20Va.,%20North%20%26%20part%20of%20South%20Carolina,%20Maryland%20%26%20Delaware,%20showing%20the%20situation%20in%20these%20states%20of%20schools%20taught%20by%20graduates%20of%20the%20Hampton%20Normal%20%26%20Agricultural%20Institute%20from%201871%20to%201876. Schools situation North Carolina 1871 to 1876]
 
** HydroSHEDS data (check if resolution is high enough) [http://www.worldwildlife.org/freshwater/hydrosheds.cfm URL], but only non-commercial use
 
** HydroSHEDS data (check if resolution is high enough) [http://www.worldwildlife.org/freshwater/hydrosheds.cfm URL], but only non-commercial use
*Derived data (from vector maps, maybe not needed? - this should go into specialized packages - e.eg. terrain analysis, image processing):
+
 
** DEM from Lidar vector points
+
*Derived data (generated when working with the book):
** complete set of topographic parameters (slope, aspect, different types of curvature)
+
** DEM from Lidar vector points, complete set of topographic parameters (slope, aspect, different types of curvature), streams from DEM, landforms, derived satellite indices, hydrologically conditioned SRTM V2
** streams from DEM
 
** landforms
 
** derived satellite indices
 
** hydrologically conditioned SRTM V2
 
  
 
=== VECTOR ===
 
=== VECTOR ===
====Status: Framework data layers (minimum needed to replace Spearfish) - put into PERMANENT====
+
====Baseline vector maps in PERMANENT====
* point data:
+
* point data: schools, climate stations, geodetic control points, bridges, firestations, hospitals, USGS stream gages, bus stops, points of interest, lidar point clouds (bare earth, multiple return)
**schools (to replace bugsites and archsites): done
+
 
**climate stations
+
* line data: hydrography (streams), major roads, streets, railroads, bus routes, contours, planimetry (one tile)
**geodetic control points: done
+
 
**bridges: done
+
* polygon data: urban areas, counties, zip codes, census blocks, soils, geology
**geographic place names
+
 
* line data
+
* 3D buildings - 84 buildings for NCSU campus available from NCSU DELTA (http://sketchup.google.com/3dwarehouse/cldetails?mid=fda4a8505b42649c30e65e54bef87cc6&start=60)
** hydrography (streams)
 
** roads,  
 
** railroads: done
 
** contours
 
* polygon data
 
** boundaries: cities, counties: done
 
** zip codes
 
** parcels (to replace fields)
 
** soils: done
 
** building footprints
 
  
 
====Additional data for general and specialized data sets====
 
====Additional data for general and specialized data sets====
*needs cleanup, can be nicely organized using Ramona GIS inventory that also shows which data layers are available for our area
+
*organize using Ramona GIS inventory that also shows which data layers are available for our area
** Lidar point clouds for DSM/DEM
 
 
** climate stations: temp., precip., frost, wind, cloud cover, ... (from [http://www.nc-climate.ncsu.edu/cronos/map.php State Climatology Office])
 
** climate stations: temp., precip., frost, wind, cloud cover, ... (from [http://www.nc-climate.ncsu.edu/cronos/map.php State Climatology Office])
** Air quality data ([http://www.epa.gov/empact/region4.htm#north EPA EMPACT])
+
** Air quality data ([http://www.epa.gov/empact/region4.htm#north EPA EMPACT]), http://www.scorecard.org
 
** human and animal diseases incident data
 
** human and animal diseases incident data
 
** wildlife data
 
** wildlife data
Line 133: Line 106:
 
*** flood data ([http://www.ncfloodmaps.com NC digital flood maps])
 
*** flood data ([http://www.ncfloodmaps.com NC digital flood maps])
 
** placenames (gazetteer data) ([http://www.census.gov/geo/www/gazetteer/gazette.html US Gazetteer])
 
** placenames (gazetteer data) ([http://www.census.gov/geo/www/gazetteer/gazette.html US Gazetteer])
** pollution and contamination data
+
** pollution and contamination data, hydrogeochemical and sediment data http://pubs.usgs.gov/of/1997/ofr-97-0492/quad/q_ralegh.htm, http://tin.er.usgs.gov/geochem/
 
** results of elections
 
** results of elections
** soils ([http://soildatamart.nrcs.usda.gov/Download.aspx?Survey=NC183&UseState=NC Wake County Soil data])
+
** complete soils attributes ([http://soildatamart.nrcs.usda.gov/Download.aspx?Survey=NC183&UseState=NC Wake County Soil data])
* line data:
+
** additional geology http://wfs.enr.state.nc.us/NCGeologicMaps/
** railroads
 
** roads (TIGER and simplified)
 
** streams
 
* area data:
 
** countyboundaries
 
** geology
 
** landowners (should cover owners, fields)
 
** landuse/landcover
 
** parcels (land owners, fields)
 
 
** quadrangles
 
** quadrangles
** soils
 
** urban areas
 
** US Census 2000 maps
 
 
** voting districts
 
** voting districts
 
** zoning districts
 
** zoning districts
** ZIP codes
+
** traffic (see wral.com, http://www.wakegov.com/news/22107.htm
 +
** 3D downtown Raleigh http://www.raleighnc.gov/portal/server.pt/gateway/PTARGS_0_2_306_203_0_43/http%3B/pt03/DIG_Web_Content/project/public/livable_streets/Drawing_board.htm
  
 
== Packaging issues ==
 
== Packaging issues ==
  
* convert feet to meters if needed
+
* File Naming convention (approximately) - feature_origin_time_extent_resolution for raster data or feature_origin_time_extent_type for vector data
* File Naming convention
+
* Metadata management (most data include standard FGDC metadata, store in history file) ,
* Metadata management (most data include standard FGDC metadata)  
+
* provided as GRASS LOCATION, also in common GIS formats ? if yes, which?
* provide in common GIS formats as well as GRASS location
+
... see [[Geodata Packaging Working_Group#Tasks]].
 +
 
 +
== Dataset Download ==
 +
 
 +
First test versions are available now. Some changes of map names are expected, FGDC metadata and full description will be added, but the data set can already be used for testing:
  
... see [[Geodata Packaging Working_Group#Tasks]].
+
* Ready-to-use [http://www.grassbook.org/data_menu3rd.php GRASS location] (see there for list of available maps)
 +
* [http://www.grassbook.org/ncexternal/index.html Original datasets]
 +
* [http://skagit.meas.ncsu.edu/~helena/grasswork/grassbookdat07/ncexternal/ More original datasets] (yet to be ordered)
 +
* [http://www.grassbook.org/ncexternal/nc_datalist.html Table with the official data sources and metadata]
 +
 
 +
== Data set license ==
 +
 
 +
This data set is licensed under [http://creativecommons.org/licenses/by-sa/3.0/ Creative Commons Attribution-ShareAlike (CC-BY-SA)]
  
 
== Additional proposed themes for data sets ==
 
== Additional proposed themes for data sets ==
Line 170: Line 143:
 
# Hydrology and earth surface processes
 
# Hydrology and earth surface processes
 
# Vegetation, land use (image processing, multitemporal)
 
# Vegetation, land use (image processing, multitemporal)
# Note, for example, the NCOne hydrography, has the old USGS streams but also the high resolution streams derived from the lidar data using the methodology developed for EDNA, so one can nicely demonstrate how the small scale (in cartographic sense) national data are inaccurate when one would try to use it a local scale (also how you would damage your DEM if you would have tried to use them for stream enforcement).
+
# Working with real-time data (traffic, wolfline buses, meteorological data, stream gauges, webcams, ...)
 +
 
 +
[[Category:Public Geospatial Data Committee]]

Latest revision as of 09:35, 11 January 2008

This page includes information about a new sample data set provided for software testing and development of educational materials (tutorials, workshops, courses). The modern North Carolina based data set aims to replace the GRASS Spearfish dataset. This page is being worked on by Helena Mitasova and Markus Neteler.

   New Image Gallery: http://www.grassbook.org/gallery/

North Carolina related Data Sources


New sample GIS data set from North Carolina

Area in and around Wake county (eastern section of the Triangle) has public data for all basic data layers that were in Spearfish and there is an easy access to USGS data, local county data, EPA, State Climate Office, NCFlood maps and others through the above listed web sites. There is a big enough city, but also some rural areas that still have some agriculture with rolling topography. SE of it is Johnston county that has FOSS-based county GIS, running Mapserver and PostGIS on-line.

Core region (15km x 13.5km, city + rural) in stp[m]:

  • north: 228500 south: 215000 west: 630000 east: 645000
  • nsres: 30 ewres: 30 rows: 450 cols: 500

in lat/long

  • long: -78.77428134 lat: 35.80960938 (north/west corner)
  • long: -78.60830318 lat: 35.80918894 (north/east corner)
  • long: -78.60889171 lat: 35.68750730 (south/east corner)
  • long: -78.77462049 lat: 35.68792712 (south/west corner)

For DEFAULT region we use entire NC (slightly beyond) to accomodate various subregions for thematic data sets (e.g. coast, mountains, urban)

  • LL(SW) corner: -85.0000, 33.0000
  • UR(NE) corner: -75.0000, 37.0000


Coordinate systems:

  • Geographic Coordinate System in decimal degrees WGS84 horizontal datum (close to NAD83 but there are submeter differences). This is EPSG:4326 SRS/CRS that all the OGC compliant software tends to use
  • NC State Plane Coordinate System, NAD 83 datum, units of meters: EPSG 3358
  • lot of local data and NCFlood mapping lidar come in US Survey feet (new EPSG 3404, the old wrong 3359 is deprecated)
  • xy for scanned historical maps and other non-georeferenced data

In addition to GRASS databases, the data will be provided in their original formats (shapes files, Arcgrid, tiff/tfw, etc.) and links to sites where the data can be downloaded will be provided too.

The data set is packaged as a baseline dataset and thematic sets (aka GRASS Mapsets concept).

The below list needs update, Ramona GIS inventory terminology http://nc.gisinventory.net/ is used in some sections.

RASTER

Baseline raster maps in PERMANENT

  • elevation: 30m, 10m lidar-based NED, 30m SRTM-V1, 6m NCFlood DEM, 1m lidar raster for rural subarea (Lake Wheeler farm LW)
  • landcover: LULC 1996 (1992, 1998 and 2001 will go into thematic mapset), landuse 1m for rural area
  • orthoimagery: 2001, 1m resolution tile for rural area (additional years and higher resolution will go into imagery mapset)
  • landsat7

rasters derived from elevation and vector data:

  • derived from DEM:
    • slope, aspect, contributing area, stream network
  • derived from vector data
    • counties, zipcodes, soils, geology (needs labels), major roads, hydrography - lakes

Additional data that can go into the baseline, imagery, terrain/hydrology, or history mapsets

  • Derived data (generated when working with the book):
    • DEM from Lidar vector points, complete set of topographic parameters (slope, aspect, different types of curvature), streams from DEM, landforms, derived satellite indices, hydrologically conditioned SRTM V2

VECTOR

Baseline vector maps in PERMANENT

  • point data: schools, climate stations, geodetic control points, bridges, firestations, hospitals, USGS stream gages, bus stops, points of interest, lidar point clouds (bare earth, multiple return)
  • line data: hydrography (streams), major roads, streets, railroads, bus routes, contours, planimetry (one tile)
  • polygon data: urban areas, counties, zip codes, census blocks, soils, geology

Additional data for general and specialized data sets

Packaging issues

  • File Naming convention (approximately) - feature_origin_time_extent_resolution for raster data or feature_origin_time_extent_type for vector data
  • Metadata management (most data include standard FGDC metadata, store in history file) ,
  • provided as GRASS LOCATION, also in common GIS formats ? if yes, which?

... see Geodata Packaging Working_Group#Tasks.

Dataset Download

First test versions are available now. Some changes of map names are expected, FGDC metadata and full description will be added, but the data set can already be used for testing:

Data set license

This data set is licensed under Creative Commons Attribution-ShareAlike (CC-BY-SA)

Additional proposed themes for data sets

  1. Coast
  2. Urban
  3. Atmospheric 3D
  4. Hydrology and earth surface processes
  5. Vegetation, land use (image processing, multitemporal)
  6. Working with real-time data (traffic, wolfline buses, meteorological data, stream gauges, webcams, ...)