GSoC 2017 - 3D OSM Plugin API for ESA-NASA Web World Wind

Introduction
The goal of this project is to create a plugin API to display OpenStreetMap (OSM) data on NASA Web World Wind (https://github.com/NASAWorldWind/WebWorldWind) virtual globe in three dimensions. The API fetches the OSM data in real time based on a bounding box, or uses a local file or the data itself in GeoJSON format. The project focuses on the performance of the visualization of 3D OSM buildings, and for this purpose creates triangle meshes based on the footprint available in OSM database. Height to the buildings can be assigned using the OSM database, a property in a GeoJSON file or an arbitrary value. The API can also create a heatmap based on the heights of the buildings. The API also offers caching and sectorizing (tiling) to improve the performance. (https://summerofcode.withgoogle.com/projects/#4767034027016192) The project can be found at https://github.com/kilsedar/3dosm.

Tentative Schedule
ɤ In case time remains, existing tiling procedures will be investigated and the most suitable one will be implemented.

Week 1

 * 1) What did I complete this week? - Configured the environment for unit testing. - Implemented OSMLayer and OSMBuildingLayer classes, which fetch the OSM data given a bounding box, render the data in 3D, zoom on the data.
 * 2) What am I going to achieve next week? - I will do research on how tiling can be implemented for OSM data in NASA Web World Wind and start its implementation.
 * 3) Is there any blocking issue? - There is no blocking issue.

Week 2

 * 1) What did I complete this week? - Fixed some bugs of the OSMBuildingLayer. - To improve the performance, instead of tiling my mentor suggested me to use triangle meshes. I started to triangulate the 3D polygons, by creating a subclass of GeoJSONParser. I first triangulated the lateral surfaces and started to read how to make polygon triangulation including the concave ones and the ones with holes. Found https://github.com/mapbox/earcut.
 * 2) What am I going to achieve next week? - If my mentor approves I will triangulate also the top of the polygons using the ear cut algorithm. - I will assess the performance improvements gained using triangulation. - I will improve the documentation.
 * 3) Is there any blocking issue? - There is no blocking issue.

Week 3

 * 1) What did I complete this week? - Managed to triangulate the top surfaces of polygons. - Made performance tests to evaluate the effect of rendering. Found out that it is always better to use triangulation instead of extrusion. - Added documentation.
 * 2) What am I going to achieve next week? - There are currently two problems, first in the triangulation, second in the type of OSM data I am getting, I will solve those. - I will check JSDoc to see how the documentation works, and improve the documentation. - I will assess the feasibility of getting the real heights of the OSM buildings using DSM, and search for possible ways to achieve that.
 * 3) Is there any blocking issue? - There is no blocking issue.

Week 4

 * 1) What did I complete this week? - Added triangulation also for MultiPolygons and changed the Overpass query to exclude Points and LineStrings (solved the bugs of the past week). - Learned the syntax of JSDoc and added documentation. - Started to learn about the ways to extract NDSM (Normalized Digital Surface Model) from the ASC files that I have for the first & last returns, and started to think about a possible workflow to match this data with GeoJSON OSM data.
 * 2) What am I going to achieve next week? - I will go on the with the documentation. - I will work on the NDSM and GeoJSON matching in JavaScript.
 * 3) Is there any blocking issue? - There is no blocking issue.

Week 5

 * 1) What did I complete this week? - I came to understand that it is not difficult to extract building heights from the DSM and DTM data I have, but this process cannot be integrated with the Web World Wind (WWW) API. I talked with my supervisor at the university and I decided to do that outside GSoC with GRASS and only display the results using WWW API (I will update the OSM GeoJSON for Milan with the height information and display this static data for showcasing). - Completed the documentation for the current state of the project. - Added code to apply building heights if the information exists in OSM. I also added coloring based on the building heights (heatmap).
 * 2) What am I going to achieve next week? - At this point in the project I don't know the next steps.
 * 3) Is there any blocking issue? - There is no blocking issue.

Week 6

 * 1) What did I complete this week? - I created a new class for setting building heights and colors. - Started to cache the OSM GeoJSON.
 * 2) What am I going to achieve next week? - I will add unit tests. - I will look into tiling.
 * 3) Is there any blocking issue? - There is no blocking issue.

Week 7

 * 1) What did I complete this week? - Added a new class for triangulating OSM polygons (https://github.com/kilsedar/3dosm/commit/53d157c5d8f0bd7db5858c35ffbb2a6b3cfc0c9f). - Enabled adding the OSMBuildingLayer by specifying a bounding box or a local file directory (https://github.com/kilsedar/3dosm/commit/269af231af5c04c70de1f6fdec5da28088040333, https://github.com/kilsedar/3dosm/commit/1bf9a93b3f150f854e6d00001d822deffe307ccc). - Added helper functions for tiling (https://github.com/kilsedar/3dosm/commit/cb938c2e14d95185b5ab679c6327eb46f31c2884).
 * 2) What am I going to achieve next week? - I will finish the implementation of tiling if the data is fetched from OSM. This will involve checking the visibility of sectors in case of panning, zooming, etc. and starting using the cache actively. If the input to construct the OSMBuildingLayer is a local GeoJSON file a new approach must be followed.
 * 3) Is there any blocking issue? - There is no blocking issue.

Week 8

 * 1) What did I complete this week? - Improved the way GeoJSON file is used (https://github.com/kilsedar/3dosm/commit/9b5dcc9a58b4b810171eca4134b4d2480ed3d1dc). - Started to use gestures, sectors and cache to improve the rendering performance (https://github.com/kilsedar/3dosm/commit/1d8cce7cad3f72df477a8e6a44718a5f491064ab).
 * 2) What am I going to achieve next week? - The use of gestures, sectors and cache gave me good hints on improving the rendering performance, however making many requests to OSM might result in error "too many requests". I should make one request and create sectors out of the data. This way I can sectorize also the local GeoJSON file. I will try to use https://github.com/mapbox/geojson-vt or implement a simpler version of its logic this week.
 * 3) Is there any blocking issue? - There is no blocking issue.

Week 9

 * 1) What did I complete this week? - This week I did not work on sectorizing the GeoJSON. Instead I improved the example (https://kilsedar.github.io/3dosm/example/) I have. This required *creating a new branch "gh-pages", *learning how Grunt works and *creating the user interface (https://github.com/kilsedar/3dosm/commit/18181d8fe74eea302e4197aa0a74dc01e3a57597). - Added license (https://github.com/kilsedar/3dosm/commit/84cafc1b5e6e31d897389a959fa59394ce04f5a8).
 * 2) What am I going to achieve next week? - I will make the example I have fully functional, which will require to extend the OSMLayer class to add layers based on their geometry using any OSM tag.  The week after I will go back on sectorizing the GeoJSON.
 * 3) Is there any blocking issue? - There is no blocking issue.

Week 10

 * 1) What did I complete this week? - I completed the example I started last week. I am currently working in the "gh-pages" branch (https://github.com/kilsedar/3dosm/tree/gh-pages). The application can be found at https://kilsedar.github.io/3dosm/example/. The example required me to work on the OSMLayer class to be able visualize all OSM data once their tag and type(s) are provided. The important relevant commits are: https://github.com/kilsedar/3dosm/commit/9e13bfd856b38d19df716925288206232f9ac8f2, https://github.com/kilsedar/3dosm/commit/f2286c8a4d1a9440e36d4b97bd48622261e7a087, https://github.com/kilsedar/3dosm/commit/f62ac13a2b8edf29037d01eb5a5f7492d0f16391.
 * 2) What am I going to achieve next week? - I will start working on sectorizing the GeoJSON data and work on the OSMTBuildingLayer.
 * 3) Is there any blocking issue? - There is no blocking issue.

Week 11

 * 1) What did I complete this week? - I made the example I have mobile-friendly (http://osm.eoapps.eu/ & http://osm.eoapps.eu/application/). The commits related to that can be found on the gh-pages branch (https://github.com/kilsedar/3dosm/commits/gh-pages). - Made tests to understand better the issue related to GestureRecognizers (https://github.com/NASAWorldWind/WebWorldWind/issues/185). - Integrated geojson-vt (https://github.com/mapbox/geojson-vt) to the project (https://github.com/kilsedar/3dosm/commit/99145152504ee1e9816cd7f08594c93af07804a6).
 * 2) What am I going to achieve next week? - I will go on with the vector tiling API, geojson-vt, and try to find ways to use it with NASA Web World Wind.
 * 3) Is there any blocking issue? - There is no blocking issue.

Week 12

 * 1) What did I complete this week? - Did a basic implementation of tiling & caching GeoJSON. Tested only for Polygon and MultiPolygon geometry types. Relevant commit can be found at https://github.com/kilsedar/3dosm/commit/f1872fa8df2d212ee719d03b471b26bbba6f9751, especially in the OSMTBuildingLayer.js file.
 * 2) What am I going to achieve next week? - I will wrap up the work and get ready for final submission.
 * 3) Is there any blocking issue? - There is no blocking issue.

ɤ Final report can be found at https://lists.osgeo.org/pipermail/soc/2017-August/003844.html.

Student's Biography
Candan Eylül Kilsedar studied Computer Science and Engineering in BSc and MSc. Currently she is working in the Department of Civil and Environmental Engineering, in GEOlab, Politecnico di Milano, Italy as a research fellow and PhD student. GEOlab focuses highly on Free and Open Source Software (FOSS) and as a result she has experience in using FOSS. She wants to take the opportunity of joining GSoC 2017 to extend her knowledge in FOSS development. Her mentors for this project are Patrick Hogan, the project manager of the NASA World Wind and Jakub Balhar, software Engineer at Gisat. Eventually she wants to develop & use FOSS to build better communication and collaboration inside and among cities. She is interested in Web technologies and geographic data visualization techniques. Her GitHub repositories can be found at https://github.com/kilsedar.