View source for GSoC'21 Integrating 3D scene builder as WPS services within MapMint administration User Interface

== Introduction ==
=== Introduction ===
MapMint is a geographic information system (GIS) software on the Internet designed to facilitate the deployment of spatial data infrastructures (SDI). MapMint is for individuals and organizations wishing to manage and optimize SDI establishment and deployment of dynamic mapping applications. The application centralizes and simplifies several GIS and WebGIS functionality. The different levels of user rights divide tasks according to the public (system administrators, GIS specialists, GIS technicians, cartographers, webmasters, etc). The SDI comprises users, tools (MapMint web software and MapMint4ME Android app), and the data (currently, images and location ) recorded by the users. MapMint4ME  is an Android app for MapMint web-service which is built on top of the ZOO-Project. The ZOO-Project is an SDI manager which gives the users the capability of building their own map and web applications using Mapserver as the   Open Geospatial Consortium (OGC) services engine. MapMint allows multiple tasks relating to the implementation of an SDI from a modular and user-friendly administration interface.
The user can use MapMint, according to his rights:
* Import and store vector and raster GIS data
* Query database server and WMS / WFS external
* Publish geographic data in the form of WMS, WFS, and WMTS
* Treat, edit, and Data Sources style
* Compose and save maps in the form of projects (map files)
* Configure and generate mapping applications
* Configure and run a cartographic portal
* Visualize and share maps

===Project proposal===
My proposal for GSoC 2021 can be found at: [[https://drive.google.com/file/d/1v7fiWu5ZMIhbkc50Bu5oGjJZACBb_rjY/view?usp=sharing Proposal]].

=== Abstract ===
At present, the user can only visualize the 2D datasets and can even generate one of them. So, my major approach in this project would be to create a user interface as a WPS service within MapMint such that the user cannot only visualize the 3D dataset but can also build the same. This can be done either by either simply capturing via camera or importing drone-captured images. Further, I am planning to integrate the various functions as a toolbox to process the 3D datasets and 2D datasets as well. 
If time permits the various additional functionality such as enhancement, smoothing, sharpening, brightness control will also be implemented.

'''Link to Github repository:''' [[https://github.com/Sandeep10021/mapmint Link]]

== Timeline ==

=== 17th May - 7th June ===
'''Community bonding period''':
'''What I have done during this period?'''
* Introduced myself over the channel and shared my proposal over the mailing list for suggestions 
* Joined MapMint Slack Workspace created by mentors which we will be used as a mode of communication over the GSoC 2021 period
* Mentors set up initial video call for primary discussion and explained to me about my tasks during Community Bonding Period
* Created a wiki page for the project "Integrating 3D scene builder as WPS services within MapMint administration User Interface" 
* Updated wiki User page and added my personal information [[https://wiki.osgeo.org/wiki/User:SandeepSaurav Link]]
* Updated links on the wiki Google_Summer_of_Code_2021_Accepted page [[https://wiki.osgeo.org/wiki/Google_Summer_of_Code_2021_Accepted Link]]
* Discussed with the mentors their expectations over the GSoC 2021 project period
* Setting up MapMint Developing Environment on my personal system
* Gone through the documentation for MapMint web application
* Gone through the working of the application user interface and code
* Analyzed structure of MapMint web application and understand it thoroughly

''' 7th June - 12th July '''
=== Coding Phase-1 ===

==== Week 1 (7th June - 13th June) ==== 
* Discussed with mentors regarding WPS UI and how to set it up on the MapMint web application.
* Designed a user interface for loading images from a folder and also capturing images via camera with several other functionalities.
* Read several publications regarding Structure From Motion.
* Wrote code for feature extraction from multiple images and matching them.

==== Week 2 (14th June - 20th June) ==== 
* Wrote code to generate 3D point cloud using a set of images.
* Applied Multiple Filters to filter out outliers.
* Extracted the 3D point cloud generated file(sample.ply) from 85 sets of images and visualized it using meshlab.

==== Week 3 (21st June - 27th June) ==== 
* Integrated the generation and visualization of multiple format 3D point cloud files in the UI.
* Applied some more advanced algorithm to remove noise from the generated 3D point clouds
* Integrated camera setup for taking live images and videos and convert them into point clouds

==== Week 4 (28th June - 4th July) ==== 
* Wrote code to convert point cloud (.ply) to mesh (.stl) file and visualize using o3d.
* Read OGC standards for WPS services and their basic requests and response.
* Read workshop 2017 post for creating zoo-services.
* Started creating simple web service-based UI for setting up various WPS services.

==== Week 5 (5th July - 11th July) ==== 
* Created a web service-based UI using Flask and added some functionality to it.
* Added Point cloud generation and visualization(using Three.js) to the UI.
* Initiated setting up Potree Visualization in the UI.

==== Week 6 (12th July - 18th July) ==== 
* Resolved some existing errors in the UI.
* Completed Potree visualization setup.
* Working on setup instructed by mentors during the meeting.
* Working on ZOO-Project configuration needed to create zoo-services.

==== Week 7 (19th July - 25th July) ====
* Created Sample getCapapbilities and describeService request using python modules for WPS services 
* Exploring more about Cheetah Template and their syntax 
* Creating a sample template to redirect to 3D point cloud UI or load UI

==== Week 8 (26th July - 1st August) ====
* Created a template for UI and added the volume to the mapmint container
* Exploring and understanding the mapmint-UI code to create a widget on the main UI

==== Week 9 (2nd August - 8th August) ====
* Created WPS service to run all the python scripts for Point Cloud Generation
* Working on storing the input and output data in the database

==== Week 10 (9th August - 15th August) ====
* Added 3D Point Cloud scene builder as a widget to the Mapmint Dashboard
* Cleaned up the code and added comments in the code
* Working on Documenting the work that I have completed till now and its future aspects

=== Final Report ===

'''Google Summer of Code 2021'''
* Author - [https://wiki.osgeo.org/wiki/User:SandeepSaurav Sandeep Saurav]
* Mentor 1 - [https://wiki.osgeo.org/wiki/User:Djay Gérald Fenoy]
* Mentor 2 - [https://wiki.osgeo.org/wiki/User:RajatShinde Rajat Shinde]
* Organization - [https://www.osgeo.org/ OSGeo]
* Developer Community - [http://www.mapmint.com/ MapMint], [http://www.zoo-project.org/ ZOO-Project]

'''Abstract'''

MapMint is a geographic information system (GIS) software on the Internet designed to facilitate the deployment of spatial data infrastructures (SDI). MapMint is for individuals and organizations wishing to manage and optimize SDI establishment and deployment of dynamic mapping applications. The application centralizes and simplifies several GIS and WebGIS functionality. The different levels of user rights divide tasks according to the public (system administrators, GIS specialists, GIS technicians, cartographers, webmasters, etc). The SDI comprises users, tools (MapMint web software and MapMint4ME Android app), and the data (currently, images and location ) recorded by the users. MapMint4ME  is an Android app for MapMint web-service which is built on top of the ZOO-Project. The ZOO-Project is an SDI manager which gives the users the capability of building their own map and web applications using Mapserver as the   Open Geospatial Consortium (OGC) services engine. MapMint allows multiple tasks relating to the implementation of an SDI from a modular and user-friendly administration interface.

The user can use MapMint, according to his rights:
* Import and store vector and raster GIS data
* Query database server and WMS / WFS external
* Publish geographic data in the form of WMS, WFS, and WMTS
* Treat, edit, and Data Sources style
* Compose and save maps in the form of projects (map files)
* Configure and generate mapping applications
* Configure and run a cartographic portal
* Visualize and share maps

MapMint includes several free software in a complete and coherent web mapping platform, whose operation is based on the use of open standards geomatics and the internet. At the heart of MapMint, we have the ZOO-Project, an application to simply and efficiently deploy [https://www.ogc.org/standards/wcs WPS](Web Processing Service) data processing. A set of web services is available MapMint, from simple display of a web page in the application to complex geographical treatments. Other types of web services are implemented, including visualization and querying spatial data, [https://www.ogc.org/standards/wms WMS](Web Map Service). Access to geographic data in vector format is via the [WFS](https://www.ogc.org/standards/wfs) (Web Feature Service) or access to image data via [https://www.ogc.org/standards/wcs WCS](Web Coverage Service). All these web services are provided by MapServer. The various files necessary for the functioning of MapServer and dynamic mapping applications are managed by MapMint services which provide a friendly interface for interacting with these web services. ZOO-Project and MapServer applications based on an Apache webserver to access the application via HTTP and HTTPS communication protocols. In the Windows environment, IIS can be used instead of Apache.

[[File:Mapmint-arch.png|700x433px|center]]

                                                      '''Overview of Mapmint Architecture'''

'''Motivation'''

The project aims to Integrate 3D scene builder as WPS services within MapMint administration User Interface. At present, the user can only visualize the 2D maps and can even generate one of them. So, my major motivation for this project is to create a user interface as a WPS service within MapMint such that the user cannot only visualize the 3D GIS maps but can also build the same. Similar to 2D maps, 3D GIS maps depict objects in greater detail by adding another dimension (z). 3D technology in GIS maps is an explanatory illustration that represents the scale of real-world objects. 3D models assist appearance, a survey in a large number of different domains. For instance, 3D maps can show the height of a hotel or a mountain and not just its location. The 3D tools have to be used along with 2D GIS and then imagined in a 3D setting.

Over the years, GIS has significantly impacted creating mapping as an essential tool to solve problems. Conventionally, GIS information was based on a two-dimensional recording, which apparently limited its usage in most applications. Incorporating 3D technology in GIS will customize the whole experience, make it more personal and enable detailed visualization. 

'''MapMint before GSoC 2021'''

With the current state of MapMint Software the user can import and store vector and raster GIS data, visualize, share maps, query database server, and WMS / WFS external, publish geographic data in the form of WMS, WFS, and WMTS, treat and edit data source style, compose and save maps in the form of projects. The interoperability of MapMint is provided by the standards used and implemented. It is thus possible to interact with data and services. MapMint application is not only accessible from a web browser but also a desktop client type QGIS.

'''Updates in MapMint during GSoC 2021'''

Integrated 3D scene builder as WPS services within MapMint administration User Interface.

Now the user can easily create a 3D Point Cloud scene using multiple sets of images from the same scene. 3D Point Cloud generation was done using [https://github.com/Sandeep10021/mapmint/wiki/StructureFromMotion Structure From Motion] algorithm.

[[File:UIView.jpg|center]]
                                               '''Overview of MapMint UI for 3D Point Cloud Generation'''

'''Future Opportunities'''

With the increase in applications of 3D scenes, huge opportunities are coming along with it. One can convert this Point Cloud model into a Mesh model and can even color them for presentation. There are various functions that can be added to process the model such as Sharpening, Masking, Smoothing, etc. Further one can even extend this 3D scene generation support for raster data as well that can be used in various applications.

'''Testing 3D Point Cloud Generation'''

'''As MapMint User Interface'''

* Setup OSGeoLive 13.0 from [https://github.com/Sandeep10021/mapmint/wiki/Setup-OSGeoLive-13.0 here]
* Setup ZOO-Project and MapMint using Docker from [https://github.com/Sandeep10021/mapmint/wiki/Setup-ZOO-Project-and-Mapmint-Using-Docker here] 
* Copy paste the folder ''PC_WPS'' from [here] to ''ZOO-Project/Zoo-project/zoo-services'' directory in order to create WPS services
* Next, make an widget similar to others on left hand side of the MapMint UI by making changes in [https://github.com/Sandeep10021/mapmint/blob/master/GSoC-21/Final%20Week/Skeleton_bs.py Skeleton_bs.py] and [https://github.com/Sandeep10021/mapmint/blob/master/GSoC-21/Final%20Week/Skeleton_bs.tmpl Skeleton_bs.py] in ''ZOO-Project/mapmint/mapmint-ui/templates'' directory
* Copy paste the folder ''PC_Generation'' to ''ZOO-Project/mapmint/public_map/'' directory
* Now, finally create the volumne by copy-pasting following code to ''docker-compose.yml'' file

  web:
    image: httpd:latest
    ports:
      - "9000:80"
    volumes:
      - ./mapmint/public_map/PointCloud:/usr/local/apache2/htdocs/PointCloud
      - ./mapmint/public_map/PointCloud/httpd.conf:/usr/local/apache2/conf/httpd.conf
      - ./mapmint/public_map/PointCloud/upload.py:/usr/local/apache2/cgi-bin/upload.py

  zookernel:
    build: .
    image: geolabs/mapmint
    ports:
      - "80:80"
    volumes:
      - ./mapmint/mapmint-ui/templates/generation_bs.tmpl:/mapmint/mapmint-ui/templates/generation_bs.tmpl
      - ./zoo-project/zoo-services/PC_WPS/cgi-env/generation.py:/usr/lib/cgi-bin/generation.py  
      - ./zoo-project/zoo-services/PC_WPS/cgi-env/PointCloud.zcfg:/usr/lib/cgi-bin/PointCloud.zcfg   
      - ./mapmint/public_map/PC_Generation:/var/www/html/PC_Generation
      - ./mapmint/public_map/PC_Generation/upload.py:/usr/lib/cgi-bin/upload.py 
      - ./mapmint:/mapmint

* Now you are ready to go !!
* Test if the WPS service for Point Cloud generation is running properly by running the following URL

 GetCapabilities: http://localhost/cgi-bin/zoo_loader.cgi?request=GetCapabilities&service=WPS
 DescribeProcess: http://localhost/cgi-bin/zoo_loader.cgi?request=DescribeProcess&service=WPS&version=1.0.0&Identifier=PointCloud

* Now, finally open the MapMint dashboard and enjoy the service! 
 
'''As Flask Web Application'''

* Setup OSGeoLive 13.0 from [here](https://github.com/Sandeep10021/mapmint/wiki/Setup-OSGeoLive-13.0)
* Clone the Point Clone Generation Repository from [here](https://github.com/Sandeep10021/mapmint/tree/master/GSoC-21/Week%201-9/PointCloudGeneration) 
* Install all modules to run this app by running following command

 pip install -r requirements.text

* Run app.py file using the following command

 python app.py
 
* Go to ''https://localhost:5000'' from any browser and now you can see Point Cloud Generation UI 
* Select '''at least 10 sets of images''' to generate 3D Point Cloud and Click ''Submit''
* Wait for some time until you receive a message '''Point Cloud has been generated successfully!'''
* Now your 3D Point Cloud is ready 
* Now you can download your 3D Point Cloud file by clicking on ''Download Point Cloud'' button
* Also, you can visualize it by clicking on ''Visualize'' button
* Alternatively you can also download the 3D Point Cloud file and visualize it in ''Meshlab'' by running the ''meshlab'' command in the terminal and selecting downloaded ''merged.ply'' file

'''Working Example'''

Demonstration video about how to use 3D Point Cloud service and UI [https://drive.google.com/file/d/1ZsurhJ2fp3_wR2_-Mol1I3446dRqIqZi/view?usp=sharing (Demo)]

'''Related Links'''
* Mapmint Repository : [https://github.com/Sandeep10021/mapmint Mapmint]
* Zoo-Project Repository : [https://github.com/Sandeep10021/ZOO-Project ZOO-Project]
* Project's Wiki Page : [https://wiki.osgeo.org/wiki/GSoC%2721_Integrating_3D_scene_builder_as_WPS_services_within_MapMint_administration_User_Interface 3D Scene Builder as WPS Service]

== Student's Biography ==
My name is Sandeep Saurav. I am an Electronics and Communication Engineering graduate from Maharaja Agrasen Institute of Technology, Delhi with a CPI of 8.68, pursuing my Master's degree in '''Geoinformatics and Natural Resources Engineering''' under Centre of Studies in Resources Engineering, Indian Institute of Technology Bombay, Mumbai, India.
Some more information about me can be obtained by following the link: [[https://wiki.osgeo.org/wiki/User:SandeepSaurav Link]]

== Mentors ==
* Fenoy Gerald [https://wiki.osgeo.org/wiki/User:Djay Fenoy Gerald ]
* Venkatesh Raghavan [https://wiki.osgeo.org/wiki/User:Venka Venkatesh Raghavan]
* Rajat Shinde [https://wiki.osgeo.org/wiki/User:Rajatshinde Rajat Shinde ]
* Sittichai Choosumrong [https://wiki.osgeo.org/wiki/Sittichai_choosumrong Sittichai Choosumrong]
* Samuel Souk Aoun




[[Category: Google Summer of Code]]