Integrating a module to 3d scan a house within the MapMint4ME android application

Introduction of MapMint
MapMint4ME is an Android application to record your data on the field. MapMint4ME is an Android application for MapMint web-mapping services. MapMint4ME gives the users capability to store data without using the internet locally. The data can be a file stored in alpha-numeric format, an audio file, a video file or readings obtained from a Sensor Observation Service (SOS). When the user returns back to a place with network connectivity, the recorded data can be uploaded back to the server.

MapMint4ME is an android application for MapMint web-service. MapMint is an internet-based Geographic Information System(GIS), designed to facilitate the deployment of Spatial Data Infrastructure(SDI). In an SDI, geographic data, metadata, tools, and the users are connected in an interactive manner in a framework so as to use the spatial information in an efficient and flexible way.

Project proposal
My proposal for GSoC 2021.

Abstract
This project allows a minimalist 3D scan (taking multiple pictures, recording camera position, using opendrone map to rebuild the 3D scene) with the house faces (accessible/visible faces) then load it as georeferenced data and finally export the data back on MapMint for 3D viewing.

Link to Github repository

17th May - 7th June (Community bonding period)

 * What I have done during this period:
 * Joined MapMint Slack Workspace created by mentors
 * Introduced myself over the channel and shared my proposal over mailing list for suggestions
 * Created a wiki page for the project Integrating a module to 3d scan a house within the MapMint4ME android application
 * Updated wiki User page and added my personal information
 * Updated links on the wiki Google_Summer_of_Code_2021_Accepted page
 * Sat 29 may, we had our first meeting with mentors, where we have the chance to:
 * first introductions
 * discuss some administrative questions
 * discuss some blockings of the projects
 * Created 3D scene using ODM (OpenDroneMap).
 * Setup MapMint.
 * Installed Android Studio and forked MapMint4ME repository (have not tried it yet)

Week One: 7th - 13th June

 * What have I done this week:
 * Gone through some of the code of the app with mentor
 * Solved the soft link problem in Windows
 * Added a button for 3D scene in the home page
 * Started the layout for the 3D scene page
 * Gone through some documentation regarding javascript in WebView in Android


 * Am I blocked on anything:
 * How to make the binding between javascript and java
 * How the whole app is working, and how everything is linked to everything


 *  My plans for next week:
 * Working harder on my blockings
 * Begin implementing the pictures taking within the app

Week Two: 14th - 20th June

 * What have I done this week:
 * Worked on UI elements on the android app
 * Discussed with mentor some issues of the SQL queries in the newly set up online MapMint live instance


 * Am I blocked on anything:
 * I am not blocked on anything.


 *  My plans for next week:
 * Debug online MapMint instance
 * Collect images from phone, experiment with creating a form

Week Three: 21th - 27th June

 * What have I done this week:
 * Setup MapMint on a docker container on my local machine.
 * Made the sample form for collecting images from MapMint4ME.


 * Am I blocked on anything:
 * I am not blocked on anything.


 * My plans for next week:
 * Connect the android MapMint4ME app to MapMint.

Week Four: 28th June - 4th July

 * What have I done this week:
 * Gone through this workshop to create a zoo service.
 * Created the Zoo Service configuration file needed to create the zoo service to get the data from ODM.
 * Created the serviceProvider get_3d_model.py : the python script that will be used to get the 3D model from ODM.


 * Am I blocked on anything:
 * I am not blocked on anything.


 * My plans for next week:
 * Make zoo-service to get data from odm

Week Five: 5th - 11th July

 * What have I done this week:
 * Still working on the Zoo Service configuration file needed to create the zoo service to get the data from ODM.
 * Still working on the serviceProvider get_3d_model.py : the python script that will be used to get the 3D model from ODM.


 * My plans for next week:
 * Continue working on getting the zoo service up and running


 * Am I blocked on anything:
 * I am not blocked on anything.

Week Six: 12th - 18th July

 * What have I done this week:
 * Had a meeting with mentor, which was highly productive, and resulted in the other accomplishments of this week.
 * Removed a symbolic link to make sure the service will be running well see in my GitHub wiki
 * Added a template that will be generated by the service code here
 * Tried getting the template working


 * My plans for next week:
 * Continue working on getting the zoo service up and running


 * Am I blocked on anything:
 * I am not blocked on anything.

Week Seven: 19th - 25th July

 * What have I done this week:
 * Added a volume to the zoo-kernel container to get access to the templates
 * Added a volume to the zoo-kernel container to get access to 3D models
 * Visualized a 3D model (.gltf) whitin cesium


 * My plans for next week:
 * Continue working on getting the zoo service up and running


 * Am I blocked on anything:
 * I am not blocked on anything.

Week Eight: 26th July - 1st August

 * What have I done this week:
 * - Switched from Cesium to threeJS as viewer of the 3D data
 * Added odm 3D results to 3dModels volume
 * Created a simple user interface to view the 3D model
 * Working on creating a drop list using cheetah template to allow the user to choose what model (from the existing ones) to view


 * My plans for next week:
 * Continue working on UI


 * Am I blocked on anything:
 * I am not blocked on anything.

Week Nine: 2nd - 8th August

 * What have I done this week:
 * Finished the final draft for my UI (to render 3D models)
 * Created a drop list to allow the user to choose what model (from the existing ones) to view
 * Loaded successfully multiple sample models (**Tajjin** / **Rock** / **GerrardHall_Building**) all processed by me usig odm . [See here.]


 * My plans for next week:
 * Refactor & clean up the code for loading the 3D models
 * Connect the android MapMint4ME app to MapMint.


 * Am I blocked on anything:
 * I am not blocked on anything.

Week Ten: 9th - 15th August

 * What have I done this week:
 * Fixed the apt-key issue
 * Clean up the code used to view the 3D models
 * Working on documenting the numerous parts of my project


 * Am I blocked on anything:
 * I am not blocked on anything.

Abstract:
This project allows a minimalist 3D scan (taking multiple pictures, recording camera position, using opendrone map to rebuild the 3D scene) with the house faces (accessible/visible faces) then load it as 3D model and export the data back on MapMint for 3D viewing.

The state of the art BEFORE GSoC:
Have you ever wished that you could easily transfer a real object into a digital file? The technology is here, but most of us simply do not have access to high-end 3D scanners that can reconstruct the digital geometry of a physical model.

The added value:
Now it is possible to take multiple pictures using MapMint4ME, then visualize the 3D model issued from the taken pictures in MapMint.

Future Work:

 * One challenging part of my project, not yet done, is to get to link the MapMint4ME with a MapMint instance.
 * I did not get enough time to automate the step of creating the model using openDroneMap, here is what i was thinking to do.

Code Repositories:

 * ZOO-Project
 * MapMint4ME

Videos:

 * Loading Models

Other Important Links:

 * Wiki on GitHub
 * Project on OSGeo Wiki

Documentation
My work consist of three parts:


 * One graphical image describing my project:



Part One: Get the pictures using MapMint4ME
Here are the commits for this part.


 * Created a button that direct you to the this layout.
 * I have created a form that can be filled with a fixed number of images taken from the mobile MapMint4ME app, and will be transfered to MapMint, here is how to do it.
 * I have some issues regarding linking MapMint to MapMint4ME, so I did get the datasets for buildings from here, and I have taken the tajjin pictures by myself.

Part Two: Create the 3D model using ODM

 * I have created a 3D model from pictures, here is the tutorial.

Part Three: Visualize the 3D model in MapMint
Here are the commits for this part.


 * Got MapMint working on virtual machine tutorial here, then I finished up with configuring it on Docker tutorial here.
 * Added 3D models to the volume see here.
 * Used docker volumes to get access to the 3D models to be visualized.
 * Using threeJS, I was able to render the 3D models in MapMint, using a template.

Mentors

 * Samuel Souk aloun
 * Fenoy Gerald
 * Rajat Shinde
 * Venkatesh Raghavan
 * Sittichai choosumrong