Integrating a module to 3d scan a house within the MapMint4ME android application

Introduction

Introduction of MapMint

MapMint4ME is an Android application to record your data on the field. MapMint4ME is an Android application for MapMint web-mapping services. MapMint4ME gives the users capability to store data without using the internet locally. The data can be a file stored in alpha-numeric format, an audio file, a video file or readings obtained from a Sensor Observation Service (SOS). When the user returns back to a place with network connectivity, the recorded data can be uploaded back to the server.

MapMint4ME is an android application for MapMint web-service. MapMint is an internet-based Geographic Information System(GIS), designed to facilitate the deployment of Spatial Data Infrastructure(SDI). In an SDI, geographic data, metadata, tools, and the users are connected in an interactive manner in a framework so as to use the spatial information in an efficient and flexible way.

Project proposal

My proposal for GSoC 2021.

Abstract

This project allows a minimalist 3D scan (taking multiple pictures, recording camera position, using opendrone map to rebuild the 3D scene) with the house faces (accessible/visible faces) then load it as georeferenced data and finally export the data back on MapMint for 3D viewing.

Link to Github repository

Timeline

17th May - 7th June (Community bonding period)

What I have done during this period:
- Joined MapMint Slack Workspace created by mentors
- Introduced myself over the channel and shared my proposal over mailing list for suggestions
- Created a wiki page for the project Integrating a module to 3d scan a house within the MapMint4ME android application
- Updated wiki User page and added my personal information
- Updated links on the wiki Google_Summer_of_Code_2021_Accepted page
- Sat 29 may, we had our first meeting with mentors, where we have the chance to:
  - first introductions
  - discuss some administrative questions
  - discuss some blockings of the projects
- Created 3D scene using ODM (OpenDroneMap).
- Setup MapMint.
- Installed Android Studio and forked MapMint4ME repository (have not tried it yet)

Week One: 7th - 13th June

What have I done this week:
- Gone through some of the code of the app with mentor
- Solved the soft link problem in Windows
- Added a button for 3D scene in the home page
- Started the layout for the 3D scene page
- Gone through some documentation regarding javascript in WebView in Android

Am I blocked on anything:
- How to make the binding between javascript and java
- How the whole app is working, and how everything is linked to everything

My plans for next week:
- Working harder on my blockings
- Begin implementing the pictures taking within the app

Week Two: 14th - 20th June

What have I done this week:
- Worked on UI elements on the android app
- Discussed with mentor some issues of the SQL queries in the newly set up online MapMint live instance

Am I blocked on anything:
- I am not blocked on anything.

My plans for next week:
- Debug online MapMint instance
- Collect images from phone, experiment with creating a form

Week Three: 21th - 27th June

What have I done this week:
- Setup MapMint on a docker container on my local machine.
- Made the sample form for collecting images from MapMint4ME.

Am I blocked on anything:
- I am not blocked on anything.

My plans for next week:
- Connect the android MapMint4ME app to MapMint.

Week Four: 28th June - 4th July

What have I done this week:
- Gone through this workshop to create a zoo service.
- Created the Zoo Service configuration file needed to create the zoo service to get the data from ODM.
- Created the serviceProvider get_3d_model.py : the python script that will be used to get the 3D model from ODM.

Am I blocked on anything:
- I am not blocked on anything.

My plans for next week:
- Make zoo-service to get data from odm

Week Five: 5th - 11th July

What have I done this week:
- Still working on the Zoo Service configuration file needed to create the zoo service to get the data from ODM.
- Still working on the serviceProvider get_3d_model.py : the python script that will be used to get the 3D model from ODM.

My plans for next week:
- Continue working on getting the zoo service up and running

Am I blocked on anything:
- I am not blocked on anything.

Week Six: 12th - 18th July

What have I done this week:
- Had a meeting with mentor, which was highly productive, and resulted in the other accomplishments of this week.
- Removed a symbolic link to make sure the service will be running well see in my GitHub wiki
- Added a template that will be generated by the service code here
- Tried getting the template working

My plans for next week:
- Continue working on getting the zoo service up and running

Am I blocked on anything:
- I am not blocked on anything.

Week Seven: 19th - 25th July

What have I done this week:
- Added a volume to the zoo-kernel container to get access to the templates
- Added a volume to the zoo-kernel container to get access to 3D models
- Visualized a 3D model (.gltf) whitin cesium

My plans for next week:
- Continue working on getting the zoo service up and running

Am I blocked on anything:
- I am not blocked on anything.

Week Eight: 26th July - 1st August

What have I done this week:
- - Switched from Cesium to threeJS as viewer of the 3D data
- Added odm 3D results to 3dModels volume
- Created a simple user interface to view the 3D model
- Working on creating a drop list using cheetah template to allow the user to choose what model (from the existing ones) to view

My plans for next week:
- Continue working on UI

Am I blocked on anything:
- I am not blocked on anything.

Week Nine: 2nd - 8th August

What have I done this week:
- Finished the final draft for my UI (to render 3D models)
- Created a drop list to allow the user to choose what model (from the existing ones) to view
- Loaded successfully multiple sample models (**Tajjin** / **Rock** / **GerrardHall_Building**) all processed by me usig odm . [See here.]

My plans for next week:
- Refactor & clean up the code for loading the 3D models
- Connect the android MapMint4ME app to MapMint.

Am I blocked on anything:
- I am not blocked on anything.

Week Ten: 9th - 15th August

What have I done this week:
- Fixed the apt-key issue
- Clean up the code used to view the 3D models
- Working on documenting the numerous parts of my project

Am I blocked on anything:
- I am not blocked on anything.

GSoC 2021 Final Report

Abstract:

This project allows a minimalist 3D scan (taking multiple pictures, recording camera position, using opendrone map to rebuild the 3D scene) with the house faces (accessible/visible faces) then load it as 3D model and export the data back on MapMint for 3D viewing.

The state of the art BEFORE GSoC:

Have you ever wished that you could easily transfer a real object into a digital file? The technology is here, but most of us simply do not have access to high-end 3D scanners that can reconstruct the digital geometry of a physical model.

The added value:

Now it is possible to take multiple pictures using MapMint4ME, then visualize the 3D model issued from the taken pictures in MapMint.

Future Work:

One challenging part of my project, not yet done, is to get to link the MapMint4ME with a MapMint instance.
I did not get enough time to automate the step of creating the model using openDroneMap, here is what i was thinking to do.

Permanent links & documentation:

Code Repositories:

Videos:

Loading Models

Documentation

My work consist of three parts:

One graphical image describing my project:

Part One: Get the pictures using MapMint4ME

Here are the commits for this part.

Created a button that direct you to the this layout.
I have created a form that can be filled with a fixed number of images taken from the mobile MapMint4ME app, and will be transfered to MapMint, here is how to do it.
I have some issues regarding linking MapMint to MapMint4ME, so I did get the datasets for buildings from here, and I have taken the tajjin pictures by myself.

Part Two: Create the 3D model using ODM

I have created a 3D model from pictures, here is the tutorial.

Part Three: Visualize the 3D model in MapMint