Enhancing AR Support for MapMint4ME and adding GPS Satellite Tracking

From OSGeo
Jump to navigation Jump to search

Introduction

Introduction

MapMint4ME (MM4ME) is an android application with the capability to record data on the field without internet connectivity. It gives users the facility to store any kind of alphanumeric data. Users are able to take photos, record their position, and view their current location on a map based on the configuration settings of their MapMint server. When the user returns back to a place with access to the internet, the recorded data can be uploaded back to the MapMint server. In 2019, we added the AR support to the MapMint4ME. The support was to implement ARScale and CloudAnchor support to the MapMint4ME application. The CloudAnchor was used to implement anchors for representing virtual world object. In this project, we aims to Enhance ARScale and CloudAnchor support to MapMint4ME Android Application. MapMint4ME is an android application for MapMint web-service. MapMint is an internet-based Geographic Information System(GIS), designed to facilitate the deployment of Spatial Data Infrastructure(SDI). In an SDI, geographic data, metadata, tools, and the users are connected in an interactive manner in a framework so as to use the spatial information in an efficient and flexible way.

The MapMint is built on top of the ZOO-Project. The ZOO-Project is an SDI manager which gives the users the capability of building their own map and web-applications.

Project proposal

My proposal for GSoC 2020 can be found at: [proposal].

Abstract

The aim of the project is to extend Augmented Reality (AR) support in the MapMint4ME application. The motivation for the project was to improve field data acquisition. AR Simulation module provides users capability to visualize 3D objects simulation in real world scenes. AR Area module allows users to export measured "AR measurements" into text format and which can be used for further analysis. MapMint4ME works without the internet too, so adding a GPS Satellite information module which gives detailed analysis about GPS Coordinates and GPS Satellite information improves the information gathering process and makes it more efficient.


Link to Github repository: [Link]

Timeline

4th May - 24th May

Community bonding period: What I have done during this period?

  • Joined MapMint Slack Workspace created by mentors which we will be used as a mode of communication over the GSoC 2020 period
  • Introduced myself over the channel and shared my proposal over mailing list for suggestions
  • Mentors setup initial video call for primary discussion and explained me about my tasks during Community Bonding Period.
  • Created a wiki page for the project "Enhancing ARScale and CloudAnchor support to the MapMint4ME Android Application"
  • Updated wiki User page and added my personal information [Link]
  • Updated links on the wiki Google_Summer_of_Code_2020_Accepted page [Link]
  • Environment setup of MapMint4ME android application on my personal system on Android Studio.
  • Gone through documentation of MapMint4ME Android Application.
  • Gone through documentation for MapMint web application.
  • Gone through the working of the application user interface and code
  • Gone through working and setup which had implemented as part of last GSoC.
  • Analyze structure of MapMint4ME app and understand it thoroughly.
  • Discussed with the mentors their expectations over the GSoC 2020 project period.

What am I going to achieve for next week?

  • Implement basic Html UI elements in app.
  • Try to resolve firebase error which I am getting.
  • Try to implement basic retrieve Location ability within MapMint4ME.

Is there any blocking issue?

  • Currently, I am stuck to connect android studio and firebase.
  • Getting constant error from firebase.

Week 1 - 5 (25th May - 28th June)

Coding Phase 1:

  • Week 1 (25th May- 31st May)  :

What I completed this week?

  • Studied how to add location support in Android Application Link
  • Studied and implemented "Get Current Location in Android"[Link]
  • Tested the simple Location support added to Android Application Link
  • Tested this location application on different android devices

What am I going to achieve for next week?

  • Show number of satellites and their position relatively to the current position
  • Add this "Get Current Location" to MapMint4ME application

Is there any blocking issue?

  • MapMint4ME have different language support while combining new part I have to take care of that
  • Week 2 (1st June- 7th June)  :

What I completed this week?

  • Studied more about GpsSatellite class[Link] and GpsStatus class[Link]
  • Implemented module that shows number of satellites relatively to current position[Link]
  • Display both number of satellites used to fix location and total number of available satellites with respect to current Gps Coordinates
  • Tried to show satellites' position with respect to current position

What am I going to achieve for next week?

  • Will complete implementation of satellites' position with respect to current position
  • Integrate GPS Coordinate finder and satellite tracking module to MapMint4ME app
  • Will test this module on different android devices

Is there any blocking issue?

  • No blocking issue till now
  • Week 3 (8th June- 14th June)  :

What I completed this week?

  • I added the following methods which give more information regarding GPS satellites currently used to fix locations, Satellite information is required because MapMint4ME acquires data without internet
  • getPrn() method returns pseudo random number for satellite
  • getAzimuth() method returns azimuth of the satellite in degrees
  • getElevation() method returns elevation of the satellite in degrees
  • getSnr() method returns signal to noise ratio
  • usedInFix() method returns Returns true if the satellite was used by the GPS engine when calculating the most recent GPS fix
  • Currently all above methods give outputs on the android studio console which can be seen at[Link]
  • Integrated last week’s work on MapMint4ME app, for this I created some UI elements and modified the edit.html file to integrate the GPS Satellite Information module

What am I going to achieve for next week?

  • To show output of getPrn(), getAzimuth(), getSnr(), getElevation(), usedInFix() on UI, discussion is going on with mentors about how we can display it on screen
  • Will integrate this in MapMint4ME
  • Test integration with different devices

Is there any blocking issue?

  • No blocking issue till now
  • Week 4 (15th June- 21st June)  :

What I completed this week?

  • Added output of getPrn(), getAzimuth(), getSnr(), getElevation(), usedInFix() on UI[Link]
  • Integrated this new module with existing MapMint4ME applicaton[Link]
  • Resolved all errors and warnings while integrating this new module with existing MapMint4ME
  • Tested MapMint4ME with new functionalities on different devices and it runs smoothly without crashing
  • Test video link for MapMint4ME with Satellite Information Module[Link]

What am I going to achieve for next week?

  • Will start working on the first objective i.e. storing AR Scene and visualizing it in app, discussion is going on with mentors regarding this.

Is there any blocking issue?

  • No blocking issue till now
  • Week 5 (22nd June- 28th June)  :

Unfortunately I lost one of my close relatives to Covid-19 this week, so wasn't able to do the coding work this week. I will certainly fill this gap in next week’s work.

What I completed this week?

  • Made pull request to MapMint4ME/gsoc-2020 branch[Link]
  • Pushed my implementation of Satellite information module on this new branch[Link]
  • Discussed with mentors about how we can proceed with Task 1
  • After discussion with mentors we got to know, that there is no way we can store cloud anchors locally at present[Link],[Link]
  • ARKit allows to place 3d model accurately with respect to location but this is not possible with ARCore and Sceneform. So finally after discussion with mentors we will follow another approach for task 1
  • We will do AR Simulation. For this we will place a 3d object on the plane. Adjust its position, size according to the user's need. Accelerate or Rotate that object in the real world. We decided to follow this approach for task 1

What am I going to achieve for next week?

  • Start working on the plan mentioned above
  • Start coding for task 1

Is there any blocking issue?

  • No blocking issue till now

Week 6 - 9 (29th June - 25th July)

Coding Phase 2:

  • Week 6 (29th June- 5th July)  :

What I completed this week?

  • Studied about Sceneform and ARCore[Link]
  • Created an AR app in which the user will select which 3d object he/she wants to place on the surface virtually. When the grid is detected, the user will just tap on the surface and that object will be placed on the surface virtually. More details can be found at[Link]
  • ARCore is supported by very limited devices[Link] and have tested on devices available to me

What am I going to achieve for next week?

  • Write a function to accelerate(move/shift) 3d objects in the real world
  • Write a function to adjust the size of 3d objects in the real world
  • Test app on different ARCore compatible devices

Is there any blocking issue?

  • Currently I am taking 3d objects from [Link], converting that objects to sfa and sfb files using “Import sceneform” in android studio is not giving desired output
  • Week 7 (6th July- 12th July)  :

What I completed this week?

  • Created UI element for adjusting size of 3d object in activity_main.xml file. Wrote java function for the same in MainActivity.java file.
  • Studied more about MotionEvent[Link]
  • Created a button on the UI for Accelerating 3d objects. Wrote java functions for the same with the help of MotionEvent[Link]
  • Tested implemented function
  • Test video link for ARSimulation Module[Link]

What am I going to achieve for next week?

  • Write a function for turning 3d object left or right based on user interest in real world scene
  • will write function for rotating objects in the real world
  • Integration of AR Simulation with MapMint4ME
  • Testing this on different ARCore supported devices.

Is there any blocking issue?

  • No blocking issues
  • Week 8 (13th July- 19th July)  :

What I completed this week?

  • Completed function for rotating 3d objects. Now the user can easily rotate the 3d object based on user’s preference
  • Wrote function for left and right button. Which will move objects gently right and left based on the user's preference[Link]
  • Tested implementation on various ARCore supported devices
  • Found that Sceneform 1.16.0 only supports glTF instead of SFA and SFB Sceneform formats[Link]
  • So now before integrating with MapMint4ME, the challenge is to use glTF format which previously we used SFA and SFB format in android studio. So I am working on adding glTF format for importing 3d objects

What am I going to achieve for next week?

  • Will create 3d objects in android studio this time with glTF
  • Integration of implemented module with MapMint4ME
  • Will make pull request and push changes to the main repository after integration.
  • Test app with different android devices

Is there any blocking issue?

  • No blocking issues
  • Week 9 (14th July- 26th July)  :

What I completed this week?

  • Solved all problems related to integration of 3d objects with MapMint4ME
  • Integrated AR Simulation module with MapMint4ME
  • Tested app on various android devices
  • Committed AR Simulation module on new branch[Link]
  • Made pull request to MapMint4ME/gsoc-2020 branch[Link]
  • Here is a link to working of the AR Simulation in the app[Link]

What am I going to achieve for next week?

  • Will discuss with mentors for next task
  • Start coding for next task

Is there any blocking issue?

  • No blocking issues

Week 10 - 13 (26th July - 23rd Aug)

Coding Phase 3:

  • Week 10 (26th July- 2nd Aug)  :

What I completed this week?

  • For finding the area or volume of selected anchor space we need to store measurements but previous implementation does not store any measurements. After discussing with mentors we decided to store those measurements in text format, so that it can later retrieved whenever needed and it will then be easy to calculate area or volume of that selected anchor space
  • Started working on measurements taken from ARScene
  • Start coding for storing those measurements and still working on it
  • Made new branch ARArea and push my changes to that branch[Link]

What am I going to achieve for next week?

  • Complete module for storing measurements
  • Will test implemented functionality

Is there any blocking issue?

  • No blocking issues
  • Week 11 (3rd Aug - 9th Aug)  :

What I completed this week?

  • Added AR Fragment and other UI Elements for finding vertical and horizontal distance in selected Anchor space
  • Wrote functions for finding horizontal and vertical distance
  • For storing those measurements, till now i am able to display it on Android studio console and working to export those measurements as text file[Link]

What am I going to achieve for next week?

  • Complete module for exporting measurements
  • Will test implemented functionality
  • Integrate this with main MapMint4ME repo

Is there any blocking issue?

  • No blocking issues
  • Week 12 (10th Aug - 16th Aug)  :

What I completed this week?

  • Studied how to export measured data in Android[Link]
  • Added UI Element for exporting data in corresponding layout file(armeasure_save.xml)
  • Wrote function for exporting measurements of current AR scene(Armeasure_save.java)
  • Tested the implemented functionality on different devices
  • Integrated this implementation with main MapMint4ME Repo[Link]

What am I going to achieve for next week?

  • Will make final implementation video of contributions for the functions which I implemented
  • Will make final report based on guidelines provided by GSoC and suggestions from mentor
  • Make a PR to the develop branch with all the files

Is there any blocking issue?

  • No blocking issues
  • Week 13 (17th Aug - 23rd Aug)  :

What I completed this week?

  • Modified some UI elements for better understanding of implemented functionality
  • Tested modified functionalities and pushed my changes to repo[Link]
  • Working on code refactoring and started with documentation

What am I going to achieve for next week?

  • Complete my documentation and prepare video for final implementation
  • Wrap up my project and submit the final evaluation of my mentors

Is there any blocking issue?

  • No blocking issues

GSoC 2020 Final Report

Augmented Reality support extended in MapMint4ME. Following three different features were added to the MapMint4ME using Google ARCore. The added features are as follows:

Final (1).jpg
  • AR Simulation : A simulation is an approximate imitation of the operation of a process or system. Augmented Reality simulation serves the purpose of experimentation and usability evaluation. In this module, the user can place any 3D object in the real world scene virtually. Users are virtually able to adjust an object's position and adjust its size according to its need. Simulation of 3D object module also includes accelerating which provides the ability of left, right and forward direction movements of the 3D object in the real world domain. Added functionality for rotating this 3D object.
  • AR Scale + Export: Measuring real world things with the help of augmented reality is the objective of this task. Users are able to measure length, width and height for multiple objects and save all those measurements in text format. Later after taking all measurements users can export those measurements, and are easily able to save in google drive or internally as text file.
  • GPS Satellite Module: As MapMint4ME works without internet also, So for offline data collection GPS Coordinate plays an important role. Apart from GPS Coordinates, GPS Satellite information is also necessary. GPS Satellite information module provides following information for each detected satellite using [GpsSatellite] in Android.
     1) PRN No  : Psuedo Random No for satellite

     2) SNR : Signal to noise ratio for satellite

     3) Azimuth: Azimuth angle for satellite

     4) Elevation : Elevation angle for satellite

     5) UsedInFix: Boolean value representing if satellite used to fix location or not

This module also shows how many satellites are detected, and out of which how many actually are used to fix GPS Coordinates. All the above information about the satellites is real time and changes dynamically.

Future Opportunities : There are endless possibilities what we can do with Augmented Reality . Adding more Augmented Reality support based on requirements can improve field data collection surely. At present AR Scale + Export & AR Simulation both are different modules,but if both of them can be merged so that we can use AR Scale + Export module to take measurements and visualise object of that particular size using AR Simulation. In GPS Satellite module we can provide feature to visualise those tracked satellites in an animation.

Working and Examples: Implementation video about how to use added functionalities in MapMin4ME. [Example]

Link to Repository : Repository

Link to Project's OSGeo Wiki-page: OSGeo Wiki Page

Final APK: APK

Here is a link to the Final report of GSoC 2020. Final Report

Other repositories:

Other Important Links:

  • GitHub Wiki page: Link
  • Google Drive Folder with Videos & images: Link

Student's Biography

My name is Aditi Mohan Sawant. I am a Computer Science graduate from Mumbai University with CPI 8.81, pursuing my Master's degree in Geoinformatics and Natural Resources Engineering under Centre of Studies in Resources Engineering, Indian Institute of Technology Bombay, Mumbai, India. Some more information about me can be obtained by following the link: [Link]

Mentors

Retrieved from "http://wiki.osgeo.org/w/index.php?title=Enhancing_AR_Support_for_MapMint4ME_and_adding_GPS_Satellite_Tracking&oldid=125270"