GSoC 17: istSOS-Data analysis and statistical tools suite

This is where I'll be posting information about istSOS-Data analysis and statistical tools suite GSoC 2017 project to the OSGeo community.

Introduction

istSOS is sensor data management tool that allows collection, maintenance and publishing of monitoring observations using the Open Geospatial Consortium (OGC) Sensor Observation Service (SOS)standard.

OAT(Observation Analysis Tools) can be used as a pre-processor for calibration observations, integrating the creation of observations for calibration directly from sensor time-series. The tool consists in an expandable Python library of processing methods and an interface integrated in the QGIS FREEWAT plug-in which includes a large number of modelling capabilities, data management tools and calibration capacity.

OAT library is an expandable Python library that can easily interact with istSOS.

The goal of my project primarily is to create OAT extension in RESTFull Web api and OAT extension having data analysis and statistical tools for istSOS which will be used to automate the creation of statisticate documents using OAT library and harvesting the data from an istSOS server and then publishing the result on Open Data Portals(CKAN).

Idea

The aim of my project primarily is to create OAT extension in RESTFull Web api and OAT extension having data analysis and statistical tools for istSOS which will be used to automate the creation of statisticat documents using OAT library and harvesting the data from an istSOS server and then publishing the result on Open Data Portals(CKAN).

Project

Title: istSOS-Data analysis and statistical tools suite.

Student: Rahul Chauhan

Mentors: Massimiliano Cannata, Milan Antonovic.

Details

Development process can be monitored at the following links:

• https://summerofcode.withgoogle.com/projects/#6301175977607168 - Project GSoC 2017 application Link
• https://github.com/istSOS/istsos2 - Project Main Repo. Link
• https://github.com/rahulworld/istsos2 - Forked Github Repo. Link
• https://github.com/rahulworld/Data-analysis - Project GitHub testing Repo. Link

Schedule

Dates	Task
Week 1 (May, 30 - June, 4)	User interface implementation.
Week 2 (June, 5 - June, 11)	Implementation of resample method(OAT) in istSOS api.
Week 3 (June, 12 - June, 18)	Testing resample(OAT method) with istSOS api.
Week 4 (June, 19 - June, 25)	Implementation of regularisation time series analysis method(OAT) in istSOS api.
Week 5 (June, 26 - July, 2) Phase 1 Evaluation	Test the working of these operations.
Week 6 (July, 3 - July, 9)	Implement OAT.methods i.e. , interpolation, digital filter.
Week 7 (July, 10 - July, 16)	Implement OAT.methods i.e. filling, validation, data quality assessment.
Week 8 (July, 17 - July, 23)	Implement remain OAT.methods.
Week 9 (July, 24 - July, 30) Phase 2 Evaluation	Testing on OAT method in istSOS api.
Week 10 (July, 31 - August, 6)	Implement CKAN (Open Data Portals) plugin.
Week 11 (August, 7 - August, 13)	Implement publish datasets ready to be harvested using CKAN.
Week 12 (August, 14 - August, 20)	Testing, Debugging, Cleaning and Optimizing the codes.
Week 13 (August, 21 - August, 27)	Take a week to scrub code, write a few test cases and test them, improve the current documentation.
Week 14 (August, 27 - August, 29) Final evaluation	Buffer period. ( Final Week. Prepare documentation for final submission.)

Reports

May 4 - May 14

• created github repo. https://github.com/rahulworld/Data-analysis
• Installed all devlopment requsites.
• Tried to understand the code flow of istSOS and OAT.
• Created Project wiki.

May 15 - May 29

What did I complete this week?

• Designed GUI for all Process time series methods.
• Also Designed GUI for
  • Add Time Series
  • Process Time Series
  • Compare Sensor
  • Manage Sensor
• This week, I’ve been indentifying all the process time series methods in OAT(freewat), I mean:
  • digital filter
  • exceedance
  • hydro events
  • hydro indices
  • hydro quality
  • resample
  • data values
  • hydro separation
  • integrate
  • compare
  • subtract
  • fill
  • statistics
  • hargreaves

What am I going to achieve for next week?

• Adding GUI(OAT extension and its sub-extension) in istSOS.
• Adding functionality of OAT method resampling in istSOS.

Is there any blocking issue?

• Currently, everything is going according to plan.

May 30 - June 4 (Week 1)

What did I complete this week?

• User interface design completed.
• Adding user interface implementation in istSOS api.

What am I going to achieve for next week?

• User interface implemented of OAT method resampling in istSOS api.

Is there any blocking issue?

• Currently, everything is going according to plan.

June, 5 - June, 11 (Week 2)

What did I complete this week?

• User interface implemented completely.
• Implementation of resample method.

What am I going to achieve for next week?

• Completing functionality of resample method(OAT) in istSOS api.

Is there any blocking issue?

• Currently, everything is going according to plan.
• Yes, I blocked on dygraph for plotting resampled data in istsos api.

June, 11 - June, 18 (Week 3)

What did I complete this week?

• Implemented resample functionality in istsos api.
• I have also updated the project wiki page.

What am I going to achieve for next week?

• Adding functionality of regularization method(OAT) in istSOS api.

Is there any blocking issue?

• Currently, everything is going according to plan.
• Yes, I blocked on dygraph for plotting resampled data in istsos api.

June, 19 - June, 25 (Week 4)

What did I complete this week?

• Implemented regularization functionality in istsos api.
• I have also updated the project wiki page.

What am I going to achieve for next week?

• Testing on these methods.
• Documentation.

Is there any blocking issue?

• Currently, everything is going according to plan.

Student's Biography

My name is Rahul Chauhan. I am 3rd year Computer Science & Engineering student at National Institute of Technology, Srinagar, India. Some more information about me can be obtained by following the link: http://rahulchauhan.net