IstSOS Web API
Currently in progress...
Introduction
istSOS[1][2] is an OGC SOS server implementation written in Python. IstSOS allows managing and dispatching observations from monitoring sensors according to the Sensor Observation Service standard.
istSOS Web API is one of the accepted project ideas for Google Summer of Code 2016 and it is currently being improved as part of Google Summer of Code 2017. Project name this year is known as Improving IstSOS Web API and finalizing IstSOS Javascript Core Library.
Improving IstSOS Web API and finalizing IstSOS Javascript Core Library is focused on two parts:
- rewritting IstSOS JavaScript core library to ES6, modernize project build system logic(webpack and babel transpiler) and publish the library to npm.
- rewritting IstSOS Web Widget Creator, using React JavaScript library, improving existing functionality and introducing new tools.
Development process can be monitored at the following links:
- https://github.com/istSOS/javascript-core - Project repository
- https://github.com/Luka-G/javascript-core - Forked repository
- https://github.com/istSOS/web-widget-creator - Project repository
- https://github.com/Luka-G/web-widget-creator - Forked repository
Student working on improvements of the istSOS Web API: Luka Glušica
Mentors: Milan Antonovic, Luca Delucchi
[1] https://wiki.osgeo.org/wiki/IstSOS
[2] http://istsos.org
Implementation details
Libraries
Openlayers 3
OpenLayers 3 is an open source JavaScript library for displaying map data in web browsers.
OL3 makes it easy to put a dynamic map in any web page. It can display map tiles, vector data and markers loaded from any source. OpenLayers has been developed to further the use of geographic information of all kinds. It is completely free, Open Source JavaScript, released under the 2-clause BSD License (also known as the FreeBSD).
Map tool of the istSOS Web widget creator, found OL3 very useful for generating map widgets containing sensor location data.
Moment
Moment is a lightweight JavaScript date library for parsing, validating, manipulating, and formatting dates.
Moment was designed to work both in the browser and in Node.js. All code should work in both of these environments, and all unit tests are run in both of these environments. Currently the following browsers are used for the ci system: Chrome on Windows XP, IE 8, 9, and 10 on Windows 7, IE 11 on Windows 10, latest Firefox on Linux, and latest Safari on OSX 10.8 and 10.11.
Moment has proven useful in converting input dates and time to istSOS allowed formats. Also, several methods use Moment to compare two date&time strings and return older or newer date.
istSOS JavaScript core
Google Summer of Code 2016
istSOS JavaScript core library exposes communication with the istSOS Rest API interface. Idea was to identify all rest API HTTP requests and implement them as methods of newly implemented classes. Most important request for developers and users is getObservations request, because it returns actual measurements collected by the sensors and istSOS Javascript core library is focused on several ways to get observation data from the server. Class structure describes the conceptual logic of the istSOS core library and it's based on couple of "container" classes. Every class has a defined namespace "istsos" before it's actual name (example: istsos.Server) and library starts with IstSOS class as a container of Server instances.
- IstSOS class - contain list of Server instances
- Server class - contain list of Service instances
- Service class - contain Procedure instance, VirtualProcedure instance, Offering instance, ObservedProperty instance, DataQuality instance and UnitOfMeasure instance lists.
There are classes like Database and Configuration, that are related to Server class as default and to Service class as service specific.
Rest of the classes:
- Date clas - used for converting time and date integer inputs to valid istSOS format.
- ProcedureBase class - class from which Procedure and VirtualProcedure classes inherit properties and methods.
- Output class - used to simplify use of Procedure and VirtualProcedure instances. Holds ObservedProperty and UnitOfMeasure instances.
Other than mentioned, istSOS JavaScript core has a special way of event handling with lots of custom event types like (PROCEDURE, NEW_PROCEDURE, SERVICES, DELETE_SERVICE etc). It has 2 global methods for interacting with tha data that came asynchronously from the server:
- istsos.on()
- istsos.once()
Both methods, take event type as a parameter and the data is to be manipulated through callback function. Here is the getObservations usage example, which also explains event handling in istSOS JavaScript core library.
Google Summer of Code 2017
This year, changes in library design and class structure have been introduced. Here is the list:
- Closure library has been removed from the project. To compensate some of the functionalities of the closure library, that has been used, new functionalities were introduced.
- Whole library is rewritten to ES6
- Dependencies are handled by ES6 import/export modules logic.
- istsos.HttpAPI is a small API, designed to handle HTTP requests and promisify them. API methods:
- get (event, opt_config)
- put (event, opt_config)
- post (event, opt_config)
- delete (event, opt_config)
- istsos.EventEmitter class - new event handling system is implemented and with it, major issue of the library is covered. Every class, that uses HTTP requests, inherits from istsos.EventEmitter class. Event handling methods:
- fire (event, data)
- on (event, callback)
- once (event, callback)
- off (event, callback)
- unlistenAll ()
- Webpack was integrated to handle the library build
- Basic HTTP authentication was also added as a usage possibility.
istSOS Online Web Widget Creator
IstSOS Web widget creator an Online Web Application for building featurerich web widget that will be easily embeddable into existing web pages. It was build upon istSOS JavaScript core library and small istSOS widget JavaScript library. Bootstrap and JQuery were used for general design of the application. Bootstrap plugin - datetimepicker was also used for helping with date&time inputs.
IstSOS Web widget creator consists of three tools:
- Map tool
- Box tool
- Chart tool
Map tool uses OpenLayers 3 to build a map with sensor locations as Point features, showing the last observations, date and time of the last observation and name of related sensor(procedure).
Upon selecting the map tool in the menu, user is offered to select service, offering, procedure/procedures and observed property and to set up other configurable properties(See Usage instructions). Upon page load, list of services is collected from the server. When user selects a service, list of offerings based on selection is collected from the server and so on in the order mentioned above. User is allowed to select multiple procedures, but is limited to select only one observed property, which differs from the box and chart widgets(below).
As a result, in the preview section, Map widget is shown and in the code section, generated embedded code is shown, which user imports to his own Web page in order to display built widget.
Interesting OL3 functionality, called ol.source.Cluster, is used to handle overlaping features(sensors). Whenever two or more features overlap, they join into one and icon show number and names and names of all sensors that are clustered.
Box tool uses Bootstrap for responsivity and design control.
Upon selecting the box tool in the menu, user is offered to select service, offering, procedure and observed property/observed properties and to set up other configurable properties(See Usage instructions). User is allowed to select only one procedure and multiple observed properties, that are related to selected procedure.
Resulting box widget contains name of the selected sensor, date and time of the last observation, and observation values of each selected observed property.
Also, user is offered to choose a layout for output widget, which can be horizontal and vertical.
Chart tool represents integration of another project - VistSOS - Data visualization framework(See VistSOS).
Upon selecting the map tool in the menu, user is offered to select service, offering, procedure/procedures and observed property/observed properties and to set up other configurable properties. Chart tool is specific, because usage of it requires additional configuration.(See Usage instructions).
User is allowed to select multiple procedures and multiple observed properties, and also to select time interval in which chart data will be shown.
There are several chart types at disposal, like line, multivariable, overview and histogram chart. More info about VistSOS projectpage
Schedule
Development of istSOS Web API consists of two parts:
- JavaScript Core Library
- Web App for creating Web Widgets, based on previosly developed core library
Working schedule presented below, is arranged in consultance with the mentors and other students working on the istSOS projects. The planning isn't strict for "in-between" deadlines, because of varying nature of our projects, but it is strict for two major parts, mentioned above (June 20 and August 22).
Timeline planning for istSOS JavaScript Core
Task From To Requirements analysis May 23 May 30 Coding May 30 June 13 Testing, bug fix, release June 13 June 20
Timeline planning for istSOS Online Web Widget Creator
Task From To Design and Requirements analysis June 20 June 27 Coding June 27 July 18 Coding(VistSOS Web Component Integration) July 18 August 8 Coding, Testing, Bug Fixing, Documentation August 8 August 22
Google Summer of Code 2016 - Reports
Week 1
What did you get done this week?
This week, I’ve been indentifying all the admin HTTP requests to the server, that are targeting the istSOS RESTful API. By identifying, I mean writting down:
- URL
- type of request (GET, POST, PUT, DELETE)
- response data
- request payload
Also, I’ve started designing the class structure, for the purpose of developing JavaScript core library.
All of the mentioned work, I’m doing in partnership with a GsoC student Florin-Daniel Cioloboc, who’s working on an Java core library for istSOS and our work overlaps in this section.
What do you plan on doing next week?
Next week, I will be finishing the class structure and begin with the actual development/coding of JavaScript core library. I will be also creating some test pages along the way, to check for possible errors.
Are you blocked on anything?
Currently, everything is going according to plan.
Week 2
What did you get done this week?
This week, I've been continuosly working on class structure design for JavaScript core library and also started commiting some code. The was to implement IstSOS, Server and Database classes, but let's say that 90% of it is done.
What do you plan on doing next week?
Next week, I will be completing the class structure and will try to implement more classes like Service, Offering, Procedure.
Are you blocked on anything?
Actually, the reason why 90% of mentioned classes were done, is that I was unable to store JSON response object into an outside variable. For the information, I've been using Google Closure library for executing AJAX requests.
Week 3
What did you get done this week?
This week, I’ve implemented Configuration, Service, Offering, ObservedProperty, DataQuality, UnitOfMeasure and Date classes. Service class is partially implemented, because it depends on other classes that should be implemented soon. Also, I’ve created an html and JavaScript file for testing and I’ve already tested some of the previously implemented classes, but the new ones are about to be tested. Appart from mentioned, I’ve updated OSgeo wiki page with timeline and weekly reports, also you can take a look at README file in my GitHub forked repository, because I’ve started with creating API documentation for the core library I’ve been working on.
What do you plan on doing next week?
Next week, I plan to implement last two classes (Procedure, VirtualProcedure) and to do all the testing, bug fixes and refining the code and documentation. Hopefully, by June 20-th, I will be releasing istSOS JavaScript core library.
Are you blocked on anything?
Currently, everything is fine. I solved the issue with AJAX requests in consultance with my mentors.
Week 4
What did you get done this week?
I have implemented all the classes and did the testing of all the GET requests, which can be checked if you take a lok at istsos-test.html and istsos-test.js ( GitHub forked repository).
I have also updated the GitHub wiki page with the API documentation and the class structure of the library.
What do you plan on doing next week?
I plan to test all requests (POST/PUT/DELETE). Hopefully, this part will go easier and quicker, because I already fixed most of the minor problems during GET request tests.
Also, I will be updating documentation along the way.
We have regular meetengs with our mentors every Tuesday, so If everything goes as planned, I will prepare the first version of a library until then.
Are you blocked on anything?
Currently, everything is going according to plan.
Week 5
What did you get done this week?
It's a midterm evaluation period and I did a lot of testing of the library, there is a test html and test js on my GitHub forked repository. Also, I've been updating the API documentation and Class structure on the GitHub wiki page.
Next part of my job is to develop JavaScript Framework based on previously developed library, so I started thinking about how it should be implemented, how should interface look like etc.
What do you plan on doing next week?
I plan to finish the documentation in the next few days, do some more testing of a library and generate a compiled version of a library, so other people can use it easily.
Also, I will work on conceptual design of JavaScript Framework, I will be working together with GSoC student Felipe Poveda, because I need to integrate his implementation into my project and he needs to use my library for his own project.
Are you blocked on anything?
Currently, everything is fine.
Week 6
What did you get done this week?
I've been working on a design of the JavaScript Framework. No coding this week, more thinking and drawing on a piece of paper. Also, I started writing into a sreadsheet, every functionallity that should be implemented with description, so I can better organize my self, when the coding starts.
What do you plan on doing next week?
I plan to finish conceptual design and start implementing/coding the features. Of course, updates&fixes of the previously developed library along the way, is inevitable.
Are you blocked on anything?
Currently, everything is fine.
Week 7
What did you get done this week?
I've been designing home page, with some of the tools (without functionality, just visual aspect) . Usage of Bootstrap helped with the page repsonsivness. Code can be checked and tested here.
What do you plan on doing next week?
Next week, I plan to finish with design and implement a lot of tools, like Map creator tools, Admin tools for creating, updating and deleting data etc. I will be thinking and adding some new tools allong the way. Of course, fixing of design errors may be necessary, depending on the feature requirements.
Are you blocked on anything?
Currently, everything is fine.
Week 8
What did you get done this week?
This week, in consultance with my mentors, I've reduced number of tools that I've had planned to implement, because they seem irrelevant. Also, I changed approach a little bit, so I needed to start over with implementation. New plan is to develop small library for creating widgets. So far, I have implemented istsos.widget.Widget base class and partially istsos.widget.Map class. Code can be checked here.
What do you plan on doing next week?
Next week, I plan to improve Widget base class and finish the Map class implementation. Also, I will start with developing Box class and working on a way to integrate this project as a Chart class.
Are you blocked on anything?
Currently, everything is fine.
Week 9
What did you get done this week?
This week, I finished with the implementation of Map creator tool. Of course, further improvements will be made along the way, but the main functionality is covered and tested. I also started with the implementation of a Box creator tool. Also I've been updating and fixing the istsos core library with some changes regarding the getObservations method and compatibility issues with Internet Explorer 11. All variations of getObservations method can now pass multiple procedures as parameter. Code can be checked at following links:
What do you plan on doing next week?
Are you blocked on anything?
Currently, everything is fine.
Week 10
What did you get done this week?
This week, I almost finished the implementation of Box widget creator tool. Generated box widget rellies on Bootstrap and this is not good, because it forces the user to import Bootsrap to his Web page, so, design part needs to be rearranged. Functionality is fully implemented, except the "decision making", when it comes to choosing the right icon to represent selected observed properties. This will be solved in consultance with the mentors.
I also spent a lot of time, familiarizing with the VistSOS data visualization framework and I'm still working on a plan how to integrate it into my project. So far, I thought of general concept of how the user input form is going to look like, how am I going to connect it to the istsos.widget.Chart class and what parts of VistSOS project I'm going to use.
What do you plan on doing next week?
Are you blocked on anything?
Currently, everything is fine.
Week 11
What did you get done this week?
This week, I implemented the form for generating chart widgets, there is still "build" method left to implement, but that will be quick, because whole logic is about dynamically creating the <istsos-chart> element. This implementation rellies on Vistsos - Data visualization framework and I prepared it for possible updates. If any change comes up, the only thing that needs to be updated is chart-types specification file and application does all the rest.
What do you plan on doing next week?
Are you blocked on anything?
Other than mentioned above, everything is fine.
Week 12 (last report)
The state of the project as it was before your GSoC?
Before my GSoC, IstSOS data couldn't be published on the Web, without hard coding from scratch. Developers would have had to learn about WA REST interface, to implement those requests in order to get the data, to define how they want to publish that data, etc...
The addition that your project brought to the software?
With istSOS Web API, developers can use powerfull istSOS core JavaScript library that covers most of the WA REST HTTP requests. Also, developers now have istSOS Web widget creator , at their disposal. This is an application that uses istSOS core Javascript library and offers the set of widget creator tools:
- Map - developers can easily build map widgets, that show sensor locations, last observation and date&time of the last observation for each sensor
- Box - developers can easily build boxes, that contain sensor name, date&time of the last observation, and last observation values for each observed property related to selected sensor
- Chart - integrated framework from the project - VistSOS - Data visualization framework. Developers can easily configure and build several types of interactive charts, that represent observation data in selected time span.
Links:
JavaScript Core - usage instructions:
https://github.com/WebPractice-LukaG/javascript-core#usage-instructions
Web widget creator - usage instructions:
https://github.com/WebPractice-LukaG/javascript-core#usage-instructions
Google Summer of Code 2016 - Reports
Who will use results of this project
Project output is targetet for:
- Users of the IstSOS Web application - data publishing
- Developers - further development using the istSOS JavaScript core library
- Students - interesting in using observation data for their workshops or similar.
Student's biography
My name is Luka Glušica and I'm a MSc student at the Faculty of Civil Engineering, Department of Geodesy and Geoinformatics, University of Belgrade, Serbia. Master's degree program, that I'm currently enrolled in, is oriented towards GIS (including programming), Web Cartography and Remote Sensing.
I'm familiar with Windows and Linux (Ubuntu, Linux Mint) operating system environments, including Microsoft Office (Word, Excel, Access) and Libre Office (Writter, Calc, Base) solutions, and I'm using them on daily basis.
During my studies, I've developed the set of skills listed below:
- Web development - HTML, CSS, JavaScript (Jquery, OpenLayers 3), Bootstrap
- Object oriented programming - Java, Python
- CAD solution - Certified Professional in AutoCAD 2014 2D Drafting
- GIS - QGIS, ArcGIS, Manifold System, Idrisi, SAGA
- Database systems - PostgreSQL/PostGIS
Some of my practice work: