Difference between revisions of "IstSOS Web API"
| Line 86: | Line 86: | ||
===istSOS Online Web Widget Creator=== | ===istSOS Online Web Widget Creator=== | ||
| + | ==== <i>Google Summer of Code 2016</i> ==== | ||
[[File:General.png|thumb|right|alt=istsos web widget creator]] | [[File:General.png|thumb|right|alt=istsos web widget creator]] | ||
<b>IstSOS Web widget creator</b> an Online Web Application for building featurerich web widget that will be easily embeddable into existing web pages. It was build upon istSOS JavaScript core library and small istSOS widget JavaScript library. [http://getbootstrap.com/ <b>Bootstrap</b>] and [https://jquery.com/ <b>JQuery</b>] were used for general design of the application. Bootstrap plugin - [https://eonasdan.github.io/bootstrap-datetimepicker/ <b>datetimepicker</b>] was also used for helping with date&time inputs. <br/><br/> | <b>IstSOS Web widget creator</b> an Online Web Application for building featurerich web widget that will be easily embeddable into existing web pages. It was build upon istSOS JavaScript core library and small istSOS widget JavaScript library. [http://getbootstrap.com/ <b>Bootstrap</b>] and [https://jquery.com/ <b>JQuery</b>] were used for general design of the application. Bootstrap plugin - [https://eonasdan.github.io/bootstrap-datetimepicker/ <b>datetimepicker</b>] was also used for helping with date&time inputs. <br/><br/> | ||
| Line 107: | Line 108: | ||
User is allowed to select multiple procedures and multiple observed properties, and also to select time interval in which chart data will be shown.</br> | User is allowed to select multiple procedures and multiple observed properties, and also to select time interval in which chart data will be shown.</br> | ||
There are several chart types at disposal, like line, multivariable, overview and histogram chart. More info about VistSOS project[https://wiki.osgeo.org/wiki/VistSOS_Data_Visualization_Framework page] | There are several chart types at disposal, like line, multivariable, overview and histogram chart. More info about VistSOS project[https://wiki.osgeo.org/wiki/VistSOS_Data_Visualization_Framework page] | ||
| + | ==== <i>Google Summer of Code 2017</i> ==== | ||
| + | This year, changes in application design and dev structure have been introduced. Here is the list: | ||
| + | <ol> | ||
| + | <li> Application is rewritten using ReactJS, ES6, Babel transpiler and Webpack 2. | ||
| + | <li> Dependencies are handled by ES6 import/export modules logic. | ||
| + | <li> Simple design changes are made. | ||
| + | <li> Configuration is simplified. | ||
| + | <li> Webpack handles application build. | ||
| + | <li> Chart widget is temporarly down due to VistSOS integration problems. | ||
| + | </ol> | ||
| + | <hr/> | ||
== Schedule == | == Schedule == | ||
Revision as of 03:32, 29 August 2017
Currently in progress...
Introduction
istSOS[1][2] is an OGC SOS server implementation written in Python. IstSOS allows managing and dispatching observations from monitoring sensors according to the Sensor Observation Service standard.
istSOS Web API is one of the accepted project ideas for Google Summer of Code 2016 and it is currently being improved as part of Google Summer of Code 2017. Project name this year is known as Improving IstSOS Web API and finalizing IstSOS Javascript Core Library.
Improving IstSOS Web API and finalizing IstSOS Javascript Core Library is focused on two parts:
- rewritting IstSOS JavaScript core library to ES6, modernize project build system logic(webpack and babel transpiler) and publish the library to npm.
- rewritting IstSOS Web Widget Creator, using React JavaScript library, improving existing functionality and introducing new tools.
Development process can be monitored at the following links:
- https://github.com/istSOS/javascript-core/ - Project repository
- https://github.com/Luka-G/javascript-core/commits/new-js-core?author=Luka-G - Forked repository(use this repository, for now)
- https://github.com/istSOS/web-widget-creator - Project repository
- https://github.com/Luka-G/web-widget-creator/commits/new-wwc?author=Luka-G - Forked repository(use this repository, for now)
Student working on improvements of the istSOS Web API: Luka Glušica
Mentors: Milan Antonovic, Luca Delucchi
[1] https://wiki.osgeo.org/wiki/IstSOS
[2] http://istsos.org
Implementation details
Libraries
Openlayers 3
OpenLayers 3 is an open source JavaScript library for displaying map data in web browsers.
OL3 makes it easy to put a dynamic map in any web page. It can display map tiles, vector data and markers loaded from any source. OpenLayers has been developed to further the use of geographic information of all kinds. It is completely free, Open Source JavaScript, released under the 2-clause BSD License (also known as the FreeBSD).
Map tool of the istSOS Web widget creator, found OL3 very useful for generating map widgets containing sensor location data.
Moment
Moment is a lightweight JavaScript date library for parsing, validating, manipulating, and formatting dates.
Moment was designed to work both in the browser and in Node.js. All code should work in both of these environments, and all unit tests are run in both of these environments. Currently the following browsers are used for the ci system: Chrome on Windows XP, IE 8, 9, and 10 on Windows 7, IE 11 on Windows 10, latest Firefox on Linux, and latest Safari on OSX 10.8 and 10.11.
Moment has proven useful in converting input dates and time to istSOS allowed formats. Also, several methods use Moment to compare two date&time strings and return older or newer date.
istSOS JavaScript core
Google Summer of Code 2016
istSOS JavaScript core library exposes communication with the istSOS Rest API interface. Idea was to identify all rest API HTTP requests and implement them as methods of newly implemented classes. Most important request for developers and users is getObservations request, because it returns actual measurements collected by the sensors and istSOS Javascript core library is focused on several ways to get observation data from the server. Class structure describes the conceptual logic of the istSOS core library and it's based on couple of "container" classes. Every class has a defined namespace "istsos" before it's actual name (example: istsos.Server) and library starts with IstSOS class as a container of Server instances.
- IstSOS class - contain list of Server instances
- Server class - contain list of Service instances
- Service class - contain Procedure instance, VirtualProcedure instance, Offering instance, ObservedProperty instance, DataQuality instance and UnitOfMeasure instance lists.
There are classes like Database and Configuration, that are related to Server class as default and to Service class as service specific.
Rest of the classes:
- Date clas - used for converting time and date integer inputs to valid istSOS format.
- ProcedureBase class - class from which Procedure and VirtualProcedure classes inherit properties and methods.
- Output class - used to simplify use of Procedure and VirtualProcedure instances. Holds ObservedProperty and UnitOfMeasure instances.
Other than mentioned, istSOS JavaScript core has a special way of event handling with lots of custom event types like (PROCEDURE, NEW_PROCEDURE, SERVICES, DELETE_SERVICE etc). It has 2 global methods for interacting with tha data that came asynchronously from the server:
- istsos.on()
- istsos.once()
Both methods, take event type as a parameter and the data is to be manipulated through callback function. Here is the getObservations usage example, which also explains event handling in istSOS JavaScript core library.
Google Summer of Code 2017
This year, changes in library design and class structure have been introduced. Here is the list:
- Closure library has been removed from the project. To compensate some of the functionalities of the closure library, that has been used, new functionalities were introduced.
- Whole library is rewritten to ES6
- Dependencies are handled by ES6 import/export modules logic.
- istsos.HttpAPI is a small API, designed to handle HTTP requests and promisify them. API methods:
- get (event, opt_config)
- put (event, opt_config)
- post (event, opt_config)
- delete (event, opt_config)
- istsos.EventEmitter class - new event handling system is implemented and with it, major issue of the library is covered. Every class, that uses HTTP requests, inherits from istsos.EventEmitter class. Event handling methods:
- fire (event, data)
- on (event, callback)
- once (event, callback)
- off (event, callback)
- unlistenAll ()
- Webpack was integrated to handle the library build
- Basic HTTP authentication was also added as a usage possibility.
istSOS Online Web Widget Creator
Google Summer of Code 2016
IstSOS Web widget creator an Online Web Application for building featurerich web widget that will be easily embeddable into existing web pages. It was build upon istSOS JavaScript core library and small istSOS widget JavaScript library. Bootstrap and JQuery were used for general design of the application. Bootstrap plugin - datetimepicker was also used for helping with date&time inputs.
IstSOS Web widget creator consists of three tools:
- Map tool
- Box tool
- Chart tool
Map tool uses OpenLayers 3 to build a map with sensor locations as Point features, showing the last observations, date and time of the last observation and name of related sensor(procedure).
Upon selecting the map tool in the menu, user is offered to select service, offering, procedure/procedures and observed property and to set up other configurable properties(See Usage instructions). Upon page load, list of services is collected from the server. When user selects a service, list of offerings based on selection is collected from the server and so on in the order mentioned above. User is allowed to select multiple procedures, but is limited to select only one observed property, which differs from the box and chart widgets(below).
As a result, in the preview section, Map widget is shown and in the code section, generated embedded code is shown, which user imports to his own Web page in order to display built widget.
Interesting OL3 functionality, called ol.source.Cluster, is used to handle overlaping features(sensors). Whenever two or more features overlap, they join into one and icon show number and names and names of all sensors that are clustered.
Box tool uses Bootstrap for responsivity and design control.
Upon selecting the box tool in the menu, user is offered to select service, offering, procedure and observed property/observed properties and to set up other configurable properties(See Usage instructions). User is allowed to select only one procedure and multiple observed properties, that are related to selected procedure.
Resulting box widget contains name of the selected sensor, date and time of the last observation, and observation values of each selected observed property.
Also, user is offered to choose a layout for output widget, which can be horizontal and vertical.
Chart tool represents integration of another project - VistSOS - Data visualization framework(See VistSOS).
Upon selecting the map tool in the menu, user is offered to select service, offering, procedure/procedures and observed property/observed properties and to set up other configurable properties. Chart tool is specific, because usage of it requires additional configuration.(See Usage instructions).
User is allowed to select multiple procedures and multiple observed properties, and also to select time interval in which chart data will be shown.
There are several chart types at disposal, like line, multivariable, overview and histogram chart. More info about VistSOS projectpage
Google Summer of Code 2017
This year, changes in application design and dev structure have been introduced. Here is the list:
- Application is rewritten using ReactJS, ES6, Babel transpiler and Webpack 2.
- Dependencies are handled by ES6 import/export modules logic.
- Simple design changes are made.
- Configuration is simplified.
- Webpack handles application build.
- Chart widget is temporarly down due to VistSOS integration problems.
Schedule
Google Summer of Code 2016
Development of istSOS Web API consists of two parts:
- JavaScript Core Library
- Web App for creating Web Widgets, based on previosly developed core library
Working schedule presented below, is arranged in consultance with the mentors and other students working on the istSOS projects. The planning isn't strict for "in-between" deadlines, because of varying nature of our projects, but it is strict for two major parts, mentioned above (June 20 and August 22).
Timeline planning for istSOS JavaScript Core
Task From To Requirements analysis May 23 May 30 Coding May 30 June 13 Testing, bug fix, release June 13 June 20
Timeline planning for istSOS Online Web Widget Creator
Task From To Design and Requirements analysis June 20 June 27 Coding June 27 July 18 Coding(VistSOS Web Component Integration) July 18 August 8 Coding, Testing, Bug Fixing, Documentation August 8 August 22
Google Summer of Code 2017
Working schedule for improvement of the IstSOS Web API:
Timeline planning
Task From To Rewritting library to ES6 and integrating project structure into webpack May 30 June 11 Solving event handling issue and implementing authentication June 11 June 18 Documentation and testing June 18 June 26 First evaluation period June 26 June 30 Preparing new project structure for IstSOS Web Wdiget Creator June 30 July 9 Design new IstSOS Web Widget Creator with new functionalities July 9 July 16 Start rewritting and reimplementing existing tools, using React July 16 July 24 Second evaluation period July 24 July 28 Finish with implementation of existing tools July 28 August 6 Implement new functionalities and test them August 6 August 13 Test and fix, both library and web widget creator, update documentation, publish library to npm August 13 August 21 Final evaluation period August 21 August 29
Reports - Google Summer of Code 2016
Week 1
What did you get done this week?
This week, I’ve been indentifying all the admin HTTP requests to the server, that are targeting the istSOS RESTful API. By identifying, I mean writting down:
- URL
- type of request (GET, POST, PUT, DELETE)
- response data
- request payload
Also, I’ve started designing the class structure, for the purpose of developing JavaScript core library.
All of the mentioned work, I’m doing in partnership with a GsoC student Florin-Daniel Cioloboc, who’s working on an Java core library for istSOS and our work overlaps in this section.
What do you plan on doing next week?
Next week, I will be finishing the class structure and begin with the actual development/coding of JavaScript core library. I will be also creating some test pages along the way, to check for possible errors.
Are you blocked on anything?
Currently, everything is going according to plan.
Week 2
What did you get done this week?
This week, I've been continuosly working on class structure design for JavaScript core library and also started commiting some code. The was to implement IstSOS, Server and Database classes, but let's say that 90% of it is done.
What do you plan on doing next week?
Next week, I will be completing the class structure and will try to implement more classes like Service, Offering, Procedure.
Are you blocked on anything?
Actually, the reason why 90% of mentioned classes were done, is that I was unable to store JSON response object into an outside variable. For the information, I've been using Google Closure library for executing AJAX requests.
Week 3
What did you get done this week?
This week, I’ve implemented Configuration, Service, Offering, ObservedProperty, DataQuality, UnitOfMeasure and Date classes. Service class is partially implemented, because it depends on other classes that should be implemented soon. Also, I’ve created an html and JavaScript file for testing and I’ve already tested some of the previously implemented classes, but the new ones are about to be tested. Appart from mentioned, I’ve updated OSgeo wiki page with timeline and weekly reports, also you can take a look at README file in my GitHub forked repository, because I’ve started with creating API documentation for the core library I’ve been working on.
What do you plan on doing next week?
Next week, I plan to implement last two classes (Procedure, VirtualProcedure) and to do all the testing, bug fixes and refining the code and documentation. Hopefully, by June 20-th, I will be releasing istSOS JavaScript core library.
Are you blocked on anything?
Currently, everything is fine. I solved the issue with AJAX requests in consultance with my mentors.
Week 4
What did you get done this week?
I have implemented all the classes and did the testing of all the GET requests, which can be checked if you take a lok at istsos-test.html and istsos-test.js ( GitHub forked repository).
I have also updated the GitHub wiki page with the API documentation and the class structure of the library.
What do you plan on doing next week?
I plan to test all requests (POST/PUT/DELETE). Hopefully, this part will go easier and quicker, because I already fixed most of the minor problems during GET request tests.
Also, I will be updating documentation along the way.
We have regular meetengs with our mentors every Tuesday, so If everything goes as planned, I will prepare the first version of a library until then.
Are you blocked on anything?
Currently, everything is going according to plan.
Week 5
What did you get done this week?
It's a midterm evaluation period and I did a lot of testing of the library, there is a test html and test js on my GitHub forked repository. Also, I've been updating the API documentation and Class structure on the GitHub wiki page.
Next part of my job is to develop JavaScript Framework based on previously developed library, so I started thinking about how it should be implemented, how should interface look like etc.
What do you plan on doing next week?
I plan to finish the documentation in the next few days, do some more testing of a library and generate a compiled version of a library, so other people can use it easily.
Also, I will work on conceptual design of JavaScript Framework, I will be working together with GSoC student Felipe Poveda, because I need to integrate his implementation into my project and he needs to use my library for his own project.
Are you blocked on anything?
Currently, everything is fine.
Week 6
What did you get done this week?
I've been working on a design of the JavaScript Framework. No coding this week, more thinking and drawing on a piece of paper. Also, I started writing into a sreadsheet, every functionallity that should be implemented with description, so I can better organize my self, when the coding starts.
What do you plan on doing next week?
I plan to finish conceptual design and start implementing/coding the features. Of course, updates&fixes of the previously developed library along the way, is inevitable.
Are you blocked on anything?
Currently, everything is fine.
Week 7
What did you get done this week?
I've been designing home page, with some of the tools (without functionality, just visual aspect) . Usage of Bootstrap helped with the page repsonsivness. Code can be checked and tested here.
What do you plan on doing next week?
Next week, I plan to finish with design and implement a lot of tools, like Map creator tools, Admin tools for creating, updating and deleting data etc. I will be thinking and adding some new tools allong the way. Of course, fixing of design errors may be necessary, depending on the feature requirements.
Are you blocked on anything?
Currently, everything is fine.
Week 8
What did you get done this week?
This week, in consultance with my mentors, I've reduced number of tools that I've had planned to implement, because they seem irrelevant. Also, I changed approach a little bit, so I needed to start over with implementation. New plan is to develop small library for creating widgets. So far, I have implemented istsos.widget.Widget base class and partially istsos.widget.Map class. Code can be checked here.
What do you plan on doing next week?
Next week, I plan to improve Widget base class and finish the Map class implementation. Also, I will start with developing Box class and working on a way to integrate this project as a Chart class.
Are you blocked on anything?
Currently, everything is fine.
Week 9
What did you get done this week?
This week, I finished with the implementation of Map creator tool. Of course, further improvements will be made along the way, but the main functionality is covered and tested. I also started with the implementation of a Box creator tool. Also I've been updating and fixing the istsos core library with some changes regarding the getObservations method and compatibility issues with Internet Explorer 11. All variations of getObservations method can now pass multiple procedures as parameter. Code can be checked at following links:
What do you plan on doing next week?
Are you blocked on anything?
Currently, everything is fine.
Week 10
What did you get done this week?
This week, I almost finished the implementation of Box widget creator tool. Generated box widget rellies on Bootstrap and this is not good, because it forces the user to import Bootsrap to his Web page, so, design part needs to be rearranged. Functionality is fully implemented, except the "decision making", when it comes to choosing the right icon to represent selected observed properties. This will be solved in consultance with the mentors.
I also spent a lot of time, familiarizing with the VistSOS data visualization framework and I'm still working on a plan how to integrate it into my project. So far, I thought of general concept of how the user input form is going to look like, how am I going to connect it to the istsos.widget.Chart class and what parts of VistSOS project I'm going to use.
What do you plan on doing next week?
Are you blocked on anything?
Currently, everything is fine.
Week 11
What did you get done this week?
This week, I implemented the form for generating chart widgets, there is still "build" method left to implement, but that will be quick, because whole logic is about dynamically creating the <istsos-chart> element. This implementation rellies on Vistsos - Data visualization framework and I prepared it for possible updates. If any change comes up, the only thing that needs to be updated is chart-types specification file and application does all the rest.
What do you plan on doing next week?
Are you blocked on anything?
Other than mentioned above, everything is fine.
Week 12 (last report)
The state of the project as it was before your GSoC?
Before my GSoC, IstSOS data couldn't be published on the Web, without hard coding from scratch. Developers would have had to learn about WA REST interface, to implement those requests in order to get the data, to define how they want to publish that data, etc...
The addition that your project brought to the software?
With istSOS Web API, developers can use powerfull istSOS core JavaScript library that covers most of the WA REST HTTP requests. Also, developers now have istSOS Web widget creator , at their disposal. This is an application that uses istSOS core Javascript library and offers the set of widget creator tools:
- Map - developers can easily build map widgets, that show sensor locations, last observation and date&time of the last observation for each sensor
- Box - developers can easily build boxes, that contain sensor name, date&time of the last observation, and last observation values for each observed property related to selected sensor
- Chart - integrated framework from the project - VistSOS - Data visualization framework. Developers can easily configure and build several types of interactive charts, that represent observation data in selected time span.
Links:
JavaScript Core - usage instructions:
https://github.com/WebPractice-LukaG/javascript-core#usage-instructions
Web widget creator - usage instructions:
https://github.com/WebPractice-LukaG/javascript-core#usage-instructions
Reports - Google Summer of Code 2017
Week 1
What did you get done this week?
This week, I've been preparing the project structure for transitioning from old version of the istsos core library to the new one. I've installed dev tools(like npm and webpack), that are going to be used in development process and did some initial configuration of those tools. I've also prepared source files from the old version, that are going to be refactored in the next weeks.
What do you plan on doing next week?
Next week, I will refactor as much classes as I can. Following set of changes are going to be introduced:
- ES6 syntax will be used
- Class constructors will be simplified a lot
- Code will be refined.
I will also, along the way, work on the major issue of the library - Event Handling.
Are you blocked on anything?
Currently, everything is going according to plan.
Week 2
What did you get done this week?
This week, I've refactored and rewritten ServerContainer, Date, Configuration, EventType, Events and Database classes. I've been also exploring possible solutions to the Event Handling problem, there was some encouraging progress.
What do you plan on doing next week?
Next week, I will continue with refactoring and rewritting remaining classes. Hopefully I will fix the EventHandling issue and start working on Basic HTTP authentication integration.
Are you blocked on anything?
Currently, everything is going according to plan.
Week 3
What did you get done this week?
This week I rewritten and refactored all of the remaining classes to ES6. I've also added a new class - EventEmitter, which introduces a simple event handling system.
What do you plan on doing next week?
The plan for the next week, until evaluation:
- Update documentation (Wiki, gh-pages for javascript-core)
- Integrate EventEmitter into all of the classes and implement authentication
- Update webpack config in order to adopt the new changes
- Testing
Are you blocked on anything?
Currently, everything is going according to plan.
Week 4
What did you get done this week?
This week, I dealt with next set of tasks:
- Removed closure library, to improve efficiency and make webpack easier to handle the bundling
- Integrated EventEmitter into all classes as a super class.
- Added authentication logic to the istsos core
- Updated Wiki page
- Created API documentation for the library
What do you plan on doing next week?
The plan for the next week, until evaluation: Next week, I will continue to test and refine the library, if needed and to prepare it for publishing, although this is planned for the end of the project.
Are you blocked on anything?
Currently, everything is going according to plan. Sorry for a slight delay with the documentation updates, I have had some private stuff to handle, so I tried to compensate the missing work with couple of 'no sleep' nights.
Who will use results of this project
Project output is targetet for:
- Users of the IstSOS Web application - data publishing
- Developers - further development using the istSOS JavaScript core library
- Students - interesting in using observation data for their workshops or similar.
Student's biography
My name is Luka Glušica and I'm a MSc student at the Faculty of Civil Engineering, Department of Geodesy and Geoinformatics, University of Belgrade, Serbia. Master's degree program, that I'm currently enrolled in, is oriented towards GIS (including programming), Web Cartography and Remote Sensing.
I'm familiar with Windows and Linux (Ubuntu, Linux Mint) operating system environments, including Microsoft Office (Word, Excel, Access) and Libre Office (Writter, Calc, Base) solutions, and I'm using them on daily basis.
During my studies, I've developed the set of skills listed below:
- Web development - HTML, CSS, JavaScript (Jquery, OpenLayers 3), Bootstrap
- Object oriented programming - Java, Python
- CAD solution - Certified Professional in AutoCAD 2014 2D Drafting
- GIS - QGIS, ArcGIS, Manifold System, Idrisi, SAGA
- Database systems - PostgreSQL/PostGIS
Some of my practice work: