Difference between revisions of "VistSOS Data Visualization Framework"

Revision as of 09:59, 5 May 2016

This wiki will be in continous change specially during the summer 2016. If you have any suggestion, complain or idea that you want to share, please write me felipe07 at gmail dot com.

Welcome. This is the wiki of Vistsos: The istSOS Data Visualization Framework project.

Introduction

Proposal summary

In order to extent the available characteristic offered by the istSOS server, I propose the use of an open source Javascript library for the implementation of VistSOS: the istSOS Data Visualization Framework. The proposed solution shall allow users of the istSOS ecosystem to configure, customize and include a wide set of chart, bar, grid and map visualization types in their websites. The framework will offer complete support for the connection of the visualization widgets with multiple data-exchange formats like CSV, TSV and JSON. Considering the real-time nature of the data sources the framework will support the generation of real-time visualization widgets specially crafted to improve the readiness to display data of this kind. The framework shall be implemented with robustness, reliability and customization as its core development principles. Recommended practices for software development like Version Control Systems use and continous testing will be followed.

Implementation details

There are several candidates libraries for the implementation of the framework, many of them based on Javascript. Considering the goal and requirements of the project, one suitable candidate for the implementations of the framework can be the D3.js library which offers an important set of data visualization types both interactive and non-interactive.

Besides the fact that D3.js is subject to the terms of an open license like BSD-3, two important points in favor of D3.js is its well documented API and its capacity of rendering of real-time data. For a detailed comparison of charting frameworks please refer to: [1].

In order to improve the compatibility with the majority of browsers, standardized languages like HTML and CSS should be properly used. The proposed library D3.js support the generation of graphs in SVG format, supported by the majority of browsers that includes an SVG rendering engine. The manipulation of the browser’s DOM supported by D3.js is also tested against Firefox, Chrome (Chromium), Safari (WebKit), Opera and IE9 [2].

The multiple support of data-interchange formats like CSV, TSV or JSON should be implemented with any pre-processing routine needed to normalize the incoming data. However, many charting frameworks support by default this formats and D3.js is not the exception.

About me

Schedule

There is an initial schedule for the project which should be re-defined with the tutors during next weeks. Provisionally, I would like to share the activities and milestones defined by me chronologically ordered (also subject of change during coming weeks):

Project planning with tutor:
- Timeline adjustments
- Definition of charts to be implemented (divided in 3 groups)
- Definition of the technical limitations of the framework

First prototype implementation:
- Real-time data unsupported
- Non-interactive widgets
- Minimum customizable options: Color, size and position.
- Implementation of
- 1st group of charts
- Hard-coded data

Prototype evaluation

Addition of feature:
- Processing of CSV data
- Processing of TSV data

Prototype evaluation

Addition of feature:
- Processing of JSON data

Prototype evaluation

Second prototype implementation:
- Implementation of 2nd group of charts
- Interactive widgets
- Customizable options: Color, size, position and scale

Prototype evaluation

Third prototype implementation:
- Implementation of the 3rd group of charts
- Real-time data supported

Prototype evaluation

Prototype evaluation

Final product presentation

Competitors

What new functionality this project brings

The framework will provide a wide set of interactive and non-interactive data visualization charts, grids, plots and maps able to encode data coming from a network of sensors, external databases or files.

The framework will allow the user to customize the color, scale, size or position of any available widget.

Real-time data visualizations shall be available.

The framework will support the processing of incoming data represented as CSV, TSV and JSON.

Who will use results of this project

Organizations and individuals interested in visualization of real time series provided by istSOS.

Student's Biography

I was born in Bogotá, Colombia. I have a bachellor in Systems Engineering from the District University of Bogotá and currently I'm doing a M.Sc. in Computer Science at Politecnico de Milano. I worked before in some companies doing software development however since my mom bought and old IBM computer and she brought it home I been finding ways to crash computers and write bugs, luckily I learned from many nice and humble people ways to improve my coding skills to the point that now I am working in this exciting project. I'm interested in interdisciplinary projects so I have the opportunity to learn not only new stuff but also new points of view, about everything.

@@ Line 72: / Line 72: @@
 == Competitors ==
-Proprietary applications like: [http://www.arc2earth.com/ Arc2Earth] have similar functionality. Several (mostly proprietary) generators for Google Earth tiles were mentioned on http://www.ogleearth.com/.
-Microsoft Research project [http://research.microsoft.com/mapcruncher/ MapCruncher] also generates tiles, but with metadata only for their MS Virtual Earth.
-[http://www.zoomify.com/ Zoomify] could be now used as tiling program for SuperOverlay in GoogleEarth with free [http://www.zoomify.com/express.htm ZoomifyerEZ] and generator [http://www.staremapy.cz/zoomifykml/ ZoomifyKML] (which I created as a part of my Master Thesis).
 == What new functionality this project brings ==
@@ Line 90: / Line 85: @@
 == Who will use results of this project ==
-Several organizations and individuals who handle processing of maps.
+Organizations and individuals interested in visualization of real time series provided by istSOS.
-Institutions which will use the 'gdal2tiles' utility from this project, as I am cooperating with them:
-* Moravian Library in Brno, Czech Republic
-* Historical institute of Czech Academy of Sciences
-* Czech National Library
-* National Library of Scotland
-* University of Connecticut Library (with their grant [http://www.rlg.org/en/page.php?Page_ID=20522#article1 "Building a Globally Distributed Historical Sheet Map Set of Austro-Hungarian Topographic Maps, 1877-1914"], cooperating with New York Public Library, University of Connecticut Library, and Library of the American Geographical Society. It is a grant about building a scalable workflow and toolset for map libraries to add cartographic content to the Web.)
 == Student's Biography ==
-My name is Klokan Petr Přidal, I have finished Master degree in Applied Informatics at [http://www.muni.cz/ Masaryk University in Brno], now I am in the first semester of postgraduate study of Cartography at [http://www.cvut.cz/ Czech Technical University in Prague]. My Master thesis subject was [http://is.muni.cz/th/39656/fi_m/annotation_english.txt "Processing and Digital Publishing of Historical Documents"] ([http://www.staremapy.cz/th.pdf PDF of thesis in Czech]).
+I was born in Bogotá, Colombia. I have a bachellor in Systems Engineering from the District University of Bogotá and currently I'm doing a M.Sc. in Computer Science at Politecnico de Milano. I worked before in some companies doing software development however since my mom bought and old IBM computer and she brought it home I been finding ways to crash computers and write bugs, luckily I learned from many nice and humble people ways to improve my coding skills to the point that now I am working in this exciting project. I'm interested in interdisciplinary projects so I have the opportunity to learn not only new stuff but also new points of view, about everything.
-I am familiar with open-source projects. I have experience with Linux/Solaris system administration. I like Python language, but I have also used other languages (C language, PHP).
-I have supplied some patches into Gnumeric ([http://bugzilla.gnome.org/show_bug.cgi?id=330129 #330129], [http://bugzilla.gnome.org/show_bug.cgi?id=334257 #334257]).
-I founded a http://mplayerosx.sf.net/ project and I also won a competition in programming application for Mac using http://dictosx.sf.net/ (Objective-C).
-My primary desktop is Gnome and Linux, but I also use Mac, Windows and Solaris. I did programming for all of these platforms (mostly smaller projects).
-I am interested in historical cartography, I have several presentations about old maps and free software. I made a czech web page about it at http://www.staremapy.cz/ and registered domain http://www.oldmapsonline.org/ (which is not in use yet).
 [[Category:Google Summer of Code]]