VistSOS Data Visualization Framework

This wiki will be in continous change specially during the summer 2016. If you have any suggestion, complain or idea that you want to share, please write me felipe07 at gmail dot com.

Welcome. This is the wiki of Vistsos: The istSOS Data Visualization Framework project.

Introduction

"THE PURPOSE OF COMPUTING IS INSIGHT, NOT NUMBERS". RICHARD HAMMING.

Visualization is the human activity of representing by means of images phenomena of a different nature. Therefore, data visualization, is the activity of representing data obtained from many different sources with the ultimate goal of helping users to understand the patterns of information behind. The objective of this project is to create a customizable framework that will allow users to define and publish data visualizations charts linked with istSOS. The objective mentioned before imposes two main challenges: one of technical nature an the other more associated with the "effective" application of design.

Proposal summary

In order to extent the available characteristic offered by the istSOS server, I propose the use of an open source Javascript library for the implementation of VistSOS: the istSOS Data Visualization Framework. The proposed solution shall allow users of the istSOS ecosystem to configure, customize and include a wide set of chart, bar, grid and map visualization types in their websites. The framework will offer complete support for the connection of the visualization widgets with multiple data-exchange formats like CSV, TSV and JSON. Considering the real-time nature of the data sources the framework will support the generation of real-time visualization widgets specially crafted to improve the readiness to display data of this kind. The framework shall be implemented with robustness, reliability and customization as its core development principles. Recommended practices for software development like Version Control Systems use and continous testing will be followed.

Implementation details

• There are several candidates libraries for the implementation of the framework, many of them based on Javascript. Considering the goal and requirements of the project, one suitable candidate for the implementations of the framework can be the D3.js library which offers an important set of data visualization types both interactive and non-interactive.

• Besides the fact that D3.js is subject to the terms of an open license like BSD-3, two important points in favor of D3.js is its well documented API and its capacity of rendering of real-time data. For a detailed comparison of charting frameworks please refer to: [1].

• In order to improve the compatibility with the majority of browsers, standardized languages like HTML and CSS should be properly used. The proposed library D3.js support the generation of graphs in SVG format, supported by the majority of browsers that includes an SVG rendering engine. The manipulation of the browser’s DOM supported by D3.js is also tested against Firefox, Chrome (Chromium), Safari (WebKit), Opera and IE9 [2].

• The multiple support of data-interchange formats like CSV, TSV or JSON should be implemented with any pre-processing routine needed to normalize the incoming data. However, many charting frameworks support by default this formats and D3.js is not the exception.

Schedule

There is an initial schedule for the project which should be re-defined with the tutors during next weeks. Provisionally, I would like to share the activities and milestones defined by me chronologically ordered (also subject of change during coming weeks):

• Project planning with tutor:
  • Timeline adjustments
  • Definition of charts to be implemented (divided in 3 groups)
  • Definition of the technical limitations of the framework

• First prototype implementation:
  • Real-time data unsupported
  • Non-interactive widgets
  • Minimum customizable options: Color, size and position.
  • Implementation of
  • 1st group of charts
  • Hard-coded data

• Prototype evaluation

• Addition of feature:
  • Processing of CSV data
  • Processing of TSV data

• Prototype evaluation

• Addition of feature:
  • Processing of JSON data

• Prototype evaluation

• Second prototype implementation:
  • Implementation of 2nd group of charts
  • Interactive widgets
  • Customizable options: Color, size, position and scale

• Prototype evaluation

• Third prototype implementation:
  • Implementation of the 3rd group of charts
  • Real-time data supported

• Prototype evaluation

• Prototype evaluation

• Final product presentation

What new functionality this project brings

• The framework will provide a wide set of interactive and non-interactive data visualization charts, grids, plots and maps able to encode data coming from a network of sensors, external databases or files.

• The framework will allow the user to customize the color, scale, size or position of any available widget.

• Real-time data visualizations shall be available.

• The framework will support the processing of incoming data represented as CSV, TSV and JSON.

Who will use results of this project

Organizations and individuals interested in visualization of real time series provided by istSOS.

Student's Biography

I was born in Bogotá, Colombia. I have a bachellor in Systems Engineering from the District University of Bogotá and currently I'm doing a M.Sc. in Computer Science at Politecnico de Milano. I worked before in some companies doing software development. I learned from many kind and humble people ways to improve my coding skills to the point that now I am working on this exciting project. I'm interested in interdisciplinary projects.