Cluj München 1140km

Gânduri clujene din Bavaria

München - Munich - Monaco di Baviera

12. martie 2014 18:46
by skorpionking
0 Comentarii

Big Data and Data Visualization - tools for visual quality

12. martie 2014 18:46 by skorpionking | 0 Comentarii

This is gonna be again a long post, so hold on folks :-) During my software engineering work in the last month I had to do something new, analyze some tools for big data visualization, somehow something new for me which I was only using some charts controls from Office, SQL Reporting, Visual Studio Reports and some other 3rd party components (ChartFX, Telerik, Infragistics).

Big Data is a powerful discovery tool for companies seeking to glean new insights. But without the right framework for understanding it, much of that knowledge may go unrecognized. Oftentimes, it’s data visualization that allows Big Data to unleash its true impact. Data visualization is the necessary ingredient in bringing the power of Big Data to the mainstream.
Progressive organizations today are using a wide array of data visualization (dataviz) tools to ask better questions of their data – and thus make better business decisions. Of course, many companies have long been using rudimentary dataviz tools, such as a Microsoft Excel graph or chart. But that’s unlikely to promote true data discovery. On the contrary, at the most advanced companies, the employees are doing a great deal more than creating simple graphs, bar charts, and pivot tables. Employees are interacting with their data, and learning new things about their businesses in the process.
Two factors have contributed to this momentum for data visualization: the rise of Big Data and the growing public awareness of its power. Today more than ever, professionals are being asked to argue their cases and make their decisions based on data. A new, data-oriented mind-set is permeating the business world.
But that push outside IT circles means that many non-technical professionals must now produce and comprehend insights from Big Data. Visualization can help, and a raft of new tools makes that possible. Microsoft, Oracle, IBM, Cognos, SAS, and other enterprise BI (business intelligence) oriented companies are still providing great tools and solutions but they are no longer the only game in town. Today, an organization need not spend hundreds of thousands or millions of dollars to get going with dataviz. These new tools have become progressively more powerful and democratic over the last decade. Long gone are the days in which IT needed to generate reports for non-technical employees. They have made it easier than ever to for employees to quickly discover new things in increasingly large datasets.

"By 2015, enterprise buyers of BI platforms will predominantly purchase platforms that support both strong and broad business-user-accessible data discovery capabilities and IT-driven enterprise features for data reuse, governance, security and scalability.
By 2015, the shifting tide of BI platform requirements, moving from reporting-centric to analysis-centric, will mean the majority of BI vendors will make governed data discovery capabilities an expansion of, and the prime competitive capability for, their BI platform offerings.
By 2015, smart data discovery,  which includes natural-language query and search, automated, prescriptive advanced analytics and interactive data discovery capabilities, will be the most in-demand BI platform user experience paradigm, enabling mainstream business consumers to get insights (such as clusters, segments, predictions, outliers and anomalies) from data.
By 2016, 25% of net-new BI and analytics platform deployments will be in the form of subscriptions to cloud services."

- Gartner Magic Quadrant for Business Intelligence and Analytics Platforms  by Rita L. Sallam, Joao Tapadinhas, Josh Parenteau, Daniel Yuen and Bill Hostmann, 20 February 2014

 A picture is worth a thousand words, but creating cool infographics can be time-consuming.  While not everyone can make infographics from scratch, there are tools available on the Web that will help you create your very own infographics. Here are some amazing tools to make it easier.


It isn't graphically flexible, but Excel is a good way to explore data: for example, by creating 'heat maps' like this one

You can actually do some pretty complex things with Excel, from 'heat maps' of cells to scatter plots. As an entry-level tool, it can be a good way of quickly exploring data, or creating visualizations for internal use, but the limited default set of colours, lines and styles make it difficult to create graphics that would be usable in a professional publication or website. Nevertheless, as a means of rapidly communicating ideas, Excel should be part of your toolbox.

Excel comes as part of the commercial Microsoft Office suite

Many Eyes

Many Eyes was developed by IBM

Many Eyes is the unique data visualization tool put out by IBM. They offer a whole slew of categories to explore for customized data topics. Some of the most popular examples are featured on the home page, many of which are user-created. To create a set you’ll need to first organize your data. This can be practically anything, but should be something relatable and easy to display. If you want to publicly store the visualizations you can create a free account on the service. There are a ton of features available to members plus the added benefit and security of storing personal data sets. If you get lost spend some time browsing the FAQ/Tour page to learn a bit more about the Many Eyes’ interface.

 Google Charts


Google Charts has an excellent selection of tools available

The seminal charting solution for much of the web, Google Charts is highly flexible and has an excellent set of developer tools behind it. It's an especially useful tool for specialist visualizations such as geocharts and gauges, and it also includes built-in animation and user interaction controls.

Tableau Public

How fast do successful tech companies grow? The Wall Street Journal posted this visualization that compares the performance of 100 fast growing software companies.

Used by more than 17,000 organizations worldwide, Tableau's award-winning software delivers fast analytics and rapid-fire business intelligence. Create visualizations and dashboards in minutes, then share in seconds. What is the result? You get answers from data quickly, with no programming required.

It is a free application for the Windows computer that brings data to life. You can create and share interactive charts and graphs, stunning maps, live dashboards and fun applications in minutes. Anyone can do it, it’s that easy. makes data visualization as simple as it can be is built with social networking features in mind to connect members all around the world. Designers are able to submit their own projects on data visualization and infographics into their site gallery. The showcase can be broken down and sorted into further categories like Food, Environment, Technology, etc. If you check out their labs page it includes some fantastic links about what the team is building. The ideal goal is to offer an interface for creating dynamic infographics directly within your browser.


 This handy JavaScript library offers a range of data visualization options

This handy JavaScript library offers a wide range of data visualization options which are rendered using SVG. This makes for a flexible approach that can easily be integrated within your own web site/app code, and is limited only by your own imagination. That said, it's a bit more hands-on than some of the other tools featured here (a victim of being so flexible), so unless you're a hardcore coder, you might want to check out some of the more point-and-click orientated options first!

jQuery Visualize

 jQuery Visualize Plugin is an open source charting plugin

Written by the team behind jQuery's ThemeRoller and jQuery UI websites, jQuery Visualize Plugin is an open source charting plugin for jQuery that uses HTML Canvas to draw a number of different chart types. One of the key features of this plugin is its focus on achieving ARIA support, making it friendly to screen-readers. It's free to download from this page on GitHub.

Better World Flux

 Making the ugly beautiful - that's Better World Flux

Better World Flux is an interactive information graphic encompassing important ideas around the world. Select a single country or countries of your choosing, followed by indicators such as life expectancy or access to water.

The refined condition of BWF’s design comes as no surprise. The amount of data available is so pristine; you can easily track the global improvements in society over the past 50 years. Though one will admit the data graph is originally very difficult to understand. Try playing around with just your country to start getting comfortable with the interface.


 Nothing can be more interesting than our history on this earth. There have been a lot of events over the past 10 or 20 years – let alone a decade or century! Dipity is a wonderful tool to create and externally embed custom interactive timelines. You can pin markers on important dates to include photos, links, audio/video, and other forms of media.

The service requires that you sign up for an account before creating timelines. They do offer a free plan with the option to upgrade to premium plan at a later date. Luckily the most popular member timelines are offered public, so you can easily sort through an exciting laundry list of dynamic timelines.


You can render some amazing diagrams with D3

D3.js is a JavaScript library that uses HTML, SVG, and CSS to render some amazing diagrams and charts from a variety of data sources. This library, more than most, is capable of some seriously advanced visualizations with complex data sets. It's open source, and uses web standards so is very accessible. It also includes some fantastic user interaction support.

JavaScript InfoVis Toolkit

JavaScript InfoVis Toolkit includes a handy modular structure

A fantastic library written by Nicolas Belmonte, the JavaScript InfoVis Toolkit includes a modular structure, allowing you to only force visitors to download what's absolutely necessary to display your chosen data visualizations. This library has a number of unique styles and swish animation effects, and is free to use (although donations are encouraged).

Visualize Free

Make visualizations for free!

Visualize Free is a hosted tool that allows you to use publicly available datasets, or upload your own, and build interactive visualizations to illustrate the data. The visualizations go well beyond simple charts, and the service is completely free plus while development work requires Flash, output can be done through HTML5.


jpGraph is a PHP-based data visualization tool

If you need to generate charts and graphs server-side, jpGraph offers a PHP-based solution with a wide range of chart types. It's free for non-commercial use, and features extensive documentation. By rendering on the server, this is guaranteed to provide a consistent visual output, albeit at the expense of interactivity and accessibility.


Highcharts has a huge range of options available

Highcharts is a JavaScript charting library with a huge range of chart options available. The output is rendered using SVG in modern browsers and VML in Internet Explorer. The charts are beautifully animated into view automatically, and the framework also supports live data streams. It's free to download and use non-commercially (and licensable for commercial use). You can also play with the extensive demos using JSFiddle.


Crossfilter in action: by restricting the input range on any one chart, data is affected everywhere. This is a great tool for dashboards or other interactive tools with large volumes of data behind them

As we build more complex tools to enable clients to wade through their data, we are starting to create graphs and charts that double as interactive GUI widgets. JavaScript library Crossfilter can be both of these. It displays data, but at the same time, you can restrict the range of that data and see other linked charts react.


A powerful free software environment for statistical computing and graphics, R is the most complex of the tools listed here

How many other pieces of software have an entire search engine dedicated to them? A statistical package used to parse large data sets, R is a very complex tool, and one that takes a while to understand, but has a strong community and package library, with more and more being produced. The learning curve is one of the steepest of any of these tools listed here, but you must be comfortable using it if you want to get to this level.


CartoDB provides an unparalleled way to combine maps and tabular data to create visualizations

CartoDB is a must-know site. The ease with which you can combine tabular data with maps is second to none. For example, you can feed in a CSV file of address strings and it will convert them to latitudes and longitudes and plot them on a map, but there are many other users. It's free for up to five tables; after that, there are monthly pricing plans.


Create animated visualisations with this jQuery plugin

Flot is a specialised plotting library for jQuery, but it has many handy features and crucially works across all common browsers including Internet Explorer 6. Data can be animated and, because it's a jQuery plugin, you can fully control all the aspects of animation, presentation and user interaction. This does mean that you need to be familiar with (and comfortable with) jQuery, but if that's the case, this makes a great option for including interactive charts on your website.


Timeline creates beautiful interactive visualizations

Timeline is a fantastic widget which renders a beautiful interactive timeline that responds to the user's mouse, making it easy to create advanced timelines that convey a lot of information in a compressed space. Each element can be clicked to reveal more in-depth information, making this a great way to give a big-picture view while still providing full detail.


iCharts can have interactive elements, and you can pull in data from Google Docs

The iCharts service provides a hosted solution for creating and presenting compelling charts for inclusion on your website. There are many different chart types available, and each is fully customizable to suit the subject matter and color scheme of your site. Charts can have interactive elements, and can pull data from Google Docs, Excel spreadsheets and other sources. The free account lets you create basic charts, while you can pay to upgrade for additional features and branding-free options


The official Revisit project is a way to redefine how we look at Twitter. With this tool you are able to create custom line maps of data connecting tweets related to one or many keywords. You can additionally add a title to your graph and share the link online (even onto Twitter).Clicking on an individual breakaway line off the graphic will display further details. Tweets will often include metadata such as the time posted and important/related keywords. The search criteria are limited to standard Twitter notation which uses a comma separated list of keywords. For the best results keep your queries below 4-5 words since Twitter often has a difficult time matching overly-complicated content.


StatSilk offers web-based and desktop software to make data analysis easy, efficient and enjoyable, to cater to diverse mapping and visualisation needs.


Speaking of unique visualizers Wikipedia is also a network you don’t see developers playing with as much. This is surprising, since the main Wiki contains a ridiculously large amount of data! WikiMindMap lets you select a region and enter the URL for a page. If your keyword doesn’t exactly match up with a page ID the app will offer you the closest suggestion. The link generated inside the circle will lead out to the main Wiki page, while the refresh link opens a tree of options. These are all related links pulled off the main wiki page coordinating to your keyword. It’s also really easy to switch onto a new root node by clicking the green refresh icon.


Aimed more at specialist data visualisers, the Polymaps library creates image and vector-tiled maps using SVG

Polymaps is a mapping library that is aimed squarely at a data visualization audience. Offering a unique approach to styling the the maps it creates, analagous to CSS selectors, it's a great resource to know about.


It isn't easy to master, but OpenLayers is arguably the most complete, robust mapping solution discussed here

OpenLayers is probably the most robust of these mapping libraries. The documentation isn't great and the learning curve is steep, but for certain tasks nothing else can compete. When you need a very specific tool no other library provides, OpenLayers is always there.


Kartograph's projections breathe new life into our standard slippy maps

Kartograph's tag line is 'rethink mapping' and that is exactly what its developers are doing. We're all used to the Mercator projection, but Kartograph brings far more choices to the table. If you aren't working with worldwide data, and can place your map in a defined box, Kartograph has the options you need to stand out from the crowd.


Processing provides a cross-platform environment for creating images, animations, and interactions

Processing has become the poster child for interactive visualizations. It enables you to write much simpler code which is in turn compiled into Java. There is also a Processing.js project to make it easier for websites to use Processing without Java applets, plus a port to Objective-C so you can use it on iOS. It is a desktop application, but can be run on all platforms, and given that it is now several years old, there are plenty of examples and code from the community.