Author Archive

Choosing Web Chart Technologies

Friday, December 9th, 2011

Web-based charting has emerged as another great use of the browser. Traditionally, business analysts were the ones crunching numbers on spreadsheet to build charts. The spreadsheet would be e-mail as a file attachment. This process of compiling numbers and chart building could take hours. There is the saying that once information is on paper, it is already outdated. It will not be long before the saying becomes: Once information is in an office file document, it is already outdated. As a result, the web is quickly becoming the preferred platform for data charting and analysis.

In this article, we will discuss approaches to creating charts on the web. Currently, there are two general approaches to creating charts on the web: Image-based versus Scalable Vector Graphics (SVG)-based. There are also Flash-based solutions but we cannot recommend them because Flash is not available on iPhones and iPads and the future of Flash-based solution is uncertain.

Image charts
Here, the server generates a static image (e.g., a JPEG) of the chart from the data and the image gets displayed in the browser. There are a plethora of server-side charting implementations to choose from — a noteworthy one is Google Image Charts: http://code.google.com/apis/chart/image. Since the chart is simply an image, there are no cross-browser issues and the look of the chart is also consistent across browsers. The chart can easily be exported for usage in presentation slides since the chart is just an image file. However, the disadvantage of this approach is that users cannot interact with the chart; they cannot click into a pie chart slice or hover over line graph points to better understand the data being presented. While image-based charts provide a consistent and reliable view of the data, this approach does not give your end users necessary usability needed to make effective use of the information.

SVG charts
Here, the rendering of the chart is strictly on the client browser: There is no server-side rendering of the chart. Data are passed into a JavaScript SVG library which renders the chart as SVG markup (or VML markup if the user is on Internet Explorer 6/7/8). The browser produces a chart from the resulting markup that is mostly consistent across browsers. (We have run into an edge case issue where labels on pie charts get cut-off in Internet Explorer.) Exporting the chart into a file for use in presentation slides is a little trickier — since SVG charts are derived from JavaScript libraries and you cannot create files using JavaScript (at least not easily or consistently across browsers). One possible solution is to post the serialized SVG output back to the server-side and convert into the desired target format (e.g., JPEG, PDF, etc.). These relatively minor issues are offset by the huge gain in chart interactivity. SVG charting libraries allows developers to attach mouse click and hover events to the charts, such that a user can drill down into a pie slice of a chart or hover over line graph points for a more comprehensive view of the data. The SVG charting libraries tend to be relatively easy to use for developers comfortable with JavaScript, relative to the server-side charting tools.

Choosing a solution
When choosing a charting solution, it is necessary to consider the following requirements:

  • Does the chart have to be interactive? If the answer is a resounding yes, then SVG approaches are the only options.
  • How important is exporting chart into other file format? If very important, then it may be worthwhile to look at image-based solutions or possibly dual solutions of both SVG and static images.
  • Are we dealing with tens of thousands of data points? If yes, then it may be neccessary to sacrifice the interactivity of SVG charts for performance of static image charts.

SVG solutions
Below is a brief descriptive of three SVG libraries. This is by no means a comprehensive list.

  • gRaphael: http://g.raphaeljs.com
    • Only completely free solution of the three.
    • Not as mature the others and poorly documented.
    • Programming somewhat easy.
    • Fewer eye candy.
    • Bottom-line: Check back in 6 months; the library is improving rapidly.
  • Google Chart Tools: http://code.google.com/apis/chart
    • Does not cost money but source code is not available and hard dependency on Google’s server.
    • Mature and well-documented.
    • Programming not as easy but powerful.
    • Better eye candy — see Geographic chart.
    • Bottom-line: Good solution but the charts and data security are heavily dependant on Google’s server.
  • Highcharts: http://www.highcharts.com
    • Costs money (couple of hundreds) but open source.
    • Mature; Good documentation.
    • Programming is easy.
    • Best eye candy — charts will “pop” to your users.
    • Bottom-line: Best overall general-purpose charting solution.

Posted in Technology, Web Development, Website Design | No Comments »

JavaScript Unit Testing

Monday, May 16th, 2011

As recently blogged on GORGES, unit testing is important to the success of projects. Unit testing reduces bugs, improves a project’s robustness to requirement changes, and sustains a high rate of programming productivity. It is well-accepted that most if not all, non-trivial web applications should include unit tests as part of the project specification.

Web developers typically write unit tests on server side code and rarely on client side code, which is where JavaScript resides. This approach is reasonable given that the majority of the application logic of web applications run on the server side. However, as the web become increasingly more dynamic, so does the number of lines of JavaScript code needed to support a rich user experience. The need for unit testing in JavaScript should be apparent, yet JavaScript unit testing have been largely ignore.

One reason why developers don’t write JavaScript unit tests is that JavaScript testing is front end testing. Front end tests are hard to write and we are better off leaving it to manual testers. Keep in mind, however, that application logic tend grow exponentially in complexity. So unless the project has the resources to hire manual testers at an exponential rate, it is far more cost effective to devote the time to write tests.

Another reason for not writing JavaScript unit tests is the belief that client side programming logic should be minimized and pushed into the server side wherever possible. The rationale is that JavaScript unit testing is not needed if the code base is small. In other words, not writing JavaScript means not having to write unit tests. However, as mentioned, JavaScript is needed to support a rich web experience. Not writing JavaScript is not an option if a modern web experience is desired.

There are many free and open source testing frameworks available for JavaScript. Three such frameworks are: Selenium, JS Test Driver, and QUnit.

SELENIUM, http://seleniumhq.com

Selenium is not a JavaScript testing framework. Rather, it is an end-to-end testing framework for web applications. So Selenium implicitly tests the JavaScript portions of a web application. Selenium works as follows:

  1. Write your Selenium scripts. The scripts can be written in many languages including PHP, Ruby, C#, and Java. If you don’t know how to program, you can use Selenium IDE which is a Firefox plugin that records your mouse clicks and typing web browsing behavior to create test scripts
  2. Your unit test runner (e.g., JUnit) sends the scripts to the Selenium server
  3. The Selenium server launches browsers and runs the tests. The Selenium server can be configured to launch different browsers on different computer platforms (e.g., PC, Mac, Linux).

Note that Selenium runs your tests using real browsers so you get comprehensive tests of your web application against various browsers. The downside is that the comprehensiveness also leads to complexity of setting up the Selenium server and stand-by machines waiting for Selenium tests. Selenium is best suitable for large web projects with the resources needed to configure various servers.

JS TEST DRIVER, http://code.google.com/p/js-test-driver/

JS Test Driver can be thought of as Selenium-lite but specifically designed for unit testing your JavaScript code (as opposed to end-to-end integration testing as is the case with Selenium). The test runner for JS Test Driver runs your unit tests in the same manner as Selenium. JS Test Driver launches your browsers and executes your tests inside the browser. JS Test Driver can also be integrated into the Eclipse or IntelliJ IDE as a plugin which provides a one click interface to running your JavaScript tests.

QUnit, http://docs.jquery.com/Qunit

QUnit is the simplest of the three framework. You write your tests in JavaScript and embed it in an HTML page with qunit.js and qunit.css include. To run your tests, you simply refresh the page in your browser. There is almost no learning curve for developers familiar with any of the xUnit framework. Because of its simplicity, there is no out-of-the-box support for running your tests on multiples browsers simultaneously, unlike Selenium and JS Test Driver.

Which framework?

These three frameworks all serve different testing needs and are by no means mutually exclusive. If you are only interested in end-to-end integration test, then Selenium is a good choice if you have the resources to set up your testing environment. If you see the value of unit testing JavaScript in your web application, then QUnit is the easiest way to start but look into JS Test Driver as your testing need evolves.

Tags: , , , ,
Posted in Technology, Uncategorized, Web Development | No Comments »

Add REST-style URLs to older Java web applications using UrlRewriteFilter

Thursday, November 11th, 2010

Say you have an old old Java Web application (jsp-based) that uses URLs like this:

/queryVehicles.jsp?make=Ford&model=Fusion

But we’re in 2010 where URLs should be pretty and more REST-style like so:

/vehicles/ford/fusion

Well, you have two options. You can re-write your web application to use newer web frameworks that supports REST-style URLs such as Spring MVC 3.0, which will probably take many months to do. Or you can use UrlRewriteFilter which should take no more than 30 minutes and will work with any existing Java web application.

UrlRewriteFilter very easy to use. You basically:

  1. Add UrlRewriteFilter as a servlet filter in your WEB-INF/web.xml
  2. Populate the configuration file WEB-INF/urlrewrite.xml with mappings of REST-style URLs to the corresponding URLs of your legacy web application (with query params and what not). UrlRewriteFilter is similar to mod_rewrite of Apache HTTP Server. So your rewrite rule would look something like this:
     <rule>
         <from>^/vehicles/([a-z]+)/([a-z]+)$</from>
         <to>/queryVehicles.jsp?make=$1&amp;model=$2</to>
     </rule>

That’s pretty much it.

UrlRewriteFilter is extremely powerful. You can perform pretty complex mapping since it uses regular expression pattern matching. Check out the manual page for details.

Tags: , ,
Posted in Technology, Web Development | No Comments »

Introduction to data recovery using open source tools

Sunday, September 19th, 2010

I have an old hard disk with corrupt NTFS volumes. I’m not sure how they got corrupt but they cannot be fixed using the standard chkdsk /f command. Fortunately, there’s a plethora of open source data recovery tools available. Two such tools are foremost and photorec which specialize in combing through hard drive partitions to recover files based on header information. They can even recover files after the disk or memory card has been accidentally erase.

Foremost is a Linux command-line tool originally developed by the U.S. Air Force. If you’re using a debian-based Linux distribution (e.g., Ubuntu), you can grab it using the command sudo apt-get install foremost. Foremost can recover common file types such as txt, jpg, avi, and etc.. It was last updated in 2008 which means that its knowledge of file headers is, at best, two years dated.

Photorec is part of the testdisk suite, which is a set of Linux command-line tools. If you’re using debian-based Linux, you can install it using the command sudo apt-get install testdisk. Testdisk not only “tests your disk” but also rebuilds your partition table. This is the tool to use if your hard drive’s master boot record or partition table is corrupt. Photorec, like foremost, recovers files (not just photos) based on file headers. In fact, photorec supports more file types and is more up-to-date than foremost, which is evident by the fact that I was able to recover more files with photorec than foremost.

The problem with both foremost and photorec is that they recover file content but not file names. So you end up with directories of randomly named files with only the file extension preserved. It’s not ideal but it’s still better than not having the data at all.

See also:

http://help.ubuntu.com/community/DataRecovery

Posted in General, System Administration, Technology | No Comments »

©2012 GORGES - All rights reserved
where programming meets design and lives happily ever after