design intelligence

Tag Swarms - Part II

Posted on 30th March 2006
When I initially started thinking of how to implement the tag swarm I preferred the option of coding it all in one language (personal choice only - I don't particularly like having to constantly switch 'mind-sets' when programming) but it was pretty obvious that this wasn't going to be an option. When deciding on the best way of proceeding I considered the following points:

As the algorithm would be getting its original data from a database It seemed pretty clear that this part of the process would be coded in PHP - as far as I know JavaScript doesn't have the ability to query a database directly.

Although it would be perfectly acceptable to produce a static HTML page of the results directly from PHP I wanted to give the viewer some indication of what the algorithm was doing behind the scenes - therefore I had to find a way of animating the HTML elements - JavaScript was the obvious choice for this part of the process.

Since the process would be split between the two programming languages I had to find a method of transferring data from one to the other. A little bit of research turned up JSON (JavaScript Object Notation) which allowed me, very easily, convert a PHP data structure into its equivalent JavaScript object. The method used to transfer the data was an XMLHTTPRequest from JavaScript (I had managed, without even intending, to opt for the 'full AJAX' solution)  

I also had to decide which would be responsible for running the main part of the algorithm. Initially I tried the following setup:

PHP - Query database, package results into JSON format and transfer to JavaScript.

JavaScript - Run the algorithm that organises the data into "swarms", animate the process and produce the final HTML output.

It became clear that JavaScript really wasn't suited to handling the main sorting algorithm due to the fact that most browsers have an automatic 'timeout' on JavaScript code (On my installation of Firefox its just 5 seconds). PHP code also has a timeout - but its far longer than that for JavaScript - so I adapted the code to the following scheme:

PHP - Query database, run sorting algorithm, package results into JSON format and transfer to JavaScript

JavaScript - Animate the process and produce the final HTML output.

For those that are wondering - The final scheme gets round the JavaScript timeout by issuing an XMLHTTPRequest for the data which, upon its return, is handled by a callback function - in effect this means that the initial bit of JavaScript code finishes running before the timeout and the callback function is only run once the data has been delivered.

Doing this means that the animation produced by the JavaScript is only an approximation to the working of the sorting algorithm (being a linear interpolation between each elements starting and ending points) - but its good enough to demonstrate whats going on.

There are a few points that I think need working on:

  1. The algorithm is still hampered by the timeouts on the PHP code. Ideally the algorithm would just run until it has finished (btw working out when that is isn't that easy) but because of the timeout it needs to be terminated by a specific time so it can produce partially unsorted data.
  2. Following on from the last point, the algorithm probably won't scale very well as the amount of time required to produce accurate results rises exponentially in relation to the number of tags being sorted - I've yet to test the extent that this problem might have so its difficult to give accurate results.
  3. The code as it stands is still very much a work in progress so I'm on the lookout for bugs constantly - there are a few that I know of but I've not had time to fix them yet.

The main algorithm used in the tag swarm process is available in the source code section and, for those that want to know, these are the various bits of external code that were used in the design:

Dojo AJAX Toolkit - Handles the animation and XMLHTTPRequest for the data. Although I've only used a fraction of the package here its a very well though out package.

JSON.php - Used for converting PHP objects into JavaScript objects. Its part of the PEAR repository of PHP code.