Archive for the ‘ Software ’ Category

Node.JS would really benefit from LibFFI

Having recently built a simple LDAP library that required a bit of C++, I think Node would really benefit from LibFFI bindings.

LibFFI allows C functions to be called dynamically at runtime from another language. There are LibFFI bindings for Python (ctypes), Ruby (Ruby-FFI) and Java (JNA), amongst others.

If FFI bindings existing for Node, you’d be able to call native C functions without ever leaving the comfort of JavaScript. Apart from making the code a lot simpler, it also avoids the need for a build step (nothing to compile), and hide the complexities of working with threads in C++.

Here’s how it would work…

Step 1: Write some JavaScript that bindings to a native library and exports C functions as JavaScript.

var ffi = require('ffi'); //
var ldap = ffi.createLibrary('ldap', { // load
  // exporting these functions
  // (C function name, [C arg types], C return type)
  open: ffi.exportAsync('ldap_open',
    [ffi.string, ffi.string], ffi.pointer),
  simpleBind: ffi.exportAsync('ldap_simple_bind_s',
    [ffi.pointer, ffi.string, ffi.password],,
  unbind: ffi.exportAsync('ldap_unbind',
    [ffi.pointer], ffi.integer)

Yes, this looks messy, but it’s a lot less painful than doing it in C++.

Step 2: Call your native libraries through the JavaScript bindings.'somehost', 389,
  function(err, ldap) {
    if (!err) {
      ldap.simpleBind(ldap, someUserName, somePassword, 
        function(err, result) {
          if (!err) {
            print('Authentication result = ' + result);

The native functions would still execute asynchronously in a background thread, so as not to tie up the main Node event loop. The usage pattern is the same as the rest of Node.

Writing the code still involves a basic knowledge of C, in that you need to read the documentation for native libraries and understand what it means. But you’d write all your code in JavaScript and avoid the hassle of the build system and platform specific linking.

There’s a lot of native libraries out there – this would open up a lot of doors.

Who’s going to take a crack at it?

LDAP authentication with Node.JS

I’m a big fan of Node.JS.

It has a large collection of third party add-ons, but it lacked anything for LDAP authentication – something I need for web-apps.

So, I created a small library with a single function that authenticates a username/password against an LDAP server.

var ldapauth = require('ldapauth'); 

ldapauth.authenticate('', 389, 'someuser', 'somepassword',
  function(err, result) {
    if (err) {
    } else {
      print('Credentials valid = ' + result); // true or false

(Full example)

Under the hood it’s a native C++ module using OpenLDAP.

To use it, grab the ldapauth.node binary and include in your Node path (or reference directly when calling require()). The binary is built for 64bit Linux – if you use another platform, you’ll have to get the code and build it yourself.

Introducing Smoothie Charts

There’s tons of great JavaScript charting libraries out there. Too many to choose from. I’ve just made it worse by adding yet another to the mix.

Before I go on, I want to balance karma a little – if you’re looking for a JavaScript charting library, just use Flot. I don’t know anything about your requirements, but I like my chances – everyone loves Flot (including me). It’s small, simple, great looking and flexible.

So why another library? Well, I had a very specific problem… I want to show real time streaming data pushed over a WebSocket. Although many of the charting libraries allow you to dynamically update data, none have really been optimized for a constant stream of data. I tried many (seriously, at least 40 libraries) and the constant updates just looked too jerky.

So Smoothie Charts was born. It’s not big, and it’s not clever. And it only works on Chrome and Safari. And it doesn’t support legends. Or pie-charts. In fact, it’s inferior in virtually every way to a library like Flot. But, it’s very good at displaying streaming data.

I use it for building web dashboards monitoring active system load (that look something like this).

Use it, hack on it, and contribute back.

New QDox lead – Robert Scholte

I’d like to announce the new lead for QDox, Robert Scholte.

Anyone who’s been following the project for the past year would have seen that Robert has made a tremendous number of contributions to all areas of QDox, including bug fixes, build, docs/website, releasing, user support and admin. If you follow the project further back, you’ll see that Robert has actually done more on the project in the past year than was done for the combined 6 years before that…. impressive.

I originally wrote QDox back in 2002, as a stop gap until annotations were added to the Java language. It was quickly picked up tools such as XDoclet. I always thought (and hoped) that once annotations made it into the language, it would be redundant. Yet it lived on. Unlike the JDK Annotation Processing Tool (APT), it could process source code rather than byte code in a simpler and faster manner. And it provided access to information that the JDK didn’t make available (e.g. parameter names). And it was useful where annotations still did not exist, like in JavaME. So, even in 2010, it still lives on and is used in tools such as Maven and Eclipse. But I cannot invest the time into it that I once could.

Given Robert’s commitment to the project, I can think of no better person to lead the project. He’s done an astounding job. Robert has kindly accepted this role and I’m delighted to hand it over to him.

I’d also like to use this opportunity to thank Paul Hammant, Mauro Talevi and Dennis Lundberg for their ongoing efforts on the project.

JUnit/TestNG tip: Diffing Lists

A little cheap and cheerful JUnit/TestNG tip…

Problem: You’ve got two lists that you want to compare using assertEquals(). But when the assertion fails, the error message kinda sucks.

java.lang.AssertionError: expected:<[Person{name=Joe,starsign=Virgo,sex=Male}, Person{name=Jaimie,starsign=Libra,sex=Female}, Per
son{name=Ewan,starsign=Capricorn,sex=Male}]> but was:<[Person{name=Joe,starsign=Virgo,sex=Male}, Person{name=Jaimie,starsign=Scor
   at org.junit.Assert.failNotEquals(
   at org.junit.Assert.assertEquals(

It’s really tricky to see exactly where the differences are in the list.

Solution: Join both the lists into a string, delimited by newlines and assert that instead. That will force your IDE’s multiline differ to kick in.

junit-diff(click image to see the rest of it)

This relies on having a sensible toString() method on your list items.

If your IDE doesn’t do this, or you can’t run your tests from your IDE, you should really get that seen to.

Here’s a method to do it:

public static <T> void assertList(List<T> actual, T... expected) {
  assertEquals(join(expected, "\n"), join(actual, "\n"));

That’s all. Now get back to work.

SiteMesh 3 preview

I’m pleased to announce a preview of the next generation of SiteMesh.

Before I go on, here’s a big disclaimer: This is not a stable release version yet… it’s not even a beta… it’s an alpha… a first alpha.

But anyway, I decided to launch it in this state, as I think it’s a huge improvement over SiteMesh 2, and I’d like to get your feedback earlier so it can be fed into the final product.

Quick links, for the impatient:

So what’s new? Well, everything. The 10 year old code base has been thrown away and rebuilt from the ground up. At the core of it is a new content processing architecture that performs roughly three times as fast as SiteMesh 2 with half the memory usage.

Along with that:

  • Decorator chaining. Compose pages from multiple layouts and decorators.
  • Decoupled from templating systems. Decorators can be produced with any technologies – you are free from JSP.
  • Offline site generation. Apply decorators as part of a build step. Accessible through Java API, Ant task and command line tool.
  • Simplified configuration. Use XML, Java or plug in your own system.
  • Clean API provides extension points for adding custom processing rules (e.g. on the fly page transformations).
  • Licensed under Apache Software License v2.0.

So what’s the same? The values of SiteMesh have not changed – simplicity, robustness and performance.

You can learn more about SiteMesh 3 on its new website:

What about compatibility with SiteMesh 2?

This was a hard decision to make, but SiteMesh 3 is not backwards compatible with SiteMesh 2. Upgrading will require work beyond just upgrading a library. Over the coming weeks, I shall produce some guides and tools to make this transition easier. Though this may be frustrating to many users, it was becoming too hard to make changes to SiteMesh 2 based on limitations to its design. SiteMesh 3 addresses these and will make way for a lot of great features and improvements in years to come.

Meanwhile, SiteMesh 2 will continue to be supported until 2011 – though this will mostly be critical bug fixes only. The majority of development effort will be focussed on SiteMesh 3.

So give it a go, and give us feedback. But please remember that it’s only an alpha!

SiteMesh logo design contest

I’m looking for a gorgeous new logo for SiteMesh, so I”ve just launched  a logo design contest on It will run for 7 days and has a prize of $400 – of course, the real prize is the good feeling of contributing back to the SiteMesh project ;).

The brief…

Clean, simple, modern.

Feel free to use whatever font you like, or create your own. Remember that SiteMesh is associated with clean and simple. If you use (or derive from) an existing font, please specify what font this is and where it can be obtained.

The logo should be on a flat (single color) background, so it is easy to place in different places. If the background is not white, please also supply a variation of the logo that has a white background (e.g. to use in print outs).

Color scheme:
Up to you.

Would like two variations of the logo:
1. The worded logo: This focus of this must be the word ‘SiteMesh’. It will typically be used in the header of every page of the website, documentation, etc.
2. The icon based logo: This would typically appear in small square placements (e.g. a desktop icon). It does not need to contain the text ‘SiteMesh’ but should be easy to recognize as associated with the worded logo. Should look good at small sizes (e.g. 42×42).

Please supply original high-resolution files of whatever tool you use to create the logo (e.g PSD), so I can make modifications and rescale in the future as necessary.

Running SiteMesh on AppEngine

Does SiteMesh work on Google AppEngine (Java preview)?

Short answer: Yes.

Longer answer: Yes. It does. So long as you check 2 things…

1. Upgrade

You need SiteMesh version 2.4.2 or greater. Earlier versions do not support AppEngine.

Download here

2. Disable static file serving

If you want decorators to be applied to static content (e.g. to .html files), the following needs to be added to WEB-INF/appengine-web.xml:

  <exclude path="**"/> 

This forces the static resources to be served by the Servlet engine, rather than a separate static web-server. Without this, the files served by the static web server will not be decorated.

Happy AppEngining.

SiteMesh 2.4 released

After a long wait, SiteMesh 2.4 has just been released.

This mostly consists of minor bug fixes and improvements.

Our Marmites

Marmite is a spread available in the UK and a few other countries. If you’ve come across Vegemite, it’s similar. The taste is very distinctive, to the point where anyone who’s ever tried it either loves it or hates it. There is no in between. Whereas representing a persons feelings for certain types of food may require a complicated data structure, for Marmite you’d only ever need a single bit. You get the idea.

The marketing slogan in the UK is “Marmite: Love it or Hate it?”.

I often ask different people what they think about a certain product (for example, a web framework). Usually I’ll get varying feedback, but once in a while I’ll come across something that only gets two categories of answer – “Love it” or “Hate it”. It’s usually accompanied this person getting very excited or angry, going a bit red in the face and thumping the table (in a positive or negative way). There are never any inbetweeners. If people have not actually used a product, I’ll discard their opinion.

These are the Marmites of our world.

Here are some of the Marmites I’ve come across, with typical explainations I get from lovers and haters.

Maven is the Marmite of the build tool world.

  • Lovers: Consistency across projects. Common build layout and tasks. Dependency management. Does lots.
  • Haters: Poor documentation. Need the head a size of a planet to understand it.

Mac OS X is the Marmite of the desktop OS world.

  • Lovers: Beautiful. Stable. Less clutter. Less hassle. Unixy + usability. It’s not Windows.
  • Haters: Forever staring at the spinny beachball.

Ruby on Rails is the Marmite of the web application framework world.

  • Lovers: Ruby. And web sites. And ActiveRecord. And it’s oh so agile.
  • Haters: Irritating fan club. Lack of abstractions can make it hard to evolve larger systems.

Google Web Toolkit is the Marmite of the AJAX world.

  • Lovers: Allows you to use Java instead of JavaScript for building apps.
  • Haters: Allows you to use Java instead of JavaScript for building apps.

ReiserFS is the Marmite of the file system world.

  • Lovers: Excellent performance with many files and big disks. Scales well. Fast recovery through journaling. Uses space effectivley.
  • Haters: Numerous problems with fsck operations corrupting the filesystem. Future uncertain since Mr Reiser was convicted of murdering his wife.

IBM Visual Age for Java is the Marmite of the Java IDE world.

  • Lovers: A magical image of objects that can be navigating, interacted with and modified on the fly. Like Smalltalk.
  • Haters: Uhhmm. WTF is this? Where’s my source code? How do I use this thing? Isn’t this supposed to be Java?

Behaviour Driven Development is the Marmite of the development process world.

  • Lovers: It bridges the gap from requirements understanding to a live running system, with a simple metaphor.
  • Haters: It’s what now? You’ve changed the prefix from ‘test’ to ‘should’ and introduced a few words. Get away from me.

The thing to remember is to never get into a conversation with anyone about a Marmite. Either you will agree with someone and learn very little from it, or you’ll get into a fight.

So to help others avoid these awkward moments, contribute your own list of Marmites here (comments below, or trackback). Criteria: Must be something that everyone who has experience of either hates or loves – there can be no inbetween.