Time to announce my first published Rails plugin: ie-specific. And, more importantly, time to explain it.

I work as a front-end developer for a large scientific publishing firm. We’re very much committed to standards compliance and accessibility, as such, make every effort to ensure our sites work in as many browsers as possible. And that includes Internet Explorer 5.

Now, prior to IE7, we did this (by and large) by producing standards-compliant stylesheets, and then adding browser hacks where they were the only solution, in order to achieve consistent look/feel in many browsers. Fast forward to IE7, and we’ve now got a problem: many browser hacks (notably * html) no longer work in IE7 and yet it still has its own raft of styling quirks.

So we’ve decided to change the way we implement browser-hacking. On the project I’m currently working on, we now have a main.css which contains all the standards-compliant, “correct” CSS. We then place the IE-specific styles inside IE conditional comments. Each browser version gets a css file of its own where necessary, named for the name of the relevant IE conditional filter – eg, lte-ie-6.css, ie-6.css, lte-ie-5.500, etcetera.

When I started work on the front-end integration for this Rails project, it was clear that this was obviously something that could be automated in Ruby – and, given that we were probably going to use it again, it made sense to package it as a plugin so that the next time I came to need it for a project, a quick script/install would be enough.

Enter ie-specific, currently hosted at Google Code given that a) it’s open-source and b) it’s an easy way to get anonymous public svn.

It’s really simple. First, install the plugin:

script/plugin install http://ie-specific-rails-plugin.googlecode.com/svn/ie-specific

Now, place any IE-specific stylesheets into #{RAILS_ROOT}/public/stylesheets/ie_specific, named with the appropriate conditional filter (eg: lte-ie-6.css). Similarly, you can place IE-specific javascript into #{RAILS_ROOT}/public/javascripts/ie_specific. We use IE-specific Javascript to simulate min/max-width capabilities in Internet Explorer.

Finally, in the <head> of your layout, include the following ERb:

<%= ie_specific_styles_and_scripts %>

Each file in the ie_specific directories will be automatically included inside appropriate conditional comments – CSS files will be imported via an @import directive, and JS files will be appropriately escaped as CDATA (important for us, because we write XHTML 1.0 strict). There’s no unnecessary duplication, either – lte-ie-6.css and lte-ie-6.js will automatically be placed inside the same conditional comment.

That’s it, really – it’s very simple, but it’s also a convenient way to keep markup and styles clean and let other developers know exactly what’s a hack and what’s not. For more info, check out the README. And let me know what you think!

So, Matt pointed out that when you do things like that, the scores for different Models are all ranked seperately because they’re coming out of different indexes. The trick, obviously, is to use the multi_search method, which generates a multi-model-index, and that’s probably a better approach. This approach is very well documented in the RDoc. So all it shows is: probably should have RTFM. The thing below isn’t by any means bad, it’s just a little like re-inventing the wheel.

Never mind, eh?

(This mail from Jonas is also worth a read on this matter).

If you’re building a Rails applciation with search, chances are you’ve run across acts_as_ferret. And if you haven’t, check it out – it allows any model to be searchable by Ferret, the Ruby port of Lucene. Ferret’s a pretty nifty search engine – reasonably fast, pretty accurate – and it’s nice to be able to use it so simply in your Rails app.

Of course, what makes acts_as_ferret really handy is that it’s neatly designed to perform multi-model searches. Or, rather, the interface to do so is there, you just have to glue the results together. After some rough stumbling about, here’s what I came up with.

Firstly: don’t use find_by_contents, that’s not much use. find_by_contents is a wrapper around find_id_by_contents. find_id_by_contents is useful because it returns you only three things: a relative score, a model name, and a model id. That means you can merge everything into a big list, and then perform individual queries on the relevant models.

So how did I implement this?

My first stab was this (borrowing some real code from what I’m working on):

articles = Article.find_id_by_contents(query)
authors = Author.find_id_by_contents(query)
matches = (articles + authors).sort_by {|match| match[:score] }

This gives you an array of model/score/id hashes, which you can then do lookups on. The problem is it’s not very DRY, and it requires editing every time you make a new model that’s Ferretable. This is my more final solution, which I came up with. This time, I’ll walk you through each step.

First, somewhere like environment.rb, define a constant array:

FERRETABLE_MODELS = %w[Article Author]

That means we can update things at a later date easily. Then, in your search controller action, start with this:

klasses = FERRETABLE_MODELS.collect {|klass| Kernelt.const_get klass}

That should give you an array containing the Article and Author objects – or whatever ActiveRecord objects you’ve chosen in your constant array. Next, let’s get the array that we arrived at last time:

matches = klasses.inject([]) {|out, klass| out << klass.find_id_by_contents(query)}.flatten

Bit more complex, but also more succinct. All this does is iterate over each klass, using an inject method with an empty array passed in, and tells each class to call find_id_by_contents, passing in the query. It then flattens that lot, so that the array is only one-deep. We're now where we were before.

Finally: let's generate an array of the actual objects we referred to, sorted by ranking. I'm going to generate an array of hashes. Each hash has two keys: :object, the actual data object we want; and :score, the rank that ferret assigned it. We get that out like so:

results = matches.collect {|match|
   :score => match[:score], 
   :object => (Kernel.const_get(match[:model]).find(match[:id]))
}.sort_by {|o| o[:score]}.reverse

Again, possibly a bit ugly. :score remains the same; we run find on the appropriate ActiveRecord model, passing in the appropriate id to obtain the :object. Finally, we sort the array by :score and flip it, so that results[0] is the most popular search record. Obviously, you can pass extra parameters into that "find" method.

All that remains is to display that lot in your view, perhaps paginate it, and build a conditional to determine how to display each kind of object.

And that's it. I made a few changes to my dummy code when typing this up, so if something's broken, tell me and I'll fix it. I think that's a more maintainable way of searching across a range of models with ferret, and it takes advantage of some useful Ruby dynamics - finding objects through Kernel.const_get, in particular. That's my Ruby fun for today, then.

Update

My colleague Ben proposes this much tidier (but untested) solution:

results = []

FERRETABLE_MODELS.each  do |klass|
  k = Kernel.const_get klass
  k.find_id_by_contents(query).each do |m|
    results.push {
       :score => m[:score], 
       :object => k.find(match[:id]) 
     }
  end
end

results = results.sort{ |a,b| b[:score] <=> a[:score] }

I like the nested loop much more - should have thought of that myself - and will admit to being lazy wrt the sort_by and .reverse trick.

I’m going to be speaking tonight at LRUG. The talk is called “Ruby on Rails from the other side of the tracks“, and it’s about how client-side developers fit into Rails, and how you (as a back-end developer) can work with them rather than against them. If that sounds interesting (or, more to the point, you want to hear Tiest talk about Domain Specific Languages, which should be great), do come along.

Didn’t notice this when it happened (because it didn’t necessarily appear on the frontpage) but… my review of David Black’s Ruby for Rails has now gone live over at Vitamin.

My first piece of technical writing. It’s a good book, too – clarified an awful lot about the hierachy of classes within the language, and explained the nuances of Ruby dynamism very well; strongly recommended to anyone coming to Ruby (through Rails) afresh, whether you’re an experienced programmer or not. I hope I conveyed that in the review.

ChinaDialogue.net

31 July 2006

I recently did some consultancy for openTrust, the parent company of openDemocracy, and now that the project in question – chinadialogue – has gone live I wanted to mention it, mainly because I’m so impressed by how the final product turned out.

The best way to describe chinadialogue is as an entirely bilingual online publication about the Chinese environment, built on top of an entirely bilingual CMS.

By “entirely bilingual”, I mean that all content appears (eventually) in both English and Chinese on the site – not just links and headings, but the full text of every article, and of every comment. The site is designed so that whilst everything appears in both languages, the original source language is always highlighted. The translation between languages is performed by Mark 1 Human Beings, incidentally. I found a certain frisson to seeing English and Chinese standing side-by-side everywhere you look; it feels very subversive, given all the issues around Chinese state censorship.

My role in the project was admittedly very limited. I did some early-stages exploratory work around publishing platforms, considering whether to use a pre-existing, open source CMS/blogging tool and extend it either through plugin APIs, or a more major fork of the source code, or whether to build from scratch – and if so, in what. One of the major factors in this decision was the bilingual nature of the project: extending any existing system would require heavy use of the plugin API, but that would mean one language’s content would exist as the primary “content” for an entry, and the other would be banished to the meta-fields. Given that either could come “first” in the workflow of the site, and that both are of equal importance, I suggested that both should also be of equal importance in the database schema.

In the end, they went with the final option, and built the project from scratch in Ruby on Rails. We discussed this option at some length, as whilst there was a strong internal desire to build in Rails, the first thing that comes to mind when you say “Chinese” and “Ruby” in the same sentence is “holy Unicode support, Batman!”

But Unicode-in-Ruby can be stepped around if you know what you’re doing (and try nothing too fancy), so it’s great to see that they not only made Rails work for them – and, by all accounts, had a good time doing it – but also they managed to step around one of the more common Ruby gotchas.

Best of all, I note that they’re planning to release the CMS that runs chinadialogue as open source towards the end of the year. I’m really looking forward to seeing some of that code.

All in all, a pleasant experience, and very cheering to see the results. If you’re working on social publishing projects of any form, and want someone to throw ideas around with (for a reasonable rate) do get in touch.

A quick heads-up to two brief speaking engagments I’ve got coming up on the horizon.

First, I’ll be talking about software to tell stories with at the London Techa Kucha night on 25th July that Steve and Tom are running. That’ll be a radio-edit of the talk I gave at Reboot, more or less.

Then, I’ll be talking at LRUG on August 8th, looking at how as a Rails developer you can work effectively with front-end designers and client-side developers, and how you can keep the integrity of your front-end code at the same level as the back-end.

Come along if you feel like it. And if you found out about these events through the blog, do say hi.

It’s a strong title for a post, I know, but having discovered that two friends didn’t know this at last night’s LRUG, I wanted to share it.

Let’s put this in bold for impact.

When you add <%= javascript_include_tag :defaults %> to the top of your Rails layout, you’re adding 146kb to your page load.

And, being Javascript, that all loads serially. This is slowing page load down a lot.

Now, I’m sure if you’re building a whizz-bang AJAX app you need all that. But I reckon a lot of Rails projects don’t use anywhere near all that. So throw some out!

The Scriptaculous libraries are quite big: dragdrop.js is 30kb, controls.js is 29kb, effects.js is 34kb. If you don’t need that lot, get rid of it. prototype.js alone is 56kb. The case for the Prototype library is easier to make… but if all you’re doing is some simple DOM scripting – show/hide, for instance, do you really need 56kb of library functions to do it? Or can you do it in < 10kb with some home-made functions? If so, strongly consider doing that. And if you’re not using any Javascript in your application… why have you get any of it in there? It can all go – that’ll speed page load up no end!

I’m not saying there’s no place for this stuff – there is. Should it be included in Rails Core? Absolutely. I just don’t think that it should be called the “default” option; <%= javascript_include_tag :all %> would make a lot more sense. And by sticking it into the basic scaffolding layout, it becomes a part of many people’s first experience with Rails – and they assume it’s default behaviour. And so I also think it should be left out of scaffolding – perhaps replaced with <%= javascript_include_tag "prototype", "application" %> at the least.

User experience, usability, and accessibility aren’t just about the content or code of the page; they’re about how the user experiences the page. If it takes an age to load, it makes the app less usable. If you’ve got huge page size, you’re excluding anyone on a slow connection. So next time you’re skeletoning out a Rails app, take a moment to think if you really need any Javascript. If you don’t, <%= javascript_include_tag :defaults %> can go straight in the bin. Then, when you come to progressively enhance your app at the end with Javascript (which is, let’s face it, how you should be doing it), you can include only the files you need – and keep your users happy at the same time.

One thing most web applications need is a static page template. Now, whilst the page content might be static, the template itself might need to be dynamic – either because it’s going to change in future, or because you’ve got dynamic user information that appears in the tempalte.

So the most obvious way around this is the static controller. Dead easy, this: generate a new controller called static (or whatever you want). Then just write views for it named after pages you want. For instance: your about.rhtml file can contain all your “about” information. Then, when you hit up /static/about (to use the default routing), you get your static content, without having to make a whole page from scratch in public_html. You could even write a new 404 page on this controller.

All that remains, once it’s working, is to write some dedicated routes, and then the “static” controller can be hidden from existence – just route /about to :controller => "static", :action => "about" and you’re done. No need to write any controller logic at all!

Going on from there: one view I’ll always add to that template is the “foo” action.

So: when you’re mocking up a page, you’ll probably use lots of dummy links. Everyone expects this, because it’s obviously just a flat mock. But when you mock up an application, and show it to stakeholders in a working state, they click on things, and wonder why they get ActionController exceptions when it breaks. Also, they wonder why the link that breaks stuff is always /foo.

Obviously, it’s because I’ve left link_to "/foo" all over the shop. Have no fear: the easy way around this is to route /foo to :controller => "static", :action => "foo", and then write a static page called foo.

When you do this, the page should explain that it represents functionality that hasn’t been added yet, but will be added soon, and that the developers haven’t forgotten it.

This (from experience) reassures stakeholders that the thing that is not working will be working soon. It also means that when they do find “grey screen” errors, it either means that something’s genuinely broken, or it means that the developers really have forgotten something. Time to update that link to point to foo.

It sounds trivial, but it turned out to be an effective communication of diligence on the developer’s part, and saved much time in meetings explaining that “yes, we’re working on that”. Consider it, next time you’re developing for external stakeholders.

Where I’ve been

19 June 2006

It appears that I’ve been AWOL for a little while. For that, and many other things: apologies.

It appears to have taken a while to decompress from Reboot and get things back on track. The to-do list floating above my desktop has not really got much shorter for a few weeks now. Even though I’m making my way through it, there always seems to be new stuff that needs adding. Also, the stuffy heat is contributing to a lack of both sleep and energy, so maybe that’s why I’ve been posting here less.

Still, I have been doing a few things behind the scenes. My Flickr stream has seen a lot of updates recently, and you’ll probably have noticed the string of del.icio.us links filling the blog up. As ever, they provide rough hints to where my brain currently is.

One thing I’ve been working on is… redeveloping this blog. Boring, I know. But the last time I redesigned, the purpose was to get me writing again. It kind-of worked, but it was also an experiment at developing a truly generic WordPress theme (which almost succeeded). Now I want to make something just for me (although, it turns out that it shares some aesthetic sensibilities). At the same time, it seems like a good enough resaon to upgrade to WordPress 2.0. That’s a story in itself, I tell you. I’ll write more about the technicalities of redeveloping it once it’s launched; there’s a post in the works on patching tagging plugins, the ups and downs of upgrading and the hell that is wp_rewrite, and also another on faking “offset” with the WordPress Loop. Until then, expect to see this place updated soon.

Other things I’ve been working on: a moderately-sized Rails project, on which I’m sole developer and designer. It’s going quite slowly, due to the concentration it’s now demanding – I’m at the “complicated” part of the development, when things go beyond CRUD, and also start demanding the final templates – so there’s been a big diversion into XHTML work. Still, this is also the stage where it gets *really* satisfying. I’m not going to be able to opensource the project as it stands, but there’s at least one plugin to be released from it, not to mention some useful experience. I’ve currently been adding fragment cacheing, which has been most satisfying – both (relatively) simple and elegant in its implementation.

We’ve also been busy at work, having finally launched Nature Network Boston in beta. There’s still some work that needs to be done there – on my part, fixing the microformats (again, more in the future) and tidying the HTML output – but it’s great to see it in “the wild”, as it were. Out of that, I’ve got an article I’d like to write for this site on “Rails from a design team perspective” – there’s an existing “Rails for Designers” piece on the web doesn’t quite cover the ground I’d like it to, or in enough details, so, you know, rather than whinging, I thought I’d write my own. Expect to see that in a week or two.

So, in short: expect to see more, longer content here. I need to get back into writing, and it seems sensible to start by writing up the stuff I’ve been doing that others might find interesting – or at least want to correct me on. In short: this place is going to be revitalised. I’m looking forward to the challenge I’ve set myself.