So there I went after mentally memorising the map from Liverpool Street Station to LBi‘s offices in Brick Lane. Funny because I have done the Brick Lane-Liverpool St. route several times already, but since it was dark and I got an excellent guide, I never paid much attention. Anyway, I arrived, gave my name to the security guy which was kind of confused with my Spanish name and then went downstairs where the people with laptops on the couches were.

I took a look and didn’t know anyone — so I decided the direct approach would be better: waved and said “hi” to whoever was looking at me at that point, and got a friendly response. Yay!

In a minute I was happily talking with Francesca, a nice front end web developer, then with someone else whose name I didn’t find out, then I got to associate Julian’s name with his face. A little while after, Paul Carvill came downstairs with lots of pizza boxes –all vegetarian, and yummy!– and as we were discussing about Rails 3.0, Joanna Geary made a little speech, more or less on the lines of this:

‘Journalists, raise your hands!’

They do.

‘Journalists, find a developer and explain them what do you have in mind that could be done with code, and let’s see if it’s doable or not!’

And then a girl approached Francesca and me. She looked strangely familiar but I couldn’t really tell why. We sat in a table, took the laptops out of their sleeves and set out to try and find if the data we needed was available. But the wireless didn’t quite work. So we chatted about something fascinating and kind of related to the topic: Volcanoes! It really helped that she had a degree in Geology so she could really offer interesting insight about how volcanoes work. Also, amazing as it sounds, she could pronounce Eyjafjallajökull in one go without hesitation!

We ate some more pizza, and then the wireless started working. So we checked and saw that the data wasn’t available — but we agreed she would contact the organisation and ask them for it. Maybe they already have it but it’s not publicly advertised.

Then it’s when the real coding part began; she said she would love to learn programming, but didn’t know where to start, or the guides she had tried were boring. I immediately proposed Processing; I know there’s Hackety Hack by _why, which is ruby based, but I don’t have any experience with it, so I went for the safer option.

I actually hadn’t done much Processing recently and as such I always forget the names of the basic functions (was it setup, or was it load? was it loop or was it play?) but it was even better, so that way they could follow along. She downloaded Processing and was typing in stuff in the sketchbook immediately… which was amazing! No setup, no preparation, no rebooting the computer after the installation; just download and experiment!

We’ve done just extremely simple things such as opening a window, setting its background colour, changing its size and… drawing a line! Still, I find it really enlightening to observe how ‘normal people’ (i.e. not hardcore developers) react to all these concepts when exposed to them.

Interesting things I’ve noticed (teachers, take note!):

• people love to see results immediately – no theory to start with!
• people love to have some starting snippets, which they can modify and experiment with, learning how they work meanwhile

So that makes Processing an excellent programming environment for beginners, because it’s got that beatiful reference written in pretty much plain language and with example snippets that can be copied and pasted for each function/class.

In a few minutes the word had spread: there were people doing stuff with Processing in the room! And so a little audience gathered, another guy (a journalist) came with his laptop, downloaded Processing and was doing little simple things as the aforementioned ones, copying code from her wife’s screen –isn’t it sweet?

On the other side of the table I was showing Paul and Francesca my word processing and statistical experiments with Tolkien works and we also spoke about Processing versus other environments such as Javascript or Flash.

In between, the girl had told us her name, and if the face looked familiar, the name was insanely familiar! She humbly told me she was in lots of events and etc, but didn’t make a big thing of it. I asked her about events I had been in (that I could recall at that point) to no avail –we didn’t seem to have been in the same events. But then I came home and looked for “Suw Charman” and found that not only had I seen her speak at resfest’05 but she had also posted a few messages to a mailing list I follow. I think that’s why the name sounded sooo familiar!

Of course, there couldn’t be a nice event without Rob McKinnon in it! I just love how I always find him everywhere I go, whether it’s a full fledged event or just a coffee shop anywhere in London, it’s just so funny. Couldn’t exchange more than a few “hey hey hi!” waves because we both were busy with our new friends, so I don’t know which sorts of interesting stuff did he show this time (the last time he spoke about Hpricot).

The event ended at around 22h, but mainly because people had to leave and take trains and all those things, but I’m pretty sure otherwise we could have stayed and playing with Processing for a long while :-)

And that’s how ruby in the pub, “with hardly any ruby and certainly no pub”, was. See you in the next one?

As you can see, I didn’t download any source file, instead I was happy with using apt-get and the hpricot version from ubuntu repositories, although they are relatively old (for example rubygems is more than a year old). If I find any problem and need to update to newer versions I’ll report that here ;-)

It’s a pity they don’t have a metapackage for ruby’s development files (the ruby1.8-dev package), the same way there’s a ruby metapackage which depends on the ruby1.8 package, so whenever ruby is updated it will update the ruby version as well, without the user having to worry about the version number.

Even more, I instinctively tried a naive sudo apt-get install rubydev and was greeted with a sad “Couldn’t find package rubydev”. It somehow proves that a metapackage called rubydev would be quite useful… at least for instinctive users.

Enjoy your screen scrapping!

But how do you detect if there’s a description? It seems the answer was rather simple: use term.next and if the next element’s name is dd, we’re lucky and have a description. The only problem was that I didn’t know how to access the name of an element, until I just thought: what if I simply use name? and guess what… it worked! So term.next.name was exactly what I looked for :-)

Source at supersnippets.

I also extended this a bit to save the results into a database, using ActiveRecord, but since each db schema is a different world, I didn’t post that version here. If anybody thinks it might be useful just let me know.

Also, this code is not very rubyesque yet, suggestions in order to improve it will be really appreciated. I’m specially thinking about the if … else parts, I’m pretty sure there’s a way to shorten those lines :-)

First I created a simple test page. It’s got some html elements, some have id’s, some contain certain text nodes. It looks like this:

The question was how to remove certain individual elements given certain conditions – more specifically, when the element attributes matched a condition. I don’t see why he had problems removing stuff with the remove method, since that’s what I have used. Since search returns a collection of elements, you just need to get a collection which contains only the element you want to remove, and then apply remove to that collection.

Here are three examples:

Removing the paragraph with id = bad_attribute

We find out the element using CSS selectors, where the hash means ‘id’.

Removing all the unordered lists (ul’s) which have an style attribute

Again, using CSS selectors:

There’s more info about CSS selectors in the Hpricot CSS search documentation. One can get very creative with this and allows for filtering almost everything!

Removing elements whose contents match certain conditions

When it’s not enough with CSS selectors, we can perfectly take advantage of ruby!

For example, if you want to remove list items (li’s) whose text doesn’t begin with E, you could do it with this:

which is the same as saying:

• Look for every list item in the document
• Take the results of that search (which is an Array of Hpricot Elements) and apply the collect! function to them
• collect! executes the code in the block for each element and stores the return value in an array
• But as it can return nils (when the inner_text doesn’t begin with ‘E’ and hence doesn’t match our little regular expression), we remove nil values from the array with compact, so that we don’t get errors when removing.
• And finally, remove the elements which are in the resulting array, with the classical Hpricot remove

Note how I used collect! instead of just collect, so that the changes are applied over the search results, and we don’t get a new array instead.

You should try using collect instead of collect!, and removing compact from the chain, to see what happens.

Final result

If one applies all these evil removals to the original code, the final result is this:

Pretty empty, isn’t it?!

Download these examples

I’ve uploaded the hpricot_remove_elements.rb and test.html together in a zip file: hpricot_remove_elements.zip. For running it, just unpack, and type ruby hpricot_remove_elements.rb

Or open with textmate and press Option+R ;-)

What can you do? Well, you could do like me and create a little script to generate an HTML file, with thumbnails and even watermarked images (just in case some creepy individual decides to use your stuff without asking first).

Requirements

This script requires a couple of gems to be installed: RMagick and builder (but if you’ve done some stuff with Rails you might already have them). RMagick is used for dealing with the images and builder is used for generating the XHTML. This is because I didn’t want to write any html by hand, with their less than and greater than signs, attributes, etc.

Using it

1. Create a folder in your computer. For example: holidays.
2. Then you copy there the pictures you want to show to the world.
3. Open a terminal and cd to that directory. E.g.

   cd ~/Desktop/holidays

4. Execute the script! E.g.

   ruby ~/code/superminigallery.rb

5. Wait until it finishes

When it finishes you’ll find there’s an output folder in the holidays folder. That’s where the index.html file, as well as all the thumbnails and watermarked images are. Simply upload the contents of this folder to your host and let everybody know about it!

Ok, but show me the code

The first lines act like a configuration area. You can change the output folder name, so that it is called superoutput, gallery, whatever you like (as long as it is a valid path name).

You may change the sizes of the generated pictures; these sizes are defined in the versions variable. Each pair means [width, height]. For example, the thumbnails are 300 pixels wide and 150 pixels high.

You can also configure which EXIF tags need to be retrieved. Since their names are too obscure for non-technical savvy people I decided to create this hash for storing the key (Exif tag) and the nice name to show with the value. So instead of showing DateTimeOriginal, it will simply output Taken.

There are way more tags you could show, but they can be confusing for normal people and only entertain geeks, so it’s better to keep them down to a minimum.

Declare the builder object, and initialize it with the XHTML header.

Now it would be amazing to have some styling in the page so that it doesn’t look so ugly. We can put an style tag inside the head, and use the text! method for adding literal text to the builder object:

(the … means there’s more code but I have reduced it for clarity purposes)

Now, we need to create the output directory. I haven’t bothered with outputting error messages if the directory already exists or anything. It will always try to create it:

We need to create a Magick::Draw object for watermarking the images, and define its parameters:

Basically we are saying: use Helvetica bold 64pt, painting it with white (ffffff) and some transparency (99 for alpha channel). If you don’t have Helvetica installed in your system, replace it with your favourite font.

(But since 2007 is Helvetica’s 50th anniversary, you should do everything possible to use Helvetica!)

Now we open the current directory (where the script was executed) and find all files with jpg and JPG extensions, and sort them. That’s because sometimes the images don’t get listed in alphabetical order, and us humans like to see things in sequential order. Specially because they usually are numbered incrementally, and older numbers mean older images, so IMG001 should appear before IMG100.

Read each file into a Magick::Image object:

And for each version…

… create the version filename by appending the version name to the filename, like big_IMG_1234.jpg, and the output filename, by prepending the output path to the version filename:

If the version is ‘thumbnail’, we’ll add the image metadata to the builder object. Note how you don’t need to open or close tags, but just include things in blocks or parenthesis to get the mark up done.

Resizing the image is as simple as

crop_resized returns another Image object which we store in the version variable.

Now, if we are dealing with the ‘big’ version, we’ll add the watermark that we prepared at the beginning. That is done with

You can replace my (c) soledadpenades.com with your text, of course!

And for writing the resulting image to disk:

Very very important: do not forget to call the Garbage Collector. For some reason which I still haven’t been able to elucidate, the RMagick gem leaks memory furiously. So if you forget to do a

as I did with the first version of the script, your computer will mostly hung if you make it generate a lot of thumbnails. If you look at the current processes with top or a similar tool, you’ll find a ruby process eating more and more memory with each picture it processes.

And finally, we just need to output the generated XHTML to index.html:

Here’s the result and here’s the source code. With only 120 lines of code (excluding the license text :D), it’s way easy to modify to suit your tastes.

Don’t tell anyone but…

I must confess I got the inspiration for this from herotyc’s jGallery (edit: link pointed to a dead site so I removed it). But he used a bash script and I thought there should be a way of doing the same with ruby :-)