Tag Archives: ruby

ladieswhocode 20130508: adventures in paranoia with sinatra+sequel, and networking

ladieswhocode is an informal meeting that takes place every month in London (and I understand other cities too, but I haven’t been there so I can’t tell!). At the beginning I was a bit skeptical as I’m not a big proponent of gender division–I’d rather prefer everyone, no matter the gender, to be able to sit in the same room and listen to speakers of any gender as well. But after attending a few meetings I can understand why some women rather not go to the usual, 99% male, technical meetings. There’s quite a bit of fear about not being good enough, and it’s quite interesting to listen to more female developers share experiences that I thought were unique and isolated to my case… but apparently are not.
Continue reading

A first impression on Ruby’s Mechanize

I had to do some screen scrapping yesterday and, while I previously have used hpricot for these kinds of things (or maybe even just plain cURL or similar), this time the page required me to be logged in, and not using any HTTP “standard” way of Auth that cURL could deal with, but a custom CMS html-based login form. So I understood that I needed something more powerful; something that could pass as a “proper” browser and not just a simple crawler.

I had heard quite nice successful experiences with Mechanize before, so I decided I would give it a try, and it turned out to vastly exceed what I expected from it! :-)

Mechanize is a port of the original Perl Mechanize library; there is a Python port too, and there might be ports for other languages too but I wasn’t interested in that.

For some reason my mind was in the ruby-mood yesterday, so I decided to go for the ruby port. Everything was flowing nicely between ruby and me. I even dared prototyping what I wanted to do in a terminal, using irb, before writing the final script I needed to write. My ruby muscle was getting toned again, so to speak.

I got some false starts, though. I first invoked just ‘ruby’ in the terminal, only to be greeted with a waiting pipe, i.e. nothing happened as –I guess– ruby was expecting me to provide him with something to run. I remembered that the interactive ruby executable was called irb. A Control+C and four other keystrokes later, I was in immediate-mode ruby.

Another gotcha: not having to

require 'rubygems'

when running in irb, but having to when writing a script. Thankfully that I remembered and was able to fix quickly.

But I’m digressing. Back to Mechanize: it simulates a real browser interacting with a website. You can even fake the user agent. But unlike the most basic screen-scrappers, which simply download a page and then do something with it and then download another one and do something else, without keeping any sort of continuity between downloads and connections, Mechanize is stateful. Which means that it keeps the state between ‘visited’ pages. To all effects, and for the visited websites, there’s a normal browser at the other end of the line. Unless you go crazy and begin hammering a server with tons of requests, crawling websites this way might be almost invisible to servers.

Which is exactly what I wanted!

The syntax is quite nice and intuitive. Borrowing shamelessly from the manual:

require 'rubygems'
require 'mechanize'

agent = Mechanize.new
page = agent.get('http://google.com/')

# List links on the page
page.links.each do |link|
  puts link.text, link.href
end

# Click on the first link with text 'News'
page = agent.page.link_with(:text => 'News').click

# Do a Google search, using the search form
google_form = page.form('f')
google_form.q = 'ruby mechanize'
page = agent.submit(google_form, google_form.buttons.first)

Something that made me even more happier is that Mechanize also uses Nokogiri internally for parsing HTML–which means we can do the same style of nice DOM tree traversing that I used to do with Hpricot, only even better! (Nokogiri is the successor to Hpricot).

I didn’t need that last feature for this particular case, but it left me wondering what I could use Mechanize for–in order to use this! I will have a look at my TODO list and see where can I use my newly acquired Mechanize skills!

My favourite GIMP plug-ins

When I used Ubuntu I normally installed a fancy package called gimp-plugin-registry that came with most of the plug-ins I liked, and then some more stuff that I never used. I had learnt to ignore that, but there was still the pain of not knowing where things were. Some filters went to “Colors”, others to “Light and shadow”, etc. It was extremely hard to remember where each filter was meant to be.

Now that I’m using Arch I couldn’t find any equivalent package but I thought that it was OK–that way I would only install what I really needed to install. In the process I’ve found a couple more plug-ins that I didn’t know about, so it’s been quite productive.

What I’ve done is I’ve written a script that takes care of downloading the scripts from the specified URLs, placing them in the appropriate gimp folder (~/.gimp2-6/scripts in my case), and the best of all is that it also patches them so that all are under the Filters/Photo menu entry. Yay! No more wandering around menus and submenus! My brain is now happy, I can just focus on playing with the pictures!

If you want to use this script just download it (it’s on my snippets repository). You might want to modify it as you wish. There are a couple of TO DO’s but the file is fully functional (as long as you’ve got Ruby installed).

Oh, and pardon my rudimentary Ruby. It’s been ages since I last wrote anything with Ruby. I still don’t know what I want to use. Ideally I’d like something as easy as php but with a syntax that is a mix of Python and Ruby, and lots of easy functional and metaprogramming.

I guess my problem is that I know too much, but still so little, not enough. Maybe if I just knew one language, I would be happy with that one. In any case, I’ll keep on experimenting and try to decide what I like best. Maybe my Japanese side will win over the European one? Who knows!

Meanwhile, be sure to enjoy the filters! Wooo!

ruby in the pub #4 :after

Julian Burgess (who I had virtually met via Twitter) suggested me I should attend the next ruby in the pub. It sounded decidedly odd: mixing journalists with an interest for code with ruby developers; even more strange considering I’m not a ruby expert as much as I try, so I thought it could be interesting to go and see what an event like that could turn out to be.

So there I went after mentally memorising the map from Liverpool Street Station to LBi‘s offices in Brick Lane. Funny because I have done the Brick Lane-Liverpool St. route several times already, but since it was dark and I got an excellent guide, I never paid much attention. Anyway, I arrived, gave my name to the security guy which was kind of confused with my Spanish name and then went downstairs where the people with laptops on the couches were.

I took a look and didn’t know anyone — so I decided the direct approach would be better: waved and said “hi” to whoever was looking at me at that point, and got a friendly response. Yay!

In a minute I was happily talking with Francesca, a nice front end web developer, then with someone else whose name I didn’t find out, then I got to associate Julian’s name with his face. A little while after, Paul Carvill came downstairs with lots of pizza boxes –all vegetarian, and yummy!– and as we were discussing about Rails 3.0, Joanna Geary made a little speech, more or less on the lines of this:

‘Journalists, raise your hands!’

They do.

‘Journalists, find a developer and explain them what do you have in mind that could be done with code, and let’s see if it’s doable or not!’

And then a girl approached Francesca and me. She looked strangely familiar but I couldn’t really tell why. We sat in a table, took the laptops out of their sleeves and set out to try and find if the data we needed was available. But the wireless didn’t quite work. So we chatted about something fascinating and kind of related to the topic: Volcanoes! It really helped that she had a degree in Geology so she could really offer interesting insight about how volcanoes work. Also, amazing as it sounds, she could pronounce Eyjafjallajökull in one go without hesitation!

We ate some more pizza, and then the wireless started working. So we checked and saw that the data wasn’t available — but we agreed she would contact the organisation and ask them for it. Maybe they already have it but it’s not publicly advertised.

Then it’s when the real coding part began; she said she would love to learn programming, but didn’t know where to start, or the guides she had tried were boring. I immediately proposed Processing; I know there’s Hackety Hack by _why, which is ruby based, but I don’t have any experience with it, so I went for the safer option.

I actually hadn’t done much Processing recently and as such I always forget the names of the basic functions (was it setup, or was it load? was it loop or was it play?) but it was even better, so that way they could follow along. She downloaded Processing and was typing in stuff in the sketchbook immediately… which was amazing! No setup, no preparation, no rebooting the computer after the installation; just download and experiment!

We’ve done just extremely simple things such as opening a window, setting its background colour, changing its size and… drawing a line! Still, I find it really enlightening to observe how ‘normal people’ (i.e. not hardcore developers) react to all these concepts when exposed to them.

Interesting things I’ve noticed (teachers, take note!):

  • people love to see results immediately – no theory to start with!
  • people love to have some starting snippets, which they can modify and experiment with, learning how they work meanwhile

So that makes Processing an excellent programming environment for beginners, because it’s got that beatiful reference written in pretty much plain language and with example snippets that can be copied and pasted for each function/class.

In a few minutes the word had spread: there were people doing stuff with Processing in the room! And so a little audience gathered, another guy (a journalist) came with his laptop, downloaded Processing and was doing little simple things as the aforementioned ones, copying code from her wife’s screen –isn’t it sweet?

On the other side of the table I was showing Paul and Francesca my word processing and statistical experiments with Tolkien works and we also spoke about Processing versus other environments such as Javascript or Flash.

In between, the girl had told us her name, and if the face looked familiar, the name was insanely familiar! She humbly told me she was in lots of events and etc, but didn’t make a big thing of it. I asked her about events I had been in (that I could recall at that point) to no avail –we didn’t seem to have been in the same events. But then I came home and looked for “Suw Charman” and found that not only had I seen her speak at resfest’05 but she had also posted a few messages to a mailing list I follow. I think that’s why the name sounded sooo familiar!

Of course, there couldn’t be a nice event without Rob McKinnon in it! I just love how I always find him everywhere I go, whether it’s a full fledged event or just a coffee shop anywhere in London, it’s just so funny. Couldn’t exchange more than a few “hey hey hi!” waves because we both were busy with our new friends, so I don’t know which sorts of interesting stuff did he show this time (the last time he spoke about Hpricot).

The event ended at around 22h, but mainly because people had to leave and take trains and all those things, but I’m pretty sure otherwise we could have stayed and playing with Processing for a long while :-)

And that’s how ruby in the pub, “with hardly any ruby and certainly no pub”, was. See you in the next one?