soledad penadés

I didn’t really feel like doing too much today but I managed to convince myself to finish the loading of relevant data from the song’s file.

I now only have to instantiate patterns, pattern entries and synth voices and send them that relevant data when loading. And adapt my old tracker-style playing routine to work with Renoise 2’s new philosophy (they got rid of the “speed” global song parameter which we were carrying on from the old-tracker era, but I was using that parameter in my player).

There’s not much to show yet, but anyway here’s an extract of the information the test program is outputting while loading a song:

So that you can see that I’m not slacking ;)

I have finally decided that I’ll load directly Renoise’s Song.xml file instead of preprocessing it as I did with the .it files in to_the_beat.

Right now, I’m manually uncompressing the .xrns file and copying the contained Song.xml file to my data folder, but in the future I would like to automate the process so that I just leave a song.xrns file in the data folder and it gets uncompressed “on the fly”, and the .xml file is extracted and used.

Any suggestion for a lightweight unzipping library that doesn’t need to be installed to /usr/local or anything like that and can be easily added to a C++ project will be really appreciated. Yes, I know I can ‘Google it’, but I am asking for opinions from real library users. I wonder if there exists something like TinyUNZIP… :-)

Back to the XML front, I am using TinyXML for reading/parsing/traversing the XML, as I have been doing since 2005 since Shine suggested it to me, but I have also updated it to the latest version :D

Since Renoise’s song format has way more nested elements than the common script.xml that I use, I have written a getElementByTagName function (like a mix of getElementById and getElementsByTagName functions from JavaScript) so that I can look for child nodes with a given name, instead of linearly processing the full document as I had been doing when parsing the demo script. This way I can ignore things that I don’t care about such as the samples, envelopes, etc, and focus only on the data I am interested in – such as patterns, plugin parameters, etc :)

I guess the function could be really optimised since each time it looks for something it has to start at the parent node again — but then again there’s not much data to parse, and initial look ups are pretty much immediate so I’m not going to bother about it.

I haven’t finished yet extracting the relevant data, but there’s something else I have done: I tried yesterday’s test in a real Windows machine (mrdoob’s Eee) and it worked, quite better than in Wine. So things look like progressing in the right direction!

No screenshot, nothing to see today, move along!

“Blue Tuesday” is a direct evolution from the codecolors code base. Since it was all done in a hurry, there were lots of things which didn’t work as expected. I somehow got rid of some of them when I ported the demo to mac, then I got rid of some extra things these days when I made the demo work in Linux, prior to releasing these sources. I am not happy with this code; with so many modifications it has grown way too much to be still readable. There are effects whose behaviour is not predictable, things like the ribbons are very inefficient and so on.

I’m still not completely happy with the synchronization. In the first version trace did a simple flash app in which he tapped a key each time an event happened (e.g. a snare hit) and then generated a list from there. But due to the way the demo is structured, I had to split the list in three parts so that I could check whether an event happened in each effect. Also with so many changes in the code, the synch had gone slightly awry and when I managed to compile the code in Linux, it clearly was asking for a revision.

So I thought about using Audacity for re-recording the synchronization. I would store each event as a label (Audacity has something called Label Tracks) and then exporting those labels as txt files and processing them to convert them into a .h file with all the events in an array. I had the idea of opening Audacity’s project with a text editor and found out that it was a simple XML file — which would make even easier the task of importing the labels’ information, without having to go through the step of exporting to a text file and parsing it.

I then tried to process the XML file with Python, but it was a horrible experience. Since it has a namespace on it and I wanted to do a simple XPath search, it seems I was doomed to fail without installing a couple of Python libraries — which I didn’t want to install. My main premise is to make my demos simple to compile, and having to install an extension just for parsing XML is not what I consider “simple”. So I resorted to good old PHP’s XML functions and in 20 minutes the import script was done. That was the easy part.

What was really painful was recording the synch points! Audacity took hanging itself up as a hobby. Thankfully, I have the CTRL+S tic –in which whenever I’m working with any program I tend to press CTRL+S regularly, pretty much each time I have typed in something, just in case– and that prevented me from losing my changes more than once. But it was just annoying to have to kill the program, open it again, say yes to “yes please recover the project I was working on when you decided to crash”. Even worse: at some point, the project got corrupted, and when I reopened it again, Audacity just got stuck switching between an sleep/idle status. I had to create a new project with the same mp3, save it and then edit the new project in a text editor, and copy and paste the old label tracks from the initial source project. Luckily it was all XML. I can’t imagine what would have I done if it had been binary data! (Probably scream a good lot!)

So lesson learnt: for making good synchronization you need a dedicated, and probably integrated, piece of software. If I need to do something like this again, I will probably spend a good amount of time in preparing something like the editor blackpawn created. Because it wasn’t only a problem of crashing, it was also the problem of not having a comfortable interface which lets you go back and forth the song, reduce its speed, etc, without having to use the mouse.

This is probably one of the reasons why I prefer to make demos with my own music: it’s easier to access the original song source files and build a list of events from there if I wanted to :-)

Enough chattering, go watch the demo or get a nice headache just by looking at the code :P

Following the discussion in the previous post (Reasons for using UTF-8) I thought it could be interesting to gather a series of steps needed to get a UTF-8 friendly environment.

I’m going to focus on php and mysql, because using mysql and ruby/rails and utf8 tends to be kind of easier (specially since newer Rails versions suggest mysql which charset to use when connecting automagically for you), but the advice can be applied to both platforms in any case.

In your preferred editor

Make sure your editor is set to use UTF-8, specially when editing templates and any other file which is used for building output content. If you include any non-ASCII content (for example, accented words) and they are mixed with more UTF-8 content (from other templates or sources), things will get messed up.

In your html/xml code

Make sure the document’s charset is specified.

In HTML documents you would accomplish this with the content/type meta tag, which you should place in the header:

In XML documents this is done in the XML declaration, which needs to be placed at the immediate beginning of the document:

I have heard recommendations for placing the page title immediately after the Content-type has been declared, so that browsers can switch to the right charset immediately if the page title includes any non-ASCII code, but it sounds a little urban myth to me.

I have also seen people recommending to specify the encoding in every form you have in your site, but I haven’t found any difference between doing it or not.

In Apache

Make sure the content is being served as UTF-8. A good AddDefaultCharset utf-8 should do. You don’t need to change all your hosted content into UTF-8 if you don’t want to, charsets can be configured per virtual hosts as well.

In the database

Make sure mysql is configured to use UTF-8 at server level.

Or if you can’t modify my.cnf (shared servers anyone?), before issuing any other query, and right after you’ve connected to the server, execute

It seems otherwise mysql will not recognize properly the character set that the client is using (!!) and will return bad data.

Of course, when you create the tables, make sure that every column which can include text data (this includes TEXT, VARCHAR, etc) is using utf8 as well.

In php

I rarely need to send a header specifying the content-type, but if things are not working totally fine you could also try to manually specify the content type with the header:

Or application/xml, whatever you need!

In Flash (or Flex)

Do not use systemCodepage. Flash assumes XML is UTF-8 so if everything else has been properly set up, Flash will be served UTF-8 content, which is what it expects, and we’ll be happy :-)

One more thing

Although generally it’s a good idea to run a validator, it’s even a better idea if you’re doing utf-8 stuff and want to make sure you’re not outputting bad stuff inadvertently. Validators are picky (even more if we speak about XML validators) and will cheekily reveal your inappropriate characters.

Missing something? Anything wrong?

I’m not an expert and I may easily have forgotten something so if you think something is missing or plainly wrong you know what to do ;-)

And before you ask – I don’t have any experience with Pylons, Django, TurboGear, Zope or whatever your favourite Python framework is, so feel free to share with us your experience.

If you need to parse some XML with namespaces in ActionScript 3 you may find this error quite soon: VerifyError: Error #1025: An invalid register 1 was accessed

It all comes because of this line:

It is needed in order to access the values in the namespaced value so removing it is not an option. The problem appears when you call any other function in your object… and you get that weird error.

I haven’t managed to find out why. It kind of seems that changing the namespace (as the very example from Flash’s reference on how to read an RSS feed suggests) also alters the namespace of later function calls, and so the interpreter can’t find the functions.

I found the simplest solution was to reset the namespace back to nothing when I was done with accessing the XML:

There should be an explanation for this but I haven’t yet found it. Meanwhile this can save your life :-)