Projects

RegExpert - the postmortem

Are you a RegExpert?

I don't really remember when did I get the idea for this game. It was probably some time when writing a text parser. Probably when writing the code for the Wiki in escena.org. Back then it seemed like one of those crazy ideas I get so often, but then I thought: hey, this could be really fun!, and wrote it down on my TO-DO list.

Meanwhile I got busy with several other matters, but still kept the idea on the backlog. On January, I met Belén Albeza (Lady BenKo), a clever engineer from Alicante who had recently moved to London. We had found out we were living right in the same neighbourhood, so we decided to meet and have a geek chat. It was a cold humid afternoon, and wind was blowing down Vauxhall Bridge Road. We desperately looked for an open pub to get some shelter, and sadly The Windsor Castle (ex The Cardinal) was closed for refurbishment. So we settled for The Warwick, which was incredibly noisy, but that was alright: no one would be able to hear about our world domination plans!

Apart from devising evil ideas and absolutely agreeing that there was a real need for a Pimlico Hackers meeting, we had a very interesting discussion about game development. See, my problem is that pretty much all of my professional life has consisted in programming back-end stuff. I think in terms of transactions, batches, and HTTP requests. So when presented with the very real issue of interactivity I get a dizzy feeling, and I don't know where to start. The user can click anywhere at any point in the game? He can leave the game without warning? What about physics? Levels? Editing them? Points!? Bonus?!?! Aaargh! Brain explodes!

But Belén was really encouraging, and explained me how she would program a game from scratch. A basic game, mind you, but it was enough to understand the basics, and most important of all, demystify the entire process. Evidently it's all a matter of refinement, as usual: you start with an amazingly blunt and draft-esque prototype and once you get something running, you keep on iterating and improving things here and there.

First iteration

Proof

Fast forward a few months and it's April already. Not cold and humid any more, but a nice warm Spring midday in London. Fragrant scents in the air when walking under the park trees; young ducklings are learning to swim in the parks' lakes --under the vigilant gaze of their parents--, and all that. I think it was a Friday when I decided to hack a quick proof of concept Android application that would help me develop this regular expressions game idea. This first incarnation helped me basically to experiment with the Java RegEx possibilities, and also with user keyboard input, since the way it worked was: the application would show you a word and you had to type in a regular expression that would match it. It seemed like a great idea!

I even managed to highlight with green the parts that were matched, and leave in red the parts that didn't. It looked like a perfect prototype!

So a few weeks later Mr.doob and me were waiting at Heathrow for the first flight in our exciting World Tour 2011, and I proudly showed him this secret prototype. The result?

Pretty disappointing.

When presented with the "question", what did he type in so that it would match?

Yes: literally the question text. Evidently, that matched the regular expression, since a literal character always matches itself. But it also completely defeated the idea of the game. How do you force on the user to type in an 'intelligent answer', such as one that used an elegant and concise regular expression instead of just repeating the question? Yes, I could maybe devise some algorithm that rewarded 'elegant' and shorter regular expressions. But I could already see the wild card issue looming in the distance: everyone would just answer with .* instead of coming up with a proper regular expression.

Besides, Mr.doob said, typing text in an on screen keyboard is so incredibly tedious, no one will do it.

But... but... tablets!, I tried to reason out. It wasn't very convincing--I didn't even believe it myself. Besides, it was time to board the plane.

As usual he was right, but the game had to wait until we returned to London.

Second iteration

It was early June and I thought I would be done with the game in a month time. Ha! Let's keep it short and say I had an exceptionally event-packed summer. So from June we jump to middle August and I have rebuilt the game and replaced the player types the regular expression in mechanic with player chooses the right expression from a list. That way I would get rid of both issues: typing in the minuscule phone screen, and 'cheating' by writing literals instead of expressions.

Second iteration

Of course, there's always a 'but': I showed the game again to Mr.doob, and he was right again: the game wasn't attractive at all, because there wasn't a clear objective in sight, and there weren't obvious incentives to keep playing either.

I could have noticed it myself if I had written the mechanics of the game down first. If not even I could explain it in a clear, convincing manner, how could I expect any player to engage with the game? The idea was that you started playing in a certain level, and if you filled the time bar, you would get "bonus". Then the time bar was reset to its default value, and you had to fight again for more "bonus". And repeat the process until you earned enough bonus that you reached "the next level". But there wasn't any indication of which level you were in, or --and this was the worst part-- how long was left until you would reach the next level.

The problem, in other words, was that the game wasn't crystal clear. It wasn't self-explanatory. It was like one of those infinite scroll loaders: you go on and on and on and you never know where's the end. Suddenly you get a bunch of extra items at the end of the page, quite unannounced, but you never know exactly when or why. So if you're bored you might go on for a little while, but if you're not really in love with regular expressions... you quit the game. And probably get a refund too, while you're at it.

Third iteration (and final!)

I had started glancing through The art of innovation while fsck did its work, on an incredibly warm evening when my brain was on the verge of melting, and I found a quote that instantly struck a nerve:

No idea is so good that it can't be improved upon

Of course!!, I said to myself. It was like a reminder of Benko's recipe for game development: iterate, iterate, iterate. And with that in mind I went on until the game was published.

First thing to do was changing the gameplay. Instead of featuring an infinite game mode, it would be divided into rounds with a defined amount of questions. This would be shown on screen so the player knew where he was. Since the rounds are relatively short, players will probably try to at least finish the round (OK it's just a couple of questions more!) instead of just abandoning. Then because the game offers some incentives he will try to keep on playing anyway!

Which incentives?

First the time bar would not jump back to a lower value when reaching the "bonus" point. Instead now players score more points the more time left they have. So it's in their best interest to ensure they have as much time as possible. The bar gets more white the more time is left, and turns to red when time is low, to provide for a visual cue.

Also, the remaining time is carried on to the next round. This helps to establish a continuity, as in: you were playing this round, and in the next one you only have the time you brought with you, because we're still in the same game, and also an incentive, as the player must make sure he's got enough time for the next round: there are no refills when starting a round.

Finally, there are the combos: every time the player answers two or more questions correctly, she gets even more points, and an encouraging message on the lines of Amazing!, Incredible!, etc, appears.

I also added different, more varied sounds. Until now, I was using only three samples: one for 'right' answers, another for 'bonus' answers and finally another for 'wrong' answers. It was a bit monotonous. Mr.doob suggested playing a higher note sample with each consecutive right answer. I had tried to do something of the sort programmatically, but I had stumbled upon the fact that some phones (namely the Xperia Play) do not have a proper implementation of SoundPool, and that meant I couldn't dynamically alter the samples' pitch by simply changing a parameter in the method call. I could write a sound mixer and do the processing and resampling myself, but I thought that was a bit too much overkill for my needs.

Now that there was a limited number of questions per round, I could simply prepare a fixed number of samples playing a musical scale, and use them along the round questions. And that's what I did. Not only did the game lose its musical monotony; it also added another incentive, as the player can tell how far he is in the round simply by listening to the sound effects, and will get encouraged to answer the right question so that he can listen to the 'nice' sample instead of the more dissonant 'wrong' one.

I also tried to make the game nice-looking, albeit keeping it very simple. I think it's important that the game is not only funny but feels well designed as well.

I'm not saying this one is a paradigm of game design, but at least it's got margins, paddings, alignments and a hierarchy of elements in the screen. I was also advised on this by Mr.doob: the scores and time bar and everything were first on the lower side of the screen, which made it very difficult to keep track of what happened underneath your thumb and above on the top of the screen at the same time. Following his advice, I moved these elements to the top side of the screen, and removed some of the information, making it less distracting on the process. Here are two images so you can compare:

Before

Before

After

After

And when I showed this version of the game to Mr.doob he at last felt it interesting enough to keep playing--even if Regular Expressions don't particularly interest him. If that isn't a proper test, I don't know what else can be it!

Tricks of the trade

As usual when developing something new, I learnt several things about the platform, and also came up with some techniques to work around the inconveniences. I'd like to share some of these here, so maybe they'll help you (by warning you) or maybe you'll help me (by letting me know I was doing it wrong).

Strings, formatting and arrays

This is hardly mentioned in the usual tutorials or articles, but turns out there are several useful resources for working with resource strings in Android, and if you ever do anything with a decent amount of text you'll probably need them. The first one is the static String.format method. This is the Java equivalent of C's printf, and sadly it took me a lot of time to start using it.

So if you wrote this in C:


printf("Hello, ducks: %d", 22);

you would write this in Java:


System.out.println(String.format("Hello, ducks: %d", 22));

The first solution that I used for using localised strings with formatted text was to combine two calls, like this:


String.format(getResources().getString(R.string.hello_ducks), 22);

But Android has another very useful version of the getString method built in the Resources object, which saves you from writing one of the method calls. The above code is functionally equivalent to this:


getResources().getString(R.string.hello_ducks, 22);

And it gets even better, because you can also specify positions in the arguments when writing the strings. This is great for translations. So instead of writing


<string name="hello_ducks">Hello, ducks: %d</string>

you must write it as this (note the added 1$):


<string name="hello_ducks">Hello, ducks: %1$d</string>

That doesn't make much of a difference with this simple example, but if you have translatable strings with more arguments it makes quite a lot of sense when translating for a language in which the arguments must be in a different position. I'll let you explore and find its usefulness.

Another interesting 'trick' I found was the ability to use arrays in resource strings. This is incredibly useful for things such as month names and in general any sort of string list. They are declared in the strings.xml file and there are two versions. The first one is pretty literal:


<string-array name="friends">
    <item>Dick</item>
    <item>Tom</item>
    <item>Harry</item>
</string-array>

For accessing this in the java code, you'd simply use this:


String[] friends = context.getResources().getStringArray(R.array.friends);

There's another version in which you can refer to already defined strings in order to build a new array:


<string name="friend1">Dick</string>
<string name="friend2">Tom</string>
<string name="friend3">Harry</string>

<array name="friends">
    <item>@string/friend1</item>
    <item>@string/friend2</item>
    <item>@string/friend3</item>
</array>

The 'efficiency' of SharedPreferences

Although you don't see it, there are many things happening at the same time on the game, and these need to be stored, to make sure the player can get interrupted by a phone call, then go back to the game and keep on playing as if nothing had happened. I decided to use SharedPreferences to read and store the game status, pretty much out of habit.

Later on, I somehow regretted this decision. It turns out that every time you write a preference, a full blown xml serialiser is invoked. That wasn't a total blocker, but it bothered me nevertheless. I alleviated this by adding a check on my own Preferences class, to make sure that values were written only if they were different from the ones on 'disk'. Still, it is using Android's SharedPreferences internally. In future projects I'd like to write my own simple, barebones preferences manager, to avoid having this worry on mind.

Script everything!

This wasn't a new discovery, but a confirmation that if you invest some time in writing a script, that will show its value later on. I scripted all that could be scripted in this game. Previously I used an script to prepare the different bitmap versions that Android requires, as I wrote here.

This time, I also added scripting for converting the game sounds into something Android liked. The workflow was like this:

  1. Compose song / samples in Renoise. Each pattern represents a sample in the final output.
  2. Render the song with Renoise, selecting the option to render each pattern to a different file.
  3. With a Python script, sox and oggenc, maximise sound levels, trim as appropriate, and convert each sample to OGG. Copy them to the project 'raw' folder.

It might seem that automating only one part (the third step) is insignificant. But when you edit the samples several times and want to know how they sound in the game, it isn't very productive to have to manually open, edit and save every single sample using Audacity.

Web app or native app?

Benko thought that the game was actually a web page, embedded on the device as a webKitView or something of the sort. No, it's a native app!, I said. But also confessed I had had my doubts about which way to go at the beginning of the project. I settled for 'native' because I thought that it would offer the maximum performance, although after developing the game I'm not sure if that was the best way to go.

If you're happy with the way Android widgets look and behave, using them is pretty straight-forward. Unfortunately, the very moment that you want to make a native widget look or act differently, things get slightly complicated--specially if compared with the ease of HTML+CSS.

Let's take a very simple example: using a custom font face on a button.

You could be tempted to guess that it only takes an XML attribute on the layout file to describe the font that the button will use. Well, you'd be wrong (as I was). If you want to use a non-native font in an Android app, you have to copy it to the 'raw' folder, then create a typeface when the application starts:


typeface = Typeface.createFromAsset(getAssets(), "fonts/myfont.ttf");

and then set it to be the button's typeface. Since this is done in runtime, you first need to grab a reference to the button, and then set the typeface:


Button b = (Button) findViewById(R.id.myButton);
b.setTypeface(typeface);

You can parameterise it, apply as much DRY as you want, etc, etc, but the fact remains that this is overly complicated--specially when compared to simply writing a couple of CSS rules and letting the WebKit engine do its job.

Android also offers some sort of 'skinning' by the use of 'themes'. This is also so obscure and complicated that I finally decided to write some of my own classes or extend existing ones to programmatically change widget attributes instead of using themes, because there was no way I could exactly accomplish what I wanted to do. And I just wanted to change some padding and margin in the buttons! It was even more insulting to think of how easy it is to do it with a CSS class.

Which led me to think that maybe I took the wrong decision when 'going native', since it also locked me in with a lot of Java code that cannot be ported elsewhere. Even my brain began to have quirky unsolicited ideas halfway the project: strange hybrids using C/C++, Lua and NativeApplications, or maybe WebKit and Javascript and some Java glue for storing preferences, and whatnot...

In the future, I'd like to investigate more on these alternatives, because clearly the Android UI toolkit is just getting out of control, and my time is too valuable to spend it fighting a megalomaniac framework.

Conclusion

Although I have had my share of frustrations with this project, I'm happy with the final result. I've learned that even the simplest games have lots of work behind, but I've also gained some confidence on how to approach game design, which was exactly the aim of this project: to lose my irrational fear to all things interactive.

I've also found several no-no's by myself--which is the perfect way of not forgetting them and repeating the errors.

Hopefully future games will be better!