Notes on Web Audio Hackday Singapore 2016

The last installment of notes from my DevFest Asia 2016 trip! Here’s the official day website.

We did three brief presentations:

  • someone (I forgot his name, sorry 😳) introduced BandLab, which is an online service that lets you compose music and also collaborate on it, all together (a bit like “git but for music” except they didn’t use “fork” because it can sound “funny”). Very interesting project, it was enlightening to hear the features that are available depending on specific platforms and limitations, and also how they do use real instruments in their user testing because this is more of a tool to help musicians, not a tool to replace them
  • Matt McKegg introduced the Web Audio node school workshop, for anyone who wants to learn Web Audio but doesn’t know how / where to start
  • And I introduced the MediaRecorder API (here’s an article I wrote), and an experiment I had built for this hackday

This hackday was small, and I didn’t get to stay until the end to see the projects that people worked on, as I had to catch my flight to Australia, but it was good to catch up with Matt, Joshua and Chinmay and learn what they have been up to recently.

For example, Matt has been busy building Ferment, a sort of decentralised peer to peer website/app/service/whatever to share music from creator to creator. It’s built with Electron, Scuttlebot and also WebTorrent—I find it super interesting! I have watched a few talks on all these concepts before but I hadn’t really seen any application that “clicked” everything in place like this one. It’s super exciting to consider that each time you listen to a song you’re becoming another peer and can share it with more people, making the song way more accessible to more listeners. I am already using it, what about you?

Joshua has a folder full of Work-In-Progress projects so there’s no point in me listing all of them—but he’s recently been working on controlling the speed of videos using a bezier curve editor. He was super cool and tried to combine that with MediaRecorder but we found a bug, ooops! Here’s the bug report 😎

I also had the chance to talk to more local developers, and again I was terrible at catching names, but it was fun to see what other people are building with or using Web Audio for. Someone had built a sound board of silly sounds when she was bored with something else, someone else was using Pedalboard.js to experiment with applying live sound effects to the output of his electric ukulele without carrying a whole bunch of hardware with him.

Notes on Live.JS at JSConf.Asia 2016

I had sketched some ideas for the Web Audio Hackday before flying to Singapore, but I really had to finish them before the actual day came. So I spent Friday doing an intense labour of testing, testing and more testing with the latest versions of the browsers that implement MediaRecorder. Since I was on the business of talking and promoting this API, I was determined to get more people to try it out, but that meant that I needed to be prepared for the “surprises” that working with new and upcoming APIs entail. Namely, that what worked yesterday might not work anymore, and what didn’t might have started to work without prior warning.

That said, my infinitely curious self would not resist flying to such a beautiful city as Singapore and not exploring a little bit of it: I squeezed some time out of the morning to visit the Marina Bay gardens, which again I found baffling (botanical gardens which are cooled down instead of heated!), then did a lot of work, and then later in the afternoon I went to the CSS Conf/JS Conf venue, as Live.JS was going to happen.

And “What is Live.JS?”, you might be wondering. Basically it’s a collective that is dedicated to making audio visual shows using JavaScript. JavaScript!!! It’s a bunch of people, but that doesn’t mean that you will always get the same people in each “show”. In this case, it was

  • Matt McKegg, which would play music using his own Loop Drop instrument for live performances. He built it using Google Web Apps initially, then migrated to Electron when GWA was starting to stagnate. He uses two Novation launchpad controllers, and the sound generation itself is via Web Audio. And it is really impressive, if you ask me!
  • Ruth John, who was VJing. Not sure what the name of her software is, but she makes heavy use of CSS variables (to animate stuff), SVG to draw elements on screen, and of course Web Audio’s Analyser nodes to modify the values of variables, in response to the music being played. Also, Web MIDI so she can use external MIDI controllers to do things such as change layer opacities and switching between effects at the turn of a knob, etc.
  • Martin Schuhfuss, who was controlling the lights in the venue using the DMX512 protocol (which happens to be a close MIDI relative) and a Monumental Hack with CSS property interpolation to run light animations: colour, rotation, focus, etc. This was quite impressive to hear!

I personally I’m a big fan of Matt’s music since I saw him live in CampJS, so I was excited about this event. And of course, I was quite intrigued about the visual side of things. Hearing about the techniques underneath made me giggle and be in awe at the same time: “can’t this be done in a better, proper way, or are these hacks just used for the sheer pleasure of subverting the original purpose of the technology?”, I wondered.

In the meantime, “special cocktails” were served—although they were actually called “codetails” in the leaflet:

  • ES4: Old Fashioned
  • jQuery: Rum Punch
  • Outsourced: Curry and tonic
  • Unicode: Unicorn tears (non-alcoholic)

I had “a jQuery” as it seemed the sweetest of the punch, and chatted around with a bunch of people, then went and admired Matt’s total LoopDrop skills, and wondered again if I’d ever understand how it works for reals.

Then I focused my attention on what the lights were doing: many things. I guess my issue was exactly that: they were doing many things, and I was kind of expecting some sort of carefully built progression which would not happen, because the LiveJS people had just met a day or two ago, and Martin had only had access to the light equipment the same day, because obviously you don’t travel from Germany with a rack of professional light equipment. So there’s only so much you can prepare, and given that Matt’s music is so improvisational, I should probably not expect a heavily synchronised audiovisual show. Still, I couldn’t but admire two facts:

  • a browser was interpolating between CSS values and ultimately this was generating MIDI commands that caused spotlights to move and change colours.
  • that the DMX512 standard did indeed work well enough that he could just show up and control DMX512 lights with his software, without further changes

That was super cool!

People often fixate in the business value of JavaScript, but it’s important to also consider JavaScript as a creative medium which has lots of potential expressivity and also reach by virtue of being online. I’m glad that Live.JS exists to inspire and support this, and I’d love to see even more exploration in this field. Bring it on! 🤘🏼

Here’s the video of this event:

One demo: two new bugs!

I thought that since I was going to the Web Audio Hack Day here in Singapore, it would be great to have a more Web Audio focused Media Recorder demo, so I built a little demo that essentially asks for audio permission, then records a short clip of audio, decodes it as an audio buffer and uses it to loop a BufferSourceNode in Web Audio.

You wouldn’t believe what happened next… or would you?

Two Things That Happened When Sole Ran This Demo

1. Chrome Canary Just Totally Gave Up

Aw, snap!

I filed a bug, since there wasn’t really much I could do…

2. Nightly Sometimes Insisted In Returning Some Mysterious 596 Samples Length Which Was Totally Not What I Expected, And Did So Without Following Any Meaningful Reproducible Pattern

596 samples for some reason

I tried to dump the contents of the returned blob once read as a buffer, using String.fromCharCode–perhaps looking at binary data while having my breakfast coffee would enlighten me?

I eat hex dumps for breakfast

As it turns out, no. So I filed another bug.

Try this yourself maybe?

instalooper

The demo is here: instalooper (sources) … just in case you want to play with the code–maybe it will be fixed next month, who knows!!? Or maybe it is just a bug in my machine and it works in everyone else’s!

Disclaimer: I have not tested this anywhere else than Nightly and Canary. So I’ve no idea of how it works (or not) in mobile.

I have a more complicated version that not only loops the sample but also randomly changes its playback frequency. Sometimes the generated sounds are strangely fascinating. It still needs a bit of interface work, but hey… watch out, Steve Reich! 😎

And now… time to prepare to go to the airport and fly to Melbourne for JSConf.AU! More jetlag, YAY!

Talking about Servo in Hackference Birmingham 2016

I visited Birmingham for the very first time last week, to give a talk at Hackference. Apparently the organiser always swears that it will always be the last hackference, and it has been “the last one” for the last four editions. Teehehe!

I spoke about Servo. They didn’t record the talks but I did an screencast, so here’s it:

📽 Here are the slides, if you want to follow along (or maybe run the demos!). The demos come from the servo-experiments repository, if you want to try more demos than the ones I showed.

If you watched this talk at ColdFrontConf, this one has more clarifications added to it, so complicated aspects should be easier to follow now (specially the explanations about layout calculations, optimisations, parallelisation and work stealing algorithms).

People enjoyed the talk!

Someone even forked one the dogemania experiment to display other images:

And Martin got so intrigued about Servo, he even sent a PR!

I didn’t get to see much of the city, to be honest, but two things caught my attention:

a) it was quite empty even during ‘rush hours’
b) people were quite calm and chill

That’s perhaps why Jessica Rose is always saying that Birmingham is The Absolute Best place. I will have to find out some other time!

A very funny thing / activity / experiment happened in the slot before my talk. It was a reenactment of the BBC’s Just A Minute show, which I have never watched in my life. Essentially you have 4 participants and each one has a little device to “stop the show” when the active participant messes up. The active participant has to speak for a minute on a given topic, but they cannot hesitate or repeat words, so it starts getting challenging! This was organised and conducted by Andrew Faraday, who also helped run the Web Audio London meetup a while ago and is always an interesting nice person to talk to.

So this, this was hilarious beyond anything I expected. I guess because I didn’t expect any funny thing in particular, and also because I didn’t have any preconception of any of the participants being a “funny person” per se, so the whole comedy came from the situation and their reactions. It had some memorable moments, such as Terence Eden’s “unexploded item in bagging area” (in relation to the Samsung Galaxy Note 7 exploding fiasco, plus the very annoying voice that anyone who’s ever used a self-service checkout till in the UK will recognise 😏).

After so much laughing, I was super relaxed when the time for my talk came! Every conference should have Andrew doing this. It was excellent!

Other interesting talks and things:

  • Felienne Hermans’ on machine learning and bridge playing AIs built with DSLs built with F# – I basically don’t know much about any of these subjects, so I figured this could be an interesting challenge. You can watch this recording of this talk from another conference, if intrigued.
  • Martin Splitt’s aka geekonaut talk on WebGL – if you have the chance to watch it, it will be quite informative for people who want to get started in WebGL and learn about what it can do for you
  • I’m certainly not a Web Audio beginner, but I tend to watch those talks anyway as I am always curious to see how other people present on Web Audio. Hugh Rawlinson‘s presentation on Web Audio had a few interesting nuggets like Audiocrawl, which showcases the most interesting things in Web Audio. He also worked on meyda, which is a library to do feature detection using Web Audio.
  • Jonathan Kingsley gave one of the most depressing and hilarious talks I’ve seen in a long time. IoT is such a disaster, and the Dyn DDoS attack via IoT devices, just a couple hours after this talk, was so on point, it almost seemed deliberate.
  • Finally Remy declared his love for the web and encouraged everyone to get better and be better to others – and also stressed that you don’t need to use all the latest fashions in order to be a web developer. It’s good when renowned speakers like Remy admit to not to like frameworks, despite also seeing their strengths. A bit of balance never hurt anyone!

The conference had a very low key tone, let’s say that it was a bit “organise as you go”, but due to the small scale of the conference that wasn’t much of a problem. As I mentioned before, everything was pretty chill and everyone was very approachable and willing to help you sort things out. It’s not like a had a terrible problem, anyway: my biggest problem was that my badge had been temporarily lost, but no one told me off for not wearing a badge while inside the venue, and I eventually got it, heheh. So yeah, nice and friendly people, both attendees and organisers.

I also liked that everything was super close to the train station. So there was no need for additional transportation, and we could use the many food places in the station to have lunch, which was super convenient.

Oh and Jessica as MC was the best, I really enjoyed the introductions she prepared for each speaker, and the way she led the time between talks, and she was really funny, unless presenters that think they are funny (but aren’t).

If you have the chance, attend the next Hackference! It might the last one! 😝

Here are other conference write ups: from Dan Pope and from Flaki (who stayed for the hackathon during the week-end).

Talking about Web Audio in WeCodeSign Podcast

I recorded an episode for the WeCodeSign podcast. It’s in Spanish!

You can download / listen from their website.

We actually talked about more than Web Audio; there’s a list of links to things we mentioned during the episode. From progressive enhancement to Firefox’s Web Audio editor, to the old PCMania tracking stories, to Firefox for iOS… lots of things!

I was really pleased with the experience. The guys were really good at planning, and did a great job editing the podcast as well (and they use Audacity!).

Totally recommended—in fact I suggested that both my fantastic colleague Belén and the very cool Buriticá are interviewed at some point in the future.

I’d love to hear what they have to say!

Throwback to the last time I recorded a podcast in Spanish–at least this time I wasn’t under a massive cold! 🙃