Notes on Live.JS at JSConf.Asia 2016

I had sketched some ideas for the Web Audio Hackday before flying to Singapore, but I really had to finish them before the actual day came. So I spent Friday doing an intense labour of testing, testing and more testing with the latest versions of the browsers that implement MediaRecorder. Since I was on the business of talking and promoting this API, I was determined to get more people to try it out, but that meant that I needed to be prepared for the “surprises” that working with new and upcoming APIs entail. Namely, that what worked yesterday might not work anymore, and what didn’t might have started to work without prior warning.

That said, my infinitely curious self would not resist flying to such a beautiful city as Singapore and not exploring a little bit of it: I squeezed some time out of the morning to visit the Marina Bay gardens, which again I found baffling (botanical gardens which are cooled down instead of heated!), then did a lot of work, and then later in the afternoon I went to the CSS Conf/JS Conf venue, as Live.JS was going to happen.

And “What is Live.JS?”, you might be wondering. Basically it’s a collective that is dedicated to making audio visual shows using JavaScript. JavaScript!!! It’s a bunch of people, but that doesn’t mean that you will always get the same people in each “show”. In this case, it was

  • Matt McKegg, which would play music using his own Loop Drop instrument for live performances. He built it using Google Web Apps initially, then migrated to Electron when GWA was starting to stagnate. He uses two Novation launchpad controllers, and the sound generation itself is via Web Audio. And it is really impressive, if you ask me!
  • Ruth John, who was VJing. Not sure what the name of her software is, but she makes heavy use of CSS variables (to animate stuff), SVG to draw elements on screen, and of course Web Audio’s Analyser nodes to modify the values of variables, in response to the music being played. Also, Web MIDI so she can use external MIDI controllers to do things such as change layer opacities and switching between effects at the turn of a knob, etc.
  • Martin Schuhfuss, who was controlling the lights in the venue using the DMX512 protocol (which happens to be a close MIDI relative) and a Monumental Hack with CSS property interpolation to run light animations: colour, rotation, focus, etc. This was quite impressive to hear!

I personally I’m a big fan of Matt’s music since I saw him live in CampJS, so I was excited about this event. And of course, I was quite intrigued about the visual side of things. Hearing about the techniques underneath made me giggle and be in awe at the same time: “can’t this be done in a better, proper way, or are these hacks just used for the sheer pleasure of subverting the original purpose of the technology?”, I wondered.

In the meantime, “special cocktails” were served—although they were actually called “codetails” in the leaflet:

  • ES4: Old Fashioned
  • jQuery: Rum Punch
  • Outsourced: Curry and tonic
  • Unicode: Unicorn tears (non-alcoholic)

I had “a jQuery” as it seemed the sweetest of the punch, and chatted around with a bunch of people, then went and admired Matt’s total LoopDrop skills, and wondered again if I’d ever understand how it works for reals.

Then I focused my attention on what the lights were doing: many things. I guess my issue was exactly that: they were doing many things, and I was kind of expecting some sort of carefully built progression which would not happen, because the LiveJS people had just met a day or two ago, and Martin had only had access to the light equipment the same day, because obviously you don’t travel from Germany with a rack of professional light equipment. So there’s only so much you can prepare, and given that Matt’s music is so improvisational, I should probably not expect a heavily synchronised audiovisual show. Still, I couldn’t but admire two facts:

  • a browser was interpolating between CSS values and ultimately this was generating MIDI commands that caused spotlights to move and change colours.
  • that the DMX512 standard did indeed work well enough that he could just show up and control DMX512 lights with his software, without further changes

That was super cool!

People often fixate in the business value of JavaScript, but it’s important to also consider JavaScript as a creative medium which has lots of potential expressivity and also reach by virtue of being online. I’m glad that Live.JS exists to inspire and support this, and I’d love to see even more exploration in this field. Bring it on! 🤘🏼

Here’s the video of this event:

On CSSConf + JSConf 2014

TL;WR*: a mostly social event, great for meeting the authors of those modules you see scroll past when you run npm install and it installs half of the internet. Also, lots of presentations on somewhat hipster stuff which I not always understood, but that’s great–I like not understanding it all from the get go, so I can learn something. And some discussion about physical and mental health and better community building and other important non purely technical stuff that usually never gets the chance to be discussed in tech conferences.
Continue reading “On CSSConf + JSConf 2014”

What will 2013 bring you?

What will 2013 bring you?

This is a little funny toy? experiment? thing? that I’ve built with three.js to wish you all merry festivities everywhere and also maybe even make you smile. Or laugh, which would be even better!

I wrote some more details on the project page, and I’d rather not repeat myself, so pop over there if you want to have a look 🙂

Macabre Pool

Macabre Pool

This is the result of my fascination with skeletons* and me wanting to learn about Box2D in Javascript.

Since everything is randomised, you might get two big skeletons or a big one and a small one, or two small ones. They sometimes look like they are alive, with spasms and all, while they jump around the screen with the head in their hands. It’s very creepy.

Try it if you dare!

Box2DWeb is quite funny; I should do more stuff with it. I had tried to learn Box2D.js before, without success, but this time I stumbled upon Seth Ladd’s series of brilliant Box2DWeb tutorials, and you can deduct the rest.

The graphics are SVG based. I drew them all with Inkscape. I then use CanVG for loading SVGs into Canvas (everything is rendered into a big Canvas), as there isn’t a native SVG method for getting a bitmap from a rendered SVG object. Mr.doob suggests using SVG directly instead of going through canvas, because it might be faster. I guess I’ll try that with something else; for now I’m just rendering to bitmap/canvas once, so it’s a relatively low overhead. I also cache the SVG to Canvas results, so even if you request the same asset several times it won’t be read and regenerated each time.

Finally I also used a couple of web fonts and some CSS 3 stuff such as box and text shadows, which can be finally used without -moz or -webkit prefixes (at least with Chromium 15 and Firefox 7). Pretty cool!

And of course a huge source of inspiration was Mr.doob’s Ball Pool!

* Proof and proof of my skeleton-mania.

Programmatically building drawables

There’s something that really bothers me and it’s doing things that a computer should do. One of these is saving different versions of drawables, in low, medium and high resolutions, for the drawables-ldpi, drawables-mdpi and drawables-hdpi folders respectively.

Coincidentally, I found a couple of weeks ago that the wonderful Inkscape accepts command line parameters and thus can be used for batch processing files.
Continue reading “Programmatically building drawables”