Audio Tags:
Web Components + Web Audio = ♥

Soledad Penadés

Apps Engineer at Mozilla

I'm totally into music

2008

Sorollet

Sorollet V1 example

(fragment)


if(synth->pitch_envelope_active)
{
  actual_note += (int) (map(
    sorollet_adsr_update(&synth->pitch_envelope, t),
    0, 1, -12, 12)
  );
}

2010

Sorollet V2

Sorollet V2 example

(fragment)


float pitchEnv = mPitchADSR->update(bufferTime);

my sample song

Sorollet V2 VSTi example

(fragment)


case PARAM_VOLUME1: vst_strncpy(label, "dB", kVstMaxParamStrLen);
break;
case PARAM_OCTAVE1: vst_strncpy(label, "Octave", kVstMaxParamStrLen);
break;
case PARAM_WAVE1: vst_strncpy(label, "Wave", kVstMaxParamStrLen);
break;
case PARAM_PHASE1: vst_strncpy(label, "", kVstMaxParamStrLen);
break;
// ...

Tedious!

People used it!

*\o/*

2011

Audio in the browser

Audio Data vs Web Audio

Why did Web Audio win?

2013

W3C Web Audio API

2013

Web Components

... but what is a "component"?

"A constituent part; element; ingredient."

Screws

Resistors

Doorbell

Web components

== custom DOM elements

And here comes the crazy idea...

Why not audio components?

Perfect sense

Web Audio is modular

Web Components are modular

Audio Tags

Custom audio... with less boilerplate

Let's build a simple synthesiser

The audio context

Where things happen

The audio context

<audio-context>
</audio-context>

Oscillator

Generates periodic waveforms

Oscillator

<audio-context>
  <audio-oscillator>
  </audio-oscillator>
</audio-context>

Example

Mixer

Adds the output of its children into a single output

Mixer

<audio-context>
  <audio-mixer>
    <audio-oscillator frequency="220"></audio-oscillator>
    <audio-oscillator frequency="440"></audio-oscillator>
  </audio-mixer>
</audio-context>

Example

Chain

Connects the output of each child to the next child

Chain

<audio-context>
  <audio-chain>
    <audio-oscillator frequency="220"></audio-oscillator>
    <audio-oscilloscope></audio-oscilloscope>
  </audio-chain>
</audio-context>

Example

Oscilloscope

Displays whatever it's connected to its input

The input gets through to the output, unmodified

Filter

Modifies the input by letting some frequencies pass

Filter

<audio-context>
  <audio-chain>
    <audio-oscillator frequency="220"></audio-oscillator>
    <audio-filter type="lowpass"></audio-filter>
  </audio-chain>
</audio-context>

Example

Minisynth

It will have...

Minisynth

<audio-chain>
  <audio-mixer>
    <audio-oscillator></audio-oscillator>
    <audio-oscillator></audio-oscillator>
  </audio-mixer>
  <audio-filter type="lowpass"></audio-filter>
</audio-chain>

Minisynth

With raw Web Audio

var mixerGain = context.createGain();

var osc1 = context.createOscillator();
var osc2 = context.createOscillator();
osc1.connect(mixerGain);
osc2.connect(mixerGain);

var filter = context.createBiquadFilter();
mixerGain.connect(filter);

// output is at *filter*

Minisynth

start

start: function(when) {
  var oscGain = this.oscillators.length > 0 ?
    1.0 / this.oscillators.length : 1.0;
  this.oscillators.forEach(function(osc) {
      osc.gain = oscGain;
      osc.start(when);
  });
}

Minisynth

stop

stop: function(when) {
  this.oscillators.forEach(function(osc) {
      osc.stop(when);
  });
}

Minisynth

noteOn

noteOn: function(noteNumber) {
  this.oscillators.forEach(function(osc, index) {
    var oscNoteNumber = noteNumber + 12 * index;
    var frequency = 
      MIDIUtils.noteNumberToFrequency(oscNoteNumber);
    osc.frequency = frequency;
  });
}

Triggering notes

What about a keyboard?

Keyboard

<audio-keyboard octaves="2"></audio-keyboard>

Keyboard events

noteon

keyboard.addEventListener('noteon', function(e) {

  var noteIndex = e.detail.index;
  minisynth.noteOn(parseInt(noteIndex, 10) + 40);
  minisynth.start();
  
}, false);

Keyboard events

noteoff

keyboard.addEventListener('noteoff', function(e) {

  minisynth.noteOff();
  
}, false);

DEMO

Now... we have our instrument

We are rockstars!

And rockstars need to look COOL

They have signature guitars...

We have CSS!!!

So let's turn our humble synth...

into a fiery hot object of desire

Wrap up

Audio tags allow you to experiment with Web Audio without drowning in details

But this is just a prototype

Call for forks

github.com/sole/audio-tags

Thank you!

Soledad Penadés

@supersole

sole@mozilla.com