Berlin Web Audio Hack Day 2014

Soledad Penadés

@supersole

Why are we here?

Because we want to make noise in the web

... without plug-ins!

Oh, I know, with <audio>, eh?

We could use it...

<audio src="awesomesong.ogg" controls preload></audio>

This would...

It could also trigger some events!

And has methods you can use

But it has shortcomings...

Is it all over?

Do we just give up and start writing native apps?

NO

... or maybe we could use Flash...?

Get out of here.

Have a nice day!

Web Audio to the rescue!

Web Audio...

So how does it work?

Let me tell you a story...

In the beginning there was the nothingness...

And we created an audio context

var audioContext = new AudioContext();

AudioContext

"Where everything happens"

AudioContext

The audio graph? ô_Ô

Show, don't tell

Let's make some noise

var audioContext = new AudioContext();
var oscillator = audioContext.createOscillator();
oscillator.connect(audioContext.destination);

Starting/stopping

// start it now
oscillator.start(0);

// 3 seconds from now
oscillator.start(audioContext.currentTime + 3)

// stop it now
oscillator.stop(0);

// start it again
oscillator.start(0); // But nothing happens!?

Why can't oscillators be restarted?

Welcome to your first Web Audio...

GOTCHA!

Because of performance reasons

One-use only nodes

Write your own wrappers

Oscillator.js (1/3)

function Oscillator(context) {
  var node = null;
  var nodeNeedsNulling = false;
  
  this.start = function(when) {
    ensureNodeIsLive();
    node.start(when);
  };

Oscillator.js (2/3)

// continues
this.stop = function(when) {
  if(node === null) {
    return;
  }
  nodeNeedsNulling = true;
  node.stop(when);
};

Oscillator.js (3/3)

// continues
  function ensureNodeIsLive() {
    if(nodeNeedsNulling || node === null) {
      node = context.createOscillator();
    }
    nodeNeedsNulling = false;
  }
}

Using it

var ctx = new AudioContext();
var osc = new Oscillator(ctx);

function restart() {
  osc.stop(0);
  osc.start(0);
}

osc.start(0);

setTimeout(restart, 1000);

Self regenerating oscillator

But before I continue...

It would be nice to see what is going on!

Let's use an

AnalyserNode

AnalyserNode, 1

var analyser = context.createAnalyser();
analyser.fftSize = 2048;
var analyserData = new Float32Array(
  analyser.frequencyBinCount
);

oscillator.connect(analyser);

AnalyserNode, 2

requestAnimationFrame(animate);

function animate() {
  analyser.getFloatTimeDomainData(analyserData);
  drawSample(canvas, analyserData);
}

Analyser

Now,

Can we play something other than that beep?

Yes!

Nodes have properties we can change

e.g. oscillator.type

oscillator.type = 'square';

Wave types

oscillator.frequency

Naive attempt:

oscillator.frequency = 880;

It doesn't work!

oscillator.frequency is an AudioParam

It is special

// Access it with
oscillator.frequency.value = 880;

So what is the point of AudioParam?

Superpowers.

Superpower #1

Scheduling changes with accurate timing

What NOT to do

Stepped sounds

AudioParam approach

Web Audio keeps a list of timed events per parameter

Go from 440 to 880 Hz in 3 seconds

osc.frequency.setValueAtTime(
  440,
  audioContext.currentTime
);
osc.frequency.linearRampToValueAtTime(
  880,
  audioContext.currentTime + 3
);

Let's build an ADSR envelope

ADSwhat...?

We need a new node for controlling the volume

GainNode

var ctx = new AudioContext();
var osc = ctx.createOscillator();
var gain = ctx.createGain(); // *** NEW

osc.connect(gain); // *** NEW
gain.connect(ctx.destination); // *** NEW

ADSR part 1

// Attack/Decay/Sustain phase
gain.gain.setValueAtTime(
  0,
  audioContext.currentTime
);
gain.gain.linearRampToValueAtTime(
  1,
  audioContext.currentTime + attackLength
);
gain.gain.linearRampToValueAtTime(
  sustainValue,
  audioContext.currentTime + decayLength
);

ADSR part 2

// Release phase
gain.gain.linearRampToValueAtTime(
  0,
  audioContext.currentTime + releaseLength
);

Envelopes

Cancelling events!

osc.frequency.cancelScheduledEvents(
  audioContext.currentTime
);

Superpower #2

Modulating properties

Connect the output of one node to another node's property

LFOs

LFOs

We can't hear those frequencies...

but can use them to alter other values we can notice!

SPOOKY SOUNDS

Watch out!

var context = new AudioContext();
var osc = context.createOscillator();
var lfOsc = context.createOscillator();

var gain = context.createGain();
lfOsc.connect(gain);

// The output from gain is the [-1, 1] range
gain.gain.value = 100;
// now the output from gain is in the [-100, 100] range!

gain.connect(osc.frequency); // NOT the destination

KEEP watching out

osc.frequency.value = 440;

// oscillation frequency is 1Hz = once per second
lfOsc.frequency.value = 1;

osc.start();
lfOsc.start();

spooky LFOs

Playing existing samples

AudioBufferSourceNode, 1

var context = new AudioContext();
var pewSource = context.createBufferSource();
var request = new XMLHttpRequest();
request.open('GET', samplePath, true);
request.responseType = 'arraybuffer'; // we want binary data 'as is'
request.onload = function() {
  context.decodeAudioData(
    request.response,
    loadedCallback, errorCallback
  );
};

AudioBufferSourceNode, 2

function loadedCallback(bufferSource) {
  buffer = bufferSource;
}

function errorCallback() {
  alert('No PEW PEW for you');
}

var abs = context.createBufferSource();
abs.buffer = buffer;

AudioBufferSourceNode, 3

Just like oscillators!

abs.start(when);
abs.stop(when);

AudioBufferSourceNode even die like oscillators!

Pssst:

You can create them again and reuse the buffer

pewpewmatic

MediaElementAudioSourceNode

Takes the output of <audio> or <video> and incorporates them into the graph.

var video = document.querySelector('video');
var audioSourceNode = 
  context.createMediaElementAudioSource(
    video
  );
audioSourceNode.connect(context.destination);

Media element

Better open the iframe in a new tab...

Further altering sounds

Standard Web Audio nodes you can use

Their parameters can also be modulated and automated!

A brief pause for self reflection...

Being mobile friendly

Web Audio Workshop 202?

Or just in case you got excited!

QUESTIONS?

Thanks!

@supersole

soledadpenades.com