Audio Tags:
Web Components + Web Audio = ♥
Soledad Penadés
Apps Engineer at Mozilla
- @supersole
- #apps in irc.mozilla.org
- chat.meatspac.es
I'm totally into music
- 150+ songs
- Writing audio "toys" for the last 5 years
2008
Sorollet
- "Tiny noise" in Catalan
- Pure C
- No GUI
- The GUI was me
Sorollet V1 example
(fragment)
if(synth->pitch_envelope_active)
{
actual_note += (int) (map(
sorollet_adsr_update(&synth->pitch_envelope, t),
0, 1, -12, 12)
);
}
2010
Sorollet V2
- VSTi
- C++
- I had a GUI!
Sorollet V2 example
(fragment)
float pitchEnv = mPitchADSR->update(bufferTime);
Sorollet V2 VSTi example
(fragment)
case PARAM_VOLUME1: vst_strncpy(label, "dB", kVstMaxParamStrLen);
break;
case PARAM_OCTAVE1: vst_strncpy(label, "Octave", kVstMaxParamStrLen);
break;
case PARAM_WAVE1: vst_strncpy(label, "Wave", kVstMaxParamStrLen);
break;
case PARAM_PHASE1: vst_strncpy(label, "", kVstMaxParamStrLen);
break;
// ...
Tedious!
People used it!
*\o/*
- Ubuntu PPA
- Fedora
- In cool songs!
2011
Audio in the browser
- <audio> tag hacks
- Firefox's AudioData
- Chrome's Web Audio
Audio Data vs Web Audio
Why did Web Audio win?
- Audio Data was 'just a buffer'
- Web Audio is modular
- ... and includes many useful ready-made units
- Plus 3D audio support
2013
W3C Web Audio API
- Chrome's API became the standard
- Implemented in Firefox too!
- It's even in iOS
- Chrome mobile
- Firefox for Android: in the works
- It's all very exciting!
2013
Web Components
... but what is a "component"?
"A constituent part; element; ingredient."
Screws
Resistors
Doorbell
Web components
== custom DOM elements
- custom behaviour
- custom appearance
- encapsulation
- reusability
And here comes the crazy idea...
Why not audio components?
Perfect sense
Web Audio is modular
Web Components are modular
Audio Tags
Custom audio... with less boilerplate
Let's build a simple synthesiser
The audio context
Where things happen
The audio context
<audio-context>
</audio-context>
Oscillator
Generates periodic waveforms
Oscillator
<audio-context>
<audio-oscillator>
</audio-oscillator>
</audio-context>
Example
Mixer
Adds the output of its children into a single output
Mixer
<audio-context>
<audio-mixer>
<audio-oscillator frequency="220"></audio-oscillator>
<audio-oscillator frequency="440"></audio-oscillator>
</audio-mixer>
</audio-context>
Example
Chain
Connects the output of each child to the next child
Chain
<audio-context>
<audio-chain>
<audio-oscillator frequency="220"></audio-oscillator>
<audio-oscilloscope></audio-oscilloscope>
</audio-chain>
</audio-context>
Example
Oscilloscope
Displays whatever it's connected to its input
The input gets through to the output, unmodified
Filter
Modifies the input by letting some frequencies pass
Filter
<audio-context>
<audio-chain>
<audio-oscillator frequency="220"></audio-oscillator>
<audio-filter type="lowpass"></audio-filter>
</audio-chain>
</audio-context>
Example
Minisynth
It will have...
- Two oscillators
- A filter
Minisynth
<audio-chain>
<audio-mixer>
<audio-oscillator></audio-oscillator>
<audio-oscillator></audio-oscillator>
</audio-mixer>
<audio-filter type="lowpass"></audio-filter>
</audio-chain>
Minisynth
With raw Web Audio
var mixerGain = context.createGain();
var osc1 = context.createOscillator();
var osc2 = context.createOscillator();
osc1.connect(mixerGain);
osc2.connect(mixerGain);
var filter = context.createBiquadFilter();
mixerGain.connect(filter);
// output is at *filter*
Minisynth
start
start: function(when) {
var oscGain = this.oscillators.length > 0 ?
1.0 / this.oscillators.length : 1.0;
this.oscillators.forEach(function(osc) {
osc.gain = oscGain;
osc.start(when);
});
}
Minisynth
stop
stop: function(when) {
this.oscillators.forEach(function(osc) {
osc.stop(when);
});
}
Minisynth
noteOn
noteOn: function(noteNumber) {
this.oscillators.forEach(function(osc, index) {
var oscNoteNumber = noteNumber + 12 * index;
var frequency =
MIDIUtils.noteNumberToFrequency(oscNoteNumber);
osc.frequency = frequency;
});
}
Triggering notes
What about a keyboard?
Keyboard
<audio-keyboard octaves="2"></audio-keyboard>
Keyboard events
noteon
keyboard.addEventListener('noteon', function(e) {
var noteIndex = e.detail.index;
minisynth.noteOn(parseInt(noteIndex, 10) + 40);
minisynth.start();
}, false);
Keyboard events
noteoff
keyboard.addEventListener('noteoff', function(e) {
minisynth.noteOff();
}, false);