innocenat

Web Audio API is one of the cutting edge technology Javascript has offered. (It isn’t even supported in IE11! But both Chrome and Firefox now support it without prefix). It allows direct low-level manipulation of audio data (to sample level).

While .wav file format is supported by all browsers (Support here is defined by ability to be decoded via AudioContext.decodeAudioData, which is normally the same as those supported in <audio> tag), many other format is not, such as MP3/AAC/Ogg due to various patent problems. Because audio codecs, as opposed to video codec, require very low power to decode, it has been done in pure Javascript before (jsMAD, etc). I actually aim to write some audio decoder in pure Javascript too, so the .wav file playing is the first stepping stone. (audiocogs/etc is Not Invented Here™)

tl;dr here’s the code: https://gist.github.com/innocenat/5c5a48365930b691c863 Read on for detail.

The implementation is pretty straightforward. First it fetch the file via XMLHttpRequest as an array buffer. All .wav file are in RIFF file format, so it looks for RIFF header and format specifier. It then reads some metadata of the stream from “fmt ” chunk, and then read the actual sample data and convert to Float32Array to feed into audio context API.

Now, what is the problem with this implementation? The entire content is stored in memory, twice!. First inside arraybuffer from XmlHTTPRequest, and second from AudioBuffer. This seriously wastes memory (both Firefox and Chrome use roughly 300MB of memory from reading 63MB file). Normally file of this length (it was 6:16 minutes long) isn’t supported to be played via AudioBuffer, but to stream from <audio> element via MediaElementSource. The problem is that we don’t want to use browser’s audio decoder! (Which is why we are writing the decoder ourselves)

In normal situation though (i.e. not for .wav file), currently, for mp3/aac/etc file to be decoded in Javascript, we should decode to .wav file, convert to Blob and store to browser via URL.createObjectURL. Then use blob url via <audio> to stream content to the Web Audio API.

In ideal situation we should be able to write the file via Filesystem API (only support in Chrome right now) and the stream file to decoder from disk, as with most media player do.

I have yet to see how the Aurora.js audio engine works though. This all is my initial approach to this problem. We will see.