AudioWorklet Internals: Real-Time Audio Processing on a Dedicated Thread

Audio is unforgiving in a way most web work isn't: if your code is even a few milliseconds late delivering the next buffer of samples, the user hears it — a click, a pop, a glitch, a dropout. Audio runs on a hard real-time deadline (the sound card needs the next buffer now, every few milliseconds, forever), and the main thread — busy with layout, JavaScript, garbage collection, and user input — cannot meet that deadline reliably. The Web Audio API's original answer, ScriptProcessorNode, ran custom audio processing on the main thread, and it was a disaster: every main-thread hiccup (a GC pause, a heavy render) caused audible glitches, because the audio processing was competing with everything else for the single main thread. AudioWorklet fixes this fundamentally: it runs your custom audio processing code on a dedicated, high-priority audio rendering thread, isolated from the main thread, so the audio callback fires reliably on its real-time deadline regardless of what the main thread is doing. This is what makes serious in-browser audio — synthesizers, effects, real-time DSP, music apps — actually viable. Understanding why audio needs a real-time thread, how AudioWorklet's process callback works, the main-thread/audio-thread communication model, and the rules of real-time audio code (no allocation, no blocking) is essential for any audio-heavy web application.

The Problem: Audio Has a Hard Real-Time Deadline

Audio processing's timing requirements are unlike normal web code:

The core problem: audio must deliver the next buffer of samples by a hard deadline (every ~3ms), forever, or the user hears a glitch — and the main thread, busy with everything else (and subject to GC pauses), cannot meet that deadline reliably. The old ScriptProcessorNode ran audio processing on the main thread, so any main-thread stall glitched the audio. AudioWorklet solves this by running your processing on a dedicated audio rendering thread with real-time priority, isolated from the main thread's chaos — so the audio callback fires on schedule regardless of layout/JS/GC. The real-time nature also imposes strict rules on the audio code itself (no allocation, no blocking), because anything that delays the callback causes an audible glitch.

Architecture: Main Thread + Dedicated Audio Thread

┌──────────────────────────────────────────────────────────────────────┐
│  MAIN THREAD                                                          │
│   AudioContext, node graph setup, UI, parameter changes               │
│   audioContext.audioWorklet.addModule('processor.js')  ← load worklet │
│   new AudioWorkletNode(ctx, 'my-processor')            ← create node   │
│        │ communicate via node.port.postMessage  ◄──────────┐          │
└────────┼───────────────────────────────────────────────────┼─────────┘
         ▼ (message)                                          │ (message)
┌────────────────────────────────────────────────────────────┼─────────┐
│  AUDIO RENDERING THREAD (dedicated, real-time priority)     │         │
│   class MyProcessor extends AudioWorkletProcessor {         │         │
│     process(inputs, outputs, parameters) {                  │         │
│       // called every ~3ms (128 samples) — MUST be fast,    │         │
│       // NO allocation, NO blocking → or AUDIBLE GLITCH     │         │
│       // fill outputs[][] with samples → returns to audio HW│         │
│     }                                                        │         │
│   }  registerProcessor('my-processor', MyProcessor);        │         │
└──────────────────────────────────────────────────────────────────────┘
                            ▼ samples
                     Audio hardware (speakers)

The split: the main thread sets up the AudioContext, the audio node graph, and the UI, and loads the worklet module (audioWorklet.addModule); the actual processing runs in an AudioWorkletProcessor on the dedicated audio rendering thread, whose process() callback is invoked every render quantum (128 samples, ~3ms at 48kHz) to produce output samples. The two threads are isolated and communicate via a MessagePort (node.port) — the main thread can't directly call into the audio thread (that would risk blocking it). This isolation is the whole point: the audio thread does only audio, on its real-time deadline, undisturbed by the main thread.

The process() Callback: The Real-Time Heart

The process() method is where audio is generated/transformed, called repeatedly on the audio thread. It receives input buffers, output buffers (to fill), and parameter values, all as arrays of Float32Array (channels of samples). The cardinal rule: it must be fast and must never do anything that could delay it — because it runs on a hard deadline, every quantum.

// processor.js — runs on the dedicated audio thread.
class GainProcessor extends AudioWorkletProcessor {
  // declare audio-rate parameters (smoothly automatable from the main thread)
  static get parameterDescriptors() {
    return [{ name: 'gain', defaultValue: 1, minValue: 0, maxValue: 2 }];
  }
  process(inputs, outputs, parameters) {
    const input = inputs[0];           // [channel][sample] Float32Arrays
    const output = outputs[0];
    const gain = parameters.gain;
    for (let ch = 0; ch < output.length; ch++) {
      const inCh = input[ch], outCh = output[ch];
      for (let i = 0; i < outCh.length; i++) {   // 128 samples per quantum
        // gain may be a single value or per-sample (if automated)
        outCh[i] = inCh[i] * (gain.length > 1 ? gain[i] : gain[0]);
      }
    }
    return true;   // keep the processor alive (return false to let it be GC'd)
  }
}
registerProcessor('gain-processor', GainProcessor);

process() is called every ~3ms to fill 128 samples per channel. Returning true keeps the processor alive; returning false signals it can be torn down. Parameters declared via parameterDescriptors are AudioParams — they can be automated smoothly from the main thread (e.g., ramp the gain over time) and arrive in process() as either a single value (constant this quantum) or a per-sample array (if changing). This is the real-time loop: in, process, out, every quantum, forever — and it must complete well within the ~3ms budget every single time.

The Rules of Real-Time Audio Code: No Allocation, No Blocking

Because process() runs on a hard real-time deadline, the code inside it must obey strict rules that ordinary JavaScript ignores — violating them causes audible glitches:

  In process() (the real-time audio callback), NEVER:
   • ALLOCATE memory (new arrays/objects) → triggers GC → GC pause → GLITCH
   • BLOCK (sync I/O, waiting, locks) → misses the deadline → GLITCH
   • do heavy/unbounded work → overruns the ~3ms budget → GLITCH
   • console.log heavily, etc. → anything slow → GLITCH

  DO:
   • preallocate all buffers/state in the constructor (reuse them)
   • keep process() tight, bounded, and fast (it runs every ~3ms)
   • pass data in/out via the port (async) or SharedArrayBuffer (lock-free)

The most important rule: don't allocate memory in process(). Allocation triggers garbage collection, and a GC pause — even a few milliseconds — means the callback misses its deadline and the user hears a glitch. So you preallocate everything (buffers, state) in the constructor and reuse it in process() (no new, no array literals that allocate). Similarly, never block (no synchronous I/O, no waiting) — any wait misses the deadline. And keep the work bounded and fast (it must reliably finish within ~3ms). These are the rules of all real-time audio programming (the same rules apply in native audio code), and they're why the audio thread is isolated and why communication with it is async — you can't risk blocking it.

Communication: Messages and SharedArrayBuffer

The audio thread is isolated, so getting data in/out requires care that doesn't block the real-time callback. Two mechanisms: the MessagePort (node.port.postMessage ↔ this.port.onmessage) for asynchronous control messages (change settings, send events) — but message-passing copies data and isn't suitable for high-rate sample streaming; and SharedArrayBuffer for sharing audio data lock-free between threads (e.g., streaming samples from the main thread or a worker to the audio thread without copying), using atomics for coordination — the high-performance path.

// Main thread → audio thread: async control messages (non-blocking).
node.port.postMessage({ type: 'setWaveform', value: 'sawtooth' });

// In the processor (audio thread):
constructor() {
  super();
  this.buffer = new Float32Array(1024);   // PREALLOCATE — not in process()!
  this.port.onmessage = (e) => {           // handle control messages
    if (e.data.type === 'setWaveform') this.waveform = e.data.value;
  };
}
// For high-rate data: a SharedArrayBuffer ring buffer (lock-free, no copy)
// lets you feed samples to the audio thread without postMessage overhead.

The communication design respects the real-time constraint: control changes go via async messages (the audio thread reads them between quanta, never blocking), and bulk/high-rate data uses SharedArrayBuffer (shared memory, no copy, coordinated with atomics — never locks, which would block the audio thread). You handle incoming messages in the constructor's onmessage (storing state to read in process()), not by doing work in process() itself. This keeps the real-time callback fast and unblocked.

Production Realities and Incidents

Incident 1: The Glitchy ScriptProcessorNode

A music app used the old ScriptProcessorNode for custom effects; users heard clicks and dropouts whenever the page did anything heavy (scrolling, rendering, GC). Root cause: ScriptProcessorNode runs audio processing on the main thread, competing with everything else — any main-thread stall glitches the audio. Fix: migrate to AudioWorklet, moving processing to the dedicated real-time audio thread, isolated from main-thread chaos. Real-time audio cannot run on the main thread; AudioWorklet (or native) is required.

Incident 2: The Allocation-Induced Glitch

An AudioWorklet processor allocated a temporary array inside process() each callback; the app glitched periodically. Root cause: allocating in the real-time callback triggers GC, and GC pauses miss the audio deadline → audible glitch. Fix: preallocate the buffer in the constructor and reuse it in process() — zero allocation in the callback. The #1 real-time audio rule: never allocate in the audio callback.

Incident 3: The Blocked Audio Thread

A processor tried to do synchronous work waiting on a result inside process(), blocking the audio thread and causing dropouts. Root cause: blocking (waiting) in the real-time callback misses the deadline. Fix: move the work off the audio thread (do it on the main thread or a worker), communicate results via the port or a SharedArrayBuffer (async, non-blocking), and have process() just read preprocessed data. The audio thread must never block — feed it data asynchronously.

Tradeoffs and Engineering Decisions

• AudioWorklet vs ScriptProcessorNode. AudioWorklet runs on a dedicated real-time thread (glitch-free, isolated from the main thread) — the only viable option for serious real-time audio — at the cost of a more complex model (separate module file, thread isolation, async communication). ScriptProcessorNode (deprecated) is simpler but runs on the main thread and glitches under any load. Always use AudioWorklet for custom processing.
• AudioWorklet vs built-in nodes. The Web Audio API provides built-in nodes (gain, filter, oscillator, etc.) that are highly optimized (native) — use them when they suffice (no custom code needed). AudioWorklet is for custom DSP that built-in nodes can't do (custom synthesis, novel effects). Don't reimplement a gain node in a worklet; use the built-in. Reach for AudioWorklet only for custom processing.
• Real-time rules: correctness vs convenience. The no-allocation/no-blocking rules make audio code more constrained and harder to write (preallocate, avoid idioms that allocate, no async/await in the callback) but are mandatory for glitch-free audio — violating them for convenience causes audible artifacts. Accept the discipline; real-time audio has hard constraints.
• postMessage vs SharedArrayBuffer. Messages are simple and safe for low-rate control (settings, events) but copy data and have overhead unsuitable for high-rate sample streaming; SharedArrayBuffer is lock-free and zero-copy for bulk/high-rate data but more complex (atomics, ring buffers, cross-origin isolation requirement). Use messages for control, SharedArrayBuffer for streaming audio data.
• Computation on vs off the audio thread. Keeping work in process() is simplest but risks overrunning the deadline; offloading heavy computation to the main thread/worker (feeding results to the audio thread asynchronously) keeps the callback fast but adds coordination. Do only the tight, bounded, time-critical work in process(); precompute/offload the rest.

Key Takeaways

• Audio has a hard real-time deadline (deliver the next ~128-sample buffer every ~3ms, forever, or the user hears a glitch), and the main thread cannot meet it reliably (it's busy with layout/JS/GC) — which is why the old main-thread ScriptProcessorNode was glitchy.
• AudioWorklet runs custom audio processing on a dedicated, real-time-priority audio rendering thread, isolated from the main thread, so the audio callback fires on schedule regardless of main-thread activity — making serious in-browser audio viable.
• The process(inputs, outputs, parameters) callback (in an AudioWorkletProcessor) is invoked every render quantum to fill output sample buffers; it must be fast and complete within the deadline every time, and parameterDescriptors give smoothly-automatable AudioParams.
• The rules of real-time audio code are strict and mandatory: never allocate in process() (GC pause → glitch — preallocate in the constructor), never block (any wait misses the deadline), and keep work bounded and fast. Violating them causes audible artifacts.
• The audio thread is isolated; communicate via MessagePort (async control messages) for settings/events and SharedArrayBuffer (lock-free, zero-copy) for high-rate sample streaming — never block the audio thread; feed it data asynchronously.
• Use built-in Web Audio nodes when they suffice (native, optimized); reach for AudioWorklet only for custom DSP — and accept the real-time discipline (preallocate, no blocking, offload heavy work) that glitch-free audio requires.