How options become ffmpeg arguments

Implementation reference for AudioProxy.Ffmpeg.Command. Nothing here is needed to use the proxy — the option grammar in audio-proxy-api-v1.md covers that. Read this when a render produces something you did not expect, or before changing the argument table.

AudioProxy.Ffmpeg.Command.build/3 turns a validated options struct plus an input URL into an argv list. It is a pure function and it is the last leg of the round-trip: equal cache keys imply byte-identical commands, which is what makes a cache hit a claim about bytes rather than about a URL.

{:ok, opts} = AudioProxy.Options.parse("f:opus/br:96/t:12.5:30/fade:0.5:1")
AudioProxy.Ffmpeg.Command.build(opts, "https://masters.example/piece.wav",
                                type: :http)
# => ["-nostdin", "-hide_banner", "-loglevel", "error",
#     "-protocol_whitelist", "https,tls,tcp",
#     "-ss", "12.5", "-t", "30", "-i", "https://masters.example/piece.wav",
#     "-vn", "-sn", "-dn", "-af", "afade=t=in:st=0:d=0.5,afade=t=out:st=29:d=1",
#     "-c:a", "libopus", "-b:a", "96k", "-f", "ogg", "pipe:1"]

There is no shell anywhere in this path. The argv is a flat list of complete arguments, so a source URL containing ;, $(…) or spaces is one element and stays data; dl and cb never reach the command at all.

The type: is the resolved source's own tag and it is required — see Audio only, at the argv for what it decides and why there is no default.

Option → ffmpeg mapping

Every command writes to pipe:1 behind an explicit -f, since stdout has no filename for ffmpeg to infer a muxer from, and every command runs with -nostdin -hide_banner -loglevel error so stderr carries diagnostics only.

Audio only, at the argv

Two things appear in every argv regardless of the options, and neither comes from the URL:

The protocol set is a function of the source's type, never of the input string and never of configuration:

The two sets are disjoint by construction, which is the property that makes them worth having: a local render cannot fetch, a remote render cannot read disk, and concat:, subfile:, data: and the rest are reachable from neither. Because the set is derived from the source type, build/3 requires type: rather than defaulting it — a default would be a guess about which side of that boundary a render sits on, and the wrong guess is a hole rather than a crash. A source type with no entry in protocols/1 raises for the same reason.

These are defence in depth, and one concrete case shows why they are not redundant. The gate exempts cover art by trusting the attached_pic disposition, which lives in the container — bytes the requester may control — so a crafted file can wear it and pass. What that buys is bounded here rather than there: with -vn -sn -dn the video stream is never mapped, so the render is an audio-only encode of the audio track. The attacker gets audio extraction from a video file, which is what the gate exists to refuse, and not a video transcode, which is what would actually cost the operator. Layer one is the policy; layer two is what makes forging layer one uninteresting.

The two layers are independent by construction, and one of them is a seam. AudioProxy.VideoPolicy holds the verdict on a source ffprobe called video, as a closed two-valued enum: :reject, which is the 415 above and the default, or :extract, which admits the source and renders its audio track. AudioProxy.Ffprobe.has_video?/1 still answers the question identically under either. What cannot move is the row in the table above: Command.build/3 takes the options, the input and the source type, and no policy is among its arguments, so -vn -sn -dn is not conditional on a verdict it never sees and the format vocabulary contains no video encoder to reach. An ingest policy can change what is admitted; it has no argument through which to change what is emitted. The knob is Application.get_env(:audio_proxy, :video_policy) and deliberately not an AP_ variable: the intended consumer is an embedding release that owns its own application env, not an operator holding the published image, which is why the configuration surface has no entry for it.

The gate that actually refuses a video source with 415 is an ffprobe run in AudioProxy.Plugs.RenderAction, before the semaphore; the flags above are what still holds if that gate is bypassed, reordered, or handed a source it cannot see inside. Placement matters for both: -protocol_whitelist after -i would bind the output format context and protect nothing, and -vn before -i would be an input option ffmpeg reads differently.

Command.allowed_flags/0 publishes the complete flag vocabulary, and takes_value?/1 says which flags carry a value. The property suite walks a generated argv position by position against both, so "no URL content can become an ffmpeg flag" is a checked claim rather than a design intention. The walk is necessary rather than decorative: ogg's quality scale starts at −1, so f:ogg/q:-1 renders ["-q:a", "-1"] and a leading-hyphen check would have to be loosened to tolerate it.

Two subprocesses read the source, not one, so both carry a whitelist. The decode's is in the argv above; the probe's comes from Ffprobe.args/2, which takes the protocol set as an argument rather than an option precisely because AudioProxy.Peaks.Render builds its own probe argv (see below) and would otherwise be the one route reading a source unrestricted.

The enhance:voice chain, and why it cannot change

enhance:voice emits exactly this, as the first filters in the graph:

highpass=f=80,
afftdn=nr=12:nf=-30,
deesser=i=0.4:m=0.5:f=0.5:s=o,
acompressor=threshold=0.125:ratio=3:attack=20:release=250:makeup=2,
alimiter=limit=0.977:level=disabled

Every filter is stock ffmpeg — the preset adds no dependency and no build flag. What each is aimed at, on speech:

The de-esser's band follows the source's sample rate

deesser's f is a fraction of the sample rate rather than a frequency in Hz, and the preset runs before aresample — so the band it works on is a fraction of the source's rate, not of any sr the request asked for. The same preset de-esses around 11 kHz on a 44.1 kHz master and around 5.5 kHz on a 22 kHz one.

This costs the cache nothing: the source is part of the cache key, so equal keys still imply the same source, the same argv and the same bytes. Enhancing before the resample also remains the right order — denoise at full bandwidth, then downsample. It is worth knowing when a preset sounds different on two masters that differ only in rate.

Why the limiter is there, and why level=disabled

The chain shipped without a limiter, and measurement is what added it. The compressor's 20 ms attack lets a transient shorter than that through uncompressed, and the 2× makeup then adds its full 6 dB on top. A fixture of 5 ms bursts peaking at −3.1 dBFS came back at 0.0 dBFS — the preset introducing clipping that the source did not have. With alimiter it comes back at −0.2 dBFS, and the render's duration is unchanged.

level=disabled is load-bearing rather than tidy. alimiter's level option defaults to enabled, which auto-normalizes the output back up to full scale, so the obvious spelling alimiter=limit=0.977 still measured 0.0 dBFS and read exactly like a limiter doing nothing.

Why it is first

The chain conditions the source, so every later stage is a statement about what comes out of it. Running it after loudnorm would mean measuring loudness on audio the compressor was about to change, and the render would miss its own target. enhance and norm are therefore orthogonal rather than alternatives: the preset shapes dynamics, norm hits a number, and asking for both means both, in that order.

Why the parameters are pinned rather than tuned

A variant is addressed by a cache key derived from the option name and served Cache-Control: immutable. Retuning voice in place would give two different renders one key: a warm CDN keeps serving the old bytes, a cold cache produces the new ones, and no part of the URL distinguishes them. So an improved chain ships as a new preset value (voice2) and the old value keeps its bytes.

That rule is enforced rather than remembered. AudioProxy.Ffmpeg.Command.enhance_chain/1 is compared against a literal in test/audio_proxy/ffmpeg/command_test.exs, so editing the chain fails a test that says so — and a failure there is never an expectation to update, it is a decision between "this is a new preset" and "this would have silently re-rendered every cached enhance:voice variant".

What the chain does is asserted separately and spectrally, in command_enhance_ffmpeg_test.exs — one fixture per stage, because a single whole-chain assertion is not enough. An earlier version measured three bands on one fixture and stayed green with acompressor deleted; mutating each filter in turn showed afftdn was unasserted too. Two of the stages were pinned only as characters.

Each assertion was checked by deleting its stage and confirming that exactly that test fails. Two of them needed a fixture of their own to say anything at all: on steady tones a compressor is indistinguishable from a gain, and a "noise band" measured on the tone fixture reads mostly skirt leakage from the 7 kHz tone rather than noise — which is how a first attempt at the afftdn assertion measured the wrong thing and concluded the stage was useless.

A sixth assertion belongs to no stage: the speech band must move ≤ 3 dB. It is what stops every "drops by" line above being satisfied by volume=-20dB.

Measured identically on ffmpeg 7.1.5 and 8.1.1. Golden bytes were rejected for this: they would pin the encoder and the ffmpeg version alongside the behaviour, and fail on a distro bump that changed nothing the preset promises.

Why f:peaks runs ffmpeg twice, and why it is mono

Peaks are the one format where ffmpeg does not produce the response. It decodes to raw interleaved s16le on stdout and AudioProxy.Peaks reduces those samples to pts min/max pairs; the PCM is folded in chunk by chunk and dropped, so a ten-minute source costs a few kilobytes of resident state rather than the tens of megabytes it decodes to.

Streaming that reduction is what forces the leading ffprobe. Bucket boundaries are ceil(frames / pts) and have to be known before the first sample arrives; the alternative is buffering the whole decode and counting afterwards, which trades a header read for memory proportional to the source. So a peaks render is a probe and then a decode, both spawned through the same render pipeline — same kill discipline, same AP_RENDER_TIMEOUT, same stderr classification, which is why a 404 source fails a peaks request with the status it would have failed an audio one with.

Probe and decode can disagree about the sample count by a frame or two, and neither direction is reported: extra samples fold into the final bucket, and a short decode leaves trailing pairs at 0, 0. length is always the pts the URL asked for.

The -ac 1 is the other peaks-only rule. Every other format follows the source when ch is absent; peaks downmix, because a waveform UI draws one shape and following a stereo source would double the payload for a picture nobody asked for. The reducer also has to know the interleaving before it reads a byte, so "whatever the source had" is not an option the argv can leave open. ch:2 still gives per-channel pairs, and the mono default is materialized into the cache key so f:peaks and f:peaks/ch:1 are one variant.

Why m4a fragments on duration

-movflags frag_keyframe starts a new fragment at each video keyframe. An audio-only stream has none, so empty_moov alone produces exactly one fragment, which ffmpeg flushes when the input ends — a valid file on a non-seekable pipe, but not a stream. Measured on a 20 s source fed at realtime:

One-second fragments cost nothing measurable in size and make the stream a stream, so that is what the builder emits. The :ffmpeg-tagged suite counts the fragments and measures time-to-first-byte, so the regression cannot come back quietly.

Filters run in the order loudnorm → volume → aresample → afade, and the order is load-bearing. loudnorm goes first because normalizing after a static gain would undo it; aresample follows it because single-pass loudnorm resamples its output to 192 kHz; afade goes last so the fade shape survives the stages above it.

That 192 kHz has one visible consequence: norm without an explicit sr appends an aresample of its own. Without it every normalized render would be a 192 kHz file. The rate it targets is the source's own, for every format, because §3.1's default is the source's rate for every format — so norm on a 96 kHz master stays at 96 kHz and on a 44.1 kHz source stays at 44.1. The stage undoes an implementation detail; landing anywhere but where the render would have been without it would make norm a rate option as well as a loudness one. Where no probe supplied a rate the fallback is 48 kHz, which is what this emitted unconditionally before the probe reached the builder.

No lossy ceiling is applied here, deliberately. §3.1's 48 kHz cap governs what a request may ask for — sr:96000 with f:aac is a 422 — not what a source may be. Clamping this stage would mean f:aac/norm:ebu downsampling a 96 kHz master that plain f:aac returns at 96 kHz, which is a loudness option quietly changing the rate.

The builder learns the source's rate the same way it learns its bit depth: the render action's audio-only gate probes every miss, and passes both into build/3 beside the source type. That keeps build/3 a pure function of its arguments — which is what the round-trip property rests on — while letting two options documented as following the source actually do it.

This is also what makes gain and norm safe under f:peaks. The reducer budgets its buckets from the source's probed rate before a byte is decoded, so a filter chain that re-rated the decode would overrun that budget and fold the overrun into the final bucket as a spike — measured at 240000 frames against a 220500-frame budget for a 5 s 44.1 kHz source. With the resample returning the decode to the source's rate, the frame count is exact, and the :ffmpeg-tagged suite asserts that equality rather than assuming it.

ffmpeg version

The argv is a contract with a specific ffmpeg, not with "ffmpeg" in the abstract: encoder names, muxer names and filter option spellings all drift between versions. The devcontainer and the release image therefore install ffmpeg from the same distro packaging, and the :ffmpeg-tagged tests (test/audio_proxy/ffmpeg/command_ffmpeg_test.exs) run every format and every filter through the real binary, so a codec name that a build does not carry fails a test rather than a request. Pinning an exact ffmpeg version — and whether to build it from source with a trimmed codec set — is decided in add-docker-release.

One known gap. libopus encodes at 48/24/16/12/8 kHz only, so sr:44100 with f:opus is resampled to 48 kHz by ffmpeg's own negotiation and produces the same bytes as f:opus alone, under a different cache key. That costs a duplicate cache object rather than a wrong render, and it is tracked alongside the semantic no-ops described under cache-key semantics.

The same negotiation has a second, quieter cost now that norm follows the source: f:opus/norm:ebu on a 44.1 kHz source emits aresample=44100 and libopus then converts 44100 → 48000 itself, so the signal is resampled twice where the old fixed 48 kHz target converted once. Nothing is wrong with the output — it is the same path an explicit sr:44100 has always taken — and special-casing opus here would mean the builder knowing each encoder's supported rates, which is a larger contract than the one option it would fix. Pass sr:48000 on that path if the extra conversion matters.

Worth knowing when reading an argv: a plain render emits no aresample at all, so the encoder is handed whatever the source had and f:aac on a 96 kHz master encodes at 96 kHz. f:mp3 and f:opus on that source come back at 48 kHz anyway, because libmp3lame and libopus accept nothing higher and ffmpeg negotiates it down. See §3.1.

Two fallbacks remain, and both are reachable only when the probe could not answer: with no bd, f:wav encodes 16-bit when the source's depth is unknown — which includes a 32-bit source, since the depth alone cannot distinguish pcm_s32le from the float pcm_f32le that bd:32f means — and norm with no sr resamples to 48 kHz when the source's rate is unknown. Neither is reachable through the mounted pipeline, which probes every miss.