FFix: from a conversion to a media pipeline

Mix.install(
  [
    {:ffix, github: "akash-akya/ffix"},
    {:kino, "~> 0.19"},
    {:req, "~> 0.7"}
  ],
  consolidate_protocols: false
)

Setup

Use FFix to convert audio, edit pictures, and package video. You'll start with a little birdsong, give sound a visible shape, and build up to moving mosaics and multiple outputs in one FFmpeg run.

Evaluate the cells in order. When you change a control, re-evaluate the cell below it; moving a slider doesn't start an encode. Audio never plays automatically, but start with your volume low.

Open this notebook in Livebook, or launch it from the repository with:

livebook server livebooks/intro.livemd

The setup cell installs FFix from GitHub, Kino for previews and controls, and Req for downloads. Keep it above the first section so Livebook recognizes it as the dependency setup cell.

The Livebook runtime needs Elixir 1.16+, Git, FFmpeg with libx264, aac, and libwebp_anim, and ffprobe. In a container or remote runtime, those programs must be available there. Installing dependencies and downloading the two small media samples requires internet access.

These notebook helpers handle file paths, downloads, previews, and stream inspection. They keep the examples focused on FFix; command construction and execution stay explicit.

defmodule Demo do
  alias FFix.{Encoder, Muxer}

  @work_dir Path.join(System.tmp_dir!(), "ffix-intro-#{System.system_time(:microsecond)}")

  def path(filename \\ ""), do: Path.join(@work_dir, filename)

  def fetch!(url, path) do
    %{status: 200, body: bytes} = Req.get!(url)
    File.write!(path, bytes)
    path
  end

  def audio(path, type \\ :wav), do: Kino.Audio.new(File.read!(path), type)
  def video(path), do: Kino.Video.new(File.read!(path), :mp4)
  def image(path, type \\ :png), do: Kino.Image.new(File.read!(path), type)
  def size_label(path), do: "#{Float.round(File.stat!(path).size / 1024, 1)} KiB"

  def download(path) do
    Kino.Download.new(fn -> File.read!(path) end, filename: Path.basename(path))
  end

  def image_card(path, title, type \\ :png) do
    Kino.Layout.grid([
      Kino.Markdown.new("**#{title}** — #{size_label(path)}"),
      image(path, type),
      download(path)
    ])
  end

  def probe_streams!(path) do
    {inventory, 0} =
      System.cmd("ffprobe", [
        "-v",
        "error",
        "-show_entries",
        "stream=index,codec_type,codec_name",
        "-of",
        "compact=p=0",
        path
      ])

    inventory
  end

  def png_output(picture) do
    picture
    |> Encoder.png(threads: 1)
    |> Muxer.mux("image2pipe", :stdout, output_options: ["frames:v": 1])
  end
end

alias FFix.{Decoder, Demuxer, Encoder, Filter, Graph, Muxer}

File.mkdir!(Demo.path())
global_options = [y: :flag]
IO.puts("Your files will be in #{Demo.path()}")

Demo.path("filename") builds paths under this notebook's temporary directory. Paths stay the same when you re-run examples; re-evaluating the helper cell creates a fresh directory.

y: :flag allows replacing this notebook's outputs when you re-run cells. Keep your own source files separate from the output paths. The preview helpers load whole files, which is fine for these small samples.

1. Convert a nature recording

Let's start outdoors with a short birdsong recording by NCPRIME. Download the five-second MP3, about 150 KiB, then turn it into a WAV file you could use in an editor.

This audio uses the Pixabay Content License.

sound_path =
  Demo.fetch!(
    "https://cdn.pixabay.com/download/audio/2025/01/23/audio_7b7cfbe3d0.mp3",
    Demo.path("birds.mp3")
  )

wav_path = Demo.path("sound.wav")

command =
  FFix.input(sound_path)
  |> FFix.audio(0)
  |> FFix.output(wav_path)
  |> FFix.command(global: global_options)

IO.puts(FFix.to_shell_string(command))
FFix.run!(command)
Demo.audio(wav_path)

audio(0) selects the first audio stream; indexes start at zero. FFmpeg chooses the WAV format and its default encoder from the filename. No filter is needed.

An output says what to write and where. A command assembles outputs and their inputs. Neither runs FFmpeg until you call run!.

2. Choose the encoding and container

For a smaller web asset, encode the sound as AAC and put it in an MP4 container. Audio-only MP4 files commonly use the .m4a extension.

aac_path = Demo.path("sound.m4a")

command =
  FFix.input(wav_path)
  |> FFix.audio(0)
  |> Encoder.aac(b: "96k")
  |> Muxer.mp4(aac_path)
  |> FFix.command(global: global_options)

FFix.run!(command)
IO.puts("WAV: #{Demo.size_label(wav_path)}; AAC: #{Demo.size_label(aac_path)}")
Demo.audio(aac_path, "audio/mp4")

The encoder controls compression; the muxer packages the encoded streams. b: "96k" requests 96 kbit/s. Try "48k", re-run the cell, and compare the sound and file size.

Muxer helpers create outputs, so they replace FFix.output here—not FFix.command.

3. Give the sound a shape

Now give the birdsong a shape. Draw a waveform to see the calls rise above the quieter background—useful for an audio editor or a clip preview.

waveform =
  FFix.input(sound_path)
  |> FFix.audio(0)
  |> Filter.aformat(channel_layouts: :mono)
  |> Filter.showwavespic(size: "640x160", colors: "DodgerBlue", scale: :sqrt)

command =
  waveform
  |> Demo.png_output()
  |> FFix.command()

waveform_image = FFix.run!(command, stdout: :collect)
Kino.Image.new(waveform_image.stdout, :png)

Time runs left to right. The mono downmix gives one trace, and square-root scaling makes quieter details easier to see. This filter takes audio but produces a picture, so its result goes to an image encoder.

showwavespic reads the selected audio before drawing the image. Keep the input short; try showwaves when you want an animated visualization instead. The waveform guide also covers separate channels and backgrounds.

4. Copy tracks without re-encoding

Now download MDN's CC0 flower clip, about 1.1 MB. It has H.264 video and a silent AAC audio track.

clip_path =
  Demo.fetch!(
    "https://interactive-examples.mdn.mozilla.net/media/cc0-videos/flower.mp4",
    Demo.path("flowers.mp4")
  )

clip = FFix.input(clip_path)
Demo.video(clip_path)

To archive it in Matroska, copy the video and any audio or subtitle tracks. optional: true lets a missing match disappear instead of failing the command.

archive_path = Demo.path("flowers.mkv")

tracks = [
  clip |> FFix.video(0) |> FFix.stream_copy(),
  clip |> FFix.audio(:all, optional: true) |> FFix.stream_copy(),
  clip |> FFix.subtitle(:all, optional: true) |> FFix.stream_copy()
]

command =
  tracks
  |> Muxer.matroska(archive_path)
  |> FFix.command(global: global_options)

FFix.run!(command)

IO.puts(Demo.probe_streams!(archive_path))
Demo.download(archive_path)

Stream copy preserves encoded media, but the destination must support its codecs. Omitting an encoder does not request copy; use FFix.stream_copy explicitly.

:all and optional selections are output queries, not lists of tracks. Filters need an individual stream such as video(clip, 0). For language matching and other queries, see stream selection and FFmpeg's stream specifiers.

5. Resize a video and compare quality

Let's make a smaller web preview and give the flowers some birdsong. Choose a width and quality, then run the next cell. Lower CRF values generally give better quality and larger files.

width_control = Kino.Input.range("Width", min: 160, max: 640, step: 160, default: 320)
crf_control = Kino.Input.range("CRF", min: 18, max: 36, step: 1, default: 23)
Kino.Layout.grid([width_control, crf_control])

width = round(Kino.Input.read(width_control))
crf = round(Kino.Input.read(crf_control))
small_path = Demo.path("flowers-small.mp4")

picture =
  clip
  |> FFix.video(0)
  |> Filter.scale(w: width, h: -2)
  |> Encoder.libx264(crf: crf, preset: "veryfast")

birdsong =
  FFix.input(wav_path, stream_loop: -1)
  |> FFix.audio(0)
  |> Encoder.aac(b: "96k")

output =
  Muxer.mp4([picture, birdsong], small_path,
    movflags: [:faststart],
    output_options: [shortest: :flag]
  )

command = FFix.command(output, global: global_options)
FFix.run!(command)
IO.puts("#{width}px wide; CRF #{crf}; #{Demo.size_label(small_path)}")
Demo.video(small_path)

h: -2 preserves the aspect ratio and gives an even height. faststart moves the MP4's index toward the beginning for progressive download.

stream_loop: -1 repeats the bird recording indefinitely; shortest: :flag stops the output when the video ends. Keep them together so the command stays finite. We map only the picture and birdsong, leaving out the clip's original silent audio.

Filters change decoded frames, so the resized video needs encoding. The WAV audio becomes AAC for the MP4. The H.264 encoding guide explains CRF, presets, and bitrate controls.

6. See what contrast changes

Now look at the picture and its RGB histogram together. Unlike CRF, contrast changes the picture itself, not the compression settings. Use split to give the filtered picture two consumers: the preview and the measurement.

contrast_control = Kino.Input.range("Contrast", min: 0.5, max: 2.0, step: 0.1, default: 1.0)

contrast = Kino.Input.read(contrast_control)

[picture, histogram_input] =
  clip
  |> FFix.video(0)
  |> Filter.scale(w: 320, h: 180)
  |> Filter.eq(contrast: contrast)
  |> Filter.format(pix_fmts: "gbrp")
  |> Filter.split()

histogram =
  Filter.histogram(histogram_input,
    display_mode: :overlay,
    levels_mode: :logarithmic,
    level_height: 160,
    scale_height: 20
  )

command =
  [picture, histogram]
  |> Filter.hstack()
  |> Demo.png_output()
  |> FFix.command()

histogram_image = FFix.run!(command, stdout: :collect)
Kino.Image.new(histogram_image.stdout, :png)

gbrp is planar RGB, so the histogram shows red, green, and blue rather than YUV components. Overlay mode puts the three distributions in one panel; its 180px height matches the picture for hstack.

Darker values sit on the left, brighter values on the right. Higher contrast can push values into the edges and clip detail. Logarithmic scaling makes less frequent values visible. See the histogram examples.

7. Extract and transform images

Seek to one second and write one PNG to stdout. You don't need a temporary image file.

command =
  FFix.input(clip_path, ss: 1)
  |> FFix.video(0)
  |> Filter.scale(w: 480, h: -2)
  |> Encoder.png(threads: 1)
  |> Muxer.mux("image2pipe", :stdout, output_options: ["frames:v": 1])
  |> FFix.command()

poster = FFix.run!(command, stdout: :collect)
Kino.Image.new(poster.stdout, :png)

Input options such as ss apply before reading that input. Make a new declaration when you need a different seek or decoder configuration; existing selections retain the configuration they captured.

You can also feed bytes into FFmpeg. This time, take the poster's left half and give it a mirrored twin. Usually FFmpeg detects file formats and decoders for you; explicit choices help with pipes and raw media.

image_input =
  Demuxer.demux(:stdin, "image2pipe")
  |> Decoder.decode("png", {:video, 0})

[left, right] =
  image_input
  |> FFix.video(0)
  |> Filter.crop(w: "iw/2", h: "ih", x: 0, y: 0)
  |> Filter.split()

command =
  [left, Filter.hflip(right)]
  |> Filter.hstack()
  |> Demo.png_output()
  |> FFix.command()

mirrored = FFix.run!(command, stdin: [poster.stdout], stdout: :collect)
Kino.Image.new(mirrored.stdout, :png)

iw and ih are FFmpeg expressions for the input frame's width and height. Crop once, split into two branches, then join them back into one picture. This is the mirror effect adapted to an in-memory image.

stdin: accepts chunks of bytes; stdout: :collect retains the result in memory. That's convenient for small images. Use files or streamed stdout events for large media.

Generic operations such as Muxer.mux and Decoder.decode accept literal FFmpeg names without requiring a named helper. The equivalent filter escape hatch is Filter.filter(stream, "hflip", [:video]): you supply its output media explicitly.

Optional: try a blend mode

Instead of placing halves side by side, blend the full poster with its reflection. Try screen for a lighter double exposure, multiply for a darker one, or difference to reveal where the pictures disagree.

blend_control =
  Kino.Input.select("Blend mode",
    screen: "Screen",
    multiply: "Multiply",
    difference: "Difference"
  )

blend_mode = Kino.Input.read(blend_control)

[original, reflection] =
  image_input
  |> FFix.video(0)
  |> Filter.format(pix_fmts: "gbrp")
  |> Filter.split()

command =
  original
  |> Filter.blend(Filter.hflip(reflection), all_mode: blend_mode)
  |> Demo.png_output()
  |> FFix.command()

blended = FFix.run!(command, stdin: [poster.stdout], stdout: :collect)
Kino.Image.new(blended.stdout, :png)

Blend math runs on color components. Converting to RGB gives familiar image-editor results; doing the same operations on YUV can produce surprising colors. Both inputs need matching dimensions and pixel formats. The blend gallery has more modes to explore.

Compare looping GIF and WebP images

A small loop is handy when you don't need a video player. Resize and sample one short excerpt, then make GIF and animated WebP in the same FFmpeg run. Compare their appearance and file sizes side by side.

The GIF follows High quality GIF with FFmpeg: build a custom palette, then apply it with deliberate dithering. We need three branches—one to build that palette, one to apply it, and one to encode WebP.

[gif_frames, palette_frames, webp_frames] =
  FFix.input(clip_path, ss: 1, t: 3)
  |> FFix.video(0)
  |> Filter.fps(fps: 10)
  |> Filter.scale(w: 240, h: -1, flags: :lanczos)
  |> Filter.split(outputs: 3)

gif_path = Demo.path("flowers.gif")
webp_path = Demo.path("flowers.webp")
palette = Filter.palettegen(palette_frames, stats_mode: :diff)

gif_output =
  gif_frames
  |> Filter.paletteuse(palette, dither: :bayer, bayer_scale: 3, diff_mode: :rectangle)
  |> Muxer.mux("gif", gif_path, loop: 0)

webp_output =
  webp_frames
  |> Filter.format(pix_fmts: "yuv420p")
  |> Encoder.encode("libwebp_anim", quality: 75)
  |> Muxer.mux("webp", webp_path, loop: 0)

command = FFix.command([gif_output, webp_output], global: global_options)
FFix.run!(command)

Kino.Layout.grid(
  [
    Demo.image_card(gif_path, "GIF", :gif),
    Demo.image_card(webp_path, "Animated WebP", "image/webp")
  ],
  columns: 2
)

palettegen chooses up to 256 colors, then paletteuse applies them to the GIF frames. stats_mode: :diff favors changing pixels; try :full when background colors matter equally.

Bayer dithering trades a visible pattern for less frame-to-frame noise, which often compresses better. diff_mode: :rectangle limits reprocessing to the changing area. Try dither: :sierra2_4a for smoother-looking gradients, then compare the file size.

WebP doesn't need a palette or dithering. libwebp_anim uses lossy compression here; raise quality toward 100 for higher quality, usually at a larger size. Its YUV pixel format is separate from the GIF's palette conversion. The generic encoder and muxer calls work without named WebP helpers, and Kino previews it using the MIME type "image/webp".

Ten fps, 240px width, and a three-second duration keep both animations small. Lanczos scaling preserves detail. loop: 0 repeats both forever; neither format carries audio.

The GIF palette is ready only after the excerpt ends, so this single-command version buffers resized frames. For longer clips, use the article's two-pass approach with the same excerpt and filters in both passes.

8. Generate and combine audio

Here's a different job: make a short notification sound. Two source filters generate tones, amix combines them, and a fade softens the ending. There are no input files.

cue_path = Demo.path("notification.wav")
low_tone = Filter.sine(frequency: 440, duration: 2)
high_tone = Filter.sine(frequency: 660, duration: 2)

command =
  [low_tone, high_tone]
  |> Filter.amix(inputs: 2, duration: :shortest)
  |> Filter.afade(type: :out, start_time: 1.5, duration: 0.5)
  |> FFix.output(cue_path)
  |> FFix.command(global: global_options)

FFix.run!(command)
Demo.audio(cue_path)

Lists connect several streams to one filter. Source filters create streams; sinks consume them without producing another stream. You can combine these with ordinary file inputs in the same graph.

9. Watch four moments together

Make a moving 2×2 mosaic from four starting points in the flower clip. These are four independent input declarations, each with its own seek. Swap the offsets to move the panels around.

panels =
  Enum.map([0, 1, 2, 3], fn offset ->
    FFix.input(clip_path, ss: offset)
    |> FFix.video(0)
    |> Filter.fps(fps: 24)
    |> Filter.scale(w: 320, h: 180)
    |> Filter.format(pix_fmts: "yuv420p")
    |> Filter.setpts(expr: "PTS-STARTPTS")
  end)

mosaic_path = Demo.path("mosaic.mp4")

command =
  panels
  |> Filter.xstack(inputs: 4, layout: "0_0|w0_0|0_h0|w0_h0", shortest: true)
  |> Encoder.libx264(crf: 23, preset: "veryfast")
  |> Muxer.mp4(mosaic_path, movflags: [:faststart])
  |> FFix.command(global: global_options)

FFix.run!(command)
Demo.video(mosaic_path)

The layout lists x_y coordinates in input order: top-left, top-right, bottom-left, bottom-right. w0 and h0 mean the first panel's width and height.

Matching sizes and pixel formats keep the grid predictable. setpts starts each panel's timeline at zero, and shortest: true ends the mosaic when the shortest excerpt runs out—about two seconds here. This combines simultaneous video streams, unlike a contact sheet that collects frames over time. Audio isn't mixed automatically.

See the xstack mosaic guide for larger layouts.

10. Reuse an existing filtergraph

Already have a filtergraph from an FFmpeg command? Parse it, bind its input to a declaration, and continue composing in Elixir.

template = Graph.parse!("[0:v]scale=320:-2,hflip[preview]")
IO.puts(FFix.to_filtergraph(template))

excerpt = FFix.input(clip_path, ss: 2)
bound = Graph.bind(template, %{0 => excerpt})
reused_path = Demo.path("reused-preview.mp4")

command =
  bound[:preview]
  |> Filter.fps(fps: 12)
  |> Encoder.libx264(crf: 25, preset: "veryfast")
  |> Muxer.mp4(reused_path)
  |> FFix.command(global: global_options)

FFix.run!(command)
Demo.video(reused_path)

preview is the graph's output label. Bind the same template again to create another instance of its filters. You can also build templates with named inputs using FFix.Graph, or simply put a filter pipeline in a regular Elixir function.

11. Make several deliveries in one run

Let's produce a main video, a two-second preview, and a contact sheet. Resize once, then split the filtered pictures into three branches.

A filtered output must have exactly one consumer. Use split for video or asplit for audio when branching. Direct input streams can be reused without a split.

[main, preview, sheet] =
  clip
  |> FFix.video(0)
  |> Filter.scale(w: 640, h: -2)
  |> Filter.split(outputs: 3)

main_path = Demo.path("main.mp4")
preview_path = Demo.path("preview.mp4")
sheet_path = Demo.path("contact-sheet.png")

main_tracks = [
  Encoder.libx264(main, crf: 20, preset: "veryfast"),
  clip |> FFix.audio(0) |> FFix.stream_copy()
]

main_output = Muxer.mp4(main_tracks, main_path, movflags: [:faststart])

preview_output =
  preview
  |> Filter.scale(w: 320, h: -2)
  |> Encoder.libx264(crf: 28, preset: "veryfast")
  |> Muxer.mp4(preview_path, output_options: [t: 2])

sheet_output =
  sheet
  |> Filter.fps(fps: 1)
  |> Filter.scale(w: 160, h: -2)
  |> Filter.tile(layout: "2x2")
  |> Encoder.png(threads: 1)
  |> Muxer.image2(sheet_path, update: true, output_options: ["frames:v": 1])

delivery_command =
  FFix.command([main_output, preview_output, sheet_output], global: global_options)

FFix.run!(delivery_command)

Kino.Layout.grid([
  Demo.video(main_path),
  Demo.video(preview_path),
  Demo.image(sheet_path)
])

Output options stay with their destination: the preview's t: 2 doesn't shorten the main video. tile collects four sampled frames into one image.

Reusing clip opens that input once. Outputs may also use entirely different inputs. They share one FFmpeg process, so use separate commands when jobs need independent cancellation or retries.

Want to inspect the full composition? This prints it without running it again:

IO.puts(FFix.to_shell_string(delivery_command))

12. Package adaptive streaming with shared audio

An HLS presentation needs coordinated encodes and playlist references. We'll make 640px and 320px video renditions that share one audio rendition.

At 24 fps, g: 48 and sc_threshold: 0 give regular two-second keyframe intervals. This aligns the renditions with hls_time: 2 segments.

hls_dir = Demo.path("hls")
File.mkdir_p!(hls_dir)

[high, low] =
  clip
  |> FFix.video(0)
  |> Filter.fps(fps: 24)
  |> Filter.split()

encode_rendition = fn video, width, bitrate ->
  video
  |> Filter.scale(w: width, h: -2)
  |> Encoder.libx264(b: bitrate, g: 48, sc_threshold: 0, preset: "veryfast")
end

high = encode_rendition.(high, 640, "600k")
low = encode_rendition.(low, 320, "180k")

audio =
  clip
  |> FFix.audio(0)
  |> Encoder.aac(b: "64k")

output =
  Muxer.hls([sound: audio, low: low, high: high], Path.join(hls_dir, "%v.m3u8"),
    hls_time: 2,
    hls_playlist_type: :vod,
    hls_flags: [:independent_segments],
    master_pl_name: "master.m3u8",
    var_stream_map: fn streams ->
      "#{streams.high.specifier},agroup:audio,name:high " <>
        "#{streams.low.specifier},agroup:audio,name:low " <>
        "#{streams.sound.specifier},agroup:audio,name:audio,default:yes"
    end
  )

hls_command = FFix.command(output, global: global_options)
FFix.run!(hls_command)
IO.puts(File.read!(Path.join(hls_dir, "master.m3u8")))
File.ls!(hls_dir) |> Enum.sort()

The callback receives final output-local positions. Here, sound is a:0, low is v:0, and high is v:1. Reorder the named mappings and re-run: the references still follow the right rendition.

Callbacks require individual streams with known media, not :all or optional queries. Keep callbacks free of side effects: rendering a command and running it each resolve these options.

The master playlist should list both resolutions and one audio group. To play it, serve the whole HLS directory over HTTP and use an HLS-capable player. See HLS options and named output mappings.

13. Watch progress and handle failures

A live frame can show progress while FFmpeg works. Start with a waiting message, then update it as progress and exit events arrive. This example reads at playback speed (re: :flag) so you have time to see updates. Omit that input option for normal batch processing.

observation_command =
  FFix.input(clip_path, re: :flag)
  |> FFix.video(0)
  |> Muxer.null(:stdout)
  |> FFix.command()

progress_frame = Kino.Frame.new() |> Kino.render()
Kino.Frame.render(progress_frame, "Waiting for progress…")

observation_command
|> FFix.stream!(progress: true)
|> Enum.each(fn
  {:progress, progress} ->
    Kino.Frame.render(progress_frame, "Processed #{progress.frame || 0} frames")

  {:exit, result} ->
    Kino.Frame.render(progress_frame, "Finished in #{result.duration_ms} ms")

  _event ->
    :ok
end)

The null muxer discards the media; decoding still runs. Event streams are lazy: each enumeration starts a fresh process, and halting enumeration cancels that run. Consume events as they arrive rather than collecting an entire long job's output.

run! raises on execution failure. Use run when your application needs to handle it. Here's a deliberate missing-file error:

missing_command =
  FFix.input(Demo.path("missing.mp3"))
  |> FFix.audio(0)
  |> FFix.output(Demo.path("missing.wav"))
  |> FFix.command()

case FFix.run(missing_command) do
  {:ok, result} -> result
  {:error, error} -> %{kind: error.kind, message: Exception.message(error)}
end

Keep exploring

Named helper docs describe recorded FFmpeg metadata. To check the build installed on your machine, query it explicitly:

{:ok, encoder} = FFix.Discovery.help(:encoder, "libx264")
encoder.properties

• FFix.Filter: cropping, overlays, expressions, concatenation, and audio processing.
• FFix.Runner: streaming bytes, diagnostics, and choosing an executable with ffmpeg:.
• FFix.Discovery: installed codecs, formats, and filters. Use ffprobe to inspect a media file instead.

Your generated files remain in the directory returned by Demo.path(). Download anything you want to keep before removing that temporary directory.