Fil.Plugin.Validation (Fil v0.2.0)

Copy Markdown View Source

Refuses writes that break the disk's rules: too large or too small, of a type the disk doesn't take, with the wrong extension, or anything a check of your own refuses.

avatars =
  Fil.disk(adapter: Fil.Adapter.Local, root: "priv/storage/avatars")
  |> Fil.Plugin.Validation.attach(max_size: 2_000_000, content_types: ["image/png", "image/jpeg", "image/webp"])

Fil.write(avatars, "1.png", {:file, "cat.gif"})
#=> {:error, %Fil.InvalidContentError{reason: {:content_type, "image/gif"}}}

A refused write returns Fil.InvalidContentError with a :reason that says which rule it broke, and writes nothing, on every adapter. What can be checked before the content is read (a size: the caller declared, the extension, the content_type:, your check:) is checked first, and the adapter never runs. The content itself is checked while the adapter reads it, with the size of one chunk plus 1445 bytes in memory, and a stream of known size keeps its size, so S3 still sends it in one request. A stream that turns out too large fails before the chunk that crosses the limit is passed on. Fil.Plug answers a refused upload with a 413 for {:too_large, _}, a 415 for a content type or an extension, and a 422 for anything else.

Writes are checked whole or streamed, including the write half of a copy across disks. A copy or a move within the disk has no content to check, but its destination has to have an allowed extension (and one whose content type is allowed), because local disks serve files with the type of their extension. Reads aren't checked.

iex> disk =
...>   Fil.disk(adapter: Fil.Adapter.Memory)
...>   |> Fil.Plugin.Validation.attach(max_size: 10, extensions: ["txt"])
iex> {:error, error} = Fil.write(disk, "notes.txt", "more than ten bytes")
iex> error.reason
{:too_large, 10}
iex> {:error, %Fil.InvalidContentError{reason: {:extension, "md"}}} = Fil.write(disk, "notes.md", "short")
iex> Fil.write(disk, "notes.txt", "short")
{:ok, Fil.ref(disk, "notes.txt")}

Content types

With content_types:, the type comes from the content's first bytes, its magic bytes, and has to be allowed. Images, PDF, ZIP and the formats built on it (Office documents, EPUB), audio, video, archives and executables have a signature. So do HTML that starts the way browsers recognise it (the tags of the WHATWG MIME Sniffing standard), SVG and XML, so they can't pass as plain text. Text that starts like HTML counts as HTML, which also catches Markdown that starts with <p> or a comment. A type that's declared too has to match the content:

  • the call's content_type: (the request's content-type on an upload through Fil.Plug) has to be allowed, and the content has to be of that type. A GIF declared as image/png fails with {:content_type_mismatch, "image/png", "image/gif"}. application/octet-stream counts as no type
  • the extension's type, when MIME knows it, has to be allowed. So x.html fails even if its content is a PNG
  • content without a signature, such as CSV, JSON or plain text, is checked by its declared type, or else by its extension's type. That type has to be allowed and has to be one without a signature: a file declared as image/png without the PNG signature fails with {:content_type_mismatch, "image/png", nil}. Without a type it fails with {:content_type, nil}

Office documents are ZIP files inside, and an .xlsx is checked as application/zip with the type it declares. So content_types: ["application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"] takes ZIP files named or declared as Excel files, and no ZIP named or declared as anything else. A file whose content doesn't match its extension, such as a PNG renamed to .jpg, fails when its declared type comes from the extension, as it does in browsers.

HTML, SVG, XML and the +xml types are documents a browser renders and runs scripts in, also XML with an XHTML root (detected as text/html) or a stylesheet. A wildcard such as image/* or text/* doesn't take them, so they have to be listed by name, and only for disks whose files are never served from your application's origin.

A copy across disks writes without a content_type:, so content without a signature is checked by the destination's extension, and fails with {:content_type, nil} without one.

Upload URLs

An upload through Fil.Plug (a disk with Fil.Plugin.URL and a :secret) is an ordinary write, and is checked like any other. A presigned PUT on S3 goes to the storage directly, so this plugin never sees its content. It checks what it can when the URL is signed with Fil.signed_url/3 and method: :put: check:, the extension, and the signed size: and content_type:, which S3 enforces (see Uploads). What it can't check depends on :presigned_uploads:

  • :refuse (the default) refuses to sign the URL with a Fil.UnsupportedError when content_types: is set, because nobody checks the magic bytes, with reason: {:unchecked, :content_types}. So does a URL without size: when max_size: or min_size: is set ({:unchecked, :max_size})
  • :check_declared takes the signed content_type: as the upload's type, without the magic bytes, and still refuses a URL without content_type: or size: when a rule needs it

Uploads through Fil.LiveView.external/2 sign their URLs with Fil.signed_url/3, so the same applies to them. On a disk that Fil.Plug serves, the URL is checked when it's signed only if this plugin is attached before Fil.Plugin.URL, which answers Fil.signed_url/3 without the plugins after it. The upload is checked either way.

Content checks of your own

check: sees the op, not the content. A check of the content is a plugin of its own on Fil.Op.scan_content/4, which raises Fil.InvalidContentError (see Inspecting content). Attach this plugin before plugins that change the content, such as compression, so it checks what the caller wrote.

Options

  • :max_size (pos_integer/0) - The largest content in bytes. Larger content fails with reason: {:too_large, max_size}.

  • :min_size (non_neg_integer/0) - The smallest content in bytes, 1 to refuse empty files. Smaller content fails with reason: {:too_small, min_size}.

  • :content_types - The content types the disk takes, such as "image/png", or "image/*" for every image type but SVG. They're checked against the content itself, see Content types.

  • :extensions (list of String.t/0) - The extensions the disk takes, such as "png", with or without the dot, in any case. A path with another extension, or none, fails with reason: {:extension, extension} ("" for none).

  • :check - A check of your own: a function, or {module, function, args} called with the op first. It gets the op before the content is read and returns :ok, or {:error, reason} to refuse it with that reason.

  • :presigned_uploads - What to do with an upload URL the storage signs itself, such as a presigned PUT on S3, whose upload never reaches this plugin. See Upload URLs. The default value is :refuse.

Summary

Functions

Attaches the plugin to disk under the name Fil.Plugin.Validation.

Functions

attach(disk, opts \\ [])

@spec attach(Fil.Disk.t(), keyword()) :: Fil.Disk.t()

Attaches the plugin to disk under the name Fil.Plugin.Validation.

The same as plugins: [{Fil.Plugin.Validation, :call, opts}] in Fil.disk/1, except that the options are validated here instead of on the first write.