Fil.Plug (Fil v0.2.0)

Copy Markdown View Source

Serves the files of a disk over HTTP: the URLs Fil.Plugin.URL signs, or with public: true, every file.

plug Fil.Plug, at: "/storage/uploads", disk: &MyApp.Storage.uploads/0

Signed URLs make direct downloads and uploads work on every disk: local and memory disks, which can't sign URLs themselves, and S3 disks whose files should go through your application.

Needs Plug, an optional dependency of Fil.

Options

  • :disk - Required. The disk to serve. A Fil.Disk, or a capture of a named 0-arity function or {module, function, args} that returns one. A function is called each time the disk is used, so it can build the disk from runtime config.

  • :at (String.t/0) - The path to serve under, e.g. "/storage/uploads". Requests outside it pass through untouched, which is how the plug sits in an endpoint. Without it, the plug serves every request it gets, as with forward.

  • :public (boolean/0) - Serves GET and HEAD without a signature, like Plug.Static for a disk, so every file on the disk can be downloaded by anyone who knows its path:

    plug Fil.Plug, at: "/avatars", disk: &MyApp.Storage.avatars/0, public: true

    GET /avatars/1.png then returns 1.png from the disk, on every adapter. There are no directory listings, and a path can't leave the disk root. With Fil.Plugin.URL and base_url: "http://localhost:4000/avatars" on the disk, Fil.url/2 builds these URLs.

    Uploads still need a signed URL, and without a :secret for Fil.Plugin.URL on the disk, a PUT gets a 403. A signed download URL that has expired or was changed still works on a public disk, but without the content-disposition it was signed with.

    The default value is false.

  • :max_body_size (pos_integer/0) - The largest upload in bytes, 100 MiB by default. A larger PUT gets a 413, and nothing is written. Uploads are streamed into the disk, so the limit is about storage, not memory. S3 keeps one part of an upload without a content-length in memory (see Fil.Adapter.S3). The default value is 104857600.

Mounting

In a Phoenix app, the plug goes into the endpoint or behind a route. The endpoint takes uploads in any format, and the router lets plugs of your own run first, such as authentication.

Via Endpoint

Put it in the endpoint before Plug.Parsers, at the path the plugin's :base_url points to:

plug Fil.Plug, at: "/storage/uploads", disk: &MyApp.Storage.uploads/0

The :base_url of the disk's Fil.Plugin.URL is then "http://localhost:4000/storage/uploads", and every URL Fil.signed_url/3 builds for it is served here.

Requests pass through untouched when the disk doesn't sign URLs with Fil.Plugin.URL (and the plug isn't public). An S3 disk without it signs URLs that go to S3 directly, so the plug can stay in the endpoint when production uses S3.

Via Router

forward runs the plug behind a router pipeline, for example to let only signed-in users download from a public disk:

pipeline :storage do
  plug :require_authenticated_user
end

scope "/storage" do
  pipe_through :storage
  forward "/avatars", Fil.Plug, disk: &MyApp.Storage.avatars/0, public: true
end

forward removes /storage/avatars from the path, so the plug needs no :at. The :base_url of Fil.Plugin.URL is still the full URL, "http://localhost:4000/storage/avatars", because signatures cover the whole request path.

Two things in a Phoenix app get in the way of uploads there:

  • Plug.Parsers in the endpoint reads the bodies it has a parser for before the router runs (in a new Phoenix app JSON, form and multipart bodies). A PUT with one of those content types reaches the plug without its body and gets a 400 (the plug compares what it can read with the content-length). Other content types, such as image/png or application/pdf, pass through unread. Use the endpoint for uploads in any format.
  • the :browser pipeline's protect_from_forgery rejects a PUT without a CSRF token. Use a pipeline of your own, as above.

Sending a file from a controller

send_file/3 sends one file as the response of a controller action, with the same headers, conditional requests and ranges as the mounted plug. The controller decides who gets the file, so no signed URL is involved:

def download(conn, %{"id" => id}) do
  with {:ok, report} <- Reports.file(conn.assigns.current_scope, id),
       {:ok, conn} <- Fil.Plug.send_file(conn, report, disposition: {:attachment, "report-#{id}.csv"}) do
    conn
  end
end

A missing file returns the error, so the controller's action_fallback can answer it.

Responses

  • GET and HEAD on a URL signed for :get return the file, with its stored content type or one guessed from the extension, and the content-disposition the URL was signed with (disposition:). The file is streamed as a chunked response, so it never has to fit in memory
  • downloads send etag and last-modified from Fil.stat/3 and accept-ranges: bytes. A request whose if-none-match lists the etag (or is *) gets a 304 without a body, and so does one without if-none-match whose if-modified-since isn't older than the file. The etag is sent as a strong validator on every adapter, including the "size-mtime" tag of Fil.Adapter.Local, as nginx and Plug.Static do
  • a GET with one byte range (range: bytes=0-1023, bytes=1024- or bytes=-1024) gets a 206 with that part and its content-range, read from the storage with Fil.stream/3's offset: and length:. A range that starts at or after the end of the file is a 416. Several ranges, a range the plug can't parse, and an if-range that's neither the etag nor the exact last-modified get a 200 with the whole file. Ranges work with signed URLs too, because the signature covers the method, the path and the query, not the headers. Offsets count bytes of the content as plugins return it, and the total in content-range is the size Fil.stat/3 returns, so a plugin that changes the size of the content should change it in the stat too
  • PUT on a URL signed for :put writes the request body, with the request's content-type, the same as a presigned PUT on S3. The body is streamed into Fil.write/4 as it's read, with the content-length as size:. Plugins attached to the disk run as for any other write. A URL signed with content_type: or size: takes only an upload with that content-type and content-length, and one signed with if_exists: :error writes with it, so a second upload to the same path gets a 409 (see Uploads)
  • a request that doesn't match its signature, or comes after the URL expired, gets a 403, and so does an upload whose content-type or content-length isn't the one its URL was signed with
  • an upload larger than :max_body_size gets a 413, and one whose body something else already read (see Mounting) a 400. Neither writes anything
  • a missing file gets a 404, and so does a file the storage denies access to, so a client can't tell which files exist
  • a failed write gets a 409 if the file already exists or changed meanwhile, 422 if the disk refuses the content (a Fil.InvalidRequestError about the content, such as from Fil.Plugin.Thumbnails for a file that isn't an image), 507 if the storage is full and 503 if it's unavailable. Any other error is a 500 with a generic body, and its message goes to the Logger. So is S3's InvalidRequest, on a download too, because S3 sends it for problems with the request or the bucket's configuration as well
  • an upload a plugin refuses with a Fil.InvalidContentError, before or while the body is read, gets a 413 for content that's too large, a 415 for a content type or an extension the disk doesn't take, and a 422 for anything else. Nothing is written, and the body doesn't say which rule the upload broke

Summary

Functions

Sends a file as the response, from a controller or a plug of your own.

Sends a file as the response. See send_file/2.

Sends a file as the response. See send_file/2.

Same as send_file/2, raising the error on failure.

Same as send_file/3, raising the error on failure.

Same as send_file/4, raising the error on failure.

Functions

send_file(conn, ref)

@spec send_file(Plug.Conn.t(), Fil.Ref.t()) :: Fil.result(Plug.Conn.t())

Sends a file as the response, from a controller or a plug of your own.

{:ok, conn} = Fil.Plug.send_file(conn, report, disposition: {:attachment, "Q3 report.pdf"})

The file goes out as the mounted plug sends a download (see Responses), on every adapter: streamed in a chunked response, with etag, last-modified and accept-ranges, a 304 for a matching if-none-match or if-modified-since, a 206 for one byte range and a 416 for a range outside the file. A HEAD gets the headers only. When the client closes the connection, the rest of the file isn't read. Headers set on conn before the call are kept, so you can add a cache-control, for example.

Conditional requests and ranges apply to GET and HEAD. Any other method, such as a POST to an export action, gets the whole file.

Returns {:ok, conn} with the response sent, whatever its status. A missing file, a directory (Fil.NotFoundError with reason: :eisdir) and any other error of Fil.stat/2 or Fil.stream/3 return {:error, exception} before anything is sent, so the caller can answer it, for example in an action_fallback. An error while the file is streamed comes after the status line, so it raises and the client gets a truncated response.

Options

  • :content_type (String.t/0) - The content-type of the response. Without it, the file's stored content type, or one guessed from its extension.

  • :disposition - The content-disposition of the response. :inline lets the browser show the file, :attachment saves it under the file's name, and {:attachment, filename} under another name, which can be any UTF-8 string. Without it, no content-disposition is sent.

send_file(conn, disk, path)

@spec send_file(Plug.Conn.t(), Fil.Disk.t(), Path.t()) :: Fil.result(Plug.Conn.t())
@spec send_file(Plug.Conn.t(), Fil.Ref.t(), keyword()) :: Fil.result(Plug.Conn.t())

Sends a file as the response. See send_file/2.

send_file(conn, disk, path, opts)

@spec send_file(Plug.Conn.t(), Fil.Disk.t(), Path.t(), keyword()) ::
  Fil.result(Plug.Conn.t())

Sends a file as the response. See send_file/2.

send_file!(conn, ref)

@spec send_file!(Plug.Conn.t(), Fil.Ref.t()) :: Plug.Conn.t()

Same as send_file/2, raising the error on failure.

send_file!(conn, a, b)

@spec send_file!(Plug.Conn.t(), Fil.Disk.t(), Path.t()) :: Plug.Conn.t()
@spec send_file!(Plug.Conn.t(), Fil.Ref.t(), keyword()) :: Plug.Conn.t()

Same as send_file/3, raising the error on failure.

send_file!(conn, disk, path, opts)

@spec send_file!(Plug.Conn.t(), Fil.Disk.t(), Path.t(), keyword()) :: Plug.Conn.t()

Same as send_file/4, raising the error on failure.