Getting Started

View Source

Bulkinup validates attrs maps through your schemas' changeset functions, then writes a parent and its nested associations across multiple tables in one call. This guide sets up the basics; see Nested Associations for the multi-table part.

Installation

Add the package to your list of dependencies in mix.exs, then run mix deps.get:

def deps do
  [
    {:bulkinup, "~> 0.6.0"}
  ]
end

A schema to work with

Here is a contrived migration and schema:

priv/repo/migrations/0001_create_persons.exs

defmodule YourProject.Repo.Migrations.CreatePersons do
  use Ecto.Migration

  def change do
    create table(:persons) do
      add :name, :string
    end
  end
end

lib/your_project/persons/person.ex

defmodule YourProject.Persons.Person do
  use Ecto.Schema
  import Ecto.Changeset

  schema "persons" do
    field :name, :string
  end

  def changeset(person \\ %__MODULE__{}, attrs) do
    person
    |> cast(attrs, [:id, :name])
    |> validate_required([:id, :name])
  end
end

The changeset function must be callable with a single argument (the attrs map) — a changeset/2 whose first argument defaults to an empty struct, as above, is the usual shape.

Inserting and upserting

After running the migration (mix ecto.migrate), try it in an IEx shell (iex -S mix). Bulkinup.insert/4 is a pure bulk insert:

iex> Bulkinup.insert(
...>   YourProject.Repo,
...>   YourProject.Persons.Person,
...>   [%{id: 1, name: "Alice"}, %{id: 2, name: "Bob"}]
...> )
{:ok, %{inserted: 2, skipped: 0}}

Running the same call again raises, because the rows already exist. Bulkinup.upsert/4 updates existing rows instead:

iex> Bulkinup.upsert(
...>   YourProject.Repo,
...>   YourProject.Persons.Person,
...>   [%{id: 1, name: "Alicia"}, %{id: 2, name: "Bobby"}]
...> )
{:ok, %{upserted: 2, skipped: 0}}

iex> YourProject.Repo.all(YourProject.Persons.Person) |> Enum.map(& &1.name)
["Alicia", "Bobby"]

Repo-scoped calls with use Bulkinup

Instead of passing the repo on every call, add use Bulkinup to your repo module. It injects bulk_insert/3 and bulk_upsert/3, and lets you declare app-wide defaults once:

defmodule YourProject.Repo do
  use Ecto.Repo,
    otp_app: :your_project,
    adapter: Ecto.Adapters.Postgres

  use Bulkinup,
    upsert: [replace_all_except: [:inserted_at]]
end

YourProject.Repo.bulk_upsert(YourProject.Persons.Person, attrs_list)

See Bulkinup.__using__/1 for the defaults and precedence rules.

Invalid rows are skipped, visibly

Rows whose changesets are invalid are skipped rather than written. The counts in the return value make this visible, and each call that skips rows emits one :warning log summarizing them (with per-row detail at the :debug level):

iex> Bulkinup.insert(
...>   YourProject.Repo,
...>   YourProject.Persons.Person,
...>   [%{id: 3, name: "Carol"}, %{id: 4}]
...> )
{:ok, %{inserted: 1, skipped: 1}}

A database error is different: it raises, and (by default) the surrounding transaction rolls back every change from the call. To recover invalid rows instead of skipping them, see the :recover_changeset_errors option in the Recipes guide.

Where to next