Tornex.Scheduler.Bucket (Tornex v0.3.0)

View Source

Representation of a user's bucket for API requests used for rate-limiting and prioritization.

A bucket can contain as many API requests as the BEAM VM and the memory can handle. However, each bucket will only process 10 API requests every 6 seconds.

The bucket provides the enqueue/1 and enqueue/2 operations providing the core functionality of this module:

  • creating the bucket if necessary,
  • handling telemetry,
  • and request handling.

Bucket Creation

Creating buckets is not required, but you can create and store buckets in a different manner if you need to do so.

A buckets can be created with the following two methods:

  • new/1 to create a bucket and store the PID of the GenServer manually
  • enqueue/1 to create a bucket and let Tornex store the PID of the GenServer in a pre-initialized registry Tornex.Scheduler.BucketRegistry

Making API Requests

API requests are made with the enqueue/1 and the enqueue/2 operations with a Tornex.Query or Tornex.SpecQuery struct. The operations will then wait until the API call has been scheduled and the Torn API has responded instead of ending the invocation early and using a later await-like function.

However, for example, this can still be done with the built-in Task module (including to handle many API requests at once):

1..10
|> Enum.map(fn n ->
  Task.async(fn ->
    Tornex.Scheduler.Bucket.enqueue(pid, query)
  end)
end)
|> Task.await_many(timeout)

Summary

Functions

Returns a specification to start this module under a supervisor.

Enqueues a Tornex.Query or Tornex.SpecQuery to the queue of the Tornex.Scheduler.Bucket for the API key's user.

Retrieves the pid of the bucket by the bucket's user ID.

Attempts to start a Tornex.Scheduler.Bucket and return the pid of a bucket.

Starts the bucket for a user.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

enqueue(query, opts \\ [])

@spec enqueue(query :: Tornex.Query.t() | Tornex.SpecQuery.t(), opts :: Keyword.t()) ::
  term()

Enqueues a Tornex.Query or Tornex.SpecQuery to the queue of the Tornex.Scheduler.Bucket for the API key's user.

The pid of the bucket belonging to the API key's user will be retrieved with Tornex.Scheduler.Bucket.get_by_id/1, and will be used to enqueue the query for that specific bucket.

Options

  • :timeout - Timeout of the GenServer call in milliseconds (default: 60_000)

get_by_id(user_id)

@spec get_by_id(user_id :: integer()) :: {:ok, pid()} | :error

Retrieves the pid of the bucket by the bucket's user ID.

Using the Tornex.Scheduler.BucketRegistry that creates the relationship between the bucket and the user ID upon bucket creation, the pid of the bucket will be returned. If a bucket is not registered for that user, :error will be returned.

new(user_id)

@spec new(user_id :: integer()) :: pid() | nil

Attempts to start a Tornex.Scheduler.Bucket and return the pid of a bucket.

The Tornex.Scheduler.Supervisor will attempt to start the bucket as a child. If the bucket does not exist, the bucket will be created and the pid of the bucket will be returned. If the bucket is already a child of Tornex.Scheduler.Supervisor, the pid of the existing bucket will be returned.

If there is an error in retrieving the pid of the bucket or an error in creating the bucket (e.g. maximum number of children in the supervisor reached), nil will be returned.

Examples

iex> Tornex.Scheduler.Bucket.new(2383326)
#PID<0.105.0>

start_link(opts \\ [])

Starts the bucket for a user.

Options

  • :user_id - (any) A required unique identifier for the user the bucket belongs to

Examples

iex> Tornex.Scheduler.Bucket.start_link(user_id: 2383326)
{:ok, process}
iex> is_pid(process)
true