Recollect.Learner behaviour (recollect v0.5.1)

Behaviour for learning modules that extract knowledge from external sources.

Each learner implements this behaviour to provide a source of knowledge that can be automatically ingested into Recollect's memory.

Summary

Types

extract()

pattern()

Callbacks

detect_patterns(list)

Detect patterns across multiple events (optional).

extract(event)

Extract learnable content from an event.

fetch_since(since, scope_id)

Fetch new events/items to learn from since the last check.

source()

Example implementation

summarize(events, scope_id)

Summarize a batch of events into synthesized memory entries (optional).

Types

extract()

@type extract() :: %{
  content: binary(),
  entry_type: atom(),
  emotional_valence: atom(),
  tags: [binary()],
  metadata: map()
}

pattern()

@type pattern() :: %{type: atom(), events: [map()], summary: binary()}

Callbacks

detect_patterns(list)

(optional)

@callback detect_patterns([map()]) :: [map()]

Detect patterns across multiple events (optional).

Called with a batch of events to find cross-event patterns like migrations.

Returns

List of pattern maps with :type, :events, :summary

extract(event)

@callback extract(event :: map()) :: {:ok, map()} | {:skip, binary()} | {:error, term()}

Extract learnable content from an event.

Parameters

event - An event map from fetch_since

Returns

{:ok, extract} - Extract map with :content, :entry_type, :emotional_valence, :tags, :metadata
{:skip, reason} - Skip this event (not learnable)
{:error, term} - Processing error

fetch_since(since, scope_id)

@callback fetch_since(since :: DateTime.t() | String.t(), scope_id :: binary()) ::
  {:ok, [map()]} | {:error, term()}

Fetch new events/items to learn from since the last check.

Parameters

since - DateTime or string (e.g., "2024-01-01")
scope_id - The scope to learn for

Returns

{:ok, [event]} - List of events to process
{:error, reason} - If fetching fails

source()

@callback source() :: atom()

Example implementation:

defmodule MyApp.Learner.CI do
  @behaviour Recollect.Learner

  @impl true
  def source, do: :ci

  @impl true
  def fetch_since(since, scope_id) do
    # Fetch CI failures since the given date
  end

  @impl true
  def extract(event) do
    {:ok, %{
      content: "CI Failure: #{event.failure_message}",
      entry_type: :observation,
      emotional_valence: :negative,
      tags: ["ci", "failure", event.workflow],
      metadata: %{source: :ci, job_id: event.id}
    }}
  end
end

end

@doc """ Return the source name (e.g., :git, :terminal, :ci).

summarize(events, scope_id)

(optional)

@callback summarize(events :: [map()], scope_id :: binary()) :: [map()]

Summarize a batch of events into synthesized memory entries (optional).

Called after all individual events have been extracted. Allows learners to produce higher-level insights (e.g., commit grouping, deprecation events).

Returns

List of extract maps (same shape as extract/1 return value)