Bedrock.JobQueue.Consumer.Scanner (bedrock_job_queue v0.2.0)
View SourceScans the pointer index for queues with visible items.
Per QuiCK paper: The Scanner continuously scans the pointer index to find queues that have items ready to process.
Fairness (QuiCK-inspired)
To prevent unfair prioritization of queues that happen to appear first:
- Queue IDs are shuffled before notification
- Last-notified tracking ensures round-robin across scans
- Jittered scan intervals prevent thundering herd
Pointer GC
The Scanner also periodically garbage collects stale pointers from empty queues. This is done as part of the scan cycle to avoid an extra process.
Configuration
:repo- Required. The Bedrock Repo module:manager- Required. The Manager process name/pid to notify:name- Process name (default:Bedrock.JobQueue.Consumer.Scanner):root- Root keyspace (default:Keyspace.new("job_queue/")):interval- Base scan interval in ms (default: 100):batch_size- Max pointers to scan per cycle (default: 100):jitter_percent- Random jitter as percentage of interval (default: 20):selection_frac- Fraction of visible queues to notify (default: 0.5):selection_max- Maximum queues to notify per scan (default: 10):gc_interval- How often to run pointer GC in ms (default: 60_000):gc_grace_period- Grace period before considering pointer stale in ms (default: 60_000):gc_batch_size- Max stale pointers to GC per cycle (default: 100):now_fn- Function returning current time in ms (default:fn -> System.system_time(:millisecond) end):worker_pool- Required. Task.Supervisor for checking worker availability:concurrency- Required. Max concurrent workers
Summary
Functions
Returns a specification to start this module under a supervisor.
Functions
Returns a specification to start this module under a supervisor.
See Supervisor.