erllama_model_evictor behaviour (erllama v0.10.0)

View Source

Behaviour for proactive model eviction under memory pressure.

erllama_scheduler is the engine-level memory-pressure responder, but the model fleet (which models are loaded, their request recency, how to unload one cleanly) is owned by the server layer, which the engine must not depend on. This behaviour is the seam: the scheduler calls evict_one/0 on an operator-configured module (scheduler.model_evictor) when cache eviction could not relieve sustained pressure, and the server ships an implementation.

evict_one/0 returns {unloaded, ModelId} when it unloaded an idle model, or none when nothing could be freed (no idle model, or all candidates turned busy). It must not raise; the scheduler additionally guards the call.

Summary

Callbacks

evict_one()

-callback evict_one() -> {unloaded, binary()} | none.

Functions

evict_one(Mod)

-spec evict_one(module()) -> {unloaded, binary()} | none.