LMStudio.NeuralArchitecture.AttentionMechanism (lmstudio v0.1.0)

Multi-head attention mechanism

Summary

Functions

forward(attention, query, key, value, mask \\ nil)

new(model_dim, num_heads \\ 8, opts \\ [])