PromptVault.TokenCounter.PretendTokenizer (PromptVault v0.1.0)

View Source

Token counter implementation using the fnord library.

This implementation uses AI.PretendTokenizer.guesstimate_tokens/1 from the fnord package to provide token count estimates.