tiktoken v0.2.0
  • Pages
  • Modules

    View Source Tiktoken (tiktoken v0.2.0)

    Documentation for Tiktoken.

    Link to this section Summary

    Functions

    decode(model, ids)
    encode(model, text, allowed_special \\ [])
    encode_ordinary(model, text)
    encode_with_special_tokens(model, text)
    encoding_for_model(model)

    Link to this section Functions

    Link to this function

    decode(model, ids)

    View Source
    Link to this function

    encode(model, text, allowed_special \\ [])

    View Source
    Link to this function

    encode_ordinary(model, text)

    View Source
    Link to this function

    encode_with_special_tokens(model, text)

    View Source
    Link to this function

    encoding_for_model(model)

    View Source

    Hex Package Hex Preview (current file) Download ePub version

    Built using ExDoc (v0.29.2) for the Elixir programming language