View Source Tokenizers.Model (Tokenizers v0.1.1)
The struct and associated functions for the tokenizer model.
Link to this section Summary
Functions
Retrieves information about the model.
Link to this section Types
Specs
Link to this section Functions
Specs
Retrieves information about the model.
Infomration retrieved differs per model but all include model_type
.