View Source tflite_beam_full_tokenizer (tflite_beam v0.3.8)
Runs end-to-end tokenization.
Related link: https://github.com/tensorflow/examples/blob/master/lite/examples/bert_qa/ios/BertQACore/Models/Tokenizers/FullTokenizer.swift
Runs end-to-end tokenization.
Related link: https://github.com/tensorflow/examples/blob/master/lite/examples/bert_qa/ios/BertQACore/Models/Tokenizers/FullTokenizer.swift