Listen

Description

History of tokenization going back to 70s language modeling, modern tokenization approaches with BPE, and the future of token-free bitestreams