V8 · 1.4 / SENTENCEPIECE

Unigram end to end

Learn probabilities from a corpus, then find the best complete segmentation.

Train → encode → decodeStep through · revisit any stage
ChaptersChoose a chapter, then explore its steps

Input
Output

This step: how the data changes

Updates with the step and vocabulary · rounding is for display only
Input

        
Output

      

      
Next

    

    

  

  
← → Change step · Space Play / pause