[entravial]

~AlphaFlame~

Tokenizing works by splitting what you say into tokens which can be read using #t(index)... take this for instance...

tokenize I am a bear;

would give you these tokens
#t(0) I
#t(1) am
#t(2) a
#t(3) bear

But, if you should do...

tokenize "I am" a bear;

you would get these tokens

#t(0) I am
#t(1) a
#t(2) bear

Since anything in " " is read as one token

Now that I've explained, all that's left is for Kaimetsu to swing by, say "I wouldn't have used those words, but I'm sure it's harmless enough", and then elabricate a bit.