why not use tokenize2?
tokenize2 delims,str; - divides str into tokens, using 'delims' as additional delimiters
example:
tokenize2 $,Bob$went to the$market;
that will return a token for each one
#t(0) = Bob
#t(1) = went to the
#t(2) = market
Why add tokenize3 that does the samething as tokenize2? Delims, as indicated from the commands.rtf is the delimiter you want the command to use to seperate each word in the string. If you make delims blank then yes, each space in str will be the cutoff point each time it occurs.