Search⌘ K
AI Features

Splitting Strings Related Tips

Explore safe and effective ways to split strings in Python and understand the limitations of the str.split() method. Learn about tokenization concepts and how to use NLP tools like nltk to process natural language strings accurately.

str.split() by white spaces

The str.split() method can be called in four ways, though two tend to cause problems. Those are:

  • Without the separator
  • With a separator

When the method is called with a separator, the method removes the separator string as a whole (not the individual separator characters) from the split string and returns a list of leftovers:

Python 3.8
print('Why, eh, do you, eh, want, eh, it?'.split(', eh, '))

If the split string contains two instances of the separator ...