Stop Words
Learn how to use stop words with the help of examples.
We'll cover the following...
We have developed several helper methods for extracting words, counting their frequency, etc.
A common issue arising when we perform text analysis or mining is the frequency of common words within natural languages.
Consider the following sample text:
The group of otherwise ordinary but eccentric teenagers ran towards the barn under destress, with nothing but a crumpled newspaper for covering, before the storm hit fully.
While we can get through this text relatively quickly, this English sentence contains many words or phrases that help glue the individual words together and help set the pacing. Using our commonWords
helper method on this text produces the following list:
array:24 [0 => "the"1 => "but"2 => "group"3 => "of"4 => "otherwise"5 => "ordinary"6 => "eccentric"7 => "teenagers"8 => "ran"9 => "towards"10 => "barn"11 => "under"12 => "destress"13 => "with"14 => "nothing"15 => "a"16 => "crumpled"17 => "newspaper"18 => "for"19 => "covering"20 => "before"21 => "storm"22 => "hit"23 => "fully"]
In the list of words produced, prepositions, coordinating ...