Member-only story
When Should You Remove Stop Words?
Welcome to the last post of the year — data science interview challenge!
Here it goes:
When should you remove stop words?
Here are some tips for readers’ reference:
Stop words are more important if you’re counting words and don’t care about their location. So you should consider removing them in case of a “ bag-of-words “ type of model where you care about frequency or co-occurrence frequency, or don’t care about the order of the words.
On the other hand, in general you shouldn’t remove them if you are using transformer type of models if you care about the relationship among the words.
Watch how Dr. Rachael Tatman from RASA explains this:
Happy practicing!
And, happy early holidays! I will be sharing job postings info for MLEs and MLDS roles in the new year with @DataSciNews on twitter. Please tune in to learn more about the opportunities!
And… a letter to YOU😊:
Dear readers!
As the year comes to a close, we’d like to take a moment to express our heartfelt gratitude for your support and loyalty. Your readership and engagement have been instrumental…
