Member-only story
What Are The Main Problems With Word Embeddings Like GloVe or Word2vec?
There are a lot of deep explanations on word embeddings, GloVe or w2v models elsewhere so here I’d like to share tips on what you can say during an interview setting.
What are the main problems with word embeddings like GloVe or Word2vec?
Here are some example answers for readers’ reference:
One of the main limitations of word embeddings is that (word vector space models in general) is that words with multiple meanings are conflated into a single representation (a single vector in the semantic space). In other words, polysemy and homonymy are not handled properly. For example, in the sentence “The club I tried yesterday was great!”, it is not clear if the term club is related to the word sense of a club sandwich, baseball club, clubhouse, golf club, or any other sense that club might have.
The necessity to accommodate multiple meanings per word in different vectors (multi-sense embeddings) is the motivation for later development in NLP including ELMO, LSTM, BERT and so on.
Watch explanation by Dr. Jacob Devlin from Google AI Language:
Happy practicing!
