A comparison of word2vec models for Identifying similar words using Deep Learning Approaches^

Abstract

In text analysis, frequency-based techniques like Count Vectorizer, Term Frequency-Inverse document frequency are used for term weighting. These approaches are based on the frequency of the words in the document. They do not consider the semantics of the word for weighting.  So, word embedding is used in this work to overcome the limitations of frequency-based approaches. Word embedding produces the vector representation based on the similarity between the words in the sentence. Deep Learning performs better than traditional machine learning algorithms because of its ability to perform automatic feature extraction from raw data and its scalability. There are two word2vec models namely continuous bag-of-words (CBOW) and Skip-gram model. Both these techniques are shallow neural networks which map words to the target variable which is also a word. They learn weights which act as word vector representations. From the experiments conducted on the Gutenberg book review dataset, it is found that skip gram model performs better than CBOW model.