![\"Embeddings](\"https:\/\/i0.wp.com\/www.blopig.com\/blog\/wp-content\/uploads\/2018\/08\/ci-2017-00616a_0005.jpeg?ssl=1\")
<\/a>2D projections (t-SNE) of Mol2vec vectors of amino acids (bold arrows). These vectors were obtained by summing the vectors of the Morgan substructures (small arrows) present in the respective molecules (amino acids in the present example). The directions of the vectors provide a visual representation of similarities. Magnitudes reflect importance, i.e. more meaningful words. [Figure from Ref. 1]<\/p><\/div>Natural Language Processing (NLP)<\/a> algorithms are usually used for analyzing human communication, often in the form of textual information such as scientific papers and Tweets. One aspect, coming up with a representation that clusters words with similar meanings, has been achieved very successfully with the word2vec<\/a> approach. This involves training a shallow, two-layer artificial neural network on a very large body of words and sentences \u2014 the so-called corpus<\/em> \u2014 to generate “embeddings” of the constituent words into a high-dimensional space. By computing the vector from “woman” to “queen”, and adding it to the position of “man” in this high-dimensional space, the answer, “king”, can be found.<\/p>\n A recent publication of one of my former InhibOx-colleagues, Simone Fulle, and her co-workers, Sabrina Jaeger and Samo Turk, shows how we can embed molecular substructures and chemical compounds into a similarly high-dimensional, continuous vectorial representation, which they dubbed “mol2vec<\/a>“.1<\/sup> They also released a Python implementation, available on Samo Turk’s GitHub repository<\/a>.<\/p>\n <\/p>\n The paper described how they assembled a corpus of 19.9 million molecules from ChEMBL<\/a> version 23 and ZINC 15<\/a>, two databases of small molecules, one storing bioactivity data, and the other commercially-available compounds; converted them into canonical SMILES strings using RDKit<\/a> 2017.03.3, and extracted the substructures that contributed to a Morgan fingerprints with radius of one. By using the gensim implementation of word2vec, and exploring both CBOW (Continuous Bag Of Words) and Skip-gram models, as well as various window sizes (CBOW: w=5 & 10; Skip-gram: w=10 & 20) and dimensional embeddings (100D and 300D), the best embedding was found using Skip-gram, w=10, and 300D.<\/p>\n Mol2vec vectors were also combined with ProtVec2<\/sup> vectors to explore the influence of adding information about the protein (when known) to models built using chemical compound features. Such models are sometimes referred to as proteochemometric models<\/em> (PCM).<\/p>\n Using a rigorous four-level cross validation scheme, which they referred to as CV1, CV2, CV3, and CV4, they investigated the influence of knowledge about compounds, target proteins, and both, on the accuracy of their PCM models:<\/p>\n Validation studies were carried out by generating models using Random Forests (RF), Gradient Boosted Machines (GBM), and Deep Neural Networks (DNN), in both regression and classification tasks:<\/p>\n Using mol2vec <\/i>embeddings as compound features, the following models were built:<\/p>\n All models were trained using 20 x 5-fold cross validations, and compared using Wilcoxon signed rank tests. <\/span>Regression tasks were <\/span>evaluated using coefficient of determination <\/span>R<\/i>2<\/sup>cv<\/sub><\/i>, Mean absolute error, MAE; and Mean squared error, MSE. <\/span>Classification tasks were assessed using the <\/span>Area under the Receiver Operating Characteristic curve, AUC; <\/span>Sensitivity, TPR; and <\/span>Specificity, TNR.<\/span><\/p>\n While there were some instances where mol2vec performed better than other features, and scenarios where mol2vec RF models did better than Support Vector Machines (SVM), Na\u00efve Bayes Classifiers (NBC), Convolutional Neural Networks (CNN), and DNNs, overall they concluded that:<\/p>\n Perhaps the greatest advantages of 300D mol2vec over 2048-bit Morgan fingerprints are that they require less memory when training models, and are faster to train. They probably require less training data, as evidenced by the significantly better performance in predicting the aqueous solubility of 1144 compounds in the ESOL set (R<\/i>2<\/sup>cv<\/sub><\/i>-value of 0.86 for mol2vec-GBM, versus 0.66 for Morgan FP-GBM models).<\/p>\n To try mol2vec out, there are some great Jupyter Notebooks \u2014 and documentation \u2014 worth looking at:<\/p>\n\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
References<\/h1>\n