dm.cs.tu-dortmund.de/mlbits/neural-nlp-beyond-word2vec/
Beyond Word2Vec – Lecture Notes
w_i) = 1 \\ \end{align*}\]
Loss function for CBOW:
\[\begin{align*} L_{\text{CBOW}} =\, & -\tfrac {1}{|S|} \sum \nolimits _{i\in S} \log P\left(w_i\middle\vert \sum \nolimits _{j=-c,\ldots ,-1,+1,\ldots [...] for Skip-Gram:
\[\begin{align*} L_{\text{skip-gram}} =\, & -\tfrac {1}{|S|} \sum \nolimits _{i\in S} \sum \nolimits _{j=-c,\ldots ,-1,+1,\ldots ,c} \log P(w_{i+j}\vert w_i) \\ \end{align*}\]
where \(|S|\) [...] skip-gram objective:
\[\begin{align*} L_{\text{skip-gram}} =\, & -\tfrac {1}{|S|} \sum \nolimits _{i\in S} \sum \nolimits _{j=-c,\ldots ,-1,+1,\ldots ,c} \log P(w_{i+j}\vert w_i) \end{align*}\]
becomes
\[\beg …
