https://d2l.ai/chapter_natural-language-processing-pretraining/word2vec-pretraining.html
What exactly nn.embbeding layer did? it seems just a linear layer. then the skip_gram() function looks different with the paper showed flow. I did not understand this part. I saw some skip_gram implementation and this one is exactly same with the paper showed flow: https://www.kaggle.com/karthur10/skip-gram-implementation-with-pytorch-step-by-step
The nn.embedding is a kind of lookup table, it will return the weights, then the implementation is same as the paper.
There is a typo in cell 2: print(f’Parameter embedding_weight ({embed.weight.shape}, ’
‘dtype={embed.weight.dtype})’)
should be: print(f’Parameter embedding_weight ({embed.weight.shape}, ’
f’dtype={embed.weight.dtype})’)
Currently embed.weight.dtype is being treated as a string and is printed as it is