A code-walkthrough on how to code a transformer from scratch
Very nice post, thanks!
Just a question:
```
class SimpleTransformer(nn.Module):
...
embedding += self.pos_encoder(embedding) # This line seems to add an extra embedding, right?
Great catch. Yes, that's right. I've updated the post and the GitHub code with the fix. Appreciate you bringing this up.
Very nice post, thanks!
Just a question:
```
class SimpleTransformer(nn.Module):
...
embedding += self.pos_encoder(embedding) # This line seems to add an extra embedding, right?
...
```
Great catch. Yes, that's right. I've updated the post and the GitHub code with the fix. Appreciate you bringing this up.