Domain specific LLM fine tuning always confused me. After several iterations I along with @MojoAnalytics published our first blog on best practices and insights on fine tuning LLMs using @huggingface and managing experiments using @weights_biases 👉
#LLM
https://t.co/jSEodhi7Jg
I finally understand how GPT generates text.
Really helps to code it from scratch in Python.
There are 5 components:
• token embeddings
• positional embeddings
• transformer blocks
• layer normalization
• output head
It sounds complex, but grokking GPT is simple.
Token embeddings turn input text into meaningful vectors that capture semantic mea... See more
AI + Blockchain research. How gpt works from scratch
The 2 biggest rumors I've seen for iOS 18 are both hilarious:
Being able to put your icons wherever you want on your homescreen
AI-generated emojis based on what you're texting