Sublime
An inspiration engine for ideas
GPT-4.1 (and -mini and -nano) are now available in the API!
these models are great at coding, instruction following, and long context (1 million tokens).
benchmarks are strong, but we focused on real-world utility, and developers seem very happy.
GPT-4.1 family i... See more
Sam Altmanx.com
📽️ New 4 hour (lol) video lecture on YouTube:
"Let’s reproduce GPT-2 (124M)"
https://t.co/QTUdu8b0qh
The video ended up so long because it is... comprehensive: we start with empty file and end up with a GPT-2 (124M) model:
- first we build the GPT-2 network
- then we optimize it to train very fast
- then we set up the training run optimization and hyperparameters by referencing GPT-2 and GPT-3 papers
- then we bring up model evaluation, and
- then cross our fingers and go to sleep.
In the morning we look through the results and enjoy amusing model generations. Our "overnight" run even gets very close to the GPT-3 (124M) model. This video builds on the Zero To Hero series and at times references previous videos. You could also see this video as building my nanoGPT repo, which by the end is about 90% similar.
Github. The associated GitHub repo contains the full commit history so you can step through all of the code changes in the video, step by step.
https://t.co/BOzkxQ8at2
Chapters.
On... See more
The upcoming Llama-3-400B+ will mark the watershed moment that the community gains open-weight access to a GPT-4-class model. It will change the calculus for many research efforts and grassroot startups. I pulled the numbers on Claude 3 Opus, GPT-4-2024-04-09, and Gemini.
Llama-3-400B is still training and will hopefull... See more
GPT-4.5 is ready!
good news: it is the first model that feels like talking to a thoughtful person to me. i have had several moments where i've sat back in my chair and been astonished at getting actually good advice from an AI.
bad news: it is a giant, expensive model. we really wanted to la... See more
Sam Altmanx.comI'm excited to announce the release of GPT4All, a 7B param language model finetuned from a curated set of 400k GPT-Turbo-3.5 assistant-style generation.
We releaseđź’°800k data samplesđź’° for anyone to build upon and a model you can run on your laptop!
Real-time Sampling on M1 Mac https://t.co/HgEgnlwYV8
Andriy Mulyarx.com
GPT 4.5 + interactive comparison :)
Today marks the release of GPT4.5 by OpenAI. I've been looking forward to this for ~2 years, ever since GPT4 was released, because this release offers a qualitative measurement of the slope of improvement you get out of scaling pretraining compute (i.e. simply training a bigger model)... See more
Andrej Karpathyx.comTL;DR: we are excited to release a powerful new open-weight language model with reasoning in the coming months, and we want to talk to devs about how to make it maximally useful: https://t.co/nZ5JQ19CN6
we... See more
Sam Altmanx.com
OpenAI just launched GPT-4.
It leaves ChatGPT in the dust.
Here's everything you need to know: https://t.co/Oc6TrygvNO