A lot of people are talking about how long-context LLMs are great as an "ask my life" app where, you deposit your journal entires, photos, even company docs -- and then ask q's. So, I've been doing this for 6-mo... It's the founder coach I never knew I needed -- it's saved me + my company multiple times now and has 100X'd my journaling. Will inc... See more

“ The agent runs in a loop of Thought, Action, Observation, Thought, ... The Thought and Action (with the Action Input to the action) are the parts which are generated by an LLM The Observation is generated by using a tool (for example the print outputs of Python or the text result of a Google search)”

Really neat to see the parallels to OODA loops,... See more

In three words: deep learning worked.

In 15 words: deep learning worked, got predictably better with scale, and we dedicated increasing resources to it.

That’s really it; humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data). To a shocking deg... See more

Some thoughts: I think this is 1 of the most important papers in a while because its the first open model that is genuinely at the frontier and not just riding on the goodwill of being open. The paper is really really simple as you can probably tell from the thread because the approach is really really simple. It really is exactly what OpenAI is... See more

I've been asked by few first year PhD about how to start LLM research on X, say long context modeling. My number one suggestion -- though it seems a bit of unconventional -- is *not* to read any papers related to long-context, but to talk to the model - Talk to the model about a text book, course slides, financial reports, novels, nonfictions, any... See more