Long-Context Retrieval Models with Monarch Mixer
hazyresearch.stanford.edu
Long-Context Retrieval Models with Monarch Mixer

The limited context window is one of the biggest limitations of LLMs today. The reasoning capabilities are great but the relevant knowledge it can reason with is very limited.
Question: nobody understands tokens. Let's talk about a regular page. How many pages of context can it uses?
