GitHub - mit-han-lab/streaming-llm: Efficient Streaming Language Models with Attention Sinks
mit-han-labgithub.comSaved by Darren LI and
GitHub - mit-han-lab/streaming-llm: Efficient Streaming Language Models with Attention Sinks
In streaming settings, StreamingLLM outperforms the sliding window recomputation baseline by up to 22.2x speedup.
mit-han-lab • GitHub - mit-han-lab/streaming-llm: Efficient Streaming Language Models with Attention Sinks
StreamingLLM can enable Llama-2, MPT, Falcon, and Pythia to perform stable and efficient language modeling with up to 4 million tokens and more.