GitHub - microsoft/LLMLingua: To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
updated 8mo ago
GitHub - microsoft/LLMLingua: To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
- Boosting annotator efficiency with Large Language Models
In July and August, we released LLM-assisted recipes for data annotation and prompt engineering:- NER: ner.llm.correct, ner.llm.fetch
- Spancat: spans.llm.correct, spans.llm.fetch
- Textcat: textcat.llm.correct, textcat.llm.fetch
- Prompt engineering and terms: ab.llm.tournament terms.llm.fetch
Prodigy's... See morefrom Prodigy in 2023: LLMs, task routers, QA and plugins · Explosion
Nicolay Gerold added
Easy data labelling for NLP tasks.
- LLMs struggle when handling tasks which require extensive knowledge. This limitation highlights the need to supplement LLMs with non-parametric knowledge. This paper Prompting Large Language Models with Knowledge Graphs for Question Answering Involving Long-tail Facts analyze the effects of different types of non-parametric knowledge, such as textu... See more
from Shortwave — rajhesh.panchanadhan@gmail.com [Gmail alternative]
Nicolay Gerold added
- DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference
Table of Contents- Introduction
- Key LLM Serving Techniques
- Dynamic SplitFuse: A Novel Prompt and Generation Composition Strategy
- Performance Evaluation
- DeepSpeed-FastGen: Implementation and Usage
- Try out DeepSpeed-FastGen
- Acknowledgements
1. Introduction
Large langu... See morefrom DeepSpeed-FastGen by microsoft
Nicolay Gerold added