GitHub - microsoft/LLMLingua: To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
GitHub - mit-han-lab/streaming-llm: Efficient Streaming Language Models with Attention Sinks
mit-han-labgithub.com22365_3_Prompt Engineering_v7 (1)
The content covers prompt engineering for large language models, including techniques, output configurations, and best practices to optimize prompts for various tasks while enhancing model performance and response accuracy.
Link
Meta 刚刚低调发布了一个最小且快速的 LLM大模型训练和推理框架库Meta Lingua。 目的是让更多人来训练 llama。可以 24 小时训练出一个 llama 7B，MMLU达到 48%。在许多下游任务上获得了非常强大的性能，并且与DCLM 基线 1.0的性能相匹配。
国内大模型要笑醒，训练一个国产大模型只需要 10 万人民币，费用计算： $2.50/h（租用一个 H100） × 256个（h100 gpu） × 24H = $15,360。
Meta Lingua主要特点 ：
- 允许用户快速入门，而无需安装和配置大量依赖项。
- 用于研究的最小且快速的 LLM 训练/推理库。
- 使用可修改的 PyTorch 组件来试验架构、损失、数据
- 支持端到端训练、推理和评估... See more
Data-Juicer: A One-Stop Data Processing System for Large Language Models
Data-Juicer is a one-stop data processing system to make data higher-quality, juicier, and more digestible for LLMs. This project is being actively updated and maintained, and we will periodically enhance and add more features and data recipes. We welcome you to join us in pro... See more
Data-Juicer is a one-stop data processing system to make data higher-quality, juicier, and more digestible for LLMs. This project is being actively updated and maintained, and we will periodically enhance and add more features and data recipes. We welcome you to join us in pro... See more
alibaba • GitHub - alibaba/data-juicer: A one-stop data processing system to make data higher-quality, juicier, and more digestible for LLMs! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大语言模型提供更高质量、更丰富、更易”消化“的数据！
StreamingLLM can enable Llama-2, MPT, Falcon, and Pythia to perform stable and efficient language modeling with up to 4 million tokens and more.