GitHub - google/maxtext: A simple, performant and scalable Jax LLM!

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax. EasyLM can scale up LLM training to hundreds of TPU/GPU accelerators by leveraging JAX's pjit functionality.

Building on top of Hugginface's transformers and datasets, this repo provides an easy to use and easy... See more

GPT4All: An ecosystem of open-source on-edge large language models.

Important

GPT4All v2.5.0 and newer only supports models in GGUF format (.gguf). Models used with a previous version of GPT4All (.bin extension) will no longer work.

GPT4All is an ecosystem to run powerful and customized large language models that work locally on consumer grade CPUs a... See more

StreamingLLM can enable Llama-2, MPT, Falcon, and Pythia to perform stable and efficient language modeling with up to 4 million tokens and more.

LLM-PowerHouse: A Curated Guide for Large Language Models with Custom Training and Inferencing

Welcome to LLM-PowerHouse, your ultimate resource for unleashing the full potential of Large Language Models (LLMs) with custom training and inferencing. This GitHub repository is a comprehensive and curated guide designed to empower developers, researche... See more

DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference

Table of Contents

1. Introduction


2. Key LLM Serving Techniques


3. Dynamic SplitFuse: A Novel Prompt and Generation Composition Strategy


4. Performance Evaluation


5. DeepSpeed-FastGen: Implementation and Usage


6. Try out DeepSpeed-FastGen


7. Acknowledgements

1. Introduction

Large langu... See more

Meta 刚刚低调发布了一个最小且快速的 LLM大模型训练和推理框架库Meta Lingua。 目的是让更多人来训练 llama。可以 24 小时训练出一个 llama 7B，MMLU达到 48%。在许多下游任务上获得了非常强大的性能，并且与DCLM 基线 1.0的性能相匹配。 国内大模型要笑醒，训练一个国产大模型只需要 10 万人民币，费用计算： $2.50/h（租用一个 H100） × 256个（h100 gpu） × 24H = $15,360。 Meta Lingua主要特点 ： - 允许用户快速入门，而无需安装和配置大量依赖项。 - 用于研究的最小且快速的 LLM 训练/推理库。 - 使用可修改的 PyTorch 组件来试验架构、损失、数据 - 支持端到端训练、推理和评估... See more