LLM-Stack

Portkey's AI Gateway is the interface between your app and hosted LLMs. It streamlines API requests to OpenAI, Anthropic, Mistral, LLama2, Anyscale, Google Gemini and more with a unified API.

✅ Blazing fast (9.9x faster) with a tiny footprint (~45kb installed)

✅ Load balance across multiple models, providers, and keys

✅ Fallbacks make sure your app... See more

Original creator : Jesse Zhang (GH: emptycrown, Twitter: @thejessezhang), who courteously donated the repo to LlamaIndex!

This is a simple library of all the data loaders / readers / tools / llama-packs that have been created by the community. The goal is to make it extremely easy to connect large language models to a large variety of knowledge... See more

Introducing Prompts: LLM Monitoring

W&B Prompts – LLM Monitoring provides large language model usage monitoring and diagnostics. Start simply, then customize and evolve your monitoring analytics over time.

dstack is an open-source toolkit and orchestration engine for running GPU workloads. It's designed for development, training, and deployment of gen AI models on any cloud.

Supported providers: AWS, GCP, Azure, Lambda, TensorDock, Vast.ai, and DataCrunch.

Latest news ✨

• [2024/01] dstack 0.14.0: OpenAI-compatible endpoints preview (Release)


• [2023/12] dst

... See more

Overview

AIConfig saves prompts, models and model parameters as source control friendly configs. This allows you to iterate on prompts and model parameters separately from your application code .

1. Prompts as configs : a standardized JSON format to store generative AI model settings, prompt inputs/outputs, and flexible metadata.


2. Model-agnostic SDK :

... See more

LanceDB

LanceDB is an open-source vector database for AI that's designed to store, manage, query and retrieve embeddings on large-scale multi-modal data. The core of LanceDB is written in Rust 🦀 and is built on top of Lance, an open-source columnar data format designed for performant ML workloads and fast random access.

Both the database and the... See more

Marker

Marker converts PDF, EPUB, and MOBI to markdown. It's 10x faster than nougat, more accurate on most documents, and has low hallucination risk.

• Support for a range of PDF documents (optimized for books and scientific papers)


• Removes headers/footers/other artifacts


• Converts most equations to latex


• Formats code blocks and tables


• Support for multiple

... See more

Higher performance and lower cost than any single LLM

We invented the first LLM router. By dynamically routing between multiple models, Martian can beat GPT-4 on performance, reduce costs by 20%-97%, and simplify the process of using AI

Boosting annotator efficiency with Large Language Models

In July and August, we released LLM-assisted recipes for data annotation and prompt engineering:

• NER: ner.llm.correct, ner.llm.fetch


• Spancat: spans.llm.correct, spans.llm.fetch


• Textcat: textcat.llm.correct, textcat.llm.fetch


• Prompt engineering and terms: ab.llm.tournament terms.llm.fetch

Prodigy's... See more