LLM-Stack

by Nicolay Gerold · updated 4mo ago

dstack is an open-source toolkit and orchestration engine for running GPU workloads. It's designed for development, training, and deployment of gen AI models on any cloud.

Supported providers: AWS, GCP, Azure, Lambda, TensorDock, Vast.ai, and DataCrunch.

Latest news ✨

[2024/01] dstack 0.14.0: OpenAI-compatible endpoints preview (Release)

[2023/12] dst
... See more
from GitHub - dstackai/dstack: dstack is an open-source toolkit for running GPU workloads on any cloud. It works seamlessly with any cloud GPU providers. Discord: https://discord.gg/u8SmfwPpMd by dstackai
Nicolay Gerold added 8mo ago
They will start to support autoscaling in March. You can configure multiple clouds and they deploy to the cheapest one.
LanceDB

LanceDB is an open-source vector database for AI that's designed to store, manage, query and retrieve embeddings on large-scale multi-modal data. The core of LanceDB is written in Rust 🦀 and is built on top of Lance, an open-source columnar data format designed for performant ML workloads and fast random access.

Both the database and the un... See more
from LanceDB - LanceDB
Nicolay Gerold added 8mo ago
Boosting annotator efficiency with Large Language Models

In July and August, we released LLM-assisted recipes for data annotation and prompt engineering:

NER: ner.llm.correct, ner.llm.fetch

Spancat: spans.llm.correct, spans.llm.fetch

Textcat: textcat.llm.correct, textcat.llm.fetch

Prompt engineering and terms: ab.llm.tournament terms.llm.fetch

Prodigy's

from Prodigy in 2023: LLMs, task routers, QA and plugins · Explosion

Easy data labelling for NLP tasks.

Overview

AIConfig saves prompts, models and model parameters as source control friendly configs. This allows you to iterate on prompts and model parameters separately from your application code .

Prompts as configs : a standardized JSON format to store generative AI model settings, prompt inputs/outputs, and flexible metadata.

Model-agnostic SDK :

from GitHub - lastmile-ai/aiconfig: aiconfig -- config-driven, source control friendly AI application development by lastmile-ai

Nicolay Gerold added 10mo ago

Original creator : Jesse Zhang (GH: emptycrown, Twitter: @thejessezhang), who courteously donated the repo to LlamaIndex!

This is a simple library of all the data loaders / readers / tools / llama-packs that have been created by the community. The goal is to make it extremely easy to connect large language models to a large variety of knowledge sour... See more

from GitHub - run-llama/llama-hub: A library of data loaders for LLMs made by the community -- to be used with LlamaIndex and/or LangChain

Nicolay Gerold added 10mo ago

🐂 What is Oxen?

Oxen is a lightning fast data version control system for structured and unstructured machine learning datasets. We aim to make versioning datasets as easy as versioning code.

The interface mirrors git, but shines in many areas that git or git-lfs fall short. Oxen is built from the ground up for data, and is optimized to handle large... See more

from GitHub - Oxen-AI/oxen-release: Official repository for docs and releases of the Oxen CLI by Oxen-AI

Nicolay Gerold added 10mo ago

Data Structuring & Storage

Large variety of ready-to-use LLM evaluation metrics (all with explanations) powered by

ANY

LLM of your choice, statistical methods, or NLP models that runs

locally on your machine

:

G-Eval

Summarization

Answer Relevancy

Faithfulness

Contextual Recall

Contextual Precision

RAGAS

Hallucination

Toxicity

Bias

etc.

LLM-Stack

from GitHub - dstackai/dstack: dstack is an open-source toolkit for running GPU workloads on any cloud. It works seamlessly with any cloud GPU providers. Discord: https://discord.gg/u8SmfwPpMd by dstackai

from LanceDB - LanceDB

from Prodigy in 2023: LLMs, task routers, QA and plugins · Explosion

from GitHub - lastmile-ai/aiconfig: aiconfig -- config-driven, source control friendly AI application development by lastmile-ai

from GitHub - run-llama/llama-hub: A library of data loaders for LLMs made by the community -- to be used with LlamaIndex and/or LangChain

from GitHub - Oxen-AI/oxen-release: Official repository for docs and releases of the Oxen CLI by Oxen-AI

from GitHub - confident-ai/deepeval: The LLM Evaluation Framework

from GitHub - skypilot-org/skypilot: SkyPilot: Run LLMs, AI, and Batch jobs on any cloud. Get maximum savings, highest GPU availability, and managed execution—all with a simple interface. by skypilot-org

from GitHub - VikParuchuri/marker: Convert PDF to markdown quickly with high accuracy by VikParuchuri