LLMs

Take a look at our official page for user documentation and examples: langtest.org

Key Features

• Generate and execute more than 50 distinct types of tests only with 1 line of code


• Test all aspects of model quality: robustness, bias, representation, fairness and accuracy.​


• Automatically augment training data based on test results (for select models)​


• Sup

... See more

LLMTuner

LLMTuner: Fine-Tune Llama, Whisper, and other LLMs with best practices like LoRA, QLoRA, through a sleek, scikit-learn-inspired interface.

• Query the RAG anyway and let the LLM itself chose whether to use the the RAG context or its built in knowledge


• Query the RAG but only provide the result to the LLM if it meets some level of relevancy (ie embedding distance) to the question


• Run the LLM both on it's own and with the RAG response, use a heuristic (or another LLM) to pick the best answer

... See more

⚡ LitGPT

Pretrain, finetune, evaluate, and deploy 20+ LLMs on your own data

Uses the latest state-of-the-art techniques:

✅ flash attention ✅ fp4/8/16/32 ✅ LoRA, QLoRA, Adapter (v1, v2) ✅ FSDP ✅ 1-1000+ GPUs/TPUs

Lightning AI • Models • Quick start • Inference • Finetune • Pretrain • Deploy • Features • Training recipes (YAML)

Finetune, pretrain and... See more

📦 Service Deployment - Ray Serve (https://lnkd.in/eAV-Y6RN)

🧰 Data Transformation - Ray Data (https://lnkd.in/e7wYmenc)

🔌 LLM Integration - AIConfig (https://lnkd.in/esvH5NQa)

🗄 Vector Database - Weaviate (https://weaviate.io/)

📚 Supervised LLM Fine-Tuning - HuggingFace TLR (https://lnkd.in/e8_QYF-P)

📈 LLM Observability - Weights & Biases Traces (https... See more

Today, we’re releasing the Assistants API, our first step towards helping developers build agent-like experiences within their own applications. An assistant is a purpose-built AI that has specific instructions, leverages extra knowledge, and can call models and tools to perform tasks. The new Assistants API provides new capabilities such as Code... See more

We went to OpenAI's office in San Francisco yesterday to ask them all the questions we had on Quivr (YC W24), here is what we learned:

1. Their office is super nice & you can eat damn good croissant in SF!

2. We can expect GPT 3.5 & 4 price to keep going down

3. A lot of people are using the Assistants API to build their use cases

4. It costs 2M$ to... See more

Principles for growable tools

There are three critical pieces to building a tool that can grow around its users over time.

1. Design around play . Sometimes I call this design around experimentation . Using the tool for day-to-day work should involve playing and experimenting with what’s possible with the tool. Whether that’s writing small programs to

... See more

The context size of the input is too small for when you want to analyse CSV's with 1000's of rows and embedding doesn't really work because it loses context.