Testing framework for LLM Part

Dynamically route every prompt to the best LLM. Highest performance, lowest costs, incredibly easy to use.

There are over 250,000 LLMs today. Some are good at coding. Some are good at holding conversations. Some are up to 300x cheaper than others. You could hire an ML engineering team to test every single one — or you can switch to the best one fo

... See more

🎄 lakera.ai

An Overview of Lakera Guard — Bringing Enterprise-Grade Security to LLMs with One Line of Code

At Lakera, we supercharge AI developers by enabling them to swiftly identify and eliminate their AI applications’ security threats so that they can focus on building the most exciting applications securely.

Businesses around the world are in

... See more

🌳 Galileo LLM Studio

Algorithm-powered LLMOps Platform

Find the best prompt, inspect data errors while fine-tuning, monitor LLM outputs in real-time. All in one powerful, collaborative platform.

Building production-ready LLM-powered applications is currently very difficult. It involves countless iterations of prompt engineering, parameter tuning, and architectures.

Agenta provides you with the tools to quickly do prompt engineering and 🧪 experiment , ⚖️ evaluate , and 🚀 deploy your LLM apps. All without imposing any restrictions on your

... See more

NeMo Guardrails enables developers to set up three kinds of boundaries:

• Topical guardrails prevent apps from veering off into undesired areas. For example, they keep customer service assistants from answering questions about the weather.


• Safety guardrails ensure apps respond with accurate, appropriate information. They can filter out unwanted langu

... See more

promptfoo is a tool for testing and evaluating LLM output quality.

With promptfoo, you can:

Systematically test prompts & models against predefined test cases

Evaluate quality and catch regressions by comparing LLM outputs side-by-side

Speed up evaluations with caching and concurrency

Score outputs automatically by defining test cases

Use as a

... See more

Welcome to prompttools created by Hegel AI! This repo offers a set of open-source, self-hostable tools for experimenting with, testing, and evaluating LLMs, vector databases, and prompts. The core idea is to enable developers to evaluate using familiar interfaces like code, notebooks, and a local playground.

In just a few lines of codes, you can t

... See more

baserun.ai💪💪💪

Testing & Observability Platform for LLM Apps

From prompt playground to end-to-end tests, baserun helps you ship your LLM apps with confidence and speed.

DeepEval — It’s a tool for easy and efficient LLM testing. Deepeval aims to make writing tests for LLM applications (such as RAG) as easy as writing Python unit tests.