GitHub - sqrkl/lm-evaluation-harness: A framework for few-shot evaluation of language models.
updated 4mo ago
GitHub - sqrkl/lm-evaluation-harness: A framework for few-shot evaluation of language models.
- Large variety of ready-to-use LLM evaluation metrics (all with explanations) powered by
ANY
LLM of your choice, statistical methods, or NLP models that runs
locally on your machine
:- G-Eval
- Summarization
- Answer Relevancy
- Faithfulness
- Contextual Recall
- Contextual Precision
- RAGAS
- Hallucination
- Toxicity
- Bias
- etc.
from GitHub - confident-ai/deepeval: The LLM Evaluation Framework
Nicolay Gerold added
- Large variety of ready-to-use LLM evaluation metrics (all with explanations) powered by
- GitHub - arthur-ai/bench: A tool for evaluating LLMs
from GitHub - arthur-ai/bench: A tool for evaluating LLMs
BA Builder added
promptfoo is a tool for testing and evaluating LLM output quality.
... See more
With promptfoo, you can:
Systematically test prompts & models against predefined test cases
Evaluate quality and catch regressions by comparing LLM outputs side-by-side
Speed up evaluations with caching and concurrency
Score outputs automatically by defining test cases
Use as afrom Testing framework for LLM Part
Nicolay Gerold added