GitHub - THUDM/AgentTuning: AgentTuning: Enabling Generalized Agent Abilities for LLMs

AgentBench: Evaluating LLMs as Agents

Our extensive test over 25 LLMs (including APIs and open-sourced models) shows that, while top commercial LLMs present a strong ability of acting as agents in complex environments, there is a significant disparity in performance between them and open-sourced competitors.

AutoGen's design offers multiple advantages: a) it gracefully navigates the strong but imperfect generation and reasoning abilities of these LLMs; b) it leverages human understanding and intelligence, while providing valuable automation through conversations between agents; c) it simplifies and unifies the implementation of complex LLM workflows as... See more

SteerLM leverages a supervised fine-tuning method that empowers you to control responses during inference. It overcomes the limitations of prior alignment techniques, and consists of four key steps:

1. Train an attribute prediction model on human-annotated datasets to evaluate response quality on any number of attributes like helpfulness, humor, and cr

... See more

a couple of the top of my head:

• LLM in the loop with preference optimization


• synthetic data generation


• cross modality "distillation" / dictionary remapping


• constrained decoding