Building Effective AI Agents

Implementing guardrails where one model instance processes user queries while another screens them for inappropriate content or requests. This tends to perform better than having the same LLM call handle both guardrails and the core response.

Barry Zhang • Building Effective AI Agents

KKenny

aaaah interesting

During execution, it's crucial for the agents to gain “ground truth” from the environment at each step (such as tool call results or code execution) to assess its progress.

Barry Zhang • Building Effective AI Agents

For many applications, however, optimizing single LLM calls with retrieval and in-context examples is usually enough.

anthropic.com • Building Effective AI Agents

Start with simple prompts, optimize them with comprehensive evaluation, and add multi-step agentic systems only when simpler solutions fall short.

Barry Zhang • Building Effective AI Agents

KKenny

need to make propmt much simpler and have a better iteration loop

Routing classifies an input and directs it to a specialized followup task. This workflow allows for separation of concerns, and building more specialized prompts. Without this workflow, optimizing for one kind of input can hurt performance on other inputs.

Barry Zhang • Building Effective AI Agents

KKenny

routing for handling categories?

Parallelization is effective when the divided subtasks can be parallelized for speed, or when multiple perspectives or attempts are needed for higher confidence results.

Barry Zhang • Building Effective AI Agents

KKenny

“multiple perspecitves are needed for higher confidence” kinda od though… too expensive. though this might be what we doing when we need to check for multiple things like breathiness, confidence etc

Prompt chaining decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one. You can add programmatic checks (see "gate” in the diagram below) on any intermediate steps to ensure that the process is still on track.

Barry Zhang • Building Effective AI Agents

KKenny

use prompt chaining for feedback system?

This workflow is particularly effective when we have clear evaluation criteria, and when iterative refinement provides measurable value. The two signs of good fit are, first, that LLM responses can be demonstrably improved when a human articulates their feedback; and second, that the LLM can provide such feedback. This is analogous to the iterative... See more

Barry Zhang • Building Effective AI Agents

KKenny

hmmm this might be interesting if done right