AI Engineering: Building Applications with Foundation Models

However, translating is a much simpler task than planning and can be done by weaker models with a lower risk of hallucination.

only after this plan is validated

Without oversight, an agent can run those steps for hours, wasting time and money on API calls, before you realize that it’s not going anywhere.

If the model’s accuracy is 95% per step, over 10 steps, the accuracy will drop to 60%, and over 100 steps, the accuracy will be only 0.6%.

“Given the following conversation, rewrite the last user input to reflect what the user is actually asking”.

it’s important to be aware of how they work under the hood to avoid unnecessary costs and headaches.

CoT is to add “think step by step” or “explain your decision”

it’s common to use a weaker model for intent classification and a stronger model to generate user responses.

you don’t want preambles.