AI & I
Ideas about AI, LLMs, and Machines
AI & I
Marty Cagan's wisdom still shines bright for AI PMs. It all boils down to mastering four critical dimensions.
Miss one, and... your AI product might just fall flat.
Miss one, and... your AI product might just fall flat.
- Valuable (The Ultimate Win): Does it solve a genuine, burning problem for your users?
- Feasible: Can your team actually build this within reasonable constraints?
- Usable: Can real people
Full Roadmap: Become an AI PM in 2025 | Ankit Shukla, HelloPM Founder
Here’s your blueprint to the essential layers:
- Models: The brain of the operation. This includes foundational models, fine-tuned versions, or custom builds (think Grok, OpenAI).
- Data Layer: The fuel that feeds the brain. This means robust collection, meticulous cleaning, precise labeling, secure storage, and constant, real-time updating.
- Infrastru
Full Roadmap: Become an AI PM in 2025 | Ankit Shukla, HelloPM Founder
Building Blocks for AI Products
- User feedback: Did the summary help? Were the questions relevant?
- Automated checks: Does the output meet quality standards? Is it free of hallucinations?
- Comparison: How does it perform against a human expert?
Full Roadmap: Become an AI PM in 2025 | Ankit Shukla, HelloPM Founder
Eval Loop for AI
Eval Cheat Sheet
Introductory Terms in AI PMing
Quantization
A method to shrink the size of a model and make it run faster. It works by converting the model’s internal numbers from high precision (like 32-bit floats) to smaller ones (like 8-bit integers). This helps reduce memory usage and speed up inference with only a small drop in accuracy.
A method to shrink the size of a model and make it run faster. It works by converting the model’s internal numbers from high precision (like 32-bit floats) to smaller ones (like 8-bit integers). This helps reduce memory usage and speed up inference with only a small drop in accuracy.
AI Product Manager Glossary
Time to First Token (TTFT)
The time it takes from sending a request to when the first token of the response appears. Even if the full answer takes longer, a fast TTFT makes the system feel more responsive to the user.
The time it takes from sending a request to when the first token of the response appears. Even if the full answer takes longer, a fast TTFT makes the system feel more responsive to the user.
AI Product Manager Glossary
Inference
The process of using a trained model to make predictions on new data. In the case of large language models, this means generating a response based on the input prompt. It’s what happens when you “ask” the model something.
The process of using a trained model to make predictions on new data. In the case of large language models, this means generating a response based on the input prompt. It’s what happens when you “ask” the model something.
AI Product Manager Glossary
Failure Modes
Coherent and non-overlapping categories of errors that emerge from analyzing LLM traces (e.g., hallucination, incorrect format, or missed instruction). Each binary failure type is easy to recognize and forms the basis for targeted metrics.
Coherent and non-overlapping categories of errors that emerge from analyzing LLM traces (e.g., hallucination, incorrect format, or missed instruction). Each binary failure type is easy to recognize and forms the basis for targeted metrics.