Nicolay Gerold

@nicolaygerold

All cardsCollections

Analyze user queries and feedback to identify topic clusters, capabilities, and areas of user dissatisfaction. This will help you prioritize improvements.

Why should we do this? Let me give you an example. I once worked with a company that provided a technical documentation search system. By clustering user queries, we identified two main issues:

Systematically Improving Your RAG - jxnl.co

RAG

Pulumi Examples

This repository contains examples of using Pulumi to build and deploy cloud applications and infrastructure across major programming languages.

Each example has a two-part prefix, <cloud>-<language> , to indicate which <cloud> and <language> it pertains to. For example, <cloud> could be:

aws for Amazon

pulumi/examples: Infrastructure, containers, and serverless ... - GitHub

Cloud Engineering

Getting Started

Keyv is a simple key-value storage system that supports multiple backends. It's designed to be a simple and consistent way to work with key-value stores.

To learn how to use Keyv, check out the keyv README. To learn how to use a specific storage adapter, check out the README for that adapter under Storage Adapters.

jaredwray • GitHub - jaredwray/keyv: Simple key-value storage with support for multiple backends

Data Storage

Each log message in the system should be unique.

If I query for a log in a specfic service, I will be confused to see the exact same logs at different flows inside the service.

More than that, I’ll just have to start debugging for the issue, since the logs are now offically useless.

One way to keep the logs unique is to denote the service name and... See more

Logging practices I follow

Software Engineering

MLEs are happy to delegate experiment tracking and execution work to ML experiment execution frameworks, such as Weights & Biases 3 , but prefer to choose subsequent experiments themselves. To be able to make informed choices of subsequent experiments to run, MLEs must maintain awareness of what they have tried and what they haven’t (Lg2 calls it... See more

Shreya Shankar • "We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learning.

learnings from one experiment into the next, like a guided search to find the best idea (Lg2, Sm4,

Lg5). Lg5 described their ideological shift from random search to guided search:

Previously, I tried to do a lot of parallelization. If I focus on one idea, a week at a time,

then it boosts my productivity a lot more.

By following a guided search, engineers are, essentially, significantly pruning a large subset of

experiment ideas without executing them. While it may seem like there are unlimited computational

resources, the search space is much larger, and developer time and energy is limited. At the end of

the day, experiments are human-validated and deployed. Mature ML engineers know their personal

tradeoff between parallelizing disjoint experiment ideas and pipelining ideas that build on top of

each other, ultimately yielding successful deployments

Some tasks are easier to improve via system design. While LLMs appear to follow remarkable scaling laws that predictably yield better results with more compute, in many applications, scaling offers lower returns-vs-cost than building a compound system. For example, suppose that the current best LLM can solve coding contest problems 30% of the

Matei Zaharia, Omar Khattab, Lingjiao Chen, et al. • The Shift From Models to Compound AI Systems

LLMs

Equally, you could use ChatGPT with a code interpreter, some plugins and a text file full of saved prompts - but it might be better for those tasks to be unbundled into tools. This year there’s been a wave of hundreds and perhaps thousands of first drafts of this - the so-called ‘thin ChatGPT wrappers.’ The prompt is an API call and ‘prompt... See more

Benedict Evans • Unbundling AI

If you’re just an API call with a GUI, you probably don’t have much of a moat. But on the other hand, everyone in tech today is wrapping something else at some level. Snap uses Google Cloud for storage, but we don’t call it a ‘thin GCP Wrapper’. A legal software company might use GCP translation and AWS sentiment analysis just as it used AWS data storage, but it used them to build something that law firms could buy, with a lot of other product around that, and a sales force and understanding of what lawyers needed, and it didn’t worry at all about competition from AWS. And then, as part of that product, it might built its own models with its own data doing things that the hyperscalers did not.

wrappers do have their place too. A wrapper has the following benefits:

Create a more focused and guided experience for the user, instead of an open-ended chat.

Less trial and error for the user to figure out how to prompt to get a specific result.

Custom integrations make it easy to interface with third-party services, for example, displaying book

Helghardt Avenant • Building AI Wrappers

AI wrapper is a simple application that sits between the AI and the user. The wrapper fulfills a specific function to augment the AI’s ability, for example, browse online, summarize a PDF, send an email, etc.

Shortwave — rajhesh.panchanadhan@gmail.com [Gmail alternative]

LLMs