Choosing your metrics, KPIs, etc...
how to measure what matters + philosophical musings on whether the important things can be measured
sari and
Choosing your metrics, KPIs, etc...
One of the most bizarre contradictions in modern software is that the people building these engagement systems don’t like them either!
Ask anyone who works on onboarding popups, feature tours, lifecycle messaging, or in-app announcements how they feel when an app interrupts them mid-flow to announce something they didn’t ask for. The answer is... See more
Ask anyone who works on onboarding popups, feature tours, lifecycle messaging, or in-app announcements how they feel when an app interrupts them mid-flow to announce something they didn’t ask for. The answer is... See more
Article
tensions of product decisions/metrics
How did people respond? Did I strike an emotional nerve? This goes beyond easy, legible metrics like pageviews or social media likes, or even comments, which are at best hazy approximations of external resonance. It’s about looking for signals that something genuinely MATTERED to one or more humans, and elicited a response that’s out of proportion with the average digital interaction. Lots of likes is an okay-ish signal. Lots of comments is a clearer signal. A small handful of comments or private replies from people saying they’ve never felt so seen or understood by a piece of writing – that’s the kind of thing I’m trying to discern and quantify here.
Which leads on to my next point. See, not only do numbers not capture what matters (love, happiness, awe, spirituality, creativity are by definition unquantifiable beyond mere surface appearances) they are also extremely open to manipulation.
thomasjbevan.substack.com • The Tyranny of Numbers - By Thomas J Bevan - The Commonplace
We believe that being data-informed comes down to internalizing a set of values. These are simple, few, yet exceedingly powerful:
- Conviction around a purpose rather than searching for meaning in numbers
- Setting verifiable goals rather than vague aspirations
- Company-wide familiarity with metrics rather than outsourcing to “data people”
- Active
Julie Zhuo • The Data-Informed Manifesto
We don’t track time on site on Sublime. We also don’t track passive consumption. We only track people actively building and curating their library (which itself is not a perfect metric - because we don’t want to incentivize collecting. we want to incentivize creating with your knowledge) but is better than time on platform.
If a platform enables
... See moreThere are things that can be measured. There are things that are worth measuring. But what can be measured is not always what is worth measuring; what gets measured may have no relationship to what we really want to know. The costs of measuring may be greater than the benefits. The things that get measured may draw effort away from the things we... See more