Choosing your metrics, KPIs, etc...
how to measure what matters + philosophical musings on whether the important things can be measured
sari and
Choosing your metrics, KPIs, etc...
How did people respond? Did I strike an emotional nerve? This goes beyond easy, legible metrics like pageviews or social media likes, or even comments, which are at best hazy approximations of external resonance. It’s about looking for signals that something genuinely MATTERED to one or more humans, and elicited a response that’s out of proportion with the average digital interaction. Lots of likes is an okay-ish signal. Lots of comments is a clearer signal. A small handful of comments or private replies from people saying they’ve never felt so seen or understood by a piece of writing – that’s the kind of thing I’m trying to discern and quantify here.
We don’t track time on site on Sublime. We also don’t track passive consumption. We only track people actively building and curating their library (which itself is not a perfect metric - because we don’t want to incentivize collecting. we want to incentivize creating with your knowledge) but is better than time on platform.
If a platform enables
... See moreToo Much Efficiency Makes Everything Worse: Overfitting and the Strong Version of Goodhart's Law
Jascha Sohl-Dicksteinsohl-dickstein.github.io
In my six years at Google, I got to observe this force up close, relentlessly killing features users loved and eroding the last vestiges of creativity and agency from our products. I know this force well, and I hate it, but I do not yet know how to fight it. I call this force the Tyranny of the Marginal User .
Simply put, companies building apps... See more
Simply put, companies building apps... See more
Ivan Vendrov • The Tyranny of the Marginal User
We believe that being data-informed comes down to internalizing a set of values. These are simple, few, yet exceedingly powerful:
- Conviction around a purpose rather than searching for meaning in numbers
- Setting verifiable goals rather than vague aspirations
- Company-wide familiarity with metrics rather than outsourcing to “data people”
- Active
Julie Zhuo • The Data-Informed Manifesto
The concept of efficiency — how much one can accomplish per unit of time (or per dollar, etc.) — requires a quantitative numerator as well as a denominator. It requires a metric. Therefore, it tells us nothing about results we cannot quantify or measure. When we gear our society around efficiency, we produce more and more of the measurable, while... See more
