Luc Cheung

Self-Attention in Detail

Let’s first look at how to calculate self-attention using vectors, then proceed to look at how it’s actually implemented – using matrices.

The first step in calculating self-attention is to create three vectors from each of the encoder’s input vectors (in this case, the embedding of each word). So for each word, we create a... See more

The thing about smart people is that they tend to think that if they think really hard about something, they might figure it out, when the truth is, in strategy (and life in general), there is never one right answer. Strategy requires making choices about a future that is not yet known. I’m one of those people that tends to over-intellectualize... See more

When a precocious yet nonconformist teenager asks why they need to learn calculus, what should you say?

You know they will never use it in adulthood, outside of certain career choices.

You could say, “It’ll help you get into college,” but then they’re left wondering why college cares if you know calculus. And once they’re in college, maybe you could... See more

There are tens of thousands of people sharing insights on a long-tail of topics, but their content is buried in the deep corners of the interwebs, found only by chance, and consumed in fleeting social media feeds that strip context and discourage reflection.

If you want to know what the smartest people have to say about the future of fandom, there... See more