Luc Cheung

Self-Attention in Detail

Let’s first look at how to calculate self-attention using vectors, then proceed to look at how it’s actually implemented – using matrices.

The first step in calculating self-attention is to create three vectors from each of the encoder’s input vectors (in this case, the embedding of each word). So for each word, we create a... See more

Anyone can say no to bad ideas, but only a focused person can say no to good ideas. Focus is about saying no.

When talking about one of the biggest lessons he learned from Steve Jobs, Jonny Ive said it was focus.

This sounds really simplistic, but it still shocks me how few people actually practice this, and it’s a struggle to practice, but is this

... See more