Okay, if you had infinite time, what would you build?”

If you’ll indulge me, here’s my list of potential projects, with light editorializing.

Energy supply

• Synthetic fuel.


• Better polysilicon production tech. The Siemens process feels paleolithic.


• Flexible silicon solar arrays, ground mounted and ultra cheap (<$30k/MW).


• Radioactivity applications

... See more

Ideas sometimes seem to need days or weeks or months to get to a point where they feel fully formed. If you try to force a solution to a problem into a preset window of time, you will almost certainly reach a suboptimal solution.

I’ll often have ideas sitting in my focus folder for weeks or months and keep tossing thoughts into them, and then one d... See more

I think that greatness is best described as “invisibility.” Great writing is invisible. It might take you a minute or two to get into the flow, but once you’re reading it, you’re flowing through it and don’t even realize you are reading words. Your mind is hallucinating the story for you, so seamlessly that it feels like you’re imagining it for you... See more

Focus forces brutal prioritization

Making few bets forces you to make hard decisions. It’s extremely hard to measure the value of something against some abstract and absolute notion of value.

Proponents of diversification argue that it takes the edge off of making a mistake. That would be a good argument if people acted the same way independent of t... See more

Only decades ago, the average person had one source of information, if any — the local newspaper. It’d take an hour, tops, out of their day. 1 hour out of 16 waking hours, or 6%. The rest of the day was spent making, creating value, conversing with others — 94%. Desires were simple — work for food and housing and a way to get around, find love, rai... See more

With three young kids, my relationship with time is complicated. I am at the stage where my alarm clock is a tiny pair of lungs and the idea of a vacation is a 15-minute drive to Whole Foods. These words by Charles Bukowski stopped me in my tracks, I think because my honest answer right now is freedom, but one day, when the kids are grown up, I kno... See more

One thing I do look back on fondly was how incredibly focused we were. Resources and time were so tight that you could feel the weight of all the things you weren’t working on. You had real conviction that the thing you were doing was the most important thing.

To pick a somewhat trivial example, at fireside chats with Mark (the predecessor to the co... See more

Self-Attention in Detail

Let’s first look at how to calculate self-attention using vectors, then proceed to look at how it’s actually implemented – using matrices.

The first step in calculating self-attention is to create three vectors from each of the encoder’s input vectors (in this case, the embedding of each word). So for each word, we create a ... See more

Every few years a great unbundling occurs

In 2010, Andrew Parker wrote a defining post about the "unbundling" of Craigslist where he outlined the opportunity to carve out niche products from broad horizontal networks like Craigslist. The broad networks are everything to everyone, the niche ones are something special to a certain group. If you add u... See more