What Builders Talk About When They Talk About AI

What Builders Talk About When They Talk About AI | Andreessen Horowitz

RelatedHighlights

Getting models into the hands of users will help us discover new use cases

Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz

We have lots and lots of customers who want to have specialized models that are cheaper, smaller, and have really high accuracy and performance. They’re saying, “Hey, this is what I want to do. I want to classify this particular defect in the manufacturing process from these pictures really well.” And there, the accuracy matters. Every ounce of acc... See more

Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz

I think ChatGPT will bring a bunch of companies really into AI. They will start out trying to solve things with GPT and then try to figure out how to solve this better with bespoke, smaller model. It could trigger a new AI summer for deep learning.

Very simply, Jevons Paradox states: if the demand is elastic and the price drops, the demand will more than make up for it. Normally, far more than make up for it. This is absolutely the case of the internet. You get more value and more productivity. I personally believe when it comes to any creative asset or work automation, the demand is elastic.... See more

Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz

I think this will be true for AI in general. ChatGPT starts out entering the enterprise, they get accustomed to it, they see the tangible results, and want to try it elsewhere without wanting an exact quantification in cost savings / revenue increases (which is often too hard to do for AI).

If you look at every technological shift or platform shift so far, it’s resulted in more things to design. You got the printing press, and then you have to figure out what you put on a page. More recently, mobile. You would think, “Okay. Less pixels, less designers.” But no, that’s when we saw the biggest explosion of designers.

—Dylan Field, Figma

Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz

Dylan Field on why AI will not mean less jobs

If you made a thousand versions of an LLM, that’s good at a thousand different things, and you have to load each of those into the GPUs and serve them, it becomes very expensive. The big holy grail right now that everybody’s looking for is: are there techniques, where you can just do small modifications where you can get really good results? There

Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz

PEFT in a nutshell.

My basic view is that inference will not get that much more expensive. The basic logic of the scaling laws is that if you increase compute by a factor of n, you need to increase data by a factor of the square root of n and the size of the model by a factor of square root of n. That square root basically means that the model itself does not get that... See more

Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz

I have a Turing test question for AI: if we took AI in 1633 and trained on all the available information at that time, would it predict that the Earth or the sun is the center of the solar system—even though 99.9% of the information is saying the Earth is the center of the solar system? I think 5 years is right at the fringe, but if we were to run ... See more

Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz

There was a study out there that did this "time-line" training.

If you have to be correct and you’ve got a very long and fat tail of use cases, either you do all of the work technically or you hire people. Often, we hire people. That’s a variable cost. Second, because the tails of the solutions tend to be so long—think something like self-driving where there are so many exceptions that could possibly happen—the... See more

Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz

I personally believe when it comes to any creative asset or work automation, the demand is elastic. The more that we make, the more people consume. We’re very much looking forward to a massive expansion in productivity, a lot of new jobs, a lot of new things, just like we saw with the microchip and the internet.

—Martin Casado, a16z