What Builders Talk About When They Talk About AI | Andreessen Horowitz
But just like the internet, someone will show up later and think about something like Uber and cab driving. Someone else showed up and thought, “hey, I wanna check out my friends on Facebook.” Those end up being huge businesses, and it’s not just going to be one model that OpenAI or Databricks or Anthropic or someone builds, and that model will dom... See more
Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz
Not sure about this one. Just an interesting snippet. I figure there are some reinforcing loops in the data, where the models get better with more data, attracting more users, generating more data. At the same time, I believe there are huge advantages in the knowledge on how to train and how to manage inference at scale, which makes a huge difference. I do not see anyone catching up to OpenAI at the moment, especially with their new finetuning offer.
An interesting factor might be figuring out the right data mix for pre-training and using a better screeing to weed out unwanted behavior. Whoever can figure that out at scale might have a huge advantage, if they can keep it a secret.
Getting models into the hands of users will help us discover new use cases
Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz
If you look at every technological shift or platform shift so far, it’s resulted in more things to design. You got the printing press, and then you have to figure out what you put on a page. More recently, mobile. You would think, “Okay. Less pixels, less designers.” But no, that’s when we saw the biggest explosion of designers.
—Dylan Field, Figma
—Dylan Field, Figma
Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz
Dylan Field on why AI will not mean less jobs
I have a Turing test question for AI: if we took AI in 1633 and trained on all the available information at that time, would it predict that the Earth or the sun is the center of the solar system—even though 99.9% of the information is saying the Earth is the center of the solar system? I think 5 years is right at the fringe, but if we were to run ... See more
Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz
There was a study out there that did this "time-line" training.
I personally believe when it comes to any creative asset or work automation, the demand is elastic. The more that we make, the more people consume. We’re very much looking forward to a massive expansion in productivity, a lot of new jobs, a lot of new things, just like we saw with the microchip and the internet.
—Martin Casado, a16z
—Martin Casado, a16z
Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz
My basic view is that inference will not get that much more expensive. The basic logic of the scaling laws is that if you increase compute by a factor of n, you need to increase data by a factor of the square root of n and the size of the model by a factor of square root of n. That square root basically means that the model itself does not get that... See more
Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz
a lot of the focus today is on the development of foundational large language models (LLMs), the transformer architecture was invented only 6 years ago, and ChatGPT was released less than a year ago. It will likely take years, or even decades, before we have a full tech stack for generative AI and LLMs and a host of transformative applications—thou... See more
Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz
When folks in these communities have access to very powerful tools, they tend to do remarkable things that create economic opportunities for themselves, their families, and their communities. They solve problems that you or I are not going to solve just because we don’t see the whole problem landscape of the world. We don’t have their point of view... See more
Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz
I really think that we could be entering a third epoch of computing. The microchip brought the marginal cost of compute to 0. The internet brought the marginal cost of distribution to 0. These large models actually bring the marginal cost of creation to 0.
Sarah Wang • What Builders Talk About When They Talk About AI | Andreessen Horowitz
Not just the cost of creation but also the cost of intelligence.