Are.na

There are some promising transfer and semisupervised learning techniques that may provide alternatives to gathering a great deal of labeled data,

So, the condition of superabundance creates a need for aggregation. LLMs will amplify that abundance. It’s a good bet that this strengthens the strategic importance of aggregation.

The second challenge we call the illusory progress gap: mistaking progress in AI on easy problems for progress on hard problems.

Deep learning is greedy. In order to set all the connections in a neural net correctly, deep learning often requires a massive amount of data.

OK, so let’s say one’s settled on a certain neural net architecture. Now there’s the issue of getting data to train the network with. And many of the practical challenges around neural nets—and machine learning in general—center on acquiring or preparing the necessary training data. In many cases (“supervised learning”) one wants to get explicit ex