Apple announces LLM in a flash: Efficient Large Language Model Inference with Limited Memory
paper page: https://huggingface.co/papers/2312.11514... See more
Today Google released Gemini with a 60-page report in which they repeatedly say the training data is key ("We find that data quality is critical to a highly-performing model"), while providing almost no information about how it was made, how it was filtered, or its contents.
The rise of statistical NLP in the early 2000s was another such cycle. But major methodological shifts come with major cultural shifts as well. Statistical NLP introduced a culture laser-focused on benchmarks and saw the end of a small, “high trust” research community.