Openish ML

Apple announces LLM in a flash: Efficient Large Language Model Inference with Limited Memory paper page: https://huggingface.co/papers/2312.11514... See more

Today Google released Gemini with a 60-page report in which they repeatedly say the training data is key ("We find that data quality is critical to a highly-performing model"), while providing almost no information about how it was made, how it was filtered, or its contents.

The rise of statistical NLP in the early 2000s was another such cycle. But major methodological shifts come with major cultural shifts as well. Statistical NLP introduced a culture laser-focused on benchmarks and saw the end of a small, “high trust” research community.

The lack of acknowledgment of the threat of prompt injection attacks in this new Purple Llama initiative from Meta AI Is baffling to me