Nice paper for a long read across 114 pages. "Ultimate Guide to Fine-Tuning LLMs" Some of the things they cover 📊 Fine-tuning Pipeline Outlines a seven-stage process for fine-tuning LLMs, from data preparation to deployment and maint... See more

New Anthropic research: We elicit capabilities from pretrained models using no external supervision, often competitive or better than using human supervision. Using this approach, we are able to train a Claude 3.5-based assistant that beats its human-supervised counterpart. https://t.co/p0wKBtRo7q

The 'data engine' idea of defensibility in AI may not be as defensible as we thought: In SELF-INSTRUCT, authors get GPT-3 to generate it's *own* dataset for instruction tuning, outperforming vanilla GPT-3 and comparable to InstructGPT. https://t.co/xlsA5pCwnS Here... See more