togethercomputer/RedPajama-Data-V2 · Datasets at Hugging Face
![Thumbnail of togethercomputer/RedPajama-Data-V2 · Datasets at Hugging Face](https://cdn-thumbnails.huggingface.co/social-thumbnails/datasets/togethercomputer/RedPajama-Data-V2.png)
updated 10mo ago
updated 10mo ago
sari and added
madisen added
sari added
Nicolay Gerold added
Kasper Jordaens added
Darren LI added
Meta AI released LLaMA ... and they included a paper which described exactly what it was trained on. It was 5TB of data.
2/3 of it was from Common Crawl. It had content from GitHub, Wikipedia, ArXiv, StackExchange and something called “Books”.
What’s Books? 4.5% of the training data was books. Part of this was Project Gutenberg, which is public dom
Johann Van Tonder added
Nicolay Gerold added