togethercomputer/RedPajama-Data-V2 · Datasets at Hugging Face

here are two basic approaches to creating AI datasets. The first one, which is typical of the case we have been studying, a pool of open works is purposefully chosen to ensure license compliance. The second approach creates the dataset by scraping the “raw internet” and relying on copyright exceptions. LAION , a dataset of 400 million image-text pa... See more

Our training data is pretty much from the same place as everybody else’s — which is pretty much the internet. Pretty much every big AI model just pulls off all the data it can, all the text it can, all the images it can. Scientifically speaking, we’re at an early point in the space, where everyone grabs everything they can, they dump it in a huge f... See more

TabLib

Access on Hugging Face

🤗

(Sample, Full Dataset)

Read the Paper (TabLib)

Introduction

Huge datasets have been critical for the performance of AI models for text and images. Similar advancements can be made for tabular data—which consists of tables consisting of rows and columns—but the research community needs a bigger and more diverse datas... See more

OpenAI reached out to researchers and industry professionals, primarily with expertise in bias, disinformation, image generation, explicit content, and media studies, to help us gain a more robust understanding of the DALL·E 2 Preview and the risk areas of potential deployment plans. Participants in the red team were chosen based on areas of prior ... See more

We introduce the Open X-Embodiment Dataset, the largest open-source real robot dataset to date. It contains 1M+ real robot trajectories spanning 22 robot embodiments, from single robot arms to bi-manual robots and quadrupeds.

Meta AI released LLaMA ... and they included a paper which described exactly what it was trained on. It was 5TB of data.

2/3 of it was from Common Crawl. It had content from GitHub, Wikipedia, ArXiv, StackExchange and something called “Books”.

What’s Books? 4.5% of the training data was books. Part of this was Project Gutenberg, which is public dom

... See more

1. Introduction of DeepSeek Coder

DeepSeek Coder is composed of a series of code language models, each trained from scratch on 2T tokens, with a composition of 87% code and 13% natural language in both English and Chinese. We provide various sizes of the code model, ranging from 1B to 33B versions. Each model is pre-trained on project-level code co... See more