Hugging Face's logo

FineData
Team
community
Activity Feed

AI & ML interests

We release large pre-training datasets to accelerate open LLM development. Part of the Hugging Face Science team (hf.co/science)

Recent Activity
joelniklaus  updated a Space 9 days ago
HuggingFaceFW/finephrase
joelniklaus  authored a paper 15 days ago
Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior
joelniklaus  updated a Space 15 days ago
HuggingFaceFW/finephrase
View all activity

Papers

How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data

FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language

View all Papers

HuggingFaceFW 's collections 8

📀 Dataset comparison models
1.8B models trained on 350BT to compare different pretraining datasets
🧪 FineWeb v1 data experiments
Ablation models trained for our data experiments.
📀 Dataset comparison models
1.8B models trained on 350BT to compare different pretraining datasets
🧪 FineWeb v1 data experiments
Ablation models trained for our data experiments.