HuggingFaceFW-Dev

HuggingFaceFW-Dev's activity

• All
• Models
• Datasets
• Spaces
• Buckets
• Papers
• Collections
• Community
• Posts
• Articles

loubnabnl 

posted an update 3 days ago

view post

Post

4,024

𝐈𝐧𝐭𝐫𝐨𝐝𝐮𝐜𝐢𝐧𝐠 🧬 𝐅𝐢𝐧𝐞𝐃𝐍𝐀: 𝐭𝐡𝐞 𝐛𝐞𝐬𝐭 𝐝𝐚𝐭𝐚𝐬𝐞𝐭 𝐭𝐨 𝐭𝐞𝐚𝐜𝐡 𝐚𝐧𝐲 𝐥𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐦𝐨𝐝𝐞𝐥 𝐃𝐍𝐀.
Training Qwen3-14B on it produces Carbon-X, the strongest open model on DNA benchmarks, on par with GPT-6.1 Sol.

It contains:
- Midtraining: 300B+ tokens of genomic sequences, papers and web pages mixing DNA with text
- SFT: 200K samples, from variant effect prediction to sequence question answering
- RL: 98K prompts with verifiable rewards

Data, models and the full recipe are open.
HuggingFaceBio/FineDNA

Enjoy this late Halloween treat! 🎃

See translation

• 

• 
• 
• 4 replies

·

🔥

15

15

😎

1

1

+

Reply

mjaggi 

authored 19 papers 20 days ago

Apertus 1.5: From Open Text Models to Open Multimodal Reasoners

Paper • 2610.15542 • Published 24 days ago • 40

Adapting English Quality Classifiers for Multilingual LLM Pretraining Data Selection

Paper • 2610.11585 • Published 27 days ago

Toward Cross-Lingual Quality Classifiers for Multilingual Pretraining Data Selection

Paper • 2604.20549 • Published Apr 22 • 1

Federated Learning for Malware Detection in IoT Devices

Paper • 2104.09994 • Published Nov 19, 2021

MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference

Paper • 2607.17733 • Published Jul 20

DoGE: Domain Reweighting with Generalization Estimation

Paper • 2310.15393 • Published Oct 23, 2023 • 1

Enhancing Multilingual LLM Pretraining with Model-Based Data Selection

Paper • 2502.10361 • Published Feb 14, 2025 • 2

Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs

Paper • 2502.05087 • Published Feb 7, 2025

Multiplication-Free Transformer Training via Piecewise Affine Operations

Paper • 2305.17190 • Published May 26, 2023 • 2

Special Properties of Gradient Descent with Large Learning Rates

Paper • 2205.15142 • Published May 30, 2022

Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations

Paper • 2405.18392 • Published May 28, 2024 • 12

Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors

Paper • 2606.25971 • Published Jun 24

Tying the Loop -- Tied Expert Layers in Mixture-of-Experts Language Models

Paper • 2606.16825 • Published Jun 15

Apertus LLM Family Expansion via Distillation and Quantization

Paper • 2605.29128 • Published May 27

Beyond URLs: Metadata Diversity and Position for Efficient LLM Pretraining

Paper • 2511.21613 • Published Nov 26, 2025 • 2

Learning from History for Byzantine Robust Optimization

Paper • 2012.10333 • Published Dec 18, 2020

TiMoE: Time-Aware Mixture of Language Experts

Paper • 2508.08827 • Published Aug 12, 2025

Training Dynamics of the Cooldown Stage in Warmup-Stable-Decay Learning Rate Scheduler

Paper • 2508.01483 • Published Aug 2, 2025 • 3

Masking as an Efficient Alternative to Finetuning for Pretrained Language Models

Paper • 2004.12406 • Published Apr 26, 2020 • 1