RoboAtom: Large-scale Atomic-Action Supervision for Robot Learning

RoboAtom-Anno · 9B and 27B video annotation models

🌐 Project page · 💻 Code · 📄 Paper · 🤗 Data & benchmark · 🤖 Models

Models · Download · Inference · Output and evaluation · License and citation

Overview

RoboAtom-Anno predicts atomic actions from robot videos and task instructions. Each predicted segment contains a skill label, an action description, and frame boundaries. The release provides merged 9B and 27B weights, with tokenizers, video processors, and generation settings.

The accompanying RoboAtom-Data release provides atomic-action annotations for 11 datasets. RoboAtom-Bench provides 500 evaluation trajectories with 721 videos.

Models

Model	Weights	Architecture	Precision
RoboAtom-Anno-9B	RoboAtom-Anno-9B/	Fine-tuned from Qwen3.5-9B	BF16
RoboAtom-Anno-27B	RoboAtom-Anno-27B/	Fine-tuned from Qwen3.8-27B	BF16

Use Python 3.11+, transformers==5.8.0, and compatible CUDA-enabled PyTorch. The RoboAtom inference implementation handles synchronized camera composition, frame sampling, and frame-index prompts. GPU requirements depend on the model and video input.

Download

These examples use ./model for weights and ./data for the benchmark:

hf download InternRobotics/RoboAtom --repo-type model \
  --include 'RoboAtom-Anno-9B/**' --local-dir ./model
hf download InternRobotics/RoboAtom --repo-type dataset \
  --include 'RoboAtom-Bench/**' --local-dir ./data

Replace RoboAtom-Anno-9B with RoboAtom-Anno-27B to select the larger model. The Hugging Face CLI (hf) is required for these download commands.

Inference

1. Install the code

From your RoboAtom code checkout:

python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[inference]'

2. Start the model service

The following paths match the download layout above; replace them if you chose another location:

roboatom serve \
  --model ./model/RoboAtom-Anno-9B \
  --media-root ./data/RoboAtom-Bench

The service listens on http://127.0.0.1:3000. Change the model directory to select the 27B weights.

3. Generate predictions

In another terminal with the same environment activated:

roboatom infer \
  --input ./data/RoboAtom-Bench/benchmark.jsonl \
  --output outputs/roboatom_predictions.jsonl \
  --base-url http://127.0.0.1:3000

Choose a new output filename for each run; the inference client refuses to overwrite an existing prediction file.

Setting	Released inference behavior
Sampling	Uniform, target 2 FPS, including both endpoints, at most 448 frames
Precision	BF16
Generation	Deterministic, at most 4,096 new tokens
Time convention	Original-video frame indices, zero-based, both endpoints included

Output and evaluation

The client writes one prediction record per benchmark input, with an identity key and ordered action segments. A segment covers start_frame through end_frame inclusively; frames 0–74 contain 75 frames. Keep ground truth separate from model inputs.

From the code checkout, score the predictions:

roboatom evaluate \
  --gt ./data/RoboAtom-Bench/gt.jsonl \
  --benchmark-input ./data/RoboAtom-Bench/benchmark.jsonl \
  --prediction outputs/roboatom_predictions.jsonl \
  --protocol configs/roboatom_bench.yaml \
  --output-dir outputs/roboatom_evaluation

The scorer writes metrics.json, summary.tsv, and episodes.jsonl. See the benchmark card for data fields and the code README for the evaluation protocol. The code package also includes a visual annotation UI for editing and human review.

License and citation

The model directories contain the 9B Apache-2.0 license and 27B Apache-2.0 license. See the project's third-party notices for component attribution and citation file for publication metadata.

If you use RoboAtom in your research, please cite:

@misc{wei2026roboatomlargescaleatomicactionsupervision,
      title={RoboAtom: Large-scale Atomic-Action Supervision for Robot Learning},
      author={Xueyuan Wei and Xuanxu Wang and Xudong Li and Yuping He and Wanshui Gan and Jiaqi Peng and Wenqi Guo and Jianjun Zhou and Junting Dong and Chunhua Shen and Haoyu Guo and Mulin Yu and Tao Lu},
      year={2026},
      eprint={2610.40207},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2610.40207},
}