ZachW/CollabLLM-Qwen3-4B-SFT-Math

CollabLLM Qwen3-4B SFT — Math

Task-specific supervised fine-tuning LoRA adapter for Qwen3-4B, trained on the original CollabLLM interactive math-hard data. This is an independent reproduction artifact, not an official checkpoint from the CollabLLM authors. Base weights are downloaded separately when loading the adapter.

Training

• Base revision: 1cfa9a7208912126459214e8b04321603b3df60c.
• Data: interactive-math-hard, from the original authors' archived repository.
• One SFT conversation per dialogue ID: select the longest prompt and append its chosen assistant response. Split: 450 training / 50 validation conversations, seed 42.
• Three epochs, 24 optimizer steps, effective batch size 64; learning rate 1e-5, warmup ratio 0.1.
• Assistant-only loss. Maximum sequence length: 1536 tokens.
• LoRA rank 32, alpha 16, dropout 0.1; attention and MLP projection layers.
• Non-thinking mode. The included tokenizer chat template fixes enable_thinking = false.

Exact hyperparameters, training/validation metrics and the adapter SHA-256 are in training_metadata.json. The original assistant system prompt is included as system_prompt.txt (attributed to CollabLLM under the accompanying COLLABLLM_LICENSE).

Load

Install compatible releases of torch, transformers, peft, accelerate and huggingface_hub with Qwen3 support.

import torch
from pathlib import Path
from huggingface_hub import hf_hub_download
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

adapter_id = "ZachW/CollabLLM-Qwen3-4B-SFT-Math"
tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-4B",
    revision="1cfa9a7208912126459214e8b04321603b3df60c",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id).eval()
system_prompt = Path(hf_hub_download(adapter_id, "system_prompt.txt")).read_text()
messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "Can you help me with a task?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

Scope and attribution

This release contains the final SFT adapter and tokenizer, not GRPO, PPO, or DPO weights. It uses Qwen3-4B, so it does not establish reproduction of the original paper's Table 1 results. No benchmark score is claimed by this model card. The model can produce incorrect responses and should be evaluated for its intended application.

Method and source data: CollabLLM paper and original code. Base model: Qwen3-4B, Apache 2.0. The source data remain subject to their original terms. The included CollabLLM prompt retains its original MIT notice.