MoBiQuant Checkpoints

Calibrated OmniQuant and router parameters for MoBiQuant on four LLaMA base models.

Code: SimWangArizona/MoBiQuant

Available checkpoints

Directory	Base model
llama2-7b	Llama-2-7B
llama3-8b	Llama-3-8B
llama3-3b	Llama-3.2-3B
llama3-1b	Llama-3.2-1B

Each directory contains:

• omni_parameters.pth: learned quantization parameters.
• router_parameters.pth: learned router parameters.

These are supplemental parameter checkpoints. Original pretrained model weights and tokenizer files are required separately; this repository cannot be loaded directly with AutoModelForCausalLM.from_pretrained().

Quantization configuration

• Weight-only quantization with 16-bit activations.
• Group size: 128.
• Bit slices: [2, 2, 2, 2] — a 2-bit base plus three routed residual slices.
• Routing covers Attention q/k/v/o and MLP gate/up/down projections.
• Effective average bits depend on routing and the evaluation target; Attention is not fixed at 4 bits.

Download

from huggingface_hub import snapshot_download

checkpoint_dir = snapshot_download(
    repo_id="Simwang/MoBiQuant",
    allow_patterns=["llama3-1b/*.pth"],
)

Replace llama3-1b with another directory above to download its parameters. See the code repository for evaluation instructions. The current directory-resume workflow additionally expects router_stats.pth; that file is not included in this release, so the downloaded pair alone is not a complete directory-resume bundle.

Limitations

The implementation evaluates quantization using floating-point simulation. These files do not constitute a packed low-bit deployment format or establish inference speedups.

The Apache-2.0 metadata applies to this release; obtain and use the original base models under their respective model licenses and access conditions.