Ternary-Bonsai-2-27B-Abliterated-v2-PQ2_0-MTP-MLX

An MLX-native conversion of BoldingBuilds' Ternary-Bonsai-2-27B-Abliterated-v2-PQ2_0-MTP, preserving exact ternary weight representations and enabling self-speculative decoding (Multi-Token Prediction) on Apple Silicon.

Key Technical Facts

• Lossless Quant-Native Weight Preservation: Converted directly from the official PQ2_0 GGUF without re-quantization. All 866 tensors (27.3 billion elements) are mapped bitwise-exact to MLX affine packed representation.
• Thinking Mode Fix (v2 Update): Incorporates BoldingBuilds' v2 output-projection adjustment, preventing runaway reasoning loops and ensuring proper closure (</think>) before emitting the final answer.
• MTP Self-Speculative Decoding: Integrates the grafted Qwen3.8 MTP draft head. Correctly resolves the Sylvester-Walsh-Hadamard orthogonal rotation coordinates across both primary and MTP embedding projections, achieving an empirical ~87.5% draft acceptance rate in oMLX.
• Low Memory Footprint: Runs within ~8.6 - 9.4 GB unified memory (VRAM), making 27B parameter reasoning accessible on 16GB Apple Silicon machines (MacBook Air / Pro / Mac mini).

Architecture & Specifications

Parameter	Specification
Base Architecture	Qwen3.5 / Bonsai 2 (Hybrid GDN + Full Attention)
Parameters	27B total
Effective Bit-Width	2.13 bpw (Ternary weights with block scales)
Context Length	Up to 131,072 tokens
Vocabulary Size	248,320 tokens (ByteLevel BPE, full tail tokens preserved)
Speculative Engine	Multi-Token Prediction (MTP) depth=1
Runtime Target	oMLX (native Metal kernel & MTP pipeline support)

Quickstart (oMLX)

For optimal performance with speculative MTP decoding on Apple Silicon, run using oMLX (free, open-source macOS-native LLM runner with smart caching and native MTP acceleration).

This model requires custom architecture code (model.py) to handle Hadamard orthogonal transformations and MTP draft grafting.

1. Python Inference via oMLX Runtime

from omlx.model_settings import ModelSettings
from omlx.utils.model_loading import lm_load_compat, maybe_apply_pre_load_patches
import mlx_lm

model_path = "path/to/Ternary-Bonsai-2-27B-Abliterated-v2-PQ2_0-MTP-MLX"

# Enable MTP speculative decoding (depth=1)
settings = ModelSettings(mtp_enabled=True, mtp_fixed_depth=1)
maybe_apply_pre_load_patches(model_path, model_settings=settings)

# Load model (strict=True, trust_remote_code=True)
model, tokenizer = lm_load_compat(model_path, trust_remote_code=True, lazy=False)

# Generate
prompt = "<|im_start|>user\nExplain quantum entanglement in simple terms.<|im_end|>\n<|im_start|>assistant\n"
response = mlx_lm.generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True)
print(response)

2. Standalone MLX-LM CLI

When using standard mlx_lm, run with --trust-remote-code:

mlx_lm.generate \
  --model path/to/Ternary-Bonsai-2-27B-Abliterated-v2-PQ2_0-MTP-MLX \
  --prompt "Hello!" \
  --trust-remote-code

(Note: Speculative MTP acceleration requires the oMLX patch layer or an MTP-enabled mlx-lm runtime branch).

Provenance & Attribution

• Base Weights & Architecture: Prism ML (Sylvester-Walsh-Hadamard transform + Qwen3.5-based hybrid).
• Abliteration & Output Tuning: BoldingBuilds (quant-native ternary bit flip, thinking-closure fix).
• MTP Architecture: Qwen Team, Alibaba Cloud.
• License: Apache-2.0.