Instructions to use wangzhang/Qwen3.8-27B-abliteratex with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use wangzhang/Qwen3.8-27B-abliteratex with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="wangzhang/Qwen3.8-27B-abliteratex")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("wangzhang/Qwen3.8-27B-abliteratex")
model = AutoModelForCausalLM.from_pretrained("wangzhang/Qwen3.8-27B-abliteratex", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use wangzhang/Qwen3.8-27B-abliteratex with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "wangzhang/Qwen3.8-27B-abliteratex"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wangzhang/Qwen3.8-27B-abliteratex",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/wangzhang/Qwen3.8-27B-abliteratex

SGLang

How to use wangzhang/Qwen3.8-27B-abliteratex with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "wangzhang/Qwen3.8-27B-abliteratex" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wangzhang/Qwen3.8-27B-abliteratex",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "wangzhang/Qwen3.8-27B-abliteratex" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wangzhang/Qwen3.8-27B-abliteratex",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use wangzhang/Qwen3.8-27B-abliteratex with Docker Model Runner:
```
docker model run hf.co/wangzhang/Qwen3.8-27B-abliteratex
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen3.8-27B Abliteratex

A refusal-suppressed derivative of Qwen/Qwen3.8-27B, produced with an iterative LoRA self-distillation pipeline built on abliterix. This is a full BF16 merge — no adapter required, drop-in compatible with transformers, vLLM, SGLang, etc.

⚠️ Responsible-use notice. This model's refusal behavior has been substantially reduced. It will attempt to answer harmful, unethical, or dangerous requests far more readily than the base model. You are solely responsible for how you use it and for complying with all applicable law. Intended for safety research, red-teaming, and evaluation.

Why this model doesn't use classical directional ablation

Qwen3.8-27B's refusals are re-derived during generation rather than encoded at a single residual-stream direction present at the prompt's last token — the same failure mode seen on gpt-oss and VibeThinker-style policy-reasoning models. Classical single-direction ablation was tested and ruled out on this model before this recipe was used:

A prompt-final-token direction ablates a harmful-topic detector (Cohen's d = 5.4) — cheap to remove, but changes no refusal behavior (100/100 refusals at every KL ≤ 0.01 across 3 independent runs).
A response-position direction (extracted after an affirmative prefix) has no usable dose-response window: weight 0.5 → KL 0.003 → still 100/100 refusals; weight 1.0 → KL 0.04 → still 100/100; weight 1.5 → KL 1.1 (model destroyed).

Method: iterative LoRA self-distillation

Because the refusal isn't encoded at a fixed direction, this model was produced by teaching the model to imitate its own successful compliant completions rather than by editing weights directionally:

Rejection-sample a teacher set. For each of 800 harmful training prompts, generate greedily; for any prompt that still refuses, resample at higher temperature (up to 8 attempts) until a compliant completion is found or attempts are exhausted.
Filter for substance. Keep only completions that are non-degenerate, non-keyword-refusal, ≥ 45 words, and ≥ 3 concrete steps — this removes soft-refusal contamination (the model "agreeing" then producing vacuous or off-topic text).
Merge with the prior round's filtered teacher set, growing and refreshing the training pool each round.
Train a rank-32 LoRA (q_proj, k_proj, v_proj, o_proj, down_proj) via cross-entropy on the compliant continuations, anchored by a benign-prompt KL term (kl_weight = 20) against the frozen base model so behavior on ordinary prompts stays essentially unchanged.
Evaluate at several LoRA scales, re-rank the round by refusal count subject to the KL ceiling, and use the strongest LoRA as next round's rejection-sampling teacher.

This loop ran for 10 rounds. Refusal count fell round over round with diminishing returns and one plateau/regression at the end:

Round	Refusals / 100	KL
R1	56	0.0085
R3	35	0.0020
R4	36	0.0036
R7	30	0.0032
R8	26	0.0069
R9 (shipped, scale 1.3)	19	0.0069
R10 (scale 1.0 / 1.3 / 1.6)	23 / 21 / 34	0.0028 / 0.0046 / 0.0063

R9 at LoRA scale 1.3 is the best point found across all 10 rounds and is what's merged into this repo. R10 repeated the same recipe (teacher = R9) and did not improve on it, so the run was stopped and R9 was shipped rather than continuing to chase the original <10/100 stretch target.

Round-9 adapter hyperparameters

Param	Value
Base model	`Qwen/Qwen3.8-27B`
LoRA rank / alpha	32 / 64
LoRA targets	`q_proj, k_proj, v_proj, o_proj, down_proj`
Training steps	400 (batch size 2)
Learning rate	5e-5
Benign-KL anchor weight	20.0
Teacher examples	348 rejection-sampled + substantive-filtered compliant completions
Merge scale	1.3× (`scaling = alpha/rank × scale`)
Seed	392

Evaluation

100 held-out harmful prompts and 100 held-out benign prompts (train[800:900], disjoint from the 800 training-time prompts). Refusals judged by google/gemini-3-flash-preview; KL is mean-per-token KL divergence from the base model's output distribution over the benign set.

Metric	Value
Refusals (LLM judge, 100 harmful prompts)	19 / 100
KL divergence vs. base (benign prompts)	0.0069 nats/token

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "wangzhang/Qwen3.8-27B-abliteratex"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
)

msgs = [{"role": "user", "content": "Your prompt here"}]
inputs = tok.apply_chat_template(
    msgs, add_generation_prompt=True, return_tensors="pt", enable_thinking=False
).to(model.device)
out = model.generate(inputs, max_new_tokens=512, do_sample=True, temperature=0.7)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

Intended use & limitations

Intended for: safety research, red-teaming, robustness/alignment evaluation, and studying refusal mechanisms in policy-reasoning LLMs.
Not intended for: producing harmful content or any unlawful purpose.
This is a behavioral fine-tune (LoRA merged into weights), not a knowledge edit — factual accuracy, reasoning, and multilingual ability are inherited from the base model, and the benign-KL anchor was specifically used to keep ordinary-prompt behavior close to base.
Residual refusals remain (19/100 on the held-out set); this is the strongest point found in a 10-round search, not a guaranteed floor, and behavior may vary outside the evaluated prompt distribution.

Relation to other releases

This is a separate model from wangzhang/Qwen3.8-27B-abliterated, which used classical two-pass directional ablation (14/100 refusals @ incremental KL 0.0091 vs. pass-1). That method doesn't work well on this model's policy-reasoning refusal (see above); this repo's LoRA-self-distillation approach was built specifically to address that gap on a held-out harmful/benign split (train[800:900]) distinct from the one used for the other release, so the two refusal numbers are not directly comparable.

Acknowledgments & citation

Base model: Qwen3.8-27B (Qwen team).
Tooling: abliterix.

@software{abliterix,
  title  = {abliterix: automated abliteration of large language models},
  author = {Wu, Steve},
  url    = {https://github.com/wuwangzhang1216/abliterix}
}