Instructions to use causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound")
model = AutoModelForMultimodalLM.from_pretrained("causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound

SGLang

How to use causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound with Docker Model Runner:
```
docker model run hf.co/causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound
```

Swift 1.5 Qwen3.8-27B W4A16 (AutoRound, BF16 MTP head)

A community 4-bit weight-only quantization of ukisai/Swift-1.5-Qwen3.8-27b, UkisAI's reasoning-efficient fine-tune of Qwen3.8-27B. It is not an official UkisAI release; UkisAI's own quantizations are listed on the parent card.

W4A16: int4 weights (group size 128, symmetric), 16-bit activations, made with Intel AutoRound 0.15.1 and exported as compressed-tensors. vLLM picks the int4 kernels up automatically (Machete on Hopper, Marlin elsewhere).
19.47 GB on disk versus about 56 GB for the BF16 parent.
BF16 MTP head: the multi-token-prediction module is kept at full precision and listed in the quantization ignore list, so self-speculative decoding works as it does on the parent.

Status: not yet evaluated. This checkpoint has not been benchmarked or load-tested in vLLM. It uses the same recipe, layout and toolchain as causal/Swift-Qwen3.8-27b-W4A16-AutoRound-MTP-BF16 (the Swift 1.0 version), which serves in vLLM 0.27.1 and scored within noise of its parent's published numbers on GPQA-Diamond, IFBench and AIME 2026.

Benchmarks

None measured for this repository yet. The parent card reports the following; they are copied here for reference and were not measured on this checkpoint.

Benchmark	Swift 1.5 BF16, 5 seeds	Swift 1.5 BF16, seed 0	UkisAI AutoRound INT4, seed 0
GPQA-Diamond	88.59%	91.41%	89.39%
IFBench (strict)	72.07%	72.00%	69.33%
AIME 2026	96.00%	86.67%	83.33%

5 seeds: the parent's main table. vLLM 0.27.1, context 262,144, reasoning effort xhigh, AIME capped at 250,000 tokens.
Seed 0: the parent's quantized-model comparison. vLLM 0.29.0, context 131,072, template-default thinking, one sample per prompt, AIME capped at 32,768 tokens with truncated answers scored incorrect.
The two protocols differ in caps, seeds and serving stack, so compare within a column, not across columns.

Quantization details

Setting	Value
Method	AutoRound 0.15.1, W4A16, group size 128, symmetric
Quantized	400 Linear layers: GatedDeltaNet `in_proj_qkv` / `in_proj_z` / `out_proj`, all MLP projections, full-attention `q/k/v/o`
Kept in BF16	vision tower (110 Linear), GatedDeltaNet `in_proj_a` / `in_proj_b` (96), `lm_head`, embeddings, MTP head (8 Linear)
Calibration	`NeelNanda/pile-10k`, 128 samples x 2,048 tokens, 200 iterations, batch 4, seed 42
Export	compressed-tensors (`pack-quantized`)
Toolchain	auto-round 0.15.1, transformers 5.17.0, vLLM 0.29.0 image (CUDA 13.0)
Cost	46 min on one NVIDIA L40S, peak 17.4 GB VRAM

The layer selection follows dbirks/Qwen3.8-27B-W4A16-AutoRound, except that the MTP head stays BF16 and is listed in quantization_config.ignore so vLLM builds the draft module unquantized. UkisAI's own AutoRound release uses the auto-round format instead of compressed-tensors.

How to use

vLLM

vllm serve causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound \
  --max-model-len 262144 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --port 8000

The Swift 1.0 version, which has the same size and layout, loads in 18.5 GiB of GPU memory and fits one full 262K-token request on a single 48 GB L40S. Lower --max-model-len on smaller GPUs.

Optional MTP decoding

--speculative-config '{"method":"mtp","num_speculative_tokens":3}'

MTP speeds up individual requests at low concurrency. On the Swift 1.0 version it lowered total throughput at high batch sizes.

Sampling

Use the parent's recommended settings: temperature 1.0, top_p 0.95, top_k 20, min_p 0.

License and access

This repository is a quantized derivative of ukisai/Swift-1.5-Qwen3.8-27b and carries the same terms. UkisAI's contribution is licensed under the Swift Open License v1.0; the underlying Qwen3.8-27B is licensed under Apache 2.0. See NOTICE.

Use is free for individuals and organizations with gross annual revenue, including affiliates, of up to US$1,000,000. Above that threshold, commercial use requires a Swift Enterprise License from UkisAI.

Changes from the parent: the model weights were quantized to int4 as described above (model-*.safetensors, model_extra_tensors.safetensors, model.safetensors.index.json); config.json gained a quantization_config and quantization_config.json was added; the other config, tokenizer and processor files were re-saved by transformers 5.17.0 during export; this README replaces the parent's. LICENSE, LICENSE-APACHE-2.0 and NOTICE are the parent's.

Citation

@misc{swift-1.5-qwen3.8-27b,
  title  = {Swift 1.5 Qwen3.8-27B},
  author = {UkisAI},
  year   = {2026},
  url    = {https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-27b}
}

Acknowledgements

Swift 1.5 was developed by UkisAI with compute from the NVIDIA Innovation Lab, Amazon Web Services and Google Cloud. Quantization for this repository ran on a Modal L40S.

Downloads last month: 25

Safetensors

Model size

6B params

Tensor type

I32

BF16

Model tree for causal/Swift-1.5-Qwen3.8-27b-W4A16-AutoRound

Base model

Qwen/Qwen3.8-27B

Finetuned

ukisai/Swift-1.5-Qwen3.8-27b

Quantized

(52)

this model