ThreatMon/qwen36-secura

ThreatMon | Website
License: Apache 2.0 | Authors: enes0o0 & Gökhan Yüceler @ ThreatMon

qwen36-secura is a domain-specialized Cyber Threat Intelligence (CTI) model, fine-tuned from Qwen3.6-27B using a 2-phase pipeline: Continued Pretraining (CPT) on a curated cybersecurity corpus, followed by reasoning-augmented Supervised Fine-Tuning (SFT) with full <think> Chain-of-Thought across 4 CTI tasks. Trained on RunPod H100 SXM 80GB with Unsloth, served via vLLM with native Multi-Token Prediction (MTP) speculative decoding.

qwen36-secura is a 27B-parameter cybersecurity-specialized language model built by ThreatMon. Starting from Qwen3.6-27B, it has been fine-tuned on authoritative cybersecurity knowledge sources — the NIST National Vulnerability Database (NVD) API 2.0 and MITRE ATT&CK Enterprise STIX repository — to deliver precise, evidence-based threat intelligence analysis.

The model outperforms both GPT-5.4 and Gemini 3.5 Flash on the CTIBench COMBINED score (68.41 vs 68.20 vs 66.55), while being a fully open-weight model that can be self-hosted. It leads in ATT&CK technique extraction (ATE Recall: 81.09% vs GPT-5.4: 43.76% vs Gemini: 37.88%) and format reliability (100% valid CVSS vector output vs 92% for GPT-5.4 vs 45% for Gemini).

---

Model Overview

Base Model	Qwen/Qwen3.6-27B (Dense, Gated DeltaNet hybrid)
Parameters	27.7B total, 335M trainable (1.21% LoRA)
Fine-Tune Method	bf16 LoRA (Unsloth), CPT + Reasoning-Augmented SFT
Training Hardware	RunPod H100 SXM 80GB, single GPU
Domain	Cyber Threat Intelligence (CVE, CVSS, CWE, MITRE ATT&CK)
Inference	vLLM + MTP speculative decoding
Context Window	16,384 tokens (serve) / 8,192 tokens (training)
Thinking Mode	Hybrid — configurable per-request (enable_thinking, thinking_token_budget)

---

Training Pipeline

qwen36-secura was trained in 2 phases, each building on the previous:

Qwen3.6-27B ➜ Phase 1: CPT (domain knowledge) ➜ Phase 2: SFT (reasoning ability) ➜ Merged Model + vLLM

	Phase 1 — CPT	Phase 2 — SFT	Inference
Input	Qwen3.6-27B base	CPT adapter	Merged 16-bit model
Data	Cybersecurity corpus, 16 public sources	4 CTI tasks with <think> CoT	—
Method	LoRA r=64, 1 epoch	Curriculum learning, enable_thinking=True	vLLM + MTP speculative decoding
Output	Domain knowledge	Reasoning ability	~90 tok/s (H100)

Phase 1 — Continued Pretraining (CPT)

Domain adaptation on a curated corpus from 16 cybersecurity sources:

Source	Share	What it provides
NVD (NIST)	84.1%	CVE descriptions, CVSS scores, CWE mappings
AlienVault OTX	5.5%	Community threat pulses, IOCs
GitHub Security Advisories	2.3%	Package vulnerabilities
MITRE ATT&CK + CWE + CAPEC	2.0%	Technique taxonomy, weakness catalog
Exploit-DB	1.8%	PoC exploits, vulnerability details
CISA Advisories + KEV	1.3%	Government advisories
Security Blogs (Krebs, ZDI, P0, etc.)	1.2%	Threat research, incident reports
Vendor IR (Mandiant, Kaspersky, Unit42, Talos, Microsoft, DFIR, CrowdStrike, Check Point)	1.8%	APT reports, incident analyses

Config: bf16 LoRA rank 64 / alpha 128, including embed_tokens and lm_head. RSLoRA enabled. 1 epoch, LR 2e-5 cosine.

Phase 2 — Reasoning-Augmented SFT

Full <think> Chain-of-Thought training across 4 CTIBench-aligned tasks:

Task	Source	Ground-Truth
VSP — CVSS Scoring	NVD API CVE descriptions	Official CVSS v3.1 vector
RCM — CWE Mapping	NVD API CVE descriptions	Primary CWE from weaknesses
ATE — ATT&CK Extraction	MITRE STIX software/groups	uses relationship techniques
MCQ — CTI Knowledge	MITRE STIX techniques/mitigations	Generated 4-choice questions

Key design decisions:

• Jackrong-inspired reasoning distillation: Teacher model (Gemini 3.1 Flash Lite, thinking_level=medium) generates full step-by-step <think> chains, but the ground-truth answer is fixed — the teacher only produces the reasoning that leads to it. This prevents hallucination from leaking into final labels.
• Curriculum learning: Stage 1 trains on short examples (≤6000 chars) for format stability, Stage 2 on longer ATE examples for complex reasoning.
• Eval contamination filter: CTIBench evaluation IDs (CVE, software, technique) are automatically excluded from training data.
• enable_thinking=True: The model is trained to think step-by-step before answering, leveraging Qwen3.6's native hybrid reasoning capability.

Config: bf16 LoRA on CPT adapter, 2 epochs per stage, LR 2e-5, train_on_responses_only.

---

Benchmark Results

Evaluated on CTIBench, the standard cybersecurity LLM benchmark. All models tested with the same prompts, parsing, and scoring pipeline. Each model used its lowest reasoning setting for a fair comparison: qwen36-secura with thinking_token_budget=1024, GPT-5.4 with reasoning_effort=low, and Gemini 3.5 Flash with thinking_level=low.

Main Results

	qwen36-secura	GPT-5.4	Gemini 3.5 Flash
CTI-MCQ Accuracy	77.60%	82.04%	81.64%
CTI-RCM Accuracy	63.10%	74.60%	74.60%
CTI-ATE F1	59.75%	46.54%	28.21%
CTI-ATE Precision	48.01%	70.41%	85.90%
CTI-ATE Recall	81.09%	43.76%	37.88%
CTI-VSP MAD	0.77	0.90	0.65
CTI-VSP Severity Acc	73.20%	69.63%	81.74%
VSP Format Reliability	100% (1000/1000)	92.2% (922/1000)	44.9% (449/1000)
COMBINED Score	68.41	68.20	66.55

All models benchmarked at their lowest reasoning level for fair comparison.

Key Findings

• ATT&CK Technique Extraction (ATE): qwen36-secura leads with 1.85x higher recall than GPT-5.4 (81.09% vs 43.76%) and 2.14x higher than Gemini (37.88%). The model systematically scans for behavioral indicators — a direct result of reasoning-augmented training.
• CVSS Scoring (VSP): 100% format reliability — every response produces a valid CVSS v3.1 vector. GPT-5.4 loses 8% of responses, Gemini loses 55%.
• COMBINED Score: Surpasses both GPT-5.4 and Gemini 3.5 Flash (68.41 vs 68.20 vs 66.55), demonstrating that a fine-tuned open-weight 27B model can outperform frontier commercial models on domain-specific tasks.
• Knowledge gaps (MCQ/RCM): ~5-11 points behind commercial models on pure knowledge recall — expected for a 27B open model vs trillion-parameter commercial systems.

Inference Performance (MTP)

Metric	Value
MTP Draft Acceptance Rate	~89.7%
MTP Throughput	~90 tok/s (H100 SXM)
VSP Exact Match (full CVSS vector)	54.60%

---

Intended Use

qwen36-secura is designed for Cyber Threat Intelligence (CTI) analysis tasks:

• CVSS v3.1 Vulnerability Scoring — Analyze CVE descriptions and calculate base scores with metric-by-metric justification
• CWE Root-Cause Mapping — Map CVE descriptions to their underlying CWE weakness classifications
• ATT&CK Technique Extraction — Extract MITRE ATT&CK technique IDs from threat intelligence reports
• CTI Knowledge Q&A — Answer questions about ATT&CK techniques, tactics, mitigations, and general cybersecurity concepts

When to Use Thinking Mode

Task Type	Thinking	Why
Simple knowledge (CVE lookup, CWE definition)	Off	Speed > marginal accuracy gain
CVSS calculation, CVE→CWE mapping	On (budget 1024)	Multi-step analysis benefits from reasoning
ATT&CK extraction, threat report analysis	On (budget 1024)	Systematic scanning improves recall
High-throughput batch processing	Off	Token savings, speed

---

Getting Started

vLLM (Recommended)

pip install vllm

vllm serve ThreatMon/qwen36-secura \
    --served-model-name qwen36-secura \
    --host 0.0.0.0 --port 8000 \
    --tensor-parallel-size 1 \
    --language-model-only \
    --max-model-len 16384 \
    --gpu-memory-utilization 0.90 \
    --reasoning-parser qwen3 \
    --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}' \
    --no-enable-prefix-caching

Python (OpenAI-compatible API)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

# Simple query (thinking off — fast)
response = client.chat.completions.create(
    model="qwen36-secura",
    messages=[
        {"role": "user", "content": "What is CVE-2021-44228 (Log4Shell)?"}
    ],
    temperature=0.7, max_tokens=512,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}}
)
print(response.choices[0].message.content)

# CVSS analysis (thinking on — accurate)
response = client.chat.completions.create(
    model="qwen36-secura",
    messages=[
        {"role": "user", "content": "Calculate CVSS 3.1 base score for: A remote unauthenticated attacker can execute arbitrary code via a crafted HTTP request to the admin panel..."}
    ],
    temperature=1.0, top_p=0.95, max_tokens=2048,
    extra_body={
        "chat_template_kwargs": {"enable_thinking": True},
        "thinking_token_budget": 1024
    }
)
print(response.choices[0].message.content)

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "ThreatMon/qwen36-secura"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

messages = [
    {"role": "user", "content": "List MITRE ATT&CK techniques used in Log4Shell exploitation"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1024)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))

---

Architecture

qwen36-secura inherits Qwen3.6-27B's hybrid architecture:

Component	Detail
Architecture	Gated DeltaNet (Mamba + Attention hybrid)
Parameters	27.7B
Layers	Interleaved linear-attention (DeltaNet) + full attention
Vocabulary	248K tokens
Precision	bf16
LoRA Rank	64 (CPT) / 16 (SFT)
LoRA Alpha	128 (CPT) / 32 (SFT)
MTP Head	Native (trained by Qwen, preserved through merge)

---

Hardware Requirements

Setup	VRAM	Notes
Inference (bf16)	~55 GB	H100/A100 80GB recommended
Inference (bf16 + MTP)	~56 GB	+1 GB for MTP head
Inference (FP8)	~28 GB	A100 40GB / RTX 4090
Training (bf16 LoRA)	~56 GB	H100 SXM 80GB

---

Training Data

CPT Corpus (Phase 1)

Metric	Value
Sources	16 public cybersecurity sources (NVD, MITRE ATT&CK, CISA, Exploit-DB, OTX, GHSA, vendor IR blogs, etc.)
Date window	3 years (2023–2026)
Deduplication	Cross-source CVE dedup applied

SFT Data (Phase 2)

Metric	Value
Tasks	CVSS Scoring, CWE Mapping, ATT&CK Extraction, CTI Knowledge
Method	Teacher-generated <think> CoT with fixed ground-truth answers (Gemini 3.1 Flash Lite, thinking_level=medium)
Contamination filter	CTIBench eval IDs excluded

Training data is not publicly released. It was curated from public APIs (NVD API 2.0, MITRE ATT&CK STIX) with reasoning traces produced by a teacher model.

---

Limitations

• Knowledge scope: Trained primarily on CVE/CWE/ATT&CK data. May underperform on topics outside NVD/MITRE coverage (e.g., cloud-specific security, IoT vulnerabilities not in NVD).
• Knowledge cutoff: Training data spans 2023–2026. CVEs after the cutoff are unknown.
• MCQ/RCM gap: ~10 points behind GPT-5.4/Gemini on pure knowledge recall tasks — inherent limitation of 27B vs frontier-scale models.
• ATE precision trade-off: High recall (81%) comes with lower precision (48%) — the model over-generates technique candidates. Post-processing filtering recommended for production use.
• No safety alignment: Contains offensive security content (exploits, attack techniques). Not suitable for consumer-facing applications without additional safety tuning.
• English-centric: CVE/ATT&CK data is primarily English. Performance on non-English CTI text is not evaluated.

---

Citation

@misc{qwen36-secura-2026,
  title={qwen36-secura: Reasoning-Augmented Cyber Threat Intelligence Model},
  author={Enes Özgözler and Gökhan Yüceler},
  year={2026},
  url={https://huggingface.co/ThreatMon/qwen36-secura}
}

---

Acknowledgments

• Gökhan Yüceler (ThreatMon Co-Founder & CTO) for co-developing the model and guiding the project direction
• Jackrong for pioneering reasoning distillation with Trace-Inversion — the Qwopus series and published datasets directly inspired our SFT approach
• Qwen Team for the Qwen3.6-27B base model
• Unsloth for efficient fine-tuning framework
• vLLM for high-throughput inference with MTP support
• CTIBench for the cybersecurity benchmark suite
• NIST NVD and MITRE ATT&CK for public training data