hiteshluke/zyot-decider-12b-adapter

Zyot Decider · 12B

A production System One decision adapter at the 12-billion-parameter class, built on google/gemma-4-12b-it with structured-decision soft-label training. Returns typed decisions over the three Jev primitives:

• noul — Boolean / True-False
• choice — multiple-choice
• score — ordinal 0–5 rating

The 12B flagship of the Zyot Lab decision-model lineup by Codekins Pvt Ltd.

For local inference via llama.cpp / Ollama / LM Studio, see the GGUF companion repo: hiteshluke/zyot-decider-12b-gguf.

---

Benchmarks

Benchmark	Primitive	Accuracy
JevBench boolq (n = 200)	noul (T / F)	0.885

Measured head-to-head against the Laya reference at the same prompt format (Laya 0.860 on the same split).

---

What makes it a 12B flagship

• Gemma 4 12B base — strong multilingual, long-context foundation
• Soft-label distillation training on SargeDev/jev-distill-corpus-v3
• LoRA-adapted — ~250 MB adapter instead of a full 24 GB model weight
• Standard transformers + peft deployment on bf16 or 4-bit quantized
• GGUF companion repo for direct llama.cpp / Ollama / LM Studio use

---

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch

BASE = "google/gemma-4-12b-it"
REPO = "hiteshluke/zyot-decider-12b-adapter"

tok = AutoTokenizer.from_pretrained(BASE)
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16,
                         bnb_4bit_use_double_quant=True)
model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb,
                                             dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, REPO).eval()

T_ID = tok("T", add_special_tokens=False).input_ids[0]
F_ID = tok("F", add_special_tokens=False).input_ids[0]

@torch.no_grad()
def decide(passage: str, question: str) -> str:
    prompt = (
        f"State: {passage}\n\n"
        f"Question: {question}\n\n"
        f"Options:\nT. Yes / True\nF. No / False\n\nAnswer:"
    )
    ids = tok(prompt, return_tensors="pt", truncation=True, max_length=1024).input_ids.to(model.device)
    logits = model(ids).logits[0, -1]
    return "T" if logits[T_ID] > logits[F_ID] else "F"

print(decide(
    "The Eiffel Tower is located in Paris, France.",
    "Is the Eiffel Tower in France?"
))  # -> T

---

Local inference (GGUF)

For CPU / consumer-GPU local inference without PyTorch, use the GGUF companion repo:

# Example with llama.cpp (merge base + adapter into one GGUF offline, or load
# the adapter at runtime where supported).
llama-cli \
  -m gemma-4-12b-it-UD-IQ3_XXS.gguf \
  --lora zyot-decider-v4-adapter.gguf \
  -p "State: ...\n\nQuestion: ...\n\nOptions:\nT. Yes / True\nF. No / False\n\nAnswer:" \
  -n 2

See hiteshluke/zyot-decider-12b-gguf for the quantized files.

---

Files

• adapter_config.json + adapter_model.safetensors — LoRA adapter on Gemma 4 12B

---

Training

• Base: google/gemma-4-12b-it
• LoRA: adapter targeting the standard attention + MLP projections
• Data: SargeDev/jev-distill-corpus-v3, soft-label distilled from a Jev reference
• Objective: verbalizer-token soft-label distillation
• Hardware: NVIDIA A100 / Kaggle T4 (QLoRA 4-bit)

---

Positioning

• 12B parameter class — largest open Zyot Decider release
• Multi-primitive — one adapter, three decision types
• Verbalizer-compatible output — reads T / F from LM head logits
• PyTorch and GGUF both shipped

---

License

Apache-2.0 for the LoRA adapter and code in this repository. Base google/gemma-4-12b-it is governed by the Gemma license.

---

Citation

@misc{zyot-decider-12b-2026,
  title   = {Zyot Decider · 12B — a System One decision adapter on Gemma 4 12B},
  author  = {Codekins Pvt Ltd · Zyot Lab},
  year    = {2026},
  url     = {https://huggingface.co/hiteshluke/zyot-decider-12b-adapter}
}