MSGEncrypted/ocf-typed-decisions-mbert-base

OCF typed-decisions CE smoke (ModernBERT-base)

Not a foundation classifier. Not RLCD. No Jev comparison.

Open Classification Foundation (OCF) specialist smoke: non-autoregressive System-1 decision head on answerdotai/ModernBERT-base, fine-tuned with cross-entropy on LocalLLaMA/typed-decisions train, temperature-calibrated on a held-out train slice (cal_frac=0.1).

Architecture

ModernBERT-base (bidirectional)
  + 2-layer decision Transformer head
  + OptionScorer at each option [MASK]
  → softmax(/T) over options

Knob	Value
Params	~164M
max_len	640
Option token budget	48 / option
Primitives	choice / score / noul

Training

Field	Value
Objective	CE on teacher-argmax labels (soft teachers in dataset)
Epochs / eff. batch	4 / 8
LR encoder / head	2.5e-5 / 1e-4
Seed (this upload)	1 (best of 0/1/2; mean acc 0.731, SD 0.028)
Temperatures (shipped)	choice 1.5 · score 1.25 · noul 1.5

Evaluation (LocalLLaMA/typed-decisions test, 2,000 decisions)

Arm	acc	NLL	ECE as-run	ECE after equal crossfit
Laya typed-decisions (reference)	0.767	0.707	0.214	0.021
This checkpoint (s1)	0.751	0.629	0.044	0.030
OCF stub (prior, ~8M)	≈0.61	≈0.90	≈0.04	—

Latency (warm, RTX 3050 Laptop): p50 59.6 ms / p95 87.8 ms per question (n=200). Not a cross-vendor claim.

Full board: see source-repo snapshot results-snapshot-2026-09-28-ocf-typed-decisions-mbert.md.

Files

• config.json — ClassifierConfig (includes temperatures)
• temperatures.json — fitted per-type T
• model.pt — full ClassifierBundle state_dict
• recipe.json — train hyperparams + acc

How to run

Requires the OCF stack from the source repository (packages.classify + models_ai.classify):

from classify import load

agent = load("path/to/this/repo")  # config.json + model.pt + temperatures.json
out = agent.predict(
    {"subject": "Refund", "body": "Billed twice"},
    {
        "department": {
            "type": "choice",
            "instructions": "Which department?",
            "criteria": {"billing": "invoices", "tech": "bugs", "other": "else"},
        }
    },
)

Not a drop-in for the external laya package.

Concurrent baseline

Laya — open System-1 concurrent (ModernBERT-large). Matched estimand only; not a dependency.

Non-claims

• Not a foundation / zero-shot System-1 model
• Not RLCD-trained
• Does not claim to beat TypeSafe Jev or Laya
• Soft teacher labels; teachers disagree on ~41% of test decisions