Kuroko

Kuroko is a turn-taking system for two-channel English conversation, built by PatientDesk AI for TurnBench. It decides when a speaker's turn has ended, and whether a sound from the other speaker is an interruption or an "mhm".

Write-up: https://spero.patientdesk.ai/blog/kuroko

This repository holds Kuroko's weights, the one file needed to run them, its settings and its predictions for TurnBench's development split. The training code and the training data are not public, and neither are the predictions for the test split, at the benchmark's request.

Run the model

The model is a 14.9M-parameter causal transformer. It hears both speakers and answers 25 times a second for each of them, from the past only.

pip install torch numpy scipy safetensors soundfile huggingface_hub

import sys

import soundfile as sf
from huggingface_hub import snapshot_download

path = snapshot_download("cloud0day3/kuroko")
sys.path.insert(0, path)
import kuroko

model = kuroko.load(path)
audio, rate = sf.read("conversation.wav", dtype="int16")  # (samples, 2): one speaker on each channel
answers = kuroko.answers(model, kuroko.features(audio, rate))  # (2, 11, n): speaker, answer, 25 a second
claims = answers[:, kuroko.ANSWERS.index("claim")]

answers[s, a, k] is a probability for speaker s that has heard the audio up to (k + 1) / 25 seconds. The 11 answers are named in kuroko.ANSWERS:

Answer	Meaning
voice	The speaker is making a sound
claim	That sound claims the floor. This is the answer the events are read from
hold	The speaker holds the floor, pauses included
8 more	Whether the speaker, and whether the partner, will talk for half or more of 0 to 0.2 s, 0.2 to 0.6 s, 0.6 to 1.2 s and 1.2 to 2 s ahead

The model was trained on 16-bit recordings at 48 kHz with one speaker per channel, brought to 16 kHz as kuroko.features does it. Read that way, on the GPU the predictions in this repository were made on, the released files give the answers behind them on all 38 dev conversations. On a CPU an answer differs by less than 0.001, which can move a threshold crossing by one answer, 40 ms. Audio of more bits read as floats moves an answer by up to 0.02.

Only claim has been measured against annotations. Offline, a recorded 12.6-minute conversation is answered in 11 s on four CPU threads, in 40-second stretches. The model has not been run as a live stream.

From answers to events

The model is one of three parts, and this repository has the code of the model only.

1. Detector. Silero VAD 6.2.3 runs on each channel in 32 ms steps and marks stretches of sound. It is certain that a stretch has ended 0.26 s after its last sound.
2. Rules. The timing rules p99lab published for turn-1-mini, without that system's learned parts, with the numbers chosen again on the dev split (policy.json). Every stop opens a candidate end of turn, which is committed once the partner has taken over or 2.45 s have passed, unless the speaker is back.
3. Model. claim commits an interruption when it reaches 0.9, read from 0.32 s after a sound begins, before the sound has lasted the 1.3 s the length rule needs. It also tells the end-of-turn rules that the partner has taken over, when it reaches 0.5 for the partner's sound.

The write-up gives the rules in full.

Results

TurnBench development split: 38 conversations, 1,904 ends of turn, 347 interruptions, scored by the benchmark's scorer. The settings were chosen on this split within a false-alarm budget of 8%. "Held out" is the same choice made on half of the conversations and scored on the other half, over 500 random halves. Kuroko was submitted to TurnBench on November 13, 2026. The test split is scored by the benchmark's maintainers, and its results are theirs to publish on the leaderboard.

	Recall	False-alarm rate	Median delay	Held-out recall	Held-out false-alarm rate
End of turn	0.987 (1,879 of 1,904)	0.079 (84 of 1,063)	630 ms	0.983	0.082
Interruption	1.000 (347 of 347)	0.073 (272 of 3,733)	545 ms	0.988	0.061

To check these numbers, drop predictions-dev.json on the benchmark's dev scorer. It scores the file in the browser.

What each part adds, on the same split (recall / false-alarm rate / median delay):

System	End of turn	Interruption
Rules, turn-1-mini's published settings, on our detector	0.968 / 0.051 / 879 ms	0.899 / 0.026 / 1,579 ms
Rules, settings chosen on dev	0.983 / 0.080 / 692 ms	0.977 / 0.076 / 1,078 ms
Kuroko: rules and the model	0.987 / 0.079 / 630 ms	1.000 / 0.073 / 545 ms

Training data

1,025 hours of two-speaker conversation: 786 hours of Open Yap 1K and about 240 hours of PatientDesk's own recordings, made by contributors who agreed to this use. The labels come from a timing rule applied in hindsight. None of the data is in this repository.

No audio and no label from TurnBench's dev or test split was trained on, and otoSpeech was not used. The dev split chose the settings, checked the label rule and compared the systems above.

Files

File	Contents
model.safetensors	The weights, kept at step 12,000
config.json	The model's shape
kuroko.py	The model, its features and how a conversation is answered. Nothing of the training
policy.json	The settings of the rules, in seconds and answer levels: eot for ends of turn, int for interruptions
predictions-dev.json	Kuroko's events for the 38 dev conversations, in TurnBench's submission format
training-log.jsonl	Training loss and the loss of each answer on held-out conversations, every 600 steps

Each block of policy.json is a full settings object, and each rule reads only part of it. The end-of-turn rules read wait_s, partner_s, deadline_s, resumed_s, held_s, confirm_s, partner_claim and claim_from_s from eot. The interruption rule reads sustained_s, claim_fire, claim_from_s and claim_rise from int. known_s and begun_s are the detector's delays.

Limits

• These are development numbers on 38 conversations, with settings chosen on them. Zero missed interruptions is an in-sample number; held out, recall is 0.988.
• TurnBench records two people, so the end-of-turn rules can wait for the second person to start talking. A voice agent is the second speaker, and in an agent these rules would be left with the 2.45 s timer.
• A turn's end can be committed a second time, and an interruption is committed for every sound taken as a claim. The scorer counts at most one false alarm per pause, and looks for interruptions only near annotated interruptions and backchannels. Kept at least 1 s apart, the events score 0.975 at 0.074 on ends of turn and 1.000 at 0.070 on interruptions.
• Studio audio with one clean channel per speaker, English only. Not measured on phone audio, with echo or as a live stream.

The write-up has the rest: the label rule and its check against the annotators, the ablations, the causality test and the credits.

License

model.safetensors, config.json and kuroko.py are released under Apache-2.0.

Credits

TurnBench is by Sesame AI Labs, with the evaluation corpus from Mundo AI and the otoSpeech training set from Oto (Jiang et al., 2026). The timing rules are p99lab's. The detector is Silero VAD. The model's future-activity targets follow voice activity projection (Ekstedt and Skantze, 2022).

Training data: The Agentic Data Company. Open Yap 1K: channel-separated English natural two-speaker conversations. 2026. Version 1.0. https://theagenticdatacompany.com/open-yap-1k