Post
5254
πΌ DaisyChain-Web: train a language model with friends or by yourself with multiple devices, in the browser, no install
Open a webpage, share a room link, and every device that joins becomes part of the training cluster. Phones, laptops, old PCs: they connect peer-to-peer over WebRTC and train one shared transformer together, entirely in the browser.
What's actually happening under the hood:
π§ A mini transformer LM trains on FineWeb-Edu, streamed live from the HuggingFace Hub. Each device pulls its own slice (data parallelism), tokenized with our 16.5k-token Spikewhale tokenizer
β‘ Every single multiply runs through verified INT8 neural units, no float fallback. On WebGPU browsers it uses the GPU's DP4A integer dot-product hardware, admitted only after proving bit-identical results against the verified units, with a 3ΓINT8 fast-accurate scheme (CUTLASS's 3xTF32 trick, ported to 8-bit)
π Devices average gradients every step under a sync guard: a per-step roster protocol plus weight-hash verification keeps every device's model bit-identical. If anything drifts, training stops instead of silently forking
π Live logs show exactly what every device contributes, step by step
πΎ When you're done: test generations right on the page, download a checkpoint, or grab the inference kit, a single self-contained HTML file with the weights baked in that runs generations offline, anywhere
Works solo too. Every extra device just grows the effective batch.
π Try it: Quazim0t0/DaisyChain-Web
π Training framework: DaisyChainAI/DaisyChain-Train
Updates:
- Block-scaled INT8 quantization
- Batched attention GEMM
- Fused dequant+ReLU epilogue
- Weight-tied unembedding
- WebSocket relay fallback
- Server keepalive ping/pong every 30s
- disconnected-state redial
- Visibility/network-change reconnect
Open a webpage, share a room link, and every device that joins becomes part of the training cluster. Phones, laptops, old PCs: they connect peer-to-peer over WebRTC and train one shared transformer together, entirely in the browser.
What's actually happening under the hood:
π§ A mini transformer LM trains on FineWeb-Edu, streamed live from the HuggingFace Hub. Each device pulls its own slice (data parallelism), tokenized with our 16.5k-token Spikewhale tokenizer
β‘ Every single multiply runs through verified INT8 neural units, no float fallback. On WebGPU browsers it uses the GPU's DP4A integer dot-product hardware, admitted only after proving bit-identical results against the verified units, with a 3ΓINT8 fast-accurate scheme (CUTLASS's 3xTF32 trick, ported to 8-bit)
π Devices average gradients every step under a sync guard: a per-step roster protocol plus weight-hash verification keeps every device's model bit-identical. If anything drifts, training stops instead of silently forking
π Live logs show exactly what every device contributes, step by step
πΎ When you're done: test generations right on the page, download a checkpoint, or grab the inference kit, a single self-contained HTML file with the weights baked in that runs generations offline, anywhere
Works solo too. Every extra device just grows the effective batch.
π Try it: Quazim0t0/DaisyChain-Web
π Training framework: DaisyChainAI/DaisyChain-Train
Updates:
- Block-scaled INT8 quantization
- Batched attention GEMM
- Fused dequant+ReLU epilogue
- Weight-tied unembedding
- WebSocket relay fallback
- Server keepalive ping/pong every 30s
- disconnected-state redial
- Visibility/network-change reconnect
- 11 replies
