Slayer1B 1.00 — training checkpoints
Full trainer checkpoints from the training run of SlayerLab/Slayer1B-1.00: every 150 steps, 10 files. Each file holds the model weights and the optimizer state, so training can be resumed from any of them.
| Model | 1 073 912 851 parameters, 20 layers × d_model 2048 × 16 heads (details in the model card) |
| Steps | 1430 × 1 048 576 tokens = 1 499 463 680 tokens |
| Schedule | warmup 150 steps, constant lr 6e-4, then 1 − sqrt decay from step 1144 to lr / 10 |
| Last checkpoint before the decay | ckpt_1050.pt — used as the starting point of Slayer1B 1.01 |
| Training curve | https://track.fabryka.ai/run/48744ed2-3155-4294-b899-a6fda1f851da |
File format
torch.save dictionary of the trainer train_gpt_ref_r23.py (in the model repository):
model (state dict), opt (Muon for hidden matrices + AdamW for the rest), step, config.
Load with torch.load(path, map_location="cpu", weights_only=True); resume with the trainer's --resume.
Each file is 8 860 285 999 bytes.
SHA-256
A full SHA-256 recorded off the machine before the copy exists for steps 150, 300, 750, 1050 and 1430. For 450, 600, 900, 1200 and 1350 only the first 8 hex characters were recorded beforehand; the full sums in the table for these steps were computed from the files downloaded back from this repository and match those prefixes.
8f113dd02835bf3cc6737980f1664464d24c26168c0b245e9b8ed365ce4fb6c9 ckpt_150.pt
2765fc1b71989ac85cb44084e8083bd86cf6a33ba67b7df1453afdaabd355f16 ckpt_300.pt
d940b909d2d0db48e4e0184405c96a472ec44f6d676798e358dd44a22c49f85f ckpt_450.pt
13b5e60a27e4732cb1f48d144d2651408d0a3bb4b6c6dabf5d165a664622df56 ckpt_600.pt
3e5b03e6d546b139060503b932419c5ee04f1241822bb36b29951769f1a7e1d5 ckpt_750.pt
6fa366b77f4d3460cd4ecaf5bf86f299eadf2bd3d38ac1728e7667095756099b ckpt_900.pt
236108e8f204995f4adcfea7d9a99437d230c1af12708c64c68c74afbc2910fd ckpt_1050.pt
2fe897999f7fc76a3805c9234e69f56df8743b03f13c0e0f12752d575fbc5102 ckpt_1200.pt
551728562d7121f94980be587ed03dfb81619f968a91de1f29bbab9b42646cf7 ckpt_1350.pt
7d092d4d0bf838da1115a5369bf0a6729fe4c170e0d5ad303873f7320d6c1866 ckpt_1430.pt
License
Apache-2.0, as the model. Training data: see the model card.