SlayerLab/Slayer1B-1.05-checkpoints

Slayer1B 1.05 — training checkpoints

Full trainer checkpoints of the run that produced Slayer1B 1.05: pre-training continued from ckpt_10010.pt (sha256 04438959…). Each file holds the model weights and the optimizer state. Release 1.05 is the state at step 12870 (ckpt_12870.pt).

Status (28 October 2026): complete — 58 files. All copies have completed anonymous read-back verification. The LFS object ids and sizes of all 58 files were independently compared with the verified source list on 28 October 2026: no missing files or mismatches. The same files are also in Slayer1B-1.04-checkpoints, which collects the whole continuation 7200–13848.

Start	ckpt_10010.pt (step 10010, constant lr 6e-4)
Steps	10011 → 12870 (2860 steps × 1 048 576 tokens = 2 998 927 360 tokens), 256 sequences × 4096 tokens per step
Schedule	constant lr 6e-4, no decay
Data	SlayerLab/slayer-pl-8x3b revision d50df04a, continuing the single pass over the corpus.
Leak-scan exclusion	from pack-04 on (step 7193), the 5133 documents with n-gram overlap with BoolQ / MMLU / ARC test items are excluded (list sha256 84caed78…); this whole segment is filtered (steps up to 7192 were not)
Hardware	8 × NVIDIA L40S (48 GB), DDP; global batch identical to the 4 × L40S run of 1.04
Checkpoints	every 50 steps, and the final step 12870; all are constant-lr states
Trainer	train_gpt_ref r24.2 (sha256 46f83b51…); model definition train_gpt_ref_r23.py (unchanged)
Training curve and evaluations	https://track.fabryka.ai/run/f056bb5c-f1da-56d5-9776-cc33063f10a5

SHA-256

step	sha256
10050	1e293777223c9db9388b724c2fc2bf821e25fcd50e68ce396b7e97b738793935
10100	c890275e08a5f152cbee78cd0d19c65d2c6d2f583dd6768f78130a3dc29a7dcc
10150	64b6a44829cfb6cf57c2baa21b1e3b8b20ad46db58554cf8ef40ad63a7bb6e2d
10200	5fef664ffd13f4059b3da0f86cfe938637161aeb4913820dab2c85bbc6497407
10250	fd68cef94410d4853416476e7b0f40e12f940d61c4edf0138c6f36953decee4c
10300	da36427a84f4598c78590f0189f6d111f805c65d9e9058f0344693ae7c9dc786
10350	3e39419093ca7e3dde5d1ddfe05207eb46dbfe63ca184622010c57ce80f00a15
10400	4f019c45d5302654200b8498d077e6cd6fc05cb83322b55f25b580b083a5d51b
10450	d89c9795a3c607e52c6b639f60df1c64d5dce4e1000a8018530f5aac429f1ec2
10500	fa093dd48a381a4167710c475ab5ec7d1b39e3e545ca9a7d41dca16a0931dd36
10550	d9a9a622da8a78d3b3d8d493edb3e48ae8d286e23b550f7e3fc5bb456b4c7790
10600	c18e0df52ea7146a53c6a750432f04bb760c3ebac316de30e954fbed0a50ff17
10650	aa7c287f10957a073237edcce300cc7b99a95ddb5ae56243e9c5db321b7c4045
10700	6f8a60a06d459d4a86a2022eea88821dbc03157b4ac2eda0678de624e67b5778
10750	758e472fcc18c58cf80b7997cb48802dc3bf9ee729102c82238b9a0642f05c58
10800	1de3505af0540ce8854210ee8d55bf1365e71ebc2a1a5ccd95894cf3a4b2b79c
10850	ea2b46c734cc037fe8c1f36a27bd1cf72c6c4133e65be4d4efcd00ddc79dd1f0
10900	83de1b3383642669dfa94077e9144785ab35e0857bcfae197b203b7da67750da
10950	80f883029b536ee0bd8fc5a6724262c6f6d7b6e43bdbc0d754344d868b3baae8
11000	a61be69ed50ed9ad96a8d9137ec2e877173273a11365ddb91e7a6c3c9ca23e98
11050	3263d00197322a13280623e0265448b3c880ee1d59edf2b67b0394e48bb8e0e8
11100	63ae4e51a79d57ce4c6ecbd7e1d1289c5e256cb2d91276ffe5d92d67d4153531
11150	01d22003471d08c1e038878b3d7ad2804a66286841eb616674c96564c6df2369
11200	7e115cbfb90fc954e66f4038f4509446f50eb7697f7df6d73bf938d7d8a43621
11250	4895150ac9ba77245032722168dee10eb1cdc52c0fbae21ab35f95db9ba1c08f
11300	bdf6df165319d361000e930a80324a7d4411275980ebd7e05ef919dedd380803
11350	866d766db9cf871ca4921e583da0ddc8f11088a5ce72b0dfe472265c8b9f0fd1
11400	5b54820f710cfaad70a3af661c1ce31484788b53ade195c7794ae64156e6d709
11450	b60b0d28fedf35d49f4ff36bdf2bc870e8e23343a67b2ba4f40f2c62981788b0
11500	14b3ce222780fdc9db13c4d38823e205754795806cf224b34ec9571d946135cc
11550	fe882eb7d0d830ab98a3e6de2f675c5e8cee1adbae4849b828c34380ecae24f0
11600	fd8a3d588918836edcfc66825e944acfae3f41e0bda8f35919b4d1f9f9c5a576
11650	de619b01c95a8bbd18e17019a4a45a3e0eec6799fe446dfc2c40d3b45aca3084
11700	4bda56891d65fb1481dd9dfc2009c3601a4d7c11faf89ddb16778926eae6a50a
11750	ddae521b41a978a7a9c6bb98606ae51ee5946e88e32b892c8793bfe11d2248e5
11800	aaf62d32005e272fb9e98058050ef12bc686aea349352bd134842533214b3363
11850	8270afa40b28e164297b05f48d807b7bb6c83402be856197315a2698bbe111e3
11900	47b4d765c9e57e2e790ac8458971654d61589926d1740c4bc1e22716258aa9da
11950	0b0400680cf4f7d061802e54529cc25999f3377361a53c1b6874c0ded687eda4
12000	c81cf23f2e50375c2569dc2ef863b5a77cb90db2e36998abca8fbb80711b76dc
12050	83f77ee11b30f5875539b81c4946d44422cc8f346ef640d3d04555f3730b1842
12100	d4ca1c2f85d92613eb309f74d9edf7558547d472c3a6b6ad4dd2c029f6b5ff00
12150	ec2c9f325c5c26c86d58bdaf872baf114889cca9c0eeeb26432cffd9e856b0dd
12200	bca0bf3a5e025342a90d58faa6bc13d104f4db72fafb197bbbc78e8aefab4481
12250	8a67f281a835f3bc53502386eca126d6212ae8ffd52a9e2ff91a510cb467c907
12300	da181e3e067b3b015eaf9910780468aa7414456c6da86934b2c285cc4fdc001b
12350	8d8313b0cdf00df30dbc62bd27daca8d82061717de73c59c1645d29e90a3ea32
12400	90482d11b8863ecff0f59ea97f864ed30a9fdeda6ae3523f569217a56606de1c
12450	b84627ecbe11e5e68cb1bd88293f59738d791835a92649ab2c63ce083347bb74
12500	729087cc7a1809151aab99011c29bb13fc5e6a669986144f4afa3f53fd573bbb
12550	a7f8a2e7259f0950f66bffa67f8b8504c007c7af4532615dfeb98ba839e03a72
12600	54ab72c1682cc955b871e0306674de60c876c203b41f3933f219dad5f3b0aee2
12650	8aa7062b36e13a43dec3c77f8733dc7bc1a38ee8252354247bfc783bbcb3cd6f
12700	aafe19bebf7b931f6a4f2379d9bd9711674a44faf6809a5d449a7095640e3d8f
12750	6e4274c649674adfa0d75360f1c503494627f76f03eea0d5529ab6e7ce90e46a
12800	d03cda6f885f0d1e88e60c0aa1f043959de564f9284d8803928b10a3b0628e70
12850	c4216f9f34e2daabd84b1edc309277678a35266b42ba1b1a557d35347c42be54
12870	435082ff36becfbb96803622139f38d2afa0ba1a9d2fc284bad96f5a8edc1181

File format

torch.save dictionary of the trainer: model, opt (Muon + AdamW), step, config. Load with torch.load(path, map_location="cpu", weights_only=True). Model class: train_gpt_ref_r23.py in the model repositories (1.00–1.06). Every file is 8 860 287 151 bytes.

License

Apache-2.0 for the checkpoints. The training data includes CC BY-SA sources (shared/core_sa, shared/legal_sa of slayer-pl-8x3b); see the dataset card for the licenses of the data.