AbstractPhil/alephllm-nano-beatrix-training
Updated • 1
AbstractPhila PRO
AbstractPhil
AI & ML interests
datasets, research papers, experimentation, vision, classification, text encoders, tokenization, llms, diffusion, distillation, and more.
Recent Activity
updated a bucket 11 minutes ago
AbstractPhil/alephllm-chat-storage updated a model about 2 hours ago
AbstractPhil/alephllm-nano-beatrix-training repliedto their post about 3 hours ago
Been a hectic week since the completion of the mini-beatrix-3 as I tested new formats for the model's v4 state. Two new model formats now exist; Castor and Pollux. They will be trained with the same data as the beatrix-2s model was given, around 30b tokens.
The V4 variant of the AlephLLM dubbed Beatrix is a pair of twins particularly designed with a series of context window curated frame tokens.
Castor, and Pollux. Each housing a curated frame-spaced context window with curated depth embedded frame curations. For example; a divided 4096 context window can be 1024, 512, 256, 256, 64, 64, and the rest padding.
Nano-Castor-V1, the rotary softmax attention variant with around 8m params. Structurally capable of better curated "needle in a haystack" behavior. Stronger at depth and houses a cheaper format of attention than Pollux. Faster, stable by design, and only 6 blocks. Trains at about 900k tok/s.
Pollux, the rotary splat/softmax hybrid trained with a format of Canon LLM inspired by music theory lookup and multiple attention hubs at around 9m params. This model houses a deeper understanding and a more complex relational behavior than simple frame lookup, but the formula is a little more complex so it takes a little longer per step. Trains at around 450k tok/s.
All attributions will be given to the teams for the released papers utilized to create the components.
The experiment is based on training the four models to be capable of deciding and deducting high complexity predictions of frames and the context of those frames within their own frame deduction, creating a pseudo-language. The ultimate goal to make this context window look like gibberish with a full lookup table for debugging, and out comes the correct next unicode byte.
Will it work? Maybe not at this size. The tests show the frames work, the structure lines up, and the system does conform. Even if it doesn't work, the four prototypes will be ready by tomorrow evening and with those results new dataOrganizations
AbstractPhil/alephllm-nano-castor-pollux
Updated • 1
AbstractPhil/alephllm-mini-beatrix-training
Updated • 1
AbstractPhil/beatrix-tokenizers
Updated
AbstractPhil/mega-liminal-lora
Text-to-Image • Updated
AbstractPhil/mini-beatrix-1
Text Generation • 0.1B • Updated • 1.7k • 2
AbstractPhil/mini-beatrix-2s
Text Generation • 0.2B • Updated • 1.69k • 1
AbstractPhil/mini-beatrix-2.5s
Text Generation • 0.2B • Updated • 335
AbstractPhil/mini-beatrix-3
Text Generation • 0.4B • Updated • 1.38k • 1
AbstractPhil/clip-vitb-mini-distilled
Image Feature Extraction • 8.93M • Updated • 46
AbstractPhil/geolip-beatrix-anima
Text-to-Image • Updated • 2
AbstractPhil/geolip-beatrix-sana
Text-to-Image • Updated
AbstractPhil/sd15-flow-lune
Text-to-Image • Updated • 64
AbstractPhil/geolip-bytelex
Updated
AbstractPhil/aleph-splat-0
Updated
AbstractPhil/rnn-cifar10-t0
Updated
AbstractPhil/alephlm-0
Feature Extraction • Updated
AbstractPhil/alephlm-adopt-0
Text Generation • Updated
AbstractPhil/captionbert-8192-v2-b
Feature Extraction • 58.3M • Updated • 20 • 2
AbstractPhil/captionbert-8192-v2
Feature Extraction • 58.3M • Updated • 28 • 1
AbstractPhil/loss-manifest
Updated
AbstractPhil/geolip-bertenstein
Feature Extraction • Updated
AbstractPhil/geolip-vit-captionbank-coco
Image Feature Extraction • Updated
AbstractPhil/geolip-vit-base-x3
11.7M • Updated • 13
AbstractPhil/geolip-vit-large-x3
78.3M • Updated • 10
AbstractPhil/geolip-aleph-diffusion
Updated • 2
AbstractPhil/geolip-aleph-qwen-3.5-0.8b-instruct
Updated • 1
AbstractPhil/amoe-lora
Updated
AbstractPhil/aleph-diffusion-adapters
Updated
AbstractPhil/qwen3.5-0.8b-relay-caption
Updated