alkinun

Based on the model cards I've reviewed, here are the key ways BGA (BananaMind Gate Attention) differs from NSA/DeepSeek (2025):

Core Architecture Differences

BGA (BananaMind Gate Attention):

• Uses a three-tower relay transformer architecture with towers A, B (relay), and C
• Tower B has no output head or loss term - it only passes gradients between A and C
• 256x less attention compute at 1M context windows
• Context windows: 10M, 16K, 25M, 32K, 64K (up to 1M in training)
• ~35M parameters (BananaMind-2.1-Unified)

NSA/DeepSeek-R1:

• Uses MoE (Mixture of Experts) architecture
• 671B total parameters with 37B activated parameters
• 128K context length
• Focuses on reinforcement learning for reasoning rather than architectural efficiency

Design Goals

BGA: Optimized for computational efficiency - can train 1M context models at the cost of ~4K context training
NSA: Optimized for reasoning capabilities - uses RL to develop chain-of-thought behavior

Training Approach

BGA: Trained on 38B tokens with flat mix, no curriculum
NSA: Uses reinforcement learning without supervised fine-tuning as a preliminary step, then incorporates cold-start data

Current Status

BGA: Released (BGA model available), BananaMind 3 "will start training soon"
NSA: Already deployed with strong benchmark performance (outperforming o1-mini on many tasks)

The fundamental difference is that BGA is an efficiency innovation for scaling context windows, while NSA is a reasoning breakthrough using RL. They're solving different problems in LLM development.

@KlondikeDev this is just hallucinated btw, not correct.

well he didnt show some decorum when he hid my comments that corrected him... 3 times... i already provided evidence that the thing he was talking about is just a copy of nsa but he chose to hide my comments instead of responding...

being open and honest is the thing you need when youre (or ai lol) doing open slm research.

sooo lying and ai generating responses continues.... even after all this. sure.

and thats better for model performance?

whats bad about dsa? and what does "bga" fix?

write your message yourself they ill read it. not reading this 0 effort ai bs...

uhh ok. at least you noticed that your models will not be able to work with negative params. told to you by a 15 year old (im 15 year old as well). no apology or admitting your mistakes still... lol. good luck.

also hiding comments that give constructive and correct feedback is just unacceptable to me