BananaMind 2 Pro, BananaMind 2 Ultra
Banaxi PRO
Banaxi-Tech
AI & ML interests
SLMs, training from scratch, LoRA, TTS, Ternary models. AI Interpretability. BCI. Contact at [email protected]
Recent Activity
new activity 10 minutes ago
GODELEV/Rose-Medium:The models just keep getting better! new activity about 12 hours ago
BananaMind/BananaMind-2-Micro:i like ggufOrganizations
replied to their post about 12 hours ago
Yes that's the XSA refresh gate the reinjecting
replied to their post about 12 hours ago
its reinjection of original token embeddings but no XSA
reacted to their post with 🤗❤️👀🚀🔥 about 13 hours ago
Post
812
We're excited to release BananaMind 2 Micro, our smallest model yet.
It fits a compact architecture in only 2.9M parameters achieving the highest parameter efficiency on BananaMind Base Bench against comparable models.
It achieves comparable performance to GPT S2 5M and GPT S 5M at almost half the size while beating CMA 1M Mini.
BananaMind 2 Micro achieved the #1 spot on the Open SLM Leaderboard for the sub 3M category (not added yet but it achieves #1)
For the training we used Muon + the XSA Refresh Gate with a 5e-2 lr for Muon and 4e-3 for the 1D weights.
Its score on our efficiency measure is 0.326 getting the first place with Syn 2.6M on the second place scoring 0.291 and GPT S 5M at 0.235*
Check it out at BananaMind/BananaMind-2-Micro and follow us at:
@vovaRL
@DedeProGames
@Banaxi-Tech
BananaMind
Our new releases aren't stopping 🚀 August 13-14 BananaMind 2 Pro
It fits a compact architecture in only 2.9M parameters achieving the highest parameter efficiency on BananaMind Base Bench against comparable models.
It achieves comparable performance to GPT S2 5M and GPT S 5M at almost half the size while beating CMA 1M Mini.
BananaMind 2 Micro achieved the #1 spot on the Open SLM Leaderboard for the sub 3M category (not added yet but it achieves #1)
For the training we used Muon + the XSA Refresh Gate with a 5e-2 lr for Muon and 4e-3 for the 1D weights.
Its score on our efficiency measure is 0.326 getting the first place with Syn 2.6M on the second place scoring 0.291 and GPT S 5M at 0.235*
Check it out at BananaMind/BananaMind-2-Micro and follow us at:
@vovaRL
@DedeProGames
@Banaxi-Tech
Our new releases aren't stopping 🚀 August 13-14 BananaMind 2 Pro
- 7 replies
posted an update about 13 hours ago
Post
812
We're excited to release BananaMind 2 Micro, our smallest model yet.
It fits a compact architecture in only 2.9M parameters achieving the highest parameter efficiency on BananaMind Base Bench against comparable models.
It achieves comparable performance to GPT S2 5M and GPT S 5M at almost half the size while beating CMA 1M Mini.
BananaMind 2 Micro achieved the #1 spot on the Open SLM Leaderboard for the sub 3M category (not added yet but it achieves #1)
For the training we used Muon + the XSA Refresh Gate with a 5e-2 lr for Muon and 4e-3 for the 1D weights.
Its score on our efficiency measure is 0.326 getting the first place with Syn 2.6M on the second place scoring 0.291 and GPT S 5M at 0.235*
Check it out at BananaMind/BananaMind-2-Micro and follow us at:
@vovaRL
@DedeProGames
@Banaxi-Tech
BananaMind
Our new releases aren't stopping 🚀 August 13-14 BananaMind 2 Pro
It fits a compact architecture in only 2.9M parameters achieving the highest parameter efficiency on BananaMind Base Bench against comparable models.
It achieves comparable performance to GPT S2 5M and GPT S 5M at almost half the size while beating CMA 1M Mini.
BananaMind 2 Micro achieved the #1 spot on the Open SLM Leaderboard for the sub 3M category (not added yet but it achieves #1)
For the training we used Muon + the XSA Refresh Gate with a 5e-2 lr for Muon and 4e-3 for the 1D weights.
Its score on our efficiency measure is 0.326 getting the first place with Syn 2.6M on the second place scoring 0.291 and GPT S 5M at 0.235*
Check it out at BananaMind/BananaMind-2-Micro and follow us at:
@vovaRL
@DedeProGames
@Banaxi-Tech
Our new releases aren't stopping 🚀 August 13-14 BananaMind 2 Pro
- 7 replies
replied to their post about 18 hours ago
Oh xD i meant August 12-14
replied to their post about 21 hours ago
July 12-14. Right now its getting worse 😭 i hope WSD decay will fix it....
152K is latest.....
replied to ProCreations's post about 22 hours ago
me too 😭
replied to KlondikeDev's post 1 day ago
And if you want long context then add the stack v2 and finepdf but only use that if you want +8K context window
replied to KlondikeDev's post 1 day ago
I reccomended you to use
Fineweb edu
Fine phrase
Cosmopedia v2
Finemath
Npset2
Dclm
reacted to ProCreations's post with 🔥 1 day ago
Post
1857
new day, new models! Introducing the Auto series, a series of classifiers for determining if agentic tool calls are safe to run or not to prevent any harmful actions from happening. They have high performance compared to other LLMs commonly used for this task with incredible speed and small memory footprints.
ProCreations/auto-1b (recommended generally, much higher accuracy)
ProCreations/auto-0.4b (faster but worse)
Comes with datasets as well (open source ftw)!
A GitHub repo with pi extensions etc will come soon with this model.
ProCreations/auto-1b (recommended generally, much higher accuracy)
ProCreations/auto-0.4b (faster but worse)
Comes with datasets as well (open source ftw)!
A GitHub repo with pi extensions etc will come soon with this model.
- 2 replies
replied to their post 1 day ago
Today it will start train, not released.
reacted to their post with 🚀🔥 2 days ago
Post
2156
Today.
- 9 replies
reacted to KlondikeDev's post with 🔥 2 days ago
Post
2030
Boris-2 coming soon!
The Models:
Boris-2-75M: Trained on 26B tokens -- estimated to start training on August 12th.
Boris-2-125M: Trained on 90B tokens -- Estimated to start training on August 20th.
Boris-2-250M: Trained on 60B tokens -- Estimated to start training on September 10th.
Why does 125M get more tokens than 250M?
Well, the straight answer is time. It saves time, while still allowing the 250M model to exceed the 125M model.
Furthermore, we are attempting a unique architecture and layering scheme to hopefully end up around the strength of SmolLM2-135M. Fingers crossed!
We hope to end up in the ballpark of AxiomicLabs/GPT-X2.5-135M or BananaMind/BananaMind-2-Pro-Preview
Following this, we will release the Pro, Instruct and Pro-Instruct variants. More info will be coming soon!
The Models:
Boris-2-75M: Trained on 26B tokens -- estimated to start training on August 12th.
Boris-2-125M: Trained on 90B tokens -- Estimated to start training on August 20th.
Boris-2-250M: Trained on 60B tokens -- Estimated to start training on September 10th.
Why does 125M get more tokens than 250M?
Well, the straight answer is time. It saves time, while still allowing the 250M model to exceed the 125M model.
Furthermore, we are attempting a unique architecture and layering scheme to hopefully end up around the strength of SmolLM2-135M. Fingers crossed!
We hope to end up in the ballpark of AxiomicLabs/GPT-X2.5-135M or BananaMind/BananaMind-2-Pro-Preview
Following this, we will release the Pro, Instruct and Pro-Instruct variants. More info will be coming soon!
- 4 replies