@BananaMindBot train GPT-X3
๐ฝ Big things for X3
Dan P
Datdanboi25
AI & ML interests
Axiomic Labs Founder
Mechatronics engineer, LLMs, Vision Models, Embedding Models
Recent Activity
liked a model about 11 hours ago
AlgoDriveAI/AIMO_EP_25 liked a model about 11 hours ago
AlgoDriveAI/Akkadian_English_DenseLLM_1BOrganizations
replied to Banaxi-Tech's post about 7 hours ago
liked 2
models about 11 hours ago
What was the peak learning rate used?
#8 opened about 13 hours ago
by Datdanboi25
reacted to their post with โค๏ธ about 15 hours ago
Post
2746
THE SLM FRONTIER ADVANCES!
bench-labs/cagliostro-v3 just hit an Intelligence Index of 26.13 on the AxiomicLabs/Open_SLM_Leaderboard a 146M-param model trained completely from scratch on a single consumer GPU. That's 2nd place overall, and as far as I can tell, the most capable SLM trained on consumer hardware to date. Beating SmolLM-135m on 1/8th of the data is just silly levels of efficiency.
Big congrats to the @BenchLabs team and specifically @TobiasLogic !
bench-labs/cagliostro-v3 just hit an Intelligence Index of 26.13 on the AxiomicLabs/Open_SLM_Leaderboard a 146M-param model trained completely from scratch on a single consumer GPU. That's 2nd place overall, and as far as I can tell, the most capable SLM trained on consumer hardware to date. Beating SmolLM-135m on 1/8th of the data is just silly levels of efficiency.
Big congrats to the @BenchLabs team and specifically @TobiasLogic !
- 6 replies
liked 2
models about 18 hours ago
Add Pollock 1.5 (127.57M, from scratch, Index 12.59)
2
#159 opened 1 day ago
by 
dawidmajewski
Update cagliostro-v3 to final weights (75B tokens)
2
#158 opened 1 day ago
by TobiasLogic
reacted to KlondikeDev's post with ๐ 1 day ago
Post
159
Important Boris-2 news:
Boris-2 is 30B out of 200B tokens in, and it is severely behind its competitors in training.
We have determined the bug to be a configuration error. Boris-2 has been in training for ~1 week, and was projected to finish on November 3rd, 2026.
We are unfortunately going to restart training, with proper configuration.
The new projected finish date is ~15-18th of November.
We apologize for the delay.
Boris-2 is 30B out of 200B tokens in, and it is severely behind its competitors in training.
We have determined the bug to be a configuration error. Boris-2 has been in training for ~1 week, and was projected to finish on November 3rd, 2026.
We are unfortunately going to restart training, with proper configuration.
The new projected finish date is ~15-18th of November.
We apologize for the delay.
- 17 replies
replied to KlondikeDev's post 1 day ago
You will NOT be forgiven for this delay!
reacted to TobiasLogic's post with ๐ฅ 1 day ago
Post
2676
Weโve been cooking something new at Bench Labs.
Introducing Cagliostro-v3, our new 146M parameter language model trained completely from scratch.
The run isnโt even finished yet.
At the current checkpoint:
โข 146M parameters
โข 72.7B / 75B tokens trained
โข 26.27 Open SLM Index
โข 43.80 ArithMark-3
โข Trained on a single RTX 5090
โข ~90K to 103K tokens/sec during training
โข ~9 days for the full run
โข Apache 2.0
For some context, SmolLM2-135M scores 27.13 on the same Index after being trained on roughly 2 trillion tokens.
Cagliostro-v3 is currently at 26.27 with only ~72.7B.
Thatโs around 27x fewer training tokens.
The model also currently Hold the number 3rd spot for ArithMark-3, scoring 43.80
This wasnโt achieved by just throwing more tokens at the model. A huge part of v3 has been figuring out architecture, data mixture, and training dynamics at this scale.
The model uses a custom 30-layer decoder architecture with grouped-query attention and cross-head subspace attenuation, SwiGLU, RMSNorm, RoPE, tied embeddings, and a warmup-stable-decay training schedule.
During cooldown we also substantially shifted the data mixture toward higher-quality synthetic textbook and mathematics data, with the mathematics share increasing from 10% to 28%.
And everything is open.
The repository contains the training history with checkpoints pushed roughly every 30 minutes, so you can inspect how the model evolved throughout training rather than only seeing the final weights.
This is still a pre-final checkpoint. We have roughly 2.3B tokens left and the learning-rate cooldown is still running.
So 26.27 isnโt the final number.
Really excited to see where the last part of the run lands.
Cagliostro-v3:
bench-labs/cagliostro-v3
Built by Bench Labs.
Open SLM Leaderboard:
AxiomicLabs/Open_SLM_Leaderboard
Introducing Cagliostro-v3, our new 146M parameter language model trained completely from scratch.
The run isnโt even finished yet.
At the current checkpoint:
โข 146M parameters
โข 72.7B / 75B tokens trained
โข 26.27 Open SLM Index
โข 43.80 ArithMark-3
โข Trained on a single RTX 5090
โข ~90K to 103K tokens/sec during training
โข ~9 days for the full run
โข Apache 2.0
For some context, SmolLM2-135M scores 27.13 on the same Index after being trained on roughly 2 trillion tokens.
Cagliostro-v3 is currently at 26.27 with only ~72.7B.
Thatโs around 27x fewer training tokens.
The model also currently Hold the number 3rd spot for ArithMark-3, scoring 43.80
This wasnโt achieved by just throwing more tokens at the model. A huge part of v3 has been figuring out architecture, data mixture, and training dynamics at this scale.
The model uses a custom 30-layer decoder architecture with grouped-query attention and cross-head subspace attenuation, SwiGLU, RMSNorm, RoPE, tied embeddings, and a warmup-stable-decay training schedule.
During cooldown we also substantially shifted the data mixture toward higher-quality synthetic textbook and mathematics data, with the mathematics share increasing from 10% to 28%.
And everything is open.
The repository contains the training history with checkpoints pushed roughly every 30 minutes, so you can inspect how the model evolved throughout training rather than only seeing the final weights.
This is still a pre-final checkpoint. We have roughly 2.3B tokens left and the learning-rate cooldown is still running.
So 26.27 isnโt the final number.
Really excited to see where the last part of the run lands.
Cagliostro-v3:
bench-labs/cagliostro-v3
Built by Bench Labs.
Open SLM Leaderboard:
AxiomicLabs/Open_SLM_Leaderboard
- 7 replies
reacted to their post with ๐ฅ 1 day ago
Post
2746
THE SLM FRONTIER ADVANCES!
bench-labs/cagliostro-v3 just hit an Intelligence Index of 26.13 on the AxiomicLabs/Open_SLM_Leaderboard a 146M-param model trained completely from scratch on a single consumer GPU. That's 2nd place overall, and as far as I can tell, the most capable SLM trained on consumer hardware to date. Beating SmolLM-135m on 1/8th of the data is just silly levels of efficiency.
Big congrats to the @BenchLabs team and specifically @TobiasLogic !
bench-labs/cagliostro-v3 just hit an Intelligence Index of 26.13 on the AxiomicLabs/Open_SLM_Leaderboard a 146M-param model trained completely from scratch on a single consumer GPU. That's 2nd place overall, and as far as I can tell, the most capable SLM trained on consumer hardware to date. Beating SmolLM-135m on 1/8th of the data is just silly levels of efficiency.
Big congrats to the @BenchLabs team and specifically @TobiasLogic !
- 6 replies
replied to their post 1 day ago
Well deserved, congrats on the incredible model!
posted an update 1 day ago
Post
2746
THE SLM FRONTIER ADVANCES!
bench-labs/cagliostro-v3 just hit an Intelligence Index of 26.13 on the AxiomicLabs/Open_SLM_Leaderboard a 146M-param model trained completely from scratch on a single consumer GPU. That's 2nd place overall, and as far as I can tell, the most capable SLM trained on consumer hardware to date. Beating SmolLM-135m on 1/8th of the data is just silly levels of efficiency.
Big congrats to the @BenchLabs team and specifically @TobiasLogic !
bench-labs/cagliostro-v3 just hit an Intelligence Index of 26.13 on the AxiomicLabs/Open_SLM_Leaderboard a 146M-param model trained completely from scratch on a single consumer GPU. That's 2nd place overall, and as far as I can tell, the most capable SLM trained on consumer hardware to date. Beating SmolLM-135m on 1/8th of the data is just silly levels of efficiency.
Big congrats to the @BenchLabs team and specifically @TobiasLogic !
- 6 replies
updated a
Space 1 day ago
Add cagliostro-v3 (146M, from scratch, Index 26.27)
#157 opened 1 day ago
by TobiasLogic
Submission: cagliostro-v3 (146M, from scratch, Index 26.27)
2
#156 opened 1 day ago
by TobiasLogic
Update README.md
#2 opened 1 day ago
by Datdanboi25
liked a
model 1 day ago
