Also G1s training takes 7 hours to schedule lol, i started it once before school and it only started 3 hours after
🏗️ Building on HF
Boning Cui
Bc-AI
AI & ML interests
He/Him. I like LLM's and VLM's. I work with my other friends to make stuff. We are in year 7 and we are enthusiastic about AI. We are based in Australia 🇦🇺
Recent Activity
repliedto their post about 5 hours ago
Hello everyone!
Me and the team are working on G1-MINI and G1. Right now, G1-MINI is aimed at a launch in mid to late September, depending on how fast we fix the minor issues.
As for G1, it's looking like a late October to mid-November launch, based on current trajectory. If things go terribly wrong, we could postpone it to December, as we prefer to ship confidently, not ship a half-done dogs' breakfast of a model. 🤣
All dates could be changed at any moment, as we are high school students not full-time ML engineers 😅.
Other things to look out for is an overhaul of the UI and the information on my website. Thanks to my beta testers: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations reacted to theirpost with 🔥 about 5 hours ago
Hello everyone! A small update on things:
1. G1 series status. G1 is training nicely, and the loss is dropping nicely. The metrics are publicly available and i made a small space you can use to see the nice graphs: https://huggingface.co/spaces/hugging-science/Loss-Plot-G1-Large
G1-MINI is a lot slower in converging for reasons unknown yet, but we are investigating it.
2. I have built a small chat app for open SLMs here: https://huggingface.co/spaces/ml-intern-explorers/slm-arena
Feel free to add your models in a pull request!
That's all for now, early G1 versions will be available for beta testers soon. Thanks to our beta testers: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations posted an update about 5 hours ago
Hello everyone!
Me and the team are working on G1-MINI and G1. Right now, G1-MINI is aimed at a launch in mid to late September, depending on how fast we fix the minor issues.
As for G1, it's looking like a late October to mid-November launch, based on current trajectory. If things go terribly wrong, we could postpone it to December, as we prefer to ship confidently, not ship a half-done dogs' breakfast of a model. 🤣
All dates could be changed at any moment, as we are high school students not full-time ML engineers 😅.
Other things to look out for is an overhaul of the UI and the information on my website. Thanks to my beta testers: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreationsOrganizations
replied to their post about 5 hours ago
reacted to their post with 🔥 about 5 hours ago
Post
2238
Hello everyone! A small update on things:
1. G1 series status. G1 is training nicely, and the loss is dropping nicely. The metrics are publicly available and i made a small space you can use to see the nice graphs: hugging-science/Loss-Plot-G1-Large
G1-MINI is a lot slower in converging for reasons unknown yet, but we are investigating it.
2. I have built a small chat app for open SLMs here: ml-intern-explorers/slm-arena
Feel free to add your models in a pull request!
That's all for now, early G1 versions will be available for beta testers soon. Thanks to our beta testers: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations
1. G1 series status. G1 is training nicely, and the loss is dropping nicely. The metrics are publicly available and i made a small space you can use to see the nice graphs: hugging-science/Loss-Plot-G1-Large
G1-MINI is a lot slower in converging for reasons unknown yet, but we are investigating it.
2. I have built a small chat app for open SLMs here: ml-intern-explorers/slm-arena
Feel free to add your models in a pull request!
That's all for now, early G1 versions will be available for beta testers soon. Thanks to our beta testers: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations
posted an update about 5 hours ago
Post
25
Hello everyone!
Me and the team are working on G1-MINI and G1. Right now, G1-MINI is aimed at a launch in mid to late September, depending on how fast we fix the minor issues.
As for G1, it's looking like a late October to mid-November launch, based on current trajectory. If things go terribly wrong, we could postpone it to December, as we prefer to ship confidently, not ship a half-done dogs' breakfast of a model. 🤣
All dates could be changed at any moment, as we are high school students not full-time ML engineers 😅.
Other things to look out for is an overhaul of the UI and the information on my website. Thanks to my beta testers: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations
Me and the team are working on G1-MINI and G1. Right now, G1-MINI is aimed at a launch in mid to late September, depending on how fast we fix the minor issues.
As for G1, it's looking like a late October to mid-November launch, based on current trajectory. If things go terribly wrong, we could postpone it to December, as we prefer to ship confidently, not ship a half-done dogs' breakfast of a model. 🤣
All dates could be changed at any moment, as we are high school students not full-time ML engineers 😅.
Other things to look out for is an overhaul of the UI and the information on my website. Thanks to my beta testers: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations
- 2 replies
@Hoglet-33 is it possible to run pebble on cpu? As when I tried it says it need mamba and the conv1d library’s which don’t install where there isn’t CUDA. Is there a workaround as I really want to try it lol
Oh nice! Super long context SLMs will be cool
replied to Banaxi-Tech's post about 10 hours ago
@Banaxi-Tech why 2 years?
nice!
replied to Banaxi-Tech's post about 17 hours ago
@Banaxi-Tech is Luna Labs your new thing?
posted an update 1 day ago
Post
2238
Hello everyone! A small update on things:
1. G1 series status. G1 is training nicely, and the loss is dropping nicely. The metrics are publicly available and i made a small space you can use to see the nice graphs: hugging-science/Loss-Plot-G1-Large
G1-MINI is a lot slower in converging for reasons unknown yet, but we are investigating it.
2. I have built a small chat app for open SLMs here: ml-intern-explorers/slm-arena
Feel free to add your models in a pull request!
That's all for now, early G1 versions will be available for beta testers soon. Thanks to our beta testers: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations
1. G1 series status. G1 is training nicely, and the loss is dropping nicely. The metrics are publicly available and i made a small space you can use to see the nice graphs: hugging-science/Loss-Plot-G1-Large
G1-MINI is a lot slower in converging for reasons unknown yet, but we are investigating it.
2. I have built a small chat app for open SLMs here: ml-intern-explorers/slm-arena
Feel free to add your models in a pull request!
That's all for now, early G1 versions will be available for beta testers soon. Thanks to our beta testers: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations
@Hoglet-33 @Banaxi-Tech Also dont put too much trust into my opinion i am not very experienced in the SLM field lol. also anything i said was not meant in an offence way just to be clear.
reacted to Banaxi-Tech's post with 🤗🔥 2 days ago
Post
2781
We're excited to release BananaMind 2.1 Pico Preview!
It includes the first preview of our BananaMind 2.1 architecture!
This model gets near BananaMind 2 Micro performance at half the parameters and 37.5x less tokens!
Thats insane!
The current architectures includes about 500K parameters of the total 1.5M parameters in n-gram embeddings and the layer 2 is run twice.
It also includes XSA and the XSA refresh gate.
We're still going to improve the architecture in the final release.
Check it out at:
Follow us for more models:
BananaMind
@Banaxi-Tech
@vovaRL
@DedeProGames
It includes the first preview of our BananaMind 2.1 architecture!
This model gets near BananaMind 2 Micro performance at half the parameters and 37.5x less tokens!
Thats insane!
The current architectures includes about 500K parameters of the total 1.5M parameters in n-gram embeddings and the layer 2 is run twice.
It also includes XSA and the XSA refresh gate.
We're still going to improve the architecture in the final release.
Check it out at:
Follow us for more models:
@Banaxi-Tech
@vovaRL
@DedeProGames
- 3 replies
replied to Banaxi-Tech's post 2 days ago
Damn nice!
replied to KlondikeDev's post 2 days ago
lol
oh you can host one on huggingface zerogpu like use gradio.server thingy expose restAPI or smth. then you can use your huggingface token to call it. its free
what model powers that
i should make my own bot too lol. just less mean
oh lol
for small models should the vocab also be small? because for my prototypes i used my 131K vocab one lol huge embedding
hmm the dude above is not wrong. 2K vocab is a bit small for a chat LLM. @Hoglet-33 Maybe for the next model consider increasing the vocab size a bit unless you know the tokenizer can very effectively split the text, or you could be wasting context :)