Hoglet-33/3.33BT-Pre-Training-Mix-Three-Of-Three
Viewer • Updated • 3.9M • 189
🔄 Pebble 50M in training
Ash
Hoglet-33
AI & ML interests
Open source AI, datasets, parameter efficiency, SLMs, AI for the betterment of humanity. Contact at [email protected]
Recent Activity
updated a Space less than a minute ago
basically-ai/README liked a model about 4 hours ago
basically-experimental/Notio-3.7M-RNN-v1 repliedto their post about 20 hours ago
Pebble-25M and Pebble-25M-Chat are out now!
We’re excited to release Pebble-25M and Pebble-25M-Chat!
Both models use our 3:1 Mamba2/Transformer hybrid architecture and were pretrained on 25B tokens. Pebble-25M-Chat was then further fine-tuned on an additional 250M tokens from smol-smoltalk, following the same approach used for the Pebble-10M models.
We hope you enjoy experimenting with them!
Pebble-50M is coming in a few days.
Models
Pebble-25M: https://huggingface.co/basically-ai/Pebble-25M
Pebble-25M-Chat: https://huggingface.co/basically-ai/Pebble-25M-Chat
Pebble-10M GGUFs
In case you missed it, our friend @ContextReq made GGUF versions of the Pebble-10M models:
https://huggingface.co/ContextReq/Pebble-10M-GGUF
https://huggingface.co/ContextReq/Pebble-10M-Chat-GGUF
Follow us if you don’t want to miss future releases and updates!
@Hoglet-33
https://huggingface.co/basically-ai
https://huggingface.co/basically-experimentalOrganizations
