Post
314
tokenizers 0.23.3 is out!https://github.com/huggingface/tokenizers/releases/tag/v0.23.3
The release is python-only and fixes a few very annoying issues
👋 Open to Work
Simon Brandeis
sbrandeis
AI & ML interests
None yet
Recent Activity
updated a Space about 1 hour ago
sbrandeis/tokenizers-v1-compat liked a model about 13 hours ago
mistralai/Mistral-Large-4-1T-A52B upvoted a collection about 13 hours ago
Mistral Large 4Organizations
Posts 2
https://github.com/huggingface/tokenizers/releases/tag/v0.23.3
The release is python-only and fixes a few very annoying issues
Papers I want to read, at some point.
-
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
Paper • 2108.12409 • Published • 5 -
YaRN: Efficient Context Window Extension of Large Language Models
Paper • 2309.00071 • Published • 88 -
MIMIC-IT: Multi-Modal In-Context Instruction Tuning
Paper • 2306.05425 • Published • 12 -
Music ControlNet: Multiple Time-varying Controls for Music Generation
Paper • 2311.07069 • Published • 44
Papers I want to read, at some point. Focused on Reinforcement Learning papers.
-
Deep reinforcement learning from human preferences
Paper • 1706.03741 • Published • 4 -
Training language models to follow instructions with human feedback
Paper • 2203.02155 • Published • 26 -
Direct Preference-based Policy Optimization without Reward Modeling
Paper • 2301.12842 • Published -
Woodpecker: Hallucination Correction for Multimodal Large Language Models
Paper • 2310.16045 • Published • 18
Papers I want to read, at some point.
-
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
Paper • 2108.12409 • Published • 5 -
YaRN: Efficient Context Window Extension of Large Language Models
Paper • 2309.00071 • Published • 88 -
MIMIC-IT: Multi-Modal In-Context Instruction Tuning
Paper • 2306.05425 • Published • 12 -
Music ControlNet: Multiple Time-varying Controls for Music Generation
Paper • 2311.07069 • Published • 44
Papers I want to read, at some point. Focused on Reinforcement Learning papers.
-
Deep reinforcement learning from human preferences
Paper • 1706.03741 • Published • 4 -
Training language models to follow instructions with human feedback
Paper • 2203.02155 • Published • 26 -
Direct Preference-based Policy Optimization without Reward Modeling
Paper • 2301.12842 • Published -
Woodpecker: Hallucination Correction for Multimodal Large Language Models
Paper • 2310.16045 • Published • 18
spaces 6
Running
tokenizers v1 compatibility
🧩
Check tokenizers v1 compatibility with existing models
Running
Tokenmaxxing BERT
🏎
Step through huggingface tokenizers normalizers
Running
8
Tokenizers in your browser
🚀
tokenizers 1.0.0-rc.2 performance overview
Running
Quickdraw
🐢
Play a drawing game where AI guesses your sketches
Paused
MCP
OmniGen2
👀
OmniGen2: Unified Image Understanding and Generation.
No application file
Agents
Jobs Runner
🚀