The304network/PhysicalAI-WorldModel-Synthetic-Warehouse-Operations-Scenes-bucket
8.62 TB
Jordan Dickens
The304network
AI & ML interests
None yet
Recent Activity
updated a bucket 16 days ago
The304network/PhysicalAI-WorldModel-Synthetic-Warehouse-Operations-Scenes-bucket published a bucket 16 days ago
The304network/PhysicalAI-WorldModel-Synthetic-Warehouse-Operations-Scenes-bucketOrganizations
None yet
updated a
bucket 16 days ago
published a
bucket 16 days ago
reacted to Avifenesh's post with 🔥 16 days ago
Post
2,188
I added a small gate to the Gated DeltaNet layers of Qwen3.5 and trained it with GRPO under a length penalty.
On Qwen3.5-4B it reasons 48% shorter than LoRA on MATH-500, and anytime accuracy goes from 0.786 to 0.852. A LoRA with about the same parameter count barely moves.
The gate sits on beta, the per-token write strength. One matrix per layer, zero init, so it starts as a no-op.
It started as a negative eigenvalue experiment. The negative range turned out not to matter. The gate did.
Write-up: https://huggingface.co/blog/Avifenesh/beta-gate-qwen35
Adapters: Avifenesh/gate-qwen35-6accb63081800b2f2d583b80
Code: https://github.com/avifenesh/gdn-beta-gatehttps://huggingface.co/collections/Avifenesh/gate-qwen35
On Qwen3.5-4B it reasons 48% shorter than LoRA on MATH-500, and anytime accuracy goes from 0.786 to 0.852. A LoRA with about the same parameter count barely moves.
The gate sits on beta, the per-token write strength. One matrix per layer, zero init, so it starts as a no-op.
It started as a negative eigenvalue experiment. The negative range turned out not to matter. The gate did.
Write-up: https://huggingface.co/blog/Avifenesh/beta-gate-qwen35
Adapters: Avifenesh/gate-qwen35-6accb63081800b2f2d583b80
Code: https://github.com/avifenesh/gdn-beta-gatehttps://huggingface.co/collections/Avifenesh/gate-qwen35
updated a
bucket 5 months ago
published a
bucket 5 months ago