Appreciate it, but efficiency wasn't the point, and the post already framed it the way you did: a research LM whose channel-mixing block isn't an MLP, just a few differentiable steps of colony growth read back into the hidden state. The only thing I was testing is whether a weird dynamical system can carry the channel-mixing role and still learn language under a fixed pipeline. "Curiosity or cheaper mixer" is a great question, just a different experiment than the one I ran.
That said, I measured your number: active-channel fan-out (Jacobian participation ratio, /768) was ~469 for the growth block vs ~489 for a matched dense MLP, at 3.86M vs 3.54M params and ~1.07x MACs/token. So you're right, it densifies to roughly MLP cost. The growth math is cheap (O(NΒ·F)), but it's bracketed by dense projections, so it's a hair more expensive, not less. Curiosity, plainly. The version that'd make your point true is a sparsified readback (each tip writing to a local slice of channels, or top-k tip-to-channel routing) so fan-out drops well under d_model. That one I'd like to run. Thanks for pushing on it.
