🏗️ Building on HF

River Rider PRO

RiverRider

3 2 14

Space-Bacon

AI & ML interests

Computational semiotics is empirically proven. It takes three to tango 💃🪩🕺

Recent Activity

posted an update 1 day ago

Train Once, Read Everywhere Paper title: Train Once, Read Everywhere: Substrate Invariance of the Linearly Readable Structure in Frozen Language Models Paper URL: https://github.com/space-bacon/SRT/blob/main/arxiv_program/paper.md Repository URL: https://github.com/space-bacon/SRT The consolidated findings of the SRT research program are now available. The program treats frozen production-scale language models as substrates whose internal states carry structure that small, inspectable instruments can read. Results include: - A ~12 M-parameter adapter that surfaces per-token semiotic signals from a frozen 7 B backbone with zero cross-entropy degradation - An activation verbalizer that recovers text from single hidden states up to a calibrated paraphrase ceiling - Linear readout ports spanning dense 3 B models to 94-layer 235 B mixture-of-experts models - A 22 MB linear head that gives a frozen multimodal chat model image-to-text retrieval performance matching fully trained 2018 dual encoders on the COCO benchmark The central claim is substrate invariance. The readable structure is a stable property of the model class. A head trained once on one host reads, with no retraining and at most a 42 KB recalibration, across: - Hosts ten times smaller (31 B → 3 B) - 4-bit weight precision - Entirely different silicon and kernels (CUDA/bf16 to Apple Silicon/MLX-Q4) Deployment tiers differ in latency and cost, never in capability. All instruments, measurement protocols, invariance evidence, negative results, and artifacts are in the repository.

updated a model 1 day ago

RiverRider/srt-sunstone-linear-head

updated a model 1 day ago

RiverRider/srt-nla-gemma4-artifacts

View all activity

Organizations

Posts 13

Post

Train Once, Read Everywhere

Paper title:
Train Once, Read Everywhere: Substrate Invariance of the Linearly Readable Structure in Frozen Language Models

Paper URL:
https://github.com/space-bacon/SRT/blob/main/arxiv_program/paper.md

Repository URL:
https://github.com/space-bacon/SRT

The consolidated findings of the SRT research program are now available.

The program treats frozen production-scale language models as substrates whose internal states carry structure that small, inspectable instruments can read. Results include:

- A ~12 M-parameter adapter that surfaces per-token semiotic signals from a frozen 7 B backbone with zero cross-entropy degradation
- An activation verbalizer that recovers text from single hidden states up to a calibrated paraphrase ceiling
- Linear readout ports spanning dense 3 B models to 94-layer 235 B mixture-of-experts models
- A 22 MB linear head that gives a frozen multimodal chat model image-to-text retrieval performance matching fully trained 2018 dual encoders on the COCO benchmark

The central claim is substrate invariance. The readable structure is a stable property of the model class. A head trained once on one host reads, with no retraining and at most a 42 KB recalibration, across:

- Hosts ten times smaller (31 B → 3 B)
- 4-bit weight precision
- Entirely different silicon and kernels (CUDA/bf16 to Apple Silicon/MLX-Q4)

Deployment tiers differ in latency and cost, never in capability.

All instruments, measurement protocols, invariance evidence, negative results, and artifacts are in the repository.

Post

144

🔮 Gemma-4-31B-it SRT-Sunstone

A read-out that reads images — trained only on words.

A 12.3M side-channel head on a frozen google/gemma-4-31B-it, taught meaning from text alone — to separate discourse communities in the residual stream. It never saw a single picture in training. Hand it a picture and it names what the image means, with zero image training.

Cross-modal transfer. Give it a photo and it lands next to the right words: bicycle → bicycle, rose → flower, dog → pet. It groups images by what they mean, not how they look: image→referent kNN 0.64 against chance 0.10.

Why this is a semiotic read-out, not an image classifier. A classifier is trained on labelled images and learns a fixed map from pixels to a closed label set; it only knows the labels it was shown. This read-out is different in kind. It never saw an image in training — it was trained only on text, to separate discourse communities in the residual stream of a frozen gemma-4-31B. It can interpret a picture because gemma-4 already fuses image and word into one representational stream, and the read-out taps the shared interpretant: the meaning a sign carries, whatever form it arrived in. So it does not classify the image. It tells you what the image means to a system that learned meaning from words — a transfer across modality, from a head that was never cross-trained. That is the result.

Each picture gets two readings: the words it means — the load-bearing evidence — and the discourse it evokes, the nearest of 35 communities it learned from text. Read the second as a flavour, not a category: cars into the automotive community, deer and mushrooms into gardening, cats and dogs into the cozy-domestic communities. Never a class label.

Scored offline through a frozen google/gemma-4-31B-it (62.5 GB, too large to run live)

Try it: RiverRider/srt-sunstone
Model: RiverRider/Gemma-4-31B-it-SRT-Sunstone
Code: https://github.com/space-bacon/SRT