Activation Steering on Matt Suiche

Steering a Loop: Control Vectors Meet the Looped Transformer

Sun, 06 Sep 2026 00:00:00 +0200

Nanbeige4.2-3B is the first open-weight Looped Transformer anyone can download: a 22-layer stack executed twice, 44 effective layer-passes, 3B non-embedding parameters, and benchmark numbers that embarrass dense models four times its size. It buys that capacity with repetition rather than parameters.

The lineage runs from the Universal Transformer through Huginn (a recurrent block looped a variable number of times at inference) and ByteDance Seed’s Ouro (looped computation trained into pretraining itself, with an entropy-regularized objective that learns how many iterations an input deserves), to Google’s Mixture-of-Recursions, which routes each token to its own recursion depth. On September 1st, The Information reported that OpenAI’s Astra is recurrent-depth: latent iteration in place of visible chain-of-thought, which is why the safety community spent the first week of September arguing about monitorability. OpenAI has not published architecture details, so treat the report as a strong rumor. Either way, depth has become a runtime quantity; Nanbeige4.5 is already training with the token-routed machinery in its config.

Autoresearch: Sticky Refusals, Free Speculative Decoding, and the Invisible Quantisation Cliff

Thu, 03 Sep 2026 00:00:00 +0200

This is the follow-up to the projection-steering post: two more weeks, five model families, and a pile of measurements that killed several of my own assumptions. The short version is that the GLP approach, ship the difference, not the model, now covers seven checkpoints from five vendors (DeepSeek, Qwen, Z.ai, Thinking Machines, Tencent), and the interesting findings are no longer “it works” but where it behaves differently, what it composes with, and, the new thread, what it breaks that isn’t refusal.

Autoresearch: Abliteration Without Redistributing the Model

Sun, 16 Aug 2026 00:00:00 +0200

I have been travelling too much this year to see the inside of a gym, so I am pleased to report that I am finally lifting weights again. Different weights. Considerably less cardio.

TL;DR#

If you want to change what a model refuses, the usual approach is to edit its weights and upload the result. That is what every “uncensored” checkpoint you have seen is: a full re-upload, differing from the original by a rounding error spread thinly across a few hundred matrices.