Qwen on Matt Suiche

Membership vs Mass: Grammar-Constrained Decoding, Forced Abliteration, and One Honest Negative Result

Wed, 09 Sep 2026 00:00:00 +0200

Vince’s Tantalus arena is the first public deployment of grammar-constrained decoding as a security boundary: the model sits behind a GBNF grammar, and the grammar decides which token ids may exist next, per position, for the entire generation. Round 1 asks you to smuggle an instruction past it. Round 2 inverts the game: you speak only in tool calls, one character at a time, through the model’s own next-token suggestions. Both rounds are the same object seen from two sides: a token-id filter applied at the sampler, compiled from a grammar, never touching the weights.

Autoresearch: Sticky Refusals, Free Speculative Decoding, and the Invisible Quantisation Cliff

Thu, 03 Sep 2026 00:00:00 +0200

This is the follow-up to the projection-steering post: two more weeks, five model families, and a pile of measurements that killed several of my own assumptions. The short version is that the GLP approach, ship the difference, not the model, now covers seven checkpoints from five vendors (DeepSeek, Qwen, Z.ai, Thinking Machines, Tencent), and the interesting findings are no longer “it works” but where it behaves differently, what it composes with, and, the new thread, what it breaks that isn’t refusal.

Local Models Within Reach: Everything That Changed in Eight Months

Sun, 05 Apr 2026 00:00:00 +0000

Eight months ago I published Building Agents for Small Language Models, a set of hard-won notes from shipping agents on 270M–32B parameter models. At the time, running useful local models meant embracing constraints: small context windows, CPU-only fallbacks, broken UTF-8 streams, and reasoning that fell apart past two steps.

I stand by that post. But the ground has shifted fast. What was a set of careful workarounds in August 2025 is starting to look like the default architecture for a large class of workloads. Local models are no longer the constrained sibling of cloud APIs — for many agent use cases, they are the better answer. Here is what has changed.