Inkling on Two DGX Sparks: The Only vLLM Lane, and the Two Walls Behind It

Fri, 04 Sep 2026 23:00:00 +0200

Thinking Machines shipped Inkling-Small at the end of July: a ~300B-total, ~10B-active hybrid MoE (short-conv plus relative-bias attention), natively multimodal, 1M context, and - the part that matters for homelab hardware - released as NVFP4 from day one. 170.7 GB of weights. Two DGX Sparks hold 243 GB of unified memory. You can see where this is going.

For five weeks, though, every Spark recipe for it - drowzeys’ champion image, MiaAI’s wrapper, and the half-dozen forks - ran on SGLang with a custom-baked image, for a simple reason: SGLang had Inkling support in early August, and vLLM only gained it this week, in v0.28.0. The engine we build everything on could not even load the model before Monday.

Autoresearch: Sticky Refusals, Free Speculative Decoding, and the Invisible Quantisation Cliff

Thu, 03 Sep 2026 00:00:00 +0200

This is the follow-up to the projection-steering post: two more weeks, five model families, and a pile of measurements that killed several of my own assumptions. The short version is that the GLP approach, ship the difference, not the model, now covers seven checkpoints from five vendors (DeepSeek, Qwen, Z.ai, Thinking Machines, Tencent), and the interesting findings are no longer “it works” but where it behaves differently, what it composes with, and, the new thread, what it breaks that isn’t refusal.

NVFP4 on Matt Suiche

Inkling on Two DGX Sparks: The Only vLLM Lane, and the Two Walls Behind It

Autoresearch: Sticky Refusals, Free Speculative Decoding, and the Invisible Quantisation Cliff