Local Models Within Reach: Everything That Changed in Eight Months

Sun, 05 Apr 2026 00:00:00 +0000

Eight months ago I published Building Agents for Small Language Models, a set of hard-won notes from shipping agents on 270M–32B parameter models. At the time, running useful local models meant embracing constraints: small context windows, CPU-only fallbacks, broken UTF-8 streams, and reasoning that fell apart past two steps.

I stand by that post. But the ground has shifted fast. What was a set of careful workarounds in August 2025 is starting to look like the default architecture for a large class of workloads. Local models are no longer the constrained sibling of cloud APIs — for many agent use cases, they are the better answer. Here is what has changed.

The Hidden Math Bug That Makes AI Unpredictable

Sun, 14 Sep 2025 00:00:00 +0200

This tweet from Awni Hannun demonstrates in one line of MLX code the nondeterminism phenomenon detailed in Thinking Machines’ research. We will explore the PyTorch equivalent that reveals a fundamental issue in AI systems, because I’ve found that tweet extremely helpful to understand what the original blogpost was about.

Here's a one-line code summary in MLX of the @thinkymachines blog post on non-determinism in LLM inference.

I'd guess the difference is larger the lower the precision, as you get larger affects from non-associativity of FP math.

Interestingly, that implies that training at low… pic.twitter.com/jYcDK9GiLn

MLX on Matt Suiche

Local Models Within Reach: Everything That Changed in Eight Months

The Hidden Math Bug That Makes AI Unpredictable