This tweet from Awni Hannun demonstrates in one line of MLX code the nondeterminism phenomenon detailed in Thinking Machines’ research. We will explore the PyTorch equivalent that reveals a fundamental issue in AI systems, because I’ve found that tweet extremely helpful to understand what the original blogpost was about. Here's a one-line code summary in MLX of the @thinkymachines blog post on non-determinism in LLM inference. I'd guess the difference is larger the lower the precision, as you get larger affects from non-associativity of FP math. Interestingly, that implies that training at low… pic.twitter.com/jYcDK9GiLn

The landscape of AI agents has been dominated by large language models (LLMs) like GPT-4 and Claude, but a new frontier is opening up: lightweight, open-source, locally-deployable agents that can run on consumer hardware. This post shares internal notes and discoveries from my journey building agents for small language models (SLMs) – models ranging from 270M to 32B parameters that run efficiently on CPUs or modest GPUs. These are lessons learned from hands-on experimentation, debugging, and optimizing inference pipelines.

The Genesis: When Signatures Aren’t Enough 🔗In the world of mobile security research, there’s a recurring frustration that keeps many of us up at night: the most sophisticated exploits - the ones that really matter - are rarely shared. When Citizen Lab and Google TAG discover NSO Group’s latest 0-click exploits targeting journalists and activists, we get brilliant technical writeups, CVE numbers, and patches. What we don’t get? The actual samples. This isn’t a criticism - there are excellent reasons for limiting access to weaponized exploits. But it creates a fundamental problem: How do you protect against threats you’ve never seen?