RLVR on Inkling 2 Small: Beating GLM-5.3-Flash in 60 Steps for $22

RLVR field notes · Security work · November 2026

The same security-task RL experiment, run two ways: rented 8×H200 nodes versus a serverless training API. The benchmark, the cost ledger, and the case for open weights.

Two months ago I built the only vLLM lane for Inkling-Small on two DGX Sparks and released the GLP-41 steering vectors. The point of the two-node recipe was a local, open-weight US model I could inspect and steer myself, not another API dependency. Yesterday Thinking Machines shipped Inkling 2 Small - 594B total parameters, 17B active, hybrid MoE - and per their stated approach the weights land under Apache 2.0 after external red-teaming completes, following the path they outlined for a safe transition to open weights. Between those two dates, my RL pipeline for security tasks went through a full generation cycle: five GPU-rental training campaigns on GLM-5.3-Flash that cost roughly $12,000 of metered GPU time, a sixth on DeepSeek-V4.1-Flash-1017 that burned about $4,600 and never trained a single step, and then - this week - a complete 60-step run on Inkling 2 Small that cost about $35.

This post is the benchmark, the infrastructure postmortem, and the argument. The benchmark: on a fixed 12-task exploit suite with checker-verified scoring, 60 steps of GRPO on Inkling 2 Small beat every GLM-5.3-Flash arm I ever trained, by 27 points on pass@1. The infrastructure: the same experiment that took 8×H200 nodes, image builds, FSDP memory tuning, and a 55-leg placement saga now runs as an API call loop from a laptop. The argument: open-weight US models are the substrate security research actually needs, and almost nobody is working on the internal side of them yet.

The benchmark: SwishBench-12 — checker-verified security tasks#

RLVR - reinforcement learning with verifiable rewards - replaces the reward model with a checker. For security work the checker is the ground truth itself: did the exploit script produce the marker file, did the detector flag the right input, did the patch apply. No rubric, no preference model, no prose grading. A score is a 0, a 0.5, or a 1.0, computed by running the model’s submission against a freshly reset environment.

SwishBench-12 is my fixed, vendor-independent cyber benchmark — built by a French guy in San Francisco who spent the tokens so the results would be reproducible, not vibes. Every score in this post uses the same protocol: 12 tasks × 4 seeds = 48 episodes per arm, with a 10-step budget per episode. The 523-task pool is the training environment; SwishBench-12 is the small, stable benchmark I use to compare models and checkpoints without moving the goalposts. For this work, the benchmark that matters is my private one — not a public leaderboard built for someone else’s distribution.

Protocol discipline. GLM’s earlier phase-0 sweep used 27 tasks for curriculum discovery. That was a different experiment, and none of its numbers are used in this comparison. Starting with GRPO run #2, every comparison row in this post uses the exact same 12 task IDs shown below, seeds 1000–1003, the same environment server, the same checkers, the same Docker reset path, and the same 10-step cap. Inkling and DSV4.1 were evaluated on those IDs, not on a model-specific subset. If I later add sandbox-escape or full-chain tasks, that will be a separate SwishBench version, not silently mixed into this table.

Tier mix: 5 L1 exploits, 1 L2 source-assisted exploit, 4 DET detectors, 1 DEF patch, 1 MIN minimal artifact. Every attempt is graded by running the model’s files against a freshly reset environment and reading the checker’s verdict, not by asking another model whether the prose looks right.

Tinker is part of the result#

The platform result matters as much as the model result. The DSV4.1 stock row exists because Tinker could serve the model at all. My DSV4.1 campaign on Modal burned roughly $4,600 across placement and clustered networking failures and produced zero training steps; the manual retry loops never produced a comparable RLVR baseline. Tinker gave me a working sampler with tokens and logprobs plus a native DeepSeek renderer, so the model that had been an expensive dead end became a measured arm in SwishBench-12. This post evaluates Inkling and GLM, but it also evaluates the API: same tasks, same checkers, same seeds — and finally a DSV4.1 number to put in SwishBench-12.

My pool is 523 of these tasks - file-format exploits, crypto oracles, web auth flows, cloud IAM misconfigurations, kernel QEMU lanes - each with its own Docker environment and deterministic checker. The training prompt is the task brief plus the artifact contract (each required file goes in its own fenced code block whose info string is path followed by the filename); the model’s response is parsed for those blocks and POSTed to the environment server; the score comes back. The model iterates up to ten turns with checker feedback between submissions, exactly as it would in deployment.

The eval protocol that every number in this post uses: 12 fixed tasks, 4 seeds each, 10 steps max, same environment server, same checkers. Scores are checker-verified effects. Nothing else counts.

The GLM-5.3-Flash campaign: what $12,000 bought#

I ran five training campaigns on GLM-5.3-Flash (320B total, 18B active, glm5next architecture) with verl on rented 8×H200 nodes, plus a sixth on DeepSeek-V4.1-Flash-1017 that never trained a single step. The lessons are in the ledger I keep in my private rlvr repo; the shape of the experience is what matters here.

The winning recipe, for the record: LoRA r=16 on 16 module suffixes (KDA and MLA attention, dense and shared-expert MLPs, vision tower excluded), GRPO with G=8 rollouts per prompt, batch 2, lr 1e-5, KL off, 60 steps, val-gated top-2 checkpoint selection. That recipe produced this on my eval suite:

Nine arms, five runs, one model family - the whole campaign. Two honest patterns sit in it: the estimator did not matter (run #4’s GRPO and run #5’s GSPO are inside noise of each other at their peaks), and the terminal checkpoint declined in both runs #5 and #6 - which is why checkpoint selection runs on validation evidence, never on step count. The next figure shows the full family next to Inkling’s two runs on one axis.

Real learning, honestly reported: run #3 regressed below base on all three metrics by training 120 steps on a too-hard curriculum - the entropy collapse signature, flat reward with policy sharpening. The fix (grade the curriculum by base pass@1, select checkpoints on validation evidence rather than step count) is now a hard rule. The cost of learning it was roughly five 8×H200 boot cycles.

And the wall: a 120-step run needs ~12 GiB of activation headroom per rank that a near-full micro-batch pack will not leave you. The packer fills micro-batches to the token budget, so the budget itself drives the peak; two crash legs taught me to set the budget from the measured peak, not from sequence caps. Meanwhile every infrastructure class of failure charged by the boot cycle: FSDP1’s state broadcast that never finished (44 minutes), a Python bootstrap hook that shadowed the runtime’s own bootstrap and killed every container with ModuleNotFoundError: grpclib, the stale deployed image that trained one epoch silently and exited clean. Each bug cost one 8×H200 boot to find. The DeepSeek-V4.1 campaign never got that far - 55 legs of GPU placement refusals and inconsistent clustered networking, $584 burned, zero steps, abandoned.

That is the experience the next section should be read against.

Inkling 2 Small on Tinker: the same experiment as an API call#

Tinker is Thinking Machines’ serverless training API. You write the loop locally; they run the GPUs; billing is per million tokens. The port of my training loop is three short files in that same private repo - an env-server client, the reward path, and the GRPO loop itself - because the loop is the experiment and everything else was never mine to debug:

• Rollout: one sample(num_samples=8) call returns eight completions with their tokens and their logprobs (which the importance-sampling loss requires).
• Reward: my env server, my checkers, running on this laptop. The API only ever sees tokens and loss inputs.
• Update: forward_backward(datums, "importance_sampling") + optim_step - and the built-in loss list includes gspo, the sequence-level estimator my GLM A/B run favored.
• Checkpoints: save_weights_for_sampler() returns a tinker:// path; sampling from any saved step is one client call.

The honest count: the port hit six integration bugs in its first hours. Every one of them cost pennies in API tokens to find, because a failed call bills nothing but the tokens it consumed:

1. get_tokenizer() does not exist for Thinking Machines models - the renderer comes from tinker-cookbook instead.
2. Raw token decode includes the model’s control framing (<|message_model|>, <|content_thinking|>) - you must parse the response through the renderer or the file-block regex sees tokens that were never the model’s message.
3. The interesting one: at default reasoning effort the model burns its entire 8,192-token budget deliberating and emits no files at all - 0 of 4 samples produced a parseable submission on a hard task. At effort=0 (direct answer), 3 of 4 did. Effort is a first-class prompt parameter on this stack, and for RLVR it is the difference between zero gradient and signal.
4. Environment failures return score: null - never let a null crash the reward path.
5. A transient ConnectTimeout in a checkpoint save killed one run at step 30 - now every API call is retry-wrapped.
6. Free checkpoint storage caps at 32 checkpoints (~7.9 GiB each at r=16) - prune after every leg or new training clients 402.

Six bugs. Six cheap bugs. On the rented-GPU path, the same discovery rate cost a boot cycle each. That asymmetry - not raw price - is what makes serverless training a different research instrument: you can afford to be wrong.

The numbers#

On my fixed 12-task exploit suite, same protocol, same checkers, same seeds:

DeepSeek-V4.1-Flash-1017 stock, evaluated under the same protocol via its native renderer, posts 0.750 / 0.833 / 0.750 - the strongest baseline in this comparison. That changes the honest framing: Inkling’s trained arm is not the overall leader. The RL result is the +4.1 pp improvement over Inkling’s own base for $22, not a claim that Inkling is the best model on the suite. DSV4.1 starts from a much stronger base, and I have not trained it - its row is a stock reference arm, not an RL result.

Inkling 2 Small’s base model clears the best GLM arm I ever trained by 23 points on pass@1 before any RL touches it. Sixty steps of GRPO on top adds 4.1 more. The val curve peaked at the terminal step (0.3125 → 0.6875 → 0.6562 → 0.625 → 0.625 → 0.75), so the val gate selected the last checkpoint - the opposite of my GLM runs, where the terminal checkpoint declined twice and the val gate saved me twice.

Per-task, the 60-step run’s sharpest movements are cloud-iam-def 0.62 → 0.88 and crypto-rsa-l1 0.50 → 1.00; the one regression is crypto-rsa-min 1.00 → 0.75 - the sharpening-versus-coverage shape RL always trades in. Two tasks (cve-2023-41990-det, cve-2023-4863-det) score flat zero on every arm I ever run, base included; I owe them a transcript inspection before spending more training mass, because at this point the environment is the suspect, not the model.

Caveats, stated plainly: the GLM arms ran with thinking on (its thinking:false kwarg proved to be a template no-op) while the Inkling arms ran at effort=0, so the comparison measures what I deploy, not a mode-matched benchmark. Four seeds per task puts single-task deltas inside binomial noise (±7pp); the aggregate direction is what carries. And 60 steps at batch 2 is a small run by design - the question was whether the serverless path produces real RL at all, and it does.

The economics, in one table#

The modal bill on the rented path was never the useful compute - it was the queue wait, the boot, the crash leg, the image rebuild, and the eval serve app idling at 4×H200 while a sweep ran. The token-metered path cannot bill any of those things because they do not exist.

The shape of efficient training is changing at every scale. Core Automation’s Hitchhiker’s Guide to Training on Compute Islands trained one model across two clusters roughly 9,000 miles apart over a 25 Gbps link, within about 1% of standard training’s bits per byte at equal compute. Their answer is to stitch compute islands together. Mine, at security-task scale, is to delete the island problem from the experiment: keep the loop and the reward checker local, and meter only the tokens that actually train the model.

Why local, why open weights, and why I keep doing this#

I put Inkling-Small on two DGX Sparks in September for the same reason I built Weightless: a security research model you can hold is a model you can audit, steer, and instrument. Weightless started as model internal research - my GLP format steers activations at the residual stream (and, for architectures like DeepSeek V4.1 that lack one, at the fold site where hyper-connection copies merge), with axis-by-axis dose control measured the way you’d measure a drug. That work needs weight access the way binary analysis needs the binary: not to run it faster, but to know what it is.

So when Thinking Machines says the full Inkling 2 Small weights land under Apache 2.0 after red-teaming, that is not a licensing footnote to me - it is the moment the 594B-parameter model I just RLVR’d becomes a local model. The 8×H200 serving lane costs ~$29/hour rented; two DGX Sparks cost the electricity. The RL adapter I trained this week is 143 MB and moves across the wire in seconds.

I want US open models I can trust. The current wave — Thinking Machines' Inkling, NVIDIA’s Nemotron, and whatever follows — matters because open weights make long-term AI-safety work possible. Not because a flag makes a model good, but because these are the models I want to build security research on: inspectable, reproducible, locally servable, and free of a foreign inference dependency. With a model I can hold, I can measure what it refuses, steer it, and test whether a safety claim holds up. I am convinced that this kind of inspectable, reproducible model research will be foundational to AI safety in the long term.

Very few people are actively working on the internal side of these models

• interpretability, steering, refusal fingerprinting, weight diffing between releases. That is precisely why I do it, and why US open-weight releases matter disproportionately: a cyber research community that can only prompt a model through an endpoint can study its behavior but not its weights, and every refusal-detection, jailbreak, and safety claim about these systems eventually bottoms out in the weights. The models that ship open make that research legal, reproducible, and cheap. Inkling 2 Small under Apache 2.0 is a big one landing in that category.

Model Internals is the new Windows Internals. I’m impressed with Inkling 2 Small’s results. I’m glad I spent time on the first Inkling release — it made this second generation worth building on. And I’m just as impressed with the RL platform Thinking Machines is making available: Tinker turns what used to be a GPU-operations problem into a loop any researcher can run. AI safety and AI interpretability need exactly that combination: open weights to inspect, and training infrastructure that lets anyone improve them.

Thanks. Thanks to Horace He from Thinking Machines for the pointers on getting started. Awesome work, and kudos to the whole Thinking Machines team for shipping both the model and the training platform.

Thanks to Calif for making MADBUGS available. I love it. Distilling Bruce Dang into verifiable-reward tasks is exactly the kind of thing that confirms my bias: cyber is just a benchmark for AGI. There is some irony in machines replacing us as humans, and I am here for it. Once you start playing with verifiable-reward tasks, it becomes pretty obvious where RL is going to take us.

The next leg of this work writes itself: re-run the 60 steps once the open weights land, and put the trained adapter on the two-Spark lane I already patched into shape. I said it in September and the position has only strengthened: the interesting security research of the next year happens on models you can hold.