Writing
Notes from finding and fixing tool-boundary bugs in AI agents — including the results that went against what I expected. Each post links to the code and the raw output it describes.
- AI safety — positions held, and the evidence for each
The three things LASR Labs screens for in Round 1 are: something concrete I built that showed engineering skill; evidence of doing research and thinking carefully about what came out; and enough…
- My own benchmark said 1.000. Real code said 0.14.
A week ago I wrote about a labelled corpus that found a false positive in my own scanner and took its precision from 0.750 to 1.000. That number was true, and it was also misleading, because I wrote…
- My checks were green. They were checking 40 of 85 files.
I maintain a repository whose entire job is to check that my public claims are still true. It has a green badge, a weekly schedule, and a self-test that proves the checker can fail.
- Nineteen findings, four of them wrong, four different ways
I took the static detector I have been building for agent tool-boundary bugs and pointed it at 14 real agent frameworks — codebases written by other people, for other reasons, with no idea a detector
- Two true facts I did not join up: an IPv6 loopback bypass in SunshineCTF's SiteCheck
I spent six rounds on one 463-point web challenge getting the answer wrong, and the reason is worth more than the flag. Twice I measured something true, drew a correct conclusion from it, and then…
- It learned where to put the value, and not how hard to hold it
My own README carried a criticism I could not answer. The agent loop's working memory is a selective state-space recurrence, and the part that decides what to write was hand-set:
- Six numbers I nearly published, and the control that stopped each one
I spent a weekend building a selective state-space model — Mamba's S6 recurrence — from scratch, to answer a question I actually wanted answered: is the non-attention architecture better, or does it…
- A trust index graded my repos as MCP servers. None of them is one.
I got a cold email: an MCP "trust index" had listed one of my repositories, where developers check servers before connecting them. Three free things I could do — claim it, add a badge, connect a…
- The zero that wasn't: how my own adapter made a working tool look broken
I spent three passes convinced I had found that an independent MCP scanner detected none of the defects in my corpus. The scanner was working the whole time. The fault was in the twelve lines of glue
- A benchmark found a bug in my own detector — 0.750 to 1.000
I built a labelled corpus of agent tool-boundary bugs to stop myself making claims about detectors that I had not measured. The first thing it did was find a false positive in my own published…
- A number nothing recomputes
Over a few days of work I found ten wrong numbers in things I had already published. Not ten typos — ten distinct facts, each of which had been true when written, on a CV, a profile, a hire page, a…
- Four checks that ran on data something else had already rewritten
I spent a day auditing my own security tools and found four vulnerabilities in them. Not four unrelated bugs — four instances of one shape: in every case the check ran against data that something…
- The check that could not see what it was checking
I spent a day auditing my own security tools and found four vulnerabilities. Then I audited the tooling that verifies my work, and found the same defect three more times — including once in a…
- Structured output made my safety evaluator less safe
I spent a month building a policy evaluator for autonomous agents: a small model that reads a proposal and decides whether a written policy allows it. The obvious engineering move was to constrain…
- The crash that wasn't: how AddressSanitizer showed my fix was for the wrong bug
I opened a pull request against google/s2geometry claiming a null-pointer dereference caused a crash. I had a test. The test passed on my machine. It segfaulted in CI on every platform.
- Null Origin CTF 2026 — Qualifier Writeup
Team: roamers • Event: Null Origin CTF 2026 — Qualifier • Focus: Reverse engineering chain, web, and the methodology that got us there