Classic devops trick. Schedule your jobs at prime intervals - this way it's much less likely to get concentrated waves of traffic at 0:00, or 12:00, or in exact 15-minute increments.
back
6 comments
Excellent advice. Good for programming in general when it comes to sleeps or waits and such. If you have multiples of 10s everywhere when an error happens every, say, 10 or 20 seconds, it might be hard to tell where it's coming from. But if you make them relatively prime to each other, you can can easily tell: "Oh it happens every 37 seconds? I know exactly what that is!"
Thia avoids collisions as long as the process takes less than one unit of time to complete.
If not, processes that take more than one unit will overlap eventually[0]
[0] Advent of code 2020, day 13. https://adventofcode.com/2020/day/13
Addendum: Don't just pick any prime numbers. The Babylonians devised our time system because it's highly composite or 'antiprime'.
So obvious primes such as 2, 3 and 5 won't do.
Wow. This one simple trick actually just made my day. Cheers.
It would be kind of cool if cron could incorporate this sort of trick.
Like have a deterministic way of starting stuff that is not urgent, but has to be done in a certain window.
for example, I have cron jobs that run every night, and I run them at times like 2:13 am and 2:27 am.
it would be nice to say: run this at 2:<prime> at night, different every night
It would be nicer to say: run these groups of things at night, in order, but at randomized non-overlapping times or similar
Many cron systems have this functionality. At least anacron and cronie (RHEL) supports the RANDOM_DELAY variable. Check your man 5 crontab.
There's also systemd timers which can replace cron jobs with RandomizedDelay.
Before that, people used to include a random sleep in the actual job.
Personally, I tend to not use these things and instead pick a random start time when provisioning the job. That way the time delta between run is constant, which is often desired.
The systems I use seem to have cron, but I will check it out. Thanks for the pointer.
Multiples of phi will do this even better.