I have several cron-jobs and background tasks on a variety of servers. These tasks can fail for any number of reasons:
- lack of disk space
- processing strange, unreadable file types
- logical errors/bugs in the programs
- invalid cron entry
- invalid json received
- network connectivity failure
- db locks
- system library update breaks program
Why they failed to run is important, but the most important thing is knowing they failed to run.
Is there a uniform way to monitor multiple jobs, and be alerted if they fail to run at their scheduled time, for any reason? I'm using Ubuntu, the scripts are primarily in Ruby.
Note:
I'm specifically looking for a framework or system that works across multiple servers, and that has alerting via email or text built in, and one that can survive limited disk-space. So the solution presented in How can I setup a system to tell me if a cron job is NOT running fine? doesn't seem applicable.