I have a process that fails regularly & sometimes starts duplicate instances..

When I run: ps x |grep -v grep |grep -c "processname" I will get: 2 This is normal as the process runs with a recovery process..

If I get 0 I will want to start the process if I get: 4 I will want to stop & restart the process

What I need is a way of taking the result of ps x |grep -v grep |grep -c "processname"

Then setup a simple 3 option function

ps x |grep -v grep |grep -c "processname"
if answer = 0 (start process & write NOK & Time to log /var/processlog/check)
if answer = 2 (Do nothing & write OK & time to log /var/processlog/check)
if answer = 4 (stot & restart the process & write NOK & Time to log /var/processlog/check)

The process is stopped with killall -9 process The process is started with process -b -c /usr/local/etc

My main problem is finding a way to act on the result of ps x |grep -v grep |grep -c "processname".

Ideally, I would like to make the result of that grep a variable within the script with something like this:

process=$(ps x |grep -v grep |grep -c "processname")

If possible.

share|improve this question
    
What doesn't work with what you propose? It looks like the right syntax. (Though you might want to use pgrep if you have that.) – Mat Nov 23 '13 at 13:15
    
What you are doing is very fragile as the results of grep can be influenced even by unprivileged users. You should have a look at monitd or at least list only roots processes (or the daemon user) – hek2mgl Nov 23 '13 at 13:17
    
OK, I tried ps aux | grep "[p]rocess" but the result was more complicated than ps x |grep -v grep |grep -c "process"` so I am not too sure how to interpret the printout & use it in my script.. In what scenario could the original grep be fragile? I will be the only user ever logged into this server.. Also, how would I incorporate "list only roots processes (or the daemon user)" in the grep?? Thankyou for your time – linuxnoob Nov 23 '13 at 13:36
up vote 57 down vote accepted

Here is a script I use to monitor if a process on a system is running.
Script is stored in crontab and runs once every minute.

#! /bin/bash

case "$(pidof amadeus.x86 | wc -w)" in

0)  echo "Restarting Amadeus:     $(date)" >> /var/log/amadeus.txt
    /etc/amadeus/amadeus.x86 &
    ;;
1)  # all ok
    ;;
*)  echo "Removed double Amadeus: $(date)" >> /var/log/amadeus.txt
    kill $(pidof amadeus.x86 | awk '{print $1}')
    ;;
esac

0 If process is not found, restart it.
1 If process is found, all ok.
* If process running 2 or more, kill the last.


A simpler version. This just test if process is running, and if not restart it.
It just tests the exit flag $? from the pidof program. It will be 0 of process is running and 1 if not.

#!/bin/bash
pidof  amadeus.x86 >/dev/null
if [[ $? -ne 0 ]] ; then
        echo "Restarting Amadeus:     $(date)" >> /var/log/amadeus.txt
        /etc/amadeus/amadeus.x86 &
fi
share|improve this answer
    
OK, I will test this and report back, The wild card might be usefull also in my case because there could be (3,5,6,7...) process's running and all of those results would be a problem. Will the * in your script mean all instances of the process other than 1 or 2? – linuxnoob Nov 23 '13 at 13:40
    
Yes, corrected my comment to reflect that. You should also try to use pidof instead of ps – Jotne Nov 23 '13 at 14:01
    
I think you should additionally check who is the owner of the process – hek2mgl Nov 23 '13 at 14:43
    
Not needed, since for this there always would be one owner and one process. – Jotne Nov 23 '13 at 21:07
    
Very nice! Not sure if it's suitable for you, but depending on what linux you're working with, you might have specific tools to achieve a nice process monitoring (for instance, procd for OpenWRT, daemontools, etc.) – Pipetus Jan 24 '17 at 15:38

I adopted the @Jotne solution and works perfectly! For example for mongodb server in my NAS

#! /bin/bash

case "$(pidof mongod | wc -w)" in

0)  echo "Restarting mongod:"
    mongod --config mongodb.conf
    ;;
1)  echo "mongod already running"
    ;;
esac
share|improve this answer

I have adopted your script for my situation Jotne.

#! /bin/bash

logfile="/var/oscamlog/oscam1check.log"

case "$(pidof oscam1 | wc -w)" in

0)  echo "oscam1 not running, restarting oscam1:     $(date)" >> $logfile
    /usr/local/bin/oscam1 -b -c /usr/local/etc/oscam1 -t /usr/local/tmp.oscam1 &
    ;;
2)  echo "oscam1 running, all OK:     $(date)" >> $logfile
    ;;
*)  echo "multiple instances of oscam1 running. Stopping & restarting oscam1:     $(date)" >> $logfile
    kill $(pidof oscam1 | awk '{print $1}')
    ;;
esac

While I was testing, I ran into a problem.. I started 3 extra process's of oscam1 with this line: /usr/local/bin/oscam1 -b -c /usr/local/etc/oscam1 -t /usr/local/tmp.oscam1 which left me with 8 process for oscam1. the problem is this.. When I run the script, It only kills 2 process's at a time, so I would have to run it 3 times to get it down to 2 process..

Other than killall -9 oscam1 followed by /usr/local/bin/oscam1 -b -c /usr/local/etc/oscam1 -t /usr/local/tmp.oscam1, in *)is there any better way to killall apart from the original process? So there would be zero downtime?

share|improve this answer

Use Systemctl in linux. It is new way to monitor systemD services and Units. Learn more here https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.