Check for a Process and Start It if It’s Not Started (Linux/Unix)

I came across a conundrum some time ago. I have a CentOS Linux based VPS running a few various different ‘processes’ for different purposes. My problem was that from time to time, these processes would stop or die, for whatever reason. Given that I require absolute uptime from these processes (or at least as close as I can get) I needed a script, a way to check if the process was running, and if not, to start it, all without my manual help. Now, Before you read any further, I just want to make it clear, I am talking about uncommon processes (not system services or daemons). To name a few:

Initially, I searched and searched the internet and found all sorts of solutions dating back to the 1990’s all the way through to 2010. I even managed to cobble something together, but it was pretty terrible and failed sometimes (more often than not actually).

Some readers may have noticed that one of my examples above, already contains an almost perfectly good solution!

Services for IRC (developed primarily by Andrew Church), contains an additional script (once compiled and installed) under the file name of ircservces-chk (located here: /usr/local/sbin/ircservices-chk on most linux distributions). The contents of this file looks like this:

#!/bin/sh
#
# Script to check whether IRC Services is running, and restart it if not.
# Usage: ircservices-chk [-q] [ircservices-options]
# -q: don't write any output
# ircservices-options: options to pass to ircservices executable
# If you change PIDFile in ircservices.conf, also change PIDFILE below.
#
# IRC Services is copyright (c) 1996-2009 Andrew Church.
# E-mail:
# Parts written by Andrew Kempe and others.
# This program is free but copyrighted software; see the file GPL.txt for
# details.

PIDFILE=ircservices.pid

if [ "X$1" = "X-q" ] ; then
exec 1>/dev/null
exec 2>/dev/null
shift
fi

ok=
if [ -f "/usr/local/lib/ircservices/$PIDFILE" ] ; then
pid=`cat "/usr/local/lib/ircservices/$PIDFILE"`
if echo "0$pid" | grep -q '[^0-9]' ; then
rm -f "/usr/local/lib/ircservices/$PIDFILE"
elif kill -0 $pid ; then
ok=1
fi
fi

if [ ! "$ok" ] ; then
"/usr/local/sbin/ircservices" "$@"
fi

This script is almost perfect for my use (it does an exactly perfect job for ircservices), but not quite. This script uses the process id (PID) of ircservices to check if it is running or not. That’s ok, but not all programs output their current process ID to a file based location on the system or in any way that’s useful to the end user. I added a line to the beginning of the script, using the pgrep command to output the PID of my selected process to a file.

pgrep process-name > /home/username/a-directory-of-your-choosing/a-file-name-of-your-choosing.pid

Obviously you would replace process-name with the name of the process you want to monitor and username with your actual username.

You can use the command ps aux to list all the processes currently running on your system and locate the process you want to monitor.

The ‘.pid’ file extension isn’t really necessary, however it might be handy for identifying what the file is, should you come across it later on down the track. Here’s my example for checking that ircbot is running:

#!/bin/sh
#
# Script to check whether IRC Services is running, and restart it if not.
# Usage: ircservices-chk [-q] [ircservices-options]
# -q: don't write any output
# ircservices-options: options to pass to ircservices executable
# If you change PIDFile in ircservices.conf, also change PIDFILE below.
#
# IRC Services is copyright (c) 1996-2009 Andrew Church.
# E-mail:
# Parts written by Andrew Kempe and others.
# This program is free but copyrighted software; see the file GPL.txt for
# details.
pgrep ircbot > /home/username/ircbot/ircbot.pid
PIDFILE=ircbot.pid

if [ "X$1" = "X-q" ] ; then
exec 1>/dev/null
exec 2>/dev/null
shift
fi

ok=
if [ -f "/home/username/ircbot/$PIDFILE" ] ; then
pid=`cat "/home/username/ircbot/$PIDFILE"`
if echo "0$pid" | grep -q '[^0-9]' ; then
rm -f "/home/username/ircbot/$PIDFILE"
elif kill -0 $pid ; then
ok=1
fi
fi

if [ ! “$ok” ] ; then

"/home/username/ircbot/ircbot" "$@"
fi

Here’s my final version.

#!/bin/sh
#
# Script to check whether a process is running, and restart it if not.
# Usage: running [-q]
# -q: don't write any output
# Props to Andrew Church (http://www.ircservices.za.net) for creating
# the original script ircservices-chk of which this script you are using
# now is a slight modification by Quin Rose of Mokona Modoki
# http://www.mokonamodoki.com/

pgrep ircbot > /home/username/ircbot/ircbot.pid

PIDFILE=ircbot.pid

if [ "X$1" = "X-q" ] ; then
exec 1>/dev/null
exec 2>/dev/null
shift
fi

ok=
if [ -f "/home/username/ircbot/$PIDFILE" ] ; then
pid=`cat "/home/username/ircbot/$PIDFILE"`
if echo "0$pid" | grep -q '[^0-9]' ; then
rm -f "/home/username/ircbot/$PIDFILE"
elif kill -0 $pid ; then
ok=1
fi
fi

if [ ! "$ok" ] ; then
"/home/username/ircbot/ircbot" "$@"
fi

To use this script, just copy and past the code above into your favourite editor and be sure to save the script, and make it executable with:  chmod +x filename

Last but not least, you don’t want to have to run this script all the time, and manually by yourself do you? Wondering if you can just get put it in a cron job and let it rip? Well, Yes! You can! Add the following line to cron for automatic (once a minute) process checking goodness:

* * * * * /home/andrew/ircbot/ircbot-chk -q

There are plenty of resources on the web relating to cron, so if you would rather have cron execute the script less often, you’ll need to do a little googling.

Adding the -q sends all output from the script to /dev/null (a black hole of sorts) thus not showing any output on the screen. You can test it with or without the -q.

If you have any questions or would like for information or examples, please do let me know in the comments, and I will be only too happy to be of assistance! This certainly helped me, so I thought I would share, so that it could help you too!