Monitor with Monit

A few days ago I posted a message about creating a pid file for linbpq. So that it becomes possible to monitor the process with watchdog.

You can read that here.

Now I use Monit to display this graphically. Why, because it looks nice. Now you immediately have an overview of running processes and their status.

The configuration file that I use for this.


## Start Monit in the background (run as a daemon):
  set daemon 300            # check services at 5-minute intervals
  set logfile /var/log/monit.log
  set pidfile /var/run/
  set idfile /var/lib/monit/id
  set statefile /var/lib/monit/state

  set httpd port 2812
        use address
        allow admin:pancake

  set mailserver localhost               # primary mailserver

  set eventqueue
      basedir /var/lib/monit/events # set the base directory where events will be stored
      slots 100                     # optionally limit the queue size
  set alert                      # receive all alerts

  check system $HOST
    if loadavg (1min) > 4 then alert
    if loadavg (5min) > 2 then alert
    if cpu usage > 95% for 10 cycles then alert
    if memory usage > 75% then alert
    if swap usage > 25% then alert

  check process Linbpq with pidfile /home/pd9q/linbpq/run/
  if changed pid then alert
  check file bpq32.cfg with path /home/pd9q/linbpq/bpq32.cfg

  check process Direwolf with pidfile /home/pd9q/linbpq/run/
  if changed pid then alert
  check file direwolf.conf with path /home/pd9q/linbpq/direwolf.conf
 check host Router with address
    if failed ping then alert

  check network Eth0 with interface eth0
    if failed link then alert
    if changed link then alert
    if saturation > 90% then alert
    if download > 10 MB/s then alert
    if total uploaded > 1 GB in last hour then alert

check host with address
      if failed icmp type echo count 5 with timeout 15 seconds
         then alert
      if failed port 80 proto http then alert
      if failed port 443 type TCPSSL proto http then alert

   include /etc/monit/conf.d/*
   include /etc/monit/conf-enabled/*



I use watchdog to monitor my RPI. Do services go offline or the RPI is not responding, watchdog kicks in and restart the system. Now I also want watchdog to watch my Linbpq. This is possible with watchdog, watchdog looks at a PID file. As far as I know, no PID file is created when starting Linbpq.

Now there is a possibility that when starting Linbpq with systemd a command can be given so that a PID file is created.

This is my linbpq.service file from systemd

pd9q@packet:~ $ sudo cat /etc/systemd/system/linbpq.service
Description=Linbpq Daemon

ExecStartPost=/bin/sh -c "echo $MAINPID > /home/pd9q/linbpq/run/"
ExecStart=/bin/bash /home/pd9q/linbpq/linbpq.start


This command creates a file, now it is possible that watchdog monitors linbpq.

ExecStartPost=/bin/sh -c "echo $MAINPID > /home/pd9q/linbpq/run/"
pd9q@packet:~ $ sudo ls -l /home/pd9q/linbpq/run
-rw-r--r-- 1 root root 4 Oct 14 18:36
pd9q@packet:~ $

At the end of my watchdog configuration file (/etc/watchdog.conf) I have added the following line.

# Check if Linbpq is running
pidfile         = /home/pd9q/linbpq/run/

Take a look at what happens when you stop Linbpq.

sudo systemctl stop linbpq.service

Yes the system is ReBoOtInG (I hoop)