Toolbox

It helps optimizing the usage of the sever (as it helped us to terminate some unused instances).
It helps make servers more secure (as it helped us to detect the incoming SSH failure hits).
It helps you manage your server effortlessly.
etc.

Tools

Munin: It is the server node, which collects data from various munin-nodes or muninlite working on different servers and uses that data to generate the graphs. By default it runs after every 5 minutes (cron).

Munin-node: It is a client node which collects the data from the system (proc and other log files) as specified by its plugins when ever munin (server-node) requests for it. By default this runs on port 4949.

Muninlite: It is a lighter version of munin-node, this is lighter as this is built only using bash script. But it has a draw back too, it has less plugins and it does not works properly on ubuntu systems.

Monit: This tools has an upper hand as it deals with the services (apache/ssh/mysql etc) running on the system and is capable of restarting/stopping/starting/monitoring from the web interface. By default this runs on port 2812.

M/monit: This is a fancy dashboard which in terms of working is same as monit's web interface but is more presentable. By default it runs on port 8080.

Installation & Configuration

Munin and Munin-node

Gentoo

Install And Configure munin

emerge -av munin
emerge --config net-analyzer/munin

Configuring Plugins

sudo -u munin munin-node-configure --shell

Best way to add plugins is to creating a symlink

ln -s /usr/libexec/munin/plugins/<plugin name> /etc/munin/plugins/
rc-update add munin-node default
/etc/init.d/munin-node start

Note: Before restarting edit /etc/munin/munin-node.conf to accept connections from server ip.
Note: If using several nodes add them to /et/munin/munin.conf of server.

Debian

Install And Configure munin

apt-get install munin munin-node

Edit /etc/munin/munin.conf

dbdir   /var/lib/munin
htmldir /var/www/munin/
logdir  /var/log/munin
rundir  /var/run/munin

tmpldir /etc/munin/templates

[server1.example.com]
    address 127.0.0.1
    use_node_name yes
# Note:/var/www/munin/ should be your apache root directory or somewhere inside that.

mkdir -p /var/www/munin
chown munin:munin /var/www/munin
/etc/init.d/munin-node restart

After sometime you can see the graphs in http://<server IP>/<Path to munin dir.>

Muninlite

Mininlite requires xinetd/inetd to work

General

These steps are common for both gentoo and debian systems.

cd /usr/local
wget http://downloads.sourceforge.net/project/muninlite/muninlite%20-%20stable/1.0.2/muninlite-1.0.2.tar.gz?use_mirror=biznetnetworks
tar -xvzf muninlite-1.0.2.tar.gz
mv muninlite-1.0.2 muninlite
cd muninlite
make
cp munin-node /usr/local/bin
chmod +x /usr/local/bin/munin-node
echo "munin           4949/tcp        lrrd            # Munin" >>/etc/services

With xinetd

add local ip of munin's servers ip to examples/xinetd.d/munin, by this you tell which is munin server to connect to

vi examples/xinetd.d/munin 
only_from       = 10.226.51.178 #local ip of irc.openbravo.com

cp examples/xinetd.d/munin /etc/xinetd.d
/etc/init.d/xinet.d restart

With inetd

cat examples/inetd.conf >> /etc/inetd.conf
/etc/init.d/inet.d restart

Monit and M/monit

Install And Configure munin (x86 specific).

Note: M/monit needs monit 5.0 or later.

Monit

cd /usr/local
wget http://mmonit.com/monit/dist/monit-5.0.3.tar.gz
tar -xvzf monit-5.0.3.tar.gz
cd monit-5.0.3
./configure
make
make install

Edit /usr/local/monit-5.0.3/monitrc according to what all services you need to monitor.

set mmonit http://monit:monit@<M/monit IP>:8080/collector
      set httpd port 2812 and use address <local IP>
          allow localhost
          allow <server IP>

check process apache with pidfile /var/run/apache2.pid
   group www
   start program = "/etc/init.d/apache2 start"
   stop program  = "/etc/init.d/apache2 stop"
   if failed host www.example.com port 80 protocol http
      and request "/monit/token" then restart
   if cpu is greater than 60% for 2 cycles then alert
   if cpu > 80% for 5 cycles then restart
   if totalmem > 500 MB for 5 cycles then restart
   if children > 250 then restart
   if loadavg(5min) greater than 10 for 8 cycles then stop
   if 3 restarts within 5 cycles then timeout

  check system localhost
    if loadavg (1min) > 4 then alert
    if loadavg (5min) > 2 then alert
    if memory usage > 75% then alert
    if cpu usage (user) > 70% then alert
    if cpu usage (system) > 30% then alert
    if cpu usage (wait) > 20% then alert

set daemon  60
set logfile syslog facility log_daemon
set mailserver localhost
set mail-format { from: monit@monitoring-server.com }
set alert email@domain.com
set httpd port 2812 and
     SSL ENABLE
     PEMFILE  /var/certs/monit.pem
     allow <user>:<password>

M/monit

cd /usr/local
wget wget http://mmonit.com/dist/mmonit-2.0.3-linux-x86.tar.gz
tar -xvzf mmonit-2.0.3-linux-x86.tar.gz
cd mmonit-2.0.3
./bin/mmonit start

Now you can browse through using http://<server IP>:8080/

Gentoo

Install And Configure munin (if using only monit and no M/monit).

emerge -va monit
/etc/init.d/monit start

Edit /etc/monitrc like this.

set daemon  60
set logfile syslog facility log_daemon
set mailserver localhost
set mail-format { from: monit@server1.example.com }
set alert root@localhost
set httpd port 2812 and
     SSL ENABLE
     PEMFILE  /var/certs/monit.pem
     allow <user>:<password>

check process proftpd with pidfile /var/run/proftpd.pid
   start program = "/etc/init.d/proftpd start"
   stop program  = "/etc/init.d/proftpd stop"
   if failed port 21 protocol ftp then restart
   if 5 restarts within 5 cycles then timeout

check process sshd with pidfile /var/run/sshd.pid
   start program  "/etc/init.d/ssh start"
   stop program  "/etc/init.d/ssh stop"
   if failed port 22 protocol ssh then restart
   if 5 restarts within 5 cycles then timeout

check process mysql with pidfile /var/run/mysqld/mysqld.pid
   group database
   start program = "/etc/init.d/mysql start"
   stop program = "/etc/init.d/mysql stop"
   if failed host 127.0.0.1 port 3306 then restart
   if 5 restarts within 5 cycles then timeout

check process apache with pidfile /var/run/apache2.pid
   group www
   start program = "/etc/init.d/apache2 start"
   stop program  = "/etc/init.d/apache2 stop"
   if failed host www.example.com port 80 protocol http
      and request "/monit/token" then restart
   if cpu is greater than 60% for 2 cycles then alert
   if cpu > 80% for 5 cycles then restart
   if totalmem > 500 MB for 5 cycles then restart
   if children > 250 then restart
   if loadavg(5min) greater than 10 for 8 cycles then stop
   if 3 restarts within 5 cycles then timeout

check process postfix with pidfile /var/spool/postfix/pid/master.pid
   group mail
   start program = "/etc/init.d/postfix start"
   stop  program = "/etc/init.d/postfix stop"
   if failed port 25 protocol smtp then restart
   if 5 restarts within 5 cycles then timeout

Debian

Install And Configure munin (if using only monit and no M/monit).

apt-get install monit

Edit vi /etc/monit/monitrc same as above (gentoo).

Configuring apache and secure connection.

mkdir /var/www/monit
echo "hello" > /var/www/monit/token
mkdir /var/certs
cd /var/certs

Create /var/certs/monit.cnf to look like this.

# create RSA certs - Server

RANDFILE = ./openssl.rnd

[ req ]
default_bits = 1024
encrypt_key = yes
distinguished_name = req_dn
x509_extensions = cert_type

[ req_dn ]
countryName = Country Name (2 letter code)
countryName_default = IN

stateOrProvinceName             = State or Province Name (full name)
stateOrProvinceName_default     = State

localityName                    = Locality Name (eg, city)
localityName_default            = Locality

organizationName                = Organization Name (eg, company)
organizationName_default        = Organization

organizationalUnitName          = Organizational Unit Name (eg, section)
organizationalUnitName_default  = Dept.

commonName                      = Common Name (FQDN of your server)
commonName_default              = commonName

emailAddress                    = Email Address
emailAddress_default            = email@domain.com

[ cert_type ]
nsCertType = server
openssl req -new -x509 -days 365 -nodes -config ./monit.cnf -out /var/certs/monit.pem -keyout /var/certs/monit.pem
openssl gendh 512 >> /var/certs/monit.pem
openssl x509 -subject -dates -fingerprint -noout -in /var/certs/monit.pem
chmod 700 /var/certs/monit.pem

Edit /etc/default/monit

startup=1
CHECK_INTERVALS=60

/etc/init.d/monit start

Retrieved from "http://wiki.openbravo.com/wiki/Release_Management/System_Monitoring_with_munin/monit"

This page has been accessed 9,542 times. This page was last modified on 20 April 2015, at 17:08. Content is available under Creative Commons Attribution-ShareAlike 2.5 Spain License.

Category: RM Internal Docs

Toolbox

PDF Books

Search

Participate

Release Management/System Monitoring with munin/monit

Contents

Introduction

Why

Tools

Installation & Configuration

Munin and Munin-node

Gentoo

Debian

Muninlite

General

With xinetd

With inetd

Monit and M/monit

Gentoo

Debian