Transmission This Stats!

Damn!
That's all I can say, damn!
2 Terabytes uploaded.

Gotta See The Godzilla

The biggest download yet, at least, AFAIK and as far as I remember... All in name of love... The GF wanted her Godzilla and she has to have her daily Godzilla, what can I say?

Work && Work and more work

Lots & lots of work lately, not a single minute to post around here.
Been migrating a huge and mastodontic old Nagios setup from an ancient Fedora server to a new state of the art Ubuntu Server.
That, and also the amazing (and ultra ball breaking) heat wave around here left me with really not much strength to post crap on Oesediez.

(Quick) Nagios setup

A quick & dirty "guide" to get Nagios monitoring on a local Linux server in no time. This has been done on CentOS, but the same applies to just about any other distro, as long as you take care of the paths to the configuration files.

cp template.HOSTNAME.FQDN.cfg $HOSTNAME.cfg
emacs $HOSTNAME.cfg

Meta X replace-regex HOSTNAME host
Meta X replace-regex $HOSTNAME.FQDN $HOSTNAME

Section "define host":
Replace "Description here" with the real data
Replace "ip.address.here" with the real IP address of the host

Section "define contact":
Replace "Description here" with the real data, a concise description.
email Replace with the real email addresses of the contacts, or the contact that should receive notifications from this server.

Section "define contactgroup":
Replace "Description here" with the real data of the Linux server.
Be sure to add more services to suit your needs, or the needs of the monitored server.
Copy the file to the Nagios directory:

sudo cp $HOSTNAME.cfg /etc/nagios/objects/

Edit the commands file:

sudo emacs /etc/nagios/objects/commands.cfg

Specifically, edit the strings "***** Nagios *****" on 'notify-host-by-email' & 'notify-service-by-email' to something that identifies the box from which the emails are coming from.
I usually leave like this:

** Nagios $HOSTNAME **

Edit Nagio's master configuration file, so it uses the cfg file just created, and discard the default localhost.cfg one.

sudo emacs /etc/nagios/nagios.cfg

Comment the line:

cfg_file=/etc/nagios/objects/localhost.cfg

And add a line with the cfg file just created.

cfg_file=/etc/nagios/objects/$HOSTNAME.cfg

Before launching Nagios, tight a bit the web access:

sudo emacs /etc/httpd/conf.d/nagios.conf

And setup basic security (enable SSL, setup the password protection, limit IP range, etc).

Test it

sudo /usr/bin/nagios -v /etc/nagios/nagios.cfg

And if its OK, launch it.

Here is a copy of "template.HOSTNAME.FQDN.cfg":

## -----------------------------------------------

## History

## -----------------------------------------------

## sudo /usr/bin/nagios -v /etc/nagios/nagios.cfg

define host{
 name                   HOSTNAME-host
 use                    generic-host
 check_period           24x7
 check_interval         5
 retry_interval         1
 max_check_attempts     10
 check_command          check-host-alive
 notification_period    24x7
 notification_interval  120
 notification_options   d,u,r
 contact_groups         HOSTNAME-contactgroup
 register               0
 }

define host{
 use                    HOSTNAME-host
 host_name              HOSTNAME.FQDN
 alias                  Description here
 address                ip.address.here
 }

define contact{
 contact_name           HOSTNAME-contact
 use                    generic-contact
 alias                  Description here
 email                  1@email.com, 2@email.com, 3@email.com
 }

define contactgroup{
 contactgroup_name      HOSTNAME-contactgroup
 alias                  Description here
 members                HOSTNAME-contact
 }

# SERVICE DEFINITIONS

define service{
 name                         HOSTNAME-generic-service
 active_checks_enabled        1
 passive_checks_enabled       1
 parallelize_check            1
 obsess_over_service          1
 check_freshness              0
 notifications_enabled        1
 event_handler_enabled        1
 flap_detection_enabled       1
 failure_prediction_enabled   1
 process_perf_data            1
 retain_status_information    1
 retain_nonstatus_information 1
 is_volatile                  0
 check_period                 24x7
 max_check_attempts           3
 normal_check_interval        5
 retry_check_interval         2
 contact_groups               HOSTNAME-contactgroup
 notification_options         w,u,c,r
 notification_interval        120
 notification_period          24x7
 register                     0
 }

define service{
 name                   HOSTNAME-service
 use                    HOSTNAME-generic-service
 max_check_attempts     4
 normal_check_interval  5
 retry_check_interval   1
 register               0
 }

define service{
 use                  HOSTNAME-service
 host_name            HOSTNAME.FQDN
 service_description  Connectivity
 check_command        check_ping!100.0,20%!500.0,60%
 }

# Define a service to check the disk space of the root partition
# on the local machine.  Warning if < 20% free, critical if
# < 10% free space on partition.

define service{
 use                   HOSTNAME-service
 host_name             HOSTNAME.FQDN
 service_description   Partition /
 check_command         check_local_disk!20%!10%!/
 }

define service{
 use                   HOSTNAME-service
 host_name             HOSTNAME.FQDN
 service_description   Partition /boot
 check_command         check_local_disk!20%!10%!/boot
 }

# Define a service to check the number of currently logged in
# users on the local machine.  Warning if > 20 users, critical
# if > 50 users.

define service{
 use                   HOSTNAME-service
 host_name             HOSTNAME.FQDN
 service_description   Current Users
 check_command         check_local_users!20!50
 }

# Define a service to check the number of currently running procs
# on the local machine.  Warning if > 250 processes, critical if
# > 400 users.

define service{
 use                   HOSTNAME-service
 host_name             HOSTNAME.FQDN
 service_description   Total Processes
 check_command         check_local_procs!250!400!RSZDT
}

define service{
 use                   HOSTNAME-service
 host_name             HOSTNAME.FQDN
 service_description   Current Load
 check_command         check_local_load!5.0,4.0,3.0!10.0,6.0,4.0
}

define service{
 use                   HOSTNAME-service
 host_name             HOSTNAME.FQDN
 service_description   Swap Usage
 check_command         check_local_swap!20!10
}

define service{
 use                   HOSTNAME-service
 host_name             HOSTNAME.FQDN
 service_description   Service SSH
 check_command         check_ssh
}

define service{
 use                   HOSTNAME-service
 host_name             HOSTNAME.FQDN
 service_description   Service HTTP
 check_command         check_http
}

# EoF #

MRTG: Reload Ok file on Ubuntu Server

I had a problem with an MRTG server while graphing the interface use on a Cisco 2600 & another 2800 router, out of the blue, the MRTG started to graph a completely pegged interface, and printing errors such as this when "executed by hand" from the CLI:

WARNING: Can not determine ifNumber for xxxx@XXX.XXX.XXX.XXX:       ref: 'Name'     key: 'Et0/0'
WARNING: Can not determine ifNumber for xxxx@XXX.XXX.XXX.XXX:       ref: 'Name'     key: 'Se0/0'
WARNING: Can not determine ifNumber for xxxx@XXX.XXX.XXX.XXX:       ref: 'Name'     key: 'Se0/0.240'
ERROR: Target[XXX.XXX.XXX.XXX_et0_0][_IN_] ' $target->[1]{$mode} ' did not eval into defined data
ERROR: Target[XXX.XXX.XXX.XXX_et0_0][_OUT_] ' $target->[1]{$mode} ' did not eval into defined data
ERROR: Target[XXX.XXX.XXX.XXX_se0_0][_IN_] ' $target->[2]{$mode} ' did not eval into defined data
ERROR: Target[XXX.XXX.XXX.XXX_se0_0][_OUT_] ' $target->[2]{$mode} ' did not eval into defined data
ERROR: Target[XXX.XXX.XXX.XXX_se0_0.240][_IN_] ' $target->[3]{$mode} ' did not eval into defined data
ERROR: Target[XXX.XXX.XXX.XXX_se0_0.240][_OUT_] ' $target->[3]{$mode} ' did not eval into defined data

After Googling for a while the error message, all the references pointed towards of "getting rid of the '*.ok' files", but since I had none of those on my system that I could tell, what I did was renaming the file for that particular device on the directory defined on the directive 'ThreshDir':

sudo mv /var/lib/mrtg/_etc_mrtg_cfgs_xxxxxx.+++++++++.cfg  /var/lib/mrtg/_etc_mrtg_cfgs_xxxxxx.+++++++++.cfg.OLD

After that, it started to work A Ok once again...

The files did actually changed, the original looked like this:

xxxx@XXX.XXX.XXX.XXX_    Descr   Ethernet0/0     1
xxxx@XXX.XXX.XXX.XXX_    Descr   Null0   3
xxxx@XXX.XXX.XXX.XXX_    Descr   Serial0/0       2
xxxx@XXX.XXX.XXX.XXX_    Descr   Serial0/0.240   4
xxxx@XXX.XXX.XXX.XXX_    Ip      XXX.XXX.XXX.XXX 1
xxxx@XXX.XXX.XXX.XXX_    Name    Et0/0   
xxxx@XXX.XXX.XXX.XXX_    Name    Nu0             3
xxxx@XXX.XXX.XXX.XXX_    Name    Se0/0   
xxxx@XXX.XXX.XXX.XXX_    Name    Se0/0.240       
xxxx@XXX.XXX.XXX.XXX_    Type    32              2
xxxx@XXX.XXX.XXX.XXX_    Type    6               1

While the new one, automagically generated right after the first MRTG run, looks like this:

xxxx@XXX.XXX.XXX.XXX_    Descr   Ethernet0/0     1
xxxx@XXX.XXX.XXX.XXX_    Descr   Null0   3
xxxx@XXX.XXX.XXX.XXX_    Descr   Serial0/0       2
xxxx@XXX.XXX.XXX.XXX_    Descr   Serial0/0.240   4
xxxx@XXX.XXX.XXX.XXX_    Eth             Dup
xxxx@XXX.XXX.XXX.XXX_    Eth     xx-xx-x-x-x-x-x-x-x-x-x-x-x-x       1
xxxx@XXX.XXX.XXX.XXX_    Ip      XXX.XXX.XXX.XXX 1
xxxx@XXX.XXX.XXX.XXX_    Ip      10.16.254.22    4
xxxx@XXX.XXX.XXX.XXX_    Name    Et0/0           1
xxxx@XXX.XXX.XXX.XXX_    Name    Nu0             3
xxxx@XXX.XXX.XXX.XXX_    Name    Se0/0           2
xxxx@XXX.XXX.XXX.XXX_    Name    Se0/0.240       4
xxxx@XXX.XXX.XXX.XXX_    Type    1               3
xxxx@XXX.XXX.XXX.XXX_    Type    32              Dup
xxxx@XXX.XXX.XXX.XXX_    Type    6               1

PF Sense setup (II)

A little package to show (in real time) the bandwidth use on the PF Sense firewall.
It is called "rate", and it is available from the "Available Packages" menu entry.

This package adds a table of realtime bandwidth usage by IP address to Status -> Traffic Graphs

Install MRTG on CentOS

Installed this morning on a remote server, it already had a basic setup of Apache up & running.

# rpm --test -ivh mrtg-2.14.5-2.i386.rpm 
warning: mrtg-2.14.5-2.i386.rpm: Header V3 DSA signature: NOKEY, key ID e8562897
error: Failed dependencies:
 perl-Socket6 is needed by mrtg-2.14.5-2.i386
 perl-IO-Socket-INET6 is needed by mrtg-2.14.5-2.i386
# rpm --test -ivh mrtg-2.14.5-2.i386.rpm perl-Socket6-0.19-3.fc6.i386.rpm perl-IO-Socket-INET6-2.51-2.fc6.noarch.rpm 
warning: mrtg-2.14.5-2.i386.rpm: Header V3 DSA signature: NOKEY, key ID e8562897
Preparing...                ########################################### [100%]

Install iftop on the Endian Firewall

I love iftop, it is the answer, the inmediate answer, to the most asked question "who the fuck is turning the network to molases".
So after reading this Install VMWareTools On Endian Firewall Community 2.2 RC3, I decided to try and install some RPM extras on the firewall.

So, I got the iftop RPM package from DAG, this one:

http://dag.wieers.com/rpm/packages/iftop/iftop-0.16-0.1.fc3.rf.i386.rpm

Setup things a bit tidy on the Endian:

~ # mkdir local.rpms
~ # cd local.rpms/

And then used wget, that I already installed using the instructions from the first link, downloaded the RPM, and tested if I could install it or if there was a dependency:

~ # rpm --test -ivh iftop-0.16-0.1.fc3.rf.i386.rpm
warning: iftop-0.16-0.1.fc3.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
Preparing...                ########################################### [100%]

No dependencies, everything went A Ok, so simply install and enjoy:

~ # rpm -ivh iftop-0.16-0.1.fc3.rf.i386.rpm 
warning: iftop-0.16-0.1.fc3.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
Preparing...                ########################################### [100%]
   1:iftop                  ########################################### [100%]

Installing Zabbix on RHEL 5

New server, new setup, a lot of work, what's the problem?

First of all, had to install these two, to fulfill dependencies

server$ rpm -q net-snmp net-snmp-devel
net-snmp-5.3.2.2-5.el5
net-snmp-devel-5.3.2.2-5.el5

The second RPM package its specially important, if not installed, you'll get an error like this upon running the './configure'

configure: error: Invalid NET-SNMP directory - unable to find net-snmp-config

After those are installed, all there is left to is executing the configure script:

./configure --enable-server --enable-agent --with-mysql --with-net-snmp

To setup the webfront, in order to finish the instal, create a directory, for instance called 'zabbix' somewhere on your Apache web accessible directories, and then, copy everything that its in the 'zabbix-1.6.2/frontends/php/*' directory to the new you have just created. On my case I something like this:

cd /opt/www_ssl/zabbix/
sudo cp -r -v ~/downloads/zabbix-1.6.2/frontends/php/* ./

After that you can access the webfront thru 'https://yoursite/zabbix/' to wrap the whole thing up.
It is really, really advisable to serve your Zabbix content form an SSL encrypted page, for instance, during the last steps of the configuration thru the webfront you'll be asked for the username/ password to the Data Base, besides that, you'll have to login to the page every time you access it, so there you have another instance that you should avoid for plain text login information.

The only problem I had (see the picture) was during the final steps of the basic configuration, I had to increase the value of one of the PHP environment variables, no big deal.

Nagios install on CentOS 5

There are a ton of sites/ HOWTOs, examples out there, this is just a reminder of the damn RPM necessary packages to get the Nagios up & running in no time on a CentOS 5 boxen.
All the packages (that can not be installed via yum) are from DAG

fping is the only dependency that has not other one but itself, and perl(Net::SNMP) is the one that has all the other ones.
After the install, Nagios' configuration files will be on '/etc/nagios/' (with a ready made minimalist configuration on the file 'localhost.cfg') and the check plugins will reside on '/usr/lib/nagios/plugins/'.

There will be also the web access configuration file on '/etc/httpd/conf.d/mrtg.conf'.
The Nagios RPM sets itself to start automagically on RunLevels 3,4,5.

# rpm --test -ivh fping-2.4-1.b2.2.el5.rf.i386.rpm 
warning: fping-2.4-1.b2.2.el5.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
Preparing...                ########################################### [100%]

# rpm --test -ivh nagios-2.9-1.el5.rf.i386.rpm 
warning: nagios-2.9-1.el5.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
error: Failed dependencies:
        libltdl.so.3 is needed by nagios-2.9-1.el5.rf.i386

[root@ltfs351 nagios]# rpm --test -ivh nagios-2.9-1.el5.rf.i386.rpm libtool-libs-1.5.6-4.EL4.2.i386.rpm 
warning: nagios-2.9-1.el5.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
warning: libtool-libs-1.5.6-4.EL4.2.i386.rpm: Header V3 DSA signature: NOKEY, key ID db42a60e
Preparing...                ########################################### [100%]

[root@ltfs351 nagios]# rpm -ivh nagios-2.9-1.el5.rf.i386.rpm libtool-libs-1.5.6-4.EL4.2.i386.rpm 
warning: nagios-2.9-1.el5.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
warning: libtool-libs-1.5.6-4.EL4.2.i386.rpm: Header V3 DSA signature: NOKEY, key ID db42a60e
Preparing...                ########################################### [100%]
   1:libtool-libs           ########################################### [ 50%]
   2:nagios                 ########################################### [100%]

# rpm --test -ivh nagios-plugins-1.4.9-1.el5.rf.i386.rpm 
warning: nagios-plugins-1.4.9-1.el5.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
error: Failed dependencies:
        perl(Net::SNMP) is needed by nagios-plugins-1.4.9-1.el5.rf.i386

# rpm --test -ivh nagios-plugins-1.4.9-1.el5.rf.i386.rpm perl-Net-SNMP-5.2.0-1.2.el5.rf.noarch.rpm
warning: nagios-plugins-1.4.9-1.el5.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
error: Failed dependencies:
        perl(Crypt::DES) is needed by perl-Net-SNMP-5.2.0-1.2.el5.rf.noarch
        perl(Digest::HMAC) is needed by perl-Net-SNMP-5.2.0-1.2.el5.rf.noarch
        perl(Digest::SHA1) is needed by perl-Net-SNMP-5.2.0-1.2.el5.rf.noarch

# rpm --test -ivh nagios-plugins-1.4.9-1.el5.rf.i386.rpm perl-Net-SNMP-5.2.0-1.2.el5.rf.noarch.rpm \
perl-Crypt-DES-2.05-3.2.el5.rf.i386.rpm perl-Digest-HMAC-1.01-2.rf.0.rh7.rf.noarch.rpm \
perl-Digest-SHA1-2.11-1.el5.rf.i386.rpm
warning: nagios-plugins-1.4.9-1.el5.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
Preparing...                ########################################### [100%]

# rpm -ivh nagios-plugins-1.4.9-1.el5.rf.i386.rpm perl-Net-SNMP-5.2.0-1.2.el5.rf.noarch.rpm \
perl-Crypt-DES-2.05-3.2.el5.rf.i386.rpm perl-Digest-HMAC-1.01-2.rf.0.rh7.rf.noarch.rpm \
perl-Digest-SHA1-2.11-1.el5.rf.i386.rpm
warning: nagios-plugins-1.4.9-1.el5.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
Preparing...                ########################################### [100%]
   1:perl-Digest-SHA1       ########################################### [ 20%]
   2:perl-Digest-HMAC       ########################################### [ 40%]
   3:perl-Crypt-DES         ########################################### [ 60%]
   4:perl-Net-SNMP          ########################################### [ 80%]
   5:nagios-plugins         ########################################### [100%]

Mailgraph install on CentOS 5

Been with a lot of work this week, most of the time busy and pressured with our mail server migration, so, I took a bit of a time off from the box yesterday.

Mailgraph is a very cool utility to graph (duh !) the behavior of your mail server's ability to send, reject & bounce emails.

Installing it is pretty simple, it is a Perl script after all, simply untar the package and copy the Perl script and the init script, edit the some values of the init script, and you are done.
The thing is, the Perl script has some dependencies, most of them dealing with the RRD package.

Personally, I prefer not to touch the repos that the default yum install use on the servers, so here is a round down of the necessary steps to make the installation work, using all the packages from DAG.

[web10 rpms]# sudo rpm --test -ivh rrdtool-1.2.23-1.el5.rf.i386.rpm
warning: rrdtool-1.2.23-1.el5.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
error: Failed dependencies:
        libart_lgpl_2.so.2 is needed by rrdtool-1.2.23-1.el5.rf.i386
        perl(RRDp) is needed by rrdtool-1.2.23-1.el5.rf.i386
        perl(RRDs) is needed by rrdtool-1.2.23-1.el5.rf.i386

According to Egopoly, you need this packages first hand:

[web10 rpms]# sudo yum install libpng-devel \
freetype freetype-devel libart_lgpl-devel

After that, you'll still have problems (Egopoly seems to be compiling from source):

[web10 rpms]# sudo rpm --test -ivh rrdtool-1.2.23-1.el5.rf.i386.rpm
warning: rrdtool-1.2.23-1.el5.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
error: Failed dependencies:
        perl(RRDp) is needed by rrdtool-1.2.23-1.el5.rf.i386
        perl(RRDs) is needed by rrdtool-1.2.23-1.el5.rf.i386

So, get Perl-RRDTool first:

[web10 rpms]# sudo rpm --test -ivh perl-rrdtool-1.2.23-1.el5.rf.i386.rpm
warning: perl-rrdtool-1.2.23-1.el5.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
error: Failed dependencies:
        librrd.so.2 is needed by perl-rrdtool-1.2.23-1.el5.rf.i386
        rrdtool = 1.2.23 is needed by perl-rrdtool-1.2.23-1.el5.rf.i386

There is still a dependency problem, so, you'll have to install both packages at the same time:

rpm --test -ivh perl-rrdtool-1.2.23-1.el5.rf.i386.rpm \
rrdtool-1.2.23-1.el5.rf.i386.rpm rrdtool-devel-1.2.23-1.el5.rf.i386.rpm

You'll still need to install the last rpm package to get Mailgraph up & running:

[web10 rpms]# sudo rpm -ivh perl-File-Tail-0.99.3-1.2.el5.rf.noarch.rpm
warning: perl-File-Tail-0.99.3-1.2.el5.rf.noarch.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
Preparing...                ########################################### [100%]
   1:perl-File-Tail         ########################################### [100%]

Another one of those days...

I tend to use screen a lot, because I need to be connected to many servers at the same time, via SSH, and I like to have only one PuTTY window open, instead of more than 10, totally cluttering my desktop.
So usually I SSH to a server, and from that one use SSH + screen to access all the others I need to.

Nothing wrong with that I guess, just trying to keep a tidy desktop :D
The only problem is, today we had another network outage, not as masive as the other one, again, related to a power suply problem.

That's how, thanks to having my session running on screen, I ended up with that piece of automatic poetry.

Another shot at MRTG

And yet, another post dealing with an MRTG configuration file... Being working a lot finessing the setup on many servers that don't have a running Zabbix server around to monitor & trend them.

HtmlDir: /var/www_ssl/default-website/mrtg/
ImageDir: /var/www_ssl/default-website/mrtg/images
LogDir: /var/www_ssl/default-website/mrtg/logs
ThreshDir: /var/lib/mrtg

Refresh: 600
XSize[_]: 250
Ysize[_]: 67
Colours[_]: --#7aafff,--#1000ff,--#006600,--#ff00ff
MaxBytes[_]: 125000000

## -----------------------------------------------

###############################
## Number of procs
###############################
Target[local_procs]: `/usr/local/etc/scripts/dataGathering/stat.pl procs`
Options[local_procs]: nopercent,gauge,noinfo,nobanner,noi
Title[local_procs]: Number of processes
MaxBytes[local_procs]: 1000000
YLegend[local_procs]: Processes
ShortLegend[local_procs]:
LegendO[local_procs]:
LegendI[local_procs]:
Legend1[local_procs]:
Legend2[local_procs]:
PageTop[local_procs]: Number of processes
WithPeak[local_procs]: wmy
Legend4[local_procs]:

## -----------------------------------------------

###############################
## Number of TCP connections
###############################
Target[local_tcpopen]: `/usr/local/etc/scripts/dataGathering/stat.pl tcpopen`
Options[local_tcpopen]: nopercent,gauge,noinfo,nobanner,noi
Title[local_tcpopen]: Open TCP connections
MaxBytes[local_tcpopen]: 1000000
YLegend[local_tcpopen]: Connections
ShortLegend[local_tcpopen]:  
LegendO[local_tcpopen]: Open TCP connections: 
Legend2[local_tcpopen]: Connections
PageTop[local_tcpopen]: Open TCP connections
WithPeak[local_tcpopen]: wmy
Legend4[local_tcpopen]: Max number of open TCP connections

## -----------------------------------------------

###############################
## Uptime
###############################
Target[local_uptime]: `/usr/local/etc/scripts/dataGathering/stat.pl uptime`
Options[local_uptime]: nopercent,gauge,noinfo,nobanner,noi
Title[local_uptime]: Uptime
MaxBytes[local_uptime]: 1000000
YLegend[local_uptime]: Days
ShortLegend[local_uptime]: days
LegendO[local_uptime]: Uptime: 
LegendI[local_uptime]:  
Legend1[local_uptime]:  
Legend2[local_uptime]: Server uptime (in days)
PageTop[local_uptime]: Uptime
WithPeak[local_uptime]: wmy
Legend4[local_uptime]: Server uptime (in days)

## -----------------------------------------------

###############################
## Load
###############################
Target[local_load]: `/usr/local/etc/scripts/dataGathering/stat.pl load`
Options[local_load]: nopercent,gauge,noinfo,nobanner,noi
Title[local_load]: Load
MaxBytes[local_load]: 100
YLegend[local_load]: Load
ShortLegend[local_load]: Load
LegendO[local_load]: Load: 
Legend2[local_load]: load
PageTop[local_load]: Load
WithPeak[local_load]: wmy
Legend4[local_load]: Max system load

## -----------------------------------------------

###############################
## Users on the server
###############################
Target[local_users]: `/usr/local/etc/scripts/dataGathering/stat.pl users`
Options[local_users]: nopercent,gauge,noinfo,nobanner,noi
Title[local_users]: Number of users
MaxBytes[local_users]: 1000
YLegend[local_users]: Users
ShortLegend[local_users]:  
LegendO[local_users]:
LegendI[local_users]:
Legend1[local_users]:
Legend2[local_users]:
PageTop[local_users]: Users logged in
WithPeak[local_users]: wmy
Legend4[local_users]: Max number of users logged in

## -----------------------------------------------

################################
## Volume on eth0
################################
Target[local_vol_eth0]: `/usr/local/etc/scripts/dataGathering/netVol.sh eth0`
Options[local_vol_eth0]: nopercent,noinfo,gauge,nobanner
Maxbytes[local_vol_eth0]: 10000000
kMG[local_vol_eth0]: M,G,T,P
Title[local_vol_eth0]: Incoming on eth0
PageTop[local_vol_eth0]: Volume on eth0
YLegend[local_vol_eth0]: Megabytes
ShortLegend[local_vol_eth0]:  
Legend1[local_vol_eth0]: Incoming
Legend2[local_vol_eth0]: Outgoing
LegendI[local_vol_eth0]: Incoming: 
LegendO[local_vol_eth0]: Outgoing: 
WithPeak[local_vol_eth0]: wmy
Legend3[local_vol_eth0]: Max incoming
Legend4[local_vol_eth0]: Max outgoing

## -----------------------------------------------

LoadMIBs: /usr/share/snmp/mibs/UCD-SNMP-MIB.txt
LoadMIBs: /usr/share/snmp/mibs/HOST-RESOURCES-MIB.txt
Target[local_memory]: memAvailReal.0&memTotalReal.0:XXXXXXXXXXXXX@localhost
Title[local_memory]: Free Memory
PageTop[local_memory]: Free memory
MaxBytes[local_memory]: 100000000000
ShortLegend[local_memory]:  
YLegend[local_memory]: Megabytes
LegendI[local_memory]: Free: 
LegendO[local_memory]: Total: 
Legend1[local_memory]: Free memory, not including swap, in bytes
Legend2[local_memory]: Total memory
Options[local_memory]: gauge,nopercent,noinfo
kMG[local_memory]: k,M,G,T,P,X

LoadMIBs: /usr/share/snmp/mibs/UCD-SNMP-MIB.txt
Target[local_swap]: memAvailSwap.0&memAvailSwap.0:XXXXXXXXXXXXX@localhost
PageTop[local_swap]: Swap memory
Options[local_swap]: nopercent,gauge,noinfo
Title[local_swap]: Free swap memory
MaxBytes[local_swap]: 4192924
kMG[local_swap]: k,M,G,T,P,X
YLegend[local_swap]: Gigabytes
ShortLegend[local_swap]:  
LegendI[local_swap]: Free swap: 
LegendO[local_swap]:
Legend1[local_swap]: Swap memory avail, in bytes

## -----------------------------------------------

###############################
## RAM & Load use
###############################
Target[local_perf]: `/usr/local/etc/scripts/dataGathering/getLoad2.sh`
Title[local_perf]: CPU load & memory use
MaxBytes[local_perf]: 100
PageTop[local_perf]: CPU load & memory use
Options[local_perf]: noinfo,gauge,nopercent,nobanner
YLegend[local_perf]: Percentage
ShortLegend[local_perf]: %
WithPeak[local_perf]: wmy
Legend1[local_perf]: CPU
Legend2[local_perf]: Memory usage
LegendI[local_perf]: Load: 
LegendO[local_perf]: Mem: 

## -----------------------------------------------

################################
## eth0 traffic
################################
Target[local_traffic]: `/usr/local/etc/scripts/dataGathering/mrtgstat-net.sh eth0`
Title[local_traffic]: Traffic analysis eth0
MaxBytes[local_traffic]: 1400000
PageTop[local_traffic]: Traffic analysis on network interface eth0
Options[local_traffic]: bits,noinfo,nopercent,nobanner
YLegend[local_traffic]: eth0 use
WithPeak[local_traffic]: wmy
LegendI[local_traffic]: Incoming: 
LegendO[local_traffic]: Outgoing: 
Legend1[local_traffic]: Incoming
Legend2[local_traffic]: Outgoing

## -----------------------------------------------

################################
## HDD space
################################
Target[df-root]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /`
Options[df-root]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-root]: 18577596
kmg[df-root]: KB,MB,GB
Title[df-root]: Disk usage for /
PageTop[df-root]: Disk usage /
YLegend[df-root]: Gigabytes
ShortLegend[df-root]:  
LegendI[df-root]: avail 
LegendO[df-root]: used 

Target[df-boot]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /boot`
Options[df-boot]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-boot]: 194442
kmg[df-boot]: KB,MB,GB
Title[df-boot]: Disk usage for /boot
PageTop[df-boot]: Disk usage /boot
YLegend[df-boot]: Megabytes
ShortLegend[df-boot]:  
LegendI[df-boot]: avail 
LegendO[df-boot]: used 

Target[df-usr]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /usr`
Options[df-usr]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-usr]: 30961696
kmg[df-usr]: KB,MB,GB
MaxBytes[df-usr]: 30961696
kmg[df-usr]: KB,MB,GB
Title[df-usr]: Disk usage for /usr
PageTop[df-usr]: Disk usage /usr
YLegend[df-usr]: Gigabytes
ShortLegend[df-usr]:  
LegendI[df-usr]: avail 
LegendO[df-usr]: used 

Target[df-usr-ora]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /usr/local/oracle`
Options[df-usr-ora]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-usr-ora]: 30961696
kmg[df-usr-ora]: KB,MB,GB
Title[df-usr-ora]: Disk usage for /usr/local/oracle
PageTop[df-usr-ora]: Disk usage /usr/local/oracle
YLegend[df-usr-ora]: Gigabytes
ShortLegend[df-usr-ora]:  
LegendI[df-usr-ora]: avail 
LegendO[df-usr-ora]: used 

Target[df-home]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /home`
Options[df-home]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-home]: 41990128
kmg[df-home]: KB,MB,GB
Title[df-home]: Disk usage for /home
PageTop[df-home]: Disk usage /home
YLegend[df-home]: Gigabytes
ShortLegend[df-home]:  
LegendI[df-home]: avail 
LegendO[df-home]: used 

Target[df-tmp]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /tmp`
Options[df-tmp]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-tmp]: 1035660
kmg[df-tmp]: KB,MB,GB
Title[df-tmp]: Disk usage for /tmp
PageTop[df-tmp]: Disk usage /tmp
YLegend[df-tmp]: Megabytes
ShortLegend[df-tmp]:  
LegendI[df-tmp]: avail 
LegendO[df-tmp]: used 

Target[df-var]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /var`
Options[df-var]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-var]: 25806268
kmg[df-var]: KB,MB,GB
Title[df-var]: Disk usage for /var
PageTop[df-var]: Disk usage /var
YLegend[df-var]: Gigabytes
ShortLegend[df-var]:  
LegendI[df-var]: avail 
LegendO[df-var]: used 

## -----------------------------------------------

# EoF #

Mute pings

Which are far more worse than the screaming ones...

Ok, Yes, like I said on quiet days, you can hear the pings reply, that's right... But, obviously, ITOH a rough day can be heard from miles, ping or no pings, and of course, everything can start by getting a single friendly email from your Nagios or Zabbix server.

Today there was a big problem, totally outside of our hands, it was a nationwide power outage problem -not in Argentina- that obviously affected some of the servers at work, from right on it was clear that the thing was huge.

The only 'advantage' of such a big event, like to call it Extinction Level Event (such a geek...), is that everyone is aware of the problem.

Monitor & trending, beauty style

Being pretty busy the last days with a lot of work, mostly getting some Zabbix servers up & running once again, and also, on the boxes that Zabbix wasn't an option, making a whole make up.

A sample mrtg.cfg file:

Htmldir: /var/www/mrtg
Imagedir: /var/www/mrtg/images
Logdir: /var/www/mrtg/logs

Refresh: 600
XSize[_]: 250
Ysize[_]: 67

Colours[_]: --#7aafff,--#1000ff,--#006600,--#ff00ff

## -----------------------------------------------

Target[local_tcpopen]: `/usr/local/etc/scripts/dataGathering/stat.pl tcpopen`
Options[local_tcpopen]: nopercent,gauge,noinfo,nobanner,noi
Title[local_tcpopen]: Open TCP connections
MaxBytes[local_tcpopen]: 1000000
YLegend[local_tcpopen]: Connections
ShortLegend[local_tcpopen]:  
LegendO[local_tcpopen]: Open TCP connections: 
Legend2[local_tcpopen]: Connections
PageTop[local_tcpopen]: TCP Open connections
WithPeak[local_tcpopen]:wmy
Legend4[local_tcpopen]: Max number of open TCP connections

## -----------------------------------------------

Target[local_uptime]: `/usr/local/etc/scripts/dataGathering/stat.pl uptime`
Options[local_uptime]: nopercent,gauge,noinfo,nobanner,noi
Title[local_uptime]: Uptime
MaxBytes[local_uptime]: 1000000
YLegend[local_uptime]: Days
ShortLegend[local_uptime]: days
LegendO[local_uptime]: Uptime: 
LegendI[local_uptime]:  
Legend1[local_uptime]:  
Legend2[local_uptime]: Server uptime (in days)
PageTop[local_uptime]: Uptime
WithPeak[local_uptime]: wmy
Legend4[local_uptime]: Server uptime (in days)

## -----------------------------------------------

Target[local_load]: `/usr/local/etc/scripts/dataGathering/stat.pl load`
Options[local_load]: nopercent,gauge,noinfo,nobanner,noi
Title[local_load]: DB server - Load
MaxBytes[local_load]: 100
YLegend[local_load]: Load
ShortLegend[local_load]: Load
LegendO[local_load]: Load: 
Legend2[local_load]: load
PageTop[local_load]: Load
WithPeak[local_load]:wmy
Legend4[local_load]: Max system load

## -----------------------------------------------

Target[local_users]: `/usr/local/etc/scripts/dataGathering/stat.pl users`
Options[local_users]: nopercent,gauge,noinfo,nobanner,noi
Title[local_users]: DB server - Number of logged users
MaxBytes[local_users]: 1000
YLegend[local_users]: Users
ShortLegend[local_users]:  
LegendO[local_users]:
LegendI[local_users]:
Legend1[local_users]:
Legend2[local_users]:
PageTop[local_users]: Users logged in
WithPeak[local_users]:wmy
Legend4[local_users]: Max number of users logged in

## -----------------------------------------------

Target[local_vol_eth0]: `/usr/local/etc/scripts/dataGathering/netVol.sh eth0`
Options[local_vol_eth0]: nopercent,noinfo,gauge,nobanner
Maxbytes[local_vol_eth0]: 10000000
kMG[local_vol_eth0]: M,G,T,P
Title[local_vol_eth0]: DB server - Incoming on eth0
PageTop[local_vol_eth0]: Volume on eth0
YLegend[local_vol_eth0]: Megabytes
ShortLegend[local_vol_eth0]:o  
Legend1[local_vol_eth0]: Incoming
Legend2[local_vol_eth0]: Outgoing
LegendI[local_vol_eth0]: Incoming: 
LegendO[local_vol_eth0]: Outgoing: 
WithPeak[local_vol_eth0]: wmy
Legend3[local_vol_eth0]: Max incoming
Legend4[local_vol_eth0]: Max outgoing

## -----------------------------------------------

Target[local_perf]: `/usr/local/etc/scripts/dataGathering/getLoad2.sh`
Title[local_perf]: DB server - CPU load & memory stats
#MaxBytes[local_perf]: 1400000
MaxBytes[local_perf]: 100
PageTop[local_perf]: CPU / memory use
Options[local_perf]: noinfo,gauge,nopercent,nobanner
YLegend[local_perf]: Percentage
ShortLegend[local_perf]: %
WithPeak[local_perf]: wmy
Legend1[local_perf]: CPU
Legend2[local_perf]: Memory usage
LegendI[local_perf]: Load: 
LegendO[local_perf]: Mem: 

## -----------------------------------------------

Target[local_traffic]: `/usr/local/etc/scripts/dataGathering/mrtgstat-net.sh eth0`
Title[local_traffic]: DB server - Traffic analysis
MaxBytes[local_traffic]: 1400000
PageTop[local_traffic]: Traffic analysis on network interface eth0
Options[local_traffic]: bits,noinfo,nopercent,nobanner
YLegend[local_traffic]: eth0 use
WithPeak[local_traffic]: wmy
LegendI[local_traffic]: Incoming: 
LegendO[local_traffic]: Outgoing: 
Legend1[local_traffic]: Incoming
Legend2[local_traffic]: Outgoing

## -----------------------------------------------

Target[df-root]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /`
Options[df-root]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-root]: 5162796
kmg[df-root]: KB,MB,GB
Title[df-root]: Disk usage for /
PageTop[df-root]: Disk usage /
YLegend[df-root]: Megabytes
ShortLegend[df-root]:  
LegendI[df-root]: avail 
LegendO[df-root]: used 

Target[df-boot]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /boot`
Options[df-boot]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-boot]: 101107
kmg[df-boot]: KB,MB,GB
Title[df-boot]: Disk usage for /boot
PageTop[df-boot]: Disk usage /boot
YLegend[df-boot]: Megabytes
ShortLegend[df-boot]:  
LegendI[df-boot]: avail 
LegendO[df-boot]: used 

Target[df-home]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /home`
Options[df-home]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-home]: 10325780
kmg[df-home]: KB,MB,GB
Title[df-home]: Disk usage for /home
PageTop[df-home]: Disk usage /home
YLegend[df-home]: Megabytes
ShortLegend[df-home]:  
LegendI[df-home]: avail 
LegendO[df-home]: used 

Target[df-tmp]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /tmp`
Options[df-tmp]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-tmp]: 2063504
kmg[df-tmp]: KB,MB,GB
Title[df-tmp]: Disk usage for /tmp
PageTop[df-tmp]: Disk usage /tmp
YLegend[df-tmp]: Megabytes
ShortLegend[df-tmp]:  
LegendI[df-tmp]: avail 
LegendO[df-tmp]: used 

Target[df-usr]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /usr`
Options[df-usr]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-usr]: 25806268
kmg[df-usr]: KB,MB,GB
Title[df-usr]: Disk usage for /usr
PageTop[df-usr]: Disk usage /usr
YLegend[df-usr]: Megabytes
ShortLegend[df-usr]:  
LegendI[df-usr]: avail 
LegendO[df-usr]: used 

Target[df-usr-ora]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /usr/local/oracle`
Options[df-usr-ora]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-usr-ora]: 14769076
kmg[df-usr-ora]: KB,MB,GB
Title[df-usr-ora]: Disk usage for /usr/local/oracle
PageTop[df-usr-ora]: Disk usage /usr/local/oracle
YLegend[df-usr-ora]: Megabytes
ShortLegend[df-usr-ora]:  
LegendI[df-usr-ora]: avail 
LegendO[df-usr-ora]: used 

Target[df-var]: `/usr/local/etc/scripts/dataGathering/mrtg-df.sh /var`
Options[df-var]: nopercent,nobanner,nolegend,noinfo,integer,gauge
MaxBytes[df-var]: 6190664
kmg[df-var]: KB,MB,GB
Title[df-var]: Disk usage for /var
PageTop[df-var]: Disk usage /var
YLegend[df-var]: Megabytes
ShortLegend[df-var]:  
LegendI[df-var]: avail 
LegendO[df-var]: used 

## -----------------------------------------------

Target[local_swap]:`/usr/local/etc/scripts/dataGathering/mrtg-swap.sh`
Unscaled[local_swap]: dwym
MaxBytes[local_swap]: 2799232
Title[local_swap]:Swap memory on server
ShortLegend[local_swap]:  
kmg[local_swap]:kB,MB
kilo[local_swap]:1024
YLegend[local_swap]: Swap use
Legend1[local_swap]: Total swap
Legend2[local_swap]: Used swap
LegendI[local_swap]: Total swap: 
LegendO[local_swap]: Used swap: 
Options[local_swap]: gauge,nopercent,noinfo,nobanner
PageTop[local_swap]:Swap usage

## -----------------------------------------------

Target[local_diskperf-root]: `/usr/local/etc/scripts/dataGathering/mrtg-diskperf-root.sh`
Title[local_diskperf-root]: / (sda9) partition I/O usage
Unscaled[local_diskperf-root]: dwym
MaxBytes[local_diskperf-root]: 100
PageTop[local_diskperf-root]: / (sda9) partition I/O usage
kmg[local_diskperf-root]: KB,MB,GB
LegendI[local_diskperf-root]: Disk I/O KBread/sec: 
LegendO[local_diskperf-root]: Disk I/O KBwrite/sec: 
Legend1[local_diskperf-root]: Disk I/O KBread/sec
Legend2[local_diskperf-root]: Disk I/O KBwrite/sec
YLegend[local_diskperf-root]:  Megabytes
ShortLegend[local_diskperf-root]:  
Options[local_diskperf-root]: gauge,nopercent,noinfo,nobanner

Target[local_diskperf-boot]: `/usr/local/etc/scripts/dataGathering/mrtg-diskperf-boot.sh`
Title[local_diskperf-boot]: /boot (sda3) partition I/O usage
Unscaled[local_diskperf-boot]: dwym
MaxBytes[local_diskperf-boot]: 100
PageTop[local_diskperf-boot]: /boot (sda3) partition I/O usage
kmg[local_diskperf-boot]: KB,MB,GB
LegendI[local_diskperf-boot]: Disk I/O KBread/sec: 
LegendO[local_diskperf-boot]: Disk I/O KBwrite/sec: 
Legend1[local_diskperf-boot]: Disk I/O KBread/sec
Legend2[local_diskperf-boot]: Disk I/O KBwrite/sec
YLegend[local_diskperf-boot]:  Megabytes
ShortLegend[local_diskperf-boot]:  
Options[local_diskperf-boot]: gauge,nopercent,noinfo,nobanner

Target[local_diskperf-tmp]: `/usr/local/etc/scripts/dataGathering/mrtg-diskperf-tmp.sh`
Title[local_diskperf-tmp]: /tmp (sda6) partition I/O usage
Unscaled[local_diskperf-tmp]: dwym
MaxBytes[local_diskperf-tmp]: 100
PageTop[local_diskperf-tmp]: /tmp (sda6) partition I/O usage
kmg[local_diskperf-tmp]: KB,MB,GB
LegendI[local_diskperf-tmp]: Disk I/O KBread/sec: 
LegendO[local_diskperf-tmp]: Disk I/O KBwrite/sec: 
Legend1[local_diskperf-tmp]: Disk I/O KBread/sec
Legend2[local_diskperf-tmp]: Disk I/O KBwrite/sec
YLegend[local_diskperf-tmp]:  Megabytes
ShortLegend[local_diskperf-tmp]:  
Options[local_diskperf-tmp]: gauge,nopercent,noinfo,nobanner


Target[local_diskperf-usr]: `/usr/local/etc/scripts/dataGathering/mrtg-diskperf-usr.sh`
Title[local_diskperf-usr]: /usr (sda5) partition I/O usage
Unscaled[local_diskperf-usr]: dwym
MaxBytes[local_diskperf-usr]: 100
PageTop[local_diskperf-usr]: /usr (sda5) partition I/O usage
kmg[local_diskperf-usr]: KB,MB,GB
LegendI[local_diskperf-usr]: Disk I/O KBread/sec: 
LegendO[local_diskperf-usr]: Disk I/O KBwrite/sec: 
Legend1[local_diskperf-usr]: Disk I/O KBread/sec
Legend2[local_diskperf-usr]: Disk I/O KBwrite/sec
YLegend[local_diskperf-usr]:  Megabytes
ShortLegend[local_diskperf-usr]:  
Options[local_diskperf-usr]: gauge,nopercent,noinfo,nobanner

Target[local_diskperf-var]: `/usr/local/etc/scripts/dataGathering/mrtg-diskperf-var.sh`
Title[local_diskperf-var]: /var (sda10) partition I/O usage
Unscaled[local_diskperf-var]: dwym
MaxBytes[local_diskperf-var]: 100
PageTop[local_diskperf-var]: /var (sda10) partition I/O usage
kmg[local_diskperf-var]: KB,MB,GB
LegendI[local_diskperf-var]: Disk I/O KBread/sec: 
LegendO[local_diskperf-var]: Disk I/O KBwrite/sec: 
Legend1[local_diskperf-var]: Disk I/O KBread/sec
Legend2[local_diskperf-var]: Disk I/O KBwrite/sec
YLegend[local_diskperf-var]:  Megabytes
ShortLegend[local_diskperf-var]:  
Options[local_diskperf-var]: gauge,nopercent,noinfo,nobanner

Target[local_diskperf-home]: `/usr/local/etc/scripts/dataGathering/mrtg-diskperf-home.sh`
Title[local_diskperf-home]: /home (sda2) partition I/O usage
Unscaled[local_diskperf-home]: dwym
MaxBytes[local_diskperf-home]: 100
PageTop[local_diskperf-home]: /home (sda2) partition I/O usage
kmg[local_diskperf-home]: KB,MB,GB
LegendI[local_diskperf-home]: Disk I/O KBread/sec: 
LegendO[local_diskperf-home]: Disk I/O KBwrite/sec: 
Legend1[local_diskperf-home]: Disk I/O KBread/sec
Legend2[local_diskperf-home]: Disk I/O KBwrite/sec
YLegend[local_diskperf-home]:  Megabytes
ShortLegend[local_diskperf-home]:  
Options[local_diskperf-home]: gauge,nopercent,noinfo,nobanner

Target[local_diskperf-usr-oracle]: `/usr/local/etc/scripts/dataGathering/mrtg-diskperf-usr-oracle.sh`
Title[local_diskperf-usr-oracle]: /usr/local/oracle (sda10) partition I/O usage
Unscaled[local_diskperf-usr-oracle]: dwym
MaxBytes[local_diskperf-usr-oracle]: 100
PageTop[local_diskperf-usr-oracle]: /usr/local/oracle (sda10) partition I/O usage
kmg[local_diskperf-usr-oracle]: KB,MB,GB
LegendI[local_diskperf-usr-oracle]: Disk I/O KBread/sec: 
LegendO[local_diskperf-usr-oracle]: Disk I/O KBwrite/sec: 
Legend1[local_diskperf-usr-oracle]: Disk I/O KBread/sec
Legend2[local_diskperf-usr-oracle]: Disk I/O KBwrite/sec
YLegend[local_diskperf-usr-oracle]:  Megabytes
ShortLegend[local_diskperf-usr-oracle]:  
Options[local_diskperf-usr-oracle]: gauge,nopercent,noinfo,nobanner

## -----------------------------------------------

# EoF #

Polling ethernet info from Windows using Zabbix

Effing Windows keeps crawling onto my life, no matter what I do...
Today at work we incorporated the first Windows server to production duties, it is dedicated server, with a hosting company in the ole US of A.

For websites, the company is, was, actually, 100% Linux; but there is a client that used to have a prior version of their site running on Windows, so we started developing the newer products on Linux, but the legacy has to be maintained on a Windows platform for the time being...

Ok, the occasion was rather good to test a bit the monitoring side of Windows,(even more than here).
On that particular site, I have setup my favorite complex monitoring and trending solution, Zabbix, so after installing the Zabbix Agent on the Windows 2003 Server, (with SP2 installed), used one of the templates on the Zabbix server, they are an amazing way of getting all sorts of data right away, without having to spend a lot of time configuring things.

The thing I noticed is that they were no stats for the Ethernet interface at all, that seemed really strange since that particular information is basically one of the most important, if not the most important information you can poll remotely from a server.

On the Windows box, add this to the bottom of the 'zabbix_agentd.conf' file:

PerfCounter = NetOut,"\Network Interface(Broadcom NetXtreme Gigabit Ethernet)\Bytes Sent/sec",60
PerfCounter = NetIn,"\Network Interface(Broadcom NetXtreme Gigabit Ethernet)\Bytes Received/sec",60

Replace 'Broadcom NetXtreme Gigabit Ethernet' with the one that your Windows box has! You can get the value running 'ipconfig /all' on a CMD window on the Windows box; it's the "Description" field.

Once you did that, restart the "Zabbix Win32 Service", and move onto the Linux server that runs the Zabbix server.

Login to the Zabbix's webfront, as an administrator, goto 'Configuration' -> 'Items', select you Windows Zabbix monitored box from the drop down, and then, 'Create item', and take a look at the screen shot to get the values for each field. You'll need to create one Item for the 'NetOut' and another for the 'NetIn'.
Once you save both Items, you are done, you'll begin to get information from the NIC on the Windows box.

Back to basics

Smoke or no smoke I had a lot of work last week.
Had to setup MRTG on some servers, and this time decided to implement it using SNMP instead of the regular way I use, executing shell scripts to get the data.

## /etc/mrtg/mrtg.cfg from
## xxxxxxxxxxxxxxxxxxxxxxxxxxxx

Htmldir: /var/www/html/mrtg
Imagedir: /var/www/html/mrtg/images
Logdir: /var/www/html/mrtg/logs

Refresh: 600
XSize[_]: 250
Ysize[_]: 67

Colours[_]: --#7aafff,--#1000ff,--#006600,--#ff00ff

MaxBytes[_]: 125000000

LoadMIBs: /usr/share/snmp/mibs/UCD-SNMP-MIB.txt
LoadMIBs: /usr/share/snmp/mibs/HOST-RESOURCES-MIB.txt

## -----------------------------------------------

Title[local_mempercent]: Percentage free memory
PageTop[local_mempercent]: Percentage free memory
Target[local_mempercent]: ( memAvailReal.0&memAvailReal.0:RaBinG0sting7320@localhost ) * 100 / ( memTotalReal.0&memTotalReal.0:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost )
options[local_mempercent]: gauge,transparent,nopercent,noinfo
Unscaled[local_mempercent]: ymwd
MaxBytes[local_mempercent]: 100
YLegend[local_mempercent]: Memory %
ShortLegend[local_mempercent]: Percent
LegendI[local_mempercent]: Free
LegendO[local_mempercent]: Free
Legend1[local_mempercent]: Percentage free memory
Legend2[local_mempercent]: Percentage free memory

Target[local_memory]: memAvailReal.0&memTotalReal.0:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost
Title[local_memory]: Free Memory
PageTop[local_memory]: Free memory
MaxBytes[local_memory]: 100000000000
ShortLegend[local_memory]: B
YLegend[local_memory]: Bytes
LegendI[local_memory]: Free
LegendO[local_memory]: Total
Legend1[local_memory]: Free memory, not including swap, in bytes
Legend2[local_memory]: Total memory
Options[local_memory]: gauge,nopercent,noinfo
kMG[local_memory]: k,M,G,T,P,X

Target[local_load]:ssCpuRawUser.0&ssCpuRawUser.0:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost + ssCpuRawSystem.0&ssCpuRawSystem.0:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost + ssCpuRawNice.0&ssCpuRawNice.0:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost
Title[local_load]: Server CPU load
PageTop[local_load]: CPU load
MaxBytes[local_load]: 100
ShortLegend[local_load]: %
YLegend[local_load]: CPU Utilization
Legend1[local_load]: Current CPU percentage load
LegendI[local_load]: Used
LegendO[local_load]:
Options[local_load]: nopercent,noinfo
Unscaled[local_load]: ymwd

Target[local_eth0]: 2:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost:
Options[local_eth0]: nopercent,noinfo
SetEnv[local_eth0]: MRTG_INT_IP="161.196.132.11" MRTG_INT_DESCR="dc0"
MaxBytes[local_eth0]: 12500000
Title[local_eth0]: Traffic analysis for ethernet card eth0
PageTop[local_eth0]: Ethernet card eth0
WithPeak[local_eth0]:wmy

Target[local_newconns]: tcpPassiveOpens.0&tcpActiveOpens.0:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost
Title[local_newconns]: Newly created TCP connections
PageTop[local_newconns]: New TCP connections
MaxBytes[local_newconns]: 10000000000
ShortLegend[local_newconns]: c/s
YLegend[local_newconns]: Conns / Min
LegendI[local_newconns]: In
LegendO[local_newconns]: Out
Legend1[local_newconns]: New inbound connections
Legend2[local_newconns]: New outbound connections
Options[local_newconns]: nopercent,perminute,noinfo

Target[local_estabcons]: tcpCurrEstab.0&tcpCurrEstab.0:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost
Title[local_estabcons]: Currently established TCP connections
PageTop[local_estabcons]: Established TCP connections
MaxBytes[local_estabcons]: 10000000000
ShortLegend[local_estabcons]:
YLegend[local_estabcons]: Connections
LegendI[local_estabcons]: In
LegendO[local_estabcons]:
Legend1[local_estabcons]: Established connections
Legend2[local_estabcons]:
Options[local_estabcons]: nopercent,gauge,noinfo

Target[local_df_root_boot]: dskPercent.1&dskPercent.2:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost
Title[local_df_root_boot]: Disk partition usage
PageTop[local_df_root_boot]: Disk partition usage / and /boot
MaxBytes[local_df_root_boot]: 100
ShortLegend[local_df_root_boot]: %
YLegend[local_df_root_boot]: Utilization
LegendI[local_df_root_boot]: /
LegendO[local_df_root_boot]: /boot
Options[local_df_root_boot]: gauge,nopercent,noinfo
Unscaled[local_df_root_boot]: ymwd

Target[local_df_data1_home]: dskPercent.3&dskPercent.4:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost
Title[local_df_data1_home]: Disk partition usage
PageTop[local_df_data1_home]: Disk partition usage /data1 and /home
MaxBytes[local_df_data1_home]: 100
ShortLegend[local_df_data1_home]: %
YLegend[local_df_data1_home]: Utilization
LegendI[local_df_data1_home]: /data1
LegendO[local_df_data1_home]: /home
Options[local_df_data1_home]: gauge,nopercent,noinfo
Unscaled[local_df_data1_home]: ymwd

Target[local_df_tmp_usr]: dskPercent.5&dskPercent.6:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost
Title[local_df_tmp_usr]: Disk partition usage
PageTop[local_df_tmp_usr]: Disk partition usage /tmp and /usr
MaxBytes[local_df_tmp_usr]: 100
ShortLegend[local_df_tmp_usr]: %
YLegend[local_df_tmp_usr]: Utilization
LegendI[local_df_tmp_usr]: /tmp
LegendO[local_df_tmp_usr]: /usr
Options[local_df_tmp_usr]: gauge,nopercent,noinfo
Unscaled[local_df_tmp_usr]: ymwd

Target[local_df_usr_oracle_var]: dskPercent.7&dskPercent.8:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost
Title[local_df_usr_oracle_var]: Disk partition usage
PageTop[local_df_usr_oracle_var]: Disk partition usage /usr/local/oracle and /var
MaxBytes[local_df_usr_oracle_var]: 100
ShortLegend[local_df_usr_oracle_var]: %
YLegend[local_df_usr_oracle_var]: Utilization
LegendI[local_df_usr_oracle_var]: /usr/local/oracle
LegendO[local_df_usr_oracle_var]: /var
Options[local_df_usr_oracle_var]: gauge,nopercent,noinfo
Unscaled[local_df_usr_oracle_var]: ymwd

Target[local_process]: .1.3.6.1.2.1.25.1.6.0&.1.3.6.1.2.1.25.1.6.0:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost
Options[local_process]: gauge,nopercent,noinfo
MaxBytes[local_process]: 1000
YLegend[local_process]: Processes
ShortLegend[local_process]: 
Legend1[local_process]: 
Legend2[local_process]:
LegendI[local_process]: 
LegendO[local_process]:
Title[local_process]: Number of processes
PageTop[local_process]: Number of processes

Target[local_user]: hrSystemNumUsers.0&hrSystemNumUsers.0:XXXXXXXXXXXXXXXXXXXXXXXXXx@localhost
Options[local_user]: nopercent,gauge,noinfo
Title[local_user]: Logged users 
PageTop[local_user]: Logged users
MaxBytes[local_user]: 100
YLegend[local_user]: # users 
ShortLegend[local_user]: users 
LegendI[local_user]:  Users: 
LegendO[local_user]: 
Legend1[local_user]: Logged users


# EoF #

Enabling SNMP on Vista

So you can monitor and graph the performance of the box over the network.
The process itself is simple enough, and doesn't even require a reboot of the box.
Start -> Control Panel -> Programs -> Turn Windows features on or off -> SNMP feature

With what I did have a little problem was getting the comunity string on all the right place, so I'll post this screen shots to get a future reference of the process.



Also, make sure to enable the SNMP access thru the Windows Firewall, the vista firewall -using it with the Advanced Security- has a very, very good GUI, and has a ton of config options extra than the XP one.

Now, setting it up and enabling it is one thing, but ITOH, getting the OIDs it is proving to be a royal PITA.

Monitoring Tiger via SNMPD over the network (III)

Changed the way that the used percentage from the HDDs are being polled from Thor, because the old way was way too cumbersome, changed from using 'snmpdf' (and having to parse the result) to use 'snmpget' to poll only the necesary data.
Querying the 'dskPercent' MIB, in which, 'dskPercent.1' will be the first partition (whether is a disk or not) that you defined on your '/etc/snmpd.conf' file


From this:

/usr/local/bin/snmpdf -v 2c -Cu -c XXXXXXXXX thor | awk '{print $5}' | tail -r -n 2 | tail -n 1 | cut -d% -f 1

To this:

/usr/local/bin/snmpget -v 2c -Oqv -c XXXXXXXXX thor dskPercent.2

Where, 'dskPercent.2' is the disk/ partition that you want to query. Leaner, faster and easier on the eyes :D

So, the snip from the /etc/mrtg.cfg file on Judith looks like this now:

Title[thor_root_df]: Percentage use /dev/disk1s3 file system
PageTop[thor_root_df]: Use /dev/disk1s3/ file system
Target[thor_root_df]: `/usr/local/bin/snmpget -v 2c -Oqv -c XXXXXXXXXXXX thor dskPercent.1`
options[thor_root_df]: gauge,transparent,nopercent,noinfo,nobanner
Unscaled[thor_root_df]: ymwd
MaxBytes[thor_root_df]: 100
YLegend[thor_root_df]: Use %
ShortLegend[thor_root_df]: Percent
LegendI[thor_root_df]: Use
LegendO[thor_root_df]: Use
Legend1[thor_root_df]: Use /dev/disk1s3/ file system
Legend2[thor_root_df]: Use /dev/disk1s3/ file system

Title[thor_root_data1]: Percentage use /dev/disk1s5/ file system
PageTop[thor_root_data1]: Use /dev/disk1s5/ file system
Target[thor_root_data1]: `/usr/local/bin/snmpget -v 2c -Oqv -c XXXXXXXXXXXX thor dskPercent.2`
options[thor_root_data1]: gauge,transparent,nopercent,noinfo,nobanner
Unscaled[thor_root_data1]: ymwd
MaxBytes[thor_root_data1]: 100
YLegend[thor_root_data1]: Use %
ShortLegend[thor_root_data1]: Percent
LegendI[thor_root_data1]: Use
LegendO[thor_root_data1]: Use
Legend1[thor_root_data1]: Use /dev/disk1s5/ file system
Legend2[thor_root_data1]: Use /dev/disk1s5/ file system

Title[thor_root_data2]: Percentage use /dev/disk0s3/ file system
PageTop[thor_root_data2]: Use /dev/disk0s3/ file system
Target[thor_root_data2]: `/usr/local/bin/snmpget -v 2c -Oqv -c XXXXXXXXXXXX thor dskPercent.3`
options[thor_root_data2]: gauge,transparent,nopercent,noinfo,nobanner
Unscaled[thor_root_data2]: ymwd
MaxBytes[thor_root_data2]: 100
YLegend[thor_root_data2]: Use %
ShortLegend[thor_root_data2]: Percent
LegendI[thor_root_data2]: Use
LegendO[thor_root_data2]: Use
Legend1[thor_root_data2]: Use /dev/disk0s3/ file system
Legend2[thor_root_data2]: Use /dev/disk0s3/ file system

Monitoring Tiger via SNMPD over the network (II)

Today while I was over looking some looooooong backups from server to server, continued enabling/ testing the hard disk drive monitoring with SNMPD on Tiger, the plain vanilla one, not the server version.

After enabling SNMPD on Thor, I'm using it to report the percentage of HDD used, running snmpdf, one of the programs from the net-snmp suite.
On Thor I have 2 HDDs, spliced onto 3 partitions.

When executed, the command returns info on all of the HDDs that box you are monitoring has, so the biggest annoyance it parsing the result, but it only takes a couple of minutes to figure it how to get the desired result printed.

This is the result, without parsing:

$ /usr/local/bin/snmpdf -v 2c -Oqv -Cu -c XXXXXXXXXXXXXXXXX thor
Description              size (kB)            Used       Available Used%
/                         25952256         4844504        21107752   18%
/Volumes/data1            51936304        21595244        30341060   41%
/Volumes/data2            79912152        60396756        19515396   75%

And here is the snip from the /etc/mrtg.cfg file from Judith:

Title[thor_root_df]: Percentage use /dev/disk1s3 file system
PageTop[thor_root_df]: Use /dev/disk1s3/ file system
Target[thor_root_df]: `/usr/local/bin/snmpdf -v 2c -Cu -c XXXXXXXXXXXXXXXX thor | awk '{print $5}' | tail -r -n 3 | tail -n 1 | cut -d% -f 1`
options[thor_root_df]: gauge,transparent,nopercent,noinfo,nobanner
Unscaled[thor_root_df]: ymwd
MaxBytes[thor_root_df]: 100
YLegend[thor_root_df]: Use %
ShortLegend[thor_root_df]: Percent
LegendI[thor_root_df]: Use
LegendO[thor_root_df]: Use
Legend1[thor_root_df]: Use /dev/disk1s3/ file system
Legend2[thor_root_df]: Use /dev/disk1s3/ file system

Title[thor_root_data1]: Percentage use /dev/disk1s5/ file system
PageTop[thor_root_data1]: Use /dev/disk1s5/ file system
Target[thor_root_data1]: `/usr/local/bin/snmpdf -v 2c -Cu -c XXXXXXXXXXXXXXXX thor | awk '{print $5}' | tail -r -n 2 | tail -n 1 | cut -d% -f 1`
options[thor_root_data1]: gauge,transparent,nopercent,noinfo,nobanner
Unscaled[thor_root_data1]: ymwd
MaxBytes[thor_root_data1]: 100
YLegend[thor_root_data1]: Use %
ShortLegend[thor_root_data1]: Percent
LegendI[thor_root_data1]: Use
LegendO[thor_root_data1]: Use
Legend1[thor_root_data1]: Use /dev/disk1s5/ file system
Legend2[thor_root_data1]: Use /dev/disk1s5/ file system

Title[thor_root_data2]: Percentage use /dev/disk0s3/ file system
PageTop[thor_root_data2]: Use /dev/disk0s3/ file system
Target[thor_root_data2]: `/usr/local/bin/snmpdf -v 2c -Cu -c XXXXXXXXXXXXXXXX thor | awk '{print $5}' | tail -n 1 | cut -d% -f 1`
options[thor_root_data2]: gauge,transparent,nopercent,noinfo,nobanner
Unscaled[thor_root_data2]: ymwd
MaxBytes[thor_root_data2]: 100
YLegend[thor_root_data2]: Use %
ShortLegend[thor_root_data2]: Percent
LegendI[thor_root_data2]: Use
LegendO[thor_root_data2]: Use
Legend1[thor_root_data2]: Use /dev/disk0s3/ file system
Legend2[thor_root_data2]: Use /dev/disk0s3/ file system

I'm looking at other options available, so I can avoid making all those operations to parse the data.