RTG (Bandwidth and 95th Percentile Monitoring ) Configuration and RTG Customer Interface tools

RTG (Bandwidth and 95th Percentile Monitoring ) Installation and Configuration

What is RTG ?

RTG Stands for Real Traffic Grabber.RTG is a flexible, scalable, high-performance SNMP statistics monitoring system. It is designed for enterprises and service providers who need to collect time-series SNMP data from a large number of targets quickly. All collected data is inserted into a relational database that provides a common interface for applications to generate complex queries and reports. RTG includes utilities that generate configuration and target files, traffic reports, 95th percentile reports and graphical data plots. These utilities may be used to produce a web-based interface to the data.

RTG Features

Runs as a daemon, incurring no cron or kernel startup overhead

Written entirely in C for speed, incurring no interpreter overhead

Multi-threaded for asynchronous polling and database insertion

Inserts data into a relational database where complex queries and reports may be generated

Performs no data averaging in order to support billing, etc.

Can poll at sub-one-minute intervals

Download RTG

http://sourceforge.net/projects/rtg/

RTG Documentation

http://rtg.sourceforge.net/docs/index.html

RTG Screenshots

http://rtg.sourceforge.net/examples/index.html

Prerequisites

RTG comes as C source code and is intended to run on UNIX systems. RTG requires a UNIX system with POSIX thread support, 64-bit long integers and a sane compiler (gcc works great). RTG requires two external packages:

- MySQL (http://www.mysql.com/)
- Net SNMP (http://net-snmp.sourceforge.net/)

RTG can also use the older UCD SNMP, but we recommend sticking with Net SNMP.

To use the built in reports and configuration generators and to generally get the most out of RTG, it is strongly recommended that you also install:

- Apache (http://www.apache.org)
- PHP (http://www.php.net)
- Perl DBI (http://www.mysql.com/downloads/api-dbi.html)

Installing RTG in Debian Linux

Before Installing RTG we need to install some of the required packages

If you want to install apache2 with php support check here

#apt-get install libapache2-mod-php4 mysql-server php4-mysql php4-snmp rrdtool php4-pear webmin-mysql libmysqlclient14-dev libmysqlclient14 php4-pear webmin-mysql webmin-core php4-cgi php4-cli php4-gd php4-mysql php4-snmp libsnmp-base libsnmp5-dev libsnmp5 php4-snmp snmp gcc g++ make patch unp

Important Note:- Your Mysql Database root user should have password otherwise it will give error when we try to create database.

Now you need to download the Latest Stable RTG form here

We are dowloading version 0.7.4 under /usr/local/src/ directory

#cd /usr/local/src

#wget http://puzzle.dl.sourceforge.net/sourceforge/rtg/rtg-0.7.4.tar.gz

#unp rtg-0.7.4.tar.gz

#cd rtg-0.7.4

#./configure

#make

#make install

This will install all the configuration files under /usr/local/rtg folder

Now you need to copy rtgplolt file using the following command

# cp /usr/local/src/rtg-0.7.4/src/rtgplot /usr/local/bin/rtgplot

Configuring database

If you want to configure your database you must know the MYSQL root password. The createdb script will setup the database for RTG.

#/usr/local/rtg/etc/createdb mysqlroot

Configuring RTG

If you want to configure Rtg you need to edit two configuration files in /usr/local/rtg/etc folder i.e routers

router file

routers file, adding each router you wish to SNMP poll, one per line. To specify a per-router non-default community string, add "router:community" in the routers file. To specify a per-router non-default response bit width, add "router:community:bits" in the routers file.

Example

rtr-1.my.net
rtr-2.my.net:public:32
rtr-3.my.net:foobar:64

Target Maker file

Edit rtgtargmkr.pl(located at /usr/local/rtg/etc) to change the default SNMP read community if necessary. To use 64bit OIDs (recommended if supported by the device), change the "bits" field in rtgtargmkr.pl to 64.

rtgtargmkr.pl script to manage the RTG target file. The first run will create a targets.cfg file.

# ./rtgtargmkr.pl

This will create a targets.cfg file with all your routers and switches interfaces details in this.One more important thing is if you do any changes to your router configuration file you need to run the rtgtargmkr.pl otherwise it will not show the new changes after running this script you need to update your database to see these changes.When you run the perl script it will give the command name you need to run in mysql database command you can see one example as follows

Interface description changed.
Was: "global network 100Mbps"
Now: "global_network 100Mbps"
Suggest: UPDATE interface SET description='global_network 100Mbps' WHERE id=7

now you need to login in to your mysql database using the following command

#mysql rtg -p -u snmp
Enter PASSWORD
mysql> UPDATE interface SET description='global_network 100Mbps' WHERE id=7;

this will update the changes in your database.

Presently you need to do manually every time you do some changes and this one i don't like personally.

Starting Poller

Start up the poller to use the target list

# /usr/local/rtg/bin/rtgpoll -v -t /usr/local/rtg/etc/targets.cfg

If the poller does not find a configuration file, it will create one in the current directory called "rtg.conf". You may need to edit this file if your installation is non-standard. If the poller is successful, the "Polls" counter in the statistics banner will increase and then a countdown to the next poll is displayed. The DBInserts should increment after the second polling round.

If rtgpoll or rtgplot cannot find an rtg.conf file in any of these paths, it will attempt to create one in the current working directory.Most users maintain a master rtg.conf file customized for their environment. rtg.conf contains the following configurable fields:

Interval 300
HighSkewSlop 3
LowSkewSlop .5
OutOfRange 93750000000
SNMP_Ver 1
SNMP_Port 161
DB_Host localhost
DB_Database rtg
DB_User snmp
DB_Pass rtgdefault
Threads 5

Interval is the time between successive polls of the target list,default is 300 seconds (5 minutes). HighSkewSlop defines the maximum number of Intervals allowed between two consecutive poll values before the time in seconds between said points is deemed too large to calculate a valid rate. With the default Interval and HighSkewSlop values, that time would be 300 * 3 (15 minutes). LowSkewSlop defines the minimum number of Intervals alloweed between two consecutive poll values before the time in seconds between said points is deemed two low to calculate a valid rate. With the default Interval and LowSkewSlop values, that time would be 300 * 0.5 (2.5 minutes). OutOfRange defines an upper bound above which rtgpoll will never attempt an insert into the database.

OutOfRange should be a multiple of the maximum number of bytes possible in the defined Interval for your highest speed link. The default OutOfRange value will suffice in most installations. SNMP_Ver specifies the SNMP version the poller will use. The number of threads rtgpoll will use is defined in the variable Threads.

Variables in rtg.conf must match the names above exactly.Comments and blank lines are allowed and the ordering of variables in rtg.conf does not matter.

The target file specifies the objects to be SNMP polled. Comments must be preceded with a '#' sign. Elements in the target file are tab delimited.

The format of the target file is fixed:

# Host OID 64/32 Community Table ID Description

where

Host = IP or hostname of target
OID = Full SNMP OID, e.g. .1.3.6.1.2.1.31.1.1.1.10.19
64/32/0 = Specify 64/32 bit objects or 0 for gauge objects
Community = SNMP Community
Table = MySQL table in the database to use
ID = A unique ID that is used with each insert
Description = Free text

rtgpoll first reads the configuration file, then the target file.For each SNMP poll, rtgpoll will attempt an SQL INSERT of the form:

INSERT INTO Table VALUES (ID, NOW(), bigint)

Where Table is the name of the database table and ID is an integer. Both Table and ID come from the target list, NOW() is the current timestamp and bigint is the delta value between successive SNMP polls.

RTG makes no attempt at determining rate; one must look at the time difference successive entries in the database. The RTG graphing and reporting tools automatically calculate rate.

Adding poller to crontab

#crontab -e

@reboot /usr/local/rtg/bin/rtgpoll -v -t /usr/local/rtg/etc/targets.cfg -c /usr/local/rtg/etc/rtg.conf

The above command automatically starts when you reboot your server.

Web interface and Graphs

your configuration is ready and now you need to configure the web interface to view graphs and 95th percentile data for this you need to configure the following steps

#cp /usr/local/rtg/web/ /var/www/rtgweb/

Copy the rtgplot binary to the /var/www/rtgweb/ directory and *add* .cgi

#cp /usr/local/rtg/bin/rtgplot /var/www/rtgweb/rtgplot.cgi

or

#ln -i -s /usr/local/bin/rtgplot /var/www/rtgweb/rtgplot.cgi

Now you need to create a apache2 website enable file name rtg with the following content

# vi /etc/apache2/sites-enabled/rtg

<Directory "/var/www/rtgweb/">

AllowOverride AuthConfig
AddType application/x-httpd-php .php
AddHandler cgi-script .cgi
Options +ExecCGI

</Directory>

Save and exit this file.Now you need to enable this site using the following

#a2ensite rtg

Restart apache server

# /etc/init.d/apache2 restart

Now you need to secure rtg web interface with .htaccess file for this we need to create a .htaccess file in /var/www/rtgweb/ directory

#cd /var/www/rtgweb/

#touch .htaccess

Now you need to enter the content in .htaccess file as follows

AuthUserFile /var/www/rtgweb/.htpasswd
AuthName Members
AuthType Basic
require valid-user

Now you need to create admin user or users using the following command

#htpasswd -b /var/www/rtgweb/.htpasswd masteradmin password

the above command will create a username as masteradmin with password as password.

You can access rtg using the following command

http://yourserverip/rtgweb/

Point your web browser at the URL (substitute your site in)

http://yourserverip/rtgweb/rtg.php (Interactive Reports)
http://yourserverip/rtgweb/95.php (95th Percentile Queries)
http://yourserverip/rtgweb/view.php (MRTG-style overview + day/wk/mo plots)

RTG Customer Web interface Tools

1) http://www.zhaba.kiev.ua/rtg/

2) RTGums

The RTG User Management System is a wrapper for RTG which allows Datacenters to give thier customers real-time views into link utilization and trending analysis.

http://www.zhaba.kiev.ua/rtg/

3) RTG Catam

http://traffic.carpathiahost.com/rtg-0.7.4_catam_customers/

RTG db split per month using MERGE mysql tables

http://gogu:c4rp4th1a@traffic.carpathiahost.com/rtg-0.7.4_catam_mysql_merge_tables

targetmaker

targetmaker is a drop in replacement for the rtgtargmkr.pl application that comes with RTG. It greatly increases the functionality of the default application and adds a great deal more automation.

http://www.misplaced.net/~btoneill/rtg/