Apache and Munin

Posted: 13/12/2010 in System Administration
Tags: , ,

Hi,

during last months a very interesting question comes from my job: how fast is a site? This may appear as a speculative question, but it harder than you could think.

I’m going to show an example: you have a site who needs uptime 24/7 but during the night from a part of earth someone says to you that your site is unavailable. In this case, all sysadmins starts to install their loved tools to monitoring the connections, traffic, load, cpu  usage and else.

Intro

In my case this tool is munin, an these are my self developed plugins:

  • apache latency aggregation
  • apache latency (max, avarange)
  • apache http coder

At this moment they are very poors because are something  like an alfa release, but I think to make them better asap.

This is a screenshot (click on it to see larger):

The first is graph show how many pages are served in time: in this way you can evaluate not only if your site is fast but how many cases it is slow.

The second one show the max latency, the averange latency and the number of errors in apache.

These two graph works togheter: if the max latency is high but the avarange is low may means nothing because it not evaluate how many high-latency pages are used, for example.

The third is related http response codes.

How to use

I should say that they are not very clear, but I’ll clear it asap. To simplify the manage, I decided to create a script called “generate.py” to create all munin needs. After downloading file, you have to untar the file in some folder (/opt/MuninPlugins for me, but you can choose another one). Next step is to run generate.py, but is necessary to explain a thing about that: this script is realized for a Debian installation (ubuntu is the same) and gets all it needs by /etc/apache/sites-enabled folder. If you have a differend situation, please contact me to get help, because it’s a little bit boring explain what it does. But what you have to know is that generate.py creates a list of symbolic links from /etc/munin/plugins to the script folder you have created.

The scripts are divided in two types: runners and workers. Runners scripts manipolate the parameteres for workers, and workers returns the result for munin.

Call a worker is very easy, is enough to call passing a title, a group and an apache log file (zipped or not). This is an example:

$ worker_http.py 'Http codes' Apache /var/log/apache2/access.log

Some scripts need a specific LogFormat that contains response time, this is what I used:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D" combined_with_time

Look in worker file to exactly how to use.

Runners are a little bit complicated: a runner script takes its parameters as difference from symbolic link name and real file name. This is an example:

$ ls -al runner_http_www.reflab.com.80_apache_www.reflab.com_access.log.py
lrwxrwxrwx 1 root root 46 2010-12-13 09:32 runner_http_www.reflab.com.80_apache_www.reflab.com_access.log.py -> /opt/MuninPlugins/runner_http.py

When you call it, the script parses the symlink name getting  www.reflab.com.80 as title, apache as group and www.reflab.com_access.log as apache log file. After that it calls the worker.

For more details, please contact me.

Download

Check it on github.

Cheers.

Advertisements
Comments
  1. Jorg Rødsjø says:

    I know this is an old post, but I just found it, and it looks really useful! But the download-link is broken:/ Is there any chance of getting a copy via another channel?

  2. cippino says:

    I uploaded it on github. May be it differs from the article. Contact me if you need.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s