In previous posts we talked about installing graphite. Now we should send it some data to store, and then start to play with the webapp. Graphite can graph lots of things, but we will start with the basics: the state of a linux server.
We will decide if a server is healthy or not looking at the typical things in a linux: memory, disk space, CPU, iowait and disk read/write operations. There are a lot of ways to gather that information. I’ve chosen a shell-script executing standard commands (free, sar and awk, basically) and then sending then to carbon via netcat. You can find the script at github in graphite-monitoring repository, but I will paste it here as well because is simple and short:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
#!/bin/bash #Carbon server where data should be stored for graphite to show - El servidor carbon on s'han de guardar les dades que mostra el graphite carbon_server=graphite.domain.tld # Tree structure where we want information to be stored - L'estructura de l'arbre on volem que es guardin les dades a graphite. tree="servers" #In this case, info will be shown in graphite as "servers.servername.loadavg_1min". We could use "pro" and "pre" to separate environments: "servers.pro.servername.loadavg_1min" - En el nostre cas es veuran a "servers.servername.loadavg_1min". Podriem posar "prod" i "pre" per separar entorns: "servers.pro.servername.loadavg_1min" now=`date +%s` host=`hostname` #Load average - Carrega read un cinc quinze resta < /proc/loadavg data="$tree.$host.loadavg_1min $un $now n $tree.$host.loadavg_5min $cinc $now n $tree.$host.loadavg_15min $quinze $now n" #Memory - Memoria data="$data `free -o|awk -v host=$host -v now=$now '(NR==2) {printf("servers.%s.memory %s %s \n ", host, $3/$2*100, now)} (NR==3) {printf("servers.%s.swap %s %s\n ", host, $3/$2*100, now)}'`" #CPU Used - Recollim CPU data="$data `sar -u 3|awk -v host=$host -v now=$now 'END {printf("servers.%s.cpu %s %s \n ", host, 100-$8, now)}'`" #Disk data - Recollim dades de disc data="$data `sar -b 3|awk -v host=$host -v now=$now 'END {printf("servers.%s.disk.totalops %s %s \n servers.%s.disk.readops %s %s \n servers.%s.disk.writeops %s %s \n servers.%s.disk.breads %s %s \n servers.%s.disk.bwrites %s %s \n ", host, $2, now, host, $3, now, host, $4, now, host, $5, now, host, $6, now)}'`" #Show data for debugging purpose - Mostrem les dades per depurar errors echo $data #Send data to graphite - Enviem dades a graphite echo -e $data |nc -w 5 $carbon_server 2003 2>&2 exit $? |
Now we just need to execute this regularly, using cron for instance. I run it with monit (someday I will talk about it), because it handles execution errors. And with that we will have some useful data in graphite to start to play.