Collectd Graph Panel v0.4

After 2,5 years and about 100 commits I’ve tagged version 0.4 of Collectd Graph Panel.

This version includes a new interface with a sidebar for plugin selection.

Javascript library jsrrdgraph has been integrated. Graphs will be rendered in the browser using javascript and HTML5 canvas by setting the “graph_type” configuration option to “canvas”. This saves a lot of CPU power on the server. Jsrrdgraph has some nice features. When rendered, you can move through time by dragging the graph from left to right and zoom in and out by scrolling on the graph.

Demo:

The Collectd compatibility setting has been changed to Collectd 5. If you’re still using Collectd 4, please set the “version” configuration setting to “4”, otherwise the graphs of a couple plugins won’t be showed right (like the interface, df, users plugins).

In this version of CGP, total values are added to the legend of I/O graphs and generated colors will be created using a rainbow palette instead of 9 predefined colors. Please read the changelog or git log for more information about the changes.

New plugins:

Special thanks for this version go to Manuel Luis, who developed jsrrdgraph, xian310 for the new interface, Manuel CISSÉ, Rohit Bhute, Matthias Viehweger, Erik Grinaker, Peter Chiochetti, Karol Nowacki, Aurélien Rougemont, Benjamin Dupuis, yur, Philipp Hellmich, Jonathan Huot, Neptune Ning and Nikoli for their contributions.

I’ve been using GitHub for a while now. You can download or checkout this version of CGP from the GitHub URL’s below. When you have improvements or fixes for CGP, don’t hesitate to send in a Pull Request on GitHub!

v0.4.1 update

I just removed the dependency on mod_rewrite when using jsrrdgraph to draw the graphs. This may solve javascript error: Invalid RRD: “Wrong magic id.” undefined.

githubGitHub: https://github.com/pommi/CGP
Download: https://github.com/pommi/CGP/archive/v0.4.1.zip
Git: git clone https://github.com/pommi/CGP.git

56 thoughts on “Collectd Graph Panel v0.4

  1. Conrad

    Pretty Cool! :)
    Hope, there will be in future also plugins for openvpn and thermal :)

  2. Tom

    Hi!
    I’ve check out the newest version from git. I’ve enabled canvas mode :)
    But it still give me only Invalid RRD: “Wrong magic id.” undefined. :(

    Is this, because I am using nginx? I don’t have apache and mod_rewrite

  3. pommi Post author

    Hi Tom! mod_rewrite isn’t required, so that shouldn’t be the problem.

    I’ll try to explain what is going on. With canvas enabled, the graphs are created by javascript. To provide the rrd files to javascript, I created rrd.php. So javascript downloads the needed RRD file for the Load graph via for example: http://domain.tld/rrd.php/your-hostname/load/load.rrd. When you look at the HTML source you can see what RRD files are downloaded between the canvas tags.

    You’ll get this message when downloading the required RRD files fails some way. To debug this problem, please download one of the RRD files manually and see what you get. An error?

  4. Tom

    Hi Pommi,
    thanks for your anwer.

    When I try to load : http://www..de/cgp/rrd.php//load/load.rrd i am getting a 404 error my nginx.

  5. Tom

    Sorry, forgot to attach log :/

    2013/06/01 00:02:30 [error] 25941#0: *253 open() “/var/www//htdocs/cgp/rrd.php//load/load.rrd” failed (20: Not a directory)

  6. Conrad

    Hi,
    I’ve tried to create an openvpn-plugin. But is only half working :/
    I get the users graph, but the correct ds-name is not set.

    The traffic graph seems not to be created. PHP-Error says:
    Undefined index in GenericIO.class.php on line 46
    Undefined index in GenericIO.class.php on line 46
    (also line 52,53,50,60)

    Don’t know why. Could you have a look?

    http://pastebin.com/5vNDuKaZ

    Structure is:
    openvpn-
    openvpn-/if_octets-.rrd
    openvpn-/users-.rrd

    Thanks!

  7. l13t

    Also i’ve trouble with NUT plugin.
    I’ve errors when web interface try to show information about UPS frequency.
    There is difference between original variant of plugin and my resolution of problem.

    diff cgp.orig/plugin/nut.php cgp/plugin/nut.php master * ] 11:18 AM
    20c20
    data_sources = array(‘value’);

    > $obj->data_sources = array(‘frequency’);

  8. yeah

    is this new version compatible with the rrd format created by collectd 4.10? 4.10 is what’s available for redhat/centos 6. for my centos 6 hosts, graphs aren’t being displayed for df, interface, ntpd, and users. cgp-0.3 displays all graphs just fine. also, cgp-0.4 displays all graphs fine on my debian wheezy hosts.

  9. pommi Post author

    hi yeah, please read the blog again, and in particular the part about collectd 4…

  10. yeah

    yep, actually fully reading the blog post and implementing the needed change in config.local.php worked great. sorry to bother…

  11. Conrad

    Btw pommi, can you have a look at my git pull? :) Would be nice, if this could be merged for nginx-users.

    Cheers,
    Conrad

  12. pommi Post author

    l13t: tnx for your post about the NUT plugin. It should be fixed now.

  13. Tasslehoff

    Hi, I got an error that seems like the problem Conrad reported.

    I configured collectd for a tplink access point interfaces via snmp plugin, rrd file seems fine but collectd returns these errors on apache error_log:
    PHP Notice: Undefined index: eth0-rx in /cgp/type/GenericIO.class.php on line 59, referer: https://serverhostname/cgp/host.php?h=tplink.local&p=snmp

    I’m using the last version of cgp (via git) and collectd version 5.1.0 (debian wheezy main repository).

    Any suggestion?

    Thanks!

  14. Conrad

    Hi!
    There seems to be a problem with ping :/ When I open it, I can see a graph for latency, but there is also an popup with an error “Unknown argument: undefinied”

    PHP Logs here: http://pastebin.com/Rt7JCNa6

    Conrad

  15. Nik

    Hi.
    Goog products.
    But when graph_type is “canvas” I can’t open details graph (detail.php)

  16. pommi Post author

    Hi Nik, indeed, this isn’t possible at the moment.

    @Conrad: fixed!

  17. Conrad

    Thank you for fixing :)
    Btw, it would be very nice, if it would be possible to define the names for myself (vertical_label, ds-names), like its possible with collection3 iirc.

  18. djtremors

    Works nicely as expected. Was looking and liking jarmon which had tabs and dynamic building of the page without having to reload the whole page. only downside of the lacking examples of how to support multiple hosts and configs to suit.
    Although I managed to get Jarmon working with multiservers, configs were a pain.
    CGP simply reads all the dirs/files and renders whats available as any gui should.

    Only would like to suggest a more dynamic page where jquery to load the page would make this more ultimate. Also give ability to segregate the areas like jarmon, ie system, network, disk, etc as so far I see only a page reload possible.

  19. Radu Rambet

    Great plugin. Just awesome.

    Would it be possible to write additional plugin for openwrt type of data
    Collectd is great for embedded applications like openwrt but theirs plugin are a little bit different and especially designed for wireless.

    I would like to be Abe to write myself additional plugin for iwinfo for instance. Is there any tutorial how to add additional plugin to graph panel v. 4?

    Thank you!

  20. djtremors

    For those of you who have nginx and issues with collectd 5 with canvas mode still getting “wrong magic id”. this is a workaround I wrote since rrd.php expects $_SERVER['PATH_INFO'] to exist which is Apache only. I fudged the 2 files below to make this work with no rewrite or anythign and works for my nginx setup.

    I’ve commented out one line and added one below, further below added a section under the EITHACK comment.

    “type/Default.class.php”
    function parse_filename($file) {
    if ($this->graph_type == 'canvas') {
    // EITHACK
    //$file = 'rrd.php/' . str_replace($this->datadir . '/', '', $file);
    // EITHACK
    $file = 'rrd.php?' . str_replace($this->datadir . '/', '', $file);
    }
    return $this->rrd_escape($file);
    }

    “rrd.php”
    require_once 'inc/html.inc.php';
    // EITHACK
    if (!isset($_SERVER['PATH_INFO']))
    {
    $_SERVER['PATH_INFO']=substr($_SERVER['REQUEST_URI'],strpos($_SERVER['REQUEST_URI'],"rrd.php")+8);
    }
    if ($file = validateRRDPath($CONFIG['datadir'], $_SERVER['PATH_INFO'])) {

  21. pommi Post author

    @Philipp: nope, unfortunately not. I test every plugin I initially add. So I need to install redis, connect collectd and understand a little bit what is going on. This is time consuming… But when I have time, I add it ;-)
    @djtremos: Please have a look at https://github.com/pommi/CGP/pull/11, for more info about the PATH_INFO variable and Nginx. Thanks for the jquery suggestion. Too bad I’m not a very good frontend developer. Thomas Harold has done some work on jquery. You can probably try his version of CGP some time.
    @Radu: Which plugins aren’t working with OpenWRT, only iwinfo? About writing your own CGP plugin, just try to copy the php of a plugin that looks a bit alike, and change/adjust it to make it work.

  22. gdi67

    I can’t drag the graph like you did in the video. I installed CGP in the NGINX with php-fpm. All of them are in the latest version. Do I need any further configuration?

  23. Radu

    Problem solved .
    iwinfo IS working on openwrt it is just it does not have proper php file on my server.(Debian) I just updated by hand and everything is ok.

    Abut my own plugins i read a bit and managed to do it myself .Easy.
    Great work anyhow

  24. tom wang

    i am using 0.4 and 0.41 to monitor our test website, it reported that error: could not parse color in ‘ ‘ on the interface rrd file. I do not know what is wrong, anyone could help?

  25. Alex Juarez

    Just for anybody else who comes across the “wrong magic id” issue. This is an example of the fastcgi_split_path_info that worked for me.

    location ~ ^(.*\.php)(.*)$ {
    root /var/www/html;
    fastcgi_pass 127.0.0.1:9000;
    fastcgi_index index.php;

    fastcgi_split_path_info ^(.+php)(.*)$;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param PATH_INFO $fastcgi_path_info;
    include fastcgi_params;
    }

  26. Didi

    Hi,

    I want to make a Python Plugin which should displayed like the load plugin… How can I realise this? I can make “normal” graphes but this one with just lines like the load plugin I cant… i hope somepne can help me ;)

  27. chris

    Hi,

    I want to use the mongodb write plugin, is there a chance to use CGP with a mongo data source?

  28. pommi Post author

    Hi Chris, currently there is no support in CGP for that.

    @Markus: See js/RrdGraph.js; search for font.

  29. Boris

    Hello,
    thanks a lot for your last release. canvas option is very useful.
    Do you forecast to develop a plugin for jmx rrd files >> GenericJMX?
    It would be quite interesting to know about the memory used by my tomcat server(garbage collector…).

  30. Tasslehoff

    Hi, I have a suggestion.
    Is it possibile to leave the 2hr/8hr/day/week/month/ etc etc links also when selecting graph_type=canvas?

  31. Pingback: Simple web based system monitoring on Ubuntu Server : cognito.me.uk

  32. Ilya

    Hi, first of all thanks for the great interface.
    One question: I wrote a simple script which is executed via exec plugin and returns the ldap response types:
    PUTVAL myhost/LdapSearch/duration interval=10 N:41
    In the interface I am getting an error. Is there any easy way to include some arbitrary exec plugin or any workaround for me to see that graph?

  33. Ilya

    Yes, I got no error, but the graph simply was not there. Now I got it: There need to be a php file for each plugin one needs (as ldapsearch was a custom one, I simply had to write one ;) ). so I modified the load plugin the following way

    data_sources = array('seconds');
    $obj->ds_names = array(
    'seconds' => 'seconds',
    );
    $obj->colors = array(
    'seconds' => '00ff00',
    );
    $obj->rrd_title = 'Request Duration';
    $obj->rrd_vertical = 'Seconds';
    $obj->rrd_format = '%.2lf';

    collectd_flush($obj->identifiers);
    $obj->rrd_graph();

    Only one question (I hardly speak php): Is there a way to put the list of machines supervised by collectd (as shown on the index page) beside the plugin list?

    Thanks a lot

  34. pommi Post author

    @Ilya please download the latest version of CGP. I’ve dropped the PHP file per plugin recently. This is not required anymore. The plugin PHP files have been replaced by JSON files that just makes the graphs prettier.

  35. FamilyMan

    Hey pommi. Great work!
    I’m not aware about web servers and PHP but I’d like to setup CGP to monitor collectd statictics from time to time. Can you please share with me a short guide how can I do it based on nginx, apache, any other web server software?
    P.S. My system is up-to-date ArchLinux x64, I can install and configure everything on it.
    Btw, I’d prepare a package for Arch if you explain me what and where shoud be copied/installed, explain dependencies etc.

  36. FamilyMan

    Hi pommi,
    I found an explanation (http://blog.cognito.me.uk/computers/simple-web-based-system-monitoring-on-ubuntu-server/) and have installed the lighttpd + CGP.
    Then I got an error: Error: No Collectd hosts found in /var/lib/collectd/rrd
    After a short research I found the answer:

    Hi Xan, This message can mean 2 things:
    1. Your collectd datadir is not in /var/lib/collectd/rrd. Set $CONFIG['datadir'] in conf/config.local.php to set the correct one.
    2. Your webserver user (for example: user www-data when Debian/Apache2) is not allowed to read the files in /var/lib/collectd/rrd

    The #1 is not my case for sure.
    Regarding #2. The /var/lib/collectd/rrd contains the data for sure:

    # ls -R /var/lib/collectd/rrd
    /var/lib/collectd/rrd:
    localhost.localdomain

    /var/lib/collectd/rrd/localhost.localdomain:
    cpu-0 hddtemp interface-lo interface-net1 memory sensors-atk0110-acpi-0
    cpu-1 interface-cipsec0 interface-net0 load nut sensors-coretemp-isa-0000
    [skip]

    The #2 I’ve tried to fix by set chown -R http.http on /var/lib/collectd:

    ls -lR /var/lib/collectd

    /var/lib/collectd/rrd:
    total 4
    drwxr-xr-x 14 http http 4096 Jul 27 10:33 localhost.localdomain

    /var/lib/collectd/rrd/localhost.localdomain:
    total 48
    drwxr-xr-x 2 http http 4096 Jul 26 19:04 cpu-0
    drwxr-xr-x 2 http http 4096 Jul 26 19:04 cpu-1
    [skip]

    But it doesn’t help.
    The lighttpd is running under http.http … Any thoughts?

  37. FamilyMan

    Ok, I found the answer: php.ini should be updated to allow access into /var/lib/collectd/rrd and /usr/bin folders.

  38. suli

    Hello,

    I am running collectd on my router and I don’t need to view the statistics real time. So I copy the /collectd/rdd to my windows laptop, download the rddtool for windows (with cygwin) and run CGP in XAMMP.
    It doesn’t surprise me that I can not get it work straightly with path like “C:\collectd\rdd” and “C:\collectd\rddtool”.

    What format of path should I put in config.local.php file, so it can be recogonized by CGP in XAMMP on windows?

    thanks a lot.

  39. suli

    Forget the XAMMP on windows, I saved my life by intalling it on a debian VM. And then I managed to make it run on my openwrt box too (uhttpd+php+rrdtool), with a few code fix ( although it take me a long time).

    My suggestion for someone else have troubles running CGP:
    1. make sure your collectd work well. Go to the rrd file directory “ls -l” to corfirm that all rrd files are there and updated
    2. Follow the address of the error page. If it is a broken image icon, right click it and copy the image link, paste it into a new browser window to see the detail.
    3. There are php file name in the link. Go to /www/CGP directory, find the exact php file, open it with edit, try to understand how it work. Follow the procedure, check output of every variable using echo and print_r. Google the function or the output error message to understand how it work and the reason of error, so you can know why it didn’t work as you expected and then google the fix solution.
    4. The working copy on the vm is very useful, as it provide something I can compare to.
    5. understand how collectd manage its plugin and rrd file is also required

    I am not a php programmer and not a native English user either, so it is not as difficult as you think ^_^

  40. pommi Post author

    Fantastic suli! This is amazing. :D I saw your 1st reply a month ago and thought: OMG not Windows :-? I was pleasantly surprised when red you installed it on Debian! 8-) Great way to go. Thanks for your debugging steps as well.

  41. suli

    Some information about collectd csv file to graphic :

    RRD is great for trend, but I have some data saved to csv so that I can retrieve the exact value. So I need some tools that can get the csv file from server and make chart out of it. Finally, I find dygraphs, it seems just suit my needs. I will do some homework to make it work for me, or even try to combine it with CGP code (the part that phrase the collectd data directory)

    Maybe someone have done this already, please let me know, thanks.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>