Category Archives: Collectd Graph Panel

Linux bcache SSD caching statistics using collectd

In October 2012 I started using bcache as an SSD caching solution for my Debian Linux server. I’ve been very happy about it so far. Back then I used a manually compiled 3.2 Linux kernel based on the bcache-3.2 git branch provided by Kent (which has been removed). This patch needed to be applied to make bcache work with grsecurity. I also created a Debian package of the bcache-tools userspace tools to be able to create the bcache setup.

At the start of this year I moved to a 3.12 kernel, also manually compiled. It’s quiet a relief that bcache is included in mainline since the 3.10 kernel. :)

This is my setup:

  1. 500GB backing device – 20GB caching device (qcow2 images)
  2. 1.3TB backing device – 36GB caching device (file storage)

The past year I’ve definitely noticed the performance difference using bcache. But I was still curious about when and how bcache was using the attached SSD. Is it using the write-back cache a lot? How many times can bcache read it’s data from the SSD cache instead of accessing the HDD?

I created a python script to collect all kinds of bcache statistics (parts of the code in this script are copied from bcache-status). This script outputs the statistics to STDOUT in a collectd exec plugin compatible way. The collectd exec plugin can be configured in collectd.conf this way:

<Plugin exec>
Exec "user:group" "/path/to/collectd-bcache"
</Plugin>

To visualize the collected data I created a bcache plugin for CGP. This is the result:

bcache-cache-hit-ratio bcache-access bcache-usage bcache-bypassed

Write-back to HDD throttled

At some time I noticed that in my case flushing data from the write-back cache to the HDD was somehow rate-limited to ~3 MB/s. You can nicely see this in these graphs:

bcache-dirty-data bcache-throughput

These threads on the mailinglist of bcache mention the same thing:

Kent explained that this is managed by the PD controller in bcache. The PD controller has been rewritten in the 3.13 Linux kernel, so I’m very interested if this behavior changed. I didn’t upgrade my kernel to 3.13 yet because I’m a very cautious about it. Still a lot of development is going on at the bcache project. But I’m looking forward to upgrading to 3.13, 3.14 or probably 3.15.

githubcollectd-bcache: github.com/pommi/collectd-bcache
CGP bcache plugin: github.com/pommi/CGP/…/bcache.php

Collectd Graph Panel v0.4

After 2,5 years and about 100 commits I’ve tagged version 0.4 of Collectd Graph Panel.

This version includes a new interface with a sidebar for plugin selection.

Javascript library jsrrdgraph has been integrated. Graphs will be rendered in the browser using javascript and HTML5 canvas by setting the “graph_type” configuration option to “canvas”. This saves a lot of CPU power on the server. Jsrrdgraph has some nice features. When rendered, you can move through time by dragging the graph from left to right and zoom in and out by scrolling on the graph.

Demo:

The Collectd compatibility setting has been changed to Collectd 5. If you’re still using Collectd 4, please set the “version” configuration setting to “4″, otherwise the graphs of a couple plugins won’t be showed right (like the interface, df, users plugins).

In this version of CGP, total values are added to the legend of I/O graphs and generated colors will be created using a rainbow palette instead of 9 predefined colors. Please read the changelog or git log for more information about the changes.

New plugins:

Special thanks for this version go to Manuel Luis, who developed jsrrdgraph, xian310 for the new interface, Manuel CISSÉ, Rohit Bhute, Matthias Viehweger, Erik Grinaker, Peter Chiochetti, Karol Nowacki, Aurélien Rougemont, Benjamin Dupuis, yur, Philipp Hellmich, Jonathan Huot, Neptune Ning and Nikoli for their contributions.

I’ve been using GitHub for a while now. You can download or checkout this version of CGP from the GitHub URL’s below. When you have improvements or fixes for CGP, don’t hesitate to send in a Pull Request on GitHub!

v0.4.1 update

I just removed the dependency on mod_rewrite when using jsrrdgraph to draw the graphs. This may solve javascript error: Invalid RRD: “Wrong magic id.” undefined.

githubGitHub: https://github.com/pommi/CGP
Download: https://github.com/pommi/CGP/archive/v0.4.1.zip
Git: git clone https://github.com/pommi/CGP.git

Collectd Graph Panel v0.3

It has been a while and many people were already using one of the development versions of CGP. Time to release a new version of CGP: v0.3.

Special thanks for this version go to Manuel CISSÉ, Jakob Haufe, Edmondo Tommasina, Michael Stapelberg, Julien Rottenberg and Tom Gallacher for sending me patches to improve CGP. Also thanks to everyone that replied after the previous release.

In this version there are a couple of minor improvements and some new settings to configure. But the most important: Support has been added for 13 Collectd plugins. CGP now supports 26 Collectd plugins. Also a couple of plugins have been updated. Please read the changelog or git log for more information about the changes.

New plugins:

Download the .tgz package, the patch file to upgrade from v0.2 or checkout the latest version from the git repository.

Download: http://pommi.nethuis.nl/…/cgp-0.3.tgz
Patch: http://pommi.nethuis.nl/…/cgp-0.2-0.3.patch.gz
Git: http://git.nethuis.nl/pub/cgp.git

Collectd Graph Panel v0.2

Version 0.2 of Collectd Graph Panel (CGP) has been released. This version has some interesting new features and changes since version 0.1.

A new interface is introduced, based on Daniel Von Fange’s styling. A little bit of web 2.0 ajax is used to expand and collapse plugin information on the host page. The width and heigth of a graph is configurable, also the bigger detailed one. UnixSock flush support is added to be able to let collectd write cached data to the rrd file, before an up-to-date graph of the data is generated. CPU support for Linux 2.4 kernel and Swap I/O support is added. And some changes under the hood.

Download the .tgz package, the patch file to upgrade from v0.1 or checkout the latest version from my public git repository.

CGP Overview Page CGP Server Page CGP Detailed Page

Download: http://pommi.nethuis.nl/…/cgp-0.2.tgz
Patch: http://pommi.nethuis.nl/…/cgp-0.1-0.2.patch.gz
Git: http://git.nethuis.nl/pub/cgp.git

Collectd Graph Panel v0.1

Collectd Graph Panel (CGP) is a graphical web front-end for Collectd written in php. I’ve developed CGP because I wasn’t realy happy with the existing web front-ends. What I like to see in a front-end for Collectd is clear overview of all the hosts you are managing and an easy way to get detailed information about a plugin.

This first release is a very basic version of what I have in mind. Not all plugins are supported yet and the user interface can be done better. Still a lot to work on!

Please give it a try! It should work out-of-the-box. Download the tgz package or checkout the latest version from my public git repository.

Download: http://pommi.nethuis.nl/…
Git: http://git.nethuis.nl/pub/cgp.git