Category Archives: Development

Collectd Graph Panel v1

v1 is here. CGP is finished ūüėÜ

Joking aside. It has been requested multiple times. So let’s get it over with. The last version was more then 3.5 years ago. This will be the last tagged version of CGP. Every commit in the master branch after this release can be considered as a new release. ūüėČ

Use git and “git pull” to keep up-to-date or download the latest version here.

Notable Changes since v0.4.1:

  • mobile support (responsive design)
  • automatic support for all plugins (markup/styling in json)
  • hybrid graph type (canvas graph on detail page, png on the others)
  • svg graph support
  • support for newer PHP versions
  • deprecate support for collectd 4

Special thanks for this version go to Peter Wu for improving security, Manuel Luis for maintaining jsrrdgraph and Vincent Brillault for his amount of contributions.

Git: git clone

Linux bcache SSD caching statistics using collectd

In October 2012 I started using bcache as an SSD caching solution for my Debian Linux server. I’ve been very happy about it so far. Back then I used a manually compiled 3.2 Linux kernel based on the bcache-3.2 git branch provided by Kent (which has been removed). This patch needed to be applied to make bcache work with grsecurity. I also created a Debian package of the bcache-tools userspace tools to be able to create the bcache setup.

At the start of this year I moved to a 3.12 kernel, also manually compiled. It’s quiet a relief that bcache is included in mainline since the 3.10 kernel. ūüôā

This is my setup:

  1. 500GB backing device – 20GB caching device (qcow2 images)
  2. 1.3TB backing device – 36GB caching device (file storage)

The past year I’ve definitely noticed the performance difference using bcache. But I was still curious about when and how bcache was using the attached SSD. Is it using the write-back cache a lot? How many times can bcache read it’s data from the SSD cache instead of accessing the HDD?

I created a python script to collect all kinds of bcache statistics (parts of the code in this script are copied from bcache-status). This script outputs the statistics to STDOUT in a collectd exec plugin compatible way. The collectd exec plugin can be configured in collectd.conf this way:

<Plugin exec>
Exec "user:group" "/path/to/collectd-bcache"

To visualize the collected data I created a bcache plugin for CGP. This is the result:

bcache-cache-hit-ratio bcache-access bcache-usage bcache-bypassed

Write-back to HDD throttled

At some time I noticed that in my case flushing data from the write-back cache to the HDD was somehow rate-limited to ~3 MB/s. You can nicely see this in these graphs:

bcache-dirty-data bcache-throughput

These threads on the mailinglist of bcache mention the same thing:

Kent explained that this is managed by the PD controller in bcache. The PD controller has been rewritten in the 3.13 Linux kernel, so I’m very interested if this behavior changed. I didn’t upgrade my kernel to 3.13 yet because I’m a very cautious about it. Still a lot of development is going on at the bcache project. But I’m looking forward to upgrading to 3.13, 3.14 or probably 3.15.

CGP bcache plugin:…/bcache.json

Mendix shipped in a Docker container

Imagine… Imagine if you could setup a new Mendix hosting environment in seconds, everywhere. A lightweight, secure and isolated environment where you just have to talk to a RESTful API to deploy your MDA (Mendix Deployment Archive) and start your App.

Since the 2nd quarter of this year a great piece of software became very popular to help to achieve this goal: Docker. Docker provides a high-level API on top of Linux Containers (LXC), which provides a lightweight virtualization solution that runs processes in isolation.

Mendix on Docker


Run a Mendix App in a Docker container in seconds:

root@host:~# docker run -d mendix/mendix
root@host:~# curl -XPOST -F model=@project.mda
File uploaded.
root@host:~# curl -XPOST
Runtime downloaded and Model unpacked.
root@host:~# curl -XPOST -d "DatabaseHost=" -d "DatabaseUserName=docker" -d "DatabasePassword=docker" -d "DatabaseName=docker"
Config set.
root@host:~# curl -XPOST
App started. (Database updated)


There has been a lot of buzz around Docker since its start in March 2013. Being able to create an isolated environment once, package it up, and run it everywhere makes it very exciting. Docker provides easy-to-use features like Filesystem isolation, Resource isolation, Network isolation, Copy-on-write, Logging, Change management and more.

For more details about Docker, please read “The whole story”. We’d like to go on with the fun stuff.

Mendix on Docker

Once a month a so-called FedEx Day (Research Day, ShipIt day, Hackatron) is organized at Mendix. On that day, Mendix developers have the freedom to work on whatever they want. We’ve been playing with Docker a couple of Research Day’s ago. Just see how it works, that kind of stuff. But this time we really wanted to create something we’d potentially use in production. A proof of concept how to run Mendix on Docker.

The plan:

  1. Create a Docker Container containing all software to run Mendix
  2. Create a RESTful API to upload, start and stop a Mendix App within that container

What about the database, you may be wondering? We’ll just use a Docker container that provides us a PostgreSQL service! You can also build your own PostgreSQL container or use an existing PostgreSQL server in your network.

Start off with an image:


This is what we are building. A Docker container containing:

  • All required software to run a Mendix App, like the Java Runtime Environment and the m2ee library
  • A RESTful API (m2ee-api) to upload, start and stop an App (listening on port 5000)
  • A webserver (nginx), to serve static content and proxy App paths to the Mendix runtime (listening on port 7000)
  • When an App is deployed the Mendix runtime will be listening on port 8000 locally

Building the base container

Before we can start to install the software, we need a base image. A minimal install of an operating system like Debian GNU/Linux, Ubuntu, Red Hat, CentOS, Fedora, etc. You could download a base container from the Docker Index. But because this is so basic and we’d like to create a Mendix container we can trust 100% (a 3rd party base image could contain back-doors), we created one ourselves.

A Debian GNU/Linux Wheezy image:

debootstrap wheezy wheezy
tar -C wheezy -c . | docker import - mendix/wheezy

That’s all! Let’s show the image we’ve just created:

root@host:~# docker images
mendix/wheezy    latest    1bee0c7b9ece   6 seconds ago     218.6 MB

Building the Mendix container

On top of the base image we just created, we can start to install all required software to run Mendix. Creating a Docker container can be done using a Dockerfile. It contains all instructions to provision the container and information like what network ports to expose and what executable to run (by default) when you start using the container.

There is an extensive manual available about how to run Mendix on GNU/Linux. We’ve used this to create our Dockerfile. This Dockerfile also installs files like /home/mendix/.m2ee/m2ee.yaml, /home/mendix/nginx.conf and /etc/apt/sources.list. They must be in your current working directory when running the docker build command. All files have been published to GitHub.

To create the Mendix container run:

docker build -t mendix/mendix .

That’s it! We’ve created our own Docker container! Let’s show it:

mendix/mendix    latest    c39ee75463d6   10 seconds ago    589.6 MB
mendix/wheezy    latest    1bee0c7b9ece   3 minutes ago     218.6 MB

Our container has been published to the Docker Index: mendix/mendix


When you look at the Dockerfile, it shows you it’ll start the m2ee-api on startup. This API will listen on port 5000 and currently supports a limited set of actions:

GET  /about/        # about m2ee-api
GET  /status/       # app status
GET  /config/       # show configuration
POST /config/       # set configuration
POST /upload/       # upload a new MDA
POST /unpack/       # unpack the uploaded MDA
POST /start/        # start the app
POST /stop/         # stop the running app
POST /terminate/    # terminate the running app
POST /kill/         # kill the running app
POST /emptydb/      # empty the database


Now that we’ve created the container and published it to the Docker Index we can start using it. And not only we can start using it. Everyone can!

Pull the container and start it.

root@host:~# docker pull mendix/mendix
Pulling repository mendix/mendix
c39ee75463d6: Download complete
eaea3e9499e8: Download complete
855acec628ec: Download complete
root@host:~# docker run -d mendix/mendix
CONTAINER ID        IMAGE                      COMMAND                CREATED             STATUS              PORTS                NAMES
bd7964940dfc        mendix/mendix:latest       /bin/su mendix -c /u   19 seconds ago      Up 18 seconds       5000/tcp, 7000/tcp   tender_hawkings
root@host:~# docker inspect bd7964940dfc | grep IPAddress | awk '{ print $2 }' | tr -d ',"'

In this container the RESTful API started and is now listening on port 5000. We can for example ask for its status or show its configuration.

root@host:~# curl -XGET
The application process is not running.
root@host:~# curl -XGET
"DatabaseHost": "",
"DTAPMode": "P",
"MicroflowConstants": {},
"BasePath": "/home/mendix",
"DatabaseUserName": "mendix",
"DatabasePassword": "mendix",
"DatabaseName": "mendix",
"DatabaseType": "PostgreSQL"

To run an App in this container, we first need a database server. Pull a PostgreSQL container from the Docker Index and start it.

root@host:~# docker pull zaiste/postgresql
Pulling repository zaiste/postgresql
0e66fd3d6a6f: Download complete
27cf78414709: Download complete
046559147c70: Download complete
root@host:~# docker run -d zaiste/postgresql
root@host:~# docker ps
CONTAINER ID        IMAGE                      COMMAND                CREATED             STATUS              PORTS                NAMES
9ba56a7c4bb1        zaiste/postgresql:latest   /bin/su postgres -c    22 seconds ago      Up 21 seconds       5432/tcp             jolly_darwin
bd7964940dfc        mendix/mendix:latest       /bin/su mendix -c /u   30 seconds ago      Up 29 seconds       5000/tcp, 7000/tcp   tender_hawkings
root@host:~# docker inspect 9ba56a7c4bb1 | grep IPAddress | awk '{ print $2 }' | tr -d ',"'

Now configure Mendix to use this database server.

root@host:~# curl -XPOST -d "DatabaseHost=" -d "DatabaseUserName=docker" -d "DatabasePassword=docker" -d "DatabaseName=docker"
Config set.
root@host:~# curl -XGET
"DatabaseHost": "",
"DTAPMode": "P",
"MicroflowConstants": {},
"BasePath": "/home/mendix",
"DatabaseUserName": "docker",
"DatabasePassword": "docker",
"DatabaseName": "docker",
"DatabaseType": "PostgreSQL"

Upload, unpack and start an MDA:

root@host:~# curl -XPOST -F model=@project.mda
File uploaded.
root@host:~# curl -XPOST
Runtime downloaded and Model unpacked.
root@host:~# # set config after unpack (unpack will overwrite your config)
root@host:~# curl -XPOST -d "DatabaseHost=" -d "DatabaseUserName=docker" -d "DatabasePassword=docker" -d "DatabaseName=docker"
Config set.
root@host:~# curl -XPOST
App started. (Database updated)

Check if the application is running:

root@host:~# curl -XGET
-- a lot of html --
root@host:~# curl -XGET
-- a lot of html --

Great success! We’ve deployed our Mendix App in a completely new environment in seconds.


Docker is a very powerful tool to deploy lightweight, secure and isolated environments. The addition of a RESTful API makes it very easy to deploy and start Apps.

One of the limitations after finishing this is that the App isn’t reachable from the outside world. The port redirection feature from Docker can be used for that. To run more Mendix containers on one host there must be some kind of orchestrator on the Docker host that administrates the containers and keeps track of what is running where.

The RESTful API provides a limited set of features in comparison with m2ee-tools. When you start your App using m2ee-tools and your database already contains data, the CLI will ask you kindly what to do. Currently the m2ee-api will just try to upgrade the database scheme if needed and start the App without a notice.

Collectd Graph Panel v0.4

After 2,5 years and about 100 commits I’ve tagged version 0.4 of Collectd Graph Panel.

This version includes a new interface with a sidebar for plugin selection.

Javascript library jsrrdgraph has been integrated. Graphs will be rendered in the browser using javascript and HTML5 canvas by setting the “graph_type” configuration option to “canvas”. This saves a lot of CPU power on the server. Jsrrdgraph has some nice features. When rendered, you can move through time by dragging the graph from left to right and zoom in and out by scrolling on the graph.


The Collectd compatibility setting has been changed to Collectd 5. If you’re still using Collectd 4, please set the “version” configuration setting to “4”, otherwise the graphs of a couple plugins won’t be showed right (like the interface, df, users plugins).

In this version of CGP, total values are added to the legend of I/O graphs and generated colors will be created using a rainbow palette instead of 9 predefined colors. Please read the changelog or git log for more information about the changes.

New plugins:

Special thanks for this version go to Manuel Luis, who developed jsrrdgraph,¬†xian310 for the new interface, Manuel CISS√Č, Rohit Bhute, Matthias Viehweger, Erik Grinaker, Peter Chiochetti, Karol Nowacki, Aur√©lien Rougemont, Benjamin Dupuis, yur, Philipp Hellmich, Jonathan Huot, Neptune Ning and Nikoli for their contributions.

I’ve been using GitHub for a while now. You can download or checkout this version of CGP from the GitHub URL’s below. When you have improvements or fixes for CGP, don’t hesitate to send in a Pull Request on GitHub!

v0.4.1 update

I just removed the dependency on mod_rewrite when using jsrrdgraph to draw the graphs. This may solve javascript error:¬†Invalid RRD: “Wrong magic id.” undefined.

Git: git clone

Bonnie++ to Google Chart

Bonnie++ is the perfect tool to benchmark disk I/O performance. When you want to run heavy disk I/O consuming applications on a server, you need to know what the disk I/O limits are to have an indication what you can run on it. Bonnie gives you information about read-, write-, rewrite Block I/O performance and the performance of file creation and deletion.

Below is a sample piece of Bonnie++ output. It is hard to read, and when you’ve done a couple of runs on different hosts, it’s hard to compare these results.

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
host3           16G  1028   0 94051   0 40554   0  1668   0 121874   0 380.3   0
Latency             16101us    7694ms    7790ms   13804us     124ms   76495us
Version  1.96       ------Sequential Create------ --------Random Create--------
host3               -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                256 50115   0 332943   0  2338   0 51591   0 +++++ +++  1189   0
Latency               312ms    1088us   10380ms     420ms      15us   15792ms

Inspired by bonnie-to-chart I’ve developed a tool called bonnie2gchart. With the CSV data Bonnie provides after a run, you can visualize that information in a graph with bonnie2gchart. bonnie2gchart uses the Google Chart API to create the graphs. It’s compatible with Bonnie++ 1.03 and 1.96. And it is also able to show the results of both versions in one graph.

Example chart

Git: git://

OTRS the RT way

I have used RT (Request Tracker) for quite a while. Starting with RT 2.0.15 + MySQL. Later upgraded that version manually (with a couple of nasty PHP scripts) to RT 3.6.x + PostgreSQL. It worked and stayed consistent. I’ve never been very happy about RT. It now contains about 80K tickets and it is slow. I have been reading a lot of forum/mailinglist threads about migrating from OTRS to RT and RT to OTRS because RT or OTRS should be faster. Unfortunately I’ve not found a real conclusion.

While starting a new project I gave OTRS a try. Just because I think it doesn’t really matter if you use RT or OTRS. I liked the way some things worked in RT, so I created some patches to make OTRS work a little bit like RT.

Empty body check

First of all the body part in OTRS is required for some reason. This is annoying. For example when a ticket is resolved and a customer replies to thank you, you just want to close the ticket. Or when you just want to change the owner of a ticket. This patch disables the emtpy body check:


RT-like Dashboard

OTRS: RT-like DashboardIn RT you are responsible for your own tickets. The RT 3.6 dashboard shows you the most recent open tickets you own. And it shows the most recent unowned new tickets. OTRS shows all open tickets, instead of only the ones you own. OTRS also shows all new tickets including the ones that are already locked.

This patch adds a new module called DashboardTicketRT. It’s a customized DashboardTicketGeneric module to make your dashboard work like RT. After you have applied the patch, you can use this module with the following steps:

  1. Login as admin
  2. Go to Sysconfig
  3. Go to Frontend::Agent::Dashboard (Group: Ticket)
  4. In TicketNew change:
    • Attributes: StateType=new; to StateType=new;OwnerIDs=1;
    • Module: Kernel::Output::HTML::DashboardTicketGeneric to Kernel::Output::HTML::DashboardTicketRT
  5. In TicketOpen change:
    • Attributes: StateType=open; to StateType=open;StateType=new;OwnerIDs=UserID;
    • Module: Kernel::Output::HTML::DashboardTicketGeneric to Kernel::Output::HTML::DashboardTicketRT
    • Title: Open Tickets / Need to be answered to My Tickets / Need to be answered
  6. Restart your server


Instant Close / Spam

OTRS: InstaClose linkTo work a little bit more efficient this patch adds an InstaClose and InstaSpam link to each ticket. With a single click you can close a ticket or move a ticket to the Junk queue. The patch adds a new module called AgentTicketInsta. You must add this code to Kernel/ to enable it:

$Self->{'Frontend::Module'}->{'AgentTicketInsta'} = {
 Group       => [ 'admin', 'users' ],
 Description => 'Instant Ticket Actions',
 Title       => 'Insta',


Collectd Graph Panel v0.3

It has been a while and many people were already using one of the development versions of CGP. Time to release a new version of CGP: v0.3.

Special thanks for this version go to Manuel CISS√Č, Jakob Haufe, Edmondo Tommasina, Michael Stapelberg, Julien Rottenberg and Tom Gallacher for sending me patches to improve CGP. Also thanks to everyone that replied after the previous release.

In this version there are a couple of minor improvements and some new settings to configure. But the most important: Support has been added for 13 Collectd plugins. CGP now supports 26 Collectd plugins. Also a couple of plugins have been updated. Please read the changelog or git log for more information about the changes.

New plugins:

Download the .tgz package, the patch file to upgrade from v0.2 or checkout the latest version from the git repository.


Collectd Graph Panel v0.2

Version 0.2 of Collectd Graph Panel (CGP) has been released. This version has some interesting new features and changes since version 0.1.

A new interface is introduced, based on Daniel Von Fange’s styling. A little bit of web 2.0 ajax is used to expand and collapse plugin information on the host page. The width and heigth of a graph is configurable, also the bigger detailed one. UnixSock flush support is added to be able to let collectd write cached data to the rrd file, before an up-to-date graph of the data is generated. CPU support for Linux 2.4 kernel and Swap I/O support is added. And some changes under the hood.

Download the .tgz package, the patch file to upgrade from v0.1 or checkout the latest version from my public git repository.

CGP Overview Page CGP Server Page CGP Detailed Page


Collectd Graph Panel v0.1

Collectd Graph Panel (CGP) is a graphical web front-end for Collectd written in php. I’ve developed CGP because I wasn’t realy happy with the existing web front-ends. What I like to see in a front-end for Collectd is clear overview of all the hosts you are managing and an easy way to get detailed information about a plugin.

This first release is a very basic version of what I have in mind. Not all plugins are supported yet and the user interface can be done better. Still a lot to work on!

Please give it a try! It should work out-of-the-box. Download the tgz package or checkout the latest version from my public git repository.