Promcon 2016

Last week, I was fortunate enough to be invited to speak alongside an old friend, Chris Urwin, at Promcon in sunny Berlin. Promcon 2016 was the inaugural Conference for all things Prometheus. For those of you unfamiliar with Prometheus, it’s in essence a queryable metrics store built upon a time series database.

Sounds interesting, but why would you need such a thing? Prometheus is a powerful tool, able to store multi-dimensional data and search it efficiently through PromQL, its custom built query language. From where I’m standing, Prometheus is one of a number of new generation monitoring tools that have recently come of age.

These are the tools that will help us finally deliver the quality of monitoring we’ve not often enjoyed.

Confrence Action Shot

The conference

Spanning two days, we heard from a fantastic selection of speakers, all eager to share their experiences and to learn from the wider community. We had great examples at scale from the likes of SoundCloud and DigitalOcean, demonstrating how Prometheus can be used at absolutely massive scale. We also had some equally important talks from companies like Shuttlecloud who’ve been able to take Prometheus and drop it straight into their platforms with great success.

Of the talks, I have to say my personal favourite was Björn Rabenstein’s talk about how Prometheus stores and handles data. It’s something he’s clearly passionate about and it’s fascinating when engineering is taken to such high levels of optimisation. I’d highly recommend watching his talk and many of the others available to view online at the links below.

The conference offered fascinating insight into how successful open source projects can be run and importantly, how you can engage the wider community in the ongoing development of the product. For me personally, the focused nature of the conference allowed for a greater level of detail to be explored, making it very useful, even for the more experienced attendees.

Our experience

So how did we come about Prometheus? We first discovered Prometheus when on the hunt for something simple yet powerful that could be leveraged to monitor a POC build we undertook for one of our clients. The POC itself, an on-premise internal CaaS built upon Docker, utilising Rancher for its container management and orchestration. We wanted something that could match our shiny new dynamic infrastructure without requiring the manual intervention or complex configuration usually associated with this genre of tooling.

After some positive initial prototyping, we started to expand our usage and understanding of Prometheus. As happens when you find an exciting new technology, one thing led to another and before we knew it we were participating in the community through our small contribution of a catalog entry available in Rancher. Rancher’s catalog function aims to lower the bar in adopting new technology, much like Docker’s application bundles which were announced at this year’s Dockercon.

In this catalog item was a subset of open source tools that work out of the box with Prometheus, the idea being you can one click deploy and dynamically monitor your entire platform, or at least start people on their way. To our surprise, this proved popular, with now over 10,000 downloads to date. We also saw high demand of a plugin we wrote to make the Rancher API scriptable by Prometheus, currently somewhere over 50,000 downloads on DockerHub thus far. It’s awesome to see how active this community has become, and to see real demand for tools like Prometheus.

Example PromQL An example of how PromQL and labels allow for much smarter queries

Example Visualisation An example of data collected by Prometheus, visualised in Grafana

Further Info

Brian Brazil, one of the founders of Prometheus, heads up consultancy called Robust Perception. Through this Brian offers training and consultancy for Prometheus users; he also runs a highly useful blog that goes into detail around the different features Prometheus has to offer.

Brian’s been kind enough to agree to talk at our offices in Leeds. If you’re in Leeds on the 13th of October and want to drop by for his talk, get in touch at and I’ll make sure we reserve a seat for you!

Watch the talks online

Offical blog is now up, and the talks are available to watch on youtube.

Written on August 30, 2016 by:
Ed Marshall