Sunday, December 7, 2014

Picking open source technology: pick the community too

A brief post on picking (open source) technology. When we need something for our project, say, a database, we have a large range of choices. There is old and staid software that may not be hip, but comes with a well known track record and lots of features. There are new shiny frameworks that are blazing trails and publishing impressive benchmarks, written in languages still being standardized.

What do we pick? First of course, our dependency has to meet (most of) our needs. Next, we can look at its current reputation - if a project is known to be a bloated mess, or is well known to crash if you wave at it, we can discount the project for now.

But, even then, we are left with a lot of choice. What I care about these days is the community around a project. Because it turns out that the community is the best predictor of the future of a project. We can’t actually predict the future, but we can be sure that we’ll have new needs for our dependencies. We can also be sure we’ll have questions and discover bugs.

And a healthy community almost guarantees that things will end up well. As a recent example, PowerDNS has recently become involved with a customer where we are helping setup a PowerDNS based anycast environment. For this we needed a BGP implementation. A quick consultation with our community reported that ExaBGP would be a very good choice, and indeed, it offered all the features we needed.

On deployment however, we found two small issues that were holding up our deployment. PowerDNS employee of the month Peter van Dijk worked with the ExaBGP people, fixed the two issues, and both fixes have now been merged by the ExaBGP project. They are happy, we are happy. The next release of ExaBGP won’t just meet our needs, it will suit the needs of many more people.

Last week, a developer reported that the most excellent Valgrind tool found some potential errors in the LMDB project, and I was shocked to see LMDB lead developer tell the reported to ‘learn how to use his tools’, and not address his report in any meaningful way.

I asked Howard Chu to reconsider his stance, given my belief that open source can’t be great without a functioning community, something you don’t build this way. Howard told me that how he treated the reporter was entirely intentional. Shortly afterwards, the following was posted on the LMDB list:

“if you post to this list and your message is deemed off-topic or insufficiently researched, you *will* be chided, mocked, and denigrated. There *is* such a thing as a stupid question. If you don't read what's in front of you, if you ignore the list charter, or the text of the welcome message that is sent to every new subscriber, you will be publicly mocked and made unwelcome.”

This is entirely intentional and by design.”

Now of course, everyone is free to run their project in their own way. But I’m also free to pick my dependencies, and I care about the development community being sane and inclusive. “Bitchslapping” reporters of potential valid issues isn’t that. Formalizing this behaviour most definitely isn’t. I won’t be picking LMDB for any future projects unless this changes.

As a parting gift to LMDB, I worked to document the (powerful) semantics of LMDB, and dragged details out of Howard. This document can be found on If you use LMDB, it may be helpful for you.

I spent this evening working with PowerDNS contributor Ruben Kerkhof to merge several of his fixes for PowerDNS issues. Ruben’s company Tilaa uses PowerDNS, and we are Tilaa customers. I’m truly proud of our community and what we achieve together, and I recommend that every open source project works to foster its community as well.

So in short.. before picking a technology to depend on, investigate how they deal with feature requests, bug reports and questions. What you will learn is the best predictor of how the project will serve you (and vice versa!) over the coming years.