Sunday, December 7, 2014

Picking open source technology: pick the community too

A brief post on picking (open source) technology. When we need something for our project, say, a database, we have a large range of choices. There is old and staid software that may not be hip, but comes with a well known track record and lots of features. There are new shiny frameworks that are blazing trails and publishing impressive benchmarks, written in languages still being standardized.

What do we pick? First of course, our dependency has to meet (most of) our needs. Next, we can look at its current reputation - if a project is known to be a bloated mess, or is well known to crash if you wave at it, we can discount the project for now.

But, even then, we are left with a lot of choice. What I care about these days is the community around a project. Because it turns out that the community is the best predictor of the future of a project. We can’t actually predict the future, but we can be sure that we’ll have new needs for our dependencies. We can also be sure we’ll have questions and discover bugs.

And a healthy community almost guarantees that things will end up well. As a recent example, PowerDNS has recently become involved with a customer where we are helping setup a PowerDNS based anycast environment. For this we needed a BGP implementation. A quick consultation with our community reported that ExaBGP would be a very good choice, and indeed, it offered all the features we needed.

On deployment however, we found two small issues that were holding up our deployment. PowerDNS employee of the month Peter van Dijk worked with the ExaBGP people, fixed the two issues, and both fixes have now been merged by the ExaBGP project. They are happy, we are happy. The next release of ExaBGP won’t just meet our needs, it will suit the needs of many more people.

Last week, a developer reported that the most excellent Valgrind tool found some potential errors in the LMDB project, and I was shocked to see LMDB lead developer tell the reported to ‘learn how to use his tools’, and not address his report in any meaningful way.

I asked Howard Chu to reconsider his stance, given my belief that open source can’t be great without a functioning community, something you don’t build this way. Howard told me that how he treated the reporter was entirely intentional. Shortly afterwards, the following was posted on the LMDB list:

“if you post to this list and your message is deemed off-topic or insufficiently researched, you *will* be chided, mocked, and denigrated. There *is* such a thing as a stupid question. If you don't read what's in front of you, if you ignore the list charter, or the text of the welcome message that is sent to every new subscriber, you will be publicly mocked and made unwelcome.”

This is entirely intentional and by design.”

Now of course, everyone is free to run their project in their own way. But I’m also free to pick my dependencies, and I care about the development community being sane and inclusive. “Bitchslapping” reporters of potential valid issues isn’t that. Formalizing this behaviour most definitely isn’t. I won’t be picking LMDB for any future projects unless this changes.

As a parting gift to LMDB, I worked to document the (powerful) semantics of LMDB, and dragged details out of Howard. This document can be found on https://github.com/ahupowerdns/ahutils/blob/master/lmdb-semantics.md. If you use LMDB, it may be helpful for you.

I spent this evening working with PowerDNS contributor Ruben Kerkhof to merge several of his fixes for PowerDNS issues. Ruben’s company Tilaa uses PowerDNS, and we are Tilaa customers. I’m truly proud of our community and what we achieve together, and I recommend that every open source project works to foster its community as well.

So in short.. before picking a technology to depend on, investigate how they deal with feature requests, bug reports and questions. What you will learn is the best predictor of how the project will serve you (and vice versa!) over the coming years.

Monday, November 3, 2014

Tin cans, can openers and solar power: explaining the snail’s pace of innovation

Innovation is fascinating. It brings us (by definition) all improvements in technology. Simultaneously, it proceeds at a snail’s pace. An example I love to share is the invention of the tin can, which was a breakthrough in food preservation (there were no refrigerators at the time, and tin cans allowed people to safely transport and store previously perishable goods).

Made practicable in the early 19th century, tin cans were sold bearing notices like “Cut round the top near the outer edge with a chisel and hammer”. And then.. it took over 50 years for the first home operable tin opener to arrive in 1870 (in the 1850s, an opener was available for use in stores, who would open your cans for you). The modern easy to use and safe opener was only invented in 1925.


Until the arrival of the opener we know today, opening tin cans was a dangerous exercise. In fact, it was a common cause of injuries and infections, highly dangerous before the advent of antibiotics. Yet it took decades before the problem was solved, even when the need for a solution stared people in the face!


This example is historical, and hard to come to grips with - why did no one invent the current can opener earlier? The mind boggles. Here’s a far more recent example and one that is still developing. It may help shed light on why innovation proceeds so slowly.


(It looks like this post is about how much I like solar, and while this might be the case, the point I’m trying to make is not that solar power is great. The point is to elaborate on why innovation operates so slowly, using a currently happening development. In fact, if you find yourself disagreeing with my description of the bright future of renewable energy, ponder where your disagreement is coming from. That very disagreement likely is what this post is about!)

Renewable energy

Renewable energy sources have long been treated with derision. “What about when there’s no wind and no sunlight?”. Industry professionals and engineers alike shared this disdain, because as seen through their eyes, solar power compares very badly with (for example) a coal powered plant. The coal powered plant delivers energy against low prices (perhaps 3-7 eurocents/kWh), and does so continuously within a short few hours of turning it on, which you can do at any time you desire.


If you hold this next to solar power, photovoltaics do come off very poorly. They roughly operate in three modes - no output (night), 10% output (overcast) or 80-100% output (direct sunlight). One can try to predict when they’ll deliver this output, but you’ll never get it quite right. And, to add insult to injury, the production cost per kWh currently is a lot higher than coal powered plants! So not only don’t you know if and when you’ll get energy, you have to pay more for it too. A mostly similar story applies to wind power.


The upshot of the intermittent nature of availability is that by the time you’ve built enough solar and wind farms to cover your average needs, you’ll have generated vast overcapacity for when the sun shines and it is windy. Your capital costs meanwhile have also been huge compared to boring old power plants.


In short, from the viewpoint of the seasoned energy professional, solar and wind power makes for very unreliable expensive capacity that generates imbalance between power needs and power production. The professional then points out that energy storage (which could average out the imbalance) is prohibitively expensive, and won’t save the day. Repeat this story for decades and decades, and you are in 2008 or so.


Personally, I was fully on board with this line of thinking. I advocated the development of large fusion or even fission plants as the (remote) future.

Then, politics happened

Then, through various plans that weren’t too well thought out (I think), all of a sudden whole countries started building heavily subsidized wind farms and suddenly cheap roof-top solar panels at a massive clip. Germany comes to mind. The Netherlands and other countries are now following behind, even with dwindled subsidies.
How come? Well.. the original energy professionals were thinking entirely on the production side (and who can blame them, that’s where they worked!). The reality is that in many countries, no consumer actually pays anywhere near 3-7 (euro)cents/kWh for electrical power. Transport costs and heavy taxes increase actual ‘grid costs’ to over 20 eurocents/kWh in most parts of Europe.


And since producing energy you use yourself is untaxed, domestic energy users compare the production costs of solar against the delivery and taxed costs of grid power. And guess what, since there is a factor 3 to 6 between these numbers, all of a sudden ‘grid parity’ arrived. Unsubsidised solar energy is now cheaper than grid power in many places.


Google trends for “grid parity


Note how the concept of grid parity, or at least the term, burst on the scene in January 2008 (the Wikipedia page was created in June of that year). Before that date, the old view of solar power as a sort of retarded coal fired plant prevailed. Production parity was what mattered. Grid parity, which is what matters for consumers, was not a factor. (By the way, I checked - the flatline of the graph pre-2008 is real, and does not reflect a lack of data).


Grid parity is when solar is interesting for consumers - production parity is only relevant to power producers.

The new reality

What the staid electricity people thought unacceptable (and perhaps unimaginable) is now the new reality. The centralized generation of power has become an uneconomic and highly ungrateful enterprise to be in. If the sun shines and the wind blows, there is literally no need for the coal fired plant. Effective energy prices now routinely plummet to 0 cents, and have gone negative on occasion.

aemo-solar-price.jpg
Note zero pricing around 03:00 and negative prices around 14:20 http://reneweconomy.com.au/wp-content/uploads/2014/07/aemo-solar-price.jpg


And while this is clearly unsustainable (nobody can generate power this way and stay in business), this reality isn’t going to go away, although some localities have so far succeeded in outlawing (Cyprus) or heavily taxing (Hawaii) grid connected solar panels.


The practical upshot though is that even today, substantially lower amounts of coal and gas are being shoveled into power plants.


Meanwhile, wily entrepreneurs have cottoned on the new reality of highly variable energy prices, and are scheduling their business needs around them. Processes that lose money at 20 cents/kWh make a lot more sense at 3 cents/kWh! This is the future, and attempting to emulate the flat power prices of the past century is not.


In another shocking development, a decade ago, California suffered from brownouts and even rolling blackouts, mostly on hot days when lots of air conditioners were running. Now it turns out that a lot of this can be blamed on energy fraud by Enron, but the problem of peak energy use was real.




Welcome to the new reality - already in 2012 the peak was mostly gone, and in 2014 it will have become a dip. The new challenge is the ramp-up when the sun goes down. All of this because in 2012 photovoltaics met the concept of ‘grid parity’, a concept very rarely discussed before 2008.


Only now that minds have been liberated, new thinking has arrived. One could for example ponder potential mass adoption of electrical cars, cars that will need to be charged before their owners drive off with them, but conceivably could charge ‘on demand’ during the day. They could even deliver power during unexpected peak net-demand periods (!). Nearly free energy and “the internet of things” could make this a reality.


Lots of other things could happen too, for example electrical heating of houses currently heated by gas, electrical (on demand) heating of water reservoirs for residential use (‘peak shaving’), bulk energy storage centrally, regionally or even in homes, etc. It is a whole new world out there. How things will end up is hard to predict, but we can already be sure that everything will be cheaper and a lot friendlier to the environment. And this leaves undiscussed the geopolitical impact of a world far less reliant on oil and gas!


All this thinking was previously rejected out of hand, “because it did not act like a power plant we know”.

What this means for innovation

In hindsight, the commonly held belief that solar (or renewable energy in general) will never fly until it performs exactly like a traditional power plant was and is ludicrous. But you still hear it today. A lot. And when you hear it, ponder the decades it took to invent a reliable can opener. Innovation happens at a snail’s pace, and only appears obvious in hindsight.


This inability to see the beyond the present concepts appears to be a feature (bug?) of human nature.


If you are personally trying to innovate, read up on the history of previous inventions, and see how they were rejected by experts and professionals. Read on until you get a feel for proper rejection (“the flying car” with present day technology) and failure-of-the-imagination rejection (British Post Office comment on phones “no need, we have lots of messenger boys”).


To innovate, we must free yourselves of dearly held convictions and concepts. Ironically, the people with the most knowledge and thus the best credentials to invent the future tend to cling fastest to previously held concepts. This is the challenge of innovation - using the knowledge out there, but without having it hold us back.


Good luck!
(Thanks to Nicholas Miell, Tsjoi Tsim and Jan van Kranendonk for editing & constructive criticism - all mistakes remain mine!)

Tuesday, August 19, 2014

The absolute minimum difficulty recipe with maximum impact: authentic chicken soup

I like cooking, and I like to share that joy. If  you can’t cook however, your first efforts are likely to be mediocre, which is not very encouraging. “Why bother?!”.


In this post, I’ll share something that can barely be called a recipe, that’s how simple it is. BUT! The results are entirely authentic and spectacular. You can take this with you to a home cooked dinner, and your friends will be very happy with your efforts. They will ask you for the recipe! Little will they know how easy it is. Also, it is almost impossible to mess this up. I dare you to try ;-)


Ingredients: 10-20 raw chicken wings (plain or spiced, NOT breaded!), 2-4 onions, a few large carrots (or more smaller ones), salt, pepper.
Required equipment: oven-proof container, oven, pan, stove, strainer or colander.
Time spent in the kitchen: 15 minutes


Pre-heat oven to 180C (350F). Meanwhile, put the chicken wings in the oven-proof container. If you got plain wings, add some salt and pepper.  If you have it, sprinkle some oil over the chicken wings.


Put container with chicken in oven (if at temperature). Then you can wait 25 minutes, you can wait 35 minutes, you can wait 45 minutes, and in every case you’ll have some pretty good chicken wings. If after 25 minutes they look ready to eat, they ARE. Do not eat unless thoroughly hot inside!


(if you don’t have an oven, you can also fry the chicken wings in a pan on a stove, works just as well, but you need to pay some closer attention to getting them browned all round and well done inside. Will require more oil).


Now for the surprising part. Eat as many of the chicken wings as you feel like. No need to finish them all. This was not your special dinner ;-)


Next, fill a pan with a few liters (quarts) of water, then throw in all your chicken wings, the ones you ate, the ones you didn’t. Peel onions, cut in large chunks, add to water. Chuck the carrots in there too. Bring water to a boil, turn down heat and leave to simmer for 3, 4, 5, hell 6 hours if you feel like it. If during the boiling you note brown stuff floating on the water, remove that with a spoon. Your house will smell lovely meanwhile.


Afterwards, use colander or strainer to filter out all the bones and pieces of meat. (You can add back the pieces of meat if you want.) Now taste the soup and add salt to taste. This part is not optional, it really needs salt.


That’s it! You can take the soup with you to a party and heat it there. In a fridge it will last for days.


Note: to improve on this soup, add more vegetables (doesn’t really matter which ones), or instead of chicken wings, use an entire chicken.

Tuesday, July 8, 2014

Are the media right and are we all going to die from antibiotic resistance?

A brief post - I've long been unhappy with the media's reporting on science. It appears that news sites aren't there to disseminate truth and educate the populace, but to sell advertisement and clicks. Took me long enough to find that out.

So we get lots of news on how we are all going to die because all antibiotics have failed. We even see repeated claims that nobody is working on new antibiotics, that the pipeline for new discoveries is empty, and nothing is on the horizon. I saw so much of this that I actually started to believe it too.

Remember MRSA and how it would kill us all? Oddly enough it didn't, and while it is a nasty bug, there are now protocols to get a handle on it. You probably missed the article telling you so because "we're all going to die" gets a lot more clicks than "with careful work, MRSA outbreaks can be controlled".

"We're all going to die" is a potent way to draw in an audience. Simultaneously, scientists are happy to big it up, since it is a real problem, and it deserves more attention than it gets. Don't get me wrong on that! The world of medicine spends way way more on cholesterol lowering (to dubious effect) than on antibiotics (which actually save lives). And a dead patient needs no cholesterol lowering, so they should get on it pronto.

So, where are we? Bacteria can be divided (roughly) into "gram positive" and "gram negative". On the gram positive front, a whole new class of antibiotics has now been trialed for a few years, and they are called lipoglycopeptides. The two most recently tested (dalbavancin and oritavancin) show excellent effectiveness against MRSA. Interestingly, these new antibiotics require a single dose which does its work for the next weeks, so you can't even forget to take the pills. The great Richard Lehmann discusses these over at the BMJ (and here, even more here).

Gram negative bacteria are different, and have an outer membrane that protects them against many antibiotics in the first place, and the situation there is less hopeful, and currently far more worrying than MRSA. The main worry now are carbapenem-resistant Enterobacteriaceae (CRE), equipped with the scarily named New Delhi metallo-beta-lactamase (NDM-1) enzyme. These are the superbugs that hit the news a lot.

What did not hit the news was a result published last week where a whole swath of NDM-1 carrying bacteria was effectively treated (in mice) with regular antibiotics plus a known compound previously considered as a hypertension drug (which sadly failed to lower blood pressure). An infection that with best current treatment killed 100% of test animals now only killed 5%. Of mainstream media, only the Wall Street Journal reported on it. "It it bleeds, it leads" - and modest but important progress does not, it appears.

Now, are we all saved? No, not yet. In the end it is a battle of bacterial genes against our whits, but our arsenal of methods these days is astounding. There is every reason to be sure we'll keep on winning this battle. The next stage will be winning it cheaply and durably.

But my main point is - whenever you see the news reporting "there are no new antibiotics" and that nobody is working on it, think back to this page. Not only are those claims false, a whole new class of antibiotics are now coming out of the lab (53 at last count) and we're also finding ways of revitalizing our existing arsenal.

Meanwhile "we're all going to die" gets a lot more clicks than "we're working on it and making tangible progress"...

Tuesday, June 10, 2014

Guns & Bandsaws: how to feel the scale of a technology?

When we pick technologies (languages, compilers, CPUs, computers, routing platforms, databases etc), we need ones that are able to deal with the size or scale of our problem. Now, what is this scale? And does it matter?

If you picked it wrong, you will find out. If you pick an operating system, open a million files, and find that the kernel bogs down because it has lots of linear searches over open file descriptors, you know the designers of the operating system weren't thinking of your use case. And perhaps in this specific example, you could work around the issue (you don't actually _need_ a million open files at once), but you'll be guaranteed to run into other linear walks. They just weren't thinking at your scale.

So what determines the 'natural scale' for which a technology is suited? Mostly, this turns out to be in all the individual parts that break when scaling to larger volumes, larger data rates, larger change rates etc. And to scale up a technology, you need to address all these individual squeaking parts, of which there might be many.

If you settled on something that scales to 'X', and then your use grows to '10X', you might find yourself being held back by a myriad small things that slow you down. And this will not change quickly - you picked your infrastructure as, well, infrastructure. It is there. It is a given. It will change, but only on multi-year timescales. So it matters, since now you'll be toast for years to come. Pick correctly!

Now, to drive home this point, and this is the real excuse for this post, here's a fair-use busting quote from the most wonderful Cryptonomicon by Neal Stephenson. You should read it. Many times. The first 100 pages are a bit slow, but THEN it delivers. Here is one of the book's heroes, Bobby Shaftoe, brilliantly explaining the concept of a technology's natural scale:

"Now when Bobby Shaftoe had gone through high school, he’d been slotted into a vocational track and ended up taking a lot of shop classes. A certain amount of his time was therefore, naturally, devoted to sawing large pieces of wood or metal into smaller pieces. Numerous saws were available in the shop for that purpose, some better than others. A sawing job that would be just ridiculously hard and lengthy using a hand saw would be accomplished with a power saw. 
Likewise, certain cuts and materials would cause the smaller power saws to overheat or seize up altogether and therefore called for larger power saws. But even with the biggest power saw in the shop, Bobby Shaftoe always got the sense that he was imposing some kind of stress on the machine. It would slow down when the blade contacted the material, it would vibrate, it would heat up, and if you pushed the material through too fast it would threaten to jam." 

I'd like to focus a bit on this feeling - "imposing stress on the machine". In the computing world, we recognize this. We have an intuitive feeling that our MySQL database will become severely unhappy with a billion rows, but will zoom with 50 million rows, for example. That Bobby Shaftoe talks about this same feeling indicates it might have broader, more universal, technological roots.

"But then one summer he worked in a mill where they had a bandsaw. The bandsaw, its supply of blades, its spare parts, maintenance supplies, special tools and manuals occupied a whole room. It was the only tool he had ever seen with infrastructure. It was the size of a car. The two wheels that drove the blade were giant eight-spoked things that looked to have been salvaged from steam locomotives. Its blades had to be manufactured from long rolls of blade-stuff by unreeling about half a mile of toothed ribbon, cutting it off, and carefully welding the cut ends together into a loop.  
When you hit the power switch, nothing would happen for a little while except that a subsonic vibration would slowly rise up out of the earth, as if a freight train were approaching from far away, and finally the blade would begin to move, building speed slowly but inexorably until the teeth disappeared and it became a bolt of pure hellish energy stretched taut between the table and the machinery above it. Anecdotes about accidents involving the bandsaw were told in hushed voices and not usually commingled with other industrial-accident anecdotes.  
Anyway, the most noteworthy thing about the bandsaw was that you could cut anything with it and not only did it do the job quickly and coolly but it didn’t seem to notice that it was doing anything. It wasn’t even aware that a human being was sliding a great big chunk of stuff through it. It never slowed down. Never heated up."

And again, we can recognize this feeling. Setup a million routes on your big BGP router? It'll just smile at it. It is doing what it was meant to do. It is not impressed. Reload them all you want. Similarly, do something like that on your Raspberry Pi, and you know you are "imposing on the machine".

Bobby now expands a bit on the concept:

"In Shaftoe’s post-high-school experience he had found that guns had much in common with saws. Guns could fire bullets all right, but they kicked back and heated up, got dirty, and jammed eventually. They could fire bullets in other words, but it was a big deal for them, it placed a certain amount of stress on them, and they could not take that stress forever."

The phrase "it was a big deal for them" is one to recognize & compare with your feelings for technology you are considering. Is it a big deal for it to do what you want?

"But the Vickers in the back of this truck was to other guns as the bandsaw was to other saws. The Vickers was water-cooled. It actually had a f*cking radiator on it. It had infrastructure, just like the bandsaw, and a whole crew of technicians to fuss over it. But once the damn thing was up and running, it could fire continuously for days as long as people kept scurrying up to it with more belts of ammunition. After Private Mikulski opened fire with the Vickers, some of the other Detachment 2702 men, eager to pitch in and do their bit, took potshots at those Germans with their rifles, but doing so made them feel so small and pathetic that they soon gave up and just took cover in the ditch and lit up cigarettes and watched the slow progress of the Vickers’ bullet-stream across the roadblock. 
Then he ceased firing at last. Shaftoe felt like he should make an entry in a log book"

Again note the awe the technology inspires, and how it is pitted against the rifles which just operate in a very different class of scale.

As a case in point, over at PowerDNS we've been taking LMDB for a spin. And I can tell you, it is a water cooled Vickers with a radiator. We've thrown everything we've had at it, and it just zooms along.. like it is enjoying the challenge. It chomps on the zones like they aren't even there.

So, wrapping up this brief post - whenever you select new technology for a project, be it a compiler, a language, a computer, a router, a database - try to feel its scale. And the feelings of 'imposing a strain', 'being a big deal', 'not even noticing the work', described above may well guide you to pick the right technology. Good luck!

Monday, May 12, 2014

Will your startup idea be successful? I don’t know. But here’s a checklist before you approach people who could help you.


(this post contains two startup ideas that you might consider doing, if so, please contact me!) 

Judging startup ideas. This is notoriously difficult but simultaneously enormously relevant. It is hard to predict if an idea (or your execution of it) will be successful. However, it may be possible at an early stage to determine that it likely won't be.

This post is intended to be useful to rid your startup idea of easily avoidable errors before you approach (angel) investors, incubators or other people that could help you.

PowerDNS
My personal history in judging startup ideas is shoddy - when PowerDNS was launched late 20th century, I did not realise our idea was doomed by design.

We wanted to sell expensive DNS software to a world used to getting it for free with the operating system. The people we tried to sell it to (system administrators) understood our sales pitch all too well: Database driven DNS would save a lot of manpower. Back in 2000 that meant the people we aimed our marketing at would personally lose their jobs if they took us up on our offer. And we didn’t know anyone higher in the food chain. We didn’t get anywhere.

However, over the years, my batting average probably improved. PowerDNS is doing very well, both as open source technology and as a company. My joint-venture with Fox-IT had sound beginnings (because I learned from PowerDNS), and achieved the three-year goals we set for it in only 5.

Recently, I was asked by the good people of Yes!Delft if I could participate in a startup event, but I decided I had better things to do, since I judged the attending startups to be hopeless. And I was rightfully chided for that - it is incredibly hard to judge a startup idea

New ideas sound bad
Almost by definition, great new ideas sound really bad when you first hear of them. Because if the idea were so obviously good, it would’ve been done already! So prima facie plausibility is not a good way to judge an idea. For example, when launched, Twitter looked like a truly stupid blogging service (2 sentence blog posts?!). When the first digital cameras arrived, the only available storage devices were fragile power hungry hard disks. It made no sense.

But still we need ways to distinguish the right kind of bad idea from the truly bad idea. Are there things that just have to be right, even if we believe in the concept itself?

Here’s a checklist of things to consider:

  1. Is the idea (near-future) implementable?
    • But not so doable that anyone can do it
    • If it is possible, can you explain why it hasn’t been done yet?

You'd be surprised how often startups ideas flunk these simple tests. Wishing something is possible doesn't make it so. The laws of physics (generally) do not change for you. It is important to separate the two kinds of “impossibilities”:
  • breaking the speed of sound (hard work, requires lots of R&D),
  • breaking the speed of light (requires Nobel prizes, whole new laws of physics).
Usually you should prefer 'speed of sound'-hard problems.

  1. Do you have a good mental model of who would buy and/or use your technology
    • How do they buy things?
    • Would they buy it from you?
    • Is it conceivable that they’d pay your prices?
    • How much better do you have to be than what they are currently buying?
    • If they won’t be paying, do you have a plan to make money anyhow?

Note that this item does not demand a 5-year business plan. No such thing makes sense, least of all for a startup. The item does ask if you have a working mental model of your potential customer base. What makes them tick? Does your team have personal experience? Do you know people who could help you there? Or are you operating on a “of course they'll buy our stuff, it is cheaper”-grade model? 

  1. Do you have (people with) special knowledge, capabilities, access that competitors don’t?
    • Technologies
    • Skills
    • Execution
    • Unique team
    • Relations
    • Intellectual property rights

Companies, startups, are in perpetual competition. If you don't have some kind of head start, don't enter the game. Note that you don't have to have all of these, but never play fair. Make sure you bring something special to the game.

  1. Are you serving a market that isn’t too crowded already? 
    • Or one that nobody previously wanted to serve, or thought of serving?

This one is key too. Unless you have vast resources, don't try to make a better iPhone from scratch. It is tempting to target a well known problem, but the problem with well known problems is that the are, well, well known. Again, don't play fair in this respect. Tackle something in a field that is not too crowded. Don't try to out-Google Google! Do however try to do things other people found beneath their scale, or too boring. At least don't do what everybody is doing already, unless you can do it 100 times better.

  1. Do you have access to your market, or can someone easily block you from entering it?

When pondering revolution, don't expect help from the establishment. They've been at it for decades. Intel kept AMD processors away from everyone by promising sweet prices only in case of total vendor loyalty. Microsoft set up its licensing so they got paid per computer shipped, not per copy of Windows – thus removing any incentive for shipping anything other than Windows. Don't expect to compete with Amazon if you need AWS to do it.

Does it all have to be perfect?
These things can compensate each other, and it is a rare startup idea that will score well on all items of the checklist.

For example, great execution can make up for an idea that is not too novel in a sparingly addressed market. Conversely, a truly great technology can take off even in the face of adverse markets and lack of relevant business experience (the launch of Google comes to mind).

A cautionary note
No battle plan survives contact with the enemy. And it is a rare startup that delivers what they originally thought they’d be doing. WhatsApp did not start as a texting service. American Express was a courier service which later added credit facilities, and only then dropped out of the delivery business. Google did not start with advertising. “The Facebook” launched as a college hangout.

But if you don't have an initial narrative on how you could succeed, you'll have a hard time gathering funding, talent and advice!

Some worked examples.
A cell phone battery that never dies. If we score this idea with the list above:
  1. yes, it might be possible (pacemakers used to have nuclear batteries, and many decades old spacecraft are still powered by them. Mars Curiosity is a recent example)
  2. Everybody who is sick of short battery life would want to buy it. So everyone. Possibly cell phone companies too, if they weren’t tied up with their own technology yet
  3. But no, I don’t actually have the special technology, nor do I know anyone who does. The nuclear batteries are not compatible with today's world.
  4. Everybody is already trying to serve this market.
  5. Batteries are highly patented, so I could easily be blocked even if I had an idea.
So this idea likely won’t get anywhere. Sorry.

Let’s try another one. Cheap hearing aids. The most basic hearing aids sell for 500.00 euros a piece at least in large parts of Europe, since that is what insurance companies reimburse. However, they only reimburse once every three to five years. If in that time, you lose one, or break it, you are on your own. Basic hearing aids are very basic, there is no need for them to cost 500 euros. And the fabled ‘programming’ consists of three knobs. Changing the clock on your microwave is harder. So, does this idea fly?

  1. Yes, the idea is possible. A quick web search shows that hearing aids, somewhat different from the ones we know, sell for less than 10 dollars a piece in other parts of the world.
  2. I personally don’t know a lot about retail, but I know enough about hearing & speech, compliance, the medical world, that I’m sure I can get these hearing aids acceptable to European markets and regulators.
  3. I know quite some (younger) hearing aid users, and they are all unhappy and want change.
  4. No one is doing this since it is easy to make stupendous amounts of money with regular hearing aids.
  5. Siemens, Phonak and others have lots of patents on advanced hearing aids, but basic ones have been around for ages and should be public domain. Traditional hearing aid shops won’t touch it (they are part of the racket), so we’ll have to work around them. Don't expect this to be easy.

Summing it up, I guess it is worth it to ponder this idea. Doesn’t mean it would work, but it would get out of the starting gate. Please contact me if you want to do it, I have some more ideas on this field. There's a (little) bit more to it.

Early stage startup patterns
Here are some common early startup patterns, some of which work:

  • Having a powerful idea, but you need to find people and suppliers for everything, since your own team (you?) lacks the native ability to implement the core of the idea.

    This means you might fail item 1, because you don’t know. You fail on item 2. And all the suppliers you need to approach might be beholden to your competition, so you fail 4 too. This never works. If you need to pay for all the work (instead of doing it as a company and owning the result), you are doomed. Gather more talent first, and give them equity (stock).
  • Launching a ‘platform’ for something that would be great once it was there and everybody was on it.

    I recently received a pitch: “.. will be an online multimedia platform that brings together four customer groups: biologists, press officers of research organisations, journalists and the public. We will identify the supply and demand related to nature information and will organise and facilitate the efficient, easy and quick search and exchange of content and contacts via web, apps and social media”.

    Once there, it would be great. But the greatness of the idea does not make it happen - the challenge is how to get there. Half a platform is no use. The probable reason it hasn’t been done is that it is very hard to get it off the ground. So by the metrics above, this fails 4 and 6. It can be done, it is not too easy, but lots of people would want to do it already, and everybody who has achieved partial success will try very hard to block you.

  • Great execution versus sleepy industry. I keep trying to get people to do this. Traffic lights. They are stupendously boring, and nobody’s main industry. An afterthought for an intersection. At least in Europe, most traffic light installations come with a giant cabinet that is placed somewhere next to the road. Refrigerator sized. What the f*ck do traffic lights require that even in 2014 they need controllers that big?

    So even without innovating a lot, you could enter this market simply by being nimble and efficient. You could power 250 intersections from the computing abilities of a cell phone. And once you have connectivity, add lovely control centers for the city that manages the intersections.

    Gather statistics, hand out magic devices to ambulances that pre-announce their trajectory to the hospital miles in advance, guaranteeing green lights all round!

    There’s no fundamental innovation going on, but it requires great execution to get this sleepy industry moving! And by the time the competition wakes up, the only thing they can do is buy you. Again, if you want to do this, contact me, I have lots more ideas on traffic lights. I'm like that.

  • Relying on partners as a shortcut to scaling. Thing is, actually scaling an idea is hard work. With notable exceptions, you’ll have to get people to sell, support and market the stuff around the world. And even if you have one of the rare ideas that sell themselves, you still need to build the infrastructure to support your customer base. It is a common theme for early stage startups to focus on their idea, and claim to find partners for the rest to speed up their time to market. I’ve fallen for this one too.

    So here’s the problem. Partners that could truly help you, since they are in the right place and have the capabilities, are likely to be invested in your competition. Even if not literally, they will ‘think like the established players’, and not like you. Secondly, to them, you are really small fry. How much of their attention will they focus on what to them might end up as 1% of their revenues? Will you get their best work?

    So any idea that starts with “we’ll focus on the core and speed up time to market with partners” needs more work, like for example, which partners would be interested and why. Count on having to do a lot of the heavy lifting yourself. (Note that it is fine for many places to buy commodity services like hosting, but this is different from ‘partnering’).

  • Ideas that deliver ‘millions’ in savings to ‘billions’ sized companies. So the thinking goes like this. You have a technology that objectively saves money, for example by allowing for slightly more efficient procurement. Say, 5% savings. For large enterprises, that adds up to millions and millions. If you’d be able to appropriate 20% of those savings, you’d be making good money. Business case done! I encounter a lot of this thinking among technology students, and I admit I used to fall for it too. (This is an example of having a bad mental model of your customer base, by the way).

    There are countless opportunities for increasing efficiency and saving 5% here and there - if only people would sign up. And there’s the rub. Your 5% savings are worth their while in absolute terms. However, unless you make *everything* 5% more efficient over at your customer, their bottom line (‘profit’) impact will be a lot smaller. However, the disruption you cause by getting them to move to your new technology is real. This means that a company has to decide to move to a startup’s technology to eventually increase profits by 1% - whereas profits go up and down by 30% all the time for other reasons. Seen this way, it is not a compelling case.

    To overcome this, your startup will need solid credentials and a great network. As a great example, see “Looking bigger”.

Wrapping up
Even good ideas tend to look bad initially, but as a founder, you’ll love your idea anyhow. But the idea is not yet a company, and you should try to work out the obvious kinks before looking for help. A potential investor or employee that can shoot instant & serious holes in your idea will not join you. If you show up with a compelling narrative that at least in theory could work, you might get the help you need!

Further reading