The Business Moped Theory of New Product Rejection

So, you or your company invented something radical & new: congratulations! This has happened to me twice, once with PowerDNS and once with Fox-IT. Both times my products had unique features, and both struggled to gain market acceptance initially. Potential customers told us things that were wrong with our product, we’d fix them, and they’d come up with new reasons not to buy our stuff.

After a few iterations of this, I formed what I like to call “The Business Moped Theory of New Product Rejection”, and this theory has served me well (a moped is also known as a scooter or “brommer” in Dutch). In this post, I want to share the Business Moped idea, and how it spells bad news, but also what you can do with your new invention to make it sell better.

Commuting to the office

Back when I lived in Delft, parking my car was a big hassle, and more often than not I had to park it (semi-illegally) over a kilometer away from my home. If you live in a city center, this is quite common. Additionally, the office where I worked 10km away had 750 employees and 200 parking places, leading to similar drama there.

Now, I could go to work on my bicycle, but many of my colleagues lived too far away for that. But we had one guy who’d been commuting on a moped for 20 years now, and he was never late because of traffic and had no trouble parking either at home or at the office. Also, it was very cheap both in maintenance & fuel. He found a great solution, but nobody copied him.

http://www.tomosforum.nl/uploads/images/galerij/Brommer.jpg

And why not? Well, the honest reason is of course that it would look silly! You don’t go to your office in business clothes on a moped!

For evidence:

http://thumbs.dreamstime.com/x/business-man-scooter-8685824.jpg

“Dork”

But when you ask people why they continue to spend an hour stuck in traffic and parking everyday, and not just copy the guy with the moped, they don’t come out and say “it would look silly”.

No, they’d say things like “the car keeps me dry in the rain”, “In winter it is too cold”, “a moped isn’t safe”. So, this set people thinking, as moped vendors had long been trying to crack the business market. So they addressed the concerns, starting with this attempt:

http://upload.wikimedia.org/wikipedia/commons/5/5f/Benelli_Adiva_01.jpg

“Benelli Adiva”

With a roof, surely people will stop complaining about getting wet in the rain? Well, after 5 years of trying to sell the thing, Benelli gave up. I’ve never seen one in any case. But, the industry kept working on it, perhaps fixing the “rain” issue wasn’t enough?

Enter the BMW C1, which had everything:

• BMW: A cool business brand
• Roof
• Safety harness, no need to wear a helmet
• Heated seats!
• Heated steering handles!
• Storage for (rain) clothes

They even sold it to racing teams, police forces and other promotionally interesting customers:

http://upload.wikimedia.org/wikipedia/commons/b/b5/BMW_C1_-_Bayerische_Polizei_1.jpg

So now that all customer issues were addressed, it should sell well right? Think again, they sold hundreds!

But, this is BMW we are talking about, they had the resources to fix whatever complaints were left & up the marketing even further. So, they also offered:

• Anti-lock brakes
• “Executive leather seats”
• A navigation system
• Powerful audio, with automatic sound volume adjustment to speed
• An immobilizer system to prevent theft
• A sunroof!
• Glove box with power socket
  
  
  
  

http://upload.wikimedia.org/wikipedia/commons/7/73/C1RollerBMW.jpg

“Executive edition”

So, now that every *possible* customer complaint was fixed, sales surely took off? Thousands were sold! And after 5 years of trying real hard, even mighty BMW had to give up.

As a last salvo, Italian Piaggio came up with this marvel, for the people who complained that they didn’t want a “bike that could fall over”:

Also, not ever

http://www.webwombat.com.au/motoring/images/piaggio-mp3-scooter-3-big.jpg

Why didn’t it work?

People want to buy what they’ve been used to buying. They don’t want to be too different. You can get away with selling something that is vastly more powerful than customers were used to, but still is fundamentally the same thing - as long as buyers trust your company. But even trust in BMW was not enough to convince people that the moped was suitable for taking to the office, no matter how hard they tried to get their scooter to emulate the features of a car. When people secretly want a car, they don’t buy a scooter!

When as an innovative company you are trying to do something that is too different, that is not what you’ll hear from your customers (they might say it, but hearing it is something else). No, you’ll hear specific complaints that you can address. The pitfall is that even when you solve those complaints, they’ll come up with new ones. 

If you are trying to sell something that is too different, fixing the superficial issues is not going to save you - even though your potential customers assure you it will. They are just dishing up excuses. They want something “normal”.

Not being too different from something they DO know

I formed this theory a few years ago, and lately something interesting has happened. While it proved to be impossible to convince people to get a weird looking moped for commuting to the office.. it was entirely possible to electrify the venerable bicycle, and keep it looking “normal”:

http://www.post-fietsen.nl/site/webfiles/content/koga-e-bike.jpg

Note the discreet battery pack in the rear

And this is now taking off in a big way, without any excuses about getting wet etc. The issue was never with getting wet in the rain (they still had their cars as backups, right?). The issue was not wanting something too different from what everybody else had! And an e-bike is sufficiently close to a bike that droves are now using one.

So summarizing - be careful not to follow BMW in blowing a billion Marks on satisfying customers that aren’t telling you the real reason they don’t want your stuff.

Instead, find people that DO have an open mind, or try being close enough to what your customers already know, while still delivering superior value. Another solution is to wait it out until sufficient people start agreeing that what you offer is just better. Might take longer than you have though!

Good luck!

PS: I’ve also been told that the BMW C1 was a pretty crappy bike. That never helps!

Boekbespreking: Doe eens normaal man (in 7 stappen naar een betere politiek)

"Doe eens normaal man (in 7 stappen naar een betere politiek)" van Kustaw Bessems en Dirk Jacob Nieuwboer

Laat ik beginnen met zeggen dat ik vind dat dit boek in 250-voud over het Binnenhof verspreid zou moeten worden, een exemplaar voor ieder kamerlid, minister en staatssecretaris. (Update: Dirk Jacob meldt dat alle 150 Tweede Kamerleden een exemplaar gekregen hebben!) Tevens zou nog een flinke oplage neergelegd kunnen worden in de diverse kamerrestaurants & toiletgelegenheden, voor het geval iemand het gemist had.

Van de achterflap "Hoe komt het dat politici zo gek praten? Waarom vinden ze het zo moeilijk sorry te zeggen? Mogen ze trots zijn op een compromis?". En dat raakt me. Ik kom uit een politiek nest, je kon in het verleden meerdere malen stemmen op mijn vader, mijn moeder en zelfs oma, voor provinciale staten, gemeenteraad en we hebben zelfs tweemaal een gedeputeerde gehad. 

Als kind stonden mijn broer en ik vooraan met prinsjesdag in het gebouw van de tweede kamer, omringd door de kinderen van kamerleden & andere kamermedewerkers. Dolle pret. Mijn moeder zette haar kooktalent (naast haar beleidsmatige werk!) maar al te graag in voor politieke doelen, wat resulteerde in dinertafels vol politici. Van huis uit was de politiek ons dierbaar, en iets van heel dichtbij. 

En daarom doen de afgelopen decennia mij zo'n pijn. Een politicus breekt een verkiezingsbelofte schofterig, en staat slechts uren later uit te leggen dat hij zo blij is met het resultaat. Er zijn grote fouten gemaakt en er volgt een excuus dat je van een peuter niet zou accepteren ("als ik mensen gekwetst zou hebben betreur ik dat"). Maar ook, een politicus presenteert een visie en krijgt kritiek dat ie geen concreet plan heeft, een andere politicus presenteert een concreet plan en wordt verweten geen visie te hebben. Het is ook nooit goed!

In "Doe eens normaal man" leggen Bessems en Nieuwboer (beide journalisten) uit hoe dat nou zo gekomen is, en uniek, ze bieden ook suggesties hoe het op te lossen. 

Het eerste hoofdstuk begint gelijk heel sterk, en ik denk eigenlijk dat dit als wortel van het boek zelfs nog wat onderbelicht blijft. Een politicus moet weten waarom hij of zij de politiek in ging. Als ie eenmaal aangekomen is in Den Haag moet ie het niet vergeten, en hij moet het ook duidelijk uit blijven spreken. Bij afwijking van de idealen mag je dat best zeggen. Met pakkende voorbeelden over Wouter Bos, Job Cohen, Diederik Samsom en Jeanine Hennis maken de schrijvers duidelijk hoe het (niet) moet.

Persoonlijk denk ik dat als je als politicus geen helder antwoord hebt (en durft te geven) op de vraag waarom je nou de politiek in wilde je net zo goed gelijk kunt stoppen. Dit is echt de wortel van het verhaal, wat overigens uitgaat van politici die oprecht hun best willen doen.

Het volgende hoofdstuk beschrijft het vermaledijde "draaien". Normale mensen veranderen, gegeven veranderende inzichten of omstandigheden, van mening. Bij politici heet dit gelijk "draaien" en is dat "niet eerlijk". Wederom met voorbeelden leggen de auteurs uit hoe je fatsoenlijk moet en kunt draaien, en dat als je het uitlegt je publiek dat ook zal begrijpen. Zo dom zijn die stemmers niet namelijk.

Volgende hoofdstukken geven gelijksoortige inzichten in het sluiten van compromissen (wees eerlijk dat het jouw keuze niet was, leg het niet zo uit alsof je toch gewonnen had), excuses aanbieden (oprecht en zeg wat je geleerd hebt), "geen commentaar geven"  (leg uit waarom je niks zegt, ontken niet dat je niks zegt!) en hoe normaal te blijven praten.

De rode draad in dit verhaal is dat als je een helder en eerlijk verhaal hebt, en je niet poogt te verstoppen achter Haags gekonkel of slimpraterij, je electoraat dat zal waarderen. En wat hoop ik dat dat waar is! Bessems en Nieuwboer zijn beide goede verstaanders en denken na over wat ze horen en lezen. Hun inzichten (ook buiten dit boek) bewijzen dat. 

En het doet hun pijn hoe politici vol met wijsheden als "in een vlek moet je niet wrijven" en "stilzitten als je geschoren wordt" steeds vreemder praten en schijnbaar nooit meer met minder dan absoluut succes uit een vergadering komen, en zeker nooit ongelijk hebben gekregen!

Ik ben er echter niet zeker van of de door de auteurs aanbevolen duidelijkheid (en ze geven vele goede voorbeelden die op mij erg overtuigend overkomen) ook aansluiting zal vinden bij het publiek en met name de media.

Het siert de auteurs dan ook dat ze zelf ook nog een afsluitend hoofdstuk 'media' opgenomen hebben, over hoe journalisten zelf bijgedragen hebben aan de vervreemding van politici. Met prachtige voorbeelden laten ze zien dat "het ook nooit goed is". De media zullen nieuwe duidelijkheid niet moeten gebruiken als excuus gehakt te maken van de helder pratende politicus!

Tussen de concrete hoofdstukken bovenstaand genoemd zitten prachtige epistels over het wisselen van zwakke ministers, vijf nieuwe "droompartijen" (ik zou erop stemmen!), de gevaren van tot laat door vergaderen en de onzin van bezuinigen op de politiek zelf.

Afsluitend, zoals eerder gezegd zou ik het geweldig vinden als ieder aanstaand kamerlid bij de eerste vergadering een exemplaar van die boek zou aantreffen op zijn of haar zetel. Samen met "Je hebt het niet van mij" van Joris Luyendijk (wat een stuk cynischer is) zou dit boek tot de verplichte introductiecursus moeten behoren.

En ik hoop van harte dat de media en het publiek de resulterende helderheid en duidelijkheid niet zullen afstraffen, maar waarderen.

Being a sophisticated & credible customer, or how to deal with one that isn't

Some years ago I wrote a post on the pains of (public) procurement in large organisations and governments. It appears this post struck a chord since it continues to receive lots of visitors.

Perhaps because of this, while on holiday I became inspired to write up some more detail on how I think things often go wrong. In passing, this may explain why your government ended up with a 3 billion euro public transport ticketing system that still doesn't work well, or why your city spent half a billion dollars on time tracking software. Both examples are real, by the way.

I've written this post with the most experience on the seller side, and you can read this as advice to ethical sellers and to all buyers. If you are selling stuff and you encounter a hapless buyer, the story below may prevent you from earning a lot of money, but it might save your soul.

For start-ups, knowing if your first customers are credible and sophisticated or not will tell you if they should guide you, or if you should be guiding them.

Sophisticated buyers
When you are a sophisticated buyer, you know what you are buying, and harbor no illusions about the product or what it should cost. As an example, most people I know are sophisticated buyers of beer. You know what it is ('drink with alcohol'), where it is sold, and that to open the bottle, you need an opener, which is typically not included with the crate. Even though the marketing indicates that beer leads to parties, an attractive lifestyle, and it appears, sporting events, you aren't fooled. The bottle contains beer. If you want the attractive lifestyle too, it is your own problem.

Credible buyers
A credible buyer is someone in a position NOT to buy, and can buy somewhere else, and is also able to complain bitterly if the product is not up to scratch. Most people buying beer are credible buyers, as there are loads of places that sell the stuff, prices are well known, and the seller can rightfully fear that the customer will leave forever if he doesn't get a refund on beer that was sold past its date.

Unsophisticated buyers
To stick with beer, we can turn ourselves into unsophisticated buyers simply by scaling up. If we need to arrange for beer for 5000 people on a hot summer day, all of a sudden we don't know what to do. Just buying a 500 crates of beer won't do it, as we'll never get them cold enough in time. We might have heard about kegs, and coolers and whatnot, but we can't just order some stuff and hope that it will work. We'll likely discover we don't have enough cooling capacity, not enough electricity to power the coolers etc, or that the coolers only function after having been left alone for 24 hours. Who knows.

Other typical unsophisticated buyer behaviour includes ordering things that don't exist ("we'd like the 35L Heineken kegs", whereas only the Bud kegs come in that size, and then being massively unhappy), or trying to buy things no one single seller sells.

As an example, when the request for a quote for beer comes in it all of a sudden includes also serving a sizable BBQ dinner. When asked why, the customer says last time round they had the folks supplying the beer vying with the BBQ vendor for power outlets "and this time round we want one single supplier" - despite the fact they are sending the quote to a company that rents out beer supplies!

In short, the unsophisticated buyer needs help, and isn't in a position to second guess the supplier. We'll just have to state our desires ('beer, 5000 people, hot summer day, next week') and hope they show up with the right stuff. Our seller becomes our consultant who tells us what to buy.. from him.

Non-credible buyers
If we are not a credible buyer we are not in a position to negotiate. As an example, let's say we are a large corporation and we've invited 5000 people to the opening of our new building on Friday, and it is now Tuesday. If we approach vendors to supply us with cold beer three days from now, we aren't really in a position to negotiate. We just have to hope we'll not be extorted too badly.

But there are more ways not to be a credible buyer. Particularly governments and large enterprises have a hard time convincing sellers that they actually want to get a good deal, but let me explain.

If you are a buyer in a big place, chances are the product or service you are going to procure will never be used by you personally. In fact, the situation is often so highly specialised that the buyer physically will actually never see the product, and may not even know what it does (or worse, what it is supposed to do).

And in fact, the buyer may have a mission to shave off a few percentage points of costs, by the time the ordering process hits procurement, a budget has already been allocated. So the seller knows the money is sitting there, and he just needs to get it, with the minimal amount of costs involved. And worse, the seller also knows that the buyer personally is worried about other things than price (it is not his own money he is spending!)

Buyers worry about not getting blamed for things, and an effective seller will make sure the buyer personally can feel safe on this front.. and worry less about price.

Another way not to be a credible buyer is when your CEO or minister or whatever has already said that a project will happen, and that it will happen on time, and that he's personally committed to that. That's like a blank check.

An optimum
There is an optimum situation when buyers are sophisticated and credible. Although as a seller you won't be making lots of money, at least everybody knows what is being expected, and knows not to fleece each other. As a start-up, this may be your best initial customer.

An unsophisticated but credible buyer can still get decent service. Since they are able to walk away from a deal, or can credibly switch suppliers, a vendor had better provide them with the goods.. eventually.  For a start-up, an unsophisticated customer can lead you to chase products or services nobody should be wanting.

A sophisticated but non-credible buyer knows what he wants, and will at least know when he is being fleeced. Damage limitation is possible, for example by setting a maximum price. This can be very good for start-ups, as the buyer is willing to take a risk to at least get a product or service.

A pessimum
The unsophisticated, non-credible buyer. Ahhhh. Getting back to the beer example, this is when people are unable to tell apart the marketing ('sport, attractive people, parties') from the product ('water with carbs and alcohol'), and end up thinking they bought the one and getting the other. Or, ordering 100 kegs of beer, and not realizing the need to also order 100 cooling dispensers.

In terms of the timekeeping software example mentioned above, this might consist of ordering the data entry technology and then discovering that no software is included to actually make reports or derive statistics from that data.

Since by this time a buyer can no longer be credible ("we'll get to the bottom of this and deploy before the end of the year!"), even if he was in the first place, we've now reached the pessimum: the buyer doesn't know what he should be buying, and can't negotiate about it.

So, now even more software has to be brought in, against ridiculous prices. The timekeeping example is from New York City, read all about it here.

Further exacerbating the situation is that almost nobody would describe themselves as an unsophisticated non-credible buyer, and instead pretends to know what they are talking about, and are mean negotiators - making it even more likely that costs & and problems will explode!

Non-solutions
Buyers often attempt to cover over their lack of credibility and sophistication with massive amounts of legalese. This rarely works. It is typical for example for a buyer to write giant lists of specifications, like "must be able to serve beer to 5000 people", but this will still not save you if you neglected to specify that they would train your people ('how hard can it be?') or would supply servers.

Also, threats of lawsuits do not save your event this Friday!

How to solve this
Becoming a "credible buyer" is very tricky if your organisation isn't like that. Even if you try to get three potential suppliers for your megaproject, for which the ministry has already allocated and published a budget, all three of them will attempt to exhaust your entire supply of money. There are also tricks like awarding a deal not to the cheapest bidder but to the second cheapest, which sets up game theory in your favour. But it is hard to do. Within a large government typically only people who can change or set budgets can be credible buyers.

Becoming a sophisticated buyer is easier however, but it has to start with the realization you aren't one yet. That may in fact even be the biggest step. And don't feel bad about it, most organisations rarely buy new buildings or mega IT projects - while sellers sell such things all the time. It is very normal not to be sophisticated about things you don't do that often!

It is tempting to hire consultants or an integrator to help you buy your project, but this puts a great strain on the integrity and quality of your partner, as you will be relying on them completely again. Are you a sophisticated & credible buyer of procurement services?

A better way is to invest in your own staff and make sure you gain sufficient understanding that your own people can become "sophisticated buyers". This requires time and money, but it will pay itself back a thousand times over. Easily.

Another good thing to realize is that the experts might already be working for you, but that you actually need to involve them.

Rounding up
Successful transactions are very common with sophisticated, credible buyers. Complete and utter failures with giant cost overruns are common for unsophisticated, non-credible buyers.

Don't try to buy stuff you don't have experience buying.  Get people working for you that have that knowledge and experience. And listen to them - they might be working for you already.

Try not to buy stuff unless the seller knows you are at liberty not to buy or buy somewhere else.  Make sure that the actual people involved in buying things are not so separated from the issue that their own job assurance is more important then actually getting a good deal.

A common support anti-pattern: the stale issue that comes back to haunt you

So, here's a scenario if you are supporting users of (your) software or systems.  An urgent issue is reported, and you get to work on addressing it. After a while, a workaround is discovered and for now, the problem has gone away.  Or, what also happens frequently, the problem goes away by itself.

As a diligent supporting organization, you might 'ping' the user once to figure out of they are happy, and perhaps you still have some outstanding questions for them (log files, packet traces, versions installed etc). But otherwise, both user and vendor move on to more pressing issues, and we don't get to the bottom of it. It is not in most organizations' nature to focus on things that are not broken.

Time passes, and maybe a few months later, the customer is fuming, the issue is back, "and we reported this MONTHS ago, we have a 2 hour SLA, and it STILL isn't solved!" The blame is put squarely on the vendor, because the individual corporate employee most certainly isn't going to blame himself. It is just not done, and this is to be understood.

Meanwhile, you or your people dig out the old email exchange and note that "well yeah, but you didn't get back to us on X!", or the weaker variant "the workaround worked, and you went silent on it".

Escalation ensues, and it is noted that a more professional support organization would've kept nagging about the open question, or working on (what appeared to be) the low-priority remaining issue.

By now everybody is seriously pissed off at each other.

This anti-pattern is well known, and occurs everywhere. A common first-order approach to prevent it is for supporting organizations to attempt to proactively close issues that aren't progressing.

This sometimes works, but most often it makes the customer feel that their vendor is trying to artificially "solve" the issue, and not actually help.

Additionally, it doesn't feel good for people to have to agree to non-solved issues to be 'closed', or even 'solved'. In corporate environments, such things might come back to haunt the employee ('why did you sign off on that?!').

So, often a low-level stalemate develops where the customer is unwilling to spend time with the vendor to get to the bottom of the issue, but also not agreeing to close it.  And a few months down the road, BOOM, "this problem STILL isn't solved, and we've been at it for MONTHS!".

Neither side wants this, but it keeps on happening, and it keeps on pissing people off. It is human nature and corporate realities working against us.

So - what is the solution? Clearly we need some indication that is acceptable to all sides, but saves a lot of shouting later on.  One suggested way to achieve this is to add another status flag to an issue: 'Paused'.  This does not in any way imply the issue is solved, or  unimportant, or that anyone has agreed the fault is on their side.

It means what it says - this issue is paused. And if later on the problem becomes urgent again, it can be unpaused. Of course, the people that now shoulder more of the blame won't be too happy about it, but at least there is a reflection of the fact that *nobody* was working on it.

Supporting organizations meanwhile should remind supported users to respond to outstanding questions, and note that it is perfectly fine to agree to 'pause' the issue. This might even happen automatically after a few reminders.

So summarizing, by not angering people by closing issues the user is not actively working on, but by adding a 'Paused' status, when the problem  resurfaces, we can all get to work faster because the mutual screaming about issues being left unresolved for months 'while we have a 4 hour SLA with you!'

PS: And yes, if you really think this post is about you.. it might well be ;-)

A "null result" bonus to improve science & science reporting

Every week we get at least one, but usually more, hype filled press releases & news items about how certain foods, medicines or lifestyle choices will either kill or save you. The vast majority of these weekly claims don't turn out to hold water.

As examples lifted from this actual week, I offer;

• Fish oil supplements linked to prostate cancer
  Not quite: this was observation of people with existing prostate cancer, in all likelihood influenced by their earlier diagnosis. More analysis here.
• Diet soda drinkers might get fatter and unhealthier than their 'regular' soda counterparts
  Not quite: this was an 'opinion' piece full of theoretical ways this might be the case, not backed up by actual research. More analysis here.

If you actually spend time on the press releases and underlying papers (if they even exist!), you often discover that:

• there is no actual (new) research to back up the claims, or
• that the claims bear scant relation to what is in the paper, or
• that the data has been massaged heavily until some correlation popped out (and massaged & weak correlation is pretty far from causation, and most often proof of no causation).

These days, the discerning internet user can find sites that take the time to debunk over-hyped claims, but the brave souls dissecting the research behind the headlines will always be 'late', and secondly, they don't make Fox News or the New York Post.

So, the average person worried or interested in her health is bombarded by multiple confusing and conflicting headlines per week. This does nothing to improve our actual health, and in all likelihood worsens it ("forget that, the story changes every month").

What is behind this avalanche of weak or even bogus results in the news? It goes like this. Scientists perform expensive research, and very often, nothing spectacular comes out. Healthy people are healthier, people that exercise have lower blood pressure, folks that do things in moderation do lots better etc. 

Scientists are people too, and they have to justify their work, so they start the first round of trawling the data. And if you've measured enough, some interesting correlation always pops up! To counter this, Bonferroni correction should be applied to statistics, but not doing is so a common but helpful oversight. I mean, the research was expensive enough, something should come out!

So we have a claim, for example: 'Overweight post-menopausal women with pre-diabetes who eat fifth quintile amounts of avocados have lower insulin resistance'. This is typically what you'll find in a research paper, and where such a claim (had it survived Bonferroni correction, which it would likely have not) actually is worth reporting. Meanwhile, the claim is flagged with 'p < 0.05' which means the result is statistically significant; in actual effect, the impact can still be clinically insignificant (and often is). 

Next, the research institute also wants to look good, so its PR department takes the paper, speaks with the scientists and writes a press release: "Benefit of eating avocados on insulin resistance, preventing diabetes". Note that they lobbed off all the qualifications, plus extrapolated the claim into preventing disease.

Finally, journalists fed this press release are eager for clicks on their articles, so they liven up the press release with some further human interest quotes and headline the piece: 'Scientists say: Eat avocados to ward off diabetes'. 

And there we go - from an investigation with no really significant results, we end up with a pretty stonking headline with incorrect advice. 

So what do we do?

Here's an odd idea. Zappos, an online shoe store, has a 'quit now' bonus for new hires. If after training you decide to leave, the company pays you $3000. The net effect of this is that people have an incentive to leave if they feel Zappos is not going to be a great place for them. 

And, although I don't know how it works in practice, in theory this should be a big win - anyone who stuck around against their will but thus inticed to leave will 1) not be a drag on Zappos 2) be able to move on to better pastures all the quicker.

The relevance to our scientists feeling pressured to publish should be obvious. Launch a fund, perhaps at department or institute level, or make it a national prize, for researchers honest enough to claim 'no significant results' from their research if there were none.

Compare the (at best misleading) headline 'Eat avocados to ward off diabetes' with 'Different levels of fruit consumption did not meaningfully change levels of diabetes among 3500 randomly selected staff of healthcare institutes'. 

The latter headline would admittedly not make the evening news. But it would allow investigators to move on to new research, and not further confuse the public. And very importantly, it would also make sure that even negative or null results make it to (the academic) press. 

As Ben Goldacre of www.alltrials.net often points out, not reporting unwelcome results leads to a statistical excess of positive results, thus "proving" that ineffective treatments actually work!

Now, I admit the details of this 'Zappos prize' would be daunting, and it would also require a significant fund to have any impact. It would need prestige too - scientists (who, as noted above are people too), are less swayed by money than most.

But something has to change. Today, mediocre research grabs the headlines while researchers honest with themselves struggle to get their voices heard!

Your thoughts are more than welcome.

How to give a decent presentation

Hi everybody,

I frequently attend conferences, and about as frequently give presentations there. Sadly, over the years, I've seen many smart and gifted people struggle to share their work with their audiences. Luckily, over time, watching & doing presentations has taught me a little bit about what makes a good presentation.

In general materials on presentations, there is a lot of emphasis on using the right fonts, maintaining eye contact with the audience and otherwise being "convincing". Such advice is of little use for the attendants & presenters at technical conferences though.

We want good content, not suave presentations! And that is a good thing since many of us in the tech community tend to be a lot better with computers than with being 'flashy'.

This year we'll again be seeing the four-yearly cycle of great hacker conferences in The Netherlands continue with OHM2013: Observe Hack Make. These events are volunteer organized, and as part of doing my bit, I thought I'd compile what I've learned about doing presentations. This will make me feel less guilty also when I see people digging trenches etc.

On http://tinyurl.com/decent-presentation you can find a Google document that contains a presentation on doing just that: giving a decent presentation. And, since slides can't and shouldn't tell the whole story, I've narrated this presentation here and here on YouTube.

This presentation outlines a process of getting to great content (and also touches on how to present that content well). This process starts with answering questions: WHY, WHO, WHAT and HOW. The WHY and WHO determine WHAT to tell, and at which level of knowledge your presentation should start.

The HOW tells you how to replicate your knowledge in the minds of the audience.

At OHM, I too will be presenting, and as an example, I'll go through these four questions here for my presentation on "What you need to know about what you eat: health & weight".

• WHY: We all get more and more obese, even people perfectly following government advice on how to eat and exercise. Over the past decade, a new consensus has arisen on why we get fat, and we now know that the conventional wisdom has it all upside down, and is making us sick. I'm presenting because I want to share what I've learned in order to let everybody share this new knowledge, so we can save their health!
• WHO: Hackers with shoddy exercise and eating habits. Many of us where at GHP and HEU over 20 years ago, and I can tell you, the hacker community.. is getting bigger (or at least larger). Especially us 'older' folks are starting to care about what we eat and do. The audience will care, but will not necessarily know the finer distinctions between cis- and trans-fats etc.
• WHAT: We have one hour, so we can't explain the full modern nutritional theory. So, we'll be explaining basics, plus specific things people can do to improve their health. Also, pointers so people know where to go to learn more.
• HOW: I have to build it up. If I just get on stage and start ranting about glucolipotoxicity, nobody will know what to make of the story. Introduction is my own story, and that of my family. This makes it personal and interesting. Then we demolish the conventional wisdom with powerful and horrifying graphs. Next we explain some basics that make it obvious current advice is all upside down. Then, once that is clear, clarify what does work. Finally, we round off with a highlight of the most interesting people, books & groups.

With these questions answered, I know what content I need to write, what pictures and graphs I need to gather, and how to keep people paying attention!

It is my sincere hope that if you'll present at OHM, or at any other geeky conference, that you'll be able to benefit from the presentation, and that you'll be better able to get your ideas across!

Finally: to anyone aiming to present at OHM, please contact me if you think I could help with your presentation, for example, by brainstorming on WHO you'll be explaining to and WHAT!

Some notes on medical statistics

Over the past year, I've been reading more and more about the causes of obesity and the (related) epidemic of diabetes, since both run in my family.  In my readings, I've encountered a lot of dodgy statistics to bolster research claims.


Statistics allow us to make statements like 'the chance that these dice are unfair is less than 1%', based on throwing them n times and observing the results. We call such results 'significant', where the threshold for significance is often set at '5% chance of results being random and not because of some effect'.

(and for the statistics professionals, I know my terminology is sloppy. Have this comic to make up for it:)

http://xkcd.com/795/

The world of medical research also tries hard to do statistics, and by and large fails at this. Partially this is due to a misunderstanding of how statistics work, and partially this is a problem of language.

For example, a pill which causes a 1% absolute reduction in the number of heart attacks in a population can easily result in a 'statistically significant effect'.  This is because we might be *very* sure that the "Odds Ratio" of having a heart attack is 0.99 and not 1.  "p < 0.05".  This number is not clinically significant though, or more concretely, it is an irrelevant number.

Public relations departments, funding considerations and industry relations however just scream to turn this mathematical, statistical significance into a bold press release reporting an actual significant medical advance.

However, since heart attacks are rare, hundreds of people would spend decades taking this particular pill before a single actual heart attack would be prevented.  And who knows how many side effects there would have been!  So, statistical significance does not equal practical significance.

A far better metric is called The Number (of people) Needed To Treat (NNT) to get benefit.  For example, the NNT of common painkillers for treating a normal headache is very close to 1, since they almost always work.

The NNT is far more powerful than "relative statistical significance". For example, although 25% of the over 45 population in the US is now prescribed statin pills, its NNT for preventing a heart attack for people without prior heart disease is 300 person years, or, described differently, if 60 of those people take statins for 5 years, 59 of them receive no benefit.  All 60 are at risk for potential side effects however.

The NNT for preventing a *fatal* heart attack in this population is in fact immeasurably high ('infinite'). For people who have had a heart attack already, the NNT for preventing death is around 80 for 5 years.

There is also "the NNT for harm", which for statins is about 10 after 5 years. In other words, of those 60 people treated for 5 years, 6 of them would have a serious side effect.

The NNT & NNT for harm are medical statistics done right; and it is therefore no surprise these numbers are exceedingly unpopular in press releases and articles.

So next time you read about a medical breakthrough.. look beyond the reported statistical "significance" and see if you can find the NNT.

Some good links for further reading:

• NNT numbers for many treatments & diagnostic tests can be found on the most
  excellent site www.thennt.com.
• Wikpedia: http://en.wikipedia.org/wiki/Number_needed_to_treat
• The Cholesterol Conundrum: http://www.saturdayeveningpost.com/2012/04/24/wellness/cholesterol-conundrum.html
• Bad Pharma book: http://www.amazon.co.uk/dp/0007350740/ref=nosim?tag=bs0b-21