Monday, July 15, 2013

A common support anti-pattern: the stale issue that comes back to haunt you

So, here's a scenario if you are supporting users of (your) software or systems.  An urgent issue is reported, and you get to work on addressing it. After a while, a workaround is discovered and for now, the problem has gone away.  Or, what also happens frequently, the problem goes away by itself.

As a diligent supporting organization, you might 'ping' the user once to figure out of they are happy, and perhaps you still have some outstanding questions for them (log files, packet traces, versions installed etc). But otherwise, both user and vendor move on to more pressing issues, and we don't get to the bottom of it. It is not in most organizations' nature to focus on things that are not broken.

Time passes, and maybe a few months later, the customer is fuming, the issue is back, "and we reported this MONTHS ago, we have a 2 hour SLA, and it STILL isn't solved!" The blame is put squarely on the vendor, because the individual corporate employee most certainly isn't going to blame himself. It is just not done, and this is to be understood.

Meanwhile, you or your people dig out the old email exchange and note that "well yeah, but you didn't get back to us on X!", or the weaker variant "the workaround worked, and you went silent on it".

Escalation ensues, and it is noted that a more professional support organization would've kept nagging about the open question, or working on (what appeared to be) the low-priority remaining issue.

By now everybody is seriously pissed off at each other.

This anti-pattern is well known, and occurs everywhere. A common first-order approach to prevent it is for supporting organizations to attempt to proactively close issues that aren't progressing.

This sometimes works, but most often it makes the customer feel that their vendor is trying to artificially "solve" the issue, and not actually help.

Additionally, it doesn't feel good for people to have to agree to non-solved issues to be 'closed', or even 'solved'. In corporate environments, such things might come back to haunt the employee ('why did you sign off on that?!').

So, often a low-level stalemate develops where the customer is unwilling to spend time with the vendor to get to the bottom of the issue, but also not agreeing to close it.  And a few months down the road, BOOM, "this problem STILL isn't solved, and we've been at it for MONTHS!".

Neither side wants this, but it keeps on happening, and it keeps on pissing people off. It is human nature and corporate realities working against us.

So - what is the solution? Clearly we need some indication that is acceptable to all sides, but saves a lot of shouting later on.  One suggested way to achieve this is to add another status flag to an issue: 'Paused'.  This does not in any way imply the issue is solved, or  unimportant, or that anyone has agreed the fault is on their side.

It means what it says - this issue is paused. And if later on the problem becomes urgent again, it can be unpaused. Of course, the people that now shoulder more of the blame won't be too happy about it, but at least there is a reflection of the fact that *nobody* was working on it.

Supporting organizations meanwhile should remind supported users to respond to outstanding questions, and note that it is perfectly fine to agree to 'pause' the issue. This might even happen automatically after a few reminders.

So summarizing, by not angering people by closing issues the user is not actively working on, but by adding a 'Paused' status, when the problem  resurfaces, we can all get to work faster because the mutual screaming about issues being left unresolved for months 'while we have a 4 hour SLA with you!'

PS: And yes, if you really think this post is about you.. it might well be ;-)

Sunday, July 14, 2013

A "null result" bonus to improve science & science reporting

Every week we get at least one, but usually more, hype filled press releases & news items about how certain foods, medicines or lifestyle choices will either kill or save you. The vast majority of these weekly claims don't turn out to hold water.

As examples lifted from this actual week, I offer;

If you actually spend time on the press releases and underlying papers (if they even exist!), you often discover that:
  • there is no actual (new) research to back up the claims, or
  • that the claims bear scant relation to what is in the paper, or
  • that the data has been massaged heavily until some correlation popped out (and massaged & weak correlation is pretty far from causation, and most often proof of no causation).
These days, the discerning internet user can find sites that take the time to debunk over-hyped claims, but the brave souls dissecting the research behind the headlines will always be 'late', and secondly, they don't make Fox News or the New York Post.

So, the average person worried or interested in her health is bombarded by multiple confusing and conflicting headlines per week. This does nothing to improve our actual health, and in all likelihood worsens it ("forget that, the story changes every month").

What is behind this avalanche of weak or even bogus results in the news? It goes like this. Scientists perform expensive research, and very often, nothing spectacular comes out. Healthy people are healthier, people that exercise have lower blood pressure, folks that do things in moderation do lots better etc. 

Scientists are people too, and they have to justify their work, so they start the first round of trawling the data. And if you've measured enough, some interesting correlation always pops up! To counter this, Bonferroni correction should be applied to statistics, but not doing is so a common but helpful oversight. I mean, the research was expensive enough, something should come out!

So we have a claim, for example: 'Overweight post-menopausal women with pre-diabetes who eat fifth quintile amounts of avocados have lower insulin resistance'. This is typically what you'll find in a research paper, and where such a claim (had it survived Bonferroni correction, which it would likely have not) actually is worth reporting. Meanwhile, the claim is flagged with 'p < 0.05' which means the result is statistically significant; in actual effect, the impact can still be clinically insignificant (and often is)

Next, the research institute also wants to look good, so its PR department takes the paper, speaks with the scientists and writes a press release: "Benefit of eating avocados on insulin resistance, preventing diabetes". Note that they lobbed off all the qualifications, plus extrapolated the claim into preventing disease.

Finally, journalists fed this press release are eager for clicks on their articles, so they liven up the press release with some further human interest quotes and headline the piece: 'Scientists say: Eat avocados to ward off diabetes'. 

And there we go - from an investigation with no really significant results, we end up with a pretty stonking headline with incorrect advice. 

So what do we do?

Here's an odd idea. Zappos, an online shoe store, has a 'quit now' bonus for new hires. If after training you decide to leave, the company pays you $3000. The net effect of this is that people have an incentive to leave if they feel Zappos is not going to be a great place for them. 

And, although I don't know how it works in practice, in theory this should be a big win - anyone who stuck around against their will but thus inticed to leave will 1) not be a drag on Zappos 2) be able to move on to better pastures all the quicker.

The relevance to our scientists feeling pressured to publish should be obvious. Launch a fund, perhaps at department or institute level, or make it a national prize, for researchers honest enough to claim 'no significant results' from their research if there were none.

Compare the (at best misleading) headline 'Eat avocados to ward off diabetes' with 'Different levels of fruit consumption did not meaningfully change levels of diabetes among 3500 randomly selected staff of healthcare institutes'. 

The latter headline would admittedly not make the evening news. But it would allow investigators to move on to new research, and not further confuse the public. And very importantly, it would also make sure that even negative or null results make it to (the academic) press. 

As Ben Goldacre of www.alltrials.net often points out, not reporting unwelcome results leads to a statistical excess of positive results, thus "proving" that ineffective treatments actually work!

Now, I admit the details of this 'Zappos prize' would be daunting, and it would also require a significant fund to have any impact. It would need prestige too - scientists (who, as noted above are people too), are less swayed by money than most.

But something has to change. Today, mediocre research grabs the headlines while researchers honest with themselves struggle to get their voices heard!

Your thoughts are more than welcome.

Thursday, July 4, 2013

The biggest bet of your life

As I'm preparing my presentation for Observe Hack Make 2013 on modern nutritional science ('what you should eat to stay healthy and lose weight'), I keep running into people who have grave problems with the fact that I'm no longer eating bread, pasta or sugar, nor most processed foods.

Their issues continue even when I explain that I lost a ton of weight without hardship, and that many of my friends have done likewise.  Nor does it sway them that many in my family have normalized their blood pressure and diabetes by likewise adopting a low-carb "paleo" lifestyle.

So what I want to talk about is the big bet we are all taking. The bet is on these graphs:

Source: Adele Hite
Number of Americans with Diabetes. Source: Centers for Disease Control
Note the explosion of obesity (doubled) and, worse, diabetes (> tripled). These two are closely related.  Combined, obesity and diabetes will in the coming decades lead to skyrocketing rates of (at least) cancer, heart disease and dementia.  This in turn is already leading to reduced life expectancy, but far worse, a very strong reduction in the number of 'healthy years of life'.

Mentally, we are all  betting that we'll personally somehow escape this fate by our lifestyle or perhaps genetics. 

If you consume what has been described as the Standard American Diet ('SAD'), or an international equivalent, you are betting that the cause of exploding obesity and diabetes is not the food itself, even as you munch on your government recommended low-fat breakfast cereal with extra fibers.

The theory is then that our expanding waistlines are in fact caused by lack of willpower, since we're unwilling to eat a low-fat diet in moderation, nor are we willing to exercise enough. Otherwise we'd be fine.

You'll have to believe there has been a massive decrease in willpower, starting in the late 1970s, causing us to suddenly overeat and underexercise.  This is the general public health message, amplified by the food companies: there's no such thing as bad food, don't blame us, exercise more!

Oddly enough, in the 1950s and beyond, people didn't exercise enough either.  Nor did they eat a low-fat diet

Like me, many people these days are not betting on the 'collective lack of willpower' theory. I've done willpower, starving myself and spending 2+ hours a day exercising. It caused me to lose 50 kilos, and a constant fight to keep them off.  When I was unable to keep that up, most of it came back in a hurry.

Instead, these days many of us are betting on the fact that the food we've been eating contains far too many "heart healthy" carbohydrates, whole grain or not. We believe that we've been needlessly vilifying fats & cholesterol, which turn out not to be behind obesity and heart disease.

We are betting that the 20+ ingredients of a typical breakfast cereal can't all be healthy for us. We are betting that the highly processed vegetable oil and sugars that lace our foods are making us ill. And while we believe in the benefits of exercise, we know you can't exercise yourself out of a bad diet.

And unlike people starving themselves to get into smaller size pants, we are finding that eating actual full fat food, very little sugar & restricted carbs, is something you can keep up for life, while delivering rapid results in terms of weight and health.

So next time you meet one of us Paleo folks living the hard life of quality meats, fine fruit, actual vegetables, delicious fish & super soups,  instead of balking at our refusal of bread, margarine & Oreos.. ponder what bet you want to take:

Collective lack of willpower.. or perhaps eating overprocessed food too rich in sugar and carbohydrates?

(to read more, please head on to http://ds9a.nl/new-consensus/ )