Tuesday, April 1, 2014

Seen that sketch, "The Expert"? Well, I blame the expert too.

So this post has been brewing for years now, but this short movie triggered me to finally write it up:

(Written & Directed by Lauris Beinerts, based on a short story "The Meeting" by Alexey Berezi).

It has been widely shared within the computing community, and is often described as too painful to watch and frequently as 'hitting too close to home'. It took me several tries until I was able to view it all the way to the end. It is painfully magnificent. 

In the sketch, five people meet, two from the customer, and three from the supplier. The customer voices artsy requirements for the project, involving red lines drawn in blue ink. Such impossible requirements are hauntingly familiar to anyone in the tech field.

And, viewed on one level, the film is about the plight of 'the expert' surrounded by fools, finally capitulating into spouting the same nonsense: can I inflate the lines as a cat shaped balloon? Why yes I can.

And seen in this way, the movie has been widely described as being indicative of people's working lives. I can personally share several stories that while not involving inflatable kittens with transparent lines in red, drawn in blue ink, are just as unlikely. 

My favorite story involves one of the first 'professionally run' Internet Service Providers in The Netherlands that decided to do a promotion around Valentines Day. And in their wisdom, the marketing people had decided not to involve anyone who knew what they were doing, and based it all on a website to be launched on XXX.NL, where XXX is Dutch for three kisses. Of course, XXX.NL was already a hardcore porn site. But everyone involved assumed that the experts could just launch a new site on XXX.NL. In the end, since all the materials had been printed already, XXX.NL had to be rented from the owner at great expense, and marketing still found a way to blame it on the techies. 

As such, the sketch is a great way for us geeks to show the world how we feel about being surrounded by what is often called 'the Damagement', a cheap joke on 'management', which by the way includes everyone not actually touching or creating new technology.

And in fact, the psyche of the engineer here typically is that they consider themselves the only smart people in the company, and that the rest are bumbling idiots. To be fair, this attitude is typically shared about truly senior management ('Use smaller words and bigger letters, this is for the board').

In the movie, we see a painfully honest expert (dressed up in short sleeves, with required pen in pocket) being flummoxed by boneheaded stupidity around him. 

It is my interpretation however that, we, experts & engineers, are just as much to blame for this happening. It is not just about stupid 'damagement'. But allow me to explain.

In established fields, whereby I mean, things that have been around for centuries, people tend to realize their lack of knowledge. For example, most of us would not attempt to build a house ourselves from parts. In fact, most of us would go one step farther: we wouldn't be qualified to even hire bricklayers etc to do that for us. Instead, we rely of a chain of suppliers to make that happen.

In analogy with the sketch, at the beginning of a building project, customers use artsy terms like 'sense of space and direction', 'challenging the environment'. This stuff we then lay on an actual architect, who is also an artist. They get this artsy stuff and can deal with it.

In turn, the architect draws a very nice vision of the building and goes over it with us until we like it. But then something interesting happens. The architect hands over the nice drawings to a design engineer ('Constructeur' in Dutch, or architectural engineer in other places).

The design engineer, who does not actually construct the house, checks if the drawing COULD be built. He may come back with 'well, you could do that 12 meter arch, but you'd have to build the house out of solid titanium'. He might instead suggest a nice sturdy pillar to mess with your "sense of space and direction". Thus, design engineer & architect hash it out, and may further negotiate with the customer on what is possible and at what cost. 

Once this process has finished, the 'what' of the architect, the 'how' of the design engineer gets turned into an actionable plan by a builder. And thence, actual bricklayers get involved, and get provided charts where to lay their bricks, and what kind etc.

Over the centuries, this chain has developed, and while cost overruns are frequent, as are late deliveries, we generally get buildings right. Short of the Berlin Airport, for example, we rarely hear of building projects that just didn't work. They may be late and expensive, but they get there. IT projects typically go for 3 out of 3: too late, too expensive, don't do what we actually wanted.

Now compare this with the sketch. In theory they got it all right! So we have a customer, two people in between and then an expert. However, both as geeks and management we go about this all wrong.

Programmers will ALWAYS lament that customers never articulate their requirements (or even specifications) properly. In this way, they are operating like bricklayers "tell me where the wall should go, how high it should be, what colour, and we'll get on it". 

Meanwhile, the rest of the world thinks they are talking to all knowing architects. They aren't providing detailed requirements, because they can't. 

Ah, the IT architect, we have those! And I've yet to meet one worth his salt. The problem is that the IT architect typically has no recent hands on experience (if he has any at all), but is also held in higher regard than the actual people that have to implement it. In the world of building houses, the design engineer overrules the architect. In the world of IT, the IT architect can just goof off, since he's supposed to know how to do it. If it fails, blame the folks doing the typing or the customer.

So why don't we have IT design engineers? Well, there are a few, but unlike the building industry, we don't yet have 100 years of experience. Check back in a few decades, and we might have learned how to do it. (Oh, and if you feel I badmouthed you because *you* ARE a good IT architect, I'd venture you are probably one of the few good IT design engineers, which is great!)

And verily, we geeks often reinforce the perception that we know everything by frequently reminding everyone of what idiots they are. I personally was made unwelcome in an organization after branding all the non-techies "art school graduates". So is it any wonder that people just lay their bare feelings on us and say 'now you solve it, expert?'

But, back to the sketch - in a better world, the expert would still be an expert. But the two people in between would know more than just hand on the question (and even making it worse along the way). 

They should be bridging the gap 'tell me more about those lines'. Actually draw out the customer, help them articulate their vision, and start the process of figuring out what they really want. 

Because the dark secret is, 'the perfect customer' that can tell you exactly what they want doesn't exist. This is for the simple reason that a customer that is that skilled doesn't need any outside help! So in reality, you'll always have to work with them to clarify their needs.

But, back to the sketch, the two people in the middle should have cushioned all this vagueness, and in private consultations with the expert, have worked on making it 'crisp', something that could be done.

This would have allowed the expert to be much happier, and not be painted like an idiot. (And, on a side note, even though many claim this sketch describes their working lives, that obviously can't be the case for long. This is not how anyone stays in business!)

However. The world in which we live is such that the expert is often pitted directly against the 'art school graduates'. Right now, many of us geeks subscribe to the notion that WE are all brilliant, but that management universally sucks (whereby management = anyone that doesn't actually touch or create the technology). And this notion allows us to continue acting like the 'expert', going home frustrated, telling our friends how much everything sucks, and our friends agree.

But this can't be the truth. This 'management' drives home in fancier cars than ours. And their homes are fancier too. Plus, they are never on call! They must be doing something right! I'm sure their lives must be very empty, knowing nothing but art, but it is not tenable for us geeks to pretend we have it so much better and that everyone else is an idiot!

So finally getting to the point of my post, as experts and implementors, we should stop complaining about idiotic requirements. Society has for now cast us in the role of architect, design engineer AND bricklayer. 

And for better or worse, if we want to get ahead, we should act like it. This may not be easy, but for example, our expert in the sketch started out with stating 'impossible' and informing the customer they are idiots ('your scheme only works if you are colour blind'). This does not aid communication, but it is what honest geeks do: tell you exactly in gory and imposing detail why it can't be done ('.. and even if it COULD be done, you should not be doing this').

At one point, our expert asks the customer to elaborate on what they want, but he's quickly cut off by his colleagues, and this is indeed quite common, as often discussion then continues with FURTHER "impossibles" being raised. However, this is exactly what we as enlightened experts should be doing: talking to the people with the requirements and getting them to elaborate on them.

If people think we are full-blown architects, we should oblige them. 

So, hold off on the 'impossible', or imputing a lack of education. Turn the tables on the customer (or the people with the requirement), and get them to talk. 

Don't lament that they are unable to provide decent specifications. And most CERTAINLY hold off on complaining that once you delivered according to bad specifications, it is all their problem for not properly asking what they wanted. That is BRICKLAYER thinking. And guess what, that leads to videos like these.

(Don't get me wrong on manual laborers - they are important, and I respect them. But don't lay "a sense of space and direction" on them and expect a proper wall that does that).

Techniques to employ are:

  • Assume they are skilled too at what they do. It may not be true, but it will help you tremendously. If you go into a meeting thinking they'll all be fools, you'll quickly find confirmation, and they'll see that you are not talking them seriously. Just assume they know what they are doing, and they may act like it. Honestly.
  • Get people to talk more, draw them out, work on usecases, or 'stories'
  • If they ask for something truly impossible, by all means don't explain them in gory detail why it is not possible. Definitely hold off on the "but you shouldn't be wanting that"
  • Non-technical people typically converse one level away from the ground truth. So, "impossible" becomes "exceedingly difficult, possibly a research project", which they will understand as "impossible". In fact, a straight "impossible" is understood as "you are a damn fool". In some circles this is so extreme that "We're taking your proposal seriously" means exactly not that. So, adjust your own verbiage. Avoid "impossible".
  • If you truly think everyone around you is an idiot, and this may be true, ponder working somewhere else. 

So, summarizing - the plight of the expert in the sketch is real. But the expert is also making things worse, and with proper technique could have prevented the alienation and having to succumb to making unrealistic promises.