Why do (almost) all projects fail?

You might find the question in the title a little bit dishonest as it suggests that most projects fail. And in that sense you are right, it is a wrong a question. But it is also wrong for a less more obvious reason. The problem with the question is that it does not tell you what is meant by "fail". If I would ask you to define project failing I guess you would come up with something like "delivered above budget" or "not delivered on time". It is true that project performance most often is measured via these two basic KPIs (see for example my blog on IT projects within Government).

But let's not ignore the fact that many projects do indeed fail to deliver on budget and on time. When was the last time you were involved in a project that was either on time or on budget (let alone both)? So even when it is the essence of the Project Managers job to keep their projects within the GREEN, it almost never happens. In my opinion this is because we are measuring the wrong things. My (maybe bold) statement is that these two KPIs are useless to measure project performance. Of course it does say something about the progress of the project but not about the actual performance. Focussing on just these two KPIs is like the mouse that is staring in the two headlights of a car. It blinds you from the real "danger". So what are these "real" dangers that we should focus on when executing projects?

For starters, it is safe to assume that your budget estimate was wrong in the first place. We are masters in "short-term-predictions". That's what our brains are doing all day long. However when it comes to long term predictions we are just terrible. In general we are biased towards optimism (optimism bias), we ignore obvious warning signals (confirmation bias), take previous events out of context (context effect) and tend to remember things more positively (Egocentric bias).

Secondly, the actual risks that materialize during your project (endangering your delivery time) are not the ones that you summed beforehand. The ones that you can think of beforehand, are the ones that were probably copied from the previous project starting documents and are most often already taken into account when defining the time window of the project. The real issue was already addressed in the blog "When KPIs fail". It's the problem of Black Swans, who are always unexpected but impactful.

So using time and budget as your KPIs is a recipy for failure. Unfortunately this is not without far-reaching consequences. Especially when the GREEN status of these KPIs

become the goal. Requirements are de-scoped or time goes before quality.

But apart from these hidden sides regarding budget and time there are more dangers that lurk in the dark (if not measured properly). When focussing on budget and time we tend to forget that the real "performance" of the project is measured by the quality of the thing it is implementing. How often is a Business Case drafted in the beginning of the project and never checked during the project? Or when it is checked it is altered to fit the new timelines and budget. Even the Business Case itself is most often as "light as a feather", presenting three "scenarios" to choose from: Doing Nothing, Doing Everything, Doing the Halfway Solution.  Furthermore most Business Cases do not take into account things like it-debt, increased complexity, maintenance costs, imbedding in Business As Usual, governance aspects, etc.

Projects implement change and change has an effect on people. All sorts of behavioural effects can take place (both inside and outside the project) that have an impact on the project results. Don't underestimate the behavioural effect. Coping with change is one of the hardest things to do for everybody. People can (directly or indirectly) sabotage the projects. Early adapters might lose interest (and the project loses a sponsor). Quality might go down when people feel the pressure to deliver. People within or outside the project do not believe in the change (even when you have a communication professional). People might mistrust the external people you hired. And so forth. But most importantly, people won't admit that they were wrong and keep on doing what they were doing, believing and assuming it is the right thing. This is especially true for sponsor, project manager, and project members as they have invested the most. Of course people will deny all of the above when you ask them.

So next time you do a project. Please make sure your KPIs are set on measuring the Business Case on a frequent basis and not only listen to the people involved, but observe what they actually do. And be brave: dare to stop projects.