This was originally published in October 2009

In this blog, I discuss the need to get to the root cause of why so many teams are not having success with Scrum.  Merely saying management is not removing impediments or labeling it “ScrumBut” does not give many indications as to what or where the problems are. The question is “why does this happen and what can we do about it?” I will suggest one of Lean’s problem-solving tools – “5-whys” – may assist Scrum teams in moving forward.

In an Agile Collab interview, Ken Schwaber acknowledged that only 1 in 4 Scrum implementations achieve the success desired by the companies trying it. He attributes this to the organizations accommodating the impediments the team faces instead of solving them. This number has been generally accepted by the Scrum community as accurate (as discussed at Scrum gatherings and on user groups). What surprises me isn’t the low rate of success but rather the general lack of interest a why it occurs. I’ve looked at both sides of this (see Challenging Why (not if) Scrum Works and Challenging Why (not if) Scrum Fails). My own belief is that one has to take a scientific approach and do an inquiry into why things work or don’t work. The intent of both blogs was to spark a conversation about how to improve Scrum.  Unfortunately, neither blog achieved this.

In the past, the generally accepted reason simply seemed to be people were not disciplined, educated or motivated enough to do Scrum properly.  More recently, the belief that developers don’t properly understand what they need to do has resulted in much discussion about having Certified Scrum Developer courses to solve this problems.  But I am afraid both of these are superficial answers.

The issue of teams doing a pseudo Scrum by cherry picking their practices has reached epidemic proportions. One hears of this so much that there has become a name for it “Scrum but.” That is, “we do Scrum but for this”.  However, why do people do this?  This is an important question.   In some situations this is probably a good thing.  After all, Scrum is not a prescriptive methodology but rather a lightweight framework. One would suppose that would mean that there aren’t any absolutes in Scrum, but I’ve heard several CSTs say if you aren’t doing daily stand-ups or retrospectives then you aren’t doing Scrum. The reticence of many Scrum CSTs to present a model of why and how Scrum works (many claim such a thought process is actually counter-productive) only exacerbates this situation. In other words, if one doesn’t have a set of principles to work from, one is left with little understanding of why practices work or not.

So what’s supposed to be in Scrum and what’s not supposed to be in Scrum and who is to decide? If one believes that practices are context dependent and that teams need to be self-organizing then one would expect the teams to decide what they should be doing. Tailoring Scrum should therefore be left to the team and an outsider shouldn’t pass judgment on the team’s efforts. However, since teams don’t understand Scrum well at the beginning of their adoption of it the advice is to require teams to follow a starting set of rules and to only tailor them after you understand Scrum. But why should we believe the starting set works in all situations?

Ironically, many Scrum thought leaders believe software is so complex there is no causality you can understand. In other words, you need to just solve the problems as they show up. In other words, after you’ve tried things and learned them, then you should do things in a way that works for your situation. While this makes sense, it seems to have led to the current situation that if it works, we’ll call it “Scrum” but if it doesn’t, well, we’ll call that “Scrum but.”

The dilemma of Scrum becomes evident. While we don’t want to be prescriptive, when you learn Scrum there are rules you need to follow.  Then, as you get better, you can tailor it to your needs.  The unexamined assumption is that you can always do this.  I do not think it is as simple as this. I believe we need a real analysis of why teams accommodate their problems or why teams don’t follow certain Scrum practices. In other words, let’s get to the root cause of Scrum not working or teams doing “Scrum but.”  To do this, let’s use a Lean tool called “5-whys.”

Five Whys

When something goes wrong, we often jump to conclusions about what caused it. Even when correct, we often stop there – instead of asking ourselves, “well, what caused that?” In other words, we don’t get to the root cause.  Thus, even if we fix this problem, if the root cause is still present, it’ll likely cause another problem.  A popular Lean tool to get to root cause is called “Five whys.”  It basically is an a technique where one keeps asking “why something happened” until the root cause of the original incident which set off the query is uncovered. This may take anywhere from 2-9 levels of questioning – so “5” should not be taken as the exact correct number of “whys” to ask.

Let’s use Five Whys to determine root cause for accommodating impediments and “Scrum but”.

The root cause of accommodating impediments

One of the common impediments teams face in corporate-wide Scrum implementations is that team members are often assigned to work on several projects. It is easy to dismiss this on the lack of commitment of management. If it’s available for them to solve it, there is a kind of implication (and insult) that they’re not up to it – either in intelligence or in commitment. But the truth is somewhat different when one looks at the issue.  The question “why don’t they solve the impediments?” and “what do they need to know to remove the impediments the team is facing?” are questions that should be asked, but rarely are.

Teams often run into impediments whose cause are outside the team. The teams therefore can’t solve them. But why doesn’t management? For example, many Scrum teams are composed of members who work on other things besides their main project. This causes thrashing an delays.

Why does this happen? Because management believes that one of their jobs is to make sure that everyone has productive work to do.

Why? Because management tends to look at people’s productivity in making management decisions.

Why? Because productivity is what they have been taught and used.

Why haven’t they been taught more useful agile methods? Shouldn’t this change in Agile? Not necessarily.

Why? Because many Scrum thought leaders believe you shouldn’t teach much, just create a framework.

Implicit assumptions that may not be correct

Two (unexamined) tenets of Scrum are impediments that slow the team down will be recognized as impediments and that the team is capable of removing them. Besides my earlier statement that impeded teams may not have the capacity to remove the impediment I will go further and state that teams may not even recognize all of their impediments as just that. Many things that slow a team down are often just viewed as the way it is. There is no notion that it is an impediment. For example, I don’t consider the fact that I have to take planes when I want to travel long distances an impediment – it’s just the way it is. Now in this case, I think my perception of reality is correct (but it would be cool if one day I woke up and found out I could fly on my own!).

The whys of “Scrum But”

Let’s take another example of applying 5-whys to a common “Scrum but” – retrospectives.

Why is the team not doing retrospectives?
Because they don’t see the value of the retrospectives.

Why don’t they see the value of the retrospectives?
Because they think there is more value in doing their work.

Why do they think there is more value in doing their work?
Because the retrospectives haven’t really done anything for them.

Why haven’t the retrospectives done anything for them?
Because they often don’t get agreement on what to do.

Why don’t they get agreement on what to do?
Because they think software development is about intelligent people using their judgment – that is, there are no real rules to use or a scientific method to apply.

Why do they think this?
Because software is so complex and no one has presented them with a theory to explain it.

BTW: This is one reason I talk a lot about the theory of flow underneath Lean-thinking. I have seen that it can provide a thought process that helps unite the team.

The dilemma (faulty thinking?) of Scrum

Scrum is in a dilemma here. On the one hand, Scrum is based on being a light framework that doesn’t give a model to work from. On the other, without such a model many Scrum practitioners don’t realize the importance of the practices or how to manifest the affect of the practices in their own context. It seems that the reaction has been to insist that certain practices be done (at least at the start) – which is ironic because Scrum is not intended to be prescriptive. If there isn’t a model, then how can one tell if one is doing Scrum? The common thinking is that you’re doing Scrum if you are removing your impediments, but you’re not doing Scrum if you are accommodating them. This, of course, assumes that Scrum can always work – a dangerous assumption, in my mind. It also makes it very difficult for new teams to know how well they are doing.

Getting beyond “Scrum But”

To get beyond “Scrum But” one needs to be able to understand when the practices of Scrum work and when they don’t.  I believe this requires both an understanding of why Scrum works and a scientific approach in how to apply these rules.  Unfortunately, I have not seen the Scrum community embrace this approach.