The Five Whys of Lean as an Answer to the “But” of Scrum

This was originally published in October 2009

In this blog I discuss the need to get to the root cause of why so many teams are not having success with Scrum.  Merely saying management is not removing impediments or labeling it “Scrum but” does not give much indications as to what or where the problems are. The question is “why does this happen and what can we do about it?” I will suggest one of Lean’s problem solving tools – “5-whys” – may assist Scrum teams in moving forward.

In an Agile Collab interview, Ken Schwaber acknowledged that only 1 in 4 Scrum implementations achieve the success desired by the companies trying it. He attributes this to the organizations accommodating the impediments the team faces instead of solving them. This number has been generally accepted by the Scrum community as accurate (as discussed at Scrum gatherings and on user groups). What surprises me isn’t the low rate of success but rather the general lack of interest into why it occurs. I’ve looked at both sides of this (see Challenging Why (not if) Scrum Works and Challenging Why (not if) Scrum Fails). My own belief is that one has to take a scientific approach and do an inquiry into why things work or don’t work. The intent of both blogs was to spark a conversation about how to improve Scrum.  Unfortunately, neither blog achieved this.

In the past, the generally accepted reason simply seemed to be people were not disciplined, educated or motivated enough to do Scrum properly.  More recently, the belief that developers don’t properly understand what they need to do has resulted in much discussion about having Certified Scrum Developer courses to solve this problems.  But I am afraid both of these are superficial answers.

The issue of teams doing a pseudo Scrum by cherry picking their practices has reached epidemic proportions. One hears of this so much that there has become a name for it “Scrum but.” That is, “we do Scrum but for this”.  However, why do people do this?  This is an important question.   In some situations this is probably a good thing.  After all, Scrum is not a prescriptive methodology but rather a lightweight framework. One would suppose that would mean that there aren’t any absolutes in Scrum, but I’ve heard several CSTs say if you aren’t doing daily stand-ups or retrospectives then you aren’t doing Scrum. The reticence of many Scrum CSTs to present a model of why and how Scrum works (many claim such a thought process is actually counter-productive) only exacerbates this situation. In other words, if one doesn’t have a set of principles to work from, one is left with little understanding of why practices work or not.

So what’s supposed to be in Scrum and what’s not supposed to be in Scrum and who is to decide? If one believes that practices are context dependent and that teams need to be self-organizing then one would expect the teams to decide what they should be doing. Tailoring Scrum should therefore be left to the team and an outsider shouldn’t pass judgment on the team’s efforts. However, since teams don’t understand Scrum well at the beginning of their adoption of it the advice is to require teams to follow a starting set of rules and to only tailor them after you understand Scrum. But why should we believe the starting set works in all situations?

Ironically, many Scrum thought leaders believe software is so complex there is no causality you can understand. In other words, you need to just solve the problems as they show up. In other words, after you’ve tried things and learned them, then you should do things in a way that works for your situation. While this makes sense, it seems to have led to the current situation that if it works, we’ll call it “Scrum” but if it doesn’t, well, we’ll call that “Scrum but.”

The dilemma of Scrum becomes evident. While we don’t want to be prescriptive, when you learn Scrum there are rules you need to follow.  Then, as you get better, you can tailor it to your needs.  The unexamined assumption is that you can always do this.  I do not think it is as simple as this. I believe we need a real analysis of why teams accommodate their problems or why teams don’t follow certain Scrum practices. In other words, let’s get to the root cause of Scrum not working or teams doing “Scrum but.”  To do this, let’s use a Lean tool called “5-whys.”

Five Whys

When something goes wrong, we often jump to conclusions about what caused it. Even when correct, we often stop there – instead of asking ourselves, “well, what caused that?” In other words, we don’t get to the root cause.  Thus, even if we fix this problem, if the root cause is still present, it’ll likely cause another problem.  A popular Lean tool to get to root cause is called “Five whys.”  It basically is an a technique where one keeps asking “why something happened” until the root cause of the original incident which set off the query is uncovered. This may take anywhere from 2-9 levels of questioning – so “5” should not be taken as the exact correct number of “whys” to ask.

Let’s use Five Whys to determine root cause for accommodating impediments and “Scrum but”.

The root cause of accommodating impediments

One of the common impediments teams face in corporate-wide Scrum implementations is that team members are often assigned to work on several projects. It is easy to dismiss this on the lack of commitment of management. If it’s available for them to solve it, there is a kind of implication (and insult) that they’re not up to it – either in intelligence or in commitment. But the truth is somewhat different when one looks at the issue.  The question “why don’t they solve the impediments?” and “what do they need to know to remove the impediments the team is facing?” are questions that should be asked, but rarely are.

Teams often run into impediments whose cause are outside the team. The teams therefore can’t solve them. But why doesn’t management? For example, many Scrum teams are composed of members who work on other things besides their main project. This causes thrashing an delays.

Why does this happen? Because management believes that one of their jobs is to make sure that everyone has productive work to do.

Why? Because management tends to look at people’s productivity in making management decisions.

Why? Because productivity is what they have been taught and used.

Why haven’t they been taught more useful agile methods? Shouldn’t this change in Agile? Not necessarily.

Why? Because many Scrum thought leaders believe you shouldn’t teach much, just create a framework.

Implicit assumptions that may not be correct

Two (unexamined) tenets of Scrum are impediments that slow the team down will be recognized as impediments and that the team is capable of removing them. Besides my earlier statement that impeded teams may not have the capacity to remove the impediment I will go further and state that teams may not even recognize all of their impediments as just that. Many things that slow a team down are often just viewed as the way it is. There is no notion that it is an impediment. For example, I don’t consider the fact that I have to take planes when I want to travel long distances an impediment – it’s just the way it is. Now in this case, I think my perception of reality is correct (but it would be cool if one day I woke up and found out I could fly on my own!).

The whys of “Scrum But”

Let’s take another example of applying 5-whys to a common “Scrum but” – retrospectives.

Why is the team not doing retrospectives?
Because they don’t see the value of the retrospectives.

Why don’t they see the value of the retrospectives?
Because they think there is more value in doing their work.

Why do they think there is more value in doing their work?
Because the retrospectives haven’t really done anything for them.

Why haven’t the retrospectives done anything for them?
Because they often don’t get agreement on what to do.

Why don’t they get agreement on what to do?
Because they think software development is about intelligent people using their judgment – that is, there are no real rules to use or a scientific method to apply.

Why do they think this?
Because software is so complex and no one has presented them with a theory to explain it.

BTW: This is one reason I talk a lot about the theory of flow underneath Lean-thinking. I have seen that it can provide a thought process that helps unite the team.

The dilemma (faulty thinking?) of Scrum

Scrum is in a dilemma here. On the one hand, Scrum is based on being a light framework that doesn’t give a model to work from. On the other, without such a model many Scrum practitioners don’t realize the importance of the practices or how to manifest the affect of the practices in their own context. It seems that the reaction has been to insist that certain practices be done (at least at the start) – which is ironic because Scrum is not intended to be prescriptive. If there isn’t a model, then how can one tell if one is doing Scrum? The common thinking is that you’re doing Scrum if you are removing your impediments, but you’re not doing Scrum if you are accommodating them. This, of course, assumes that Scrum can always work – a dangerous assumption, in my mind. It also makes it very difficult for new teams to know how well they are doing.

Getting beyond “Scrum But”

To get beyond “Scrum But” one needs to be able to understand when the practices of Scrum work and when they don’t.  I believe this requires both an understanding of why Scrum works and a scientific approach in how to apply these rules.  Unfortunately, I have not seen the Scrum community embrace this approach.

 

One important aspect of systems-thinking that has long been ignored by the Agile community is that a system is not its components. Rather, it must be recognized that these interact in a way that creates the behavior.

But the nature of these interactions are very complex – meaning that they can’t be predicted. Unforeseen events occur, unexpected interactions and sometimes small, even obscure events happen that cause huge side effects. This is a reflection that product development (creating the unknown) by a group of human beings (by nature unknowable) comprise what is known as a complex system.

Many people have gotten caught up in the theory of complexity going further than what I need is necessary for pragmatic effect. While it is true that complex systems can’t be predicted, there are many patterns of behavior exhibited by them. In the same way engineers were quite effective in building magnificent edifices (e.g., the Pyramids) without a full understanding of the science underneath the methods used, it is possible to adjust the behavior of complex systems without understanding the full nature of the principles involved.

We mostly need to know that 1) our changes may produce unexpected behavior and 2) we are embedded in a system where small errors can cause big, undesirable affects (the essence of Chaos Theory). However, instead of giving up and saying we can’t predict things, we can move forward with an knowing our understanding is always incomplete. We do, of course, need quick feedback, both of our actions in our work and in any actions we take towards its improvement.

Both agility of development and improvement of our methods is required.

 

Systems thinking and complexity

An excerpt from the preface of my book.

One important aspect of systems-thinking is that a system is not its components but is defined by the way these interact in a way that creates the behavior.

But the nature of these interactions are very complex – meaning that they can’t be predicted. Unforeseen events & interactions occur. And sometimes small mis-understandings cause huge side effects. This is a reflection that product development is both complex (unknowable in advance) and chaotic (small things can have big effects).

While I believe a deep understanding of complexity can be useful, very little is needed to attend to it. In the same way engineers were quite effective in building magnificent edifices (e.g., the Pyramids) without a full understanding of the science underneath them, it is possible to adjust the behavior of complex systems without understanding the exact results of proposed changes.

We mostly need is to know that 1) our changes may produce unexpected behavior and 2) we are embedded in a system where small errors can cause big changes. This requires quick feedback both about the actions we take and process improvements we try. Both agility of development and improvement of our methods is required.

Impediments to flow

Flow-thinking is my latest thing. It’s looking at how to achieve value in the quickest way. This is not necessarily going faster – but mostly avoiding delays. I have observed that when I consult with clients I look for the following:

  1. hand offs (these delay value because they typically cause a degradation in understanding)
  2. delays in workflow (i.e., waiting for someone)
  3. interruptions to workflow (often caused by working on too many things)
  4. delays in feedback – a symptom is when work goes upstream (e.g., dev back to product owners)
  5. doing less valuable work than what could be worked on
  6. doing work manually that could be automated

As you minimize these you almost certainly will improve your value delivery. Quality typically goes up as well.

Question to ask when you hear something you know is wrong

When we hear something that we know is wrong we typically ignore it or argue with it. But there is often truth in something we don’t yet understand. One can’t tell at this point of course. It could be wrong. But it could hold some value if we’d explore a little further.

I have found it valuable to ask the question:

“what if this (absurd) statement is true?”

A good way to explore this question is to pretend the statement _is_ true – at least for a little while. This opens the door to many other questions such as “what am I not seeing?” “what am I looking at that is in my way?” “what does this person see that I don’t?” “what are they not looking at that I am looking at?”

You can even (for a limited time) act as if it is true and see what happens. Maybe you’ll learn something. Worst case is you’ll learn you were right (or still think so).

It’s not GETTING Agile that takes the time, it’s the NOT getting it

I had a mentor a long time ago who would say “it’s not getting it that takes the time, it’s the NOT getting it that takes the time.” I have seen this over and over again. I remember when I was learning design patterns 20+ years ago I spent 6 months NOT getting it. Then, in one 15 minute (imaginary) conversation with Chris Alexander I had the epiphany that led to my truly understanding what patterns were (they are well beyond “solutions to a recurring problem in a context” and was the basis for Jim Trott and my Design Patterns Explained book).

6 months NOT getting it.

15 minutes GETTING it.

Most of the 6 MONTHS of NOT getting it was studying and thinking about patterns. The 15 minutes of GETTING it was preceded by 6 HOURS of working on my problems.

The insight was not based on information but came from a slight mind-shift.

Maybe we need to learn by doing and not by being in a course where we are not working on our own tasks. This is the basis of scaled, flipped classroom learning.

Agile Developer Habits 101 – Focus on Finishing to Manage Work in Process

Kanban suggests we manage work in process (WIP). Many people interpret this to mean add WIP limits on queues. But that’s not the only way to do it. And it often takes a very disciplined team to do that.

An easy way to start managing WIP is to simply build a habit of looking to finish something before starting something new.

Habit – manage Work in Process

Trigger: When you complete a task.

Action: Look to see if there’s another task that’s already been started that you can finish. If it’s yours, just finish it. If it’s someone else’s see how you can help them.

Intention: Avoid increasing WIP above capacity without having to put a lot of effort on it while also creating opportunities for cross-learning and collaboration.

Announcing On the Job, Online Training for Scrum & Kanban Coaches

The best way for people to learn is on the job over time. The worst is in a 2-4 day course with where you learn information about something but don’t get a chance to try it. Studies have shown that 90% of what was learned in a classroom is forgotten in less than a week.

5 years ago I asked myself – “what would I do if a company asked me to train their thousands of Agile coaches who needed to become more proficient?”

We think we’ve come up with the best way to deliver on this. It also turns out to also be the most economical. The format we use is called “flipped classroom”, a method developed at Harvard. Essentially each lesson starts with a reading and video. An assignment is then given that participants work on with their team. Two live sessions are also provided so participants can ask questions before and after their assignment. Participants learn by doing while having the support they need. In addition, participants are provided access to an online support system that covers virtually all of the common challenges faced by teams. In this way participants can learn both the curriculum they need and get help on their own specific problems. The next program starts 1/28. Contact me if interested.

Elephants in the Room and No Clothes on the Emperor

There are many things I, and many other consultants believe, that are not politically correct to talk about. I am going to talk about them now. The reason is if you are a practitioner and these resonate with you, then you might want to consider working with people like me to help you get better. If you don’t want to work with me, then use this as a checklist to vet your potential providers.

Both coaching & training is more expensive than they need to be. Much of this happens because the certifying bodies control the number of trainers by quantity. Many of the top trainers in the industry have no certification by choice.

The biggest problem with the Scrum-Kanban confrontation is that neither are the right thing to do all of the time. Usually some combination of the two is best.

Intense training, except where people do hands on work is effective. True learning takes place over time in the learner’s workplace.

People need a support system that can be used to provide them insights into their own problems. Reinventing the wheel should not be required.

AFAIK our Advanced Scrum Master / Kanban Online Workshop is the only workshop that fits the above. It’s worth checking out.

How I define Lean-Scrum (how Net Objectives does Scrum)

I mostly love Scrum, but I am concerned with that it:
1) doesn’t work everywhere but is promoted as if it does
2) sets up a sub-conscious resistance to go beyond it as because the case where “We do Scrum, but we do X instead of Y” is virtually always considered negative whether it is or isn’t. Also, when you are certified in Scrum and go beyond it the value of your certification is lessened

Lean-Scrum differs from Scrum in the following ways:
1) Scrum practices & roles are considered to be ways of achieving particular intentions
2) The team needs to fit into the context of the organization
3) Any part of Scrum can be modified as long its intention is still retained. This may have you no longer be doing Scrum but you should still be doing some form of team-agility
4) It considers Scrum as a tool in a toolbox with other tools being needed (e.g., Kanban)
5) Scrum mostly works iterations & cross-functional teams are advisable
6) We don’t really care if we’re doing Scrum or not as long as we’re consistent with Lean principles
Lean-Scrum is an approach where use take what’s useful and leave what isn’t.

Be clear, Lean-Scrum is not Scrum. But my 19 years of doing & helping others do Scrum has demonstrated it is more effective