|
|
(Fr)Agile Software Development. Ha ha!
Detractors of Agile Software Development sometimes use the term Fragile Software Development. It’s an easy pun, you don’t have to think too hard about it. And then you can dismiss the whole idea of agile software development without a further thought.
This accusation is mostly aimed at Extreme Programming. XP has a set of interlocking practices that reinforce and balance each other. Take one away and the whole house of cards comes crumbling down.
XP is fragile. And that’s how it’s supposed to be.
Huh? Agile: good! Fragile: bad! Surely?
What if that fragility contributes to XP’s agility?
Let’s look at another fragile method and all the ways it can break: Lean. Let’s count the ways we can shut down a Lean plant:
- Just in time delivery: if one delivery from one supplier is late, what little inventory you have runs out very quickly.
- Everyone can stop the line: if someone discovers an error, the whole line is shutdown while the cause of the error is removed (actually, the line shuts down incrementally: first one cell, then a group of cells, then the line, depending on how long it takes to fix the problem).
- Pull: there are very small buffers of work in progress between cells, regulated by Kanban cards. If one cell fails to replenish the buffer, the other cells quickly shut down due to lack of input. Ideally, you have “one piece flow”, no buffers!
- Small batches: each production run produces only small batches, which are consumed quickly. If the process that produces the batches breaks down, the consumers quickly break down due to lack of resources.
And yet… every metric and statistic available clearly indicates that Lean plants are more efficient and production lines break down less often than “traditional” plants with lots of safety margin.
Making the system more fragile: lowering the water level to expose the rocks
In between cells or parts of the production line, there are small buffers of unfinished work (see the “Drum-buffer-rope” entry) that protect a consumer from variations in output of its producers. The Toyota Way describes how the size of these buffers is determined: by using the lower the water level to expose the rocks technique.
Whenever a process runs smoothly, the size of the buffer is reduced. If the system keeps working smoothly, the buffer is reduced again. This continues until the system breaks down. The buffer is enlarged to the previous value and the root cause of the system breakdown is researched. When that problem has been solved, the buffer is reduced again.
 The metaphor that gave this technique its name goes like this: imagine you’re sailing a boat. Below the water surface, there are rocks, but you don’t know where they are. If the water level is high enough, you’ll never hit any rocks. If you gradually lower the water level, you will hit the highest rock. Now you can remove that rock and you can sail freely again. To hit (and find) more rocks you lower the water level again.
Fragility keeps you on your toes
How would you act if a mistake could shut the whole plant down? You would make damn sure that everything is in place to avoid mistakes or to fix them very quickly. You’d try to find and fix root causes of problems. You would constantly look to improve and refine your processes. You would actively search out problems. That is… IF you worked in an extraordinary, learning organisation.
If you work in the average company, you would be scared to do anything for fear of being blamed (and fired). That’s one of the reasons that the Toyota Way says “Whatever the problem, it’s never a people problem, it’s always a process problem”
A while ago, I had a discussion about how to handle a daily build failure. I advocated letting the whole team work on fixing the build problem, before they started to work on new features. Someone else proposed to let one developer fix the build, so that the others could get on with their work. He thought I was a bit excessive, extreme even. Now why would I want to waste good developer’s time? Maybe to make our process a little more fragile, to make the cost of a build break a bit higher, so that we would think that little bit harder about how to avoid the root causes of broken builds.
XP is fragile. XP is Lean
XP has even more ways to break down than a Lean plant. Let’s list a few:
- What if the customer can’t supply stories fast enough (just in time)?
- What if your story cards get lost/blown away/stolen? (I get this question a lot)
- What if your tests don’t catch all the errors?
- What if the build breaks a lot?
- What if developers don’t sign up for stories/tasks?
- What if… (your favourite method of breaking down here)
What keeps XP projects alive?
- A team with a common vision and open communication.
- Discipline to practice all the agreed practices, not just the easy ones. Even when there’s pressure. Especially then.
- A blame-free environment, so that you can solve problems and learn.
- Reflecting on what you do.
- Constantly looking for ways to improve.
- Being open-minded to accept solutions when you find them, even if that means you’re not doing XP by the book. Especially then.
And that’s just for starters. Who knows where you would end up if you kept doing that? Just imagine… How fragile can you make your process?
Tags: agile, theory of constraints, lean, Toyota Way
Thinking for a Change at SPA 2006
 Marc Evers and I will host a session on the Theory of Constraints’ “Thinking Processes” at the SPA 2006 conference.
 The session is, not coincidentally, called “Thinking for a Change”, after the Thinking Processes book by Lisa Scheinkopf, that introduced me to the subject.
We will introduce “current reality trees”, “transition trees” and “future reality trees” by applying them to real problems that participants bring to the session. We will be “learning by doing” by taking small steps: explain part of the technique, apply the technique, reflect, explain another part…
We don’t have the time to deal with the “Evaporating Cloud” technique during the session, but we will run a “birds of a feather” session about it. The previous entry contains an example of the use of the evaporating cloud. Expect more examples of the techniques in upcoming entries.
The conference takes place in St Neots, Bedfordshire, from 26 to 29 March.
Some of the interesting sessions of the conference:
See you there!
Tags: SPA2006, theory of constraints, thinking processes
 It’s always a people problem (Gerald Weinberg)
In “The Secrets of Consulting“, Gerald Weinberg tells us that Whatever the problem is, it’s always a people problem.
All too often, problems are labeled as “Technology problems”, to hide the real problem and avoid blame. This strategy is very effective in a field like IT, where there is lots of technology churn. However, you can always trace even these problems back to people: who selected the technology and why, do people know how to use the technology, were the users of the technology consulted…
 It’s always a process problem (The Toyota Way)
The Toyota Way states that Whatever the problem is, it’s always a process problem.
Not surprising, coming from a company which has one of its principles: “The right process will deliver the right results.”
An example situation: Mike the micromanager
Imagine an all too common situation: someone who micro-manages their team. As the number of interventions rises, so does the disruption to team member’s flow. This reduces the team’s productivity. When the lower results become visible, this increases the need the manager feels to intervene, which leads to more interventions. And the vicious circle goes on an on…
Micromanagement as a people problem
What questions would Gerald Weinberg ask?
- Where does this need to intervene come from?
- Is the manager aware of the effects of his actions? Could we clarify this by using a Systems Thinking approach as above?
- Can we improve the way the manager and the team communicate? E.g. the manager could “batch” his requests of the team; the team could provide better visibility into the state of their work, so that the manager need not fear that “things are getting out of control”.
- How do the team members react to the interventions? Are they acting incongruently themselves?
Micromanagement as a process problem
What questions would Toyota ask?
- What is wrong with our hiring or promotion process, that put this manager in a situation he wasn’t able to handle?
- What is wrong with our training and coaching process, that we left this manager in a difficult situation without any help?
- What is wrong with our Genchi Genbutsu (go see on the workfloor) process, that this manager’s manager didn’t go and see how the team was, so that he could see the problem?
- What is wrong with our Hourensou (report, inform, consult) process, that this manager didn’t inform his manager of the problem and consult him for ways to solve the problem?
- What is wrong with our evaluation and “Stop the line to fix mistakes”, that nobody stopped to report the problem and find its cause?
People or Process? Evaporating the cloud
We clearly have a conflict: either every problem is a people problem or it’s a process problem. The “Thinking Processes” provides us with the “Evaporating Cloud” technique to examine and resolve such conflicts.
How do you read this diagram?
- In order to have “An effective organisation” (goal) we need “Effective people” (requirement 1), because the effectiveness of the people determines the qualities of the result (assumption 1)
- In order to have “An effective organisation” (goal) we also need “Effective processes” (requirement 2), because effective processes enable people to work well together (assumption 2).
- In order to have “Effective people” (requirement 1), we need to “Treat every problem as a people problem” (prerequisite 1), because this will help us focus on solving the fundamental people problems and not get sidetracked by superficial symptoms (assumption 3).
- In order to have “Effective processes” (requirement 2), we need to “Treat every problem as a process problem” (prerequisite 2), because this will help us focus on continuously improving our processes and thereby making our solutions available to the whole organisation (assumption 4).
- There is a conflict between D1 and D2: either all problems are people problems OR all problems are process problems.
How can we resolve the conflict?
- Are the relationships between goal and requirements valid? I think so. I believe we need a combination of effective people and processes to succeed. Take away one of them and you can’t have an effective organisation (if you have effective people, they will probably grow a process anyway).
- Are the relationships between prerequisites and requirements valid? I believe the only way to be effective is to be continually be solving problems, be they people or process problems.
- Are the assumptions correct? I believe so, I can’t find any counter-examples for the moment.
- Why can’t D1 and D2 co-exist? Are they really mutually exclusive? What’s the assumption behind this? Hmmmmmm. Let me think…
Imagine that both statements are true: Every problem is a people problem AND a process problem. Would this assumption lead to a contradiction? Let’s see what we would do in Mike’s case:
- First, we could call in Jerry Weinberg and resolve this manager and team’s issues, so that this team can work effectively.
- Then, we could call in a Toyota sensei to question the processes that led up to this problem and failed to detect and correct it. They would help us to improve the processes, so that this kind of situation could be avoided in the future, so that in the future, all teams can work effectively.
I believe this resolves the conflict: every problem is a people problem that must be solved now AND an underlying process problem that must be solved, so that this problem doesn’t reoccur in the future. That means we have to clarify our goal: “To have an effective organisation, now and in the future“. With these changes, the diagram looks like this:
Reconciling people and process, present and future
Both types of problem solving have their place. We need to deal with the problems that people are having now. We need to continuously improve our processes so that these problems won’t reappear for other teams in the future.
An analogy I like is that improvement needs to be like a ratchet: relentless reflection and continuous improvement (Hansei & Kaizen) drive the gear wheel forward; processes encode our growing and changing knowledge to act as the “click” that prevents the wheel from turning back.
Tags: Systems Thinking, theory of constraints, thinking processes, Toyota Way
A few gems from the Toyota Way Fieldbook on organisational change:
“…we’re more likely to change what people think by changing what they do, rather than changing what people do by changing what they think.”
“Similarly, changing culture is not going to happen because of a classroom education process. We can teach people what is politically correct to say and sophisticated ways of saying it, but not affect deeply held values and assumptions. This is the unfortunate truth, though it might seem a lot easier to change culture en masse through an educational program than to have to remake the structure and processes of organizations in order to begin to change what people think. But Lean is not about doing what is easy. It is about doing what works. It is about confronting reality and having the confidence that we can shape the reality to achieve our goals.“
Team organisation: horizontal vs vertical. Or is it?
How is your team organised ?
In my travels through IT-land I’ve seen basically two ways of people cooperating in a team or teams cooperating: the vertical and the horizontal organisation. In the vertical organisation, one person (or team) is responsible for one functionality, from start to finish. In the horizontal organisation, one person (or team) is responsible for a common layer of several functionalities and you need each one of them to complete a functionality. As a result of Conway’s Law, this organisation is reflected in the architecture of the systems.
Each of these ways of organizing has distinct advantages and drawbacks. Many organisations go through regular flip-flops, switching from vertical to horizontal and back, only to trade one set of drawbacks for another.
Flip: vertical
 The advantage of the vertical model (the integrated team) is that they have their eye firmly on the goal: delivering a useful system for the customer.
We have all seen the disadvantages: silo systems. Systems that are not integrated, have trouble exchanging information, have nothing in common, no reuse, enormous amounts of duplication. A too narrow focus on the local goal (the current system), without looking at the global goal (all systems that support the business).
All good news voor EAI consultants and vendors…
Flop: horizontal
The advantage of the horizontal model is that the people in each layer are very knowledgeable in their area and can focus on one specific area. There’s natural reuse as all applications use common layers.
The big disadvantage is that it’s hard to see the goal of the whole system. Each layer starts to define their own local goal. The local goal can be misaligned with the global goal, or even go against the global goal. Sometimes, the means and the goal get confused: framework authors try to increase the amount of framework code that is used in applications, whether the framework helps the application writers or not; methodologists start to measure conformance to process or documents instead of results; DBAs try to increase database stability by denying all change, at the expense of the applications using the database…
Beyond the flipflop
Of course, most (larger) organisations are not organized along such clear lines: there’s a mixture of horizontal and vertical teams. In these cases there’s often a tension between the two types of groups.
 Thinking Beyond Lean by Michael Cusumano and Kentaro Nobeoka describes a refined and pragmatic organisation with a combination of specialists vs generalists, reuse vs uniqueness, integrated vs separated in use at Toyota since a few years. See the previous entry for a more extensive description.
If I can choose, I prefer to start with a vertical organisation, because they have a clear idea of the goal. Then factor out commonalities. Get the different teams, one by one, to communicate with each other, to find commonalities and ways to share useful resources.
With a horizontal organisation, it’s important to make the global goal clear again for everybody involved. One effective way to do this is to create a “virtual crossfunctional team” for the duration of the project. Get the layers to communicate with each other, preferably using a “pull” model, where the layers closest to the customer request services from the layers further away.
Been there, done that!
With the exception of one case, I’ve always worked in “vertically” oriented, customer and delivery focused groups. In the one horizontal case, the goals of the group actually ended up working against the global goal.
Tomorrow, I start to work for a “horizontal” team. More stories will surely follow.
Note to self: keep your eye on the goal!
|
|
