Thinking for a Change

Backup soldiers

I just don’t seem to learn, do I?

High availability soldiers

As described in the previous entry, the CSM had to schedule soldiers to perform all sorts of tasks. What happens when someone isn’t available to perform a task? That would create chaos and we can’t have that, can we? The solution: assign two soldiers to each task: the primary and the backup. If the primary isn’t available, the backup performs the task. That should increase the availability of the soldier resources.

Of course, this made the planning job even more difficult for the CSMs: they had to schedule double the number of soldiers, with even more potential for scheduling conflicts.

Did it work?

Not really. There were lots of primary/backup scheduling conflicts, which made the plans even more slippery. But having double the number of required soldiers assigned should have made it easier to form complete work-teams each day, shouldn’t it? No, because of a very annoying dynamic.

Soldiers had a tendency to become ill or otherwise unavailable on those days that they were assigned as primary. That meant their backup had to perform the job, but who cares? They didn’t know Joe Random Soldier who’d been assigned as their backup. It’s everyone for themselves here!

Of course, when a backup had to take on an extra job, this would have to be compensated. That meant even more schedule churn. And soldiers started to get ill when they were backups too. Imagine the chaos as the officer on duty tries to form complete teams, with all the people on his duty roster calling in sick.

A simple solution: pair soldiering

We faced the same issues when we made our own schedules. We solved it by grouping the soldiers in pairs. Each soldier paired up with someone with whom they’d been in basic training. Whenever one of them was primary, the other was backup. And vice versa.

When it came to choosing tasks, the pair chose together. This way, they could avoid primary/backup schedule clashes. We formed pairs out of soldiers who had been through basic training together because:

• they had the same seniority and thus the same priority in choosing tasks
• we reasoned that you wouldn’t deliberately welsh on someone you knew and worked with every day. If a backup ever had to replace a primary, it was easy to compensate by switching roles on the next task (of the same value). If one of the pair defaulted on their duty, the other could easily retaliate.

How did that work?

Very well. Scheduling wasn’t difficult at all: each pair just had to make sure they didn’t introduce any conflicts for the both of them. Primaries never got sick (except for one case in 1 year), so backups never had to fill in. Whenever there was a problem, we could simply “swap” tasks or roles, with minimal disruption to the schedule.

The “pair with someone from basic training” rule was useful for scheduling, but not necessary as a deterrent. With tasks swaps (see the previous story), you got other soldiers (from the same division) as backup anyway. That’s not a problem, as long as it’s someone you know and work with (and thus trust).

And what have we learned from this?

Back in the real world, there were no backups. Everyone worked on the modules they had been assigned. Some people got very possessive of their code: if the owner wasn’t available (or if they were in a bad mood), the module didn’t get changed, no matter how badly you needed the change. No-one else was allowed to (or could) change these modules. And at the end of the year, you’d get a performance appraisal for your work.

In one case (I’ll tell the full story later), I was leaving this company. I wanted to tell someone about the last project I’d worked on, but nobody wanted to listen. Finally, someone told me why: “If ‘they’ know I understand that module, they’ll assign it to me. And I’ll be stuck maintaining it, for the rest of my life!”

And things were back to normal: work had to wait to be completed, because the one programmer who knew some module was unavailable. No two modules worked alike; each of them reflected the eccentricities of their (sometimes many) authors. When people left, they took all their knowledge with them. Taking over someone else’s module felt like archaeology, even when the module was documented… That’s how it’s supposed to be, isn’t it?

How many points for a soldier?

I just don’t seem to learn, do I?

You’re in the army now

After my studies I had to perform a compulsory military service. What does a soldier do all day in peace time? Apart from the “job” you’re supposed to do (in my case, working at the data processing center), you’re mostly performing boring, menial tasks like cleaning the barracks, pealing spuds, washing dishes, guarding the base.

Now there are lots of soldiers and lots of tasks. How do you allocate who does what, when? This was the job of the CSM (Company Sergeant-Major). Each division’s CSM had to make a monthly plan to allocate the tasks to their division’s soldiers. Combine all the division’s plans and you’ve got the work plan for the whole base.

Apart from the usual constraints (one can’t do two tasks at the same time), there were constraints on the number of tasks you could be assigned each week. There were two types of tasks: “work duty” and “guard duty”. After being on guard for 24h, you got the next day off to get some sleep. Which introduces another constraint. And you want the allocation of work to be reasonable fair. Not so easy…

A good plan is hard to find

How did that work? It didn’t.

Nobody thought the system was fair; it was always easy to find other soldiers who had considerably fewer tasks than you had. The plan changed daily as constraint violations were fixed, people complained, tasks got switched. You never knew if you were going to be assigned to perform some task the next day or not. There were always several different versions of the plan; nobody really knew which one was the “correct” one.

The result: total chaos, tasks not being performed, bad morale, poor discipline. And lots of stress for the CSMs who had to manage this chaos. And what’s the standard CSM way of solving problems? Shouting. Very loudly. It doesn’t help, but at least they feel as if they’ve done everything they could…

A simple plan

Most of us in the army data processing center were engineers or computer scientists. How could we solve the scheduling problem? With scheduling software? No, in our experience software only made things worse. We needed a simple solution™! We made our CSM an offer he couldn’t refuse: we would fill in the work assignments for him.

We started by giving each task a number of points. The more we disliked doing that task, the more points. This depended on the type of the task and the day. E.g. a task on a weekday cost less than one in the weekend. The easiest task got 2 points; the hardest task got 7 points. From then on, it was easy: we played the “Duty Game” each month.

The Duty Game, round 1

Each month we received a duty roster to fill in. The soldier with the best maths skills assigned the points to each task and added them up. Now we knew “how many task points we had to do” this month. Our math wizard then divided this number of points by the number of soldiers in our division. Now we knew how many task points each of us had to do this month.

Then, each of us played the first round of the game, with the following rules:

• you have to put your name on as many tasks as needed to equal or exceed the required number of points.
• you may not violate a scheduling constraint

Everyone got to choose in turn. Old-timers first, rookies last. With people rotating in and out, you’d get closer to the head of the queue each month.

The Duty Game, round 2

Of course, the guys who had to choose last had a very limited choice. They might be forced to violate a constraint or pick an inconvenient date. To solve this problem, we had a second iteration. In this round, you could swap tasks with another soldier, with the following rules:

• you could only swap tasks if their number of points was (approximately) equal, irrespective of the number of tasks. E.g. I could swap a 5 point task for a 3 point task + a 2 point task.
• you may not introduce schedule constraint violations

This second round quickly and easily solved any problems in the schedule. We didn’t have a backup plan in case we couldn’t solve issues, but we never needed one. We were always able to solve the problems to everyone’s satisfaction.

Did it work?

We always filled in the plan quickly, within the time limits set by the CSM. The schedule was fair and took into account each soldier’s preferences and personal constraints. We produced a conflict-free schedule with a minimum of effort. Our monthly schedules stayed mostly constant, we (almost) never needed to change them. If we needed to change something, we would swap tasks (with the same rules as in the second round), so that only the two people involved in the swap were affected.

And what have we learned from this?

After a year, I was released back into the “real world”. I started to work as a programmer. How did I know what to work on? Well, there were project managers who would make plans for groups of developers. Lots of programmers, lots of work, lots of planning constraints to satisfy. The plans changed constantly, there were several versions of “the plan” and nobody knew which one was the “definitive one”, you had no idea what you were going to work on the next day…

And things were back to normal: chaos, work not being performed, bad morale, poor discipline. And lots of stress for the project managers who had to manage this chaos… That’s how it’s supposed to be, isn’t it?

The incredible shrinking program

I just don’t seem to learn, do I?

A classic experiment

Years ago, during a course on software engineering, the professor described a classic experiment: several programmers were given the same application to write. Each was asked to optimize one non-functional requirement: speed of implementation, speed of application, size of application, readability of the code, run-time memory use… Each developer was able to optimize what was asked of them, at the expense of one or more of the other qualities. No surprise there. TANSTAAFL.

I decided to perform this experiment on myself.

At the time, I was writing a hypertext help compiler and viewer, so that I could always pop up the programming reference documentation I needed while programming. The first useable version of the executable was about 20K (yes 20480 bytes) long. It might be fun to see how long I could keep the application at that size. As this was a resident application, the smaller the program was, the more memory was available for the IDE, compiler, debugger and the application I was developing.

A simple game with the expected results

So, I played a game with the following simple rules:

1. Get the most important feature from the feature list and implement it, until it works
2. If the application is larger than 20K, rework it until it’s smaller than 20K. All the features must still work!
3. Again…

The results were as expected: I managed to keep the application under 20K. This came at the cost of implementation time: sometimes it took a lot of ingenuity and work to squeeze the code back into that 20K. I had to find duplicated stuff, places where I could simplify the code, error handling I could remove because I had made it impossible for common user mistakes to happen…

And some unexpected results

I expected code readability to suffer: I might have to abandon my object-based C writing style, replace C code with more efficient assembler or use complicated tricks to keep the application small. I never needed to do that. If anything the code became more readable…

There were some unexpected side effects. In my experience, the longer you worked on a program, the more features you added, the more difficult it was to add (and debug!) new features. This program was really weird: it kept getting easier to add new features. Wow! That hadn’t happened to me until then and wouldn’t happen again for a long time after that.

This was the first program where I and my users (the application was released as shareware by then) ran out of ideas before my programming ran out of steam.

And what have we learned from this?

I stopped playing with limited home computers and started to work on real men’s computers: Unix workstations. I stopped making my programs small. There was no need: there was plenty of memory (megabytes!) in those machines. I was taught that it was cheaper to upgrade memory than to spend expensive programmer time.

And things were back to normal: with each new feature, it became more difficult to add another one. That’s how it’s supposed to be, isn’t it?

When I give a talk on Agile or Extreme Programming, there are two stereotypical answers I get from the participants:

• That won’t work (where I work).
• That’s just common sense. I’ve been doing that for years.

The first reaction is an ideal opening to ask the question: “You’re right. But… what’s the worst thing that’s holding you back?” That way you might discover their constraint.The second answer is a bit more tricky to deal with. Often, the answer indicates a lack of understanding of Agile software development: “I’ve been working without specs all my life. I was XP when you were still in diapers, son!“. If not, get their stories out and learn about their successes and failures.

I was like that, until a few years ago. Since then I’ve learned two things:

• It has worked everywhere I worked for the past 5 years. In fact, it’s the only thing I’ve seen work in that time.
• If it’s common sense, I must not have much common sense. Or any sense at all. I did things consistently as I had been taught them and was consistently unhappy with the results. Not that this made me change the way I worked; I just tried harder. Someday, it will work…

It’s not that I haven’t had many opportunities to learn. I just missed them all. I was probably trying too hard to notice the obvious solutions staring me right in the face.

So, let me tell you about some of these learning opportunities I missed…