Thinking for a Change

The one without a bug.

I just don’t seem to learn, do I?

Daddy, where do bugs come from?

My project manager wanted to know where all the bugs came from. Well, there’s this stork and… Yeah, the “Bug Stork”…

Well, no. We make mistakes and introduce errors into our programs. We had a lot of errors. We had so many, we needed a database to keep track of them.

Were some modules more error-prone than others? If so, why? That’s what the project manager wanted to know. He had written a script that counted the number of lines in each module and divided this by the number of bugs assigned to this module. He was going to make a graph of bug density. That would tell us where to concentrate our effort, because we couldn’t fix all the bugs.

He asked me to help him debug his script, because something was going wrong. His script always threw a “division by zero” error when calculating #lines/#bugs. We discovered that there was one module with zero bugs! Clearly, a requirements error: nobody expects a zero-defect module!

The perfect module

Which module had no bugs? Who was the super-programmer who had implemented this perfect module?

Turns out that this module was written by M., a programmer in my team. It was the first thing he ever wrote for the company, fresh out of school. His later work was riddled with errors, like everyone else’s. What had happened?

As M. was new and this was his first programming job, I was asked to mentor him and help him find his way around our code, so I had a pretty good idea how this module was written.

The recipe for a perfect module

I divided the project into small pieces. Before starting on each piece, M. and I would discuss how we could implement it. M. would then go off to implement a bit, asking me questions whenever something was unclear.

When M. had written a piece, we’d go over the code. M. would explain the code to me and I asked questions and gave feedback. Often, the fastest way to explain what I meant was to take over the keyboard and change a bit of code or write another bit. And then M. would change something. We would rework each bit of code until we were happy with it and it worked. And then on to the next small piece.

And what have we learned from this?

It took a lot of effort on my part, effort that wasn’t being put into productive work. But the goal was to teach M. how to code independently.

After that first project, M. didn’t need any more handholding. He could now implement modules on his own. I could go back to my real work: writing and (most of all) debugging modules. Now, we could each work on our own modules. That’s a lot more efficient than two people working on one module.

And things were back to normal: neither M. nor I ever wrote a (even remotely) defect-free module ever again… That’s how it’s supposed to be, isn’t it?

Into the heart of darkness. And back again.

I just don’t seem to learn, do I?

Don’t touch anything!

The previous story told how I did one last project before I left this company. This project involved customisation (for one customer) of the way data was acquired and processed by two applications. In the past, when I changed these applications, I would always introduce errors. My project manager’s motto was “If it works, don’t touch it!”. As if this application was a house of cards… Unfortunately most of the code didn’t work. I had to change some critical code to implement this customisation.

There be monsters here!

Why did I get this project? Probably because no one else dared to take on this project. The main part of the coding would be deep in the bowels of the applications, written long ago. It was like a dark, dank, dangerous dungeon nobody dared to enter. There had been some stories about people going in, but they hadn’t been seen again.

Test yourself!

This feature was intended for one customer. The number one requirement was that for all other customers, the results should be totally compatible with the previous versions of the software. Our customers needed to be able to compare new measurements to their historical data. Hmmm, how could I be sure that my changes didn’t introduce any incompatibilities in the resulting data? This piece of code processed megabytes of data as they came in from the data acquisition hardware. I would never be able to verify its correctness by hand.

Who can verify this data fast enough? The application, of course! If it was fast enough to process the data, it was fast enough to verify the data. I restructured the affected code to look like this:

Cleaning the dungeon

At first, “Changed code” was a copy of the original code. As expected, the “Compare” stage indicated that there were no differences between the two streams.

This code was a real mess: it was hard to understand, full of low-level C pointer manipulations, long functions, function names without meaning, 1 or 2-letter variable names… To understand the code, I started to make small changes. After each small change, I would re-run the application with test data to verify that I hadn’t introduced any incompatibilities. If anything was broken, I undid the change.

After a few days of making these small changes, I had transformed the scary dungeon into a light, airy, cosy little room: the code was now simple and easy to understand. It gave the same results as the original disgusting code. No monsters had been seen or harmed during this transformation.

Now I could implement that feature. It was extremely easy and only took me a couple of hours. I used the same technique to verify that my changes were compatible if the feature was not activated.

And what have we learned from this?

This time I had learned something: you should always have tests for your code. On the next major project, we wrote a whole set of tests that exercised the code we were writing. The test called all of our methods with certain test data and print out the results. I looked at the output and see if everything was well. I ran tests every day, so that we discovered errors early. I was the TestRunner.

When I left that project, nobody ran the tests any more. It was too difficult and time-consuming to run the test application and interpret the results. They might run it once in a while if there was a serious bug, but the test was finally forgotten.

And things were back to normal: people were scared to make changes, bugs weren’t caught quickly, more bug reports came in from testers and customers … That’s how it’s supposed to be, isn’t it?

One for the road

I just don’t seem to learn, do I?

I quit!

After working for a few years for this company, I decided to leave. I had to negotiate with my manager and the HR guy how long I would have to stay on.

The manager proposed that I stay on until I finished “Project X”, a customisation of two of our products for one of our biggest customers. At this company we had a habit of starting projects with many, unclear feature requests and then piling on more features as we were developing. We were all too busy with frantically churning out code to notice the whooshing sound of the deadlines as they flew by. We felt like Mr Creosote, being fed one “wafer-thin” feature at a time, until we exploded. If past performance was an indicator, it would be at least a year before that project would be done and I could get out.

We finally agreed that I would stay on another 5 weeks, the legal minimum. I would do as much of Project X as I could in these last few weeks.

An offer he couldn’t refuse

I asked the product manager to select the few most important features that would satisfy the customer AND that I could implement in a few weeks. He was not amused; he wanted “everything”, as usual. My final offer: “either you get some features OR you get none. It’s your choice, I’m off in 5 weeks in any case”. He grumblingly took the first option and complained to everyone who’d listen about “those developers getting uppity”.

We selected a set of features that was both feasible and provided the customer with a coherent implementation of their requirement. The metaphor we used was: “This is like a software patch panel, so that the customer doesn’t have to waste time manipulating the wiring on the hardware patch panel.”

The knights who say ni no!

Everything had gone smoothly: no features had been added during development. Not that the product manager didn’t try, but I could now afford to say no. Everything had been done according to the company’s process, including all the documents and documentation. I had worked at a relaxed, leisurely pace and yet I had accomplished more than on any other project at that company. In the process I had refactored a particularly nasty piece of code (more on that later) and created a reused piece of code.

4 weeks later…

The product manager performed the acceptance tests. He grumbled about all those missing features. He found a few small errors. We found a major oversight in the requirements: the UI didn’t indicate how the “patch panel” was (re)wired, which was very confusing. Oops!

Next day, I fixed the small errors and we performed the acceptance tests again. Passed.

Around the process in one day

Next day, we discussed how the “wiring visualisation” feature should work, I adapted the specification document, adapted the design, modified our MVC framework, used that to update all affected UIs in both applications and we performed the acceptance tests again. Passed. We had just gone through our whole development process in a single day.

Done. That left me two days to say goodbye to everyone and buy the drinks and snacks for the farewell party. This the most fun AND productive project I’d ever done at that company. And it was the only one that finished on time. I should have quit more often!

And all those extra features? They were never implemented.

And what have we learned from this?

I left for bigger and better things. Well, not bigger and not always better, either. I still scheduled based on features. I still believed that the customer needed all the features they asked.

And things were back to normal: deadlines slipped, goalposts moved. Developers got demotivated. Except for the small (1 month or so) projects… That’s how it’s supposed to be, isn’t it?