Re: Timeless Classics of Software Engineering

Richard Riehle wrote:
> XP does make a contribution to software engineering thought, and
> an important contribution at that. Many of the ideas in XP have been
> around for a long time and used, however informally, in the creation
> of successful software products. I can recall a rule in a
> programming shop where I worked in the mid-1970's that was a lot like
> "pair programming."

IMHO, XP is a lot about things that people assume implicitely and which in practice don't happen when you keep them implicit. You have to force people to do them.

Pair Programming and Move People Around: People within a project need to talk to each other. I've seen many engineers who think that this is a waste of time, and that you can work better when you are alone in a quiet corner, and nobody makes outsider suggestions to your design. If you look into someone else's work, and you find a severe problem, people often take it as insult (depends on how long you looked to see the problem - when it takes a few seconds, and you make an ugly face, it's worst case ;-). I've even seen other engineers jump into defense of the "offended" person, declaring that "my analysis was wrong". Unfortunately, the problem was real. This sort of "MY code" "YOUR code", i.e. ego, is in the way.

Pair programming formalizes the process to have more than one person look at what's done. There are alternative approaches like open sourcing the software, and have the users look into the source. However, uses don't really look at sources (and if they do, they are not good at it), except if there's something that doesn't work. This is also too late (e.g. the last time I had to look into GCC, I saw the horrors of legacy code; fortunately, the GCC team feels the same pain, and inserted a new, saner intermediate representation).

User stories: Customers fail to set proper requirements. They usually (strongly!) suggest an implementation that supposedly solves the problem. Since the customer is not an experienced architect (that's why he's a customer, not the chief architect), this implementation must have severe drawbacks. The proper way is to divide the requirements into input/output relations, where independent requirements are separated. XP formalizes that with "user stories" which have to fit on a card. Note that the "user stories" both provide requirements and test cases.

Unit tests: I think this is also related to refactoring. Traditional programming languages make it difficult to write tests along with the code, so people tend to just hack the code in and test the complete application. That's wrong. As you can build a house only on bricks that sustain the pressure, you can build software (or hardware) only on functions (circuits) that are robust, and deliver what they should. Thing that haven't been tested won't work.

Refactoring comes into the game by forcing the programmer to actually build the system out of bricks, and not out of concrete walls (the equivalent of monster-functions which are casted into stone).

Iterations: Due to the fact that the customer has no clear idea what he really wants, even the user stories are misleading. An iterative search approach is formalized to find what the user actually wants. It is important to realize this (the customer is wrong) on both sides, because typically, you can't sell a project where you already plan the necessary iterations ahead. You have to charge the customer afterwards, it's difficult, and you'll usually take the blame.

To summarize XP: XP is about handling incompetence. Since at least 95% of all engineers think they have more than sufficient competence, and it's only the other 95% of engineers surrounding them, which are incompetent, something has to be done. If you feel better that way: once you use XP techniques, you can say that you do everything to increase your competence.

The waterfall model also is about handling incompetence. It tries to avoid incompetence by assigning tasks to competent people (e.g. "choose an architect to do the design, not a coder"), and tries to achieve competence by specializing. The main problem here is that this model assumes that somehow you are able to find competent people for each step of the waterfall model, and that somehow the specialists don't fail due to their shortsightness.

Note that you still need competent people. The productivity of excellent people in software development is so high that the average coffee-drinking Wallies really do negative work. This is no good news for people who want to solve unemployment, because the simpler jobs just go away (to China and to robots, and finally to robots in China). The good news is that when we really get there, the remaining people who have work are true hackers, and do it for the fun of it.

BTW: The real hacker way to deal with customer requirements is to throw them into the bit bucket, and analyze the problem yourself (only care about it if it is interesting, and redefine it until the solution is trivial). Unfortunately, this often leads to software where user and developer have to be the same sort of people, and which doesn't solve the problems of average users (problems hackers don't have at all).

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://www.jwdt.com/~paysan/