Re: Timeless Classics of Software Engineering

From: Eric Hamilton <eric.nospam.hamilton_at_hp.com>
Date: Fri, 27 Aug 2004 19:45:40 GMT
Message-ID: <EbMXc.8961$PN5.729@news.cpqcorp.net>

Thanks for a stimulating topic.

I heartily agree that Mythical Man Month is essential reading for anyone who wants to understand large scale software projects.

The other essential on my book case is Lakos' "Large Scale C++ Software Design". It's applicable to any language and has enough rationale that's grounded in real development practices and the problems of large scale projects that I think it's relevant to the original topic.

A few years ago, I happened to reread Brooks and wrote up a collection his insights that resonated with me. I've attached it below in hopes of whetting the appetite of anyone who hasn't already read it and as a reminder for those who haven't reread it recently. I encourage everyone to (re)read the full book.

Eric

---

Notes from re-reading "The Mythical Man-Month" by Fredrick P. Brooks, Jr.

I went looking for a quotation about the value of "system design" and ended up reading most of the book because it has many insights into the challenges of producing large-scale software projects.

Some highlights:

In the preface Brooks says that while OS/360 had some "excellencies in design and execution", it had some noticable flaws that stem from the design process.

• "Any OS/360 user is quickly aware of how much better it should be."
• OS/360 was late, took more memory than planned, cost several times the estimate, and did not perform very well until several releases after the first.

His central argument is:

• "Briefely I believe that large programming projects suffer management problems different in kind from small ones, due to division of labor. I believe the critical need to be the preservation of the conceptual integrity of the product itself."

Why are industrial teams apparently less productive than garage duos?

• Must look at what is being produced.
  • program: complete in itself, written by author for own use
    • programming product: more generalized, for use by others
    • programming system: collection of interacting programs with interfaces and system interactions.
    • programming system product: both product and system

Interesting exposition of the "joys of the craft" of programming:

• joy of making things
• pleasure of making things that are useful to others
• fascination of making complex puzzle-like objects
• joy of always learning due to non-repeating nature of the task
• delight in working in such a tractable medium

Also "woes of the craft"

• one must perform perfectly
• others set objectives, provide resources, furnish information
• dependence on others' poor programs
• finding nitty bugs is just work
• linear convergence of debugging
• appearance of obsolescence by time you ship when you compare what you ship to what others imagine

An analysis of sources of programmer optimism:

• creative activity comprises the idea, the implementation, the interaction with the user
• tractability of medium leads us to believe it should implement easily
• other media place constraints on what can be imagined and limitations of media mask mistakes in the ideas

Man-month: fallacy of lack of communication or serialization

Naive preference for "small sharp team of first-class people" ignores the problem of how to build a *large* software system.

Harlan Mills proposed "surgical team" approach. [Not applicable everywhere.]

Conceptual integrity:

• Analogy to architectural unity of Reims cathedral vs. others that were "improved" inconsistently.
• "I will contend that conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas."
• The purpose of a programming system is to make a computer easy to use. [We may modify purpose to be to make it easy to do the things that our customers need done.]
  • Ratio of function to conceptual complexity is the ultimate test of system design.
    • For a given level of function, that system is best in which one can specify things with the most simplicity and straightforwardness.

Careful division of labor between architecture and implementation allows conceptual integrity in large projects.

• Architecture: complete and detailed specification of the user interface (for OS/360 the programming manual).
  • We may want to consider what is the right specification for <project>
• Architect is "user's agent". Brings "professional and technical knowledge to bear in the unalloyed interest of the user."
• Architecture tells what happens, implementation tells how.

Argues that designing implementations is equally creative work as architecture. Cost-performance ratio depends most heavily on implementer; ease of use most heavily on architect.

External provision of architecture enhances creativity of implementors. They focus on what they uniquely do. Unconstrained most thought and debate goes into archtectural decisions with not enough effort on implementation.

Experience shows that integral systems go together faster and take less time to test.

Need coordination and feedback between architect and builder to bound architectural enthusiasm.

Second system effect:

• overextend and add too many bells and whistles
• may spend too much optimizing something being superceded by events

Communication & decision making

• Strong belief in written specifications
• architects meetings
  • emphasis on creativity in discussions
    • detailed change proposals come up for decisions
• chief architect presides & has decision making power
• broad "supreme court" sessions handle backlog of issues, gripes, etc.

Organization:

• Talks of "producer" & "technical director or architect"
• Either can report to the other depending on circumstances & people

"Plan to throw one away; you will, anyhow."

"The most pernicious and subtle bus are system bugs arising from mismatched assumptions made by the authors of various components. ... Conceptual integrity of the product not only makes it easier to use, it also makes it easier to build and less subject to bugs." Received on Fri Aug 27 2004 - 14:45:40 CDT