Re: Timeless Classics of Software Engineering

> From: Bernd Paysan <bernd.paysan_at_gmx.de>
> When you start programming a function, you define it on the command
> prompt, as you do with the manual unit tests.

There are at least three UI configurations where this is the opposite of what's actually done:
- When using a CL IDE, such as Macintosh Allegro CommonLisp that I used on my Mac Plus before it died: The user doesn't type code into the command window, but rather composes code in the edit window, then uses command-E to execute whatever s-expression is adjacent to the cursor. - When using EMACS-LISP, something similar I presume. - What I'm doing currently: McSink text editor on Macintosh, VT100 emulator connecting me to Unix, CMUCL running on Unix. I compose code in a McSink window, copy and paste to VT100 window whereupon it is transmitted to Unix and fed on stdin to CMUCL.

> If it works (and the tests are supplied with the proper commands),
> you can cut&paste them from the command history into your source
> code, and then you have automated unit tests.

I don't know about EMACS-LISP, but in both MACL and McSink/VT100/CMUCL, there's a scrolling transcript of the current Listener or dialup session respectively. I already have the "command" (the s-expression fed into READ-EVAL) locally in my edit window. If I want to retain a copy of the output (from PRINT), that's the only part I need to copy from the Listener/dialup window and paste into my edit window. But for manual tests, I just eyeball the result when testing, it's obvious whether it worked or not. Sometimes if the result is just a number, such as an index into a string, which I can't tell if correct or not just by looking at it, then I'll copy the result into a comment alongside the "command" expression (after first doing an additional check to make sure it was correct, for example to test the result of a SEARCH or POSITION, I do a SUBSEQ call to see the part of the string starting or ending with what it had allegedly found so I know it found the correct thing, especially for example in my recent work where I'm parsing 30k HTML files which are Yahoo! Mail output and the index where something was found is typically 15-20k into the long string).

So anyway, it's pretty easy to collect whatever you need, input and/or output from a test, as you go along, except for very long strings and large structures where you don't want to include verbatim the whole thing but instead want to save it to a file and have your test rig read the file to get input for the function under test. Very flexible what to actually do from moment to moment as needed...

If I were getting paid, and I'm not the only programmer working on the code, I'd want to set up something more formal: Each test-data input file would be formally registered and kept as-is with nobody allowed to change it without permission. Then a test suite for a batch of code could confidently perform some sort of read of that file to get the first item of test data, pass that data through the first function and compare output with what was supposed to be the output, then pass that output and possibly more canned test data to the next function, etc. testing all the various functions one-by-one in sequence from raw input through processing stages to final output.

Often any single one of those major data-processing-pipeline functions is composed of calls to several auxiliary functions. It'd be easy to use that sequence of calls to directly produce a test rig, based on the master data flow, for each of those auxiliary functions. So the finished test suite would, after loading the canned-test-data file, first test all calls to auxiliary functions in sequence within the dataflow of the single first major pipeline function, then test that one main dataflow function as a gestalt, then likewise test inside then all of second, etc. down the line. Of course if one of the auxiliary functions is itself composed of pieces of code that needs to be tested, the same breakdown could be done another level deeper as needed (test parts in sequence before testing whole as gestalt). Of course for functions that take small easily expressed parameters, instead of huge strings, they could be tested with literal constant data instead of data from dataflow from canned-test-data file. For testing boundary conditions, error conditions, etc., this would be useful. What about functions that take huge inputs but where it'd be nice to test boundary cases? Well then we just have to contrive a way to generate boundary-case input from the given valid canned-test-data file, or create a new canned-test-data file just for these exceptional cases.

I wish somebody would hire me to do programming work for them, so I could put my ideas into commerical practice... Received on Fri Aug 20 2004 - 23:47:26 CEST