Re: Development as Configuration

From: Kenneth Downs <knode.wants.this_at_see.sigblock>
Date: Thu, 05 May 2005 18:23:46 -0400
Message-Id: <g3jqk2-tq6.ln1@pluto.downsfam.net>

dawn wrote:

> Kenneth Downs wrote:

>> dawn wrote:
>>
>> > erk wrote:
>> >> dawn wrote:
>> >> > ugh.
>> >> > On the one hand we want to bump up some levels so we are using
>> >> objects
>> >> > of types that are more compex than Strings and ints and so we

> are

>> > not
>> >> > hand-coding such things as threading logic every time we write

> an

>> >> > application. On the other hand, we should retire one-way code
>> >> > generation for developers.  Bea might be talking about

> round-trip

>> >> tools
>> >> > here, but there is no reason to think so from the above quote.
>> >>
>> >> Depends on what you mean. All compilation is "one-way code
>> > generation."
>> >
>> > OK, I'll be more precise and say that the only code generation of

> any

>> > interest to me is that which either provides a round-trip between
>> > spec/picture and code or that which keeps the code hidden and

> almost

>> > always unncessary for any software developer to read.  In that

> latter

>> > case, the "spec" becomes the new code, so there need to be good

> tools

>> > for reading and debugging at the level of the spec/new language.
>>
>> Good.  Once the spec has become the new code, there is no more round

> trip.

>> We generate code, and when we change the spec we through away the

> code and

>> regenerate it.

>
> I'm good with that -- the source is then not a "code generation tool"
> so much as it is a computer "language"

A declarative language, one that can only state propositions. In our case the language is a thin disguise for what is really atomic data.

>>
>> >
>> > 1) The spec'ing tools are proprietary, while the underlying

> generated

>> > language is more likely an industry standard.
>>
>> What if the tool were free software (free as in freedom, GPL, that

> stuff),

>> or open source? Or perhaps simply all of the standards were published

> but

>> the software was still held proprietary?

>
> It is a question of what "languages" one wants to be locked into as a
> company or developer. Not everyone will have the same answer to that.
> I'm OK being locked into a single language that has compilers (or
> interpreters, or generators) from multiple parties, where anyone can
> legally write such a compiler.

If the spec'ing tool is any good, it feeds on raw data, not code. The data can be in any file format, such as XML, CSV, or spit out into a text file in some declarative language.

Because it is fundamentally data, not code, you are only locked in if you are unable to manipulate data.

>

>> > 2) The generated code might be portable, but the specs must be
>> > converted and that is rarely 100%, not an easy job, and rarely

> yields

>> > quality, easily maintained software after the port.
>>
>> Fascinating.  How can specs not be portable?  What examples are you

> thinking

>> of?  To me a spec is just data and that means it is always portable.

>
> Sure, if you are saying that all code is portable in that it is text
> that one can move to other OS's. Clearly that was not the meaning of
> "portable" in my comment (OK, clear to me). What I mean is that if I
> want to ditch vendor xyz whose product I'm using to take my code from
> spec to the OS/VM and use vendor abc instead, then I want my spec to be
> portable so that it can be used "as is" with the new vendor.

This is an extreme requirement, but I like it. It's also a 1-10 scale kind of thing rather than a true-false kind of thing. Personal experience has always gotten me closer to 10 by following a RISC strategy of using as few primitive technologies as possible, like sticking to core SQL 92 and staying away from proprietary extensions, or at least staying away from those that have no analog on the competition's product.

But to return to my original question, how can a spec, which is data, not be portable? Data is far more portable than code will ever be.

>

>> > 3) The application is often easier to debug at the level of the

> code,

>> > rather than from the specs
>>
>> Ouch.  This is true too often.

>
> Yes and it is NEVER considered up front as the thing to do, but then at
> some point you reaize everyone is having to learn to read the ugly
> generated code (of course we used to read hex in order to debug the
> simplest things, but ...)

I am going to tuck this whole idea into the back of my head and revisit it from time to time.

I will say a few things for the record:

1. my own generated code is "assembled" rather than generated, and contains comments. No "paragraph" is more than a few lines long. It is not ugly.
2. We never look at the generated code. The reason I believe we have avoided your trap is that we use only a precious few primitives that have been quite thoroughly debugged. I'm not saying they are perfect, just simpler and therefore they will mature faster, and have already shown a tendency to do so.
3. By Ken's law that "people understand tables just fine", we keep both our specs and our applications very close to the tables, trying turn everything we can into simple table maintenance. This is sort of the application equivalent of WYSIWYG.

>

>> We have since passed the break-even point
>> where there are more problems in our specs than in our code, but

> still by

>> reflex we end up looking at code a lot.

>
> No, I don't think that is the reason. The reason, I believe, is that
> the debugging tools for the spec are not good enough to find and solve
> problems. Would you say that it is as easy for a developer to find
> problems in their specs with your tool as if they debugged at the level
> of the gen'd code?

It is clear here that you have no small experience in the matter, but in this case I really do think our debugging abilities in the spec are ok. The reason again comes down to the fact that it is data, not code that we are debugging.

Our language format is a thinly veiled disguise for typing of tabular information in a text file, something like this:

column address1 { description: Address 1; type_id: char.....}

this is crucially important because it means our validation is composed of (drumroll......) simple unique and referential constraints! That is, a column may not be defined more than once, and you cannot assign security priveleges to a table for a group that does not exist.

Then of course at parsing there is simple stuff like missing curly-braces, mistyped keywords and so forth.

>

>> > 4) While languages are often careful about backward compatibility,

> code

>> > generation tools more often require you to regen an entire

> application

>> > to take advantage of a new feature.
>>
>> Why is this bad?  What would be better?

>
> What do you mean, "Why is this bad?"? Are you asking me why backward
> compatibility of source code is a good thing? There are at least a
> couple of compatibility issues I care about and I'll use Java as an
> example. 1) If I have code written in Java 1.4, when I upgrade my VM
> to Java 1.5 I don't have to recompile everything I have written under
> 1.5 in order to run it and 2) If I upgrade the compiler to 1.5, I don't
> have to change the code that has already been written in order to
> compile it under 1.5.
>

Ooops, not sure we are talking about the same thing. The nature of generated apps is that the code has no particular value, by virtue of how easy it is to get it and replace it. So regenerating an entire app for any reason is not seen as a problem. Maybe I am not understanding where you are coming from.

-- 
Kenneth Downs
Secure Data Software, Inc.
(Ken)nneth@(Sec)ure(Dat)a(.com)

Received on Thu May 05 2005 - 17:23:46 CDT