Re: database systems and organizational intelligence

"Dawn M. Wolthuis" <dwolt_at_tincat-group.com> wrote in message news:c97doc$i8i$1_at_news.netins.net...
> Yes. I took a look at Haskell when you brought it up earlier and if Java
> didn't have the industry backing, tools, and numbers of developers that I
> need for my efforts right now, I'd be playing (perhaps even working) with
> it.
>
> It's all data. It's all functions. That's how I think about databases as
> well as languages.

True, within a given context. But it's like the "what" vs. "how" distinction - "what" you do at "abstraction layer N" becomes part of "how" you're implementing layer N-1 (where lower numbers are higher layers of abstraction). You have to know where you're at, and during a project you find yourself at many different levels. Within a given context, not all code is data, or at least not all of it benefits from being treated that way. Not all apps need a domain-specific language with an interpreter for users to write scripts. You have to choose your battles, and avoid cases where you're treating code as data but are unable to predict or mandate anything about its structure - then you've just got a String.

Java is less powerful than Lisp/Scheme and its kin partially because functions are not first-class citizens (though perhaps if Reflection were part of the core language we might get somewhere). You have to jump through hoops, and use the Command pattern, but that's still only first-order (unless you jump through additional flaming hoops). In addition, the only data structure native to the language (as opposed to placed into libraries) is an object graph - what Java specifically lacks is some declarative way of relating objects.

I did stumble across an interesting project the other day - JXPath, from Apache/Jakarta (never can keep them straight). It allows you to evaluate XPath expressions against object graphs - you supply a root object, and it navigates the path using JavaBeans conventions. While I'm not a huge XML/XPath fan, it beats the hell out of nested loops looking for objects that meet various criteria... and allows those expressions to be dynamically-generated. The biggest failing (I think) that (most) OO languages have is that the navigation is hard-coded, meaning that you have to choose 1 path, when something like relations allows you to evaluate one of N logically-equivalent expressions to get what you want. The JXPath stuff above, while not as powerful as relations, at least allows you to avoid coding some of that navigation.

> > In another
> > vein, rules engines and any application with a built-in interpreter
(e.g.
> > Jython, Guile, etc.) allow the end-user (or at least one of their
> > developers, who may not be familiar with your software) to add scripts
> which
> > are invoked at appropriate times by the framework.
>
> thus putting a new twist on the old "user exits" where it can now be the
> end-user, not just an IT professional, writing the code to be executed at
> various points.

Yes, as long as your framework is built well enough to detect and/or recover from the inevitable user mistakes. Then again, that's not terribly different from "firewalling" oneself from programmer mistakes... even my own...

> It makes sense that it be the end-user who maintains any
> constraints that are not related to the databases internal integrity --

I agree, but that's a very grey line. What's core vs. potentially-changeable? That's far less decideable than most mathematical propositions. At some point it seems you have to decide "right now, this code is just code, not data". Although I have to admit that since I'm not a fulltime Lisp programmer, I could be wrong...

> for example, a size constraint (for display and input) on an attribute or
> whether to validate a particular value against a list of possible values.
> Setting up test environments can be tricky when using this approach,
> however. What was once "coding" is then often done in the live
environment.

Agreed. That's also why it's critical to keep the constraints declarative - procedural logic opens up a can of worms. And since above machine code all our languages are "just notation", it's clear that notation matters a great deal. "Syntactic sugar" is rhetoric - syntax matters, just as human language matters.

Thus the value of something like Guile (an embeddable version of Scheme, which is a version of Lisp), which allows the development of domain-specific languages on top of the core language - and with relatively little "code." Obviously there are other possibilities - it's possible to do Python, for example, in a more functional than OO style. You can even do Java that way... it's just harder.

> You seem to have a broad language background-- with your appreciation of
> relational theory (perhaps including declarative languages?), functional
> languages, OO, and scripting languages (procedural) -- an off-topic
question
> that I might have asked before:
>
> Iin what (type of) language would you like to see data constraints encoded
> and constraint handling logic written so that these same "services"
> (components) can be used for both data entry (when needed for tossing up a
> UI or validating entered or sent values) and data storage (if the DBMS
> didn't lock you in)?

Thank you - but rats, now I have to commit to something...

I like Tutorial D, although it's a bit wordy. It allows one to express much more than typical foreign-key constraints; in fact, the foreign key clauses are simply shortcut for a longer expression in the full language. The critical part is that these are declarative statements of what relates to what - not how to get there (using what and how loosely). By requiring a relation to be a predicate, different "paths" from A to B are logically equivalent, which I don't believe is true of most XML docs I've seen - the nesting is limiting, I think.

Note that (I'm pretty sure) views are allowed to be parts of integrity constraints in D, allowing for a gradual migration in some cases where even the logical structure has changed.

I also like Alloy very much - it uses only relations to express and check constraints, though its niche is model checking, not implementation. A language like it might be very useful, however, for development.

> Lament -- I'm teaching Java in the Fall and the last time I taught a
college
> language course was over 20 years ago --COBOL, Fortran, and BASIC. Where
we
> were once able to teach a language and in short order a student could take
> in data, validate it, use it, modify it, store it, and retrieve it (i.e.
> data processing), we might now need to teach Java, SQL, XML (including
many
> xml-based "languages" such as ant), jsp, html (along with taglibs etc) to
> write one little example of taking in data, storing it and retrieving it
> again the way a production system might work. (and then to get the
software
> run-time environment set up...!!)

Agreed, it's a lot to digest, although I think presenting the overall environment in technology-neutral terms (discuss a "web server" and an "app server" in general, a UI in general, etc.) could help set the stage for how to implement each of those pieces, and what constraints tie them together. Then show one piece at a time - for teaching, I've found an XP approach very useful.

I think the vast majority of our technology is repetitive, and indicative of language and platform failings. If it were easy to extend Java, for example, we might not need quite so many design patterns. I am encouraged by emerging declarative frameworks - for example, Tiles and Struts (though there are other frameworks that claim to abstract even further) on the UI and web structure side, various interception architectures like Guarana. I hope that some of the better-designed of these make their way into complete languages; Java is a fine assembly language, but our industry should be further ahead by now. At least Sun finally got C++ right (mostly).

All these indicate that different parts of our solution space have their own needs, and yet derive certain fundamentals from the data being manipulated. I think that relations combined with something like Tiles (and maybe a Swing or SWT templating mechanism) could be used to derive UIs, based on the relational structure and the domains of the attributes, each of which would have a mapped visual representation. And I believe a relational language could simplify a great deal of the code we write to query and update our in-memory data (independent of a DBMS). I see the problem as a distributed database issue; the client program is manipulating data internally, and after a number of manipulations (which have been validated against the schema, as declared in one place, probably the RDBMS), when the user saves those updates are synchronized with the server. Not an easy problem in itself, but something there are standard ways of resolving.

Hopefully soon we will have 2 sets of declarations: data (in all its glory, with full constraints that express the meaning), and components (how processing components relate - navigation and delegation). Much of the guts of the components should be generated by the basic data constraints. That's my hope for the future... and no, I don't mean just one choice, but certainly fewer choices, constrained by education in basic logic and minimal thinking (stating what you want and letting the implementation engines enforce it and generate code).

I don't think I've directly answered your questions, so forgive my ramblings. I haven't truly settled on one set of technologies, but look for minimal expression, minimal ordering (i.e. procedures), and maximum generation. I think Dataphor is doing good work, but have to look at it in detail... I've only caught some conversations by its developers.

• erk