Re: database systems and organizational intelligence

From: Dawn M. Wolthuis <dwolt_at_tincat-group.com>
Date: Wed, 2 Jun 2004 21:16:35 -0500
Message-ID: <c9m1m5$i93$1@news.netins.net>

"Eric Kaun" <ekaun_at_yahoo.com> wrote in message news:p41vc.5086$Zq.1808_at_newssvr32.news.prodigy.com...
> "Dawn M. Wolthuis" <dwolt_at_tincat-group.com> wrote in message
> news:c98fu9$4mj$1_at_news.netins.net...

<snip>

> > > > Of all of the languages in the mix, the "relational one" (SQL) is
one
> of
> > > the
> > > > most problematic, causing us to treat stored data differently than
> other
> > > > data, for one thing
> > >
> > > I agree with you here, but probably not for reasons you'd like. I wish
> > > programming languages were "more relational", as opposed to making the
> > data
> > > "more graph-like" like XML and OO.
> >
> > the data IS more graph-like -- I just want to view it as it is ;-)
> > [over-simplification]

>

> Well, then I envy you. If you've only ever had the need to view data as a
> single graph, then your requirements have been much kinder to you than
mine
> have to me. Typically I see the need for many different graphs in
different
> contexts, which is difficult to support through graph-like transforms, but
> easy when based on relations.

Hmmm. I'm thinking of the namespace as the root, so I guess I work with just one graph.

> > > > (and then tied to that bloomin' 1NF for another).
> > >
> > > Well, as Date says, 1NF is something of a matter of perspective,
> >
> > as well as the premise for all of relational theory, being that all
> > normalization requires 1NF to start with

>

> So take it in its generous form, and use it as you wish. As you said, you
> still get value from 2NF and 3NF, right, even in Pick?

Yes, the entire functional dependency approach gets at some key aspects of maintainability for any model.

> Date's point is
> simply that the DBMS sees a single value at any given tuple/attribute
> "point". That single value can be "complex", but that's a type you've
> defined, and the DBMS doesn't care about that - you have to do the work.
No
> special treatment for types.

Which means you have to do the work when retrieving the data too. There is more than one database hosted by a SQL-DBMS where the users opted to put a delimted list in a field, but then they have work to cope with that.

<snip>

> Agreed, except that I'd be fine with spec'ing them in XML and then leaving
> > them in this tree structure for use - no need to do that XML to
SQL/RDBMS
> > mapping at all.

>

> Well, if you can use constraints specified in XML format, you're a better
> human than me.

Since XML is hardly a dictatorial format, it is pretty easy to specify a constraint with it -- just stick a SQL statement between tag delimiters ;-) [ignore that]

> I've yet to see a simple logic or data structure that XML
> can't render so opaque as to be unusable. Even the cleanest XML is dirty.

>

> > Additionally, because I want more constraints to be
> > specified by the users and/or end-users AND IGNORED by the database
> (that's
> > what gets put in the app code today, right?), I see the DBMS constrains
as
> a
> > projection of the application constraints.

>

> But certainly there are some things you actually want the users not to
> override, right? In any event, you argue for a domain-specific language
> interpreted in your app - a good thing. But you have to keep in mind that
> just because users can specify some things doesn't mean they'll be good at
> it, and your app has to have enough structure to not fall apart when they
> screw up.

Yes, it is a risk-assessment thing. SQL-RDBMS implementors often treat all constraints as if they have an equal risk related to every constraint.

> > We can let a user say they want
> > to enter and view address lines as max of 30 characters without the DBMS
> > nosing in on that and fixing that length in the schema, for example.
Sure
> > there is a risk, but it is a risk worth taking when there is no fixed
> > reason -- just the whim of the current user -- for sticking to 30
> > characters.

>

> I agree. I'd say there needs to be more analysis, though, to avoid things
> like a user who changes this and subsequently screws up the formatting of
> various reports going to other departments.

Agreed.

> > We don't want users knocking down supporting walls without
> > involving IT professionals, but we have got to let them hang pictures
and
> do
> > other makeovers of their applications without requiring an extreme
> makeover
> > (using doctors).

>
> Yes, true. This can be a slippery slope, but I think this sort of
necessity
> argues for better typing, not less typing.

I'm still trying to figure out what I think about typing -- what, when, where, how -- there is something in current SQL-DBMS approach that really bugs me and I haven't pinpointed it but it is related to the brittle nature of many database implementations.

> > Relational theory says it is about how to view the data, not how it is
> > stored. In that case, the users know how they want to view it and the
> > constraints should originate from there.

>

> Well, if by users you mean all of them, then yes. VPs, CEOs, accountants,
> auditors, etc. are all users, to some extent. You have to take the body of
> requirements into account, not just Joe at the data-entry desk.

Yup, I had even forgotten about Joe, but it was a collective statement, not singling out one user's requirements.

>

> > I want to design constraints from front to back and not the other way
> around.

>
> Why? They come from both directions. It depends on which interested
parties
> ("users" is too specific) you mean. The government has requirements,
too...

When I typed that, I knew you would have legitimate disagreement, but I still pretty-much agree with my statement because that government requirement comes into play when a user identifies it as required for the organization. The government doesn't interface directly with DBA's (or shouldn't) in non-government organizations. So I don't see another direction, other than end-users, from which these requirements should come, but if conflicting requirements come in, that must be negotiated and we wouldn't want to implement something that would reduce the system qualtiy (reliability, security). But semantic constraints should all arise from the users of the system.

> > Some can stay at the front as
> > "local constraints" (aka application software). Relational theorists
seem
> > to have "centralized control" as the key concept. I think I'm a more
> > progressive parent than that. Sure there are risks, but it seemed to
have
> > worked with my kids. Users can take responsibility too and recognize
the
> > implications of changing the max size of a field by one character, for
> > example, or even switching from storing only numbers as a product ID to
> > permitting alpha-numeric.

>
> Overconstraint is a problem too, as you indicate. No need to specify
numbers
> only for an ID that the users determine, typically. And even if it is
> system-generated (and thus probably numeric), that's no reason to store it
> only as digits.

Agreed.

> > > The common schema can be a relational engine. I think you'll find it
> much
> > > harder using a tree structure, since rules don't tend to respect
> > hierarchies
> > > very well (you can writes rules about hierarchies, but that doesn't
mean
> > the
> > > rules themselves are easily expressed hierarchically!).
> >
> > Rules can be built by specifying one function to apply after another --
> > ands, ors, comparison operators, booleans, etc. A fork-in-the-road
> function
> > might lead to function A or function B as the next function -- rules are
> > simply trees, even if a simple list tree, right? [I'll admit I didn't
> think
> > that through completely]

>
> A given rule might be seen as a tree, but the set of rules for a business
is
> much more complex than that. And in any event, you're talking purely about
> flow of decisions, not about the data to which those decisions are
applied.
> That forms an overlapping structure. And given the complexity of the
> functions that can be specified, I'd doubt that a simple tree is better
than
> a full language of some sort. Yes, parsing is more difficult, but it only
> has to be written once.

>

> > > I'd be interested to
> > > hear more about the common services for applying them... what do you
> mean?
> >
> > One thought on it -- With the distribution of processes across machines
> and
> > across the network, there are times when you really need to use a
> > synchronous service (or "subroutine", for example), but more often times
> > when an asynch service would be fine (the software analogy to voice mail
> > being highly underused in software applications). I'd like to see a GUI
> > with a message window that doesn't keep interrupting the user like
dialog
> > boxes do (hate those things!) but provides information, warning, and
error
> > messages.
> >
> > So, for example, if some aspect of the UI cannot be set up to ensure
that
> > the user can only pick a valid option (by providing a list, for
example),
> > but must be validated after the user enters data, then after the user
> > addresses that field, but before an entire document is submitted and
> without
> > tying up the user, the GUI could be a client of a constraint engine
> service
> > to validate that field. After that service is completed, any messages
> from
> > the validation could go into the message window, along with messages
from
> > other asynch services. The user might then find that they entered an
> > invalid value somewhere before they submit the page.
>
> Absolutely - and if they change the value to a "good" one prior to the
first
> validation being done, the user should never see a message.

>

> > When they submit a page, the process will know whether it has all data
> back
> > from the validation services and, if so, will know whether the data has
> been
> > validated or not.

>

> Absolutely. This is where declarative constraints become absolutely
> essential - the engine should be able to do all of the above, with little
> work from the developer other than stating the required constraint, and
> perhaps a little about how the result is to be reported.

That works for me. I don't care if the constraints are declarative as long as the application can use a service to get what it needs. That service (future-style DBMS, perhaps) can have declarative constraints, XML docs, OO objects or whatever as input to the constraints engine.

> > > > I just haven't seen the fruit from that relational seed
> > >
> > > None of us have, although I'd still be interested to hear more from
> those
> > > who dealt with old network databases as SQL was coming to the
> forefront...
> > > SQL was embraced as solving problems, but did it? Did it solve some
and
> > > introduce new ones?
> > >
> > > > and now that it is almost dead, I think I'll skip it ;-)
> > > > [yes -- just pokin'] Cheers! --dawn
> > >
> > > Well, unfortunately (for you) I don't think it's dead yet, and have
> great
> > > hopes for its revival.
> >
> > only if it is greatly fixed, and my take is that the starting point for
> > fixing it is not the current RDBMS implementations.

>

> Agreed!

>

> > As disturbing as almost
> > the entire IT industry seems to think XML to be from a theory
perspective,
> > it aligns much better with the way that users think of their data
(primary
> > due to its lack of 1NF and variable length strings that might or might
not
> > have tight constraints).

>
> Any SQL DBMS worth its salt can handle variable-length strings, but I
still
> disagree about the XML. I have yet to see a case where XML didn't
completely
> annoy both end user and developers.

I can sympathize with that, but having coded many JCL (job control language) "card decks" in a previous life, as well as passing lots of common-quote files around, I'm fine with replacing parm declarations and data passing with XML documents. It lacks elegance on the one hand, but provides a viable standard that is AT LEAST NOT FLAT RELATIONS!! (ducking)

> It's used because it's widely-discussed;
> that's all. Software developers, for some reason still unknown to me,
jumped
> onto the text markup bandwagon and sold the farm to do so. It's biting the
> asses' asses as we speak - hopefully it will be recognized as the snake it
> is, rather than just pushing those with I.T. budgets to further offshore,
> because gawrsh, this I.T. stuff is just too darned expensive (just like
> education - it is expensive, but consider the price of the alternative).
>
> - erk

I really, really would like to see some of the expense of data handling removed across the board in our industry and it appears to me that it is in the interface with the SQL-DBMS that there is a huge cost that could be lessened. But that's just a hunch at this point ... still workin' on it. Cheers! --dawn Received on Wed Jun 02 2004 - 21:16:35 CDT