Re: Multiple specification of constraints

"Eric Kaun" <ekaun_at_yahoo.com> wrote in message news:05m1c.30683$u64.30224_at_newssvr16.news.prodigy.com...
> A good discussion - first my response to Dawn's original thoughts:
>
> Dawn wrote:
> > Where should constraints be specified?
>
> In one place, the best place, where they can be specified declaratively,
and
> which won't allow violations of them. If you require violations, your
> consraints are wrong.

Agreed.

<snip>
> Any file system does CRUD. No one must be allowed to update data
independent
> of the "data validation service." Given that we're trying to abstract
above
> the level of a file system, I see no good reason to differentiate the
"CRUD
> services" - that's what the DBMS vendor does.

The reason to do so is not to sever the relationship between the database CRUD services and the validation services, but to ensure the validation services need not require that CRUD requests be directed to them. We need to be able to access validation/constraint information and apply it in cases that have nothing to do with the CRUD services. It is typically too tightly coupled with CRUD to do this today.

>
> > However, if you look at software applications, it is common, even
> required,
> > to have a GUI employ such business rules, for example, in giving users a
> > list of valid codes from which to choose, or even sometimes in finishing
> the
> > data entry based on prior entries (as with Excel) or based on valid
> entries.
> > Additionally, once a set of data is passed to the Validation Service,
> prior
> > to heading to the CRUD Services (whether Validation and CRUD are tightly
> or
> > loosely coupled), these business rules are employed again.
>
> The GUI data "validation" isn't really validation in the same sense,
though
> its rules are derived directly from constraints. I'd say the GUI should be
> "declarative" - rather than validating, give the user no choice to enter
bad
> data. A drop-down list doesn't validate - it's the sole input mechanism
for
> that field.

Agreed.

> > Isn't it rather common, for example, for there to be a drop down box
> locking
> > in the user to a valid set of choices, with the specification for this
> drop
> > down box in one place/language and then these choices again validated
when
> > changes are made to the database? We are then taking our industry's
data
> > (specs/code) and duplicating it, thus making it difficult to maintain
> while
> > keeping integrity in this data.
>
> Agreed - that's the trouble with multi-tier systems, and it's the reason
> code generation is so popular these days. Declared constraints give a
much,
> MUCH stronger basis on which to generate code, boilerplate, data access
> mechanisms, etc. They are for more than just validation - they are are the
> meaning of your application.

Agreed.

>
> > My point is that with the way that software is written today, quite
often
> > data constraints are coded in multiple places, even multiple languages.
>
> With multi-tier systems, they need to be in multiple places, in slightly
> different forms. But that code doesn't need to be written and maintained
> separately.

Agreed.
>

<snip>
> If your data has much value to the organization, then it will be accessed
by
> a wide variety of programmers. Declaring constraints does more than just
> defend against corruption (though that's extremely important). It also
> communicates the meaning of the data and the system.
>
> > If the front-end services are written in Java, for example, then I would
> > prefer that the GUI and the Validation Services make use of the exact
same
> > constraint specs and logic to validate the data against the constraints.

Agreed.

>
> Agreed, the specs should be the same. Their implications within a tier are
> not. Only the GUI needs a drop-down list, for example, but the contents of
> that drop-down are derived using constraints. If you have an HTML and a
> Swing interface to your data, you'll need different code for the same
> drop-down, but it's still the same constraints which define its essence.

Yup.
>
> Separating essence (the constraints) from accident (the implementation
> language) is critical, and I hope it's the "next big thing" in software
> development.

This statement is somewhat meaningless to me -- I'm pretty sure you are not saying that you would not encode these constraints in some language, right? Whether that language is declarative, procedural, OO, or whatever, I just don't care -- that's an implementation detail. It is key that the constraints and the application of the constraint logic (interpretation of specs, if that is how it is implemented) be accessible (as "services" the way I view it) to the database CRUD services and any software application UI services, for example.

>
> > Even if we tap into the specifications that are placed in a DBMS, taking
> the
> > logic of the DBMS that is used to apply the constraints and using that
up
> > front in the GUI isn't happening in current software applications, in
> > general, for at least a few reasons I can think of:
> >
> > 1) The DBMS's logic is applied when CRUD services are launched, while
the
> > logic in the GUI is not ready to launch such services, so there is not
an
> > easy means (or perhaps any means) for the application software developer
> to
> > tap into the DBMS's logic.
>
> There can and should be. The system catalog should tell the GUI most of
what
> it needs to know.

Yes, in a similar way that it is "telling" the database CRUD services what they need to know.

> > This is true for type constraints as well as ref
> > int constraints where the GUI might need to come back to a user and ask
> the
> > equivalent of "Are you sure you want to delete this folder and all of
its
> > contents?" when a user requests that a parent be removed, for example.
>
> That's derivable.

Yup.

> > 2) The application developer cannot easily see the specs and logic
applied
> > by the DBMS to know whether that is the same as what they believe they
> need
> > for this application. So they do not even make the decision on whether
> the
> > constraints they are applying are tighter in this case than what the
> > database spec permits. It is easier for the application developer to
> treat
> > the DBMS constraints as a black box where they try not to bump into the
> > limits, but they don't think of them as related.
>
> Code generation (or direct higher-level language execution) based on
> constraints doesn't give them as much of a choice in the matter.

Of course I would prefer to see it as giving them the full choice in the matter.

> > 3) Reading a database to get constraints when trying to respond quickly
to
> > the end-user is sometimes considered by the application developer to be
> out
> > of the question compared to employing perhaps byte code that runs in an
> app
> > server JVM.
>
> The constraints can be read up-front; JSPs and ASPs are
> dynamically-generated GUIs, and can read whatever they like on the server,
> before the HTML is streamed to the browser. The app build process can use
> the DBMS to generate much, so the code won't run slower than something
with
> hard-coded duplicated constraints. The structure of your database is going
> to change far, far less than its contents.

why would we gen code from the DBMS for use with the GUI and not think of gen'ing code for similar applications of constraint logic with the CRUD services? Simply RUN (not gen) the same services for both.

> > One solid design could be to have all of the constraints specified and
the
> > validations applied in a Validation Service that is able to reuse the
same
> > constraint specifications that the GUI uses and that the CRUD routines
> use,
> > all specified outside of the DBMS.
>
> I still haven't seen a compelling reason to be outside the DBMS. You're
> DESCRIBING a DBMS!. It's more than just a file system!

Fine -- call the entire application software development environment a DBMS. But I haven't seen such an animal yet, so whether there are, in the future, pre-packaged DBMS's that include this functionality or whether we write all such components today, it is still functionality I would not want to compromise.

> > We could code and run all such services
> > in a JVM on an app server (rather than the database server).
>
> And you were worried about speed before? :-)

smiles. Unfortunately whether tuning Java code or Oracle, there is still expertise required to get the desired performance, no matter how much bigger boxes are each year.

>
> > In this case,
> > employing a file system that might not be considered a DBMS would be
> > acceptable provided that we still have the door locked.
>
> And feel like doing a lot more work.
>
> > This can be done by
> > permitting CRUD services on the stored data only via the designated
> > services.
>
> Could be, but why?

so as not to duplicate the constraint specs, logic, and engine

>
> > The constraint specs and logic need to be moved right up to the front of
> the
> > application -- to the GUI.
>
> Depends on what you mean by "moved." Certainly they need to be manifest
> there.

granted.
>
> > I know some have advocated for the database
> > language to deliver the applications, but that tends to tightly couple
way
> > too much of the code, rather than employing a good design pattern, such
as
> a
> > services pattern to the application.
>
> I'm unclear as to why a services pattern is "good," and what the nature of
> the coupling you mention is. I don't see it.

I understand why -- it was poorly stated and perhaps even poorly thought out -- I'll get back to that someday if words to describe my thoughts are less muddy. Sorry not to have provided clean lines for that one. That isn't even a hypothesis -- it is just intuition, so ...
>
> > I'm just thinking out loud and I'm sure something I wrote was not
> brilliant,
> > but help me out here -- isn't this a real issue in application software
> > development written today?
>
> It's the very heart of the matter, and I thank you for raising this
> discussion.

And thank-you for taking the questions seriously and not responding with any "stupid" accusations, Eric!

>
> > I'm suggesting something rather radical --
> > decouple the constraint specification and validation logic from the
> > database. Does anyone agree?
>
> Not I. A database is defined by constraints and validation - decouple
them,
> and you have what we already have: file systems. You have what DBMSs were
> created to abstract above! But again, a good topic.

My intent with the "decoupling" suggestion is not to make it so that CRUD services do not make use of the validation services (multiple negatives there) but to ensure that these validation/constraint services are available wherever they are needed in software applications so they are not duplicated.

--dawn Received on Wed Mar 03 2004 - 20:30:38 CET