Re: Multiple specification of constraints

From: Dawn M. Wolthuis <dwolt_at_tincat-group.com>
Date: Tue, 2 Mar 2004 08:24:49 -0600
Message-ID: <c225fl$rob$1_at_news.netins.net>


"Tony" <andrewst_at_onetel.net.uk> wrote in message news:c0e3f26e.0403020320.79552ae8_at_posting.google.com...
> "Dawn M. Wolthuis" <dwolt_at_tincat-group.com> wrote in message
news:<
c20eth$aq7$1_at_news.netins.net>...
> > "Tony" <andrewst_at_onetel.net.uk> wrote in message
> > news:c0e3f26e.0403010301.48d3c24c_at_posting.google.com...
> > > So you currently have 4 choices:
> > > 1) No constraints at all
> > > 2) DBMS constraints only
> > > 3) UI constraints only
> > > 4) Both DBMS and UI constraints
> >
> > Or you could partition the software application services in this way:
> > a -- interface to user and/or "batch" (e.g. web services)
> > b -- constraint/validation services
> > c -- CRUD services
> >
> > whereboth a and c are clients of the b services.
>
> If all you want from a DBMS is what you call "CRUD services", then you
> may as well store the data in flat files. A true DBMS incorporates
> both b and c by definition, and a is a client of the DBMS. If you
> want to avoid duplication of constraint validation, surely it would be
> better to integrate a more closely with the DBMS, rather than decouple
> b from the DBMS?

I honestly don't care what aspects of a software application are provided by a database management service and which are not. If the DBMS becomes an IDE, that's fine. Then the question is not theory, but a company would decide whether to build into this particular vendor to that extent. I am also fine with the application being developed with a file management system (such as an OS) that provides primarily CRUD (with security) services. I just don't want the data constraints (which are a significant part of a software application's specification and logic) to be duplicated and require different skills and often different people to make a change when one is required.

>
> > This means decoupling the CRUD services from the constraint services,
while
> > still ensuring that data that gets stored has been through the
> > constraint/validation services. This possibly puts additional power in
the
> > hands of the application developers to botch up the database in a big
way,
> > but it is not as if that isn't the case anyway. It also tends to put
the
> > data architecture roles together in the application development process,
> > rather than having one role, such as a dba, do data modeling for data
> > storage and another, such as a an application software engineer, do the
data
> > modeling throughout the rest of the application (for the data in the
> > interface with the UI and the web services, for example).
>
> What sort of data modelling in the rest of the application do you
> mean?

The data model immediately behind a UI, for example, is typically a function from a candidate key to a set of elements, many of which are themselves complex types. There is some additional metadata related to accepting data from a user in a particular instance than there is related to the data as stored. So, the metadata (of which constraints and typing informaion is a part) needs to be extended for the UI as well.

> > > 2) DBMS constraints only: this GUARANTEES that the data will never get
> > > corrupted, even though the UI allows the user to enter bad data. It
> > > gives the user a poor experience, but at least the data can't be
> > > corrupted for sure.
> >
> > Nothing guarantees accurate data but if we take a wholistic approach to
data
> > integrity where we want to give the best shot we can at capturing
accurate,
> > meaningful, retrievable, validated, properly interpreted, etc data (not
just
> > what is currently in the DBMS constraint specs) we will give it a good
shot.
> > Good data management crosses the entirety of a software application, not
> > just the "backend".
>
> OK, nobody is perfect, granted. But what I mean is that if a correct
> constraint is correctly defined in the DBMS, the DBMS guarantees that
> it WILL be applied. It is not optional. BTW, do you mean "holistic"?

Yes, oops, I do mean "holistic"! Thanks.

> > > 3) UI constraints only (your favoured option, Dawn?):
> >
> > By no means -- I am not suggesting that we ditch the idea of applying
> > database validation against constraint specifications AT ALL! I simply
> > don't think we (as an industry) are taking a solid constraint/validation
> > approach that makes for both data integrity and software development &
> > maintenance efficiency & effectiveness.
>
> Good.
>
> > > this guarantees
> > > the user a good experience, and guantees that the data will never be
> > > corrupted IF IT IS ENTERED VIA THIS UI. It does not and CANNOT
> > > guarantee that the data will not be corrupted by other applications,
> > > or by bypassing the UI altogether. You have to legislate that this UI
> > > is THE ONLY way to access the database.
> >
> > An application developer could technically bypass the data integrity
engine
> > and employ the crud services in a test environment, with my scenario.
> > However, the production environment could be established so that the s/w
> > developer does not have the rights for that type of access -- the only
> > access to the production CRUD services would be by way of the data
integrity
> > engine. This is not locked down in the same way as it is in a
traditional
> > DBMS and there are times when "traditional" DBMS lock down would be
> > worthwhile but there is more than one way to accomplish our goals.
> > Decoupling, while still interfacing, CRUD services from data integrity
> > services gives some advantages:
> >
> > 1) maintainability, one place to make integrity changes; local & global
> > typing specified using the same tools
> > 2) cost of development; potentially faster and better quality with fewer
> > "people interfaces" (and if you have not seen any example of a poor
> > interface between DBAs and software developers, ...!)
> > 3) cost of ownership, stems from combination of 1 & 2
> > 4) data accuracy, by taking a wholistic approach to data quality, it
shows
> > in the resulting software
>
> Agreed. So create the constraints ONCE, declaratively, in the DBMS -
> tightly coupled with the data they are meant to protect. The issue is
> then how to make the UI aware of those constraints where it would be
> helpful (i.e. to be user-friendly). Duplicating them manually is one
> costly way. Duplicating them by some kind of code generation process
> is better. A Utopian vision would be a UI that was so intimately
> acquainted with the DBMS that it could see what constraints existed
> there and automatically apply them as appropriate in the UI.

When I suggest that the constraints be decoupled, I am not suggesting that they are not employed by the dbms (so maybe "loosely coupled" would be a better way to state it), but that I don't want to have to check in with the storage CRUD services in order to "run" the constraint logic.

The ugliest scenario I can see would be for a database management system to permit specification of constraints for its engine to apply and then make the specification, but not the engine, available for the rest of the application (e.g. UI). Then the application developer has to read the specs from a database management system and duplicate the logic of the dbms (why did we even invest in that dbms then?)

I'm still looking for all parts of the application that need to employ constraint logic to use a) the same constraint specs b) the same logic that interprets the specs and c) the same engine / run-time environment. Today that is possible if the backend storage consists of CRUD services and is not a full blown DBMS, but if it can be accomplished with other approaches, that is fine with me.

>
> > > 4) Both DBMS and UI constraints: this obviously GUARANTEES that the
> > > database will never get corrupted, and ALSO gives the end user a good
> > > experience, which is nice. Of course, there is a cost in duplicating
> > > the constraints: at worst, re-writing them in another language; at
> > > best, pressing the "re-generate UI constraints" button in some tool.
> > > But the 2 sets of constraints serve different purposes: the DBMS
> > > constraints are for data integrity, and the UI constraints are for
> > > enhancing user experience/productivity. The UI constraints are NOT in
> > > any way responsible for data integrity.
> >
> > GUARANTEES?? If the software application has "Date of Birth" as a
"screen
> > prompt" for an attribute that is treated as "Date of Skull Fracture" (my
> > creativity isn't up right now, but you get the idea), then you test the
type
> > of the data all you want and your data is definitely corrupted!
>
> It GUARANTEES that any attempt by the application/user to put a value
> in the Date Of Skull Fracture column does not violate any constraints
> on the Date of Skull Fracture column (or indeed any other constraint).
> Of course if you enter a Date of Skull Fracture in the mistaken
> belief that you are entering Date of Birth, you will enter the wrong
> date. Constraints can only ensure that data is consistent with the
> rules, not that it is "correct" in the real world. If I enter my Date
> of Birth as 01-Jan-1972 (which it isn't), there is no conceivable
> constraint that will say "Error: lying about your age! And I know you
> are a Taurus!"

Yes -- and that shows that data type constraints are but one aspect of data integrity.

> I think we are in agreement that data integrity is vital and should be
> strongly enforced. However, you seem to favour an application-centric
> approach (relegating the DBMS to "CRUD services") where others
> (including me) favour a data-centric approach (relegating the
> application to "UI services", perhaps).

It is the quality of each software application that makes the biggest difference in both the quality of the data and the quality of the maintenance effort. I don't think that the way we partition software applications from DBMS's today yields quality software. I have no problems with a DBMS being where constraints are spec'd, but today the DBMS is doing only a part of its job and there are more downsides to using a DBMS that doesn't handle all aspects of constraint management in an application then there are reasons to use it. So today we would be better off (as a profession), for s/w quality and overall cost, to have developers work with the services partitioning I listed earlier.

It is ironic that we spend so much time designing our data structures so as not to duplicate data entries and the very tools we employ for that are what cause us to duplicate a major portion of our software applications (constraints and their processing).

--dawn Received on Tue Mar 02 2004 - 15:24:49 CET

Original text of this message