Re: MV Keys

Bob Hairgrove wrote:
> On 3 Mar 2006 16:36:13 -0800, "dawn" <dawnwolthuis_at_gmail.com> wrote:
>
> >Bob Hairgrove wrote:
> [lots snipped...]
>
> >> Of course, the finest database does nothing until
> >> some application *uses* it.
> >
> >Yes, a database would not be production software without having a means
> >for data to get in and out. A database is part of one or more
> >solutions, not one we can lop off as an entity unto itself.
>
> Not the database as a whole, but the particular view, or perspective
> of the data as seen by any one solution. There is a big difference.

One database is typically used by multiple apps. One app often uses multiple data repositories, possibly multiple dbms products.

> >> The fact that applications can reside in
> >> the database itself in the form of scheduled jobs which run stored
> >> procedures, etc. shouldn't blur the distinction between the two
> >> ("application != database" ... say this 120 times every day when you
> >> get up in the morning, then maybe you will see things a little
> >> clearer).
> >
> >Applications include one or more databases as components of an overall
> >solution.
>
> No, they include their particular views of the data, not the database
> itself.

Yes, I realize I was inciting such a response ;-) Databases are components of one or more applications, but not proper subsets of such.  A client node might be a component of an application too. I'll grant this might be an improper use of the word "component." I don't know.

> >Databases can be incorporated into one or more applications.
> > Does that cover it for you? If not, what is inaccurate?
>
> The fact that one database can be used by different applications with
> different priorities, objectives and constraints means that the model
> cannot cater to one application or the other in an effort to reduce
> complexity.

I almost agree with that statement. My hesitation is that this type of thinking can lead to the mistake of trying to create one huge enterprise logical data model in the absense of any particular purpose for any aspect of it. Each data element should be added to a model only by way of a project that requires it. While I like to have some thinking-beyond-the-end-of-our-noses on a project, when a data element is added, it should be defined in light of the glossary for that project, mitigating risks of future anticipated changes to an appropriate extent.

For example, if we are writing software for a college and we start with financial aid software, we might need to have the concept of a full-time student defined very precisely since only full-time students can receive certain scholarships. If we define that only for the FA app, we will then undoubtedly run into difficulty when writing an accounts receivable system where we need to determine how much to charge a full-time student as defined by some other campus office. So, when writing an FA app, it is good to take into account that multiple areas of the enterprise need to be in step about what a full-time student is.

However, the most critical aspect of this particular FA project is that we need to address the requirements of this particular project. If we try to anticipate every nuance for future projects, we could end up with one of those projects where the requirements grow vastly during the design phase and the project never ends. It is not an impossible situation for the next project to rework what we done in a prior iteration in accordance with the requirements for that project, even if more expensive post-implementation. And, yes, I've read Barry Boehm and others on this topic and I understand that if you get the requirements right up front yada yada yada

> This is why most applications which use some RDBMS today
> will typically only see views of the data specific to that application
> and not work directly with the tables.

I've seen plenty of reporting or BI approaches that use only views of the data and not base tables, but have not seen any successful deployments of enterprise software that performs all updates through views. I don't doubt that there could be some, but that is not my experience.

> An application need not know,
> for example, that the data they see in a certain column is actually
> calculated on the fly by a function in the database.

In theory, it need not know stored data from derived data for read-only, but are you suggesting an application attempt to update stored data via derived data? Even if that were feasible, it doesn't sound smart.

> The same goes for
> related lists which are easy to generate from a normalized schema. The
> opposite might hold true for one application, but not for another.
> Therefore, we normalize the tables to provide for a consistent
> handling of the data by many different applications.

I understand the thinking behind that and am in full agreement with functional-dependency normalization. We need not always remove remove attribute lists, however.

> >> The RM is a model for storage and manipulation of data which can be
> >> used by very many different applications, and the applications need
> >> not know anything about each other.
> >
> >Agreed.
> >
> >> That is the beauty, and the
> >> strength, of the concept. It is up to the application to provide an
> >> interface to the user which is easy to use; this shouldn't be a
> >> requirement of the database!
> >
> >Agreed.
>
> You agree that it shouldn't be a requirement of the database? Then why
> did you say:
>
> >"Surely you can implement a list, for example, using the RM, but the
> >tool is not doing much work for you. It doesn't have enough power.
> >It isn't simple enough from a user standpoint."

There are two different users here - the end-user of an app and the developer using the database. I was referring to the latter. I agree that the application provides the interface to the end-user. It makes sense for the dbms to provide attributes with list values to its users.  If I had not worked with embedded lists, I might miss the elegance of this approach too, however.

> It sure sounds to me like you are criticizing the RM for not being
> simple enough for users...

I would like it to provide better productivity for developers (as users), so yes. If there is something here that sounds like a contradiction, let me know again and I'll try to state it more clearly.

> >> In most respects, the duty of a good
> >> database design is to *protect the data* from abuse by misinformed
> >> users (and I include application developers here as well).
> >
> >The duties of good software development, whether a database component
> >or any other, include mitigating risks, facilitating quality data, etc.
> > I'll agree it is important to protect data integrity and that we need
> >to take appropriate measures to mitigate risks that anyone on the
> >project team, whether someone writing application code, database
> >constraints or anything else, would corrupt the data.
> >
> >When it comes to software projects, a team is likely more productive if
> >all software development resources, including people who make changes
> >to the database schema and constraints, those who do software builds,
> >etc are on the same project team, with cross-training, working for the
> >same project leader. Some sites are organized to have "teams trying to
> >do projects" and "teams trying to inhibit projects" using bureaucracy,
> >for example. Checks and balances are good, policing with "keepers of
> >the gate" is an overrated strategy IMO.
>
> I agree 100% that communication between the teams is of utmost
> importance. During the Swiss census, we had a situation where the
> DBA's removed a column from a table and didn't tell everyone about it,
> or somehow the information didn't make it through all the channels.
> There was at least one application which broke because of it.

This seems to happen more in shops that have prescriptive schema and have a DBA-like team than in those that employ descriptive schema and where "dba" and "application developer" are simply two hats worn by most software developers.

> Also,
> some of my code in stored procedures worked at the office and failed
> when we delivered for them to sign off on it because someone had added
> a NOT NULL constraint to a column I didn't know about, and the
> reference db at work, which was supposed to be an exact clone of the
> real db, obviously wasn't up to date.

Other than the fact that I don't prefer to work with SQL-style NULLS, this type of problem is not uncommon no matter what the database.

> But these are all *management* issues which have nothing to do with
> the RM per se.

The latter is more likely than the former in a non-RDBMS environment in my experience, but sure.

> Although such incidents are unfortunate, they could
> just as easily have happened with some other database model. Also, any
> antagonisms which arise between DBA's and application developers are
> management or personnel issues.

But it is much more rare to have such issues in an MV/Pick shop.

> Why do DBA's implement audit trails,
> for example? Do you think that every user and application developer
> should know (a) that audits are in place, (b) the tables where the
> audit data is stored, and (c) passwords to gain read/write access to
> that data? Surely not! Some things should remain hidden from all but
> the sysadmin.

Who has access to what can be determined on a project by project basis, but I agree in concept.

> >> A good database design is usually never "simple enough" to the average
> >> user. But it really doesn't matter because "simple" shouldn't be a
> >> requirement for database design.
> >
> >As simple as possible, but no simpler, right?
>
> Yes.
>
> >> At best, it might serve as a
> >> requirement for a user interface design.
> >
> >I'm asking that people think of the API or language used for working
> >with a database and its schema as a "user interface" too. If those
> >working on DBMS software see software developers as their users, I
> >suspect we can improve both productivity and quality of all aspects of
> >the resulting software.
> >
> >I know I'm saying this with a programmer's hat on, but I think we need
> >to move more in that direction. I know how to think about it from a
> >"we must protect the database and those programmers are requesting
> >changes that will muck it up" viewpoint too, but I figure this group
> >already has too much of that perspective already. Cheers! --dawn
>
> As you can see, I have also been on the other side of the fence. It
> has nothing to do with the model itself, but everything to do with
> management and attitudes, as well as a lack of understanding of what
> is important on both sides of the fence.

Oddly enough, it really does seem to have something to do with the database. It might not be the data model, per se, but I perhaps one key difference here is related to the descriptive, rather than prescriptive nature of the schema. If a new project wants to describe data differently than a prior one, it can add a new description of the data (as a synonym, for example), which sounds dangerous, but has a different set of risks. The corporate cultures in some ways reflect the dbms tools, it seems. That's just an opinion based on my experiences plus anecdotes of others and not a scientifically-arrived at conclusion, however.

Cheers! --dawn Received on Sun Mar 05 2006 - 05:44:01 CET