Re: Development as Configuration

From: mAsterdam <mAsterdam_at_vrijdag.org>
Date: Thu, 05 May 2005 23:04:54 +0200
Message-ID: <427a8a77$0$165$e4fe514c_at_news.xs4all.nl>


dawn wrote:

> mAsterdam wrote:

>>dawn wrote:
>>>mAsterdam wrote:
>>>>dawn wrote:
>>>>
>>>>>... and then often SQL for persistence, ...
>>>>
>>>>Why use SQL for persistence? Any filesystem could provide that, no?
>>>
>>>An obvious mAstersam response ;-) so here is mine:
>>>
>>>There are two reasons for using SQL as the API for persistance.
>>>1) If someone else determined at some other time that
>>>a SQL-DBMS would be used for storing the data with which
>>>my app is working or

>>Portability of the interface to your data - not persistence.

> both. From an application developer's standpoint, the API is what
> "does" the persistence.

That is part of the deal: it is like the DBMS says to the application: ok, I'll manage your data, but you have to let me take care of storing it or else I cannot garantee anything.

>>>2) If I determined that there were features of a 
>>>SQL-DBMS that gave an advantage over a simple file 
>>>system so that I chose to use PostgreSQL,
>>>for example, for a given project.  
>>>Because SQL is so pervasive in tools
>>>that have as a PRIMARY feature (smiling) an API 
>>>for creating, reading, updating, and deleting data 
>>>on secondary storage devices, it is often
>>>used in even the simplest database applications.
>>
>>Glad to see you smile :-) But why bother to go the extra kilometer
>>if a filesystem provides perfect persistence? I think it is good to
>>use an interface for the right reasons. I suggest persistence is not
>>the right reason to use a database or SQL. To identify what the true
>>reasons are,

>
> OK, I havn't looked at a list of DBMS features lately, but off the top
> of my head, I want the API I am using for CRUD to be the same as others
> use and to port to other "filesystems". I also want reliability,
> recovery, self-healing in the event of failures, transactions, and
> logging. I want the OS/VM, the underlying data structions, allocation
> of disk space and memory, and any movement of data on secondary storage
> to be optimized by the implementation and not need adjustment using the
> API. I might also want the DBMS to include security administration,
> but I might not care about that feature (preferring not to have to
> address security management in every layer, I might want to handle that
> in the OS and the apps).

Well, you won't escape _some_ security considerations at the level of your visible DBMS, but minimizing them when you have security covered in other layers makes sense.

> I might not want a restriction on what types of data can be stored
> (such as requiring a type to be defined to the DBMS before using it),
> what lengths, etc. and I might not want any constraint handling
> "hard-coded" into the database (catalog), prefering to put it in the
> applications.

Why?
Constraint handling is an important part of what constitutes data management. DBMS have good facilities for guarding against constraint violations. Application programmers have learned to trust those to alert them. You'll have to speciallly train your programmers for your project if you take that away.

You say "applications", plural.
How does one application know what one of the other applications considers a constraint? Accidental violations seem inevitable.

> I might want the DBMS to consider everything it has
> learned about the data as descriptive, rather than proscriptive.

If so you'ld want data monitoring, not data management.

>>just imagine you would have a filesystem for
>>persistence. Which problems will you have? To address those maybe
>>SQL comes back in. But now you have the right reasons, and more
>>realistic expectations.
>>
>>>Sure, there are other features that are important
>>>for DBMS tools, such as security and referential
>>>integrity, but many a software application
>>>has included code for those features too.
>>>To date it appears to me that people license DBMS 
>>>tools for the PRIMARY, not negotiable, feature of having 
>>>an API for creating, reading, updating, and deleting data on
>>>secondary storage devices, even if there are other important
>>>features too.  Right?
>>
>>IMHO DBMS tools can deliver their benefits by encapsulating, hiding
>>storage considerations from the application. Delivering storage to
>>the application is the means, not the end. You don't want a room,
>>you already have it. The DBMS is the locks on the doors and the
>>organization of the room - not the room itself.

>
> DBMS's often give many more features than most companies or developers
> are looking for and some of those features become the prison in which
> they end up living for the life of the software applications using that
> DBMS.
>
> Here is one little picture of a not-so-great contribution a DBMS
> sometimes ends up providing a project during the process of developing
> software:
>
> Imagine construction workers building a house within the constraints of
> having to, at every point during the construction process, meet with
> building code for a ready-to-live-in house.
>
> I'm sure I've gotten off topic here, but I think there are a lot of
> features in a typical DBMS, including a lot of the constraint handling,
> that are road blocks prohibiting behavior that might be the way we want
> to change things or the way we want to work when developing software.
  1. disable constraint checking (on test-tables)
  2. code
  3. spit the data into empty test-tables
  4. enable contraint checking
  5. read the errors (learn)
  6. errors? repeat from 1.
  7. populate from the test-tables

> These same features could be turned into warnings or suggestions for
> the developers instead of stopping developers in their progress. There
> might be features in some DBMS's that permit all constraints to be on
> or off with a "switch" sent to it from an application, but I'm not
> aware of any such feature. There are surely projects where it makes
> sense not to give keys for this type of easy use of the DBMS to
> application developers, but many more where it doesn't make sense at
> all.
>
> Of course this is just one tiny concern about how the requirement for a
> standard API for reliable persistence has taken us down a path that has
> made many software projects much more expensive than they need to be in
> my opinion.

Investing in things you don't need because you already have them does that. Received on Thu May 05 2005 - 23:04:54 CEST

Original text of this message