Re: The wisdom of the object mentors (Was: Searching OO Associations with RDBMS Persistence Models)

From: Sasa <sasa555_at_gmail.com>
Date: Sat, 03 Jun 2006 18:54:51 +0200
Message-ID: <e5sesh$sat$1_at_sunce.iskon.hr>


mAsterdam wrote:

[snip]

I've purposely avoided continuing the first part of discussion, as it seems to me it doesn't lead to something essential or constructive anymore and this posts are getting fairly long.

( Update: it took me almost an hour to write the post :-( this is getting to be exhausting and time consuming)

If you think I've avoided something relevant, feel free to introduce it back to the discussion.

>>>> but in my current view - I regard both primarily as a persistence
>>>> from my OO biased perspective. Why do I usually choose RDBMS over
>>>> flat file? Because of this great features you and other have mentioned.
>>>> Do I treat then RDBMS as a flat file? Most certainly not - I have a
>>>> isolation layer which decouples my OO code from my RDBMS code.
>>>> Behind that layer I can fully leverage the power of RDBMS.
>>>
>>>
>>> That seems like a sensible design heuristic for many applications.
>>>
>>> > From the client side I'm not aware of RDBMS existence.
>>>
>>> If you would have said "The application user does not
>>> for day-to-day use of the application need to be aware
>>> of the existence of the DBMS" I could agree.
>>> Is that what you meant?
>>
>>
>> Not really - the OO code which uses database is not aware of the
>> specific implementation.

> 
> 
> Along these antropmorfic lines:
> 
> For an object to be persistent, regardless of the
> specific implementation, it has to somehow be able
> to provide and package all information, necessary
> for a subsequent incarnation.

Agreed, and this is irelevant of the persistence mechanism.

> At some point in development this ability has to be designed.
> When this has been taken care of one can choose to be agnostic
> about that as long as the common reference does not change.
> In the case of a filesystem the common reference is bytes.

Do I understand correctly that "common reference" signifies that what two system have in common?

Only if a object is stored as a pure byte after byte image?

If you write your isolated mini persistence API with general services, and then a layer on top of it which knows how to persist instances of specific classes using that API, the common reference is entirely represented in the layer.

>>>> When analysing or debugging the data I most certainly use the query
>>>> editor and write ad hoc queries to find out what interests me.
>>>
>>>
>>> Do you also relax the isolation requirement for user ad hoc queries?
>>
>>
>> What kind of app would it be i.e. for what kind of users? I consider
>> SQL statements other than simplest ones to be fairly inappropriate for
>> the non programmer user.

> 
> 
> How do you provide the users with ad hoc query facilities?

Where I had to do this, the 'ad hoc' query feature amounted to combinations of simple filters (which would map to the WHERE part of the query). When multiple filters were used in most cases only logical and was supported. The set of attributes available in filtering was predefined. There was no relational filtering (e.g. give me all orders for customers which satisfy...)
That was more than enough for the end users.

Having these prerequisites in mind, the internal representation of the filter would be made in dedicated classes, representation of its combination would be the collection of instances of such classes, with the isolation layer translating this into implementation specific (e.g. parts of the select statement).
This then makes the user independent of the specific on how the data is
searched.

I didn't have to provide this feature very often, so I have limited experience, so feel free to give your own, more complex real life examples.

>>>> I worked on a legacy app which used proprietary flat files for
>>>> persistence. It was a nightmare when I had to analyze what's in there.
>>>>
>>>> So no, I don't think I'm fully underestimating the power of the dark
>>>> side :-) Sorry just couldn't resist - no harm meant there, I meant
>>>> the power of the RDBMSes. I am open to the possibility that RDBMSes
>>>> can be ever more used than I am currently using them, and are much
>>>> more appropriate for some stuff I currently do in OO. But to accept
>>>> this notion I would need more specific details on your preferred
>>>> approaches.
>>>>
>>>>> Why not? Simply serializing/deserializing all objects
>>>>> (like smalltalk did - the "system image")
>>>>> to a file on secure media gives you perfect persistence.
>>>>> That you want more shows that you already are
>>>>> aware of some desiderata beyond just that.
>>>>
>>>>
>>>> Well to me, most of these other features are rather "persistence
>>>> related".
>>>
>>>
>>> Sharing data can be persistence related.
>>> Naked persistence only gives you very clumsy ways of sharing.
>>>
>>> Please give some specifics regarding "these other features".
>>
>>
>> If I regard entire DBMS as a persistence than all of its features
>> (querying, security, reliability etc.) are obviously persistence related.

> 
> 
> 
> Queries give user meaningful data (or information if you
> care to define etc.) as results, not objects.
> 
> How then is querying data related to objects, or more specifically 
> object-persistence? (m.m. security, reliability).

To expand my statement. Since I use the DBMS primary for persistence services, then

1. Querying the DBMS means querying the persisted data.
2. Security features of the DBMS relate to the persisted data.
3. Reliability applies to the persisted data.

What is in my database is then the data, and there are no notion of objects there. I could think of possible situation where two applications use the same database, one written in OO and the other in non OO language.

> 

>>>>>> Given the fact the DBMS can serve more apps (but actually even one
>>>>>> can mean trouble if SQL code is spread around the app code), and
>>>>>> given the fact that one app might want to switch DBMSes I think it
>>>>>> would be smart to decouple both as much as possible, which to me
>>>>>> means no client code in DBMS, and very localized DBMS specific
>>>>>> statements in client code.
>>>>>
>>>>>
>>>>> "no client code in DBMS" - could you give an example of what you'ld
>>>>> consider typical client code?
>>>>
>>>>
>>>> As can be seen from my other posts, I write my domain logic (the
>>>> core business data and behavior) in OO and use RDBMS for persistence.
>>>
>>>
>>> Don't. If persistence is all you need, a DBMS is
>>> terrible overkill. How do you justify the additional
>>> investment (not talking licenses, but mainly the creation
>>> and maintenance of the staff's DBMS skills)?
>>
>>
>> Most businesses want to have their data at their own network, so the
>> DBMS existence and supporting staff is usually inherently there.
> 
> 
> I'll summarize that as:
> The additional investment is not visible/out of pocket.

Very good summary.

>> Other than that, the features listed above should support the cause.

> 
> 
> Those would be: Normalization, querying, security, integrity, 
> reliability, concurrency, and atomicity, right?

Yes, with additional remark that this list is not ordered in any way, nor is it complete.

>  > But ...
> 

>>
>>>> The data is hence in the database only because it must persist.
>>>
>>>
>>> A secure, backed-up filesytem will do nicely for persistence only.
>>
>>
>> ... it depends on the case (the specific app being developed).
> 
> 
> Could you give an example where a DBMS purely for persistence
> is /not/ gross overkill?

Banking loan application serving couple of hundred clients simultaneously, processing couple of hundred thousands pieces of information (client, loans etc.)

The extreme counter example could be simple address book tuned for only couple of dozens of entries, each entry containing only few attributes (e.g. name, last name, address and phone) which is designed to be used as a single user desktop application. I could imagine using XML for such app.

I suspect this will not shed the light, though. It would be more interesting to see the cases where it is not so clear whether to use simple files or relational database as a storage.

Could you provide your own examples?

> 
> I apologize for being persistent about this - maybe you have
> the impression that you already answered it, but I can't
> make anything out it, so I'll just repeat the question:

No problem, I would like to compliment you for additional explanations. It is easy to misunderstand the intention of the text when not talking face to face.

> "no client code in DBMS" - could you give an example of what you'ld > consider typical client code?

UI would be the most drastical example. The the business model (data and behavior). In my present view - the complex financial calculations in the example below would qualify as the client code.

Business rules and constraints would qualify as the client code (e.g. date of birth must be today or earlier).

> 

>>>> Moving the behavior to the DBMS would mean moving client code (the
>>>> non DBMS code which somehow directly or indirectly uses the
>>>> database) to the database.
>>>>
>>>> I was working on a financial application once, where a lot of such
>>>> logic was implemented in the SQL database. There was number of large
>>>> stored procedures which performed complex financial calculations.
>>>> The code looked horribly - with lot of verbose, bulky IF THEN ELSE
>>>> blocks, etc. it reminded me more of my BASIC days than of the
>>>> professional code.
>>>>
>>>> In addition, the app was being distributed to different clients.
>>>> Between any two clients some subset of the calculation algorithm
>>>> varied to some degree, with most of the logic being the same.
>>>>
>>>> This was being resolve by simply switching the stored procedures
>>>> code - for each client we had the SPs scripted. The structure being
>>>> used (tables) was the same.
>>>
>>>
>>>
>>> So, you had some server-side, customer specific code
>>> (I say customer here to disambiguate from client as in client/server
>>> architecture). You did provide your customer with the schema DDL
>>> in the distribution package, didn't you?
>>
>>
>>
>> Yes, where we were the customers (in your dictionary). The end user
> 
> 
> :-) At c.d.t. I /do/ maintain a glossary, and 'client' is not
> one of the entries because we never had a real misunderstanding
> about the term. I still don't think it is necessary to enter it.

What is the client in your language?
I use the term for everything which is being served by something else. E.g. if function a uses function b then a is the client and b is the server. The end user is the client of the app etc.

This is probably to broad of a usage, sometimes my coworkers don't understand me (especially when I use server for something that doesn't represent web or database server).

>> would receive entire app (client code and database). There were
>> multiple end users. So I think you misunderstood me a little (or I
>> didn't describe it accurately).

> 
> 
> Let's see (below).
> 

>>>> So if we wanted to work on an app for another clients, we would
>>>> execute appropriate scripts which would then regenerate SPs and then
>>>> we would have the appropriate code. Needless to say, there was fair
>>>> amount of duplication (or rather n-plication) with slight variations
>>>> in some places.
>>>
>>>
>>> Customer specific issues at the client-side have to be
>>> resolved as well. I don't see a fundamental difference.
>>
>>
>> Now I don't understand this statement. In this passage I was pointing
>> out the duplication problem.
> 
> 
> Maybe an additional question helps:
> You did have some server-side, customer specific
> code (some stored procedures).
> Did you have n-plication at the client-side?

I understand now (I think) - you mean to say that n-plication is not related to the database (right?) and I can agree. However as I stated in some other post in this thread (no idea whatsoever which one exactly :-( ) I think OO has some very powerful mechanisms to make reusability easier.

As for the SQL (maybe my problem is that RDBMS to me implies SQL) - the language has some very elegant constructs which would take days if not more to achieve in OO. However, when writing sequential code, you basically have BASIC like possibilities at your disposal. No stuff as polymorphism, interfaces, classes or structures etc.

>>>> To summarize - there are two problems:
>>>> 1. Complex ifological sequential logic
>>>
>>>
>>> Please elaborate.
>>
>>
>> Caculating algorithm consists of many sequential steps, after almost
>> any step if else if ... branch occurs, as the algorithm (as defined by
>> end users in their financial domain) is complex per se.
>>
>>> BTW: Where was this in what you are summarizing?
>>
>>
>> In the "lot of verbose, bulky IF THEN ELSE blocks" but very poorly
>> elaborated. Hope this previous paragraph sheds some better light.

> 
> 
> It does, thank you.
> I see what you mean - not to be miscontrued as agreement :-)

Will not be :-)

> I will not go into the sequential/non sequential nature
> of infological complexities here. This post is already way
> to long. Maybe later. Maybe in another thread.

As you please. I'm sorry though, as this is in my view one of the more important reasons why I prefer OOL to RDBMS for domain logic. I would be very glad to hear some oppinion from the "opposite camp" (don't take this term to seriously).

>>>> 2. Code reuse which supports variations on appropriate places
>>>
>>>
>>> There may be customer specific code on both client and server.
>>> You either prevent that or you don't. How does this relate to
>>> persistence?
>>
>>
>> This statement again I don't understand. The problem I was referring
>> to is that end users use the same app (visually) with same business
>> entities, and almost the same business rules (in this case financial
>> calculations).
>> How do you promote maximal reuse via DBMS?

> 
> 
> Maximal reuse is not a goal in itself.

No, but it's a desired property.

> By viewing the DBMS as a persistence
> engine you condemn yourself to having
> lots of redundant code. Some of it redundant
> because it is elsewhere, but because it isn't
> at all necessary. A lot of business rules

Can you elaborate on that, or provide some example?

> (in theory; some say all, also in practise,
> I don't know about that) can be implemented
> as constraints and data-derivations.

What do you mean by the term "data derivations"?

>>>> Now, I have fairly good idea how to resolve these in OO, but have no
>>>> idea how do you deal with such issues in DBMSes. Simply, when it
>>>> comes down to the behavioral, sequential code, and as far as I know
>>>> it has to come down to it sooner or later, I regard OO as a better
>>>> choice.
>>>
>>>
>>> IMO There is no choice here.
>>> There is just appropriateness of metaphors and discipline.
>>> Pretending there is a choice here culminates in a
>>> counterproductive us vs. them attitude.
>>
>>
>> What do you mean? There is a choice as there are multiple paradigms,
>> languages, technologies and tools available to solve the problem. You
>> have to select which ones will you use and distribute the
>> responsibility to each one selected. Is that not the act of choosing?
>> Given the fact that there are multiple possibilities isn't one faced
>> with a choice?

> 
> 
> Let's step out of the box: Given that there are so many different 
> musical instruments isn't one faced with a choice? Sure - at the time
> one chooses an instrument to play. The drummer doesn't have to know
> about ii-V-I chord sequences, the piano player doesn't need to know
> about paradiddles. At time of making music, though, it is just a
> matter of orchestration, listening very carefully and playing together.

I'm not sure I follow your analogy properly. If the instruments correspond to the technology, then it backs my claim. You have to conciously choose which instruments will you use and what will each one do at each specific time. They don't just play like that (assuming you're not refering to the improvised jam session). There is a composition (the task), the band selection (selected technologies), the arrangement (detailed responsibility division and implementation specifics of each) and the actual performance (the running app).

I'm still puzzled to your notion that there is no choice (I assume I just cannot fully understand what are you trying to say).

> 

>>>> Disclaimer: I can agree that the example just presented was simply
>>>> bad implementation (it most certainly was).
>>>
>>>
>>> Let's talk about good implementations :-)
>>
>>
>>
>> That's what I want to hear. How do you envision the role of DBMS?
> 
> 
> For that to be discussed we need more common ground.
> Sorry if this sounds arrogant, but at this point I really
> have to put a lot of effort into making a lot of huge
> moves around (what I see as, but might not be)
> basic misconceptions.

No problem, when you find it fit, introduce the answer to this.

>>>> Do you think OOLs are inferior to RDBMSes in all aspects,
>>>> and if so are they redundant?
>>>
>>>
>>> No I don't.
>>
>>
>>
>> Great (would be also great if you had answered with a yes)!
>> What are then in you view strengths of OO, when is it appropriate and
>> when not?

> 
> 
> How to organise your code?

Could you be a bit more specific?

> OO is strongest when it comes to partitioning large software projects.
> OO gives a nice set of metaphors, to make it easy to achieve
> modularity (like in Myers-Constantine etc..SASD, cohesion, coupling)
> in a relatively easy, but more importantly, commonly understood way.
> 

>> The same question for RDBMS.
>
> How to organise your data?

Could you be a bit more specific?

>> How do you divide responsibilities between two if they are used in the
>> same system?

> 
> 
> The more important, and nowadays often neglected question is:
> what do they /need to/ have in common?

And what is your answer to that question?

Sasa Received on Sat Jun 03 2006 - 18:54:51 CEST

Original text of this message