Re: 3vl 2vl and NULL

From: Frank Hamersley <terabitemightbe_at_bigpond.com>
Date: Fri, 03 Feb 2006 01:03:17 GMT
Message-ID: <pZxEf.234166$V7.107919_at_news-server.bigpond.net.au>


dawn wrote:
> Frank Hamersley wrote:
>>dawn wrote:
>>>Frank Hamersley wrote:
>>>>dawn wrote:
>>[....]
>>
>>>>>This is where there could be some measurement. My hunch is that there
>>>>>are fewer project failures in the MV world than in the SQL-DBMS world.
>>>>
>>>>The WTF presentations suggest otherwise, although as Murphys Law would
>>>>have it todays gem is SQL related!
>>>
>>>I checked and, yes, this is a typical SQL problem that is rare in the
>>>MV world.
>>
>>I guess in the MV world performance is more likely to be linear and
>>remain so. This can be reassuring to management but the downside is
>>there is limited scope for a "cheap correction" as described in the WTF
>>to bring about a huge improvement. Of course you could hack into the
>>code but then "not broke" resistance will need to be overcome.
>>
>>>Can you point me to some entries at
>>>http://www.thedailywtf.com/ that talk about project failures with
>>>MV/Pick systems?
>>
>>No - my visits to the site are infrequent and haphazard!
>>
>>>I can certainly think of incidents and project
>>>failures with attempts to combine Pick and SQL, but would like to read
>>>some anecdotes about major project failures where no SQL was in the
>>>mix.
>>
>>I have only one first hand observation - a financial accounting system
>>that could not accurately compute simple interest on a call balance. It
>>used to drive the accountants absolutely nuts but the Pickies couldn't
>
> I definitely doubt that

Which bit do you doubt? The (a) lack of accuracy, (b) the frustrated accountants (and auditors for that matter) or (c) that Pick was involved?

>>and wouldn't
>
> this sounds like a peopleware issue, but I do attribute some of those
> to the influence of tools too. One thing that is associate with Pick
> is that often business experts morphed into computer folks since it was

Well that is a better state of affairs than in general IT! Most types I encounter aren't even business experts let alone IT competent!

> easy enough (not as easy today when it is not a single-server with dumb
> terminals). So some Pickies do or did not consider themselves
> computing professionals but subject-matter experts. In that case, they
> would not respond as IT professionals, but might rely on their own
> opinion of whether the change was warranted too much. This was clearly
> not a professional response to such a bug.

True - this is a universal malaise within IT.

>>find a way to correct it - in their minds it was only a few
>>cents here and there - never more than $1.00 per annum so why the big
>>fuss? Why spend lots of time and money hacking the system for a few $
>>when they couldn't be sure what else they would bust re-engineering the
>>accrual mechanism. Eventually the Treasurer got jack of their attitude
>>and bought an external package solution that was SQL based so he didn't
>>have to keep explaining to the board why the organisation routinely got
>>slagged by its somewhat captive customers about its capacity to
>>calculate simple interest. Some of the Pickies subsequently got a
>>Darwin award for their troubles!
>
> It sounds like that was well-deserved.

It was shades of drowning horses - an extinct gene line nonetheless!

>>>>>However, then folks will argue that you pay the price down the road in
>>>>>MV (which I do not see to be the case).
>>
>>My anecdote confirms at least 1 contra example -
 >
> Oh, I could give you a lot more. There are such anecdotes throughout
> the profession, whether SQL-based solutions or not. I am not
> contesting that at all. I guess you could see this as a failed
> project, although not quite the same way I think of it.

Agreed - my continuance on this thread branch was not to pin it to Pick but more to ensure it was not rated as a silver bullet and secondly that the RM was not labeled obsolete. On the latter point we disagree and I expect either stance will take more time than we spend on the NG to vindicated.

>>there is no empiricism
>>though. My strike rate is 100% but you will argue even accepting my
>>case it is still close to 0%. This is reasonable - my purpose is to
>>illustrate MV does not preclude failure.
>
> Oh, I never intended to argue that. I just don't sense the same
> massive failure rate.

Just to restate my view I don't believe the RM failure rate is any more or less "massive" than the MV rate. That the rate of IT failure is massive is beyond conjecture and I suspect for the most part it is due to the carbon rather than the silicon component.

At this point our subjective anecdotal assessments diverge - ce la vie!

>>>>>Measurements would need to
>>>>>measure the total cost of developing and maintaining systems, but also
>>>>>some measure to ensure the systems are effective for the users.
>>>>
>>>>Yup - but IT isn't about waiting around for epidemiological studies to
>>>>complete.
>>>>
>>>>>I'll go out on a limb and say that it is unlikely that a development
>>>>>team working with a SQL-DBMS tool would end up writing the same system
>>>>
>>>>>from the user perspective as a team working with an MV system. The
>>>>
>>>>>data modelers for the SQL-DBMS tool will work to avoid multivalues so
>>>>>they don't have to form new tables for short lists that don't seem too
>>>>>important, for example. I've seen more FormerNames fields in MV
>>>>>systems, collecting all former names as they become such in cases where
>>>>>it would not be warrented in a SQL-DBMS system -- a single-valued
>>>>>"Maiden Name" attribute (or whatever is deemed politically correct now)
>>>>>is more likely.
>>>>
>>>>Interestingly your example would be a non-event if support for the "time
>>>>varying" (temporal) aspect of Codds laws were provided in the various
>>>>main stream SQL implementations.
>>>
>>>Yes, and I'll grant that it might be a better approach than the
>>>one-at-a-time, pick and choose fields about which to keep a history.
>>>However, the "current value on top" pattern is used repeated in Pick
>>>and works well for end-users.
>>
>>How would retrospective states be analysed in your stack oriented record?
>
> If I understand the question, what people do sometimes is instead of
> taking an attribute that would be single-valued in a SQL-DBMS, they add
> to the multiplicity and arity by making it a stack and adding a
> timedate field, so you turn an attribute into a little embedded table
> of value-date. This is often use with status codes. Then you create a
> virtual field (derived data) that is just the current (top) entry for
> today's value, but you can analyze for how long things were pending,
> for example.
>
>>Consider a model where a "Person" was designated as having a "Rank" and
>>that the Rank affected automatically generated "Correspondence".
>>
>>Say an operator of the model incorrectly changed a Persons Rank from
>>"First Officer" to "Captain" and they did not rectify the error for
>>several days.
>>
>>How would you using the MV model alone determine the number of cases and
>>the parties affected by an event where Correspondence for the eyes of
>>Captains only was forwarded to a First Officer?
>
> Did my explanation work to answer that? If not, I can expand on it.

Explanation is fine.

>>I expect you would have to code it, and if insufficient data had been
>>retained by the designer it might prove impossible!
>
> Yes, that is the case.
>
>>This also the case
>>for the mainstream SQL engines today - if it is not designed in it won't
>>be feasible.
>
> It makes sense to me that the requirements of the project are what is
> implemented. What is a little different is that if there is this "hint
> of requirement" in the air, the SQL developer might be less likely to
> include it in the model bz it seems to cost more in that environment
> than in Pick.

Maybe - this is all subject even as to how actively a project is managed. In times of stress a good project manager will of course keep the staff focused on the documented requirements - any extension to accommodate possible requirements is another domain of responsibility. Sadly in some cases very poor choices are made when the differential cost is almost unmeasurable, but the instantiator just uses what they are comfortable with rather than looking for a corporate outcome.

>>However it is an active area of research in academia at least to build
>>integral temporal support into the SQL standard. Progress to date has
>>been very limited based on my review of the papers published so far.
>>
>>>>I guess temporality will never become
>>>>integral in the non RM space, and at the current rate of progress in RM
>>>>research not in the SQL standard very soon to boot!
>>>
>>>I don't think that any future SQL standards are likely to be very
>>>relevant, do you? --dawn
>>
>>Whats the alternative?
>
> I cannot bear to respond "XQuery" outloud, and I'm also interesting in
> seeing what kind changes are spurred by RSS and search engines.

Is is a fad or not - too early to tell IMO.

>> Something like MV standards - ie only those that
>>any particular developer feels like instantiating in their code? Sounds
>>like more anarchy to me! :-)
>
> The balance between enforcement of good standards and room for creative
> excellence is important. One of the reasons that it is likely are more
> agile environment is because the RM or at least SQL-DBMS tools enforce
> unnecessary compliance DURING the development cycle. What I mean by
> that is that in an analogy to construction, if you insist that at every
> point of building or rennovating a house, you must have a working
> toilet, then you unnecessarily tie the hands of the builders in how
> they must do the construction.

I disagree strongly with this "house of cards" concept of construction.

Your association with the toilet is facile (sorry to be harsh) at best given a site portaloo is mandated by regulation (here at least).

Instead what is reasonable to consider is ...

1. the foundations are constructed before walls are built,
2. the services are installed before the slab is poured,
3. the walls are constructed to load bearing capacity (if not complete) 
before the roof is constructed,
4. the roof is covered before the plaster is rendered. ...and so it goes.

All these processes are simply common sense. Neglect them at your peril.   Just ask the builder who fails to install the plumbing before the concrete is poured about how cost effective core drilling is. Builders who stuff up in this way go out of business but in IT there is no reliable sanction mechanism to ensure that this occurs.

Building systems in IT is no different. Where I do have a bone to pick (pun intended) is with a methodology or tool that facilitates or encourages deviant behaviour. IMO agile is not about bending or breaking established rules but about responsiveness.

Cheers,
Frank. Received on Fri Feb 03 2006 - 02:03:17 CET

Original text of this message