Re: A new proof of the superiority of set oriented approaches: numerical/time serie linear interpolation

On 5 mai, 14:27, "David Cressey" <cresse..._at_verizon.net> wrote:
> "Brian Selzer" <b..._at_selzer-software.com> wrote in message
>
> news:8kY_h.7025$2v1.1573_at_newssvr14.news.prodigy.net...
>
>
>
>
>
> > "David Cressey" <cresse..._at_verizon.net> wrote in message
> >news:QvJ_h.11$rk5.8_at_trndny06...
>
> > > "Brian Selzer" <b..._at_selzer-software.com> wrote in message
> > >news:eFF_h.7202$rO7.3983_at_newssvr25.news.prodigy.net...
>
> > >> "Cimode" <cim..._at_hotmail.com> wrote in message
> > >>news:1178260494.811737.293620_at_y5g2000hsa.googlegroups.com...
> > >> > On 4 mai, 02:35, "Brian Selzer" <b..._at_selzer-software.com> wrote:
> > >> >> "Cimode" <cim..._at_hotmail.com> wrote in message
>
> > >> >>news:1178221012.371056.145700_at_u30g2000hsc.googlegroups.com...
>
> > >> >> > On 3 mai, 21:04, "Brian Selzer" <b..._at_selzer-software.com> wrote:
> > >> >> >> "Cimode" <cim..._at_hotmail.com> wrote in message
>
> > >> >> >> > On 3 mai, 18:45, "David Cressey" <cresse..._at_verizon.net> wrote:
> > >> >> >> >> "Bob Badour" <bbad..._at_pei.sympatico.ca> wrote in message
> > >> >> >> > [Snipped]
> > >> >> >> >> Maybe I cut him too much slack. If the past is any guide,
> > >> >> >> >> optimizers
> > >> >> >> >> will
> > >> >> >> >> get better, and Brian won't. It's instructive that Both
> Cimode
> > > and
> > >> >> >> >> Kevin
> > >> >> >> >> provide actual data, while Brian merely states his claim.
> > >> >> >> > For my defense, I am trying to trigge questions here. It would
> > >> >> >> > be
> > >> >> >> > illusional ton my part to hope to get definite answers.
>
> > >> >> >> >> But even if he wins this particular race, he will not
> convince
> > > me
> > >> >> >> >> that
> > >> >> >> >> cursors are the way to go. I've seen too much evidence to the
> > >> >> >> >> contrary.
> > >> >> >> > You will soon realize that Brian mainly tries to convince
> > >> >> >> > himself.
> > >> >> >> > Getting out of procedural mindset is nothing but *natural*
> > > instinct.
>
> > >> >> >> Could you please elaborate? I don't need convincing. Sometimes
> I
> > > dip
> > >> >> >> into
> > >> >> >> waters that a bit too deep and end up chewing on my foot, but
> when
> > >> >> >> I
> > >> >> >> know
> > >> >> >> I'm right, I don't need convincing, and I certainly don't try to
> > >> >> >> convince
> > >> >> >> myself.
> > >> >> > You are in denial. Don't you realize that at least 4 people have
> > > told
> > >> >> > you the exact opposite of your claims and you are refusing to
> admit
> > >> >> > they may just be right. Is 'nt there a slight doubt in your mind
> > > that
> > >> >> > you may be missing something ? that some aspect of fundamental
> > >> >> > theory
> > >> >> > may have elluded you?
>
> > >> >> There is an understandable but unreasonable bias against cursors in
> > > this
> > >> >> group. I wouldn't be surprised if 15 people told me I was full of
> it,
> > >> >> since
> > >> >> most of the time, cursors are used incorrectly or where a set-based
> > >> >> solution
> > >> >> would perform better, and once someone has had a bad experience,
> it's
> > >> >> difficult to set aside emotionalism and examine a similar solution
> > >> >> dispassionately.
> > >> > Have you considered the idea that some people here were once in your
> > >> > exact shoes and they once may had the unshakable belief in what you
> > >> > just wrote.
>
> > >> I only know what I have experienced, and to me it makes sense.
>
> > > This raises the following question: does "what you have experienced"
> > > include what you have learned from other people? If not, what will it
> > > take
> > > to persuade you that some of the writers in this newgroup are onto
> > > something
> > > you might find worth while.
>
> > I admit that I am more willing to trust my own personal experience than
> that
> > of others. And even more so when I have a logical rationale to back it
> up.
> > What will it take? In this case, hard evidence.
>
> > >> Every
> > >> optimizer step is implemented in the engine by using some form of
> > > iteration.
> > >> What I'm suggesting effectively replaces several optimizer steps with a
> > >> single user-defined one that makes a single pass through the
> > >> data--similar
> > >> to what a step that computes aggregates does.
>
> > > I think you are greatly underestimating what a good optimizer can do.
> I'm
> > > judging from a limited understanding of the Rdb/VMS optimizer, going
> back
> > > to 1994. I can only assume that things have gotten better since then.
> > > Even
> > > if they haven't gotten better, I'm sure they haven't gotten worse.
>
> > When you realize that you have to squeeze a batch process that's taking
> over
> > 30 hours to run into a 2-hour window...when you've tweaked, rewritten, and
> > rewritten again a set-based solution, spending weeks pouring over
> execution
> > plans and traces...when you've tried indexes, optimizer hints, forced
> > plans--even thrown hardware at it, and still have a solution that performs
> > better, but not good enough to meet requirements...when you've thrown up
> > your hands and polished your resume...you gain a far too intimate
> > understanding of what an optimizer can and cannot do. The most efficient
> > set-based solution I could come up with still took 7 hours to run--even
> > after adding hardware. But after introducing 5 cursors at key points in
> the
> > process, the execution time dropped to just over an hour. In three
> > instances a cursor replaced a complex section of an execution plan
> resulting
> > from multiple self-joins; in another instance, two cursors executing
> > simultaneously replaced a complex section of an execution plan resulting
> > from several correlated subqueries.
>
> And yet, someone else in this newgroup (not me) reported that he took a
> batch job that was taking 45 hours using legacy code, rewrote it using
> about 300 lines of SQL, and got something to run in about 45 minutes. Do
> you think he was lying? Do you think he was using cursors?
>
> I also have a long history of personal experience to draw on, including
> optimizer workarounds. But I am not as dismissive of what others report as
> you are. Cimode is offering as close to "hard evidence" as anything that
> can be squeezed through the narrow pipeline of a newgroup. And yet you've
> dismissed his conclusion out of hand, without really putting it to the
> test.
Like the pipeline analogy ;) . Sharp (wish I had such proficiency in english but I am no english native speaker).

Don't be too harsh on Brian. Let's keep in mind that online exchange is indeed an uneffective media and that misinformation is a systemic problem promoted by vendors. As we, Brian is just a chain link and a db practitionner who has not realize yet the full potential of set operations. I am positive he speaks of good faith and would be willing to consider all options including the one that he might learn something from some people in this NG. As I said I will try to adapt a more concise formula (if possible based on Vadim's regression formula) to the train stop example to allow Brian to run a few test of his own. My hope is that if Brian simply realizes the possibility of optimization offered in set based approach, he may reconsider his position. If not I would have gained a sharper formula that may prove enligntenning.

> (Cimode, I realize that your main point was about interpolation, and not
> about performance. But it seems to me that, as long as someone dismisses
> your inputs because "he can make it run faster by manual algorithm
> selection" that your flowers are wasting their freshness on the desert
> air.)
Thank you for your friendly warning but do not worry about me. To me, it's a matter of expectations and feedback quality. I have already came to the conclusion that a few good points made by some people here may worth the hassle of accepting noise, off topic or uninformed (unaware?) feedback.

In my quest of refining and characterizing a computing model that would correctly represent relations and relation operations, such quality input is quite refreshing and allows me to double check that I am not following a dead end conceptual lead. Received on Sat May 05 2007 - 16:53:56 CEST