Re: MYSQL Error 2013 load infile 15mln rec 6gb CSV

On Mon, 20 Oct 2014 23:11:45 +0100, Eric wrote:

> idiot: someone who disagrees with you on a single point?

Nope. Idiot is someone who can say that hints are not necessary because Postgres is not for profit.

>
> As for needing hints, how often, on any forum, is an OP told to step
> above their actual question and state the goal behind it. Are you sure
> that you are not just like them, assuming that the way to do something
> in product X must exist in the same form in product Y.

Nope. The hints are basically means to override the optimizer decision and require the execution plan to go differently than the optimizer has originally calculated. Insisting that optimizer decision should never be overridden means telling your users that you think that a computer program is smarter than them, when it comes to their data. State of affairs in technology is simply not there yet. Optimizer is right in the 99% of the cases, but for the remaining 1% percent, hints are needed, period. PostgreSQL is the only major database which doesn't have hints. And the idiot I quoted before is trying to make a technical inadequacy, something that PostgreSQL is missing, as opposed to all other major databases, into a phylosophical sticky point. The attitude is even more curious because there already are methods to change the execution plans, only the scope is session, not a single SQL. Unrelated to that, one of the main people designing Postgres is Bruce Momjian who works for the company which sells a commercial version of Postgres which supports hints. One of the replies I received on the mailing lists was "if you want hints, you will have to pay for them". Funny, isn't it? Of course I will pay, only not to EnterpriseDB or other commercial versions of Postgres. If I have to pay for hints, it will be to the makers of Oracle, DB2 or MS SQL Server, who also sell databases which support hints. The makers of those databases do not have problems with hints. One of the points in the article by our former pastry baker is that "commercial databases have hints because DBAs want them". True, if you want me to use your software, you better make sure that it fulfills my needs. And that is why Postgres is remaining an obscure solution which I do not recommend to anyone, unless its developers change attitude and implement hints and some other things that users need.

>
> OK, I use hints because they are the best way (or the only way) to
> achieve a desired situation in Oracle, but I am quite prepared to
> believe that they are not a universal answer.

Your religion is not my problem. There are people who are prepared to believe that they will go to heaven and be greeted by 69.98 virgins if they blow themselves up in a bus full of infidels. Dealing with the occasional optimizer hiccups is my problem. And I need hints in order to be able to do that. In other words, when I have SQL that needs to use specific index or access path, I must be able to tell that to the optimizer. If I cannot do that with the PostgreSQL, I will not use it, period.
As for the "universal answer", it has already been found: "42". The only problem is what is the question. Hints are not the answer, hints are means of controlling the optimizer. The question is whether users should have the ability to control the optimizer or should not have that ability. And the users themselves answered that question by overwhelmingly flocking to the databases which offer them such control, which is everything else except PostgreSQL.

I am just helping Postgres to remain obscure by constantly pointing out the attitude of developers who think that users do not need to have control of their optimizer. Such a condescending and rude attitude is hard to defend. It amounts to telling the users that they are too stupid to control the optimizer. I didn't accept that message too well and, judging by the size of Postgres conferences, neither have the vast majority of the other database users. I do believe in evolution. Ingres has lost to Oracle and Postgres is on its way to lose to Oracle too.

-- 
Mladen Gogala
The Oracle Whisperer
http://mgogala.byethost5.com