Oracle FAQ Your Portal to the Oracle Knowledge Grid
HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US
 

Home -> Community -> Usenet -> c.d.o.misc -> Re: Strange precision related math errors...

Re: Strange precision related math errors...

From: Alexander Z <ziryanov_at_lycos.com>
Date: Wed, 26 Sep 2001 23:02:31 GMT
Message-ID: <b8ts7.12841$w7.2807936@news02.optonline.net>


for a,b and c you have precision and scale set: NUMBER(p,s) but not for a/b or (a/b)/c they are as just NUMBER. use CAST(expr AS type_name), for example

select a,b,c, CAST((a/b)/c as number(16,4)) from test;

                Alexander.

I live and i will .....

"Andreas Koch" <mail_at_kochandreas.com> wrote in message news:9ot325$vcj$03$1_at_news.t-online.com...
> Hi,
>
> i have a table test with
> a number(14,2)
> b number(14,2)
> c number(16,4)
>
>
> If i (example from mind - numbers may be too small)
>
> select a,b,c,a/b,(a/b)/c,1-((a/b)/c) from test
>
> i get for example
>
> 19558001,1000000,1.9558,1.9558001,1.000000051129,-5.1129
>
> well, all but the last value is correct.
> As soon as ((a/b)/c) has more than 4 0's before relevant
> digits, 1-((a/b)/c) does neither return -0.0000000051129
> nor 0 (what i would have expected), but shifts the
> result left til it is larger(abs) than 1 !!!
>
> This happens on both 8.0.4 and 8.1.7 ...
>
> What is the best way to get correct results without
> redefining the tables.
>
>
>
> --
> Andreas
> Who doesn't live can never die
>
Received on Wed Sep 26 2001 - 18:02:31 CDT

Original text of this message

HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US