Oracle FAQ | Your Portal to the Oracle Knowledge Grid |
Home -> Community -> Usenet -> c.d.o.misc -> Re: Strange precision related math errors...
for a,b and c you have precision and scale set: NUMBER(p,s)
but not for a/b or (a/b)/c they are as just NUMBER.
use CAST(expr AS type_name), for example
select a,b,c, CAST((a/b)/c as number(16,4)) from test;
Alexander.
"Andreas Koch" <mail_at_kochandreas.com> wrote in message
news:9ot325$vcj$03$1_at_news.t-online.com...
> Hi,
>
> i have a table test with
> a number(14,2)
> b number(14,2)
> c number(16,4)
>
>
> If i (example from mind - numbers may be too small)
>
> select a,b,c,a/b,(a/b)/c,1-((a/b)/c) from test
>
> i get for example
>
> 19558001,1000000,1.9558,1.9558001,1.000000051129,-5.1129
>
> well, all but the last value is correct.
> As soon as ((a/b)/c) has more than 4 0's before relevant
> digits, 1-((a/b)/c) does neither return -0.0000000051129
> nor 0 (what i would have expected), but shifts the
> result left til it is larger(abs) than 1 !!!
>
> This happens on both 8.0.4 and 8.1.7 ...
>
> What is the best way to get correct results without
> redefining the tables.
>
>
>
> --
> Andreas
> Who doesn't live can never die
>
Received on Wed Sep 26 2001 - 18:02:31 CDT