Oracle FAQ | Your Portal to the Oracle Knowledge Grid |
![]() |
![]() |
Home -> Community -> Usenet -> c.d.o.misc -> Strange precision related math errors...
Hi,
i have a table test with
a number(14,2) b number(14,2) c number(16,4)
If i (example from mind - numbers may be too small)
select a,b,c,a/b,(a/b)/c,1-((a/b)/c) from test
i get for example
19558001,1000000,1.9558,1.9558001,1.000000051129,-5.1129
well, all but the last value is correct. As soon as ((a/b)/c) has more than 4 0's before relevant digits, 1-((a/b)/c) does neither return -0.0000000051129 nor 0 (what i would have expected), but shifts the result left til it is larger(abs) than 1 !!!
This happens on both 8.0.4 and 8.1.7 ...
What is the best way to get correct results without redefining the tables.
-- Andreas Who doesn't live can never dieReceived on Wed Sep 26 2001 - 12:21:26 CDT
![]() |
![]() |