Re: Pro*C rounding problem on SUN

Hi,
|> culberso_at_tybrin.com (Mike Culberson) writes:
|>
|> > The problem I am having is when I fetch a NUMBER column from
|> >a table the value referenced in the host variable is not the exact
|> >number value in table. I am working with an accounting system and numbers
|> >need to be exact. Example of code follows.
 

|> >oracle_table
|> >val NUMBER(11,2) with a value of 1.55
 

|> > EXEC SQL SELECT val
|> > INTO :value
|> > FROM oracle_table;
 

|> >If val is 1.55 the value printed out would be
|> >1.55000000000000004440892098501

Well, you didn't say how you defined value, but I guess from the precision of your result (17 valid digits, approx 56 bits), that it must be some kind of (64 bit) long float. Since floats are always stored internally in a binary format, I'm sorry, but I think you'll have to live with this kind of errors when converting the numbers back to decimal.

However, if you're really in bad need of absolute precision, and are ready to pay lots of performance for it, you should take a glance at :

• calc
• C-style arbitrary precision calculator (version 2.9.0)

It should be available at your nearest unix-ftp-server ( or contact

    dbell_at_canb.auug.org.au
    chongo_at_toad.com {uunet,pyramid,sun}!hoptoad!chongo)

and provides arithmetic functions with arbitrary precision, using strings of digits for internal representation of integers and fractions for rational numbers (which should be sufficient for accounting purposes).

It is implemented as a kind of desktop calculator, but also provides a library of the internal procedures for general use.

Still left to the user is the problem of a) adapting the software to this library b) exchanging the values with oracle.

Hih, Andreas Received on Tue Dec 20 1994 - 09:50:10 CET