Re: Pro*C: Loss of Decimal Places INSERTing From "double" Host Var into NUMBER(16,6) Column

I think I've figured it out!

This works without loss of precision right of the decimal place:

    TVTime=(double)123456789.123456;
    EXEC SQL INSERT INTO TESTPRECISION (KEY,VALUE)

        VALUES


        (

        -3,
        :TVTime
        );

But this doesn't:

    TVTime=(double)1234567890.123456;
    EXEC SQL INSERT INTO TESTPRECISION (KEY,VALUE)

        VALUES


        (

        -3,
        :TVTime
        );

Notice that left of the D.P., the top one, that works, has one fewer digit.

I think the key to it is what I mentioned earlier about the fact that a C double type is starting to run out of precision at about 16 decimal digits.

I *think* what's happening is that internally, Pro*C, when inserting/updating
from a C double type, truncates at 15 decimal places "to be safe", because I guess it considers that at least the most significant 15 places are "safe",
and after that, things start getting dicey, so it just uses the most significant
15 places.

So in the first example above, there are 15 decimal digits, so even the rightmost
digit is preserved.

In the second example, it discards the least significant digit after rounding
to 15 sig figs.

In the Oracle domain, when it gets sent the full 16 digits as a string, from a string host variable, it knows that the (16,6) column can store the full 16 decimal digits, so it does the conversion and stores the value without loss of precision.

In conclusion, I think Pro*C is "trying to be safe" here, and taking the 15 most significant digits from the double C host variable that it "knows" can be trusted.

Martin Received on Thu May 22 2008 - 03:16:30 CDT