Re: Pro*C: Loss of Decimal Places INSERTing From "double" Host Var into NUMBER(16,6) Column
Date: Thu, 22 May 2008 11:00:35 +0200
"Fleetie" <fleetie_at_nospam.com> schreef in bericht
> "Shakespeare" <whatsin_at_xs4all.nl> wrote
>> Or it reserves one digit for the sign..
> I don't think so, because a NUMBER(1) column can take negative values,
> i.e. values from -9 to +9.
> Here are some more findings I've just made, in the C domain:
> Additionally, In C domain, you start losing the 6th place right of the
> after the number to the left of the D.P. exceeds about 2^33=8589934592:
> TVTime=(double)8589934590.123456; -> TVTime=8589934590.123456
> TVTime=(double)8589934591.123456; -> TVTime=8589934591.123456
> TVTime=(double)8589934592.123456; -> TVTime=8589934592.123455 <-- Loss
> starts now.
> TVTime=(double)8589934593.123456; -> TVTime=8589934593.123455
> TVTime=(double)8589934594.123456; -> TVTime=8589934594.123455
> So with a C double, you can't represent a FULL 16 digits in decimal, i.e.
> you can't represent 9999999999.123456 without losing precision at the
> I guess that's why Pro*C plays safe and rounds to 15 places.
Ok thank you. Now I see what you mean with 'safe decimal places'. Looks like a bit falling off somewhere ;-)
Shakespeare Received on Thu May 22 2008 - 04:00:35 CDT