Re: Pro*C: Loss of Decimal Places INSERTing From "double" Host Var into NUMBER(16,6) Column

"Fleetie" <fleetie_at_nospam.com> schreef in bericht news:g13b8g$ul9$1_at_aioe.org...
> "Shakespeare" <whatsin_at_xs4all.nl> wrote
>> Or it reserves one digit for the sign..
>>
>> Shakespeare
>
> I don't think so, because a NUMBER(1) column can take negative values,
> i.e. values from -9 to +9.
>
> Here are some more findings I've just made, in the C domain:
>
> ---
>
> Additionally, In C domain, you start losing the 6th place right of the
> point
> after the number to the left of the D.P. exceeds about 2^33=8589934592:
>
> TVTime=(double)8589934590.123456; -> TVTime=8589934590.123456
> TVTime=(double)8589934591.123456; -> TVTime=8589934591.123456
> TVTime=(double)8589934592.123456; -> TVTime=8589934592.123455 <-- Loss
> starts now.
> TVTime=(double)8589934593.123456; -> TVTime=8589934593.123455
> TVTime=(double)8589934594.123456; -> TVTime=8589934594.123455
>
> So with a C double, you can't represent a FULL 16 digits in decimal, i.e.
> you can't represent 9999999999.123456 without losing precision at the
> right.
>
> I guess that's why Pro*C plays safe and rounds to 15 places.
>
> ---
>
>
> Martin
>
>

Ok thank you. Now I see what you mean with 'safe decimal places'. Looks like a bit falling off somewhere ;-)

Shakespeare Received on Thu May 22 2008 - 04:00:35 CDT