Re: Floating Point Approximations.

From: Bob Badour <bbadour_at_pei.sympatico.ca>
Date: Thu, 29 Mar 2007 00:37:56 GMT
Message-ID: <ETDOh.16737$PV3.172688@ursa-nb00s0.nbnet.nb.ca>

paul c wrote:

> Bob Badour wrote:
>

>> ...
>> The epsilon would be 16 times larger for BCD floats.
>> ie. 5e-7 versus 2^-25

>
> Sorry, not hip to epsilon in this context. Does the above correspond
> with 50 millionths versus 32 millionths, or thereabouts?

Nope: 0.5 millionths versus 0.0298 millionths scaled by the exponent part of the float.

For floating point representations, epsilon is the rounding factor, which is half the least significant digit of the mantissa. If I recall correctly, a 32-bit BCD float has a 6 decimal digit mantissa with a 2 decimal digit exponent. A 32-bit binary float has a 24 bit mantissa with an 8 bit exponent. Received on Wed Mar 28 2007 - 19:37:56 CDT