Re: Precision in numbers

Bruce Murphy wrote:
>
> I work for a research lab where precision is very important. I was asked
> today to allow for a numeric field to be able to allow different precision
> to be held and not be changed by the database. For example:
>
> 3.12 would be added for a sample.
> 3.124 would be added for another sample
>
> The user (engineers) does not want the precision to be lost. If the field
> is defined as a double or float with four decimal places the numbers are
> stored as:
>
> 3.1200
> 3.1240
>
> How can one get the precision to be maintained for both reporting and
> calculations? Thanks for any help!
>
> One thought we had was to use a text field and then use the conversion to
> number, but I don't like this approach.
>
> Bruce Murphy

How about storing the numbers in two columns, one for the number of decimal places, and one for the number converted to an integer?

         Decimals  Number
3.1200 =    2        312
3.1240 =    3       3124

Do any math with the integer components, then use the decimal column to calculate the number of decimal places in the result. If the original, measured number was 10.0 or 1.2 x10^2, then store it as:

            Decimals  Number
     10.0 =    1        100
1.2 x10^2 =   -2         12

I haven't done any advanced math in years, but I think it would work... Received on Tue Jun 03 1997 - 00:00:00 CDT