Oracle FAQ Your Portal to the Oracle Knowledge Grid
HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US
 

Home -> Community -> Usenet -> c.d.o.misc -> Sum of double values not the same after fetch from dbase

Sum of double values not the same after fetch from dbase

From: Ahsan ul Haque <ahaque_at_mcmgtm.com>
Date: 1997/05/12
Message-ID: <01bc5ee8$056526b0$6f000059@auh>#1/1

Hello.

Using MSVC 4.1 on NT and Oracle 7.2, (the same situation with MS SQLServer 6.0 seems not to give us a problem), we are experiencing the following problem with the summing of double values.

double variable adSum is a sum of n double values. It has to be compared to adLimitSum, which is another double value.

After the individual values which sum up to adSum are saved to the database and retrieved, the newly calculated adSum value is slightly greater than it was before the save. Hence, the comparison fails.

The representation in the database is "float(53)" for Oracle. The data type in C++ is double for each of the elements.

We need to come up with some tolerance value within which the sum should be. Using the above compiler and database, how should this tolerance be defined? It probably will need to take into account the value of n.  

Any help would be appreciated.

Please e-mail a copy of the response to:  

ahaque_at_mcmgtm.com
Ahsan ul Haque
MCM, Inc. Received on Mon May 12 1997 - 00:00:00 CDT

Original text of this message

HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US