Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1   Report Post  
Posted to microsoft.public.excel.programming
Udo Udo is offline
external usenet poster
 
Posts: 48
Default How to avoid rounding when comparing numbers?

Try to simply divide the numbers. If the result is not exactly 1, then
there is a difference.

Udo

Maury Markowitz schrieb:

We're moving data from an Excel sheet into a table in our DB. We copied the
data over in the past, but recently found that it was all cut off at the 4th
decimal place. This did not occur in the DB itself as far as I can tell, the
fields are set to FLOAT, not MONEY.

Anyway I've written a macro to roll over the original data again, find if
there's a single matching row in the DB, and if so, update it. As a sanity
check I look at the values in the DB just to be sure we have to change
something. For instance...

if rstTrans!Value = theTotal then...

this test always passes, even when the numbers are actually different! For
instance, in the first case the number on the left is 1,933.46385, and the
one on the right is 1,933.4639

So how do I get VB to properly compare these things?

Maury


  #2   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 86
Default How to avoid rounding when comparing numbers?

"Udo" wrote:

Try to simply divide the numbers. If the result is not exactly 1, then
there is a difference.


That doesn't really help. When I read the cell value, even though it clearly
has a five-decimal-place number in it, the value returned in VB is four
decimals long. In other words if I try to fix the database, it gets "fixed"
with the wrong number.

Why is it that when I refer to the cell I get a rounded number?

Maury
  #3   Report Post  
Posted to microsoft.public.excel.programming
Udo Udo is offline
external usenet poster
 
Posts: 48
Default How to avoid rounding when comparing numbers?

Most probably the problem is the initialization of the variables. The
following piece of code gives the correct result and shows in the
local-window all the decimals which are appropriate. The numbers I used
we First = 3,4587921, Second = 3,4587919.

Dim First As Double
Dim Second As Double
Dim Result As Double

First = Cells(8, 3)
Second = Cells(8, 4)
Result = First / Second
If Result < 1 Then
MsgBox ("Not identical")
End If

Good luck
Udo

Maury Markowitz schrieb:

"Udo" wrote:

Try to simply divide the numbers. If the result is not exactly 1, then
there is a difference.


That doesn't really help. When I read the cell value, even though it clearly
has a five-decimal-place number in it, the value returned in VB is four
decimals long. In other words if I try to fix the database, it gets "fixed"
with the wrong number.

Why is it that when I refer to the cell I get a rounded number?

Maury


Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
rounding and comparing cells BSantos Excel Discussion (Misc queries) 1 May 7th 07 03:21 PM
Avoid duplicate number entry when only start and stop numbers given. almk05 Excel Discussion (Misc queries) 1 March 26th 06 02:42 AM
How to Avoid Rounding Errors John Pritchard[_3_] Excel Programming 2 January 30th 06 12:40 PM
How can I generate random numbers but avoid duplications? specialk4life Excel Discussion (Misc queries) 1 December 5th 05 05:42 PM
Rounding numbers then doing a sum of those numbers produces incorrect result. GzusRox Excel Programming 2 July 26th 03 01:42 PM


All times are GMT +1. The time now is 03:15 PM.

Powered by vBulletin® Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"