View Single Post
  #1   Report Post  
Posted to microsoft.public.excel.programming
animal1881[_7_] animal1881[_7_] is offline
external usenet poster
 
Posts: 1
Default Rounding Defect?

Here is a tough problem. I have an array of singles (the numerica
kind) that I need to search through to find a match for an inpu
number. The input number and the array numbers are decimals of typ
"00.0", but stored as a singles. I search like this:

If Round(Single(i) , 1) = Round(Input , 1) Then
Msgbox(Single(i) & " " & Input)

The numbers in the msgbox are different!!! It matches 50.6 with a
input of 50.5. I have also tried FormatNumber(# , 1), and I trie
Int(10 * #) * .1 but those give the same problem. So, I lined up th
numbers in a spreadsheet to I could look at the result of each roundin
method. Using Int() to convert gave many rounding errors, bu
FormatNumber and Round and the Excel Format Cell Function show the sam
result in the spreadsheet, 50.5. According to the spreadsheet I mad
there is no problem. You can see for yourself:

Array (i).Value What I am looking up
V Format(Array (i),1) V format# (Input,1)
50.09999847 50.1 1.25 50.5
50.20000076 50.2 1.25 50.5
50.29999924 50.3 1.25 50.5
50.40000153 50.4 1.25 50.5
50.5 50.5 1.25 50.5
50.59999847 50.6 1.26 50.5
50.70000076 50.7 1.26 50.5
50.79999924 50.8 1.26 50.5

But just asking the computer to compare using an if statement causes
1.26 to be returned and if I look at the Array value that is bein
matched the computer is matching 50.5 to 50.6. Please help

--
Message posted from http://www.ExcelForum.com