Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 1
Default Inputbox error? 10 divided by 100 equals 0.100000001490116 ???


Hey, guys.

I was setting up an inputbox for percentile value input, and the
weirdest thing happened. The code follows.

====================
Option Explicit

Sub inputboxtest()

Dim aa As Single

aa = Application.InputBox(Prompt:="Type in 10 for 10 percent", Type:=1)
/ 100
MsgBox aa
ActiveSheet.Range("a1").Value = aa

End Sub
====================

Why is this happening? Msgbox shows 0.1 but the value in the cell A1 is
not the same value.

What am I missing?

Thanks.


--
staying
------------------------------------------------------------------------
staying's Profile: http://www.excelforum.com/member.php...o&userid=23230
View this thread: http://www.excelforum.com/showthread...hreadid=385805

  #2   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 129
Default Inputbox error? 10 divided by 100 equals 0.100000001490116 ???

Hi,
0.1 of Double and 0.1 of Single are not same and not just 0.1.
See
http://support.microsoft.com/default...kb;en-us;78113
Floating-point arithmetic may give inaccurate results in Excel
(BTW, "0001100110011100110011 (and so on)" is wrong.)

You should use Double type for cell values, like:
Dim aa As Double

--
HTH
okaizawa

staying wrote:
Hey, guys.

I was setting up an inputbox for percentile value input, and the
weirdest thing happened. The code follows.

====================
Option Explicit

Sub inputboxtest()

Dim aa As Single

aa = Application.InputBox(Prompt:="Type in 10 for 10 percent", Type:=1)
/ 100
MsgBox aa
ActiveSheet.Range("a1").Value = aa

End Sub
====================

Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Percentage equals 0 error Richard Horn[_2_] Excel Worksheet Functions 7 March 4th 10 02:16 AM
count if equals first non error cell John Excel Worksheet Functions 6 January 12th 10 05:51 PM
Validation - Error message if equals Left formula JICDB Excel Worksheet Functions 6 October 29th 07 08:45 PM
custom filter does not work when selecting 'equals' X AND 'equals' plindman Excel Discussion (Misc queries) 1 June 1st 05 11:29 PM
inputbox msgbox error lost again Excel Programming 3 April 14th 04 07:56 PM


All times are GMT +1. The time now is 07:17 PM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"