LinkBack Thread Tools Search this Thread Display Modes
Prev Previous Post   Next Post Next
  #1   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 5,651
Default Is this rounding error?

In the below routine, are the instances where i=0 due to not being able to
express certain values exactly in binary?

And can it be reliably solved for any 'r' by ROUNDing to 15 places?

Or is there something else I'm missing?

Thanks.

===================
Sub foo()
Dim r
Dim i

For r = 2 To 16

i = Fix(Log(r) / Log(r))
Debug.Print ("For r = " & r & ", i = " & i)
Next r

End Sub
================


--ron
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
rounding error pkeegs Excel Discussion (Misc queries) 2 January 13th 10 07:53 AM
My rounding error in a simple IF garron Excel Worksheet Functions 2 August 6th 08 04:11 AM
rounding error? Steve Excel Discussion (Misc queries) 5 June 1st 08 02:58 PM
Percentage rounding error in charts Tracey Excel Discussion (Misc queries) 4 May 14th 05 04:01 AM
Rounding Error Matthew W Excel Programming 5 April 19th 04 10:11 AM


All times are GMT +1. The time now is 07:11 PM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"