![]() |
Single versus Double
I just noticed that x=Application.Round(0.6, 1) is rounded
when x is double, and isn't when x is single: I get 0.600000023841857 . Is this a bug ? Should I change all my singles to doubles ? |
Single versus Double
I get 0.6 for both single and double, as expected.
-- HTH Bob Phillips (remove nothere from email address if mailing direct) "Sinus Log" wrote in message ... I just noticed that x=Application.Round(0.6, 1) is rounded when x is double, and isn't when x is single: I get 0.600000023841857 . Is this a bug ? Should I change all my singles to doubles ? |
Single versus Double
I think I'd never use Single or Integer--just Double and Long.
Sinus Log wrote: I just noticed that x=Application.Round(0.6, 1) is rounded when x is double, and isn't when x is single: I get 0.600000023841857 . Is this a bug ? Should I change all my singles to doubles ? -- Dave Peterson |
Single versus Double
Singles only have about 7 Digits of precision. when you look at 15 digits,
then the last 8 digits are garbage. particularly when your forcing your single to a double: ? format(cdbl(.6!),"0.000000000000000") 0.600000023841858 -- Regards, Tom Ogilvy "Sinus Log" wrote in message ... I just noticed that x=Application.Round(0.6, 1) is rounded when x is double, and isn't when x is single: I get 0.600000023841857 . Is this a bug ? Should I change all my singles to doubles ? |
Single versus Double
Dave Peterson: I think I'd never use Single or Integer--just Double and Long. I couldn't agree more. I'd rather use a bit of extra memory to play it safe. I sometimes even wonder why Single Precision is still left as a variable option. Myles -- Myles ------------------------------------------------------------------------ Myles's Profile: http://www.excelforum.com/member.php...o&userid=28746 View this thread: http://www.excelforum.com/showthread...hreadid=507121 |
Single versus Double
Maybe to support old code???
Myles wrote: Dave Peterson: I think I'd never use Single or Integer--just Double and Long. I couldn't agree more. I'd rather use a bit of extra memory to play it safe. I sometimes even wonder why Single Precision is still left as a variable option. Myles -- Myles ------------------------------------------------------------------------ Myles's Profile: http://www.excelforum.com/member.php...o&userid=28746 View this thread: http://www.excelforum.com/showthread...hreadid=507121 -- Dave Peterson |
Single versus Double
Dave Peterson wrote:
Maybe to support old code??? Myles wrote: Dave Peterson: I think I'd never use Single or Integer--just Double and Long. I couldn't agree more. I'd rather use a bit of extra memory to play it safe. I sometimes even wonder why Single Precision is still left as a variable option. Myles -- Myles ------------------------------------------------------------------------ Myles's Profile: http://www.excelforum.com/member.php...o&userid=28746 View this thread: http://www.excelforum.com/showthread...hreadid=507121 I checked again in a blank workbook: Option Explicit Public Function z(x As Single) As Single z = Application.Round(x, 1) End Function And in the worksheet: =z(.6) When I enlarge the column, I can see the same 15-digit value I pasted in my first post. Garbage or bug, it looks the same to me. In any case, thank you all for your replies. I am going to eliminate singles and integers. I hope I won't have too many surprises in the process. |
All times are GMT +1. The time now is 05:30 PM. |
Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
ExcelBanter.com