Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1   Report Post  
Posted to microsoft.public.excel.worksheet.functions
external usenet poster
 
Posts: 11,058
Default Excel 2003 Linest Function Bug

Your solution may be ill-conditioned. Plot your data and it appears nearly
linear.

If you fit using only two parameters (slope & intercept) you get
a = -9.62307E-05
b = 0.000329459



2499.89 -0.24042 -0.240236766 3.35746E-08
4499.83 -0.43274 -0.432692441 2.26186E-09
6499.77 -0.62507 -0.625148116 6.10209E-09
8999.69 -0.86558 -0.865717228 1.88316E-08
11499.83 -1.10609 -1.106307511 4.73112E-08
13499.91 -1.29861 -1.298776658 2.7775E-08
15999.74 -1.53922 -1.53933711 1.37148E-08
18499.93 -1.7798 -1.779932205 1.74781E-08
20500.05 -1.972495 -1.972405201 8.06385E-09
21999.93 -2.11685 -2.11673974 1.21573E-08
2499.89 -0.2404 -0.240236766 2.66453E-08
4499.83 -0.43274 -0.432692441 2.26186E-09
6499.77 -0.62507 -0.625148116 6.10209E-09
8999.69 -0.865535 -0.865717228 3.32071E-08
11499.83 -1.106125 -1.106307511 3.33104E-08
13499.91 -1.29861 -1.298776658 2.7775E-08
15999.74 -1.539235 -1.53933711 1.04265E-08
18499.93 -1.77985 -1.779932205 6.75762E-09
20500.05 -1.97241 -1.972405201 2.30297E-11
21999.93 -2.11683 -2.11673974 8.14689E-09
2499.89 -0.24043 -0.240236766 3.73393E-08
4499.83 -0.43277 -0.432692441 6.01539E-09
6499.77 -0.62515 -0.625148116 3.55003E-12
8999.69 -0.865625 -0.865717228 8.50604E-09
11499.83 -1.106175 -1.106307511 1.75593E-08
13499.91 -1.29868 -1.298776658 9.34286E-09
15999.74 -1.53924 -1.53933711 9.43036E-09
18499.93 -1.779895 -1.779932205 1.38419E-09
20500.05 -1.97244 -1.972405201 1.21097E-09
21999.93 -2.11684 -2.11673974 1.00521E-08
4.4277E-07
Where the third column is the linear model and the fourth column is the
square of the error.

It may not be meaningful to go beyond two parameters.
--
Gary''s Student


"JimK" wrote:

The Linest Function has severe errors when curve fitting my data as follows:

X Y
2499.89 -0.240420
4499.83 -0.432740
6499.77 -0.625070
8999.69 -0.865580
11499.83 -1.106090
13499.91 -1.298610
15999.74 -1.539220
18499.93 -1.779800
20500.05 -1.972495
21999.93 -2.116850
2499.89 -0.240400
4499.83 -0.432740
6499.77 -0.625070
8999.69 -0.865535
11499.83 -1.106125
13499.91 -1.298610
15999.74 -1.539235
18499.93 -1.779850
20500.05 -1.972410
21999.93 -2.116830
2499.89 -0.240430
4499.83 -0.432770
6499.77 -0.625150
8999.69 -0.865625
11499.83 -1.106175
13499.91 -1.298680
15999.74 -1.539240
18499.93 -1.779895
20500.05 -1.972440
21999.93 -2.116840

D C B
A
4.384643E-17 -4.849642E-12 -9.613158E-05 -7.173873E-05

If you change the first 'Y' value to -0.240419 (the real value), the Linest
Equation blows up and is wrong, (Value of 'D' is zero). The results are
below:
D C B A
0 -4.83784E-12 -9.61317E-05 8.03804E-05

Is there any fix by Microsoft in the works or is there something I can do to
ensure data sent to the customer is correct?
--
Jim
National Research Council of Canada

  #2   Report Post  
Posted to microsoft.public.excel.worksheet.functions
external usenet poster
 
Posts: 20
Default Excel 2003 Linest Function Bug

I guess I should clarify the accuracy required. These are high precision
Load Cell calibrations which we hope are in the order of 0.005 percent
accurate. While the curve is almost linear, the second order is very
important and the third less so, but we supply the equation to our
customers. It would be nice to know that it is accurate.
--
Jim
National Research Council of Canada


"Gary''s Student" wrote:

Your solution may be ill-conditioned. Plot your data and it appears nearly
linear.

If you fit using only two parameters (slope & intercept) you get
a = -9.62307E-05
b = 0.000329459



2499.89 -0.24042 -0.240236766 3.35746E-08
4499.83 -0.43274 -0.432692441 2.26186E-09
6499.77 -0.62507 -0.625148116 6.10209E-09
8999.69 -0.86558 -0.865717228 1.88316E-08
11499.83 -1.10609 -1.106307511 4.73112E-08
13499.91 -1.29861 -1.298776658 2.7775E-08
15999.74 -1.53922 -1.53933711 1.37148E-08
18499.93 -1.7798 -1.779932205 1.74781E-08
20500.05 -1.972495 -1.972405201 8.06385E-09
21999.93 -2.11685 -2.11673974 1.21573E-08
2499.89 -0.2404 -0.240236766 2.66453E-08
4499.83 -0.43274 -0.432692441 2.26186E-09
6499.77 -0.62507 -0.625148116 6.10209E-09
8999.69 -0.865535 -0.865717228 3.32071E-08
11499.83 -1.106125 -1.106307511 3.33104E-08
13499.91 -1.29861 -1.298776658 2.7775E-08
15999.74 -1.539235 -1.53933711 1.04265E-08
18499.93 -1.77985 -1.779932205 6.75762E-09
20500.05 -1.97241 -1.972405201 2.30297E-11
21999.93 -2.11683 -2.11673974 8.14689E-09
2499.89 -0.24043 -0.240236766 3.73393E-08
4499.83 -0.43277 -0.432692441 6.01539E-09
6499.77 -0.62515 -0.625148116 3.55003E-12
8999.69 -0.865625 -0.865717228 8.50604E-09
11499.83 -1.106175 -1.106307511 1.75593E-08
13499.91 -1.29868 -1.298776658 9.34286E-09
15999.74 -1.53924 -1.53933711 9.43036E-09
18499.93 -1.779895 -1.779932205 1.38419E-09
20500.05 -1.97244 -1.972405201 1.21097E-09
21999.93 -2.11684 -2.11673974 1.00521E-08
4.4277E-07
Where the third column is the linear model and the fourth column is the
square of the error.

It may not be meaningful to go beyond two parameters.
--
Gary''s Student


"JimK" wrote:

The Linest Function has severe errors when curve fitting my data as follows:

X Y
2499.89 -0.240420
4499.83 -0.432740
6499.77 -0.625070
8999.69 -0.865580
11499.83 -1.106090
13499.91 -1.298610
15999.74 -1.539220
18499.93 -1.779800
20500.05 -1.972495
21999.93 -2.116850
2499.89 -0.240400
4499.83 -0.432740
6499.77 -0.625070
8999.69 -0.865535
11499.83 -1.106125
13499.91 -1.298610
15999.74 -1.539235
18499.93 -1.779850
20500.05 -1.972410
21999.93 -2.116830
2499.89 -0.240430
4499.83 -0.432770
6499.77 -0.625150
8999.69 -0.865625
11499.83 -1.106175
13499.91 -1.298680
15999.74 -1.539240
18499.93 -1.779895
20500.05 -1.972440
21999.93 -2.116840

D C B
A
4.384643E-17 -4.849642E-12 -9.613158E-05 -7.173873E-05

If you change the first 'Y' value to -0.240419 (the real value), the Linest
Equation blows up and is wrong, (Value of 'D' is zero). The results are
below:
D C B A
0 -4.83784E-12 -9.61317E-05 8.03804E-05

Is there any fix by Microsoft in the works or is there something I can do to
ensure data sent to the customer is correct?
--
Jim
National Research Council of Canada

Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Running Excel 2000 VBA Application on Excel 2003 Excel Worksheet Functions 0 August 8th 06 06:04 PM
FUNCTION GETPIVOTDATA EXCEL 2003 v EXCEL 2004 FOR MAC FRANCISCO PEREZ-LANDAETA Excel Worksheet Functions 0 July 6th 06 12:56 PM
Sharing read-write Excel 2003 files ttt8262 Excel Discussion (Misc queries) 0 April 1st 06 09:39 PM
WHY WAS THE CONVERT FUNCTION DROPPED IN EXCEL 2003??? dan Excel Worksheet Functions 2 May 5th 05 06:03 AM
CHR() function not working in Excel 2003 Richard Jonas Excel Discussion (Misc queries) 4 February 16th 05 08:45 AM


All times are GMT +1. The time now is 10:30 AM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"