Generally speaking, you need to:
Decide what your testers are going to test for, and how they will score it. For example, if you
were testing Pizza, you might have the panelists test the crust for lightness, crispness,
doughiness, etc; the sauce for saltiness, tomato flavor, spices; the topping for greasiness, taste,
stringiness....

You may want to train the panelists on how to differentiate between the qualities that you are
looking for.

1) Keep your testers in the dark about how many different products they are testing
2) Randomly present the products to the individials - sometimes product 1 first, other times product
2 first, and strive to keep the products consistent in their presentation
3) Repeat your tests, so that each individual "tests" each product multiple times
4) Hire a PhD Statistician......err,
4a) Look at the mean, variance, spread, standard deviation of your data, based on overall, each
judge, order presented, and over time. Look for trends, and then you need to decide what is
meaningful. For example, you could decide that the mean values need to differ by at least the
largest of the standard deviations, the mean values need to differ by the sum of the two different
standard deviations, etc.

Remember, there are liars, da*ned liars, and statisticians.

HTH,
Bernie
MS Excel MVP


"Pizza" wrote in message
...
I am running a test where a group of individuals will sample 2 products and
provide a preference for one or the other, as well as an overall rating for
each product.
Okay - what test will tell me a) that the preference rating indicates a
"true win" for one product or the other?; and, b) that the average overall
ratings for the 2 products are or are not significantly different?
I have some ideas, but would appreciate any help that's out there!!! THANKS!