Purpose
Establish the power of a discrimination test given a set
sample size or to calculate the sample size required to get a desired power.
This can be done specifying the expected difference as a
proportion of discriminators (Pd) or as a Thurstonian difference
(d’).
- Discrimination.xlsx
- Results of the discrimination test are binary (1
= correct answer, 0 = incorrect answer)
- At
present this analysis requires an uploaded dataset – although this is not used for
the analysis.
Background
The power of a test is defined as the probability of the
test to correctly reject the null hypothesis given the alternative hypothesis
is true. For a difference test, the power of the test is the probability a
difference will be detected, given there is a difference. For a similarity test
the power of the test is the probability that no difference will be detected,
given there is no difference.
Before a test is run it should be ensured that it is powered
correctly. The Discrimination Test Settings modules allow the user to calculate
the power of a test, or to find the correct sample size for a test for a
desired power. The Pd option allows the user to specify test parameters in
terms of the proportion of discriminators (Pd). The d’ (dprime) option allows
the user to specify test parameters in terms of the Thurstonian difference.
Discrimination tests can be conducted using one of 2 models.
- The Guessing model is panellist oriented and
determines the proportion of panellists who can detect differences between
products (i.e. proportion of distinguishers) Pd.
- The Thurstonian model is product oriented. It
estimates the difference between products through a signal/noise ratio (d’).
The dprime is specific to each test protocol.
The power of the test depends on several parameters.
- Protocol: Type of discrimination test (2-AFC,
3-AFC, Duo-Trio, Triangle or Tetrad).
- Type of Test: Similarity or difference test.
- Pd or d’: The relevant level of difference in
the products. Either measured as the proportion of discriminators (Pd) or
d-prime (d’) the size of the difference between the products.
- Significance: Type I error risk (alpha) – the chance
that the null is rejected if the null is true. For a difference test this is the risk of claiming a difference when there
isn't one. For a similarity test this is the risk of claiming equivalence when the
products are different.
- Sample Size: The total number of tests in the
study.
Similarly, the required sample size of test for a chosen
power depends on the same parameters.
- Power: the probability (value between 0 and 1)
of the test to reject the null hypothesis if the alternate hypothesis is true. This is equivalent to 1 – Beta, where Beta
is the Type II error – the chance of accepting the null hypothesis when the
alternate is true. For example, if Beta = 0.2 then the Power = 0.8.
Options
- Protocol: Type of discrimination test (2-AFC,
3-AFC, Duo-Trio, Triangle or Tetrad)
- Type of test: Similarity or difference test.
- Prop of Discriminators (Pd): Proportion of panellists who can detect difference between products.
Or - D-prime (d’): The estimated difference between products.
- Estimation: Calculate the required sample size
for a given power (N) or Calculate
the power achieved for a specified sample size (Power)
- Power: (If Estimation = N) The required power as
a proportion.
- Total Number: (If Estimation = Power) The specified
sample size.
- Alpha: Type I error or false positive rate.
- Number of Decimals: Required number of decimals
for values given in the results.
Results and Interpretation for Sample Size Calculation
- N Exact: The smallest number of samples which
will obtain the specified poer for the test.
- N Stable: Power is not a monotonic function of
sample size (in this case, it is not strictly increasing). Therefore, slightly higher
samples sizes than N Exact will usually have less than the specified power. N stable
is therefore the sample size for which no larger sample sizes will have a power
less than that specified.
Results and Interpretation for Power calculation
Prob Guess: The probability of a correct guess.
This is a function of the test protocol selected and is not dependent on other
parameters.
Power: The power of the tests for the specified
sample size, protocol and type of test, alpha and proportion of distinguishers
(Pd) or d-prime (d’).
Min Correct:
If the test statistic is more extreme than the critical value of a
hypothesis test, then the null hypothesis is rejected.
For a difference test, if the
number of correct tests is greater or equal to the minimum correct (for the
given alpha) then the null hypothesis is rejected.
For a similarity test, if the
number of correct tests is less than or equal to the minimum correct (for a
given alpha) then the null hypothesis is rejected.
The minimum correct is therefore the minimum/maximum
number of correct responses that must be returned for the null to be rejected
depending on whether it is a similarity or difference test.
- The R package sensR (Rune Christensen and Per B.
Brockhoff) is used.
- The power calculation is done using the ‘exact’ binomial
calculation.
References
Related Articles
Same/Different Test Analysis
Available from version: 5.0.8.6 Purpose The Same Different Test is a discrimination test that is a variation of the paired comparison test. The assessor is presented with two samples and is asked to decide whether these samples are the same or ...
Two Step Double-Faced Applicability Test
Introduction In the 'double-faced applicability' test, each attribute is presented with a “double-faced” approach, featuring two descriptors (a pair of semantic-differential descriptors) separately presented in the questionnaires to represent both ...
Penalty Analysis
Purpose To provide a penalty analysis of a consumer data set, that is to investigate how liking or acceptability of product decreases when product attributes are not at the optimal intensity. Data Format Consumer.xlsx Note: for EyeOpenR to read your ...
Tetrad
Introduction The Tetrad method is a testing procedure designed to understand the presence of a perceptible sensory distinction between two products. Panelists are given four samples, with two being identical and two being different. The panellists ...
Triangle
Introduction The triangle method is used to understand the presence or absence of a perceptible sensory difference or similarity between samples of two products through a forced-choice approach. Participants are presented with three samples, two of ...