2-AFC, 3-AFC, Duo Trio, Triangle, Tetrad Analysis

2-AFC, 3-AFC, Duo Trio, Triangle, Tetrad Analysis

Purpose 

Analyse results from one of the following tests: 2-AFC, 3-AFC, Duo-Trio, Triangle, Tetrad. 

Data Format

  1. Discrimination.xlsx
  2. Results of the discrimination test are binary (1 = correct answer, 0 = incorrect answer) 

Background 

2-AFC Test

This is an alternative forced choice (AFC) discrimination test where panellists are presented with 2, 3 or more products and asked to select one product based on a pre-specified attribute.  The 2-AFC test, also known as a directional difference test, is used to establish if a directional difference exists in the perceived intensity of a specified attribute between 2 products.  The panellist is required to state if a 2nd product is more or less intense in the specified attribute compared to the 1st presented product.
The guessing probability (probability of getting a correct answer by guessing only) is ½.

3-AFC Test

This is an alternative forced choice (AFC) discrimination test where panellists are presented with 2, 3 or more products and asked to select one product based on a pre-specified attribute.  The3-AFC test is used to establish if there is a discernible difference between 2 products in respect of a specified attribute.  The panellist is required to select 1 product from the set of 3 that differences in the specified attribute.
 The guessing probability (probability of getting a correct answer by guessing only) is ⅓.

Duo Trio Test

A discrimination (difference) test to determine if an unspecified difference exists between two products.  The panellist is presented with 2 products (A and B) and a reference product (R), and asked to determine which of A or B is most similar to R. Useful for products that are fairly similar but not totally identical.
The guessing probability (probability of getting a correct answer by guessing only) is ½.

Triangle Test

During a triangle test, a panellist is presented with three samples of which two are equal and one is different. The panellist must state which sample is different. The results indicate whether a detectable difference exists between two samples. The method is statistically more efficient than the duo trio test but has limited use with products that have a strong and/or lingering flavour.
 If there is no difference detected between sample A and B the panellist must choose a random sample. The chance a panellist chooses the odd sample is 1/3. If there is no difference between the samples you would expect one third of the panellists to choose the odd sample, while two third of the panel chooses one of the equal samples. If there is a detectable difference more than one third of the panellists will choose the right sample. 
 The guessing probability (probability of getting a correct answer by guessing only) is ⅓.

Tetrad

An unspecified or specified attribute discrimination test which aims to establish if 2 products are different or similar.  Panellists are presented with 2 samples of the 1st product and 2 samples of the 2nd product and asked to sort them into two groups of 2 products where products in a group are more similar to each other.
 There is a ⅓ chance of selecting of selecting any combination of 2 groups of 2 products.

Theory 

The statistical principle behind every discrimination test should be to reject a null hypothesis (H0). For a difference test the null hypothesis states there is no detectable difference between two (or more) products. The alternative hypothesis H1 is that there is a detectable difference. If there is sufficient evidence to reject H0 in favour of the alternative hypothesis H1, then a difference can be recorded.

For a similarity test the null hypothesis states that there is a non-negligible difference. The size of difference that will be considered non-negligible must be pre-specified. The alternative hypothesis is that there is no difference. If there is sufficient evidence to reject H0 in favour of the alternative hypothesis H1, then it can be concluded the products are similar.

The data is processed using the binomial test to test for difference among the samples.
​
The number of correct and incorrect responses are counted.  The proportion of correct responses is calculated and from there the proportion of true distinguishers is calculated as follows: where pc is the proportion of correct responses. Pg is the guessing probability and pd is the proportion of distinguishers.

pc = pd + (1-pd)*pg                                          
pd =(pc-pg)/(1-pg)            

Options

  1. Treat Sessions/Replicates separately: If the data has been gathered over different sessions, or there are different replicates, these can be analysed separately.
  2. Type of test: Similarity or difference test.
  3. Model: Guessing or Thurstonian model.
  4. Prop of Discriminators Threshold (Pd): If the test is a similarity test and the Guessing Model is used, the threshold that will be considered non-negligible. That is, H0 is that Pd is greater than this threshold.
    Or
  5. D-prime Threshold (d’): If the test is a similarity test and the Thurstonian Model is used, the threshold that will be considered non-negligible. That is, H0 is that d’ is greater than this threshold.
  6. Confidence level: The probability for the confidence intervals. That is, a 0.95 confidence interval means that in 95% of population samples, the true value would lie in this confidence interval.
  7. Number of Decimals for Values: Required number of decimals for values given in the results.
  8. Number of Decimals for P-Values: Required number of decimals for any p-values given in the results. 

Results and Interpretation 

Summary

  1. N total: The total number of tests in the results set
  2. N correct: The number of correct tests
  3. N incorrect: The number of incorrect tests (N incorrect + N correct = N total)
  4. Proportion correct: The proportion of correct tests as a percentage (100 x N correct/N tot)
  5. P-value: The p-value indicates the probability of obtaining the result if the Null hypothesis is true.
  6. Min correct (0.1%): The minimum number of correct responses for the result to be significant with 99.9% probability.
  7. Min correct (1%): The minimum number of correct responses for the result to be significant with 99% probability. (For a similarity test this is the maximum number).
  8. Min correct (5%): The minimum number of correct responses for the result to be significant with 95% probability.
  9. Min correct (10%): The minimum number of correct responses for the result to be significant with 90% probability.
  10. p < 0.1%: Is the p-value less than 0.001 i.e. Is the result significant at 99.9% level?
  11. p < 1%: Is the p-value less than 0.01 i.e. Is the result significant at 99% level?
  12. p < 5%: Is the p-value less than 0.05 i.e. Is the result significant at 95% level?
  13. p < 10%: Is the p-value less than 0.1 i.e. Is the result significant at 90% level?

Results

  1. Proportion correct: The proportion of correct responses (within the bounds of the guessing probability) and confidence interval, calculated using the exact method.
  2. Proportion Discriminators: The proportion of responses that are true distinguishers and confidence interval, calculated using the exact method.
  3. D-prime: is an estimation of the distance between the products according to the Thurstonian scale. It is the difference between the mean values of the two signals divided by the standard deviation.
When testing for a difference, the p-value indicates if the samples are significantly different, and at what level.  You will decide whether to conclude if the samples are different based on the risk you want to take.
In the case of similarity, the interpretation is similar, but will be in the context of the pd or d-prime value you specified as a non-negligible difference. 

Beta-Binomial

If the data contains 3 or more replicates the Beta-Binomial model is fitted. This is to check for loss of independence in the replicates. This is also called over-dispersion in the data.
The beta-binomial model is parameterized in terms of mu and gamma, where mu corresponds to a probability parameter and gamma measures over-dispersion. Both parameters are restricted to the interval (0, 1).
The parameters of the standard (i.e. not corrected) beta-binomial model refers to the mean (i.e. probability) and dispersion on the scale of the observations, i.e. on the scale where we talk of a probability of a correct answer (Pc).
The following parameters are returned, with estimate, standard error, and confidence interval limits.
  1. Probability (mu)
  2. Over-dispersion (gamma)
  3. Pc – the probability of a correct response.
  4. Pd – the probability of true discrimination.
  5. d-prime – the ‘distance’ between the products. 

Test (Beta-Binomial)

The test shows:

  1. a likelihood ratio test of over-dispersion on one degree of freedom.
  2. and a likelihood ratio test of association (i.e. where the null hypothesis is "no difference" and the alternative hypothesis is "any difference") on two degrees of freedom (chi-square tests).
  3. If the data is over-dispersed, the p-value for the test for over dispersion should be smaller than the desired alpha. 

Technical Information 

  1. The R package sensR (Rune Christensen and Per B. Brockhoff) is used.
  2. The confidence intervals are calculated using the ‘exact’ binomial method. 

References

  1. ISO 4120:2004 Sensory Analysis – Methodology – Triangle Test
  2. ASTM-E1885-04 (2011) Standard Test Method for Sensory Analysis – Triangle Test
  3. Lawless, H.T. and Heymann, H. (2010).  Sensory Evaluation of Food – Principles and Practices.  Springer.
  4. Ennis. J. M., and Jesionka, V. (2011) – The Power of Sensory Discrimination Methods Revisited.  Journal of Sensory Studies, 26, 371-382.
  5. Ennis. J. M. (2012).  Guiding the Switch from Triangle Testing to Tetrad Testing.  Journal of Sensory Studies, 27, 4, 223-231.
  6. Garcia, K., Ennis, J.M. and PrinyawIwatkul, W. (2013).  Reconsidering the Specified Tetrad Test.  Journal of Sensory Studies, 28, 6, 445-449.
  7. O’Mahony, M. (2013).  The Tetrad Test – Looking Back, Looking Forward.  Journal of Sensory Studies, 28, 4, 259-263.
  8. ISO 10399:2004 Sensory Analysis – Methodology – Duo-Trio Test
  9. ASTM-E2610-08 (2011) Standard Test Method for Sensory Analysis – Duo-Trio Test
  10. Christensen, R.H.B. (2014).  Statistical Methodology for Sensory Discrimination Tests and Its Implementation in SensR.
  11. Christensen, R.H.B. and Brockhoff, P.B (2014).  Package ‘sensR’.
  12. http://cran.r-project.org/web/packages/sensR/sensR.pdf
  13. Christensen, R.H.B. and Brockhoff, P.B (2014).  Sensory Discrimination Testing with the sensR Package.
  14. http://user2014.stat.ucla.edu/abstracts/talks/125_Christensen.pdf


    • Related Articles

    • Duo-Trio

      Introduction The Duo-Trio Test is a sensory test that can determine the presence or absence of sensory differences between two samples. This method is particularly valuable in: The Duo-Trio test can be comparable to the triangle test, maintaining ...

    • Tetrad

      Introduction The Tetrad method is a testing procedure designed to understand the presence of a perceptible sensory distinction between two products. Panelists are given four samples, with two being identical and two being different. The panellists ...

    • Triangle

      Introduction The triangle method is used to understand the presence or absence of a perceptible sensory difference or similarity between samples of two products through a forced-choice approach. Participants are presented with three samples, two of ...

    • Three-Alternative Forced Choice (3-AFC)

      Introduction The three-alternative forced choice (3-AFC) method is a discrimination method where panelists are presented with three samples simultaneously and instructed to identify the sample(s) that are higher or lower in a specified sensory ...

    • Two-Alternative Forced Choice (2-AFC)

      Introduction The Two-Alternative Forced Choice (2-AFC) method, is a specific way of comparing two samples in sensory science. In this method, the only allowed difference between the two samples is a specified sensory characteristic. It works well ...