Paired Comparison Analysis

Paired Comparison Analysis

Purpose 

To analyse the results of a paired comparison test. 

Data Format      

  1. paired_comparison.xlsx
  2. Datatype for the attribute is pairedcomp. 

Background 

A paired comparison test is a directional / specified test. It is used as a difference test with 2 products, to determine if one product has more or less of an attribute than another product. It is also called a paired preference test if preference is the attribute of interest. It should be used in situations where there is no definitive ‘right’ answer.

The test is carried out as follows:

Two blind-coded samples are presented to each assessor (typically consumer).

  1. Assessors are asked to focus on a specified attribute or preference
    (the latter for consumer testing)
  2. Balance the presentation order across assessors
  3. Chance of each product being selected = 0.5
  4. If there is a ‘no-preference’ option when running a consumer test. Assessors that pick the no-preference option can be removed or split equally between the 2 products.

The user should consider in advance if their test is 1-sided or 2-sided as part of the study design. 

Options 

  1. No Choice: What action to take if assessors have selected a no-preference option. This can be ‘Delete’ which removes those assessors. Or ‘Split’ which divided the assessors equally between products.
  2. Type of test: One-sided or two-sided. This depends on the context of the test and whether the comparison is one sided or two sided.
  3. Threshold: The significance level or type-I error for the comparison.
  4. Number of Decimals for Values: Required number of decimals for values given in the results.
  5. Number of Decimals for P-Values: Required number of decimals for any p-values given in the results. 

Results and Interpretation 

Contingency

A n x n contingency table of results. The cell totals are the number of times the product in the row was selected when compared to the product in the column.
This can be considered as a combined set of 2 x 2 tables. 

Binomial 

  1. One tab per Attribute. A binomial test is carried out on the results of each comparison.
  2. The following totals are returned:
    1. The number of times each product was selected.
    2. N Total: total number of tests
    3. Minimum: The number of times a product would need to be selected over the other to detect a significant difference at the given threshold.
    4. P-value: From the binomial test – the probability of obtaining the result under the null hypothesis of no difference. 

Interpretation

If the p-value is less than the selected threshold (significance level) for the binomial test, conclude that the samples are significantly different.  You may choose to have a higher level of confidence to minimise your risk.

Note that even if you conclude a difference, this does not imply similarity. 

References 

  1. ISO 5495:2005 Sensory Analysis – Methodology – Paired Comparison Test
  2. ASTM-2263-12 Standard Test Method for Paired Preference Test


 


    • Related Articles

    • Paired Comparison

      Introduction The paired comparison method is a handy method to use when dealing with products that have a lasting effect or are in short supply. Unlike other tests like the triangle or duo-trio, it works well when showing three samples at once isn't ...

    • Design Types Available in EyeQuestion

      Introduction EyeQuestion provides a diverse range of design types to meet your every need. On the "Design Type" tab within the Design Generator, you can select from various design options. This page offers an overview of each design type available. ...

    • ANOVA with Multiple Comparison Tests

      Purpose To provide an analysis of variance (ANOVA) test per selected attribute. ANOVA examines sources of variation in the data: this is often used in sensory science to investigate whether variation in attributes is due to products, samples and ...

    • Penalty Analysis

      Purpose To provide a penalty analysis of a consumer data set, that is to investigate how liking or acceptability of product decreases when product attributes are not at the optimal intensity. Data Format Consumer.xlsx Note: for EyeOpenR to read your ...

    • Same/Different Test Analysis

      Available from version: 5.0.8.6 Purpose The Same Different Test is a discrimination test that is a variation of the paired comparison test. The assessor is presented with two samples and is asked to decide whether these samples are the same or ...