Panelist Outliers Analysis

Panelist Outliers Analysis

Mar 2022
Author: JM

Purpose

To detect panellists that are possible outliers, highlighting those with extremely high or low values.

Data Format

  1. See the profiling dataset.

Options

  1. Type of Panel Mean: Adjusted or Arithmetic. Arithmetic means are the well known means, calculated by summing the observations then dividing by the number of observations. Adjusted means, commonly called Least-Squares means (LSmeans) or Estimated Marginal Means (EMMs), use a regression model to calculate the means adjusting for the balance of the data.
  2. Number of standard deviations: Assessors whose mean falls outside plus or minus this number of standard deviations from the panel mean are highlighted as possible outliers.
  3. Y-axis Scale: Automatic or Manual. Automatic chooses the plotting range from the range of each attribute. Manual uses the plotting range you specified.
  4. Y-axis min value: The smallest value that will appear on all plots when specifying the plotting range manually.
  5. Y-axis max value: The largest value that will appear on all plots when specifying the plotting range manually.
  6. Anonymise Assessors? Choose to replace the assessor names or not. There are options for randomly generated names or names from the assessor metadata.
  7. Anonymise Products? Choose to replace the product names or not. There are options for randomly generated names or names from the product metadata.
  8. Anonymise Attributes? Choose to replace the attribute names or not. There are options for randomly generated names or names from the attribute metadata.

Results and Interpretation

  1. For each product and attribute a bar plot of panellist means is displayed. These plots overlay the panel mean as a black line and the chosen number of standard deviations from this mean as two dotted lines. Any panellist outside this range is highlighted as a possible outlier.
  2. If “Type of Panel Mean” was set to “Adjusted” then the panel means will be adjusted means from models of the form: Attribute = Product + Assessor + Replicate + Residuals or Attribute = Product + Assessor + Residuals if replications are not present.
  3. Outliers should not necessarily be removed.
  4. ​For a large number of draws from a normal distribution it is expected that around 5% of observations would be outside of plus or minus 1.96 standard deviations from the mean.

Technical Information

  1. R packages: This analysis uses SensoMineR to calculate the adjusted means.
    • Related Articles

    • Direct Feedback, Panelist Feedback on a scale, Panelist Feedback

      Direct Feedback The direct feedback function provides the feedback live to panelists.  The mean and sd need to be specified for each attribute for each product, so you need to know ahead of the mean and the standard deviation value.  When panelists ...
    • Panelist Feedback Analysis

      Mar 2022 Author: JM Purpose Plots summarising of the scoring range for each judge on each of the samples and attributes. Data Format See the profiling dataset. The attributes should be of scale or interval type. Options Type of Panel Mean: Adjusted ...
    • Panelist Performance Analysis

      Mar 2022 Author: JM Purpose This analysis looks at the overall panel performance in terms of Discrimination, Agreement and Repeatability or Reproducibility, and then the performance of each individual in the panel in these terms. Data Format See the ...
    • Panelist Strip Plot Analysis

      Mar 2022 Author: JM Purpose Plots for each attribute the panellists means across the products, for visual comparison and interpretation. Data Format See the profiling dataset. Background This analysis calculates the mean response from each assessor ...
    • Why are the results from panel and panelist performance different?

      Although the panel performance and the panellist performance seems to be linked, at least from an interpretation point of view (one could expect that the panel is discriminating if panellists are discriminated), the analyses involved different data ...