Panelist Feedback Analysis

Panelist Feedback Analysis

Mar 2022
Author: JM

Purpose

Plots summarising of the scoring range for each judge on each of the samples and attributes.

Data Format

  1. See the profiling dataset.
  2. The attributes should be of scale or interval type.

Options

  1. Type of Panel Mean: Adjusted or Arithmetic. Arithmetic means are the well known means, calculated by summing the observations then dividing by the number of observations. Adjusted means, commonly called Least-Squares means (LSmeans) or Estimated Marginal Means (EMMs), use a regression model to calculate the means adjusting for the balance of the data.
  2. Show results by: Should each individual plot be for a given attribute or a given product.
  3. Y-axis Scale: Automatic or Manual. Automatic chooses the plotting range from the range of each attribute. Manual uses the plotting range you specified.
  4. Y-axis min value: The smallest value that will appear on all plots when specifying the plotting range manually.
  5. Y-axis max value: The largest value that will appear on all plots when specifying the plotting range manually.
  6. Anonymise Assessors? Choose to replace the assessor names or not. There are options for randomly generated names or names from the assessor metadata.
  7. Anonymise Products? Choose to replace the product names or not. There are options for randomly generated names or names from the product metadata.
  8. Anonymise Attributes? Choose to replace the attribute names or not. There are options for randomly generated names or names from the attribute metadata. 

Results and Interpretation

  1. The FEEDBACK ALL are line plots, either one plot for each product with panellist means plotted against the attributes, or one plot for each attribute with panellist means plotted against the products. The panel mean is also included. Note that while this information is plotted with lines, this is only to make the plots easier to follow.
  2. The FEEDBACK plots are split by panellist and either plot for each product said panellist’s mean across attributes or for each attribute said panellist’s mean across products. The panel level mean is overlaid and the closer a panelist’s mean is to the overall panel, the closer to the panel consensus. Intervals of plus or minus the standard error of the mean are included, small intervals suggest closer agreement in scoring over replicated assessments.
  3. If “Type of Panel Mean” was set to “Adjusted” then the panel means will be adjusted means from models of the form: Attribute = Product + Assessor + Replicate + Residuals or Attribute = Product + Assessor + Residuals if replications are not present.

Technical Information

  1. R packages: This analysis uses SensoMineR to calculate the adjusted means.

    • Related Articles

    • Direct Feedback, Panelist Feedback on a scale, Panelist Feedback

      Direct Feedback The direct feedback function provides the feedback live to panelists.  The mean and sd need to be specified for each attribute for each product, so you need to know ahead of the mean and the standard deviation value.  When panelists ...
    • Panelist Performance Analysis

      Mar 2022 Author: JM Purpose This analysis looks at the overall panel performance in terms of Discrimination, Agreement and Repeatability or Reproducibility, and then the performance of each individual in the panel in these terms. Data Format See the ...
    • Panelist Strip Plot Analysis

      Mar 2022 Author: JM Purpose Plots for each attribute the panellists means across the products, for visual comparison and interpretation. Data Format See the profiling dataset. Background This analysis calculates the mean response from each assessor ...
    • Panelist Outliers Analysis

      Mar 2022 Author: JM Purpose To detect panellists that are possible outliers, highlighting those with extremely high or low values. Data Format See the profiling dataset. Options Type of Panel Mean: Adjusted or Arithmetic. Arithmetic means are the ...
    • Why are the results from panel and panelist performance different?

      Although the panel performance and the panellist performance seems to be linked, at least from an interpretation point of view (one could expect that the panel is discriminating if panellists are discriminated), the analyses involved different data ...