Why are the results from panel and panelist performance different?

Why are the results from panel and panelist performance different?

Although the panel performance and the panellist performance seems to be linked, at least from an interpretation point of view (one could expect that the panel is discriminating if panellists are discriminated), the analyses involved different data (the panellist performance is based on subset of data, while the panel performance uses all the data at once). Hence, it can happen that assessors are not so well discriminating while once combined, the overall results show good discrimination.

The panel and panelist performance do not rely on the same statistical analysis. Panel performance is based on ANOVA, and the agreement is based on the product:assessor interaction.The panelist performance is taking only the panelist data, and the agreement at an individual level is based on the correlation between that assessor scores and the average across panels.These 2 stats are completely different, have completely different implications/assumptions, so cannot be compared one to one. Also, in some situations, the lack of disagreement can be simply due to a different use of the scale, meaning that the assessors are still “ranking” the samples in the same way, BUT are emphasizing more or less certain differences.


    • Related Articles

    • How do I interpret panel performance?

      The panel repeatability is evaluated through the product:replicate interaction whereas in the other case (reproducibility), the panel reproducibility is evaluated through the MS error. For individual panelist performance, the agreement is evaluated ...
    • Direct Feedback, Panelist Feedback on a scale, Panelist Feedback

      Direct Feedback The direct feedback function provides the feedback live to panelists.  The mean and sd need to be specified for each attribute for each product, so you need to know ahead of the mean and the standard deviation value.  When panelists ...
    • How can I import a panel?

      You can invite panellists to participate in your panel by: - Transferring them from the Panel Database to the panel access list - Using Import Open a project and click the panel tab. Panelists can be added to the panel by adding them to the access ...
    • Why do I have empty cells in panel performance (panelist agreement) ?

      The empty cell in the panel/panellist performance is usually due to the fact that you do not have variability in your data (the assessor gives the same score to all the samples). Indeed, most of the analysis requires dividing by the standard ...
    • How to rapidly visualize categorial results for monitoring the progress of the research?

      Category data such as hedonic data, demographic data (age, gender, income, education) or frequencies and often collected in sensory or consumer research questionnaires. In order to rapidly visualize these results, you can make use of an effective ...