Why Do I Have Empty Cells In Panel Performance (Panelist Agreement)

Why Do I Have Empty Cells In Panel Performance (Panelist Agreement)

The empty cell in the panel/panellist performance is usually because there is no variation in your data. This happens when an assessor gives the same average score to all samples, where the average score is taken across all replicate measurements given by an assessor to one sample.  So, it is not necessary that all the individual replicate scores are the same, just that the average product scores are the same. Mathematically the problem is a division by zero error, because the assessor panel performance measure depends on the standard deviation of the assessor's average scores, a calculation that is not possible if that standard deviation is zero.

You are more likely to experience empty cells when there are only two samples, since the fewer samples there are, the greater chance that all means will be the same.  At the panellist level, agreement is measured through the correlation coefficient, and note that it is not recommended to compute correlation coefficients with only two samples since the coefficients are always -1, +1, or undefined owing to no variation.  We recommend that at least 5 samples are necessary to get robust diagnostics based on the correlation coefficient.

More generally, when running panel/panellist performance, the more samples there are, the more valuable the results.
    • Related Articles

    • Why Are The Results From Panel And Panelist Performance Different?

      Although the panel performance and the panellist performance seems to be linked, at least from an interpretation point of view (one could expect that the panel is discriminating if panellists are discriminated), the analyses involved different data ...
    • Panelist Performance Analysis

      Purpose This analysis looks at the overall panel performance in terms of Discrimination, Agreement and Repeatability or Reproducibility, and then the performance of each individual in the panel in these terms. Data Format See the profiling dataset. ...
    • Panel Scheduling in EyeQuestion 5

      Introduction One of the more challenging tasks in conducting sensory and consumer research is the process of manually checking the availability of your panellists participation by calling or emailing them. This traditional approach is not ideal as it ...
    • Export Accounting (Panelist History)

      If you want to create an overview (in Excel) of your projects and participants or of your panelist' participation, then the Export Accounting is the way to go. Go to Tools > Data tools > Export Accounting Use the filters to filter on date or time ...
    • Panel Restriction

      Introduction Panel Restrictions allow you to filter out panellists based on their past or future appointments or their participation in specific projects. This feature helps you create more targeted panels by excluding panellists who may not be ...