Direct Feedback, Panelist Feedback on a scale, Panelist Feedback

Direct Feedback

The direct feedback function provides the feedback live to panelists. 

The mean and sd need to be specified for each attribute for each product, so you need to know ahead of the mean and the standard deviation value. 

When panelists are filling the questionnaire they will see how their answer performed compared to the mean and the sd set ahead. 

Step 1. Create a project: Create Project>Panel Training>Direct Feedback.

Step 2. Modify the settings in the Line Scale. 

In the Line Scale Standard used in this template, the Feedback function is already activated (Advanced settings>Feedback). Under mean and sd options you can indicate what mean and standard deviation for each product should be. 

There are two ways to indicate the mean score and the standard deviation:

First methods:

You can create and import an Excel file where you indicate the mean and sd value for each product. In the excel file, you need to have a column with the attribute name + mean, and another attribute with the attribute name + sd.  Each row in your excel file corresponds to your products. E.g. 25 is the mean score of product 1 indicated in your design (not the first product that panelists will evaluate).

You can then import this Excel file in Design>Products>Import:

Once imported you will see that the mean and sd per product are in. If you want you can also directly make the changes here. 

When you go back to the line-scale question, in the mean and sd section you can indicate with a placeholder that the information regarding the mean and sd will be taken from the product characteristics that you have just imported.

Second methods:

Alternatively, you can directly set the mean value and the sd directly from the line scale: in the example below for instance the mean score for product 1 = 40 with sd 10, product 2 = 30with sd 5, and product 3 = 50 with sd 15 . 

Show a panelist how he/she rated all the products on the scale (B. Panelist Feedback on a scale): 

Step 1. After you have added a linescale question type in your design questions sections, go to the "End screen"  

Add question Panel Feedback>Panelist feedback on a scale 

In the Standard Setting, you should set from which question the feedback is based. 

In this feedback, the panelist will only see their individual personal answer. 

Step 2. Make sure that the line scale that you use during your test and for your feedback has the same anchor. 

After participants complete the session and reach the final screen, they will be presented with the data they evaluated on a line intensity scale, displayed in the order based on the intensity they provided.

Show the feedback of also the other participants (A. Feedback)

Through this function, you will show the result of EyeOpenR analysis to your panelists.  

In the End Question, add a Question type>Panel Feedback>Feedback.

In the Standard settings, set the Analysis that you would like to show to your panelists. And you can also indicate the Output. 

It is important to set up the screen settings on the feedback page. Enable the options 'Close Session' and 'Feedback Screen'.

Proceed to deploy your questionnaire. The feedback page will be displayed to all panelists after the threshold has been met. The panelists will see a loading screen until the last panelist before the threshold has filled in the questionnaire.

After all panelists have finished filling in the questionnaire, the feedback screen will be visible.

The unique logins of the other panelists will be anonymised (U47297 for example). The sessions will automatically be closed when this screen is visible.

You can read more about the different statistical analyses available in the following link: https://support.eyequestion.nl/portal/en/kb/articles/panelist-feedback

• Related Articles

• Direct Feedback

  Introduction This template is designed for training panellists. Within this template, you can preset the mean score and standard deviation for each attribute queried. Panelists will review the list of attributes and then conduct their evaluation. ...

• Panelist Feedback Analysis

  Purpose Plots summarising the scoring range for each judge on each of the samples and attributes. Data Format See the profiling dataset. The attributes should be of scale or interval type. Options Type of Panel Mean: Adjusted or Arithmetic. ...

• How to Display Feedback of Triangle and Tetrad Test

  Introduction During a Triangle test a panellist is presented with three samples of which two are equal and one is different. A panellist must point out the odd sample. A user can add the screen condition in the triangle test to display message when a ...

• Why Are The Results From Panel And Panelist Performance Different?

  Although the panel performance and the panellist performance seems to be linked, at least from an interpretation point of view (one could expect that the panel is discriminating if panellists are discriminated), the analyses involved different data ...

• Panelist Performance Analysis

  Purpose This analysis looks at the overall panel performance in terms of Discrimination, Agreement and Repeatability or Reproducibility, and then the performance of each individual in the panel in these terms. Data Format See the profiling dataset. ...