Available from version: 5.4.4
The Quality Index is an ANOVA based analysis, with the idea based on the paper by Verhoef (2015) for the purpose of measuring the reliability of univariate sensory descriptive data.
The data to import should be in an ANOVA-type of configuration, i.e. the response variables should be numerical (measured on a linear scale or a categorical scale), and there should be data available to capture assessor, session, replicate or both replicate and session. These explanatory variables should have different levels (at least 2 products and 2 panellists in a non-monadic test situation).
Data layout is standard descriptive data on the number of variables of interest across products and sessions and/or replicates.
The ANOVA model considered here is:
Attribute = Product + Assessor + Occasion + Product:Assessor + Product:Occasion + Assessor:Occasion + Product:Assessor:Occasion + error
The univariate quality index (uQI) is then computed from the variance estimates in the resulting ANOVA model using the following formula:
Definition of the calculation of these variables is defined in the analysis specification.
It is important to understand that 2 situations are possible which cause the definition of στ2 and σδ2 to vary slightly. These are controlled via options defined in the EO setup (see below).
- Occasion: Defines
which variable should be considered to measure as the occasion, i.e. session,
replicate or session by replicate.
- Type of assessor facet: Defines whether or not the user wishes to specify the assessor facet as ‘fixed’ or ‘random’. It is important to note here this is not how the variable is defined in the ANOVA model setup, the model always uses fixed effects, but rather how the assessor facet is considered in the calculation of στ2 and σδ2 and hence the Quality Index.
- Number of Decimals for Values: Required number of decimals for values given in the results.
- Number of Decimals for P-Values: Required number of decimals for p-values given in the results.
Results and Interpretation
On the first tab of the output, the table containing the results is shown. The first column is the name of the attribute that has been analysed.
Then the following columns are presented in the table:
- Significance level: The p-value for the product effect from the model
- uQI: The univariate quality index colour coded according to it’s reliability level
- Product: Variance of the product effect
- Product*Assessor: Variance of the product by assessor interaction
- Product *Occasion: Variance of the product by occasion interaction
- Product * Assessor * Occasion: Variance of the product by assessor by occasion interaction
- Error: Mean square error
- Universe Score: Universe score variance στ2
- Relative Error: Relative error variance σδ2
The last 7 columns are a decomposition of the quality index. These values of variances are combined to calculate the uQI value.
This output table can also be exported to Excel by clicking on the green Excel link to the right of the table.
Finally, on the second tab, there is an information table that may provide additional information regarding your data and/or the analysis. This should be checked each time, but particularly if unusual results are seen as there may be error messages or warnings given here.
The outputs provide insights on the proportion of variability in that data that is related to the different parts of the model as described in the outputs section above. In general we want variability due to assessors to be low values, demonstrating that the assessor are providing reliable and repeatable data. The quality index is considered as ‘good’ if it is above 0.8 and the colour coding on the output demonstrates this clearly.
• R packages used:
The key R package used for this analysis is the ‘car’ package to perform ANOVA.
• Statistical Notes Regarding Model Fitting:
Type II sums of squares have been used as these are considered most appropriate. (If the data is complete there will be no difference between type I, type II or type III sums of squares).
In addition, when the model does not contain enough data to fit all model terms (i.e. there is not enough data to fit a 3-way interaction) the model fits as many terms as possible and the F-tests use the remaining data as residual error term. This will be seen in the results table when the error variance is shown as zero. Where the model does not have enough degrees of freedom to fit a complicated model reliably, it will report an error/warning message – this is likely due to a very unbalanced design resulting in considerable numbers of design cells with missing values.
These models have been fitted in R and validated against SAS and XLSTAT for completeness due to the numerous ways that different softwares handles type I, II and III sums of squares with unbalanced designs/missing data.
The p-value for the product effect uses a simple standard consumer science model including terms for assessor, product and assessor by product interaction. In the case of replicates the product effect F-test is computed against the interaction MS, otherwise it is test against the MS error.
Verhoef, A., Huijberts, G., Vaessen, W. (2015). Introduction of a quality index, based on Generalizability theory, as a measure of reliability for univariate- and multivariate sensory descriptive data. Food Quality & Preference, 40, 296-303
Purpose To analyse the results of an R-index test. Data Format R_index_rank_withMD.xlsx Background The R-index applies signal detection theory as an alternative approach to discrimination testing. It is often used where there are a large number ...
Purpose To provide an analysis of data collected using the napping methodology. Data Format Napping.xlsx For EyeOpenR to read your data the first five columns must include the following in the specified order: Assessor, Product, Session, Replica and ...
Same/Different Test Analysis
Available from version: 18.104.22.168 Purpose The Same Different Test is a discrimination test that is a variation of the paired comparison test. The assessor is presented with two samples and is asked to decide whether these samples are the same or ...
MAM Model Analysis
Purpose This analysis looks at the overall panel performance in terms of Discrimination, Agreement and Repeatability or Reproducibility, and then the performance of each individual in the panel in these terms. Using a more sophisticated model, than ...
Principal Component Analysis (PCA)
Purpose To provide a Principal Components Analysis (PCA) of the imported data. PCA is a popular dimensionality reduction method in sensory and consumer science. In non-technical terms, the analysis reduces an initial set of variables to a smaller ...