Quality Index Analysis

Quality Index Analysis

Available from version: 5.4.4

Purpose

The Quality Index is an ANOVA based analysis, with the idea based on the paper by Verhoef (2015) for the purpose of measuring the reliability of univariate sensory descriptive data.

Data format

  1. profiling.xlsx
The data to import should be in an ANOVA-type of configuration, i.e. the response variables should be numerical (measured on a linear scale or a categorical scale), and there should be data available to capture assessor, session, replicate or both replicate and session. These explanatory variables should have different levels (at least 2 products and 2 panellists in a non-monadic test situation).

Data layout is standard descriptive data on the number of variables of interest across products and sessions and/or replicates.

Background

The ANOVA model considered here is:
      Attribute = Product + Assessor + Occasion + Product:Assessor + Product:Occasion + Assessor:Occasion + Product:Assessor:Occasion + error

The univariate quality index (uQI) is then computed from the variance estimates in the resulting ANOVA model using the following formula:


Definition of the calculation of these variables is defined in the analysis specification.
It is important to understand that 2 situations are possible which cause the definition of  στ2 and σδ2 to vary slightly. These are controlled via options defined in the EO setup (see below).

Options

  1. Occasion: Defines which variable should be considered to measure as the occasion, i.e. session, replicate or session by replicate.
  2. Type of assessor facet: Defines whether or not the user wishes to specify the assessor facet as ‘fixed’ or ‘random’. It is important to note here this is not how the variable is defined in the ANOVA model setup, the model always uses fixed effects, but rather how the assessor facet is considered in the calculation of  στ2 and σδ2 and hence the Quality Index.
  3. Number of Decimals for Values: Required number of decimals for values given in the results.
  4. Number of Decimals for P-Values: Required number of decimals for p-values given in the results.

Results and Interpretation

On the first tab of the output, the table containing the results is shown. The first column is the name of the attribute that has been analysed.
Then the following columns are presented in the table:
  1. Significance level: The p-value for the product effect from the model
  2. uQI: The univariate quality index colour coded according to it’s reliability level 


  3. Product: Variance of the product effect
  4. Product*Assessor: Variance of the product by assessor interaction
  5. Product *Occasion: Variance of the product by occasion interaction
  6. Product * Assessor * Occasion: Variance of the product by assessor by occasion interaction
  7. Error: Mean square error
  8. Universe Score: Universe score variance στ2
  9. Relative Error: Relative error variance σδ2
The last 7 columns are a decomposition of the quality index. These values of variances are combined to calculate the uQI value.

This output table can also be exported to Excel by clicking on the green Excel link to the right of the table.

Finally, on the second tab, there is an information table that may provide additional information regarding your data and/or the analysis. This should be checked each time, but particularly if unusual results are seen as there may be error messages or warnings given here.

The outputs provide insights on the proportion of variability in that data that is related to the different parts of the model as described in the outputs section above. In general we want variability due to assessors to be low values, demonstrating that the assessor are providing reliable and repeatable data. The quality index is considered as ‘good’ if it is above 0.8 and the colour coding on the output demonstrates this clearly.

Technical Information

• R packages used:
The key R package used for this analysis is the ‘car’ package to perform ANOVA.

• Statistical Notes Regarding Model Fitting:
Type II sums of squares have been used as these are considered most appropriate. (If the data is complete there will be no difference between type I, type II or type III sums of squares). 

In addition, when the model does not contain enough data to fit all model terms (i.e. there is not enough data to fit a 3-way interaction) the model fits as many terms as possible and the F-tests use the remaining data as residual error term. This will be seen in the results table when the error variance is shown as zero. Where the model does not have enough degrees of freedom to fit a complicated model reliably, it will report an error/warning message – this is likely due to a very unbalanced design resulting in considerable numbers of design cells with missing values.

These models have been fitted in R and validated against SAS and XLSTAT for completeness due to the numerous ways that different softwares handles type I, II and III sums of squares with unbalanced designs/missing data.

The p-value for the product effect uses a simple standard consumer science model including terms for assessor, product and assessor by product interaction. In the case of replicates the product effect F-test is computed against the interaction MS, otherwise it is test against the MS error.

References

Verhoef, A., Huijberts, G., Vaessen, W. (2015). Introduction of a quality index, based on Generalizability theory, as a measure of reliability for univariate- and multivariate sensory descriptive data. Food Quality & Preference, 40, 296-303

    • Related Articles

    • R-Index

      Introduction The R-index is a tool in sensory science that helps quantify the difference between two samples. The R-index is designed to quantify the area under an empirical Receiver Operating Characteristics (ROC) curve within the framework of ...

    • R-Index (ranked data)

      Purpose R-index is a signal detection measure that assesses the degree to which assessors can discriminate between a pair of products for a given attribute. R-index is expressed as a proportion that represents the probability that the one product is ...

    • R-index Analysis

      Purpose To analyse the results of an R-index test. Data Format R_index_rank_withMD.xlsx Background The R-index applies signal detection theory as an alternative approach to discrimination testing. It is often used where there are a large number of ...

    • Quality Scoring / Rating / Grading

      Introduction This template lets panelists assign a numerical score to each sample, and all samples are displayed at once. Template Description The template starts with a screen which can be used for welcoming panellists or asking additional questions ...

    • Penalty Analysis

      Purpose To provide a penalty analysis of a consumer data set, that is to investigate how liking or acceptability of product decreases when product attributes are not at the optimal intensity. Data Format Consumer.xlsx Note: for EyeOpenR to read your ...