R-Index (ranked data)

R-Index (ranked data)

Purpose 

R-index is a signal detection measure that assesses the degree to which assessors can discriminate between a pair of products for a given attribute. R-index is expressed as a proportion that represents the probability that the one product is stronger for an attribute in a paired comparison, so the r-index of product A compared to product B is 100 minus the r-index of product B compared to product A.

Sometimes products are assessed by asking assessors to rank them for a particular attribute, for example asking them to rank products by their sweetness. R-index is a distribution-free method that can use the information about how different assessors rank the products to determine the degree of difference between them.

It is a pairwise test where often there are test products being compared to a standard.

Data Format

  1. R-index_rank.xlsx

For EyeOpenR to read your data, the first five columns must be in the following order: assessor (consumer), product, session, replicate and order. The attribute data with the ranks should be in the sixth column (column F) onwards. There should be one column for each attribute.

The analysis requires a complete design and if an incomplete design is detected the analysis will not run and a note will be added to the information tab in the results. The session, replicate and order information is not used for this analysis and these columns should contain the value ‘1’ in each cell (order can be recorded as ‘NA’).

If two or more products are tied for an assessor the rank must be the average of the positions that they represent. Consider the example where each assessor has ranked the sweetness of 5 products, A, B, C, D and E. Here are a selection of ranks that could be recorded:

  1. The assessor can discriminate between all five products and ranks them in the following order: B, D, A, C, E. The ranks are recorded as A=3, B=1, C=4, D=2, E=5.
  2. The assessor cannot discriminate between D and A and ranks them in the following order: B, D & A, C, E. The ranks are recorded as A=2.5, B=1, C=4, D=2.5, E=5.
  3. The assessor cannot discriminate between D, A and C  and ranks them in the following order: B, D & A & C, E. The ranks are recorded as A=3, B=1, C=3, D=3, E=5.

The data sheet is necessary, the other sheets provide metadata and if they are not in the spreadsheet you can assign the metadata in EyeOpenR.

See the example spreadsheet for an illustration of the data format.

Background 

R-index is a signal detection measure that assesses the degree to which assessors can discriminate between a pair of products for a given attribute. As well as being used to determine which pairs of products can be discriminated it can also be used to determine how different two samples must be before assessors are confident enough to report a difference.

R-index is reported as a probability that product A is preferred to product B. An r-index of 100% indicates that the products are perfectly discriminated as different and product A is stronger for the attribute being measured than product B. An r-index of 50% indicates that the products are indistinguishable from each other. An r-index of 0% indicates that the products are perfectly discriminated as different and product B is stronger for the attribute than product A.

Untrained assessors may find it easier to rank products for a given attribute rather than scoring on a rating scale, e.g. sort products based on sweetness. It is generally quicker and easier to collect data in this way. R-index is a distribution-free method that can use the ranking information to measure how well assessors can discriminate between the products.

R-index is a pairwise test and therefore it is run for each pair of products tested. It is not a multiple comparison test and no adjustment is made for the multiple comparisons.

Options

  1. Levels of significance: There are three options, each with three levels of significance. These significance levels are used in the ‘critical values’ and ‘r-index and significance’ tabs in the results for the assessment of the size of r-index of each pair of products.
  2. Number of decimals for values: Specify the number of decimal places shown for values in the results.
  3. Number of decimals for p-values: Specify the number of decimal places shown for p-values in the results.

Results and Interpretation

The results described below are shown for each attribute separately. They are accessed by selecting the attribute from the tabs below the results tab.

  1. Frequency: This shows a table with the ranks in the columns and the products in the rows with each cell in the table showing the number of assessors who assigned each product each rank. This table is useful for checking whether there is a product that divides opinion between assessors.
  2. Sum of Rank: This table show the sum of the ranks for each product. Letters are shown as superscripts next to the sum of ranks. Products with the same letters are not significantly different at 5%. This is based on the r-index probability values.
  3. R-index: This shows a matrix of the pairwise comparisons, with each cell representing the r-index of the product in the row relative to the product in the column. If the value is greater than 50 then more assessors ranked the product in the row higher than the product in column, less than 50 is the other way around. The maximum r-index is 100 and the minimum is 0.
  4. P-values: This shows the same matrix with each cell representing the probability observing the r-index shown in the r-index tab if the assessors could not discriminate between the products. A low p-value indicates that assessors are more likely to be able to discriminate between the products.
  5. Critical values: This table shows the values of r-index that relate to the three levels of significance selected in the option ‘levels of significance’. There is one value for low r-index and one for high r-index. Values lower than the low value or higher than the high value have reached those levels of significance. The values shown in this table vary depending on the number of assessors.
  6. R-index and significance: This table combines the information from the previous results tabs and shows the matrix of r-index values, with stars to indicate the level of significance (relative to the critical values) and ‘ns’ if the r-index is between the low and high value for the lowest level of significance. The cells of the table are coloured blue if the product in the row is more likely to be ranked higher than the product in the column and coloured red if the product in the column is more likely to be ranked higher than the product in the row.

Technical Information

There are no specialist R packages used.

References 

  1. Brown J. 1974. Recognition assessed by rating and ranking. Br. J. Psychol. 65, 13–22.
  2. O’Mahony M. 1992. Understanding Discrimination Tests: A user friendly treatment of response bias, rating and ranking R-index tests and their relationship to signal detection. Journal of Sensory Studies 7, 1-47
  3. Bi J. 2006. Statistical analyses for R-index. Journal of Sensory Studies 21, 584–600.

    • Related Articles

    • R-Index

      Introduction The R-index is a tool in sensory science that helps quantify the difference between two samples. The R-index is designed to quantify the area under an empirical Receiver Operating Characteristics (ROC) curve within the framework of ...
    • Ranking and Ranking (continuous data)

      Purpose Sometimes products are assessed by asking assessors to rank them for a particular attribute, for example asking them to rank products by their sweetness. The Ranking module uses Friedman’s test to assess whether there are any differences ...
    • Ranking

      Introduction The aim of ranking as a sensory methodology is to compare a set of products in terms of their perceived intensities on a set of defined sensory attributes. The process is designed to be simple: for each attribute separately in turn, the ...
    • R-index Analysis

      Purpose To analyse the results of an R-index test. Data Format R_index_rank_withMD.xlsx Background The R-index applies signal detection theory as an alternative approach to discrimination testing. It is often used where there are a large number of ...
    • Quality Index Analysis

      Available from version: 5.4.4 Purpose The Quality Index is an ANOVA based analysis, with the idea based on the paper by Verhoef (2015) for the purpose of measuring the reliability of univariate sensory descriptive data. Data format profiling.xlsx The ...