Purpose
To make a statistical test for differences between two means. Or in the case that there are more than two samples, make separate difference tests for each possible pair of samples.
The test is quite general and can be applied in any scenario where numeric data has been collected on two or more samples. T-tests were developed specifically for occasions when the amount of data available is limited, so there is no minimum sample size.
Background
There are two variants of the t-test, firstly a 2-sample, or unpaired, t-test where data is collected independently on each sample – for example there are two meal kits under test, one group of consumers is given kit A to prepare and test at home, while a different group of consumers is assigned kit B to test at home. All consumers provide a liking rating of their kit, and the unpaired test is used to see if one product is liked more than the other.
The second variant is a paired t-test where data is collected in pairs. In sensory and consumer science these data pairs nearly always come from two products evaluated sequentially by the same assessor. The paired test then seeks to find out if the average difference between the data pairs is different from zero. The paired test is more sensitive than the unpaired test since unwanted variation, such as different use of scale by different assessors, is removed by taking the difference at the individual level.
Options
- Comparison between - Is the comparison you want to make, between products (default), or between attributes? If the latter option is selected, then there must be two or more attributes for analysis and the test is made between all the scores for each attribute. Note that ‘between attributes’ may be particularly relevant if you have ‘wide’ format data where the scores for different products are each in their own columns.
- Type of T-test – either paired (default) or unpaired (see Background section above for description of test types).
- Assume equal variances – yes (default) or no. In the standard unpaired t-test, equal variances for each sample are assumed. If you believe that the data for each sample might have very different variances, consider checking ‘no’ to perform Welch’s t-test that makes a correction to the degrees of freedom using the Welch–Satterthwaite equation and uses an adjusted standard error to account for the unequal variation. Note that this option has no effect when a paired t-test is performed.
- Significance Level – specify the alpha level for the t-test as a percentage (default 5% which corresponds to alpha=0.05).
- Number of decimals for values – controls the number of decimals printed in the output for the mean difference, t-statistic, and upper and lower confidence limits.
- Number of decimals for p-values – controls the number of decimals printed for p-values.
Results and Interpretation
- If the comparison between products option is selected, then there are two tables of output for each attribute analysed.
- The ‘T-test’ table contains the main results, where each row corresponds to a test of one sample against another. Note that the sign of the difference is important, and the first column of the table indicates the direction e.g. ‘Product 1 – Product 2’ should be read as product 1 minus product 2. The 2nd column of the table gives the average value of this directional difference. The next three columns give the results of the performing the t-test itself: the t-statistic, the number of degrees of freedom associated with the test, and the p-value under the null hypothesis of no difference. The final two columns give the upper and lower confidence interval (at the selected significance level) for the difference printed in the 2nd column. Confidence intervals that do not include zero will be associated with the smaller p-values that suggest statistically significant differences between samples. It is down to the user to choose their own threshold, but it is customary to declare mean differences with an associated p-value smaller than 0.05 as being significant.
- The ‘P-values’ table presents the p-values from the 5th column of the T-test table in a square symmetric format. Both rows and columns are indexed by the products, and so the table presents the results of all pairwise comparisons between products in a convenient format.
- If the comparison between attributes option is selected then there will only be two tables of output in the same format as described above, however, the rows of the T-test table are indexed by all possible comparisons between attributes.
- The R function t.test( ) from the package ‘stats’ is used for all calculations.
- All tests are two-sided.
References
- Martin Bland (2015) “An Introduction to Medical Statistics – 4th Edition”, Oxford University Press. See chapter 10 “Comparing the means of small samples”.