Comparisons (GLM)

Multiple Comparisons
Sidak Method - Hypothesis Test

  

Use the adjusted p-values provided with the hypothesis tests to determine whether pairs of means are different. These p-values are adjusted to maintain the family error rate:

·    If the adjusted p-value for a comparison is less than or equal to your chosen a-level, the difference between the means is significant.

·    If the adjusted p-value is greater than your chosen a-level, the difference between means is not significant.

The simultaneous confidence level is particularly important when you perform multiple comparisons. For any given individual confidence level, the chance that at least one confidence interval does not contain the true difference increases with the number of comparisons. Consequently, the simultaneous confidence level is always lower than the individual confidence level for multiple comparisons.

The individual confidence level is the percentage of times that a single confidence interval includes the true difference between factor levels if the study were repeated multiple times.

The simultaneous confidence level is the percentage of times that a group of confidence intervals all include the true differences between factor levels if the study were repeated multiple times.

Example Output

Sidak Simultaneous Tests for Differences of Means

 

Difference

of Subject  Difference       SE of   Simultaneous 95%            Adjusted

Levels        of Means  Difference          CI          T-Value   P-Value

2 - 1           0.6567      0.0664  ( 0.4708,  0.8425)     9.89     0.000

3 - 1           0.9633      0.0708  ( 0.7651,  1.1615)    13.60     0.000

4 - 1           0.6900      0.0750  ( 0.4802,  0.8998)     9.20     0.000

3 - 2           0.3067      0.0632  ( 0.1299,  0.4834)     4.86     0.000

4 - 2           0.0333      0.0678  (-0.1563,  0.2230)     0.49     0.997

4 - 3          -0.2733      0.0721  (-0.4751, -0.0716)    -3.79     0.004

 

Individual confidence level = 99.15%

Interpretation

Pairwise comparisons were conducted for the subject factor of the salary analysis. Because there are four levels of subject, this produces six pairwise comparisons. The hypothesis tests for the comparisons reveal the following:

·    The adjusted p-values for the differences between the mean for subject 1, and the means for subjects 2 (0.000), 3 (0.000), and 4 (0.000) are all lower than the chosen a-level of 0.05, which indicates that these differences are significant. Furthermore, the differences between the means (Difference of Means) are all positive, which indicates that teaching subjects 2, 3, and 4 each paid better than teaching subject 1.

·    The adjusted p-value for the difference between the means for subjects 2 and 3 (0.000) indicates that these means are significantly different as well. Furthermore, the difference (0.3067) is positive, which indicates that the mean for subject 3 is greater than that for subject 2.

·    The adjusted p-value for the difference between the means for subjects 2 and 4 (0.997) is greater than the chosen a-level, which indicates that these means are not significantly different.

·    The adjusted p-value for the difference between the means for subjects 3 and 4 (0.004) is lower than the chosen a-level, which indicates that the mean for subject 4 is significantly different from that of subject 3. Furthermore, the difference between the means (-0.2733) is negative, which indicates that teaching subject 3 paid better than teaching subject 4.