Comparisons (GLM)

Multiple Comparisons
Dunnett Method - Hypothesis Test

  

Use the adjusted p-values provided with the hypothesis tests to determine whether pairs of means are different. These p-values are adjusted to maintain the family error rate at the desired a-level (in this case 0.05):

·    If the adjusted p-value for a comparison is less than or equal to your chosen a-level, the difference between the means is significant.

·    If the adjusted p-value is greater than your chosen a-level, the difference between means is not significant.

The simultaneous confidence level is particularly important when you perform multiple comparisons. For any given individual confidence level, the chance that at least one confidence interval does not contain the true difference increases with the number of comparisons. Consequently, the simultaneous confidence level is always lower than the individual confidence level for multiple comparisons.

The individual confidence level is the percentage of times that a single confidence interval includes the true difference between factor levels if the study were repeated multiple times.

The simultaneous confidence level is the percentage of times that a group of confidence intervals all include the true differences between factor levels if the study were repeated multiple times.

Example Output

Dunnett Simultaneous Tests for Level Mean - Control Mean

 

Difference

of Subject  Difference       SE of   Simultaneous 95%            Adjusted

Levels        of Means  Difference          CI          T-Value   P-Value

1 - 4          -0.6900      0.0750  (-0.8734, -0.5066)    -9.20     0.000

2 - 4          -0.0333      0.0678  (-0.1991,  0.1324)    -0.49     0.921

3 - 4           0.2733      0.0721  ( 0.0970,  0.4497)     3.79     0.002

 

Individual confidence level = 98.00%

Interpretation

For the salary analysis, comparisons were requested between the mean for subject 4 (the designated control level) and the means for all other levels of the subject factor. The confidence intervals for the comparisons reveal the following:

·    The adjusted p-value for the difference between the means for subject 4 and 1 (0.000) is lower than the chosen a-level of 0.05, which indicates that this difference is significant. Furthermore, the difference (Differences of Means, -0.6900) is negative, which indicates that teaching subject 1 paid less than teaching subject 4.

·    The adjusted p-value for the difference between the means for subjects 4 and 2 (0.921) is greater than the chosen a-level, which indicates that these means are not significantly different.

·    The adjusted p-value for the difference between the means for subjects 4 and 3 (0.002) is lower than the chosen a-level, which indicates that the mean for subject 4 is significantly different from that of subject 3. Furthermore, the difference between the means (0.2733) is positive, which indicates that teaching subject 3 paid better than teaching subject 4.