PC in groups not sig, but difference between groups sig/ MGA

[beautifulgarbage]

Hello everyone,

I am conducting a MGA between two groups and for one path, strangely, I find a significant difference between the groups (MGA test), although this path coefficient is NOT significant in either of the two groups. I am searching for a plausible explanation of this fact- does anyone have one? Generally, having significant path coefficients in the single groups is not a prerequisite for applying a MGA, right?

I would be very grateful if someone could help me out.

Thanks again and best regards

BG

[jmbecker]

It is definitely a strange result and should be further investigated.
Generally, the difference between significant and insignificant must not be significant. This is, however, something different that your situation, but closely related. The difference test and the normal test are two different test and results from the one should not matter for the other. Both effects could be barely insignificant, but on different sides of the zero. If the one is negative and the other positive, but barely insignificant it could be that the difference is indeed significant, although both effects separately are not significant. It would be a statistical artifact that is hard to interpret from theory, but which could happen without rendering the methods invalid.

However, I would try to rule out possible confounding effects first, like, for example, measurement inequality between groups, etc...

[beautifulgarbage]

Dear Dr. Becker,

thank you very much for the quick reply. Indeed, the path coefficient in one of the groups is "barely insignificant at the 5% level" (significant at the 10% level) and also one is below 0 and one above, so it is probably exactly the situation you are talking about. Another question in that regard: do you have a nice source explaining the related situation you mentioned, namely one significant and one insignificant path, but un insignificant difference between them? In my model I also have this one, as well, and would like to inform myself in details on that.

Thank you very uch once again and best regards
BG

[jmbecker]

There are several articles, if you search for it. This is one prominent example: Gelman, A., & Stern, H. (2006). The difference between “significant” and “not significant” is not itself statistically significant. The American Statistician, 60(4), 328-331.

[beautifulgarbage]

Thank you very much!