MICOM: variances invariance established but not means invariance
Posted: Fri Nov 02, 2018 11:33 am
I have two data sets, based on the same survey questions but collected with one year apart. The first set contains 1249 cases and the second set contains 875 cases. I have performed a MICOM analysis to see whether pooling the two data sets is possible.
Configural invariance (step 1) has been established and after running a permutation test in SmartPLS 3, partial compositional invariance (step 2) could also be established. However, concerning step 3, testing whether full measurement invariance holds, I get statistically significant p-values for means but not for variances. That is, if means were the sole focus for this test, full measurement invariance would not have been established. If, on the other hand, variances were the sole focus for this test, full measurement invariance would have been established.
Since the litterature states that both means and variances should be equal for full measurement invariance to be established, I interpret my results as supporting that partial but not full compositional measurement invariance has been established. Thus, multigroup analyses are feasible while pooling data is not.
But what does it mean when the third step of the MICOM test results in contradicting p-values for means and variances? Is this common? Should this be interpreted as a different kind of "failed step 3" compared to when both means and variances produce significant p-values?
[EDIT] I forgot to mention that in my PLS path model, I make use of higher order constructs, measured with latent variable scores. In a comparison of the results from a permutation test of constructs based on manifest indicator variables with the results from a test based on latent variable scores, it appears as if the latent variable score based constructs perform worse in terms of achieved full measurement invariance. My interpretation is that using latent variable scores will mess up the means invariance but not the variance invariance. Or am I reaching?
Configural invariance (step 1) has been established and after running a permutation test in SmartPLS 3, partial compositional invariance (step 2) could also be established. However, concerning step 3, testing whether full measurement invariance holds, I get statistically significant p-values for means but not for variances. That is, if means were the sole focus for this test, full measurement invariance would not have been established. If, on the other hand, variances were the sole focus for this test, full measurement invariance would have been established.
Since the litterature states that both means and variances should be equal for full measurement invariance to be established, I interpret my results as supporting that partial but not full compositional measurement invariance has been established. Thus, multigroup analyses are feasible while pooling data is not.
But what does it mean when the third step of the MICOM test results in contradicting p-values for means and variances? Is this common? Should this be interpreted as a different kind of "failed step 3" compared to when both means and variances produce significant p-values?
[EDIT] I forgot to mention that in my PLS path model, I make use of higher order constructs, measured with latent variable scores. In a comparison of the results from a permutation test of constructs based on manifest indicator variables with the results from a test based on latent variable scores, it appears as if the latent variable score based constructs perform worse in terms of achieved full measurement invariance. My interpretation is that using latent variable scores will mess up the means invariance but not the variance invariance. Or am I reaching?