Dear everyone,
I am conducting a cross-cultural analysis, testing it with PLS-MGA, and I am not sure about how to interpret the data of the MICOM report in order to assess measurement invariance. The problem is that when I have to confirm whether the means and the variances are equal, I do not know which colum establish full measurement invariance. Do I have to check the "permutation p-value"? I have checked several papers and, with the data they show in the table of the results of the invariance measurement testing, I do not understand which is the cutoff of differences in means and variances, or how different can be the cofidence interval, and they do not report the permutation p-value.
I would apppreciate if someone could help me with the interpretation of the MICOM steps to establish measurement invariance.
Thank you very much beforehand!
Carolina
I have already checked the following papers:
Jörg Henseler, Christian M. Ringle, Marko Sarstedt, (2016) "Testing measurement invariance of composites using partial least squares", International Marketing Review, Vol. 33 Issue: 3, pp.405-431.
Schlägel, C., Sarstedt, M, (2016) "Assessing the measurement invariance of the four-dimensional cultural intelligence scale across countries: A composite model approach", European Management Journal, Vol.34 Issue: 6, pp. 633-649.
Interpret MICOM report
-
- SmartPLS Developer
- Posts: 1284
- Joined: Tue Mar 28, 2006 11:09 am
- Real name and title: Dr. Jan-Michael Becker
Re: Interpret MICOM report
You should check whether the original difference in means and variances falls within the 95% confidence interval generated by the permutation approach.
The two-sided permutation p-value that we report in this section does more or less the same: It is the proportion of sampled permutations where the absolute difference |du| are larger than or equal to the (absolute) original difference |d|.
Generally, the two approaches should give you the same result (unless the sampled permutation distribution is strongly asymmetric)
The two-sided permutation p-value that we report in this section does more or less the same: It is the proportion of sampled permutations where the absolute difference |du| are larger than or equal to the (absolute) original difference |d|.
Generally, the two approaches should give you the same result (unless the sampled permutation distribution is strongly asymmetric)
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
-
- PLS Junior User
- Posts: 2
- Joined: Fri Jan 05, 2018 6:13 pm
- Real name and title: Carolina Herrando (PhD Student)
Re: Interpret MICOM report
Dear Professor Becker,
Thank you very much for your help!
All the best,
Carolina
Thank you very much for your help!
All the best,
Carolina
Interpreting MICOM results Step 2 and Step 3
Dear Experts/ Prof. Becker
Thanks Prof. Becker for the guide.
May I ask the following questions:
- in interpreting the MICOM report as per Step 2. We only read whether the original mean difference falls within the 95% CI generated through permutation, OR, we have to simultaneously fulfil TWO conditions, whereby i) original difference means falls within the 95% CI AND ii) permutation p value > 0.05?
- In determining the configural variance, it is possible to have the original mean difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original mean difference falls outside the 95% CI BUT, p value > 0.05?
- In determining the compositional variance, it is possible to have the original variance difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original variance difference falls outside the 95% CI BUT, p value > 0.05?
Looking forward to your reply.
Thanks and Regards
mei peng
Thanks Prof. Becker for the guide.
May I ask the following questions:
- in interpreting the MICOM report as per Step 2. We only read whether the original mean difference falls within the 95% CI generated through permutation, OR, we have to simultaneously fulfil TWO conditions, whereby i) original difference means falls within the 95% CI AND ii) permutation p value > 0.05?
- In determining the configural variance, it is possible to have the original mean difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original mean difference falls outside the 95% CI BUT, p value > 0.05?
- In determining the compositional variance, it is possible to have the original variance difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original variance difference falls outside the 95% CI BUT, p value > 0.05?
Looking forward to your reply.
Thanks and Regards
mei peng
jmbecker wrote:You should check whether the original difference in means and variances falls within the 95% confidence interval generated by the permutation approach.
The two-sided permutation p-value that we report in this section does more or less the same: It is the proportion of sampled permutations where the absolute difference |du| are larger than or equal to the (absolute) original difference |d|.
Generally, the two approaches should give you the same result (unless the sampled permutation distribution is strongly asymmetric)
-
- SmartPLS Developer
- Posts: 1284
- Joined: Tue Mar 28, 2006 11:09 am
- Real name and title: Dr. Jan-Michael Becker
Re: Interpret MICOM report
Usually the p value and the CI agree. Therefore, there should be no difference between AND or OR. Only in situations where the confidence interval is not symmetric and the value is on the edge of the confidence interval, the two might disagree. In this case, the CI, however, should give a better representation as it accounts for the skewness of the sampling distribution, while the p value assumes a symmetric distribution allow it does not exist.MeiPeng wrote: - in interpreting the MICOM report as per Step 2. We only read whether the original mean difference falls within the 95% CI generated through permutation, OR, we have to simultaneously fulfil TWO conditions, whereby i) original difference means falls within the 95% CI AND ii) permutation p value > 0.05?
Configural invariance in MICOM is Step 1. It is always assured within SmartPLS 3. You do not look at any results for this.MeiPeng wrote: - In determining the configural variance, it is possible to have the original mean difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original mean difference falls outside the 95% CI BUT, p value > 0.05?
Compositional invariance is assessed in Step 2 by analyzing the correlations. The In can be confirmed independent of the results of Step 3, which you seem to look at (i.e., equal means and equal variances).MeiPeng wrote: - In determining the compositional variance, it is possible to have the original variance difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original variance difference falls outside the 95% CI BUT, p value > 0.05?
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
Re: Interpret MICOM report_Report Writing
Dear Prof. Becker
Thank you for your swift reply and clarying my doubts.
- CI (confidence interval) is the main reference to establish compositional invariance
- it is possible yet unlikely that CI and p value DO NOT agree with each other
- if the CI and p value do not agree with each other, it doesn't mean that there is an error in data, it just implies that there is no symmetric distribution
My next query is pertaining to report writing
- if CI and p value does not agree with each other, I should refer to CI as it is the main reference to determine compositional invariance. Do I need to explain why p value is < 0.05, yet it is compositional invariance being established? Or I should take it as it is known and no explanation required?
Thank you.
Best regards
mei peng
Thank you for your swift reply and clarying my doubts.
Am I correct to say the following:jmbecker wrote:Usually the p value and the CI agree. Therefore, there should be no difference between AND or OR. Only in situations where the confidence interval is not symmetric and the value is on the edge of the confidence interval, the two might disagree. In this case, the CI, however, should give a better representation as it accounts for the skewness of the sampling distribution, while the p value assumes a symmetric distribution allow it does not exist.
- CI (confidence interval) is the main reference to establish compositional invariance
- it is possible yet unlikely that CI and p value DO NOT agree with each other
- if the CI and p value do not agree with each other, it doesn't mean that there is an error in data, it just implies that there is no symmetric distribution
My next query is pertaining to report writing
- if CI and p value does not agree with each other, I should refer to CI as it is the main reference to determine compositional invariance. Do I need to explain why p value is < 0.05, yet it is compositional invariance being established? Or I should take it as it is known and no explanation required?
Thank you.
Best regards
mei peng
-
- SmartPLS Developer
- Posts: 1284
- Joined: Tue Mar 28, 2006 11:09 am
- Real name and title: Dr. Jan-Michael Becker
Re: Interpret MICOM report_Report Writing
Yes.MeiPeng wrote: Am I correct to say the following:
- CI (confidence interval) is the main reference to establish compositional invariance
- it is possible yet unlikely that CI and p value DO NOT agree with each other
- if the CI and p value do not agree with each other, it doesn't mean that there is an error in data, it just implies that there is no symmetric distribution
That is your decision on how much space you want to devote to that discussion.MeiPeng wrote: My next query is pertaining to report writing
- if CI and p value does not agree with each other, I should refer to CI as it is the main reference to determine compositional invariance. Do I need to explain why p value is < 0.05, yet it is compositional invariance being established? Or I should take it as it is known and no explanation required?
I would probably only report the CI as it is the better approach and would not bother readers with (in this case) unnecessary information. For me, the p-values are good to easily spot what is going on at the first glance, but for the reporting and final analysis I would always use confidence intervals.
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
Re: Interpret MICOM report
Thanks, Prof. Becker for your quick reply. Appreciate it very much.
best regards
mei peng
best regards
mei peng