Interpret MICOM report

PLS is broadly applied in modern business research. This forum is the right place for discussions on the use of PLS in the fields of Marketing, Strategic Management, Information Technology etc.
Post Reply
cherrando
PLS Junior User
Posts: 2
Joined: Fri Jan 05, 2018 6:13 pm
Real name and title: Carolina Herrando (PhD Student)

Interpret MICOM report

Post by cherrando » Fri Jan 05, 2018 6:53 pm

Dear everyone,

I am conducting a cross-cultural analysis, testing it with PLS-MGA, and I am not sure about how to interpret the data of the MICOM report in order to assess measurement invariance. The problem is that when I have to confirm whether the means and the variances are equal, I do not know which colum establish full measurement invariance. Do I have to check the "permutation p-value"? I have checked several papers and, with the data they show in the table of the results of the invariance measurement testing, I do not understand which is the cutoff of differences in means and variances, or how different can be the cofidence interval, and they do not report the permutation p-value.

I would apppreciate if someone could help me with the interpretation of the MICOM steps to establish measurement invariance.

Thank you very much beforehand!
Carolina

I have already checked the following papers:
Jörg Henseler, Christian M. Ringle, Marko Sarstedt, (2016) "Testing measurement invariance of composites using partial least squares", International Marketing Review, Vol. 33 Issue: 3, pp.405-431.

Schlägel, C., Sarstedt, M, (2016) "Assessing the measurement invariance of the four-dimensional cultural intelligence scale across countries: A composite model approach", European Management Journal, Vol.34 Issue: 6, pp. 633-649.

jmbecker
SmartPLS Developer
Posts: 821
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Interpret MICOM report

Post by jmbecker » Sat Jan 06, 2018 5:02 pm

You should check whether the original difference in means and variances falls within the 95% confidence interval generated by the permutation approach.

The two-sided permutation p-value that we report in this section does more or less the same: It is the proportion of sampled permutations where the absolute difference |du| are larger than or equal to the (absolute) original difference |d|.

Generally, the two approaches should give you the same result (unless the sampled permutation distribution is strongly asymmetric)
Dr. Jan-Michael Becker, University of Cologne, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Ja ... v=hdr_xprf
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de

cherrando
PLS Junior User
Posts: 2
Joined: Fri Jan 05, 2018 6:13 pm
Real name and title: Carolina Herrando (PhD Student)

Re: Interpret MICOM report

Post by cherrando » Sun Jan 07, 2018 5:36 pm

Dear Professor Becker,

Thank you very much for your help!

All the best,
Carolina

MeiPeng
PLS User
Posts: 14
Joined: Sun May 24, 2015 3:33 am
Real name and title: LowMeiPeng

Interpreting MICOM results Step 2 and Step 3

Post by MeiPeng » Wed Jan 17, 2018 6:08 am

Dear Experts/ Prof. Becker

Thanks Prof. Becker for the guide.

May I ask the following questions:
- in interpreting the MICOM report as per Step 2. We only read whether the original mean difference falls within the 95% CI generated through permutation, OR, we have to simultaneously fulfil TWO conditions, whereby i) original difference means falls within the 95% CI AND ii) permutation p value > 0.05?

- In determining the configural variance, it is possible to have the original mean difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original mean difference falls outside the 95% CI BUT, p value > 0.05?

- In determining the compositional variance, it is possible to have the original variance difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original variance difference falls outside the 95% CI BUT, p value > 0.05?

Looking forward to your reply.

Thanks and Regards
mei peng
jmbecker wrote:You should check whether the original difference in means and variances falls within the 95% confidence interval generated by the permutation approach.

The two-sided permutation p-value that we report in this section does more or less the same: It is the proportion of sampled permutations where the absolute difference |du| are larger than or equal to the (absolute) original difference |d|.

Generally, the two approaches should give you the same result (unless the sampled permutation distribution is strongly asymmetric)

jmbecker
SmartPLS Developer
Posts: 821
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Interpret MICOM report

Post by jmbecker » Wed Jan 17, 2018 9:35 am

MeiPeng wrote: - in interpreting the MICOM report as per Step 2. We only read whether the original mean difference falls within the 95% CI generated through permutation, OR, we have to simultaneously fulfil TWO conditions, whereby i) original difference means falls within the 95% CI AND ii) permutation p value > 0.05?
Usually the p value and the CI agree. Therefore, there should be no difference between AND or OR. Only in situations where the confidence interval is not symmetric and the value is on the edge of the confidence interval, the two might disagree. In this case, the CI, however, should give a better representation as it accounts for the skewness of the sampling distribution, while the p value assumes a symmetric distribution allow it does not exist.
MeiPeng wrote: - In determining the configural variance, it is possible to have the original mean difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original mean difference falls outside the 95% CI BUT, p value > 0.05?
Configural invariance in MICOM is Step 1. It is always assured within SmartPLS 3. You do not look at any results for this.
MeiPeng wrote: - In determining the compositional variance, it is possible to have the original variance difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original variance difference falls outside the 95% CI BUT, p value > 0.05?
Compositional invariance is assessed in Step 2 by analyzing the correlations. The In can be confirmed independent of the results of Step 3, which you seem to look at (i.e., equal means and equal variances).
Dr. Jan-Michael Becker, University of Cologne, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Ja ... v=hdr_xprf
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de

MeiPeng
PLS User
Posts: 14
Joined: Sun May 24, 2015 3:33 am
Real name and title: LowMeiPeng

Re: Interpret MICOM report_Report Writing

Post by MeiPeng » Thu Jan 18, 2018 6:19 am

Dear Prof. Becker

Thank you for your swift reply and clarying my doubts.
jmbecker wrote:Usually the p value and the CI agree. Therefore, there should be no difference between AND or OR. Only in situations where the confidence interval is not symmetric and the value is on the edge of the confidence interval, the two might disagree. In this case, the CI, however, should give a better representation as it accounts for the skewness of the sampling distribution, while the p value assumes a symmetric distribution allow it does not exist.
Am I correct to say the following:
- CI (confidence interval) is the main reference to establish compositional invariance
- it is possible yet unlikely that CI and p value DO NOT agree with each other
- if the CI and p value do not agree with each other, it doesn't mean that there is an error in data, it just implies that there is no symmetric distribution


My next query is pertaining to report writing
- if CI and p value does not agree with each other, I should refer to CI as it is the main reference to determine compositional invariance. Do I need to explain why p value is < 0.05, yet it is compositional invariance being established? Or I should take it as it is known and no explanation required?

Thank you.

Best regards
mei peng

jmbecker
SmartPLS Developer
Posts: 821
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Interpret MICOM report_Report Writing

Post by jmbecker » Fri Jan 19, 2018 3:15 pm

MeiPeng wrote: Am I correct to say the following:
- CI (confidence interval) is the main reference to establish compositional invariance
- it is possible yet unlikely that CI and p value DO NOT agree with each other
- if the CI and p value do not agree with each other, it doesn't mean that there is an error in data, it just implies that there is no symmetric distribution
Yes.
MeiPeng wrote: My next query is pertaining to report writing
- if CI and p value does not agree with each other, I should refer to CI as it is the main reference to determine compositional invariance. Do I need to explain why p value is < 0.05, yet it is compositional invariance being established? Or I should take it as it is known and no explanation required?
That is your decision on how much space you want to devote to that discussion.
I would probably only report the CI as it is the better approach and would not bother readers with (in this case) unnecessary information. For me, the p-values are good to easily spot what is going on at the first glance, but for the reporting and final analysis I would always use confidence intervals.
Dr. Jan-Michael Becker, University of Cologne, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Ja ... v=hdr_xprf
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de

MeiPeng
PLS User
Posts: 14
Joined: Sun May 24, 2015 3:33 am
Real name and title: LowMeiPeng

Re: Interpret MICOM report

Post by MeiPeng » Mon Jan 22, 2018 11:49 pm

Thanks, Prof. Becker for your quick reply. Appreciate it very much.

best regards
mei peng

Post Reply