Interpret MICOM report

Questions about the implementation and application of the PLS-SEM method, that are not related to the usage of the SmartPLS software.
Post Reply
cherrando
PLS Junior User
Posts: 2
Joined: Fri Jan 05, 2018 6:13 pm
Real name and title: Carolina Herrando (PhD Student)

Interpret MICOM report

Post by cherrando »

Dear everyone,

I am conducting a cross-cultural analysis, testing it with PLS-MGA, and I am not sure about how to interpret the data of the MICOM report in order to assess measurement invariance. The problem is that when I have to confirm whether the means and the variances are equal, I do not know which colum establish full measurement invariance. Do I have to check the "permutation p-value"? I have checked several papers and, with the data they show in the table of the results of the invariance measurement testing, I do not understand which is the cutoff of differences in means and variances, or how different can be the cofidence interval, and they do not report the permutation p-value.

I would apppreciate if someone could help me with the interpretation of the MICOM steps to establish measurement invariance.

Thank you very much beforehand!
Carolina

I have already checked the following papers:
Jörg Henseler, Christian M. Ringle, Marko Sarstedt, (2016) "Testing measurement invariance of composites using partial least squares", International Marketing Review, Vol. 33 Issue: 3, pp.405-431.

Schlägel, C., Sarstedt, M, (2016) "Assessing the measurement invariance of the four-dimensional cultural intelligence scale across countries: A composite model approach", European Management Journal, Vol.34 Issue: 6, pp. 633-649.
jmbecker
SmartPLS Developer
Posts: 1282
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Interpret MICOM report

Post by jmbecker »

You should check whether the original difference in means and variances falls within the 95% confidence interval generated by the permutation approach.

The two-sided permutation p-value that we report in this section does more or less the same: It is the proportion of sampled permutations where the absolute difference |du| are larger than or equal to the (absolute) original difference |d|.

Generally, the two approaches should give you the same result (unless the sampled permutation distribution is strongly asymmetric)
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
cherrando
PLS Junior User
Posts: 2
Joined: Fri Jan 05, 2018 6:13 pm
Real name and title: Carolina Herrando (PhD Student)

Re: Interpret MICOM report

Post by cherrando »

Dear Professor Becker,

Thank you very much for your help!

All the best,
Carolina
MeiPeng
PLS User
Posts: 15
Joined: Sun May 24, 2015 3:33 am
Real name and title: LowMeiPeng

Interpreting MICOM results Step 2 and Step 3

Post by MeiPeng »

Dear Experts/ Prof. Becker

Thanks Prof. Becker for the guide.

May I ask the following questions:
- in interpreting the MICOM report as per Step 2. We only read whether the original mean difference falls within the 95% CI generated through permutation, OR, we have to simultaneously fulfil TWO conditions, whereby i) original difference means falls within the 95% CI AND ii) permutation p value > 0.05?

- In determining the configural variance, it is possible to have the original mean difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original mean difference falls outside the 95% CI BUT, p value > 0.05?

- In determining the compositional variance, it is possible to have the original variance difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original variance difference falls outside the 95% CI BUT, p value > 0.05?

Looking forward to your reply.

Thanks and Regards
mei peng
jmbecker wrote:You should check whether the original difference in means and variances falls within the 95% confidence interval generated by the permutation approach.

The two-sided permutation p-value that we report in this section does more or less the same: It is the proportion of sampled permutations where the absolute difference |du| are larger than or equal to the (absolute) original difference |d|.

Generally, the two approaches should give you the same result (unless the sampled permutation distribution is strongly asymmetric)
jmbecker
SmartPLS Developer
Posts: 1282
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Interpret MICOM report

Post by jmbecker »

MeiPeng wrote: - in interpreting the MICOM report as per Step 2. We only read whether the original mean difference falls within the 95% CI generated through permutation, OR, we have to simultaneously fulfil TWO conditions, whereby i) original difference means falls within the 95% CI AND ii) permutation p value > 0.05?
Usually the p value and the CI agree. Therefore, there should be no difference between AND or OR. Only in situations where the confidence interval is not symmetric and the value is on the edge of the confidence interval, the two might disagree. In this case, the CI, however, should give a better representation as it accounts for the skewness of the sampling distribution, while the p value assumes a symmetric distribution allow it does not exist.
MeiPeng wrote: - In determining the configural variance, it is possible to have the original mean difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original mean difference falls outside the 95% CI BUT, p value > 0.05?
Configural invariance in MICOM is Step 1. It is always assured within SmartPLS 3. You do not look at any results for this.
MeiPeng wrote: - In determining the compositional variance, it is possible to have the original variance difference falls within the 95% CI BUT, p value < 0.05 and
the reverse whereby original variance difference falls outside the 95% CI BUT, p value > 0.05?
Compositional invariance is assessed in Step 2 by analyzing the correlations. The In can be confirmed independent of the results of Step 3, which you seem to look at (i.e., equal means and equal variances).
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
MeiPeng
PLS User
Posts: 15
Joined: Sun May 24, 2015 3:33 am
Real name and title: LowMeiPeng

Re: Interpret MICOM report_Report Writing

Post by MeiPeng »

Dear Prof. Becker

Thank you for your swift reply and clarying my doubts.
jmbecker wrote:Usually the p value and the CI agree. Therefore, there should be no difference between AND or OR. Only in situations where the confidence interval is not symmetric and the value is on the edge of the confidence interval, the two might disagree. In this case, the CI, however, should give a better representation as it accounts for the skewness of the sampling distribution, while the p value assumes a symmetric distribution allow it does not exist.
Am I correct to say the following:
- CI (confidence interval) is the main reference to establish compositional invariance
- it is possible yet unlikely that CI and p value DO NOT agree with each other
- if the CI and p value do not agree with each other, it doesn't mean that there is an error in data, it just implies that there is no symmetric distribution


My next query is pertaining to report writing
- if CI and p value does not agree with each other, I should refer to CI as it is the main reference to determine compositional invariance. Do I need to explain why p value is < 0.05, yet it is compositional invariance being established? Or I should take it as it is known and no explanation required?

Thank you.

Best regards
mei peng
jmbecker
SmartPLS Developer
Posts: 1282
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Interpret MICOM report_Report Writing

Post by jmbecker »

MeiPeng wrote: Am I correct to say the following:
- CI (confidence interval) is the main reference to establish compositional invariance
- it is possible yet unlikely that CI and p value DO NOT agree with each other
- if the CI and p value do not agree with each other, it doesn't mean that there is an error in data, it just implies that there is no symmetric distribution
Yes.
MeiPeng wrote: My next query is pertaining to report writing
- if CI and p value does not agree with each other, I should refer to CI as it is the main reference to determine compositional invariance. Do I need to explain why p value is < 0.05, yet it is compositional invariance being established? Or I should take it as it is known and no explanation required?
That is your decision on how much space you want to devote to that discussion.
I would probably only report the CI as it is the better approach and would not bother readers with (in this case) unnecessary information. For me, the p-values are good to easily spot what is going on at the first glance, but for the reporting and final analysis I would always use confidence intervals.
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
MeiPeng
PLS User
Posts: 15
Joined: Sun May 24, 2015 3:33 am
Real name and title: LowMeiPeng

Re: Interpret MICOM report

Post by MeiPeng »

Thanks, Prof. Becker for your quick reply. Appreciate it very much.

best regards
mei peng
Post Reply