High difference in composite reliability between two models

Small talk on SmartPLS that does not correspond to the other forums!
Post Reply
Jules17
PLS Junior User
Posts: 4
Joined: Wed Sep 06, 2017 9:43 am
Real name and title: Julia Lanzl, M.Sc.

High difference in composite reliability between two models

Post by Jules17 » Wed Sep 06, 2017 9:55 am

Hi,

ich cannot think about the reason for my following problem. I hope, you can help me:

I have a data set with 259 data points. The bootstrapping results for one of the constructs in my model regarding composite reliability are:

Original Sample: 0.160
Sample Mean: 0.828
Standard Deviation: 0.923
T Statistics: 0.173
P Values: 0.356

Is this high difference between the original sample and the sample mean possible?

Also, if I put the construct into another/similar model with same data set, composite reliability is much higher (over 0.7)

Can anybody explain the reason for this high differences?

Thanks!

jmbecker
SmartPLS Developer
Posts: 879
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: High difference in composite reliability between two mod

Post by jmbecker » Wed Sep 06, 2017 11:40 am

Possible, yes, but it indicates some more severe model problems. Generally, the original value for your model is quite low. First, I would check the inter-item correlations (and also Cronbach’s Alpha). If they are sufficiently high, the reason could be that the construct cannot build a stable estimate, because it only has near-zero (insignificant) links to other constructs. That would also explain, why the CR is much higher in a different model.
Dr. Jan-Michael Becker, University of Cologne, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Ja ... v=hdr_xprf
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de

Jules17
PLS Junior User
Posts: 4
Joined: Wed Sep 06, 2017 9:43 am
Real name and title: Julia Lanzl, M.Sc.

Re: High difference in composite reliability between two mod

Post by Jules17 » Wed Sep 06, 2017 12:19 pm

Thanks for the answer.
The inter-item correlations are all above 0.6, Cronbach's Alpha is above 0.8.

What else is interesting is that the outer loadings for three of the four items are negative (but I do not think this is due to reverse coded items). Is it also necessary to say that the construct is a moderating variable?

jmbecker
SmartPLS Developer
Posts: 879
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: High difference in composite reliability between two mod

Post by jmbecker » Wed Sep 06, 2017 7:29 pm

It is not a moderating effect (interaction effect) created by the product indicator approach, right? Because it that case you should not worry about the measurement model. Otherwise, it does not matter whether it is a normal predictor or moderator variable.

The negative loadings are certainly a problem. Do your indicators have all positive correlations or are there also negative correlations? Then you have problems with reverse coded or otherwise reverse measured items.
Dr. Jan-Michael Becker, University of Cologne, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Ja ... v=hdr_xprf
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de

Jules17
PLS Junior User
Posts: 4
Joined: Wed Sep 06, 2017 9:43 am
Real name and title: Julia Lanzl, M.Sc.

Re: High difference in composite reliability between two mod

Post by Jules17 » Thu Sep 07, 2017 6:57 am

No, not created by product indicator approach. The indicators' correlations are all positive.

If I calculate the same model with another but similar data set, outer loadings are all positive (about 0.8) and composite reliability is all fine.

jmbecker
SmartPLS Developer
Posts: 879
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: High difference in composite reliability between two mod

Post by jmbecker » Thu Sep 07, 2017 12:24 pm

Sounds to me that this is still a possible explanaition:
jmbecker wrote: If they are sufficiently high, the reason could be that the construct cannot build a stable estimate, because it only has near-zero (insignificant) links to other constructs. That would also explain, why the CR is much higher in a different model.
Dr. Jan-Michael Becker, University of Cologne, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Ja ... v=hdr_xprf
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de

Jules17
PLS Junior User
Posts: 4
Joined: Wed Sep 06, 2017 9:43 am
Real name and title: Julia Lanzl, M.Sc.

Re: High difference in composite reliability between two mod

Post by Jules17 » Thu Sep 07, 2017 12:31 pm

Thanks again.

Then, how do I explain it in the text? Do I have to exclude the construct after measurement model evaluation or can I keep it and explain for the structural model that the moderating effect does not exist?

Post Reply