different reliability results in SPSS, PLS, PLSc

Small talk on SmartPLS that does not correspond to the other forums!
Post Reply
Stef
PLS Junior User
Posts: 5
Joined: Sat Aug 27, 2016 1:54 pm
Real name and title: Stefanie, doctoral student

different reliability results in SPSS, PLS, PLSc

Post by Stef » Sat Aug 27, 2016 2:26 pm

Hello all,

I have a problem which I cannot resolve on my own, after already trying some things. I am working with the prof. license and I have a reflective construct with four indicators. In SPSS and with the PLSc I get wonderful reliability values for that scale/ indicators (value of 0,856 without deleating items). But for my research model I am using the normal PLS algorithm (fits the research question better than does the PLSc) and there I get totally strange values (one is negative and two are under 0,3) - I just tried the PLSc because of these strange results for the construct and was very surprised about the results: with PLSc the reliability checks were all well. I do not understand what could be the reason for that and appreciate very much help on that!

Thanks in advance for any ideas and BR
Stefanie

jmbecker
SmartPLS Developer
Posts: 1099
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: different reliability results in SPSS, PLS, PLSc

Post by jmbecker » Mon Aug 29, 2016 1:44 pm

At what reliability indices are your looking?
There are indices that are model dependent and those that are not model dependent (e.g., Cronbachs alpha). Those that are not model dependent, but are only based on the correlation/covariance between the indicators should show the same results.
All the other could differ as you are implying a different model with different assumptions (i.e., factor model / composite model, etc.). If you have a reflective / factor type measurement models and the reliability for factor-based methods (e.g., PLSc) are better than those for composites-based methods (normal PLS) than you should probably better use those methods.
BTW: I have never seen any negative reliability measures, except for the fact that you have inappropriate measurement models. Do you have negative weights & loadings? Which indices are negative?
Dr. Jan-Michael Becker, University of Cologne, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de

Stef
PLS Junior User
Posts: 5
Joined: Sat Aug 27, 2016 1:54 pm
Real name and title: Stefanie, doctoral student

Re: different reliability results in SPSS, PLS, PLSc

Post by Stef » Mon Aug 29, 2016 2:26 pm

Hi and thanks already for your response and sorry for my inconcrete descriptions in the first message!
It is a reflective factor type measurement model with for indicators for the construct. So I wanted to check for the reliability and had a look on the composite reliability, which was on 0,4 (far away from above 0,7). So I had a look at the loadings to see if there are any indicators that I could delete for increasing the reliability of the scale but the outer loadings are: 0,341; 0,558; 0,256; -0,225; I was surprised because I used a common, often used and validated scale and therefore checked it in SPSS and surprisingly there the cronbachs alpha for the four items was on 0,856 (without deleating items). The cronbachs alpha in smartPLS is 0,446 running the (normal) PLS Algorithm. I uploaded the data file several times again and also deleated the indicator with the negative outer loading; I recoded the items in SPSS just to have a look if that changes anything. And than (without theoretical arguments, just trying) I run the PLSc and was very surprised because there the composite reliability of the construct is totally fine (0,850) and the cronbachs alpha is 0,857 - very close to the result of SPSS (how it should be). Any idea what could be wrong?
Thanks in advance and BR

jmbecker
SmartPLS Developer
Posts: 1099
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: different reliability results in SPSS, PLS, PLSc

Post by jmbecker » Tue Aug 30, 2016 11:23 am

Could you open a support request and send us the data?
http://support.smartpls.com/open.php
Dr. Jan-Michael Becker, University of Cologne, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de

Stef
PLS Junior User
Posts: 5
Joined: Sat Aug 27, 2016 1:54 pm
Real name and title: Stefanie, doctoral student

Re: different reliability results in SPSS, PLS, PLSc

Post by Stef » Tue Aug 30, 2016 11:43 am

Of course, thank you for that offer! Format csv? And I would reduce non-relevant content/data or is it better to leave everything as it is?

jmbecker
SmartPLS Developer
Posts: 1099
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: different reliability results in SPSS, PLS, PLSc

Post by jmbecker » Wed Aug 31, 2016 9:00 am

The best would be a model export that demonstrates the "problem" and whatever data is nessesary to run that model.
Dr. Jan-Michael Becker, University of Cologne, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de

Post Reply