Comparability of PLS resuts

Questions about the implementation and application of the PLS-SEM method, that are not related to the usage of the SmartPLS software.
Post Reply
T_Hansen
PLS Junior User
Posts: 5
Joined: Wed Nov 09, 2005 4:20 pm
Real name and title:

Comparability of PLS resuts

Post by T_Hansen »

Hello everyone,

I have a had a recent discussion with a colleague about the comparability of PLS results and would be interested in further opinions.

Assumed, I apply the exact same PLS model structure in two application scenarios:

Scenario 1: 500 datasets are collected for input.
Scenario 2: 300 datasets are collected for input.

The datasets are different in each scenario, for example once the model is used with data from the US, once with data from Europe, each time the purpose of the analysis is identical, just the origin and the amount of data varies.

Assuming we are looking at one specific relationship in the model between two latent variables. The path coefficient is assumed to be 0.1 in scenario 1 and 0.3 in scenario 2.

Is it valid to assume, that in scenario 2 despite, necessarily varying quality criteria, the relationship between two variables is stronger? Is it valid to compare PLS results that originate from the same model structure with each other, despite different input data?

Best wishes
stefanbehrens
PLS Expert User
Posts: 54
Joined: Wed Oct 19, 2005 5:53 pm
Real name and title:

Post by stefanbehrens »

Torben,

with your post we are entering the interesting world of multi-group comparisons in PLS.

As you might already have suspected, simply comparing the two path coefficients doesn't tell you whether the observed differences are really significant (or just mere coincidence).

You have several options to resolve this question:

A) Perform a permutation-based procedure to non-parametrically test the significance of the observed group difference (Chin 2003; Dibbern & Chin 2005).

B) Use the formula posted on Chin's website (http://disc-nt.cba.uh.edu/chin/plsfaq/multigroup.htm) to approximate the significance level of the observed difference assuming that path coefficient variances in both groups are approximately equal

C) Run a moderated regression of the entire dataset (800 cases) where group membership is coded as a dummy variable and an interaction term is created in the form dummy*indepvar of interest. If the interaction term is significant, the observed difference between the two models is as well.

So back to your question: Comparing obviously is always a valid approach. You just have to be careful with your conclusions ;-) If you intend to use C) I suggest you also read (Carte & Russel 2003, in particular error no. 9).

Good luck,
Stefan



[Chin 2003] Chin, W.W. "A Permutation Procedure for Multi-Group Comparison of PLS Models," Paper presented at the 3rd International Symposium on PLS and Related Methods, Lisbon, Portugal, 2003, pp. 33-43.

[Dibbern & Chin 2005] Dibbern, J., and Chin, W.W. "Multi-Group Comparison: Testing a PLS Model on the Sourcing of Application Software Services across Germany and the U.S.A. Using a Permutation Based Algorithm," in: Handbuch PLS-Pfadmodellierung: Methode, Anwendung, Praxisbeispiele, Bliemel, F., Eggert, A., Fassott, G. and Henseler, J. (eds.), Schäffer-Poeschel Verlag, Stuttgart, 2005, pp. 135-159.

[Carte & Russell 2003] Carte, T.A., and Russell, C.J. "In Pursuit of Moderation: Nine Common Errors and Their Solutions," MIS Quarterly (27:3), September 2003, pp. 479-501.
T_Hansen
PLS Junior User
Posts: 5
Joined: Wed Nov 09, 2005 4:20 pm
Real name and title:

Post by T_Hansen »

Thank you, Stefan, this is excellent advice. I wasn't aware of the term "multi group", thus the existing threads on this topic haven't been of interest for me so far.
viswadatta
PLS Expert User
Posts: 139
Joined: Wed Jul 05, 2006 1:43 pm
Real name and title:
Location: Coimbatore, India

path coefficient

Post by viswadatta »

In regression and path modelling we never use raw path coefficients to determine the relation's strength. We use only loadings and the t-coefficient to determine the strength of relationship between the variables.(Similar to the beta coefficients in ordinary regression)

This should answer your doubt.
Post Reply