Using the OTG approach to compare between 3 alternatives?

Questions about the implementation and application of the PLS-SEM method, that are not related to the usage of the SmartPLS software.
Post Reply
ge375
PLS Junior User
Posts: 3
Joined: Wed Jun 30, 2010 7:55 am
Real name and title:

Using the OTG approach to compare between 3 alternatives?

Post by ge375 »

Hello smartPLS community,
my question refers to the following paper:

Marko Sarstedt, Jörg Henseler, Christian M. Ringle (2011), Multigroup Analysis in Partial Least Squares (PLS) Path Modeling: Alternative Methods and Empirical Results, in Marko Sarstedt, Manfred Schwaiger, Charles R. Taylor (ed.) Measurement and Research Methods in International Marketing (Advances in International Marketing, Volume 22), Emerald Group Publishing Limited, pp.195-218

In this paper an omnibus test of group differences (OTG) is presented to test for differences between multiple groups in a PLS modeling framework.

I have conducted an online survey, where people were asked to evaluate 3 different alternatives (technologies). For each alternative the same items were used. Answers were given on a 5-point Likert-Scale. For example: Usefulness - Technology 1; Usefulness - Technology 2; ...

At the moment I have calculated 3 individual models by using SmartPLS. But I can't tell if there are significant differences between the path coefficients of the three single models.

Is the OTG approach the right instrument to compare for differences between the three alternatives or is there any other possibility to include 3 different groups in PLS?

Best regards

Gerhard
User avatar
Hengkov
PLS Super-Expert
Posts: 1599
Joined: Sun Apr 24, 2011 10:13 am
Real name and title: Hengky Latan
Location: AMQ, Indonesia
Contact:

Post by Hengkov »

Hello,
OTG for multigroup analysis with three group in PLS.
You must run randommization for it and compute variance ratio and error probability p.
Regards,
Hengky
ge375
PLS Junior User
Posts: 3
Joined: Wed Jun 30, 2010 7:55 am
Real name and title:

Post by ge375 »

Hello Henky,
thank you for your answer.
How do I compute variance ratio and error probability with smartPLS?

I've run the bootstrapping procedure for all three groups. But now I am not sure, where to find the "bootstrap estimates". Are these the bootstrapping "path coefficients"?
User avatar
Hengkov
PLS Super-Expert
Posts: 1599
Joined: Sun Apr 24, 2011 10:13 am
Real name and title: Hengky Latan
Location: AMQ, Indonesia
Contact:

Post by Hengkov »

Hi, Gerhad
SmartPLS cannot run OTG because not have feature randommization, OK.
For compute variance ratio used formula in paper (Sarstedt, Henseler, and Ringle, p. 207).
Best Regards,
Hengky
ge375
PLS Junior User
Posts: 3
Joined: Wed Jun 30, 2010 7:55 am
Real name and title:

Post by ge375 »

Hello Henky,
thank you again.

Now I have calculated the FR-Value with an self-developed Excel-Sheet.
The first obtained FR-Value for the difference between my three alternatives is 51,863.11. This seems to be a quite big value.

Is this calculated FR-Value enough to say that the differences between the alternative-specific path coefficients are statistically significant?

In the OTG-Approach paper there are two more steps suggested:
1. A permutation step
2. A calculation of the error probability (p)

Are these steps necessary? I don't understand why they should be computed and what information they obtain.

Best regards

Gerhard
User avatar
Hengkov
PLS Super-Expert
Posts: 1599
Joined: Sun Apr 24, 2011 10:13 am
Real name and title: Hengky Latan
Location: AMQ, Indonesia
Contact:

Post by Hengkov »

Hi Gerhard,
No. After calculate FR, you must run permutation or randommization test. SmartPLS not have this feature. So, OTG cannot implemented using SmartPLS.
Regards,
Hengky
SHU
PLS Junior User
Posts: 10
Joined: Tue Jan 03, 2017 5:58 pm
Real name and title: Susanne Hügel
Location: Wiesbaden (Germany)

Re: Using the OTG approach to compare between 3 alternatives?

Post by SHU »

Hi Gerhard,

I struggle with the same topics and came up with the same questions.
Now I have calculated the FR-Value with an self-developed Excel-Sheet.
Do you mind sharing this self-developed an Excel-Sheet for the FR-values with the community here?

How did you (or anyoneelse who is advanced regarding the OTG approach) master the permutation step and the calculation of the error probability (p) ?

Thanks,
Susanne
SHU
PLS Junior User
Posts: 10
Joined: Tue Jan 03, 2017 5:58 pm
Real name and title: Susanne Hügel
Location: Wiesbaden (Germany)

Re: Using the OTG approach to compare between 3 alternatives?

Post by SHU »

Hi guys,

I tried to proceed with the OTG test R code obtained here: https://www.pls-sem.net/downloads/advan ... pls-sem-1/ but further questions arose. I used the R code and the previously generated bootstrap estimates from SmartPLS (5,000 bootstraps, equal across all 3 groups) to compute the variance ratio FR for each structural model relation.

First, I wonder what number of permutations (MC runs) is best?
I tried different numbers of permutation (from 100 to 5,000), but the obtained F0 was always quite high (>5K) and the FR values very small (<5) constantly leading to p = 0.

Second, do you only report the highest FR value per structural model relation (or a range)?

Third, in case the OTG supports that there is a difference between the groups, which post-hoc approach should be used to detect the origin(s) of the difference:
- the permutation-based approach (Chin, 2003b; Chin & Dibbern, 2010)?
- the non-parametric PLS-MGA (Sarstedt, Henseler, & Ringle, 2011)?

Many thanks in advance!

Bests, Susanne
jmbecker
SmartPLS Developer
Posts: 1282
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Using the OTG approach to compare between 3 alternatives?

Post by jmbecker »

1)
Usually, the more the better if you are using bootstrapping or permutation based approaches, because the estimates become more stable.

2)
There is not much literature about the OTG approach or reporting guidelines. Usually, it is the best to look at some recent applications or the paper that proposes the approach.

3)
That depends on you data and groups. Generally, I would say that the permutation approach is the statistically most appropriate approach. However, it has weaknesses when the group sizes are very unequal. Usually, they only differ when your effects are at the edge of being significant. Otherwise they usually all agree in showing significance or non-significance. If they agree it would be good to report that in a footnote as a robustness check.
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
SHU
PLS Junior User
Posts: 10
Joined: Tue Jan 03, 2017 5:58 pm
Real name and title: Susanne Hügel
Location: Wiesbaden (Germany)

Re: Using the OTG approach to compare between 3 alternatives?

Post by SHU »

Thank you for your reply! I'll try to work on it as suggested.

Bests, Susanne
Post Reply