Page 1 of 1

Using the OTG approach to compare between 3 alternatives?

Posted: Thu Jul 05, 2012 9:35 am
by ge375
Hello smartPLS community,
my question refers to the following paper:

Marko Sarstedt, Jörg Henseler, Christian M. Ringle (2011), Multigroup Analysis in Partial Least Squares (PLS) Path Modeling: Alternative Methods and Empirical Results, in Marko Sarstedt, Manfred Schwaiger, Charles R. Taylor (ed.) Measurement and Research Methods in International Marketing (Advances in International Marketing, Volume 22), Emerald Group Publishing Limited, pp.195-218

In this paper an omnibus test of group differences (OTG) is presented to test for differences between multiple groups in a PLS modeling framework.

I have conducted an online survey, where people were asked to evaluate 3 different alternatives (technologies). For each alternative the same items were used. Answers were given on a 5-point Likert-Scale. For example: Usefulness - Technology 1; Usefulness - Technology 2; ...

At the moment I have calculated 3 individual models by using SmartPLS. But I can't tell if there are significant differences between the path coefficients of the three single models.

Is the OTG approach the right instrument to compare for differences between the three alternatives or is there any other possibility to include 3 different groups in PLS?

Best regards

Gerhard

Posted: Sun Jul 08, 2012 1:44 am
by Hengkov
Hello,
OTG for multigroup analysis with three group in PLS.
You must run randommization for it and compute variance ratio and error probability p.
Regards,
Hengky

Posted: Tue Jul 10, 2012 2:08 pm
by ge375
Hello Henky,
thank you for your answer.
How do I compute variance ratio and error probability with smartPLS?

I've run the bootstrapping procedure for all three groups. But now I am not sure, where to find the "bootstrap estimates". Are these the bootstrapping "path coefficients"?

Posted: Wed Jul 11, 2012 3:57 am
by Hengkov
Hi, Gerhad
SmartPLS cannot run OTG because not have feature randommization, OK.
For compute variance ratio used formula in paper (Sarstedt, Henseler, and Ringle, p. 207).
Best Regards,
Hengky

Posted: Thu Jul 12, 2012 7:44 am
by ge375
Hello Henky,
thank you again.

Now I have calculated the FR-Value with an self-developed Excel-Sheet.
The first obtained FR-Value for the difference between my three alternatives is 51,863.11. This seems to be a quite big value.

Is this calculated FR-Value enough to say that the differences between the alternative-specific path coefficients are statistically significant?

In the OTG-Approach paper there are two more steps suggested:
1. A permutation step
2. A calculation of the error probability (p)

Are these steps necessary? I don't understand why they should be computed and what information they obtain.

Best regards

Gerhard

Posted: Thu Jul 12, 2012 8:01 pm
by Hengkov
Hi Gerhard,
No. After calculate FR, you must run permutation or randommization test. SmartPLS not have this feature. So, OTG cannot implemented using SmartPLS.
Regards,
Hengky

Re: Using the OTG approach to compare between 3 alternatives?

Posted: Fri Aug 10, 2018 5:55 am
by SHU
Hi Gerhard,

I struggle with the same topics and came up with the same questions.
Now I have calculated the FR-Value with an self-developed Excel-Sheet.
Do you mind sharing this self-developed an Excel-Sheet for the FR-values with the community here?

How did you (or anyoneelse who is advanced regarding the OTG approach) master the permutation step and the calculation of the error probability (p) ?

Thanks,
Susanne

Re: Using the OTG approach to compare between 3 alternatives?

Posted: Sat Aug 11, 2018 1:08 pm
by SHU
Hi guys,

I tried to proceed with the OTG test R code obtained here: https://www.pls-sem.net/downloads/advan ... pls-sem-1/ but further questions arose. I used the R code and the previously generated bootstrap estimates from SmartPLS (5,000 bootstraps, equal across all 3 groups) to compute the variance ratio FR for each structural model relation.

First, I wonder what number of permutations (MC runs) is best?
I tried different numbers of permutation (from 100 to 5,000), but the obtained F0 was always quite high (>5K) and the FR values very small (<5) constantly leading to p = 0.

Second, do you only report the highest FR value per structural model relation (or a range)?

Third, in case the OTG supports that there is a difference between the groups, which post-hoc approach should be used to detect the origin(s) of the difference:
- the permutation-based approach (Chin, 2003b; Chin & Dibbern, 2010)?
- the non-parametric PLS-MGA (Sarstedt, Henseler, & Ringle, 2011)?

Many thanks in advance!

Bests, Susanne

Re: Using the OTG approach to compare between 3 alternatives?

Posted: Sun Aug 12, 2018 2:37 pm
by jmbecker
1)
Usually, the more the better if you are using bootstrapping or permutation based approaches, because the estimates become more stable.

2)
There is not much literature about the OTG approach or reporting guidelines. Usually, it is the best to look at some recent applications or the paper that proposes the approach.

3)
That depends on you data and groups. Generally, I would say that the permutation approach is the statistically most appropriate approach. However, it has weaknesses when the group sizes are very unequal. Usually, they only differ when your effects are at the edge of being significant. Otherwise they usually all agree in showing significance or non-significance. If they agree it would be good to report that in a footnote as a robustness check.

Re: Using the OTG approach to compare between 3 alternatives?

Posted: Mon Aug 13, 2018 12:47 pm
by SHU
Thank you for your reply! I'll try to work on it as suggested.

Bests, Susanne