Page **1** of **1**

### Using the OTG approach to compare between 3 alternatives?

Posted: **Thu Jul 05, 2012 9:35 am**

by **ge375**

Hello smartPLS community,

my question refers to the following paper:

Marko Sarstedt, Jörg Henseler, Christian M. Ringle (2011), Multigroup Analysis in Partial Least Squares (PLS) Path Modeling: Alternative Methods and Empirical Results, in Marko Sarstedt, Manfred Schwaiger, Charles R. Taylor (ed.) Measurement and Research Methods in International Marketing (Advances in International Marketing, Volume 22), Emerald Group Publishing Limited, pp.195-218

In this paper an **omnibus test of group differences (OTG)** is presented to test for differences between multiple groups in a PLS modeling framework.

I have conducted an online survey, where people were asked to evaluate 3 different alternatives (technologies). For each alternative the same items were used. Answers were given on a 5-point Likert-Scale. For example: Usefulness - Technology 1; Usefulness - Technology 2; ...

At the moment I have calculated 3 individual models by using SmartPLS. But I can't tell if there are significant differences between the path coefficients of the three single models.

Is the OTG approach the right instrument to compare for differences between the three alternatives or is there any other possibility to include 3 different groups in PLS?

Best regards

Gerhard

Posted: **Sun Jul 08, 2012 1:44 am**

by **Hengkov**

Hello,

OTG for multigroup analysis with three group in PLS.

You must run randommization for it and compute variance ratio and error probability p.

Regards,

Hengky

Posted: **Tue Jul 10, 2012 2:08 pm**

by **ge375**

Hello Henky,

thank you for your answer.

How do I compute variance ratio and error probability with smartPLS?

I've run the bootstrapping procedure for all three groups. But now I am not sure, where to find the "bootstrap estimates". Are these the bootstrapping "path coefficients"?

Posted: **Wed Jul 11, 2012 3:57 am**

by **Hengkov**

Hi, Gerhad

SmartPLS cannot run OTG because not have feature randommization, OK.

For compute variance ratio used formula in paper (Sarstedt, Henseler, and Ringle, p. 207).

Best Regards,

Hengky

Posted: **Thu Jul 12, 2012 7:44 am**

by **ge375**

Hello Henky,

thank you again.

Now I have calculated the FR-Value with an self-developed Excel-Sheet.

The first obtained FR-Value for the difference between my three alternatives is 51,863.11. This seems to be a quite big value.

Is this calculated FR-Value enough to say that the differences between the alternative-specific path coefficients are statistically significant?

In the OTG-Approach paper there are two more steps suggested:

1. A permutation step

2. A calculation of the error probability (p)

Are these steps necessary? I don't understand why they should be computed and what information they obtain.

Best regards

Gerhard

Posted: **Thu Jul 12, 2012 8:01 pm**

by **Hengkov**

Hi Gerhard,

No. After calculate FR, you must run permutation or randommization test. SmartPLS not have this feature. So, OTG cannot implemented using SmartPLS.

Regards,

Hengky

### Re: Using the OTG approach to compare between 3 alternatives?

Posted: **Fri Aug 10, 2018 5:55 am**

by **SHU**

Hi Gerhard,

I struggle with the same topics and came up with the same questions.

Now I have calculated the FR-Value with an self-developed Excel-Sheet.

Do you mind sharing this self-developed an

**Excel-Sheet for the FR-values** with the community here?

How did you (or anyoneelse who is advanced regarding the OTG approach) master the

**permutation step** and the calculation of the

**error probability** (p) ?

Thanks,

Susanne

### Re: Using the OTG approach to compare between 3 alternatives?

Posted: **Sat Aug 11, 2018 1:08 pm**

by **SHU**

Hi guys,

I tried to proceed with the OTG test R code obtained here:

https://www.pls-sem.net/downloads/advan ... pls-sem-1/ but further questions arose. I used the R code and the previously generated bootstrap estimates from SmartPLS (5,000 bootstraps, equal across all 3 groups) to compute the variance ratio F

R for each structural model relation.

First, I wonder what

**number of permutations (MC runs)** is best?

I tried different numbers of permutation (from 100 to 5,000), but the obtained F

0 was always quite high (>5K) and the F

R values very small (<5) constantly leading to p = 0.

Second, do you only

**report the highest** F

R value per structural model relation (or a range)?

Third, in case the OTG supports that there is a difference between the groups, which

**post-hoc approach** should be used to detect the origin(s) of the difference:

- the permutation-based approach (Chin, 2003b; Chin & Dibbern, 2010)?

- the non-parametric PLS-MGA (Sarstedt, Henseler, & Ringle, 2011)?

Many thanks in advance!

Bests, Susanne

### Re: Using the OTG approach to compare between 3 alternatives?

Posted: **Sun Aug 12, 2018 2:37 pm**

by **jmbecker**

1)

Usually, the more the better if you are using bootstrapping or permutation based approaches, because the estimates become more stable.

2)

There is not much literature about the OTG approach or reporting guidelines. Usually, it is the best to look at some recent applications or the paper that proposes the approach.

3)

That depends on you data and groups. Generally, I would say that the permutation approach is the statistically most appropriate approach. However, it has weaknesses when the group sizes are very unequal. Usually, they only differ when your effects are at the edge of being significant. Otherwise they usually all agree in showing significance or non-significance. If they agree it would be good to report that in a footnote as a robustness check.

### Re: Using the OTG approach to compare between 3 alternatives?

Posted: **Mon Aug 13, 2018 12:47 pm**

by **SHU**

Thank you for your reply! I'll try to work on it as suggested.

Bests, Susanne