Hello smartPLS community,
my question refers to the following paper:
Marko Sarstedt, Jörg Henseler, Christian M. Ringle (2011), Multigroup Analysis in Partial Least Squares (PLS) Path Modeling: Alternative Methods and Empirical Results, in Marko Sarstedt, Manfred Schwaiger, Charles R. Taylor (ed.) Measurement and Research Methods in International Marketing (Advances in International Marketing, Volume 22), Emerald Group Publishing Limited, pp.195-218
In this paper an omnibus test of group differences (OTG) is presented to test for differences between multiple groups in a PLS modeling framework.
I have conducted an online survey, where people were asked to evaluate 3 different alternatives (technologies). For each alternative the same items were used. Answers were given on a 5-point Likert-Scale. For example: Usefulness - Technology 1; Usefulness - Technology 2; ...
At the moment I have calculated 3 individual models by using SmartPLS. But I can't tell if there are significant differences between the path coefficients of the three single models.
Is the OTG approach the right instrument to compare for differences between the three alternatives or is there any other possibility to include 3 different groups in PLS?
Best regards
Gerhard
Using the OTG approach to compare between 3 alternatives?
Hello Henky,
thank you again.
Now I have calculated the FR-Value with an self-developed Excel-Sheet.
The first obtained FR-Value for the difference between my three alternatives is 51,863.11. This seems to be a quite big value.
Is this calculated FR-Value enough to say that the differences between the alternative-specific path coefficients are statistically significant?
In the OTG-Approach paper there are two more steps suggested:
1. A permutation step
2. A calculation of the error probability (p)
Are these steps necessary? I don't understand why they should be computed and what information they obtain.
Best regards
Gerhard
thank you again.
Now I have calculated the FR-Value with an self-developed Excel-Sheet.
The first obtained FR-Value for the difference between my three alternatives is 51,863.11. This seems to be a quite big value.
Is this calculated FR-Value enough to say that the differences between the alternative-specific path coefficients are statistically significant?
In the OTG-Approach paper there are two more steps suggested:
1. A permutation step
2. A calculation of the error probability (p)
Are these steps necessary? I don't understand why they should be computed and what information they obtain.
Best regards
Gerhard
-
- PLS Junior User
- Posts: 10
- Joined: Tue Jan 03, 2017 5:58 pm
- Real name and title: Susanne Hügel
- Location: Wiesbaden (Germany)
Re: Using the OTG approach to compare between 3 alternatives?
Hi Gerhard,
I struggle with the same topics and came up with the same questions.
How did you (or anyoneelse who is advanced regarding the OTG approach) master the permutation step and the calculation of the error probability (p) ?
Thanks,
Susanne
I struggle with the same topics and came up with the same questions.
Do you mind sharing this self-developed an Excel-Sheet for the FR-values with the community here?Now I have calculated the FR-Value with an self-developed Excel-Sheet.
How did you (or anyoneelse who is advanced regarding the OTG approach) master the permutation step and the calculation of the error probability (p) ?
Thanks,
Susanne
-
- PLS Junior User
- Posts: 10
- Joined: Tue Jan 03, 2017 5:58 pm
- Real name and title: Susanne Hügel
- Location: Wiesbaden (Germany)
Re: Using the OTG approach to compare between 3 alternatives?
Hi guys,
I tried to proceed with the OTG test R code obtained here: https://www.pls-sem.net/downloads/advan ... pls-sem-1/ but further questions arose. I used the R code and the previously generated bootstrap estimates from SmartPLS (5,000 bootstraps, equal across all 3 groups) to compute the variance ratio FR for each structural model relation.
First, I wonder what number of permutations (MC runs) is best?
I tried different numbers of permutation (from 100 to 5,000), but the obtained F0 was always quite high (>5K) and the FR values very small (<5) constantly leading to p = 0.
Second, do you only report the highest FR value per structural model relation (or a range)?
Third, in case the OTG supports that there is a difference between the groups, which post-hoc approach should be used to detect the origin(s) of the difference:
- the permutation-based approach (Chin, 2003b; Chin & Dibbern, 2010)?
- the non-parametric PLS-MGA (Sarstedt, Henseler, & Ringle, 2011)?
Many thanks in advance!
Bests, Susanne
I tried to proceed with the OTG test R code obtained here: https://www.pls-sem.net/downloads/advan ... pls-sem-1/ but further questions arose. I used the R code and the previously generated bootstrap estimates from SmartPLS (5,000 bootstraps, equal across all 3 groups) to compute the variance ratio FR for each structural model relation.
First, I wonder what number of permutations (MC runs) is best?
I tried different numbers of permutation (from 100 to 5,000), but the obtained F0 was always quite high (>5K) and the FR values very small (<5) constantly leading to p = 0.
Second, do you only report the highest FR value per structural model relation (or a range)?
Third, in case the OTG supports that there is a difference between the groups, which post-hoc approach should be used to detect the origin(s) of the difference:
- the permutation-based approach (Chin, 2003b; Chin & Dibbern, 2010)?
- the non-parametric PLS-MGA (Sarstedt, Henseler, & Ringle, 2011)?
Many thanks in advance!
Bests, Susanne
-
- SmartPLS Developer
- Posts: 1284
- Joined: Tue Mar 28, 2006 11:09 am
- Real name and title: Dr. Jan-Michael Becker
Re: Using the OTG approach to compare between 3 alternatives?
1)
Usually, the more the better if you are using bootstrapping or permutation based approaches, because the estimates become more stable.
2)
There is not much literature about the OTG approach or reporting guidelines. Usually, it is the best to look at some recent applications or the paper that proposes the approach.
3)
That depends on you data and groups. Generally, I would say that the permutation approach is the statistically most appropriate approach. However, it has weaknesses when the group sizes are very unequal. Usually, they only differ when your effects are at the edge of being significant. Otherwise they usually all agree in showing significance or non-significance. If they agree it would be good to report that in a footnote as a robustness check.
Usually, the more the better if you are using bootstrapping or permutation based approaches, because the estimates become more stable.
2)
There is not much literature about the OTG approach or reporting guidelines. Usually, it is the best to look at some recent applications or the paper that proposes the approach.
3)
That depends on you data and groups. Generally, I would say that the permutation approach is the statistically most appropriate approach. However, it has weaknesses when the group sizes are very unequal. Usually, they only differ when your effects are at the edge of being significant. Otherwise they usually all agree in showing significance or non-significance. If they agree it would be good to report that in a footnote as a robustness check.
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
-
- PLS Junior User
- Posts: 10
- Joined: Tue Jan 03, 2017 5:58 pm
- Real name and title: Susanne Hügel
- Location: Wiesbaden (Germany)
Re: Using the OTG approach to compare between 3 alternatives?
Thank you for your reply! I'll try to work on it as suggested.
Bests, Susanne
Bests, Susanne