test on small sample bias

[Fariza]

Dear SmartPLS community,

I tested the theoretical model of my thesis using SmartPLS and I am happy to share with you that this model is now under revision in a high quality journal. Unfortunately, there is one comment from the reviewers that I do not know how to tackle.

The comment says the following: ‘Sample size sounds small for such a large number of variables. A test on the “Small sample bias” can confirm whether the small sample bias is a concern’.

It should be noted that my model has 2 independent latent variables (LV), 1 dependent LV, and two mediator LVs. Except for the final dependent LV, the rest are second order variables, so I adopt a two-stage approach (also known as the latent variable score method).

The rule of thumb says that the sample size requirement would be 10 times either the dependent LV with the largest number of LVs influencing it or the LV with the largest number of formative indicators. Consequently, I look for LV with the bigger number of arrows that arrive to it, and I use this number as "number of predictors", i.e. I have 4 predictors. Consequently, I need at least 40 cases to test the model. My sample size consists of 111 cases, more than the minimum requirement.

Moreover, I have read the posts regarding this topic in the forum for the SmartPLS community and most of the conversations refer to the use of G*power. However, I have two concerns in this respect:

FIRST, does anybody know to what the reviewer is referring with "test on the small sample bias"? I am not sure if the best way to address the question is using G*power.

SECOND, when I perform a test using G*power the results sound estrange to me. I proceed as follows:

- I run F test / Linear Multiple regression: Fixed model, R2 deviation from zero.
- Post hoc
- Determine effect size: R2=0.58; which gives me an effect size of 1.38. IS IT POSSIBLE????
- Alpha = 0,05
- Total sample = 111
- Predictors = max number of arrows arriving in a LV = 4
- Calculate --> Power, which gives me a power of 1. AGAIN, IS IT POSSIBLE????

Please, could anybody give me a clue on how to proceed? I appreciate any help. Thanks in advance and best regards,

Fariza

[cringle]

Thanks. The 10 times rule is way too liberal. Use the 30 or 40 times rule; alternatively, take a look at the sample size table in the PLS-SEM book:
https://www.smartpls.com/documentation/ ... on-pls-sem

But the minimum sample size is a technical issue. In PLS more data is better. I guess that the reviewer comment also has in mind the question, if your sample is representative for the underlying population. That's in most cases the most severe issue that researchers need to justify.

Best regards
Christian

[jmbecker]

When using G*Power you really need to think about which test you want information about the power.
You are requesting information on whether your R² is different from zero. This is fine, but often not the information that you really want to have. With an R² of 0.58 usually nobody would doubt that the R² is different from zero and a power of 1 would not surprise me.

A more interesting question is, whether all significance test for the path coefficients have enough power. The correct G*Power test would be
T-Test: Linear multiple regression: Fixed model, single regression coefficient.
Then you should insert the f²-effect size for the path coefficient in question. Often this effect size is often quite small and you get a much lower Power.