forum.smartpls.com

Dear all,

I have a beginner question concerninr effect size (f2).

Based on the formula f2 is computed as follows : (R2included - R2 excluded)/(1 - R2 included)

Imagine I have a paht model with
A-->C-->D
B-->C-->D

In this model, it is possible to apply the formula for C: A is excluded and the new R2 of C is calculated; then B is excluded and another R2 of C is calculated. As a result we get 3 R2: one with A and B; one with A only; one with B only.

But how is it possible to apply this formula when there is only one exogenous variable (as is the case for the path C-->D). I cannot exclude C otherwise the model is not complete. Is it correct to pretend that R2 for D=0 when C is excluded ? Or is there another solution?

Thank you very much for your help.

David

Hi David,

See
COHEN, Jacob. (1977). Statistical Power Analysis for the Behavioral Sciences. Revised Edition. New York: Academic Press.

In the chapter 9 (F Tests of Variance Proportions in Multiple Regression / Correlation Analysis (p.407-414), we have:

R2 = f2 / (1 + f2)

f2 = R2 / (1 - R2)

where
f2 = 0.02 is small
f2 = 0.15 is medium
f2 = 0.35 is large

Best regards,

Bido

Dear Bido,

Thank you for your answer. But I'm not sure we're talking about the same thing. What I want to estimate is the f2 for each effect in the model in order to gauge the effect of each predictor in the model.
To that purpose, Henseler et al. 2008 suggested to use this formula:
f2 = (R2incl-R2excl)/(1-R2incl).
My problem is the application of this formula when there is only one predictor. Maybe your formula must then be used?

Thanks for answers.

David

Hi David,

I looked for more information, and…

1) f2 = R2 / (1 - R2) --> used in multiple regression considering all independent variables (It is not what you are trying to assess).

2) (R2included - R2 excluded)/(1 - R2 included) --> used in multiple regression considering the partial coefficients (ok for A and B variables, ok as you have done).

3) Eventually, as you have just one relation ( C --> D ) the beta = r, then, the effect size will be the value of this correlation (Cohen, 1977, p.75-80), with:
r = 0.10 = small effect
r = 0.30 = medium effect
r = 0.50 = large effect

I hope this help.

Bido

Hi!

I have a similar model A -> C -> D
B -> C -> D

But what I would like to know is if A is a stronger predictor then B, and if A has a greater contribution to the R2 (variance explained) of the endogenous variables, when compared to B.

I remembered I could test the model with and without A and compare both the R2 for C (and also D), as well as the f2 (effect sizes). Is this possible? Is it correct? How can I test for the R2-change significance? I believe it will be necessary. Do you agree?

I read something in this forum suggesting we could calculate it in spss, but I don't know how... should I compute and use global scores for each of my LV in the model?

I still have one more doubt... I realized that the R2 for D (the late endogenous variable, not directly linked to A or B) did not change with the inclusion or exclusion of A from the model. Why does this happen? Is it related with the partial nature of the analysis performed in PLS?

I appreciate it enormously if professor Bido or anyone could reply to this post and clarify my doubts.

Thanks in advance!

Eliana Carraça

Hey Eliana,

Here comes my opinion about the use of effect size in your model. Maybe it will help you:

a) I would use an excel sheet to calculate the effect size. As you described above you calculate one time your model with and the other time without A (or B). Then you put the different R2 in the formula of Cohen (R2 incl.- R2 exc. / 1-R2 incl). Thereafter you can compare the different effect sizes.

b) I would not use a F-test, because “it is more convenient to work directly with f2 rather than f.” (Cohen, 1988, p. 480) Normally you use the F-test if you are examining simultaneously the impact of “sets of predictors” for a dependent construct.

c) The R2 for D does not change, because the independent variable A has no impact on it.

Best regards

Christian

Hi Elina,

christian.nitzl wrote: c) The R2 for D does not change, because the independent variable A has no impact on it.

or to be a bit more precise, it doesn't change because PLS uses a blockwise estimation procedure. Thus, if the variables are not connected, no changes in R² occur if you omit the independent variable.

Best, Stephan

Thanks to both for your answers!!

Eliana

Hey Stephan,

Thanks for your comment.

I test such a case in my model, like mentioned from Elian (e.g. A-B-C with no direct connection from A to C).

However R2 has been changed for my model, but in a very small way (0.01).

One reason, why this is so, I can see in the iteration process of pls. The LV scores aren't fixed in the first iteration. At the beginning you calculate your LV scores than the connections between the LV variables [for ease]. This has again an impact on the estimated LV before and so on.

But something else came in my mind: perhaps the "total effect" can be giving Eliana some more information about the impact of variable A and the test of significance of it.

Best regards

Christian

Hi Christian,

I think the explanation for this phenomenon is that the PLS-algorithm switches between the estimate for the measurement model and the structural model while iterating. If I understood the algorithm correctly, it converges to a solution when the differences between the inner and the outer approximation fall below a certain (very small) threshold. Thus, unlike covariance-based techniques, the estimation of the measurement and the structural model is NOT independent from each other.

The consequence of this fact is that as soon as you omit an IV (like you did in your model), slight changes in the measurement model of the mediating variable can occur which finally lead to an (likewise small) alteration of the path coefficient to the DV. So while it is still correct that the structural model by itself is estimated blockwise, it is possible that minor changes occur due to an alteration of the measurement model of the mediating variable.

At least this would be my explanation for this phenomenon. Any comments are appreciated.

Best, Stephan

davidal wrote:Dear Bido,

Thank you for your answer. But I'm not sure we're talking about the same thing. What I want to estimate is the f2 for each effect in the model in order to gauge the effect of each predictor in the model.
To that purpose, Henseler et al. 2008 suggested to use this formula:
f2 = (R2incl-R2excl)/(1-R2incl).
My problem is the application of this formula when there is only one predictor. Maybe your formula must then be used?

Thanks for answers.

David

Is there a way of testing the significance of the difference between two paths leading to the same DV? That is, A -> C and B -> C
How can I say that one effect is significantly stronger than the other? Is there a way of testing this with PLS?

Can calculating the effect size help me...? f2 = (R2incl-R2excl)/(1-R2incl).
And if so, how could I do it? By applying the formula for C excluding A and obtaining a new R2 of C; then excluding B and obtaining another R2 of C. And finally, including A and B and obtaining a final R2, and then calculating both f2 and comparing them?

Thanks!

Eliana

Hello Everyone,

Thank you for the detailed discussion! I want to make sure I am applying the information correctly.

I need to calculate the Overall Effect Size of my model. My model is in the following form:

A-->B1-->C
A-->B2-->C
A-->B3-->C

I have three questions:

1. Does using the formula given below for f2 produce what is referred to as the "Overall Effect Size"?

f2 = R2 / (1 - R2)

2. I assume that I would use the R2 of C for this calculation. Is this correct?

3. My R2 is 0.457 and thus my f2 is 0.842 . According to the guidelines below, my f2 is huge! Is this possible?

Thank you very much!
Reem


Professor Bido wrote:

See
COHEN, Jacob. (1977). Statistical Power Analysis for the Behavioral Sciences. Revised Edition. New York: Academic Press.

In the chapter 9 (F Tests of Variance Proportions in Multiple Regression / Correlation Analysis (p.407-414), we have:

R2 = f2 / (1 + f2)

f2 = R2 / (1 - R2)

where
f2 = 0.02 is small
f2 = 0.15 is medium
f2 = 0.35 is large

Hey Reem,

For me it makes no sense to use the cited formula f2=R2/(1-R2) if you use PLS. This formula would be useful only, as Prof. Diogenes wrote, for multiple regression (or the special case of David at the beginning of this discussion). That is the case if you have only direct effects on ‘one’ variable. In this case you should use a multiple regression. As far as I understand it, this is not the case in your model. Therefore you should use the formula f2 = (R2incl-R2excl)/(1-R2incl). Then you can calculate R2 with and without e.g. B1 and put this values in the formula. As Stephan and I discussed above, it would be no difference (or only small difference) if you use A for your calculation, because A has no direct connection to C.

Best regards,

Christian

Hi Christian,

Thank you for your reply!

The reason I think the formula f2 = (R2incl-R2excl)/(1-R2incl) does not apply to my case is that I don't want to know the effect of including one of the variables--or not including it; I simply want to know the overall effect size of the whole model. I Should add that my model is where A is an independent variable, B1, B2, and B3 are mediating variables and C is the dependent variable. All are included in the model simultaneously.

1. In my case, would the formula proposed by Dr. Bido at the beginning--f2=R2/(1-R2)-- apply? From what you say it seems that it would not apply for a PLS model.

2. So, which formula/method does one have to use to calculate the overall effect size for a PLS model (with all variables included)?

Looking forward to hearing what you think about this!
Reem

Hey Reem,

ad 1) In my opinion R2 by itself is enough. Because R2 is the percentage of the explained variance of your focal variable C through your model. In other words your whole “effect” . Why do you need the effect size for your model if you won’t test the separate effects of the mediating variables?

ad 2) Perhaps the Stone-Geisser-Test is what are you searching for. It is testing how useful your model is for prediction. That is an other way for measure the “impact” of your model.

I hope this will help you!

Best Regards,

Christian