Significance of indirect effect in multiple mediator models

Questions about the implementation and application of the PLS-SEM method, that are not related to the usage of the SmartPLS software.
Post Reply
Sam
PLS User
Posts: 13
Joined: Thu Dec 11, 2014 1:53 pm
Real name and title: Daniel Fischer

Significance of indirect effect in multiple mediator models

Post by Sam »

Hi there!

Is there a way to determine significance of a single indirect path in a multiple mediator model using the bootstrapping approach? In smartpls, I can only see the significance of all indirect effects taken together. For mediation analysis, I should know the significance of every single indirect effect... And I do not want to use sobel-test ;)

Thanks for any hints!
Sam
jmbecker
SmartPLS Developer
Posts: 1282
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Significance of indirect effect in multiple mediator mod

Post by jmbecker »

Hi Sam,
At the moment you can only calculate these specific indirect effects by hand. Therefore, you have to use the data that is provided under "Path Coefficients > Samples". There you find for each bootstrap sample the path coefficients of your model. You can then calculate the indirect effects that you want by multiplying the respective direct effect estimates for each sample (e.g., in excel or any other tool). You can then calculate the standard deviation over all these newly calculated individual indirect effects. The original sample estimate (which you also have to calculate) divided by this stdev is the t-value that you are looking for.
I will also consider if makes sense to implement these individual indirect effects in SmartPLS. Thank you for your suggestion.
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
Sam
PLS User
Posts: 13
Joined: Thu Dec 11, 2014 1:53 pm
Real name and title: Daniel Fischer

Re: Significance of indirect effect in multiple mediator mod

Post by Sam »

Hi Dr. Becker,

thank you very much for your hint, that helps me a lot. Is there a chance to calculate bias corrected bootstrapping intervals for these single indirect effects by hand?

Tanks in advance!
Sam
Sam
PLS User
Posts: 13
Joined: Thu Dec 11, 2014 1:53 pm
Real name and title: Daniel Fischer

Re: Significance of indirect effect in multiple mediator mod

Post by Sam »

Hi there,

sry, but I have to ask again. I calculated a simple mediation model in pls and tried to calculate the corresponding t-value of the indirect effect to check if I'm right. While calculating, I saw that indirect effects calculated by hand are sometimes opposite in sign compared to indirect effects given by smartpls. The values are in the same height, except for rounding. The result is a slightly differing stdev so that i have no chance to get the right t-values.
To make clear what I mean, I attached a pdf-file that show my excel file; X is the independent, M the mediator, and Y the dependent variable. You can get it here: indirect-Effect_Smartpls3.pdf
Edit: pls click on the blue "Download" button...

Do you have any idea why this happens?

Tanks a lot
Sam

Edit2: I'm using the actual version of smartpls 3.1.9
jmbecker
SmartPLS Developer
Posts: 1282
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Significance of indirect effect in multiple mediator mod

Post by jmbecker »

The problem is that you are using the individual sign-change option, right? Hence, your direct effect estimates are all set to the sign of the original model even if they are in opposite direction. Unfortunately, this is not done for the indirect effects. I will look into this issue.
However, given your very small estimates (close to zero) and generally, I would not advice the use of the individual sign-change option. You are probably pushing your estimates towards significance although there is none.
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
jmbecker
SmartPLS Developer
Posts: 1282
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Significance of indirect effect in multiple mediator mod

Post by jmbecker »

Actually I have quickly illustrated what happens if you use the individual sign change option in a case, where the true population coefficient is zero (so that there is actually no relation between the two variables).
The original model coefficient (in this case -0.028) is only a result of sampling error. Fixing the sample estimates to negative values might not be a good decision. The resulting distribution is very skewed and hence t-values etc. are not really applicable.

Individual-Sign-Change:
Image

No Sign Changes:
Image


BTW: The bootstrapping histograms as shown will be a feature of the next version.
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
Sam
PLS User
Posts: 13
Joined: Thu Dec 11, 2014 1:53 pm
Real name and title: Daniel Fischer

Re: Significance of indirect effect in multiple mediator mod

Post by Sam »

Good morning Dr. Becker,

thank you very much for your detailed answer, those histograms look great! And you're right, I'm using individual sign changes. I see, this problem is also discussed in Hair et al (2014) A Primer on PLS SEM, p. 136ff. (for anyone who wants to have a citable source ;)).

Very thanks Dr. Becker!

Sam
jmbecker
SmartPLS Developer
Posts: 1282
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Significance of indirect effect in multiple mediator mod

Post by jmbecker »

These nice Histograms are now part of SmartPLS Version 3.2.0. You should use at least use 1,000 bootstapps to get a good resolution. (Examples here use 10,000 resamples).
In addition, we also fixed the problem with sign change options and indirect and total effects.
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
saddas
PLS User
Posts: 19
Joined: Wed Jan 30, 2008 12:54 pm
Real name and title:

Re: Significance of indirect effect in multiple mediator mod

Post by saddas »

Hi Dr. Becker,

Thanks for your explanation about the sign corrections. When you say you "fixed" that issue, what does this mean exactly? That the software now changes the signs also for the indirect effect? Given that the sign corrections generally truncates the bootstrap distribution at zero and thus biases the confidence intervals (Rönkkö et al. 2015), do you advise we stay away from using that setting and always use the "no sign changes" option?

Thanks,
Shamel



Rönkkö, M., McIntosh, C. N., & Antonakis, J. (2015). On the adoption of partial least squares in psychological research: Caveat emptor. Personality and Individual Differences, (87), 76–84. http://doi.org/10.1016/j.paid.2015.07.019
jmbecker
SmartPLS Developer
Posts: 1282
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Significance of indirect effect in multiple mediator mod

Post by jmbecker »

Yes, I would also advice not to use the sign change option. It is an option from the past that we still offer to give users the possibility to reproduce old results, but it produces misleading statistics and incorrect results.
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
saddas
PLS User
Posts: 19
Joined: Wed Jan 30, 2008 12:54 pm
Real name and title:

Re: Significance of indirect effect in multiple mediator mod

Post by saddas »

Ok, thanks for clarifying.
Post Reply