Indirect Effects: P-Values and Confidence Intervals

Questions about the implementation and application of the PLS-SEM method, that are not related to the usage of the SmartPLS software.
Post Reply
AdrianWorkingWithPLS
PLS Junior User
Posts: 4
Joined: Tue Aug 08, 2017 3:04 pm
Real name and title: Adrian

Indirect Effects: P-Values and Confidence Intervals

Post by AdrianWorkingWithPLS »

Hello everybody,

I'm currently looking at an indirect effect. For this effect, I obtained a p-value of 0.07 and a bias corrected 95% confidence interval of (0.005; 0.080). Against this backdrop, I wondered how I should interpret these results. From my point of view, looking at the p-value would imply that the effect is not significant at the 5% level, while considering the condifence interval would imply that the effect is significant (as the value of 0 is not included in the CI).

How would you interpret these results?

Thank you!
AmerSaeed
PLS Junior User
Posts: 4
Joined: Sun Sep 24, 2017 11:53 am
Real name and title: Amer Saeed

Re: Indirect Effects: P-Values and Confidence Intervals

Post by AmerSaeed »

According to latest articles bias corrected confidence intervals give the best indication of indirect effects. For your situation perhaps you have run bootstrap analysis with two-tail. If it is then try to run with one-tail and you will get the significant p-value as well.

Hope it will help
Regards
AdrianWorkingWithPLS
PLS Junior User
Posts: 4
Joined: Tue Aug 08, 2017 3:04 pm
Real name and title: Adrian

Re: Indirect Effects: P-Values and Confidence Intervals

Post by AdrianWorkingWithPLS »

Thank you for your answer! This could be interesting to think about. Still, could anyone explain the large differences when considering both the confidence intervals and the p-values described in my first post?

Thanks again and all the best!
jmbecker
SmartPLS Developer
Posts: 1282
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Indirect Effects: P-Values and Confidence Intervals

Post by jmbecker »

The p-value is based on t-distribution testing and therefore assumes symmetric and close to normal parameter estimates distributions. The bca confidence relax some of these assumption and are, as was said before, better suited for indirect effects, which usually do not follow the assumptions of t-distributed parameters.
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
mpenn
PLS Junior User
Posts: 2
Joined: Wed Jan 03, 2018 8:35 pm
Real name and title: Mortimer Penn, Statistician in Training

Re: Indirect Effects: P-Values and Confidence Intervals

Post by mpenn »

AmerSaeed wrote:According to latest articles bias corrected confidence intervals give the best indication of indirect effects. For your situation perhaps you have run bootstrap analysis with two-tail. If it is then try to run with one-tail and you will get the significant p-value as well.

Hope it will help
Regards
You mention the latest articles saying bias corrected confidence intervals are better indicators for indirect effects than p values. Do you have any specific citations handy.

Thank you for your help,
mp
jmbecker
SmartPLS Developer
Posts: 1282
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Indirect Effects: P-Values and Confidence Intervals

Post by jmbecker »

Particularly for the inference of the indirect effect Hayes and Scharkow (2013) is probably one of the standard sources. Although many researchers tend to cite other more general publications. For example, Hayes repeats and discusses the results together with other research in his 2013 book.

Hayes, A. F., & Scharkow, M. (2013). The relative trustworthiness of inferential tests of the indirect effect in statistical mediation analysis: Does method really matter? Psychological Science.
Hayes, A. F. (2013) Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach

Regarding PLS, I think in the new Primer book there is also the suggestion to use bias-corrected confidence intervals: https://www.smartpls.com/documentation/ ... s-sem-book

In addition, standard treatments of bootstrapping also include discussions of the superiority of bias-corrected approaches, e.g.,
Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. Boca Raton, FL: Chapman & Hall.
Davison, A. C., & Hinkley, D. V. (1999). Bootstrap methods and their application. Cambridge university press.
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
Post Reply