q² effect size

Frequently asked questions about PLS path modeling.
Post Reply
imdad7371
PLS Junior User
Posts: 2
Joined: Tue Jan 16, 2018 9:39 am
Real name and title: Mr. Imdadulah

Re: Some methodological issues

Post by imdad7371 » Tue Jan 16, 2018 9:41 am

When assessing q², I get negative values. Can I ignore the negative symbol and interpret the magnitude of the value whether positive or negative?

jmbecker
SmartPLS Developer
Posts: 842
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: Some methodological issues

Post by jmbecker » Tue Jan 16, 2018 10:35 am

Generally, q² can also be negative unlike f².
The R² (i.e., explained variance) always improves if you add more predictors (and therefore f² is always positive).
The predictive relevance (Q²) of a model can also decrease if you add predictors, due to overfitting of the model (e.g., adding irrelevant noisy predictors).
Thus, theoretically, a negative q² means that removing the focal variable actually improves the predictive power of the model (Q²). That is, in terms of predictions, you are better off without the focal variable.
However, there are some potential concerns with calculating q² in a PLS model (which are also the reason why we have not implemented such a calculation).
For example, by deleting a latent variable you are changing the model and thus the PLS estimates might change as they are dependent on the structural model. Whether this is good or not (correct or not) when calculating q² has never been finally decided.
Deleting variables from the PLS model may change the order of manifest variables in the missing value substitution and prediction process. This may positively or negatively effect the predictive relevance without having a reason in the deletion of the focal variable.
Therefore, the change in Q² may not be attributable only to the inclusion/exclusion of the focal variable. Thus, q² might not be a reliable measure, which is why I usually do not recommend it.
At the moment, more research is necessary to more fully understand this type of analysis and how it can be correctly applied.
Dr. Jan-Michael Becker, University of Cologne, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Ja ... v=hdr_xprf
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de

Post Reply