q² effect size

[imdad7371]

When assessing q², I get negative values. Can I ignore the negative symbol and interpret the magnitude of the value whether positive or negative?

[jmbecker]

Generally, q² can also be negative unlike f².
The R² (i.e., explained variance) always improves if you add more predictors (and therefore f² is always positive).
The predictive relevance (Q²) of a model can also decrease if you add predictors, due to overfitting of the model (e.g., adding irrelevant noisy predictors).
Thus, theoretically, a negative q² means that removing the focal variable actually improves the predictive power of the model (Q²). That is, in terms of predictions, you are better off without the focal variable.
However, there are some potential concerns with calculating q² in a PLS model (which are also the reason why we have not implemented such a calculation).
For example, by deleting a latent variable you are changing the model and thus the PLS estimates might change as they are dependent on the structural model. Whether this is good or not (correct or not) when calculating q² has never been finally decided.
Deleting variables from the PLS model may change the order of manifest variables in the missing value substitution and prediction process. This may positively or negatively effect the predictive relevance without having a reason in the deletion of the focal variable.
Therefore, the change in Q² may not be attributable only to the inclusion/exclusion of the focal variable. Thus, q² might not be a reliable measure, which is why I usually do not recommend it.
At the moment, more research is necessary to more fully understand this type of analysis and how it can be correctly applied.