forum.smartpls.com

Dear Forum-users,

since the available info on holdout samples with PLS-SEM - to my knowledge - is relatively limited, I tried to produce something for myself. So i created a macro with which you can easily take your models outer weights and the path cofficients and run this against the holdout sample data to obtain the R²-values for the holdout sample. As far as I understand the R²-values of the sample and the holdout sample should be fairly equal.

What I'm currently not 100% sure of is the data normalization process. Do you need to normalize all data (sample + holdout) together before doing anything in smartPLS or do you normalize both data sets separately?

Is this sufficient for a holdout sample test? Or do I need to do more?
If yes, this linked Excelsheet should do the trick with few clicks: http://bit.ly/1L2LnTY

Hope this helps somebody!
jan

Sounds like a good feature to implement into SmartPLS! Such example help us a lot to advance develoipments.

Best regards
Christian

Good to hear, Prof. Ringle! I'm glad I can help :)

Would be pleased to get some feedback on the two methodical questions, too, though.

BR, Jan Schreier

Dear All,

You say that R2 values of the sample and the holdout sample should be fairly equal. Some breaf questions please:

1- Is there any reference / rule of thumb for assessing what is considered to be "fairly equal"?

2- If R2 values look too different, I guess the predictive validity would be problematic. How would we report it? for example noting that "the findings should be treated with caution since the predictive validity of the model is not large"? is that correct?

3- Is there a new feature in smartPLS last version related to predictive validity issues with holdout samples?

Thanks in advance!

Hi Jan;
I am not sure if you have a chance to read Prof., Henesler file Assessing the Predictive Validity of PLS Path Models Using Holdout Samples?
I think that might be also a good help to confirm.
Best, Murad

janschreier wrote:Dear Forum-users,

since the available info on holdout samples with PLS-SEM - to my knowledge - is relatively limited, I tried to produce something for myself. So i created a macro with which you can easily take your models outer weights and the path cofficients and run this against the holdout sample data to obtain the R²-values for the holdout sample. As far as I understand the R²-values of the sample and the holdout sample should be fairly equal.

What I'm currently not 100% sure of is the data normalization process. Do you need to normalize all data (sample + holdout) together before doing anything in smartPLS or do you normalize both data sets separately?

Is this sufficient for a holdout sample test? Or do I need to do more?
If yes, this linked Excelsheet should do the trick with few clicks: http://bit.ly/1L2LnTY

Hope this helps somebody!
jan

Dear Murad,

do you have a more exact source/reference for Prof. Henseler's paper?

Thanks for your help!
Jan

Hi Jan,
Please send me your mail.
Best

janschreier wrote:Dear Murad,

do you have a more exact source/reference for Prof. Henseler's paper?

Thanks for your help!
Jan