Hello community!
Concerning my Master Thesis I'm doing a scientific research (please keep in mind!) regarding autonomous driving cars. Here I have 80 datasets to check whether my hypotheses are confirmed or not. Here I did the following steps:
1. Step:
To have evidence of the reliability and validity of my measurement model, I computed the correlations between constructs together with the Average Variance Extracted (AVE), Composite Reliability (CR) and the reflective item’s factor loadings by using SmartPLS.
2.Step:
I checked the path coef. and the relating T-statistics whether my hypotheses are confirmed or not.
Now I want to split my dataset into women and men (lets say 40 / 40) and later on into geographical areas to check if there are differences in the target groups.
My question now, do I have to repeat the 1. Step for each target group (women/men) and write it down in my scientific paper or won't that be necessary?
measurement model validation in a scientific paper
-
- PLS Expert User
- Posts: 116
- Joined: Fri Sep 12, 2014 2:12 pm
- Real name and title: Jan Schreier
Re: measurement model validation in a scientific paper
I would say yes, because it could be the case that for men or women some constructs are not defined clearly. But I'm also curious what others think about this question.
-
- PLS User
- Posts: 16
- Joined: Thu Oct 17, 2013 10:04 am
- Real name and title: Marc Janka
- Location: Germany
Re: measurement model validation in a scientific paper
Dear PATR:
Of course, you can test measurement invariance between genders to convince yourself. However, for gender I would not report it in the paper only if it is requested by a reviewer. There are so many aspects, even the educational or social background, the job position, the age etc. of your respondents, that may affect measurement invariance. If you use validated items in your study then pretests should have promoted that measurement invariance can be hold across genders. I guess reporting measurement invariance tests is more important in cross-national research. In these kind of research, reporting statistical tests to establish measurement invariance is very important to show that your questionnaire was translated correctly into the specific languages and that the constructs are operationalized in the same way across different cultures.
Best regards
Marc Janka
Of course, you can test measurement invariance between genders to convince yourself. However, for gender I would not report it in the paper only if it is requested by a reviewer. There are so many aspects, even the educational or social background, the job position, the age etc. of your respondents, that may affect measurement invariance. If you use validated items in your study then pretests should have promoted that measurement invariance can be hold across genders. I guess reporting measurement invariance tests is more important in cross-national research. In these kind of research, reporting statistical tests to establish measurement invariance is very important to show that your questionnaire was translated correctly into the specific languages and that the constructs are operationalized in the same way across different cultures.
Best regards
Marc Janka
Re: measurement model validation in a scientific paper
@MrcJnk thanks for your answer.
currently i'm running into problems by analyzing my different "groups". My questionaire asked for (1) gender, (2) geographical location, (3) age and (4) driving licence.
While the PLS Algorithm is able to deliver values for each sub-group, bootstrapping is not executable, it seems that the sub-sample size is to small even if I add some sub-groups to one bigger group to increase the sub-sample size (e.g. 6 geographical areas are communlated to 3.)
My questions now:
1. As I need the T-values for the significance, is there any opportunity to state the significance level based on the PLS Algorithm?
2. Is there any literature regarding the evaluation/interpretation of the differences between the path coefficients?
For example:
Perceived Traffic Safety --> Perceived Usefulness 0,323
path coefficient overall: 0,323
path coefficient men: 0,365
path coefficient women: 0,147
differences between men and woman 0,218
currently i'm running into problems by analyzing my different "groups". My questionaire asked for (1) gender, (2) geographical location, (3) age and (4) driving licence.
While the PLS Algorithm is able to deliver values for each sub-group, bootstrapping is not executable, it seems that the sub-sample size is to small even if I add some sub-groups to one bigger group to increase the sub-sample size (e.g. 6 geographical areas are communlated to 3.)
My questions now:
1. As I need the T-values for the significance, is there any opportunity to state the significance level based on the PLS Algorithm?
2. Is there any literature regarding the evaluation/interpretation of the differences between the path coefficients?
For example:
Perceived Traffic Safety --> Perceived Usefulness 0,323
path coefficient overall: 0,323
path coefficient men: 0,365
path coefficient women: 0,147
differences between men and woman 0,218
-
- PLS User
- Posts: 12
- Joined: Sat Aug 20, 2016 3:09 pm
- Real name and title: Morteza - PhD student
Re: measurement model validation in a scientific paper
Maybe this video helps you:
https://www.youtube.com/watch?v=-BI8VweLQPc
https://www.youtube.com/watch?v=-BI8VweLQPc
-
- PLS Junior User
- Posts: 3
- Joined: Tue Aug 16, 2016 7:39 pm
- Real name and title: MOHAMED TAILAB-PhD STUDENT
Re: measurement model validation in a scientific paper
Here is my advice for you:
First, you should assess your model in three steps:
(1) Evaluating Reflective Measurement Models: This can be done by consistency reliability, indicator reliability, discriminant validity try to use Cornell-Lacker and Cross-Loading; and Average Variance Extracted
(2)Evaluating Formative Measurement Models: This can be done by Multicollinearity.
(3)Evaluating PLS-SEM Structural Models. this can be done by R square, Q square, f square q, square, and path coefficient. Pay attention to R square due to your subject.
if your model met the whole requirements, then you have valid model that ready to by run, and report your results. During your evaluation, may or may be not you need to remove some indicators for the initial model, pay attention that you should not not allow your statistic results drive your theory, sometimes you need some indictors even if there loading values are very low.
As for your questions, I could not get what do you mean, but I can say based on my understanding
We use SmartPLS since it is very recommended with a small sample size. So, there is no problem with that.
for your T-values, let me tell you this. we are as researchers, we can not say that we need these results, we have to deal with our results as it is, and deliver them to the audience with objective opinion. So, there is no any problem if your t-values are not significant or R square is very low. if your t-values are insignificant, It is totally fine, the one thing you should work on is for your discussion part; compare your results with the pervious work
First, you should assess your model in three steps:
(1) Evaluating Reflective Measurement Models: This can be done by consistency reliability, indicator reliability, discriminant validity try to use Cornell-Lacker and Cross-Loading; and Average Variance Extracted
(2)Evaluating Formative Measurement Models: This can be done by Multicollinearity.
(3)Evaluating PLS-SEM Structural Models. this can be done by R square, Q square, f square q, square, and path coefficient. Pay attention to R square due to your subject.
if your model met the whole requirements, then you have valid model that ready to by run, and report your results. During your evaluation, may or may be not you need to remove some indicators for the initial model, pay attention that you should not not allow your statistic results drive your theory, sometimes you need some indictors even if there loading values are very low.
As for your questions, I could not get what do you mean, but I can say based on my understanding
We use SmartPLS since it is very recommended with a small sample size. So, there is no problem with that.
for your T-values, let me tell you this. we are as researchers, we can not say that we need these results, we have to deal with our results as it is, and deliver them to the audience with objective opinion. So, there is no any problem if your t-values are not significant or R square is very low. if your t-values are insignificant, It is totally fine, the one thing you should work on is for your discussion part; compare your results with the pervious work
-
- SmartPLS Developer
- Posts: 1284
- Joined: Tue Mar 28, 2006 11:09 am
- Real name and title: Dr. Jan-Michael Becker
Re: measurement model validation in a scientific paper
I would say that you should be very careful with splitting your sample. 80 observations is already a quite small sample. Further splitting this sample and thus reducing the sample size will give you unreliable results. It is not surprising that you run into calculation problems. Try to gather more data or to incorporate the grouping-variables into your model (as controls or moderators). That may solve some sample size problems (but 80 is still not very much).
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de