measurement model validation in a scientific paper

Questions about the implementation and application of the PLS-SEM method, that are not related to the usage of the SmartPLS software.
Post Reply
PATR
PLS Junior User
Posts: 3
Joined: Tue Aug 02, 2016 1:49 pm
Real name and title: Patrick R

measurement model validation in a scientific paper

Post by PATR »

Hello community!

Concerning my Master Thesis I'm doing a scientific research (please keep in mind!) regarding autonomous driving cars. Here I have 80 datasets to check whether my hypotheses are confirmed or not. Here I did the following steps:

1. Step:
To have evidence of the reliability and validity of my measurement model, I computed the correlations between constructs together with the Average Variance Extracted (AVE), Composite Reliability (CR) and the reflective item’s factor loadings by using SmartPLS.

2.Step:
I checked the path coef. and the relating T-statistics whether my hypotheses are confirmed or not.

Now I want to split my dataset into women and men (lets say 40 / 40) and later on into geographical areas to check if there are differences in the target groups.
My question now, do I have to repeat the 1. Step for each target group (women/men) and write it down in my scientific paper or won't that be necessary?
janschreier
PLS Expert User
Posts: 116
Joined: Fri Sep 12, 2014 2:12 pm
Real name and title: Jan Schreier

Re: measurement model validation in a scientific paper

Post by janschreier »

I would say yes, because it could be the case that for men or women some constructs are not defined clearly. But I'm also curious what others think about this question.
MrcJnk
PLS User
Posts: 16
Joined: Thu Oct 17, 2013 10:04 am
Real name and title: Marc Janka
Location: Germany

Re: measurement model validation in a scientific paper

Post by MrcJnk »

Dear PATR:

Of course, you can test measurement invariance between genders to convince yourself. However, for gender I would not report it in the paper only if it is requested by a reviewer. There are so many aspects, even the educational or social background, the job position, the age etc. of your respondents, that may affect measurement invariance. If you use validated items in your study then pretests should have promoted that measurement invariance can be hold across genders. I guess reporting measurement invariance tests is more important in cross-national research. In these kind of research, reporting statistical tests to establish measurement invariance is very important to show that your questionnaire was translated correctly into the specific languages and that the constructs are operationalized in the same way across different cultures.

Best regards

Marc Janka
PATR
PLS Junior User
Posts: 3
Joined: Tue Aug 02, 2016 1:49 pm
Real name and title: Patrick R

Re: measurement model validation in a scientific paper

Post by PATR »

@MrcJnk thanks for your answer.

currently i'm running into problems by analyzing my different "groups". My questionaire asked for (1) gender, (2) geographical location, (3) age and (4) driving licence.
While the PLS Algorithm is able to deliver values for each sub-group, bootstrapping is not executable, it seems that the sub-sample size is to small even if I add some sub-groups to one bigger group to increase the sub-sample size (e.g. 6 geographical areas are communlated to 3.)

My questions now:
1. As I need the T-values for the significance, is there any opportunity to state the significance level based on the PLS Algorithm?
2. Is there any literature regarding the evaluation/interpretation of the differences between the path coefficients?
For example:
Perceived Traffic Safety --> Perceived Usefulness 0,323
path coefficient overall: 0,323
path coefficient men: 0,365
path coefficient women: 0,147
differences between men and woman 0,218
mnabi
PLS User
Posts: 12
Joined: Sat Aug 20, 2016 3:09 pm
Real name and title: Morteza - PhD student

Re: measurement model validation in a scientific paper

Post by mnabi »

MOHAMED-2016
PLS Junior User
Posts: 3
Joined: Tue Aug 16, 2016 7:39 pm
Real name and title: MOHAMED TAILAB-PhD STUDENT

Re: measurement model validation in a scientific paper

Post by MOHAMED-2016 »

Here is my advice for you:
First, you should assess your model in three steps:
(1) Evaluating Reflective Measurement Models: This can be done by consistency reliability, indicator reliability, discriminant validity try to use Cornell-Lacker and Cross-Loading; and Average Variance Extracted
(2)Evaluating Formative Measurement Models: This can be done by Multicollinearity.
(3)Evaluating PLS-SEM Structural Models. this can be done by R square, Q square, f square q, square, and path coefficient. Pay attention to R square due to your subject.
if your model met the whole requirements, then you have valid model that ready to by run, and report your results. During your evaluation, may or may be not you need to remove some indicators for the initial model, pay attention that you should not not allow your statistic results drive your theory, sometimes you need some indictors even if there loading values are very low.

As for your questions, I could not get what do you mean, but I can say based on my understanding
We use SmartPLS since it is very recommended with a small sample size. So, there is no problem with that.
for your T-values, let me tell you this. we are as researchers, we can not say that we need these results, we have to deal with our results as it is, and deliver them to the audience with objective opinion. So, there is no any problem if your t-values are not significant or R square is very low. if your t-values are insignificant, It is totally fine, the one thing you should work on is for your discussion part; compare your results with the pervious work
jmbecker
SmartPLS Developer
Posts: 1282
Joined: Tue Mar 28, 2006 11:09 am
Real name and title: Dr. Jan-Michael Becker

Re: measurement model validation in a scientific paper

Post by jmbecker »

I would say that you should be very careful with splitting your sample. 80 observations is already a quite small sample. Further splitting this sample and thus reducing the sample size will give you unreliable results. It is not surprising that you run into calculation problems. Try to gather more data or to incorporate the grouping-variables into your model (as controls or moderators). That may solve some sample size problems (but 80 is still not very much).
Dr. Jan-Michael Becker, BI Norwegian Business School, SmartPLS Developer
Researchgate: https://www.researchgate.net/profile/Jan_Michael_Becker
GoogleScholar: http://scholar.google.de/citations?user ... AAAJ&hl=de
Post Reply