Please help with the inconsistent conclusins of POS-PLS

[AAljabr]

Dear SmartPLS team,

I used the FIMIX as a first step to determine the number of segments in my data, and all the fit indices as well as the EN indicate that I have two segments in my data (I compared the one segment versus the two segment solutions only, and I did not try more than two segments because of sample size restrictions). My model include both reflective and formative constructs (attached), and my sample size is 200. Given that POS-PLS outperforms the FIMIX in examining the unobserved heterogeneity, I used it for this purpose. In relation to its application in SmartPLS, I did the following procedure:

- In relation to the settings, I did the following:
1. Number of groups= 2
2. Maximum iterations= 1000
3. Search depth= 200 (equal to the sample size).
4. I ticked the choice related to using pre-segmentation in the first round.
5. In relation to optimization criterion, I was not sure whether I should choose:
a. Sum of all constructs R-squares.
b. Sum of target construct R-square.
I have two endogenous constructs, CSC and TMS. However, the most important one is CSC because all constructs in the model are hypothesized to be related to it. In contrast, TMS is only influenced by one construct, TMKA, and at the same time TMS is hypothesized to be related to CSC (i.e., TMS acts as a mediator construct between TMKA and CSC). Given that CSC is the main focus, I choose the second alternative (Sum of target construct R-square).

After finishing the settings, I conducted POS many times (each time with ten repetitions). I choose the solution with the highest average weighted R square while keeping an eye on the number of iterations used and the search depth used. However, what I have noticed is that the solutions vary in relation the segments' size even though they have approximately similar values of R squares. The differences of segments' size have important implications in relation to:

1. Finding an explanatory variable to describe the segments.
2. Examining whether there is measurement invariance and, subsequently, whether there are significant differences in the structural model's paths (using the permutation multi-group analysis).

My problem is that:

I conducted the analysis many times (each time with ten repetitions as described above), and I choose the best solution with the highest R square. Then, I run the permutation test to examine whether there the measurement invariance requirement has been achieved, and whether there are significant differences in the structural paths which suggests that the segments should not be combined and that I need to find an explanatory variable that describe the segments.. What I found is that the findings of the permutation test are not the same for each time. In particular, some times I find that the measurement invariance requirement has been achieved and that there are no significant differences in the structural paths. Other times, I find some significant differences in the structural paths. Please note that the sample sizes of the different POS solutions are sometimes below the minimum required sample size using the power analysis.

Can any one provide any advice?

Notes:
1. I use Mode A for formative constructs to solve the negative weights problems.
2. PC is a second order construct, and I used the two step procedure instead of the repeated measure.
3. Production type is a control construct.
4. All VIFs of the endogenous constructs are below 2.

Thanks
Abdul

[AAljabr]

Please note that although FIMIX suggested the two segments solution, the size of the second segment is relatively lower than the minimum required sample size for my model (28.2% of the sample size (n=200), which equals 56 cases).

[jmbecker]

I think the basic problem is your sample size. If one of your segments is too small, it will produce unreliable results. That might cause the instability of the POS algorithm. For such a complex model, you should have at least 100 observations per group if not more.

Moreover, a problem with regard to the permutation approach is your unequal sample size between the groups. The permutation procedure works well if your groups are more or less equally sized. Having a 75% to 25% distribution as in your case can also make results of the permutation approach less reliable. In such cases you should revert to the other MGA procedures for the test of significant group differences. Yet, your one groups will still be too small on an absolute basis.

[AAljabr]

Dear Dr. Becker,

Thank you for your response.
I understand your point about the permutation test.

In relation to POS, what is best action to investigate unobserved heterogeneity?
Given that the FIMIX suggested a two segments solution with a segment's size being 28.2% for the first segment and 61.8% for the second, can I consider unobserved heterogeneity not to affect my results?

Thanks

[jmbecker]

I would rather put it in the limitations. Note that there seems to be unobserved heterogeneity, as FIMIX points to a two segment solution and that given your model PLS-POS would be the better choice for uncovering the heterogeneity, but given your limited sample size that future research should investigate the heterogeneity with larger sample sizes.

[AAljabr]

Dear Dr. Becker,

Thanks for your response.
If I take the FIMIX output which indicates a two segment solution as a reliable result, and use it as an input when running the POS-PLS (i.e., determining a two segments choice prior to running the POS-PLS). My problem is with results of POS-PLS. In particular, it is difficult to decide whether the second segment is substantial because the results vary. Sometimes, the second segment accounts for approximately 45% to 50%, and sometimes it accounts for less than 30%. Here, the conclusion whether the second segment is substantial changes every time you run the a set of POS-PLS (set= 10 runs and choosing the solution with the highest R square).