Measurement Invariance for MGA not necessary?!

[Research016]

Hi, I have a question concerning my research. I use PLS-SEM MGA to compare three groups. Each group got a different treatment: eco-friendly fashion, recycled fashion, and second-hand fashion.

Now I wanted to test for measurement invariance. However, I think in this case of research checking for measurement invariance makes no sense, its not necessary. Measurement Invariance testing is done to rule out that possible path differences between the groups stem from different interpretations or meanings of the latent variables by the different respondents of the groups, right? For instance, path differences in a structural model that differentiates three groups divided by country of residence could be the result of translation problems or different (cultural) associations instead of true differences between the groups such as perceived value or attitudes.

However, my research compares a research model across three groups which are divided by three distinct treatments. This means that the path differences between the groups are expected to be the reason of the different treatments, not the reason of the different characteristics of the participants in each group. Eg. To check for perceived monetary value I say “The second hand fashion offers value for money” and “the recycled fashion offers value for money” etc. So the items are obviously differing. Usually, scholars conducting MGA divide their groups by gender or country e.g. while the items of the entire questionnaire are identical. This means that the difference then lies within the specific group characteristics (e.g. gender), not in the questionnaire context. With my reseach, the difference does directly lie in the questionnaire context, not within the characteristics of the respondents; respondents for each group are expected to be very similar to each other. Therefore, testing for measurement invariance is not appropriate. Is that correct? Or is it only about checking the meaning of the latent variables?

Any help would be greatly appreciated!! Thank you!

[jmbecker]

That is a good question.
In general, I think it would be necessary to show partial measurement invariance also for such research designs, because the treatment could cause changes in response styles or interpretation of the measures instead of actual differences in the conceptual variables and their relations. Therefore, it would be important to first show partial measurement invariance before looking at difference in the means and differences in path coefficients.

However, it is rarely done in such research settings, most likely with similar reasoning/assumptions as yours that measurement invariance is more important for cross-cultural research or at least comparing groups that come from different populations. And to some degree, it is probably true that in such situations it is more likely to find problems with measurement invariance. Nevertheless, if you want to be sure, you should also test and thereby rule out that your treatment caused problems with measurement invariance.

[Research016]

Thank you for your quick answer!

You said that we want to rule out that differences come from different response styles due to the different treatments. What exactly do you mean with response styles? The pattern or the answers themselves? Because I actually want the treatment to cause different responses. The hypotheses state that people evaluate, eg. value for money different for the three different business models (recycled fashion, eco-friendly fashion and second-hand fashion). So the treatment should indeed cause these changes in the variable of perceived value for money eg.

The thing is that when I check for Measurement Invariance in SmartPLS using the MICOM procedure, the majority of the pairwise comparisons come back with significant permutation p-values for some latent variables, indicating that partial measurement invariance is not established. I thought that this is due to the different contexts in my items though? In other words, I would have expected problems with measurement invariance since I am using a different context for the three groups.
Therefore I thought I already know that contextual differences exist and these are intentionally there as the difference here doesn't lie within the group characteristics (e.g. USA vs Mexico) but within the context, the wording in the items themselves (e.g. "the second-hand clothing offers value for money" vs. the other group with the item "recycled clothing offers value for money"). Does that make any sense or am I on the wrong track?

[jmbecker]

With response styles, I mean how respondents react to the measurement instrument.
You want respondents to show differences in the conceptual variable of interest and their relationships.
However, what you do not want is that people in the treatment group, for example, show higher likelihood of using extreme response categories. Some treatments might have "side effects" that people understand the scale questions differently, put more or less effort into reading the questions, or increase their willingness take extremes. Unless you want to measure those things, it would be a unwanted side effect if people suddenly change their behavior on how they react to the measurement instrument. That would confound the measurement of changes in the focal variables with the potential changes in how people react to measurement.
So yes, it is also about establishing that the meaning of the latent variable is the same across the groups and that you actually measure the same thing across the groups.

I am mot saying that this is necessarily the case in your research and many research doing experimentation with different treatments ignores these problems, but I am just saying that you usually want to make sure that measurement is invariant across different groups that you want to compare. Otherwise, you are comparing apples with oranges,