Original scientific paper
https://doi.org/10.11567/met.40.1.1
Measurement Invariance of Some Measures of Intergroup Relations Between Refugees and Members of the Receiving Communities
Jana Kiralj Lacković
; Zagreb
Una Mikac
orcid.org/0000-0001-9369-6462
; Odsjek za psihologiju, Filozofski fakultet u Zagrebu, Zagreb
*
* Corresponding author.
Abstract
The number of forcibly displaced persons has steadily grown for the past ten years (UNHCR, 2015–2021), providing a humanitarian and political challenge as well as a need to understand better the dynamics between the arriving and the receiving communities. Integration is a two-way process of mutual accommodation of members of both groups (European Commission, 2005), multidimensional and dynamic. One of the goals of integration is to reach a positive view of the other group’s members in a socially desired way and foster mutual acceptance between refugees and receiving community members (RCMs) (Kiralj Lacković et al., 2023). The involvement of both RCMs and refugees is crucial in the integration process, making it interesting to compare them to describe their dynamics, similarities, and differences.
In ethnic and socio-psychological research, group comparison is a common method for understanding group differences. However, testing measurement invariance is often neglected in such research (Boer et al., 2018) despite it being the prerequisite for interpreting group mean differences or interrelations of phenomena with regard to group membership. Measurement invariance points to the equivalence of the instrument as a measure of a construct in various groups or across time (Putnick and Borstein, 2016). Testing measurement invariance does not answer whether the groups are different in their results (e.g. “Do RCMs have higher scores than refugees?”), but rather, are there any differences in the way the measure is interpreted by the groups (“Does the measure behave in the same way for RCMs and refugees?”). If the measure is non-invariant, we cannot be sure whether the potential differences in the scores between the groups reflect the actual differences in the construct or the differences in the understanding of the measure (Leitgöb et al., 2023).
Measurement invariance has several levels, with structure, loading, and intercept invariance tested most often and with previous levels required for the following ones (Little, 2024). Each of these levels poses a restriction on a certain set of parameters – characteristics of the individual items, which can differ across groups. Structure or configural invariance points to the equivalence of the number of factors and the number of adjacent observed variables. After this level, we test for the loadings or metric invariance – one which assumes that the factor loadings of items are the same across groups. This invariance implies that the variability of each observed variable contributes to the variability of the factor to the same degree across the groups. Thus, it allows between-group comparison of correlations or regression coefficients of the invariant measure scores with other measures. Next, we test for intercept or strict invariance, which dictates that the intercepts are the same between groups. This invariance implies that the observed variables contribute to the factor’s mean to the same degree across groups. If so, we can interpret mean score differences between the two groups as construct differences. We can also differentiate between full invariance, when none of the parameters of interest differ between groups, and partial, when some, but not all of them, differ.
The goal of this study was to test the measurement invariance of two measures commonly used in socio-psychological research on the integration of refugees and RCMs – intergroup attitudes and perception of threat.
A probabilistic sample of 600 RCMs and a convenient sample of 149 refugees from Syria participated in the study as a part of the project FOCUS funded by the European Commission. The data was collected face-to-face using a survey. Two measures previously used in research on integration in Croatia were used (Ajduković et al., 2019): the adapted Attitudes Towards Refugees Scale (6 items) and the Realistic and Symbolic Threat Scale (3/4 items per subscale).
Before testing invariance, CFA was conducted to test whether the latent structure of the measures fitted the structures based on theory and prior research, and reliability was calculated as Cronbach Alfa and McDonald Omega. Measurement invariance was explored using multigroup CFA by comparing a series of more complex models: structure, loading, and intercept invariance models. In case the more complex (later) model shows an equally good fit to the data as the previous one, we have shown a level of invariance corresponding to the later model. In case the more complex (later) model does not fit the data equally well, a partial model, which equalises some but not all factor loadings or intercepts, can be tested. In the case of partial invariance, the choice of potential non-invariant items was guided by the forward method using confidence intervals described by Jung and Yoon (2016). Model parameters were estimated using robust ML and FIML methods of missing data handling (Newman, 2014). For model identification, we used the referent variable method, with the variable chosen based on the highest loading in the EFA. Model fit was evaluated using a combination of standard indicators (RMSEA, CFI, SRMR; Little, 2024) and model comparison based on the difference in the chi-square test (structure), CFI and RMSEA (invariance; Chen, 2007).
Attitudes Towards Refugees Scale (referred in this paper as the Attitudes Towards Members of Another Group Scale) showed a partial loading and intercept invariance, with five out of six factor loadings equal between refugees and RCMs, and three out of six intercepts equal between the groups. The Realistic Threat Subscale showed full loading and partial intercept invariance, with all factor loadings equal between refugees and RCMs and two out of three intercepts equal between the same groups. The Symbolic Threat Subscale showed partial loading and intercept invariance, with three out of four factor loadings and two out of four intercepts equal between refugees and RCMs.
The established levels of invariance can lead to four approaches. First, according to the most often used approach, statistical analyses can be adjusted. When full invariance is achieved, intergroup comparisons could be made at the manifest level (regressions with Realistic Threat Subscale) and with partial invariance at the latent level (means and regressions with Attitudes Towards Refugees Scale and Symbolic Threat Subscale). With no invariance, intergroup comparisons could be misleading.
The second approach would comprise the analysis of the content of invariant items to learn more about construct differences between groups. Readiness to assist a member of the other group had a non-invariant loading, meaning that refugees and RCMs perceived this item differently in the context of attitudes towards each other. It is possible that due to the nature of the minority-majority relations and the power dynamics, refugees do not perceive having a choice in the matter and see helping the RCMs as mandatory. This could alter their perception of the item itself. The item measuring the fear of the other group endangering the values and way of life of the own group had a non-invariant factor loading on the Symbolic Threat Factor between the groups. Findings imply that in the RCMs, fear of endangered values adds to the perception of symbolic threat more than it does for refugees. RCMs may be more likely to perceive the differences between themselves and refugees in this context than vice-versa, which is supported by a previous study showing that RCMs emphasise the differences between themselves and Muslim refugees, which is true for most refugees from Syria (Kiralj and Ajduković, 2022).
Third, we could form invariant measures by excluding those items that show non-invariance, although this approach should be used with caution, as it can lead to a lack of generalisability across studies. Fourth, the level of non-invariance could also be controlled statistically with methods such as Bayesian approximate measurement invariance and alignment.
These conclusions should be interpreted in the light of differences in the subsample size, the chosen grouping variable, and the analysis used. This study is based on a most used multigroup method for testing measurement invariance, which requires relatively large samples, roughly continuous variables and linear relations, theoretically clear and clean structure and is sensitive to the smallest levels of invariance (Somaraju, Nye and Olenick, 2022). Other methods for testing invariance are also available, e.g. the MIMIC model or ESEM.
The findings pointing to non-invariance should not be perceived as unwanted and discarded. They reflect interesting differences between the groups in their interpretation of the construct. This is particularly interesting in research of socio-psychological integration or other intergroup and ethnic research, as it sheds light on the understanding that different groups have of the constructs which are most often treated as universal, such as intergroup attitudes. Non-invariance can raise important questions that can lead to future research, and with a careful interpretation and conscience of the potential differences in the understanding of the measurement instruments, the results of statistical tests conducted after invariance testing can be a very valuable basis for future research. With the rise of research on socio-psychological integration, including the important research on the similarities between refugees and RCMs, invariance testing could be used not only as a preliminary analysis of measures but also as a way of revealing a more complex understanding of the constructs related to intergroup relations.
Keywords
measurement invariance; intergroup attitudes; perception of threat; refugees; receiving community
Hrčak ID:
319761
URI
Publication date:
30.6.2024.
Visits: 291 *