This is an open access article distributed under the terms of the Creative Commons
Attribution License (As society becomes more aware of the negative consequences of conspiracy theories, research interest in these beliefs and the way they are disseminated is rising, especially in the context of global crises such as the COVID-19 pandemic (e.g., Stojanov et al., 2023). However, to study a construct, it typically helps to quantify it. So far, operationalizations of conspiracy beliefs have not always adequately reflected what is commonly understood by the term. For example, items tapping into paranormal beliefs (Graeupner & Coman, 2017), corruption (van Prooijen & Acker, 2015) or paranoid ideation (Whitson & Galinsky, 2008) have served as proxies for measuring conspiracy beliefs. In other instances, researchers have developed ad-hoc scales for measuring specific conspiracy beliefs, merely relying on the face validity of those scales (e.g., Douglas & Sutton, 2008), with proper validation studies reported only in rare occasions (e.g., Shapiro et al., 2016). However, with the insight that conspiracy theory beliefs are interrelated (e.g., Goertzel, 1994; Imhoff & Lamberty, 2017; Lukić et al., 2019; Wood et al., 2012) and that they form part of a ‘conspiracy mentality’ (Imhoff et al., 2022; Imhoff & Bruder, 2014) measurement efforts have shifted towards capturing this general tendency to believe in conspiracy theories in the development of validated generic scales (Brotherton et al. 2013; Bruder et al., 2013; Stojanov & Halberstadt, 2019).
Some of the first such instruments were the Generic Conspiracist Beliefs scale (Brotherton et al., 2013) and the Conspiracy Mentality Questionnaire (Bruder et al., 2013). However, these scales were criticized by Swami et al. (2017) for unstable factor structure or tapping into rational beliefs. For example, the Generic Conspiracist Beliefs Scale has inconsistent factor structures, ranging from two to five factors (Atari et al., 2019; Brotherton et al., 2013; Drinkwater et al., 2020; Majima & Nakamura, 2020; Siwiak et al., 2019; Stojanov & Douglas, 2022), undermining its 'generic' nature due to clustering of content-specific items like those related to extra-terrestrial cover-ups. Meanwhile, the single-factor Conspiracy Mentality Questionnaire has drawn criticism for conflating rational beliefs reflecting world states with conspiracy theory beliefs (Swami et al., 2017). A psychometric instrument that overcomes these drawbacks is the Conspiracy Mentality Scale (CMS, Stojanov & Halberstadt, 2019). In this paper, we report a French and Italian validation of this particular scale, which assesses generic conspiracy beliefs by the means of two subscales: (1) conspiracy theory ideation (i.e. beliefs in conspiracy theories, seven items), and (2) skepticism (i.e. a more mundane suspiciousness, four items).
The original CMS validation study demonstrated that the conspiracy theory ideation subscale is suitable for measuring conspiracy beliefs in the US, New Zealand and Macedonia, and the subscale has been used widely for this purpose (e.g., Craig & Sadovykh, 2022; Gligorić et al., 2021). The skepticism subscale, on the other hand, has predicted belief in more mundane allegations, such as bribery and corruption, and the developers recommended its use when researchers want to partial out the variance caused by such a mundane suspiciousness from the suspicion that characterizes conspiracy beliefs.
As a relatively new scale, further validation studies and examination of the CMS’ psychometric properties in different samples and in other languages are needed. The availability of multi-lingual validations is especially important given the speed with which conspiracy theories are disseminated across geographical boundaries, and the potential negative effects their proliferations impose on individuals and society. Further multi-lingual validations also provide the measurement means for attaining insights into the psychological and social demographics that characterize individuals who produce, consume and believe conspiracy theories across the globe. The latter are particularly important, given that conspiracy proliferation is primarily a communication phenomenon. For example, studies have shown that conspiracy mentality predicts the endorsement of deceptive claims (Landrum & Olshansky, 2019), which became relevant during COVID-19 where a massive “infodemic” suddenly accompanied the spread of the virus, with a swift circulation of false information that nurtured conspiracy claims. The way messages are communicated also directly influences their perceived trustworthiness, which in turn relates to conspiracy ideation. Complex (vs. simple) messages about mask wearing, for example, reduced trustworthiness only in individuals with a conspiracy mentality during COVID-19 (Schnepf et al., 2021). These multifaceted interrelations need to be better understood, and the availability of a valid measure of conspiracy mentality in different languages is needed for improving our understanding of the factors and processes that relate conspiracy ideation to communication.
To assess construct validity of the CMS’ two new language versions, we examined the associations of each CMS subdimension with the following variables: compliance with COVID-19 pandemic control measures, perceived severity of the pandemic, functional literacy, and social validation. Below, we outline the rationale for testing these associations.
Compliance With the Pandemic Control Measures
In terms of compliance with the pandemic measures, we expected a negative correlation with conspiracy theory ideation. Our reasoning was based on previous studies across the world which have shown conspiracy beliefs to be associated with lower compliance with the recommended guidelines (Karić & Međedović, 2021). We did not expect a correlation between compliance and skepticism, since individuals with a high mundane suspiciousness might have conflicting perceptions regarding compliance (e.g. there is not enough evidence on the effectiveness of the preventive measures, so why wear a mask / better be careful), which would ultimately lead to an annulling effect and no correlation.
Perceived Severity of the Pandemic
Past research has demonstrated that conspiracy theory beliefs negatively correlate with COVID-19 risk perception (Plohl & Musil, 2021). Thus, we hypothesized a negative relationship between perceived severity of the COVID-19 pandemic and conspiracy theory ideation. We expected no relationship with skepticism, as mundane suspiciousness may operate in both directions, given that the long-term effects of COVID-19 are unknown (i.e., “the pandemic is not as severe as we are led to believe”, and “the pandemic is more severe than we are led to believe”).
Functional Literacy
Previous studies have linked conspiracy beliefs to innumeracy (Martini et al., 2022), lower health literacy (Duplaga, 2020), lower news media literacy (Craft et al., 2017) and lower scientific literacy (Luo & Jia, 2022). Therefore, we predicted a negative relationship between conspiracy ideation and functional literacy (i.e. people’s ability to read, understand and comprehend the official COVID-19 messages). We expected a positive relationship with skepticism, as the general suspiciousness that characterizes this subfactor requires a certain degree of knowledge and comprehension.
Social Validation
The need for belongingness is considered a powerful motive for conspiracy ideation (Douglas et al., 2017). Therefore, we predicted that those high in conspiracy theory ideation would tend to associate with like-minded individuals to satisfy their need for social validation. Based on a previous study (Stojanov & Hannawa, 2023b), we expected a negative relationship between social validation and skepticism, because a higher need for validation would mean that one would appraise a situation less suspiciously and critically to comply with the norm, which is contrary to what skeptics would do.
Measurement Invariance
When a survey is administered to two or more distinct (e.g., culturally or linguistically) groups, a question arises if it ‘operates’ equivalently (or invariantly) across the groups, as even though the items are the same, they may have different meaning for participants from different settings (Putnick & Bornstein, 2016). In particular, there are several points at which the instrument could behave differently across the groups. First, if configural invariance is not met, it means that the same items load on different factors in the two groups. Configural invariance is usually established by conducting confirmatory factor analysis and examining the model fit indices. Next, researchers need to establish metric invariance. This involves demonstrating that, in both groups, the items load in the same way on the respective factors, and is implemented by constraining the item loadings to be equal across groups and comparing the model fit of the constrained model the configural (i.e. baseline) mode. Once metric invariance is met, researchers examine scalar invariance. Scalar invariance ensures that the level of the construct as measured by the items is comparable across groups and is established by demonstrating equivalence of the item intercepts. Strict invariance involves demonstrating that the measurement error is equivalent across the groups. Finally, structural invariance involves demonstrating that the relationship between the latent factors is the same across groups (Byrne, 2009; Putnick & Bornstein, 2016). In the present paper we examine the measurement equivalence across the French and Italian participants.
Method
Sampling and Procedures
The data for the scale validation was collected as part of a larger Swiss national study (“COM-COVID”) that examined citizens’ retrospective perceptions around communication during the COVID-19 pandemic in February 2022. The survey was conducted by a professional survey company in Bern, which randomly selected participants from a national web panel of 50,000 Swiss residents who had been recruited through various channels (e.g. phone, social media, print- and online ads). Respondents were qualified for participation if they were at least 18 years old and had resided in Switzerland during the entire COVID-19 pandemic. The recruited respondents completed a 30-minute online questionnaire and received reward points equivalent to a five Swiss Francs value for their participation. Rigorous quality control processes were implemented to ensure individuals could only join once and remained attentive throughout the survey (e.g., asking participants to confirm that they would answer honestly and to the best of their ability prior to commencing the survey, replacing the data of those who completed the survey too fast or in a pattern with data from new participants, and excluding multiple participations by comparing the IP address of the respondents).
The English survey items were professionally translated into French and Italian by freelance interpreters. The second author then compared the backtranslated English versions to resolve any discrepancies. The study was approved by the relevant university’s Ethics committee. Participants were informed that they could withdraw from the study at any time and provided informed consent prior to starting the survey. The sampling procedure is depicted in Figure 1.
Figure 1
Recruitment Process for the COM-COVID Survey
Participants
A random sample from the Swiss national web panel, consisting of 50,000 Swiss residents, was invited via email to participate in the study. The members of the panel had been recruited via several channels (e.g. phone, social media, print- and online ads) to reach also “non-heavy” online users. The sample composition reflected the Swiss population in terms of age, gender and region.
The final French sample comprised 160 residents of the French-speaking part of Switzerland (43.8% males). Their average age was M = 43.26 years (SD = 13.73, range 18–69 years). The Italian sample encompassed 114 residents of the Italian-speaking region of Switzerland (49.1% males), who averaged 42.32 years of age (SD = 13.61, range 18–69). In addition, German participants likewise completed the survey; however, the validation of the scale using that data is reported elsewhere (Stojanov & Hannawa, 2023b).
Instruments
Conspiracy Mentality Scale (CMS)
Participants rated seven statements measuring conspiracy theory ideation, and four items measuring skepticism on a 4-point scale (1 = untrue; 4 = true). In the French sample, Cronbach alpha for the CMS overall was 0.94, and for each subfactor 0.93 (ideation) and 0.89 (skepticism). Omega hierarchical was ωh = 0.78, 0.70 and 0.69 for the scale and each subscale, respectively. In the Italian version, the overall alpha equated 0.95, with 0.93 for ideation and 0.88 for skepticism. Omega hierarchical was ωh = 0.77, 0.72 and 0.67, respectively. The English, French and Italian items are shown in Table 1. The decision to switch from a seven-point scale to a four-point scale was influenced by a number of factors. These included an intended IRT analysis, the need to avoid respondent fatigue, and the constraints of the research budget.
Table 1
Items of the Conspiracy Mentality Scale in English, French and Italian and Standardized Estimates From CFA for the French and Italian Sample
| Item in English | Item in French | Item in Italian | French | Italian |
|---|---|---|---|---|
| Ideation | ||||
| 1. The government or covert organizations are responsible for events that are unusual or unexplained | 1. Le gouvernement ou des organisations secrètes sont responsables d'événements inhabituels ou inexpliqués. | 1. Il governo o le organizzazioni segrete sono responsabili di eventi insoliti o inspiegabili. | 0.80 | 0.84 |
| 2. Many so called “coincidences” are in fact clues as to how things really happened. | 2. Beaucoup de soi-disant «coïncidences» sont en fait des indices sur la façon dont les choses se sont réellement passées. | 2. Molte cosiddette "coincidenze" sono in realtà degli indizi su come le cose siano realmente accadute. | 0.86 | 0.82 |
| 3. Some things that everyone accepts as true are in fact hoaxes created by people in power. | 3. Certaines choses que tout le monde accepte comme "vraies" sont en fait des mensonges inventés par les personnes au pouvoir qui essaient de duper les gens. | 3. Alcune cose che tutti accettano come vere sono in realtà bufale create da persone al potere. | 0.84 | 0.84 |
| 4. Many situations or events can be explained by illegal or harmful acts by the government or other powerful people. | 4. De nombreuses situations ou événements peuvent s'expliquer par des actes illégaux ou dommageables provenant du gouvernement ou d'autres personnalités puissantes. | 4. Molte situazioni o eventi possono essere spiegati con atti illegali o dannosi del governo o di altre persone potenti. | 0.79 | 0.81 |
| 5. The alternative explanations for important societal events are closer to the truth than the official story. | 5. Les explications alternatives d’événements sociétaux importants sont plus proches de la vérité que l'histoire officielle. | 5. Le spiegazioni alternative per importanti eventi sociali sono più vicine alla verità rispetto alla storia ufficiale. | 0.83 | 0.86 |
| 6. Events throughout history are carefully planned and orchestrated by individuals for their own betterment. | 6. Les événements à travers l'histoire sont soigneusement planifiés et orchestrés par des individus dans leur propre intérêt. | 6. Gli eventi nel corso della storia sono attentamente pianificati e orchestrati da individui per il proprio interesse. | 0.80 | 0.84 |
| 7. Events on the news may not have actually happened | 7. Les événements de l’actualité ne se sont peut-être pas réellement produits. | 7. Gli eventi nei notiziari potrebbero non essere realmente accaduti. | 0.72 | 0.68 |
| Skepticism | ||||
| 1. Some things are not as they seem. | 1. Certaines choses ne sont pas ce qu'elles semblent être. | 1. Alcune cose non sono come sembrano. | 0.81 | 0.77 |
| 2. There are people who don’t want the truth to come out. | 2. Il y a des gens qui ne veulent pas que la vérité éclate. | 2. Ci sono persone che non vogliono che la verità venga fuori. | 0.82 | 0.83 |
| 3. People will do crazy things to cover up the truth. | 3. Les gens feront des choses folles pour dissimuler la vérité. | 3. La gente farà cose assurde per nascondere la verità. | 0.83 | 0.83 |
| 4. Many things happen without the public’s knowledge. | 4. Beaucoup de choses se passent à l'insu du grand public. | 4. Molte cose accadono all'insaputa del pubblico. | 0.83 | 0.76 |
Social Validation
Social validation was measured with three items taken from the COM-COVID survey’s pandemic coping scale. The items assessed to what extent COVID-19 communications from the government or the news media made participants feel or act in certain ways (e.g. their messages made me want to “seek support from others to feel better emotionally”; “seek connection with other people who share my views and beliefs, to gain a sense of belonging and companionship;” “be around people who would validate my positions and beliefs regarding the situation”). Respondents were asked to indicate the degree to which each item was true of them on a 4-point scale (1 = untrue, 4 = true). Cronbach alpha was 0.74 for the French sample, and 0.73 for the Italian sample.
Functional Literacy
Five items adapted from Ishikawa et al.'s (2008) functional literacy scale asked participants to rate the frequency (1 = never, 4 = often) of their experience with the government’s or news media’s COVID-19 communications. Example items included “I found their message contents difficult to follow” or “There were words that I did not know.” Cronbach alpha amounted to 0.83 for the French sample and 0.89 for the Italian sample.
Perceived Severity
Participants were asked to rate their agreement/disagreement with six items (e.g., “I felt at risk of getting infected with COVID-19” or “I believed that the Coronavirus was a severe public health problem”) that measured their perceived severity of the pandemic on a 4-point Likert scale (1 = strongly disagree, 4 = strongly agree). Cronbach alpha was 0.87 (French sample) and 0.89 (Italian sample).
Compliance With the Pandemic Control Measures
Compliance with the pandemic measures was measured by four items:
Compliance With Hygiene Measures
Participants indicated the extent to which they adhered to the prescribed hygiene measures (1= never, 2 = rarely/sometimes, 3 = most of the time, 4 = always as prescribed).
Compliance With Social Distancing
Participants indicated the extent to which they adhered to the prescribed social distancing behaviours (1 = never, 2 = rarely/sometimes, 3 = most of the time, 4 = always as prescribed, whenever possible).
Compliance With Mask-Wearing
Participants indicated the extent to which they adhered to wearing a surgical or FFP2 face mask (1 = never for medical reasons, 2 = never because I didn’t want to/was not convinced, 3 = occasionally, 4 = always as prescribed).
Compliance With Vaccination
Participants indicated the extent to which they adhered to vaccination recommendations (1 = I wanted to get vaccinated but I couldn’t for medical reasons, 2 = I did not want to get vaccinated, 3 = I got one shot of an admitted COVID-19 vaccine; 4 = I got two shots of an admitted COVID-19 vaccine; 5 = I got at least one booster shot in addition to my initial shots of an admitted COVID-19 vaccine).
Results
Factor Structure
We tested the a priori two-factor solution of the original CMS and, to exclude the possibility that the scale is unidimensional in the Italian and French versions, an alternative one-factor structure in R using the lavaan package (Rosseel, 2012) and robust maximum likelihood estimator (MLR). We looked at indices such as CFI, TLI, RMSEA, SRMR and also considered BIC. CFI and TLI compare the postulated model to a null model (i.e. model where all variables are unrelated), while RMSEA and SRMR estimate lack of fit compared with a perfect model. For the French scale, the two-factor solution had a superior fit to the data over the one-factor solution (see Table 2). The same was true for the Italian scale. Further, as seen in Figures 2 and 3, for both the French and Italian versions, the items loaded on the respective factors as predicted. As the data from both samples replicated the hypothesized structure of the CMS, we retained the original two-factor solution. The zero-order correlation between the scale scores representing the factors was high (r = 0.79 and r = 0.80 in the French and Italian sample, respectively).
Table 2
Model Fit Statistics for the Original Two Factor Structure and One Factor Structure
| Scale | χ2(df) | χ2/df | CFI | TLI | RMSEA | SRMR | AIC | BIC |
|---|---|---|---|---|---|---|---|---|
| French scale | ||||||||
| original two factor solution | 67.004(43) | 1.55 | 0.974 | 0.967 | 0.070 | 0.043 | 3569.843 | 3640.572 |
| one factor solution | 104.731(44) | 2.38 | 0.933 | 0.916 | 0.111 | 0.053 | 3623.930 | 3691.584 |
| Italian scale | ||||||||
| original two factor solution | 54.329(43) | 1.26 | 0.983 | 0.978 | 0.057 | 0.040 | 2621.612 | 2684.544 |
| one factor solution | 74.010(44) | 1.68 | 0.953 | 0.941 | 0.093 | 0.049 | 2649.996 | 2710.193 |
Figure 2
Path Diagram of the Two-Factor Structural Equation Model With Standardized Coefficients for the French Sample
Note. All estimates are significant.
Figure 3
Path Diagram of the Two-Factor Structural Equation Model With Standardized Coefficients for the Italian Sample
Note. All estimates are significant.
Construct Validity
For testing construct validity, we adopted the same approach as in the original validation study (Stojanov & Halberstadt, 2019) and simultaneously entered conspiracy theory ideation and skepticism as predictors with each of the variables of interest as outcome variables.
Tables 3 and 4 provide the construct validity results for the French and Italian version of the CMS and summarize the expected and obtained relationships. Multicollinearity diagnostics evidenced that multicollinearity was not an issue (VIF = 2.62, tolerance = 0.38 for the French scale; VIF = 2.83, tolerance = 0.35 for the Italian scale).
Table 3
Unstandardized Coefficients, Standard Errors, Standardized Coefficients, and p-Values From the Regression Model With Conspiracy Theory Ideation and Skepticism as Predictors and the Variables in the First Column as Outcome Variables (French Scale)
| Variable | Conspiracy Theory Ideation
|
Expected | Skepticism
|
Expected | F | R2 | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| B | SE | β | t | p | B | SE | β | t | p | |||||
| Functional Literacy | -0.27 | 0.10 | -0.33 | -2.64 | .009 | — | 0.08 | 0.08 | 0.12 | 0.97 | .33 | + | 5.08 | 0.06 |
| Social Validation | 0.28 | 0.12 | 0.29 | 2.31 | .02 | + | -0.02 | 0.10 | -0.03 | -0.23 | .82 | — | 6.01 | 0.07 |
| Severity | 0.002 | 0.13 | 0.002 | 0.016 | .98 | — | -0.05 | 0.11 | -0.06 | -0.45 | .65 | None | 0.25 | 0.003 |
| Compliance social distancing | -0.08 | 0.13 | -0.08 | -0.60 | .55 | — | -0.06 | 0.11 | -0.08 | -0.60 | .55 | None | 1.68 | 0.02 |
| Compliance hygiene | -0.14 | 0.11 | -0.17 | -1.31 | .19 | — | 0.04 | 0.09 | 0.06 | 0.44 | .66 | None | 1.31 | 0.02 |
| Compliance mask | -0.24 | 0.11 | -0.26 | -0.21 | .04 | — | -0.08 | 0.10 | -0.10 | -0.78 | .44 | None | 10.18 | 0.12 |
| Compliance vaccine | -0.40 | 0.16 | -0.30 | -2.52 | .01 | — | -0.16 | 0.13 | -0.14 | -1.19 | .23 | None | 16.54 | 0.17 |
Note. Significant coefficients are marked in bold.
Table 4
Unstandardized Coefficients, Standard Errors, Standardized Coefficients, and p-Values From the Regression Model With Conspiracy Theory Ideation and Skepticism as Predictors and the Variables in the First Column as Outcome Variables (Italian Scale)
| Variable | Conspiracy theory ideation
|
Expected | Skepticism
|
Expected | F | R2 | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| B | SE | β | t | p | B | SE | β | t | p | |||||
| Functional Literacy | -0.25 | 0.12 | -0.30 | -1.99 | .05 | — | -0.03 | 0.12 | -0.04 | -0.25 | .80 | + | 6.91 | 0.11 |
| Social Validation | 0.46 | 0.14 | 0.49 | 3.25 | .002 | + | -0.23 | 0.13 | -0.27 | -1.78 | .08 | — | 6.25 | 0.10 |
| Severity | -0.20 | 0.l5 | -0.21 | -1.30 | .2 | — | 0.10 | 0.14 | 0.11 | 0.69 | .49 | None | 1.02 | 0.02 |
| Compliance social distancing | -0.50 | 0.13 | -0.55 | -3.70 | < .001 | — | 0.29 | 0.13 | 0.35 | 2.31 | .023 | None | 7.45 | 0.12 |
| Compliance hygiene | -0.51 | 0.12 | -0.61 | -4.14 | < .001 | — | 0.36 | 0.12 | 0.46 | 3.13 | .002 | None | 8.63 | 0.13 |
| Compliance mask | -0.23 | 0.11 | -0.32 | -2.08 | .04 | — | 0.12 | 0.10 | 0.18 | 1.13 | .26 | None | 2.56 | 0.04 |
| Compliance vaccine | -0.43 | 0.19 | -0.31 | -2.20 | .03 | — | -0.27 | 0.18 | -0.21 | -1.47 | .14 | None | 17.65 | 0.25 |
Note. Significant coefficients are marked in bold.
We had hypothesized a negative relationship between conspiracy ideation and social validation, and negative correlations of conspiracy ideation with compliance, perceived severity and functional literacy. As seen in Tables 3 and 4, the signs of the relationships were as predicted. However, they did not always reach significance. This was particularly true for the French sample. In this group, conspiracy theory ideation was positively related to seeking social validation and negatively related to compliance with mask-wearing, vaccination and functional literacy as predicted, but it was not significantly related to compliance with social distancing and hygienic measures and perceived severity. In the Italian group, conspiracy ideation was positively related to social validation and negatively related to all four compliance measures and functional literacy. Perceived severity of the pandemic was unrelated to conspiracy theory ideation in this group as well.
For skepticism, we had predicted a positive relationship with functional literacy, a negative association with social validation, and no relationship between with compliance or perceived severity. Our no-correlation predictions with compliance and perceived severity were confirmed, but our predictions about the relationships with functional literacy (+) and social validation (-) were not, as skepticism was not significantly associated with these variables.
To follow up the non-significant multiple regression results, we conducted equivalence analysis (Lakens, 2017), in which the lower and upper bounds of a negligible effect are specified. An effect is considered too small and not worth examining if it falls within the range of the lower and upper bounds of the negligible effect. Based on Campbell (2020), we considered standardized regression coefficients of +/-0.1 a small effect size, and values falling within the +0.1 to –0.1 range to be a negligible effect. We used the TOST (two one-sided test) in the reg.equiv function in R. The results indicated that in all cases, there was insufficient evidence for negligible effects, meaning that the true population effect could be larger/smaller than 0.1/-0.1.
Measurement Invariance
As a final step in the validation process, we examined measurement invariance of the CMS across the two language groups (see Table 5). For this purpose, we used the lavaan package in R, with robust estimator (“MLR”) where we constrained the regression weights to be equal across groups and compared the fit of that model to the configural model (no parameters constrained). Next, we constrained the intercepts to be equal across the French and Italian groups and compared the model fit indexes to that of the previous model (only regression weights constrained equal). To test for strict invariance, we next restricted the error terms to be equivalent across both groups and compared the model fit to the fit of the model where the intercepts were constrained. As a final step, we also looked at structural invariance, by constraining the factor variances, covariances and means to be equal.
Table 5
Measurement Invariance Results Across the French and Italian Version
| Model | χ2(df) | CFI | TLI | RMSEA | SRMR | BIC | AIC | Comparison | Δχ2(df) | ΔCFI | ΔTLI | ΔRMSEA | ΔSRMR | Decision |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| M1: Configural | 122.64 (86) | 0.977 | 0.971 | 0.066 | 0.039 | 6481.147 | 6235.45 | / | — | — | — | — | — | — |
| M2: Metric invariance | 131.627 (95) | 0.978 | 0.974 | 0.061 | 0.047 | 6435.428 | 6223.254 | M1 | 8.98(9) | 0.001 | 0.003 | 0.005 | 0.008 | Equivalence |
| M3: Scalar invariance | 145.628 (106) | 0.975 | 0.974 | 0.062 | 0.050 | 6400.027 | 6219.371 | M2 | 14.001(11) | 0.003 | 0.000 | 0.001 | 0.003 | Equivalence |
| M4: residual invariance | 146.608 (115) | 0.981 | 0.982 | 0.052 | 0.050 | 6344.395 | 6203.483 | 0.98(9) | 0.006 | 0.008 | 0.01 | 0 | Equivalence | |
| M5: structural invariance | 148.304(117) | 0.981 | 0.982 | 0.051 | 0.053 | 6334.584 | 6200.899 | 1.696(2) | 0 | 0 | 0.001 | 0.003 | Equivalence |
A significant Δχ2 indicates nonequivalence. Given that Δχ2 is dependent on sample size and almost always significant in large samples, change in other fit indexes should also be considered (Byrne, 2009). ΔCFI greater than 0.01 (Cheung & Rensvold, 2002), ΔRMSEA values greater than 0.015 and ΔSRMR values greater than 0.03 (Chen, 2007) are considered to indicate measurement nonequivalence. We also examined AIC and BIC; lower values compared to the less restrictive model indicate measurement invariance. As seen in Table 4, Δχ2 was not significant, indicating measurement equivalence. Likewise, ΔCFI and ΔTLI values were lower than 0.01, and ΔRMSEA values lower than 0.015, also providing evidence for equivalent measures. Likewise, BIC and AIC continued to drop as we increasingly restricted the model parameters, suggesting invariance.
As a supplementary analysis we report measurement invariance across the German (the validation associated with this sample is reported elsewhere), French and Italian sample. As can be seen in the Supplementary Materials, measurement invariance was supported across the three languages.
Discussion
The popularity and fast-paced spread of conspiracy theories in crisis situations such as COVID-19 necessitates a validated measure that captures people’s general tendency to believe in conspiracy theories, separating that tendency from a more mundane form of suspicion (e.g., corruption beliefs or plausible conspiracies). In this paper, we presented a scientific validation of new French and Italian language versions of the Conspiracy Mentality Scale (CMS). Confirmatory factor analysis supported the two-factor solution of the original scale, evidencing that ideation (capturing conspiracy beliefs) and skepticism (capturing more mundane forms of suspicion) constitute two subdimensions of conspiracy mentality also in the French and Italian versions of the scale. Furthermore, measurement invariance tests showed that the CMS operates equivalently across the two language versions. Construct validity results were descriptively in line with our predictions, with the exception of a few of the relationships that did not reach statistical significance. In particular, perceived severity of the pandemic was not significantly related to conspiracy ideation in either sample, and adherence to social distancing and hygienic measures were only related to conspiracy ideation in the Italian, but not in the French sample.
The lack of relationship between conspiracy theory ideation and perceived severity of the pandemic, once the variance of skepticism was taken into account, was surprising. One possible reason for this finding may be that the survey was conducted after the pandemic was officially “over” (all pandemic measures were lifted in Switzerland in February 2022). This contextual circumstance may have primed participants to rate the pandemic as less severe overall, thus attenuating any relationship with conspiracy ideation. Another reason may be the rather narrow response range (1–4), which may have left less space for nuanced answering, as some area of the opinion spectrum (i.e. the midpoint) was not an available option. It is also possible that the lack of relationship is genuine, and conspiracy ideation, as measured with this scale, does not correlate with perceived severity of the pandemic. However, given that we observed such relationship in a different validation study (Stojanov & Hannawa, 2023b) using a larger sample (over 400) of German-speaking Swiss residents, we are inclined to ascribe this lack of relationship to the low power of the study (i.e., smaller sample size) rather than to a problematic validity of the scale. Therefore, we see the null findings as an indication of the need for additional research, rather than as evidence against the validity of the scale.
It is also noteworthy that adherence to social distancing and hygienic measures were not related to conspiracy ideation in the French sample, controlling for skepticism. To further explicate this finding, we also checked the correlations of each subdimension with adherence to distancing and hygienic measures, without controlling for the other dimension, but the results followed the same pattern (i.e., no association). This result may have been due to the ceiling effect in terms of compliance with social distancing and hygiene, as over 91% of the participants indicated they often or always adhered to hygiene measures, and 86% did the same for social distancing. Indeed, the correlation between these two variables (distancing and hygiene) was high (r = 0.58), but the correlation between them and the other two preventive measures was small to moderate (0.07 < r < 0.32), suggesting that participants may have had nuanced and measure-specific approaches to each recommendation, and thus conspiracy mentality may have exerted influence only in those cases where the measure was deemed coercive or infringing upon one’s liberties. Of course, the non-significant findings might also reflect a noise in the data due to sampling error. As previously mentioned, the number of non-compliant individuals was rather low, making questionable the degree to which this sub-sample was representative of the other noncompliers.
Although the direction of the relationship between skepticism and social validation (controlling for ideation) was in the direction we predicted, it did not reach significance. Based on previous findings, we expected that a higher need for validation would mean that one would appraise a situation less suspiciously and critically, complying with the norm. However, the present findings seem to suggest that a more mundane form of suspiciousness, as measured by the skepticism dimension of the CMS, may not be a function of social validation. Future studies could look into this relationship in a more systematic manner.
Similarly, we did not obtain evidence for a relationship between skepticism and functional literacy, but our equivalence analyses suggested that there is insufficient evidence to conclude the effect is negligible. Thus, future studies with a larger sample size and more statistical power is needed to further investigate this association, as we believe that the hypothesized relationship may be genuine, but our study simply failed to detect it.
While supporting our hypothesis, the negative correlation between conspiracy ideation and functional literacy was especially concerning. As our study demonstrated, conspiracy ideation was also related with a number of negative outcomes, such as non-adherence to vaccine recommendations and mask wearing, implying that those high in conspiracy mentality may benefit most from communication relevant to pandemic measures. Yet, at the same time, those high in conspiracy ideation are less likely to comprehend those messages. Future studies could examine strategic ways in which pandemic control measures could be communicated so that message comprehension is particularly enhanced for low-literacy conspiracy believers. Researchers could examine, for example, if acknowledging the emotional state of the recipient, using concrete language and examples, or being particularly transparent about the cost-benefit analysis of a required action could moderate the link between functional literacy and conspiracy ideation.
Measurement invariance demonstrated that the scale operates equivalently among the three linguistic groups in Switzerland. This finding sheds some light on why it is difficult to demonstrate full measurement equivalence (e.g., Cicero, 2016; Cicero et al., 2019; Torsheim et al., 2012)—it may be the cultural context in terms of shared political and civic reality rather than linguistic differences that are at the core of non-equivalent instruments. By keeping the political and civic context relatively constant (i.e., all participants came from Switzerland and thus shared more or less similar socio-political realities), the effects of the linguistic factors on measurement invariance could be better examined. By demonstrating measurement equivalence, therefore, we in effect demonstrated that the instrument behaves the same for participants with the same cultural background in terms of a shared political and civic context, regardless of what language they speak. Therefore, it is still possible that the instrument may not behave equivalently across, for example, French speaking participants coming from Canada and France (i.e., different political and civic context), and future studies could examine this. Demonstrating equivalence is important, especially when comparing the results between groups, as without it the results may be biased or misleading.
Although the two-factor structure was confirmed, the label of the second factor may be open to interpretations. As one of the reviewers pointed out, the items of first factor refer to a collective entity, whereas the items loading on the second factor lack this element and are about “people covering things up or things happening without the public’s knowledge”, leading one to wonder what is the evidence that it is rational to believe the latter. Skepticism, reflects a measured and cautious form of doubt that prompts one to question information, especially when there may be motives for deception. This form of doubt is sometimes referred to as 'rational' because it encourages critical thinking and evidence-based conclusions rather than accepting information without scrutiny. For example, corruption cases are typically grounded in documented proof and involve legal proceedings. Therefore, the skepticism factor, that predicts belief in corruption cases (Stojanov & Halberstadt, 2019) is capturing this critical thinking approach to information. On the other hand, belief in specific conspiracy theories, which are predicted by the 'ideation' factor and which, due to the complex plot narrative, are inherently harder to prove or disprove, often results from speculation, mistrust, and pattern-seeking in the absence of evidence. Hence, 'conspiracy ideation' represents a form of thinking that might not rely as heavily on empirical evidence or critical scrutiny, and the use of the adjective ‘rational’ in relation to the second factor serves the purpose to delineate more clearly one factor from the other.
To conclude, the addition of French and Italian versions of the CMS to the literature now enables researchers from countries with French- and Italian-speaking populations to better understand conspiratorial activities and processes, particularly in the context of crisis situations that require population compliance and coherence (e.g. pandemics, wars, terrorist attacks). Such studies can now examine the factors associated with the spread and believability of conspiracy theories across an even larger number of geographical regions. In particular, they could examine how those with low vs. high conspiracy mentality react to different framings of a message, different mediums, or even different communicators. Such endeavours could yield important insights about ways in which communication could be tailored to the recipients so that it is most effective (i.e. achieves the desired outcome). Similarly, researchers could devise ‘profiles’ of individuals in terms of their conspiracy mentality and reaction to the COVID-19 related communication, which might inform efforts for the best way to communicate effectively with the larger population.
There are several limitations to this study. First, while the study was representative of the French- and Italian-speaking Swiss population, the number of participants was rather small. Furthermore, the study was cross-sectional, meaning estimation of test-retest reliability was not possible (however, the test-retest reliability reported in the original study was good). Further, the use of self-reported measures may have resulted in common-method variance; however, the Harman’s single factor test indicated the only 28% (French group) and 29% (Italian group) of the variance can be captured in one factor, suggesting that this was not an issue. Moreover, given that conspiracy beliefs are considered stigmatized beliefs (Lantian et al., 2018; Nera et al., 2022), the possibility that participants responded in a socially desirable way cannot be excluded. Further as a reviewer noted, the item pool of the current scale is somewhat limited and may not capture all facets of conspiracist ideation. Thus, future scale construction may aim to devise a more fine-grained model of conspiracy mentality. Next, we used parametric statistics, although some may argue that the variables are measured at ordinal level; however, using robust regression in R replicated the pattern of results obtained with multiple regression, with the exception of the relationship between skepticism and social validation in the Italian group, where the 95% CI for the estimate did not cross zero. Thus, overall, the results suggest that the two CMS subscales are a valid psychometric tool for measuring the respective constructs, and operate equivalently across the two language groups. Finally, the validity could have been more fully demonstrated if other variables were measured such as belief in fictitious conspiracy theories (Swami et al., 2011) or contradictory conspiracy theories (Lukić et al., 2019; Wood et al., 2012). Future studies could incorporate these variables in the design.
In summary, this study provided scientific support that the CMS consists of two dimensions: (1) Conspiracy theory ideation, which taps into conspiracy beliefs and can be used as measure of the generic tendency to believe conspiracy theories; and a more mundane form of (2) skepticism, which taps into more mundane suspiciousness. With the use of the three language versions of the scales, future studies may now examine how these two dimensions interact to predict specific beliefs in population from different countries and cultures, and whether or under what conditions skepticism may morph into conspiracist ideation.