Measuring Tolerance for Ambiguity: A German-Language Adaption and Validation of the Tolerance for Ambiguity Scale (TAS)

Conceptual and Definitional Issues

Common Instruments for Measuring Tolerance for Ambiguity

The first attempts for measuring TA date back to Frenkel-Brunswik (1949, 1951), and were conducted as part of the studies on the authoritarian personality (Adorno et al., 1950). Experimental methods and many survey instruments using Likert items for structured questionnaires were developed. Common instruments are briefly presented here (for a review see Furnham & Marks, 2013).

For a long time, the most common and widely used instrument for measuring TA was Budner’s (1962) survey instrument with 16 items. This instrument differentiates according to the potential mode of reaction (submission, denial), the levels of reaction (phenomenological, operative), and the type of ambiguous situation (novelty, complexity, and insolubility). Budner (1962) assumed TA to be a stable one-dimensional personality variable. The instrument had a low internal consistency of α = .49, a good test-retest reliability of r = .85, and a good construct validity. The fact that this instrument was widely used despite lower than acceptable psychometric properties was criticized (e.g., Furnham & Marks, 2013; Herman et al., 2010).

McLain (1993) designed the 22-item Multiple Stimulus Types Ambiguity Tolerance measure (MSTAT-I), which highlighted several stimuli of TA, described as facets of a one-dimensional TA construct. Based on the MSTAT-I, McLain (2009) developed the MSTAT-II by removing items based on feedback from researchers and respondents. The MSTAT-II is composed of 13 items and has an internal consistency of α = .83. However, it is preferred over the MSTAT-I only when survey space is limited (Furnham & Marks, 2013). This instrument also suffers in part from an overgeneralized view or overly abstract formulations (McLain, 2009, p. 986).

Regarding German-language measurement instruments, the Inventory for the Measurement of Tolerance for Ambiguity (IMA) by Reis (1996, 1997) is test-statistically very sound. It distinguishes between five different facets that have been largely confirmed by factor analysis: TA toward problems that seem insoluble, toward social conflicts, toward role stereotypes, toward parental image, and openness to new experiences. However, the instrument is composed of 40 items.

The measurement instrument of Herman et al. (2010) aims to help researchers to better understand intercultural phenomena and to improve the conceptual dimensionality and psychometric evidence of previous measurement instruments. The 12-item Tolerance for Ambiguity Scale (TAS) was designed primarily for cross-cultural contexts, arguing that items that are too general are not appropriate for the various concepts of TA or advocating for context-specific instruments. Herman et al. (2010) drew on the conceptualization and the instrument of Budner (1962) and attempted to improve the instrument by adding new items and removing other items (see the final instrument in Table A1). For evaluation, Herman et al. (2010) computed principal component analyses (PCA), internal consistency, and correlations among items. Accordingly, the scale represents an improvement on Budner’s (1962) scale in terms of factor structure and internal consistency, α(_{Herman et al., 2010}) = .73 vs. α(_{Budner, 1962}) = .49. Moreover, the authors emphasise that the instrument is designed in such a way that it is consistent with the ideas of McLain (1993, 2009). It is composed of four sub-dimensions: valuing diverse others, change, challenging perspectives, and unfamiliarity. However, regarding low internal consistencies of the individual factors, they support a one-dimensional theoretical framework. The dimensional structure emerged through an exploratory approach combined with theoretical considerations, especially regarding the newly created items. These new items were written in terms of cross-cultural relevance and to complement Budner's (1962) existing items. Herman et al. (2010) do not mention this specifically, but it stands out that the sub-dimension ‘valuing diverse others’ fits very well with considerations of Frenkel-Brunswik (1949, 1951), who defined TA as ‘tolerance of the diversity of people’. ‘Valuing diverse others’ is not included in other recent concepts (Furnham & Marks, 2013).

Translation Procedure

For this study, the 12 items of the Tolerance for Ambiguity Scale (TAS) by Herman et al. (2010) were translated from English into German using the TRAPD-approach (Translation, Review, Adjudication, Pretesting, Documentation; Harkness, 2003; Harkness et al., 2004). First, the English-language instrument was translated into German by two independent translators. Subsequently, the translations were discussed under the moderation of a ’reviewer’ and solutions were developed for each item. Next, open questions were clarified with an experienced survey expert (an ‘adjudicator’). Then, the translated items have been tested on a sample of the target population. Based on pretest results, the items were partially modified and tested for comprehensibility using cognitive interviews with probing procedures (comprehensive, general). In a last step, the entire translation process was documented in detail.⁶ In the final translation, care was also taken to ensure that the items were formulated in simple and easy-to-understand language.

Material

Responses can be given on a fully verbalized five-point response scale: 1 – does not apply at all, 2 – rather does not apply, 3 – neither, 4 – rather applies, and 5 – fully applies. The final translation of the items and the response scale can be found in Table A1 in the Appendix.

Data

The development of the survey instrument is embedded in the project ‘Social Cohesion in Times of Crisis: The Corona Pandemic and Anti-Asian Racism in Germany’. This project included an online panel study in Germany, in which the participants of the first survey were re-surveyed in two further waves (Mayer et al., 2022). The data was collected from December 2020 to May 2021 via an online access panel by respondi, with adjusted quotas for age, gender, and federal state. The target group was people aged 18 to 74 living in Germany with an oversampling of respondents who were born abroad or whose parents were born abroad. The pretest (n = 2,002)⁷ and the three waves (n = 1,370 for each wave) were used to develop and validate the survey instrument. The sample description for the analysis of the three waves (after cleansing)⁸ can be found in Table A2.

Analytical Strategy

To ensure high instrument quality, the measurement instrument and its documentation were based on the quality standards developed in 2014 by the German Data Forum (RatSWD) in the working group ’Quality Standards’ (Rammstedt, Beierlein, et al., 2014). To assess the psychometric quality of the German-language TAS, validity and reliability are considered. To do this, the following three steps were taken:

(1) Factorial validity of the TAS was tested using confirmatory factor analysis, applying structural equation modelling (SEM) with maximum likelihood estimation, and using standardised coefficients and values.⁹ After a first screening of the proposed model of Herman et al. (2010) different model assumptions were tested.

All multidimensional models (M2–M4) were models with correlated latent variables, as the concept assumes correlations between the sub-dimensions. Model fit was evaluated using CFI, TLI, RMSEA, SRMR, AIC and BIC.¹⁰ The heuristic of Hu and Bentler (1999) is used, according to which a CFI and TLI of .95 (or higher), a RMSEA of .06 (or lower) and a SRMR of about .08 (or lower) imply good model fit. Lower values of information criteria such as AIC or BIC indicate a better model fit. In addition, LR-tests were performed. In a next step, model misspecifications were examined using modification indices for model M3. Therefore, the following iterative process was used: The error correlation with the highest modification index was included if they were meaningful in term of content. However, so as not to overload the model no more than four correlations are accepted. The iteration process is documented in Table A3 in the Appendix.

(2) To test the construct validity, Spearman’s correlations were calculated with constructs that are theoretically or empirically related to TA:

To test whether one-dimensional scales indeed make it difficult to correlate TA with theoretically relevant behaviours (Kenny & Ginsberg, 1958; Lauriola et al., 2016), correlations are computed for the proposed models for the full and reduced scale, as well as for the individual dimensions.

(3) To determine reliability of the measurement instrument, the test-retest stability was computed.¹¹ In addition, the internal reliability is tested by calculating the McDonalds’s omega (Baldwin, 2019; McDonald, 1999).

Descriptive Statistics

Table 1 shows the mean, standard deviation, skewness, and kurtosis for each of the 12 items of the TAS. Mean and standard deviation separately for gender, age groups, educational background, and migration background of the translated TAS—can be found in Table A4–A7 in the Appendix.

Factorial Validity

Figure 1 shows the proposed structure by Herman et al. (2010) with four correlated factors behind the 12 items: As for the factor loadings, the most noticeable is the low correlation between item_9 and the latent factor L3 (‘challenging perspectives’, .19), which may be an indicator that this item does not fit the factor.¹² As for the correlations between the latent variables, the strong correlation between the latent variables L2 (‘change’) and L4 (‘unfamiliarity’) of .85 is striking, which could indicate that these two dimensions are not properly separable. This consideration is also plausible from a theoretical perspective. Table A8 shows which theoretical considerations Budner (1962) had already about the items. According to Budner (1962), item_5, item_6 and item_12 are items which have both the same response reaction (phenomenological submission, PS) and the same underlying ambiguous situation (novelty). The consideration that these three items lie on a common factor therefore seems to make theoretical sense, too.

Click to enlarge

Figure 1

Path Diagram of the Structural Equation Model M2

Note. Structural equation modelling (SEM) with maximum likelihood estimation; standardised coefficients; latent variables: L1 = valuing diverse others; L2 = change; L3 = challenging perspectives; L4 = unfamiliarity; wording of the items can be found in Table A1; 1st wave; n = 1,370.

Based on these findings, as described in section Generality and Dimensionality of the Construct, different models are compared: The full model with 12 items and one dimension (M1), the full model just described with 12 items and four sub-dimensions (M2); a reduced model with 11 items and four sub-dimensions (M3), and a reduced merged model with 11 items and three sub-dimensions (M4). Fit statistics are summarized in Table 2. When a one-dimensional model (M1) was fitted to the data of the first wave, fit indices pointed to a poor model fit, CFI = .654, TLI = .578, RMSEA = .109, SRMR = .085, AIC = 45630.966, BIC = 45818.978. The model specification M2 with four sub-dimensions has a better model fit than the one-dimensional model, CFI = .859, TLI = .806, RMSEA = .074, SRMR = .067, AIC = 45119.287, BIC = 45338.634, and a LR test supports that M1 is nested within M2, χ²(6) = 523.68. Model M2 is acceptable, but still fails to be a ‘good’ model fit according to Hu and Bentler (1999). Model M3 is calculated without item_9, what leads to an improvement in some model fit indices, CFI = .878, TLI = .824, RMSEA = .075, SRMR = .060, AIC = 41927.586, BIC = 42131.266. However, a LR test does not confirm a significant improvement of the model compared to M2, χ²(3) = -3185.70. When the reduced models M3 and M4 with only 11 items are compared, the LR test confirms that M4 is nested in M3, χ²(3) = 19.59, which argues for the four-dimensional structure of the model. However, the results support a strong relationship between the latent variables L2 (‘change’) and L4 (‘unfamiliarity’). In summary, the results support the theoretically derived four-dimensional structure. Omitting item_9 does not lead to a significant improvement of model fit. However, this item can be omitted because it does not fit the intended factorial structure of Herman et al. (2010). Those types of items were already criticised by McLain (2009, p. 976) in the scale of Budner (1962), since items that refer to ‘classifications of people such as teachers or experts’ may be ‘confounded by reference to specific situations that may evoke responses other than reactions to ambiguity’.

Investigating whether error terms are potentially correlated, and to check whether statistically significant paths should be added, leads to a next step to the consideration of modification indices (MI) for the reduced four-dimensional model M3. After the error correlations were added, the model fit of model M5 improved, and according to Hu and Bentler (1999), the model has nearly a good model fit, CFI = .921, TLI = .872, RMSEA = .064, SRMR = .051, AIC = 41828.558, BIC = 42053.129. However, the high correlation between L2 and L4 is still conspicuous, and presumably these latent factors cannot be properly separated from each other (see Figure 2).¹³

Click to enlarge

Figure 2

Path Diagram of the Structural Equation Model M5

Note. Structural equation modelling (SEM) with maximum likelihood estimation; standardised coefficients; latent variables: L1 = valuing diverse others; L2 = change; L3 = challenging perspectives; L4 = unfamiliarity; wording of the items can be found in Table A1; 1st wave; n = 1,370.

It was shown that a multidimensional model fits significantly better than a one-dimensional one, with the four-dimensional structural solution fitting significantly better than the three-dimensional one. In conclusion, even if a four-dimensional solution seems statistically to be best, it was shown empirically and theoretically, that the dimensions ‘change’ and ‘unfamiliarity’ are very strongly related, and that item_9 does not fit the factor very well and can therefore be omitted. Regarding, factorial validity of the construct, it should be emphasized that none of the tested models represent the empirical covariances and variances of the items sufficiently well. The model fit proved to be rather low for all tested models, especially with respect to the CFI measurement.

Construct Validity

Spearman’s correlations of this study and correlations of the reference studies can be found in Table 3.¹⁴ Correlations are calculated for the full and reduced overall measure and for the individual sub-dimensions for the proposed models.

The Big Five personality traits were measured using a validated instrument by Rammstedt, Kemper, et al. (2014). According to the studies mentioned in section Related Psychological Concepts, the translated TAS correlates positively with ‘extraversion’ (r_{s, 12 items} = .24) and ‘openness to experience’ (r_{s, 12 items} = .22). But contrary to Caligiuri and Tarique (2012), respondents who scored high in ‘agreeableness’ had higher degrees of TA. ‘Conscientiousness’ was uncorrelated to the TAS, which is analogous to Caligiuri and Tarique (2012), and ‘neuroticism’ correlated negatively, which is analogous to Jach and Smillie (2019). The translated TAS correlates positively with ‘life satisfaction’, which is in line with the results of Bardi et al. (2009). In the online panel study, ‘need for cognitive closure’ was assessed using a validated short scale by Rinke (2020), the NCC-5. The translated TAS correlates negatively with the NCC-5 (r_{s, 12 items} = -.50), which is in line with empirical and theoretical assumptions, too. As mentioned before, within the work of Adorno et al. (1950) and Frenkel-Brunswik (1949, 1951) on ethnocentrism and authoritarianism, intolerance for ambiguity is the central component of the authoritarian personality and is positively related to ‘racism’ and ‘ethnic prejudice’. As a measure of authoritarianism, a shortened version of the KSE-3 was integrated into the questionnaire. The correlation between the KSE-3 and the German TAS is strong with r_{s, 12 items} = -.45, which is again in line with empirical considerations. ‘Modern racism’ was measured via a translation of the Modern Racism Scale of McConahay (1986). The correlation between the translated Modern Racism Scale and the German TAS is r_{s, 12 items} = -.35.

Regarding construct validity, all findings are consistent with theoretical and empirical considerations. However, the comparability of correlations between this study and the reference studies is limited: First, the results of the reference studies refer in part to other measurement instruments for TA, and second, the reference studies have much smaller, mostly specific samples (students, managers, etc.). Regarding the individual sub-dimensions, in fact, there are considerable differences in places.

Reliability

In terms of test-retest stability, the full scale (12 items) and the reduced scale (11 items) have acceptable test-retest reliabilities (> .7) (see Table A9). McDonald’s omega coefficients are calculated for the overall measure and for the individual dimensions for the proposed models (see Table 4). Internal reliability of the overall measure was acceptable at ω_{TAS, 12 items} = ω_{TAS, 11 items} = .71. Regarding the individual dimensions, the moderately sized factor loadings of the confirmatory factor analysis are also reflected in moderate reliabilities of the sub-dimensions.