Item Development and Psychometric Evaluation of the Scrambled Sentences Task: A Pilot Study in Persian
,
Abstract
Objective: One significant concern regarding the cross-cultural use of psychological instruments is their adaptation to the language of the target population. The Scrambled Sentences Task (SST) exemplifies this issue. As a well-known paradigm for assessing interpretation bias (IB), the SST has been widely used across different languages; however, it remains unclear whether the SST is a valid and reliable tool to be used in languages other than English. The current study aims to develop SST items for Persian language while controlling for word frequency, word length and morphological complexity— linguistic features critical to meet SST’s unique demands. We also seek to examine the psychometric properties of the Persian version of the SST (P-SST).
Method: The verbal stimuli for the P-SST were developed by drawing on a pool of sentences from prior research as a guide. These sentences were adapted specifically to fulfill the task's unique demands, ensuring the appropriateness of the P-SST for detecting IB. Since the SST primarily measures depressogenic tendencies, the Beck Depression Inventory-II (BDI-II) was also employed as part of the assessment. The measures were administered to a sample of 222 female students, selected due to evidence suggesting higher susceptibility to depression among women. The reliability and validity of the P-SST were then calculated, along with an analysis of responses to identify areas for enhancing performance on this task.
Results: Our results provided evidence of the convergent validity of the P-SST, as it was significantly correlated with the BDI-II (rs = 0.43, 95% CI [0.29–0.57], N = 161), as well as evidence of its divergent validity (rs = 0.35, 95% CI [0.22–0.49], N = 161). Moreover, internal consistency analysis revealed a Cronbach’s alpha value of α = 0.81 and a split-half correlation value of r = 0.9.
Conclusion: Findings from this research established the psychometric properties of the P-SST as a quick and easily administered assessment tool to be used in the context of depression in Persian. The paper concludes with both linguistic and methodological recommendations to guide future development of SST items in any language.
2. Hambleton RK, Zenisky AL. Translating and adapting tests for cross-cultural assessments. 2011.
3. Shou Y, Chen HF, Takemura K, Wu J, Yang CT, Wang MC. Editorial: From West to East: Recent Advances in Psychometrics and Psychological Instruments in Asia. Front Psychol. 2022;13:875536.
4. Gudmundsson E. Guidelines for translating and adapting psychological instruments. Nordic Psychology. 2009;61(2):29-45.
5. Hambleton RK, Merenda PF, Spielberger CD. Issues, designs, and technical guidelines for adapting tests into multiple languages and cultures. Adapting educational and psychological tests for cross-cultural assessment: Psychology Press; 2004. p. 15-50.
6. Gregoire J. ITC guidelines for translating and adapting tests. Int J Test. 2018;18(2):101-34.
7. Van de Vijver F, Hambleton RK. Translating tests. European psychologist. 1996;1(2):89-99.
8. Van de Vijver F, Leung K. Methods and data analysis of comparative research. Handbook of cross-cultural psychology. 1997;1:257-300.
9. Geisinger KF. Cross-cultural normative assessment: translation and adaptation issues influencing the normative interpretation of assessment instruments. Psychol Assess. 1994;6(4):304.
10. Colina S, Marrone N, Ingram M, Sánchez D. Translation Quality Assessment in Health Research: A Functionalist Alternative to Back-Translation. Eval Health Prof. 2017;40(3):267-93.
11. Peña ED. Lost in translation: methodological considerations in cross-cultural research. Child Dev. 2007;78(4):1255-64.
12. Holmlund TB, Foltz PW, Cohen AS, Johansen HD, Sigurdsen R, Fugelli P, et al. Moving psychological assessment out of the controlled laboratory setting: Practical challenges. Psychol Assess. 2019;31(3):292-303.
13. Cruchinho P, López-Franco MD, Capelas ML, Almeida S, Bennett PM, Miranda da Silva M, et al. Translation, Cross-Cultural Adaptation, and Validation of Measurement Instruments: A Practical Guideline for Novice Researchers. J Multidiscip Healthc. 2024;17:2701-28.
14. Krach SK, McCreery MP, Guerard J. Cultural-linguistic test adaptations: Guidelines for selection, alteration, use, and review. School psychology international. 2017;38(1):3-21.
15. Kalfoss M. Translation and Adaption of Questionnaires: A Nursing Challenge. SAGE Open Nurs. 2019;5:2377960818816810.
16. Wenzlaff RM, Bates DE. Unmasking a cognitive vulnerability to depression: how lapses in mental control reveal depressive thinking. J Pers Soc Psychol. 1998;75(6):1559-71.
17. Beck A. Depression. Clinical, Experimental and Theoretical Aspects. New York (Hoeber) 1967. 1967.
18. Everaert J, Struyf S, Koster EH. Biased interpretation of ambiguity in depression and anxiety: Interactions with attention, memory, and cognitive control processes. Interpretational Processing Biases in Emotional Psychopathology: From Experimental Investigation to Clinical Practice. 2023:79-96.
19. Würtz F, Zahler L, Blackwell SE, Margraf J, Bagheri M, Woud ML. Scrambled but valid? The scrambled sentences task as a measure of interpretation biases in psychopathology: A systematic review and meta-analysis. Clin Psychol Rev. 2022;93:102133.
20. Ren Z, Li X, Zhao L, Yu X, Li Z, Lai L, et al. Effectiveness and mechanism of internet-based self-help intervention for depression: The Chinese version of MoodGYM. Acta Psychologica Sinica. 2016.
21. Novović Z, Mihić L, Biro M, Tovilović S. Measuring vulnerability to depression: The Serbian scrambled sentences test-SSST. Psihologija. 2014; 47(1): 33-48. https://doi.org/10.2298/PSI1401033N
22. Everaert J, Grahek I, Koster EH. Individual differences in cognitive control over emotional material modulate cognitive biases linked to depressive symptoms. Cogn Emot. 2017;31(4):736-46.
23. Everaert J, Grahek I, Duyck W, Buelens J, Van den Bergh N, Koster EH. Mapping the interplay among cognitive biases, emotion regulation, and depressive symptoms. Cogn Emot. 2017;31(4):726-35.
24. Suto I, Williamson J, Ireland J, Macinska S. On reducing errors in assessment instruments. Res Pap Educ. 2023;38(3):357-77.
25. Torkan H, Blackwell SE, Holmes EA, Kalantari M, Neshat-Doost HT, Maroufi M, et al. Positive Imagery Cognitive Bias Modification in Treatment-Seeking Patients with Major Depression in Iran: A Pilot Study. Cognit Ther Res. 2014;38(2):132-45.
26. Dolatshahi B, Naderi Rajeh Y, Pourshahbaz A, Zarghami M. Uncovering Negative Interpretation Bias in Remitted/Recovered Depression with Laboratory Task. Iran J Psychiatry. 2023;18(2):165-72.
27. Albert PR. Why is depression more prevalent in women? J Psychiatry Neurosci. 2015;40(4):219-21.
28. Slavich GM, Sacher J. Stress, sex hormones, inflammation, and major depressive disorder: Extending Social Signal Transduction Theory of Depression to account for sex differences in mood disorders. Psychopharmacology (Berl). 2019;236(10):3063-79.
29. Rude SS, Wenzlaff RM, Gibbs B, Vane J, Whitney T. Negative processing biases predict subsequent depressive symptoms. Cogn Emot. 2002;16(3):423-40.
30. Rude SS, Valdez CR, Odom S, Ebrahimi A. Negative cognitive biases predict subsequent depression. Cognit Ther Res. 2003;27:415-29.
31. Hammen C, Kim EY, Eberhart NK, Brennan PA. Chronic and acute stress and the prediction of major depression in women. Depress Anxiety. 2009;26(8):718-23.
32. Joormann J, Talbot L, Gotlib IH. Biased processing of emotional information in girls at risk for depression. J Abnorm Psychol. 2007;116(1):135-43.
33. Rude S S, Durham-Fowler J A, Baum E S, Rooney S B, Maestas K L.
Self-report and cognitive processing measures of depressive thinking predict subsequent
major depressive disorder. Cognitive Therapy and Research. 2010; 34(2): 107–115.
https://doi.org/10.1007/s10608-009-9237-y
34. Seeds PM. Interpretive bias in the context of life stress and depression: An examination of stress generation and diathesis-stress models: The University of Western Ontario (Canada); 2012.
35. Sfärlea A, Löchner J, Neumüller J, Asperud Thomsen L, Starman K, Salemink E, et al. Passing on the half-empty glass: A transgenerational study of interpretation biases in children at risk for depression and their parents with depression. J Abnorm Psychol. 2019;128(2):151-61.
36. Viviani R, Dommes L, Bosch JE, Stingl JC, Beschoner P. A Computerized Version of the Scrambled Sentences Test. Front Psychol. 2017;8:2310.
37. Gagné CL. Psycholinguistic Approaches to Morphology: Theoretical Issues. Oxford research encyclopedia of linguistics2017.
38. Sandra D. Morphological units: A theoretical and psycholinguistic perspective. Oxford research encyclopedia of linguistics2020.
39. Taboada M, Trnavac R, Goddard C. On being negative. Corpus Pragmat. 2017;1:57-76.
40. Gao J, Suzuki H, editors. Long distance dependency in language modeling: an empirical study. International Conference on Natural Language Processing; 2004: Springer.
41. Garrett M. Levels of processing in sentence production. Language production Vol 1: Speech and talk: Academic Press; 1980. p. 177-220.
42. Steer RA, Clark DA, Beck AT, Ranieri WF. Common and specific dimensions of self-reported anxiety and depression: the BDI-II versus the BDI-IA. Behav Res Ther. 1999;37(2):183-90.
43. Ghassemzadeh H, Mojtabai R, Karamghadiri N, Ebrahimkhani N. Psychometric properties of a Persian-language version of the Beck Depression Inventory--Second edition: BDI-II-PERSIAN. Depress Anxiety. 2005;21(4):185-92.
44. Gilbert DT, Hixon JG. The trouble of thinking: Activation and application of stereotypic beliefs. J Pers Soc Psychol. 1991;60(4):509.
45. Mostafavi P. Predicative Clauses in Today Persian-A Typological Analysis. Asian Social Science. 2015;11(15):104.
| Files | ||
| Issue | Vol 20 No 3 (2025) | |
| Section | Original Article(s) | |
| DOI | https://doi.org/10.18502/ijps.v20i3.19043 | |
| Keywords | ||
| Adaptation Depression Interpretation Bias Linguistics Psychometrics Scrambled Sentences Task | ||
| Rights and permissions | |
|
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
