EVALUATION OF THE ACCURACY OF PERSONALITY JUDGMENTS BASED ON WRITTEN VERBAL PRODUCTION
DOI:
https://doi.org/10.19090/pp.2020.3.333-348Keywords:
accuracy, personality judgment, personality traits, written verbal productionAbstract
The process of personality judgment occurs in daily social interactions and represents an attempt to identify characteristics of someone else's personality, in the way to explain past and predict future behaviors. The results of this process have implications on future decisions and actions of people. This research aimed to examine the accuracy of non-expert ratings of Extraversion and Conscientiousness based on short written texts. The sample consisted of 215 participants (Mage = 28.58, SD = 10.30; 80.5% females). The exclusion criterion was that participants were psychologists or psychology students, i.e., individuals familiar with personality research and taxonomies. Participants rated Extraversion and Conscientiousness, based on the texts written by five different individuals. Criteria used to estimate the accuracy of judgments were the agreement between self-report measures on HEXACO PI-R from people who wrote the texts and ratings from participants, as well as the agreement between multiple raters. The results showed that there was a moderate self-other agreement for Extraversion and Conscientiousness. Also, the results showed that there was a high between-raters agreement for Extraversion and Conscientiousness. This study indicates that it is possible to judge one’s personality based on written verbal production, as well that raters tend to form similar impressions about the personality from written texts.
Metrics
Metrics Loading ...
References
Albright, L., Kenny, D. A., & Malloy, T. E. (1988). Consensus in personality judgments at zero acquaintance. Journal of Personality and Social Psychology, 55(3), 387–395. https://doi.org/10.1037/0022-3514.55.3.387
Back, M. D., & Nestler, S. (2016). Accuracy of judging personality. In J. A. Hall, M. Schmid Mast, & T. V. West (Eds.), The social psychology of perceiving others accurately (pp. 98–124). Cambridge University Press.
Beer, A. (2013). Group personality judgments at zero acquaintance: Communication among judges versus aggregation of independent evaluations. Journal of Research in Personality, 47(4), 385–389. https://doi.org/10.1016/j.jrp.2013.03.008
Beer, A., & Watson, D. (2008). Personality judgment at zero acquaintance: Agreement, assumed similarity, and implicit simplicity. Journal of Personality Assessment, 90(3), 250–260. https://doi.org/10.1080/00223890701884970
Beer, A., & Watson, D. (2010). The effects of information and exposure on self-other agreement. Journal of Research in Personality, 44(1), 38–45. https://doi.org/10.1016/j.jrp.2009.10.002
Bjekić, J. (2016). Stilističke i sadržinske karakteristike verbalne produkcije kao indikatori bazične ličnosti [The style and content of verbal production as indicators of basic personality structure]. Unpublished doctoral dissertation. Department of Psychology, Faculty of Philosophy, University of Belgrade.
Borkenau, P., Mosch, A., Tandler, N., & Wolf, A. (2016). Accuracy of judgments of personality based on textual information on major life domains. Journal of Personality, 84(2), 214–224. https://doi.org/10.1111/jopy.12153
Čerović, S. L. (2018). Primena analize verbalne produkcije u selekcionom intervjuu [Application of verbal production analysis in the selection interview]. Unpublished doctoral dissertation. Department of Psychology, Faculty of Philosophy, University of Belgrade.
Christiansen, N. D., Wolcott-Burnam, S., Janovics, J. E., Burns, G. N., & Quirk, S. W. (2005). The good judge revisited: Individual differences in the accuracy of personality judgments. Human Performance, 18(2), 123–149. https://doi.org/10.1207/s15327043hup1802_2
Connelly, B. S., & Ones, D. S. (2010). An other perspective on personality: Meta-analytic integration of observers' accuracy and predictive validity. Psychological Bulletin, 136(6), 1092–1122. https://doi.org/10.1037/a0021212
Connolly, J. J., Kavanagh, E. J., & Viswesvaran, C. (2007). The convergent validity between self and observer ratings of personality: A meta‐analytic review. International Journal of Selection and Assessment, 15(1), 110–117. https://doi.org/10.1111/j.1468-2389.2007.00371.x
Darbyshire, D., Kirk, C., Wall, H. J., & Kaye, L. K. (2016). Don't Judge a (Face) Book by its Cover: Exploring judgement accuracy of others' personality on Facebook. Computers in Human Behavior, 58, 380–387. https://doi.org/10.1016/j.chb.2016.01.021
Fast, L. A., & Funder, D. C. (2008). Personality as manifest in word use: correlations with self-report, acquaintance report, and behavior. Journal of Personality and Social Psychology, 94(2), 334–346. https://doi.org/10.1037/0022-3514.94.2.334
Funder, D. C. (1995). On the accuracy of personality judgment: A realistic approach. Psychological Review, 102(4), 652–670. https://doi.org/0033-295X/95/S3.00
Funder, D. C. (2012). Accurate personality judgment. Current Directions in Psychological Science, 21(3), 177–182. https://doi.org/10.1177/0963721412445309
Funder, D. C. (2015). The personality puzzle: Seventh international student edition. WW Norton & Company.
Funder, D. C. (Ed.). (1999). Personality judgment: A realistic approach to person perception. Elsevier.
Gill, A. J., Oberlander, J., & Austin, E. (2006). Rating e-mail personality at zero acquaintance. Personality and Individual Differences, 40(3), 497–507. https://doi.org/10.1016/j.paid.2005.06.027
Gosling, S. D., Ko, S. J., Mannarelli, T., & Morris, M. E. (2002). A room with a cue: Personality judgments based on offices and bedrooms. Journal of Personality and Social Psychology, 82(3), 379–398. https://doi.org/10.1037//0022-3514.82.3.379
Hall, J. A., Back, M. D., Nestler, S., Frauendorfer, D., Schmid Mast, M., & Ruben, M. A. (2018). How do different ways of measuring individual differences in zero‐acquaintance personality judgment accuracy correlate with each other?. Journal of Personality, 86(2), 220–232. https://doi.org/10.1111/jopy.12307
Hall, J. A., Gunnery, S. D., Letzring, T. D., Carney, D. R., & Colvin, C. R. (2017). Accuracy of judging affect and accuracy of judging personality: How and when are they related? Journal of Personality, 85(5), 583–592. https://doi.org/10.1111/jopy.12262
Hirschmüller, S., Egloff, B., Nestler, S., & Back, M. D. (2013). The dual lens model: A comprehensive framework for understanding self–other agreement of personality judgments at zero acquaintance. Journal of Personality and Social Psychology, 104(2), 335–353. https://doi.org/10.1037/a0030383
Hirsh, J. B., & Peterson, J. B. (2009). Personality and language use in self-narratives. Journal of Research in Personality, 43(3), 524–527. https://doi.org/10.1016/j.jrp.2009.01.006
Holleran, S. E., & Mehl, M. R. (2008). Let me read your mind: Personality judgments based on a person’s natural stream of thought. Journal of Research in Personality, 42(3), 747–754. https://doi.org/10.1016/j.jrp.2007.07.011
IBM Corp. (2012). IBM SPSS Statistics for Windows, Version 21.0. IBM Corp.
Ireland, M. E., & Mehl, M. R. (2014). Natural language use as a marker of personality. In T. Holtgraves (Ed.), Oxford handbook of language and social psychology (pp. 22–35). Oxford university press.
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting Intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine,15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
Lee, C. H., Kim, K., Seo, Y. S., & Chung, C. K. (2007). The relations between personality and language use. The Journal of General Psychology, 134(4), 405–413. https://doi.org/10.3200/GENP.134.4.405-414
Lee, K., & Ashton, M. C. (2008). The HEXACO personality factors in the indigenous personality lexicons of English and 11 other languages. Journal of Personality, 76, 1001–1053. https://doi.org/10.1111/j.1467-6494.2008.00512.x
Letzring, T. D., & Funder, D. C. (2018). Interpersonal accuracy in trait judgments. In Zeigler-Hill, V. & Shackelford, T. K. (Eds.) The SAGE handbook of personality and individual differences: Volume III: Applications of personality and individual differences (pp. 253–282). SAGE.
Li, J., & Chignell, M. (2010). Birds of a feather: How personality influences blog writing and reading. International Journal of Human-Computer Studies, 68(9), 589–602. https://doi.org/10.1016/j.ijhcs.2010.04.001
Mehl, M. R., Gosling, S. D., & Pennebaker, J. W. (2006). Personality in its natural habitat: Manifestations and implicit folk theories of personality in daily life. Journal of Personality and Social Psychology, 90(5), 862–877. https://doi.org/10.1037/0022-3514.90.5.862
Pennebaker, J. W., & King, L. A. (1999). Linguistic styles: Language use as an individual difference. Journal of Personality and Social Psychology, 77(6), 1296–1312. https://doi.org/10.1037//0022-3514.77.6.1296
Pennebaker, J. W., Mehl, M. R., & Niederhoffer, K. G. (2003). Psychological aspects of natural language use: Our words, our selves. Annual Review of Psychology, 54(1), 547–577. https://doi.org/10.1146/annurev.psych.54.101601.145041
Perinetti, G. (2018). StaTips Part IV: Selection, interpretation and reporting of the intraclass correlation coefficient. South European Journal of Orthodontics and Dentofacial Research, 5(1), 3–5. https://doi.org/10.5937/sejodr5-17434
Pretsch, J., Heckmann, N., Flunger, B., & Schmitt, M. (2014). Agree or Disagree?. European Journal of Psychological Assessment, 30(1), 31–39. https://doi.org/10.1027/1015-5759/a000165
Shevlin, M., Walker, S., Davies, M. N., Banyard, P., & Lewis, C. A. (2003). Can you judge a book by its cover? Evidence of self–stranger agreement on personality at zero acquaintance. Personality and Individual Differences, 35(6), 1373–1383. https://doi.org/10.1016/S0191-8869(02)00356-2
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. https://doi.org/10.1037/0033-2909.86.2.420
Tskhay, K. O., & Rule, N. O. (2014). Perceptions of personality in text-based media and OSN: A meta-analysis. Journal of Research in Personality, 49, 25–30. https://doi.org/10.1016/j.jrp.2013.12.004
Watson, D., Hubbard, B., & Wiese, D. (2000). Self–other agreement in personality and affectivity: The role of acquaintanceship, trait visibility, and assumed similarity. Journal of Personality and Social Psychology, 78(3), 546–558. https://doi.org/10.1037//0022-3514.78.3.546
Witt, L. A. (2002). The interactive effects of extraversion and conscientiousness on performance. Journal of Management, 28(6), 835–851. https://doi.org/10.1016/S0149-2063(02)00188-5
Wood, A. M., Brown, G. D., Maltby, J., & Watkinson, P. (2012). How are personality judgments made? A cognitive model of reference group effects, personality scale responses, and behavioral reactions. Journal of Personality, 80(5), 1275–1311. https://doi.org/10.1111/j.1467-6494.2012.00763.x
Yarkoni, T. (2010). Personality in 100,000 words: A large-scale analysis of personality and word use among bloggers. Journal of Research in Personality, 44(3), 363–373. https://doi.org/10.1016/j.jrp.2010.04.001
Back, M. D., & Nestler, S. (2016). Accuracy of judging personality. In J. A. Hall, M. Schmid Mast, & T. V. West (Eds.), The social psychology of perceiving others accurately (pp. 98–124). Cambridge University Press.
Beer, A. (2013). Group personality judgments at zero acquaintance: Communication among judges versus aggregation of independent evaluations. Journal of Research in Personality, 47(4), 385–389. https://doi.org/10.1016/j.jrp.2013.03.008
Beer, A., & Watson, D. (2008). Personality judgment at zero acquaintance: Agreement, assumed similarity, and implicit simplicity. Journal of Personality Assessment, 90(3), 250–260. https://doi.org/10.1080/00223890701884970
Beer, A., & Watson, D. (2010). The effects of information and exposure on self-other agreement. Journal of Research in Personality, 44(1), 38–45. https://doi.org/10.1016/j.jrp.2009.10.002
Bjekić, J. (2016). Stilističke i sadržinske karakteristike verbalne produkcije kao indikatori bazične ličnosti [The style and content of verbal production as indicators of basic personality structure]. Unpublished doctoral dissertation. Department of Psychology, Faculty of Philosophy, University of Belgrade.
Borkenau, P., Mosch, A., Tandler, N., & Wolf, A. (2016). Accuracy of judgments of personality based on textual information on major life domains. Journal of Personality, 84(2), 214–224. https://doi.org/10.1111/jopy.12153
Čerović, S. L. (2018). Primena analize verbalne produkcije u selekcionom intervjuu [Application of verbal production analysis in the selection interview]. Unpublished doctoral dissertation. Department of Psychology, Faculty of Philosophy, University of Belgrade.
Christiansen, N. D., Wolcott-Burnam, S., Janovics, J. E., Burns, G. N., & Quirk, S. W. (2005). The good judge revisited: Individual differences in the accuracy of personality judgments. Human Performance, 18(2), 123–149. https://doi.org/10.1207/s15327043hup1802_2
Connelly, B. S., & Ones, D. S. (2010). An other perspective on personality: Meta-analytic integration of observers' accuracy and predictive validity. Psychological Bulletin, 136(6), 1092–1122. https://doi.org/10.1037/a0021212
Connolly, J. J., Kavanagh, E. J., & Viswesvaran, C. (2007). The convergent validity between self and observer ratings of personality: A meta‐analytic review. International Journal of Selection and Assessment, 15(1), 110–117. https://doi.org/10.1111/j.1468-2389.2007.00371.x
Darbyshire, D., Kirk, C., Wall, H. J., & Kaye, L. K. (2016). Don't Judge a (Face) Book by its Cover: Exploring judgement accuracy of others' personality on Facebook. Computers in Human Behavior, 58, 380–387. https://doi.org/10.1016/j.chb.2016.01.021
Fast, L. A., & Funder, D. C. (2008). Personality as manifest in word use: correlations with self-report, acquaintance report, and behavior. Journal of Personality and Social Psychology, 94(2), 334–346. https://doi.org/10.1037/0022-3514.94.2.334
Funder, D. C. (1995). On the accuracy of personality judgment: A realistic approach. Psychological Review, 102(4), 652–670. https://doi.org/0033-295X/95/S3.00
Funder, D. C. (2012). Accurate personality judgment. Current Directions in Psychological Science, 21(3), 177–182. https://doi.org/10.1177/0963721412445309
Funder, D. C. (2015). The personality puzzle: Seventh international student edition. WW Norton & Company.
Funder, D. C. (Ed.). (1999). Personality judgment: A realistic approach to person perception. Elsevier.
Gill, A. J., Oberlander, J., & Austin, E. (2006). Rating e-mail personality at zero acquaintance. Personality and Individual Differences, 40(3), 497–507. https://doi.org/10.1016/j.paid.2005.06.027
Gosling, S. D., Ko, S. J., Mannarelli, T., & Morris, M. E. (2002). A room with a cue: Personality judgments based on offices and bedrooms. Journal of Personality and Social Psychology, 82(3), 379–398. https://doi.org/10.1037//0022-3514.82.3.379
Hall, J. A., Back, M. D., Nestler, S., Frauendorfer, D., Schmid Mast, M., & Ruben, M. A. (2018). How do different ways of measuring individual differences in zero‐acquaintance personality judgment accuracy correlate with each other?. Journal of Personality, 86(2), 220–232. https://doi.org/10.1111/jopy.12307
Hall, J. A., Gunnery, S. D., Letzring, T. D., Carney, D. R., & Colvin, C. R. (2017). Accuracy of judging affect and accuracy of judging personality: How and when are they related? Journal of Personality, 85(5), 583–592. https://doi.org/10.1111/jopy.12262
Hirschmüller, S., Egloff, B., Nestler, S., & Back, M. D. (2013). The dual lens model: A comprehensive framework for understanding self–other agreement of personality judgments at zero acquaintance. Journal of Personality and Social Psychology, 104(2), 335–353. https://doi.org/10.1037/a0030383
Hirsh, J. B., & Peterson, J. B. (2009). Personality and language use in self-narratives. Journal of Research in Personality, 43(3), 524–527. https://doi.org/10.1016/j.jrp.2009.01.006
Holleran, S. E., & Mehl, M. R. (2008). Let me read your mind: Personality judgments based on a person’s natural stream of thought. Journal of Research in Personality, 42(3), 747–754. https://doi.org/10.1016/j.jrp.2007.07.011
IBM Corp. (2012). IBM SPSS Statistics for Windows, Version 21.0. IBM Corp.
Ireland, M. E., & Mehl, M. R. (2014). Natural language use as a marker of personality. In T. Holtgraves (Ed.), Oxford handbook of language and social psychology (pp. 22–35). Oxford university press.
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting Intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine,15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
Lee, C. H., Kim, K., Seo, Y. S., & Chung, C. K. (2007). The relations between personality and language use. The Journal of General Psychology, 134(4), 405–413. https://doi.org/10.3200/GENP.134.4.405-414
Lee, K., & Ashton, M. C. (2008). The HEXACO personality factors in the indigenous personality lexicons of English and 11 other languages. Journal of Personality, 76, 1001–1053. https://doi.org/10.1111/j.1467-6494.2008.00512.x
Letzring, T. D., & Funder, D. C. (2018). Interpersonal accuracy in trait judgments. In Zeigler-Hill, V. & Shackelford, T. K. (Eds.) The SAGE handbook of personality and individual differences: Volume III: Applications of personality and individual differences (pp. 253–282). SAGE.
Li, J., & Chignell, M. (2010). Birds of a feather: How personality influences blog writing and reading. International Journal of Human-Computer Studies, 68(9), 589–602. https://doi.org/10.1016/j.ijhcs.2010.04.001
Mehl, M. R., Gosling, S. D., & Pennebaker, J. W. (2006). Personality in its natural habitat: Manifestations and implicit folk theories of personality in daily life. Journal of Personality and Social Psychology, 90(5), 862–877. https://doi.org/10.1037/0022-3514.90.5.862
Pennebaker, J. W., & King, L. A. (1999). Linguistic styles: Language use as an individual difference. Journal of Personality and Social Psychology, 77(6), 1296–1312. https://doi.org/10.1037//0022-3514.77.6.1296
Pennebaker, J. W., Mehl, M. R., & Niederhoffer, K. G. (2003). Psychological aspects of natural language use: Our words, our selves. Annual Review of Psychology, 54(1), 547–577. https://doi.org/10.1146/annurev.psych.54.101601.145041
Perinetti, G. (2018). StaTips Part IV: Selection, interpretation and reporting of the intraclass correlation coefficient. South European Journal of Orthodontics and Dentofacial Research, 5(1), 3–5. https://doi.org/10.5937/sejodr5-17434
Pretsch, J., Heckmann, N., Flunger, B., & Schmitt, M. (2014). Agree or Disagree?. European Journal of Psychological Assessment, 30(1), 31–39. https://doi.org/10.1027/1015-5759/a000165
Shevlin, M., Walker, S., Davies, M. N., Banyard, P., & Lewis, C. A. (2003). Can you judge a book by its cover? Evidence of self–stranger agreement on personality at zero acquaintance. Personality and Individual Differences, 35(6), 1373–1383. https://doi.org/10.1016/S0191-8869(02)00356-2
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. https://doi.org/10.1037/0033-2909.86.2.420
Tskhay, K. O., & Rule, N. O. (2014). Perceptions of personality in text-based media and OSN: A meta-analysis. Journal of Research in Personality, 49, 25–30. https://doi.org/10.1016/j.jrp.2013.12.004
Watson, D., Hubbard, B., & Wiese, D. (2000). Self–other agreement in personality and affectivity: The role of acquaintanceship, trait visibility, and assumed similarity. Journal of Personality and Social Psychology, 78(3), 546–558. https://doi.org/10.1037//0022-3514.78.3.546
Witt, L. A. (2002). The interactive effects of extraversion and conscientiousness on performance. Journal of Management, 28(6), 835–851. https://doi.org/10.1016/S0149-2063(02)00188-5
Wood, A. M., Brown, G. D., Maltby, J., & Watkinson, P. (2012). How are personality judgments made? A cognitive model of reference group effects, personality scale responses, and behavioral reactions. Journal of Personality, 80(5), 1275–1311. https://doi.org/10.1111/j.1467-6494.2012.00763.x
Yarkoni, T. (2010). Personality in 100,000 words: A large-scale analysis of personality and word use among bloggers. Journal of Research in Personality, 44(3), 363–373. https://doi.org/10.1016/j.jrp.2010.04.001
