Institute for Economic and Social Research

Publications

Proceedings of the National Academy of Sciences of the United States of America

2020-02-01

Sensitivity of Self-reported Noncognitive Skills to Survey Administration Conditions

Proceedings of the National Academy of Sciences of the United States of America, PNAS

Yuanyuan Chen, Shuaizhang FengJames J. HeckmanTim Kautz

Abstract

Noncognitive skills (e.g., persistence and self-control) are typically measured using self-reported questionnaires in which respondents rate their own skills. In many applications—including program evaluation and school accountability systems—such reports are assumed to measure only the skill of interest. However, self-reports might also capture other dimensions aside from the skill, such as aspects of a respondent’s situation, which could include incentives and the conditions in which they complete the questionnaire. To explore this possibility, this study conducted 2 experiments to estimate the extent to which survey administration conditions can affect student responses on noncognitive skill questionnaires. The first experiment tested whether providing information about the importance of noncognitive skills to students directly affects their responses, and the second experiment tested whether incentives tied to performance on another task indirectly affect responses. Both experiments suggest that self-reports of noncognitive skills are sensitive to survey conditions. The effects of the conditions are relatively large compared with those found in the program evaluation literature, ranging from 0.05 to 0.11 SDs. These findings suggest that the effects of interventions or other social policies on self-reported noncognitive skills should be interpreted with caution.

Keywords: Noncognitive skills; Psychological assessment; Personality traits; Big Five; Incentives

Read more: https://doi.org/10.1073/pnas.1910731117


back

Copyright © 2019 Institute for Economic and Social Research ICP record No.: Yue ICP Bei No. 12087612