Surveys have become an important tool in economics and in social science more broadly. However, it is well known that common methods (e.g., ordered probit) used to analyze ordinal survey data require strong, and often unjustified distributional assumptions. In this paper, we propose using observable survey response times to solve that problem. Our main identifying assumption is that individual response time is decreasing in the distance between the value of the latent variable and an indecision threshold. We provide conditions under which the expected values of the latent variables can be compared across groups, even without making any distributional assumptions. We show how our method can be implemented in practice and give rise to new insights by applying it to an online survey experiment.