De Freitas et al. (2024)

Lessons From an App Update at Replika AI: Identity Discontinuity in Human-AI Relationships

Find It

Abstract

As consumers increasingly interact with AI applications specialized for social relationships, what is the nature and depth of these relationships among actual users, and can company actions influence these dynamics? We find that active users of the US-based AI companion, Replika, feel closer to their AI companion than even their best human friend, and anticipate mourning the loss of their AI companion more than any other technology. We then leverage an app-update event in which Replika removed its erotic role play (ERP) feature, preventing intimate interactions between consumers and chatbots that were previously possible. We find that this event triggered negative reactions typical of losing a partner in human relationships, including mourning and deteriorated mental health. In short, users of AI companions are forming relationships that are very close and show characteristics typical of human relationships, and company actions can perturb these dynamics, creating risks to consumer welfare. Follow-up studies investigate the psychological mechanisms and what companies can do to mitigate these risks.

Reference

De Freitas, J., Castelo, N., Uğuralp, A. K., & Oğuz-Uğuralp, Z. (2024). Lessons from an app update at Replika AI: Identity discontinuity in human-AI relationships [Preprint]. arXiv. https://arxiv.org/abs/2412.14190

What Rests on This

Cited in the argument rather than behind a score: this is one of the papers the taxonomy is built on, named where the site explains itself rather than in the findings under a mechanism.