Beyond Interaction: Investigating the Appropriateness of Human-AI Assistant Relationships

Arianna Manzini, Geoff Keeling, Lize Alberts, Shannon Vallor, Meredith Ringel Morris, Iason Gabriel
May, 2024

Abstract

This paper explores the moral limits of relationships between users and advanced AI assistants, specifically which features of such relationships render them appropriate or inappropriate. We first consider a series of values including benefit, flourishing, autonomy and care that are characteristic of appropriate human interpersonal relationships. We use these values to guide an analysis of which features of user–AI assistant relationships are liable to give rise to harms, and then we discuss a series of risks and mitigations for such relationships. The risks that we explore are - (1) causing direct emotional and physical harm to users; (2) limiting opportunities for user personal development; (3) exploiting emotional dependence; and (4) generating material dependencies.

Type
Preprint
Publication
Accepted at AAAI Conference on AI, Ethics, and Society (AIES)
Advanced AI Assistants Conversational Agents Social Interactional Harms Human-AI Relationships
Lize Alberts
Lize Alberts
Doctoral Candidate, Research Fellow

DPhil candidate in Computer Science at the University of Oxford and Research Fellow at Stellenbosch University’s Unit for the Ethics of Technology.