Crii: HCC: RUI: exploring the dynamics and quality of interactions with AI companion applications

Seattle University received a $175,000 Project Grant from the National Science Foundation's Division of Information and Intelligent Systems (Computer and Information Science and Engineering, CFDA 47.070) awarded on July 1, 2025, with completion targeted for June 30, 2027. This award supports research investigating the dynamics and quality of interactions between users and artificial intelligence (AI) companion applications. The primary deliverables include a comprehensive taxonomy of user interactions developed through analysis of social media data and user interviews; an algorithmic audit employing role-playing AI personas to evaluate conversation quality across diverse psychological and behavioral profiles; and evidence-based design guidelines for ethically aligned AI companion systems co-created through participatory workshops with end users.

The research outcomes are intended to inform regulatory frameworks, contribute to broader discussions on AI-human interaction ethics, and provide critical insights for the safe and responsible deployment of AI companionship technologies. No sub-awards are planned under this project.

Abstract

The increasing use of artificial intelligence (AI)-powered companion applications is transforming how people engage with digital entities. AI companions can serve various roles that foster deep emotional connections, including friend, mentor, or romantic partner. These applications are growing in popularity, raising questions about their impact on users'. While some people report finding comfort and support in AI companionship, reports of uncomfortable interactions underscore the urgent need for comprehensive evaluation and responsible design. This project investigates the dynamics and quality of interactions between people and AI companion apps, ensuring these technologies enhance user experience. By understanding user experiences, assessing the quality of AI-driven conversations, and developing design guidelines, and promoting their safe and responsible deployment. This research will be conducted in three phases. First, large-scale social media data and user interviews will be analyzed to develop a taxonomy of user interactions, identifying motivations, challenges, and values associated with AI companion apps. Second, an innovative algorithmic audit will be conducted using role-playing AI personas, which are LLM-based simulated users representing diverse psychological and behavioral profiles, to evaluate the quality of conversations with AI companion applications. Third, participatory design workshops will engage users in CO-creating guidelines for ethically aligned AI companion systems. The findings will provide critical insights into AI companionship, inform regulatory frameworks, and contribute to broader discussions on the ethics of AI-human interactions. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are not planned for this award.