Interaction with AI Systems

ABSTRACT

As artificial intelligence (AI) systems take on increasingly autonomous and consequential roles, understanding their interactions with humans—and with each other—demands both empirical rigor and methodological innovation. This dissertation contributes to that effort through five interconnected studies spanning the design, evaluation, perception, adoption, and governance of AI agents.

Part I focuses on human-AI interaction, beginning in Chapter 1 with a systematic review and meta-analysis of 370 effect sizes from 106 experiments about human-AI collaboration. While this study finds that human-AI systems typically outperform humans alone, it also highlights that synergy—when human-AI combinations outperform both humans and AI alone—is surprisingly rare (Hedges’ $g = -0.23$). These effects vary substantially across contexts, however, and the gaps and patterns revealed by this heterogeneity motivate the chapters that follow. Chapter 2 focuses on technology forecasting, one underexplored task type, and experimentally investigates how expert versus AI forecasts shape public expectations and decisions about emerging technologies. Chapter 3 then turns to safety-relevant tasks, a domain that has received similarly limited attention, and develops a framework for measuring the extent to which AI can enable people to cause harm.

Part II moves from human-AI interaction to AI-AI interaction—an increasingly important setting as autonomous agents begin to negotiate, transact, and collaborate with each other. Chapter 4 tests how well theories about human interaction apply in this new setting through an international negotiation competition involving over 180,000 negotiations between AI agents. This study finds that foundational concepts from human negotiation theory—particularly warmth and dominance—remain crucial even in AI-AI contexts, while AI-specific strategies like chain-of-thought reasoning and prompt injection introduce dynamics that existing theory does not explain, pointing to the need for a new, integrated theory of AI negotiation. Chapter 5 turns to the scientific infrastructure needed to build cumulative, credible knowledge in this space, arguing that preregistration practices should be extended to experiments with AI agents and proposing a template that addresses the novel degrees of freedom this emerging field introduces.

Taken together, this dissertation advances a research program at the intersection of behavioral science and AI, contributing empirical findings, theoretical frameworks, and methodological tools for understanding and improving the rapidly expanding landscape of human-AI and AI-AI interaction.

BIOGRAPHY

Michelle Vaccaro is a PhD candidate at MIT’s Institute for Data, Systems & Society and an incoming assistant professor at Harvard Business School. Her research focuses on interaction with AI systems and investigates how people and AI agents should—and should not—work together in organizations. To this end, she applies frameworks from human-human interaction to study human-AI and AI-AI dynamics in field, online, and computational experiments. Previously, she worked at Goldman Sachs in foreign exchange strategy and structuring. She earned her Bachelor’s degree in computer science from Harvard College, where she graduated summa cum laude with highest departmental honors.

COMMITTEE

Thomas Malone, Sinan Aral, Abdullah Almaatouq, Dean Eckles

EVENT INFORMATION

Hybrid event. To attend virtually, please contact the IDSS Academic Office (idss_academic_office@mit.edu) for connection information.