Outcome feedback and AI explanations may improve human-AI synergy in collaborative tasksFeedback and explanations can improve how humans work with AI

Humans collaborating with AI hold the promise of achieving superior outcomes compared to either acting alone (i.e., human-AI synergy). However, the conditions that facilitate such synergy when humans are advised by AI are not well understood. A recent meta-analysis showed that, on average, human-AI combinations do not outperform the better individual agent. We argue that this pessimistic conclusion arises from insufficient attention to human learning in experimental designs. To substantiate this claim, we reanalyzed all 74 studies included in the original meta-analysis and found that most previous research overlooked design features that foster human learning (e.g., outcome feedback to participants). Our reanalysis further revealed that studies providing outcome feedback show tentatively higher synergy than those without outcome feedback. Crucially, feedback paired with AI explanations was associated with positive synergy, while explanations without feedback were associated with negative synergy-suggesting that explanations improve synergy mainly when humans can learn to verify the AI's reliability through feedback. Our reanalysis suggests that the current literature underestimates the potential of human-AI collaboration because it predominantly relies on paradigms that do not facilitate human learning, thus hindering humans from effectively adapting their collaboration strategies. However, experiments directly varying learning opportunities are needed for stronger, causal conclusions. We advocate for a paradigm shift in human-AI interaction research that explicitly addresses human learning and thus enhances our understanding of and support for successful human-AI collaboration.