Managing expectations towards AI tools for software development: a multiple-case study

Software development organizations (SDOs) are increasingly working to adopt artificial intelligence (AI) tools, like GitHub Copilot, to meet varied expectations. Nevertheless, we know little about how SDOs manage these expectations. This paper investigates how different SDOs expect AI tools to impact software development, and how these expectations change after a period of considering and evaluating AI tools. We conducted a multiple-case study involving three SDOs. To elicit initial expectations towards AI tools, we collected data using semi-structured interviews and field visits. To assess the persistence of expectations towards AI tools, we collected data from meetings, a debriefing, and retrospectives on AI tools. We found three expectations particular to one SDO; four shared between two SDOs; and six pervasive across all SDOs. Five expectations did not persist after experiential learning with AI tools, due to platform- and SDO-related factors. SDOs must carefully manage their expectations towards AI tools due to the variety and complexity of expectations. Some expectations are niche-specific based on their compatibility with the unique SDOs’ people- and structure-related aspects, while others are becoming mainstream for a broader array of SDOs. Recognizing factors that affect the persistence of expectations and how they manifest in the individual SDO will enable SDOs to form their initial expectations and understand how these might change during adoption of AI tools, supporting expectation management.