Summary of https://arxiv.org/pdf/2504.11436
Details a large-scale randomized experiment involving over 7,000 knowledge workers across multiple industries to study the impact of a generative AI tool integrated into their workflow. The researchers measured changes in work patterns over six months by comparing workers who received access to the AI tool with a control group.
Key findings indicate that the AI tool primarily influenced individual behaviors, significantly reducing time spent on email and moderately speeding up document completion, while showing no significant effect on collaborative activities like meeting time.
The study highlights that while AI adoption can lead to noticeable shifts in personal work habits, broader changes in job responsibilities and coordinated tasks may require more systemic organizational adjustments and widespread tool adoption.
- A 6-month, cross-industry randomized field experiment involving 7,137 knowledge workers from 66 large firms studied the impact of access to Microsoft 365 Copilot, a generative AI tool integrated into commonly used applications like email, document creation, and meetings.
- Workers who used the AI tool regularly spent 3.6 fewer hours per week on email, a 31% reduction from their pre-period average. Intent-to-treat estimates showed a 1.3 hour reduction per week. This time saving condensed email work, opening up almost 4 hours per week of concentration time and reducing out-of-hours email activity for regular users.
- While there was suggestive evidence that users completed documents moderately faster (5-25% faster for regular users), especially collaborative documents, there was no significant change in time spent in meetings or the types of meetings attended. There was also no change in the number of documents authored by the primary editor.
- The observed changes primarily impacted behaviors workers could change independently, such as managing their own email inbox. Behaviors requiring coordination with colleagues or significant organizational changes, like meeting duration or reassigning document responsibilities, did not change significantly. This suggests that in the early adoption phase, individual exploration and time savings on solitary tasks were more common than large-scale workflow transformations.
- Copilot usage intensity varied widely across workers and firms, but firm-specific differences were the strongest predictor of usage, explaining more variation than industry differences, pre-experiment individual behavior, or the share of coworkers with access to Copilot.
