RESEARCH FOR ENTREPRENEURS · 18.11.2026
AI and Productivity: Lessons from Three Developer Experiments

The promise of faster work is one reason companies adopt generative AI. Yet a tool does not necessarily deliver the same value to every team. A 2026 Management Science study evaluated three randomized field experiments in which software developers either received access to an AI coding assistant or worked without it. The research offers an unusually useful example of measuring work output rather than simply asking employees whether they like a technology.
What the researchers measured
The main question was whether access to an assistant providing intelligent code completions changes how many development tasks are finished. That measure is narrower than the claim that AI improves the performance of an entire business. Task counts alone may not capture code quality, security, subsequent review effort or project economics. Identifying exactly what was measured is essential before translating a research headline into a purchasing decision.
How the experiments worked
The field trials took place at Microsoft, Accenture and an unnamed Fortune 100 firm. Randomly chosen developers were given access to the assistant. Across the three studies, the pooled analysis included 4,867 developers. Random assignment offers stronger evidence of an intervention effect than a survey of impressions. Still, the setting concerns a particular kind of specialized work rather than accounting, sales or customer support.
The main reported result
Pooling the experiments, the researchers report a 26.08% increase in completed tasks among developers using the assistant. They also warn that individual experiments were noisy and estimates differed across them. Less experienced developers showed higher uptake and greater gains. The overall result should therefore not be presented as a guaranteed 26% productivity improvement for every worker, job or company.
What this means for businesses
Before buying tools, managers should identify what output they want to improve. A software team might track not just completed tasks but also review time, rework and reliability. Employee experience may matter because the results differed across levels of expertise. When a product is useful only for selected activities, buying licenses for every employee without a deployment plan may not create the expected business value.
An example of a pilot
A company could select recurring, measurable tasks and record baseline values before testing an assistant with part of the team. Comparisons should include time, corrections, total costs and the work of experienced reviewers. This is an editorial example for assessing an investment, not a reproduction of the authors’ exact experimental design. Confidentiality and protection of internal code remain important.
Avoiding overclaims
The study does not establish that profits rise by the same percentage as completed tasks. It does not demonstrate identical benefits in other occupations or support broad claims about automatic job replacement. Faster production of weak outputs can shift costs into later review and repair. Evaluating the full workflow is therefore more informative than measuring only the speed of one stage.
Conclusion
Three real-world experiments offer a stronger basis for discussing AI than product marketing alone. The findings suggest improvements in coding task completion under defined conditions, with differences between groups of developers. For entrepreneurs, the practical lesson is measurement: judge AI investments against quality, cost and final business outcomes, not a percentage taken out of context.
Scientific source
Kevin Zheyuan Cui, Mert Demirer, Sonia Jaffe, Leon Musolff, Sida Peng, Tobias Salz (2026). The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers. Management Science. https://doi.org/10.1287/mnsc.2025.00535 https://doi.org/10.1287/mnsc.2025.00535