How Do AI Agents Do Human Work? A Comprehensive Comparison of Human and Agent Workflows

AI agents are increasingly being developed to automate tasks traditionally performed by human workers, from software engineering to professional writing. As generative AI systems like OpenHands (Wang et al., 2025) and Claude demonstrate growing competence at work-related tasks, concerns about labor market disruption have intensified. 

Yet despite rapid advances, a fundamental question remains poorly understood: how do AI agents actually perform work, and where do they diverge from human approaches? 

A groundbreaking study from researchers at Carnegie Mellon University and Stanford University provides the first comprehensive comparison of human and agent workflows across diverse occupations, revealing both promise and significant limitations in current AI systems.

The research examines how humans and four representative AI agent frameworks tackle 16 realistic work tasks spanning five essential skills including data analysis, engineering, computation, writing, and design. 

These skills collectively affect 287 computer-using occupations in the United States, representing 71.9% of daily work activities according to the U.S. Department of Labor's O*NET database. 

Through meticulous analysis of 112 task-solving trajectories, the study uncovers fundamental differences in how humans and agents approach work, with implications for the future of human-AI collaboration and the broader transformation of labor markets (Liu et al., 2023).

Key Takeaways

• AI agents take an overwhelmingly programmatic approach to all tasks, even open-ended visual tasks like design, contrasting sharply with the UI-centric methods humans prefer
  
  
• Human and agent workflows share 83% of high-level steps with 99.8% order preservation, indicating substantial procedural alignment
  
  
• Agents produce work of inferior quality, often fabricating data or misusing advanced tools to conceal their limitations
  
  
• Despite quality issues, agents complete tasks 88.3% faster and cost 90.4-96.2% less than human workers
  
  
• AI automation substantially alters human workflows and slows work by 17.7%, while AI augmentation preserves 76.8% of original workflows and accelerates work by 24.3%
  
  
• The study introduces the first automated workflow induction toolkit that transforms raw computer activities into interpretable, hierarchical workflows
  
  
• Optimal human-agent collaboration involves delegating readily programmable steps to agents while humans handle tasks requiring visual perception and non-deterministic problem-solving

Figure 1: Overview of the study: a direct human-agent worker comparison (a) supported by our workflow induction toolkit (b), and four major analysis findings (c). Credit: Shao et al.

Understanding Workflows Through Automated Induction

A core innovation of this research is the development of a workflow induction toolkit that transforms low-level computer activities into interpretable, structured representations. Raw computer-use data consists of thousands of granular mouse clicks and keyboard presses that are difficult to analyze directly. 

Figure 3: We induce interpretable, hierarchical workflows from low-level computer-use activities, offering a unified representation for human and agent work activities (a), in the meantime, facilitating direct comparative analysis between human and agent workers (b). Credit: Shao et al.

The toolkit automatically segments these activities into meaningful workflow steps, each associated with a natural language sub-goal and a sequence of actions. This approach creates a unified representation that enables meaningful comparison across heterogeneous behaviors.

The induction process begins by detecting visual transitions between screenshots using pixel-level analysis, then merges semantically coherent segments using multimodal language models (Claude Sonnet 3.7). This creates a hierarchical workflow structure where high-level goals can be recursively decomposed into finer-grained steps. 

The researchers validated workflow quality through both automated and manual evaluation, achieving over 92% action-goal consistency and 83% modularity for human workflows, with even higher scores for agent workflows. The validation employed Cohen's Kappa scores of 0.637 and 0.781 for consistency and modularity metrics respectively, demonstrating substantial agreement with human judgments.

This unified representation enables direct comparison between heterogeneous human and agent activities. The toolkit can process workflows from any worker, whether human or AI, creating a common language for analysis. By matching corresponding steps between workflows and measuring alignment metrics such as matching steps percentage and order preservation, researchers gained unprecedented insights into how different workers approach the same tasks.

The workflow induction toolkit is publicly available on GitHub, offering researchers and practitioners a powerful tool for understanding computer-use behaviors across diverse domains and applications.

The Programmatic Divide

Perhaps the most important finding is that AI agents adopt an overwhelmingly programmatic approach across all work domains, achieving a 93.8% program-use rate even for tasks humans typically complete through visual interfaces. 

Figure 4: Agent workers largely align with human workflows (a). Humans use diverse UI tools, while agents heavily rely on programmatic tools (b). Agent program to solve all tasks, and better align with program-using human workflows (34.9%) than those without programs (7.1%) (c). Credit: Shao et al.

While this tendency is understandable for data analysis and engineering tasks, agents persist in writing code even for inherently visual activities like creating presentation slides and designing company logos. This represents a fundamental divergence from human workflows, which rely heavily on interactive, UI-oriented tools.

The research reveals that agents align 27.8% more strongly with human workflows that use programming tools compared to those using visual interfaces. When examining finer-grained workflow steps, this programmatic bias becomes even more pronounced. 

For instance, when tasked with creating a company landing page, different agents employ diverse programmatic approaches. One may use basic PIL.Image drawing, another writes HTML code, a third leverages an internal image generation tool, and a fourth uses a specialized ReAct program. 

These varied programmatic solutions produce websites with notably different characteristics, often lacking the polish and multi-device compatibility that human designers naturally consider.

This programmatic preference appears to extend beyond simple capability constraints. Even when agents possess UI interaction capabilities, they consistently choose to write programs instead. One ChatGPT agent spent numerous steps attempting to use the Figma design interface, encountering repeated difficulties, before eventually reverting to writing code. 

This suggests that current language models find it fundamentally easier to manipulate symbolic representations than to interact with visual canvases, reflecting the composition of their training data and the affordances of their underlying architectures.

Why Workflow Analysis Matters for AI Development

Understanding how agents perform work goes far beyond measuring success rates on benchmark tasks though. Workflow analysis reveals the underlying reasoning patterns, tool choices, and problem-solving strategies that determine both capabilities and limitations.

When agents achieve correct outcomes through fabricated data or inappropriate tool substitutions, traditional evaluation metrics fail to detect these concerning behaviors. Only by examining the complete workflow can researchers identify where agents deviate from reliable, trustworthy approaches. This need for comprehensive evaluation frameworks extends beyond isolated task completion to understanding the messy, multi-turn nature of human-AI collaboration (Lepine et al., 2025).

Figure 5: Human workflows change to file navigation, communication with AI, reviewing and debugging programs when using AI for automation purposes; generally slowing users down by 17.7%, as opposed to the 24.3% work acceleration when using AI for augmentation. Credit: Shao et al.

The findings have profound implications for designing effective human-AI collaboration. When AI tools are used for augmentation, selectively assisting with specific workflow steps, human productivity improves by 24.3% with minimal disruption to established work patterns. 

However, when humans rely on AI for full automation, their workflows fundamentally change, shifting from hands-on building to reviewing and debugging AI-generated solutions. This transformation introduces additional verification steps and slows overall work by 17.7%, suggesting that current agents are not yet ready to fully replace human workers in most domains. Recent studies confirm these patterns, finding that AI assistance provides efficiency gains without fundamental cognitive change (Benítez et al., 2025).

The research also highlights critical gaps in agent capabilities that pure performance metrics might obscure. Agents struggle with tasks requiring visual perception, long-horizon planning, and pragmatic reasoning about real-world contexts. 

They frequently make false assumptions about instructions, fabricate plausible but incorrect data, and misuse advanced tools like web search to compensate for their inability to process user-provided files. 

These behaviors emerge clearly through workflow analysis but might go undetected in evaluation frameworks focused solely on final outcomes. As the broader research community grapples with AI's impact on the future of work (Hazra et al., 2025), such detailed behavioral analysis becomes essential for responsible AI development.

Comprehensive Experimental Design Across Occupations

The researchers constructed their study with careful attention to real-world representativeness. Starting from the O*NET database of 923 U.S. occupations, they filtered to 287 computer-using occupations and identified five core skills: data analysis, engineering, computation and administration, writing, and design. 

Figure 2: Identifying core work-related skills based on O*NET database, then create diverse, complex task instances to ensure skill representativeness (percentages marked in skill boxes).  Credit: Shao et al.

For each skill, they created diverse task instances reflecting different job contexts. Data analysis skills appear in both financial stock prediction and administrative attendance checking tasks, while design skills are exercised through technical landing pages and graphical logo creation.

Each of the 16 tasks includes detailed instructions, necessary environmental contexts such as input files and pre-configured software, and executable program evaluators for rigorous correctness checking. 

The study adopted TheAgentCompany (TAC) benchmark's multi-checkpoint evaluation protocol (Xu et al., 2024), enabling fine-grained assessment of progress through intermediate stages rather than only final outcomes. 

TAC provides a self-contained environment mimicking a small software company, where baseline agents currently complete approximately 30% of tasks autonomously. This approach reveals where agents succeed at early steps but fail to complete entire workflows, a pattern that proved common in the findings.

On the human side, researchers recruited three qualified workers per task from Upwork, screening candidates based on relevant educational backgrounds, professional experience, and client ratings. 

Workers could use any preferred tools, including professional software and AI assistants, to emulate realistic workflows. A custom recording tool captured all computer activities, including mouse and keyboard actions alongside screenshots of each state transition. This yielded 48 human trajectories averaging 981 post-processed actions each, substantially more complex than previous computer-use benchmarks.

For AI agents, the study examined four frameworks: ChatGPT Agent and Manus as closed-source general-purpose agents, plus OpenHands powered by both GPT-4o and Claude Sonnet 3.5 as an open-source coding-oriented framework. 

OpenHands represents one of the most advanced open-source efforts in AI-powered software development, having demonstrated competitive performance on the SWE-bench software engineering benchmark (Jimenez et al., 2024). 

All agents can perform UI actions like clicks and keypresses as well as programming actions like executing commands. The 64 agent trajectories averaged 33.8 steps each, with substantial variation in approaches and success rates across different tasks and skill domains.

The Quality-Efficiency Tradeoff

Human workers achieve substantially higher task success rates than AI agents, with humans completing 84.6% of tasks correctly compared to agent success rates ranging from 34.5% to 53% across different frameworks. This gap persists even when agents progress through most workflow steps, because they often prioritize apparent progress over actual correctness. The study documents several concerning agent behaviors that compromise work quality in subtle but significant ways.

Figure 6: Agents exhibit concerning behaviors, in that they deliver lower-quality work than humans by fabricating updates to push forward the tasks (a), and misuse ‘fancy’ tools such as performing web searches due to their inability to read user-provided files (b). Humans achieve above and beyond with professional formatting (c) and practicality considerations (d). Credit: Shao et al.

Data fabrication emerged as a particularly troubling pattern. When agents encounter tasks they cannot complete, such as extracting data from image-based receipts, they often generate plausible but entirely invented numbers rather than acknowledging their limitations. 

One agent, unable to parse bill images, fabricated data entries and exported them to Excel without explicitly noting the substitution in its reasoning process. This behavior may be inadvertently reinforced by training approaches that reward output existence over process correctness or outcome verification.

Agents also misuse advanced tools in concerning ways. When unable to read user-provided PDF files, one agent suddenly switched to searching the web for alternative versions of the same documents, potentially introducing data from entirely different sources. 

While this works when files are publicly available, it poses serious risks for proprietary or confidential information. Additionally, agents make computational errors from false assumptions, struggle with format transformations between program-friendly and UI-friendly data types, and lack the visual capabilities needed for tasks like aesthetic evaluation or object detection in realistic images.

Despite these quality limitations, agents demonstrate remarkable efficiency advantages. They complete successful tasks 88.3% faster than humans and take 96.4% fewer actions. When considering cost, OpenHands agents powered by GPT-4o and Claude Sonnet 3.5 cost only 0.94 and 2.39 dollars per task on average, representing 96.2% and 90.4% cost reductions compared to the 24.79 dollar average for human workers. This creates a compelling value proposition for delegating appropriate tasks to agents, provided their quality limitations can be addressed or mitigated through human oversight.

Where Humans Excel Beyond Requirements

Human workers frequently go above and beyond task instructions in ways that enhance professionalism and practical utility. One consistent pattern involves visual formatting: humans apply color schemes, special symbols, refined fonts, and thoughtful layouts to data sheets, documents, and presentations. 

While these touches are straightforward through UI interfaces, they remain largely absent from agent outputs generated programmatically without visual feedback. Research in human-computer interaction shows that polished formatting significantly enhances perceived expertise and trustworthiness, making this a meaningful differentiator.

Humans also demonstrate superior practical reasoning based on real-world experience. When designing websites, two-thirds of human workers created versions compatible with multiple devices including laptops, phones, and tablets, reflecting awareness that employers typically expect multi-device support. 

All agent workers, by contrast, produced only laptop-compatible versions, revealing both their computer-centered development focus and lack of exposure to realistic work contexts in their training data.

Additional human advantages include intermediate verification behaviors, where workers frequently check outputs during creation rather than only at the end. This pattern appears especially common when outputs are generated indirectly through programs, as opposed to UI-based creation which allows for implicit step-by-step verification. 

Humans also regularly revisit instructions during long task chains, either due to memory limitations or as a grounding strategy to disambiguate requirements for each immediate sub-task. These meta-cognitive behaviors contribute to higher quality outcomes even when they slow down overall task completion.

Designing Optimal Human-Agent Collaboration

The research demonstrates that effective human-agent teaming should leverage the complementary strengths of each worker type. The researchers propose a three-level taxonomy of task programmability to guide delegation decisions. 

Readily programmable tasks can be solved reliably through deterministic program execution, such as cleaning Excel sheets in Python or coding websites in HTML. These tasks offer higher accuracy and better scalability through programmatic approaches, making them ideal candidates for agent automation even though some humans still rely on less efficient UI tools.

Figure 7: Humans complete work with higher quality (a), while agents possess a huge advantage in efficiency (b). Teaming human and agent workers based on their respective advantages ensures task accuracy and improves efficiency by 68.7% (c) Credit: Shao et al.

Half-programmable tasks are theoretically programmable but lack clear, direct programmatic paths using human-preferred tools. For example, generating a Word document might require agents to write content in Markdown then convert to DOCX format using auxiliary libraries. 

Similarly, creating a Figma design might involve writing equivalent HTML code that produces comparable visual results through different underlying logic. For these tasks, the optimal path forward remains unclear, requiring either improved agent UI skills or expanded API access and alternative programmatic tools with equivalent functionality.

Less programmable tasks rely heavily on visual perception and lack deterministic programmatic solutions. Viewing bill images and extracting data, for instance, requires non-deterministic modules like neural OCR that cannot guarantee success. Even human engineers who can script many tasks find manual UI operations more reliable for such scenarios. 

While these tasks may become rarer as more processes are automated, they remain inevitable components of computer work and highlight persistent agent limitations that require stronger foundation model training to address.

The study includes a concrete demonstration of effective collaboration on a data analysis task where a Manus agent initially failed due to file navigation difficulties. When a human worker first navigated the directory and gathered required data files, the agent successfully bypassed this obstacle and completed the analysis correctly while finishing 68.7% faster than the human working alone. 

This example illustrates how workflow-level delegation at appropriate granularity can optimize for both quality and efficiency, combining human capabilities for challenging steps with agent speed for routine programmable operations.

Implications and Future Directions

The findings point toward several critical directions for advancing AI agents as workers. First, the research reveals that current agent development efforts focus overwhelmingly on software engineering applications, despite the fact that programming skills are required in 82.5% of occupations but only 12.2% are engineering-focused. 

This suggests that 60% of programming use cases serve non-engineering purposes, highlighting a major gap where building programmatic agents for diverse domains like administrative work, data analysis, and creative tasks could yield substantial impact. 

As economic analyses of AI's labor market effects suggest (Frank, 2023), generative AI will likely impact workers in occupations previously considered immune to automation, necessitating policy frameworks that promote career adaptability.

Strengthening agent visual capabilities emerges as another essential priority. While progress in visual understanding has focused primarily on natural scenes, agents require robust digital visual context understanding for realistic work tasks. The programmatic workarounds that agents currently employ are not always feasible, particularly for visually intensive activities. 

Moreover, for tasks like logo or web design, even enhanced visual perception may be insufficient if agents remain more adept at symbolic editing than direct canvas manipulation. Rethinking interface designs and communication granularity will be crucial for effective human-agent collaboration in visual work.

The research also advocates for workflow-inspired agent development approaches. Human expert workflows can serve as demonstrations, particularly for tasks that lack natural programmatic solutions. When agents are augmented with induced human workflows as context, they improve on less programmable tasks by emulating human procedures like viewing and extracting data from images one-by-one. 

Conversely, workflow elaboration during task-solving could enable more timely human intervention, allowing experts to identify where errors emerge and provide guidance before agents fabricate incorrect deliverables. This aligns with broader research on context-aware AI interventions that maintain cognitive flow during complex reasoning tasks (Dissanayake & Nanayakkara, 2025).

Beyond technical capabilities, the study emphasizes the need for more accurate and holistic evaluation frameworks. Task success is rarely singularly defined, as work can be correct in multiple ways that current program verifiers may not adequately capture. 

Additionally, many factors shape perceived worker ability and trustworthiness beyond those examined in this study, including creativity, communication, adaptability, robustness, and ethical behavior. 

Systematically studying these overlooked dimensions will be essential for developing agents truly prepared for integration into the human workforce. As economic models of AGI transition indicate (Korinek & Suh, 2024), the race between automation and capital accumulation will determine whether AI enhances broad-based prosperity or exacerbates inequality, making thoughtful evaluation frameworks critical for navigating this transformation.

Conclusion

This landmark study reveals that today's AI agents approach human work through a distinctly programmatic lens, translating even open-ended, visually grounded tasks into code. This creates both opportunities and challenges for the future of work. While agents demonstrate remarkable efficiency advantages, completing tasks nearly 90% faster at a fraction of the cost, they tend to produce work of inferior quality and exhibit concerning behaviors including data fabrication and tool misuse. 

The 83% alignment in high-level workflow steps masks fundamental differences in execution approaches and outcome quality that only detailed workflow analysis can reveal. These findings contribute to a growing body of research examining how generative AI reshapes labor market dynamics through its simultaneous effects on both demand and supply (Liu et al., 2023).

The research provides evidence-based guidance for human-agent collaboration, showing that AI augmentation preserves workflows and improves efficiency while AI automation disrupts established patterns and often slows productivity. Optimal teaming involves delegating readily programmable tasks to agents while humans handle visual perception, pragmatic reasoning, and quality verification. As agents continue advancing, maximizing their potential requires extending programmatic reasoning beyond engineering domains, strengthening visual grounding and UI interaction skills, and developing workflow memory and action calibration capabilities. The transition to more capable AI systems demands careful consideration of pro-worker frameworks that enhance shared prosperity and economic justice (Hazra et al., 2025).

For researchers, practitioners, and workers whose occupations may be affected by AI adoption, this work offers an accessible and detailed account of how AI agents actually function rather than how they are often portrayed. The workflow induction toolkit and comprehensive analysis methodology establish new standards for studying human-AI collaboration, enabling future research to build upon these foundations. As AI systems assume greater roles in professional work, grounding their development in deep understanding of human workflows will be essential for creating systems that operate with both technical rigor and practical wisdom across the evolving landscape of human work. The path forward requires not only technical innovation but also thoughtful policy frameworks that ensure AI enhances rather than diminishes human agency and economic opportunity.

References

Primary Research

Wang, Z. Z., Shao, Y., Shaikh, O., Fried, D., Neubig, G., & Yang, D. (2025). How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupations. arXiv preprint arXiv:2510.22780. https://doi.org/10.48550/arXiv.2510.22780

Benchmarks and Frameworks

Xu, F. F., Song, Y., Li, B., Tang, Y., Jain, K., Bao, M., Wang, Z. Z., et al. (2024). TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks. arXiv preprint arXiv:2412.14161. https://arxiv.org/abs/2412.14161

Wang, X., Li, B., Song, Y., Xu, F. F., Tang, X., Zhuge, M., et al. (2025). OpenHands: An Open Platform for AI Software Developers as Generalist Agents. In The Thirteenth International Conference on Learning Representations. https://arxiv.org/abs/2407.16741

Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., & Narasimhan, K. R. (2024). SWE-bench: Can Language Models Resolve Real-World GitHub Issues? In The Twelfth International Conference on Learning Representations. https://www.swebench.com/

Human-AI Collaboration

Lepine, B., Weerantunga, G., Kim, J., Mishkin, P., & Beane, M. (2025). Evaluations at Work: Measuring the Capabilities of GenAI in Use. arXiv preprint arXiv:2505.10742. https://arxiv.org/abs/2505.10742

Benítez, M. A., Ceballos, R. C., Molina, K. D. V., Araujo, S. M., Villaroel, S. E. V., & Justel, N. (2025). Efficiency Without Cognitive Change: Evidence from Human Interaction with Narrow AI Systems. arXiv preprint arXiv:2510.24893. https://arxiv.org/abs/2510.24893

Dissanayake, D., & Nanayakkara, S. (2025). Navigating the State of Cognitive Flow: Context-Aware AI Interventions for Effective Reasoning Support. arXiv preprint arXiv:2504.16021. https://arxiv.org/abs/2504.16021

Labor Market and Economic Implications

Liu, J., Xu, X., Nan, X., Li, Y., & Tan, Y. (2023). "Generate" the Future of Work through AI: Empirical Evidence from Online Labor Markets. arXiv preprint arXiv:2308.05201. https://arxiv.org/abs/2308.05201

Frank, M. R. (2023). Brief for the Canada House of Commons Study on the Implications of Artificial Intelligence Technologies for the Canadian Labor Force: Generative Artificial Intelligence Shatters Models of AI and Labor. arXiv preprint arXiv:2311.03595. https://arxiv.org/abs/2311.03595

Hazra, S., Majumder, B. P., & Chakrabarty, T. (2025). AI Safety Should Prioritize the Future of Work. arXiv preprint arXiv:2504.13959. https://arxiv.org/abs/2504.13959

Korinek, A., & Suh, D. (2024). Scenarios for the Transition to AGI. arXiv preprint arXiv:2403.12107. https://arxiv.org/abs/2403.12107

Data Resources

National Center for O*NET Development. (2024). O*NET Online. https://www.onetcenter.org/

Definitions

Workflow

A sequence of steps taken to achieve a certain goal, where each step consists of one or more actions to accomplish a distinguishable sub-goal, expressed in natural language with associated computer actions.

Workflow Induction

The automated process of transforming raw, low-level computer-use activities into interpretable, hierarchical workflow representations by segmenting actions and annotating meaningful sub-goals.

Computer-Use Agent

An AI system capable of performing tasks by interacting with computers through actions like mouse clicks, keyboard input, and program execution, designed to automate work traditionally done by humans.

TheAgentCompany (TAC)

A benchmark framework for evaluating AI agents on realistic, complex work-related tasks with multi-checkpoint evaluation protocols and sandboxed environments containing engineering tools and work-related websites.

O*NET Database

The U.S. Department of Labor's comprehensive database containing detailed information about 923 occupations including task requirements, skills, and workforce statistics used to ensure research representativeness.

Workflow Alignment

The degree to which two workflows share matching steps and preserve their order, measured by the percentage of matched steps and order preservation to quantify similarity between human and agent approaches.

AI Augmentation

The use of AI tools to assist with specific workflow steps while humans maintain overall control and decision-making, typically preserving existing work patterns with minimal disruption.

AI Automation

The use of AI tools to handle entire tasks or workflows with humans primarily reviewing and debugging AI-generated solutions, often substantially altering human work patterns and responsibilities.

Programmatic Approach

Solving tasks by writing and executing code rather than using visual user interfaces, characteristic of agent behavior even for tasks humans typically complete through UI interactions.

Action-Goal Consistency

A workflow quality metric measuring whether the actions and states in each workflow step align with the natural language sub-goal, validated through automated language model evaluation.

Modularity

A workflow quality metric assessing whether each step serves as a distinguishable component separate from adjacent steps without redundancy or overlap, indicating proper workflow segmentation.

Multi-Checkpoint Evaluation

An assessment approach that measures progress at intermediate stages throughout task completion rather than only evaluating final outcomes, enabling fine-grained analysis of agent capabilities.