3x3 Institute

Evaluating AI and Human Performance

September 17, 2023

So how much does AI improve human performance?

The impact of AI on human performance varies depending on the task and the specific AI system being used. In general, AI can help humans to be more productive, efficient, and accurate. However, AI is not yet able to outperform humans on all tasks, and there are some tasks where AI can actually hinder human performance.

For example, a study by McKinsey found that AI could boost global productivity by 0.8% to 1.4% per year between 2018 and 2025. However, the study also found that AI could displace up to 800 million jobs by 2030.

What is the best method for obtaining such improvements?

The rise of large language models (LLMs) like ChatGPT has sparked much interest and debate around how artificial intelligence will impact human work. Two recent studies provide some intriguing early insights into LLMs’ effects on creativity and knowledge worker productivity.

In one study, researchers compared ChatGPT to human students on a creative divergent thinking task. On average, the AI chatbot outperformed humans in generating more creative and higher quality ideas. However, the very best human ideas still matched or exceeded ChatGPT’s. The study suggests AI can enhance average creativity, but uniquely human traits likely underpin the most highly creative contributions.

Another study looked at how consultants working for a management firm used ChatGPT on realistic business tasks. For analytic activities clearly within ChatGPT’s capabilities, the AI significantly boosted consultants’ productivity and work quality. Consultants completed over 12% more subtasks and 25% faster with AI aid. Response quality rose over 40%. However, for a task designed just beyond ChatGPT’s frontier, its use actually lowered performance and accuracy.

These initial results highlight AI’s potential to augment human skills, but also the need to thoughtfully navigate its limitations. As LLMs continue rapidly evolving, understanding where to best apply them will be crucial. The frontier between human and machine strengths is likely to keep shifting. Mastering human-AI collaboration and knowing when AI reaches its breaking point appear critical to unleashing benefits. More research is still required, but these studies provide early insights into AI’s emerging impacts on knowledge work.

There are a number of different ways to use AI to improve human performance. One common approach is to use AI to automate tasks that are currently performed by humans. This can free up humans to focus on more complex and creative tasks.

Another approach is to use AI to augment human capabilities. For example, AI can be used to provide humans with real-time feedback and assistance. This can help humans to improve their performance on a variety of tasks.

Are higher or lower performers helped more?

The impact of AI on human performance can vary depending on the individual’s skill level. In general, AI can help both high and low performers to improve their performance. However, the magnitude of the improvement may be greater for low performers.

Recent studies show some interesting findings regarding how AI impacts workers of different skill levels:

So there are somewhat contradictory results on whether top versus average performers gain more from AI collaboration. The workflow study indicates AI particularly helps raise lower performers closer to top levels. But extreme creativity may be harder to augment. More research is needed, but it appears AI can enhance skills across ability levels in many real-world tasks, while truly exceptional creativity remains more uniquely human. Understanding how to best combine strengths of both humans and AI across the skills spectrum is an important area for ongoing study.

What is the best strategy?

The best strategy for using AI to improve human performance will vary depending on the specific task and the individual’s skill level. There are a few key strategies emerging for how humans can best collaborate with AI tools like large language models:

The optimal strategy likely utilizes elements of all approaches. Key factors include assessing current AI capabilities, mapping workflows, and matching AI tools to suitable tasks. Continuously training both humans and AI will further enhance collaboration. But fundamentally, combining human ingenuity and judgement with AI’s scale and speed offers the greatest potential.