Study Highlights Strengths and Weaknesses of Generative AI

A recent study conducted by Boston Consulting Group assessed their consultants’ use of Chat GPT-4 through a structured study designed to simulate actual client work. Managed by leading academics, the study employed 750 consultants divided into test groups who used AI and control groups who did not.

What was found is that those who leveraged generative AI on creative tasks (i.e. generating new product ideas and go-to-market plans) had superior results to those who worked unaided. However, consultants who used generative AI on problem solving tasks (i.e. identify the root cause of a company’s challenges based on performance data and interviews with executives) fared significantly worse that those who used no AI at all.

One might write this impact down to the fact that generative AI is not optimized for problem solving, and fair enough. But the lesson remains: Pick your AI wisely, in the same way that you would pick a colleague to assist you on a task. You want one (whether AI or colleague) who is skilled at the kind of task you are facing. Of course, with hundreds of AI start-ups generating thousands of differentiated solutions there is an ever-increasing range of options from which to choose. Which begs the question: just which AI can help me do that choosing?

(Source study summary at: https://lnkd.in/gaeeHUw7)
[Image generated by DALI-E2. Text generated by human.]

Bill Haines, Partner