The bot then spawned an even bigger Waldo.
Subbarao Kambhampati, a professor and AI researcher at Arizona State University, also put the chatbot through some testing and said he didn’t notice any notable improvements in reasoning compared to the latest version.
He presented ChatGPT with a puzzle involving blocks:
The answer is that it is impossible to arrange the blocks under these conditions, but, just as with previous versions, ChatGPT-4o consistently found a solution that involved moving block C. With this and other reasoning tests, ChatGPT was occasionally able to take feedback to get the right answer, which is antithetical to how AI is supposed to work, Kambhampati said.
“You can correct it, but when you do that you’re using your own intelligence,” he said.
OpenAI pointed to test results that showed GPT-4o scored about two percentage points better at answering general knowledge questions than previous versions of ChatGPT, demonstrating that its reasoning abilities had improved slightly.