 |
The difficulty AI faces in creating analogies for complicated matters, a skill humans possess, raises concerns about the risks of using AI in critical real-world choices.
Artificial intelligence (AI) has made remarkable strides in recent years, from generating human-like text to diagnosing diseases. Yet, a new study has revealed fundamental differences between human and AI thinking—differences that could have major implications for how we deploy AI in critical areas like law, governance, and decision-making.
The Study: Where AI Falls Short
A recent study published in Transactions on Machine Learning Research (Feb. 2025) examined how well large language models (LLMs) form analogies—a crucial aspect of human reasoning. The researchers tested AI models on both simple letter-string analogies and complex matrix problems, which required identifying missing digits. Humans excelled in these tasks, but AI struggled, especially when presented with novel patterns it had never seen before.
One of the most revealing findings came from story-based analogy problems. AI models exhibited answer-order effects, meaning their responses varied based on the order in which they received information. Additionally, they often paraphrased rather than forming true analogies, showing a lack of deep conceptual understanding.
Martha Lewis, an assistant professor of neurosymbolic AI at the University of Amsterdam and co-author of the study, highlighted a key example. When given the analogy:
If ‘abbcd’ transforms into ‘abcd,’ what does ‘ijkkl’ transform into?
Humans tend to answer ‘ijkl’ by recognizing the pattern of removing duplicate elements. However, GPT-4 and similar models often failed at such tasks, indicating they struggle with abstract reasoning beyond mere pattern recognition.
Why AI’s Weakness in Analogical Reasoning Matters
Humans naturally generalize from patterns to create broader rules, a capability AI lacks. As Lewis explained, AI is excellent at identifying and matching patterns but cannot generalize beyond them effectively. This means that even when AI is trained on vast datasets, its ability to understand and apply concepts remains limited.
This limitation has real-world consequences. For example, AI is increasingly used in legal research, case law analysis, and even sentencing recommendations. If an AI system cannot effectively recognize how legal precedents apply to slightly different cases, it may fail in high-stakes decision-making. A misinterpretation of the law due to AI's weak analogy-forming ability could lead to unjust outcomes.
Moreover, AI’s reliance on pattern recognition rather than true reasoning affects its ability to innovate. Creative problem-solving and scientific discovery often require seeing connections between seemingly unrelated concepts—something AI struggles with. This suggests that while AI can assist in research, it cannot yet replace human intuition and creative reasoning.
The Road Ahead: Ensuring AI’s Responsible Use
The study’s findings reinforce the need for caution when deploying AI in critical areas. If AI is to play a role in legal, medical, or policy decisions, developers must go beyond accuracy and assess the robustness of AI’s cognitive capabilities. Future AI systems should focus on improving generalization and reasoning, not just pattern recognition.
For now, the human brain remains unparalleled in its ability to form abstract connections, think creatively, and apply knowledge flexibly. AI, while powerful, is still far from matching our ability to reason. As we integrate AI into more aspects of life, we must remember that it is a tool—not a replacement—for human intelligence.
What do you think? Should AI be used in areas like law and governance despite these limitations? Share your thoughts in the comments!