# Study by Apple AI Scientists Reveals LLMs Lack Reasoning Abilities! 📊

**URL:** <https://onehack.st/t/study-by-apple-ai-scientists-reveals-llms-lack-reasoning-abilities/293591>\
**Category:** News & Articles\
**Tags:** technology\
**Created:** [October 14, 2024, 4:01pm UTC](https://onehack.st/t/study-by-apple-ai-scientists-reveals-llms-lack-reasoning-abilities/293591 "2024-10-14T16:01:07Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![TheJoker](https://onehack.st/user_avatar/onehack.st/thejoker/32/22660_2.png) [@TheJoker](https://onehack.st/u/TheJoker)\
**Post date:** [October 14, 2024, 4:01pm UTC](https://onehack.st/t/study-by-apple-ai-scientists-reveals-llms-lack-reasoning-abilities/293591/1 "2024-10-14T16:01:08Z")

</div>

**Summary:**

1. **LLMs and Reasoning Deficiency**  
A recent study from Apple’s AI team found that large language models (LLMs) from companies like **Meta** and **OpenAI** lack basic reasoning skills.

2. **New Benchmark Proposal**  
The researchers introduced a new benchmark called **GSM-Symbolic** to evaluate the reasoning capabilities of various LLMs.

3. **Inconsistent Answers**  
Initial tests showed that minor changes in query wording led to vastly different answers, raising concerns about the models’ reliability.

4. **Fragility in Mathematical Reasoning**  
The study highlighted that adding contextual information to math questions could decrease accuracy by up to **65%** , indicating the models’ inability to reason effectively.

5. **Pattern Matching Explained**  
The researchers concluded that LLM behavior is better described as **sophisticated pattern matching** , which is sensitive to changes in input, affecting output consistency.

**Read more at:** [Apple Insider](https://appleinsider.com/articles/24/10/12/apples-study-proves-that-llm-based-ai-models-are-flawed-because-they-cannot-reason?utm_medium=rss) | [ArXiv Paper](https://arxiv.org/pdf/2410.05229)