This article delves into GPT’s position within the Chomsky hierarchy, revealing the fundamental limitations of its computational capabilities. Through rigorous mathematical analysis, the author demonstrates that even with an infinite context window, GPT cannot achieve Turing completeness because its finite vocabulary creates a bounded embedding space, causing the model to stop outputting with probability 1 within finite steps. This limited expressiveness is actually what makes GPT easy to train, but it also restricts its ability to learn universal algorithms. The article suggests that this architectural difference may be the root cause of diverging expectations about AI capabilities and hints at potential directions for future AI development. This analysis holds significant importance for understanding the essential limitations of current large language models.
