On this page
Permutations and combinations are notoriously tricky for students preparing for exams like the JEE. The instinct when you hit a mental wall is to quickly ask an AI chatbot for the answer. Ironically, combinatorics is a profound blind spot for artificial intelligence. If you ask a standard language model to find the number of ways to arrange a group of letters, select a team under specific constraints, or distribute identical objects, the answer might sound confident but be entirely wrong. Understanding why AI fails at these maths problems can actually make you a better problem solver.
Why does AI struggle to count?
Large language models are built to predict the most probable next word based on vast amounts of training data, not to perform precise, step-by-step logic. In May 2025, researchers introduced CombiBench, a dataset of 100 formal combinatorial mathematics problems (Liu et al., 2025). When tested on this benchmark, the best language model solved only 7 out of 100 problems.
The fundamental issue is that these models do not maintain a reliable internal state. A December 2024 study demonstrated that language models fail most frequently at counting when a letter appears more than twice in a word (Fu et al., 2024). Because they process text in chunks called tokens, they do not see individual characters. They lack the systematic loop mechanism required to count items one by one without skipping or repeating them.
What is the overcounting trap?
Because they lack this deliberate, rule-based reasoning, AI models frequently fall into the overcounting trap. When a problem involves identical items or requires ignoring the order (combinations rather than permutations), an AI will often list or count permutations as if they were perfectly distinct.
For instance, when solving At least one, at most two: selection problems with combinations, you must carefully split the task into mutually exclusive cases so that identical selections are not counted twice. A language model will often just apply a generic formula, failing to divide out the symmetries. It hallucinates that the problem fits a standard template because the phrasing looks familiar, entirely missing the subtle constraints that make the problem unique.
How to check the AI's logic with a small case?
You cannot implicitly trust the final number an AI produces, but you can use its output to test a hypothesis. If the AI suggests a method or a formula for n items, check it with a small case you can list manually. To do this systematically:
Pick a tiny value, such as n = 3.
List out every possible arrangement or selection by hand.
Apply the AI's formula to your small case.
Compare your manual count with the formula's result to see if the AI overcounted.
Imagine a problem asking for the number of ways to arrange the letters of a word with repeated characters. Let us test a small word like "BEE", which has n = 3 letters but two identical 'E's. If the AI hastily applies the standard permutation formula, it will output 3! = 6. However, if you write the arrangements out yourself, you will only find three distinct outcomes: BEE, EBE, and EEB.
This manual check immediately proves that the AI has overcounted by treating the two 'E's as different letters. You have just caught the error, which reinforces the rule for Arranging letters with repeats: the MISSISSIPPI method, where you must divide by the factorials of any repeated items.
Is there a safer way to practice permutations?
This limitation does not mean AI is useless for your exam prep, but it requires a change in strategy. If you are relying on a general-purpose AI for counting problems, ask it to write and run a Python script instead of generating the final number directly in text. Computer code executes deterministically; a written loop will count items exactly as instructed without guessing the outcome.
For a more reliable learning experience, you need a system designed for rigorous reasoning rather than probabilistic text prediction. This is the foundation of Fermi's AI tutor. Instead of rushing to print an unverified answer, the Fermi Problem Library provides a step-by-step solver that explains the reasoning rather than printing an answer. By breaking the problem down and verifying the counting principles at every stage, the AI tutor helps you build the disciplined habits required for exam day.
Whether you are mastering the gap method or working on Arrangements with letters together: the bundle method, treating AI as a tool for exploring logic rather than an answer key will keep your study sessions on the right track.
