the principle of elimintation

 I'm not an AI engineer and this isn't a verified content as it's a personal view of LLMs.

today I want to talk about a pattern I noticed when working with LLM models, which I called "the principle of elimination" because it's how it works, the best way to work with LLMs isn't to tell it how to find the right answer, but to eliminate each possible wrong answer, or rather, each possible "search space"


imagine we have a small, red ball in a HUGE empty space, and this ball will automatically move, but we don't know how fast it'll move, to any direction and how many meters it'll move.. now can you give me the number of possibilities of its position after it moves?
it's essentially infinite!
the ball can move to left by 1 meter or by 2 meters, or move down by 3 or 4 meters, etc..

this is so unpredictable, how we reduce this?
by specifying some factors to reduce the combinatorial possibilities...

for example, let's say we certainly know the ball will move to left but we don't know how fast and how many meters it'll move!

now what is the number of possibilities?
well, it's still practically infinite but we narrowed down the space we'll search in to predict its position!
we'll only search in the left side of the ball!

let's specify another factor, let's say it'll move to left by exactly 5 meters!

now, can you know its position?
yes!
it'll be 5 meters left to its current position!
we needed 2 factors to be known to solve this problem!

but, what if I asked you how many seconds it'll take the ball to go to the next position?
we can't answer by only knowing the direction and distance!
we need another factor, its speed!

let's specify it's speed to be 5 m/s, meaning it moves 5 meters each second

now how many seconds it'll take it to move to the next position?
obviously, it's 1 second!

how many factors we needed to know this answer?
also 2, the distance it'll move and its speed

well after this imaginary example, let's go back to LLMs.. how to get the best of LLMs?
LLMs are more like us trying to predict the ball position but it rather tries to predict the right answer you look for, if you gave it an empty prompt, you essentially told it to predict the right answer you look for with no factors specified at all (it's not practically an infinite search space but it's exponentially vast since they're trained on the whole data of the internet..)!

how it can do such a thing?

it'll practically give the wrong answer!

what if you told it:
"make a simple calculator in python"

you dramatically reduced the search space, now it knows a lot about what you need to do, but still, it doesn't know how you specifically need it, you gave it an abstract prompt!
do you need the calculator to take the first number, operator and second number as separate inputs or do you want it to parse a string expression?
what operations do you want to support?
is it a simple project for a college assignment or an emulator for a scientific calculator?
each question of those changes the code a lot, and unfortunately LLMs aren't trained to ask such questions as they're trained to produce "useful" and "helpful" answers, only an experienced engineer can answer such questions!


now imagine you told the LLM:
"make a python implementation for a simple calculator that takes 3 inputs from the user, the first operand, the operator and the second operand and perform the operation corresponding to the operator input and it only support basic arithmetic operations (addition, subtraction, multiplication, division) with zero division error handling"

this is a way longer prompt, but I can guarantee that if you gave this prompt to different LLMs, they'll give very similar answers.. because you narrowed down the search space for the right answer to few possibilities that only differ by variable names, function names, etc..

but the logic itself is now obvious to it!

the good engineer is the one who knows the factors that affect the design and implementation of the project he is requesting from LLMs and he can give it a specific values for these factors so it doesn't guess or imagine the user's intent.