the principle of elimination | part 2

this post will not add new information or discuss something but rather will try to clear a misconception that may happen from the previous post (here).

some people may understand that I encourage longer prompts, or that I'm saying "longer prompts lead to better results from LLMs"... this is wrong, longer result doesn't ensure better results, clear prompts do!

let's go back to our small, red ball from the previous post.. we asked 3 questions to predict 2 things.. we asked:

1- how many meters will the ball move (the distance)?
2- where it'll move (the direction)?
3- how many meters it'll move per second (the velocity)?

we wanted to predict its position after it moves and how many second it'd take it to finish the movement..

does this mean the 3 questions were the same importance to predict the 2 results?
not exactly!

let's take 2 examples:

1- I asked you to predict the position of the ball after the movement and gave you one information: the ball will move to left

2- I asked you to predict the position of the ball after the movement and gave you one information: the ball will move 5 meters

the examples are pretty similar except for the information I gave to you, despite that.. they're totally different!

in the first one, the information isn't that important, without it, the search space is infinite, and still it's infinite with this information.. it's not useless but it's not the core information required to predict ball's position!

in the second one.. this single information narrowed down the search space to ONLY 4 possibilities:

1- ball's position will be 5 meters to the left of its current position.

2- ball's position will be 5 meters to the right of its current position.

3- ball's position will be 5 meters to the down of its current position.

4- ball's position will be 5 meters to the up of its current position.

can you see the magic?
one information didn't do that much and another increased the possibility of predicting the right position from practically 0% to 25%!

now let's try to predict the time it'll need to finish the movement with one information each time:
1- its velocity is 5 m/s.

2- it'll move 5 meters

try to predict the time only using 1 information of those 2... well this time it's harder.. it seems like both of the information have "equivalent" importance, we need both of them to answer the question or we'll guess..

how this maps to LLMs?
LLMs have the so-called "attention mechanism", I'll not explain it because I don't understand it but it's proposed by google in "attention is all you need" paper.. it's the mechanism LLMs use to "pay attention" to the most important info that matter the most to predict the next token!

so, it's not about giving longer prompts or more information.. it's about engineering a prompt that gives all information that actually matters!

the magic of LLMs is its attention mechanism, and this is exactly it's bottleneck!
because it scales quadratic-ally (it processes each token compared to all other token).. and this is why we don't have an infinite context window!
because each new token added to the capacity of the context windows requires huge computation power to be added, which is currently limited..