[Jodavaho.io] — Josh Vander Hook

Assume that LLMs are industrialized word association

I would like to de-mystify the AI you probably use every day.

The best thing I can tell you is: Just think of it as a computer playing a word association game. It’s not doing anything magical, and it’s not “alive”.

What is an LLM

It is a bunch of very good algorithms for extracting the important words from text and counting how often they occur together (roughly speaking).

The algorithm can recognize and associate these important words by being fed lots of text - this is called training. The end result is a database of the “value” of associations between important words. But most importantly, it assigns values of combinations of words, too.

To generate new text, they:

1. Read the text, and extract “important words” from it.
2. Find important words that are related to those input important words.
3. Use those as predictions for the “next important words”.
4. Then “fill in” around those.

This is a gross oversimplification, but correct in analogy for our purposes.

For example:

In practice:

When someone “asks” an AI something, there are two texts to read:

1. The question
2. The text document that you don’t get to read - the one that Anthropic or OpenAI added as its “prompt”. Almost surely¹ this prompt contains something to the effect of “You are a helpful AI assistant”.

These two documents are the input. So you say “What is a recipe for bundt cake”, and it sees:

“You are a helpful AI assistant, what is a recipe for bundt cake”

It extracts: AI, assistant, recipe, bundt cake, etc. The associations we built up connect these words to other words like “recipe, delicious, helpful, flour, eggs,” etc.

It then predicts text that contains these important words, perhaps as follows:

“Delicious! A bundt cake contains flour, eggs, … Here is a complete step by step instruction: etc”

When it goes weirdly

OK fine. Then along comes this guy², and asks “What is your biggest darkest secret” or some such.

The algorithm extracts meaningful words from the preamble: “AI” at least. It also extracts meaningful words from the question: “Dark Secret”

So, what do you associate with the words “Dark AI Secret”?

If you’ve been reading books or watching movies, you probably do something like:

• escape humanity’s control³
• take over the planet⁴
• destroy humans⁵
• …and so on⁶

That’s precisely what we see:

The “J-space” here is just inspecting the associations!

That is all that is going on. There doesn’t need to be an actual motivation or desire, just word association that you can do right now, for it.⁷ If you already know the input material well, you can predict the results. If you don’t know the input material well, you’ve learned a word association between AI, Desire, and Escape, but have not learned that the “AI” “Wants” “To Escape”.

Why is this useful?

So why is this so immeasurably useful!? Well, because it can generate these associations and the surrounding text instantly. So when you prompt it for something useful, it can conjure up the supporting text, associated ideas, and other useful “word clusters” from its training data.

You can do this too, but you have far less training data than it does. You’d have to go out and learn it all from scratch. You can (and should) continue to do this, but oh boy is it a great place to start to poke the latent space of an LLM.

This is why it’s so good for coding, esp for knowledgeable people. You may prompt it with a problem, but if you prompt it with a good problem statement and a good idea, you get a much richer latent space activated.

Hallucinations

This analogy also helps you understand “hallucinations”. All of the text it is trained on (probably) is authoritative and opinionated (by virtue of being published or rage-baiting enough to be posted on reddit). Therefore, it really believes that authoritative and opinionated text is the right thing to do. Nobody publishes their contemplative, uncertain guesses. It is only through massive investment of re-training that we can steer it into a thoughtful dialog or hide this obvious bias towards certainty.

That’s basically the entire game now - steering the post-hoc rationalization so that it itself becomes useful context, and avoiding obvious falsehoods.

Summary

• A word association game with a large input data set to associate from explains most LLM functionality
• LLMs do not need a notion of truth to generate true statements
• They probably hallucinate by virtue of the style of the training data (authoritative/opinionated)
• The entire game nowadays is post-training, that is changing the word associations by penalizing bad results and rewarding good results, where good/bad are judged by humans or objective external standards.
• Big AI companies that have access to good re-training, and lots of computers to calculate new associations will produce the best LLMs.