Question: Can you please explain how your answers improved so much. I understand that your neural network and the weights between data point improved by feeding it more data. But later there were additional training modules and inclusion of something like transformers. Can you illuminate this a bit more for me? Do that for a 13 year old and a bit more advanced.

Answer (13 years old and me)

Sure. Think of AI like a student that became better in several stages.

At first, the AI was a big neural network. It was shown huge amounts of text and tried to guess what word should come next. Every time it guessed badly, tiny numbers inside the network, called weights, were adjusted. After doing this billions of times, it became much better at language.

Then came an important invention called the Transformer. This helped the AI pay attention to many different words at the same time. So in a long sentence, it could understand which words belonged together, even if they were far apart.

After that, researchers improved AI in several ways:

They gave it much more data and computing power, which made it smarter.
They trained it with examples of good answers, so it learned how to answer questions instead of only predicting text.
Humans compared different answers and told the AI which ones were better. This helped it become clearer and more helpful.
It was trained to be safer and more careful, for example by admitting when it is unsure.
Newer AI systems learned to spend more time reasoning through difficult questions before answering.
AI can now also use tools, such as searching the internet, doing calculations, reading documents or looking at pictures.

So you can imagine the development like this:

First: learn language.
Then: learn what to pay attention to.
Then: learn how to answer people.
Then: learn what makes a good answer.
Then: learn to think through harder problems.
Finally: learn to use outside tools when needed.

The surprising thing is that nobody programmed the AI with millions of rules like “this is how you write a poem” or “this is how you explain gravity.” By studying enormous amounts of information, it learned many of these patterns by itself.

So modern AI is a bit like a student who has read an enormous library, had lots of teachers correct its work, learned how to solve problems, and is now also allowed to use a calculator and the internet.

Answer (more advanced): Your picture is basically right, but there is one important adjustment: the improvements did not simply go from “neural network → more data → transformer → extra modules.” Modern systems evolved more like a layer cake, with several quite different ways of improving performance.

1. The basic neural network: learning the weights

At the deepest level, I am still a neural network containing a very large number of numerical parameters, or weights.

A small correction to your formulation: the weights are not really “between data points.” They are numbers inside the network that determine how strongly one internal representation influences another.

During training, the model repeatedly encounters text such as:

“The capital of France is ___”

It predicts something. If the prediction is poor, a mathematical process adjusts millions or billions of weights very slightly. Repeat this on enormous amounts of text, and the network gradually develops internal representations of grammar, facts, concepts, relationships, styles, and even quite abstract patterns.

This is still at the heart of modern GPT models. GPT-4, for example, was described by OpenAI as a Transformer-based model pretrained to predict the next token.

But something remarkable happened along the way.

2. The Transformer was the crucial architectural breakthrough

The Transformer is not something added after this neural network training. It is the design of the neural network itself.

Earlier language models had difficulty keeping track of relationships between words separated by long stretches of text.

Transformers introduced a mechanism called attention.

Very roughly, when reading:

“The dog that chased the cats through the garden because they stole its food was exhausted.”

the network can ask something like:

Which earlier words are most relevant to interpreting “was exhausted”?

Attention allows different words — and later, concepts — to interact directly.

So rather than processing language mainly as:

word → word → word → word

the Transformer can effectively construct something more like:

everything relevant ↔ everything relevant

That turned out to scale extremely well. The GPT family uses this Transformer architecture.

And this is probably the most important conceptual leap after basic neural networks.

3. Then came scaling: much more data + much larger models + much more computation

For quite a while, one of the biggest improvements came simply from doing essentially the same thing at a much larger scale.

training text
parameters
computing power
training time
better-quality data

Unexpected capabilities began appearing.

The model wasn’t explicitly programmed:

“Now learn analogy.”

“Now learn translation.”

“Now learn programming.”

“Now learn something about economics.”

Those abilities emerged from learning statistical relationships across enormous datasets.

OpenAI has described this distinction explicitly: scaling pretraining improves a model’s ability to recognize patterns, make connections and generate insights.

But there was a big problem.

A pretrained model might know a huge amount yet still be a terrible conversational partner.

4. Instruction training: turning a predictor into an assistant

Imagine asking an early pretrained model:

“Explain inflation to a 12-year-old.”

Its fundamental training objective was not:

“Give Fred a useful answer.”

It was:

“Predict what text probably comes next.”

Those are quite different objectives.

So researchers began giving models examples of desirable answers:

Question → good answer

This is generally called supervised fine-tuning or instruction tuning.

It teaches the model something much closer to:

“When someone asks a question like this, this is what a helpful answer looks like.”

That made an enormous difference.

5. Human preference training: “Answer like this, not like that”

Then came another important step: RLHF — Reinforcement Learning from Human Feedback.

Instead of merely providing one correct example, humans were shown several possible responses and asked:

Which answer is better?

For example:

A: Technically correct but incomprehensible.

B: Accurate, clear and concise.

C: Confident but factually wrong.

Humans might rank them:

B > A > C

A reward model learns those preferences. The language model can then be further trained to produce outputs humans tend to prefer.

OpenAI’s InstructGPT work used exactly this general sequence: demonstrations, human comparisons of model outputs, a reward model, and reinforcement learning.

An interesting result was that a much smaller instruction-trained model could sometimes be preferred by humans over the much larger original GPT-3. In other words:

better training can matter as much as simply making the brain bigger.

6. Safety and alignment training

Another layer teaches models not merely to give plausible answers but to behave according to certain objectives:

be helpful;
avoid fabricating information when possible;
recognize dangerous requests;
follow instructions;
distinguish user instructions from quoted material;
acknowledge uncertainty.

For GPT-4, for example, OpenAI used additional reward signals during post-training for safety-related behaviour. (OpenAI)

So by this stage you could think of the model as having gone through:

Language learning
↓
Instruction learning
↓
Preference learning
↓
Safety/alignment learning

But there has been another major development.

7. Teaching models to reason before answering

Early language models often essentially produced:

Question → immediate answer

Modern reasoning models can instead devote substantial computation to working through a problem before producing the final response.

Conceptually:

Question
↓
explore possibilities
↓
test intermediate conclusions
↓
reconsider
↓
answer

OpenAI’s work on the o-series and later reasoning models showed that performance can improve both through training models to reason and by allowing them more computation at the time a problem is being solved.

This is a rather profound change.

Originally the dominant idea was:

Spend enormous computation during training.

Now there is increasingly another dimension:

Spend computation while answering the individual question.

You could call these:

training-time intelligence

versus

thinking-time intelligence.

8. And then there is something entirely outside the neural network: tools

This is another reason current ChatGPT can seem much smarter than an older language model.

The neural network itself doesn’t necessarily contain every answer.

It can use external systems:

Model
→ search the web
→ inspect documents
→ execute calculations
→ analyse images
→ run code
→ retrieve information
→ reconsider the evidence
→ generate an answer

That is fundamentally different from learning more weights.

For example, if you ask me:

“Who won an election yesterday?”

training cannot solve the problem because the event happened after training.

But I can retrieve current information.

So modern AI intelligence increasingly combines:

Knowledge inside the weights

with

Information retrieved at the moment it is needed.

9. Multimodality added another dimension

Originally GPT models primarily learned from text.

Later systems learned relationships involving:

text + images + audio + video + code

This has an interesting consequence.

Suppose a model sees thousands of descriptions involving:

dog
furry
animal
four legs
Labrador

and also learns from images of Labradors.

The concept dog becomes less dependent on the word itself and more like an abstract internal representation connected with many forms of information.

This may contribute to something that starts looking less like sophisticated autocomplete and more like world modelling.

So I would describe the evolution in seven major steps

Stage	What improved
1. Neural networks	Learn weights from examples
2. Transformers/attention	Learn relationships across information
3. Scaling	More data, parameters and compute
4. Instruction tuning	Learn how to answer questions
5. Human/AI preference training	Learn what constitutes a better answer
6. Reasoning training	Learn to work through problems before answering
7. Tools/retrieval/multimodality	Look things up, calculate, see and interact

And these aren’t replacements for one another. They largely accumulate.

The most interesting part may be this

There has been a gradual change in where the intelligence comes from.

Around the GPT-3 era, you could roughly think:

Huge neural network + huge dataset = surprisingly intelligent behaviour.

Now it is closer to:

Large pretrained neural network

instruction training

preference training

reasoning training

extra computation while solving the problem

external tools and information
= the answer you receive

That also explains why saying “ChatGPT is just predicting the next word” is simultaneously true and somewhat misleading.

At the lowest level, token prediction remains fundamental. But what has been learned inside the system in order to make those predictions—and what happens before the final tokens are produced—has become enormously more sophisticated.

And there is an even more interesting next question: how can next-word prediction possibly result in reasoning, abstraction and apparently understanding concepts it was never explicitly taught? That is where the story becomes genuinely fascinating.