Unit 12: Words

A network cannot read

Everything in this track eats numbers. Text is not numbers, so before anything else a sentence has to be chopped into pieces and each piece given a number.

The pieces are called tokens, and the list of all the different ones is the vocabulary.

Python

text = "the cat sat on the mat"
words = text.split()

print(words)
print(len(words), "tokens")
print(sorted(set(words)))

It prints

['the', 'cat', 'sat', 'on', 'the', 'mat']
6 tokens
['cat', 'mat', 'on', 'sat', 'the']

Every token gets a number

Sort the vocabulary and hand out numbers in order. Now a sentence is a list of numbers, which is something a network can be fed.

The numbers mean nothing by themselves — cat being 1 does not make it half of on. They are name tags, not measurements.

Python

text = "the cat sat on the mat"
words = text.split()
vocab = sorted(set(words))

ids = {}
for i in range(len(vocab)):
    ids[vocab[i]] = i

print(ids)
print([ids[w] for w in words])

It prints

{'cat': 0, 'mat': 1, 'on': 2, 'sat': 3, 'the': 4}
[4, 0, 3, 2, 4, 1]

Real models do not split on spaces

They chop into word *pieces*, so "unhappiness" might become un + happi + ness. Two reasons: a word nobody has ever seen can still be spelled out of pieces, and a vocabulary of pieces stays small where a vocabulary of every word in English would not.

It is the same idea — text in, list of numbers out — with cleverer scissors.

The entire job

A language model does one thing: given the text so far, put a number on every token in the vocabulary saying how likely it is to come next.

That is all ChatGPT does. Everything else — answering, explaining, writing a poem — is that one guess, made over and over, with its own answer fed back in.

The smallest possible version: counting

Go through the text and write down, for each word, which words followed it and how often. This is called a bigram model — bi for two, gram for piece.

Python

text = "the cat sat on the mat the cat ate the fat rat the rat ran"
words = text.split()

counts = {}
for i in range(len(words) - 1):
    here = words[i]
    nxt = words[i + 1]
    if here not in counts:
        counts[here] = {}
    counts[here][nxt] = counts[here].get(nxt, 0) + 1

for word in sorted(counts):
    print(word, counts[word])

It prints

ate {'the': 1}
cat {'sat': 1, 'ate': 1}
fat {'rat': 1}
mat {'the': 1}
on {'the': 1}
rat {'the': 1, 'ran': 1}
sat {'on': 1}
the {'cat': 2, 'mat': 1, 'fat': 1, 'rat': 1}

Counts become chances

Five words followed "the" in that text, and two of them were "cat". So the model says there is a 0.4 chance of "cat" next, and 0.2 each for the rest.

Every probability it gives is a share of what it has seen. A word it never saw follow "the" gets a flat zero, however sensible it would be.

Python

after_the = {"cat": 2, "mat": 1, "fat": 1, "rat": 1}
total = sum(after_the.values())

for word in sorted(after_the):
    print(word, round(after_the[word] / total, 2))

It prints

cat 0.4
fat 0.2
mat 0.2
rat 0.2

Feed its own answer back in

Pick a word. Ask what usually follows. Move there. Ask again. This is how every AI writes text — one token at a time, always looking only at what is now in front of it.

Always taking the favourite goes wrong

Take the commonest next word every time and the model falls into a loop almost immediately, because from "the" the answer is always "cat" and from "cat" it is always "ate".

Python

counts = {
    "the": {"cat": 2, "mat": 1, "fat": 1, "rat": 1},
    "cat": {"sat": 1, "ate": 1},
    "ate": {"the": 1},
    "sat": {"on": 1},
    "on": {"the": 1},
}

word = "the"
sentence = [word]
for i in range(9):
    choices = counts[word]
    best = sorted(choices)[0]
    for w in sorted(choices):
        if choices[w] > choices[best]:
            best = w
    word = best
    sentence.append(word)

print(" ".join(sentence))

It prints

the cat ate the cat ate the cat ate the

So pick by chance instead

Put one ticket in a bag for each time a word was seen — "cat" gets two, the others one each — and draw one out. Likelier words win more often without always winning.

Computers have no real dice, so the "random" number here is arithmetic on a seed. Same seed, same sentence, every time.

Python

counts = {
    "the": {"cat": 2, "mat": 1, "fat": 1, "rat": 1},
    "cat": {"sat": 1, "ate": 1},
    "ate": {"the": 1},
    "sat": {"on": 1},
    "on": {"the": 1},
    "mat": {"the": 1},
    "fat": {"rat": 1},
    "rat": {"the": 1, "ran": 1},
}

seed = 7

def rand():
    global seed
    seed = (seed * 75 + 74) % 65537
    return seed

word = "the"
sentence = [word]
for i in range(9):
    if word not in counts:
        break
    bag = []
    for w in sorted(counts[word]):
        for n in range(counts[word][w]):
            bag.append(w)
    word = bag[rand() % len(bag)]
    sentence.append(word)

print(" ".join(sentence))

It prints

the rat the fat rat the fat rat ran

This dial has a name

How much notice to take of the probabilities is called the temperature.

Low temperature means "always take the favourite" — safe, repetitive, and prone to loops. High temperature means "ignore the odds and pick almost anything" — surprising, and quickly nonsense. The chat models you have used sit somewhere in between, and it is a setting somebody chose.

Look at what is actually stored

Print the model. It is a table of which word followed which word, and how often. There is no fact in there. Nothing in it knows that a cat is an animal, that mats are on floors, or that anything in the sentence is true.

A huge language model is bigger and cleverer at this — it looks at thousands of previous tokens instead of one, and stores patterns rather than raw counts — but it is still a machine for continuing text plausibly.

Ask it something it never saw

Our model was trained on cats and rats. Ask it to continue "the fat" and it will answer, confidently and instantly, because continuing is the only thing it can do.

Python

counts = {
    "the": {"cat": 2, "mat": 1, "fat": 1, "rat": 1},
    "fat": {"rat": 1},
    "rat": {"the": 1, "ran": 1},
}

print(counts["fat"])
print("does it know what fat means?", "meaning" in counts)

It prints

{'rat': 1}
does it know what fat means? False

That is what a "hallucination" is

When a chatbot invents a book that does not exist, or a date that is wrong, it has not malfunctioned and it is not lying — lying needs knowing. It produced the continuation that fitted the pattern best, exactly as it does when it is right.

Which is why the answers come out equally confident either way. The model has no way of telling the two apart, and neither does its tone.