Part 11
What it cannot do
Everything on this page follows from how the model is built. These are not bugs waiting to be fixed — they are what a next-token predictor is.
It does not remember you
Every request starts from nothing. The model has no memory between messages, and no notion that it spoke to you five minutes ago. A conversation only feels continuous because the entire history is re-sent with every message and re-read from scratch each time.
Which means: close the tab, and it is gone. Nothing you said is known to the model afterwards. Anything it appears to remember about you was in the text it was just handed.
Everything shares 2,048 tokens
Roughly 1,500 English words for the instructions, the whole conversation, and the reply combined. Drag to see what happens as a chat grows.
It invents things, fluently
The model was trained to produce text that looks like a good continuation. Nothing in that objective rewards being right. A plausible-sounding false statement and a true one are, as far as training is concerned, the same kind of success — both look like reasonable text.
This is why a made-up answer arrives with exactly the same confidence as a correct one. The model has no separate sense of “I know this” versus “I am filling in a shape”. At 100 million parameters, with two billion tokens of training, there is far less room for reliable facts than in a model a thousand times the size. WillMe will be wrong often.
It also cannot do arithmetic reliably, look anything up, tell you today's date, or know what it does not know. There is no calculator, no search, no clock, and no self-check — only the next token.
A fair way to read its answers
Treat WillMe's output as a guess about what a plausible reply looks like, not as a claim about the world. It is genuinely interesting for style, tone and conversation — and it is not a source. If a fact matters, check it somewhere that actually knows.
Same question, same answer
By default WillMe always takes its highest-scoring token, so asking the same thing twice gives you the same reply word for word. That is useful — it makes behaviour reproducible and comparisons fair — but it is worth being precise about what it means. Reproducible is not correct. A wrong answer is simply wrong the same way every time.
None of this makes the model uninteresting. It makes it a small, honest, well-understood machine that does one thing. If you have read all eleven parts, you now know essentially everything about what happens when you press send.
Statelessness and the KV cache
Each invocation is stateless unless the caller re-sends the earlier
messages. During generation the model keeps a KV cache — the
rotated keys and values for every position, [B, 4, T, 64] per
layer across 12 layers — so it does not recompute the whole prefix
for each new token.
That cache is an inference speed-up, not memory. It holds only the current prompt and what has been generated so far, and it is discarded when the request ends. The model can know something about you only if it is in the current context, or was statistically absorbed into its weights during training.
The 2,048-token limit is a hard processing budget shared by the system message, the conversation history, the current message, the role markers themselves, and every token being generated.