The repetition game | Steepest Descent

Steepest Descent

prompt

warm-mat same-word-four-ids one-word-many-pieces numbers-come-apart repetition open-door

v1 · gpt-2 small · precomputed prompts

your text · 0 tokens

tap a token to focus every view on it

views

1 · tokens

39 characters

Eight tokens, three distinct words. Note the first "red" has no leading space and gets a different id from the later ones.

2 · embedding

This view needs the model. In v1 it is precomputed for the preset prompts – pick one from the strip above. The tokeniser stays live for anything you type.

3 · attention · l7 h10

This view needs the model. In v1 it is precomputed for the preset prompts – pick one from the strip above. The tokeniser stays live for anything you type.

4 · next token

This view needs the model. In v1 it is precomputed for the preset prompts – pick one from the strip above. The tokeniser stays live for anything you type.

1 · tokens

0 tokens · 39 characters

Eight tokens, three distinct words. Note the first "red" has no leading space and gets a different id from the later ones.

2 · embedding neighbourhood

This view needs the model. In v1 it is precomputed for the preset prompts – pick one from the strip above. The tokeniser stays live for anything you type.

3 · attention · layer 7, head 10

causal mask on

This view needs the model. In v1 it is precomputed for the preset prompts – pick one from the strip above. The tokeniser stays live for anything you type.

4 · next token

This view needs the model. In v1 it is precomputed for the preset prompts – pick one from the strip above. The tokeniser stays live for anything you type.