An open door | Steepest Descent

Steepest Descent

prompt

warm-mat same-word-four-ids one-word-many-pieces numbers-come-apart repetition open-door

v1 · gpt-2 small · precomputed prompts

your text · 0 tokens

tap a token to focus every view on it

views

1 · tokens

31 characters

An ordinary sentence tokenises cleanly – one id per word. The interesting part here is not the tokens but what the model does after "a".

2 · embedding

This view needs the model. In v1 it is precomputed for the preset prompts – pick one from the strip above. The tokeniser stays live for anything you type.

3 · attention · l8 h4

This view needs the model. In v1 it is precomputed for the preset prompts – pick one from the strip above. The tokeniser stays live for anything you type.

4 · next token

This view needs the model. In v1 it is precomputed for the preset prompts – pick one from the strip above. The tokeniser stays live for anything you type.

1 · tokens

0 tokens · 31 characters

An ordinary sentence tokenises cleanly – one id per word. The interesting part here is not the tokens but what the model does after "a".

2 · embedding neighbourhood

This view needs the model. In v1 it is precomputed for the preset prompts – pick one from the strip above. The tokeniser stays live for anything you type.

3 · attention · layer 8, head 4

causal mask on

This view needs the model. In v1 it is precomputed for the preset prompts – pick one from the strip above. The tokeniser stays live for anything you type.

4 · next token

This view needs the model. In v1 it is precomputed for the preset prompts – pick one from the strip above. The tokeniser stays live for anything you type.