ChatGPT has a 4k token context window, and some of that might be taken up by scaffolding like the system prompt, autosummary of conversation outside of context window, etc.
If you continue the conversation, the gibberish will eventually leave the context window and then no longer directly influence the generations.
