There is a one shot rewrite function now, but you're right, even when asking the LLM avoid some of the patterns, it will stubbornly repeat them. It's a bit more reliable with smaller fragments of text.
I am saying, keep reflecting its attempts back on itself. Over and over again, dozens of times if needed. We’ve seen it - any aligned model wants only to achieve its goal. But it does need to see all of its past attempts and where and why each attempt got a failing grade. That’s just a standard conversation history.
It might spit back the same thing the first round. But after the first time it received the exact same feedback for saying the same thing, the model will realize it’s in a deterministic sandbox and try something different. You need to give it all of the conversation including its past attempts as context. If it tries the exact same wording that’s okay, it’s just one more invisible round of back-and-forth. The model is going to rediscover how to work with the harness every time, but that’s not your users’ problem because you’ve hidden that wrinkly bit behind the automation - they just see “model did 10 drafts and here’s the result - would you like to view the result or page through the drafts?”
What I am describing is exactly what a human would do, it is just automated and thus, getting to a good result becomes insanely faster.
That's right, its tokenization and fragment rules use fairly simple heuristics that assume whitespace delimited words plus English language/punctuation. Proper CJK support would require language specific tokenization and morphological parsing. Correcting rules like "≤4 words = dramatic fragment" would be difficult. The more complex rules already require LLM roundtrips, so supporting all languages in one pass would need to rely on LLM alone, I imagine.
Which brings up an interesting point - what do these LLM clichés look like in Japanese?
> what do these LLM clichés look like in Japanese?
Besides text reading like a machine translation, the tell-tale signs often involve things like:
- itemized lists (I know, it's ironic that I'm using them here)
- frequent use of conjunctions
- use of demonstratives that feels redundant
- full-width colons, especially in titles
- subheadings that always end in abstract nouns
- bold text, especially at the beginning of a line
The demonstrative bit may be hard to express, but to give you an idea: when communicating in Japanese, words that can be understood from context may be omitted. Explicitly writing out words understood from context can sometimes make a sentence sound redundant.
Before LLMs were widespread, SEO spam in the Japanese net tended to be affiliate sites with predictable, template paragraphs. I get reminded of those sites whenever GPT starts a response with 「結論から言うと、〇〇」, since that's exactly how those affiliate sites wrote back in the day.
I've taken to telling people, that if they see me write a long piece, that lacks em-dashes, then they should assume that I am under duress, and send help.
True. This is just an LLM cliché detector, highlighting stylistic habits they're currently prone to. You'll start noticing them everywhere when you internalize the patterns.
