I try every major model as it is released and run it inside real projects. My repeated impression: GLM and Kimi solve problems from a library of known patterns, while OpenAI and Anthropic models explore alternatives when the first path fails. On simple tasks the difference is invisible. On complex tasks it decides which model finishes the work.
An impression is not a measurement. So I went looking for what other builders report and which tests capture this dimension. As of July 20, 2026, the tests exist, they have names, and most of them support the pattern — with two exceptions where Kimi beats everyone, which I cover below.
The short verdict
What I keep seeing in real projects
The pattern shows up when a task has no template. Ask any frontier model to scaffold a CRUD endpoint and they all produce roughly the same competent result. Ask for a migration strategy across two live systems with conflicting constraints, and the behavior splits.
GLM and Kimi tend to pick a canonical solution shape early and defend it. When the approach hits a wall, they often retry the same shape with small variations. Claude and GPT models more often step back, question the framing, and propose a second architecture I had not specified. That second proposal is frequently the one that ships.
I noticed the same thing in my Kimi K3 benchmark review→: K3 reached the frontier group on aggregate scores, yet the gap to Claude Fable 5 widened precisely on the hardest long-horizon knowledge work.
What other builders report
Fabio Akita runs one of the more useful independent tests I know: every major model builds the same ChatGPT-style Rails 8 app end to end, scored across eight dimensions. His July 17 Kimi K3 test gave K3 89/100 — a strong result, roughly replacing last-generation Opus 4.6 at a lower price. Opus 4.8 scored 95, and the difference was not raw correctness. K3 skipped preflight credential validation and shipped a smaller test suite; Opus added preconditions and defensive layers nobody asked for. His one-line diagnosis matches my experience: Claude keeps the edge "when the project demands defenses against details nobody puts in the prompt."
His June test of GLM 5.2 points the same direction with a twist: GLM 5.2 jumped 41 points over GLM 5.1, the biggest single-family improvement he has measured, with the cleanest dependency injection in the field. The open models are closing the gap fast. His conclusion still stands, though: "serious programming work, today, is done with Claude Opus 4.8 or GPT 5.5."
On the writing side, Adam Holter's "great agent, mediocre writer" review of Kimi K2 Thinking reads like a description of the constraint pattern: the model chains 200 to 300 tool calls without losing the plot, but on technical prose past 3,000 words it drifts and repeats. An agent brain, not a blog writer — another way of saying it executes known procedures better than it generates fresh structure.
Which tests measure creativity and exploration?
"Feels more creative" sounds unfalsifiable. It is not. Four families of tests target exactly this dimension:
| Test | What it measures | Current signal |
|---|---|---|
| --- | --- | --- |
| SWE-bench Verified vs Pro / SWE-Rebench | Memorized patterns vs fresh problem-solving | Open-weight models drop furthest on protected tasks |
| Pass@1 vs pass@k | Single-path accuracy vs solution diversity | RLVR-tuned models can fall below their own base model at high k |
| NoveltyBench | Distinct, useful answers to open-ended prompts | Every frontier model scores below humans; larger models are often less diverse |
| EQ-Bench Creative Writing v3 | Judged writing quality and craft | Claude Fable 5 leads at 2,230 Elo, Opus 4.7 at 2,203, GPT-5.5 at 2,029 |
Two of these deserve a closer look.
The contamination gap: same models, fresh tasks
SWE-bench Verified was the standard coding benchmark until OpenAI stopped reporting it in February 2026, on the grounds that frontier models could reproduce gold patches from training data. All 500 tasks come from public repositories that predate every model's cutoff. A high score can mean strong engineering or strong recall.
Compare the contaminated and protected numbers:
| Model | SWE-bench Verified | SWE-bench Pro |
|---|---|---|
| --- | ---: | ---: |
| Claude Opus 4.8 | — | 69.2% |
| GLM-5.2 | — | 62.1% |
| MiniMax M3 | — | 59.0% |
| Kimi K2.6 | 80.2% | 58.6% |
| GPT-5.2 | 80.0% | — |
| Claude Sonnet 4.6 | 79.6% | — |
| GLM-5 | 77.8% | — |
*Verified scores from BenchLM's July 2026 leaderboard; Pro scores from the SWE-bench Pro leaderboard. The two suites use different task sets, so compare the gaps, not the absolute levels.*
On Verified, four models from four labs sit within 2.4 points — a statistical tie. On Pro, where tasks come from actively maintained repositories without public ground truth, Kimi K2.6 lands more than 20 points below its Verified score and 10 points behind Opus 4.8. SWE-Rebench, which only uses GitHub tasks that post-date model training cutoffs, was built for the same reason, and analyses of its results describe disproportionately large drops for several Chinese models. When the answer cannot be recalled, the pattern library runs out.
The pass@k problem
The research literature gives my impression a mechanism. Reinforcement learning with verifiable rewards — the post-training recipe behind most 2025–2026 reasoning models — reliably improves pass@1, the chance of solving a task on the first try. But studies on diversity collapse in RL with verifiable reward and RLVR overtraining show pass@k stagnating or falling below the base model as k grows: the trained model concentrates probability on a shrinking set of known solution paths. Researchers call it diversity collapse. In daily use it feels like a model that always reaches for the same three tools.
NoveltyBench measures the related failure from the output side: across 1,100 open-ended prompts, every frontier model generates fewer distinct useful answers than a group of humans, and more capable models within a family are often less diverse. Nobody escapes this — which brings me to the honest part.
Where my impression breaks down
Three results cut against my thesis, and skipping them would make this article marketing.
First, Kimi K3 leads Arena's blind Frontend Code ranking at 1,679 Elo, ahead of both Fable 5 and GPT-5.6 Sol — human evaluators voting on runnable output with no brand attached. Second, Decrypt reports K3 jumped from #21 to #1 in a blind editorial-voice writing ranking. Third, the original Kimi K2 took first place in a tracked short-story benchmark at 8.56, ahead of o3-pro at 8.44.
So the constraint I feel is domain-shaped. In frontend work and short-form styled prose — dense, well-represented pattern spaces — Kimi's training produces output people prefer blind. The gap appears where the pattern space thins out: novel architectures, long documents, tasks where the right move is to question the brief. Diversity collapse also affects Claude and GPT; the labs seem to differ in how much breadth survives their post-training, and NoveltyBench suggests everyone loses some.
Worth stating plainly: GLM 5.2's 41-point single-generation jump and K2.6's frontier-level agentic scores mean this picture has a shelf life. I re-test every release, and I covered GLM's current API state in my GLM-5.2 benchmark notes→.
How I test for pattern-following
Benchmarks aside, three cheap tests expose this behavior in an afternoon:
A practical routing decision
My stack keeps OpenAI and Anthropic on ambiguous, complex, long-horizon work — the tasks where exploration decides the outcome, and where the decontaminated benchmarks and pass@k research say the gap is real. GLM and Kimi earn slots on well-specified tasks with known solution shapes: frontend iteration, structured agent chains, high-volume work where their pricing wins. That split follows directly from what the tests measure.
My original impression survives the evidence, but sharper than I first phrased it. GLM and Kimi are not less capable. They are narrower — outstanding inside the pattern library, weaker at leaving it. For the work I ship, the ability to leave the library is what I pay for.
