Qwen3-27B in the tower-defense harness: a decode gain and a client timeout

In the previous post, I built a local harness that could generate and test a browser tower-defense game on a Radeon 780M.

The final game passed 38 browser and static checks. The harness had learned to quarantine broken patches, inspect the running page, and route failures back to the right file. The last defect came down to one geometric predicate, which I fixed by hand.

That left a more interesting question than “can a local model write a game?”

What happens if I put a much larger local model back into the loop?

The previous setup used gpt-oss-20b-16k for planning and review, and Ling-3.0-tiny for bulk source generation. This time I used the Qwen3.8-27B GGUF already on the machine. It was the same quant from an earlier experiment. I ran it through Unsloth Desktop’s local llama.cpp runtime and gave it a repair task.

The Q3 quant fit a useful 24K context on the 780M. Its built-in multi-token prediction raised decode speed from 4.29 to 7.18 tokens per second. Qwen repaired a deliberately broken game with one minimal patch. A whole cold request improved by about 13%. A fresh 20K-token prompt still took about six minutes before the model produced anything.

The machine

The test machine is a GMKtec K8 Plus with a Ryzen 7 8845HS, Radeon 780M graphics, and 32 GB of RAM. The 780M is an integrated GPU. Its memory is shared with the CPU. Allocating more of it to inference also changes the memory available to Windows.

The model was an existing 13.1 GB Qwen3.8-27B-UD-Q3_K_XL.gguf. I pointed the runtime at the cached file directly. Nothing was copied, deleted, or converted, and the existing Ollama, Open WebUI, ROCm, and ComfyUI installations were left alone.

Unsloth launched the bundled llama.cpp server with Vulkan. The live process loaded ggml-vulkan.dll, vulkan-1.dll, and AMD’s amdvlk64.dll. Windows reported active GPU compute while requests were running. GPU inference was active, with the CPU participating through shared memory.

The first settings were intentionally ordinary: full GPU offload, one slot, Q8 KV cache, and 16K context. That worked. The useful surprise was that 24K also worked.

Finding the configuration

I swept one variable at a time against the same 3,811-token input and a 128-token response. Each configuration was freshly loaded, and I saved the request, process command line, engine timing, response, raw SSE, and memory telemetry.

Configuration	Prompt tok/s	Decode tok/s	Request time
16K, full GPU, 8 threads	62.02	4.30	90.98 s
16K, 12 threads	62.21	4.27	91.02 s
16K, 16 threads	58.88	4.28	94.44 s
16K, 56 of 65 GPU layers	48.40	3.30	117.24 s
16K, microbatch 512	56.37	4.27	97.39 s
24K, MTP disabled	59.11	4.29	94.10 s
24K, MTP with two draft tokens	59.23	7.18	82.07 s
24K, n-gram speculation	61.38	4.08	93.21 s
24K, MTP with four draft tokens	57.73	3.75	99.92 s

Eight CPU threads won by being boring. Twelve and sixteen left decode unchanged. Moving nine layers back to the CPU slowed prompt processing and generation. A larger microbatch also slowed this workload.

The standout setting was Qwen’s built-in MTP head. With two draft tokens, the server proposed 100 draft tokens, accepted 76, and reached 7.18 tokens per second. The ordinary and MTP responses were identical in the controlled comparison.

Four draft tokens was worse. It accepted 87 of 155 proposed tokens. Decode fell to 3.75 tokens per second. Two draft tokens suited this machine and workload.

The final profile was:

Qwen3.8-27B UD-Q3_K_XL
Vulkan llama.cpp, all 65 layers on the GPU
24,576 context
Q8_0 K/V cache, flash attention on
8 CPU threads
batch 512, microbatch 256
one inference slot
MTP, two draft tokens
1 GiB host prompt cache
four context checkpoints
reasoning disabled for the harness

The exact reload request is saved as a machine-readable best-config.json alongside the local evaluation artifacts. The profile was reloaded at the end of the experiment, and the active process command line confirmed the context size, GPU layers, MTP mode, cache limits, and thread count.

Twenty thousand tokens fit. They are still expensive.

A context setting describes capacity. Usable latency requires a separate measurement.

For the long-context test, I put one marker near the beginning of a repeated source corpus and another at the end, filled the context window, and asked Qwen to return both values.

At 16K, a 12,532-token input was retrieved correctly. First-token latency was 221 seconds, at 56.79 prompt tokens per second.

At 24K, a 20,032-token input was also retrieved correctly. Under the final MTP profile, first-token latency was 361 seconds. Prompt processing ran at 55.60 tokens per second. The short answer decoded at 6.67 tokens per second. The process still had at least 5.93 GiB of free physical RAM.

The practical shape of a large local model on an integrated GPU looks like this:

Short cached conversation: usable
Cold 3.8K-token prompt: about a minute before decoding
Cold 20K-token prompt: about six minutes before decoding
Decode after the prompt: around 7 tok/s with MTP-2

The 24K context helps with repository repair because it lets the model see source alongside the specification and browser diagnostics. Reading a repository still takes time. Prompt selection and context reuse matter more than advertising the largest context the model can technically load.

The cache benchmark was wrong before I found the reason

My first ten-turn test reported zero cached prompt tokens. That suggested the runtime was discarding the conversation after every request.

It was my test that was wrong.

I had supplied a fixed seed. The installed Unsloth backend deliberately disables prompt caching when a seed is present. That keeps repeated requests reproducible. Once I removed the seed, the behavior changed immediately. Follow-up turns reused most of the previous prompt and started producing tokens in roughly 1.6 to 2.0 seconds.

Ten turns all retained a remembered build key. The run reused 4,669 prompt tokens, averaged 10.61 seconds per response, and kept more than 5.91 GiB of physical RAM free.

I also tried more frequent context checkpoints. The default spacing averaged 1.77 seconds to first token across matched follow-up turns. A 256-token spacing averaged 1.82 seconds. The more aggressive setting produced no gain. I kept the default spacing and bounded the number of retained checkpoints.

The episode gave me a useful rule for local inference work: inspect the runtime source before changing the model, quantization, or backend.

The client timed out before the model

The original repair adapter used Node’s fetch to call the Unsloth API. Its own AbortController allowed fifteen minutes. Long calls still failed at around five minutes with fetch failed.

I reproduced the behavior with a local server that waited 310 seconds before responding:

Node fetch / Undici:  failed at 305.426 s, UND_ERR_HEADERS_TIMEOUT
Explicit Node HTTP:   succeeded at 310.025 s

The HTTP client had a separate headers timeout. The application timeout left that setting unchanged.

That explained several apparently failed Qwen repairs. The model was still running. The client had stopped waiting.

I replaced the Unsloth side of the private harness adapter with an explicit streaming HTTP client. It has its own total deadline. It consumes server-sent events incrementally and keeps usage and timing data. It rejects incomplete streams. It also refuses a response that ended because it hit its output limit.

The transport tests cover split SSE chunks, incomplete streams, and a stalled response. The fix changed what the harness could measure. A seven-minute model call is now a slow result with recorded output.

A real repair, with a known bug

I wanted the benchmark to prove that the repair was needed before calling a prompted patch “autonomous repair.”

The isolated test began with the working Neon Bastion and its browser validator. I introduced a projectile-damage regression and confirmed that the validator failed. Qwen then received the broken source and the browser failure report. I validated its candidate in a separate directory.

The working baseline passed 38/38 checks. The introduced regression failed exactly one check: tower-attack-damages-enemy.

Qwen received the full JavaScript source and the failing browser output. It returned one exact patch:

-      target.hp -= 0;
+      target.hp -= proj.damage;

The candidate passed syntax checking and all 38 browser checks. Its SHA-256 matched the known-good source. The original game was never touched.

The request used 8,068 input tokens and 35 output tokens. It took 140 seconds. Prompt processing ran at 60.13 tokens per second. Decode ran at 6.59 tokens per second. Almost all of that time was spent reading the source and test context. The answer itself was tiny.

The benchmark gives stronger evidence than a model’s own review. Qwen had to produce a patch the browser could execute. The initial validation proved the defect existed. The final validation proved that the patch fixed it.

The defect was deliberately introduced. Bug discovery in Neon Bastion was outside this test’s scope. The experiment demonstrates a repair loop with an executable failure report, a minimal model-produced edit, and an independent browser pass.

What changed in the harness

The published experiment showed that local models can generate a convincing game, and that a validator can expose the gap between appearance and behavior.

The planner can now be a larger model while keeping the bounded patch interface. Qwen returned a small structured edit for script.js.

The harness also treats long local inference as a systems problem. It records the backend, model process, cache settings, stream timing, and memory floor. It keeps the planner on one side of the GPU with other workloads out of the way.

The harness also tests its own assumptions. A seed disabled caching. The HTTP client imposed a five-minute timeout. An early repair experiment validated the wrong directory. Each mistake produced a better experiment once it became observable.

What the run established

Qwen3.8-27B on a Radeon 780M is more usable than the initial numbers suggested.

The Q3 XL quant fits 24K context. Vulkan is active. MTP-2 raises decode speed by about 67% on the tested workload, while a cold request is about 13% faster end to end. Cached conversations respond in a couple of seconds. A known game regression was repaired and verified at 38/38.

The limits are just as clear. A cold 20K prompt takes about six minutes. Available memory supports one large resident model comfortably. The successful repair covered one controlled regression. An overnight autonomous coding session remains untested. The 27B model makes a good evidence-driven reviewer and repairer. An integrated mini PC still has workstation-class latency limits.

The published post asked whether the harness could tell when generated code was wrong. This run made the answer more useful: Qwen supplied the patch, and the browser verified it at 38/38.