Getting local video generation running on a Radeon 780M mini PC

shino

780mcomfyuivideo-generation

A compact computer projecting a coherent snowy landscape from a blurred stream of video frames

I have a GMKtec NucBox K8 Plus sitting on my desk: a Ryzen 8845HS, a Radeon 780M integrated GPU (gfx1103), 32GB of shared DDR5, and no discrete graphics card. Nobody would call it an AI workstation. I wanted to see how far I could push local video generation on it anyway.

This is the story of getting there. Some paths dead-ended outright. Two model attempts got abandoned, then revived later once I understood the real bug. It ends with a debugging arc where a single missing boolean flag silently turned coherent video into noise.

The setup

gfx1103 is not officially supported by mainline ROCm, which rules out a lot of the easy paths. ComfyUI-TheRock, built against AMD’s TheRock ROCm 7.15 nightly with PyTorch 2.12, worked: genuine HIP-backed PyTorch targeting the 780M directly, without ZLUDA or a CUDA-translation layer in the middle. (The full TheRock switch-over is its own earlier post.) ComfyUI reports it as 14GB of usable VRAM, which in practice is system RAM being carved out for the iGPU. That number matters a lot for what follows.

First wall: MiniMax H3

The first model I tried was MiniMax H3. It hit a hard VRAM ceiling almost immediately, more model than the hardware had room for. No software workaround exists for hardware that doesn’t have the memory, so I moved on to something smaller.

LTX-2.5: four bugs and a wrong conclusion

LTX-2.5 looked more promising. It’s smaller, and Lightricks explicitly targets consumer hardware with it. Getting it running took four separate fixes:

  1. A broken optional import. ComfyUI-LTXVideo’s pyramid-blending module imported pad from kornia.geometry.transform.pyramid, which doesn’t exist in the installed kornia version. This crashed the entire custom node package on load, taking down features that had nothing to do with the broken import. Fixed by wrapping the import in try/except and only registering the node if it succeeded.
  2. The wrong VAE. ltx-2.5-video-vae-bf16.safetensors is a diffusion decoder, a different architecture that the plain VAELoader can’t parse. Swapping to ltx-2.5-video-vae-conv-bf16.safetensors (the conventional convolutional decoder) fixed it.
  3. A real upstream ComfyUI bug. Gemma4Model.process_tokens() returned a single value where the base forward() expected four, an actual bug in ComfyUI core. A git pull to a commit that removed the broken override fixed it.
  4. VRAM, then architecture. With those three cleared, text encoding hit a hard CUDA OOM (13.81GB requested, 0 free): the text encoder alone didn’t fit in the 14GB pool. Once that was worked around, model loading failed with a tensor shape mismatch: scale_shift_table came out as [9, ...] in the checkpoint but [6, ...] in the model ComfyUI built.

At that point I gave up on 2.5, on the unverified assumption that the community GGUF quant was “built against a different architecture revision.” That guess was wrong. I didn’t find out how wrong until much later.

Falling back to LTX-2.3, and getting it working

LTX-2.3 turned out to be the right size for this hardware, once the text encoder moved off the GPU. LTXAVTextEncoderLoader (and the plain CLIPLoader, which hides the same option under an easy-to-miss advanced param) supports device: "cpu", which forces the encoder fully onto system RAM instead of the 14GB pool. That let an 11GB text encoder run without ever touching the VRAM budget the diffusion model needs.

The other fix was in VAE decoding at longer lengths. The generic VAEDecodeTiled node, even with conservative tile settings, crashed the entire native process at 97 frames (LTX requires frame counts of the form 1 + 8k; 97 frames is about 4 seconds at 24fps). Switching to the LTX-native LTXVTiledVAEDecode node, which tiles by count rather than pixel size, fixed it outright.

With both fixes in place, the 2.3 pipeline worked: 9-frame test generations in a couple minutes, full 4-second clips with audio in under 5.

The bug that produced noise with zero errors

To cut the roughly 30-40 minute text-encoding step out of every iteration, I added a save/load cache for conditioning tensors (LTXVSaveConditioning / LTXVLoadConditioning, community nodes), so the encode only had to happen once and the result could be reused after that.

The cached path ran clean: correct duration, correct resolution, status: success. What came out was garbage, a flat, muddy, static-noise texture, nothing resembling the ball, the table, or any motion at all.

The debugging process here is worth being honest about, because I got it wrong twice before getting it right:

  • First guess: quantization. I was on a Q2_K quant, and maybe it was too aggressive. Downloading Q4_K_M produced the same bad output, so that wasn’t it.
  • Second guess: a missing pipeline node. LTX conditioning sometimes needs an explicit frame-rate-stamping node. Adding it changed nothing.
  • Third attempt: tensor-valued options getting dropped during save. The save node only preserved a hardcoded attention_mask tensor, so I generalized it to save all tensor-valued options, implemented the fix, and waited through a 31-minute re-encode to test it. cond_options turned out to be empty for this model: the fix was a correctly-implemented no-op. Inspecting the raw tensors directly (shape, dtype, mean/std, NaN checks) showed no corruption anywhere.

The actual bug was hiding one level up. ComfyUI’s text encoder attaches extra = {"unprocessed_ltxav_embeds": True} to the conditioning, a plain Python boolean rather than a tensor. The original save/load implementation only round-tripped tensors, so this flag was silently dropped on every save.

That single dropped boolean turned out to matter a lot. preprocess_text_embeds() in av_model.py checks whether the incoming embedding’s last dimension already matches cross_attention_dim + audio_cross_attention_dim (4096 + 2048 = 6144) to decide whether it’s already been projected. Raw Gemma embeddings for this setup happen to also be 6144-dimensional, a coincidence of the specific model config. Without the flag telling it otherwise, the model treated raw, unprojected embeddings as if they had already been through the projection layers. The shapes lined up, so nothing threw an error or a warning, and the meaningless numbers just flowed through every downstream stage unchanged.

The fix extended the save/load nodes to JSON-serialize arbitrary non-tensor options into the .safetensors file’s metadata block, alongside the tensors. Since only metadata needed patching, I retro-fitted the already-saved conditioning files in place rather than re-running the 30-40 minute encode.

for idx, (cond_tensor, cond_options) in enumerate(conditioning):
    tensors_to_save[f"conditioning_data_{idx}"] = cond_tensor.to(dtype=target_dtype).contiguous()
    for key, value in cond_options.items():
        if torch.is_tensor(value):
            tensors_to_save[f"opt_{key}_{idx}"] = value.contiguous()
        else:
            try:
                json.dumps(value)
            except (TypeError, ValueError):
                continue
            non_tensor_options[f"{idx}:{key}"] = value  # -> written into file metadata

status: success and a correct output shape meant nothing about actual correctness. I only found this because I insisted on decoding real frames and looking at them instead of trusting the pipeline’s own success signal, a habit that paid off again later.

Reviving LTX-2.5

With 2.3 solid, I came back to 2.5’s abandoned scale_shift_table mismatch, no longer willing to accept “probably a different architecture revision” as an answer.

Bug 1: the community GGUF quant was missing its architecture metadata. ComfyUI doesn’t infer every architectural flag from tensor shapes. Some, like cross_attention_adaln, are read from a config JSON blob embedded in the GGUF file’s own metadata:

# comfy/model_detection.py
if metadata is not None and "config" in metadata:
    dit_config.update(json.loads(metadata["config"]).get("transformer", {}))

The working LTX-2.3 quant (from unsloth) carries this blob. The LTX-2.5 quant I’d downloaded (from Abiray) had exactly six trivial KV fields and no config at all. Without it, cross_attention_adaln silently defaulted to False, and BasicAVTransformerBlock.__init__ built a 6-row modulation table instead of the checkpoint’s 9-row one, precisely the [9,...] vs [6,...] mismatch that had ended the first attempt.

To fix it, I diffed all 4,349 tensor names and shapes between the working 2.3 checkpoint and the broken 2.5 one. They were architecturally identical except for two things: 2.3 has extra feed-forward bias tensors that 2.5 doesn’t, and 2.5 has one extra tensor (keyframes_abs_pos_embedding) that ComfyUI detects separately anyway. That meant copying 2.3’s config blob wholesale into 2.5’s file, flipping one flag (ff_bias: False), and getting a config that matched the checkpoint, all without touching a single tensor’s data.

Bug 2, found on the very next attempt: the scale_shift_table error was gone, but three new mismatches appeared, all with suspiciously exact 2× or squeezed-dimension errors:

size mismatch for keyframes_abs_pos_embedding: checkpoint [4096] vs model [1, 4096]
size mismatch for audio_embeddings_connector.learnable_registers: checkpoint [128,4096] vs model [128,2048]
size mismatch for video_embeddings_connector.learnable_registers: checkpoint [128,8192] vs model [128,4096]

The ComfyUI-GGUF loader reshapes tensors on load, but only for F32/F16 dtypes. Its BF16 handling only covers 1-D tensors:

if tensor.tensor_type in {F32, F16}:
    torch_tensor = torch_tensor.view(*shape)
# 1D tensors shouldn't be quantized, this is a fix for BF16
if len(shape) <= 1 and tensor.tensor_type == BF16:
    state_dict[sd_key] = dequantize_tensor(...)

Multi-dimensional BF16 tensors fall through both branches untouched, keeping their raw byte-view shape instead of their logical one. A BF16 element is 2 bytes, so a [128, 4096] tensor viewed as raw bytes reports as [128, 8192], exactly the doubling shown in the error. LTX-2.3 stores these three tensors as F32, which sidesteps the bug entirely. The Abiray LTX-2.5 quant stores them as BF16, so it hits the bug on every load.

The fix converts just those three tensors to F32, an extra 1.6MB, compared to the 856MB it would have cost to convert every BF16 tensor in the file, and adds a comfy.gguf.orig_shape metadata entry so the squeezed keyframes_abs_pos_embedding gets its leading dimension back:

def bf16_to_f32(u8):
    return (u8.view(np.uint16).astype(np.uint32) << 16).view(np.float32)

With both fixes applied, now reusable as a single patch script for any Abiray LTX-2.5 quant, LTX-2.5 loaded and sampled cleanly. I confirmed it with a real generation rather than just a clean load, a full 40-minute Gemma4 CPU encode of the real prompt, followed by inspecting the actual decoded frames. The output was coherent, with correct motion. At 4 seconds, the LTX-native tiled decoder that had been rock-solid on 2.3 at 1×1 tiles crashed the whole process on 2.5, since its VAE is heavier, and needed 2×2 tiling to survive.

Chasing quality

With both models working, the obvious next question was how much headroom was left on the table. Every test below reused banked conditioning, so each iteration cost minutes instead of another 40-minute encode.

Resolution turned out to be the single biggest lever, and it was nearly free. I’d been testing at 512×288, a leftover smoke-test setting from checking whether the pipeline loaded at all, and that was throwing away most of the achievable quality. Going to 864×480, 2.8 times the pixels, cost 9 extra seconds and produced a dramatic jump in sharpness. Going all the way to LTX’s native 1344×768, 7 times the pixels, cost about 60% more time total. The wood grain came in as individual grain lines instead of a blur, and the reflections held together as reflections. Depth of field behaved the way a real lens would, foreground sharp and background falling away naturally. On this hardware, generation time is dominated by model load and CPU-GPU offload overhead rather than pixel compute, which is why resolution ends up this cheap.

Step count mattered more than expected. The 8-step schedule carried over from 2.3’s distilled model starved 2.5 badly. Bumping to 25 steps via LTXVScheduler roughly tripled the time and visibly improved fine surface detail, a strong signal that the Abiray quant is the dev model rather than a distilled one.

The “best quality” diffusion decoder made no visible difference against the standard conv decoder, even at native resolution, and dropped out as a lever entirely.

Quantization level was the one that fooled me. On a simple test prompt, a red ball on a wooden table, Q3_K_S and Q6_K (17.8GB, patched with the same two fixes above) were nearly indistinguishable. I initially concluded the quant barely mattered. That conclusion didn’t survive contact with a harder prompt.

The test that separated the quants

A red ball on a table is close to a best case for a low-bit quant: one object, shallow depth of field hiding the background, minimal fine texture, and no text anywhere in frame. Quantization damage tends to show up in fine high-frequency detail and dense, multi-object scenes under complicated lighting, none of which this scene has much of.

So I built a harder prompt on purpose: a white-haired anime witch, kneeling in a sunlit greenhouse, gardening glowing bioluminescent plants, humming a tune. It packed in hair strands, foliage texture, hand-object interaction, complex light, and an audio component to exercise the audio branch, deliberately targeting everything the ball prompt didn’t.

The same seed, conditioning, and settings (1344×768, 25 steps) went into both runs, Q3_K_S against Q6_K:

Q3_K_S was good on its own, usable output. Side by side with Q6_K, the difference showed immediately: Q6_K rendered individual hair strands with real light-catching highlights, where Q3_K_S’s hair read as a flatter mass. The hat told the same story. Q6_K modeled a band with a visible buckle and tassel; on Q3_K_S the band was a vague dark smear. The gap the ball prompt had completely hidden turned out to be obvious once fine detail was on screen.

Revised recommendation: Q6_K at 1344×768, 25 steps as the default. Q3_K_S still works for quick low-detail drafts, just not for anything with fine detail or faces.

Where it stands

LTX-2.3LTX-2.5
Text encode (once per prompt, CPU)~32 min~40 min
9-frame generation~2.4 min~3-5 min (res/step dependent)
4s (97 frames) + audio~4.5 min~5.4 min
Best settings foundQ2_K/Q4_K_M, 8 steps (distilled)Q6_K, 1344×768, 25 steps

Two subtle bugs turned out to be gating a model that runs fine on this hardware: a missing metadata blob and an unhandled dtype branch in a GGUF loader. Neither was a hardware limitation. Both were fixable by understanding exactly what ComfyUI’s loading code does with a checkpoint’s bytes, and diffing against a model that already worked.