Turning 2D sprites into 3D meshes locally on the Radeon 780M

shino

780mcomfyui3d-generation

A wireframe chibi character standing inside a transparent three-dimensional box

The question here was whether a local, no-cloud pipeline on this ROCm iGPU box (Radeon 780M, gfx1103) could turn flat 2D game sprites into usable 3D character models. The bar was pipeline viability, not a polished feature, and Craymel Ball’s existing P1, P2, and ball sprites gave a real test case since the art already existed.

Tooling progression

TripoSR came first and got rejected. Fed the existing 2D sprites through it, and it ran end-to-end without a hitch, but the output geometry was unusable: unrecognizable blobs rather than anything rough-but-workable.

Hunyuan3D-2mini was the first usable tier. Installed kijai/ComfyUI-Hunyuan3DWrapper plus hunyuan3d-dit-v2-mini.fp16.safetensors, running the shape-only pipeline: Hy3DModelLoader → Hy3DGenerateMesh → Hy3DVAEDecode → Hy3DPostprocessMesh → Hy3DExportMesh. The texture/paint stage got skipped entirely, since it needs a CUDA-only custom_rasterizer/nvdiffrast with no ROCm build. A single front-image input (518×518, composited onto flat gray) produced output that read as recognizably the source character. This is the tier that made the POC feel viable at all.

Hunyuan3D-2mv was the real quality jump. Once P2’s geometry showed a real, measurable limitation (below), the pipeline stepped up to the standard, non-mini Hunyuan3D-2mv checkpoint (roughly a 4.9GB download) and fed it real front, back, left, and right images through Hy3DGenerateMeshMultiView instead of a single image. That’s a different input contract architecturally, and reading the wrapper’s conditioner code confirmed it: real, learned per-view position embeddings, not a placeholder stub.

What running local generation on ROCm surfaced

Single-view input silently under-determines depth. This was the single biggest lesson. P1 came out with a natural 0.71 depth/height ratio from one front image; P2 came out at 0.34, visibly flat, because a cloak’s silhouette gives the model much weaker depth cues than a fitted jacket does. A single-image pipeline can’t tell you when this is happening. You only find out by measuring the output. The first patch was a manual per-asset Z-stretch multiplier, a real fix, but a guess dressed up as a number. Multi-view input fixed it at the source: 0.34 rose to 0.55 with no manual correction, because the model now had actual back and side pixels to reconstruct from instead of guessing.

Shape-only output means inheriting the texturing problem. Skipping the paint stage, forced by ROCm, means every mesh comes out flat gray. Rather than run a fresh AI texture pass, the pipeline reused the game’s own sprite art as vertex colors via orthographic projection. Two gotchas turned up specifically around generating usable assets this way:

  • Baking directional shading into the vertex colors and then letting the real-time renderer light the mesh double-darkens it. Bake color only, and let the engine’s lighting handle the shading.
  • trimesh’s GLB export emits no material at all for a vertex-color-only mesh, so glTF viewers fall back to a near-black default (metalness: 1, roughness: 1). Any vertex-color pipeline needs an explicit diffuse material attached after export, and this cost real debugging time before the cause turned up.

Multi-view conditioning has a specific, undocumented-feeling input contract. It wants either a real alpha channel or a pre-composited black background, unlike the flat gray that worked fine for single-view. Getting this wrong doesn’t throw an error. It just silently degrades the output.

More resolution isn’t automatically more detail. Bumping octree_resolution from 384 to 512 for finer marching-cubes detail turned out to be a dead end for this use case: the extra detail gets discarded by a fixed 40k-face decimation cap immediately downstream, and discarded a second time by the game’s own decimation down to roughly 8k faces, for six times the compute and a measurably identical result. Worth knowing before assuming higher resolution is a free win in any decimated pipeline.

The local environment turned out to be fragile in ways that bite mid-experiment. A stalled-looking job (0% CPU, unresponsive to interrupt) turned out to be slow rather than hung, but recovering from the premature restart surfaced missing dependencies in ComfyUI’s Python environment. Fixing those with a blanket pip install -r requirements.txt silently swapped the ROCm-enabled torch build for a CPU-only PyPI wheel. Checking torch.version.hip caught it before it caused real damage, and the correct venv turned out to be untouched. It’s a sharp edge worth flagging for anyone doing repeated local generation runs: a routine dependency install can quietly break GPU acceleration if the environment isn’t fully isolated.

The bottom line

Locally generated, shape-only meshes are a viable path for stylized low-poly game assets on this hardware: recognizable, reasonably proportioned characters, with no cloud dependency. The real quality lever turned out to be viewpoint count rather than resolution or postprocessing, since giving the model more real angles to reconstruct from moved the numbers more than anything else tried. What’s rough right now is pipeline maturity rather than the underlying capability. Flatness has to be measured by hand, since nothing in the pipeline flags it automatically. The multi-view input contract isn’t documented anywhere obvious. And the shared local environment needs real care around dependency changes.