NaTex: Seamless Texture Generation as Latent Color Diffusion

Zeqiang Lai^1,2+, Yunfei Zhao²⁺, Zibo Zhao², Xin Yang², Xin Huang², Jingwei Huang², Xiangyu Yue^1*, Chunchao Guo^2*

¹MMLab, CUHK
²Tencent Hunyuan

We present NaTex, a native texture generation framework that predicts texture color directly in 3D space. In contrast to previous approaches that rely on baking 2D multi-view images synthesized by geometry-conditioned Multi-View Diffusion models (MVDs), NaTex avoids several inherent limitations of the MVD pipeline. These include difficulties in handling occluded regions that require inpainting, achieving precise mesh-texture alignment along boundaries, and maintaining cross-view consistency and coherence in both content and color intensity. NaTex features a novel paradigm that addresses the aforementioned issues by viewing texture as a dense color point cloud. Driven by this idea, we propose latent color diffusion, which comprises a geometry-awared color point cloud VAE and a multi-control diffusion transformer (DiT), entirely trained from scratch using 3D data, for texture reconstruction and generation. To enable precise alignment, we introduce native geometry control that conditions the DiT on direct 3D spatial information via positional embeddings and geometry latents. We co-design the VAE–DiT architecture, where the geometry latents are extracted via a dedicated geometry branch tightly coupled with the color VAE, providing fine-grained surface guidance that maintains strong correspondence with the texture. With these designs, NaTex demonstrates strong performance, significantly outperforming previous methods in texture coherence and alignment. Moreover, NaTex also exhibits strong generalization capabilities, either training-free or with simple tuning, for various downstream applications, e.g., material generation, texture refinement, and part segmentation and texturing.

Comparing NaTex with previous Multi-View Diffusion (MVD) methods on core challenges

1. Occlusion: regions are an inevitable aspect of multi-view texturing; no matter the approach, they cannot be entirely avoided.
2. Alignement: MVDs have difficulties in achieving precise alignment of texture features with fine-grained geometric details.
3. Coherence: Maintaining consistency across multiple views is a costly process, and even state-of-the-art video models struggle to achieve satisfactory results in this regard.

NaTex directly predicts RGB color for given 3D coordinates via a latent diffusion approach, a paradigm that has shown remarkable effectiveness in image, video, and 3D shape generation, yet unexplored for texture generation.

We introduce Latent Color Diffusion for modeling with:

1. A Geometry-aware Color VAE for color point clouds, in which a geometric branch encodes local shape cues to achieve geometric awareness for color generation.
2. A Multi-Control Color DiT that flexibly integrates geometric, image conditions, and color condition, enabling seamless texture generation and beyond.

NaTex outperforms state-of-the-art methods, particularly in texture alignment and coherence and occlusion handling.

Commercial models use their own geometries, while other methods share the same geometry from Hunyuan3D 2.5. All methods are rendered with albedo only.

Our design enables a wide range of applications beyond image-conditioned texture generation (with geometry and image controls) to textureconditioned material generation and texture refinement (using an initial texture, named as color control).

Notably, NaTex exhibits remarkable generalization capability, enabling image-conditioned part segmentation and texturing even in a training-free manner.