GUEST EDITORS' INTRODUCTION

Generative AI for Computer Graphics

Rajesh Sharma € rich, Switzerland
, ETH Zurich, 8092, Zu
Vinicius C. Azevedo , Disney Research Studios, 8006, Zurich, Switzerland
Tomasz Bednarz , NVIDIA, Santa Clara, CA, 95051, USA
Doug Roble , Meta, Menlo Park, CA, 94025, USA

Generative AI has emerged as a transformative force in the realm of computer

graphics, offering innovative methods that push the boundaries of creativity,
efficiency, and realism. In this special issue, we delve into the myriad ways in which
generative AI is reshaping the field, with six articles that explore its applications
across a wide range of topics. These contributions cover advancements in neural
networks for image generation, AI-assisted design tools, deep learning techniques
for realistic simulations, and the future of AI-driven animation. By examining both
the theoretical and practical implications of these developments, this issue
provides a comprehensive overview of how generative AI is enhancing the art and
science of computer graphics. As the field continues to evolve, the articles in this
issue offer a glimpse into the exciting possibilities that lie ahead, illuminating the
potential of AI to redefine creative workflows and technical innovations in graphics.

T
he rise of powerful generative artificial intelli- As we enter this new era, it is clear that Gen AI is
gence (Gen AI) systems has catalyzed a revo- not just enhancing the traditional computer
lutionary paradigm shift in visual computing. graphics pipeline—it is fundamentally reshaping its
These systems generate content through probabilis- foundations and expanding the very notion of what
tic, data-driven algorithms that learn complex visual is computationally and creatively possible. In this
patterns from massive datasets—enabling them to special issue, we will address current challenges in
produce rich, diverse, and highly realistic imagery. this emergent process, along with a short summary
This stands in sharp contrast to the traditional about the articles published in this issue.
computer graphics pipeline, which has long been
grounded in deterministic processes, such as physi-
cal simulation, geometric modeling, and rule-based CHALLENGES IN AI FOR
rendering. At the intersection of these two worlds COMPUTER GRAPHICS
lies a new emerging paradigm: a fusion of statistical
learning and procedural generation that redefines Achieving Realism and Detail in
how images, animations, and virtual worlds are con- Generated Content
ceived and created. This convergence opens the One of the primary challenges of integrating Gen AI in
door to hybrid workflows where machine learning computer graphics is to match the high standards
augments artistic intent, where rules and random- required by pipelines established by the industry, may
ness coexist, and where creativity is no longer they be related to the development of real-time inter-
bound solely by manual craft or simulation fidelity. active applications or to the integration of AI into one
of the many stages of the film production pipeline.
While recent advancements in Gen AI have signifi-
cantly improved image quality and fidelity, issues,
such as inconsistencies, artifacts, and unnatural pat-
0272-1716 ß 2025 IEEE. All rights reserved, including rights for terns, still persist. These issues often happen because
text and data mining, and training of artificial intelligence and
similar technologies. the data-driven process that underlies neural
Digital Object Identifier 10.1109/MCG.2025.3574915 approaches is not constrained to the traditional deter-
Date of current version 16 June 2025. ministic laws that underpin imagery generation in

March/April 2025 Published by the IEEE Computer Society IEEE Computer Graphics and Applications 15
GUEST EDITORS' INTRODUCTION

computer graphics. Addressing these challenges accessibility and practicality for widespread usage,
requires a better understanding of how to embed especially in real-time or interactive applications.
physical constraints into generative pipelines without Research into more efficient model architectures,
hindering their creative capabilities, alongside a optimized training algorithms, and hardware accelera-
deeper understanding of perceptual cues that humans tion is essential to reduce computational costs. Prog-
associate with realism. ress in these areas can democratize Gen AI
technology, allowing smaller studios, independent cre-
Controlling the Generation and Output ators, and real-time applications to benefit fully from
Gen AI’s inherent ability to produce diverse and these AI-assisted pipelines.
novel outputs can simultaneously introduce difficul-
ties in precisely controlling the creative process. Integration Into Existing Workflows
Creators require intuitive tools and mechanisms to The integration of Gen AI technologies into estab-
direct generative models toward desired outcomes lished computer graphics pipelines poses additional
without compromising the creative potential of practical and technical challenges. Many profes-
these systems. Current generative tools are usually sional workflows depend on precise control, repro-
based on text-to-image or text-to-video pipelines; ducibility, and seamless interoperability between
textual controls, while effective to create a signifi- software systems and tools. Gen AI techniques
cant embedding of the data distribution, are not must be adapted or developed in a manner that
effective as an interface to artists when considering aligns smoothly with existing production standards,
different stages of the production pipeline. There- pipelines, and creative practices. Furthermore, the
fore, to better represent user-guided control, there direct output of many current generative models
is a need for development of more interactive inter- typically consists of finalized raster images or video
faces, conditional generation techniques, and robust sequences. This presents a limitation when consid-
methods for specifying constraints or stylistic attrib- ering downstream creative workflows that necessi-
utes. Providing artists and designers with precise tate further manipulation and refinement. To
yet flexible control mechanisms is crucial for the address this, significant research and development
widespread adoption and effective use of Gen AI in efforts are required to adapt these generative archi-
professional workflows. tectures to produce more versatile outputs. Specifi-
cally, models need to be engineered to generate
Training Data and Generalization editable layers, such as those found in professional
The performance and reliability of Gen AI systems are image editing software (e.g., Photoshop), or distinct,
heavily influenced by the quality, diversity, and repre- separable assets, such as foreground objects, back-
sentativeness of the training data. Curating compre- grounds, and individual elements within a scene.
hensive datasets that adequately capture the This capability would unlock new possibilities for
complexity of real-world visual phenomena or diverse artists and designers, allowing them to leverage the
artistic styles is a very complex endeavor. Limited or creative potential of Gen AI while retaining precise
biased datasets can cause generative models to control over the final visual outcome through subse-
exhibit poor generalization capabilities, leading to out- quent editing and compositing stages.
puts that are repetitive, biased, or overly specialized.
Effective solutions require systematic approaches to ARTICLES IN THE SPECIAL ISSUE
data collection, augmentation, and synthetic data As the field continues to evolve, this special issue
generation to ensure models generalize well beyond provides a snapshot of the latest developments and
their initial training conditions, thus expanding their perspectives on Gen AI in computer graphics.
applicability across various graphics-related tasks and Through the formal IEEE Computer Graphics and
scenarios. Applications review process of the six submissions,
we accepted six articles for this special issue. The
Computation Cost and Efficiency six articles in this issue offer insights into different
Another critical challenge is the substantial computa- facets of the subject, from theoretical advance-
tional resources required to train and deploy state-of- ments to practical applications.
the-art Gen AI models. High-resolution and realistic Ye et al.A1 presented a unified visual comparison
outputs typically demand large-scale neural networks, framework that combines neural embeddings with
which are computationally expensive, limiting their computational aesthetics to analyze and compare

16 IEEE Computer Graphics and Applications March/April 2025

 GUEST EDITORS' INTRODUCTION

human- and AI-generated paintings. Using CLIP By integrating progressive refinement with seamless
embeddings and interpretable aesthetic features scene transitions, HoloJig maintains user immersion
(such as color, composition, and edge detail), the even during compute-heavy operations. The system
authors develop an interactive visualization system to opens up new possibilities for personalized virtual
explore dataset-level differences and artist-specific experiences across domains, such as remote collab-
style evolution. Case studies, including a comparison oration, education, performance, and simulation,
between real and AI-generated Picasso works, demon- and moves us a step closer to realizing the potential
strate that while AI art approximates stylistic of AI-assisted immersive environments.
elements, it struggles to replicate the nuanced pro- RoutrayA6 presented a comprehensive explora-
gression and diversity found in human creativity. tion of the ethical challenges associated with
Kavouras et al.A2 introduced a framework that Gen AI in computer graphics, including issues of
uses Gen AI, specifically image inpainting, to support authenticity, intellectual property, cultural appropri-
architects and urban planners in creating environmen- ation, and algorithmic bias. It discusses how Gen AI
tally friendly urban interventions. The system gener- blurs the line between real and synthetic visuals,
ates multiple visual alternatives and incorporates a raising concerns about artistic integrity, privacy,
voting-based evaluation method that includes input consent, and the displacement of creative profes-
from both experts and citizens, promoting inclusive sionals. To navigate these complex implications,
and participatory planning. Through four case studies, the author calls for a multidisciplinary approach
the authors demonstrate how this approach can that combines regulatory oversight, ethical literacy,
accelerate early design stages while highlighting the technical safeguards, and inclusive design to
current limitations of AI in producing technically com- ensure responsible and equitable development of
plete solutions. Gen AI technologies.
Kunz et al.A3 introduced a novel framework for
real-time video stylization using text prompts and dif-
fusion models, enabling users to apply artistic effects, CONCLUSION
such as cartoon or painterly styles, to live video The articles in this special issue collectively illus-
streams at 30 fps on commodity GPUs. By combining trate the profound impact of Gen AI on the future
diffusion-based keyframe stylization with a light- of computer graphics, both serving as a powerful
weight, few-shot patch-based training strategy, the tool for technical advancement and a catalyst for
system achieves both high visual quality and temporal creative exploration. From real-time video styliza-
consistency, overcoming previous limitations in speed tion and AI-generated datasets to immersive
and coherence. Especially suited for video conferenc- speech-driven 3-D environments and ethics-cen-
ing, this interactive approach broadens creative tered critiques, the selected works span a rich
expression and user accessibility in real-time graphics spectrum of innovation and inquiry. As Gen AI con-
applications. tinues to evolve, its successful integration into
Mures et al.A4 presented a novel framework for visual computing will depend not only on algorith-
generating synthetic datasets for semantic segmenta- mic breakthroughs, but also on thoughtful design,
tion using controlled diffusion models guided by seg- interdisciplinary collaboration, and ongoing reflec-
mentation maps and text prompts. By bypassing tion on its impact across society, ethics, and the
traditional rendering pipelines, the authors demon- arts. We hope this issue inspires further research
strate how Gen AI can produce high-quality, structur- and responsible innovation at the intersection of AI
ally consistent labeled images across diverse and graphical creativity.
scenarios, such as adverse weather, or specific
domains, such as car part segmentation. Experimental
results show that models trained on these AI-gener- ACKNOWLEDGMENTS
ated datasets consistently outperform those trained The guest editors would like to thank all the authors
on conventionally rendered datasets, suggesting a who submitted their work, as well as the reviewers for
persuasive shift from “Should I render?” to “AI should their thoughtful and dedicated evaluations of the
generate.” many high-quality manuscripts we received. We are
Casas et al.A5 introduced HoloJig, an interactive also grateful to Pak Chung Wong and Chris Weaver for
speech-to-VR system that generates immersive 3-D their leadership at IEEE Computer Graphics and Appli-
environments in real time from spoken prompts cations and for their guidance throughout the prepa-
using diffusion models and depth-based rendering. ration of this special issue.

March/April 2025 IEEE Computer Graphics and Applications 17

GUEST EDITORS' INTRODUCTION

RAJESH SHARMA is currently working toward the Ph.D.

APPENDIX: RELATED ARTICLES degree with ETH Zurich, 8092, Zurich, Switzerland. His
research interests include global illumination in complex
A1. Y. Ye, R. Huang, K. Zhang, and W. Zeng, scenes and simulation solutions for vegetation, hair,
“Unified visual comparison framework cloth, snow, and water. He is the corresponding author of this
for human and AI paintings using article. Contact him at [email protected].
neural embeddings and computational
aesthetics,” IEEE Comput. Graphics Appl.,
vol. 45, no. 2, pp. 19–30, Mar./Apr. 2025, VINICIUS C. AZEVEDO is a research scientist at Disney
doi: 10.1109/MCG.2025.3555122. Research Studios, 8006, Zurich, Switzerland. His research
A2. I. Kavouras, I. Rallis, E. Sardis, A. Doula-
interests include visual computing, physically based anima-
mis, and N. Doulamis, “Voting-based
tions, machine learning, and generative graphics. Contact
intervention planning using AI-gener-
ated images,” IEEE Comput. Graphics him at [email protected].
Appl., vol. 45, no. 2, pp. 31–46, Mar./Apr.
2025, doi: 10.1109/MCG.2025.3553620.
A3. D. Kunz, O. Texler, D. Mould, and D. TOMASZ BEDNARZ is the director of strategic researcher
Sykora, “Meet-in-style: Text-driven real- engagement at NVIDIA Corporation, Santa Clara, CA, 95051,
time video stylization using diffusion USA. His research interests include immersive visualization,
models,” IEEE Comput. Graphics Appl., computational modeling, and AI-driven scientific discovery.
vol. 45, no. 2, pp. 47–56, Mar./Apr. 2025, Contact him at [email protected].
doi: 10.1109/MCG.2025.3554312.
A4. O. A. Mures, M. Silva, M. Lijo  -Sanchez,
E. J. Padro  n, and J. A. Iglesias-Guitian, DOUG ROBLE is a research scientist at Meta, Menlo Park, CA,
“Should I render or should AI generate?
94025, USA. His research interests include digital humans,
Crafting synthetic semantic segmenta-
machine learning, and physical simulation. Roble received his
tion datasets with controlled genera-
tion,” IEEE Comput. Graphics Appl., Ph.D. degree in computer graphics from The Ohio State Uni-
vol. 45, no. 2, pp. 57–68, Mar./Apr. 2025, versity, Columbus, OH, USA. He is a member of ACM, AMPAS,
doi: 10.1109/MCG.2025.3553494. and the Television Academy. Contact him at [email protected].
A5. L. Casas, S. Hannah, and K. Mitchell,
“HoloJig: Interactive spoken prompt
specified generative AI environments,”
IEEE Comput. Graphics Appl., vol. 45, no.
2, pp. 69–77, Mar./Apr. 2025, doi: 10.1109/
MCG.2025.3553780.
A6. S. K. Routray, “Ethical considerations and
implications of generative AI in computer
graphics,” IEEE Comput. Graphics Appl.,
vol. 45, no. 2, pp. 78–89, Mar./Apr. 2025,
doi: 10.1109/MCG.2025.3570722.

18 IEEE Computer Graphics and Applications March/April 2025