Skia Integration in Delphi: Enhancing FireMonkey's Graphics with Pros, Cons, and Performance Insights #347

Zeus64 · 2024-10-08T11:10:33Z

Zeus64
Oct 8, 2024

https://github.com/MagicFoundation/Alcinoe?tab=readme-ov-file#about-skia

I’ve written a brief article about Skia and Delphi. I’d love to hear any feedback or remarks you might have!

Skia is an open-source 2D graphics library that powers the graphics engine used by Flutter and the Android operating system. The implementation of Skia in Delphi marked a significant advancement for the FireMonkey framework, as it greatly surpasses the legacy Delphi TCanvasGPU graphics engine in nearly every aspect.

Key Advantages of Skia:

Solid Graphics Foundation: Since Skia is the same graphics engine used by Flutter, apps built with Flutter or Delphi + Skia share the same foundational graphics technology.
Performance: The Skia algorithms are optimized to the extent that they can often render directly onto the form surface without requiring an internal buffer.
Cross-Platform Consistency: Skia offers a consistent graphics engine across all platforms (Windows, iOS, macOS, Android).
Rich Features: Skia provides powerful capabilities typical of a robust graphics engine, including advanced text formatting and shadow rendering.

Drawbacks of Skia:

Increased Package Size: Adding Skia increases the package size by approximately 25 MB.
Dependency on Google: Skia is a Google product, and while Embarcadero has traditionally focused on offering an independent product free from GAFA domination, using Skia introduces a dependency on Google's technology.
OpenGL Limitations: Under OpenGL, only raster images (CPU) can be shared across different threads. This means that GPU textures cannot be created in background threads and later drawn on the main form surface. Interestingly, there seems to be no noticeable speed improvement when drawing GPU images compared to raster images.
Image Rendering Performance: While Skia excels at drawing shapes directly on the form surface, it is notably slower (4x more slower) than the legacy Delphi canvas (TCanvasGPU) when rendering images onto the form.

Performance Challenges with Skia:

The slower image rendering is particularly problematic in scenarios where we paint everything first onto an internal buffer and then render that buffer to the form surface on each paint loop—a technique often used to avoid flickering and improve performance. Unfortunately, Skia can be up to 4 times slower at drawing images onto the form surface compared to the legacy Delphi TCanvasGPU, which uses OpenGL textures.

For example, on a Google Pixel 7, I can render 2000 textures simultaneously at 90 FPS using TCanvasGPU. However, with Skia, I can only render 500 images at the same frame rate. You can verify this using the demo app located at Demos/ALFmxGraphics.

The Chosen Approach:

Given these limitations, I’ve decided to use Skia as the backend graphics engine in Alcinoe while continuing to use the legacy Delphi canvas (TCanvasGPU) for rendering on the main form surface. The exception is on Windows, where we use Skia for both the backend and the main form surface.

Platform-Specific Considerations:

On Android and iOS, the operating systems already provide powerful graphics APIs, and in many cases, these native APIs outperform Skia—often by as much as 2x. While the performance of the OS graphics APIs is superior, Skia still offers the benefit of a unified graphics engine across all platforms, along with additional features like animated images.

viniciusfbb · 2024-10-09T02:34:29Z

viniciusfbb
Oct 9, 2024
Maintainer

@Zeus64 I am impressed with the work. It seems absolutely amazing. 👏👏👏👏

As for the drawbacks, I would like to make some considerations:

OpenGL Limitations: Under OpenGL, only raster images (CPU) can be shared across different threads. This means that GPU textures cannot be created in background threads and later drawn on the main form surface. Interestingly, there seems to be no noticeable speed improvement when drawing GPU images compared to raster images.

On mobile, it seems that the draws on both GPU / CPU have similar performance, but there are specific scenarios where the GPU will draw considerably faster, like effects / shaders.

Image Rendering Performance: While Skia excels at drawing shapes directly on the form surface, it is notably slower (4x more slower) than the legacy Delphi canvas (TCanvasGPU) when rendering images onto the form.

Our TBitmap canvas implementation is raster (CPU) but there are some optimizations. The actual approach creates internally a cache of the texture according to the target of the bitmap and draw target. That is, when you have a TBitmap and draw it on a form surface, it will generate one skia texture from that TBitmap and will cache it for that target until that TBitmap changes. IOW, you can create the TBitmap, draw it, and when you use it to draw on the screen, it will generated a texture internally, that is a slow process, but only for the first draw of that bitmap in another target.

So, the tips here are:

Use TBitmap instead ISkImage directly;
Avoid create temporary TBitmap;
Use TBitmap.Assign when possible;
Enable GlobalSkiaBitmapsInParallel if you are "building" the TBitmap in a background thread;
Avoid cache system for normal controls. I also adopted this strategy in an app a few years ago but with Skia I realized that drawing directly on the screen is the best option on the end. Skia's own controls work this way too, they create a cache in a TBitmap only when TCanvas is not a Skia implementation. The exceptions are effects that are really slow to draw each frame, but the TEffect from FMX already have a cache system that avoids unnecessary repainting;
The last one is more hardcore, which I don't usually recommend, but it depends on the situation. It is possible to create a Skia texture and draw on that texture with a TCanvas and paint that texture on the screen, that is, without involving TBitmap and without any raster operation or image upload operation to the GPU. This is done in FMX.Skia.Filter.TSkTextureCacheLayer but it is a somewhat difficult code to understand. So it is not an easy way, but depending on the situation, it may be worth it;

Finally, I'm curious to know the relationship between this "4 times slower at drawing images onto the form" performance and #331, I mean, was the measurement made with your proposed changes?

0 replies

Zeus64 · 2024-10-09T08:57:34Z

Zeus64
Oct 9, 2024
Author

Thank @viniciusfbb!

On mobile, it seems that the draws on both GPU / CPU have similar performance, but there are specific scenarios where the GPU will draw considerably faster, like effects / shaders.

Yes, that's true. I was measuring the speed of drawing a simple GPU-based skImage versus a CPU-based skImage onto the form, and my tests show that the performance is similar in both cases. However, as you pointed out, when building the skImage with the GPU, certain methods are more optimized, like effects and shaders.

Our TBitmap canvas implementation is raster (CPU) but there are some optimizations. The actual approach creates internally a cache of the texture according to the target of the bitmap and draw target. That is, when you have a TBitmap and draw it on a form surface, it will generate one skia texture from that TBitmap and will cache it for that target until that TBitmap changes. IOW, you can create the TBitmap, draw it, and when you use it to draw on the screen, it will generated a texture internally, that is a slow process, but only for the first draw of that bitmap in another target.

As for my tests, I didn’t use TBitmap at all—I worked directly with skImage using the Skia API. My test was specifically to compare drawing a TTexture (TCustomCanvasGpu(ACanvas).DrawTexture) versus drawing a skImage (sk4d_canvas_draw_image_rect(TSkCanvasCustom(ACanvas).Canvas.Handle, ...)) directly on the form. The results showed that TTexture with TCustomCanvasGpu is about 4x faster to draw than skImage with TSkCanvasCustom.

Avoid cache system for normal controls. I also adopted this strategy in an app a few years ago but with Skia I realized that drawing directly on the screen is the best option on the end. Skia's own controls work this way too, they create a cache in a TBitmap only when TCanvas is not a Skia implementation. The exceptions are effects that are really slow to draw each frame, but the TEffect from FMX already have a cache system that avoids unnecessary repainting;

I don't fully agree. Yes, Skia is incredible for drawing shapes directly onto the form, but for physical images (e.g., .png, .jpg, etc.), you will always need to cache them. For example, even simple text like TText internally uses caching (via sk_paragraph_t). While it's true that simple shapes don't need caching, for other content, that's not always the case. This is why I don't question it—I cache everything in a double-buffered way. In fact, 80% of what you’ll draw on a classical form will contain text, and in those cases, caching is necessary.

Finally, I'm curious to know the relationship between this "4 times slower at drawing images onto the form" performance and #331, I mean, was the measurement made with your proposed changes?

There is no relationship between the two. The issue mentioned at #331 is primarily about allowing the possibility of continuing to work with Skia using textures created by third-party products via FMX.Context.GLES. In short, the context must remain the same between Skia and FMX.Context.GLES.ios / FMX.Context.GLES.android.

5 replies

viniciusfbb Oct 9, 2024
Maintainer

I didn’t use TBitmap at all—I worked directly with skImage using the Skia API. My test was specifically to compare drawing a TTexture (TCustomCanvasGpu(ACanvas).DrawTexture) versus drawing a skImage (sk4d_canvas_draw_image_rect(TSkCanvasCustom(ACanvas).Canvas.Handle, ...)) directly on the form.

@Zeus64 Just to clarify, there is nothing wrong on drawing with SkImage instead TBitmap, I'm just saying that it is possible to optimize this by using TBitmap or by creating your own cached SkImage with a texture internally.

It may seem confusing because there are 2 types of SkImage, SkImage from pixels (raster) and the SkImage from a texture. In short, you draw on a raster surface and generate a SkImage from that raster surface or create a raster SkImage from an image file, but then you can generate a second SkImage with an internal texture and reuse it, since the creation process is slow but once created, it will be instantaneous, just like the TTexture of TCanvasGpu. See our code:

skia4delphi/Source/FMX/FMX.Skia.Canvas.pas

Lines 2465 to 2472 in 90211b0

    
           procedure TGrSharedContext.InitializeTextureCache( 
        
             const ABitmap: TGrBitmapHandle); 
        
           var 
        
             LImage: ISkImage; 
        
           begin 
        
             LImage := TSkImage.MakeFromRaster(TSkImageInfo.Create(ABitmap.Width, ABitmap.Height, SkFmxColorType[ABitmap.PixelFormat]), ABitmap.Pixels, ABitmap.Width * PixelFormatBytes[ABitmap.PixelFormat]); 
        
             ABitmap.Cache := LImage.MakeTextureImage(ABitmap.SharedContext.GrDirectContext); 
        
           end;

Zeus64 Oct 9, 2024
Author

it will be instantaneous, just like the TTexture of TCanvasGpu

No unfortunately, it's 4x more slow than TTexture of TCanvasGpu, that the problem :( you can try this demo : https://github.com/MagicFoundation/Alcinoe/tree/master/Demos/ALFmxGraphics and you will see by yourself (compile one time with skia enabled, and second time with skia disabled. with skia disabled you can draw 4x more textures simultaneously than with skia enabled).

viniciusfbb Oct 9, 2024
Maintainer

@Zeus64 Thanks for sharing! I will do some tests on this weekend based on this and I'll share here.

viniciusfbb Oct 20, 2024
Maintainer

@Zeus64 I made a demo with 5000 TPaintBox on a TVertScrollbox with height=5000, with an automatically simulation of scrolling via touch, drawing an image using TPaintBox.OnPaint without stretch.

The results for Android on Samsung A15 device are:

FMX (without Skia) drawing TBitmap: 62 fps
Skia OpenGL drawing TBitmap: 17 fps
Skia Vulkan drawing TBitmap: 17 fps
FMX (without Skia) drawing TTexture: 64 fps
Skia OpenGL drawing SkImage from TTexture (using Stephane approach): 66 fps
Skia Vulkan drawing SkImage from TTexture: ?
Skia OpenGL drawing SkImage raster: 65 fps
Skia Vulkan drawing SkImage raster: 69 fps
Skia OpenGL drawing SkImage backed texture from another SkImage raster: 67 fps
Skia Vulkan drawing SkImage backed texture from another SkImage raster: 72 fps

Based on this, we can conclude:

There is a clear performance problem drawing TBitmap when Skia is enabled. There is something wrong on FMX.Skia.Canvas.pas, because when drawing directly using Skia APIs the performance is fast. We should investigate it;
What I said earlier about SkImage backed texture from another SkImage raster being faster may not be true. In the tests it showed very similar results, so I suspect that SkImage raster already has some internal mechanism for automatic texture caching.
The performance of TTexture is very similar to TBitmap (in fact TBitmap itself has a TTexture internally).

viniciusfbb Oct 20, 2024
Maintainer

The bad performance issue of Skia drawing TBitmap is now fixed with the latest commit 2ff08b1. The problem was that it uses a high quality filter to resize images but we were not disabling it for no stretched bitmap draws. Now the benchmark for their scenarios are:

Skia OpenGL drawing TBitmap: 58 fps
Skia Vulkan drawing TBitmap: 64 fps

Zeus64 · 2024-10-20T19:17:25Z

Zeus64
Oct 20, 2024
Author

@viniciusfbb thanks a lot! Can you share the code of your demo ? every tests i did show that sk4d_canvas_draw_image_rect is 4 times more slower than TCustomCanvasGpu(ACanvas).DrawTexture. I never used Tbitmap or any abstraction like this always work directly with skimage or TextureID

1 reply

viniciusfbb Oct 20, 2024
Maintainer

FmxFPS.zip

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Skia Integration in Delphi: Enhancing FireMonkey's Graphics with Pros, Cons, and Performance Insights #347

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 3 comments 6 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Skia Integration in Delphi: Enhancing FireMonkey's Graphics with Pros, Cons, and Performance Insights #347

Zeus64 Oct 8, 2024

Key Advantages of Skia:

Drawbacks of Skia:

Performance Challenges with Skia:

The Chosen Approach:

Platform-Specific Considerations:

Replies: 3 comments · 6 replies

viniciusfbb Oct 9, 2024 Maintainer

Zeus64 Oct 9, 2024 Author

viniciusfbb Oct 9, 2024 Maintainer

Zeus64 Oct 9, 2024 Author

viniciusfbb Oct 9, 2024 Maintainer

viniciusfbb Oct 20, 2024 Maintainer

viniciusfbb Oct 20, 2024 Maintainer

Zeus64 Oct 20, 2024 Author

viniciusfbb Oct 20, 2024 Maintainer

Zeus64
Oct 8, 2024

Replies: 3 comments 6 replies

viniciusfbb
Oct 9, 2024
Maintainer

Zeus64
Oct 9, 2024
Author

viniciusfbb Oct 9, 2024
Maintainer

Zeus64 Oct 9, 2024
Author

viniciusfbb Oct 9, 2024
Maintainer

viniciusfbb Oct 20, 2024
Maintainer

viniciusfbb Oct 20, 2024
Maintainer

Zeus64
Oct 20, 2024
Author

viniciusfbb Oct 20, 2024
Maintainer