Decode HEIC with correct color space #731

joshuamegnauth54 · 2025-09-10T07:43:04Z

Possibly closes: #710

Instead of always decoding as RGBA, I manually convert the image from its real color space to RGBA. I'm not sure if this fixes the issue, but it seems like a good thing to do anyway.

~~I still have to test this so it's a draft for now.~~

Possibly closes: woelper#710 Instead of always decoding as RGBA, I manually convert the image from its real color space to RGBA. I'm not sure if this fixes the issue, but it seems like a good thing to do anyway.

Stoppedpuma · 2025-09-12T14:36:40Z

This PR seems to break these images. This affects thumbnail generation as well.

Upclose:

joshuamegnauth54 · 2025-09-13T03:09:24Z

Do you have samples of the broken images so I can test locally? This patch was a bit hard to figure out, so I'll have to fiddle with the code more till it works. 😅

I tested it with a few HEIC images but not a large range of them because I couldn't find too many samples.

Stoppedpuma · 2025-09-13T03:16:24Z

These photos are a friends and I discovered this issue while they were viewing them on my laptop. As they are not mine, I'll ask for permission to send some of the originals for this PR. Do you have a preferred method I would be able to send a few of these privately to you?

joshuamegnauth54 · 2025-09-13T03:29:05Z

To clarify, some HEIC images worked but those specific images failed? If we can figure out the format, I can probably fix it without needing the images. If not, you could probably just email them (I think my email is in the commit).

For reference, I tested with these: https://heic.digital/samples/

Stoppedpuma · 2025-09-14T18:46:26Z

I've emailed you some sample images, please feel free to let me know if you need more! I can also send you the raws if needed.

awxkee

This implementation is missing many cases that were previously handled internally by libheif.

It’s necessary to account for the following:

8/10 bit depth ( 12 bit depth HEIC's is possible but rare, AFAIK no opensource tools can encode 12 bit, so it's probably not required for HEIC's)
YUV range changes
YUV matrix changes
GBR (identity matrix), which is technically YUV but slightly different and not especially rare.
Monochrome is YUV 4:0:0 that should be handled appropriately

There are also YCgCo images sometimes but as far as I know they are not handled out of the box in libheif so it should not be broken.

This won't help with HDR curves because they are different thing.

awxkee · 2025-09-15T19:17:06Z

src/image_loader.rs

-            let buf = image::ImageBuffer::from_vec(width as u32, height as u32, res)
-                .context("Can't create HEIC/HEIF ImageBuffer with given res")?;
-            let i = DynamicImage::ImageRgba8(buf);
+                Some(ColorSpace::YCbCr(chroma)) => {


I'm not sure if this branch is correct one.

It's missing:

Checking the image bit depth, and handling or rejecting unsupported depths. Typically, libheif supports 8-bit and 10-bit for HEIC.

Accounting that YUV range is subject to change.

Accounting for the YUV matrix, which may vary. For HEIC, it is usually BT.709 rather than BT.601. But it's needed to be queried from libheif.

awxkee · 2025-09-15T19:17:33Z

src/image_loader.rs

+            let img = lib_heif.decode(&handle, ColorSpace::Undefined, None)?;
+
+            let i = match img.color_space() {
+                Some(ColorSpace::Rgb(chroma @ (RgbChroma::Rgb | RgbChroma::Rgba))) => {


libheif can use 10 and 12 bits as well, not only 8 bit images.

awxkee · 2025-09-15T19:19:01Z

src/image_loader.rs

+                    let width = img.width();
+                    let height = img.height();
+
+                    let i = DynamicImage::ImageLuma8(


Monochrome used in HEIC is YUV image with only Y plane, so it's required to handle YUV range and YUV matrix as well and image also could be 10 bit.

joshuamegnauth54 · 2025-09-16T05:08:13Z

@awxkee I recently realized that I didn't have to do any of this. libheif seems to convert the image on load so the original code was correct. With that said, do you have any ideas concerning the original issue? I would love to fix it.

@Stoppedpuma Yes, I got them and thank you! I have a slightly busy week, but I'll get back to work on this soon as I can.

Stoppedpuma · 2025-09-16T15:01:17Z

@Stoppedpuma Yes, I got them and thank you! I have a slightly busy week, but I'll get back to work on this soon as I can.

No worries, take your time!

awxkee · 2025-09-16T20:27:21Z

@joshuamegnauth54

Typical flow to handle HDR image without gainmaps:

Check CICP in image, if it is not present then exit early.
Check transfer function stored in CICP, if it is not PQ or HLG image is not an HDR, at least for CICP case.
As you determined that stored transfer function is HDR one ( PQ or HLG ) this is the time to start actually handle it.
You need to determine at first if driver + display supports requested color space + transfer function. This is system and driver dependent step. Though, I'm not familiar with framework here. Typically needed something like this in Vulkan or in MacOS/iOS.
If HDR surface is available and you determined somehow that driver and display supports it, you just need create it, that's it.
If HDR surface is not available, then you need to perform a tonemapping. ITU-R has recommendation 2408, but this is recomendation only. Tonemapping itself is not strictly defined. It's kind of idea how to put something big into a something small. Thus, there are many possible approaches, but no single "exact" method. Here some tone mapper examples.
In most cases HDR tranfer function combined with Wide Gamut color spaces, if HDR surface is not available in most cases, wide gamut usually isn’t either. Therefore if you'll likely need along with tonemapping step convert Wide Gamut colorspace into sRGB, or some surface that you can create.

This is not actually as complicated as it may seems, but it does require some attention to details.

joshuamegnauth54 · 2025-10-07T02:01:27Z

Hmm, if this is dependent on the renderer then it may be best to wait until Oculante moves off Notan.

Stoppedpuma · 2025-10-07T15:48:48Z

Hmm, if this is dependent on the renderer then it may be best to wait until Oculante moves off Notan.

I agree, there's a lot of things that we have to change with our move away from Notan (and even for updating Notan before we move away from it).

Decode HEIC with correct color space

1cafbd3

Possibly closes: woelper#710 Instead of always decoding as RGBA, I manually convert the image from its real color space to RGBA. I'm not sure if this fixes the issue, but it seems like a good thing to do anyway.

joshuamegnauth54 marked this pull request as ready for review September 12, 2025 05:05

joshuamegnauth54 force-pushed the heic-use-correct-chroma-space branch from 3e3ef80 to 1cafbd3 Compare September 12, 2025 05:06

Merge branch 'master' into heic-use-correct-chroma-space

9fe569d

awxkee reviewed Sep 15, 2025

View reviewed changes

Decode HEIC with correct color space #731

Are you sure you want to change the base?

Decode HEIC with correct color space #731

Uh oh!

Conversation

joshuamegnauth54 commented Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Stoppedpuma commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joshuamegnauth54 commented Sep 13, 2025

Uh oh!

Stoppedpuma commented Sep 13, 2025

Uh oh!

joshuamegnauth54 commented Sep 13, 2025

Uh oh!

Stoppedpuma commented Sep 14, 2025

Uh oh!

awxkee left a comment

Choose a reason for hiding this comment

Uh oh!

awxkee Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

awxkee Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

awxkee Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

joshuamegnauth54 commented Sep 16, 2025

Uh oh!

Stoppedpuma commented Sep 16, 2025

Uh oh!

awxkee commented Sep 16, 2025

Uh oh!

joshuamegnauth54 commented Oct 7, 2025

Uh oh!

Stoppedpuma commented Oct 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

joshuamegnauth54 commented Sep 10, 2025 •

edited

Loading

Stoppedpuma commented Sep 12, 2025 •

edited

Loading