@rm5248 Thank you for your contribution.

Please note that this is mainly a duplicate of #1201.

The suggested generic solution to this should be what's proposed in #1530.

@rm5248 Thank you for your contribution.

Please note that this is mainly a duplicate of #1201.

The suggested generic solution to this should be what's proposed in #1530.

While I don't disagree, it would be nice to have this(or something similar) available in the meantime. It's unclear to me as to if and when #1530 would be implemented, so this is a stopgap measure.

Adding the .bin match here would lead to the proliferation of matches in RAUC that we want to avoid (see #1201 (comment)). Due to backwards compatibility, we could never remove it again, so I don't see how merging this would be a stopgap measure.

The proper place is still in the build-system integration or explicit file types in the manifest (which would be a larger change).

Why do you want to support casync for bootloader images? They are so small that you usually don't save much. You can use CONVERT_ARGS=--ignore-image=<your bootloader class> to avoid converting the bootloader image when using the bundle.bbclass.

Adding the .bin match here would lead to the proliferation of matches in RAUC that we want to avoid (see #1201 (comment)). Due to backwards compatibility, we could never remove it again, so I don't see how merging this would be a stopgap measure.

My assumption is that whatever the long-term change that happens in the future would effectively do what this patch is doing, turning this into a stopgap solutiojn. This may not be a valid assumption though.

Why do you want to support casync for bootloader images? They are so small that you usually don't save much. You can use CONVERT_ARGS=--ignore-image=<your bootloader class> to avoid converting the bootloader image when using the bundle.bbclass.

The casync for the bootloader images doesn't work properly at the moment anyway(at least as far as I can tell), it always downloads it. For our purposes the devices that we are updating may be on a cell connection which is very limited in the amount of data per month that is allocated to each device, so if we can avoid downloading the new bootloader(which is the same as the old bootloader) that would definitely help.

As to why I don't just do CONVERT_ARGS=--ignore-image=<your bootloader class>, the answer is that it was not clear to me that this would be needed. It is unexpected to me that the bootloader image does not support casync, while the other image types do.

Adding the .bin match here would lead to the proliferation of matches in RAUC that we want to avoid (see #1201 (comment)). Due to backwards compatibility, we could never remove it again, so I don't see how merging this would be a stopgap measure.

My assumption is that whatever the long-term change that happens in the future would effectively do what this patch is doing, turning this into a stopgap solutiojn. This may not be a valid assumption though.

Some more information about the time span for backwards compatibility is in https://rauc.readthedocs.io/en/latest/basic.html#sec-compatibility.

Why do you want to support casync for bootloader images? They are so small that you usually don't save much. You can use CONVERT_ARGS=--ignore-image=<your bootloader class> to avoid converting the bootloader image when using the bundle.bbclass.

The casync for the bootloader images doesn't work properly at the moment anyway(at least as far as I can tell), it always downloads it. For our purposes the devices that we are updating may be on a cell connection which is very limited in the amount of data per month that is allocated to each device, so if we can avoid downloading the new bootloader(which is the same as the old bootloader) that would definitely help.

What's the slot type your bootloader is on?

Also note that a simpler approach for the bootloader might be to set install-same=false for this slot.

As to why I don't just do CONVERT_ARGS=--ignore-image=<your bootloader class>, the answer is that it was not clear to me that this would be needed. It is unexpected to me that the bootloader image does not support casync, while the other image types do.

If that's a new project, did you consider using 'adaptive updates' with block-hash-index as an alternative?

The casync for the bootloader images doesn't work properly at the moment anyway(at least as far as I can tell), it always downloads it. For our purposes the devices that we are updating may be on a cell connection which is very limited in the amount of data per month that is allocated to each device, so if we can avoid downloading the new bootloader(which is the same as the old bootloader) that would definitely help.

What's the slot type your bootloader is on?

eMMC boot partition.

Also note that a simpler approach for the bootloader might be to set install-same=false for this slot.

I can look into that.

As to why I don't just do CONVERT_ARGS=--ignore-image=<your bootloader class>, the answer is that it was not clear to me that this would be needed. It is unexpected to me that the bootloader image does not support casync, while the other image types do.

If that's a new project, did you consider using 'adaptive updates' with block-hash-index as an alternative?

Yes, in the testing that we did we found that casync performed much better(fewer bytes downloaded) than the adaptive updates.

The casync for the bootloader images doesn't work properly at the moment anyway(at least as far as I can tell), it always downloads it. For our purposes the devices that we are updating may be on a cell connection which is very limited in the amount of data per month that is allocated to each device, so if we can avoid downloading the new bootloader(which is the same as the old bootloader) that would definitely help.

What's the slot type your bootloader is on?

eMMC boot partition.

The corresponding type boot-emmc actually does not support casync.
I'm not sure if this was intentional. I had a look and did some minor rework there anyway, maybe I'll come around with an update/fix.

Also note that a simpler approach for the bootloader might be to set install-same=false for this slot.

I can look into that.

👍

As to why I don't just do CONVERT_ARGS=--ignore-image=<your bootloader class>, the answer is that it was not clear to me that this would be needed. It is unexpected to me that the bootloader image does not support casync, while the other image types do.

If that's a new project, did you consider using 'adaptive updates' with block-hash-index as an alternative?

Yes, in the testing that we did we found that casync performed much better(fewer bytes downloaded) than the adaptive updates.

Thank you for the feedback! For the bootloader, the adaptive update method 'block-hash-index' should not really give a benefit. For file system images, it requires some caution when creating them, like properly aligning inodes, etc.

Do you use the original casync implementation (The issue with that is that it is currently unmaintained) or an alternative implementation like desync or casync-nano?

Also, note that we are open to extending the adaptive update approach with more flexible approaches.

The casync for the bootloader images doesn't work properly at the moment anyway(at least as far as I can tell), it always downloads it. For our purposes the devices that we are updating may be on a cell connection which is very limited in the amount of data per month that is allocated to each device, so if we can avoid downloading the new bootloader(which is the same as the old bootloader) that would definitely help.

What's the slot type your bootloader is on?

eMMC boot partition.

The corresponding type boot-emmc actually does not support casync. I'm not sure if this was intentional. I had a look and did some minor rework there anyway, maybe I'll come around with an update/fix.

FWIW this PR does at least allow the casync upgrade to install.

Thank you for the feedback! For the bootloader, the adaptive update method 'block-hash-index' should not really give a benefit. For file system images, it requires some caution when creating them, like properly aligning inodes, etc.

Do you use the original casync implementation (The issue with that is that it is currently unmaintained) or an alternative implementation like desync or casync-nano?

We're using the original casync implementation with some patches to support mTLS authentication. The fact that casync is effectively unmaintained at the moment is not ideal, but as long as it continues to work we should be fine.

As to why I don't just do CONVERT_ARGS=--ignore-image=<your bootloader class>, the answer is that it was not clear to me that this would be needed. It is unexpected to me that the bootloader image does not support casync, while the other image types do.

If that's a new project, did you consider using 'adaptive updates' with block-hash-index as an alternative?

Yes, in the testing that we did we found that casync performed much better(fewer bytes downloaded) than the adaptive updates.

Would you be able to share the details of your test setup? Perhaps even the images which show the better performance of casync?

Long term, if/when adaptive updates provide better results, we may want to deprecate casync.

Would you be able to share the details of your test setup? Perhaps even the images which show the better performance of casync?

I can't share the images, but I can probably come up with a test setup, hopefully this week.

the basic setup is that there are two slots:

1. contains FIT image with kernel, DTB, etc.
2. readonly squashfs as rootfs

For testing how much data is transferred the output is not always clear, so one way we checked was to just do a wireshark capture and see how large the capture was.

the basic setup is that there are two slots:

1. contains FIT image with kernel, DTB, etc.

For FIT images, as they are not aligned to 4k, I'd expect casync to have better data reuse if don't change kernel & initramfs at the same time. Otherwise, even for kernel/initramfs images with a "small" changes, most of the data would be different due to compression anyway.

Do you see significant reuse even if you change all parts of the FIT?

2. readonly squashfs as rootfs

Due to how squashfs compresses data, you'd probably have better reuse when using block-hash-index with EROFS.

For testing how much data is transferred the output is not always clear, so one way we checked was to just do a wireshark capture and see how large the capture was.

Since RAUC v1.13, we log "downloaded x% of the full bundle" when the installation is done. That doesn't include casync data, though, as that is fetched by a different process. So a packet capture is probably better here.

@rm5248 Is this still needed for you? If so, do you have a test setup that you could share where the download differences could be reproduces?

@rm5248 Is this still needed for you? If so, do you have a test setup that you could share where the download differences could be reproduces?

For the download size issue, yes. I started on something but I haven't had a chance to finish it up.

For this particular issue, it looks like RAUC 1.15 fixes the bin files for eMMC; I'll see about testing and making sure that it is sufficient.

For this particular issue, it looks like RAUC 1.15 fixes the bin files for eMMC; I'll see about testing and making sure that it is sufficient.

Yes. Note, it doesn't change anything related to casync on eMMC boot partitions.

FTR: adaptive updates should work in theory as a side-effect of refactoring in #1752 in.

The part I did but did not mainline (, yet) is https://github.com/ejoerns/rauc/commits/enable-casync-adaptive-emmc/.
However, this should not be a recommendation to prefer casync ;)

This is not using the newest version of RAUC, but I have made a standalone project that is sufficiently close to our real setup: https://github.com/brivolabs/yocto-pine64-rauc

Assuming that I have set everything up properly, this is properly illustrating what we saw with the adaptive vs. casync upgrades. Both of the upgrades were done with artifacts generated from the same build. The wireshark captures clearly show that the normal upgrade downloads ~16MB while the casync is only ~200k. For our purposes where bandwidth is at a premium, the smaller download size is needed.

This is getting a bit away from the original issue though, should we split this discussion out?