The associated forum post URL from https://forum.rclone.org

What is the problem you are having with rclone?

Using rclone with an S3-compatible remote (R2 in my case) requires doing a HEAD request to get the modtime of a file. This is very slow with many files, so I use --checkers=100 --transfers=100 --fast-list to speed up sync/copy commands on directories with many files (in the 100k+ range).

This normally works great, but I had a weird issue where I ran a sync command to sync from local to an R2 bucket for the first time with the following command:

rclone sync \
  /home/jh/opt/atm10-server/bluemap/web \
  r2:bluemap/maps/servers/atm10-2026 \
  -vv \
  --transfers 100 \
  --checkers 100 \
  --fast-list \
  --min-age 1m \
  --dump-headers \
  --log-file rclone-bug.log

After this transfer finished copying 752 files, I stopped it early, and then started it again. The second time, it spent several minutes with no output (not even showing "checking" output like I'm used to - it just seemed to freeze).

Looking at the logs (rclone-bug.log), I noticed something weird: it's heading files VERY slowly (maybe 2-4 per second).

When testing without --min-age, the checks run very fast, so I am suspecting that there's a bug where using --min-age, we don't respect the concurrency set by --checkers

For comparison, here's a sync without --min-age where there were 743 files already on R2, and it started transferring files within a few seconds: rclone-bug.log

What is your rclone version (output from rclone version)

rclone v1.72.1
- os/version: ubuntu 24.04 (64 bit)
- os/kernel: 6.14.0-36-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.25.5
- go/linking: static
- go/tags: none

Which OS you are using and how many bits (e.g. Windows 7, 64 bit)

Ubuntu 24.04 (AMD64)

Which cloud storage system are you using? (e.g. Google Drive)

Cloudflare R2

The command you were trying to run (e.g. rclone copy /tmp remote:tmp)

rclone sync
/home/jh/opt/atm10-server/bluemap/web
r2:bluemap/maps/servers/atm10-2026
-vv
--transfers 100
--checkers 100
--fast-list
--min-age 1m
--dump-headers
--log-file rclone-bug.log

A log from the command with the -vv flag (e.g. output from rclone -vv copy /tmp remote:tmp)

https://gist.githubusercontent.com/jahands/0eb84d4d774476776173496ecc302ffd/raw/47536b74c2dc1023d72aceb697bd79cf319b5f8b/rclone-bug.log

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.