Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@Hritik14
Copy link

What is the purpose of this change?

Fixes: #8745

Was the change discussed in an issue or in the forum before?

Forum: https://forum.rclone.org/t/accept-directory-names-in-exclude-if-present/52248/1
Issue: Fixes: #8745

Checklist

  • I have read the contribution guidelines.
  • I have added tests for all changes in this PR if appropriate.
  • I have added documentation for the changes if appropriate.
  • All commit messages are in house style.
  • I'm done, this Pull Request is ready for review :-)

Copy link
Member

@ncw ncw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would also need to change ListContainsExcludeFile I think.

These could do with tests and docs also, and the help for --exclude-if-present would need adjusting.

I am slightly concerned that this is a backwards incompatible change though.

Perhaps we should require directories in --exclude-if-present to have trailing / so you'd write --exclude-if-present .ignoreFile --exclude-if-present .ignoreDir/ which would fix the backwards incompatibility problem.

What do you think?

@Hritik14
Copy link
Author

This would also need to change ListContainsExcludeFile I think.

I'm not sure of the changes there. I suppose obj, ok := entry.(fs.Object) fails because dir is not considered an object ?

require directories in --exclude-if-present to have trailing /

Makes a lot of sense. Added this.

These could do with tests and docs also, and the help for --exclude-if-present would need adjusting.

I'm happy to make those changes after we agree on an implementation :)

Copy link
Member

@ncw ncw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like the right approach now (see notes inline).

Please carry on with docs and tests - thank you :-)

if exists {
return true, nil
}
if err == fs.ErrorIsDir {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like the right approach. The logic needs to go like this though otherwise it ignores errors on the exists check

if err != nil {
    if errors.Is(err, fs.ErrorIsDir) && strings.HasSuffix(excludeFile, "/") {
      return true, nil
    }
    return false, err
}
if exists {
    return true, nil
}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

above errors out in case exclude file doesn't have a / but name corresponds to a dir.

Eg:

test2
├── node-project
│   └── lol
└── python-project
    └── .venv/
rclone ls /Users/neo/tmp/test2 --exclude-if-present '.venv' --exclude-if-present '.git'
2025/08/30 19:05:19 ERROR : error listing: is a directory not a file
2025/08/30 19:05:19 NOTICE: Failed to ls with 2 errors: last error was: is a directory not a file

I wonder if we should even check for / suffix, since, rclone just crashes if it encounters a ignore-file name that exists as a directory in the fs.
Anyone relying on this behaviour is unlikely.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ha, yes you are right. Existing rclone has a bug! I was trying to preserve the existing behaviour, but let's fix the bug

So that makes your original solution nearly right but I think we need to add errors.Is, so

-                       if err == fs.ErrorIsDir {
+                       if errors.Is(err, fs.ErrorIsDir) {
                                return strings.HasSuffix(excludeFile, "/"), nil
                        }

Can you do that then add tests and docs (in docs/content/filtering.md also, and the help for --exclude-if-present would need adjusting.

Help: "Exclude directories if filename is present",

Maybe something like

"Exclude directories if file or dir/ is present"

Thanks

Copy link
Author

@Hritik14 Hritik14 Aug 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Exclude directories if file or dir/ is present"

given it was a bug, do we even require the trailing / for directories ? most of the other commands do not distinguish against file/dir in this manner, (say copy).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the consensus is --exclude-if-present file and --exclude-if-present dir/.

Do you want to finish of the PR assuming that and we can get it merged - thank you

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree with the reasoning by @nielash. I'm a little occupied with my day job but I do intend to finish the rest of it soon.

@ncw
Copy link
Member

ncw commented Aug 16, 2025

I think ListContainsExcludeFile needs to look something like this (untested)

// ListContainsExcludeFile checks if exclude file is present in the list.
func (f *Filter) ListContainsExcludeFile(entries fs.DirEntries) bool {
	if len(f.Opt.ExcludeFile) == 0 {
		return false
	}
	for _, entry := range entries {
		basename := path.Base(entry.Remote())
		switch entry.(type) {
		case fs.Object:
			if slices.Contains(f.Opt.ExcludeFile, basename) {
				return true
			}
		case fs.Directory:
			if slices.Contains(f.Opt.ExcludeFile, basename+"/") {
				return true
			}
		}
	}
	return false
}

@ncw
Copy link
Member

ncw commented Aug 31, 2025

The options

  1. --exclude-if-present name should exclude the containing directory if name is a file or directory
  2. --exclude-if-present name should exclude the containing directory if name is a file only and we use --exclude-if-present name/ to exclude a directory

What I normally do here is look for prior art to see what other programs do. Here is a selection of programs with an --exclude-if-present flag or similar. I didn't actually try these, I looked at the docs so they may not be 100%

Program Exclude can be a Directory
borg Yes
restic No
duplicity No
rdiff-backup Yes
tar No
bupstash No

There isn't a strong consensus for whether files and directories should be treated differently or not.

Given the fact that currently if --exclude-if-present finds a directory with that name it errors the program, I think we are free to make our own choice.

If we choose option 2) if you want files and directories you can always add --exclude-if-present name --exclude-if-present name/ which makes it more flexible.

@albertony @nielash do you have an opinion?

@nielash
Copy link
Contributor

nielash commented Sep 1, 2025

@albertony @nielash do you have an opinion?

I lean slightly towards option 2, for a few reasons.

  • It is more consistent with the "How filter rules are applied to directories" section of the docs
  • It is more flexible (to your point about including the flag twice)
  • Treating files and dirs as interchangeable could lead to unexpected results (suppose I want it to apply to dirs but not to files with the same name)

I hear the point about rclone copy accepting either a dir or file as arguments, but that's slightly different from filtering rules, which do care about the trailing slash. (I also have always found the copy behavior a bit odd, as the docs specifically instruct: "To copy single files, use the copyto command instead.")

Also, it may or may not be worth revisiting #4523 at the same time.

@albertony
Copy link
Contributor

I lean slightly towards option 2, for a few reasons.

I agree - with everything you said there!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Accept directory names in –exclude-if-present

4 participants