ENH add auto .gitignore in the Memory cache folder - #1674
Conversation
tomMoral
left a comment
There was a problem hiding this comment.
For directory that we do not create, I would not add the .gitignore.
I also added a few nipicks.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1674 +/- ##
==========================================
- Coverage 95.51% 95.43% -0.08%
==========================================
Files 46 46
Lines 7711 7731 +20
==========================================
+ Hits 7365 7378 +13
- Misses 346 353 +7 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| The directory indicated by `path` is left in place, and its subdirectories | ||
| are erased. | ||
| The directory indicated by `path` is left in place, and its subdirectories and the | ||
| .gitignore file are erased. |
There was a problem hiding this comment.
An empty directory is not tracked by git anyways, but I assume there must be some benefit of re-using this directory somehow (so the .gitignore file should also be removed), otherwise I cannot see how users would benefit from a function like this one - compared to simply deleting the whole folder.
Let's say users have added other files into the cache folder - they would be wiped as well. This is what puzzles me the most about this...
There was a problem hiding this comment.
Good catch but I would not add .gitignore logic here, mostly for simplicity reasons.
rm_subdirs seems to imply that it deletes subdirectories so it was not deleting any file before and I think we can decide to keep it like this, at least not changing anything doesn't seem to have any downside
There was a problem hiding this comment.
Thank you, @lesteve, I have removed this. So the file stays in the new folder also after running rm_subdirs() or clearing cache.
There was a problem hiding this comment.
thank you for your comments, I have fixed the doctest failure by adding a verbosity level to the memory object in this example (still not sure why this didn't raise before, since it was unrelated to the PR) and have implemented your requested changes.
This PR is ready for a review.
| The directory indicated by `path` is left in place, and its subdirectories | ||
| are erased. | ||
| The directory indicated by `path` is left in place, and its subdirectories and the | ||
| .gitignore file are erased. |
There was a problem hiding this comment.
Thank you, @lesteve, I have removed this. So the file stays in the new folder also after running rm_subdirs() or clearing cache.
ogrisel
left a comment
There was a problem hiding this comment.
I reopened a tracking issue for the str vs pathlib.Path inconsistency.
+1 for merging this PR without resolving that bug and instead fix it in a dedicated PR.
| # automatically add `.gitignore` file to the cache folder; the condition is | ||
| # necessary because in Memory.__init__, the user passed `location` param is | ||
| # modified to be either `{location}` or `{location}/joblib `depending on input | ||
| # type |
There was a problem hiding this comment.
| # automatically add `.gitignore` file to the cache folder; the condition is | |
| # necessary because in Memory.__init__, the user passed `location` param is | |
| # modified to be either `{location}` or `{location}/joblib `depending on input | |
| # type | |
| # Automatically add `.gitignore` file to the cache folder. | |
| # XXX: the condition is necessary because in `Memory.__init__`, the user | |
| # passed `location` param is modified to be either `{location}` or | |
| # `{location}/joblib `depending on input type (`pathlib.Path` vs `str`). | |
| # The proper resolution of this inconsistency is tracked in: | |
| # https://github.com/joblib/joblib/issues/1684 |
There was a problem hiding this comment.
I commited this, but what is XXX supposed to mean or stand for?
|
@StefanieSenger can you please add an entry in the changelog to document this enhancement? |
Co-authored-by: Olivier Grisel <[email protected]>
tomMoral
left a comment
There was a problem hiding this comment.
A few more comments but I think we are close to the end
Co-authored-by: Thomas Moreau <[email protected]>
Co-authored-by: Thomas Moreau <[email protected]>
|
Thank you for reviewing and taking care of absolute paths, @tomMoral. I had not been aware that you could pass them. I have committed your suggestions. |
|
Merging! |
Closes #1643
This PR adds an automatically created
.gitignorefile to the cache folder created withMemory().__init__().Memory().clear()andrm_subdirs()are modified to delete joblib's.gitignorefile while removing the other joblib folders.PR also add a corresponding test.