feat(file-watcher): start the watcher without blocking the main thread #437

altsem · 2025-10-11T12:48:21Z

@jonathanj put this back in a thread, I can't see how it wouldn't work on Mac. 🤔
This time around the watcher doesn't move around threads.

previously reverted in: #360
related: #358, #374

jonathanj · 2025-10-11T13:04:53Z

The thing I noticed last time, and maybe the improved gitignore stuff I added helps, was that starting it in an async thread caused a surge of file change events in the initial check, for whatever reason.

I'm happy to build this branch on macOS and check that the FS change events work.

jonathanj · 2025-10-11T13:14:08Z

Seems to be fine to me. Was starting this on the main thread the cause of a startup delay?

jonathanj · 2025-10-11T13:30:15Z

Here's my gitu.log for this session:

gitu.log

[00:00:00.000] (2018c60c0) DEBUG  Initializing terminal backend
[00:00:00.003] (2018c60c0) DEBUG  Starting app
[00:00:00.003] (2018c60c0) DEBUG  Finding git dir
[00:00:00.040] (2018c60c0) DEBUG  Opening repo
[00:00:00.042] (2018c60c0) INFO   Loading config file at "/Users/jonathan/.config/gitu/config.toml"
[00:00:00.066] (2018c60c0) WARN   Couldn't read /Users/jonathan/Coding/gitu/.git/rebase-merge/onto, due to No such file or directory (os error 2)
[00:00:00.066] (2018c60c0) WARN   Couldn't read /Users/jonathan/Coding/gitu/.git/MERGE_HEAD, due to No such file or directory (os error 2)
[00:00:00.066] (2018c60c0) WARN   Couldn't read /Users/jonathan/Coding/gitu/.git/REVERT_HEAD, due to No such file or directory (os error 2)
[00:00:00.114] (16b897000) DEBUG  Opening repo
[00:00:00.118] (16b897000) INFO   File watcher started (kind: Fsevent)
[00:05:31.051] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/4913" (Create(File))
[00:05:31.052] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/4913" (Remove(File))
[00:05:31.052] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/4913" (Modify(Metadata(Ownership)))
[00:05:31.053] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/4913" (Modify(Metadata(Extended)))
[00:05:31.053] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Modify(Name(Any)))
[00:05:31.053] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Create(File))
[00:05:31.053] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Modify(Metadata(Ownership)))
[00:05:31.053] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Modify(Metadata(Extended)))
[00:05:31.054] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Modify(Data(Content)))
[00:05:31.054] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml~" (Remove(File))
[00:05:31.054] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml~" (Modify(Name(Any)))
[00:05:31.167] (2018c60c0) WARN   Couldn't read /Users/jonathan/Coding/gitu/.git/rebase-merge/onto, due to No such file or directory (os error 2)
[00:05:31.167] (2018c60c0) WARN   Couldn't read /Users/jonathan/Coding/gitu/.git/MERGE_HEAD, due to No such file or directory (os error 2)
[00:05:31.167] (2018c60c0) WARN   Couldn't read /Users/jonathan/Coding/gitu/.git/REVERT_HEAD, due to No such file or directory (os error 2)
[00:05:37.724] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/4913" (Create(File))
[00:05:37.728] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/4913" (Remove(File))
[00:05:37.729] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/4913" (Modify(Metadata(Ownership)))
[00:05:37.729] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/4913" (Modify(Metadata(Extended)))
[00:05:37.730] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Create(File))
[00:05:37.730] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Modify(Name(Any)))
[00:05:37.730] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Modify(Metadata(Ownership)))
[00:05:37.731] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Modify(Metadata(Extended)))
[00:05:37.731] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Modify(Data(Content)))
[00:05:37.731] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Create(File))
[00:05:37.731] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Modify(Metadata(Ownership)))
[00:05:37.731] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Modify(Metadata(Extended)))
[00:05:37.731] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml" (Modify(Data(Content)))
[00:05:37.731] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml~" (Remove(File))
[00:05:37.732] (16baa3000) INFO   File changed: "/Users/jonathan/Coding/gitu/codecov.yml~" (Modify(Name(Any)))
[00:05:37.792] (2018c60c0) WARN   Couldn't read /Users/jonathan/Coding/gitu/.git/rebase-merge/onto, due to No such file or directory (os error 2)
[00:05:37.793] (2018c60c0) WARN   Couldn't read /Users/jonathan/Coding/gitu/.git/MERGE_HEAD, due to No such file or directory (os error 2)
[00:05:37.793] (2018c60c0) WARN   Couldn't read /Users/jonathan/Coding/gitu/.git/REVERT_HEAD, due to No such file or directory (os error 2)

There are a couple of interesting things:

Changes to whatever gitu/4913 is, that's not the file I modified. I don't think this is my editor, this file is immediately deleted but gitu incurs the cost of processing it
Errors reading .git/… which happens before I even make any changes, which is a bit curious: I'm not sure how the cost of this might vary from system to system, or how many of these could be for a large and old checkout, maybe it's nothing but it seems a bit odd regardless.
Events for metadata: Every one of these events potentially incurs some amount of processing for gitu, which may produce no visible UI changes in any way

Maybe none of these are interesting to #374 but I can't really tell since I don't have a repo/system that has these issues, unfortunately.

altsem · 2025-10-11T13:34:04Z

Nice! It seem like it is a pretty significant amount in larger repos. I found this in the man-page:

       Inotify monitoring of directories is not recursive: to monitor
       subdirectories under a directory, additional watches must be
       created.  This can take a significant amount time for large
       directory trees.

The filtering done: repo.status_should_ignore(&path) is after already having registered watches on every single file.
Feels like not watching everything could be a win as well.

jonathanj · 2025-10-11T13:50:50Z

The filtering done: repo.status_should_ignore(&path) is after already having registered watches on every single file. Feels like not watching everything could be a win as well.

Do you have any idea of how to approach this? Browsing around the docs for notify I don't see a way to cause the watcher to not watch certain paths.

jonathanj · 2025-10-11T13:52:07Z

Nice! It seem like it is a pretty significant amount in larger repos. I found this in the man-page:

I noticed you asked this same question on the other thread, but how does magit (or is it emacs?) manage not to run into the same issue?

jonathanj · 2025-10-11T13:56:33Z

Also, in this comment the log shows FsEvents not INotify as the watcher kind, so is the inotify manpage still relevant in this case?

altsem · 2025-10-11T14:16:23Z

True. But perhaps he tested it on a Mac as well.
FsEvents seems to be unique for Mac, the Linux impl relies on INotify. And INotify doesn't natively support watching recursively.

It'd be perfect if the notify crate supported filters on files, but it doesn't seem to.
I'll keep digging.

altsem · 2025-10-18T07:06:57Z

I'll merge this in and create a new issue:

File-watcher listening on gitignored files

Copilot

Pull Request Overview

Refactors the file watcher to run in a separate thread to prevent blocking the main thread during initialization. The watcher is now started asynchronously and uses std::mem::forget to prevent it from being dropped.

Moved file watching logic into a separate watch function that runs in a spawned thread
Removed the _watcher field from the FileWatcher struct since the watcher now runs independently
Added error logging for file watcher failures in the background thread

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

src/file_watcher.rs

altsem force-pushed the feat/start-watcher-async branch from e84d8f0 to f23e15c Compare October 11, 2025 12:49

perf(file-watcher): start the watcher without blocking the main thread

cf21bd1

altsem force-pushed the feat/start-watcher-async branch from f23e15c to cf21bd1 Compare October 11, 2025 12:49

altsem mentioned this pull request Oct 18, 2025

File-watcher listening on gitignored files on Linux #438

Open

altsem requested a review from Copilot October 18, 2025 07:13

Copilot AI reviewed Oct 18, 2025

View reviewed changes

src/file_watcher.rs Show resolved Hide resolved

src/file_watcher.rs Show resolved Hide resolved

altsem merged commit 3b589c6 into master Oct 18, 2025
3 checks passed

altsem deleted the feat/start-watcher-async branch October 18, 2025 07:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(file-watcher): start the watcher without blocking the main thread #437

feat(file-watcher): start the watcher without blocking the main thread #437

Uh oh!

altsem commented Oct 11, 2025 •

edited

Loading

Uh oh!

jonathanj commented Oct 11, 2025

Uh oh!

jonathanj commented Oct 11, 2025

Uh oh!

jonathanj commented Oct 11, 2025

Uh oh!

altsem commented Oct 11, 2025

Uh oh!

jonathanj commented Oct 11, 2025

Uh oh!

jonathanj commented Oct 11, 2025

Uh oh!

jonathanj commented Oct 11, 2025

Uh oh!

altsem commented Oct 11, 2025 •

edited

Loading

Uh oh!

altsem commented Oct 18, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

feat(file-watcher): start the watcher without blocking the main thread #437

feat(file-watcher): start the watcher without blocking the main thread #437

Uh oh!

Conversation

altsem commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonathanj commented Oct 11, 2025

Uh oh!

jonathanj commented Oct 11, 2025

Uh oh!

jonathanj commented Oct 11, 2025

Uh oh!

altsem commented Oct 11, 2025

Uh oh!

jonathanj commented Oct 11, 2025

Uh oh!

jonathanj commented Oct 11, 2025

Uh oh!

jonathanj commented Oct 11, 2025

Uh oh!

altsem commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

altsem commented Oct 18, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

altsem commented Oct 11, 2025 •

edited

Loading

altsem commented Oct 11, 2025 •

edited

Loading