feat(provide): detailed ipfs provide stat
#11019
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@lidel what is the best place to add a brief description about individual metrics?
|
|
@guillaumemichel ok to include in |
283382b to
763db02
Compare
ipfs provide stat for Provide.DHT.SweepEnabled
ipfs provide stat for Provide.DHT.SweepEnabledipfs provide stat
ipfs provide stat ipfs provide stat
ipfs provide stat ipfs provide stat
- add extractSweepingProvider() helper to reduce nested type switching - extract lowWorkerThreshold constant for worker availability check - fix --lan error handling to work with buffered providers
lidel
reviewed
Oct 15, 2025
lidel
reviewed
Oct 15, 2025
- prevent panic when both columns are empty - fix column alignment with UTF-8 characters - only track col0MaxWidth for first column (as intended)
- test basic functionality, flags, JSON output - test legacy provider behavior - test integration with content scheduling - test disabled provider configurations - add parseSweepStats helper with t.Helper()
- update tagline to "Control and monitor content providing" - simplify help descriptions - make error messages more consistent - update tests to match new error messages
``` Next reprovide at: Next prefix: ``` updated to: ``` Next region prefix: Next region reprovide: ```
c89cb5a to
f2d68bc
Compare
Enhance documentation for the Provide system to better explain how provider records work and the differences between sweep and legacy modes. Changes to docs/config.md: - Provide section: add clear explanation of provider records and their role - Provide.DHT: add provider record lifecycle and two provider systems overview - Provide.DHT.Interval: explain relationship to expiration, contrast sweep vs legacy behavior - Provide.DHT.SweepEnabled: rewrite to explain legacy problem, sweep solution, and efficiency gains - Monitoring section: prioritize command-line tools (ipfs provide stat) before Prometheus Changes to core/commands/provide.go: - ipfs provide stat help: add explanation of provider records, TTL expiration, and how sweep batching works Changes to docs/changelogs/v0.39.md: - Add context about why stats matter for monitoring provider health - Emphasize real-time monitoring workflow with watch command - Explain what users can observe (rates, queues, worker availability)
Merged latest changes from origin/master while preserving go-libp2p-kad-dht v0.35.2-0.20251009080632-2c5b3769ca70 from this branch.
lidel
approved these changes
Oct 17, 2025
There was a problem hiding this comment.
@guillaumemichel lgtm, feel free to merge libp2p/go-libp2p-kad-dht#1144, then we can switch this to kad-dht@main and squash merge.
lidel
added a commit
that referenced
this pull request
Oct 24, 2025
resolved conflicts: - go.mod/go.sum: prefer versions from master (PR #11019 merged first) - docs/config.md: prefer clearer wording from master - docs/changelogs/v0.39.md: keep reprovide-alert warning feature updates from make mod_tidy: - go-log/v2: v2.8.1 -> v2.8.2 - go-dsqueue: v0.0.5 -> v0.1.0
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introducing stats for the new provide system (sweep), using
ipfs provide stat.Help section
Default behavior
Brief summary of the most important metrics:
If the provide is low on workers it also includes additional metrics on workers
And if the DHT provider is
disconnectedorofflineit also prints the Connectivity:All metrics
Compact mode
Since the output of
--allstats is quite long, it may not fit on the screen. The compact mode comes to the rescue. It was made to be run withwatchto monitor the provide metrics over time (watch ipfs provide stat --all --compact).Section only
It is possible to filter a single stats section
$ ipfs provide stat --network Avg record holders: 12.2 Peers swept: 1k (1,801) Reachable peers: 1k (1,055) (58%) Avg region size: 31.4 Full keyspace coverage: false Replication factor: 20Or multiple sections
$ ipfs provide stat --schedule --operations --workers Schedule: CIDs scheduled: 7k (7,926) Regions scheduled: 499 Avg prefix length: 9 Next reprovide at: 21:30:10 Next prefix: 100011010 Operations: Ongoing provides: 0 CIDs, 0 regions Ongoing reprovides: 0 CIDs, 0 regions Total CIDs provided: 8k (8,535) Total records provided: 104k (104,403) Total provide errors: 0 CIDs provided/min: 17.8 CIDs reprovided/min: 25.7 Region reprovide duration: 37.5s Avg CIDs/reprovide: 17.2 Regions reprovided (last cycle): 63 Workers: Active workers: 0 / 16 (max) Free workers: 13 Worker stats: Periodic Burst Active: 0 0 Dedicated: 2 1 Available: 15 14 Queued: 0 0 Max connections/worker: 16LAN DHT
If using the Dual DHT (default behavior),
ipfs provide statwill show numbers about the WAN DHT provider by default. If you want to show stats about the LAN DHT provider, you can use the--lanflag.My LAN client isn't active, since no other peers on my LAN.
When using the accelerated DHT client (along with Provide Sweep), there is only one DHT to display stats (WAN).
Demo
In a terminal session:
Monitor new stats in a second session:
Import a lot of CIDs in a third session:
ipfs get -p QmdmQXB2mzChmMeKY47C43LxUdg1NDJ5MWcKMKxDu7RgQm # XKCD Archives ~7k CIDsThe newly added CIDs will be added to the provide queue, and will be reprovided as needed.
Sit back and watch as the system gracefully handles the provider announcements in real-time 🚀
Checklist
--help)go-libp2p-kad-dhtdependency onmaster