Tags: tikv/tikv
Tags
crossbeam-skiplist: fix RefRange iterator memory leak (#19284) (#19293) close #19285 Fix RefRange::next() and next_back() to properly decrement old self.head/self.tail refcount. Signed-off-by: ekexium <[email protected]> Signed-off-by: lucasliang <[email protected]> Signed-off-by: you06 <[email protected]> Co-authored-by: ekexium <[email protected]> Co-authored-by: lucasliang <[email protected]> Co-authored-by: you06 <[email protected]>
compact-log-backup: pick some of fixs of restoring. (#19258) * sst_importer: set fill cache to false when merge downloaded SST files (#19229) close #19228 set fill cache to false when merge downloaded SST files Signed-off-by: Jianjun Liao <[email protected]> * sst_importer: ensure cleanup of temporary files on error (#19240) ref #17283, close #19239 sst_importer: enhance failpoint handling and ensure cleanup of temporary files on error - Ensure temporary files are cleaned up when a download fails in sst_importer - Add a new failpoint `download_files_ext_after_download` to inject errors right after batch downloads complete, improving test coverage of cleanup paths Signed-off-by: Juncen Yu <[email protected]> * Squashed commit of the following: commit 5a14b2b Author: Juncen Yu <[email protected]> Date: Mon Dec 29 15:11:06 2025 +0800 make clippy happy Signed-off-by: Juncen Yu <[email protected]> commit ace595e Merge: bd5bff3 147cacc Author: Juncen Yu <[email protected]> Date: Fri Dec 26 17:51:36 2025 +0800 Merge branch 'master' of https://github.com/tikv/tikv into resolve-compact-conflict commit bd5bff3 Merge: fde14ce 301984c Author: Juncen Yu <[email protected]> Date: Mon Dec 22 12:10:59 2025 +0800 Merge branch 'master' of https://github.com/tikv/tikv into resolve-compact-conflict Signed-off-by: Juncen Yu <[email protected]> commit fde14ce Author: Juncen Yu <[email protected]> Date: Tue Mar 18 15:44:06 2025 +0800 added more checks Signed-off-by: Juncen Yu <[email protected]> commit d0801cb Author: Juncen Yu <[email protected]> Date: Tue Mar 18 15:27:14 2025 +0800 added more case about collapsed rollback Signed-off-by: Juncen Yu <[email protected]> commit 6d2b994 Author: Juncen Yu <[email protected]> Date: Tue Mar 18 14:52:40 2025 +0800 compact-log-backup: try to resolve conflict encountering during compacting Signed-off-by: Juncen Yu <[email protected]> Signed-off-by: Juncen Yu <[email protected]> * *: upgrade cargo-deny version to make it compatible to further developments. Signed-off-by: lucasliang <[email protected]> --------- Signed-off-by: Jianjun Liao <[email protected]> Signed-off-by: Juncen Yu <[email protected]> Signed-off-by: lucasliang <[email protected]> Co-authored-by: Jianjun Liao <[email protected]> Co-authored-by: lucasliang <[email protected]>
raftstore: reset region approximate size/keys stats when it becomes l… …eader (#19181) (#19186) close #19180 Reset region's approximate size/keys stats when peer becomes leader. This can avoid peer report outdated stats to pd and caused unnecessary balance scheduling. Signed-off-by: glorv <[email protected]>
compact_log_backup: optimize compact (#18882) (#19168) * compact_log_backup: optimize compact (#18882) close #18843, close #18950 1. Remove read_buf_exact_size for s3 hyper client 2. Use cloud::blob::read_to_end to read migrations from futures::io::AsyncRead 3. Use bytes::Bytes to speed up deallocating MetaFile Signed-off-by: Jianjun Liao <[email protected]> Signed-off-by: Jianjun Liao <[email protected]> Signed-off-by: Juncen Yu <[email protected]> * mute clippy Signed-off-by: Juncen Yu <[email protected]> * make format Signed-off-by: Juncen Yu <[email protected]> --------- Signed-off-by: Jianjun Liao <[email protected]> Signed-off-by: Jianjun Liao <[email protected]> Signed-off-by: Juncen Yu <[email protected]>
compact_log_backup: optimize compact (#18882) (#19168) * compact_log_backup: optimize compact (#18882) close #18843, close #18950 1. Remove read_buf_exact_size for s3 hyper client 2. Use cloud::blob::read_to_end to read migrations from futures::io::AsyncRead 3. Use bytes::Bytes to speed up deallocating MetaFile Signed-off-by: Jianjun Liao <[email protected]> Signed-off-by: Jianjun Liao <[email protected]> Signed-off-by: Juncen Yu <[email protected]> * mute clippy Signed-off-by: Juncen Yu <[email protected]> * make format Signed-off-by: Juncen Yu <[email protected]> --------- Signed-off-by: Jianjun Liao <[email protected]> Signed-off-by: Jianjun Liao <[email protected]> Signed-off-by: Juncen Yu <[email protected]>
tikv-ctl: pick compact-log-bakcup (#19164) * br: batch download and merge download sst before ingest (#19062) close #19086 Add a new PRC method called batch-download to download batch SST. Signed-off-by: RidRisR <[email protected]> Signed-off-by: Juncen Yu <[email protected]> * fix build Signed-off-by: Juncen Yu <[email protected]> * make format Signed-off-by: Juncen Yu <[email protected]> * implement the base library for compacting logs (#17632) close #17631 Added a new crate named `compact-log-backup`. Now it can merge some log files generated by log backup and make them become SSTs. Signed-off-by: hillium <[email protected]> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <[email protected]> * added `compact-log-bakcup` to `tikv-ctl` (#17845) close #17844 Signed-off-by: hillium <[email protected]> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <[email protected]> * compact_log_backup: record `min_input_ts` and `max_input_ts` in Compaction (#18085) close #18084 `min_input_ts` and `max_input_ts` will present in a log files compaction. Signed-off-by: hillium <[email protected]> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <[email protected]> * compact_log_backup: fix typo (#18090) ref #15990 Fixed a typo: `Migartion` -> `Migration`. Signed-off-by: hillium <[email protected]> Signed-off-by: Juncen Yu <[email protected]> * compact_log_backup: filter out meta files by migration (#18123) close #18122 Now, `StreamMetaStorage` is able to filter out files by meta edits. Signed-off-by: hillium <[email protected]> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <[email protected]> * compact_log_backup: added minimal compactions size (#18235) close #18234 Added `--minimal-compact-size` to `compact-log-backup`. Signed-off-by: hillium <[email protected]> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <[email protected]> * log backup: fix several issues during compact log backup. (#18298) close #18308 log backup compact: fix several issues during compact a log backup Signed-off-by: 3pointer <[email protected]> Signed-off-by: Juncen Yu <[email protected]> * compact_log_backup: correct version assignment in subcompaction metadata (#18389) close #18390 Fixed a bug that caused the time range of compaction generated SSTs are too huge. Signed-off-by: Juncen Yu <[email protected]> * compact_log_backup: add new field to track fully compacted data KV files and fix metafile filtering (#18837) close #18843 compact_log_backup: add new field to track fully compacted data KV files and fix metafile filtering Signed-off-by: 3pointer <[email protected]> Signed-off-by: Juncen Yu <[email protected]> * compact_log_backup: use max ts among all storage checkpoint ts (#18848) close #18847 Now, `consistency` hook checks the storage checkpoint by the max value among them. Signed-off-by: Juncen Yu <[email protected]> * compact_log_backup: fix compact meta edit filter (#18842) close #18843 Merge the same meta edit from different migrations instead of replacing. Signed-off-by: Jianjun Liao <[email protected]> Signed-off-by: Juncen Yu <[email protected]> * compact_log_backup: offload reading meta to diff cpus (#18885) close #18884 This PR spawns read s3 file tasks to remote threads. Signed-off-by: Juncen Yu <[email protected]> * compact_log_backup: read meta from checkpoint (#19068) close #19069 This PR makes `compact-log-backup` fills the migration with subcompactions skipped by checkpoint. Signed-off-by: hillium <[email protected]> Signed-off-by: Juncen Yu <[email protected]> * fix build Signed-off-by: Juncen Yu <[email protected]> --------- Signed-off-by: RidRisR <[email protected]> Signed-off-by: Juncen Yu <[email protected]> Signed-off-by: hillium <[email protected]> Signed-off-by: 3pointer <[email protected]> Signed-off-by: Jianjun Liao <[email protected]> Signed-off-by: 山岚 <[email protected]> Co-authored-by: ris <[email protected]> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Co-authored-by: 3pointer <[email protected]> Co-authored-by: Jianjun Liao <[email protected]>
Release-8.5-20251126-v8.5.4 (#19147) * br: batch download and merge download sst before ingest (#19062) close #19086 Add a new PRC method called batch-download to download batch SST. Signed-off-by: RidRisR <[email protected]> Signed-off-by: Juncen Yu <[email protected]> * fix build Signed-off-by: Juncen Yu <[email protected]> * make format Signed-off-by: Juncen Yu <[email protected]>
metrics: observe block read time at command level (#19089) (#19117) ref #19075 Retrieve `block_read_time` from the Perf Context, record it into `SchedDetails`, and add a histogram for monitoring. Also let `SchedDetails` record the duration of the async snapshot. Signed-off-by: ti-chi-bot <[email protected]> Signed-off-by: zyguan <[email protected]> Co-authored-by: zyguan <[email protected]>
[cp-v7.5.6] pd: Optimize metrics to limit metrics count (#19116) ref #18797 Limit the collection of `network_duration` Signed-off-by: okjiang <[email protected]>
PreviousNext