Thanks to visit codestin.com
Credit goes to github.com

Skip to content
This repository was archived by the owner on Aug 21, 2023. It is now read-only.

dump: always split TiDB v3.* tables through tidb rowid to save TiDB's memory (#301)#308

Merged
ti-chi-bot merged 27 commits intopingcap:release-5.1from
ti-chi-bot:cherry-pick-301-to-release-5.1
Jul 23, 2021
Merged

dump: always split TiDB v3.* tables through tidb rowid to save TiDB's memory (#301)#308
ti-chi-bot merged 27 commits intopingcap:release-5.1from
ti-chi-bot:cherry-pick-301-to-release-5.1

Conversation

@ti-chi-bot
Copy link
Member

@ti-chi-bot ti-chi-bot commented Jul 13, 2021

This is an automated cherry-pick of #301

What problem does this PR solve?

fix #295, an extension of #273
In TiDB v3.x:

  1. table INFORMATION_SCHEMA.TIKV_REGION_STATUS doesn't have table_name/table_schema.
  2. doesn't have SHOW TABLE REGIONS statement.
  3. table INFORMATION_SCHEMA.PARTITIONS doesn't actually have data.
  4. doesn't have tidb_decode_key function.
mysql> show create table information_schema.TIKV_REGION_STATUS;
+--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table              | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| TIKV_REGION_STATUS | CREATE TABLE `TIKV_REGION_STATUS` (
  `REGION_ID` bigint(21) unsigned DEFAULT NULL,
  `START_KEY` text DEFAULT NULL,
  `END_KEY` text DEFAULT NULL,
  `EPOCH_CONF_VER` bigint(21) unsigned DEFAULT NULL,
  `EPOCH_VERSION` bigint(21) unsigned DEFAULT NULL,
  `WRITTEN_BYTES` bigint(21) unsigned DEFAULT NULL,
  `READ_BYTES` bigint(21) unsigned DEFAULT NULL,
  `APPROXIMATE_SIZE` bigint(21) unsigned DEFAULT NULL,
  `APPROXIMATE_KEYS` bigint(21) unsigned DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin |
+--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

What is changed and how it works?

Split chunks by PKIsHandle for TiDB v3.*. Make sure TiDB won't OOM through splitting tables by rowids.
We do some staff by our own to solve the problem that tidb lacks some info.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    Test dump with v3.0.0. The result is looks expected.

Related changes

  • Need to cherry-pick to the release branch

Release note

  • Always split TiDB v3.* tables through tidb rowid to save TiDB's memory.

@ti-chi-bot ti-chi-bot mentioned this pull request Jul 13, 2021
Merged
@ti-chi-bot ti-chi-bot requested a review from kennytm July 13, 2021 07:05
@ti-chi-bot ti-chi-bot requested a review from lichunzhu July 13, 2021 07:05
lichunzhu
lichunzhu approved these changes Jul 23, 2021
View reviewed changes
Copy link
Contributor

@lichunzhu lichunzhu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot
Copy link
Member Author

ti-chi-bot commented Jul 23, 2021

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • lichunzhu

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Details

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added the status/LGT1 One reviewer approved (LGTM1) label Jul 23, 2021
@lichunzhu
Copy link
Contributor

lichunzhu commented Jul 23, 2021

/merge

@ti-chi-bot
Copy link
Member Author

ti-chi-bot commented Jul 23, 2021

This pull request has been accepted and is ready to merge.

DetailsCommit hash: f96c905

@ti-chi-bot ti-chi-bot merged commit 279359a into pingcap:release-5.1 Jul 23, 2021
@liuzix liuzix mentioned this pull request Jul 28, 2021
Merged
12 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@kennytm kennytm Awaiting requested review from kennytm

1 more reviewer

@lichunzhu lichunzhu lichunzhu approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@lichunzhu lichunzhu

Labels

cherry-pick-approved size/XXL status/can-merge status/LGT1 One reviewer approved (LGTM1)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ti-chi-bot @lichunzhu @zhouqiang-cl