Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Refactor clang_tidy.py#61119

Closed
1ntEgr8 wants to merge 7 commits into
gh/1ntEgr8/12/basefrom
gh/1ntEgr8/12/head
Closed

Refactor clang_tidy.py#61119
1ntEgr8 wants to merge 7 commits into
gh/1ntEgr8/12/basefrom
gh/1ntEgr8/12/head

Conversation

@1ntEgr8
Copy link
Copy Markdown
Contributor

@1ntEgr8 1ntEgr8 commented Jul 1, 2021

Stack from ghstack:

Summary:

This change spilts the clang-tidy CI job into smaller steps and uses a
refactored version of the clang_tidy.py script.

The new folder structure is as follows:

tools/linter/clang_tidy
|_ __main__py
|_ requirements.txt
|_ run.py
|_ setup.sh

__main__.py

This script will run tools/linter/clang_tidy/setup.sh if a build
directory doesn't exist, mimicing what used to be done as a separate
step in the CI job.

After that, it will invoke clang-tidy with default arguments being
declared in the script itself (as opposed to declaring them in
lint.yml).

The reasoning behind this approach is two-fold:

  • Make it easier to run clang-tidy locally using this script
  • De-duplicate the option passing

requirements.txt

Contains a list of additional python dependencies needed by the
clang-tidy script.

setup.sh

If a build directory doesn't exist, this command will run the necessary
codegen and build commands for running clang-tidy

Example usage:

python3 tools/linter/clang_tidy --parallel

Notice that we don't have to put the .py at the end of clang_tidy.

Test Plan:

Run the following command:

python3 tools/linter/clang_tidy --paths torch/csrc/fx --parallel

Differential Revision: D29568582

Summary:

This change spilts the clang-tidy CI job into smaller steps and uses a
refactored version of the clang_tidy.py script.

The new folder structure is as follows:
```
tools/linter/clang_tidy
|_ __main__py
|_ requirements.txt
|_ run.py
|_ setup.sh
```

`__main__.py`

This script will run `tools/linter/clang_tidy/setup.sh` if a `build`
directory doesn't exist, mimicing what used to be done as a separate
step in the CI job.

After that, it will invoke `clang-tidy` with default arguments being
declared into the script itself (as opposed to declaring them in
lint.yml).

The reasoning behind this approach is two-fold:

- Make it easier to run `clang-tidy` locally using this script
- De-duplicate the option passing

`requirements.txt`

Contains a list of additional python dependencies needed by the
`clang-tidy` script.

`setup.sh`

If a build directory doesn't exist, this command will run the necessary
codegen and build commands for running `clang-tidy`

Example usage:
```
python3 tools/linter/clang_tidy --parallel
```
Notice that we don't have to put the `.py` at the end of `clang_tidy`.

Test Plan:

Run the following command:
```
python3 tools/linter/clang_tidy --paths torch/csrc/fx --parallel
```

[ghstack-poisoned]
@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Jul 1, 2021

💊 CI failures summary and remediations

As of commit 0651ac5 (more details on the Dr. CI page and at hud.pytorch.org/pr/61119):


  • 2/2 failures possibly* introduced in this PR
    • 1/2 non-scanned failure(s)

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See GitHub Actions build Lint / mypy (1/1)

Step: "Run mypy" (full log | diagnosis details | 🔁 rerun)

2021-07-06T22:35:00.2340542Z aten/src/ATen/nati...ive/quantized/cpu/qnnpack/deps/clog/configure.py')
2021-07-06T22:34:35.8311098Z env:
2021-07-06T22:34:35.8311644Z   pythonLocation: /opt/hostedtoolcache/Python/3.8.10/x64
2021-07-06T22:34:35.8312330Z   LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.8.10/x64/lib
2021-07-06T22:34:35.8312992Z ##[endgroup]
2021-07-06T22:34:35.8419375Z + STATUS=
2021-07-06T22:34:35.8421787Z + for CONFIG in mypy*.ini
2021-07-06T22:34:35.8423987Z + mypy --config=mypy-strict.ini
2021-07-06T22:34:53.6795080Z Success: no issues found in 149 source files
2021-07-06T22:34:54.3525741Z + for CONFIG in mypy*.ini
2021-07-06T22:34:54.3526843Z + mypy --config=mypy.ini
2021-07-06T22:35:00.2340542Z aten/src/ATen/native/quantized/cpu/qnnpack/configure.py: error: Duplicate module named 'configure' (also at './aten/src/ATen/native/quantized/cpu/qnnpack/deps/clog/configure.py')
2021-07-06T22:35:00.2342543Z aten/src/ATen/native/quantized/cpu/qnnpack/configure.py: note: Are you missing an __init__.py? Alternatively, consider using --exclude to avoid checking one of them.
2021-07-06T22:35:00.2343756Z Found 1 error in 1 file (errors prevented further checking)
2021-07-06T22:35:01.8407648Z Please fix the above mypy warnings.
2021-07-06T22:35:01.8408435Z + STATUS=fail
2021-07-06T22:35:01.8409401Z + '[' -n fail ']'
2021-07-06T22:35:01.8410210Z + echo 'Please fix the above mypy warnings.'
2021-07-06T22:35:01.8410730Z + false
2021-07-06T22:35:01.8423575Z ##[error]Process completed with exit code 1.
2021-07-06T22:35:01.8586628Z Post job cleanup.
2021-07-06T22:35:01.9727656Z [command]/usr/bin/git version

Preview docs built from this PR

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

Summary:

This change spilts the clang-tidy CI job into smaller steps and uses a
refactored version of the clang_tidy.py script.

The new folder structure is as follows:
```
tools/linter/clang_tidy
|_ __main__py
|_ requirements.txt
|_ run.py
|_ setup.sh
```

`__main__.py`

This script will run `tools/linter/clang_tidy/setup.sh` if a `build`
directory doesn't exist, mimicing what used to be done as a separate
step in the CI job.

After that, it will invoke `clang-tidy` with default arguments being
declared into the script itself (as opposed to declaring them in
lint.yml).

The reasoning behind this approach is two-fold:

- Make it easier to run `clang-tidy` locally using this script
- De-duplicate the option passing

`requirements.txt`

Contains a list of additional python dependencies needed by the
`clang-tidy` script.

`setup.sh`

If a build directory doesn't exist, this command will run the necessary
codegen and build commands for running `clang-tidy`

Example usage:
```
python3 tools/linter/clang_tidy --parallel
```
Notice that we don't have to put the `.py` at the end of `clang_tidy`.

Test Plan:

Run the following command:
```
python3 tools/linter/clang_tidy --paths torch/csrc/fx --parallel
```

[ghstack-poisoned]
@1ntEgr8 1ntEgr8 requested a review from a team July 1, 2021 14:41
Comment thread mypy.ini Outdated
Comment thread CONTRIBUTING.md Outdated
Comment thread tools/git-pre-commit Outdated
Copy link
Copy Markdown
Contributor

@samestep samestep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just to clarify: do any of the scripts use requirements.txt? or is it expected that the user will manually run pip install -r tools/linter/clang_tidy/requirements.txt first?

@samestep samestep requested a review from a team July 1, 2021 15:54
@1ntEgr8
Copy link
Copy Markdown
Contributor Author

1ntEgr8 commented Jul 1, 2021

just to clarify: do any of the scripts use requirements.txt? or is it expected that the user will manually run pip install -r tools/linter/clang_tidy/requirements.txt first?

It is expected that the user will run the pip install command themselves. I was debating if it was a good design choice to have a script that is supposed to execute a lint install the dependencies it needs. What do you think?

@samestep
Copy link
Copy Markdown
Contributor

samestep commented Jul 1, 2021

It is expected that the user will run the pip install command themselves. I was debating if it was a good design choice to have a script that is supposed to execute a lint install the dependencies it needs. What do you think?

Yeah I think it's fine if the lint script itself doesn't install the dependencies; you could add it to the make setup_lint target though, right?

Comment thread CONTRIBUTING.md Outdated
Comment thread tools/linter/clang_tidy/requirements.txt
Comment thread tools/linter/clang_tidy/setup.sh Outdated
Summary:

This change spilts the clang-tidy CI job into smaller steps and uses a
refactored version of the clang_tidy.py script.

The new folder structure is as follows:
```
tools/linter/clang_tidy
|_ __main__py
|_ requirements.txt
|_ run.py
|_ setup.sh
```

`__main__.py`

This script will run `tools/linter/clang_tidy/setup.sh` if a `build`
directory doesn't exist, mimicing what used to be done as a separate
step in the CI job.

After that, it will invoke `clang-tidy` with default arguments being
declared in the script itself (as opposed to declaring them in
lint.yml).

The reasoning behind this approach is two-fold:

- Make it easier to run `clang-tidy` locally using this script
- De-duplicate the option passing

`requirements.txt`

Contains a list of additional python dependencies needed by the
`clang-tidy` script.

`setup.sh`

If a build directory doesn't exist, this command will run the necessary
codegen and build commands for running `clang-tidy`

Example usage:
```
python3 tools/linter/clang_tidy --parallel
```
Notice that we don't have to put the `.py` at the end of `clang_tidy`.

Test Plan:

Run the following command:
```
python3 tools/linter/clang_tidy --paths torch/csrc/fx --parallel
```

[ghstack-poisoned]
@1ntEgr8 1ntEgr8 requested review from driazati and samestep July 1, 2021 20:39
Summary:

This change spilts the clang-tidy CI job into smaller steps and uses a
refactored version of the clang_tidy.py script.

The new folder structure is as follows:
```
tools/linter/clang_tidy
|_ __main__py
|_ requirements.txt
|_ run.py
|_ setup.sh
```

`__main__.py`

This script will run `tools/linter/clang_tidy/setup.sh` if a `build`
directory doesn't exist, mimicing what used to be done as a separate
step in the CI job.

After that, it will invoke `clang-tidy` with default arguments being
declared in the script itself (as opposed to declaring them in
lint.yml).

The reasoning behind this approach is two-fold:

- Make it easier to run `clang-tidy` locally using this script
- De-duplicate the option passing

`requirements.txt`

Contains a list of additional python dependencies needed by the
`clang-tidy` script.

`setup.sh`

If a build directory doesn't exist, this command will run the necessary
codegen and build commands for running `clang-tidy`

Example usage:
```
python3 tools/linter/clang_tidy --parallel
```
Notice that we don't have to put the `.py` at the end of `clang_tidy`.

Test Plan:

Run the following command:
```
python3 tools/linter/clang_tidy --paths torch/csrc/fx --parallel
```

[ghstack-poisoned]
Copy link
Copy Markdown
Contributor

@driazati driazati left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm with some minor comments

Comment thread tools/linter/translate_annotations.py Outdated
Comment thread tools/linter/clang_tidy/__main__.py Outdated
Comment thread tools/linter/install/clang_tidy.py
Comment thread tools/linter/clang_tidy/setup.py
Comment thread Makefile
Comment thread Makefile
Comment thread tools/linter/install/download_bin.py Outdated
1ntEgr8 added 2 commits July 6, 2021 07:39
Summary:

This change spilts the clang-tidy CI job into smaller steps and uses a
refactored version of the clang_tidy.py script.

The new folder structure is as follows:
```
tools/linter/clang_tidy
|_ __main__py
|_ requirements.txt
|_ run.py
|_ setup.sh
```

`__main__.py`

This script will run `tools/linter/clang_tidy/setup.sh` if a `build`
directory doesn't exist, mimicing what used to be done as a separate
step in the CI job.

After that, it will invoke `clang-tidy` with default arguments being
declared in the script itself (as opposed to declaring them in
lint.yml).

The reasoning behind this approach is two-fold:

- Make it easier to run `clang-tidy` locally using this script
- De-duplicate the option passing

`requirements.txt`

Contains a list of additional python dependencies needed by the
`clang-tidy` script.

`setup.sh`

If a build directory doesn't exist, this command will run the necessary
codegen and build commands for running `clang-tidy`

Example usage:
```
python3 tools/linter/clang_tidy --parallel
```
Notice that we don't have to put the `.py` at the end of `clang_tidy`.

Test Plan:

Run the following command:
```
python3 tools/linter/clang_tidy --paths torch/csrc/fx --parallel
```

[ghstack-poisoned]
Summary:

This change spilts the clang-tidy CI job into smaller steps and uses a
refactored version of the clang_tidy.py script.

The new folder structure is as follows:
```
tools/linter/clang_tidy
|_ __main__py
|_ requirements.txt
|_ run.py
|_ setup.sh
```

`__main__.py`

This script will run `tools/linter/clang_tidy/setup.sh` if a `build`
directory doesn't exist, mimicing what used to be done as a separate
step in the CI job.

After that, it will invoke `clang-tidy` with default arguments being
declared in the script itself (as opposed to declaring them in
lint.yml).

The reasoning behind this approach is two-fold:

- Make it easier to run `clang-tidy` locally using this script
- De-duplicate the option passing

`requirements.txt`

Contains a list of additional python dependencies needed by the
`clang-tidy` script.

`setup.sh`

If a build directory doesn't exist, this command will run the necessary
codegen and build commands for running `clang-tidy`

Example usage:
```
python3 tools/linter/clang_tidy --parallel
```
Notice that we don't have to put the `.py` at the end of `clang_tidy`.

Test Plan:

Run the following command:
```
python3 tools/linter/clang_tidy --paths torch/csrc/fx --parallel
```

[ghstack-poisoned]
@1ntEgr8
Copy link
Copy Markdown
Contributor Author

1ntEgr8 commented Jul 6, 2021

@1ntEgr8 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary:

This change spilts the clang-tidy CI job into smaller steps and uses a
refactored version of the clang_tidy.py script.

The new folder structure is as follows:
```
tools/linter/clang_tidy
|_ __main__py
|_ requirements.txt
|_ run.py
|_ setup.sh
```

`__main__.py`

This script will run `tools/linter/clang_tidy/setup.sh` if a `build`
directory doesn't exist, mimicing what used to be done as a separate
step in the CI job.

After that, it will invoke `clang-tidy` with default arguments being
declared in the script itself (as opposed to declaring them in
lint.yml).

The reasoning behind this approach is two-fold:

- Make it easier to run `clang-tidy` locally using this script
- De-duplicate the option passing

`requirements.txt`

Contains a list of additional python dependencies needed by the
`clang-tidy` script.

`setup.sh`

If a build directory doesn't exist, this command will run the necessary
codegen and build commands for running `clang-tidy`

Example usage:
```
python3 tools/linter/clang_tidy --parallel
```
Notice that we don't have to put the `.py` at the end of `clang_tidy`.

Test Plan:

Run the following command:
```
python3 tools/linter/clang_tidy --paths torch/csrc/fx --parallel
```

Differential Revision: [D29568582](https://our.internmc.facebook.com/intern/diff/D29568582)

[ghstack-poisoned]
@1ntEgr8
Copy link
Copy Markdown
Contributor Author

1ntEgr8 commented Jul 6, 2021

@1ntEgr8 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@1ntEgr8 merged this pull request in a1ad28d.

@malfet malfet mentioned this pull request Jul 7, 2021
facebook-github-bot pushed a commit that referenced this pull request Jul 7, 2021
Summary:
Fixes CI regression caused by #61119
Unlike Python, `.ini` string lists could not  end with trailing comma.

Fixes CI on master

Pull Request resolved: #61333

Reviewed By: bhosmer

Differential Revision: D29578696

Pulled By: malfet

fbshipit-source-id: b81e5f4c0a553299c4d4bee0a9bb73748910795f
@facebook-github-bot facebook-github-bot deleted the gh/1ntEgr8/12/head branch July 10, 2021 14:16
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 25, 2026
Summary:
Pull Request resolved: pytorch#61119

This change spilts the clang-tidy CI job into smaller steps and uses a
refactored version of the clang_tidy.py script.

The new folder structure is as follows:
```
tools/linter/clang_tidy
|_ __main__py
|_ requirements.txt
|_ run.py
|_ setup.sh
```

`__main__.py`

This script will run `tools/linter/clang_tidy/setup.sh` if a `build`
directory doesn't exist, mimicing what used to be done as a separate
step in the CI job.

After that, it will invoke `clang-tidy` with default arguments being
declared in the script itself (as opposed to declaring them in
lint.yml).

The reasoning behind this approach is two-fold:

- Make it easier to run `clang-tidy` locally using this script
- De-duplicate the option passing

`requirements.txt`

Contains a list of additional python dependencies needed by the
`clang-tidy` script.

`setup.sh`

If a build directory doesn't exist, this command will run the necessary
codegen and build commands for running `clang-tidy`

Example usage:
```
python3 tools/linter/clang_tidy --parallel
```
Notice that we don't have to put the `.py` at the end of `clang_tidy`.

Test Plan:
Run the following command:
```
python3 tools/linter/clang_tidy --paths torch/csrc/fx --parallel
```

Reviewed By: walterddr, janeyx99

Differential Revision: D29568582

Pulled By: 1ntEgr8

fbshipit-source-id: cd6d11c5cb8ba9f1344a87c35647a1cd8dd45b04
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 25, 2026
Summary:
Fixes CI regression caused by pytorch#61119
Unlike Python, `.ini` string lists could not  end with trailing comma.

Fixes CI on master

Pull Request resolved: pytorch#61333

Reviewed By: bhosmer

Differential Revision: D29578696

Pulled By: malfet

fbshipit-source-id: b81e5f4c0a553299c4d4bee0a9bb73748910795f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants