cpp_wrapper: Precompile device-specific header files #144002

benjaminglass1 · 2024-12-30T21:23:39Z

Stack from ghstack (oldest at bottom):

This saves us about a second per compilation, which is massive for the OpInfo tests. Total OpInfo test runtime is down about 2x from this change alone.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang @aakhundov

Differential Revision: D69185685

[ghstack-poisoned]

pytorch-bot · 2024-12-30T21:23:43Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/144002

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 5 Pending

As of commit 7c342c1 with merge base 93d98ac ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This saves us about a second per compilation, which is _massive_ for the OpInfo tests. Total OpInfo test runtime is down about 2x. ghstack-source-id: 14933f8 Pull Request resolved: #144002

[ghstack-poisoned]

This saves us about a second per compilation, which is _massive_ for the OpInfo tests. Total OpInfo test runtime is down about 2x. ghstack-source-id: 5aef659 Pull Request resolved: #144002

benjaminglass1 · 2025-02-04T22:59:30Z

@desertfire I've stacked the remainder of the test bugfix stack underneath this, temporarily, in order to test whether the speedups here allow us to run the full suite of inductor tests. Hopefully we'll just be able to merge those out from underneath this, but if not I'll unstack whenever you need.

[ghstack-poisoned]

desertfire · 2025-02-05T16:16:40Z

@desertfire has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

desertfire

Chatted offline and test will come in a follow-up PR.

benjaminglass1 · 2025-02-10T17:04:51Z

@pytorchbot merge

pytorchmergebot · 2025-02-10T17:07:25Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

huydhn · 2025-02-11T00:40:54Z

@pytorchbot revert -m 'Sorry for reverting your change but this breaks some inductor tests running internally' -c ghfirst

======================================================================
ERROR: test_cpp_wrapper (caffe2.test.inductor.test_cudagraph_trees.CudaGraphTreeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/fbcode/platform010/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/caffe2/test/inductor/test_cudagraph_trees.py", line 2503, in test_cpp_wrapper
    compiled_result = self.run_twc(compiled, example_input)
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/caffe2/test/inductor/test_cudagraph_trees.py", line 197, in run_twc
    fn(*args, **kwargs)
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/torch/_dynamo/eval_frame.py", line 574, in _fn
    raise e.remove_dynamo_frames() from None  # see TORCHDYNAMO_VERBOSE=1
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/torch/_inductor/compile_fx.py", line 752, in _compile_fx_inner
    raise InductorError(e, currentframe()).with_traceback(
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/torch/_inductor/compile_fx.py", line 737, in _compile_fx_inner
    mb_compiled_graph = fx_codegen_and_compile(
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/torch/_inductor/compile_fx.py", line 1402, in fx_codegen_and_compile
    return scheme.codegen_and_compile(gm, example_inputs, inputs_to_check, graph_kwargs)
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/torch/_inductor/compile_fx.py", line 1122, in codegen_and_compile
    compiled_fn = graph.compile_to_module().call
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/torch/_inductor/graph.py", line 1986, in compile_to_module
    return self._compile_to_module()
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/torch/_inductor/graph.py", line 2028, in _compile_to_module
    mod = PyCodeCache.load_by_key_path(
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/torch/_inductor/codecache.py", line 2853, in load_by_key_path
    mod = _reload_python_module(key, path)
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/torch/_inductor/runtime/compile_tasks.py", line 51, in _reload_python_module
    exec(code, mod.__dict__, mod.__dict__)
  File "/re_tmp/tmpwrjbuu4z/jq/cjq53uwok6fj5qand5kfwnazczljuwix4xupx7gmoxjlon2f4hea.py", line 197, in <module>
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/torch/_inductor/codecache.py", line 2347, in load_pybinding
    return cls.load_pybinding_async(*args, **kwargs)()
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/torch/_inductor/codecache.py", line 2328, in load_pybinding_async
    get_result = cls.load_async(
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/torch/_inductor/codecache.py", line 2096, in load_async
    _precompile_header(
  File "/re_cwd/buck-out/v2/gen/fbcode/bbdd369aded5cb42/caffe2/test/inductor/__cudagraph_trees__/cudagraph_trees#link-tree/torch/_inductor/codecache.py", line 1974, in _precompile_header
    extra=f"{hashable_cmd_line} {compiler_version} {torch.version.__version__} {torch.version.build_uuid}",
torch._inductor.exc.InductorError: AttributeError: module 'torch.version' has no attribute 'build_uuid'

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information


----------------------------------------------------------------------
Ran 1 test in 4.217s

FAILED (errors=1)

I think @desertfire has been looking into the failure

pytorchmergebot · 2025-02-11T00:42:14Z

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

pytorchmergebot · 2025-02-11T00:42:25Z

@benjaminglass1 your PR has been successfully reverted.

This reverts commit de6efa1. Reverted #144002 on behalf of https://github.com/huydhn due to Sorry for reverting your change but this breaks some inductor tests running internally ([comment](#144002 (comment)))

benjaminglass1 · 2025-02-11T16:45:06Z

@desertfire @huydhn I'll open a new PR to re-land this in, to avoid any issues with the associated internal revision.

This saves us about a second per compilation, which is _massive_ for the OpInfo tests. Total OpInfo test runtime is down about 2x. ghstack-source-id: 84d4ba3 Pull Request resolved: pytorch/pytorch#144002

This saves us about a second per compilation, which is _massive_ for the OpInfo tests. Total OpInfo test runtime is down about 2x from this change alone. Relands #144002, with changes needed by fbcode internals. Pull Request resolved: #146928 Approved by: https://github.com/desertfire

Update

460bdc5

[ghstack-poisoned]

This was referenced Dec 30, 2024

ir.ExternKernel: correctly handle kwarg default arguments #141371

Closed

cpp_wrapper: Use runtime dispatched fallbacks for complex ops #143223

Closed

This was referenced Dec 30, 2024

AOTI fallback ops: remove ops that were never codegen'ed #143421

Closed

cpp_wrapper: Move #includes to per-device header files #143909

Closed

pytorch-bot bot added ciflow/inductor module: inductor labels Dec 30, 2024

benjaminglass1 added the topic: not user facing topic category label Dec 30, 2024

pytorchbot added the open source label Dec 30, 2024

benjaminglass1 added the ciflow/trunk Trigger trunk jobs on your pull request label Dec 31, 2024

Update

b07202e

[ghstack-poisoned]

benjaminglass1 self-assigned this Jan 2, 2025

Update

22a011b

[ghstack-poisoned]

Update

c30bc58

[ghstack-poisoned]

Update

99310e9

[ghstack-poisoned]

This was referenced Jan 3, 2025

cpp_wrapper AOTI: Move #includes to per-device header files #144123

Closed

cpp_wrapper AOTI: Precompile device-specific header files #144124

Closed

Update

ec67641

[ghstack-poisoned]

benjaminglass1 requested a review from desertfire January 3, 2025 18:08

benjaminglass1 marked this pull request as ready for review January 3, 2025 18:08

benjaminglass1 removed the ciflow/trunk Trigger trunk jobs on your pull request label Jan 3, 2025

Update

4724ccc

[ghstack-poisoned]

Update

7c342c1

[ghstack-poisoned]

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Feb 5, 2025

benjaminglass1 mentioned this pull request Feb 7, 2025

cpp_wrapper: persist autotune example tensors until last use #146706

Closed

desertfire approved these changes Feb 10, 2025

View reviewed changes

pytorchmergebot added the merging label Feb 10, 2025

pytorchmergebot added the Merged label Feb 10, 2025

pytorchmergebot closed this in de6efa1 Feb 10, 2025

pytorchmergebot removed the merging label Feb 10, 2025

pytorchmergebot added Reverted ci-no-td Do not run TD on this PR labels Feb 11, 2025

pytorchmergebot reopened this Feb 11, 2025

benjaminglass1 closed this Feb 11, 2025

benjaminglass1 mentioned this pull request Feb 11, 2025

cpp_wrapper: Precompile device-specific header files #146928

Closed

github-actions bot deleted the gh/benjaminglass1/47/head branch March 14, 2025 02:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

cpp_wrapper: Precompile device-specific header files #144002

cpp_wrapper: Precompile device-specific header files #144002

Uh oh!

benjaminglass1 commented Dec 30, 2024 •

edited

Loading

Uh oh!

pytorch-bot bot commented Dec 30, 2024 •

edited

Loading

Uh oh!

benjaminglass1 commented Feb 4, 2025

Uh oh!

desertfire commented Feb 5, 2025

Uh oh!

desertfire left a comment

Uh oh!

benjaminglass1 commented Feb 10, 2025

Uh oh!

pytorchmergebot commented Feb 10, 2025

Uh oh!

huydhn commented Feb 11, 2025

Uh oh!

pytorchmergebot commented Feb 11, 2025

Uh oh!

pytorchmergebot commented Feb 11, 2025

Uh oh!

benjaminglass1 commented Feb 11, 2025

Uh oh!

Uh oh!

cpp_wrapper: Precompile device-specific header files #144002

cpp_wrapper: Precompile device-specific header files #144002

Uh oh!

Conversation

benjaminglass1 commented Dec 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Dec 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/144002

⏳ No Failures, 5 Pending

Uh oh!

benjaminglass1 commented Feb 4, 2025

Uh oh!

desertfire commented Feb 5, 2025

Uh oh!

desertfire left a comment

Choose a reason for hiding this comment

Uh oh!

benjaminglass1 commented Feb 10, 2025

Uh oh!

pytorchmergebot commented Feb 10, 2025

Merge started

Uh oh!

huydhn commented Feb 11, 2025

Uh oh!

pytorchmergebot commented Feb 11, 2025

Uh oh!

pytorchmergebot commented Feb 11, 2025

Uh oh!

benjaminglass1 commented Feb 11, 2025

Uh oh!

Uh oh!

benjaminglass1 commented Dec 30, 2024 •

edited

Loading

pytorch-bot bot commented Dec 30, 2024 •

edited

Loading