Bugfix stack.py to re-enable ./xla checkout, +improvements #112
Conversation
| /jax | ||
|
|
||
| # temporary working files | ||
| /_my |
There was a problem hiding this comment.
what generates this _my folder?
There was a problem hiding this comment.
Nothing generates it, I just use it for work scripts and tired of adding it to a local .gitignore on each new checkout.
It's good to have a dedicated dir for such stuff, dumps and whatnot...
| # pylint: disable=line-too-long | ||
| assert os.path.isdir( | ||
| kernels_jax_path | ||
| ), f"XLA path (specified as '{kernels_jax_dir}') doesn't resolve to existing directory at '{kernels_jax_path}'" |
There was a problem hiding this comment.
I am confused about this "kernels_jax_dir" variable. Could you please put detailed explanation for this one?
There was a problem hiding this comment.
kernels_jax_dir is the var created by Charles in his previous PR. Essentially it's a directory for the JAX parts, such as kernels, that are needed for pjrt & plugin compilation. It might and might not coincide with ./jax tests directory. Before Charles's PR ./jax was just an independent test storage, and kernels were always taken from the upsteam jax-ml/jax. In his PR, he added an override to always use ./jax (more properly the value of --kernel-jax-dir argument if is set) instead of the upstream, so it's finally used to build pjrt&plugin where it's needed. Now, the upstream jax-ml/jax is used only if --kernel-jax-dir argument is explicitly set to an empty string.
kernels_jax_path is just an abspath(kernels_jax_dir). Absolute paths are mandatory to make debug info remapping work properly.
There was a problem hiding this comment.
I understand, thanks for the detailed explanation!
Co-authored-by: Gulsum Gudukbay Akbulut <[email protected]>
| python3 ./build/build.py build \ | ||
| --use_clang=true \ | ||
| --wheels=jax-rocm-plugin \ | ||
| --target_cpu_features=native \ |
Motivation
bd2fab3 introduced a regression that disabled checking out
./xladirectory by default. This PR fixes it.Additionally the PR:
--target_cpu_features=nativetobuild/build.pycalls in the Makefile to produce more optimal dev build fasterXLA_OVERRIDE_OPTIONmakefile variable to a more genericALL_BAZEL_OPTIONSto better reflect the scope and the purpose of the variable.refreshandrefresh_jaxlibrules as shortcuts to correspongingclean dist installrules.--fix-bazel-symbolsoption tostack.py. This lets the script assumes you need to build code in a release with symbolic info configuration to alleviate debugging. The script enables respective bazel options and adds./externalsymbolic links to corresponding workspaces pointing to bazel's dependencies storage.