|
1 | 1 | (pip-integration)=
|
2 | 2 | # Pip Integration
|
3 | 3 |
|
4 |
| -To pull in dependencies from PyPI, the `pip_parse` function is used, which |
5 |
| -invokes `pip` to download and install dependencies from PyPI. |
6 |
| - |
7 |
| -In your WORKSPACE file: |
8 |
| - |
9 |
| -```starlark |
10 |
| -load("@rules_python//python:pip.bzl", "pip_parse") |
11 |
| - |
12 |
| -pip_parse( |
13 |
| - name = "pip_deps", |
14 |
| - requirements_lock = ":requirements.txt", |
15 |
| -) |
16 |
| - |
17 |
| -load("@pip_deps//:requirements.bzl", "install_deps") |
18 |
| - |
19 |
| -install_deps() |
20 |
| -``` |
21 |
| - |
22 |
| -For `bzlmod` an equivalent `MODULE.bazel` would look like: |
23 |
| -```starlark |
24 |
| -pip = use_extension("//python/extensions:pip.bzl", "pip") |
25 |
| -pip.parse( |
26 |
| - hub_name = "pip_deps", |
27 |
| - requirements_lock = ":requirements.txt", |
28 |
| -) |
29 |
| -use_repo(pip, "pip_deps") |
30 |
| -``` |
31 |
| - |
32 |
| -You can then reference installed dependencies from a `BUILD` file with: |
33 |
| - |
34 |
| -```starlark |
35 |
| -load("@pip_deps//:requirements.bzl", "requirement") |
36 |
| - |
37 |
| -py_library( |
38 |
| - name = "bar", |
39 |
| - ... |
40 |
| - deps = [ |
41 |
| - "//my/other:dep", |
42 |
| - "@pip_deps//requests", |
43 |
| - "@pip_deps//numpy", |
44 |
| - ], |
45 |
| -) |
46 |
| -``` |
47 |
| - |
48 |
| -The rules also provide a convenience macro for translating the entries in the |
49 |
| -`requirements.txt` file (e.g. `opencv-python`) to the right bazel label (e.g. |
50 |
| -`@pip_deps//opencv_python`). The convention of bazel labels is lowercase |
51 |
| -`snake_case`, but you can use the helper to avoid depending on this convention |
52 |
| -as follows: |
53 |
| - |
54 |
| -```starlark |
55 |
| -load("@pip_deps//:requirements.bzl", "requirement") |
56 |
| - |
57 |
| -py_library( |
58 |
| - name = "bar", |
59 |
| - ... |
60 |
| - deps = [ |
61 |
| - "//my/other:dep", |
62 |
| - requirement("requests"), |
63 |
| - requirement("numpy"), |
64 |
| - ], |
65 |
| -) |
66 |
| -``` |
67 |
| - |
68 |
| -If you would like to access [entry points][whl_ep], see the `py_console_script_binary` rule documentation. |
69 |
| - |
70 |
| -[whl_ep]: https://packaging.python.org/specifications/entry-points/ |
71 |
| - |
72 |
| -(per-os-arch-requirements)= |
73 |
| -## Requirements for a specific OS/Architecture |
74 |
| - |
75 |
| -In some cases you may need to use different requirements files for different OS, Arch combinations. This is enabled via the `requirements_by_platform` attribute in `pip.parse` extension and the `pip_parse` repository rule. The keys of the dictionary are labels to the file and the values are a list of comma separated target (os, arch) tuples. |
76 |
| - |
77 |
| -For example: |
78 |
| -```starlark |
79 |
| - # ... |
80 |
| - requirements_by_platform = { |
81 |
| - "requirements_linux_x86_64.txt": "linux_x86_64", |
82 |
| - "requirements_osx.txt": "osx_*", |
83 |
| - "requirements_linux_exotic.txt": "linux_exotic", |
84 |
| - "requirements_some_platforms.txt": "linux_aarch64,windows_*", |
85 |
| - }, |
86 |
| - # For the list of standard platforms that the rules_python has toolchains for, default to |
87 |
| - # the following requirements file. |
88 |
| - requirements_lock = "requirements_lock.txt", |
89 |
| -``` |
90 |
| - |
91 |
| -In case of duplicate platforms, `rules_python` will raise an error as there has |
92 |
| -to be unambiguous mapping of the requirement files to the (os, arch) tuples. |
93 |
| - |
94 |
| -An alternative way is to use per-OS requirement attributes. |
95 |
| -```starlark |
96 |
| - # ... |
97 |
| - requirements_windows = "requirements_windows.txt", |
98 |
| - requirements_darwin = "requirements_darwin.txt", |
99 |
| - # For the remaining platforms (which is basically only linux OS), use this file. |
100 |
| - requirements_lock = "requirements_lock.txt", |
101 |
| -) |
102 |
| -``` |
103 |
| - |
104 |
| -(vendoring-requirements)= |
105 |
| -## Vendoring the requirements.bzl file |
106 |
| - |
107 |
| -In some cases you may not want to generate the requirements.bzl file as a repository rule |
108 |
| -while Bazel is fetching dependencies. For example, if you produce a reusable Bazel module |
109 |
| -such as a ruleset, you may want to include the requirements.bzl file rather than make your users |
110 |
| -install the WORKSPACE setup to generate it. |
111 |
| -See https://github.com/bazelbuild/rules_python/issues/608 |
112 |
| - |
113 |
| -This is the same workflow as Gazelle, which creates `go_repository` rules with |
114 |
| -[`update-repos`](https://github.com/bazelbuild/bazel-gazelle#update-repos) |
115 |
| - |
116 |
| -To do this, use the "write to source file" pattern documented in |
117 |
| -https://blog.aspect.dev/bazel-can-write-to-the-source-folder |
118 |
| -to put a copy of the generated requirements.bzl into your project. |
119 |
| -Then load the requirements.bzl file directly rather than from the generated repository. |
120 |
| -See the example in rules_python/examples/pip_parse_vendored. |
121 |
| - |
122 |
| - |
123 |
| -(credential-helper)= |
124 |
| -## Credential Helper |
125 |
| - |
126 |
| -The "use Bazel downloader for python wheels" experimental feature includes support for the Bazel |
127 |
| -[Credential Helper][cred-helper-design]. |
128 |
| - |
129 |
| -Your python artifact registry may provide a credential helper for you. Refer to your index's docs |
130 |
| -to see if one is provided. |
131 |
| - |
132 |
| -See the [Credential Helper Spec][cred-helper-spec] for details. |
133 |
| - |
134 |
| -[cred-helper-design]: https://github.com/bazelbuild/proposals/blob/main/designs/2022-06-07-bazel-credential-helpers.md |
135 |
| -[cred-helper-spec]: https://github.com/EngFlow/credential-helper-spec/blob/main/spec.md |
136 |
| - |
137 |
| - |
138 |
| -### Basic Example: |
139 |
| - |
140 |
| -The simplest form of a credential helper is a bash script that accepts an arg and spits out JSON to |
141 |
| -stdout. For a service like Google Artifact Registry that uses ['Basic' HTTP Auth][rfc7617] and does |
142 |
| -not provide a credential helper that conforms to the [spec][cred-helper-spec], the script might |
143 |
| -look like: |
144 |
| - |
145 |
| -```bash |
146 |
| -#!/bin/bash |
147 |
| -# cred_helper.sh |
148 |
| -ARG=$1 # but we don't do anything with it as it's always "get" |
149 |
| - |
150 |
| -# formatting is optional |
151 |
| -echo '{' |
152 |
| -echo ' "headers": {' |
153 |
| -echo ' "Authorization": ["Basic dGVzdDoxMjPCow=="]' |
154 |
| -echo ' }' |
155 |
| -echo '}' |
156 |
| -``` |
157 |
| - |
158 |
| -Configure Bazel to use this credential helper for your python index `example.com`: |
159 |
| - |
160 |
| -``` |
161 |
| -# .bazelrc |
162 |
| -build --credential_helper=example.com=/full/path/to/cred_helper.sh |
163 |
| -``` |
164 |
| - |
165 |
| -Bazel will call this file like `cred_helper.sh get` and use the returned JSON to inject headers |
166 |
| -into whatever HTTP(S) request it performs against `example.com`. |
167 |
| - |
168 |
| -[rfc7617]: https://datatracker.ietf.org/doc/html/rfc7617 |
| 4 | +See [PyPI dependencies](./pypi-dependencies). |
0 commit comments