Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 29e3ec4

Browse files
kushalkolarkushalkolar
authored
simplify CI and install extras (#698)
* simplify CI, WIP * tweak * jupyter rfb v0.5.0 not on pypi yet * tweak * twaek * ok let's make this look ugly * typical bash and sed bleh * ok so github really likes it ugly * needs more sed * see if core dump fixed * imgui and no-imgui matrix for regen screenshots * update test utils * account for imagewidget which needs imgui * modify regen * tweak * skip imgui required tests when not installed * cleanup * move check earlier * more informative artifact upload names * typo * use upload-artifact v4 * add no imgui ground truth imageS * increase jupyter_rfb version * use bash find to exclude/include imagewidget nbs depending on imgui existance * test imagewidget gc in nbs only if imgui installed * regen screenshots nb with and w/o imgui * add nb back to regen matrix * we actually only need 2 variants of regen * prefix no-imgui for nb screenshots * add no-imgui-nb screenshots * python version
1 parent c84e5ba commit 29e3ec4

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+228
-157
lines changed

‎.github/workflows/ci.yml

Lines changed: 26 additions & 67 deletions
Original file line numberDiff line numberDiff line change
@@ -15,28 +15,24 @@ on:
1515

1616
jobs:
1717
test-build-full:
18-
name: Test Linux, notebook + offscreen
18+
name: Test Linux
1919
runs-on: ubuntu-latest
20-
timeout-minutes: 30
20+
timeout-minutes: 10
2121
if: ${{ !github.event.pull_request.draft }}
2222
strategy:
2323
fail-fast: false
2424
matrix:
25-
include:
26-
- name: Test py311
27-
pyversion: '3.11'
28-
- name: Test py312
29-
pyversion: '3.12'
30-
- name: Test py313
31-
pyversion: '3.13'
25+
python: ["3.11", "3.12", "3.13"]
26+
imgui_dep: ["imgui", ""]
27+
notebook_dep: ["notebook", ""]
3228
steps:
3329
- uses: actions/checkout@v4
3430
with:
3531
lfs: true
3632
- name: Set up Python
3733
uses: actions/setup-python@v5
3834
with:
39-
python-version: ${{ matrix.pyversion }}
35+
python-version: ${{ matrix.python }}
4036
- name: Install llvmpipe and lavapipe for offscreen canvas
4137
run: |
4238
sudo apt-get update -y -qq
@@ -47,7 +43,12 @@ jobs:
4743
# remove pygfx from install_requires, we install using pygfx@main
4844
sed -i "/pygfx/d" ./setup.py
4945
pip install git+https://github.com/pygfx/pygfx.git@main
50-
pip install -e ".["tests"]"
46+
- name: Install fastplotlib
47+
run: |
48+
# create string with one of: tests,imgui,notebook; test,imgui; test,notebook ; tests
49+
# sed removes trailing comma
50+
# install fastplotlib with given extras options from above
51+
pip install -e ".[$(echo "tests,${{ matrix.imgui_dep }},${{ matrix.notebook_dep }}" | sed -e "s/,\+/,/g" -e "s/,$//")]"
5152
- name: Show wgpu backend
5253
run:
5354
python -c "from examples.tests.testutils import wgpu_backend; print(wgpu_backend)"
@@ -60,69 +61,27 @@ jobs:
6061
PYGFX_EXPECT_LAVAPIPE: true
6162
run: |
6263
WGPU_FORCE_OFFSCREEN=1 pytest -v tests/
63-
pytest -v examples
64-
FASTPLOTLIB_NB_TESTS=1 pytest --nbmake examples/notebooks/
65-
- uses: actions/upload-artifact@v4
66-
if: ${{ failure() }}
67-
with:
68-
name: screenshot-diffs
69-
path: |
70-
examples/diffs
71-
examples/notebooks/diffs
72-
73-
test-build-offscreen:
74-
name: Test Linux, only offscreen
75-
runs-on: ubuntu-latest
76-
timeout-minutes: 30
77-
if: ${{ !github.event.pull_request.draft }}
78-
strategy:
79-
fail-fast: false
80-
matrix:
81-
include:
82-
- name: Test py311
83-
pyversion: '3.11'
84-
- name: Test py312
85-
pyversion: '3.12'
86-
- name: Test py313
87-
pyversion: '3.13'
88-
steps:
89-
- uses: actions/checkout@v4
90-
with:
91-
lfs: true
92-
- name: Set up Python
93-
uses: actions/setup-python@v5
94-
with:
95-
python-version: ${{ matrix.pyversion }}
96-
- name: Install llvmpipe and lavapipe for offscreen canvas
97-
run: |
98-
sudo apt-get update -y -qq
99-
sudo apt-get install --no-install-recommends -y libegl1-mesa-dev libgl1-mesa-dri libxcb-xfixes0-dev mesa-vulkan-drivers xorg-dev
100-
- name: Install dev dependencies
101-
run: |
102-
python -m pip install --upgrade pip setuptools
103-
# remove pygfx from install_requires, we install using pygfx@main
104-
sed -i "/pygfx/d" ./setup.py
105-
pip install git+https://github.com/pygfx/pygfx.git@main
106-
pip install -e ".["tests-desktop"]"
107-
- name: Show wgpu backend
108-
run:
109-
python -c "from examples.tests.testutils import wgpu_backend; print(wgpu_backend)"
110-
- name: fetch git lfs files
111-
run: |
112-
git lfs fetch --all
113-
git lfs pull
114-
- name: Test examples
64+
WGPU_FORCE_OFFSCREEN=1 pytest -v examples/
65+
- name: Test examples notebooks, exclude ImageWidget notebook
66+
if: ${{ matrix.notebook_dep == 'notebook' }}
11567
env:
11668
PYGFX_EXPECT_LAVAPIPE: true
117-
run: |
118-
WGPU_FORCE_OFFSCREEN=1 pytest -v tests/
119-
pytest -v examples
69+
# test notebooks, exclude ImageWidget notebooks
70+
run: FASTPLOTLIB_NB_TESTS=1 pytest --nbmake $(find ./examples/notebooks/ -maxdepth 1 -type f -name "*.ipynb" ! -name "image_widget*.ipynb" -print | xargs)
71+
- name: Test ImageWidget notebooks
72+
# test image widget notebooks only if imgui is installed
73+
if: ${{ matrix.notebook_dep == 'notebook' && matrix.imgui_dep == 'imgui' }}
74+
env:
75+
PYGFX_EXPECT_LAVAPIPE: true
76+
run: FASTPLOTLIB_NB_TESTS=1 pytest --nbmake $(find ./examples/notebooks/ -maxdepth 1 -type f -name "image_widget*.ipynb" -print | xargs)
12077
- uses: actions/upload-artifact@v4
12178
if: ${{ failure() }}
12279
with:
123-
name: screenshot-diffs
80+
name: screenshot-diffs-${{ matrix.pyversion }}-${{ matrix.imgui_dep }}-${{ matrix.notebook_dep }}
12481
path: |
12582
examples/diffs
83+
examples/notebooks/diffs
84+
12685
12786
# test-build-full-mac:
12887
# name: Test Mac, notebook + glfw

‎.github/workflows/docs-deploy.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ jobs:
4242
# remove pygfx from install_requires, we install using pygfx@main
4343
sed -i "/pygfx/d" ./setup.py
4444
pip install git+https://github.com/pygfx/pygfx.git@main
45-
pip install -e ".["docs"]"
45+
pip install -e ".[docs,notebook,imgui]"
4646
- name: Show wgpu backend
4747
run:
4848
python -c "from examples.tests.testutils import wgpu_backend; print(wgpu_backend)"

‎.github/workflows/screenshots.yml

Lines changed: 23 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,10 @@ jobs:
1616
runs-on: ubuntu-latest
1717
timeout-minutes: 10
1818
if: ${{ !github.event.pull_request.draft }}
19+
strategy:
20+
fail-fast: false
21+
matrix:
22+
imgui_dep: ["imgui", ""]
1923
steps:
2024
- uses: actions/checkout@v4
2125
with:
@@ -34,21 +38,34 @@ jobs:
3438
# remove pygfx from install_requires, we install using pygfx@main
3539
sed -i "/pygfx/d" ./setup.py
3640
pip install git+https://github.com/pygfx/pygfx.git@main
37-
pip install -e ".["tests"]"
41+
- name: Install fastplotlib
42+
run: |
43+
# create string with one of: tests,imgui,notebook; test,imgui; test,notebook ; tests
44+
# sed removes trailing comma
45+
# install fastplotlib with with or without imgui depending on build matrix
46+
pip install -e ".[$(echo "tests,notebook,${{ matrix.imgui_dep }}" | sed -e "s/,\+/,/g" -e "s/,$//")]"
3847
- name: Show wgpu backend
3948
run:
4049
python -c "from examples.tests.testutils import wgpu_backend; print(wgpu_backend)"
41-
- name: Test examples
50+
- name: Generate screenshots
4251
env:
4352
PYGFX_EXPECT_LAVAPIPE: true
4453
run: |
4554
# regenerate screenshots
46-
REGENERATE_SCREENSHOTS=1 pytest -v examples
47-
FASTPLOTLIB_NB_TESTS=1 REGENERATE_SCREENSHOTS=1 pytest --nbmake examples/notebooks/
48-
- uses: actions/upload-artifact@v3
55+
WGPU_FORCE_OFFSCREEN=1 REGENERATE_SCREENSHOTS=1 pytest -v examples
56+
- name: Generate screenshots notebook, exclude image widget
57+
env:
58+
PYGFX_EXPECT_LAVAPIPE: true
59+
run: FASTPLOTLIB_NB_TESTS=1 REGENERATE_SCREENSHOTS=1 pytest --nbmake $(find ./examples/notebooks/ -maxdepth 1 -type f -name "*.ipynb" ! -name "image_widget*.ipynb" -print | xargs)
60+
- name: Generate screenshots notebook, include image widget
61+
if: ${{ matrix.imgui_dep == 'imgui' }}
62+
env:
63+
PYGFX_EXPECT_LAVAPIPE: true
64+
run: FASTPLOTLIB_NB_TESTS=1 REGENERATE_SCREENSHOTS=1 pytest --nbmake $(find ./examples/notebooks/ -maxdepth 1 -type f -name "image_widget*.ipynb" -print | xargs)
65+
- uses: actions/upload-artifact@v4
4966
if: always()
5067
with:
51-
name: screenshots
68+
name: screenshots-${{ matrix.imgui_dep }}
5269
path: |
5370
examples/screenshots/
5471
examples/notebooks/screenshots/

‎CONTRIBUTING.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ cd fastplotlib
6363

6464
```bash
6565
# install all extras in place
66-
pip install -e ".[notebook,docs,tests]"
66+
pip install -e ".[imgui, notebook, docs, tests]"
6767
```
6868

6969
5. Add the upstream remote branch:
@@ -280,7 +280,7 @@ to open source software packages.
280280
# after cloning
281281
cd fastplotlib
282282
# install dev dependencies
283-
pip install -e ".[tests, docs, notebook]"
283+
pip install -e ".[imgui, tests, docs, notebook]"
284284
```
285285

286286
3) Check out a feature branch from `main`

‎examples/notebooks/nb_test_utils.py

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -105,11 +105,21 @@ def plot_test(name, fig: fpl.Figure):
105105

106106

107107
def regenerate_screenshot(name, data):
108-
iio.imwrite(SCREENSHOTS_DIR.joinpath(f"nb-{name}.png"), data)
108+
if fpl.IMGUI:
109+
prefix = ""
110+
else:
111+
prefix = "no-imgui-"
112+
113+
iio.imwrite(SCREENSHOTS_DIR.joinpath(f"{prefix}nb-{name}.png"), data)
109114

110115

111116
def assert_screenshot_equal(name, data):
112-
ground_truth = iio.imread(SCREENSHOTS_DIR.joinpath(f"nb-{name}.png"))
117+
if fpl.IMGUI:
118+
prefix = ""
119+
else:
120+
prefix = "no-imgui-"
121+
122+
ground_truth = iio.imread(SCREENSHOTS_DIR.joinpath(f"{prefix}nb-{name}.png"))
113123

114124
img = normalize_image(data)
115125
ref_img = normalize_image(ground_truth)
@@ -140,9 +150,14 @@ def get_diffs_rgba(slicer):
140150
diffs_rgba = diffs_rgba.astype("u1")
141151
return diffs_rgba[..., slicer]
142152

153+
if fpl.IMGUI:
154+
prefix = ""
155+
else:
156+
prefix = "no-imgui-"
157+
143158
# split into an rgb and an alpha diff
144159
diffs = {
145-
DIFFS_DIR.joinpath(f"nb-diff-{name}-rgb.png"): slice(0, 3),
160+
DIFFS_DIR.joinpath(f"{prefix}nb-diff-{name}-rgb.png"): slice(0, 3),
146161
}
147162

148163
for path, slicer in diffs.items():

‎examples/notebooks/screenshots/no-imgui-nb-astronaut.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/notebooks/screenshots/no-imgui-nb-astronaut_RGB.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/notebooks/screenshots/no-imgui-nb-camera.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/notebooks/screenshots/no-imgui-nb-lines-3d.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/notebooks/screenshots/no-imgui-nb-lines-colors.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/notebooks/screenshots/no-imgui-nb-lines-data.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/notebooks/screenshots/no-imgui-nb-lines-underlay.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/notebooks/screenshots/no-imgui-nb-lines.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/notebooks/test_gc.ipynb

Lines changed: 20 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -202,57 +202,27 @@
202202
"metadata": {},
203203
"outputs": [],
204204
"source": [
205-
"movies = [np.random.rand(100, 100, 100) for i in range(6)]\n",
206-
"\n",
207-
"iw = fpl.ImageWidget(movies)\n",
208-
"\n",
209-
"# add some events onto all the image graphics\n",
210-
"for g in iw.managed_graphics:\n",
211-
" for f in g._features:\n",
212-
" g.add_event_handler(feature_changed_handler, f)\n",
213-
"\n",
214-
"iw.show()"
215-
]
216-
},
217-
{
218-
"cell_type": "markdown",
219-
"id": "189bcd7a-40a2-4e84-abcf-c334e50f5544",
220-
"metadata": {},
221-
"source": [
222-
"# Test that setting new data with different dims clears old ImageGraphics"
205+
"if fpl.IMGUI:\n",
206+
" # do image widget tests only if imgui is installed\n",
207+
" movies = [np.random.rand(100, 100, 100) for i in range(6)]\n",
208+
" \n",
209+
" iw = fpl.ImageWidget(movies)\n",
210+
" \n",
211+
" # add some events onto all the image graphics\n",
212+
" for g in iw.managed_graphics:\n",
213+
" for f in g._features:\n",
214+
" g.add_event_handler(feature_changed_handler, f)\n",
215+
" \n",
216+
" iw.show()\n",
217+
" \n",
218+
" old_graphics = [weakref.proxy(g) for g in iw.managed_graphics]\n",
219+
" \n",
220+
" # Test that setting new data with different dims clears old ImageGraphics\n",
221+
" new_movies = [np.random.rand(100, 200, 200) for i in range(6)]\n",
222+
" \n",
223+
" iw.set_data(new_movies)\n",
224+
" test_references(old_graphics)"
223225
]
224-
},
225-
{
226-
"cell_type": "code",
227-
"execution_count": null,
228-
"id": "7e855043-91c1-4f6c-bed3-b69cf4a87f84",
229-
"metadata": {},
230-
"outputs": [],
231-
"source": [
232-
"old_graphics = [weakref.proxy(g) for g in iw.managed_graphics]\n",
233-
"\n",
234-
"new_movies = [np.random.rand(100, 200, 200) for i in range(6)]\n",
235-
"\n",
236-
"iw.set_data(new_movies)"
237-
]
238-
},
239-
{
240-
"cell_type": "code",
241-
"execution_count": null,
242-
"id": "59e3c193-5672-4a66-bdca-12f1dd675d32",
243-
"metadata": {},
244-
"outputs": [],
245-
"source": [
246-
"test_references(old_graphics)"
247-
]
248-
},
249-
{
250-
"cell_type": "code",
251-
"execution_count": null,
252-
"id": "ad3d2a24-88b3-4071-a49c-49667d5a7813",
253-
"metadata": {},
254-
"outputs": [],
255-
"source": []
256226
}
257227
],
258228
"metadata": {

‎examples/screenshots/no-imgui-gridplot.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/screenshots/no-imgui-gridplot_non_square.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/screenshots/no-imgui-heatmap.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/screenshots/no-imgui-image_cmap.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/screenshots/no-imgui-image_rgb.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/screenshots/no-imgui-image_rgbvminvmax.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/screenshots/no-imgui-image_simple.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/screenshots/no-imgui-image_small.png

Lines changed: 3 additions & 0 deletions
Loading

‎examples/screenshots/no-imgui-image_vminvmax.png

Lines changed: 3 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)