Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Bug] - Poor performance deriving dummy MODFLOW6 package for MetaSWAP coupling for big model #1445

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
JoerivanEngelen opened this issue Feb 20, 2025 · 0 comments · Fixed by #1446
Closed

[Bug] - Poor performance deriving dummy MODFLOW6 package for MetaSWAP coupling for big model #1445

JoerivanEngelen opened this issue Feb 20, 2025 · 0 comments · Fixed by #1446
Assignees
@JoerivanEngelen
Labels
bug Something isn't working
Milestone
v1.0 release

Comments

@JoerivanEngelen
Copy link
Contributor

JoerivanEngelen commented Feb 20, 2025 •
edited
Loading

Bug description
@WouterSwierstra tried reading a large iMOD5 model, if I recall correctly: (30, 26, 1200, 1200), and writing it was extremely slow: Over 30 minutes. Just inferring from files, it appeared to be the MetaSWAP recharge package. I asked Wouter to profile it and just analysed the data. The profiler mentioned 87% of the time was spent in this function

imod-python/imod/mf6/wel.py

Line 531 in a5f0cd3

def gather_filtered_well_ids(
. So I think what happened: The msw-sprinkling.wel file was written, but we were still waiting for all the corresponding data files to be written, before the msw-rch.rch could be written. Reproducing it should be quite simple: Just creating some large CAP data grids and seeing what takes very long. Also performance might be worse if loaded as dask.

See Snakeviz table:

Image
@JoerivanEngelen JoerivanEngelen added the bug Something isn't working label Feb 20, 2025
@github-project-automation github-project-automation bot moved this to 📯 New in iMOD Suite Feb 20, 2025
@JoerivanEngelen JoerivanEngelen moved this from 📯 New to 🤝 Accepted in iMOD Suite Feb 20, 2025
@JoerivanEngelen JoerivanEngelen mentioned this issue Feb 21, 2025
Merged
5 tasks
@JoerivanEngelen JoerivanEngelen self-assigned this Feb 21, 2025
@JoerivanEngelen JoerivanEngelen added this to the v1.0 release milestone Feb 21, 2025
@JoerivanEngelen JoerivanEngelen moved this from 🤝 Accepted to 🧐 In Review in iMOD Suite Feb 21, 2025
github-merge-queue bot pushed a commit that referenced this issue Feb 21, 2025
@JoerivanEngelen
Issue #1445 fix performance big model (#1446)
dd96909 
Fixes #1445

It turned out this performance issue pertained from a performance issue
where calling ``xr.DataArray().isin(pd.Series())`` was very slow for
large datasets, whereas ``pd.Series().isin(pd.Series())`` was a lot
faster.

# Description
- Fix performance issue when writing large MODFLOW6 WEL packages
- Add test which tests this performance, I had to add a small pytest
plugin for this.

# Checklist
- [x] Links to correct issue
- [x] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example
@github-project-automation github-project-automation bot moved this from 🧐 In Review to ✅ Done in iMOD Suite Feb 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JoerivanEngelen JoerivanEngelen

Labels
bug Something isn't working
Projects
iMOD Suite
Status: ✅ Done
Milestone
v1.0 release
Development

Successfully merging a pull request may close this issue.

Issue #1445 fix performance big model Deltares/imod-python
1 participant
@JoerivanEngelen