Issue #1536 reduce values calls splitting #1537
Conversation
| with raise_if_dask_computes(): | ||
| assert _skip_dataarray(grid) is False | ||
| assert _skip_dataarray(xr.DataArray(True)) is True | ||
| assert _skip_dataarray(layer_da) is True |
There was a problem hiding this comment.
Line 22 and 23 is independent of the cases provided to this test. You could put them in a separate test.
Maybe name the tests something along:
test_skip_dataarray_grids_types
test_skip_dataarray_non_grid_types
There was a problem hiding this comment.
Good point, I decided to add separate cases for the layered constants and constants to GridCases, which I renamed to DataArrayCases.
imod/mf6/exchangebase.py
Outdated
| ) | ||
|
|
||
| all_geometric_vars = ["ihc", "cl1", "cl2", "hwva", "angldegx", "cdist"] | ||
| for var in all_geometric_vars: |
There was a problem hiding this comment.
You can combine these line
for var in all_geometric_vars if var in in self.dataset.data_vars:
There was a problem hiding this comment.
This doesn't work without a list/dict comprehension. So it would end up in this:
geometric_vars = ["ihc", "cl1", "cl2", "hwva", "angldegx", "cdist"]
vars_to_render.update({
var: (index_dim, self.dataset[var].data)
for var in geometric_vars if var in self.dataset.data_vars
})Which looks more complicated than currently.
| for var in all_geometric_vars: | ||
| if var in self.dataset.data_vars: | ||
| vars_to_render[var] = (index_dim, self.dataset[var].data) | ||
| datablock = xr.merge([vars_to_render], join="exact").to_dataframe() |
There was a problem hiding this comment.
What does merge on a single object do?
There was a problem hiding this comment.
xr.merge merges all dictionaries which map variable names to DataArrays into a single xr.Dataset. So with one element in the list, the dictionary is converted to a xr.Dataset. xr.merge doesn't support providing a single dictionary directly.
|
Fixes #1536
Description
.values_skip_if_datarraywhere the spatial dimension of unstructured grids was hardcodedxr.Dataset().to_dataframe(), this to carefully merge variables into a dataset with matching dimensions, then convert to pandas dataframe.I can't verify yet if this improves things with Teun's example as I do not have his scripts, but this fixes a bug and will improve performance somewhat when using dask (as it reduces unnecessary loads into memory)
Checklist
Issue #nr, e.g.Issue #737