Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Time format/origin of forcing and state inputs? #266

@dwoodson-usbr

Description

@dwoodson-usbr

Hello,

After the update to MetSim 2.4.2, I was able to run the tutorial without issue, but when trying to run my own data (Livneh historical data for the CRB) I received two errors (key error messages in bold):

  1. When running with a .conf file, Metsim appeared to run (progress bar loaded) for about 30 seconds, then I received the below error message. Primarily it seems related to variable names (lat and lon), but I did define my latitude and longitude as lat and lon in the configuration file. I also toyed around with changing the chunks definition of lat and lon to latitude and longitude as well, but in that case the progress bar doesn't even load.

(base) C:\Users\dwoodson\DOI\Colorado River Hydrology - Documents\General\CALC\metSim>ms configLivneh.conf -v -n 3
[ ] | 0% Completed | 28.7s2023-11-05 01:50:59,468 - distributed.worker - WARNING - Compute Failed
Key: wrap_run_slice-012f07b0-2219-4077-bfcc-37fbd531459c
Function: execute_task
args: ((<function wrap_run_slice at 0x000002147EEAB8B0>, (<class 'dict'>, [['period_ending', False], ['is_worker', False], ['method', 'mtclim'], ['domain', 'C:/Users/dwoodson/DOI/Colorado River Hydrology - Documents/General/DATA/metSim/input/livneh_domain.nc'], ['state', 'C:/Users/dwoodson/DOI/Colorado River Hydrology - Documents/General/DATA/metSim/input/Livneh_metsim_state.nc'], ['out_dir', 'C:\Users\dwoodson\DOI\Colorado River Hydrology - Documents\General\DATA\metSim\output'], ['out_prefix', 'livneh.historical_'], ['start', datetime.datetime(1948, 3, 31, 0, 0)], ['stop', datetime.datetime(1948, 12, 31, 0, 0)], ['forcing_fmt', 'netcdf'], ['time_step', '180'], ['calendar', 'standard'], ['prec_type', 'uniform'], ['out_precision', 'f4'], ['verbose', 10], ['sw_prec_thresh', 0.0], ['utc_offset', False], ['lw_cloud', 'cloud_deardorff'], ['lw_type', 'prata'], ['tdew_tol', 1e-06], ['tmax_daylength_fraction', 0.67], ['rain_scalar', 0.75], ['tday_coef', 0.45], ['lapse_rate', 0.0065], ['out_
kwargs: {}
Exception: 'ValueError("Dimensions {'lon', 'lat'} do not exist. Expected one or more of Frozen({'time': 90, 'latitude': 256, 'longitude': 224})")'

Traceback (most recent call last): ] | 0% Completed | 28.8s←[2K
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\dwoodson\AppData\Local\miniforge3\Scripts\ms.exe_main
.py", line 7, in
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\site-packages\metsim\cli\ms.py", line 62, in main
ms.run()
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\site-packages\metsim\metsim.py", line 343, in run
dask.compute(persisted)
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\site-packages\dask\base.py", line 628, in compute
results = schedule(dsk, keys, **kwargs)
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\site-packages\metsim\metsim.py", line 804, in wrap_run_slice
ms.run_slice()
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\site-packages\metsim\metsim.py", line 484, in run_slice
self.state.isel(**locs),
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\site-packages\xarray\core\dataset.py", line 2905, in isel
indexers = drop_dims_from_indexers(indexers, self.dims, missing_dims)
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\site-packages\xarray\core\utils.py", line 858, in drop_dims_from_indexers
raise ValueError(
ValueError: Dimensions {'lon', 'lat'} do not exist. Expected one or more of Frozen({'time': 90, 'latitude': 256, 'longitude': 224})
2023-11-05 01:51:00,271 - distributed.worker.state_machine - WARNING - Async instruction for <Task cancelled name="execute('wrap_run_slice-0195ca0c-fafa-4a63-910b-9639e0f2a35a')" coro=<Worker.execute() done, defined at C:\Users\dwoodson\AppData\Local\miniforge3\lib\site-packages\distributed\worker_state_machine.py:3610>> ended with CancelledError
2023-11-05 01:51:03,456 - distributed.nanny - WARNING - Worker process still alive after 3.1999983215332035 seconds, killing

  1. When I ran with the configuration file in .yaml format, MetSim failed but the error was different and strictly related to my datetime variable:

(base) C:\Users\dwoodson\DOI\Colorado River Hydrology - Documents\General\CALC\metSim>ms configLivneh.yaml --verbose
Traceback (most recent call last):
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\dwoodson\AppData\Local\miniforge3\Scripts\ms.exe_main
.py", line 7, in
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\site-packages\metsim\cli\ms.py", line 61, in main
ms = MetSim(setup)
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\site-packages\metsim\metsim.py", line 223, in init
self._times = self._get_output_times(
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\site-packages\metsim\metsim.py", line 531, in _get_output_times
prototype = self.met_data
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\site-packages\metsim\metsim.py", line 309, in met_data
self._validate_force_times(force_times=self._met_data['time'])
File "C:\Users\dwoodson\AppData\Local\miniforge3\lib\site-packages\metsim\metsim.py", line 260, in _validate_force_times
assert self.params['start'] >= pd.Timestamp(
TypeError: can't compare datetime.datetime to datetime.date

Both of these approaches do output a .nc file but it is appears to be corrupted.

When I dig into my netcdf files and the netcdf files from the tutorial (plus another MetSim user's files #164) with ncdump - I see that my netcdfs have the time format as 'time:units = "seconds since 1970-1-1 00:00:00"' whereas the tutorial files and other users' files have a time format of 'time:units = "days since 1900-01-01 00:00:00" '. Does the time format/origin make a difference? I don't see much else in the ncdump info that would indicate an issue.

I have put all of my input data, config files, and the output at the below link. The lat/lon coords appear to be identical between the domain, forcing, and state files. Please advise and thanks for your help!

https://o365coloradoedu-my.sharepoint.com/:f:/g/personal/dawo7917_colorado_edu/Ek1WP3K5-oJOkjQ3aI5j4vsBzAPr_viLSQzmXPAQhi6oCw?e=2ekBFG

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions