Correctly handle non-scalar rewards for dm_env - #162
Conversation
| """ | ||
| obs = dm_obs2gym_obs(timestep.observation) | ||
| reward = timestep.reward or 0 | ||
| reward = timestep.reward if timestep.reward is not None else 0 |
There was a problem hiding this comment.
| reward = timestep.reward if timestep.reward is not None else 0 | |
| if hasattr(timestep.reward, "__getitem__") and hasattr(timestep.reward, "__len__"): | |
| if len(timestep.reward) == 1: | |
| reward = timestep.reward[0] | |
| else: | |
| raise TypeError( | |
| f"Gymnasium only supports scalar reward, got {timestep.reward}" | |
| ) | |
| else: | |
| reward = timestep.reward or 0 |
There was a problem hiding this comment.
Gymnasium doesn't support vector rewards, so we should handle them properly and reject any reward with a length other than 1.
There was a problem hiding this comment.
length-1 arrays unwrap, anything else raises TypeError. kept is not None so 0.0 stays a float.
|
Well, the issue is that Gymnasium environments only support reward values of type |
Gymnasium only accepts scalar SupportsFloat rewards. Unwrap a 1-d length-1 array, pass through 0-d numpy values, and raise TypeError for other vectors including shape (1, 2). Keep is not None so 0.0 stays float.
This should be a few times faster for all cases
|
|
dm_env_step2gym_stepusedtimestep.reward or 0. That turns a real 0.0 into int 0, and a vector reward raisesValueError: The truth value of an array with more than one element is ambiguous.Swap it for
is not None. FIRST timesteps still become 0; 0.0 stays a float; arrays pass through.The three cases live in
tests/test_dm_control.pynext to the DiscreteArray dtype test from #157.I ran the three conversions against
dm_env.TimeStephere (no dm-control extra on this box).black,isort --profile black, andpydocstyle --convention=googleare clean on the two files.