Hi @MishaLaskin. Thanks for sharing your code@ I have found a bug when running the pertaining script.

At the line 49 value = time_step[spec.name] in replay_buffer.py, we have an error that return getattr(self, attr) at the line 30 in dmc.py will be called twice. This leads to an error where the second call will take as input attribute an int-type attr. It is not so clear about the exact reason for this issue, but I guess it may come from the inheritance of class ExtendedTimeStep. It can be solved by deleting such inheritance and writing an explicit __init__() function. Hope it helps:)