This is purely an idea if you wish and I understand how much work it is to get this working.
Thankfully, https://github.com/mttga/purejaxql has shown this to be possible achieving 200 million frames in 1 hour with EnvPool XLA support.
Therefore, would be cool to do hyperparameter search, etc over Atari environments