Hi,
As indicated in the document, the amp.initialize() should be called once. But in cross validation, models need to be created in different folds, and creating them before CV requires extra effort. So what is the suggested way to use amp.initialize() with cross validation? I tried to call initialize() multiple times but the gpu memory seems leakaging.