In batch mode machinery, there are some places where pydap should refactor code. Of particular importance is after downloading data and de-serializing.

The following snippet of code is not production ready, even though the result is correct:

The data is deserialized into numpy arrays (in memory) and held into a dictionary within the dataset object. However, a better way to handle this deserialized data that does not hold the arrays in memory, is the way the Dap4BaseProxy handles it. For example, in the following:

So that data, once it is read once, it clears the RAM and no longer is held into memory.

The workflow then depends on a secondary function called which retrieves the data from the batch promise, and then assigns it back to the original dataset as an inmemory numpy object. The function snippet is

Rather than having a dictionary retain the data arrays in memory and then fetched them / assign them as in-memory numpy arrays, these need to be assigned to the dataset itself from the initial step of deserializing the dap response.

This is a must before any release. A proper testing should show there is no explosion of RAM memory usage.