Closed
Description
Environment
- Pythonnet version: 2.5.1
- Python version: 3.8.2
- Operating System: Windows 10 (.Net 4.7.2, 64bit)
Details
I use the technique described here to convert a Numpy ndarray to .Net Array and use the resulting array in C# code. However, the array is not garbage collected, as the Python interpreter keeps a reference on the array (Python.Runtime.CLRObject).
I have included an example below, where each step in the iteration increases the memory usage by about 8MB, since the old arrays are not garbage collected.
I have tried using the Dispose() method on the array, and forcing the Python garbage collector (as suggested in other issues), but to no avail. I have also tried 100 other things, but there is no to re-iterate all the failures here.
Not sure where to go from here. Any suggestions are welcome!
Example code
C# code Program.cs:
using System;
using Python.Runtime;
namespace ConsoleApp1
{
class Program
{
private dynamic module;
public Program()
{
Environment.SetEnvironmentVariable("PYTHONPATH", Environment.CurrentDirectory, EnvironmentVariableTarget.Process);
PythonEngine.Initialize();
using (Py.GIL())
{
module = Py.Import("testmodule");
}
}
~Program()
{
module = null;
PythonEngine.Shutdown();
}
Array Step()
{
Array a;
using (Py.GIL())
{
a = module.step();
}
return a;
}
public void Iterate()
{
Array a;
for(int i=0; i<100; i++)
{
a = Step();
}
}
static void Main(string[] args)
{
Program p = new Program();
p.Iterate();
}
}
}Python code testmodule.py:
import numpy as np
import ctypes
import System
from System.Runtime.InteropServices import GCHandle, GCHandleType
#
# C# <-> Numpy conversion adopted from
# https://github.com/pythonnet/pythonnet/issues/514#issuecomment-350375105
_MAP_NP_NET = {
np.dtype('float32'): System.Single,
np.dtype('float64'): System.Double,
np.dtype('int8'): System.SByte,
np.dtype('int16'): System.Int16,
np.dtype('int32'): System.Int32,
np.dtype('int64'): System.Int64,
np.dtype('uint8'): System.Byte,
np.dtype('uint16'): System.UInt16,
np.dtype('uint32'): System.UInt32,
np.dtype('uint64'): System.UInt64,
np.dtype('bool'): System.Boolean,
}
def asNetArray(npArray):
"""Given a `numpy.ndarray` returns a CLR `System.Array`. See _MAP_NP_NET for
the mapping of Numpy dtypes to CLR types.
"""
dims = npArray.shape
dtype = npArray.dtype
if not npArray.flags.c_contiguous:
npArray = npArray.copy(order='C')
assert npArray.flags.c_contiguous
try:
if npArray.ndim == 1:
netArray = System.Array.CreateInstance(_MAP_NP_NET[dtype], dims[0])
elif npArray.ndim == 2:
netArray = System.Array.CreateInstance(
_MAP_NP_NET[dtype], dims[0], dims[1])
elif npArray.ndim == 3:
netArray = System.Array.CreateInstance(
_MAP_NP_NET[dtype], dims[0], dims[1], dims[2])
else:
raise NotImplementedError(
"asNetArray does not support arrays of ndim={}".format(
npArray.ndim))
except KeyError:
raise NotImplementedError(
"asNetArray does not yet support dtype {}".format(dtype))
destHandle = None
try:
# Memmove
destHandle = GCHandle.Alloc(netArray, GCHandleType.Pinned)
sourcePtr = npArray.__array_interface__['data'][0]
destPtr = destHandle.AddrOfPinnedObject().ToInt64()
ctypes.memmove(destPtr, sourcePtr, npArray.nbytes)
finally:
if destHandle is not None and destHandle.IsAllocated:
destHandle.Free()
return netArray
def step():
a = np.random.random(1000000)
return asNetArray(a)Metadata
Metadata
Assignees
Labels
No labels