Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Realign4dAlgorithm memleaks? #398

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
bpinsard opened this issue Mar 21, 2016 · 4 comments
Open

Realign4dAlgorithm memleaks? #398

bpinsard opened this issue Mar 21, 2016 · 4 comments

Comments

@bpinsard
Copy link

I have problems when using Realign4dAlgorithm, especially with large files in the nipype interface.
The memory seems no to be fully cleared then causing memory hog.
I tracked down with mprun down to cython function which seems to be the source.
_cspline_sample4d (3d/2d is certainly the same) to which an output array is given still take memory that is not released afterward.

for example running multiple times resampling (wrapped for memory_profiler to work) with the same value does take some memory (even if it stabilizes after a few). If I change the coordinates it again eat some more of the memory then stabilizes...

In [9]: %mprun -f memleaks.cspline_samp csampl=memleaks.cspline_samp(out,cdata,coords,coords,coords,coords)
Filename: /home/bpinsard/data/tests/memleaks.py

Line #    Mem usage    Increment   Line Contents
================================================
     7    896.8 MiB      0.0 MiB   def cspline_samp(*args):
     8    904.4 MiB      7.6 MiB       nipy.algorithms.registration._registration._cspline_sample4d(*args)
('',)

In [10]: %mprun -f memleaks.cspline_samp csampl=memleaks.cspline_samp(out,cdata,coords,coords,coords,coords)
Filename: /home/bpinsard/data/tests/memleaks.py

Line #    Mem usage    Increment   Line Contents
================================================
     7    904.4 MiB      0.0 MiB   def cspline_samp(*args):
     8    904.5 MiB      0.1 MiB       nipy.algorithms.registration._registration._cspline_sample4d(*args)
('',)

In [11]: %mprun -f memleaks.cspline_samp csampl=memleaks.cspline_samp(out,cdata,coords,coords,coords,coords)
Filename: /home/bpinsard/data/tests/memleaks.py

Line #    Mem usage    Increment   Line Contents
================================================
     7    904.5 MiB      0.0 MiB   def cspline_samp(*args):
     8    904.5 MiB      0.0 MiB       nipy.algorithms.registration._registration._cspline_sample4d(*args)
('',)

@alexis-roche do you have any idea? Thanks
@alexis-roche
Copy link
Member

@bpinsard As far as I can see, _cspline_sample4d does not allocate dynamic memory. The input arrays should be handled by Python's garbage collector.

@bpinsard
Copy link
Author

Thanks for you quick reply.
I tried to force garbage collection using gc.collect() but is does not free any memory.
Can this problem be dependent on the version of python I am using?
Or even, because I run it in ipython?

@yarikoptic
Copy link
Member

I wonder if running your script through valgrind with an option to track origins would help to identify possibly outside of direct python code memory allocations which were left behind

On March 22, 2016 1:50:25 PM GMT+01:00, bpinsard [email protected] wrote:

Thanks for you quick reply.
I tried to force garbage collection using gc.collect() but is does not
free any memory.
Can this problem be dependent on the version of python I am using?
Or even, because I run it in ipython?

You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
#398 (comment)

Sent from a phone which beats iPhone.

@bpinsard
Copy link
Author

here is what I get:

...

==9898== 528 bytes in 1 blocks are possibly lost in loss record 34 of 126
==9898==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==9898==    by 0x5372A5: _PyObject_GC_Malloc (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x4C81CE: PyFrame_New (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x4A1A38: ??? (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x505F95: PyObject_Call (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x5259BD: ??? (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x5038D4: ??? (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x4DCD80: PyObject_CallFunctionObjArgs (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x49DFDE: PyEval_EvalFrameEx (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x4A090B: PyEval_EvalCodeEx (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x588D41: PyEval_EvalCode (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x588DF9: PyImport_ExecCodeModuleEx (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898== 
==9898== 536 bytes in 1 blocks are possibly lost in loss record 35 of 126
==9898==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==9898==    by 0x5372A5: _PyObject_GC_Malloc (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x4C81CE: PyFrame_New (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x4A2052: ??? (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x505F95: PyObject_Call (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x5259BD: ??? (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x5038D4: ??? (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x499BE4: PyEval_EvalFrameEx (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x499EF1: PyEval_EvalFrameEx (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x4A1C99: ??? (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x4DC9CA: PyEval_CallObjectWithKeywords (in /home/bpinsard/.virtualenvs/prod/bin/python)
==9898==    by 0x4E26FC: _PyCodec_Lookup (in /home/bpinsard/.virtualenvs/prod/bin/python)

...
==9049== LEAK SUMMARY:
==9049==    definitely lost: 0 bytes in 0 blocks
==9049==    indirectly lost: 0 bytes in 0 blocks
==9049==      possibly lost: 1,064 bytes in 2 blocks
==9049==    still reachable: 463,240 bytes in 228 blocks
==9049==         suppressed: 0 bytes in 0 blocks

I only copied the possibly lost blocks, I do not know if it helps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@yarikoptic @alexis-roche @bpinsard