Reset the available animation movie writer on rcParam change #5628

jankatins · 2015-12-06T21:36:52Z

If one of the rcParams for a path to a program, which was called by a
movie writer, is changed, the the available movie writer in the
registry should be reevaluated if they are (still/became) available.

This also fixes the problem that you have to set the path to a movie
writer before importing mpl.animation, as before the state was
fixed on import time.

Closes: #5616

jankatins · 2015-12-09T10:08:45Z

lib/matplotlib/animation.py

+        """Reset the available state of all registered writers"""
+        self.avail = {}
+        for name, writerClass in self._registered.items():
+            if writerClass.isAvailable():


BTW: should that be "is_available"? It's the only method in that class(es) which use CamelCase instead of underscores...

It is annoying, but I don't think worth breaking user code over.

At most, we can rename and add an alias isAvailable -> is_available with the @deprecated decorator.

jankatins · 2015-12-27T17:08:51Z

Apart from the possible deprecation (which should go to a different PR), IMO this is ready to go in...

dopplershift · 2015-12-28T17:14:41Z

lib/matplotlib/animation.py

            if writerClass.isAvailable():
                self.avail[name] = writerClass
            return writerClass
        return wrapper

+    def endsure_not_dirty(self):


should endsure be ensure?

Omg, fix is on the way...

jankatins · 2016-02-06T13:23:48Z

Ping?

dopplershift · 2016-02-06T18:45:48Z

lib/matplotlib/rcsetup.py

+    if not isinstance(p, six.text_type):
+        raise ValueError("path must be a (unicode) string")
+    from sys import modules
+    # set dirty, so that the next call to the reistry will reevaluate


... registry will re-evaluate

done and repushed

dopplershift · 2016-02-06T18:47:08Z

Minor comment nit, otherwise this is 👍 from me.

If one of the rcParams for a path to a program, which was called by a movie writer, is changed, the the available movie writer in the registry should be reevaluated if they are (still/became) available. This also fixes the problem that you have to set the path to a movie writer before importing mpl.animation, as before the state was fixed on import time.

python is blocking, leading to timeouts...

jankatins · 2016-02-06T19:41:22Z

cleaned up the typo and repushed

WeatherGod · 2016-02-06T22:13:08Z

lib/matplotlib/tests/test_animation.py

+    animation.writers.list()  # resets
+    assert_false(animation.writers._dirty)
+    assert_false(animation.writers.is_available("ffmpeg"))
+    # something which is garanteered to be available in path


"garanteered"?

WeatherGod · 2016-02-06T22:14:34Z

Appveyor is having all sorts of failures, some of them having to do with animation stuff, so I am not entirely certain if it is the usual failures or not.

On windows the check ensures that the imagemagick path for 'convert' was set to '' if it was just 'convert' to prevent clashes with the windows built-in command 'convert' (doing filesystem conversion). Unfortunately, the command only set this values in the current rcParams and rcParamsDefault, but a use of `matplotlib.style.use("classic")` after importing mpl.animation would again overwrite it with `convert`. In this case, the image comparison decorator sets the style to classic and appveyor went BOOM...

jankatins · 2016-02-07T16:22:08Z

The imagemagic problem is that the workaround for masking convert on windows (which is always available to to the internal convert.exefrom windows) is undone by a later call to matplotlib.style.use("classic") in the image comparison decorator...

It seems that the workaround should also be applied in the is_available method...

And for your daily dose of WTF:

c:\data\external\R
λ C:\Users\jschulz\Dropbox\Programme\cmder\vendor\imagemagick\convert --version
Version: ImageMagick 7.0.0-0 Q16 x64 2016-01-23 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2015 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Visual C++: 180040629
Features: Cipher DPC HDRI
Delegates (built-in): bzlib cairo freetype jng jp2 jpeg lcms lqr openexr pangocairo png ps rsvg tiff webp xml zlib

c:\data\external\R
λ echo %errorlevel%
1
c:\data\external\R                         
λ convert                                  
Ein Dateisystem muss angegeben werden.     

c:\data\external\R                         
λ echo %errorlevel%                        
4

and running the tests with mpl.verbose.set_level("debug") triggers a different bug (pobably in subprocess)?:

======================================================================
ERROR: matplotlib.tests.test_animation.test_save_animation_smoketest('mencoder_file', 'mp4')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\portabel\miniconda\envs\matplotlib_build\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "c:\data\external\pydata\matplotlib\lib\matplotlib\testing\decorators.py", line 152, in wrapped_callable

    func(*args, **kwargs)
  File "c:\data\external\pydata\matplotlib\lib\matplotlib\tests\test_animation.py", line 137, in check_save_animation
    anim.save(F.name, fps=30, writer=writer, bitrate=500)
  File "c:\data\external\pydata\matplotlib\lib\matplotlib\animation.py", line 893, in save
    writer.grab_frame(**savefig_kwargs)
  File "C:\portabel\miniconda\envs\matplotlib_build\lib\contextlib.py", line 66, in __exit__
    next(self.gen)
  File "c:\data\external\pydata\matplotlib\lib\matplotlib\animation.py", line 174, in saving
    self.finish()
  File "c:\data\external\pydata\matplotlib\lib\matplotlib\animation.py", line 453, in finish
    self._run()
  File "c:\data\external\pydata\matplotlib\lib\matplotlib\animation.py", line 283, in _run
    creationflags=subprocess_creation_flags)
  File "C:\portabel\miniconda\envs\matplotlib_build\lib\subprocess.py", line 914, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "C:\portabel\miniconda\envs\matplotlib_build\lib\subprocess.py", line 1145, in _get_handles
    c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
io.UnsupportedOperation: fileno

jankatins · 2016-02-07T16:22:26Z

I also currently have a bug that mencoder returns 3:

======================================================================
ERROR: matplotlib.tests.test_animation.test_save_animation_smoketest('mencoder_file', 'mp4')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\portabel\miniconda\envs\matplotlib_build\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "c:\data\external\pydata\matplotlib\lib\matplotlib\testing\decorators.py", line 152, in wrapped_callable

    func(*args, **kwargs)
  File "c:\data\external\pydata\matplotlib\lib\matplotlib\tests\test_animation.py", line 140, in check_save_animation
    anim.save(F.name, fps=30, writer=writer, bitrate=500)
  File "c:\data\external\pydata\matplotlib\lib\matplotlib\animation.py", line 894, in save
    writer.grab_frame(**savefig_kwargs)
  File "C:\portabel\miniconda\envs\matplotlib_build\lib\contextlib.py", line 66, in __exit__
    next(self.gen)
  File "c:\data\external\pydata\matplotlib\lib\matplotlib\animation.py", line 174, in saving
    self.finish()
  File "c:\data\external\pydata\matplotlib\lib\matplotlib\animation.py", line 462, in finish
    + ' Try running with --verbose-debug')
RuntimeError: Error creating movie, return code: 3 Try running with --verbose-debug

I've actually no clue why that's happens :-/ Anyone with mencoder knowledge here?

jankatins · 2016-02-07T17:01:12Z

The error seems to be this (pull from self._proc._stderr_buff via the debugger):

[b"
WARNING: OUTPUT FILE FORMAT IS _AVI_. See -of help.
[mpeg4 @ 00000000019034e0]AVFrame.format is not set
[mpeg4 @ 00000000019034e0]AVFrame.width or height is not set

[mpeg4 @ 00000000019034e0]AVFrame.format is not set
[mpeg4 @ 00000000019034e0]AVFrame.width or height is not set

[mpeg4 @ 00000000019034e0]AVFrame.format is not set
[mpeg4 @ 00000000019034e0]AVFrame.width or height is not set

[mpeg4 @ 00000000019034e0]AVFrame.format is not set
[mpeg4 @ 00000000019034e0]AVFrame.width or height is not set

[mpeg4 @ 00000000019034e0]AVFrame.format is not set
[mpeg4 @ 00000000019034e0]AVFrame.width or height is not set
Assertion v>0 && v<=(1 ? 32 : 16) failed at libavutil/mem.c:233

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
"]

jankatins · 2016-02-07T17:57:04Z

It seems the problem is this: Assertion v>0 && v<=(1 ? 32 : 16) failed at libavutil/mem.c:233. I've no clue what that means but it seems that others also have no clue, at least I didn't find any proper answer by googling :-(

E.g. here: http://ehc.ac/p/mplayer-win32/bugs/179/ -> unanswered :-(

I use this build: http://mplayerwin.sourceforge.net/downloads.html (build mplayer-svn-37610-x86_64.7z)

-> not a mpl bug and so IMO if this passes (without mplayer/mencoder, which is not installed on appveyor) and no one finds another typo :-), this is IMO good to merge...

This pulls out the stderr and stdout from the internal POpen buffers (tested on py3.5 and not shown if any error occures while pulling out the messages...). It also changes the exception message to mention mpl.verbose.set_level("helpful") because adding a commandline switch is probably not helpful in todays world where everybody uses the Jupyter notebook and can't add to the kernel commandline... It would be nice if this verbose level could be set by nose / tests.py, but I have found no way yet :-(

jankatins · 2016-02-07T18:10:08Z

lib/matplotlib/animation.py

+                verbose.report("MovieWriter.finish: stdout: %s" % stdout, level='helpful')
+                verbose.report("MovieWriter.finish: stderr: %s" % stderr, level='helpful')
+            except Exception as e:
+                pass


This is not my finest code, but it works on py35...

This is just to provide improved debugging feed back right?

yes, when the called converter fails and the user adds the mpl.verbose.set_level("helpful") before calling the failing method again...

Unfortunately, it doesn't work for the unittests unless I add such a set_level call in the unittest file :-(

tacaswell · 2016-02-07T19:22:43Z

Looks like some long lines have crept in

======================================================================
FAIL: matplotlib.tests.test_coding_standards.test_pep8_conformance_installed_files
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/matplotlib/matplotlib/venv/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/tests/test_coding_standards.py", line 249, in test_pep8_conformance_installed_files
    expected_bad_files=expected_bad_files)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/tests/test_coding_standards.py", line 143, in assert_pep8_conformance
    assert_equal(result.total_errors, 0, msg)
AssertionError: 3 != 0 : Found code syntax errors (and warnings):
/home/travis/build/matplotlib/matplotlib/lib/matplotlib/animation.py:462:80: E501 line too long (90 > 79 characters)
/home/travis/build/matplotlib/matplotlib/lib/matplotlib/animation.py:463:80: E501 line too long (90 > 79 characters)
/home/travis/build/matplotlib/matplotlib/lib/matplotlib/animation.py:468:80: E501 line too long (81 > 79 characters)

tacaswell · 2016-02-07T19:23:47Z

👍 modulo small comments / pep8

jankatins · 2016-02-07T20:33:35Z

Oh damn, I really should add a pep8 precommit / prepush hook in the matplotlib repo...

Ok, added comments and hopefully fixed the PEP8 thingies...

Reset the available animation movie writer on rcParam change

efiring · 2016-10-18T19:09:53Z

@tacaswell, it looks to me like this could be backported to v2.x, in the spirit of making 2.0 less likely to trip people up.

tacaswell · 2016-10-19T14:59:30Z

I completely forget about this and was contemplating re-writing it to fix issues that Christoph reported.

Reset the available animation movie writer on rcParam change

Merge pull request #5628 from JanSchulz/ani_writer

efiring · 2016-10-22T03:05:07Z

backported to v2.x via #7306

tacaswell added the status: needs review label Dec 6, 2015

jankatins force-pushed the ani_writer branch from ad2dedd to 90b8e27 Compare December 6, 2015 21:47

tacaswell added this to the proposed next point release (2.1) milestone Dec 8, 2015

jankatins force-pushed the ani_writer branch from 90b8e27 to 0a1ae7c Compare December 9, 2015 08:37

jankatins reviewed Dec 9, 2015
View reviewed changes

jankatins force-pushed the ani_writer branch 2 times, most recently from 7d847ea to b46d8aa Compare December 9, 2015 14:53

dopplershift reviewed Dec 28, 2015
View reviewed changes

jankatins force-pushed the ani_writer branch from b46d8aa to 5a31ad3 Compare December 28, 2015 17:51

dopplershift reviewed Feb 6, 2016
View reviewed changes

jankatins added 2 commits February 6, 2016 20:40

TST: Use a better command for annimation tests

6a01829

python is blocking, leading to timeouts...

jankatins force-pushed the ani_writer branch from 5a31ad3 to 6a01829 Compare February 6, 2016 19:40

WeatherGod reviewed Feb 6, 2016
View reviewed changes

fix spelling

96b439a

jankatins reviewed Feb 7, 2016
View reviewed changes

jankatins added 2 commits February 7, 2016 21:26

Pep8 fixes

33f564a

Add comment to explain the fix in ImageMagickBase

e783867

tacaswell added a commit that referenced this pull request Feb 7, 2016

Merge pull request #5628 from JanSchulz/ani_writer

424556b

Reset the available animation movie writer on rcParam change

tacaswell merged commit 424556b into matplotlib:master Feb 7, 2016

mdboom removed the status: needs review label Feb 7, 2016

tacaswell added a commit to tacaswell/matplotlib that referenced this pull request Oct 19, 2016

Merge pull request matplotlib#5628 from JanSchulz/ani_writer

70418b9

Reset the available animation movie writer on rcParam change

NelleV added a commit that referenced this pull request Oct 19, 2016

Merge pull request #7306 from tacaswell/backport_win_animation_reg

f91dff4

Merge pull request #5628 from JanSchulz/ani_writer

QuLogic modified the milestones: 2.0.1 (next bug fix release), 2.1 (next point release) Oct 22, 2016

QuLogic added the topic: animation label Oct 22, 2016

QuLogic modified the milestones: 2.0.1 (next bug fix release), 2.0 (style change major release) Dec 7, 2016

anntzer mentioned this pull request Sep 20, 2017

after the animation encoder (e.g. ffmpeg) fails, the animation framework itself fails internally in various ways while trying to report the error #9205

Closed

Uh oh!

Reset the available animation movie writer on rcParam change #5628

Reset the available animation movie writer on rcParam change #5628

Uh oh!

Conversation

jankatins commented Dec 6, 2015

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jankatins commented Dec 27, 2015

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jankatins commented Feb 6, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dopplershift commented Feb 6, 2016

Uh oh!

jankatins commented Feb 6, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

WeatherGod commented Feb 6, 2016

Uh oh!

jankatins commented Feb 7, 2016

Uh oh!

jankatins commented Feb 7, 2016

Uh oh!

jankatins commented Feb 7, 2016

Uh oh!

jankatins commented Feb 7, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tacaswell commented Feb 7, 2016

Uh oh!

tacaswell commented Feb 7, 2016

Uh oh!

jankatins commented Feb 7, 2016

Uh oh!

efiring commented Oct 18, 2016

Uh oh!

tacaswell commented Oct 19, 2016

Uh oh!

efiring commented Oct 22, 2016

Uh oh!

Uh oh!