Thanks to visit codestin.com
Credit goes to github.com

Skip to content

MAINT: Deterministic SVG and PDF tests #7748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jan 21, 2017
Merged

MAINT: Deterministic SVG and PDF tests #7748

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
529afdc
Allow deleting pdf metadata keys
jkseppan Jan 5, 2017
8c7adbb
Make pdf test results (more) deterministic
jkseppan Jan 5, 2017
93b2eea
Reproducible svg
jkseppan Jan 5, 2017
eab297c
Document the changes in whats_new
jkseppan Jan 5, 2017
ec941b4
Fix documentation mistakes
jkseppan Jan 20, 2017
749e81f
Try to improve wording
jkseppan Jan 20, 2017
6736f7d
Another wording fix
jkseppan Jan 21, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions doc/users/whats_new/reproducible_ps_pdf.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,28 @@
Reproducible PS and PDF output
------------------------------
Reproducible PS, PDF and SVG output
-----------------------------------

The ``SOURCE_DATE_EPOCH`` environment variable can now be used to set
the timestamp value in the PS and PDF outputs. See
https://reproducible-builds.org/specs/source-date-epoch/

Alternatively, calling ``savefig`` with ``metadata={'creationDate': None}``
will omit the timestamp altogether.

The reproducibility of the output from the PS and PDF backends has so
far been tested using various plot elements but only default values of
options such as ``{ps,pdf}.fonttype`` that can affect the output at a
low level, and not with the mathtext or usetex features. When
matplotlib calls external tools (such as PS distillers or LaTeX) their
versions need to be kept constant for reproducibility, and they may
add sources of nondeterminism outside the control of matplotlib.

For SVG output, the ``svg.hashsalt`` rc parameter has been added in an
earlier release. This parameter changes some random identifiers in the
SVG file to be deterministic. The downside of this setting is that if
more than one file is generated using deterministic identifiers
and they end up as parts of one larger document, the identifiers can
collide and cause the different parts to affect each other.

These features are now enabled in the tests for the PDF and SVG
backends, so most test output files (but not all of them) are now
deterministic.
12 changes: 10 additions & 2 deletions lib/matplotlib/backends/backend_pdf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -472,7 +472,6 @@ def __init__(self, filename, metadata=None):
'Pages': self.pagesObject}
self.writeObject(self.rootObject, root)

revision = ''
# get source date from SOURCE_DATE_EPOCH, if set
# See https://reproducible-builds.org/specs/source-date-epoch/
source_date_epoch = os.getenv("SOURCE_DATE_EPOCH")
Expand All @@ -484,11 +483,13 @@ def __init__(self, filename, metadata=None):

self.infoDict = {
'Creator': 'matplotlib %s, http://matplotlib.org' % __version__,
'Producer': 'matplotlib pdf backend%s' % revision,
'Producer': 'matplotlib pdf backend %s' % __version__,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we even need the version here? It's in the Creator tag.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought including the version in both would help with debugging when the user has overridden just one of these. If you use Matplotlib as a component of some larger application, you might want to override Creator and leave Producer pointing to the PDF backend.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough.

'CreationDate': source_date
}
if metadata is not None:
self.infoDict.update(metadata)
self.infoDict = {k: v for (k, v) in self.infoDict.items()
if v is not None}

self.fontNames = {} # maps filenames to internal font names
self.nextFont = 1 # next free internal font name
Expand Down Expand Up @@ -2459,6 +2460,13 @@ def __init__(self, filename, keep_empty=True, metadata=None):
'Document Information Dictionary'), e.g.:
`{'Creator': 'My software', 'Author': 'Me',
'Title': 'Awesome fig'}`

The standard keys are `'Title'`, `'Author'`, `'Subject'`,
`'Keywords'`, `'Creator'`, `'Producer'`, `'CreationDate'`,
`'ModDate'`, and `'Trapped'`. Values have been predefined
for `'Creator'`, `'Producer'` and `'CreationDate'`. They
can be removed by setting them to `None`.

"""
self._file = PdfFile(filename, metadata=metadata)
self.keep_empty = keep_empty
Expand Down
5 changes: 5 additions & 0 deletions lib/matplotlib/testing/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,10 @@ def set_font_settings_for_testing():
rcParams['text.hinting_factor'] = 8


def set_reproducibility_for_testing():
rcParams['svg.hashsalt'] = 'matplotlib'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will also break, in a way, the determinism test.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain in more detail? The _test_determinism_save function in test_backend_svg.py file also sets this rc parameter to a constant value.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably along the same lines as @tacaswell; I was thinking it wouldn't fail if svg.hashsalt was broken. However, I see this test doesn't use the decorator anyway, so it is likely to still fail correctly.



def setup():
# The baseline images are created in this locale, so we should use
# it during all of the tests.
Expand All @@ -161,3 +165,4 @@ def setup():
rcdefaults() # Start with all defaults

set_font_settings_for_testing()
set_reproducibility_for_testing()
7 changes: 6 additions & 1 deletion lib/matplotlib/testing/decorators.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -298,7 +298,12 @@ def compare(self, idx, baseline, extension):
remove_ticks_and_titles(fig)

actual_fname = os.path.join(self.result_dir, baseline) + '.' + extension
fig.savefig(actual_fname, **self.savefig_kwargs)
kwargs = self.savefig_kwargs.copy()
if extension == 'pdf':
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this break the test of SOURCE_DATE_EPOCH ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that test starts a separate subprocess to write the output and doesn't use the image_comparison decorator at all.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, 🐑

kwargs.setdefault('metadata',
{'Creator': None, 'Producer': None,
'CreationDate': None})
fig.savefig(actual_fname, **kwargs)

expected_fname = self.copy_baseline(baseline, extension)
raise_on_image_difference(expected_fname, actual_fname, self.tol)
Expand Down