Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Fix Unicode support for PyPy #4145

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hugovk merged 2 commits into from
Nov 24, 2019
Merged

Fix Unicode support for PyPy #4145

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
837d8ae
fix support for extended unicode characters in PyPy
nulano Oct 12, 2019
64317f8
raqm now works with PyPy on Windows
nulano Oct 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .github/workflows/test-windows.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,6 @@ jobs:

# for Raqm
- name: Build dependencies / HarfBuzz
if: "!contains(matrix.python-version, 'pypy')"
run: |
set INCLUDE=C:\Program Files (x86)\Microsoft SDKs\Windows\V7.1A\Include
set INCLIB=%GITHUB_WORKSPACE%\winbuild\depends\msvcr10-x32
Expand All @@ -274,7 +273,6 @@ jobs:

# for Raqm
- name: Build dependencies / FriBidi
if: "!contains(matrix.python-version, 'pypy')"
run: |
set INCLUDE=C:\Program Files (x86)\Microsoft SDKs\Windows\V7.1A\Include
set INCLIB=%GITHUB_WORKSPACE%\winbuild\depends\msvcr10-x32
Expand All @@ -292,9 +290,7 @@ jobs:
copy /Y /B *.lib %INCLIB%
shell: cmd

# failing with PyPy3
- name: Build dependencies / Raqm
if: "!contains(matrix.python-version, 'pypy')"
run: |
set INCLUDE=C:\Program Files (x86)\Microsoft SDKs\Windows\V7.1A\Include
set INCLIB=%GITHUB_WORKSPACE%\winbuild\depends\msvcr10-x32
Expand Down
2 changes: 1 addition & 1 deletion Tests/test_imagefont.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -463,7 +463,7 @@ def test_unicode_pilfont(self):
with self.assertRaises(UnicodeEncodeError):
font.getsize("’")

@unittest.skipIf(is_pypy(), "requires CPython")
@unittest.skipIf(is_pypy(), "failing on PyPy")
def test_unicode_extended(self):
# issue #3777
text = "A\u278A\U0001F12B"
Expand Down
16 changes: 2 additions & 14 deletions src/_imagingft.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -326,24 +326,12 @@ getfont(PyObject* self_, PyObject* args, PyObject* kw)
static int
font_getchar(PyObject* string, int index, FT_ULong* char_out)
{
#if (defined(PYPY_VERSION_NUM))
if (PyUnicode_Check(string)) {
Py_UNICODE* p = PyUnicode_AS_UNICODE(string);
int size = PyUnicode_GET_SIZE(string);
if (index >= size)
return 0;
*char_out = p[index];
return 1;
}
#else
if (PyUnicode_Check(string)) {
if (index >= PyUnicode_GET_LENGTH(string))
return 0;
*char_out = PyUnicode_READ_CHAR(string, index);
return 1;
}
#endif

return 0;
}

Expand All @@ -363,7 +351,7 @@ text_layout_raqm(PyObject* string, FontObject* self, const char* dir, PyObject *
goto failed;
}

#if (defined(PYPY_VERSION_NUM))
#if (defined(PYPY_VERSION_NUM) && (PYPY_VERSION_NUM < 0x07020000))
Copy link
Contributor Author

@nulano nulano Oct 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a compile time PyPy version check here, to be removed in the future. What is Pillow's policy on supporting old versions of PyPy?

Aside: This fallback code fails on Windows anyway (due to the cast in the p_raqm.set_text call), see #4084 (comment).

Copy link
Member

@hugovk hugovk Oct 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For CPython we support the same versions as they do.

For PyPy, we generally only test the latest version, so maybe that too? If it's trivial to support older ones, we could do so, but I'm not sure of the value in testing many old versions.

Does the PyPy project have a policy on which PyPy versions they support?

(PyPy plan to support Python 2.7 "forever" http://doc.pypy.org/en/latest/faq.html#how-long-will-pypy-support-python2, but we won't support PyPy Python 2.7 after 2020-01-01.)

if (PyUnicode_Check(string)) {
Py_UNICODE *text = PyUnicode_AS_UNICODE(string);
Py_ssize_t size = PyUnicode_GET_SIZE(string);
Expand Down Expand Up @@ -392,7 +380,7 @@ text_layout_raqm(PyObject* string, FontObject* self, const char* dir, PyObject *
and raqm fails with empty strings */
goto failed;
}
int set_text = (*p_raqm.set_text)(rq, (const uint32_t *)(text), size);
int set_text = (*p_raqm.set_text)(rq, text, size);
PyMem_Free(text);
if (!set_text) {
PyErr_SetString(PyExc_ValueError, "raqm_set_text() failed");
Expand Down