Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[mypyc] Add primitive for bytes.decode - #10951

Merged
JukkaL merged 6 commits into
python:masterfrom
97littleleaf11:decode
Aug 11, 2021
Merged

[mypyc] Add primitive for bytes.decode#10951
JukkaL merged 6 commits into
python:masterfrom
97littleleaf11:decode

Conversation

@97littleleaf11

Copy link
Copy Markdown
Collaborator

Description

Implements part of mypyc/mypyc#880

@97littleleaf11
97littleleaf11 marked this pull request as ready for review August 9, 2021 13:45
Comment thread mypyc/lib-rt/str_ops.c Outdated
} No newline at end of file
}

#define PyUnicode_UTF8(op) \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better not use a macro here, as it buys us little and makes the code harder to understand and maintain. If you need to share code, use an inline function.

Comment thread mypyc/lib-rt/str_ops.c Outdated

#define PyUnicode_UTF8(op) \
(assert(PyUnicode_Check(op)), \
assert(PyUnicode_IS_READY(op)), \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to crash the process in case the string object is not ready, which is not the right thing to do? Instead, you should probably use PyUnicode_READY and return NULL if it fails.

Comment thread mypyc/primitives/str_ops.py Outdated
decode_types: List[RType] = [bytes_rprimitive, str_rprimitive, str_rprimitive]
decode_constants: List[Tuple[int, RType]] = [(0, pointer_rprimitive),
(0, pointer_rprimitive)]
for i in range(len(decode_types)):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The for loop doesn't save a lot of code and makes this harder to understand. I'd prefer just having three normal primitive definitions.

return_type=c_pyssize_t_rprimitive,
c_function_name='CPyStr_Size_size_t',
error_kind=ERR_NEG_INT)

@97littleleaf11 97littleleaf11 Aug 10, 2021

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We prefer three separate helper functions.

Comment thread mypyc/lib-rt/str_ops.c Outdated
}
}

PyObject* CPy_DecodeWithErrors(PyObject *obj, PyObject *encoding, PyObject *errors) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline, I'd prefer to have three primitive definitions that all call a single C function, i.e. we'd only duplicate the method_op declarations, not C implementations. Some primitives can provide fixed extra arguments, but without using the for loop for clarity. Sorry for the extra back and forth!

@JukkaL JukkaL left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates!

assert b.decode('gbk') == '浣犲ソ'
assert b.decode('latin1') == 'ä½\xa0好'

[case testEncode]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it would be reasonable to have them in a single test case, since these are related (all string operations) and having fewer run tests will speed up tests. But it's not a big deal. The original name (testChrOrdEncodeDecode) wasn't the clearest though. Also we already have testStringOps, which would cover these as well. Not sure what's the best way to organize our tests.

@JukkaL
JukkaL merged commit 5adb0a0 into python:master Aug 11, 2021
@97littleleaf11
97littleleaf11 deleted the decode branch February 22, 2022 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants