Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Fix potential crash when calling an overloaded function - #1327

Merged
wjakob merged 6 commits into
pybind:masterfrom
oremanj:fix-unconvertible-return-after-conversions
Sep 25, 2018
Merged

Fix potential crash when calling an overloaded function#1327
wjakob merged 6 commits into
pybind:masterfrom
oremanj:fix-unconvertible-return-after-conversions

Conversation

@oremanj

@oremanj oremanj commented Mar 20, 2018

Copy link
Copy Markdown
Collaborator

The crash would occur if:

  • dispatcher() uses two-pass logic (because the target is overloaded and some arguments support conversions)
  • the first pass (with conversions disabled) doesn't find any matching overload
  • the second pass does find a matching overload, but its return value can't be converted to Python

The code for formatting the error message assumed it still pointed to the selected overload,
but during the second-pass loop it was nullptr. Fix by setting it correctly if a second-pass
call returns a nullptr handle. Add a new test that segfaults without this fix.

oremanj added 2 commits March 20, 2018 13:59
The crash would occur if:
- dispatcher() uses two-pass logic (because the target is overloaded and some arguments support conversions)
- the first pass (with conversions disabled) doesn't find any matching overload
- the second pass does find a matching overload, but its return value can't be converted to Python

The code for formatting the error message assumed `it` still pointed to the selected overload,
but during the second-pass loop `it` was nullptr. Fix by setting `it` correctly if a second-pass
call returns a nullptr `handle`. Add a new test that segfaults without this fix.
Comment thread include/pybind11/pybind11.h Outdated
if (!result) {
// The error reporting logic below expects 'it' to be valid, as it would be
// if we'd encountered this failure in the first-pass loop.
for (it = overloads; it != &call.func; it = it->next)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ISTM this could be simplified down to just:

if (!result)
    it = &call.func;

(unless I'm forgetting something about this code).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

D'oh, right you are - I was getting confused about function_call vs function_record.

Comment thread tests/test_class.cpp Outdated
struct NotRegistered {};
struct StringWrapper { std::string str; };
NotRegistered test_error_after_conversions(StringWrapper) { return {}; }

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These classes should be moved with the module definitions, and the function rewritten as a lambda, unless there's a good reason to keep them outside. (We try to enforce that style in the test suite, where possible, to keep the test cases as localized as possible within each test suite file, to makes it easier to read).

@oremanj

oremanj commented Apr 17, 2018

Copy link
Copy Markdown
Collaborator Author

@jagerman Ping on this? The CI failure appears unrelated, e.g. just-uploaded #1362 has the same error.

@oremanj

oremanj commented Sep 25, 2018

Copy link
Copy Markdown
Collaborator Author

@wjakob @jagerman ping? I have a number of other PRs I'm sitting on which I have been holding off on contributing because this one has been stalled so long... for a while I was assuming there just wasn't development going on, but it looks like that's been more active lately.

@wjakob
wjakob merged commit e7761e3 into pybind:master Sep 25, 2018
@wjakob

wjakob commented Sep 25, 2018

Copy link
Copy Markdown
Member

Apologies for the delay -- this looks good now.

@oremanj
oremanj deleted the fix-unconvertible-return-after-conversions branch September 25, 2018 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants