Thanks to visit codestin.com
Credit goes to github.com

Skip to content

gh-99741: Clean Up the _xxsubinterpreters Module #99940

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 18 commits into from
Dec 2, 2022
Merged

gh-99741: Clean Up the _xxsubinterpreters Module #99940

merged 18 commits into from
Dec 2, 2022

Conversation

ericsnowcurrently
Copy link
Member

@ericsnowcurrently ericsnowcurrently commented Dec 2, 2022 •
edited by bedevere-bot
Loading

This cleanup up resolves a few subtle bugs and makes the implementation for multi-phase init much cleaner.

@bedevere-bot bedevere-bot mentioned this pull request Dec 2, 2022
Closed
@ericsnowcurrently ericsnowcurrently added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Dec 2, 2022
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @ericsnowcurrently for commit f5e53ef 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Dec 2, 2022
@ericsnowcurrently
Use a C11 struct initializer.
3fa77fb
@ericsnowcurrently
Clean up PyInit__xxsubinterpreters().
f2f00aa
@ericsnowcurrently
Clean up channel_exceptions_init() and interp_exceptions_init().
bb84f38
@ericsnowcurrently
Add _globals_fini() and only do init/fini once.
45e1f25
@ericsnowcurrently
Factor out module_exec().
0efa456
@ericsnowcurrently
Fix add_new_exception().
bd841f2
@ericsnowcurrently
Do not rely on exception globals in low-level functions.
36e1239
@ericsnowcurrently
Import _xxsubinterpreters in a subinterpreter if necessary.
51645f7
@ericsnowcurrently
Ensure that the XID data gets released.
db57245
@ericsnowcurrently
Return an error code from _PyCrossInterpreterData_Release().
4f9baf0
@ericsnowcurrently
Clean up for ChannelIDType.
27d9b35
@ericsnowcurrently
Always decrement _globals.module_count.
c4aa646
@ericsnowcurrently
Decref the module when appropriate.
5cb73fa
@ericsnowcurrently
Add some asserts.
5af58e8
@ericsnowcurrently
Clear the registered type during module cleanup.
f555fbe
@ericsnowcurrently
Add a dedicated ChannelIDTests.test_shareable().
ad000ad
@ericsnowcurrently
Add a TODO comment.
60a78ab
@ericsnowcurrently
Update a TODO comment.
012d61c
@ericsnowcurrently ericsnowcurrently force-pushed the clean-up-_xxsubinterpreters branch from f5e53ef to 012d61c Compare December 2, 2022 17:42
@ericsnowcurrently ericsnowcurrently merged commit 0547a98 into python:main Dec 2, 2022
@ericsnowcurrently ericsnowcurrently deleted the clean-up-_xxsubinterpreters branch December 2, 2022 18:37
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot wasm32-emscripten node (pthreads) 3.x has failed when building commit 0547a98.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1050/builds/955) and take a look at the build logs.
  4. Check if the failure is related to this commit (0547a98) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1050/builds/955

Failed tests:

  • test_capi

Summary of the results of the build (if available):

== Tests result: FAILURE ==

338 tests OK.

10 slowest tests:

  • test_tokenize: 1 min 15 sec
  • test_unparse: 51.8 sec
  • test_io: 44.7 sec
  • test_lib2to3: 36.3 sec
  • test_zipfile: 26.2 sec
  • test_unicodedata: 25.8 sec
  • test_pickle: 23.5 sec
  • test_statistics: 19.8 sec
  • test_decimal: 19.5 sec
  • test_argparse: 18.7 sec

1 test failed:
test_capi

94 tests skipped:
test__xxsubinterpreters test_asyncgen test_asyncio
test_check_c_globals test_clinic test_cmd_line
test_concurrent_futures test_contextlib_async test_ctypes
test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_doctest
test_docxmlrpc test_dtrace test_embed test_epoll test_faulthandler
test_fcntl test_file_eintr test_fork1 test_ftplib test_gdb
test_grp test_httplib test_httpservers test_idle test_imaplib
test_interpreters test_ioctl test_kqueue test_launcher test_lzma
test_mmap test_msilib test_multiprocessing_fork
test_multiprocessing_forkserver test_multiprocessing_main_handling
test_multiprocessing_spawn test_nis test_openpty test_ossaudiodev
test_pdb test_peg_generator test_perf_profiler test_poll
test_poplib test_pty test_pwd test_readline test_regrtest
test_repl test_resource test_select test_selectors test_smtplib
test_smtpnet test_socket test_socketserver test_spwd test_ssl
test_stable_abi_ctypes test_startfile test_subprocess
test_sys_settrace test_syslog test_tcl test_telnetlib test_tix
test_tkinter test_tools test_ttk test_ttk_textonly test_turtle
test_urllib2 test_urllib2_localnet test_urllib2net test_urllibnet
test_venv test_wait3 test_wait4 test_webbrowser test_winconsoleio
test_winreg test_winsound test_wmi test_wsgiref test_xmlrpc
test_xmlrpc_net test_xxlimited test_zipfile64
test_zipimport_support test_zoneinfo
0:22:53 load avg: 7.49
0:22:53 load avg: 7.49 Re-running failed tests is not supported with --python host runner option.

Total duration: 22 min 53 sec

Click to see traceback logs 
remote: Enumerating objects: 23, done.        
remote: Counting objects:   4% (1/23)        
remote: Counting objects:   8% (2/23)        
remote: Counting objects:  13% (3/23)        
remote: Counting objects:  17% (4/23)        
remote: Counting objects:  21% (5/23)        
remote: Counting objects:  26% (6/23)        
remote: Counting objects:  30% (7/23)        
remote: Counting objects:  34% (8/23)        
remote: Counting objects:  39% (9/23)        
remote: Counting objects:  43% (10/23)        
remote: Counting objects:  47% (11/23)        
remote: Counting objects:  52% (12/23)        
remote: Counting objects:  56% (13/23)        
remote: Counting objects:  60% (14/23)        
remote: Counting objects:  65% (15/23)        
remote: Counting objects:  69% (16/23)        
remote: Counting objects:  73% (17/23)        
remote: Counting objects:  78% (18/23)        
remote: Counting objects:  82% (19/23)        
remote: Counting objects:  86% (20/23)        
remote: Counting objects:  91% (21/23)        
remote: Counting objects:  95% (22/23)        
remote: Counting objects: 100% (23/23)        
remote: Counting objects: 100% (23/23), done.        
remote: Compressing objects:  16% (1/6)        
remote: Compressing objects:  33% (2/6)        
remote: Compressing objects:  50% (3/6)        
remote: Compressing objects:  66% (4/6)        
remote: Compressing objects:  83% (5/6)        
remote: Compressing objects: 100% (6/6)        
remote: Compressing objects: 100% (6/6), done.        
remote: Total 12 (delta 11), reused 6 (delta 6), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to '0547a981ae413248b21a6bb0cb62dda7d236fe45'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 0547a981ae gh-99741: Clean Up the _xxsubinterpreters Module (gh-99940)
Switched to and reset branch 'main'

../../Objects/obmalloc.c:776:1: warning: always_inline function might not be inlinable [-Wattributes]
  776 | arena_map_get(pymem_block *p, int create)
      | ^~~~~~~~~~~~~

configure: ../../configure --prefix $(PWD)/target/host --with-pydebug --without-pydebug --with-emscripten-target=node --disable-wasm-dynamic-linking --enable-wasm-pthreads --build=x86_64-pc-linux-gnu --host=wasm32-unknown-emscripten --with-build-python=../build/python
configure: WARNING: using cross tools not prefixed with host triplet
mcc: error: no input files

make: make -j2 all
../../Python/initconfig.c:2347:27: warning: format specifies type 'wint_t' (aka 'int') but the argument has type 'wint_t' (aka 'unsigned int') [-Wformat]
    printf(usage_envvars, (wint_t)DELIM, (wint_t)DELIM, PYTHONHOMEHELP);
           ~~~~~~~~~~~~~  ^~~~~~~~~~~~~
../../Python/initconfig.c:138:18: note: format string is defined here
"PYTHONPATH   : '%lc'-separated list of directories prefixed to the\n"
                 ^~~
                 %c
../../Python/initconfig.c:2347:42: warning: format specifies type 'wint_t' (aka 'int') but the argument has type 'wint_t' (aka 'unsigned int') [-Wformat]
    printf(usage_envvars, (wint_t)DELIM, (wint_t)DELIM, PYTHONHOMEHELP);
           ~~~~~~~~~~~~~                 ^~~~~~~~~~~~~
../../Python/initconfig.c:141:58: note: format string is defined here
"PYTHONHOME   : alternate <prefix> directory (or <prefix>%lc<exec_prefix>).\n"
                                                         ^~~
                                                         %c
2 warnings generated.
../../Python/pytime.c:297:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
    if (!_Py_InIntegralTypeRange(time_t, intpart)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:72:45: note: expanded from macro '_Py_InIntegralTypeRange'
    (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                         ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:61:88: note: expanded from macro '_Py_IntegralTypeMax'
    (_Py_IS_TYPE_SIGNED(type) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
../../Python/pytime.c:352:14: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
        if (!_Py_InIntegralTypeRange(time_t, intpart)) {
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:72:45: note: expanded from macro '_Py_InIntegralTypeRange'
    (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                         ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:61:88: note: expanded from macro '_Py_IntegralTypeMax'
    (_Py_IS_TYPE_SIGNED(type) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
../../Python/pytime.c:518:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
    if (!_Py_InIntegralTypeRange(_PyTime_t, d)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:72:45: note: expanded from macro '_Py_InIntegralTypeRange'
    (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                         ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:61:88: note: expanded from macro '_Py_IntegralTypeMax'
    (_Py_IS_TYPE_SIGNED(type) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
3 warnings generated.
../../Modules/expat/xmlparse.c:3116:9: warning: code will never be executed [-Wunreachable-code]
        parser->m_characterDataHandler(parser->m_handlerArg, parser->m_dataBuf,
        ^~~~~~
../../Modules/expat/xmlparse.c:3115:16: note: silence by adding parentheses to mark code as explicitly dead
      else if (0 && parser->m_characterDataHandler)
               ^
               /* DISABLES CODE */ ( )
../../Modules/expat/xmlparse.c:4059:9: warning: code will never be executed [-Wunreachable-code]
        parser->m_characterDataHandler(parser->m_handlerArg, parser->m_dataBuf,
        ^~~~~~
../../Modules/expat/xmlparse.c:4058:16: note: silence by adding parentheses to mark code as explicitly dead
      else if (0 && parser->m_characterDataHandler)
               ^
               /* DISABLES CODE */ ( )
../../Modules/expat/xmlparse.c:7703:11: warning: format specifies type 'int' but the argument has type 'ptrdiff_t' (aka 'long') [-Wformat]
          bytesMore, (account == XML_ACCOUNT_DIRECT) ? "DIR" : "EXP",
          ^~~~~~~~~
3 warnings generated.
../../Modules/socketmodule.c:4031:33: warning: comparison of integers of different signs: 'unsigned long' and 'long' [-Wsign-compare]
         cmsgh != NULL; cmsgh = CMSG_NXTHDR(&msg, cmsgh)) {
                                ^~~~~~~~~~~~~~~~~~~~~~~~
/opt/buildbot/.emscripten_cache/sysroot/include/sys/socket.h:356:44: note: expanded from macro 'CMSG_NXTHDR'
        __CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Modules/socketmodule.c:4084:33: warning: comparison of integers of different signs: 'unsigned long' and 'long' [-Wsign-compare]
         cmsgh != NULL; cmsgh = CMSG_NXTHDR(&msg, cmsgh)) {
                                ^~~~~~~~~~~~~~~~~~~~~~~~
/opt/buildbot/.emscripten_cache/sysroot/include/sys/socket.h:356:44: note: expanded from macro 'CMSG_NXTHDR'
        __CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Modules/socketmodule.c:4708:54: warning: comparison of integers of different signs: 'unsigned long' and 'long' [-Wsign-compare]
            cmsgh = (i == 0) ? CMSG_FIRSTHDR(&msg) : CMSG_NXTHDR(&msg, cmsgh);
                                                     ^~~~~~~~~~~~~~~~~~~~~~~~
/opt/buildbot/.emscripten_cache/sysroot/include/sys/socket.h:356:44: note: expanded from macro 'CMSG_NXTHDR'
        __CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 warnings generated.
../../Modules/_sqlite/connection.c:2198:19: warning: result of comparison of constant 9223372036854775807 with expression of type 'Py_ssize_t' (aka 'long') is always false [-Wtautological-constant-out-of-range-compare]
    if (data->len > 9223372036854775807) {  // (1 << 63) - 1
        ~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~
1 warning generated.
mcc: warning: USE_PTHREADS + ALLOW_MEMORY_GROWTH may run non-wasm code slowly, see https://github.com/WebAssembly/design/issues/1271 [-Wpthreads-mem-growth]

make: *** [Makefile:1860: buildbottest] Error 2

Cannot open file '/opt/buildbot/bcannon-wasm/3.x.bcannon-wasm.emscripten-node-pthreads/build/build/build_oot/host/test-results.xml' for upload

@sobolevn sobolevn mentioned this pull request Dec 4, 2022
Closed
ramvikrams added a commit to ramvikrams/cpython that referenced this pull request Dec 5, 2022
@ramvikrams
Revert "pythongh-99741: Clean Up the _xxsubinterpreters Module (pytho…
79983f0 
…ngh-99940)"

This reverts commit 0547a98.
@ericsnowcurrently ericsnowcurrently mentioned this pull request Dec 5, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
skip news
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ericsnowcurrently @bedevere-bot