Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[wip] Wean of tornado ioloop #68417

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
+235 −123
Open

[wip] Wean of tornado ioloop #68417

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e0a7de7
Wean of tornado ioloop
dwoz Oct 21, 2025
2ab0e14
Multiple test fixes
dwoz Oct 31, 2025
d1878d9
Bump relenv
dwoz Oct 31, 2025
a0e5f7e
more test fixes
dwoz Nov 1, 2025
716d4c6
Add _scheudle method to event
dwoz Nov 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -439,7 +439,7 @@ jobs:
with:
cache-seed: ${{ needs.prepare-workflow.outputs.cache-seed }}
salt-version: "${{ needs.prepare-workflow.outputs.salt-version }}"
relenv-version: "0.21.1"
relenv-version: "0.21.2"
python-version: "3.11.14"
ci-python-version: "3.11"
matrix: ${{ toJSON(fromJSON(needs.prepare-workflow.outputs.config)['build-matrix']) }}
Expand All @@ -456,7 +456,7 @@ jobs:
with:
salt-version: "${{ needs.prepare-workflow.outputs.salt-version }}"
cache-prefix: ${{ needs.prepare-workflow.outputs.cache-seed }}
relenv-version: "0.21.1"
relenv-version: "0.21.2"
python-version: "3.11.14"
ci-python-version: "3.11"
source: "onedir"
Expand All @@ -473,7 +473,7 @@ jobs:
with:
salt-version: "${{ needs.prepare-workflow.outputs.salt-version }}"
cache-prefix: ${{ needs.prepare-workflow.outputs.cache-seed }}
relenv-version: "0.21.1"
relenv-version: "0.21.2"
python-version: "3.11.14"
ci-python-version: "3.11"
source: "src"
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/nightly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -434,7 +434,7 @@ jobs:
with:
cache-seed: ${{ needs.prepare-workflow.outputs.cache-seed }}
salt-version: "${{ needs.prepare-workflow.outputs.salt-version }}"
relenv-version: "0.21.1"
relenv-version: "0.21.2"
python-version: "3.11.14"
ci-python-version: "3.11"
matrix: ${{ toJSON(fromJSON(needs.prepare-workflow.outputs.config)['build-matrix']) }}
Expand All @@ -451,7 +451,7 @@ jobs:
with:
salt-version: "${{ needs.prepare-workflow.outputs.salt-version }}"
cache-prefix: ${{ needs.prepare-workflow.outputs.cache-seed }}
relenv-version: "0.21.1"
relenv-version: "0.21.2"
python-version: "3.11.14"
ci-python-version: "3.11"
source: "onedir"
Expand All @@ -472,7 +472,7 @@ jobs:
with:
salt-version: "${{ needs.prepare-workflow.outputs.salt-version }}"
cache-prefix: ${{ needs.prepare-workflow.outputs.cache-seed }}
relenv-version: "0.21.1"
relenv-version: "0.21.2"
python-version: "3.11.14"
ci-python-version: "3.11"
source: "src"
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/scheduled.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -482,7 +482,7 @@ jobs:
with:
cache-seed: ${{ needs.prepare-workflow.outputs.cache-seed }}
salt-version: "${{ needs.prepare-workflow.outputs.salt-version }}"
relenv-version: "0.21.1"
relenv-version: "0.21.2"
python-version: "3.11.14"
ci-python-version: "3.11"
matrix: ${{ toJSON(fromJSON(needs.prepare-workflow.outputs.config)['build-matrix']) }}
Expand All @@ -499,7 +499,7 @@ jobs:
with:
salt-version: "${{ needs.prepare-workflow.outputs.salt-version }}"
cache-prefix: ${{ needs.prepare-workflow.outputs.cache-seed }}
relenv-version: "0.21.1"
relenv-version: "0.21.2"
python-version: "3.11.14"
ci-python-version: "3.11"
source: "onedir"
Expand All @@ -516,7 +516,7 @@ jobs:
with:
salt-version: "${{ needs.prepare-workflow.outputs.salt-version }}"
cache-prefix: ${{ needs.prepare-workflow.outputs.cache-seed }}
relenv-version: "0.21.1"
relenv-version: "0.21.2"
python-version: "3.11.14"
ci-python-version: "3.11"
source: "src"
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/staging.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -466,7 +466,7 @@ jobs:
with:
cache-seed: ${{ needs.prepare-workflow.outputs.cache-seed }}
salt-version: "${{ needs.prepare-workflow.outputs.salt-version }}"
relenv-version: "0.21.1"
relenv-version: "0.21.2"
python-version: "3.11.14"
ci-python-version: "3.11"
matrix: ${{ toJSON(fromJSON(needs.prepare-workflow.outputs.config)['build-matrix']) }}
Expand All @@ -484,7 +484,7 @@ jobs:
with:
salt-version: "${{ needs.prepare-workflow.outputs.salt-version }}"
cache-prefix: ${{ needs.prepare-workflow.outputs.cache-seed }}
relenv-version: "0.21.1"
relenv-version: "0.21.2"
python-version: "3.11.14"
ci-python-version: "3.11"
source: "onedir"
Expand All @@ -506,7 +506,7 @@ jobs:
with:
salt-version: "${{ needs.prepare-workflow.outputs.salt-version }}"
cache-prefix: ${{ needs.prepare-workflow.outputs.cache-seed }}
relenv-version: "0.21.1"
relenv-version: "0.21.2"
python-version: "3.11.14"
ci-python-version: "3.11"
source: "src"
Expand Down
2 changes: 1 addition & 1 deletion cicd/shared-gh-workflows-context.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
nox_version: "2022.8.7"
python_version: "3.11.14"
relenv_version: "0.21.1"
relenv_version: "0.21.2"
release_branches:
- "3006.x"
- "3007.x"
Expand Down
6 changes: 3 additions & 3 deletions salt/channel/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
import salt.utils.stringutils
import salt.utils.verify
import salt.utils.versions
from salt.utils.asynchronous import SyncWrapper
from salt.utils.asynchronous import SyncWrapper, aioloop

log = logging.getLogger(__name__)

Expand Down Expand Up @@ -422,7 +422,7 @@ def factory(cls, opts, **kwargs):

def __init__(self, opts, transport, auth, io_loop=None):
self.opts = opts
self.io_loop = io_loop
self.io_loop = aioloop(io_loop)
self.auth = auth
try:
# This loads or generates the minion's public key.
Expand Down Expand Up @@ -636,7 +636,7 @@ def __enter__(self):
return self

def __exit__(self, *args):
self.io_loop.spawn_callback(self.close)
self.io_loop.call_soon(self.close)

async def __aenter__(self):
return self
Expand Down
71 changes: 50 additions & 21 deletions salt/transport/tcp.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import asyncio
import asyncio.exceptions
import errno
import inspect
import logging
import multiprocessing
import queue
Expand Down Expand Up @@ -183,12 +184,24 @@ def run(self):
# Wait for a free slot to be available to put
# the connection into.
# Sockets are picklable on Windows in Python 3.
self.socket_queue.put((connection, address), True, None)
try:
self.socket_queue.put((connection, address), True, None)
except Exception: # pylint: disable=broad-except
log.exception(
"Failed to enqueue connection from %s into load balancer queue",
address,
)
connection.close()
except OSError as e:
# ECONNABORTED indicates that there was a connection
# but it was closed while still in the accept queue.
# (observed on FreeBSD).
name = self._socket.getsockname()
name = None
if self._socket is not None:
try:
name = self._socket.getsockname()
except OSError:
name = None
if isinstance(name, tuple):
name = name[0]
if tornado.util.errno_from_exception(e) == errno.ECONNABORTED:
Expand Down Expand Up @@ -229,7 +242,7 @@ class PublishClient(salt.transport.base.PublishClient):
def __init__(self, opts, io_loop, **kwargs): # pylint: disable=W0231
super().__init__(opts, io_loop, **kwargs)
self.opts = opts
self.io_loop = io_loop
self.io_loop = salt.utils.asynchronous.aioloop(io_loop)
self.unpacker = salt.utils.msgpack.Unpacker()
self.connected = False
self._closing = False
Expand Down Expand Up @@ -634,6 +647,7 @@ def __init__(self, message_handler, *args, **kwargs):
self._closing = False
super().__init__(*args, **kwargs)
self.io_loop = io_loop
self.asyncio_loop = salt.utils.asynchronous.aioloop(io_loop)
self.clients = []
self.message_handler = message_handler

Expand All @@ -656,9 +670,18 @@ async def handle_stream( # pylint: disable=arguments-differ,invalid-overridden-
for framed_msg in unpacker:
framed_msg = salt.transport.frame.decode_embedded_strs(framed_msg)
header = framed_msg["head"]
self.io_loop.spawn_callback(
self.message_handler, stream, framed_msg["body"], header
)
try:
log.trace("Dispatching message handler for %s", address)
result = self.message_handler(
stream, framed_msg["body"], header
)
except Exception: # pylint: disable=broad-except
log.exception(
"Unhandled exception while running message handler"
)
else:
if inspect.isawaitable(result):
self.asyncio_loop.create_task(result)
except _StreamClosedError:
log.trace("req client disconnected %s", address)
self.remove_client((stream, address))
Expand Down Expand Up @@ -703,7 +726,7 @@ def __init__(self, socket_queue, message_handler, *args, **kwargs):
super().__init__(message_handler, *args, **kwargs)
self.socket_queue = socket_queue
self._stop = threading.Event()
self.thread = threading.Thread(target=self.socket_queue_thread)
self.thread = threading.Thread(target=self.socket_queue_thread, daemon=True)
self.thread.start()

def close(self):
Expand All @@ -720,9 +743,10 @@ def socket_queue_thread(self):
if self._stop.is_set():
break
continue
# 'self.io_loop' initialized in super class
# 'salt.ext.tornado.tcpserver.TCPServer'.
# 'self._handle_connection' defined in same super class.
log.trace(
"LoadBalancerWorker received queued connection from %s", address
)
# Schedule handling the connection using the tornado IOLoop.
self.io_loop.spawn_callback(
self._handle_connection, client_socket, address
)
Expand Down Expand Up @@ -785,17 +809,19 @@ def __init__(
self.source_port = source_port
self.connect_callback = connect_callback
self.disconnect_callback = disconnect_callback
self.io_loop = io_loop or tornado.ioloop.IOLoop.current()
with salt.utils.asynchronous.current_ioloop(self.io_loop):
self._tcp_client = TCPClientKeepAlive(opts, resolver=resolver)
if io_loop is None:
io_loop = tornado.ioloop.IOLoop.current()
self.io_loop = io_loop
self.asyncio_loop = salt.utils.asynchronous.aioloop(io_loop)
self._tcp_client = TCPClientKeepAlive(opts, resolver=resolver)
# TODO: max queue size
self.send_future_map = {} # mapping of request_id -> Future

self._read_until_future = None
self._on_recv = None
self._closing = False
self._closed = False
self._connecting_future = tornado.concurrent.Future()
self._connecting_future = self.asyncio_loop.create_future()
self._stream_return_running = False
self._stream = None

Expand All @@ -806,9 +832,12 @@ def close(self):
if self._closing or self._closed:
return
self._closing = True
self.io_loop.add_timeout(1, self.check_close)
if not self.send_future_map:
self.io_loop.call_later(0, self.check_close)
else:
self.io_loop.call_later(1, self.check_close)

async def check_close(self):
def check_close(self):
if not self.send_future_map:
self._tcp_client.close()
if self._stream:
Expand All @@ -817,7 +846,7 @@ async def check_close(self):
self._closed = True
self._closing = False
else:
self.io_loop.add_timeout(1, self.check_close)
self.io_loop.call_later(1, self.check_close)

# pylint: disable=W1701
def __del__(self):
Expand Down Expand Up @@ -860,7 +889,7 @@ async def connect(self):
self._closing = False
self._closed = False
if not self._stream_return_running:
self.io_loop.spawn_callback(self._stream_return)
return_task = self.asyncio_loop.create_task(self._stream_return())
if self.connect_callback:
self.connect_callback(True)

Expand All @@ -882,7 +911,7 @@ async def _stream_return(self):
# self.remove_message_timeout(message_id)
else:
if self._on_recv is not None:
self.io_loop.spawn_callback(self._on_recv, header, body)
self.io_loop.call_soon(self._on_recv, header, body)
else:
log.error(
"Got response for message_id %s that we are not"
Expand Down Expand Up @@ -970,7 +999,7 @@ async def send(self, msg, timeout=None, callback=None, raw=False):
message_id = self._message_id()
header = {"mid": message_id}

future = tornado.concurrent.Future()
future = self.asyncio_loop.create_future()

if callback is not None:

Expand Down Expand Up @@ -998,7 +1027,7 @@ async def _do_send():

# Run send in a callback so we can wait on the future, in case we time
# out before we are able to connect.
self.io_loop.add_callback(_do_send)
send_task = self.asyncio_loop.create_task(_do_send())
return await future


Expand Down
8 changes: 6 additions & 2 deletions salt/transport/ws.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import salt.payload
import salt.transport.base
import salt.transport.frame
import salt.utils.asynchronous
from salt.transport.tcp import (
USE_LOAD_BALANCER,
LoadBalancerServer,
Expand Down Expand Up @@ -43,7 +44,10 @@ class PublishClient(salt.transport.base.PublishClient):

def __init__(self, opts, io_loop, **kwargs): # pylint: disable=W0231
self.opts = opts
if io_loop is None:
io_loop = tornado.ioloop.IOLoop.current()
self.io_loop = io_loop
self.asyncio_loop = salt.utils.asynchronous.aioloop(io_loop)

self.connected = False
self._closing = False
Expand Down Expand Up @@ -89,7 +93,7 @@ def close(self):
if self._closing:
return
self._closing = True
self.io_loop.spawn_callback(self._close)
self.asyncio_loop.create_task(self._close())

# pylint: disable=W1701
def __del__(self):
Expand Down Expand Up @@ -570,7 +574,7 @@ def __init__(self, opts, io_loop): # pylint: disable=W0231
self.sending = False
self.ws = None
self.session = None
self.io_loop = io_loop
self.io_loop = salt.utils.asynchronous.aioloop(io_loop)
self._closing = False
self._closed = False
self.ssl = self.opts.get("ssl", None)
Expand Down
Loading