[Messenger][Amqp] Add a prefetch_count option to consume messages instead of fetching them one by one - #65924

The AMQP transport asks the broker for one message at a time: Connection::get() calls AMQPQueue::get(), one basic.get per message, including when the worker wants a batch. RabbitMQ documents this as the least efficient way to consume, since every fetch is a round trip and an idle queue is polled on every loop. No consumer shows up in the management UI either, which is what #30259 reported.

This adds a prefetch_count option. Any value greater than zero registers a consumer per queue (basic.consume) and lets the broker push messages as capacity frees up:

amqp://localhost/%2f/messages?prefetch_count=20

5000 messages, ack each, against a loopback broker, which is the best case for basic.get since there is barely any round trip time to save:

get       5000 msgs in 3.050s =  1639 msg/s
consume   5000 msgs in 0.168s = 29803 msg/s   18.2x
get       5000 msgs in 2.918s =  1713 msg/s
consume   5000 msgs in 0.215s = 23302 msg/s   13.6x

The prefetch count is what buys the pipelining, so it has to be worth setting. With the default --fetch-size=1:

basic.get (today)         1565 msg/s
prefetch_count=1          1380 msg/s
prefetch_count=5          6832 msg/s
prefetch_count=10        11528 msg/s
prefetch_count=50        32884 msg/s

What changes when it is on

• Ordering across queues. getFromQueues() round-robins the queues itself today. One consume() call serves every consumer of the connection (the extension routes each message back to its queue by consumer tag), so the broker decides. Publishing L,L,L then H,H,H with the high priority queue registered first delivers L0 L1 L2 H0 H1 H2.
• read_timeout becomes bounded. It defaults to 0, which means block forever, so consuming defaults it to 1 second. That is also what makes an idle queue return, and since the worker computes its sleep from the start of the iteration, the read timeout takes the place of that sleep instead of adding to it.
• Stopping requeues more. Prefetched but unhandled messages are unacked, so the broker redelivers them. Nothing is lost, but a large prefetch_count means a larger redelivery burst per stop.
• keepalive() used to send qos(0, 0) for the traffic, which would drop the prefetch limit. It now resends the current one.

Batching

The consume callback cannot yield, so the receiver collects into an array and yields afterwards, which fits --fetch-size. It waits for the first message with the connection read timeout, then fills the rest of the batch with a 1 ms timeout: a partial batch returns at once rather than being held for the whole read timeout, the way basic.get returns as soon as a queue runs dry.

first get(5):  [0,1,2,3,4] in 0.007s
second get(5): [5,6,7,8,9] in 0.000s   (already pushed, no round trip)
third get(5):  [10,11]     in 0.000s   (partial batch, not held)
fourth get(5): []          in 1.000s   (idle, the read timeout)

The 1 ms timeout is what the extension gives us today to say "take what is buffered and come back". php-amqp/php-amqp#627 adds an AMQP_NB_CONSUME flag that does exactly that without a timeout and without throwing, which would turn that second phase into AMQP_JUST_CONSUME | AMQP_NB_CONSUME. It is open since June; I built it and it works, but the bridge cannot depend on an unreleased extension, so this is a follow-up rather than a blocker.

Also fixes the pull path

The same loss reaches basic.get: a signal that arrives while the extension waits for the broker is dropped when that wait ends by throwing, which a read timeout on a stalled connection does. Since 8.2 the receiver implements KeepaliveReceiverInterface and the keepalive alarm is rescheduled by its own handler, so one lost SIGALRM stops the keepalive for the rest of the process's life and the messages in flight are redelivered once their TTL expires. The dance moved to a private helper used by both paths, and the added test fails without it.

About signals

AMQPQueue::consume() reports the read timeout by throwing, and until php/php-src#23624 the engine drops every signal that was queued while an internal function was running when that function returns with an exception pending. A worker spends essentially all of its wall time inside that call, so it would miss every SIGTERM: 653 iterations of a 10 ms read timeout and the signal never arrives, not even through an explicit pcntl_signal_dispatch(). Hence the pcntl_async_signals() dance around the blocking call, which dispatches them by hand. Measured stop latency with it, on an unpatched PHP:

read_timeout=0.20 [email protected]: stopped, latency 0.094s
read_timeout=0.20 [email protected]: stopped, latency 0.083s
read_timeout=1.00 [email protected]: stopped, latency 0.601s

Stop latency is bounded by read_timeout, which is the same contract get() has today. The workaround stays correct once the engine no longer drops signals, so it does not need a version guard.

Kept opt-in for now, since the ordering and redelivery changes are visible. Tested against RabbitMQ 3.12.1 with ext-amqp 2.2.0; the new integration test covers the round trip.