Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[redis-messenger] remove undefined array key warnings #45619

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 4, 2022

Conversation

PhilETaylor
Copy link
Contributor

@PhilETaylor PhilETaylor commented Mar 2, 2022

Q A
Branch? 5.4
Bug fix? yes
New feature? no
Deprecations? no
Tickets Fix #45270
License MIT

check that we actually have some information back from redis before assuming its an array with 2 keys to avoid undefined array key warnings as per #45270


so the code in question is this:

public function get(): ?array
    {
        if ($this->autoSetup) {
            $this->setup();
        }
        $now = microtime();
        $now = substr($now, 11).substr($now, 2, 3);

        $queuedMessageCount = $this->rawCommand('ZCOUNT', 0, $now);

        while ($queuedMessageCount--) {
            if (![$queuedMessage, $expiry] = $this->rawCommand('ZPOPMIN', 1)) {
                break;
            }

            if (\strlen($expiry) === \strlen($now) ? $expiry > $now : \strlen($expiry) < \strlen($now)) {
                // if a future-placed message is popped because of a race condition with
                // another running consumer, the message is readded to the queue

                if (!$this->rawCommand('ZADD', 'NX', $expiry, $queuedMessage)) {
                    throw new TransportException('Could not add a message to the redis stream.');
                }

                break;
            }

            $decodedQueuedMessage = json_decode($queuedMessage, true);
            $this->add(\array_key_exists('body', $decodedQueuedMessage) ? $decodedQueuedMessage['body'] : $queuedMessage, $decodedQueuedMessage['headers'] ?? [], 0);
        }

I think what's happening, as I often can have up to 1000 workers (yup, for real, making 1000 http requests to remote sites - see mySites.guru) I think the logic might be flawed.

Because, the following line gets the number of messages using ZCOUNT

$queuedMessageCount = $this->rawCommand('ZCOUNT', 0, $now);

It will then enter a loop counting down that number of messages checking them

while ($queuedMessageCount--) {

At the same time, 999 other workers can be doing the same thing

it gets to a point where one of the workers doesnt get a message back from ZPOPMIN and thus we get undefined array keys...

You can see the result of ZPOPMIN is no longer an array once empty using the interactive test on the official redis page.

Screen Shot 2022-03-02 at 20 21 00

@carsonbot
Copy link

Hey!

Oh no, it looks like you have made this PR towards a branch that is not maintained anymore. :/
Could you update the PR base branch to target one of these branches instead? 4.4, 5.4, 6.0, 6.1.

Cheers!

Carsonbot

@PhilETaylor PhilETaylor changed the title remove undefined array key warnings [redis-messenger] remove undefined array key warnings Mar 2, 2022
@PhilETaylor PhilETaylor changed the base branch from 5.3 to 5.4 March 2, 2022 20:17
@PhilETaylor
Copy link
Contributor Author

Rebased to 5.4

@carsonbot
Copy link

Hey!

I think @Steveb-p has recently worked with this code. Maybe they can help review this?

Cheers!

Carsonbot

@nicolas-grekas nicolas-grekas added this to the 5.4 milestone Mar 4, 2022
@nicolas-grekas
Copy link
Member

Thank you @PhilETaylor.

@nicolas-grekas nicolas-grekas merged commit e091d74 into symfony:5.4 Mar 4, 2022
This was referenced Mar 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants