Fix to issue #2257 - Trivial change to make Queue<T>'s Enqueue / Dequeue twice faster #2515

omariom · 2015-07-24T23:33:26Z

The issue: https://github.com/dotnet/corefx/issues/2257

This PR replaces usage of reminder operator (which is fairly slow) in Queue's Enqueue / Dequeue / Contains methods with simple boundary check.

stephentoub · 2015-07-24T23:42:23Z

src/System.Collections/src/System/Collections/Generic/Queue.cs

-                    newcapacity = _array.Length + MinimumGrow;
-                }
-                SetCapacity(newcapacity);
+                Grow();


Why was this refactored into a separate method? Does it help with inlining?

No, it has nothing to do with inlining. It was just instinct :)

Ok. In that case please revert it; if it's beneficial to separate it out, that can be done separately.

omariom · 2015-07-25T13:15:03Z

@nguerrera @stephentoub

Ok, this is what I've found.
The original Enqueue does a million ops in not less than 10.5 ms.
The new one can do the same in 6.5 ms.
Using Increment(ref index) though hasn't impacted perf but increased number of instructions
from

lea         eax,[rdx+1]  
mov         dword ptr [rsi+1Ch],eax
cmp         r8d,eax  
jne         00007FF8F4E708E8  
xor         eax,eax  
mov         dword ptr [rsi+1Ch],eax

to

lea         rax,[rsi+1Ch]  
mov         edx,dword ptr [rax]  
inc         edx  
mov         dword ptr [rax],edx  
mov         rcx,qword ptr [rsi+8]
cmp         dword ptr [rcx+8],edx
jne         00007FF8F4E80920  
xor         edx,edx  
mov         dword ptr [rax],edx

That's probably because JIT is not yet perfect at inlining methods with ref params pointing to fields.

But with @sharwell's suggestion, looking like:

int tail = _tail;
_array[tail] = item;
Increment(ref tail);
_tail = tail;

JIT does a perfect job of inlining the call to Increment:

inc         eax  
cmp         ecx,eax  
jne         00007FF8F4E7079D  
xor         eax,eax  
mov         dword ptr [rsi+1Ch],eax

With the changes enqueuing a million ints takes about 5.5 ms.

The results for Dequeue: 9.3 ms vs 3 ms.

stephentoub · 2015-07-25T13:23:44Z

Thanks, @omariom. Sounds like adopting both Nick's and Sam's suggestions is the way to go.

sharwell · 2015-07-25T15:15:36Z

@omariom: Keep in mind the same thing (local variable to ensure single-update to the field) can be applied to the location where _tail is updated. I didn't mention it before because the intermediate update was not exposed to callers by a pure method like Peek.

omariom · 2015-07-26T17:49:09Z

I've made the discussed changes: https://github.com/omariom/corefx/commit/8c22635ac1e38433244b8e7351c7d653906a4db6

What to do now with Grow method? Should I create a separate issue or just a PR will be enough?

nguerrera · 2015-07-28T15:39:35Z

I have to say I'm disappointed that the code has gotten repetitive. I'd really like to capture everything in the increment helper if at all possible.

cc @CarolEidt to see if there's something else we can use to get around the sub-optimal inline...

omariom · 2015-07-31T21:13:04Z

I can turn it back to usage of the fields (without copies). It was good enough already.
It will get better with JIT getting better.

stephentoub · 2015-08-09T18:48:27Z

Thanks, @omariom. I'd suggest not worrying about the increased number of instructions for now; as you say, that's something that'll just get better as the backend improves. We'd have the helper like:

private void MoveNext(ref int value)
{
    int tmp = value + 1;
    value = (tmp == _array.Length) ? 0 : tmp;
}

which would be used at call sites like:

_array[_tail] = item;
MoveNext(ref _tail); // instead of _tail = (_tail + 1) % _array.Length;
_size++;
_version++;

and that should provide the bulk of the wins while keeping the call sites simple.

omariom · 2015-08-16T11:51:12Z

@stephentoub
Ready!
And perf is good.

stephentoub · 2015-08-16T12:07:51Z

Ready!

Thanks, but what about the local temp in the helper?

Can you also please squash this down to a single commit?

omariom · 2015-08-16T15:19:07Z

Fixed and squashed 👌

stephentoub · 2015-08-16T15:41:10Z

src/System.Collections/src/System/Collections/Generic/Queue.cs

+        // Increments the index wrapping it if necessary.
+        private void MoveNext(ref int index)
+        {
+            // It is tempting to use the reminder operator here but it is actually much slower 


Nit: remainder, not reminder

stephentoub · 2015-08-16T15:41:30Z

Thanks! Perf still looks good?

omariom · 2015-08-16T15:45:16Z

Dequeue is about 3.5 ms, Enqueue 4.5-5 ms. Even better than twice.

stephentoub · 2015-08-16T15:52:15Z

LGTM. Thanks!

Fix to issue #2257 - Trivial change to make Queue<T>'s Enqueue / Dequeue twice faster

paulomorgado · 2017-07-02T17:49:34Z

src/System.Collections/src/System/Collections/Generic/Queue.cs

+            int tmp = index + 1;
+            index = (tmp == _array.Length) ? 0 : tmp;
+        }
+


Wouldn't this be better?

private void MoveNext(ref int index) { index++; if (index == _array.Length) { index = 0; } }

Wouldn't this be better?

Have you tried it? I would expect "no", since every read/write on index needs to go through that ref (hence the use of the tmp here), but if you find otherwise and have data to back it up, PRs are welcome. 😄 (It's possible subsequent JIT improvements in the last year have helped, too.)

I was just wondering. But the ref access does make a difference.

Would the C# compiler automatically inline the MoveNext method in the release build? If so, then ref won't be slower than a local variable. Just wondering...

Fix to issue dotnet/corefx#2257 - Trivial change to make Queue<T>'s Enqueue / Dequeue twice faster Commit migrated from dotnet/corefx@51f757c

dnfclas added the cla-not-required label Jul 24, 2015

stephentoub reviewed Jul 24, 2015
View reviewed changes

stephentoub reviewed Aug 16, 2015
View reviewed changes

Replace slow remainder operator with comparison to the array's length.

6ec9384

stephentoub added a commit that referenced this pull request Aug 16, 2015

Merge pull request #2515 from omariom/2257_faster_queues

51f757c

Fix to issue #2257 - Trivial change to make Queue<T>'s Enqueue / Dequeue twice faster

stephentoub merged commit 51f757c into dotnet:master Aug 16, 2015

omariom deleted the 2257_faster_queues branch August 16, 2015 19:28

stephentoub added the netfx-port-consider label Apr 13, 2016

jamesqo mentioned this pull request Aug 24, 2016

Remove use of the modulo operator in Queue.Enumerator #11101

Merged

karelz modified the milestone: 1.0.0-rtm Dec 3, 2016

paulomorgado reviewed Jul 2, 2017

View reviewed changes

jamesqo mentioned this pull request Jan 31, 2020

Queue<T>.Enumerator still uses the modulo operator during MoveNext dotnet/runtime#18152

Closed

Fix to issue #2257 - Trivial change to make Queue<T>'s Enqueue / Dequeue twice faster #2515

Fix to issue #2257 - Trivial change to make Queue<T>'s Enqueue / Dequeue twice faster #2515

Uh oh!

Conversation

omariom commented Jul 24, 2015

Uh oh!

stephentoub Jul 24, 2015

Choose a reason for hiding this comment

Uh oh!

omariom Jul 25, 2015

Choose a reason for hiding this comment

Uh oh!

stephentoub Jul 25, 2015

Choose a reason for hiding this comment

Uh oh!

omariom commented Jul 25, 2015

Uh oh!

stephentoub commented Jul 25, 2015

Uh oh!

sharwell commented Jul 25, 2015

Uh oh!

omariom commented Jul 26, 2015

Uh oh!

nguerrera commented Jul 28, 2015

Uh oh!

omariom commented Jul 31, 2015

Uh oh!

stephentoub commented Aug 9, 2015

Uh oh!

omariom commented Aug 16, 2015

Uh oh!

stephentoub commented Aug 16, 2015

Uh oh!

omariom commented Aug 16, 2015

Uh oh!

stephentoub Aug 16, 2015

Choose a reason for hiding this comment

Uh oh!

stephentoub commented Aug 16, 2015

Uh oh!

omariom commented Aug 16, 2015

Uh oh!

stephentoub commented Aug 16, 2015

Uh oh!

paulomorgado Jul 2, 2017

Choose a reason for hiding this comment

Uh oh!

stephentoub Jul 2, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paulomorgado Jul 2, 2017

Choose a reason for hiding this comment

Uh oh!

fandecheng Nov 15, 2018

Choose a reason for hiding this comment

Uh oh!

Uh oh!

stephentoub Jul 2, 2017 •

edited

Loading