Two Socket/NetworkStream-related optimizations #12664

stephentoub · 2016-10-14T20:41:06Z

This PR provides two independent but related optimizations, one for NetworkStream and one for Socket. The latter was done as it showed up as a significant source of allocations once the former was done.

Add NetworkStream.CopyToAsync override. This does several custom things. First, it uses ArrayPool for the buffer used in the copy operation. Second, it uses a SocketAsyncEventArgs to avoid per-operation costs related to the socket. And third, it then uses a custom awaitable to avoid async/Task-related overheads that would otherwise occur per-ReadAsync.
Removes a per-operation SafeHandle allocation from any operation on a SocketAsyncEventArgs.

I wrote a little benchmark that connects a socket to a localhost server which serves up 10MB of data. It then creates a NetworkStream and does a CopyToAsync on it to copy all of the data to Stream.Null. And it does all of this 10 times. Prior to the changes, there are lots of allocations (I'm only showing line items with > 10K of impact):

After adding CopyToAsync, most of the allocations go away, but we end up with a significant number of SafeNativeOverlapped SafeHandles:

After the second fix to address the SafeHandles, our memory usage is much more reasonable:

(Most of what remains is unrelated to the actual operation being tested, and is coming from things elsewhere in the test app, e.g. strings created at startup.)

cc: @ericeil, @davidsh, @CIPop, @davidfowl, @benaadams
Fixes #11573
Fixes #12659

benaadams · 2016-10-14T20:51:44Z

Very nice 💯

davidfowl · 2016-10-14T21:04:31Z

This is great! Even better would be a way to pool the AwaitableSocketAsyncEventArgs /SocketAsyncEventArgs for reuse but that's an unrelated optimization.

ericeil

Very nice!

stephentoub · 2016-10-14T22:31:21Z

@dotnet-bot test this please (Jenkins rebooted)

CIPop · 2016-10-15T00:08:40Z

Thanks @stephentoub. Please validate this in both netcore as well as UWP profiles using the performance tests to ensure no regressions under stress using either of the ThreadPool implementations.

stephentoub · 2016-10-15T00:48:46Z

Please validate this in both netcore as well as UWP profiles using the performance tests to ensure no regressions under stress using either of the ThreadPool implementations.

Where are instructions for doing this?

CIPop · 2016-10-15T01:14:12Z

Please manually run the outerloop https://github.com/dotnet/corefx/tree/master/src/System.Net.Sockets/tests/PerformanceTests tests, changing the counters to something that takes a couple of minutes.

UWP: I'm asking only because I remember some interesting behavior differences for netcore50 as well as netnative modes. @ericeil can talk more about the differences between the two platforms in terms of IOCP/Overlapped/ThreadPool.
To test UWP I was using the internal ToF tests. If this isn't possible anymore, please open a tracking issue to run extended perf/stress tests for UAP whenever testing for this platform is enabled in the GitHub branch.

CIPop

A few nitpicks / questions. Thanks @stephentoub!

CIPop · 2016-10-15T00:12:52Z

src/Common/src/Interop/Windows/Winsock/SafeNativeOverlapped.cs

                unsafe
                {
-                    Debug.Assert(_safeCloseSocket != null, "m_SafeCloseSocket is null.");
+                    Debug.Assert(SocketHandle != null, "_safeCloseSocket is null.");


Nit: SocketHandle is null.

CIPop · 2016-10-15T00:46:23Z

src/System.Net.Sockets/src/System/Net/Sockets/SocketAsyncEventArgs.Windows.cs

            Debug.Assert(overlapped != null, "NativeOverlapped is null.");
-            _ptrNativeOverlapped = new SafeNativeOverlapped(_currentSocket.SafeHandle, overlapped);
+
+            if (_ptrNativeOverlapped != null && _ptrNativeOverlapped.SocketHandle == _currentSocket.SafeHandle)


Is there a case when _ptrNativeOverlapped.SocketHandle == _currentSocket.SafeHandle is not true?

Is there a case when _ptrNativeOverlapped.SocketHandle == _currentSocket.SafeHandle is not true?

If the same SocketAsyncEventArgs is used with one socket and then with another. I noticed we didn't have any tests for this, so I've added one as well.

stephentoub · 2016-10-15T03:33:44Z

@dotnet-bot Test OuterLoop Windows_NT Debug please (https://github.com/dotnet/corefx/issues/11737)

stephentoub · 2016-10-15T14:27:36Z

Please manually run the outerloop

Thanks. I ran the perf tests locally and didn't see any regressions.

stephentoub · 2016-10-15T14:28:10Z

@dotnet-bot Test Outerloop Windows_NT Debug please
@dotnet-bot Test Outerloop OSX Debug please

This commit overrides CopyToAsync on NetworkStream to provide an optimized implementation. Several optimizations: - Use ArrayPool for a pooled copy buffer rather than allocating a new one for each CopyToAsync operation - Uses a SocketAsyncEventArgs to avoid per-socket operation costs like pinning of the buffer - Uses a custom awaitable to avoid per-ReadAsync costs like the allocated Tasks and IAsyncResult objects involved

…ation Each operation ends up allocating a SafeNativeOverlapped, which results in a ton of allocations when trying to optimize code via SocketAsyncEventArgs. This commit reuses the same SafeHandle object for many / all of the operations on the event args instance.

jamesqo · 2016-10-15T21:37:52Z

src/System.Net.Sockets/src/System/Net/Sockets/NetworkStream.cs

+                {
+                    cancellationToken.ThrowIfCancellationRequested();
+
+                    int bytesRead = await ea.ReceiveAsync();


Should a ConfigureAwait(false) have been added here, or was it intentionally omitted? I noticed the second await has it, but this one does not.

This isn't a Task. The behavior of this awaitable matches that of ConfigureAwait(false) implicitly, in that it always ignores the current context.

scionwest · 2016-11-26T19:30:50Z

I just ran into this issue over yesterday! I was seeing a GC collection happen once per-second. Glad to see it has already been fixed, awesome stuff!

scionwest · 2016-11-27T20:03:41Z

This appears to only be an issue with netcoreapp projects. Using SocketAsyncEventArgs on the full framework, 4.6, does not have this issue. I was able to create the exact same app in both frameworks and see it allocate like crazy in netcoreapp, and run nice and smooth in 4.6. Interesting, looking forward to getting this into my netcoreapp project for sure.

…nnot attach. PerfView shows that the issue is solely dotnet/corefx#12664, so should target netstandard2.0.

…_copytoasync Two Socket/NetworkStream-related optimizations Commit migrated from dotnet/corefx@c9b1b94

dnfclas added the cla-already-signed label Oct 14, 2016

davidsh added the area-System.Net label Oct 14, 2016

ericeil approved these changes Oct 14, 2016

View reviewed changes

CIPop approved these changes Oct 15, 2016

View reviewed changes

stephentoub force-pushed the networkstream_copytoasync branch 2 times, most recently from dd54b66 to d092bcb Compare October 15, 2016 14:25

stephentoub added 2 commits October 15, 2016 12:31

stephentoub force-pushed the networkstream_copytoasync branch from d092bcb to 81dbf51 Compare October 15, 2016 16:32

karelz assigned stephentoub Oct 15, 2016

stephentoub merged commit c9b1b94 into dotnet:master Oct 15, 2016

jamesqo reviewed Oct 15, 2016

View reviewed changes

stephentoub deleted the networkstream_copytoasync branch October 15, 2016 22:09

stephentoub mentioned this pull request Oct 31, 2016

Optimize DeflateStream.CopyToAsync #13184

Merged

karelz modified the milestone: 1.2.0 Dec 3, 2016

buybackoff added a commit to Spreads/Spreads.IPC that referenced this pull request Jul 13, 2017

Run UDP benchmark on .NET Core. A lot of GC, slower, and dotMemory ca…

5c48f21

…nnot attach. PerfView shows that the issue is solely dotnet/corefx#12664, so should target netstandard2.0.

picenka21 pushed a commit to picenka21/runtime that referenced this pull request Feb 18, 2022

Merge pull request dotnet/corefx#12664 from stephentoub/networkstream…

2f90b56

…_copytoasync Two Socket/NetworkStream-related optimizations Commit migrated from dotnet/corefx@c9b1b94

Two Socket/NetworkStream-related optimizations #12664

Two Socket/NetworkStream-related optimizations #12664

Uh oh!

Conversation

stephentoub commented Oct 14, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

benaadams commented Oct 14, 2016

Uh oh!

davidfowl commented Oct 14, 2016

Uh oh!

ericeil left a comment

Choose a reason for hiding this comment

Uh oh!

stephentoub commented Oct 14, 2016

Uh oh!

CIPop commented Oct 15, 2016

Uh oh!

stephentoub commented Oct 15, 2016

Uh oh!

CIPop commented Oct 15, 2016

Uh oh!

CIPop left a comment

Choose a reason for hiding this comment

Uh oh!

CIPop Oct 15, 2016

Choose a reason for hiding this comment

Uh oh!

stephentoub Oct 15, 2016

Choose a reason for hiding this comment

Uh oh!

CIPop Oct 15, 2016

Choose a reason for hiding this comment

Uh oh!

stephentoub Oct 15, 2016

Choose a reason for hiding this comment

Uh oh!

stephentoub commented Oct 15, 2016

Uh oh!

stephentoub commented Oct 15, 2016

Uh oh!

stephentoub commented Oct 15, 2016

Uh oh!

jamesqo Oct 15, 2016

Choose a reason for hiding this comment

Uh oh!

stephentoub Oct 15, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

scionwest commented Nov 26, 2016

Uh oh!

scionwest commented Nov 27, 2016

Uh oh!

Uh oh!

stephentoub commented Oct 14, 2016 •

edited

Loading

stephentoub Oct 15, 2016 •

edited

Loading