Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix performance issue with ResponseStream wrapper #6937

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 26, 2022

Conversation

thrau
Copy link
Member

@thrau thrau commented Sep 26, 2022

This PR fixes a performance issue that occurred with AWS responses that have streaming bodies. The change exposes the underlying iterator of Response directly through the _ResponseStream wrapper, so invocations of the form for item in stream no longer go through the pointless read_into abstraction.

Detailed explanation

This issues surfaced with the new S3 provider, where get_object download would take exponentially longer the larger the body. When bodies are streamed, we wrap the response in a _ResponseStream object that exposes the response as a readable IO object

response_dict["body"] = _ResponseStream(response)

The issue is that the _ResponseStream is invoked in the form for item in stream. Through the RawIOBase abstraction, this creates an iterator by looping over calls to read_into with a byte buffer of size 1.
The culprit is this line in particular:

output, self._buf = chunk[:upto], chunk[upto:]

Since the underlying response is one big blob in memory, chunk will actually contain the entire s3 object. Consequently, we're always popping a single byte off the entire chunk to populate the response, and copying the remaining buffer. This will take longer depending on how big chunk is, which is the exponential runtime behavior we were seeing.

After the fix, read_into might actually be dead code now, since the underlying webserver always consumes the response stream using the for data in stream pattern. However, read_into is still necessary for the RawIOBase implementation, so I added tests to cover it.

Reproduce

Here's a simple benchmark. On my machine this takes

  • 1.3 seconds without the fix
  • 0.00016 seconds with the fix
import timeit

from werkzeug.wrappers.response import _iter_encoded, Response

from localstack.aws.client import _ResponseStream

n_bytes = 200000


def process_stream():
    b = b'0' * n_bytes
    stream = _ResponseStream(Response(b))

    for line in _iter_encoded(stream, 'utf-8'):
        assert line


def main():
    print(timeit.timeit(stmt=process_stream, number=1))


if __name__ == '__main__':
    main()

@thrau thrau requested a review from bentsku September 26, 2022 16:03
@thrau thrau temporarily deployed to localstack-ext-tests September 26, 2022 16:03 Inactive
@coveralls
Copy link

Coverage Status

Coverage increased (+0.006%) to 79.823% when pulling 44fdf98 on fix-response-stream-performance into 68c507e on master.

@github-actions
Copy link

LocalStack integration with Pro

       3 files  +    2         3 suites  +2   1h 10m 32s ⏱️ + 13m 47s
1 313 tests +  16  1 198 ✔️ +  27  115 💤  -   11  0 ±0 
1 825 runs  +528  1 566 ✔️ +395  259 💤 +133  0 ±0 

Results for commit 44fdf98. ± Comparison against base commit 68c507e.

Copy link
Contributor

@bentsku bentsku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀 Thanks a lot for fixing this deep bug, great catch. Super excited that we can soon see the results of the S3 migration. Awesome! 👍

@thrau thrau merged commit 6a734e2 into master Sep 26, 2022
@thrau thrau deleted the fix-response-stream-performance branch September 26, 2022 21:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants