Thanks to visit codestin.com
Credit goes to github.com

Skip to content

http.client.HTTPResponse.read1 does not close response after reading all data #113199

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
illia-v opened this issue Dec 15, 2023 · 7 comments
Closed

http.client.HTTPResponse.read1 does not close response after reading all data #113199

illia-v opened this issue Dec 15, 2023 · 7 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@illia-v
Copy link
Contributor

illia-v commented Dec 15, 2023 •
edited by bedevere-app bot
Loading

Bug report

Bug description:

While trying to use read1 in urllib3, we discovered that read1 unlike read never closes the response after reading all data when the content length is known (urllib3/urllib3#3235).

import http.client

conn = http.client.HTTPSConnection("www.python.org")
conn.request("GET", "/")
response = conn.getresponse()

This runs successfully:

response.read()
assert response.isclosed()

This fails:

while response.read1():
    pass
assert response.isclosed()

CPython versions tested on:

3.8, 3.9, 3.10, 3.11, 3.12, 3.13, CPython main branch

Operating systems tested on:

Linux, macOS, Windows

Linked PRs
@illia-v illia-v added the type-bug An unexpected behavior, bug, or error label Dec 15, 2023
illia-v added a commit to illia-v/cpython that referenced this issue Dec 15, 2023
@illia-v
pythongh-113199: Make HTTPResponse.read1 close IO after reading all…
f473ef5 
… data
illia-v added a commit to illia-v/cpython that referenced this issue Dec 15, 2023
@illia-v
pythongh-113199: Make HTTPResponse.read1 close IO after reading all…
7322221 
… data
@bedevere-app bedevere-app bot mentioned this issue Dec 15, 2023
Merged
@AlexWaygood AlexWaygood added the stdlib Python modules in the Lib dir label Dec 17, 2023
@serhiy-storchaka
Copy link
Member

Is it a bug? The next reading operation will return an empty bytes and close the connection. So it may be just an optimization.

The read1() method was added in 8e5d0ca (bpo-19009/#63209), several years after adding autoclose when read all data in 7066590 (bpo-1580738/#44147). Why this was not added in read1() and readline()? @kristjanvalur, @pitrou, was it just an oversight or were there reasons for it?

@pitrou
Copy link
Member

pitrou commented Dec 17, 2023

This was probably an oversight and I think this should be fixed.

@illia-v
Copy link
Contributor Author

illia-v commented Dec 17, 2023

Is it a bug? The next reading operation will return an empty bytes and close the connection.

Closing won't happen when the content length is known.

cpython/Lib/http/client.py

Lines 661 to 665 in 21d5299

if self.length is not None and (n < 0 or n > self.length):
n = self.length
result = self.fp.read1(n)
if not result and n:
self._close_conn()

After it has read all data, any subsequent call will reset n to self.length equal to 0 on line 662. Because of this closing will be skipped.

illia-v added a commit to illia-v/cpython that referenced this issue Dec 17, 2023
@illia-v
Merge remote-tracking branch 'python/main' into pythongh-113199
3f6f3cb
@kristjanvalur
Copy link
Contributor

Nice necro. Yes, probably an oversight. The original impetus was to read from streams that didn't close.

serhiy-storchaka pushed a commit that referenced this issue Dec 18, 2023
@illia-v
gh-113199: Make read1() and readline() of HTTPResponse close IO after…
41336a7 
… reading all data (GH-113200)
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Dec 18, 2023
@illia-v @miss-islington
pythongh-113199: Make read1() and readline() of HTTPResponse close IO…
efc7247 
… after reading all data (pythonGH-113200)

(cherry picked from commit 41336a7)

Co-authored-by: Illia Volochii <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Dec 18, 2023
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Dec 18, 2023
@illia-v @miss-islington
pythongh-113199: Make read1() and readline() of HTTPResponse close IO…
2d927f1 
… after reading all data (pythonGH-113200)

(cherry picked from commit 41336a7)

Co-authored-by: Illia Volochii <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Dec 18, 2023
Merged
@serhiy-storchaka
Copy link
Member

Thank you for your contribution @illia-v.

serhiy-storchaka pushed a commit that referenced this issue Dec 18, 2023
@miss-islington @illia-v
[3.11] gh-113199: Make read1() and readline() of HTTPResponse close I…
44bdb6d 
…O after reading all data (GH-113200) (GH-113260)

(cherry picked from commit 41336a7)

Co-authored-by: Illia Volochii <[email protected]>
serhiy-storchaka pushed a commit that referenced this issue Dec 18, 2023
@miss-islington @illia-v
[3.12] gh-113199: Make read1() and readline() of HTTPResponse close I…
816b635 
…O after reading all data (GH-113200) (GH-113259)

(cherry picked from commit 41336a7)

Co-authored-by: Illia Volochii <[email protected]>
@pquentin
Copy link

Thanks everyone! Can this be closed now?

@illia-v
Copy link
Contributor Author

illia-v commented Dec 19, 2023

@serhiy-storchaka @pitrou @kristjanvalur @pquentin thank you for your attention!

@illia-v illia-v closed this as completed Dec 19, 2023
ryan-duve pushed a commit to ryan-duve/cpython that referenced this issue Dec 26, 2023
@illia-v @ryan-duve
pythongh-113199: Make read1() and readline() of HTTPResponse close IO…
c7e51ec 
… after reading all data (pythonGH-113200)
aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
@illia-v @aisk
pythongh-113199: Make read1() and readline() of HTTPResponse close IO…
4bf7de8 
… after reading all data (pythonGH-113200)
@illia-v illia-v mentioned this issue Feb 27, 2024
Open
@illia-v illia-v mentioned this issue Mar 27, 2024
Open
@ghost ghost mentioned this issue May 20, 2024
Closed
Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024
@illia-v @Glyphack
pythongh-113199: Make read1() and readline() of HTTPResponse close IO…
9d0f918 
… after reading all data (pythonGH-113200)
@yurnasg yurnasg mentioned this issue Mar 12, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@pquentin @pitrou @serhiy-storchaka @kristjanvalur @illia-v @AlexWaygood