Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add Parameter read_file_handle to InputFile #4388

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Aug 2, 2024
Merged

Add Parameter read_file_handle to InputFile #4388

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
50e5ef5
Document `InputFile` as valid input object
Bibo-Joshi Jul 17, 2024
edde944
Add parameter `read_file_handle` to `InputFile`
Bibo-Joshi Jul 17, 2024
bc22b86
Update docs down the line
Bibo-Joshi Jul 17, 2024
f74331e
Merge branch 'refs/heads/master' into bug/large-files
Bibo-Joshi Jul 21, 2024
7da978e
Add a few tests
Bibo-Joshi Jul 21, 2024
40c335c
Merge branch 'refs/heads/master' into bug/large-files
Bibo-Joshi Jul 21, 2024
04e10af
Fix test-official
Bibo-Joshi Jul 21, 2024
4edb6a4
Try addressing code coverage
Bibo-Joshi Jul 21, 2024
1955010
Merge branch 'refs/heads/master' into bug/large-files
Bibo-Joshi Aug 2, 2024
ffb0da6
Review
Bibo-Joshi Aug 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 25 additions & 22 deletions telegram/_bot.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1305,8 +1305,8 @@ async def send_photo(

Args:
chat_id (:obj:`int` | :obj:`str`): |chat_id_channel|
photo (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.PhotoSize`): Photo to send.
photo (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | :obj:`bytes` \
| :class:`pathlib.Path` | :class:`telegram.PhotoSize`): Photo to send.
|fileinput|
Lastly you can pass an existing :class:`telegram.PhotoSize` object to send.

Expand Down Expand Up @@ -1465,9 +1465,9 @@ async def send_audio(

Args:
chat_id (:obj:`int` | :obj:`str`): |chat_id_channel|
audio (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.Audio`): Audio file to send.
|fileinput|
audio (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | \
:obj:`bytes` | :class:`pathlib.Path` | :class:`telegram.Audio`): Audio file to
send. |fileinput|
Lastly you can pass an existing :class:`telegram.Audio` object to send.

.. versionchanged:: 13.2
Expand Down Expand Up @@ -1617,8 +1617,8 @@ async def send_document(

Args:
chat_id (:obj:`int` | :obj:`str`): |chat_id_channel|
document (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.Document`): File to send.
document (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | \
:obj:`bytes` | :class:`pathlib.Path` | :class:`telegram.Document`): File to send.
|fileinput|
Lastly you can pass an existing :class:`telegram.Document` object to send.

Expand Down Expand Up @@ -1755,8 +1755,8 @@ async def send_sticker(

Args:
chat_id (:obj:`int` | :obj:`str`): |chat_id_channel|
sticker (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.Sticker`): Sticker to send.
sticker (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | \
:obj:`bytes` | :class:`pathlib.Path` | :class:`telegram.Sticker`): Sticker to send.
|fileinput| Video stickers can only be sent by a ``file_id``. Video and animated
stickers can't be sent via an HTTP URL.

Expand Down Expand Up @@ -1895,8 +1895,8 @@ async def send_video(

Args:
chat_id (:obj:`int` | :obj:`str`): |chat_id_channel|
video (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.Video`): Video file to send.
video (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | :obj:`bytes` \
| :class:`pathlib.Path` | :class:`telegram.Video`): Video file to send.
|fileinput|
Lastly you can pass an existing :class:`telegram.Video` object to send.

Expand Down Expand Up @@ -2059,8 +2059,9 @@ async def send_video_note(

Args:
chat_id (:obj:`int` | :obj:`str`): |chat_id_channel|
video_note (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.VideoNote`): Video note to send.
video_note (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | \
:obj:`bytes` | :class:`pathlib.Path` | :class:`telegram.VideoNote`): Video note
to send.
Pass a file_id as String to send a video note that exists on the Telegram
servers (recommended) or upload a new video using multipart/form-data.
|uploadinput|
Expand Down Expand Up @@ -2209,9 +2210,9 @@ async def send_animation(

Args:
chat_id (:obj:`int` | :obj:`str`): |chat_id_channel|
animation (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.Animation`): Animation to send.
|fileinput|
animation (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | \
:obj:`bytes` | :class:`pathlib.Path` | :class:`telegram.Animation`): Animation to
send. |fileinput|
Lastly you can pass an existing :class:`telegram.Animation` object to send.

.. versionchanged:: 13.2
Expand Down Expand Up @@ -2371,8 +2372,8 @@ async def send_voice(

Args:
chat_id (:obj:`int` | :obj:`str`): |chat_id_channel|
voice (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.Voice`): Voice file to send.
voice (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | :obj:`bytes` \
| :class:`pathlib.Path` | :class:`telegram.Voice`): Voice file to send.
|fileinput|
Lastly you can pass an existing :class:`telegram.Voice` object to send.

Expand Down Expand Up @@ -6370,8 +6371,9 @@ async def upload_sticker_file(

Args:
user_id (:obj:`int`): User identifier of sticker file owner.
sticker (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path`):
A file with the sticker in the ``".WEBP"``, ``".PNG"``, ``".TGS"`` or ``".WEBM"``
sticker (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | \
:obj:`bytes` | :class:`pathlib.Path`): A file with the sticker in the
``".WEBP"``, ``".PNG"``, ``".TGS"`` or ``".WEBM"``
format. See `here <https://core.telegram.org/stickers>`_ for technical requirements
. |uploadinput|

Expand Down Expand Up @@ -6695,8 +6697,9 @@ async def set_sticker_set_thumbnail(

.. versionadded:: 21.1

thumbnail (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path`, \
optional): A **.WEBP** or **.PNG** image with the thumbnail, must
thumbnail (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | \
:obj:`bytes` | :class:`pathlib.Path`, optional): A **.WEBP** or **.PNG** image
with the thumbnail, must
be up to :tg-const:`telegram.constants.StickerSetLimit.MAX_STATIC_THUMBNAIL_SIZE`
kilobytes in size and have width and height of exactly
:tg-const:`telegram.constants.StickerSetLimit.STATIC_THUMB_DIMENSIONS` px, or a
Expand Down
44 changes: 39 additions & 5 deletions telegram/_files/inputfile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
from typing import IO, Optional, Union
from uuid import uuid4

from telegram._utils.files import load_file
from telegram._utils.files import guess_file_name, load_file
from telegram._utils.strings import TextEncoding
from telegram._utils.types import FieldTuple

Expand Down Expand Up @@ -53,9 +53,36 @@ class InputFile:
attach (:obj:`bool`, optional): Pass :obj:`True` if the parameter this file belongs to in
the request to Telegram should point to the multipart data via an ``attach://`` URI.
Defaults to `False`.
read_file_handle (:obj:`bool`, optional): If :obj:`True` and :paramref:`obj` is a file
handle, the data will be read from the file handle on initialization of this object.
If :obj:`False`, the file handle will be passed on to the
`networking backend <telegram.request.BaseRequest.do_request>`_ which will have to
handle the reading. Defaults to :obj:`True`.

Tip:
If you upload extremely large files, you may want to set this to :obj:`False` to
avoid reading the complete file into memory. Additionally, this may be supported
better by the networking backend (in particular it is handled better by
the default :class:`~telegram.request.HTTPXRequest`).

Important:
If you set this to :obj:`False`, you have to ensure that the file handle is still
open when the request is made. In particular, the following snippet can *not* work
as expected.

.. code-block:: python

with open('file.txt', 'rb') as file:
input_file = InputFile(file, read_file_handle=False)

# here the file handle is already closed and the upload will fail
await bot.send_document(chat_id, input_file)

.. versionadded:: NEXT.VERSION


Attributes:
input_file_content (:obj:`bytes`): The binary content of the file to send.
input_file_content (:obj:`bytes` | :class:`IO`): The binary content of the file to send.
attach_name (:obj:`str`): Optional. If present, the parameter this file belongs to in
the request to Telegram should point to the multipart data via a an URI of the form
``attach://<attach_name>`` URI.
Expand All @@ -71,14 +98,18 @@ def __init__(
obj: Union[IO[bytes], bytes, str],
filename: Optional[str] = None,
attach: bool = False,
read_file_handle: bool = True,
):
if isinstance(obj, bytes):
self.input_file_content: bytes = obj
self.input_file_content: Union[bytes, IO[bytes]] = obj
elif isinstance(obj, str):
self.input_file_content = obj.encode(TextEncoding.UTF_8)
else:
elif read_file_handle:
reported_filename, self.input_file_content = load_file(obj)
filename = filename or reported_filename
else:
self.input_file_content = obj
filename = filename or guess_file_name(obj)

self.attach_name: Optional[str] = "attached" + uuid4().hex if attach else None

Expand All @@ -95,8 +126,11 @@ def __init__(
def field_tuple(self) -> FieldTuple:
"""Field tuple representing the contents of the file for upload to the Telegram servers.

.. versionchanged:: NEXT.VERSION
Content may now be a file handle.

Returns:
Tuple[:obj:`str`, :obj:`bytes`, :obj:`str`]:
Tuple[:obj:`str`, :obj:`bytes` | :class:`IO`, :obj:`str`]:
"""
return self.filename, self.input_file_content, self.mimetype

Expand Down
37 changes: 19 additions & 18 deletions telegram/_files/inputmedia.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@ class InputMedia(TelegramObject):

Args:
media_type (:obj:`str`): Type of media that the instance represents.
media (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.Animation` | :class:`telegram.Audio` | \
media (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | :obj:`bytes` | \
:class:`pathlib.Path` | :class:`telegram.Animation` | :class:`telegram.Audio` | \
:class:`telegram.Document` | :class:`telegram.PhotoSize` | \
:class:`telegram.Video`): File to send.
|fileinputnopath|
Expand Down Expand Up @@ -128,8 +128,9 @@ class InputPaidMedia(TelegramObject):

Args:
type (:obj:`str`): Type of media that the instance represents.
media (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.PhotoSize` | :class:`telegram.Video`): File to send. |fileinputnopath|
media (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | :obj:`bytes` | \
:class:`pathlib.Path` | :class:`telegram.PhotoSize` | :class:`telegram.Video`): File
to send. |fileinputnopath|
Lastly you can pass an existing telegram media object of the corresponding type
to send.

Expand Down Expand Up @@ -167,8 +168,8 @@ class InputPaidMediaPhoto(InputPaidMedia):
.. versionadded:: 21.4

Args:
media (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.PhotoSize`): File to send. |fileinputnopath|
media (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | :obj:`bytes` | \
:class:`pathlib.Path` | :class:`telegram.PhotoSize`): File to send. |fileinputnopath|
Lastly you can pass an existing :class:`telegram.PhotoSize` object to send.

Attributes:
Expand Down Expand Up @@ -207,8 +208,8 @@ class InputPaidMediaVideo(InputPaidMedia):
changed by Telegram.

Args:
media (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.Video`): File to send. |fileinputnopath|
media (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | :obj:`bytes` | \
:class:`pathlib.Path` | :class:`telegram.Video`): File to send. |fileinputnopath|
Lastly you can pass an existing :class:`telegram.Video` object to send.
thumbnail (:term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | :obj:`str`, \
optional): |thumbdocstringnopath|
Expand Down Expand Up @@ -278,8 +279,8 @@ class InputMediaAnimation(InputMedia):
|removed_thumb_note|

Args:
media (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.Animation`): File to send. |fileinputnopath|
media (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | :obj:`bytes` | \
:class:`pathlib.Path` | :class:`telegram.Animation`): File to send. |fileinputnopath|
Lastly you can pass an existing :class:`telegram.Animation` object to send.

.. versionchanged:: 13.2
Expand Down Expand Up @@ -401,8 +402,8 @@ class InputMediaPhoto(InputMedia):
.. seealso:: :wiki:`Working with Files and Media <Working-with-Files-and-Media>`

Args:
media (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.PhotoSize`): File to send. |fileinputnopath|
media (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | :obj:`bytes` | \
:class:`pathlib.Path` | :class:`telegram.PhotoSize`): File to send. |fileinputnopath|
Lastly you can pass an existing :class:`telegram.PhotoSize` object to send.

.. versionchanged:: 13.2
Expand Down Expand Up @@ -501,8 +502,8 @@ class InputMediaVideo(InputMedia):
|removed_thumb_note|

Args:
media (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.Video`): File to send. |fileinputnopath|
media (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | :obj:`bytes` | \
:class:`pathlib.Path` | :class:`telegram.Video`): File to send. |fileinputnopath|
Lastly you can pass an existing :class:`telegram.Video` object to send.

.. versionchanged:: 13.2
Expand Down Expand Up @@ -639,8 +640,8 @@ class InputMediaAudio(InputMedia):
|removed_thumb_note|

Args:
media (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.Audio`): File to send. |fileinputnopath|
media (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | :obj:`bytes` | \
:class:`pathlib.Path` | :class:`telegram.Audio`): File to send. |fileinputnopath|
Lastly you can pass an existing :class:`telegram.Audio` object to send.

.. versionchanged:: 13.2
Expand Down Expand Up @@ -743,8 +744,8 @@ class InputMediaDocument(InputMedia):
|removed_thumb_note|

Args:
media (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path` | \
:class:`telegram.Document`): File to send. |fileinputnopath|
media (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | :obj:`bytes` \
| :class:`pathlib.Path` | :class:`telegram.Document`): File to send. |fileinputnopath|
Lastly you can pass an existing :class:`telegram.Document` object to send.

.. versionchanged:: 13.2
Expand Down
3 changes: 2 additions & 1 deletion telegram/_files/inputsticker.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,8 @@ class InputSticker(TelegramObject):
order of the arguments has changed.

Args:
sticker (:obj:`str` | :term:`file object` | :obj:`bytes` | :class:`pathlib.Path`): The
sticker (:obj:`str` | :term:`file object` | :class:`~telegram.InputFile` | :obj:`bytes` \
| :class:`pathlib.Path`): The
added sticker. |uploadinputnopath| Animated and video stickers can't be uploaded via
HTTP URL.
emoji_list (Sequence[:obj:`str`]): Sequence of
Expand Down
19 changes: 13 additions & 6 deletions telegram/_utils/files.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,14 +61,21 @@ def load_file(
except AttributeError:
return None, cast(Union[bytes, "InputFile", str, Path], obj)

if hasattr(obj, "name") and not isinstance(obj.name, int):
filename = Path(obj.name).name
else:
filename = None
filename = guess_file_name(obj)

return filename, contents


def guess_file_name(obj: FileInput) -> Optional[str]:
"""If the input is a file handle, read name and return it. Otherwise, return
the input unchanged.
"""
if hasattr(obj, "name") and not isinstance(obj.name, int):
return Path(obj.name).name

return None


def is_local_file(obj: Optional[FilePathInput]) -> bool:
"""
Checks if a given string is a file on local system.
Expand Down Expand Up @@ -110,8 +117,8 @@ def parse_file_input( # pylint: disable=too-many-return-statements
attribute.

Args:
file_input (:obj:`str` | :obj:`bytes` | :term:`file object` | Telegram media object): The
input to parse.
file_input (:obj:`str` | :obj:`bytes` | :term:`file object` | :class:`~telegram.InputFile`\
| Telegram media object): The input to parse.
tg_type (:obj:`type`, optional): The Telegram media type the input can be. E.g.
:class:`telegram.Animation`.
filename (:obj:`str`, optional): The filename. Only relevant in case an
Expand Down
2 changes: 1 addition & 1 deletion telegram/_utils/types.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@
.. versionadded:: 20.0
"""

FieldTuple = Tuple[str, bytes, str]
FieldTuple = Tuple[str, Union[bytes, IO[bytes]], str]
"""Alias for return type of `InputFile.field_tuple`."""
UploadFileDict = Dict[str, FieldTuple]
"""Dictionary containing file data to be uploaded to the API."""
Expand Down
6 changes: 5 additions & 1 deletion telegram/request/_requestdata.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,11 @@ def json_payload(self) -> bytes:

@property
def multipart_data(self) -> UploadFileDict:
"""Gives the files contained in this object as mapping of part name to encoded content."""
"""Gives the files contained in this object as mapping of part name to encoded content.

.. versionchanged:: NEXT.VERSION
Content may now be a file handle.
"""
multipart_data: UploadFileDict = {}
for param in self._parameters:
m_data = param.multipart_data
Expand Down
6 changes: 5 additions & 1 deletion telegram/request/_requestparameter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,11 @@ def json_value(self) -> Optional[str]:

@property
def multipart_data(self) -> Optional[UploadFileDict]:
"""A dict with the file data to upload, if any."""
"""A dict with the file data to upload, if any.

.. versionchanged:: NEXT.VERSION
Content may now be a file handle.
"""
if not self.input_files:
return None
return {
Expand Down
Loading
Loading