Drop `python-telegram-bot-raw` And Switch to `pyproject.toml` Based Packaging #4288

Bibo-Joshi · 2024-06-01T20:25:06Z

Closes #4129 once ready. closes #4296

First draft, install work okay locally. Let's see how the the tests behave. definitely needs to be tested locally.

Must not be merged before we make a final ptb-raw release including #4270!

Check if readthedocs still works https://docs.python-telegram-bot.org/en/hatch-based-packaging/
Make a test release to test-pypi

…hatchling`

github-actions

Hey! Looks like you edited the (optional) requirements or the pre-commit hooks. I'm just a friendly reminder to keep the additional dependencies for the hooks in sync with the requirements :)

github-actions

Hey! Looks like you edited README.rst or README_RAW.rst. I'm just a friendly reminder to apply relevant changes to both of those files :)

# Conflicts: # requirements-dev.txt

Bibo-Joshi · 2024-06-03T17:38:58Z

RTD seems to run fine & other tests as well. Marking as ready for review.
before merge, I'll also make a test release to the pypi test server to double check if everything is in order

# Conflicts: # README_RAW.rst

# Conflicts: # README_RAW.rst # telegram/__init__.py

Poolitzer

LGTM

harshil21

found a couple of things to look at again

.github/CONTRIBUTING.rst

.github/workflows/pre-commit_dependencies_notifier.yml

.github/workflows/test_official.yml

pyproject.toml

setup.cfg

Co-authored-by: Harshil <[email protected]>

Bibo-Joshi · 2024-06-12T19:49:10Z

Just tried to upload to test-pypi and this threw the following error:

(venv311) ~\PycharmProjects\python-telegram-bot git:[hatch-based-packaging]
twine upload --repository testpypi dist/*
Uploading distributions to https://test.pypi.org/legacy/
Enter your API token: 
Uploading python_telegram_bot-21.3-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 662.4/662.4 kB • 00:01 • 1.9 MB/s
WARNING  Error during upload. Retry with the --verbose option for more details.
ERROR    HTTPError: 400 Bad Request from https://test.pypi.org/legacy/
         Can't have direct dependency: furo-sphinx-search@ git+https://github.com/harshil21/[email protected] ; extra == "dev-all". See
         https://packaging.python.org/specifications/core-metadata for more information.

Apparently PyPi doesn't accept dependencies to git repos & sim as dependencies. TBF, the optional dependencies dev, docs and dev-all are not actually meant to be used by users.
I guess we have several options here:

publish harshils fork as pypi package (significant overhead)
include the fork as submodule (I was really happy to get rid of submodules in v20)
drop using the fork and use the original (does that work?) or none at all
go back to listing dev-dependencies in a requirements-dev.txt file. E.g. black, fastapi do it like that
go all-in and let a packaging manager like hatch manage everything. for hatch that would be "environments" such that one would run hatch run docs:build or something like that. I don't think I'm ready for that yet …

I took a bit to write up the choices here and I think while typing I've already made up my mind for Option 4 :D

Bibo-Joshi · 2024-06-12T20:19:27Z

RTD build still works
- https://readthedocs.org/projects/python-telegram-bot/builds/24676212/
- https://docs.python-telegram-bot.org/en/hatch-based-packaging/
testpypi package is up: https://test.pypi.org/project/python-telegram-bot/21.3/. Can you install pip install -i https://test.pypi.org/simple/ python-telegram-bot==21.3 in a fresh venv and run a few of our examples for double checking that everything works as expected?

pyproject.toml

harshil21

Apparently PyPi doesn't accept dependencies to git repos & sim as dependencies

surprising, but understandable.

drop using the fork and use the original (does that work?) or none at all

the original one is just buggy, has less features, and bad design compared to this "furo compatible" one.

I took a bit to write up the choices here and I think while typing I've already made up my mind for Option 4

the only reasonable choice. Also makes it easier to update our local environment I think.

Can you install pip install -i https://test.pypi.org/simple/ python-telegram-bot==21.3 in a fresh venv and run a few of our examples for double checking that everything works as expected?

it fails because httpx doesn't have a test pypi release, so it can't find the distribution. The correct command is: pip install -i https://test.pypi.org/simple --extra-index-url https://pypi.org/simple python-telegram-bot==21.3?

Though I'm not sure which ptb version it installed, the one from test pypi or real pypi (because test pypi one fails)?

.github/CONTRIBUTING.rst

requirements-all.txt

Bibo-Joshi · 2024-06-14T10:31:58Z

Can you install pip install -i https://test.pypi.org/simple/ python-telegram-bot==21.3 in a fresh venv and run a few of our examples for double checking that everything works as expected?

it fails because httpx doesn't have a test pypi release, so it can't find the distribution. The correct command is: pip install -i https://test.pypi.org/simple --extra-index-url https://pypi.org/simple python-telegram-bot==21.3?

Though I'm not sure which ptb version it installed, the one from test pypi or real pypi (because test pypi one fails)?

Interesting! I did a new release 21.4rc0 to have something unique and that seems to work now:

(.venv) ~\PycharmProjects\aiorem git:[master]
pip install -i https://test.pypi.org/simple --extra-index-url https://pypi.org/simple python-telegram-bot==21.4rc0 --no-cache --force --verbose
Using pip 24.0 from C:\Users\hinri\PycharmProjects\aiorem\.venv\Lib\site-packages\pip (python 3.12)
Looking in indexes: https://test.pypi.org/simple, https://pypi.org/simple
Collecting python-telegram-bot==21.4rc0
  Obtaining dependency information for python-telegram-bot==21.4rc0 from https://test-files.pythonhosted.org/packages/70/cb/f1b28a9ad2f70a91a55f243386c4aeeb434a2a7c7f6ecf4951055f7a6dd4/python_telegram_bot-21.4rc0-py3-none-any.whl.metadata
  Downloading https://test-files.pythonhosted.org/packages/70/cb/f1b28a9ad2f70a91a55f243386c4aeeb434a2a7c7f6ecf4951055f7a6dd4/python_telegram_bot-21.4rc0-py3-none-any.whl.metadata (15 kB)
Collecting httpx~=0.27 (from python-telegram-bot==21.4rc0)
  Obtaining dependency information for httpx~=0.27 from https://files.pythonhosted.org/packages/41/7b/ddacf6dcebb42466abd03f368782142baa82e08fc0c1f8eaa05b4bae87d5/httpx-0.27.0-py3-none-any.whl.metadata
  Downloading httpx-0.27.0-py3-none-any.whl.metadata (7.2 kB)
Collecting anyio (from httpx~=0.27->python-telegram-bot==21.4rc0)
  Obtaining dependency information for anyio from https://files.pythonhosted.org/packages/7b/a2/10639a79341f6c019dedc95bd48a4928eed9f1d1197f4c04f546fc7ae0ff/anyio-4.4.0-py3-none-any.whl.metadata
  Downloading anyio-4.4.0-py3-none-any.whl.metadata (4.6 kB)
Collecting certifi (from httpx~=0.27->python-telegram-bot==21.4rc0)
  Obtaining dependency information for certifi from https://files.pythonhosted.org/packages/5b/11/1e78951465b4a225519b8c3ad29769c49e0d8d157a070f681d5b6d64737f/certifi-2024.6.2-py3-none-any.whl.metadata
  Downloading certifi-2024.6.2-py3-none-any.whl.metadata (2.2 kB)
Collecting httpcore==1.* (from httpx~=0.27->python-telegram-bot==21.4rc0)
  Obtaining dependency information for httpcore==1.* from https://files.pythonhosted.org/packages/78/d4/e5d7e4f2174f8a4d63c8897d79eb8fe2503f7ecc03282fee1fa2719c2704/httpcore-1.0.5-py3-none-any.whl.metadata
  Downloading httpcore-1.0.5-py3-none-any.whl.metadata (20 kB)
Collecting idna (from httpx~=0.27->python-telegram-bot==21.4rc0)
  Obtaining dependency information for idna from https://files.pythonhosted.org/packages/e5/3e/741d8c82801c347547f8a2a06aa57dbb1992be9e948df2ea0eda2c8b79e8/idna-3.7-py3-none-any.whl.metadata
  Downloading idna-3.7-py3-none-any.whl.metadata (9.9 kB)
Collecting sniffio (from httpx~=0.27->python-telegram-bot==21.4rc0)
  Obtaining dependency information for sniffio from https://files.pythonhosted.org/packages/e9/44/75a9c9421471a6c4805dbf2356f7c181a29c1879239abab1ea2cc8f38b40/sniffio-1.3.1-py3-none-any.whl.metadata
  Downloading sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB)
Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx~=0.27->python-telegram-bot==21.4rc0)
  Obtaining dependency information for h11<0.15,>=0.13 from https://files.pythonhosted.org/packages/95/04/ff642e65ad6b90db43e668d70ffb6736436c7ce41fcc549f4e9472234127/h11-0.14.0-py3-none-any.whl.metadata
  Downloading h11-0.14.0-py3-none-any.whl.metadata (8.2 kB)
Downloading https://test-files.pythonhosted.org/packages/70/cb/f1b28a9ad2f70a91a55f243386c4aeeb434a2a7c7f6ecf4951055f7a6dd4/python_telegram_bot-21.4rc0-py3-none-any.whl (635 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 635.4/635.4 kB 3.6 MB/s eta 0:00:00
Downloading httpx-0.27.0-py3-none-any.whl (75 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.6/75.6 kB 4.4 MB/s eta 0:00:00
Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 4.2 MB/s eta 0:00:00
Downloading anyio-4.4.0-py3-none-any.whl (86 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.8/86.8 kB 5.1 MB/s eta 0:00:00
Downloading idna-3.7-py3-none-any.whl (66 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.8/66.8 kB ? eta 0:00:00
Downloading sniffio-1.3.1-py3-none-any.whl (10 kB)
Downloading certifi-2024.6.2-py3-none-any.whl (164 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 164.4/164.4 kB 9.6 MB/s eta 0:00:00
Downloading h11-0.14.0-py3-none-any.whl (58 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 3.2 MB/s eta 0:00:00
Installing collected packages: sniffio, idna, h11, certifi, httpcore, anyio, httpx, python-telegram-bot
  Attempting uninstall: sniffio
    Found existing installation: sniffio 1.3.1
    Uninstalling sniffio-1.3.1:
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\sniffio-1.3.1.dist-info\
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\sniffio\
      Successfully uninstalled sniffio-1.3.1
  Attempting uninstall: idna
    Found existing installation: idna 3.7
    Uninstalling idna-3.7:
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\idna-3.7.dist-info\
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\idna\
      Successfully uninstalled idna-3.7
  Attempting uninstall: h11
    Found existing installation: h11 0.14.0
    Uninstalling h11-0.14.0:
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\h11-0.14.0.dist-info\
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\h11\
      Successfully uninstalled h11-0.14.0
  Attempting uninstall: certifi
    Found existing installation: certifi 2024.6.2
    Uninstalling certifi-2024.6.2:
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\certifi-2024.6.2.dist-info\
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\certifi\
      Successfully uninstalled certifi-2024.6.2
  Attempting uninstall: httpcore
    Found existing installation: httpcore 1.0.5
    Uninstalling httpcore-1.0.5:
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\httpcore-1.0.5.dist-info\
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\httpcore\
      Successfully uninstalled httpcore-1.0.5
  Attempting uninstall: anyio
    Found existing installation: anyio 4.4.0
    Uninstalling anyio-4.4.0:
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\anyio-4.4.0.dist-info\
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\anyio\
      Successfully uninstalled anyio-4.4.0
  Attempting uninstall: httpx
    Found existing installation: httpx 0.27.0
    Uninstalling httpx-0.27.0:
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\httpx-0.27.0.dist-info\
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\httpx\
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\scripts\httpx.exe
      Successfully uninstalled httpx-0.27.0
  Attempting uninstall: python-telegram-bot
    Found existing installation: python-telegram-bot 21.3
    Uninstalling python-telegram-bot-21.3:
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\python_telegram_bot-21.3.dist-info\
      Removing file or directory c:\users\hinri\pycharmprojects\aiorem\.venv\lib\site-packages\telegram\
      Successfully uninstalled python-telegram-bot-21.3
Successfully installed anyio-4.4.0 certifi-2024.6.2 h11-0.14.0 httpcore-1.0.5 httpx-0.27.0 idna-3.7 python-telegram-bot-21.4rc0 sniffio-1.3.1

Note that the url for ptb explicitly says test-files.pythonhosted.org/… :)

Running echobot worked fine for me, installing extra dependencies job-queue, ext & all worked fine as well.

https://black.readthedocs.io/en/stable/usage_and_configuration/the_basics.html#t-target-version

https://docs.astral.sh/ruff/settings/#target-version

harshil21

I did a new release 21.4rc0 to have something unique and that seems to work now

can verify it also works for me now.

🚀

harshil21 · 2024-06-15T01:15:08Z

pyproject.toml

+]
+webhooks = [
+    # tornado is rather stable, but let's not allow the next major release without prior testing
+    "tornado~=6.4",


Suggested change

"tornado~=6.4",

"tornado~=6.4.1",

we can take the opportunity to bump this here or another PR

No, this is on purpose, i.e. we also want to allow 6.x, x>=4 :)

oh ok, then why is cachetools on ~=5.3.3, won't that block a 5.4 release then?

we did that back then b/c we hadn't had that much experience with cachetools yet and wanted to be on the save side …

Get started on converting to pyproject.toml based packaging using `…

48def4d

…hatchling`

Bibo-Joshi added 📋 do-not-merge-yet work status: do-not-merge-yet 🛠 refactor change type: refactor labels Jun 1, 2024

github-actions bot reviewed Jun 1, 2024

View reviewed changes

Bibo-Joshi added 3 commits June 2, 2024 05:44

Address workflow failures

6a5e28f

Merge branch 'master' into hatch-based-packaging

dcefe61

# Conflicts: # requirements-dev.txt

Bump pytest

885fa70

Bibo-Joshi changed the title ~~[WIP] Drop python-telegram-bot-raw And Switch to pyproject.toml Based Packaging~~ Drop python-telegram-bot-raw And Switch to pyproject.toml Based Packaging Jun 3, 2024

Remove some other remnants

3ba5d05

Bibo-Joshi marked this pull request as ready for review June 6, 2024 15:45

Merge branch 'master' into hatch-based-packaging

3999503

# Conflicts: # README_RAW.rst

Bibo-Joshi removed the 📋 do-not-merge-yet work status: do-not-merge-yet label Jun 7, 2024

Merge branch 'refs/heads/master' into hatch-based-packaging

ec6559e

# Conflicts: # README_RAW.rst # telegram/__init__.py

Poolitzer approved these changes Jun 10, 2024

View reviewed changes

Bibo-Joshi mentioned this pull request Jun 10, 2024

[BUG] python-telegram-bot-raw deprecation warning appears incorrectly if compiled with Nuitka #4296

Closed

harshil21 requested changes Jun 11, 2024

View reviewed changes

Bibo-Joshi and others added 7 commits June 11, 2024 12:32

Update pyproject.toml

f3d0275

Co-authored-by: Harshil <[email protected]>

Update pyproject.toml

69a2808

Co-authored-by: Harshil <[email protected]>

Update pyproject.toml

611955f

Co-authored-by: Harshil <[email protected]>

Update .github/workflows/test_official.yml

ee86402

Co-authored-by: Harshil <[email protected]>

more review

12e909e

drop manifest.in, set license identifier

9db589b

fix license-files

dad9c75

Bibo-Joshi requested review from harshil21 and Poolitzer June 12, 2024 19:27

Bibo-Joshi added 2 commits June 12, 2024 22:01

Go back to using reqs-dev/all.txt

1694128

Fix test-official

b2a111c

Remove pre-commit dependency notifier

e4e4d85

Bibo-Joshi commented Jun 13, 2024

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

remove allow_direct_ref flag

9ac4558

harshil21 reviewed Jun 14, 2024

View reviewed changes

.github/CONTRIBUTING.rst Show resolved Hide resolved

requirements-all.txt Outdated Show resolved Hide resolved

New proposal for dev requirements naming

42d2353

Bibo-Joshi added 3 commits June 14, 2024 13:14

Let black infer the target version automatically

4b0e6dc

https://black.readthedocs.io/en/stable/usage_and_configuration/the_basics.html#t-target-version

Let ruff infer the target version automatically

da80b74

https://docs.astral.sh/ruff/settings/#target-version

Remove setup.py from readme

7623bd3

harshil21 approved these changes Jun 15, 2024

View reviewed changes

Bibo-Joshi merged commit 44e8292 into master Jun 15, 2024
25 of 26 checks passed

Bibo-Joshi deleted the hatch-based-packaging branch June 15, 2024 08:29

github-actions bot locked and limited conversation to collaborators Jun 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Drop `python-telegram-bot-raw` And Switch to `pyproject.toml` Based Packaging #4288

Drop `python-telegram-bot-raw` And Switch to `pyproject.toml` Based Packaging #4288

Bibo-Joshi commented Jun 1, 2024 •

edited

Loading

github-actions bot left a comment

github-actions bot left a comment

Bibo-Joshi commented Jun 3, 2024

Poolitzer left a comment

harshil21 left a comment

Bibo-Joshi commented Jun 12, 2024

Bibo-Joshi commented Jun 12, 2024

harshil21 left a comment •

edited

Loading

Bibo-Joshi commented Jun 14, 2024

harshil21 left a comment

harshil21 Jun 15, 2024

Bibo-Joshi Jun 15, 2024

harshil21 Jun 15, 2024

Bibo-Joshi Jun 15, 2024

Drop python-telegram-bot-raw And Switch to pyproject.toml Based Packaging #4288

Drop python-telegram-bot-raw And Switch to pyproject.toml Based Packaging #4288

Conversation

Bibo-Joshi commented Jun 1, 2024 • edited Loading

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

Bibo-Joshi commented Jun 3, 2024

Poolitzer left a comment

Choose a reason for hiding this comment

harshil21 left a comment

Choose a reason for hiding this comment

Bibo-Joshi commented Jun 12, 2024

Bibo-Joshi commented Jun 12, 2024

harshil21 left a comment • edited Loading

Choose a reason for hiding this comment

Bibo-Joshi commented Jun 14, 2024

harshil21 left a comment

Choose a reason for hiding this comment

harshil21 Jun 15, 2024

Choose a reason for hiding this comment

Bibo-Joshi Jun 15, 2024

Choose a reason for hiding this comment

harshil21 Jun 15, 2024

Choose a reason for hiding this comment

Bibo-Joshi Jun 15, 2024

Choose a reason for hiding this comment

Drop `python-telegram-bot-raw` And Switch to `pyproject.toml` Based Packaging #4288

Drop `python-telegram-bot-raw` And Switch to `pyproject.toml` Based Packaging #4288

Bibo-Joshi commented Jun 1, 2024 •

edited

Loading

harshil21 left a comment •

edited

Loading