Environment markers #357

dahlia · 2017-06-15T11:03:34Z

This patch aims to fix #356 and some related others.

As twilio doesn't support Python 2.5 or below, removed simplejson from conditional dependencies.
Although twilio have used environment markers, it hadn't opt out conditional dependencies when building wheels. If a wheel is once built with Python 3 pip always installs pysocks (which is a conditional dependency only for Python 3) regardless of Python version. Opting out conditional dependencies prevents this problem.
Although environment markers work well with the recent versions of pip, the older versions of pip lacks the feature. To make it work on them as well, if conditions on setup.py still remains if it's not building wheels.
As some older versions of pip doesn't have operators like </>=, it uses only ==/!=/and for backward compatibility.
Environment markers became more organized and reduced duplicated lists.

Dependencies of build wheels now look like ("run_requires" field of twilio-6.3.0.dist-info/metadata.json):

[
  {
    "requires": [
      "PyJWT (>=1.4.2)",
      "pytz",
      "requests (>=2.0.0)",
      "six"
    ]
  },
  {
    "environment": "python_version!='2.6' and python_version!='2.7'",
    "requires": [
      "pysocks"
    ]
  },
  {
    "environment": "python_version=='2.6'",
    "requires": [
      "cryptography (>=1.3.4)",
      "idna (>=2.0.0)",
      "pyOpenSSL (>=0.14)"
    ]
  },
  {
    "environment": "python_version=='2.7'",
    "requires": [
      "cryptography (>=1.3.4)",
      "idna (>=2.0.0)",
      "pyOpenSSL (>=0.14)"
    ]
  }
]

ilanbiala · 2017-06-15T16:40:16Z

setup.py

+# Environment markers
+extras_require = {
+    # Older versions of pip don't support operators other than ==/!=
+    ":python_version=='2.6'": REQUIRES_PY2,


Does

":python_version =='2.6' or :python_version='2.7'": REQUIRES_PY2,

not work?

Never tried, I will change if it works after I tried using or operator.

ilanbiala · 2017-06-15T16:40:36Z

setup.py

+    # Older versions of pip don't support operators other than ==/!=
+    ":python_version=='2.6'": REQUIRES_PY2,
+    ":python_version=='2.7'": REQUIRES_PY2,
+    ":python_version!='2.6' and python_version!='2.7'": REQUIRES_PY3,


I'd rather be explicit about the specific versions of 3 we support, because we do not currently state that we support anything below 3.3, which your environment marker currently does include.

Environment markers doesn't mean to support nor to unsupport Python versions what they're about. As like it doesn't mean to support Python 2.5, doesn't mean support Python 3.2. 😀

However, I would append and python_version!='3.0' and python_version!='3.1' and python!='3.2' to it if you advise to do that. 😜

ilanbiala · 2017-06-15T16:41:30Z

setup.py

-    REQUIRES.extend(["cryptography >= 1.3.4", "idna >= 2.0.0", "pyOpenSSL >= 0.14"])
-if sys.version_info >= (3, 0):
-    REQUIRES.append('pysocks')
+if 'bdist_wheel' not in sys.argv:


When is this if statement used and the environment markers are not used such that this block is necessary?

It's used when an older version of pip installs twilio sdist.

Do you know what versions of pip need this block? Maybe we can remove it if those versions of pip are sufficiently old?

I'm unsure about that, but as far as I remember pip installed on Ubuntu LTS versions didn't work well without this.

dahlia · 2017-06-15T16:58:23Z

Adjusted things @ilanbiala pointed. Thanks!

ilanbiala · 2017-06-15T17:20:16Z

@dahlia thanks for the changes and explanation. I actually think it might be better to just say:

":python_version=='3.3' or :python_version=='3.3' or :python_version=='3.5' or :python_version=='3.6'": REQUIRES_PY3

I'm not sure though if the : is needed before each version, but if not you can just copy and paste the line above and remove those. Anyway, it looks fine to me otherwise, so if you could squash down to 1 commit then we'll get this merged in when we do the next release.

dahlia · 2017-06-15T17:24:33Z

@ilanbiala Got it. Amended!

ilanbiala · 2017-06-15T17:48:00Z

@dahlia LGTM, can you squash down to 1 commit?

dahlia · 2017-06-15T18:00:20Z

@ilanbiala Should I squash all 5 commits into one? GitHub supports squash merging of pull requests.

ilanbiala · 2017-06-15T18:32:35Z

Oh thanks pointing that out, I didn't know GitHub added that.

dahlia · 2017-06-16T05:38:37Z

To resolve conflict with the current master I rebased it on the current master. As Python 2.6 support is dropped I remove it from environment markers as well.

dahlia · 2017-06-19T10:15:00Z

Are there anything I should do more to merge this? Or it's ready but waiting for the next release?

ilanbiala · 2017-06-19T16:23:55Z

@dahlia It's ready, and we're including it in the next release. Thanks for submitting the fix and following up!

tisdall · 2017-07-11T13:09:54Z

The only problem with this code is it breaks when Python 3.7 is released. I think the old version tests made more sense so that it included REQUIRES_PY3 if it's not one of the supported py2 versions (since no more py2 versions are going to come). I think there's also discussions about a py4.

ilanbiala · 2017-07-11T17:29:07Z

@tisdall can you clarify how it would break when Python 3.7 is released? Are you referring to just missing the python_version for pysocks for it? Also, where are there talks about Python 4?

tisdall · 2017-07-11T18:37:25Z

Python 3.7 won't match any of the extras_require statements so none of REQUIRES_PY3 will be included (which right now is just pysocks). I haven't investigated whether pysocks is necessary for py3.7, but I'm assuming it is.

extras_require = {
    # Older versions of pip don't support operators other than ==/!=
    ":python_version=='2.7'": REQUIRES_PY2,
    (":python_version=='3.3' or python_version=='3.4' or"
    " python_version=='3.5' or python_version=='3.6'"): REQUIRES_PY3,
}

Okay, I'm probably wrong about py4, please disregard that part.

ilanbiala · 2017-07-11T18:58:07Z

@tisdall Right, but we can just add or python_version=='3.7' when it is released, right? Or am I missing something more?

tisdall · 2017-07-11T20:18:44Z

@ilanbiala - You can, but I think what @dahlia had before would mean that wouldn't be necessary: ":python_version!='2.6' and python_version!='2.7'": REQUIRES_PY3,

I guess it's not a big deal, it's just one less thing to worry about when new versions of python are added.

ilanbiala · 2017-07-12T22:30:49Z

Superseded by #368. Thanks @dahlia for the initial work, but we did some further testing and this method seems to be simpler and work when we publish both sdists and bdist_wheels.

dahlia mentioned this pull request Jun 15, 2017

pysocks installed in py2.7 #356

Closed

ilanbiala reviewed Jun 15, 2017

View reviewed changes

dahlia force-pushed the environment-markers branch from d95a2e7 to ed8dd30 Compare June 15, 2017 17:24

ilanbiala added the deploy label Jun 15, 2017

dahlia added 5 commits June 16, 2017 14:36

Use envmarkers for wheels and conditions for sdist

a1160c5

Older pip doesn't support < or >=

e657be3

Unify two conditions for py2 into one

2786a2f

Be more explicit about supported Python 3 versions

f1f9c15

No more support Python 2.6

09ce75f

dahlia force-pushed the environment-markers branch from ed8dd30 to 09ce75f Compare June 16, 2017 05:37

dougblack approved these changes Jun 16, 2017

View reviewed changes

ilanbiala mentioned this pull request Jul 12, 2017

Fix package distribution #368

Merged

ilanbiala closed this Jul 12, 2017

Environment markers #357

Environment markers #357

Uh oh!

Conversation

dahlia commented Jun 15, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dahlia commented Jun 15, 2017

Uh oh!

ilanbiala commented Jun 15, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dahlia commented Jun 15, 2017

Uh oh!

ilanbiala commented Jun 15, 2017

Uh oh!

dahlia commented Jun 15, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ilanbiala commented Jun 15, 2017

Uh oh!

dahlia commented Jun 16, 2017

Uh oh!

dahlia commented Jun 19, 2017

Uh oh!

ilanbiala commented Jun 19, 2017

Uh oh!

tisdall commented Jul 11, 2017

Uh oh!

ilanbiala commented Jul 11, 2017

Uh oh!

tisdall commented Jul 11, 2017

Uh oh!

ilanbiala commented Jul 11, 2017

Uh oh!

tisdall commented Jul 11, 2017

Uh oh!

ilanbiala commented Jul 12, 2017

Uh oh!

Uh oh!

dahlia commented Jun 15, 2017 •

edited

Loading

ilanbiala commented Jun 15, 2017 •

edited

Loading

dahlia commented Jun 15, 2017 •

edited

Loading