Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit bbb14b4

Browse files
committed
Modernize packaging
PEP 517 ======= Don't migrate away from setuptools, but move "static" packaging content from `setup.py` to `pyproject.toml` and `setup.cfg`. Building should now be performed using [`build`](https://pypa-build.readthedocs.io/en/stable/), as direct invocations of `setup.py` have been deprecated for a while. Also remove the legacy `__author__` strictures from the source files, as `pyproject.toml` supports an authors section which is more suitable. Codegen Modernization ===================== The modernization via command overrides and self-invocation still generated `setup.py` warnings. Migrate the "invoke commands in commands" method to codegen sub-commands in `build`, it looks like this suffices for what we need. This reuses the existing code, though modernizes it significantly thanks to the previous commit. This is strongly inspired by @abravalheri's comments on codegen in setuptools discussions e.g. pypa/setuptools#3180 and pypa/setuptools#3762, those comments were very helpful in getting a better understanding of the sub_commands system (also @jaraco's comments but those were a touch terse so can't say I really got it until sumbling on @abravalheri's). Leverages the setuptools subcommands protocol to support editable wheels, as that might be useful. Don't use editable wheels for testing though, as I don't see the point. Note: not sure the git submodule stuff is useful during the codegen, so currently commented it out. sdists ====== I'm not entirely clear what sdists should really be about, adding codegen to sdist using subcommands seems not necessarily trivial, so for now make them basically an export: - remove `_regexes.py` from sdist - add uap-core to the manifest, and thus the sdist, at least `regexes.yaml` Currently the test files (both python and yaml) are also included, but it's not clear that they *should* be. That seems like [a whole debate](https://discuss.python.org/t/should-sdists-include-docs-and-tests/14578/117), One strong argument (to me) in favor of a sdist maximalism is that it matches the wheels and more official releases, but in that case the sdist should probably be better crafted than what I threw together. Tox === Switch tox to an explicit `pip install` as its builtin handling for installing packages (whether via sdist or develop) does not seem compatible with PEP-517. Need to think about moving to tox 4, which apparently has native support for PEP 517 packages. Cf #157. Also remove the `docs` tox env, it hasn't been a thing since 8273477 but I forgot to remove it from the envlist.
1 parent 24c8217 commit bbb14b4

File tree

8 files changed

+118
-165
lines changed

8 files changed

+118
-165
lines changed

MANIFEST.in

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
1+
exclude .*
2+
prune .github
3+
global-exclude *~
4+
15
include README.rst
26
include LICENSE
3-
global-exclude *~
7+
graft uap-core
8+
exclude uap-core/.*
9+
recursive-exclude uap-core *.js

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ format:
1414
@black .
1515

1616
release: clean
17-
python setup.py sdist bdist_wheel
17+
pyproject-build
1818
twine upload -s dist/*
1919

2020
.PHONY: all test clean format release

pyproject.toml

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
[build-system]
2+
requires = ["setuptools", "setuptools-scm", "PyYaml"]
3+
build-backend = "setuptools.build_meta"
4+
5+
[project]
6+
name = "ua-parser"
7+
description = "Python port of Browserscope's user agent parser"
8+
version = "1.0.0a"
9+
readme = "README.rst"
10+
requires-python = ">=3.8"
11+
dependencies = []
12+
optional-dependencies = { yaml = ["PyYaml"] }
13+
14+
license = {text = "Apache 2.0"}
15+
urls = {repository = "https://github.com/ua-parser/uap-python"}
16+
17+
authors = [
18+
{ name = "Stephen Lamm", email = "[email protected]"},
19+
{ name = "PBS", email = "[email protected]" },
20+
{ name = "Selwin Ong", email = "[email protected]" },
21+
{ name = "Matt Robenolt", email = "[email protected]" },
22+
{ name = "Lindsey Simon", email = "[email protected]" },
23+
]
24+
maintainers = [
25+
{ name = "masklinn", email = "[email protected]" }
26+
]
27+
28+
classifiers = [
29+
"Development Status :: 4 - Beta",
30+
"Environment :: Web Environment",
31+
"Intended Audience :: Developers",
32+
"Operating System :: OS Independent",
33+
"License :: OSI Approved :: Apache Software License",
34+
"Programming Language :: Python",
35+
"Topic :: Internet :: WWW/HTTP",
36+
"Topic :: Software Development :: Libraries :: Python Modules",
37+
"Programming Language :: Python",
38+
"Programming Language :: Python :: 3.8",
39+
"Programming Language :: Python :: 3.9",
40+
"Programming Language :: Python :: 3.10",
41+
"Programming Language :: Python :: 3.11",
42+
"Programming Language :: Python :: Implementation :: CPython",
43+
"Programming Language :: Python :: Implementation :: PyPy"
44+
]

setup.cfg

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,8 @@
1-
[bdist_wheel]
2-
universal = 1
1+
[options]
2+
packages = find:
3+
package_dir =
4+
=src
5+
setup_requires = pyyaml
6+
7+
[options.packages.find]
8+
where = src

setup.py

Lines changed: 56 additions & 155 deletions
Original file line numberDiff line numberDiff line change
@@ -1,114 +1,83 @@
11
#!/usr/bin/env python
22
# flake8: noqa
3-
import os
4-
from distutils import log
5-
from distutils.core import Command
6-
from distutils.command.build import build as _build
7-
from setuptools import setup, find_packages
8-
from setuptools.command.develop import develop as _develop
9-
from setuptools.command.sdist import sdist as _sdist
10-
from setuptools.command.install import install as _install
11-
12-
13-
def check_output(*args, **kwargs):
14-
from subprocess import Popen
15-
16-
proc = Popen(*args, **kwargs)
17-
output, _ = proc.communicate()
18-
rv = proc.poll()
19-
assert rv == 0, output
20-
21-
22-
class build_regexes(Command):
23-
description = "build supporting regular expressions from uap-core"
24-
user_options = [
25-
("work-path=", "w", "The working directory for source files. Defaults to ."),
26-
("build-lib=", "b", "directory for script runtime modules"),
27-
(
28-
"inplace",
29-
"i",
30-
"ignore build-lib and put compiled javascript files into the source "
31-
+ "directory alongside your pure Python modules",
32-
),
33-
(
34-
"force",
35-
"f",
36-
"Force rebuilding of static content. Defaults to rebuilding on version "
37-
"change detection.",
38-
),
39-
]
40-
boolean_options = ["force"]
41-
42-
def initialize_options(self):
43-
self.build_lib = None
44-
self.force = None
45-
self.work_path = None
46-
self.inplace = None
47-
48-
def finalize_options(self):
49-
install = self.distribution.get_command_obj("install")
50-
sdist = self.distribution.get_command_obj("sdist")
51-
build_ext = self.distribution.get_command_obj("build_ext")
52-
53-
if self.inplace is None:
54-
self.inplace = (
55-
(build_ext.inplace or install.finalized or sdist.finalized) and 1 or 0
56-
)
3+
from contextlib import suppress
4+
from os import fspath
5+
from pathlib import Path
6+
from typing import Optional, List, Dict
577

58-
if self.inplace:
59-
self.build_lib = "./src"
60-
else:
61-
self.set_undefined_options("build", ("build_lib", "build_lib"))
62-
if self.work_path is None:
63-
self.work_path = os.path.realpath(os.path.join(os.path.dirname(__file__)))
8+
from setuptools import setup, Command, find_namespace_packages
9+
from setuptools.command.build import build, SubCommand
10+
from setuptools.command.editable_wheel import editable_wheel
11+
12+
import yaml
13+
14+
15+
build.sub_commands.insert(0, ("compile-regexes", None))
16+
17+
18+
class CompileRegexes(Command, SubCommand):
19+
def initialize_options(self) -> None:
20+
self.pkg_name: Optional[str] = None
6421

65-
def run(self):
22+
def finalize_options(self) -> None:
23+
self.pkg_name = self.distribution.get_name().replace("-", "_")
24+
25+
def get_source_files(self) -> List[str]:
26+
return ["uap-core/regexes.yaml"]
27+
28+
def get_outputs(self) -> List[str]:
29+
return [f"{self.pkg_name}/_regexes.py"]
30+
31+
def get_output_mapping(self) -> Dict[str, str]:
32+
return dict(zip(self.get_source_files(), self.get_outputs()))
33+
34+
def run(self) -> None:
35+
# FIXME: check git / submodules?
36+
"""
6637
work_path = self.work_path
6738
if not os.path.exists(os.path.join(work_path, ".git")):
6839
return
6940
7041
log.info("initializing git submodules")
7142
check_output(["git", "submodule", "init"], cwd=work_path)
7243
check_output(["git", "submodule", "update"], cwd=work_path)
44+
"""
45+
if not self.pkg_name:
46+
return # or error?
7347

74-
yaml_src = os.path.join(work_path, "uap-core", "regexes.yaml")
75-
if not os.path.exists(yaml_src):
48+
yaml_src = Path("uap-core", "regexes.yaml")
49+
if not yaml_src.is_file():
7650
raise RuntimeError(
77-
"Unable to find regexes.yaml, should be at %r" % yaml_src
51+
f"Unable to find regexes.yaml, should be at {yaml_src!r}"
7852
)
7953

80-
def force_bytes(text):
81-
if text is None:
82-
return text
83-
return text.encode("utf8")
84-
8554
def write_params(fields):
8655
# strip trailing None values
8756
while len(fields) > 1 and fields[-1] is None:
8857
fields.pop()
8958

9059
for field in fields:
91-
fp.write((" %r,\n" % field).encode("utf-8"))
60+
fp.write((f" {field!r},\n").encode())
61+
62+
with yaml_src.open("rb") as f:
63+
regexes = yaml.safe_load(f)
9264

93-
import yaml
65+
if self.editable_mode:
66+
dist_dir = Path("src")
67+
else:
68+
dist_dir = Path(self.get_finalized_command("bdist_wheel").bdist_dir)
9469

95-
log.info("compiling regexes.yaml -> _regexes.py")
96-
with open(yaml_src, "rb") as fp:
97-
regexes = yaml.safe_load(fp)
70+
outdir = dist_dir / self.pkg_name
71+
outdir.mkdir(parents=True, exist_ok=True)
9872

99-
lib_dest = os.path.join(self.build_lib, "ua_parser")
100-
if not os.path.exists(lib_dest):
101-
os.makedirs(lib_dest)
73+
dest = outdir / "_regexes.py"
10274

103-
py_dest = os.path.join(lib_dest, "_regexes.py")
104-
with open(py_dest, "wb") as fp:
75+
with dest.open("wb") as fp:
10576
# fmt: off
10677
fp.write(b"# -*- coding: utf-8 -*-\n")
107-
fp.write(b"############################################\n")
108-
fp.write(b"# NOTICE: This file is autogenerated from #\n")
109-
fp.write(b"# regexes.yaml. Do not edit by hand, #\n")
110-
fp.write(b"# instead, re-run `setup.py build_regexes` #\n")
111-
fp.write(b"############################################\n")
78+
fp.write(b"########################################################\n")
79+
fp.write(b"# NOTICE: This file is autogenerated from regexes.yaml #\n")
80+
fp.write(b"########################################################\n")
11281
fp.write(b"\n")
11382
fp.write(b"from .user_agent_parser import (\n")
11483
fp.write(b" UserAgentParser, DeviceParser, OSParser,\n")
@@ -156,77 +125,9 @@ def write_params(fields):
156125
fp.write(b"]\n")
157126
# fmt: on
158127

159-
self.update_manifest()
160-
161-
def update_manifest(self):
162-
sdist = self.distribution.get_command_obj("sdist")
163-
if not sdist.finalized:
164-
return
165-
166-
sdist.filelist.files.append("src/ua_parser/_regexes.py")
167-
168-
169-
class develop(_develop):
170-
def run(self):
171-
self.run_command("build_regexes")
172-
_develop.run(self)
173-
174-
175-
class install(_install):
176-
def run(self):
177-
self.run_command("build_regexes")
178-
_install.run(self)
179-
180-
181-
class build(_build):
182-
def run(self):
183-
self.run_command("build_regexes")
184-
_build.run(self)
185-
186-
187-
class sdist(_sdist):
188-
sub_commands = _sdist.sub_commands + [("build_regexes", None)]
189-
190-
191-
cmdclass = {
192-
"sdist": sdist,
193-
"develop": develop,
194-
"build": build,
195-
"install": install,
196-
"build_regexes": build_regexes,
197-
}
198-
199128

200129
setup(
201-
name="ua-parser",
202-
version="0.16.1",
203-
description="Python port of Browserscope's user agent parser",
204-
author="PBS",
205-
author_email="[email protected]",
206-
packages=find_packages(where="src"),
207-
package_dir={"": "src"},
208-
license="Apache 2.0",
209-
zip_safe=False,
210-
url="https://github.com/ua-parser/uap-python",
211-
include_package_data=True,
212-
setup_requires=["pyyaml"],
213-
install_requires=[],
214-
cmdclass=cmdclass,
215-
classifiers=[
216-
"Development Status :: 4 - Beta",
217-
"Environment :: Web Environment",
218-
"Intended Audience :: Developers",
219-
"Operating System :: OS Independent",
220-
"License :: OSI Approved :: Apache Software License",
221-
"Programming Language :: Python",
222-
"Topic :: Internet :: WWW/HTTP",
223-
"Topic :: Software Development :: Libraries :: Python Modules",
224-
"Programming Language :: Python",
225-
"Programming Language :: Python :: 3.8",
226-
"Programming Language :: Python :: 3.9",
227-
"Programming Language :: Python :: 3.10",
228-
"Programming Language :: Python :: 3.11",
229-
"Programming Language :: Python :: Implementation :: CPython",
230-
"Programming Language :: Python :: Implementation :: PyPy",
231-
],
130+
cmdclass={
131+
"compile-regexes": CompileRegexes,
132+
}
232133
)

src/ua_parser/user_agent_parser.py

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,6 @@
33
import sys
44
import warnings
55

6-
__author__ = "Lindsey Simon <[email protected]>"
7-
86

97
class UserAgentParser(object):
108
def __init__(

tests/test_legacy.py

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
1-
__author__ = "[email protected] (Stephen Lamm)"
2-
31
import logging
42
import os
53
import platform

tox.ini

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
[tox]
22
envlist = py38, py39, py310, py311,
33
pypy3.8, pypy3.9,
4-
docs, flake8, black
4+
flake8, black
55
skipsdist = True
66

77
[testenv]
8-
usedevelop = True
98
deps = -rrequirements_dev.txt
109
commands =
10+
pip install .
1111
pytest -Werror --doctest-glob="*.rst" {posargs}
1212

1313
[testenv:flake8]

0 commit comments

Comments
 (0)