Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 43461c1

Browse files
msullivanmsullivan
committed
Merge https://github.com/mypyc/mypyc into merge-mypyc
The mypyc history was rewritten before this merge to fix up PR and issue references to be cross-repo references. The command was `git filter-branch -f --msg-filter 'sed "s@\(^\|[ (]\)\(#[0-9]\+\)@\1mypyc/mypyc\2@g"'`
2 parents afb1284 + 72eaed6 commit 43461c1

142 files changed

Lines changed: 70851 additions & 32 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,3 +43,12 @@ include/
4343

4444
.tox
4545
pip-wheel-metadata
46+
47+
48+
test_capi
49+
*.o
50+
*.a
51+
test_capi
52+
/.mypyc-flake8-cache.json
53+
/mypyc/lib-rt/build/
54+
/mypyc/lib-rt/*.so

‎.travis.yml‎

Lines changed: 46 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,22 @@ if: tag IS present OR type = pull_request OR ((branch = master OR branch =~ rele
44
language: python
55
# cache package wheels (1 cache per python version)
66
cache: pip
7+
# also cache the directories where we set up our custom pythons in some builds
8+
cache:
9+
directories:
10+
- $HOME/python-debug
11+
# I ran into some issues with this but will investigate again later.
12+
# - $HOME/.pyenv/versions
13+
- $HOME/Library/Caches/pip
14+
715
# newer python versions are available only on xenial (while some older only on trusty) Ubuntu distribution
816
dist: xenial
917

1018
env:
1119
TOXENV=py
1220
EXTRA_ARGS="-n 12"
1321
TEST_MYPYC=0
22+
PYTHON_DEBUG_BUILD=0
1423

1524
jobs:
1625
include:
@@ -22,6 +31,20 @@ jobs:
2231
python: 3.6 # 3.6.3 pip 9.0.1
2332
- name: "run test suite with python 3.7"
2433
python: 3.7 # 3.7.0 pip 10.0.1
34+
- name: "run mypyc runtime tests with python 3.6 debug build"
35+
language: generic
36+
env:
37+
- TOXENV=py36
38+
- PYTHONVERSION=3.6.8
39+
- PYTHON_DEBUG_BUILD=1
40+
- EXTRA_ARGS="-n 12 mypyc/test/test_run.py mypyc/test/test_external.py"
41+
- name: "run mypyc runtime tests with python 3.6 on OS X"
42+
os: osx
43+
osx_image: xcode8.3
44+
language: generic
45+
env:
46+
- PYTHONVERSION=3.6.3
47+
- EXTRA_ARGS="-n 12 mypyc/test/test_run.py mypyc/test/test_external.py"
2548
- name: "run test suite with python 3.7 (compiled with mypyc)"
2649
python: 3.7
2750
env:
@@ -58,11 +81,33 @@ install:
5881
- pip install -U pip setuptools
5982
- pip install -U tox==3.9.0
6083
- tox --notest
84+
6185
# This is a big hack and only works because the layout of our directories
6286
# means that tox picks up the mypy from the source directories instead of
6387
# the version it installed into a venv. This is also *why* we need to do this,
6488
# since if we arranged for tox to build with mypyc, pytest wouldn't use it.
65-
- if [[ $TEST_MYPYC == 1 ]]; then pip install -r mypyc-requirements.txt; CC=clang MYPYC_OPT_LEVEL=0 python3 setup.py --use-mypyc build_ext --inplace; fi
89+
- if [[ $TEST_MYPYC == 1 ]]; then pip install -r test-requirements.txt; CC=clang MYPYC_OPT_LEVEL=0 python3 setup.py --use-mypyc build_ext --inplace; fi
6690

6791
script:
6892
- tox -- $EXTRA_ARGS
93+
94+
# Getting our hands on a debug build or the right OS X build is
95+
# annoying, unfortunately.
96+
before_install: |
97+
set -e
98+
if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then
99+
if [[ $PYTHON_DEBUG_BUILD == 1 ]]; then
100+
PYTHONDIR=~/python-debug/python-$PYTHONVERSION
101+
VENV=$PYTHONDIR/env
102+
misc/build-debug-python.sh $PYTHONVERSION $PYTHONDIR $VENV
103+
source $VENV/bin/activate
104+
fi
105+
elif [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
106+
# Attempt to install, skipping if version already exists.
107+
pyenv install $PYTHONVERSION -s
108+
# Regenerate shims
109+
pyenv rehash
110+
# Manually set pyenv variables per https://pythonhosted.org/CodeChat/.travis.yml.html
111+
export PYENV_VERSION=$PYTHONVERSION
112+
export PATH="/Users/travis/.pyenv/shims:${PATH}"
113+
fi

‎LICENSE‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
1-
Mypy is licensed under the terms of the MIT license, reproduced below.
1+
Mypy (and mypyc) are licensed under the terms of the MIT license, reproduced below.
22

33
= = = = =
44

55
The MIT License
66

7-
Copyright (c) 2015-2016 Jukka Lehtosalo and contributors
7+
Copyright (c) 2015-2019 Jukka Lehtosalo and contributors
88

99
Permission is hereby granted, free of charge, to any person obtaining a
1010
copy of this software and associated documentation files (the "Software"),
@@ -26,8 +26,10 @@ DEALINGS IN THE SOFTWARE.
2626

2727
= = = = =
2828

29-
Portions of mypy are licensed under different licenses. The files
30-
under stdlib-samples are licensed under the PSF 2 License, reproduced below.
29+
Portions of mypy and mypyc are licensed under different licenses. The
30+
files under stdlib-samples as well as the files
31+
mypyc/lib-rt/pythonsupport.h and mypyc/lib-rt/getargs.c are licensed
32+
under the PSF 2 License, reproduced below.
3133

3234
= = = = =
3335

@@ -223,5 +225,3 @@ FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
223225
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
224226
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
225227
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
226-
227-
= = = = =

‎README.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -265,9 +265,10 @@ Compiled version of mypy
265265
------------------------
266266

267267
We have built an compiled version of mypy using the [mypyc
268-
compiler](https://github.com/mypyc/mypyc) for mypy-annotated Python
269-
code. It is approximately 4 times faster than interpreted mypy and is
270-
available (and the default) for 64-bit Windows, macOS, and Linux.
268+
compiler](https://github.com/python/mypy/files/mypyc) for
269+
mypy-annotated Python code. It is approximately 4 times faster than
270+
interpreted mypy and is available (and the default) for 64-bit
271+
Windows, macOS, and Linux.
271272

272273
To install an interpreted mypy instead, use:
273274

‎appveyor.yml‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,14 @@ cache:
33

44
environment:
55
matrix:
6-
76
- PYTHON: "C:\\Python37-x64"
87
PYTHON_VERSION: "3.7.x"
98
PYTHON_ARCH: "64"
9+
EXTRA_ARGS:
10+
- PYTHON: "C:\\Python37"
11+
PYTHON_VERSION: "3.7.x"
12+
PYTHON_ARCH: "32"
13+
EXTRA_ARGS: "mypyc/test/test_run.py mypyc/test/test_external.py"
1014

1115
install:
1216
- "git submodule update --init mypy/typeshed"
@@ -16,7 +20,7 @@ install:
1620
build: off
1721

1822
test_script:
19-
- "%PYTHON%\\python.exe -m tox -e py37"
23+
- "%PYTHON%\\python.exe -m tox -e py37 %EXTRA_ARGS%"
2024

2125
skip_commits:
2226
files:

‎conftest.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,10 @@ def pytest_configure(config):
99
mypy_source_root = os.path.dirname(os.path.abspath(__file__))
1010
if os.getcwd() != mypy_source_root:
1111
os.chdir(mypy_source_root)
12+
13+
14+
# This function name is special to pytest. See
15+
# http://doc.pytest.org/en/latest/writing_plugins.html#initialization-command-line-and-configuration-hooks
16+
def pytest_addoption(parser) -> None:
17+
parser.addoption('--bench', action='store_true', default=False,
18+
help='Enable the benchmark test runs')

‎misc/build-debug-python.sh‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
#!/bin/bash -eux
2+
3+
# Build a debug build of python, install it, and create a venv for it
4+
# This is mainly intended for use in our travis builds but it can work
5+
# locally. (Though it unfortunately uses brew on OS X to deal with openssl
6+
# nonsense.)
7+
# Usage: build-debug-python.sh <version> <install prefix> <venv location>
8+
#
9+
# Running it locally might look something like: mkdir -p ~/tmp/cpython-debug && cd ~/tmp/cpython-debug && ~/src/mypy/misc/build-debug-python.sh 3.6.6 ~/tmp/cpython-debug ~/src/mypy/env-debug
10+
11+
VERSION=$1
12+
PREFIX=$2
13+
VENV=$3
14+
if [[ -f $PREFIX/bin/python3 ]]; then
15+
exit
16+
fi
17+
18+
CPPFLAGS=""
19+
LDFLAGS=""
20+
if [[ $(uname) == Darwin ]]; then
21+
brew install openssl xz
22+
CPPFLAGS="-I$(brew --prefix openssl)/include"
23+
LDFLAGS="-L$(brew --prefix openssl)/lib"
24+
fi
25+
26+
curl -O https://www.python.org/ftp/python/$VERSION/Python-$VERSION.tgz
27+
tar zxf Python-$VERSION.tgz
28+
cd Python-$VERSION
29+
CPPFLAGS="$CPPFLAGS" LDFLAGS="$LDFLAGS" ./configure CFLAGS="-DPy_DEBUG -DPy_TRACE_REFS -DPYMALLOC_DEBUG" --with-pydebug --prefix=$PREFIX
30+
make -j4
31+
make install
32+
$PREFIX/bin/python3 -m pip install virtualenv
33+
$PREFIX/bin/python3 -m virtualenv $VENV
34+
ln -s python3-config $PREFIX/bin/python-config

‎misc/clean-mypyc.sh‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#!/bin/bash
2+
echo "Cleaning C/C++ build artifacts..."
3+
(cd mypyc/lib-rt; make clean)
4+
(cd mypyc/external/googletest/make; make clean)

‎mypyc-requirements.txt‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎mypyc/README.md‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
mypyc: Mypy to Python C Extension Compiler
2+
==========================================
3+
4+
*Mypyc is (mostly) not yet useful for general Python development.*
5+
6+
Mypyc is a compiler that compiles mypy-annotated, statically typed
7+
Python modules into CPython C extensions. Currently our primary focus
8+
is on making mypy faster through compilation -- the default mypy wheels
9+
are compiled with mypyc. Compiled mypy is about 4x faster than
10+
without compilation.
11+
12+
Mypyc compiles what is essentially a Python language variant using "strict"
13+
semantics. This means (among some other things):
14+
15+
* Most type annotations are enforced at runtime (raising ``TypeError`` on mismatch)
16+
17+
* Classes are compiled into extension classes without ``__dict__``
18+
(much, but not quite, like if they used ``__slots__``)
19+
20+
* Monkey patching doesn't work
21+
22+
* Instance attributes won't fall back to class attributes if undefined
23+
24+
* Metaclasses not supported
25+
26+
* Also there are still a bunch of bad bugs and unsupported features :)
27+
28+
Compiled modules can import arbitrary Python modules, and compiled modules
29+
can be used from other Python modules. Typically mypyc is used to only
30+
compile modules that contain performance bottlenecks.
31+
32+
You can run compiled modules also as normal, interpreted Python
33+
modules, since mypyc targets valid Python code. This means that
34+
all Python developer tools and debuggers can be used.
35+
36+
macOS Requirements
37+
------------------
38+
39+
* macOS Sierra or later
40+
41+
* Xcode command line tools
42+
43+
* Python 3.5+ from python.org (other versions are untested)
44+
45+
Linux Requirements
46+
------------------
47+
48+
* A recent enough C/C++ build environment
49+
50+
* Python 3.5+
51+
52+
Windows Requirements
53+
--------------------
54+
55+
* Windows has been tested with Windows 10 and MSVC 2017.
56+
57+
* Python 3.5+
58+
59+
Quick Start for Contributors
60+
----------------------------
61+
62+
First clone the mypy git repository *and git submodules*:
63+
64+
$ git clone --recurse-submodules https://github.com/mypyc/mypy.git
65+
$ cd mypy
66+
67+
Optionally create a virtualenv (recommended):
68+
69+
$ virtualenv -p python3 <directory>
70+
$ source <directory>/bin/activate
71+
72+
Then install the dependencies:
73+
74+
$ python3 -m pip install -r test-requirements.txt
75+
76+
Now you can run the tests:
77+
78+
$ pytest mypyc
79+
80+
Look at the [issue tracker](https://github.com/mypyc/mypyc/issues)
81+
for things to work on. Please express your interest in working on an
82+
issue by adding a comment before doing any significant work, since
83+
development is currently very active and there is real risk of duplicate
84+
work.
85+
86+
Note that the issue tracker is still hosted on the mypyc project, not
87+
with mypy itself.
88+
89+
Documentation
90+
-------------
91+
92+
We have some [developer documentation](doc/dev-intro.md).
93+
94+
Development Status and Roadmap
95+
------------------------------
96+
97+
These are the current planned major milestones:
98+
99+
1. [DONE] Support a smallish but useful Python subset. Focus on compiling
100+
single modules, while the rest of the program is interpreted and does not
101+
need to be type checked.
102+
103+
2. [DONE] Support compiling multiple modules as a single compilation unit (or
104+
dynamic linking of compiled modules). Without this inter-module
105+
calls will use slower Python-level objects, wrapper functions and
106+
Python namespaces.
107+
108+
3. [DONE] Mypyc can compile mypy.
109+
110+
4. [DONE] Optimize some important performance bottlenecks.
111+
112+
5. [PARTIALLY DONE] Generate useful errors for code that uses unsupported Python
113+
features instead of crashing or generating bad code.
114+
115+
6. [DONE] Release a version of mypy that includes a compiled mypy.
116+
117+
7a. More feature/compatibility work. (100% compatibility with Python is distinctly
118+
an anti-goal, but more than we have now is a good idea.)
119+
120+
7b. Support compiling Black, which is a prominent tool that could benefit
121+
and has maintainer buy-in.
122+
(Let us know if you maintain another Python tool or library and are
123+
interested in working with us on this!)
124+
125+
7c. More optimization! Code size reductions in particular are likely to
126+
be valuable and will speed up mypyc compilation.
127+
128+
8. We'll see! Adventure is out there!
129+
130+
Future
131+
------
132+
133+
We have some ideas for
134+
[future improvements and optimizations](doc/future.md).

0 commit comments

Comments
 (0)