We should probably move to pip install -v -e . as Nathaniel advocates?

Can you point me towards that discussing? I missed it.

Most of my discussions with @njsmith about this have been in person.

Do you mind explain the logic? It is not a standard way to setup a work environment and the command is unclear and confusing (considering the main use of pip is to download and install from pypi, not local source code).
I've just checked quickly (numpy, scipy, sklearn, etc) and none suggest doing so.

pip install -e just calls setup.py develop, so it doesn't matter much in practice right now. But the general movement is that setup.py is being deprecated and should always be called through pip, because in the hopefully-not-too-distant future pip will know how to invoke non-distutils-based build systems.

As a side note I hesitate to recommend setup.py develop or pip install -e because they kinda inherently screw up your virtualenv (you end up with skew between what pip thinks is installed, what python files you actually have installed, and what compiled files you have installed), but these kinds of installs are really popular anyway so I guess this is a losing battle...

you end up with skew between what pip thinks is installed, what python files you actually have installed, and what compiled files you have installed

What do you mean? An editable install only places a *.egg-link into site-packages, and that's all that it knows is installed, which seems to be fully consistent.

@QuLogic: Try e.g. checking out the matplotlib 1.5 branch, doing an editable install, and then after that switch to the master branch for more hacking: git checkout master. Now pip thinks your virtualenv has 1.5 (check out the metadata files in the .egg-info directory -- these only get regenerated if you re-do the editable install), but it actually has a mix of master's .py files and 1.5's extension modules. Rebuild the extension modules alone, and now you have a consistent installation of master (2.x) but pip still thinks it's 1.5. Then pip install some package that depends on matplotlib (>=2.0) and pip will freak out and try to uninstall your dev version, because it isn't good enough, even though it is, etc. You can make it work, but it's intrinsically flaky and you have to be constantly aware of all the weird pitfalls.

@NelleV:

I used to reinstall matplotlib every time, but it has become a pain now that the git hash is appended to the version.

I assume you're referring to the thing where pip refuses to install "matplotlib 2.1.dev1+git123" if you already have "matplotlib 2.1.dev1+git456"? Yeah :-(. This is an incredibly frustrating bug that will eventually be fixed :-(. (The pip maintainers at least have finally come around to consensus that it is a bug that needs fixing, but unfortunately the patch got tangled up in the bikeshed around fixing pip install -U and is temporarily stalled out.)

but it actually has a mix of master's .py files and 1.5's extension modules.

Yes, it fails to load then, and as noted in another PR, you need to re-install to update modules whose source files have changed.

Rebuild the extension modules alone

How would you only rebuild the modules? With setup.py build_ext? But then you've side-stepped pip anyway.

@QuLogic: I'm not saying you can't make it work if you do the right thing. But in general the only wholly reliable way to make it work is to re-run the editable install after every time you change the source; then there are a bunch of special cases where it also works. If you're willing to pay the cognitive overhead to learn and keep track of all those special cases, that's cool, I'm not going to stop you :-). I'm just explaining why I hesitate to recommend it. If you don't want to keep track of that stuff and want to be confident things work reliably, then you'll be re-running the install after every edit so you might as well use a regular install instead of an editable install and avoid the traps entirely :-). (Modulo hopefully-fixed-soon issue @NelleV points to where pip install . often fails for no good reason when pip install -e . succeeds.)

(You can also imagine in-between things, like a script that continuously runs pip install . every time you change a file.)

A vast majority of the work that needs to be done on mpl is in the pure-python layers, so a source install can be very convenient.

Add a note here that make clear changing branches is changing the c-code?

Make this stronger?

I made it slightly stronger.
If I recall correctly, matplotlib and virtualenv don't play well together, and I've ran in so much problems with matplotlib and conda that I gave up installing any packages from conda recently.