I've never been able to run the make flake8-diff command on MacOS. Does it require to install the flake8-diff Python package?

No it shouldn't. It calls scikit-learn/build_tools/circle/linting.sh which does a number of things to ensure the diff is taken against upstream/master. I agree that recommending,

git diff upstream/master -u -- "*.py" | flake8 --diff

as pandas does is likely preferable and is cross-plaftorm unlike our script. Although it assumes that contributors have registered the upstream remote. One solution is also to recommend that when cloning same as done in pandas guide.

+1 to change the docs.

Registering the upstream remote is mentioned above:

6. Add the upstream remote. This saves a reference to the main scikit-learn repository, which you can use to keep your repository synchronized with the latest changes:
   $ git remote add upstream https://github.com/scikit-learn/scikit-learn.git

And it's an important step if you want to open more than one PR (even for a single PR to solve eventual merge conflicts if the PR takes a while?).

Waiting for another point of view and opening a PR if there is an agreement :)

yeah we do have it in the docs, but not maybe in the most clear way. Open to suggestions.

@rth we should also probably only run the "check unused improts" only on the changed files? Now the linting.sh takes quite a while to finish.

we should also probably only run the "check unused improts" only on the changed files? Now the linting.sh takes quite a while to finish.

Running it on the diff wouldn't work, because some imports not touched by the diff get unused as a result of the diff. There is also mypy which would add 30s or so https://github.com/scikit-learn/scikit-learn/ . There are likely workarounds to make that CI faster.

@johannfaouzi feel free to open a PR to improve it.

then I guess I wouldn't even mind having the git diff upstream/master -u -- "*.py" | flake8 --diff in the makefile somewhere.

Note that makefile is not cross-plaform, so we shouldn't recommend using makefile in the contribution docs IMO. A lot of potential contributors are on windows.

I agree we shouldn't recommend using it for windows users, but it doesn't mean we shouldn't have it for our own sake. We could have it but not recommend it. Although I kinda find it somewhat deeply problematic or ironic that for a lot of our contributing guides it's not what we actually do. I guess we don't have the bandwidth to do it, but it'd be nice to have a contributing guide for everybody, and one for people with a POSIX system.