Doc building on PRs do not push to circleci yet

With https://github.com/scikit-learn/scikit-learn/pull/21137 do push to circleci yet.

This is being tested in https://github.com/scikit-learn/scikit-learn/pull/23508 and a possible solution is: https://github.com/scikit-learn/scikit-learn/pull/23508#issuecomment-1143087348