When loading the olivetti dataset, I got the deprecation warning:

       "sklearn.externals.joblib is deprecated in 0.21 and will be removed "
       "in 0.23. Please import this functionality directly from joblib, "
       "which can be installed with: pip install joblib. If this warning is "
       "raised when loading pickled models, you may need to re-serialize "
       "those models with scikit-learn 0.21+."

It is due that the dataset was downloaded and pickled with an old scikit-learn. I have 2 suggestions:

1. "you may need to re-serialize those models with scikit-learn 0.21+" does not seem enough. We should mention that the models should be serialized with sklearn 0.21+ and joblib directly (externals.joblib is still existing).
2. It might be interesting to detect the case of the dataset. It does not have anything to do with the models and the user might wonder what is going on. If we detect the warning after calling fetch_*** we could first remove the pickle and re-pickle using joblib which will avoid raising the deprecation warning after.