While reviewing issue #8379 I noticed this statement in the docstring explaining the return value from permutation_test_score:

If scoring is rather a loss function (i.e. when lower is better such as with mean_squared_error) then this is actually the complement of the p-value: 1 - p-value.

If I understand correctly that's not quite true. permutation_test_score returns:

pvalue = (np.sum(permutation_scores >= score) + 1.0) / (n_permutations + 1)

But one minus this value is:

pvalue = np.sum(permutation_scores < score) / (n_permutations + 1)

which is not the same as the value obtained with the appropriate comparison for a scoring function where lower is better:

pvalue = (np.sum(permutation_scores <= score) + 1.0) / (n_permutations + 1)

In particular, one minus this value can be 0.0, while the minimum p-value is 1 / ( n_permutations + 1 ) as was discussed in issue #8379.