Thanks to visit codestin.com
Credit goes to github.com

Skip to content

BatchHTTPRequest: only add access token header if we have it #434

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

snarfed
Copy link

@snarfed snarfed commented Aug 29, 2017

re-fixes #350. it was originally fixed by c669a3b following discussion in #351 (comment) , but then that fix got lost in the next commit, 282fc88 .

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Aug 29, 2017
@theacodes
Copy link
Contributor

Can you show me a case where this is problematic?

@snarfed snarfed force-pushed the batch_check_access_token branch from d94cb5b to 1919655 Compare August 30, 2017 04:59
re-fixes googleapis#350. it was originally fixed by c669a3b following discussion in googleapis#351 (comment) , but then that fix got lost in the next commit, googleapis@282fc88 .
@snarfed snarfed force-pushed the batch_check_access_token branch from 1919655 to bdf5ea8 Compare August 30, 2017 05:04
@snarfed
Copy link
Author

snarfed commented Aug 30, 2017

sure! my repro is pretty much the same as @mgilson's in his original report in #350:

from apiclient import discovery
import httplib2

service = discovery.build('plus', 'v1')
batch = service.new_batch_http_request()
batch.add(service.comments().list(activityId='123'))

# creds is an instance of oauth2client.client.OAuth2Credentials
http = creds.authorize(httplib2.Http())
batch.execute(http=http)

which results in:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File ".../oauth2client/oauth2client/util.py", line 137, in positional_wrapper
    return wrapped(*args, **kwargs)
  File ".../googleapiclient/http.py", line 1424, in execute
    self._execute(http, self._order, self._requests)
  File ".../googleapiclient/http.py", line 1340, in _execute
    body = self._serialize_request(request)
  File ".../googleapiclient/http.py", line 1211, in _serialize_request
    _auth.apply_credentials(credentials, headers)
  File ".../googleapiclient/_auth.py", line 113, in apply_credentials
    return credentials.apply(headers)
  File ".../oauth2client/oauth2client/client.py", line 621, in apply
    headers['Authorization'] = 'Bearer ' + self.access_token
TypeError: cannot concatenate 'str' and 'NoneType' objects

with this PR, it runs fine.

now, to get it to pass the tests... :P

@mgilson
Copy link

mgilson commented Aug 30, 2017

I don't know if it's helpful -- I haven't looked at this for quite a while, but way-way back I put together a branch/fork that seemed to work. You can see my changes: https://github.com/mgilson/google-api-python-client/compare/master...mgilson:app-default-creds?expand=1

Obviously don't trust that this actually works :-).

@snarfed
Copy link
Author

snarfed commented Aug 30, 2017

@mgilson it does work! i actually ran that patch in production for months, until just yesterday, and based this PR on it. thank you!

snarfed added a commit to snarfed/bridgy that referenced this pull request Sep 1, 2017
@dtheodor
Copy link

dtheodor commented Jan 3, 2018

Can you get this merged to finally fix the TypeError: cannot concatenate 'str' and 'NoneType' objects travesty?

@theacodes
Copy link
Contributor

I don't think this PR covers the case where google_auth is used as the auth library instead of oauth2client, so I need to do some more investigation to see why this is occurring and how to properly fix it.

@theacodes
Copy link
Contributor

Okay, getting to the source of this issue it's this:

  1. batch.execute() allows you to specify an http object that will be used in place of all of the request's http objects.
  2. When batch is serializing each request, the credentials attached to the http object for each request are preserved.
  3. The http object for each request is never preserved. If no http argument is passed to batch.execute, it just finds the first valid http client in the list of requests.

Because of this behavior, when you have the following situation:

  1. You are using application default credentials for discovery.build
  2. You are using different credentials in your batch.execute.

Then the batch._serialize_request tries to use the discovery.build credentials for each request instead of the credentials attached to to the batch.execute call.

I can remedy this by making batch always use the credentials specified by the http passed to execute and ignore the credentials attached the the request's http object. However, this prevents having different credentials for each request in a batch. I don't know if anyone is depending on that "feature", but I need to do some more investigation before I can commit one way or the other.

@theacodes
Copy link
Contributor

I've decided I'm going to resolve this by making apply_credentials will proactively refresh the credentials if needed and will treat a access_token of None as a credential that needs refresh.

This does not in itself fix the example given. You will need to do one more thing to make this work as expected which is explicitly tell discovery.build not to use default credentials by specifying http=httplib2.Http() to the constructor.

PR incoming, but let me know if anyone disagrees with this.

@snarfed
Copy link
Author

snarfed commented Jan 17, 2018

thank you for working on this @jonparrott! that approach sgtm. i can easily add http=httplib2.Http() to my discovery.build. i'll happily try out your PR when it's pushed and see if it works for me.

@theacodes
Copy link
Contributor

From my testing doing that alone fixes this in the batch.execute(http) case, so feel free to go ahead and try it (PR incoming, though)

theacodes pushed a commit that referenced this pull request Jan 17, 2018
…es_token as invalid.

Closes #350, Supersedes #434

Note: This change reveals surprising behavior between default credentials and batches. If you allow `googleapiclient.discovery.build` to use default credentials *and* specify different credentials by providing `batch.execut()` with an explicit `http` argument, your individual requests will use the default credentials and *not* the credentials specified to the bathc http. To avoid this, tell `build` explicitly not to use default credentials by specifying `build(..., http=httplib2.Http()`.

For context, see:
#434 (comment)
@theacodes theacodes closed this Jan 17, 2018
snarfed added a commit to snarfed/oauth-dropins that referenced this pull request Jan 18, 2018
theacodes pushed a commit that referenced this pull request Jan 18, 2018
…es_token as invalid. (#469)

Closes #350, Supersedes #434

Note: This change reveals surprising behavior between default credentials and batches. If you allow `googleapiclient.discovery.build` to use default credentials *and* specify different credentials by providing `batch.execut()` with an explicit `http` argument, your individual requests will use the default credentials and *not* the credentials specified to the bathc http. To avoid this, tell `build` explicitly not to use default credentials by specifying `build(..., http=httplib2.Http()`.

For context, see:
#434 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TypeError when trying to issue BatchHttpRequest when service was built using application default credentials
5 participants