Thanks to visit codestin.com
Credit goes to github.com

Skip to content

AsyncIterators doesn't respect to AsyncRetry config #863

@keepdying

Description

@keepdying

Environment details

  • OS type and version: Linux
  • Python version: 3.12.3
  • pip version: pip 24.0
  • google-api-core version: 2.28.1

Steps to reproduce

  1. Define an AsyncRetry config
  2. Supply config as arg to streamable rpc call. See the async pubsub example below

Code example

import asyncio
import json

from google import pubsub_v1
from google.api_core import retry

PROJECT_ID = "your-project-id"
SUBSCRIPTION_ID = "your-subscription-id"


async def main():
    async def request_generator():
        yield pubsub_v1.StreamingPullRequest(
            subscription=f"projects/{PROJECT_ID}/subscriptions/{SUBSCRIPTION_ID}",
            max_outstanding_messages=1,
            stream_ack_deadline_seconds=60,
        )

    retry_config = retry.AsyncRetry(
        predicate=retry.if_transient_error,
        initial=1.0,
        maximum=60.0,
        multiplier=2.0,
        timeout=None,
    )

    subscriber = pubsub_v1.SubscriberAsyncClient()
    stream = await subscriber.streaming_pull(requests=request_generator(), retry=retry_config)
    async for response in stream:
        for message in response.received_messages:
            print("Received message: ", json.loads(message.message.data))


if __name__ == "__main__":
    asyncio.run(main())

After a while (~60 secs) stream throws grpc.StatusCode.UNAVAILABLE and exits even though it should be retried,

I believe it is due to google.api_core.grpc_helpers_async._wrap_stream_errors returns an iterator immediately, bypassing google.api_core.retry.retry_unary_async.retry_target wrap completely.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions