Thanks to visit codestin.com
Credit goes to github.com

Skip to content

SNS Time of sending a message to the topic depends on the number of queues subscribing to this topic #2928

Closed
@a-wi

Description

@a-wi

Type of request: This is a ...

[x] bug report
[ ] feature request

Detailed description

Time of sending a message to the SNS topic with AWS SDK .NET PublishAsync() depends (~linearly) on the number of the queues subscribing to the topic. This dependency is seen only with LocalStack. For AWS service time of executing PublishAsync() is independent on the number of queues (is more less constant).

Expected behavior

Time of sending a message to the SNS topic should be independent on the number the queues subscribing to the topic (should be constant).

Actual behavior

While sending a message to the SNS topic with AWS SDK for .NET it can be observed that time of executing PublishAsync() API depends (linearly) on the number of queues subscribing to the topic. This linear dependency is seen only with LocalStack.
Here are times of sending a message measured in the sample application creating a SNS topic, subscribing queues to this topic and next sending a message:
Num. queues = 0 Publishing time = 25 ms
Num. queues = 1 Publishing time = 45 ms
Num. queues = 2 Publishing time = 77 ms
Num. queues = 3 Publishing time = 103 ms
Num. queues = 4 Publishing time = 142 ms
Num. queues = 5 Publishing time = 171 ms
Num. queues = 6 Publishing time = 184 ms
Num. queues = 7 Publishing time = 225 ms
Num. queues = 8 Publishing time = 244 ms
Num. queues = 9 Publishing time = 318 ms
Num. queues = 10 Publishing time = 308 ms
Num. queues = 11 Publishing time = 441 ms
Num. queues = 12 Publishing time = 400 ms
Num. queues = 13 Publishing time = 402 ms
Num. queues = 14 Publishing time = 434 ms
Num. queues = 15 Publishing time = 460 ms
Num. queues = 16 Publishing time = 482 ms
Num. queues = 17 Publishing time = 517 ms
Num. queues = 18 Publishing time = 549 ms
Num. queues = 19 Publishing time = 544 ms
Num. queues = 20 Publishing time = 587 ms

Steps to reproduce

This issue can be reproduced with the following steps (code snippet attached):

  • Create a SNS topic.
  • Create N queues and subscribe them the topic.
  • Send a message to the topic with PublishAsync(). Measure the time of executing this API function.

LocalStack 0.11.4
AWSSDK.SimpleNotificationService 3.3.102.17
AWSSDK.SQS 3.3.103.26
Target framework .NET Core 3.1

Command used to start LocalStack

docker run -p 4566:4566 -p 8080:8080 --rm -e SERVICES=sqs,sns -e DEBUG=1 -e DATA_DIR=/tmp/localstack/data localstack/localstack:0.11.4

Client code (AWS SDK code snippet, or sequence of "awslocal" commands)

static async Task TestPublishing()
{
    const int maxNumQueues = 20;

    AmazonSQSConfig sqsConfig = new AmazonSQSConfig
    {
        ServiceURL = "http://localhost:4566"
    };
    AmazonSimpleNotificationServiceConfig snsConfig = new AmazonSimpleNotificationServiceConfig
    {
        ServiceURL = "http://localhost:4566"
    };

    using (AmazonSQSClient SQS = new AmazonSQSClient("ignore", "ignore", sqsConfig))
    using (AmazonSimpleNotificationServiceClient SNS = new AmazonSimpleNotificationServiceClient("ignore", "ignore", snsConfig))
    {
        // Create a topic.
        CreateTopicResponse createTopicResp = await SNS.CreateTopicAsync("testTopic");

        for (int numQueues = 0; numQueues <= maxNumQueues; numQueues++)
        {
            // Create and subscribe queues to the topic.
            List<string> queues = new List<string>();
            List<string> arns = new List<string>();
            for (int i = 0; i < numQueues; i++)
            {
                string queueName = string.Format("testQueue_{0}", Guid.NewGuid().ToString());
                CreateQueueResponse createQueueResp = await SQS.CreateQueueAsync(queueName);
                queues.Add(createQueueResp.QueueUrl);

                // It looks with LocalStack special permissions are necessary.
                if (SQS.Config.RegionEndpoint == null)
                {
                     await SQS.AddPermissionAsync(createQueueResp.QueueUrl, "", new List<string> { "*" }, new List<string> { "SendMessage" });

                     int retryCount = 0;
                     while (true)
                     {
                        try
                        {
                            string subscriptionArn = await SNS.SubscribeQueueAsync(createTopicResp.TopicArn, SQS, createQueueResp.QueueUrl);
                            arns.Add(subscriptionArn);
                            break;
                        }
                        catch (Exception ex) when (ex is ThirdParty.Json.LitJson.JsonException)
                        {
                            if (retryCount++ == 10)
                                throw;

                            await Task.Delay(1000);
                        }
                     }
                }
                else
                {
                    string subscriptionArn = await SNS.SubscribeQueueAsync(createTopicResp.TopicArn, SQS, createQueueResp.QueueUrl);
                    arns.Add(subscriptionArn);
                }
            }

            // Publish a message.
            string msg = new string('x', 100);
            DateTime tm0 = DateTime.Now;
            PublishResponse pubResp = await SNS.PublishAsync(createTopicResp.TopicArn, msg);
            DateTime tm1 = DateTime.Now;
            Console.WriteLine("Num. queues = {0,2}  Publishing time = {1:F0} ms.", numQueues, (tm1 - tm0).TotalMilliseconds);

            // Cleanup subscriptions and queues.
            foreach (string arn in arns)
            {
                await SNS.UnsubscribeAsync(arn);
            }
            foreach (string url in queues)
            {
                await SQS.DeleteQueueAsync(url);
            }
        }
    }
}

┆Issue is synchronized with this Jira Bug by Unito

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions