Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Feature Description

In the current embedding server setup, if a client sends a request and then cancels it, tasks that are already queued continue processing without detecting the cancellation. This can lead to inefficiencies and potential server overload.

[Test Case]
During an actual load test, I canceled a request, but the queued requests prior to the cancellation were processed to completion, causing subsequent requests to be delayed.

[After Modification]
During a load test, when a request is cancelled, task processing should be terminated. If a new request is sent immediately afterward, it should be processed without delay.

Motivation

If a client makes a massive number of requests and then disconnects, the server could become paralyzed. I would like to remove tasks from the queue if the associated request is terminated.

Possible Implementation

I don't have experience with configuring servers in C++, so it's difficult for me to suggest implementation details. If you have an idea as to how it can be implemented, please write a detailed description. Feel free to give links to external sources or share visuals that might be helpful to understand the details better.