-
-
Notifications
You must be signed in to change notification settings - Fork 223
feat(logs): add Buffering and Batching #4310
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@sentry review |
On it! We are reviewing the PR and will provide feedback shortly. |
PR DescriptionThis pull request introduces a batch processing mechanism for Sentry logs. The goal is to improve performance and reduce overhead by collecting log messages and sending them to Sentry in batches, rather than individually. Click to see moreKey Technical Changes
Architecture Decisions
Dependencies and Interactions
Risk Considerations
Notable Implementation Details
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please add a benchmark for this too, we definitely want to get an idea of how this behaves under load. In the tests, even if you're not committing that consider writing something that writes in a tight loop using a few threads (like 1 per CPU core you have) and check how that behaves too.
Add minimal support for Buffering / Batching for Sentry Logs.
Planned follow-up to avoid having high-frequency logging drop other, potentially more important, envelopes: #4306
Changes (Updated):
SentryStructuredLogger
is nowIDisposable
Dispose
d whenHub
isDispose
dFlush
es all logs whenHub
isDispose
dSentryLog
is no longerISentryJsonSerializable
StructuredLog
isISentryJsonSerializable
SentryLog
instances for serializationDefaultSentryStructuredLogger
no longer directly captures and enqueues Log-EnvelopesStructuredLogBatchProcessor
insteadStructuredLogBatchProcessor
Flush
) whenlock
-free solution, based on aCountdownEvent
that only signals/Wait
s whenFlush
ingStructuredLogBatchBuffer
Array
Add
andFlush
operationsScopedCountdownLock
CountdownEvent
1
Enqueue/Add
operationsFlush
is requested), can no longerEnqueue/Add
and canWait
until existingEnqueue/Add
operations have completedFlush
has completed, disengage the Lock to reset the count to the offset of1
to allowEnqueue/Add
operations againStructuredLogBatchProcessor
#skip-changelog