spark: serialize event to json for logging #4029
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: EugenYushin <[email protected]>
|
Thanks for opening your first OpenLineage pull request! We appreciate your contribution. If you haven't already, please make sure you've reviewed our guide for new contributors (https://github.com/OpenLineage/OpenLineage/blob/main/CONTRIBUTING.md). |
Signed-off-by: EugenYushin <[email protected]>
Contributor
Author
|
@collado-mike could you please check this tiny "improvement" PR? |
dolfinus
reviewed
Sep 11, 2025
| .build()); | ||
|
|
||
| log.debug("Posting event for start {}: {}", executionId, event); | ||
| log.debug("Posting event for start {}: {}", executionId, OpenLineageClientUtils.toJson(event)); |
Contributor
There was a problem hiding this comment.
I'd recommend to wrap this with a check for current log level, to avoid spending CPU time if nobody will see this message anyway:
https://docs.oracle.com/javase/8/docs/api/java/util/logging/Logger.html#isLoggable-java.util.logging.Level-
Contributor
Author
There was a problem hiding this comment.
Great call! I've wrapped calls to logger with log.isDebugEnabled(), same way we do in the rest of the codebase
Signed-off-by: EugenYushin <[email protected]>
dolfinus
approved these changes
Sep 12, 2025
mobuchowski
approved these changes
Sep 12, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Events appear in logs as Java objects refs instead of their serialized json representation:
Solution
Wrap logger calls with
OpenLineageClientUtils.toJson.One-line summary:
Checklist
SPDX-License-Identifier: Apache-2.0
Copyright 2018-2025 contributors to the OpenLineage project