dbt: fix log path, more precise file reading #3793
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
3a61382 to
ee1d103
Compare
c1d1abb to
d1783a3
Compare
Signed-off-by: Maciej Obuchowski <[email protected]>
d1783a3 to
7cb28c8
Compare
|
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #3793 +/- ##
==========================================
- Coverage 85.37% 85.10% -0.27%
==========================================
Files 57 57
Lines 3705 3739 +34
==========================================
+ Hits 3163 3182 +19
- Misses 542 557 +15 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
kacpermuda
reviewed
Jun 24, 2025
kacpermuda
approved these changes
Jun 24, 2025
marccampa
pushed a commit
to marccampa/OpenLineage-Collibra
that referenced
this pull request
Jun 26, 2025
Signed-off-by: Maciej Obuchowski <[email protected]> Signed-off-by: marccampa <[email protected]>
mobuchowski
added a commit
that referenced
this pull request
Jul 3, 2025
* Update consumers.tsx * Add files via upload * Update consumers.tsx * Add files via upload * Update consumers.tsx * Update consumers.tsx * Update consumers.tsx * Update consumers.tsx * Delete Collibra-Logo-RGB.png * Update consumers.tsx * [Flink] Do not hide OpenLineage config parsing errors (#3724) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Update consumers.tsx Signed-off-by: marccampa <[email protected]> * Add files via upload Signed-off-by: marccampa <[email protected]> * Update consumers.tsx Signed-off-by: marccampa <[email protected]> * Add files via upload Signed-off-by: marccampa <[email protected]> * Update consumers.tsx Signed-off-by: marccampa <[email protected]> * [DBT] Add processing_engine facet (#3725) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * java: prevent original events from being mutated in TransformTransport (#3728) - Add deepCopy utility method to OpenLineageClientUtils for safe object cloning - Modify TransformTransport to create deep copies of events before transformation Signed-off-by: Jakub Dardzinski <[email protected]> Signed-off-by: marccampa <[email protected]> * [Flink] Add processing_engine facet (#3726) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Add Github stars statistics to Readme (#3730) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * [DBT] Document supported adapters (#3729) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Prettify Spark JSON event examples (#3740) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Prettify Flink JSON event examples (#3742) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Prettify Airflow JSON event examples (#3741) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Prettify DBT JSON event examples (#3743) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Restyle cardmedia. (#3733) Signed-off-by: merobi-hub <[email protected]> Signed-off-by: marccampa <[email protected]> * dbt-ol should not error on job complete if there is no start event (#3749) Signed-off-by: Maciej Obuchowski <[email protected]> Signed-off-by: marccampa <[email protected]> * [Flink] Add facet with Flink jobId (#3744) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Update consumers.tsx Signed-off-by: marccampa <[email protected]> * Update consumers.tsx Signed-off-by: marccampa <[email protected]> * [DBT] Initial support for Clickhouse (#3739) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * [SPEC] Add contentType to documentation facet (#3748) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * [spark] Update Spark 4 dependency to 4.0.0 (remove -preview1 suffix) (#3751) Signed-off-by: Dominik Dębowczyk <[email protected]> Signed-off-by: marccampa <[email protected]> * filter temp inner jobs for bigquery indirect mode (#3722) Signed-off-by: Pawel Leszczynski <[email protected]> Signed-off-by: marccampa <[email protected]> * [Docs] Add documentation for some facets (#3752) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Tweak the Maven signing config (#3069) This tweak allows Gradle to default on using values set in `~/.gradle/gradle.properties` Signed-off-by: Julien Phalip <[email protected]> Signed-off-by: marccampa <[email protected]> * Run prettier on .json files (#3750) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * remove native proxy (#3680) * remove native proxy Signed-off-by: Maciej Obuchowski <[email protected]> # Conflicts: # proxy/backend/gradle.properties * remove leftover proxy gradle reference Signed-off-by: Kacper Muda <[email protected]> --------- Signed-off-by: Kacper Muda <[email protected]> Co-authored-by: Kacper Muda <[email protected]> Signed-off-by: marccampa <[email protected]> * [DBT] Add DbtRun facet (#3738) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Remove Airflow < 2.5.0 support (#3669) Signed-off-by: Kacper Muda <[email protected]> Signed-off-by: marccampa <[email protected]> * nit: fix supported airflow versions (#3755) Signed-off-by: Kacper Muda <[email protected]> Signed-off-by: marccampa <[email protected]> * [Java] Speedup generateNewUUID (#3754) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * [DBT] Use adapter rows_affected as outputStatistics (#3731) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * fix variables in docs for setting of root parents in spark config (#3761) Signed-off-by: Humzah Kiani <[email protected]> Signed-off-by: marccampa <[email protected]> * [spark] Add support for Big Query Metastore catalog type (#3760) Signed-off-by: Dominik Dębowczyk <[email protected]> Signed-off-by: marccampa <[email protected]> * Fix visibility of GcpLineageTransportConfig.Mode (#3762) * Register GCP common job facet Signed-off-by: Natalia Gorchakova <[email protected]> * Add ACCEPT_CASE_INSENSITIVE_ENUMS for ObjectMapper to ensure that lower and upper case enum values are accepted for config Signed-off-by: Natalia Gorchakova <[email protected]> * Add ACCEPT_CASE_INSENSITIVE_ENUMS for ObjectMapper to ensure that lower and upper case enum values are accepted for config Signed-off-by: Natalia Gorchakova <[email protected]> * Add ACCEPT_CASE_INSENSITIVE_ENUMS for ObjectMapper to ensure that lower and upper case enum values are accepted for config Signed-off-by: Natalia Gorchakova <[email protected]> * Add ACCEPT_CASE_INSENSITIVE_ENUMS for ObjectMapper to ensure that lower and upper case enum values are accepted for config Signed-off-by: Natalia Gorchakova <[email protected]> --------- Signed-off-by: Natalia Gorchakova <[email protected]> Signed-off-by: marccampa <[email protected]> * Update consumers.tsx Signed-off-by: marccampa <[email protected]> * Delete Collibra-Logo-RGB.png Signed-off-by: marccampa <[email protected]> * update httpConfig Headers and TimeoutInMillis property values (#3767) Signed-off-by: Nidhin Varghese <[email protected]> Signed-off-by: marccampa <[email protected]> * [java] Add log if load from yaml fails (#3766) Signed-off-by: Fiore Mario Vitale <[email protected]> Signed-off-by: marccampa <[email protected]> * smart debug facet (#3715) Signed-off-by: Pawel Leszczynski <[email protected]> Signed-off-by: marccampa <[email protected]> * [Spark] Fix missing table path in InsertIntoHadoopFsRelationCommand (#3773) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Github: mark Hive PRs with proper label (#3778) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * fix configurable test failin in CI (#3782) Signed-off-by: Pawel Leszczynski <[email protected]> Signed-off-by: marccampa <[email protected]> * Column level lineage for jdbc queries load (#3763) * test column level lineage for jdbc queries load Signed-off-by: Pawel Leszczynski <[email protected]> * refactor jdbc lineage visitor Signed-off-by: Pawel Leszczynski <[email protected]> --------- Signed-off-by: Pawel Leszczynski <[email protected]> Signed-off-by: marccampa <[email protected]> * chore: Use attr.define instead of attr.s (#3776) Signed-off-by: Kacper Muda <[email protected]> Signed-off-by: marccampa <[email protected]> * [Hive] Add job sql facet (#3777) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * build(deps): bump the integration-sql group (#3704) Updates the requirements on [pyo3](https://github.com/pyo3/pyo3) and [pyo3-build-config](https://github.com/pyo3/pyo3) to permit the latest version. Updates `pyo3` to 0.25.0 - [Release notes](https://github.com/pyo3/pyo3/releases) - [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md) - [Commits](PyO3/pyo3@v0.24.0...v0.25.0) Updates `pyo3-build-config` to 0.25.0 - [Release notes](https://github.com/pyo3/pyo3/releases) - [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md) - [Commits](PyO3/pyo3@v0.24.0...v0.25.0) --- updated-dependencies: - dependency-name: pyo3 dependency-version: 0.25.0 dependency-type: direct:production dependency-group: integration-sql - dependency-name: pyo3-build-config dependency-version: 0.25.0 dependency-type: direct:production dependency-group: integration-sql ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: marccampa <[email protected]> * [Hive] Add hive_query facet (#3781) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Website: correct Node version in README (#3783) * Fix node version in website readme. Signed-off-by: merobi-hub <[email protected]> * Misc fixes. Signed-off-by: merobi-hub <[email protected]> --------- Signed-off-by: merobi-hub <[email protected]> Signed-off-by: marccampa <[email protected]> * [spark] Disable module metadata file generation (#3785) Signed-off-by: Dominik Dębowczyk <[email protected]> Signed-off-by: marccampa <[email protected]> * Add Debezium to producers (#3787) Signed-off-by: Fiore Mario Vitale <[email protected]> Signed-off-by: marccampa <[email protected]> * build(deps): bump requests from 2.32.0 to 2.32.4 in /dev (#3759) Bumps [requests](https://github.com/psf/requests) from 2.32.0 to 2.32.4. - [Release notes](https://github.com/psf/requests/releases) - [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md) - [Commits](psf/requests@v2.32.0...v2.32.4) --- updated-dependencies: - dependency-name: requests dependency-version: 2.32.4 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: marccampa <[email protected]> * [Hive] Add hive_session facet (#3786) * [Hive] Add hive_session facet Signed-off-by: Martynov Maxim <[email protected]> * [Hive] Record hive session creation time Signed-off-by: Martynov Maxim <[email protected]> --------- Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * changelog for release 1.34.0 (#3790) Signed-off-by: Maciej Obuchowski <[email protected]> Signed-off-by: marccampa <[email protected]> * Prepare for release 1.34.0 Signed-off-by: Maciej Obuchowski <[email protected]> Signed-off-by: marccampa <[email protected]> * Prepare next development version 1.35.0-SNAPSHOT Signed-off-by: Maciej Obuchowski <[email protected]> Signed-off-by: marccampa <[email protected]> * chore: Fix changelog item authors (#3791) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * [Hive] Add jobType facet (#3789) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Website: update README (#3801) * Update website readme. Signed-off-by: merobi-hub <[email protected]> * Fix code blocks. Signed-off-by: merobi-hub <[email protected]> * Fix wordiness. Signed-off-by: merobi-hub <[email protected]> * More details in deployment sec. Signed-off-by: merobi-hub <[email protected]> * Continued. Signed-off-by: merobi-hub <[email protected]> * Continued. Signed-off-by: merobi-hub <[email protected]> * Continued. Signed-off-by: merobi-hub <[email protected]> --------- Signed-off-by: merobi-hub <[email protected]> Signed-off-by: marccampa <[email protected]> * fix spotless in hive integration (#3806) Signed-off-by: Maciej Obuchowski <[email protected]> Signed-off-by: marccampa <[email protected]> * run Java SQL tests (#3808) Signed-off-by: Maciej Obuchowski <[email protected]> Signed-off-by: marccampa <[email protected]> * [Hive] Add docker-compose example for local testing (#3800) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * [DBT] Make invocation_id field optional (#3796) Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> * Remove empty Flink page. (#3810) Signed-off-by: Jakub Dardzinski <[email protected]> Signed-off-by: marccampa <[email protected]> * Flink integration: Fixed a bug incorrectly loading configuration in Event Emitter (#3799) * Flink integration: Fixed a bug incorrectly loading configuration in Event Emitter, resulting in "disabled facets" feature not working (and probably others as well). Signed-off-by: Jan Siekierski <[email protected]> * Flink integration: Fixed a bug incorrectly loading configuration in Event Emitter, resulting in "disabled facets" feature not working (and probably others as well). Signed-off-by: Jan Siekierski <[email protected]> --------- Co-authored-by: Jan Siekierski <[email protected]> Signed-off-by: marccampa <[email protected]> * Website: add missing guidance to readme (#3807) * Add missing guidance to readme. Signed-off-by: merobi-hub <[email protected]> * Img file formats. Signed-off-by: merobi-hub <[email protected]> --------- Signed-off-by: merobi-hub <[email protected]> Signed-off-by: marccampa <[email protected]> * build(deps): bump urllib3 from 1.26.19 to 2.5.0 in /dev (#3794) Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.19 to 2.5.0. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](urllib3/urllib3@1.26.19...2.5.0) --- updated-dependencies: - dependency-name: urllib3 dependency-version: 2.5.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: marccampa <[email protected]> * dbt: fix log path, more precise file reading (#3793) Signed-off-by: Maciej Obuchowski <[email protected]> Signed-off-by: marccampa <[email protected]> * Spark: fix & upgrade databricks test (#3811) Signed-off-by: Pawel Leszczynski <[email protected]> Signed-off-by: marccampa <[email protected]> * Formalize dataset naming (#3775) * Formalize dataset naming --------- Signed-off-by: Dominik Dębowczyk <[email protected]> Signed-off-by: marccampa <[email protected]> * Update consumers.tsx Signed-off-by: marccampa <[email protected]> * Apply prettier fix. Signed-off-by: merobi-hub <[email protected]> --------- Signed-off-by: Martynov Maxim <[email protected]> Signed-off-by: marccampa <[email protected]> Signed-off-by: Jakub Dardzinski <[email protected]> Signed-off-by: merobi-hub <[email protected]> Signed-off-by: Maciej Obuchowski <[email protected]> Signed-off-by: Dominik Dębowczyk <[email protected]> Signed-off-by: Pawel Leszczynski <[email protected]> Signed-off-by: Julien Phalip <[email protected]> Signed-off-by: Kacper Muda <[email protected]> Signed-off-by: Humzah Kiani <[email protected]> Signed-off-by: Natalia Gorchakova <[email protected]> Signed-off-by: Nidhin Varghese <[email protected]> Signed-off-by: Fiore Mario Vitale <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Maxim Martynov <[email protected]> Co-authored-by: Jakub Dardzinski <[email protected]> Co-authored-by: Michael Robinson <[email protected]> Co-authored-by: Maciej Obuchowski <[email protected]> Co-authored-by: ddebowczyk92 <[email protected]> Co-authored-by: pawel.leszczynski <[email protected]> Co-authored-by: Julien Phalip <[email protected]> Co-authored-by: Kacper Muda <[email protected]> Co-authored-by: Humzah Kiani <[email protected]> Co-authored-by: ngorchakova <[email protected]> Co-authored-by: Nidhin Varghese <[email protected]> Co-authored-by: Fiore Mario Vitale <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Fiore Mario Vitale <[email protected]> Co-authored-by: Maciej Obuchowski <[email protected]> Co-authored-by: Jan Siekierski <[email protected]> Co-authored-by: Jan Siekierski <[email protected]> Co-authored-by: merobi-hub <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes issue where dbt-ol read incorrect structured log file, reads log file more precisely, and adds better debug logging.