Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feature: Add plugin name or meltano as another column in the console-formatted output #9586

@edgarrmondragon

Description

@edgarrmondragon

Feature scope

CLI (options, error messages, logging, etc.)

Description

Right now, when both extractor and loader emit SDK structured logs, the output looks something like this:

2025-10-18T01:41:34.050193Z [info     ] Environment 'dev' is active
2025-10-18T01:41:34.137423Z [info     ] Running job with name 'dev:tap-fedidb-to-target-csv' and run ID '0199f4fa-b10e-7a29-bbe7-e819be8d2220'
2025-10-18T01:41:34.172081Z [info     ] Using systemdb state backend
2025-10-18T01:41:34.172644Z [warning  ] No state was found, complete import.
2025-10-18T01:41:34.308572Z [info     ] tap-fedidb v0.5.0, Meltano SDK v0.50.2 name=tap-fedidb
2025-10-18T01:41:34.308905Z [info     ] Skipping parse of env var settings... name=tap-fedidb
2025-10-18T01:41:34.309466Z [info     ] Beginning full_table sync of 'popular_accounts' name=tap-fedidb
2025-10-18T01:41:34.785773Z [info     ] METRIC                         metric_name=http_request_duration metric_value=0.468653 name=tap-fedidb
2025-10-18T01:41:34.798617Z [info     ] METRIC                         metric_name=http_request_count metric_value=1 name=tap-fedidb
2025-10-18T01:41:34.798801Z [info     ] METRIC                         metric_name=sync_duration metric_value=0.48907017707824707 name=tap-fedidb
2025-10-18T01:41:34.798890Z [info     ] METRIC                         metric_name=record_count metric_value=136 name=tap-fedidb
2025-10-18T01:41:34.798951Z [info     ] Skipping deselected stream 'servers'. name=tap-fedidb
2025-10-18T01:41:34.799008Z [info     ] Beginning full_table sync of 'software' name=tap-fedidb
2025-10-18T01:41:35.014505Z [info     ] METRIC                         metric_name=http_request_duration metric_value=0.214582 name=tap-fedidb
2025-10-18T01:41:35.023827Z [info     ] METRIC                         metric_name=http_request_count metric_value=1 name=tap-fedidb
2025-10-18T01:41:35.023996Z [info     ] METRIC                         metric_name=sync_duration metric_value=0.22496819496154785 name=tap-fedidb
2025-10-18T01:41:35.024072Z [info     ] METRIC                         metric_name=record_count metric_value=63 name=tap-fedidb
2025-10-18T01:41:35.554208Z [info     ] target-csv v0.2.1, Meltano SDK v0.50.2 name=target-csv
2025-10-18T01:41:35.554431Z [info     ] Skipping parse of env var settings... name=target-csv
2025-10-18T01:41:35.558373Z [info     ] Writing to destination file '/Users/edgarramirez/Code/meltano/triage/_console/popular_accounts.csv'... name=target-csv
2025-10-18T01:41:35.570079Z [info     ] Writing to destination file '/Users/edgarramirez/Code/meltano/triage/_console/software.csv'... name=target-csv
2025-10-18T01:41:35.573459Z [info     ] Reader 'target-csv' completed processing 203 lines of input (2 schemas, 199 records, 0 batch manifests, 2 state messages, 0 activate version messages). name=target-csv
2025-10-18T01:41:35.578880Z [info     ] Appending 136 records to file... name=target-csv
2025-10-18T01:41:35.579005Z [info     ] Appending 63 records to file... name=target-csv
2025-10-18T01:41:35.581862Z [info     ] METRIC                         metric_name=batch_processing_time metric_value=0.0028040409088134766 name=target-csv
2025-10-18T01:41:35.582075Z [info     ] METRIC                         metric_name=batch_processing_time metric_value=0.0033750534057617188 name=target-csv
2025-10-18T01:41:35.591689Z [info     ] METRIC                         metric_name=record_count metric_value=136 name=target-csv
2025-10-18T01:41:35.591811Z [info     ] METRIC                         metric_name=record_count metric_value=63 name=target-csv
2025-10-18T01:41:35.596854Z [info     ] Using systemdb state backend
2025-10-18T01:41:35.598817Z [info     ] Incremental state has been updated at 2025-10-18 01:41:35.598806+00:00.
2025-10-18T01:41:35.629481Z [info     ] Block run completed
2025-10-18T01:41:35.629633Z [info     ] Run completed

notice that the structured plugin log gets a name=<plugin name> that's displayed at the rightmost.

An arguable better experience would be for the plugin name (when available via structured logging) to be displayed in its own "column", similar to how the timestamp or the log level are displayed:

2025-10-18T01:41:34.050193Z [info     ] meltano     Environment 'dev' is active
2025-10-18T01:41:34.137423Z [info     ] meltano     Running job with name 'dev:tap-fedidb-to-target-csv' and run ID '0199f4fa-b10e-7a29-bbe7-e819be8d2220'
2025-10-18T01:41:34.172081Z [info     ] meltano     Using systemdb state backend
2025-10-18T01:41:34.172644Z [warning  ] meltano     No state was found, complete import.
2025-10-18T01:41:34.308572Z [info     ] tap-fedidb  v0.5.0, Meltano SDK v0.50.2
2025-10-18T01:41:34.309466Z [info     ] tap-fedidb  Beginning full_table sync of 'popular_accounts'
2025-10-18T01:41:34.798951Z [info     ] tap-fedidb  Skipping deselected stream 'servers'.
2025-10-18T01:41:34.799008Z [info     ] tap-fedidb  Beginning full_table sync of 'software'
2025-10-18T01:41:35.554208Z [info     ] target-csv v0.2.1, Meltano SDK v0.50.2
2025-10-18T01:41:35.578880Z [info     ] target-csv  Appending 136 records to file...
2025-10-18T01:41:35.579005Z [info     ] target-csv  Appending 63 records to file...
2025-10-18T01:41:35.629633Z [info     ] meltano     Run completed

Fortunately, structlog supports this via passing custom Column instances to ConsoleRenderer.

Related

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions