Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[extension/encoding/textencoding] Marshaling and Unmarshaling Separator configurations are ignored by the extension #42797

@mevishalr

Description

@mevishalr

Component(s)

extension/encoding/textencoding

What happened?

Description

The text_encoding extension exposes configuration options for marshaling_separator and unmarshaling_separator. However, these settings do not appear to affect the exported output when used in combination with the fileexporter.

On reviewing the implementation in extension/encoding/textencodingextension/extension.go, it seems that the configured values for marshaling_separator and unmarshaling_separator are never applied when creating the textLogCodec instance. As a result, the marshaling_separator always defaults to an empty string, regardless of the user-provided configuration.

In addition, the current logic in MarshalLogs() (extension/encoding/textencodingextension/text.go) appends the separator after every log record. This behavior introduces trailing separators (e.g., empty lines when marshaling_separator="\n") instead of joining records in the same way as a standard delimiter would (e.g., "\n".join(records)).

Steps to Reproduce

  1. Configure a filelog receiver to read logs from a file.
  2. Configure a fileexporter with text_encoding and set marshaling_separator to "\n".
  3. Observe the exported log file.

Expected Result

The following should be the expected log output written by fileexporter.

<ip_address> - - [19/Sep/2025:10:04:06 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:06 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:06 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:06 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:06 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"

Actual Result

  1. Configured marshaling_separator value is ignored resulting in log lines within a ResourceLog getting concatenated together.
<ip_address> - - [19/Sep/2025:10:04:06 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"<ip_address> - - [19/Sep/2025:10:04:06 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:06 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:06 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"<ip_address> - - [19/Sep/2025:10:04:06 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
  1. When configured properly, the separator is appended after each record, resulting in extra blank lines in the output when using newline separator for marshaling.
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"

<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"

<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"

<ip_address> - - [19/Sep/2025:10:04:07 +0000] "POST /api/v1/service/log HTTP/1.1" 200 13 "-" "LuaSocket 3.0.0"
...

Collector version

0.135.0

Environment information

Environment

OS: Debian 11

OpenTelemetry Collector configuration

# Sample Otelcol configs

receivers:
  filelog:
    include:
      - /var/log/nginx/access.log
    start_at: end
    include_file_path: true
    include_file_name: false

    resource:
      environment: development
      component: nginx

    operators:
      - type: move
        from: attributes["log.file.path"]
        to: resource["path"]
      - type: add
        field: resource.log_path
        value: EXPR(join([resource.environment,resource.component], "-") + resource.path)

processors:
  batch:
    send_batch_size: 8192
    timeout: 200ms
    send_batch_max_size: 0

  memory_limiter:
    check_interval: 5s
    limit_percentage: 75
    spike_limit_percentage: 10

exporters:
  file:
    path: /var/log/otelcol/*.log
    rotation:
      max_megabytes: 5000
      max_days: 7
      max_backups: 4
      localtime: false
    encoding: text_encoding/utf8
    group_by:
      enabled: true
      resource_attribute: log_path

extensions:
  text_encoding/utf8:
    encoding: utf8
    marshaling_separator: "\n"

service:
  extensions: [text_encoding/utf8]
  pipelines:
    logs:
      receivers: [filelog]
      processors: [memory_limiter,batch]
      exporters: [file]

Log output

Additional context

Only checked the MarshalLogs() logic. The logic used in UnmarshalLogs() was not checked for correctness.

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions