Codestin Search App

thaJeztah · 2026-02-20T20:35:05Z

This adds support for streaming logs as JSON stream either through an "Accept" header ("application/jsonl", "application/x-ndjson", or "application/json-seq"), or through a format=json query variable.

By default log messages are included as UTF-8 encoded strings, which should work for regular logs, but can be lossy. An "encoding=base64" query parameter is added to switch the response to return the message in base64 encoding.

docker rm -fv my-container3
docker run \
    -qit \
    --name my-container3 \
    --label=labelA=foo \
    --label=labelB=bar \
    --log-opt labels=labelA,labelB \
    ubuntu bash

Inside the container:

root@b36bfcd439e9:/# echo {1..9} && exit
1 2 3 4 5 6 7 8 9
exit

With the message encoded as utf-8;

curl -s --unix-socket /var/run/docker.sock 'http://localhost/v1.53/containers/my-container3/logs?stdout=1&stderr=1&format=json' | jq .
{
  "Line": "\u001b[?2004h\u001b]0;root@1065b85dd167: /\u0007root@1065b85dd167:/# \u001b[7mecho {1..9} && exit\u001b[27m\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\becho {1..9} && exit\r\n",
  "Source": "stdout",
  "Timestamp": "2026-02-20T15:39:01.083531419Z"
}
{
  "Line": "\u001b[?2004l\r1 2 3 4 5 6 7 8 9\r\n",
  "Source": "stdout",
  "Timestamp": "2026-02-20T15:39:01.088152377Z"
}
{
  "Line": "exit\r\n",
  "Source": "stdout",
  "Timestamp": "2026-02-20T15:39:01.088183336Z"
}

With base64 encoding for the message (non-lossy)

curl -s --unix-socket /var/run/docker.sock 'http://localhost/v1.53/containers/my-container3/logs?stdout=1&stderr=1&format=json&encoding=base64' | jq .
{
  "LineEncoded": "G1s/MjAwNGgbXTA7cm9vdEAxMDY1Yjg1ZGQxNjc6IC8Hcm9vdEAxMDY1Yjg1ZGQxNjc6LyMgG1s3bWVjaG8gezEuLjl9ICYmIGV4aXQbWzI3bQgICAgICAgICAgICAgICAgICAhlY2hvIHsxLi45fSAmJiBleGl0DQo=",
  "Source": "stdout",
  "Timestamp": "2026-02-20T15:39:01.083531419Z",
}
{
  "LineEncoded": "G1s/MjAwNGwNMSAyIDMgNCA1IDYgNyA4IDkNCg==",
  "Source": "stdout",
  "Timestamp": "2026-02-20T15:39:01.088152377Z",
}
{
  "LineEncoded": "ZXhpdA0K",
  "Source": "stdout",
  "Timestamp": "2026-02-20T15:39:01.088183336Z",
}

With details enabled;

curl -s --unix-socket /var/run/docker.sock 'http://localhost/v1.53/containers/my-container3/logs?stdout=1&stderr=1&format=json&details=1' | jq .
{
  "Line": "\u001b[?2004h\u001b]0;root@1065b85dd167: /\u0007root@1065b85dd167:/# \u001b[7mecho {1..9} && exit\u001b[27m\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\becho {1..9} && exit\r\n",
  "Source": "stdout",
  "Timestamp": "2026-02-20T15:39:01.083531419Z",
  "Attrs": [{"Key": "labelA","Value": "foo"},{"Key": "labelB","Value": "bar"}]
}
{
  "Line": "\u001b[?2004l\r1 2 3 4 5 6 7 8 9\r\n",
  "Source": "stdout",
  "Timestamp": "2026-02-20T15:39:01.088152377Z",
  "Attrs": [{"Key": "labelA","Value": "foo"},{"Key": "labelB","Value": "bar"}]
}
{
  "Line": "exit\r\n",
  "Source": "stdout",
  "Timestamp": "2026-02-20T15:39:01.088183336Z",
  "Attrs": [{"Key": "labelA","Value": "foo"},{"Key": "labelB","Value": "bar"}]
}

Using Accept header instead of the format=json query parameter;

curl -s -H 'Accept: application/json-seq' --unix-socket /var/run/docker.sock 'http://localhost/v1.53/containers/my-container3/logs?stdout=1&stderr=1' | jq .
{
  "Line": "\u001b[?2004h\u001b]0;root@1065b85dd167: /\u0007root@1065b85dd167:/# \u001b[7mecho {1..9} && exit\u001b[27m\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\becho {1..9} && exit\r\n",
  "Source": "stdout",
  "Timestamp": "2026-02-20T15:39:01.083531419Z"
}
{
  "Line": "\u001b[?2004l\r1 2 3 4 5 6 7 8 9\r\n",
  "Source": "stdout",
  "Timestamp": "2026-02-20T15:39:01.088152377Z"
}
{
  "Line": "exit\r\n",
  "Source": "stdout",
  "Timestamp": "2026-02-20T15:39:01.088183336Z"
}

Metadata about partial messages is included (except for the json-file logging
driver, which doesn't use it currently);

The partial messages also don't get a trailing newline, except for the last entry;

docker run --name bar alpine sh -c 'head -c 20000 /dev/zero | tr "\0" A; echo'
curl -s --unix-socket /var/run/docker.sock 'http://localhost/v1.53/containers/bar/logs?stdout=1&stderr=1&format=json'
{"Line":"AAAAAAAAAA....AAAA","Source":"stdout","Timestamp":"2026-02-21T21:45:47.075093919Z"}
{"Line":"AAAAAAAAAAAAAAAA\n","Source":"stdout","Timestamp":"2026-02-21T21:45:47.075093919Z"}

And for logging drivers other than the json-file, we have information about
partials in a slightly more structured format:

docker run --name baz --log-driver=local alpine sh -c 'head -c 20000 /dev/zero | tr "\0" A; echo'
curl -s --unix-socket /var/run/docker.sock 'http://localhost/v1.53/containers/baz/logs?stdout=1&stderr=1&format=json'
{"Line":"AAAAAAAAAA....ASAA","Source":"stdout","Timestamp":"2026-02-21T21:49:42.453363584Z","LogMetaData":{"Last":false,"ID":"46a9188a552a068c1d63a5096b7335788aea7e430817f3d2d83c5fd90c6365ef","Ordinal":1}}
{"Line":"AAAAAAAAAAAAAAAA\n","Source":"stdout","Timestamp":"2026-02-21T21:49:42.453363584Z","LogMetaData":{"Last":true,"ID":"46a9188a552a068c1d63a5096b7335788aea7e430817f3d2d83c5fd90c6365ef","Ordinal":2}}

- What I did

- How I did it

- How to verify it

- Human readable description for the release notes

- A picture of a cute animal (not mandatory but encouraged)

thaJeztah · 2026-02-20T20:38:48Z

Ah, derp; messed up extracting from another branch; fixing

thaJeztah · 2026-02-21T22:06:18Z

daemon/server/httputils/logstream/json_logstream.go

+	// LogMetaData contains metadata for partial log records that must be
+	// reassembled by the client.
+	LogMetaData *backend.PartialLogMetaData `json:"PLogMetaData,omitempty"`


I found PLogMetaData a bit awkward name, so picked LogMetadata here, but perhaps we could even inline / embed the struct (or its fields).

For JSON-file, we should consider if we want the logging driver itself to also use the structured metadata, or otherwise backfill the info when producing (check for trailing \n to see if it's a "partial" or "full" message).

thaJeztah · 2026-02-21T22:10:37Z

daemon/server/httputils/logstream/json_logstream.go

+	// Line contains the log payload as UTF-8 text when text encoding is used.
+	// When an alternate encoding is requested, this field is omitted.
+	Line string `json:"Line,omitempty"`
+
+	// LineEncoded contains the raw log payload encoded using the selected
+	// non-text encoding (currently base64). Exactly one of Line or
+	// LineEncoded is present.
+	LineEncoded string `json:"LineEncoded,omitempty"`


Looking at this feature (encoding); the JSON-file logging driver appears to just marshal as a string, so ... that would mean it's already lossy?

tail /var/lib/docker/containers/1065b85dd167dcca420d186abba394f8dcf44ede78247d314b93fab1a7fbeaef/1065b85dd167dcca420d186abba394f8dcf44ede78247d314b93fab1a7fbeaef-json.log {"log":"\u001b[?2004h\u001b]0;root@1065b85dd167: /\u0007root@1065b85dd167:/# \u001b[7mecho {1..9} \u0026\u0026 exit\u001b[27m\u0008\u0008\u0008\u0008\u0008\u0008\u0008\u0008\u0008\u0008\u0008\u0008\u0008\u0008\u0008\u0008\u0008\u0008\u0008echo {1..9} \u0026\u0026 exit\r\n","stream":"stdout","attrs":{"labelA":"foo","labelB":"bar"},"time":"2026-02-20T15:39:01.083531419Z"} {"log":"\u001b[?2004l\r1 2 3 4 5 6 7 8 9\r\n","stream":"stdout","attrs":{"labelA":"foo","labelB":"bar"},"time":"2026-02-20T15:39:01.088152377Z"} {"log":"exit\r\n","stream":"stdout","attrs":{"labelA":"foo","labelB":"bar"},"time":"2026-02-20T15:39:01.088183336Z"} {"log":"\u001b[?2004h\u001b]0;root@1065b85dd167: /\u0007root@1065b85dd167:/# ","stream":"stdout","attrs":{"labelA":"foo","labelB":"bar"},"time":"2026-02-21T13:20:21.087924553Z"}

This adds support for streaming logs as JSON stream either through an "Accept" header ("application/jsonl", "application/x-ndjson", or "application/json-seq"), or through a `format=json` query variable. By default log messages are included as UTF-8 encoded strings, which should work for regular logs, but can be lossy. An "encoding=base64" query parameter is added to switch the response to return the message in base64 encoding. ```bash docker rm -fv my-container3 docker run \ -qit \ --name my-container3 \ --label=labelA=foo \ --label=labelB=bar \ --log-opt labels=labelA,labelB \ ubuntu bash ``` Inside the container: ```bash root@b36bfcd439e9:/# echo {1..9} && exit 1 2 3 4 5 6 7 8 9 exit ``` With the message encoded as utf-8; ```bash curl -s --unix-socket /var/run/docker.sock 'http://localhost/v1.53/containers/my-container3/logs?stdout=1&stderr=1&format=json' | jq . { "Line": "\u001b[?2004h\u001b]0;root@1065b85dd167: /\u0007root@1065b85dd167:/# \u001b[7mecho {1..9} && exit\u001b[27m\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\becho {1..9} && exit\r\n", "Source": "stdout", "Timestamp": "2026-02-20T15:39:01.083531419Z" } { "Line": "\u001b[?2004l\r1 2 3 4 5 6 7 8 9\r\n", "Source": "stdout", "Timestamp": "2026-02-20T15:39:01.088152377Z" } { "Line": "exit\r\n", "Source": "stdout", "Timestamp": "2026-02-20T15:39:01.088183336Z" } ``` With `base64` encoding for the message (non-lossy) ```bash curl -s --unix-socket /var/run/docker.sock 'http://localhost/v1.53/containers/my-container3/logs?stdout=1&stderr=1&format=json&encoding=base64' | jq . { "LineEncoded": "G1s/MjAwNGgbXTA7cm9vdEAxMDY1Yjg1ZGQxNjc6IC8Hcm9vdEAxMDY1Yjg1ZGQxNjc6LyMgG1s3bWVjaG8gezEuLjl9ICYmIGV4aXQbWzI3bQgICAgICAgICAgICAgICAgICAhlY2hvIHsxLi45fSAmJiBleGl0DQo=", "Source": "stdout", "Timestamp": "2026-02-20T15:39:01.083531419Z", } { "LineEncoded": "G1s/MjAwNGwNMSAyIDMgNCA1IDYgNyA4IDkNCg==", "Source": "stdout", "Timestamp": "2026-02-20T15:39:01.088152377Z", } { "LineEncoded": "ZXhpdA0K", "Source": "stdout", "Timestamp": "2026-02-20T15:39:01.088183336Z", } ``` With `details` enabled; ```bash curl -s --unix-socket /var/run/docker.sock 'http://localhost/v1.53/containers/my-container3/logs?stdout=1&stderr=1&format=json&details=1' | jq . { "Line": "\u001b[?2004h\u001b]0;root@1065b85dd167: /\u0007root@1065b85dd167:/# \u001b[7mecho {1..9} && exit\u001b[27m\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\becho {1..9} && exit\r\n", "Source": "stdout", "Timestamp": "2026-02-20T15:39:01.083531419Z", "Attrs": [{"Key": "labelA","Value": "foo"},{"Key": "labelB","Value": "bar"}] } { "Line": "\u001b[?2004l\r1 2 3 4 5 6 7 8 9\r\n", "Source": "stdout", "Timestamp": "2026-02-20T15:39:01.088152377Z", "Attrs": [{"Key": "labelA","Value": "foo"},{"Key": "labelB","Value": "bar"}] } { "Line": "exit\r\n", "Source": "stdout", "Timestamp": "2026-02-20T15:39:01.088183336Z", "Attrs": [{"Key": "labelA","Value": "foo"},{"Key": "labelB","Value": "bar"}] } ``` Using `Accept` header instead of the `format=json` query parameter; ```bash curl -s -H 'Accept: application/json-seq' --unix-socket /var/run/docker.sock 'http://localhost/v1.53/containers/my-container3/logs?stdout=1&stderr=1' | jq . { "Line": "\u001b[?2004h\u001b]0;root@1065b85dd167: /\u0007root@1065b85dd167:/# \u001b[7mecho {1..9} && exit\u001b[27m\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\becho {1..9} && exit\r\n", "Source": "stdout", "Timestamp": "2026-02-20T15:39:01.083531419Z" } { "Line": "\u001b[?2004l\r1 2 3 4 5 6 7 8 9\r\n", "Source": "stdout", "Timestamp": "2026-02-20T15:39:01.088152377Z" } { "Line": "exit\r\n", "Source": "stdout", "Timestamp": "2026-02-20T15:39:01.088183336Z" } ``` Metadata about partial messages is included (except for the json-file logging driver, which doesn't use it currently); The partial messages also don't get a trailing newline, except for the last entry; ```bash docker run --name bar alpine sh -c 'head -c 20000 /dev/zero | tr "\0" A; echo' curl -s --unix-socket /var/run/docker.sock 'http://localhost/v1.53/containers/bar/logs?stdout=1&stderr=1&format=json' {"Line":"AAAAAAAAAA....AAAA","Source":"stdout","Timestamp":"2026-02-21T21:45:47.075093919Z"} {"Line":"AAAAAAAAAAAAAAAA\n","Source":"stdout","Timestamp":"2026-02-21T21:45:47.075093919Z"} ``` And for logging drivers other than the `json-file`, we have information about partials in a slightly more structured format: ```bash docker run --name baz --log-driver=local alpine sh -c 'head -c 20000 /dev/zero | tr "\0" A; echo' curl -s --unix-socket /var/run/docker.sock 'http://localhost/v1.53/containers/baz/logs?stdout=1&stderr=1&format=json' {"Line":"AAAAAAAAAA....ASAA","Source":"stdout","Timestamp":"2026-02-21T21:49:42.453363584Z","LogMetaData":{"Last":false,"ID":"46a9188a552a068c1d63a5096b7335788aea7e430817f3d2d83c5fd90c6365ef","Ordinal":1}} {"Line":"AAAAAAAAAAAAAAAA\n","Source":"stdout","Timestamp":"2026-02-21T21:49:42.453363584Z","LogMetaData":{"Last":true,"ID":"46a9188a552a068c1d63a5096b7335788aea7e430817f3d2d83c5fd90c6365ef","Ordinal":2}} ``` Signed-off-by: Sebastiaan van Stijn <[email protected]>

github-actions bot added area/logging area/daemon Core Engine labels Feb 20, 2026

thaJeztah force-pushed the json_logs branch from b2c9302 to a28595f Compare February 20, 2026 20:40

thaJeztah changed the title ~~[WIP] container/logs: add support for JSON streams~~ (WIP) container/logs: add support for JSON streams Feb 20, 2026

thaJeztah force-pushed the json_logs branch from a28595f to 3b3ed18 Compare February 21, 2026 21:32

thaJeztah commented Feb 21, 2026

View reviewed changes

thaJeztah force-pushed the json_logs branch from 3b3ed18 to be96dba Compare February 23, 2026 12:21

github-actions bot added the area/testing label Feb 23, 2026

thaJeztah force-pushed the json_logs branch 3 times, most recently from f4ec27c to 9217011 Compare February 24, 2026 16:24

thaJeztah force-pushed the json_logs branch from 9217011 to 13dcf57 Compare February 24, 2026 20:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(WIP) container/logs: add support for JSON streams#52073

(WIP) container/logs: add support for JSON streams#52073
thaJeztah wants to merge 1 commit intomoby:masterfrom
thaJeztah:json_logs

thaJeztah commented Feb 20, 2026 •

edited

Loading

Uh oh!

thaJeztah commented Feb 20, 2026

Uh oh!

thaJeztah Feb 21, 2026

Uh oh!

thaJeztah Feb 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

thaJeztah commented Feb 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thaJeztah commented Feb 20, 2026

Uh oh!

thaJeztah Feb 21, 2026

Choose a reason for hiding this comment

Uh oh!

thaJeztah Feb 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

thaJeztah commented Feb 20, 2026 •

edited

Loading