v3.6.3

Compare Source

Bug Fixes

• Do not override S3 region if already specified in configuration chain (backport release-3.6.x) (#​20128) (38582ac)
• Fix regression in S3 client configuration (backport release-3.6.x) (#​20118) (7d8176f)

v3.6.2

Compare Source

Bug Fixes

• deps: Loki 3.6.1 CVE updates (#​20014) (2f85998)

v3.6.1

Compare Source

Bug Fixes

• docker: missing permissions to start docker (backport release-3.6.x) (#​19948) (48b507f)
• docker: set WORKDIR to root in loki Dockerfiles (#​19952) (6040a09)

v3.6.0

Compare Source

Features

• Add method to get zone-aware partition consumers (#​17377) (4a733fc)
• analytics: Change how analytics proxy works (#​19098)
• API: Add endpoint returning tenant configured limits (#​17101) (6818c90)
• API: Loki Improvement Document (LID) for query splitting (#​16958) (66f0f0b)
• ci: Add workflow_dispatch for querytee-images workflow (#​19541)
• compactor: Compactor deletion manifest builder (#​17474) (47961f8)
• compactor HS: Add a dashboard for monitoring deletion with horizontally scalable compactor (#​18588) (10db8ab)
• compactor HS: Add a job runner for processing of deletion jobs (#​18058)
• compactor HS: Add job queue and deletion job builder (#​17843)
  (#​17843)
• compactor HS: Add jsonnet for deploying horizontally scalable compactor (#​18550) (5e28950)
• compactor HS: Add support for applying storage updates after finishing processing of deletion manifest (#​18294) (ba3c670)
• compactor HS: Add support for worker for processing of jobs from the compactor's job queue (#​18165) (d05c4bc)
• compactor HS: Fixes and changes for getting horizontally scalable compactor to work well (#​18526) (000f90d)
• compactor HS: Refactor interfaces and build a link between delete requests manager and compactor for HS of compactor (#​18326) (3646607)
• compactor HS: Store index updates per processed segment in the manifest to the object storage (#​18212) (8007e22)
• compactor HS: tune compactor worker config and add experimental feature warning to the docs (#​18539) (5bf3f74)
• compactor HS: Use streaming RPCs for interacting with job queue over grpc (#​18125) (385e317)
• compactor HS: Wire things up and add some metrics (#​18421) (bd29320)
• compactor HS: Add a metric to track the number of jobs left to conclude ongoing unit of work (#​19506)
• dataobj executor: project node (#​17312) (395030b)
• dataobj, executor: filter node execution (#​17327) (cf54d50)
• data-objects: Add query stats collection to the dataobj readers (#​17128) (9e68fb0)
• data-objects: Add p50, p95 and p99 section sizes per tenant (#​19054)
• data-objects: Add p50, p95 and p99 sections per tenant to stats (#​19044)
• data-objects: Add tenant to section builders (#​18864)
• data-objects: Add tenant to section metadata (#​18843)
• data-objects: build multi-tenant data objects (#​19034)
• data-objects: decode tenant when reading streams and logs sections (#​18846)
• data-objects: print the tenant in the dump and stats sub-commands (#​19033)
• data-objects: improve dataobj-inspect command (#​18618) (238e0b7)
• data-objects: Initial index-builder implementation (#​18297) (1f0edcd)
• distributor: Add MaxRecvMsgSize config for uncompressed message size limits (#​16915) (1d99f4d)
• distributor: log ingestion policy when log_push_request is set (#​18712) (0ec787f)
• distributor: Mark truncated log lines with identifier (#​18262) (0ee8e76)
• distributor: stop pushing to the metadata topic from distributors (#​17525) (e0f015c)
• distributor: remove tenant topic tee (#​19082)
• distributor: check for failed reason in distributors (#​18128) (104c457)
• distributor: Add agent IPs to distributor log (#​17011) (7fc9026)
• distributor: Add enforceLimits to ingest_limits.go (#​17117) (d197cda)
• distributor: Add metric to track ingest limits failures in distributor (#​17015) (7f1a1c7)
• distributor: Add tests for enforcing limits in distributors (#​17124) (69aeda1)
• engine: Implements predicate mapping for DataObjScan (#​17621) (cbb5be6)
• engine, dataobj: logql bench wiring for new engine [non-fork] (#​17627) (ccb65f9)
• engine: Add json parsing stage to v2 engine (#​19240)
• engine: Add logfmt parse support to the v2 query engine (#​18931)
• engine: More aggregations for the new query engine (#​19350)
• engine: Prefetch inputs of Merge pipeline concurrently (#​19039)
• fluentbit: Update fluent/fluent-bit to v4 which adds support for structured metadata to the out_loki plugin (#​17200) (f108e1c)
• fluentd: Add 'compress gzip' setting to fluentd plugin (#​18995)
• GEL: Remove tokengen job and kubectl dependency. (#​18803)
• goldfish: improve query performance with indexes and partitions (#​19023)
• goldfish: Add user and query engine to Goldfish UI (#​18801)
• goldfish: capture logs drilldown flag in query samples (#​19242)
• goldfish: Add traceId support to goldfish (#​18681) (2f0dc74)
• goldfish: Add traceId to Goldfish UI (#​18726) (4fabb8d)
• goldfish: Goldfish UI (#​17983) (462acb5)
• goldfish: Goldfish UI Improvements (#​18504) (b14a282)
• helm: Add block_builder config to helm chart (#​17451) (a9808a1)
• helm: Add startupProbe to memcached(#​19166)
• helm: Add startupProbe to memcached exporter(#​19165)
• helm: Add configurable livenessProbe to single binary StatefulSet(#​19072)
• helm: Add dnsconfig to all missing resources(#​18903)
• helm: Add FOLDER_ANNOTATION logic for sidecar container in loki helm chart, making multi-tenant alerting easier (#​13289) (833fcbc)
• helm: Add health probes to admin-api (#​18546) (358228a)
• helm: Add health probes to bloom-builder (#​18549)
• helm: Add health probes to bloom-gateway(#​18551)
• helm: Add health probes to bloom-planner(#​18552)
• helm: Add health probes to gateway component(#​18545)
• helm: Add health probes to loki in backend(#​18548)
• helm: Add location snippet to nginx config (#​18105) (5737751)
• helm: Add missing revisionHistoryLimit (#​18638) (d0e0d1a)
• helm: Add more specificity to affinity and topology spread constraints(#​19061)
• helm: Add namespaceOverride(#​18707)
• helm: Add non-default-values.yaml to increase test coverage (#​18360) (b0e04f4)
• helm: Add option to use deployment type instead of daemonset in canary (#​17364) (a6563cf)
• helm: Add PDB for patternIngester (#​17058) (7fad38d)
• helm: Add readiness and liveness support for all memcached components (#​18525) (9b54d9f)
• helm: Add readiness probe for memcached (#​15609) (6ed1461)
• helm: Add replicas to loki-canary deployment (#​19095)
• helm: Add startupProbe to loki-sc-rules (#​18547) (121e84e)
• helm: Add support for custom alert rule annotations (#​19193)
• helm: Add tpl support for pattern_ingester and ingester_client (#​16759) (901b2b0)
• helm: Added trafficDistribution for service-distributor (#​17742) (78f154f)
• helm: Adds a flag for tokengen rbac components so that they can be disabled optionally (#​15882) (10536b2)
• helm: Allow configuring service types (#​18708)
• helm: Allow enabling user namespaces (#​18661)
• helm: Allow external memcached setup (#​17432) (013aa17)
• helm: Allow extraObjects to be defined as either a list or dictionary.(#​13252)
  (#​13252)
• helm: Allow passing tenant password hash instead of password (#​17049) (20b97f7)
• helm: Allow setting labels for ingesters (#​18536) (ab426be)
• helm: Enable rules sidecar for the ruler pods (#​17231) (2c5ad02)
• helm: Expose all storage configuration (#​18509) (db5e021)
• helm: Expose labels on volumeClaimTemplates (#​13317) (cdefbbf)
• helm: Expose Loki UI in pure ingress (#​18400) (21f3d09)
• helm: Helm issue templates (#​18693) (e0ec815)
• helm: Implementing support of l2 chunksCache memcache (#​17556) (039e7c1)
• helm: Integrate the enterprise-provisioner into Loki helm chart (#​17616)
• helm: Make access modes for persistence on all PVCs and StatefulSets editable (#​13474)
• helm: Only delete PVC on statefulset deletion, not scaling (#​19097)
• helm: Support bypass all s3/gcs/azure config in object_store (#​17597) (d6c2a8e)
• helm: Support separate loki ruler storage configuration (#​18510)
• helm: Support templated loki.operational_config (#​17045) (078f353)
  to 3.5.3 (#​18619) (907bb61)
• helm: Update Loki version in charts (#​18447) (e2770ae)
• helm: Use named ports for memcached services targetports (#​18990)
• helm: delete statefulset if volumeClaimTemplates changes (#​19217)
• helm: Added affinity property to the loki-canary deamonset (#​17720)
• ingester: Add ingester_chunks_flush_requests_total (#​18696)
• ingester: per policy overridable stream limits (#​18994)
• ingest-limits: Re-implement ExceedsLimits in ingest-limits-frontend (#​17576) (7b88a02)
• ingest-limits: Add exceeds limits RPC with no-op implementation (#​17546) (cd5e7df)
• ingest-limits: Add partition id caching (#​16964) (54ff5dc)
• ingest-limits: Append stream metadata records to Kafka-based WAL (#​17602) (932f39b)
• ingest-limits: Enforce synchronously max stream limit per partition (#​17527) (c55f038)
• ingest-limits: Support disabling the partition consumers cache (#​17318) (1d68047)
• kafka: Support multiple Kafka clients in component (#​17757)
• kafka: Adds partition-ingester push latency histogram (#​17385) (01cf36d)
• kafka: Optionally enable Kafka histograms for read/write latency (#​17089) (e962398)
• kafka: Refactors kafka/client to improve metric collection (#​17308) (ab65ac5)
• kafka: Support consuming records from Kafka (#​17691)
• limits: Add ability to filter tenant limits endpoint by allowlist (#​18521) (73de7b5)
• limits: Add a fast path for consuming records from Kafka (#​17858)
• limits: Add metric that tracks unanswered streams (#​17966)
• limits: limits-frontend failover to other zones (#​17408)
• limits: optimize rate buckets to avoid re-allocating slice (#​17904)
• limits: quick fix to produce a stream once per minute (#​17996)
• logcli: Add delete commands to logcli (#​18375) (f38950a)
• logcli: Add --log.level parameter to help with diagnosing errors when using logcli. (#​19136)
• logs drilldown: Return metric query partial results for drilldown (#​18286) (c0c2265)
• logs drilldown: Drilldown config endpoint (#​19028)
• logs drilldown: Implement unwrap as a projection
  (#​19409)
• logs drilldown: persist patterns as aggregated metrics (#​17737)
• logs drilldown: query persisted patterns (#​17980)
• loki-canary: Update loki canary to support batched log push (#​17558)
• metrics: Adds distributor lag counter to push.go (#​18012)
• metrics: Adds User-Agent to distributor push logging (#​17764)
• mixins: Allow disabling Thanos dashboard and fix dashboard's file name (#​18749)
• operator: Add alert LokiIngesterFlushFailureRateCritical (#​18698)
• operator: Add warning alert for when LokiStack is not getting ready (#​19258)
• operator: Allow users to configure virtual-host-style for S3 secrets (#​17363) (c4dace5)
• operator: Allow using resource requests as limits (#​17484) (0b2bae8)
• operator: deploy network policies with LokiStack (#​19099)
• operator: extend LokiStack authorization to support OTel Semantics (#​16031)
• operator: s3 validation reject endpoints that contain a URL path (#​19356)
• operator: Update Loki operand to v3.5.5 (#​19187)
• operator: Update Loki operand to v3.4.3 (#​17847)
• operator: Update Loki operand to v3.5.4 (#​19122)
• policies: Default ingestion policy mappings merged with per-tenant mappings (#​18926)
• pattern ingester: Pattern ingester pushes detected level as structured metadata (#​16924) (66033bb)
• pattern ingester: implement volume-based pattern filtering (#​18838)
• planning: Adds physical and logical nodes for range aggregation (#​17974)
• testing: Add ReasonFailed (#​18055) (b15d85d)
• testing return ReasonFailed from ingest-limits frontend (#​18123) (c42ccc3)
• thanos: Allow dashes in storage_prefix config (#​16934) (c01d9c7)
• tools: Querytee Goldfish (#​17959) (7775a41)
• tools: Add stream generator (#​17214) (af97644)
• ui: Add ingest limits rings (#​16978) (1df66cc)

Fixes

• aggregated metrics: Hide __aggregated_metric__ in /series and /labels. This will alleviate confusion for people using a tool like logcli series --analyze-labels for cardinality analysis. (#​14677) (4b0aaaa)
• build: Udpate purgo package (#​17807)
• build: remove busybox from dockerfile (#​19502)
• build: remove UI from docker build (#​19425)
• build: rpm signature config (#​19476)
• build: Update ckit (#​18997)
• ci: Fix deprecation errors from staticcheck linter (#​17210) (cfe3675)
• ci: Fix lambda-promtail image build (#​18443) (8f0ba2e)
• ci: Fix workflow timeout added in wrong format (#​18348) (53ba9c7)
• ci: Read releaseLibRef from jsonnet.json lockfile (#​18538) (26417b5)
• chunkenc: avoid memory allocations when loading symbolizer from a flushed chunk (#​17953)
• compactor: Return request ID in response header (#​19444)
• compactor: Concurrent map access for processedSeries in delete requests manager (#​17469) (32c5088)
• compactor: Fix timeout and series progress marker for same requests with different shards (#​17125) (288ec8c)
• compactor: Move the check for duplicate series up the call stack to fix an issue when retention is enabled (#​17663) (a5e95ba)
• compactor: Wire up deletion series progress tracking (#​17099) (db59d0e)
• compactor: Do not try to merge the already consolidated delete requests while listing them (#​18544) (8054076)
• compactor: Ensure to return delete requests only for the requested tenant (#​18589) (cdb190b)
• compactor: Fix handling of duplicate delete requests (#​18460) (255fce2)
• compactor: Fix lint failure in retention.go (#​17894)
• compactor: Use the custom objectExists function to work around bug in Thanos object client (#​18820)
• compactor: Fix panic while applying storage updates when a whole chunk is deleted by line filters (#​19262)
• compactor: set delete request batch to empty when we load to fail requests (#​17810)
• config_wrapper: apply instance_enable_ipv6 from common to all components (#​18254) (5bc5853)
• deps: Patch mapstructure dependency to remove CVE (#​19447)
• distributor: Disable metadata topic writes (#​17437) (46b2271)
• distributor: Revert remove colons from level detection (#​16999) (e678b61)
• distributor: Short circuit level detection when already detected (#​17028) (58aa00a)
• distributor: Skip streams over limits in dry-run mode (#​17114) (2cde9b1)
• distributor: Prevent panic when ContentLength is negative in Distributor RequestBodyTooLarge metrics (#​17054) (9e9f534)
• distributor: detect "Information" log level as "info" (#​18195) (82263d3)
• distributor: correctly register allowed log level fields (#​19261)
• distributor: extended detected level for debug and critical level (#​18370)
• docs: Update configuration.md (#​17269) (70de523)
• docs: Added instructions for how to upgrade zone-aware ingesters (#​18658) (aff2cb0)
• engine: Don't attempt to release batch reference on read for merge node (#​18913)
• engine: Fix bug in SortMerge which caused rows to be skipped due to incorrect sorting (#​18334) (4fbe7c1)
• engine: Fix out-of-bounds panic in SortMerge implementation (#​17967)
• frontend: Allow resolution of v6 addresses. (#​18251) (8481b34)
• helm: Add flush=true to preStop hook (#​16063) (a375751)
• helm: Add init container configuration for backend, bloom builder, distributor, query-frontend, query-scheduler, read, write. (#​18709)
• helm: Add livenessProbe to read pods in Simple Scalable mode (#​17063)
• helm: Add missing log deletion endpoint to native ingress paths (#​14390) (6db8b1f)
• helm: Add release namespace metadata to HorizontalPodAutoscaling that lack it (#​18453) (faae40a)
• helm: Add validation for loki.storage.bucketNames (#​13781) (831c814)
• helm: admin-api should only be deployed in microservice mode (#​13655)
• helm: Allow loki to use hostPath volumes (#​17680) (8e61a5b)
• helm: Allow metrics networkpolicy only from namespace (#​17555) (3a9b8e8)
• helm: avoid double zonal ingester scrapping by not scrapping headless svc (#​19000)
• helm: Create namespaced RBAC when sidecar is enabled (#​16776) (7c9003e)
• helm: Do not strip whitespace in target definitions for querier and read deployments (#​19464)
• helm: Ensure global.extraEnv and global.extraEnvFrom applied to all resources consistently (#​16828) (ee479d0)
• helm: explicitly set registry for k8s-sidecar image (#​19233)
• helm: Fix global extra* values (#​17020)
• helm: Fix incorrect context references in Helm storage templates (#​18740) (1f362c8)
• helm: Fix PDB settings for chunksCache and resultsCache (#​18321) (e03b9b9)
• helm: Fix setting X-Scope-OrgID header (#​18414) (2f461c9)
• helm: Fix storage_config when use_thanos_objstore is set to true (#​17024) (81866d7)
• helm: Fixed statement logic to enable annotations (#​17756)
• helm: Gateway Ingester endpoints points to inexistent service when zone aware replication is enabled (#​17362) (395a711)
• helm: Make loki.storage.bucketNames are optional, if builtin minio is enabled. (#​18653) (409ccbd)
• helm: missing s3 field in lokiStorageConfig templated value (#​18791)
• helm: Move loki-sc-rules to second position in containers (#​17937) (6e72df3)
• helm: Only validate the ruler key exists under loki.storage.bucketNames when an object storage bucket is being used (#​18665) (0604477)
• helm: Remove flaky enterprise test (#​19362)
• helm: Removing deprecated admin_api_directory to avoid config crash (#​18731) (8cce03c)
• helm: Roll back change to Thanos ruler storage (#​18907)
• helm: un-deprecate several features in monitoring block (#​19012)
• helm: Update querier, read, and single binary target definitions for UI enablement (#​19461)
• helm: Use correct serviceName in zone-aware ingester statefulsets (#​18558) (2706302)
• helm: In helm chart fix indentation in nginx gateway config template handling (#​18167) (4fe20a7)
• helm: Loki chart fails to render proper YAML when add more than one extra manifest (#​12911) (cf12656)
• helm: Make helm use grpc for compactor address (#​17454) (90003f6)
• helm: Use gateway container port as nginx server port (#​18774)
• helm: Use strings in stead of integers for ports in CiliumNetworkPolicies (#​19252)
• helm: Websocket related proxy_set_header to locations back to resolve high CPU usage. (#​18800)
• helm: Use UDP/53 for DNS egress instead of named port (#​19073)
• helm: Add single-binary component to ingress NetworkPolicy (#​19229)
• helm: canary only worked when gateway enabled (#​16758)
• helm: Enable ui in helm deployments (#​17562)
• helm: Update loki-helm-test image tag to latest commit (#​19227)
• ingester: Fix missing series in monolithic mode (#​17067) (5625464)
• json parser: Fix possible JSON log line corruption caused by json parser on query path (#​18056)
• json, structured metadata: correctly handle escaping in json parsing API (#​17068)
• json, structured metadata: json-parsing cleanup (#​17072) (2aed4c3)
• kafka: Remove duplicated metric from Kafka producer (#​18614) (a67a460)
• kafka: Disable defaults address defaults (#​17825)
• kafka: Use cooperative active sticky load balancer (#​19160)
• limits: Adapt defaults and expose evict interval (#​17808)
• limits: Fix ingest-limits eviction metric (#​17779)
• limits: Read the consumer group and topic from the ingest-limits config (#​17831)
• limits: Set ready when all partitions ready (#​18092) (a07bee5)
• limits: counter variables should not have total at the end ((#​17982)
• loki: Return defaults for non-existent tenants on applied limits endpoint (#​17942)
• logql: Fix inconsistency with parsed field short circuiting (#​17104) (88beefb)
• logql: While validating logql expression, detect and validate expression in label_replace (#​18470) (d379de5)
• logql: Implement approx-topk function on querier (#​17816)
• logql: Reduce allocations for JSON and logfmt parser (#​18637)
• memberlist: Allow resolution of advertise address from v6 interface (#​18250) (8594d1c)
• mixins: Fix label generation for Loki logs dashboard (#​17412) (da2b620)
• mixin: Fix typo in Loki Reads dashboard TSDB row (#​18872)
• objstore: Ruler not starting and incorrect mkdir path when Thanos Store is enabled (#​16555) (3da9f2c)
• operator: Fix type of maximum OpenShift version property (#​18066) (f7c7dfa)
• operator: Fix typo in docs regarding forcepathstyle (#​17725)
• operator: Update maximum OpenShift version (#​17954)
• operator: Update memberlist when ingester becomes unhealthy (#​17026) (c63f0a9)
• operator: Update webhook validator for alerts/rules (#​17824)
• operator: Updated AlertingRule sample to make it comply with the validation we apply (#​18671)
• operator: upgrade OPA policy syntax for v1+ (#​18795)
• otlp: Calculate entry metadata size before adding resource/scope attributes (#​17629) (2161c8c)
• patterns: Pattern persistence feature flag (#​18285) (a206324)
• patterns: Limit volume and frequency of persisted patterns (#​18362) (c690827)
• patterns: Fix feature flag for enabling pattern persistence (#​18216) (c167800)
• push: Add guard clauses to prevent negative counter values (#​17056) (9000de1)
• push: Fix guard clauses to prevent error spam in logs (#​17372) (d70b20e)
• push: Fix push stats calculation (#​19319)
• oltp: apply global otlp config to tenant config only when it is updated in the overrides (#​19213)
• querier: Fixes panic when filtering agg metrics from nil resp (#​17662)
• querier: Allow boolean numeric values in detected labels (#​16997) (bfb935d)
• querier: Fix integer overrun when calculating parallelism for very long time range queries (#​17428) (82acbd5)
• querier: Better fix integer overrun when calculating parallelism for very long time range queries (#​17430) (d660af3)
• Revert "perf: Fix memory leak in cachedIterator (#​17628)" (#​18687) (0316740)
• ruler: correct log level verbosity in rule evaluation (#​19519)
  fix(ruler): Return StatusBadRequest on multiple org IDs (#​17850)
• storage: Fix nil pointer dereference in Volume method of the composite store (#​18064) (fc7c018)
• storage: Use default config when building s3 client (backport k277) ([#​19559](https://redirect.github.com/grafan