Thanks to visit codestin.com
Credit goes to github.com

Skip to content

generate_series (cancel_big_query test) should not leak memory #5114

@refset

Description

@refset

Test failure on my fork: https://github.com/refset/xtdb/runs/58435067849

The test:

(deftest cancel_big_query
(test-cancel "FROM GENERATE_SERIES(1,10000000) AS system(_id) ORDER BY _id"))

[Check failure on line 1 in pgwire_test__init](https://github.com/refset/xtdb/commit/9ffa47d6ebeb3ef5f5816e82655f4365944510e8#annotation_43160227074) 

@github-actions
github-actions
/ JUnit Test Report
pgwire_test__init.cancel_big_query

org.opentest4j.AssertionFailedError: ERROR in  () (core.cljc:357)
Uncaught exception, in fixtures.
expected:  nil
org.opentest4j.AssertionFailedError: ERROR in  () (core.cljc:357)
Uncaught exception, in fixtures.
expected:  nil

	at dev.clojurephant.jovial.engine.clojure_test$eval920$fn__921.invoke(clojure_test.clj:86)
	at app//clojure.lang.MultiFn.invoke(MultiFn.java:229)
	at app//clojure.test$do_report.invokeStatic(test.clj:357)
	at app//clojure.test$do_report.invoke(test.clj:351)
	at dev.clojurephant.jovial.engine.clojure_test$execute_node$fn__897.invoke(clojure_test.clj:57)
	at dev.clojurephant.jovial.engine.clojure_test$execute_node.invokeStatic(clojure_test.clj:50)
	at dev.clojurephant.jovial.engine.clojure_test$execute_node.invoke(clojure_test.clj:46)
	at dev.clojurephant.jovial.engine.clojure_test$execute_node$fn__897$fn__898.invoke(clojure_test.clj:55)
	at app//clojure.test$default_fixture.invokeStatic(test.clj:687)
	at app//clojure.test$default_fixture.invoke(test.clj:683)
	at dev.clojurephant.jovial.engine.clojure_test$execute_node$fn__897.invoke(clojure_test.clj:51)
	at dev.clojurephant.jovial.engine.clojure_test$execute_node.invokeStatic(clojure_test.clj:50)
	at dev.clojurephant.jovial.engine.clojure_test$execute_node.invoke(clojure_test.clj:46)
	at dev.clojurephant.jovial.engine.clojure_test$execute_node$fn__897$fn__898.invoke(clojure_test.clj:55)
	at dev.clojurephant.jovial.engine.clojure_test$eval870$fn__871$fn__872.invoke(clojure_test.clj:22)
	at dev.clojurephant.jovial.engine.clojure_test$execute_node$fn__897.invoke(clojure_test.clj:51)
	at dev.clojurephant.jovial.engine.clojure_test$execute_node.invokeStatic(clojure_test.clj:50)
	at dev.clojurephant.jovial.engine.clojure_test$execute_node.invoke(clojure_test.clj:46)
	at dev.clojurephant.jovial.engine.clojure_test.Engine.execute(clojure_test.clj:113)
	at dev.clojurephant.jovial.engine$eval659$fn__686$G__651__689.invoke(engine.clj:23)
	at dev.clojurephant.jovial.engine$eval659$fn__686$G__650__693.invoke(engine.clj:23)
	at app//clojure.lang.AFn.applyToHelper(AFn.java:156)
	at app//clojure.lang.AFn.applyTo(AFn.java:144)
	at app//clojure.lang.Var.applyTo(Var.java:707)
	at app//clojure.core$apply.invokeStatic(core.clj:667)
	at app//clojure.core$apply.invoke(core.clj:662)
	at app//clojure.lang.Var.invoke(Var.java:390)
	at app//dev.clojurephant.jovial.util.SimpleClojure.invoke(SimpleClojure.java:21)
	at app//dev.clojurephant.jovial.BaseClojureEngine.execute(BaseClojureEngine.java:32)
Caused by: clojure.lang.ExceptionInfo: Error on key :xtdb.query/query-source when running system {:reason :integrant.core/run-threw-exception, :system {:xtdb.cache/memory #object[xtdb.cache.MemoryCache 0x25130f26 "xtdb.cache.MemoryCache@25130f26"], :xtdb/modules {}, :xtdb.flight-sql/server #object[xtdb.api.FlightSql$Companion$open$1$1 0x58c7300 "xtdb.api.FlightSql$Companion$open$1$1@58c7300"], :xtdb.operator.scan/scan-emitter #object[xtdb.operator.scan$fn$reify__17912 0x3530ebfb "xtdb.operator.scan$fn$reify__17912@3530ebfb"], :xtdb.log/clusters {}, :xtdb/tracer nil, :xtdb/authn #xtdb.authn.UserTableAuthn{:rules [{:method #xt.authn/method :trust, :user nil, :remote-addr nil}], :q-src #xtdb.query.QuerySource{:allocator #object[org.apache.arrow.memory.ChildAllocator 0x1f537655 "Allocator(query-source) 0/1048576/6815744/2899102924 (res/actual/peak/limit)\n"], :scan-emitter #object[xtdb.operator.scan$fn$reify__17912 0x3530ebfb "xtdb.operator.scan$fn$reify__17912@3530ebfb"], :query-warning-counter #object[io.micrometer.prometheusmetrics.PrometheusCounter 0x4dcf6e87 "io.micrometer.prometheusmetrics.PrometheusCounter@5da077"], :ref-ctr #object[xtdb.util.RefCounter 0x607d36d5 "xtdb.util.RefCounter@607d36d5"], :plan-cache #object[com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalManualCache 0x72f19ffe "com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalManualCache@72f19ffe"], :metrics-registry #object[io.micrometer.prometheusmetrics.PrometheusMeterRegistry 0x49f116d9 "io.micrometer.prometheusmetrics.PrometheusMeterRegistry@49f116d9"]}, :db-cat #object[xtdb.db_catalog$fn$reify__5112 0x70c36cc7 "xtdb.db_catalog$fn$reify__5112@70c36cc7"]}, :xtdb/information-schema #object[xtdb.information_schema$fn$reify__13060 0x1403f6d3 "xtdb.information_schema$fn$reify__13060@1403f6d3"], :xtdb.query/query-source #xtdb.query.QuerySource{:allocator #object[org.apache.arrow.memory.ChildAllocator 0x1f537655 "Allocator(query-source) 0/1048576/6815744/2899102924 (res/actual/peak/limit)\n"], :scan-emitter #object[xtdb.operator.scan$fn$reify__17912 0x3530ebfb "xtdb.operator.scan$fn$reify__17912@3530ebfb"], :query-warning-counter #object[io.micrometer.prometheusmetrics.PrometheusCounter 0x4dcf6e87 "io.micrometer.prometheusmetrics.PrometheusCounter@5da077"], :ref-ctr #object[xtdb.util.RefCounter 0x607d36d5 "xtdb.util.RefCounter@607d36d5"], :plan-cache #object[com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalManualCache 0x72f19ffe "com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalManualCache@72f19ffe"], :metrics-registry #object[io.micrometer.prometheusmetrics.PrometheusMeterRegistry 0x49f116d9 "io.micrometer.prometheusmetrics.PrometheusMeterRegistry@49f116d9"]}, :xtdb/indexer #object[xtdb.indexer$fn$reify__24478 0x74a65d54 "xtdb.indexer$fn$reify__24478@74a65d54"], :xtdb.cache/disk nil, :xtdb.pgwire/server {:read-write #object[xtdb.pgwire.Server 0xc123d31 "xtdb.pgwire.Server@d387ce6d"], :read-only nil}, :xtdb/node #<XtdbNode>, :xtdb/config {:config #object[xtdb.api.Xtdb$Config 0x4f4f94e8 "Config(server=ServerConfig(host=localhost/127.0.0.1, port=0, readOnlyPort=-1, numThreads=42, ssl=null), flightSql=FlightSqlConfig(host=127.0.0.1, port=0), logClusters={}, log=Factory(instantSource=xtdb.test_util$$reify__24551@7c0bf685, epoch=0, coroutineContext=Dispatchers.Default), storage=InMemoryStorageFactory(epoch=0), memoryCache=xtdb.cache.MemoryCache$Factory@68ba4819, diskCache=null, healthz=null, defaultTz=Z, indexer=IndexerConfig(logLimit=64, pageLimit=1024, rowsPerBlock=102400, flushDuration=PT4H, skipTxs=[], enabled=true), compactor=CompactorConfig(threads=2), authn=UserTable(rules=[MethodRule(method=TRUST, user=null, remoteAddress=null)]), garbageCollector=GarbageCollectorConfig(enabled=false, blocksToKeep=10, garbageLifetime=PT24H, approxRunInterval=PT10M), tracer=TracerConfig(enabled=false, endpoint=http://localhost:4318/v1/traces, serviceName=xtdb, spanProcessor=null), txSink=null, nodeId=588c4967)"], :node-id "588c4967", :default-tz #xt/zone "Z"}, :xtdb/garbage-collector #object[xtdb.garbage_collector.GarbageCollector$Impl 0x45a5b3e6 "xtdb.garbage_collector.GarbageCollector$Impl@45a5b3e6"], :xtdb/db-catalog #object[xtdb.db_catalog$fn$reify__5112 0x70c36cc7 "xtdb.db_catalog$fn$reify__5112@70c36cc7"], :xtdb.metrics/registry #object[io.micrometer.prometheusmetrics.PrometheusMeterRegistry 0x49f116d9 "io.micrometer.prometheusmetrics.PrometheusMeterRegistry@49f116d9"], :xtdb/compactor #object[xtdb.compactor.Compactor$Impl 0x344c09f1 "xtdb.compactor.Compactor$Impl@344c09f1"], :xtdb/allocator {:allocator #object[org.apache.arrow.memory.RootAllocator 0x3c4f0087 "Allocator(ROOT) 0/1050368/6817536/2899102924 (res/actual/peak/limit)\n"], :close? true}}, :completed-keys (:xtdb/modules :xtdb/garbage-collector :xtdb.pgwire/server :xtdb.flight-sql/server :xtdb/node :xtdb/authn :xtdb/db-catalog :xtdb/indexer :xtdb/tracer :xtdb/compactor), :remaining-keys (:xtdb.operator.scan/scan-emitter :xtdb/information-schema :xtdb.log/clusters :xtdb.cache/memory :xtdb/allocator :xtdb.cache/disk :xtdb.metrics/registry :xtdb/config), :function #object[clojure.lang.MultiFn 0x13705509 "clojure.lang.MultiFn@13705509"], :key :xtdb.query/query-source, :value #xtdb.query.QuerySource{:allocator #object[org.apache.arrow.memory.ChildAllocator 0x1f537655 "Allocator(query-source) 0/1048576/6815744/2899102924 (res/actual/peak/limit)\n"], :scan-emitter #object[xtdb.operator.scan$fn$reify__17912 0x3530ebfb "xtdb.operator.scan$fn$reify__17912@3530ebfb"], :query-warning-counter #object[io.micrometer.prometheusmetrics.PrometheusCounter 0x4dcf6e87 "io.micrometer.prometheusmetrics.PrometheusCounter@5da077"], :ref-ctr #object[xtdb.util.RefCounter 0x607d36d5 "xtdb.util.RefCounter@607d36d5"], :plan-cache #object[com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalManualCache 0x72f19ffe "com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalManualCache@72f19ffe"], :metrics-registry #object[io.micrometer.prometheusmetrics.PrometheusMeterRegistry 0x49f116d9 "io.micrometer.prometheusmetrics.PrometheusMeterRegistry@49f116d9"]}}
	at app//integrant.core$run_exception.invokeStatic(core.cljc:357)
	at app//integrant.core$run_exception.invoke(core.cljc:356)
	at app//integrant.core$try_run_action.invokeStatic(core.cljc:371)
	at app//integrant.core$try_run_action.invoke(core.cljc:367)
	at app//integrant.core$run_loop.invokeStatic(core.cljc:377)
	at app//integrant.core$run_loop.invoke(core.cljc:373)
	at app//integrant.core$reverse_run_BANG_.invokeStatic(core.cljc:397)
	at app//integrant.core$reverse_run_BANG_.invoke(core.cljc:391)
	at app//integrant.core$halt_BANG_.invokeStatic(core.cljc:666)
	at app//integrant.core$halt_BANG_.invoke(core.cljc:660)
	at app//integrant.core$halt_BANG_.invokeStatic(core.cljc:663)
	at app//integrant.core$halt_BANG_.invoke(core.cljc:660)
	at app//xtdb.node.impl$open_node$fn__19222.invoke(impl.clj:376)
	at app//xtdb.node.impl.Node.close(impl.clj:272)
	at app//xtdb.util$close.invokeStatic(util.clj:80)
	at app//xtdb.util$close.invoke(util.clj:63)
	at app//xtdb.test_util$with_node.invokeStatic(test_util.clj:73)
	at app//xtdb.test_util$with_node.invoke(test_util.clj:72)
	at app//clojure.test$compose_fixtures$fn__9892$fn__9893.invoke(test.clj:694)
	at app//xtdb.test_util$with_opts.invokeStatic(test_util.clj:70)
	at app//xtdb.test_util$with_opts.invoke(test_util.clj:66)
	at app//xtdb.test_util$with_mock_clock.invokeStatic(test_util.clj:115)
	at app//xtdb.test_util$with_mock_clock.invoke(test_util.clj:114)
	at app//clojure.test$compose_fixtures$fn__9892$fn__9893.invoke(test.clj:694)
	at app//xtdb.test_util$with_allocator$fn__24499.invoke(test_util.clj:52)
	at app//xtdb.test_util$with_allocator.invokeStatic(test_util.clj:50)
	at app//xtdb.test_util$with_allocator.invoke(test_util.clj:49)
	at app//clojure.test$compose_fixtures$fn__9892$fn__9893.invoke(test.clj:694)
	at app//clojure.test$default_fixture.invokeStatic(test.clj:687)
	at app//clojure.test$default_fixture.invoke(test.clj:683)
	at app//clojure.test$compose_fixtures$fn__9892.invoke(test.clj:694)
	at app//clojure.test$compose_fixtures$fn__9892.invoke(test.clj:694)
	at app//clojure.test$compose_fixtures$fn__9892.invoke(test.clj:694)
	at app//clojure.test$compose_fixtures$fn__9892.invoke(test.clj:694)
	at dev.clojurephant.jovial.engine.clojure_test$execute_node$fn__897.invoke(clojure_test.clj:51)
	... 24 more
Caused by: java.lang.IllegalStateException: Memory was leaked by query. Memory leaked: (1048576)
Outstanding child allocators : 
  Allocator(BoundQuery/openCursor) 0/1048576/6815744/2899102924 (res/actual/peak/limit)
Allocator(query-source) 0/1048576/6815744/2899102924 (res/actual/peak/limit)

	at org.apache.arrow.memory.BaseAllocator.close(BaseAllocator.java:504)
	at xtdb.util$close.invokeStatic(util.clj:80)
	at xtdb.util$close.invoke(util.clj:63)
	at xtdb.query.QuerySource.close(query.clj:450)
	at xtdb.query$fn__23903.invokeStatic(query.clj:479)
	at xtdb.query$fn__23903.invoke(query.clj:478)
	at clojure.lang.MultiFn.invoke(MultiFn.java:234)
	at integrant.core$try_run_action.invokeStatic(core.cljc:369)
	... 56 more

Metadata

Metadata

Assignees

No one assigned

    Labels

    ciRelates test & build automation in CircleCI

    Type

    Projects

    Status

    💭 Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions