Bug: Race condition in acquiring connection after stream error, losing track of used connection #6288
Description
I noticed a problem with connection pool management after error in stream.
In the code below when await runStreamWithError() is commented out, then it behaves as expected: data is read in 2 threads concurrently, 2 connections are checked out and used with cursors.
With await runStreamWithError() there is a problem: When 2 streams are started then only one connection is used at a time, and the second stream uses a connection that is already released back to pool. From logs I see that connections pool knows that 2 connections are needed, but there is at most one connection checked out.
const db = require('knex')({
client: 'postgresql',
connection: …,
pool: { min: 0, max: 2 },
})
const runStreamWithError = async () => {
try {
for await (const row of db.raw(`SELECT 1 / 0`).stream()) {
console.info(row.id)
}
} catch (e) {
console.info('Caught error as expected:', e)
}
}
const runStream = async (threadId) => {
for await (const row of db.raw(`SELECT gs AS id FROM generate_series(1, 999) AS gs;`).stream()) {
console.info(
`thread ${threadId} row ${row.id}\tused ${db.client.pool.used.length}\tfree ${db.client.pool.free.length}\tcreates ${db.client.pool.pendingCreates.length}\tacquires ${db.client.pool.pendingAcquires.length}`,
)
await new Promise((r) => setTimeout(r, 2))
}
}
const main = async () => {
await runStreamWithError()
await Promise.all([runStream(1), runStream(2)])
await db.destroy()
}
main()I get logs like:
Caught error as expected: error: division by zero
at …
thread 1 row 1 used 1 free 0 creates 1 acquires 0
thread 1 row 2 used 1 free 0 creates 1 acquires 0
thread 1 row 3 used 1 free 1 creates 0 acquires 0 ← we see that second connection is ready to be used by second thread
…
thread 1 row 784 used 1 free 1 creates 0 acquires 0
thread 2 row 1 used 1 free 1 creates 0 acquires 0 ← first thread have fetched the last batch and second thread fetched the first batch
thread 1 row 785 used 1 free 1 creates 0 acquires 0
thread 2 row 2 used 1 free 1 creates 0 acquires 0
thread 1 row 786 used 1 free 1 creates 0 acquires 0
thread 2 row 3 used 1 free 1 creates 0 acquires 0
…
thread 1 row 998 used 1 free 1 creates 0 acquires 0
thread 2 row 215 used 1 free 1 creates 0 acquires 0
thread 1 row 999 used 1 free 1 creates 0 acquires 0
thread 2 row 216 used 0 free 2 creates 0 acquires 0 ← first thread printed all the data, second thread still uses a connection to db and calls `_getRows` every 100 rows
thread 2 row 217 used 0 free 2 creates 0 acquires 0
…
thread 2 row 999 used 0 free 2 creates 0 acquires 0
My versions:
knex: 3.1.0
pg: 8.16.3
pg-query-stream: 4.10.3
nodejs 22.13.1