processChild.js dies in production with no replacement #49623
Description
Verify canary release
- I verified that the issue exists in the latest Next.js canary release
Provide environment information
Operating System:
Platform: linux
Arch: x64
Version: #1 SMP Tue Sep 13 07:51:46 UTC 2022
Binaries:
Node: 20.1.0
npm: 9.6.4
Yarn: 1.22.19
pnpm: N/A
Relevant packages:
next: 13.4.2-canary.4
eslint-config-next: 13.4.1
react: 18.2.0
react-dom: 18.2.0
typescript: 5.0.4Which area(s) of Next.js are affected? (leave empty if unsure)
Standalone mode (output: "standalone")
Link to the code that reproduces this issue
To Reproduce
not sure, use prisma?
Describe the Bug
in my next.js 13 appDir app I'm using:
- server side components
- route handlers
- both are using prisma
once I started getting some production traffic I started observing lock up issues, timeouts and whatnot.
initially of course I went through the whole prisma/pgbouncer shebang but after some days of debugging (including checking proxied requests between server and jest workers) I just noticed that in my pod just after few requests we observe this.
PID USER TIME COMMAND
1 nextjs 0:01 node server.js
14 nextjs 0:02 /usr/local/bin/node /app/node_modules/next/dist/compiled/jest-worker/processChild.js
25 nextjs 0:13 /usr/local/bin/node /app/node_modules/next/dist/compiled/jest-worker/processChild.js
26 nextjs 0:00 /usr/local/bin/node /app/node_modules/next/dist/compiled/jest-worker/processChild.js
39 nextjs 0:00 sh
78 nextjs 0:00 ps axjf
this is the situation when the server starts, once we get some requests in:
...
Handling request GET /projects/2
Proxy target: http://localhost:33149/projects/2
Handling request GET /projects/14
Proxy target: http://localhost:33149/projects/14
Handling request GET /projects/4
Proxy target: http://localhost:33149/projects/4
Handling request GET /projects
Proxy target: http://localhost:33149/projects
Error: aborted
at connResetException (node:internal/errors:717:14)
at Socket.socketCloseListener (node:_http_client:455:19)
at Socket.emit (node:events:523:35)
at TCP.<anonymous> (node:net:335:12) {
code: 'ECONNRESET'
}
Error: socket hang up
at connResetException (node:internal/errors:717:14)
at Socket.socketOnEnd (node:_http_client:519:23)
at Socket.emit (node:events:523:35)
at endReadableNT (node:internal/streams/readable:1367:12)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
code: 'ECONNRESET'
}
Error: aborted
at connResetException (node:internal/errors:717:14)
at Socket.socketCloseListener (node:_http_client:455:19)
at Socket.emit (node:events:523:35)
at TCP.<anonymous> (node:net:335:12) {
code: 'ECONNRESET'
}
Error: aborted
at connResetException (node:internal/errors:717:14)
at Socket.socketCloseListener (node:_http_client:455:19)
at Socket.emit (node:events:523:35)
at TCP.<anonymous> (node:net:335:12) {
code: 'ECONNRESET'
}
Error: aborted
at connResetException (node:internal/errors:717:14)
at Socket.socketCloseListener (node:_http_client:455:19)
at Socket.emit (node:events:523:35)
at TCP.<anonymous> (node:net:335:12) {
code: 'ECONNRESET'
}
Error: aborted
at connResetException (node:internal/errors:717:14)
at Socket.socketCloseListener (node:_http_client:455:19)
at Socket.emit (node:events:523:35)
at TCP.<anonymous> (node:net:335:12) {
code: 'ECONNRESET'
}
Error: aborted
at connResetException (node:internal/errors:717:14)
at Socket.socketCloseListener (node:_http_client:455:19)
at Socket.emit (node:events:523:35)
at TCP.<anonymous> (node:net:335:12) {
code: 'ECONNRESET'
}
Error: aborted
at connResetException (node:internal/errors:717:14)
at Socket.socketCloseListener (node:_http_client:455:19)
at Socket.emit (node:events:523:35)
at TCP.<anonymous> (node:net:335:12) {
code: 'ECONNRESET'
}
Handling request GET /_next/static/css/65a6d53be77a4768.css
Proxy target: http://localhost:33149/_next/static/css/65a6d53be77a4768.css
Handling request GET /_next/static/chunks/webpack-a75aec2d68f60fab.js
Proxy target: http://localhost:33149/_next/static/chunks/webpack-a75aec2d68f60fab.js
...
and we can notice how a processChild is missing:
PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND
14 1 nextjs S 10.6g 32% 5 0% /usr/local/bin/node /app/node_modules/next/dist/compiled/jest-worker/processChild.js
26 14 nextjs S 10.6g 32% 0 0% /usr/local/bin/node /app/node_modules/next/dist/compiled/jest-worker/processChild.js
1 0 nextjs S 686m 2% 3 0% node server.js
39 0 nextjs S 1692 0% 3 0% sh
77 39 nextjs R 1620 0% 7 0% top
so it seems that a worker crashed but the server still thinks it's around somehow.
on further requests however there are no ECONNRESET in the logs anymore, just the server hanging indefinitely.
it is unclear what is causing this apparent crash given the same pages sometime works, especially when server just restarted.
Expected Behavior
server is stable.
Which browser are you using? (if relevant)
No response
How are you deploying your application? (if relevant)
No response