Clicking "Watch in Browser - MP4" works w/ 8GB RAM but crashes calibre-web.service w/ 2GB RAM / memory (MP4 is 1.0GB; users see "502 Bad Gateway" due to OOM crash) #79
Description
1h41m video (https://youtu.be/vbXu7WaifAo) is downloadable to IIAB even on VM's with 1.5GB RAM or less, and thumbnail/cover/poster seems to appear properly every time. However:
- Clicking "Watch in Browser - MP4" (top-right of "Book Details" pop-up) consistently causes
calibre-web.serviceto crash, on regular VM's with 2GB or less. - Whereas VM's with a gigantic amounts of RAM (e.g. 8GB) do allow clicking "Watch in Browser - MP4" to play the video.
Service crashing needs to be solved not just mitigated.
Hiding the problem is not a healthy option in my opinion — e.g. at this time I'm personally not in favor of partial solutions like these 2 proposed workarounds:
- PR [Workaround for] OOM / memory "502 Bad Gateway" crashes w/ large videos' playback or download-to-client [SEE ALSO #37 & #53] #51
- PR Restrict NGINX (public?) access to "large" files only (e.g. .mp3, .mp4, .webm) in /library/calibre-web — for PR #51 OOM workaround #57
Basic diagnostics below, towards getting us on the road to a proper fix:
root@box:~# cat /etc/issue
Ubuntu Noble Numbat (development branch) \n \l
root@box:/usr/local/calibre-web-py3# git log --oneline | head
e83c81e5 Merge branch 'janeczku:master' into master
4c64662b README.md: "Internet-in-a-Box does not support Docker"
2f7413cb Merge pull request #75 from deldesir/deldesir-tornado
93791b3a Increase Buffersize to 10GB
25fb8d93 Merge remote-tracking branch 'nl/master'
b456263d Merge pull request #72 from deldesir/deldesir-mp4-cover
525cd2fe Support case insensitive ext
bc0ebdb7 Updated Dutch translation.
cdfd9b31 Merge pull request #71 from deldesir/deldesir-description
04306ba2 Pull description
root@box:~# free -h
total used free shared buff/cache available
Mem: 1.9Gi 372Mi 1.5Gi 21Mi 162Mi 1.5Gi
Swap: 0B 0B 0B
root@box:~# df -h .
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 8.7G 5.8G 3.0G 67% /
root@box:~# cat /var/log/calibre-web.log
[2023-12-24 17:32:39,037] INFO {cps:168} *** "greenlet" version does not meet the requirements. Should: <2.1.0, Found: 3.0.3, please consider installing required version ***
[2023-12-24 17:32:39,037] INFO {cps:177} Starting Calibre Web...
[2023-12-24 17:32:39,044] DEBUG {cps.services:26} Cannot import goodreads, showing authors-metadata will not work: No module named 'betterreads'
[2023-12-24 17:32:39,045] DEBUG {cps.services:34} Cannot import simpleldap, logging in with ldap will not work: No module named 'flask_simpleldap'
[2023-12-24 17:32:39,079] DEBUG {cps.services:49} Cannot import gmail, sending books via Gmail Oauth2 Verification will not work: No module named 'google_auth_oauthlib'
[2023-12-24 17:32:39,096] WARN {py.warnings:109} /usr/local/calibre-web-py3/lib/python3.11/site-packages/flask_limiter/extension.py:336: UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend.
warnings.warn(
[2023-12-24 17:32:39,240] DEBUG {cps.gdriveutils:81} Cannot import pydrive, httplib2, using gdrive will not work: No module named 'pydrive'
[2023-12-24 17:32:39,512] INFO {apscheduler.scheduler:181} Scheduler started
[2023-12-24 17:32:39,512] INFO {apscheduler.scheduler:895} Added job "end scheduled task" to job store "default"
[2023-12-24 17:32:39,642] DEBUG {cps.comic:46} Cannot import comicapi, extracting comic metadata will not work: No module named 'comicapi'
[2023-12-24 17:32:39,643] DEBUG {cps.comic:53} Cannot import rarfile, extracting cover files from rar files will not work: No module named 'rarfile'
[2023-12-24 17:32:39,643] DEBUG {cps.comic:59} Cannot import py7zr, extracting cover files from CB7 files will not work: No module named 'py7zr'
[2023-12-24 17:32:39,721] DEBUG {cps.admin:81} Cannot import Flask-Dance, login with Oauth will not work: No module named 'flask_dance'
[2023-12-24 17:32:39,726] DEBUG {cps.gdrive:44} Cannot import googleapiclient, using GDrive will not work: No module named 'googleapiclient'
[2023-12-24 17:32:39,768] DEBUG {cps.search_metadata:62} Import error for metadata source: scholar - No module named 'scholarly'
[2023-12-24 17:32:39,799] DEBUG {cps.search_metadata:62} Import error for metadata source: lubimyczytac - No module named 'html2text'
[2023-12-24 17:32:39,809] DEBUG {cps.search_metadata:62} Import error for metadata source: amazon - No module named 'bs4'
[2023-12-24 17:32:39,810] DEBUG {cps.search_metadata:62} Import error for metadata source: douban - No module named 'html2text'
[2023-12-24 17:32:39,911] INFO {cps.server:267} Starting Tornado server on :8083
[2023-12-24 17:34:58,223] DEBUG {cps.web:1410} You are now logged in as: 'Admin'
[2023-12-24 17:34:58,243] DEBUG {cps.ub:82} Login and store session : 8b1f3e026c99c26da2710a803e79fd8c3731814f5a5be1b048a9411edb469756fa361953799483455066116aa7c0ded030ea40dafe80a1633b5985cd2bc19327
[2023-12-24 17:34:58,244] DEBUG {cps.ub:84} Found stored session: 8b1f3e026c99c26da2710a803e79fd8c3731814f5a5be1b048a9411edb469756fa361953799483455066116aa7c0ded030ea40dafe80a1633b5985cd2bc19327
[2023-12-24 17:35:05,991] DEBUG {cps.services.worker:91} Add Task for user: Admin - Download https://www.youtube.com/watch?v=vbXu7WaifAo
[2023-12-24 17:35:05,992] INFO {cps.tasks.download:28} Starting download task for URL: https://www.youtube.com/watch?v=vbXu7WaifAo
[2023-12-24 17:35:05,992] INFO {cps.tasks.download:37} Subprocess args: ['lb-wrapper', 'https://www.youtube.com/watch?v=vbXu7WaifAo']
[2023-12-24 17:35:05,996] INFO {cps.tasks.download:50} /usr/local/bin/lb
[2023-12-24 17:36:12,661] INFO {cps.editbooks:382} Requested files list: ['/library/downloads/calibre-web/Youtube/MMTV_NEWLOOK/URGENT_PANIQUE_CHEZ_FATSHI_COU.P_D_ETAT_EN_COURS_SOULEVEMENT_POPULAIRES_DANS_LA_DIASPORA_15.90k_[vbXu7WaifAo].mp4']
[2023-12-24 17:36:12,662] INFO {cps.editbooks:388} Processing file: <_io.BufferedReader name='/library/downloads/calibre-web/Youtube/MMTV_NEWLOOK/URGENT_PANIQUE_CHEZ_FATSHI_COU.P_D_ETAT_EN_COURS_SOULEVEMENT_POPULAIRES_DANS_LA_DIASPORA_15.90k_[vbXu7WaifAo].mp4'>
[2023-12-24 17:36:12,663] DEBUG {cps.uploader:371} Temporary file: /tmp/calibre_web/a47764e9ecae51b20547d3f41ae2b1d0
[2023-12-24 17:36:15,488] DEBUG {cps.helper:543} Moving title: /tmp/calibre_web/a47764e9ecae51b20547d3f41ae2b1d0 to /library/calibre-web/MMTV NEWLOOK/URGENT PANIQUE CHEZ FATSHI,COU.P D'ÉTAT EN COURS, SOULÈVEMENT POPULAIRES DANS LA DIASPORA (1)/URGENT PANIQUE CHEZ FATSHI,COU.P D'ÉTAT EN COURS, SOULÈVEMENT POPULAIRES DANS LA DIASPORA - MMTV NEWLOOK
[2023-12-24 17:36:15,509] DEBUG {cps.services.worker:91} Add Task for user: Admin - Upload URGENT PANIQUE CHEZ FATSHI,COU.P D'ÉTAT EN COURS, SOULÈVEMENT POPULAIRES DANS LA DIASPORA
[2023-12-24 17:36:15,517] INFO {cps.tasks.download:79} Successfully sent the list of requested files to http://192.168.0.173/books/meta
[2023-12-24 17:36:24,632] DEBUG {cps.web:1577} Start video watching for 1
[2023-12-24 17:36:24,634] INFO {cps.web:1214} Serving book: URGENT PANIQUE CHEZ FATSHI,COU.P D'ÉTAT E - MMTV NEWLOOK
root@box:~# systemctl status calibre-web.service
× calibre-web.service - Calibre-Web
Loaded: loaded (/etc/systemd/system/calibre-web.service; enabled; preset: enabled)
Active: failed (Result: oom-kill) since Sun 2023-12-24 17:36:33 EST; 25min ago
Duration: 3min 54.843s
Process: 3550 ExecStart=/usr/local/calibre-web-py3/bin/python3 /usr/local/calibre-web-py3/cps.py -p /library/calibre-web/config/app.db (code=killed, signal=KILL)
Main PID: 3550 (code=killed, signal=KILL)
CPU: 30.887s
Dec 24 17:32:38 box systemd[1]: Started calibre-web.service - Calibre-Web.
Dec 24 17:36:32 box systemd[1]: calibre-web.service: A process of this unit has been killed by the OOM killer.
Dec 24 17:36:33 box systemd[1]: calibre-web.service: Main process exited, code=killed, status=9/KILL
Dec 24 17:36:33 box systemd[1]: calibre-web.service: Failed with result 'oom-kill'.
Dec 24 17:36:33 box systemd[1]: calibre-web.service: Consumed 30.887s CPU time.
root@box:~# journalctl -u calibre-web
Dec 24 17:32:38 box systemd[1]: Started calibre-web.service - Calibre-Web.
Dec 24 17:36:32 box systemd[1]: calibre-web.service: A process of this unit has been killed by the OOM killer.
Dec 24 17:36:33 box systemd[1]: calibre-web.service: Main process exited, code=killed, status=9/KILL
Dec 24 17:36:33 box systemd[1]: calibre-web.service: Failed with result 'oom-kill'.
Dec 24 17:36:33 box systemd[1]: calibre-web.service: Consumed 30.887s CPU time.
root@box:~# hostname -I
10.181.233.208 192.168.0.173 10.8.0.46 fd42:fb6b:9c93:d2f1:5054:ff:fefe:fe43
root@box:~# hostname -A
t6.lxd box box t6.lxd
root@box:/library/downloads/calibre-web# tree
.
├── Youtube
│ └── MMTV_NEWLOOK
│ └── URGENT_PANIQUE_CHEZ_FATSHI_COU.P_D_ETAT_EN_COURS_SOULEVEMENT_POPULAIRES_DANS_LA_DIASPORA_15.90k_[vbXu7WaifAo].mp4
└── survey.db
3 directories, 2 files
root@box:/library/calibre-web# tree
.
├── MMTV NEWLOOK
│ └── URGENT PANIQUE CHEZ FATSHI,COU.P D'ÉTAT EN COURS, SOULÈVEMENT POPULAIRES DANS LA DIASPORA (1)
│ ├── URGENT PANIQUE CHEZ FATSHI,COU.P D'ÉTAT E - MMTV NEWLOOK.mp4
│ └── cover.jpg
├── config
│ └── app.db
├── metadata.db
└── metadata_db_prefs_backup.json
4 directories, 5 files
root@box:~# cat /var/log/xklb.log
2023-12-24 17:35:06 - Running command: lb tubeadd /library/downloads/calibre-web/survey.db https://www.youtube.com/watch?v=vbXu7WaifAo --verbose && lb dl /library/downloads/calibre-web/survey.db --prefix /library/downloads/calibre-web --write-thumbnail --format best --format-sort 'tbr~1000' --video https://www.youtube.com/watch?v=vbXu7WaifAo --verbose
2023-12-24 17:36:12 - Download completed successfully.
root@box:~# lb --version
2.2.192
root@box:~# yt-dlp --version
2023.11.16
Related:
- Some videos download as overweight (giant) files that (1) are not appropriate for schools, and (2) crash Calibre-Web with "502 Bad Gateway" #37
- Serve videos [using Video.js ?] #46
- Give everyone access to all books, intentionally ignoring Calibre-Web ACL accounts/authorization? #53
- MP4 files failed to upload [WebM too, for large video files?] #61
- PR scripts/lb-wrapper: Try yt-dlp with "--format best --format-sort 'tbr~1000'" [testing shows need for URL validation, clearly communicated, for users who accidentally request the impossible after clicking "Download to IIAB" > "Start"] #63
- Thumbnail of ["Downloaded to IIAB"] video is cut off & many files fail to be copied to /library/calibre-web [intermittent failures?] #77
- "Download to IIAB" of Vimeo video ("Staff Pick" URL or regular URL) does not download [and 1 YouTube video too, then 2 YouTube videos worked] #78
- "Download to IIAB" worked 5 out of 15 tries (trying to d/l individual YouTube videos) — "Tasks" view screenshots show the 10 failures [e.g. formerly live videos] #84