timerring · timerring · Dec 5, 2024 · Dec 5, 2024 · Dec 5, 2024
diff --git a/.gitignore b/.gitignore
@@ -393,4 +393,6 @@ cookies.json
 src/subtitle/models/base.pt
 src/subtitle/models/small.pt
 
-test/
+test/
+
+src/upload/upload.yaml
diff --git a/README.md b/README.md
@@ -4,7 +4,7 @@
     <img src="assets/headerLight.svg" alt="BILIVE" />
   </picture>
 
-*7 x 24 小时无人监守录制、渲染弹幕、自动上传，启动项目，人人都是录播员。*
+*7 x 24 小时无人监守录制、渲染弹幕、识别字幕、自动上传，启动项目，人人都是录播员。*
 
 [:page_facing_up: Documentation](#major-features) |
 [:gear: Installation](#quick-start) |
@@ -21,10 +21,10 @@
 
 ## 2. Major features
 
-- **速度快**：录制的同时可以选择启动无弹幕版视频的上传进程，下播即上线平台。
+<!-- - **速度快**：~~录制的同时可以选择启动无弹幕版视频的上传进程，下播即上线平台~~。(无弹幕版暂缓上线，等维护完成下一个版本上线) -->
 - **多房间**：同时录制多个直播间内容视频以及弹幕文件（包含普通弹幕，付费弹幕以及礼物上舰等信息）。
 - **占用小**：自动删除本地已上传的视频，极致节省空间。
-- **灵活高**：模版化自定义投稿，支持自定义投稿分区，动态内容，视频描述，视频标题，视频标签等。
+- **模版化**：无需复杂配置，开箱即用，( :tada: NEW)通过 b 站搜索建议接口自动抓取相关热门标签。
 - **检测片段并合并**：对于网络问题或者直播连线导致的视频流分段，能够自动检测合并成为完整视频。
 - **渲染弹幕版视频**：自动转换xml为ass弹幕文件并且渲染到视频中形成**有弹幕版视频**并自动上传。
 - **硬件要求极低**：无需GPU，只需最基础的单核CPU搭配最低的运存即可完成录制，弹幕渲染，上传等等全部过程，无最低配置要求，10年前的电脑或服务器依然可以使用！
@@ -85,12 +85,14 @@ graph TD
 
 ### 4.1 安装环境
 ```
+# 进入项目目录
+cd bilive
 # 安装所需依赖 推荐先 conda 创建虚拟环境
 pip install -r requirements.txt
-
 # 记录项目根目录
-./setRoutineTask.sh && source ~/.bashrc
+./setPath.sh && source ~/.bashrc
 ```
+以下功能默认开启，如果无 GPU，请直接看 4.2 节，并将 `src/allconfig.py` 文件中的 `GPU_EXIST` 参数设置为 `False`。
 如果需要使用自动识别并渲染字幕功能，模型参数及链接如下，注意 GPU 显存必须大于所需 VRAM：
 
 |  Size  | Parameters | Multilingual model | Required VRAM |
@@ -105,7 +107,6 @@ pip install -r requirements.txt
 > 1. 项目默认采用 [`small`](https://openaipublic.azureedge.net/main/whisper/models/9ecf779972d90ba49c06d968637d720dd632c55bbf19d441fb42bf17a411e794/small.pt) 模型，请自行下载所需文件，并放置在 `src/subtitle/models` 文件夹中。
 > 2. 由于 github 单个文件上限是 100MB，因此本仓库内只保留了 tiny 模型以供试用，如需试用请将 `settings.ini` 文件中的 `Mode` 参数设置为模型对应Size名称`tiny`，使用其他参数量模型同理。
 > 3. 如果追求识别准确率，推荐使用参数量 `small` 及以上的模型。
-> 4. 如果无 GPU，请勿开启此功能，并将 `src/allconfig.py` 文件中的 `GPU_EXIST` 参数设置为 `False`。
 
 ### 4.2 biliup-rs 登录
 
@@ -119,7 +120,7 @@ pip install -r requirements.txt
 然后执行：
 
 ```bash
-./startRecord.sh
+./record.sh
 ```
 ### 4.4 启动自动上传
 有弹幕版视频和无弹幕版视频的上传是独立的，可以同时进行，也可以单独启用。
@@ -129,33 +130,31 @@ pip install -r requirements.txt
 - 投稿的配置文件为 `upload_config.json`，可以参考给出的示例添加。
 - 请在将一级键值名称取为**字符串格式**的对应直播间的房间号（4位数以上）。
 
-然后执行：
-```bash
-./startUploadNoDanmaku.sh
-```
-
 #### 4.4.2 弹幕版视频渲染与自动上传
 
 > 请先确保你已经完成了 4.1 步骤，下载并放置了模型文件。
+> 否则，请将 `src/allconfig.py` 文件中的 `GPU_EXIST` 参数设置为 `False`
 
 ##### 启动弹幕渲染进程
 
-输入以下指令即可开始检测已录制的视频并且自动合并分段，自动进行弹幕转换与渲染的过程：
+输入以下指令即可检测已录制的视频并且自动合并分段，自动进行弹幕转换，字幕识别与渲染的过程：
 
 ```bash
-./startScan.sh
+./scan.sh
 ```
 
 ##### 启动自动上传进程
 
-参照 `upload/config` 文件夹内的 `22230707.yaml` 模板，添加你需要录制的房间信息，如有多个房间，请添加多个`roomid.yaml`文件，具体见[biliup-rs上传文档](https://biliup.github.io/biliup-rs/Guide.html#useage)。
-
-输入以下指令即可自动使上传队列中的视频匹配对应模版并自动上传：
-
 ```bash
-./startUpload.sh
+./upload.sh
 ```
 
+> [!TIP]
+> 上传默认参数如下，[]中内容全部自动替换。也可在 src/upload/extract_video_info.py 中自定义相关配置：
+> + 默认标题是"【弹幕】[XXX]直播回放-[日期]-[直播间标题]"。
+> + 默认描述是"【弹幕+字幕】[XXX]直播，直播间地址：[https://live.bilibili.com/XXX] 内容仅供娱乐，直播中主播的言论、观点和行为均由主播本人负责，不代表录播员的观点或立场。"
+> + 默认标签是根据主播名字自动在 b 站搜索推荐中抓取的[热搜词]，详见[bilibili-API-collect](https://github.com/SocialSisterYi/bilibili-API-collect/blob/master/docs/search/suggest.md)。
+
 > [!NOTE]
 > 相应的执行日志请在 `logs` 文件夹中查看。
 > ```
@@ -166,15 +165,10 @@ pip install -r requirements.txt
 > │   └── ...
 > ├── mergeLog # 片段合并日志
 > │   └── ...
-> ├── uploadDanmakuLog # 有弹幕版上传日志
-> │   └── ...
-> ├── uploadNoDanmakuLog # 无弹幕版上传日志
+> ├── uploadLog # 视频上传日志
 > │   └── ...
-> ├── blrec.log # startRecord.sh 运行日志
-> ├── removeEmojis.log # 移除弹幕表情日志
-> ├── scanSegments.log # startScan.sh 运行日志
-> ├── uploadQueue.log # startUpload.sh 运行日志
-> └── uploadNoDanmaku.log # startUploadNoDanmaku.sh 运行日志
+> ├── blrec.log # record.sh 运行日志
+> └── scan.log # scan.sh 运行日志
 > ```
 
 ## 特别感谢

diff --git a/logs/uploadDanmakuLog/.gitkeep → logs/uploadLog/.gitkeep b/logs/uploadDanmakuLog/.gitkeep → logs/uploadLog/.gitkeep
diff --git a/logs/uploadNoDanmakuLog/.gitkeep b/logs/uploadNoDanmakuLog/.gitkeep
diff --git a/startRecord.sh → record.sh b/startRecord.sh → record.sh
diff --git a/startScan.sh → scan.sh b/startScan.sh → scan.sh
@@ -1,7 +1,7 @@
 # kill the previous scanSegments process
 kill -9 $(pgrep -f scanSegments)
 # start the scanSegments process
-nohup $BILIVE_PATH/src/scanSegments.sh > $BILIVE_PATH/logs/scanSegments.log 2>&1 &
+nohup $BILIVE_PATH/src/scanSegments.sh > $BILIVE_PATH/logs/scan.log 2>&1 &
 # Check if the last command was successful
 if [ $? -eq 0 ]; then
     echo "success"

diff --git a/setRoutineTask.sh → setPath.sh b/setRoutineTask.sh → setPath.sh
diff --git a/settings.toml b/settings.toml
@@ -2,7 +2,7 @@ version = "1.0"
 webhooks = []
 
 [[tasks]]
-room_id = 22230707
+room_id = 173551
 enable_monitor = true
 enable_recorder = true
 

diff --git a/src/burn/only_render.py b/src/burn/only_render.py
@@ -49,9 +49,9 @@ def normalize_video_path(filepath):
         if os.path.exists(remove_path):
             os.remove(remove_path)
 
-    # For test
+    # # For test
     # test_path = original_video_path[:-4]
     # os.rename(original_video_path, test_path)
 
-    with open(f"{src.allconfig.SRC_DIR}/uploadProcess/uploadVideoQueue.txt", "a") as file:
+    with open(f"{src.allconfig.SRC_DIR}/upload/uploadVideoQueue.txt", "a") as file:
         file.write(f"{format_video_path}\n")
diff --git a/src/burn/render_and_merge.py b/src/burn/render_and_merge.py
@@ -7,7 +7,7 @@
 from src.burn.generate_danmakus import get_resolution, process_danmakus
 from src.burn.generate_subtitles import generate_subtitles
 from src.burn.render_video import render_video
-
+from src.upload.extract_video_info import get_video_info
 
 def normalize_video_path(filepath):
     """Normalize the video path to upload
@@ -19,13 +19,13 @@ def normalize_video_path(filepath):
     new_date_time = f"{date_time_parts[0][:4]}-{date_time_parts[0][4:6]}-{date_time_parts[0][6:8]}-{date_time_parts[1]}"
     return filepath.rsplit('/', 1)[0] + '/' + parts[0] + '_' + new_date_time + '.mp4'
 
-def merge_videos(in_final_video):
-    """Merge the video segments
+def merge_videos(in_final_video, title, artist, date):
+    """Merge the video segments and preserve the first video's metadata
     Args:
         in_final_video: str, the path of videos will be merged
     """
     merge_command = [
-    'ffmpeg', '-f', 'concat', '-safe', '0', '-i', 'mergevideo.txt', '-use_wallclock_as_timestamps', '1', 
+    'ffmpeg', '-f', 'concat', '-safe', '0', '-i', 'mergevideo.txt', '-metadata', f'title={title}', '-metadata', f'artist={artist}', '-metadata', f'date={date}', '-use_wallclock_as_timestamps', '1', 
     '-c', 'copy', in_final_video
     ]
     with open(src.allconfig.MERGE_LOG_PATH, 'a') as mlog:
@@ -35,6 +35,9 @@ def merge_videos(in_final_video):
 if __name__ == '__main__':
     # Define the path to your file
     same_videos_list = src.allconfig.SRC_DIR + '/sameSegments.txt'
+    title = ''
+    artist = ''
+    date = ''
     output_video_path = ''
 
     # Open the file and read it line by line
@@ -48,6 +51,10 @@ def merge_videos(in_final_video):
                 video_name = os.path.basename(stripped_line)
                 tmp = directory + '/tmp/'
                 if output_video_path == '':
+                    title, artist, date = get_video_info(stripped_line)
+                    #  = video_info['title']
+                    # artist = video_info['artist']
+                    # date = video_info['date']
                     output_video_path = normalize_video_path(stripped_line)
                     print("The output video is " + output_video_path)
                     subprocess.run(['mkdir', tmp])
@@ -79,13 +86,13 @@ def merge_videos(in_final_video):
                 if os.path.exists(remove_path):
                     os.remove(remove_path)
 
-            # For test part
+            # # For test part
             # test_path = original_video_path[:-4]
             # os.rename(original_video_path, test_path)
 
     subprocess.run(['rm', same_videos_list])
-    merge_videos(output_video_path)
+    merge_videos(output_video_path, title, artist, date)
     subprocess.run(['rm', '-r', tmp])
 
-    with open(f"{src.allconfig.SRC_DIR}/uploadProcess/uploadVideoQueue.txt", "a") as file:
+    with open(f"{src.allconfig.SRC_DIR}/upload/uploadVideoQueue.txt", "a") as file:
         file.write(f"{output_video_path}\n")
diff --git a/src/upload/__init__.py b/src/upload/__init__.py
@@ -0,0 +1,6 @@
+# Copyright (c) 2024 bilive.
+
+import sys
+import os
+# In order to test separately
+sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '../..')))
diff --git a/src/uploadProcess/biliup → src/upload/biliup b/src/uploadProcess/biliup → src/upload/biliup
diff --git a/src/upload/extract_video_info.py b/src/upload/extract_video_info.py
@@ -0,0 +1,74 @@
+# Copyright (c) 2024 bilive.
+
+import subprocess
+import re
+import json
+import os
+from datetime import datetime
+# from src.upload.query_search_suggestion import get_bilibili_suggestions
+from src.upload.query_search_suggestion import get_bilibili_suggestions
+
+def get_video_info(video_file_path):
+    """get the title, artist and date of the video file via ffprobe
+    Args:
+        video_file_path: str, the path of the video file
+    Returns:
+        str: the title of the video file, if failed, return None
+    """
+
+    command = [
+        "ffprobe",
+        "-v", "quiet",
+        "-print_format", "json",
+        "-show_format",
+        video_file_path
+    ]
+    output = subprocess.check_output(command, stderr=subprocess.STDOUT).decode('utf-8')
+    parsed_output = json.loads(output)
+    title_value = parsed_output["format"]["tags"]["title"]
+    artist_value = parsed_output["format"]["tags"]["artist"]
+    date_value = parsed_output["format"]["tags"]["date"]
+    if len(date_value) > 8:
+        dt = datetime.fromisoformat(date_value)
+        new_date = dt.strftime('%Y%m%d')
+    else:
+        new_date = date_value
+    return title_value, artist_value, new_date
+
+def generate_title(video_path):
+    title, artist, date = get_video_info(video_path)
+    new_title = "【弹幕】" + artist + "直播回放-" + date + "-" + title
+    return new_title
+
+def generate_desc(video_path):
+    title, artist, date = get_video_info(video_path)
+    source_link = generate_source(video_path)
+    new_desc = "【弹幕+字幕】" + artist + "直播，直播间地址：" + source_link + " 内容仅供娱乐，直播中主播的言论、观点和行为均由主播本人负责，不代表录播员的观点或立场。"
+    return new_desc
+
+def generate_tag(video_path):
+    title, artist, date = get_video_info(video_path)
+    tags = get_bilibili_suggestions(artist)
+    return tags
+
+def generate_source(video_path):
+    file_name = os.path.basename(video_path)
+    match_result = re.search(r'^([^_]*)', file_name)
+    if match_result:
+        part_before_underscore = match_result.group(1)
+        source_link = "https://live.bilibili.com/" + part_before_underscore
+        return source_link
+    else:
+        return None
+
+
+if __name__ == "__main__":
+    video_path = "/home/jh/Downloads/bilive/Videos/31612461/31612461_20241204-19-08-29.mp4"
+    video_title = generate_title(video_path)
+    print(video_title)
+    video_desc = generate_desc(video_path)
+    print(video_desc)
+    video_tag = generate_tag(video_path)
+    print(video_tag)
+    video_source = generate_source(video_path)
+    print(video_source)
diff --git a/src/upload/generate_yaml.py b/src/upload/generate_yaml.py
@@ -0,0 +1,39 @@
+# Copyright (c) 2024 bilive.
+
+import os
+import time
+import yaml
+import codecs
+from datetime import datetime
+from src.upload.extract_video_info import generate_title, generate_desc, generate_tag, generate_source
+
+def generate_yaml_template(video_path):
+    source = generate_source(video_path)
+    title = generate_title(video_path)
+    desc = generate_desc(video_path)
+    tag = generate_tag(video_path)
+    data = {
+        "line": "kodo",
+        "limit": 5,
+        "streamers": {
+            video_path: {
+                "copyright": 1,
+                "source": source,
+                "tid": 138,
+                "cover": "",
+                "title": title,
+                "desc_format_id": 0,
+                "desc": desc,
+                "dynamic": "",
+                "tag": tag
+            }
+        }
+    }
+    return yaml.dump(data, default_flow_style=False, sort_keys=False)
+
+
+if __name__ == "__main__":    
+    # read the queue and upload the video
+    yaml_template = generate_yaml_template("")
+    with open('upload.yaml', 'w', encoding='utf-8') as file:
+        file.write(yaml_template)
diff --git a/src/upload/query_search_suggestion.py b/src/upload/query_search_suggestion.py
@@ -0,0 +1,33 @@
+# Copyright (c) 2024 bilive.
+
+import requests
+
+def get_bilibili_suggestions(term):
+    """use the bilibili search suggestion api to get the most popular search suggestions
+
+    Args:
+        term: str,the keyword of the search.
+    Returns:
+        dict or None: if the request is successful and the response content is in JSON format, return the parsed JSON data (usually a dictionary or list structure),
+                      if the request fails, return None
+    """
+    url = "https://s.search.bilibili.com/main/suggest"
+    params = {
+        "term": term
+    }
+    try:
+        response = requests.get(url, params=params)
+        if response.status_code == 200:
+            suggestions = response.json()
+            values_list = [item['value'] for item in suggestions['result']['tag']]
+            result = ",".join(values_list)
+            return result
+        print(f"Request failed with status code: {response.status_code}")
+        return None
+    except requests.RequestException as e:
+        print(f"Request failed with exception: {e}")
+        return None
+
+if __name__ == "__main__":
+    suggestions = get_bilibili_suggestions("bilive")
+    print(suggestions)