Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Keep certstream server alive, fix CT log URLs, and prevent stack overflow#4

Draft
imfht with Copilot wants to merge 7 commits into
masterfrom
copilot/fix-automatic-exit-issue
Draft

Keep certstream server alive, fix CT log URLs, and prevent stack overflow#4
imfht with Copilot wants to merge 7 commits into
masterfrom
copilot/fix-automatic-exit-issue

Conversation

Copilot AI commented Jan 16, 2026 •
edited
Loading

Copy link
Copy Markdown

CT watcher initialization can exit after transient failures, which currently ends the entire process via tokio::select!. This explains the observed ~22s shutdown window during CT log list fetch retries.

  • Watcher supervision
    • Wrap ct_watcher::start_watchers in a restart loop with backoff on exit.
    • Log unexpected exits and continue running the web server.
  • Configurable CT log list
    • Add CT_LOG_LIST_URL environment override and log the resolved CT log list URL.
    • Document the new variable and add tests to validate default/override behavior.
  • CT API URL normalization
    • Ensure CT log API requests consistently add a trailing slash when building ct/v1/* endpoints so polling does not stall.
  • CT entry fetch stability
    • Replace recursive entry fetch retry with an iterative loop and stop on empty responses to avoid stack overflows when logs return no entries.
loop {
    match ct_watcher::start_watchers(client_manager.clone(), certificate_buffer.clone()).await {
        Ok(()) => warn!("CT watcher task exited unexpectedly. Restarting in {} seconds...", config::INIT_RETRY_DELAY_SECS),
        Err(err) => error!("CT watcher task exited: {}. Restarting in {} seconds...", err, config::INIT_RETRY_DELAY_SECS),
    }
    sleep(Duration::from_secs(config::INIT_RETRY_DELAY_SECS)).await;
}

> [!WARNING]
>
>

Original prompt

跑了22s之后自动退出了，是什么情况？

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI changed the title [WIP] Fix automatic exit after 22 seconds Keep certstream server alive when CT watcher exits Jan 16, 2026
Copilot AI requested a review from imfht January 16, 2026 03:28
@imfht

imfht commented Jan 16, 2026

Copy link
Copy Markdown
Owner

@copilot 我不知道为什么会退出，但是确实退出了

Copilot AI commented Jan 16, 2026

Copy link
Copy Markdown
Author

@copilot 我不知道为什么会退出，但是确实退出了

原因是 main 里用 tokio::select! 同时等待 watcher 和 web server，watcher 在初始化 CT log 列表或网络超时后会提前结束，导致整个进程退出。我已在 44d8132 加了 watcher 监督重启循环，watcher 退出会记录日志并自动重启，web server 不再被带退出。

@imfht

imfht commented Jan 19, 2026

Copy link
Copy Markdown
Owner

@copilot websocket 连接上去了，但是没有任何数据反馈

Copilot AI commented Jan 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

@copilot websocket 连接上去了，但是没有任何数据反馈

已在 c7ba3c0 增加 CT_LOG_LIST_URL 环境变量，可替换 gstatic 的 CT log 列表地址（被墙会导致 watcher 不启动，websocket 只有连接无数据）。现在启动日志会打印实际使用的 URL，设置该变量指向可访问的镜像后即可开始推送数据。

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • example.test
    • Triggering command: ./target/debug/certstream-server ./target/debug/certstream-server lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 --cfg feature="alloc" --cfg feature="clock" --cfg (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title Keep certstream server alive when CT watcher exits Keep certstream server alive and allow CT log list override Jan 19, 2026
@imfht

imfht commented Jan 19, 2026

Copy link
Copy Markdown
Owner

@copilot 一直是 Waiting on certificates

Copilot AI commented Jan 19, 2026

Copy link
Copy Markdown
Author

@copilot 一直是 Waiting on certificates

在 a732770 修复了 CT log URL 拼接问题：之前会把日志 URL 末尾的 “/” 去掉，导致拼出的请求变成 ...argon2020ct/v1/...，所有日志请求失败，所以一直停在 “Waiting on certificates”。现在统一补上尾部 “/”，拉取正常后就会开始推送证书。

Copilot AI changed the title Keep certstream server alive and allow CT log list override Keep certstream server alive and fix CT log URLs Jan 19, 2026
@imfht

imfht commented Jan 20, 2026

Copy link
Copy Markdown
Owner

@copilot 这个日志是不是看起来很不正常？2026-01-20T01:21:22.825443Z WARN Failed to get initial tree size for https://ct.googleapis.com/logs/argon2022 (attempt 4/5): error decoding response body: expected value at line 1 column 1. Retrying in 30 seconds...
2026-01-20T01:21:22.855135Z WARN Failed to get initial tree size for https://ct.googleapis.com/logs/us1/argon2025h1 (attempt 4/5): error decoding response body: expected value at line 2 column 1. Retrying in 30 seconds...
2026-01-20T01:21:22.868362Z WARN Failed to get initial tree size for https://ct.googleapis.com/logs/us1/argon2026h2 (attempt 4/5): error decoding response body: expected value at line 2 column 1. Retrying in 30 seconds...
2026-01-20T01:21:22.907908Z WARN Failed to get initial tree size for https://ct.googleapis.com/logs/us1/argon2027h1 (attempt 4/5): error decoding response body: expected value at line 2 column 1. Retrying in 30 seconds...

Copilot AI commented Jan 20, 2026

Copy link
Copy Markdown
Author

@copilot 这个日志是不是看起来很不正常？2026-01-20T01:21:22.825443Z WARN Failed to get initial tree size for https://ct.googleapis.com/logs/argon2022 (attempt 4/5): error decoding respo...

这条日志不正常，表示 get-sth 请求拿到的不是 JSON（通常是被代理/防火墙返回了 HTML 或空响应，所以 serde_json 报 “expected value at line 1 column 1”）。建议在服务器上直接 curl https://ct.googleapis.com/logs/argon2022/ct/v1/get-sth 看是否返回 JSON；如果返回 HTML/空白/403，说明网络被拦截或需要代理。此类错误会导致 watcher 一直重试但不会推送证书。

@imfht

imfht commented Jan 20, 2026

Copy link
Copy Markdown
Owner

@copilot curl https://ct.googleapis.com/logs/argon2022/ct/v1/get-sth
{"tree_size":1602372213,"timestamp":1768876246727,"sha256_root_hash":"rpxS5TuZlRANPfz+3+4gl9Q1l5L+bKN7uyMZX3vazfQ=","tree_head_signature":"BAMARzBFAiB1m3cCnohojjKGjOKEBWvI22PAbY/yc5DMmvrg9kmFAgIhAIHJt2aJ1OMASDKs/08958S12Z+Bi76EX4ayGYzl7H5f"}%

Copilot AI commented Jan 20, 2026

Copy link
Copy Markdown
Author

@copilot curl https://ct.googleapis.com/logs/argon2022/ct/v1/get-sth
{"tree_size":1602372213,"timestamp":1768876246727,"sha256_root_hash":"rpxS5TuZlRANPfz+3+4gl9Q1l5L+bKN7uyMZX3vazfQ=","tree_head_sign...

curl 返回的 JSON 说明这个日志本身是正常的；之前的 WARN 多半是旧版本（未修复 URL 拼接）或网络层偶发返回空/HTML（代理/防火墙/临时 5xx）。请用最新代码重新编译运行（commit e989040 之后），如果仍出现同样 WARN，请确认服务器没有 HTTPS_PROXY/透明代理，并用 curl -v 看实际响应状态和内容类型；需要的话我可以再加日志把 HTTP 状态/响应片段打印出来。

@imfht

imfht commented Jan 20, 2026

Copy link
Copy Markdown
Owner

@copilot 2026-01-20T02:53:07.223304Z WARN Failed to get initial tree size for https://mammoth2025h2.ct.sectigo.com/ (attempt 2/5): error decoding response body: expected value at line 1 column 1. Retrying in 30 seconds...

thread 'tokio-runtime-worker' (338002) has overflowed its stack
fatal runtime error: stack overflow, aborting
[1] 337994 IOT instruction (core dumped) PORT=4001 ./target/release/certstream-server

Copilot AI commented Jan 20, 2026

Copy link
Copy Markdown
Author

@copilot 2026-01-20T02:53:07.223304Z WARN Failed to get initial tree size for https://mammoth2025h2.ct.sectigo.com/ (attempt 2/5): error decoding response body: expected value at line 1 column 1. Ret...

在 f86264b 里把 fetch_and_broadcast_certs 的递归改成迭代循环，并在收到空响应时停止本批次以避免无限递归/堆栈溢出；这应能解决你看到的 stack overflow 崩溃。

Copilot AI changed the title Keep certstream server alive and fix CT log URLs Keep certstream server alive, fix CT log URLs, and prevent stack overflow Jan 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@imfht imfht Awaiting requested review from imfht

Assignees

@imfht imfht

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@imfht