Thanks to visit codestin.com
Credit goes to github.com

Skip to content

implement context checkpointing for hybrid and recurrent models #16382

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Oct 3, 2025
Merged

implement context checkpointing for hybrid and recurrent models #16382

Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
6b3d5e2
initial commit for branch 3
ddh0 Oct 2, 2025
257d492
Merge branch 'ggml-org:master' into mamba-checkpoints-3
ddh0 Oct 2, 2025
cfba346
generalize `swa_checkpoint` to `ctx_checkpoint`
ddh0 Oct 2, 2025
ba574ba
oops
ddh0 Oct 2, 2025
fa222c5
disable debug prints
ddh0 Oct 2, 2025
475e80b
keep backwards compat with `--swa-checkpoints`
ddh0 Oct 2, 2025
4fee0cc
update prompt re-processing message
ddh0 Oct 2, 2025
a3b4c17
Merge branch 'ggml-org:master' into mamba-checkpoints-3
ddh0 Oct 2, 2025
d304f02
fix off-by-one error per GG
ddh0 Oct 2, 2025
bb92d83
keep `seq_rm` log per GG
ddh0 Oct 2, 2025
126e08a
Merge branch 'ggml-org:master' into mamba-checkpoints-3
ddh0 Oct 2, 2025
9f996a7
Merge branch 'ggml-org:master' into mamba-checkpoints-3
ddh0 Oct 2, 2025
e1b68d8
Merge branch 'master' into mamba-checkpoints-3
ggerganov Oct 3, 2025
829c701
server : fix checkpoint logic to support recurrent caches
ggerganov Oct 3, 2025
85d5053
Merge branch 'master' into mamba-checkpoints-3
ggerganov Oct 3, 2025
6fc5bcd
server : cleanup and fixes
ggerganov Oct 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix off-by-one error per GG
  • Loading branch information
@ddh0
ddh0 committed Oct 2, 2025
commit d304f0291c81e099e8719a46d2f4a56747de1e65
2 changes: 1 addition & 1 deletion tools/server/server.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3552,7 +3552,7 @@ struct server_context {

const auto pos_min_thold = std::max(0, slot.n_past - n_swa);

if (pos_min > pos_min_thold) {
if (pos_min > pos_min_thold + 1) {
SLT_WRN(slot, "n_past = %d, cache_tokens.size() = %d, seq_id = %d, pos_min = %d, n_swa = %d\n", slot.n_past, (int) slot.cache_tokens.size(), slot.id, pos_min, n_swa);

// search for a context checkpoint
Expand Down