Thanks to visit codestin.com
Credit goes to github.com

Skip to content

imatrix : handle partial entries #7833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 9, 2024
Merged

imatrix : handle partial entries #7833

merged 1 commit into from
Jun 9, 2024

Conversation

@ggerganov
Copy link
Member

@ggerganov ggerganov commented Jun 8, 2024

fix #7816

Print warning messages when imatrix entries have zero counts:

compute_imatrix: tokenizing the input ..
compute_imatrix: tokenization took 22899.3 ms
compute_imatrix: computing over 4918 chunks with batch_size 256
compute_imatrix: 12.00 seconds per pass - ETA 16 hours 23.53 minutes
[1]6.6864,[2]9.1522,[3]9.9931,[4]8.8414,
save_imatrix: entry '             blk.17.ffn_down_exps.weight' has partial data (93.75%) - skipping
save_imatrix: entry '             blk.17.ffn_gate_exps.weight' has partial data (93.75%) - skipping
save_imatrix: entry '               blk.17.ffn_up_exps.weight' has partial data (93.75%) - skipping
save_imatrix: entry '             blk.16.ffn_down_exps.weight' has partial data (73.44%) - skipping
save_imatrix: entry '             blk.16.ffn_gate_exps.weight' has partial data (73.44%) - skipping
save_imatrix: entry '               blk.16.ffn_up_exps.weight' has partial data (73.44%) - skipping
save_imatrix: entry '             blk.15.ffn_down_exps.weight' has partial data (78.12%) - skipping
save_imatrix: entry '             blk.14.ffn_down_exps.weight' has partial data (60.94%) - skipping
save_imatrix: entry '               blk.14.ffn_up_exps.weight' has partial data (60.94%) - skipping
save_imatrix: entry '              blk.1.ffn_down_exps.weight' has partial data (78.12%) - skipping
save_imatrix: entry '             blk.13.ffn_gate_exps.weight' has partial data (62.50%) - skipping
save_imatrix: entry '             blk.13.ffn_down_exps.weight' has partial data (62.50%) - skipping
save_imatrix: entry '             blk.11.ffn_down_exps.weight' has partial data (59.38%) - skipping
save_imatrix: entry '               blk.11.ffn_up_exps.weight' has partial data (59.38%) - skipping
save_imatrix: entry '             blk.15.ffn_gate_exps.weight' has partial data (78.12%) - skipping
save_imatrix: entry '               blk.15.ffn_up_exps.weight' has partial data (78.12%) - skipping
save_imatrix: entry '             blk.14.ffn_gate_exps.weight' has partial data (60.94%) - skipping
save_imatrix: entry '             blk.12.ffn_down_exps.weight' has partial data (62.50%) - skipping
save_imatrix: entry '             blk.12.ffn_gate_exps.weight' has partial data (62.50%) - skipping
save_imatrix: entry '               blk.12.ffn_up_exps.weight' has partial data (62.50%) - skipping
save_imatrix: entry '              blk.3.ffn_down_exps.weight' has partial data (98.44%) - skipping
save_imatrix: entry '                blk.1.ffn_up_exps.weight' has partial data (78.12%) - skipping
save_imatrix: entry '               blk.13.ffn_up_exps.weight' has partial data (62.50%) - skipping
save_imatrix: entry '              blk.1.ffn_gate_exps.weight' has partial data (78.12%) - skipping
save_imatrix: entry '              blk.3.ffn_gate_exps.weight' has partial data (98.44%) - skipping
save_imatrix: entry '                blk.3.ffn_up_exps.weight' has partial data (98.44%) - skipping
save_imatrix: entry '              blk.2.ffn_down_exps.weight' has partial data (96.88%) - skipping
save_imatrix: entry '              blk.2.ffn_gate_exps.weight' has partial data (96.88%) - skipping
save_imatrix: entry '             blk.11.ffn_gate_exps.weight' has partial data (59.38%) - skipping
save_imatrix: entry '                blk.2.ffn_up_exps.weight' has partial data (96.88%) - skipping
save_imatrix: warning: storing only 306 out of 336 entries

save_imatrix: stored collected data after 10 chunks in imatrix.dat

Such entries are not stored in the output matrix to prevent errors when using the imatrix. To prevent this from happening, provide larger and more diverse training data

@ggerganov ggerganov force-pushed the gg/imatrix-partial-data branch from 175a179 to 5a21852 Compare June 8, 2024 09:40
This was referenced Jun 8, 2024
Closed
Closed
@ggerganov ggerganov merged commit e95beeb into master Jun 9, 2024
@ggerganov ggerganov deleted the gg/imatrix-partial-data branch June 9, 2024 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

examples

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Bug: QWEN2 MoE imatrix contains nan's after generating it

2 participants

@ggerganov