Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Convert.py fails on falcon-7b #5426

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
cmwilki opened this issue Feb 9, 2024 · 2 comments
Closed

Convert.py fails on falcon-7b #5426

cmwilki opened this issue Feb 9, 2024 · 2 comments
Labels
bug-unconfirmed

Comments

@cmwilki
Copy link

cmwilki commented Feb 9, 2024 •
edited
Loading

I cloned llama.cpp today and also performed a git checkout of falcon-7b from hugging face. Attempting to run convert fails similarly to #2717 which was supposedly closed and merged to master.

Here is my python:

python
Python 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0] on linux

And the error:

~/git/llama.cpp/convert.py ~/models/falcon-7b/ --outtype f16 --outfile falcon-7b.f16.bin
Loading model file /home/cwilkinson/models/falcon-7b/pytorch_model-00001-of-00002.bin
Loading model file /home/cwilkinson/models/falcon-7b/pytorch_model-00001-of-00002.bin
Loading model file /home/cwilkinson/models/falcon-7b/pytorch_model-00002-of-00002.bin
Traceback (most recent call last):
  File "/home/cwilkinson/git/llama.cpp/convert.py", line 1478, in <module>
    main()
  File "/home/cwilkinson/git/llama.cpp/convert.py", line 1414, in main
    model_plus = load_some_model(args.model)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cwilkinson/git/llama.cpp/convert.py", line 1276, in load_some_model
    model_plus = merge_multifile_models(models_plus)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cwilkinson/git/llama.cpp/convert.py", line 730, in merge_multifile_models
    model = merge_sharded([mp.model for mp in models_plus])
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cwilkinson/git/llama.cpp/convert.py", line 709, in merge_sharded
    return {name: convert(name) for name in names}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cwilkinson/git/llama.cpp/convert.py", line 709, in <dictcomp>
    return {name: convert(name) for name in names}
                  ^^^^^^^^^^^^^
  File "/home/cwilkinson/git/llama.cpp/convert.py", line 684, in convert
    lazy_tensors: list[LazyTensor] = [model[name] for model in models]
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cwilkinson/git/llama.cpp/convert.py", line 684, in <listcomp>
    lazy_tensors: list[LazyTensor] = [model[name] for model in models]
                                      ~~~~~^^^^^^
KeyError: 'transformer.word_embeddings.weight'

Any help is appreciated.
@VisionTheta
Copy link

You may use python convert-hf-to-gguf.py models/falcon-7b for converting huggingface models to gguf format.

convert.py is only used for converting llama models to gguf I think.

@cmwilki
Copy link
Author

cmwilki commented Mar 5, 2024

Confirmed user error. Closing this issue.

@cmwilki cmwilki closed this as completed Mar 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug-unconfirmed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@VisionTheta @cmwilki