Thanks to visit codestin.com
Credit goes to github.com

Skip to content

stable-diffusion.cpp doesn't compile with flash attention enabled #297

Closed
@OkLunixBSD

Description

@OkLunixBSD

Trying to compile the program with flash attention enabled errors out
apparently, the ggml_flash_attn() function was replaced by ggml_flash_attn_ext() in ggml in this pr: ggml-org/ggml@1a81566
cmake configuration:
cmake -B build -DSD_FLASH_ATTN=ON
compilation:

$ cmake --build build
[ 15%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml.c.o
[ 15%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml-backend.c.o
[ 23%] Building C object thirdparty/CMakeFiles/zip.dir/zip.c.o
[ 30%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml-alloc.c.o
[ 38%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml-quants.c.o
In file included from /home/kmf/pg/image/flash/thirdparty/zip.c:40:
/home/kmf/pg/image/flash/thirdparty/miniz.h:4988:9: note: ‘#pragma message: Using fopen, ftello, fseeko, stat() etc. path for file I/O - this path may not support large files.’
 4988 | #pragma message(                                                               \
      |         ^~~~~~~
[ 38%] Built target zip
[ 46%] Linking C static library libggml.a
[ 46%] Built target ggml
[ 61%] Building CXX object CMakeFiles/stable-diffusion.dir/upscaler.cpp.o
[ 69%] Building CXX object CMakeFiles/stable-diffusion.dir/model.cpp.o
[ 69%] Building CXX object CMakeFiles/stable-diffusion.dir/stable-diffusion.cpp.o
[ 76%] Building CXX object CMakeFiles/stable-diffusion.dir/util.cpp.o
In file included from /home/kmf/pg/image/flash/esrgan.hpp:4,
                 from /home/kmf/pg/image/flash/upscaler.cpp:1:
/home/kmf/pg/image/flash/ggml_extend.hpp: In function ‘ggml_tensor* ggml_nn_attention(ggml_context*, ggml_tensor*, ggml_tensor*, ggml_tensor*, bool)’:
/home/kmf/pg/image/flash/ggml_extend.hpp:592:31: error: ‘ggml_flash_attn’ was not declared in this scope; did you mean ‘ggml_flash_attn_ext’?
  592 |     struct ggml_tensor* kqv = ggml_flash_attn(ctx, q, k, v, false);  // [N * n_head, n_token, d_head]
      |                               ^~~~~~~~~~~~~~~
      |                               ggml_flash_attn_ext
In file included from /home/kmf/pg/image/flash/stable-diffusion.cpp:1:
/home/kmf/pg/image/flash/ggml_extend.hpp: In function ‘ggml_tensor* ggml_nn_attention(ggml_context*, ggml_tensor*, ggml_tensor*, ggml_tensor*, bool)’:
/home/kmf/pg/image/flash/ggml_extend.hpp:592:31: error: ‘ggml_flash_attn’ was not declared in this scope; did you mean ‘ggml_flash_attn_ext’?
  592 |     struct ggml_tensor* kqv = ggml_flash_attn(ctx, q, k, v, false);  // [N * n_head, n_token, d_head]
      |                               ^~~~~~~~~~~~~~~
      |                               ggml_flash_attn_ext
In file included from /home/kmf/pg/image/flash/preprocessing.hpp:4,
                 from /home/kmf/pg/image/flash/util.cpp:13:
/home/kmf/pg/image/flash/ggml_extend.hpp: In function ‘ggml_tensor* ggml_nn_attention(ggml_context*, ggml_tensor*, ggml_tensor*, ggml_tensor*, bool)’:
/home/kmf/pg/image/flash/ggml_extend.hpp:592:31: error: ‘ggml_flash_attn’ was not declared in this scope; did you mean ‘ggml_flash_attn_ext’?
  592 |     struct ggml_tensor* kqv = ggml_flash_attn(ctx, q, k, v, false);  // [N * n_head, n_token, d_head]
      |                               ^~~~~~~~~~~~~~~
      |                               ggml_flash_attn_ext
gmake[2]: *** [CMakeFiles/stable-diffusion.dir/build.make:118: CMakeFiles/stable-diffusion.dir/util.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[2]: *** [CMakeFiles/stable-diffusion.dir/build.make:104: CMakeFiles/stable-diffusion.dir/upscaler.cpp.o] Error 1
gmake[2]: *** [CMakeFiles/stable-diffusion.dir/build.make:90: CMakeFiles/stable-diffusion.dir/stable-diffusion.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:167: CMakeFiles/stable-diffusion.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions