Support for SD2.x has been added, welcome to try it out #41

leejet · 2023-09-03T08:06:42Z

leejet
Sep 3, 2023
Maintainer

./bin/sd -m ../models/v2-1_512-ema-pruned-ggml-model-f16.bin -p "a lovely cat" -v

juniofaathir · 2023-09-03T13:18:54Z

juniofaathir
Sep 3, 2023

Your newest windows ready version, is that mean the image generation now can be accelerated by nvidia gpu?

4 replies

leejet Sep 3, 2023
Maintainer Author

Not yet, the convolution operation can only run on the cpu for the time being

juniofaathir Sep 3, 2023

So, what does 'add CUDA RNG' mean?

leejet Sep 3, 2023
Maintainer Author

Imitiating torch cuda randn on CPU. This results in the same image as stable-diffusion-webui GPU RNG

Green-Sky Sep 3, 2023

should we make the stabe-diffusion-webui one the default?

Green-Sky · 2023-09-03T16:39:53Z

Green-Sky
Sep 3, 2023

$ bin/sd -m ../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin -p "night scene, close up 85mm photo of a beautiful cute girl, posing, look at a camera and smile, pink ponytail hair, (blue eyes:0.8), cute young face, 18 yo, soft volumetric lights, (backlit:1.3), (cinematic:1.3), intricate details, (ArtStation:1.2)"

same speed as 1.x models (~10s per sampling step for 512x512)

8 replies

Green-Sky Sep 3, 2023

@leejet id did not acually finish, it crashed very late.

Details

$ bin/sd -m ../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin -p "night scene, close up 85mm photo of a beautiful cute girl, posing, look at a camera and smile, pink ponytail hair, (blue eyes:0.8), cute young face, 18 yo, soft volumetric lights, (backlit:1.3), (cinematic:1.3), intricate details, (ArtStation:1.2)" -H 1024 -W 1024 -v
Option:
    n_threads:       12
    mode:            txt2img
    model_path:      ../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin
    output_path:     output.png
    init_img:
    prompt:          night scene, close up 85mm photo of a beautiful cute girl, posing, look at a camera and smile, pink ponytail hair, (blue eyes:0.8), cute young face, 18 yo, soft volumetric lights, (backlit:1.3), (cinematic:1.3), intricate details, (ArtStation:1.2)
    negative_prompt:
    cfg_scale:       7.00
    width:           1024
    height:          1024
    sample_method:   eular a
    sample_steps:    20
    strength:        0.75
    rng:             std_default
    seed:            42
System Info:
    BLAS = 0
    SSE3 = 1
    AVX = 1
    AVX2 = 1
    AVX512 = 0
    AVX512_VBMI = 0
    AVX512_VNNI = 0
    FMA = 1
    NEON = 0
    ARM_FMA = 0
    F16C = 1
    FP16_VA = 0
    WASM_SIMD = 0
    VSX = 0
[INFO]  stable-diffusion.cpp:2793 - loading model from '../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin'
[DEBUG] stable-diffusion.cpp:2801 - verifying magic
[DEBUG] stable-diffusion.cpp:2812 - loading hparams
[INFO]  stable-diffusion.cpp:2821 - model type: SD2.x
[INFO]  stable-diffusion.cpp:2829 - ftype: q8_0
[DEBUG] stable-diffusion.cpp:2835 - loading vocab
[DEBUG] stable-diffusion.cpp:2863 - ggml tensor size = 288 bytes
[DEBUG] stable-diffusion.cpp:2868 - clip params ctx size =  359.99 MB
[DEBUG] stable-diffusion.cpp:2887 - unet params ctx size =  1406.40 MB
[DEBUG] stable-diffusion.cpp:2908 - vae params ctx size =  95.51 MB
[DEBUG] stable-diffusion.cpp:2929 - preparing memory for the weights
[DEBUG] stable-diffusion.cpp:2945 - loading weights
[DEBUG] stable-diffusion.cpp:3048 - model size = 1858.32MB
[INFO]  stable-diffusion.cpp:3053 - total params size = 1858.69MB (clip 358.69MB, unet 1405.49MB, vae 94.51MB)
[INFO]  stable-diffusion.cpp:3059 - loading model from '../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin' completed, taking 1.33s
[DEBUG] stable-diffusion.cpp:3129 - diffusion context need 10.91MB static memory, with work_size needing 0.08MB
[INFO]  stable-diffusion.cpp:3188 - check is_using_v_parameterization_for_sd2 completed, taking 0.12s
[DEBUG] stable-diffusion.cpp:3189 - diffusion graph use 12.65MB runtime memory: static 10.91MB, dynamic 1.75MB
[DEBUG] stable-diffusion.cpp:3193 - 1024 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3082 - running in v-prediction mode
[DEBUG] stable-diffusion.cpp:813  - parse 'night scene, close up 85mm photo of a beautiful cute girl, posing, look at a camera and smile, pink ponytail hair, (blue eyes:0.8), cute young face, 18 yo, soft volumetric lights, (backlit:1.3), (cinematic:1.3), intricate details, (ArtStation:1.2)' to [['night scene, close up 85mm photo of a beautiful cute girl, posing, look at a camera and smile, pink ponytail hair, ', 1], ['blue eyes', 0.8], [', cute young face, 18 yo, soft volumetric lights, ', 1], ['backlit', 1.3], [', ', 1], ['cinematic', 1.3], [', intricate details, ', 1], ['ArtStation', 1.2], ]
[DEBUG] stable-diffusion.cpp:375  - split prompt "night scene, close up 85mm photo of a beautiful cute girl, posing, look at a camera and smile, pink ponytail hair, " to tokens ["night</w>", "scene</w>", ",</w>", "close</w>", "up</w>", "8</w>", "5</w>", "mm</w>", "photo</w>", "of</w>", "a</w>", "beautiful</w>", "cute</w>", "girl</w>", ",</w>", "posing</w>", ",</w>", "look</w>", "at</w>", "a</w>", "camera</w>", "and</w>", "smile</w>", ",</w>", "pink</w>", "ponytail</w>", "hair</w>", ",</w>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt "blue eyes" to tokens ["blue</w>", "eyes</w>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt ", cute young face, 18 yo, soft volumetric lights, " to tokens [",</w>", "cute</w>", "young</w>", "face</w>", ",</w>", "1</w>", "8</w>", "yo</w>", ",</w>", "soft</w>", "<|endoftext|>", "lights</w>", ",</w>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt "backlit" to tokens ["<|endoftext|>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt ", " to tokens [",</w>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt "cinematic" to tokens ["cinematic</w>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt ", intricate details, " to tokens [",</w>", "intricate</w>", "details</w>", ",</w>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt "ArtStation" to tokens ["<|endoftext|>", ]
[DEBUG] stable-diffusion.cpp:3230 - condition context need 10.70MB static memory, with work_size needing 0.32MB
[DEBUG] stable-diffusion.cpp:3255 - building condition graph completed: 1205 nodes, 397 leafs
[DEBUG] stable-diffusion.cpp:3263 - computing condition graph completed, taking 0.25s
[INFO]  stable-diffusion.cpp:3309 - condition graph use 373.30MB of memory: params 358.69MB, runtime 14.61MB (static 10.70MB, dynamic 3.91MB)
[DEBUG] stable-diffusion.cpp:3318 - 316416 bytes of dynamic memory has not been released yet
[DEBUG] stable-diffusion.cpp:813  - parse '' to [['', 1], ]
[DEBUG] stable-diffusion.cpp:375  - split prompt "" to tokens []
[DEBUG] stable-diffusion.cpp:3230 - condition context need 10.70MB static memory, with work_size needing 0.32MB
[DEBUG] stable-diffusion.cpp:3255 - building condition graph completed: 1205 nodes, 397 leafs
[DEBUG] stable-diffusion.cpp:3263 - computing condition graph completed, taking 0.23s
[INFO]  stable-diffusion.cpp:3309 - condition graph use 373.30MB of memory: params 358.69MB, runtime 14.61MB (static 10.70MB, dynamic 3.91MB)
[DEBUG] stable-diffusion.cpp:3318 - 316416 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3854 - get_learned_condition completed, taking 0.49s
[INFO]  stable-diffusion.cpp:3870 - start sampling
[DEBUG] stable-diffusion.cpp:3367 - diffusion context need 32.64MB static memory, with work_size needing 21.25MB
[INFO]  stable-diffusion.cpp:3496 - step 1 sampling completed, taking 61.77s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 2 sampling completed, taking 60.59s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 3 sampling completed, taking 68.61s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 4 sampling completed, taking 57.71s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 5 sampling completed, taking 58.78s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 6 sampling completed, taking 58.51s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 7 sampling completed, taking 59.81s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 8 sampling completed, taking 59.15s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 9 sampling completed, taking 57.99s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 10 sampling completed, taking 62.49s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 11 sampling completed, taking 58.17s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 12 sampling completed, taking 58.04s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 13 sampling completed, taking 58.53s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 14 sampling completed, taking 58.87s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 15 sampling completed, taking 57.54s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 16 sampling completed, taking 58.88s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 17 sampling completed, taking 58.62s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 18 sampling completed, taking 60.46s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 19 sampling completed, taking 59.86s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 20 sampling completed, taking 57.64s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3572 - diffusion graph use 6705.54MB of memory: params 1405.49MB, runtime 5300.04MB (static 32.64MB, dynamic 5267.40MB)
[DEBUG] stable-diffusion.cpp:3580 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3875 - sampling completed, taking 1192.06s
[DEBUG] stable-diffusion.cpp:3743 - vae context need 10.14MB static memory, with work_size needing 0.00MB
Segmentation fault (core dumped)

Green-Sky Sep 3, 2023

it also immediately crashes with --steps 1

leejet Sep 4, 2023
Maintainer Author

All the issues has been fixed. Also, I have set the default rng to cuda. You can pull the latest code and give it a try. Don't forget to update the submodule as well.

git pull origin master
git submodule update

Green-Sky Sep 4, 2023

Don't forget to update the submodule as well.

yes, always :)

tested and works. will run a more steps later, since large images take forever and my cpu is occupied :)

Details

 bin/sd -m ../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin -p "a lovely cat" --steps 1 -t 10 -W 1024 -H 1024
[INFO]  stable-diffusion.cpp:2802 - loading model from '../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin'
[INFO]  stable-diffusion.cpp:2830 - model type: SD2.x
[INFO]  stable-diffusion.cpp:2838 - ftype: q8_0
[INFO]  stable-diffusion.cpp:3066 - total params size = 1858.69MB (clip 358.69MB, unet 1405.49MB, vae 94.51MB)
[INFO]  stable-diffusion.cpp:3068 - loading model from '../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin' completed, taking 1.86s
[INFO]  stable-diffusion.cpp:3197 - check is_using_v_parameterization_for_sd2 completed, taking 0.24s
[INFO]  stable-diffusion.cpp:3091 - running in v-prediction mode
[INFO]  stable-diffusion.cpp:3325 - condition graph use 373.30MB of memory: params 358.69MB, runtime 14.61MB (static 10.70MB, dynamic 3.91MB)
[INFO]  stable-diffusion.cpp:3325 - condition graph use 373.30MB of memory: params 358.69MB, runtime 14.61MB (static 10.70MB, dynamic 3.91MB)
[INFO]  stable-diffusion.cpp:3864 - get_learned_condition completed, taking 0.66s
[INFO]  stable-diffusion.cpp:3880 - start sampling
[INFO]  stable-diffusion.cpp:3505 - step 1 sampling completed, taking 111.65s
[INFO]  stable-diffusion.cpp:3588 - diffusion graph use 6705.54MB of memory: params 1405.49MB, runtime 5300.04MB (static 32.64MB, dynamic 5267.40MB)
[INFO]  stable-diffusion.cpp:3885 - sampling completed, taking 111.70s
[INFO]  stable-diffusion.cpp:3805 - vae graph use 6760.65MB of memory: params 94.51MB, runtime 6666.14MB (static 10.14MB, dynamic 6656.00MB)
[INFO]  stable-diffusion.cpp:3898 - decode_first_stage completed, taking 169.61s
[INFO]  stable-diffusion.cpp:3912 - txt2img completed in 281.97s, use 6800.05MB of memory: peak params memory 1858.69MB, peak runtime memory 6666.14MB
save result image to 'output.png'

Green-Sky Sep 4, 2023

1024x1024 15steps

...
[DEBUG] stable-diffusion.cpp:3509 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3510 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3588 - diffusion graph use 6705.54MB of memory: params 1405.49MB, runtime 5300.04MB (static 32.64MB, dynamic 5267.40MB)
[DEBUG] stable-diffusion.cpp:3589 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3885 - sampling completed, taking 837.42s
[DEBUG] stable-diffusion.cpp:3754 - vae context need 10.14MB static memory, with work_size needing 0.00MB
[DEBUG] stable-diffusion.cpp:3782 - computing vae graph completed, taking 82.86s
[INFO]  stable-diffusion.cpp:3805 - vae graph use 6760.65MB of memory: params 94.51MB, runtime 6666.14MB (static 10.14MB, dynamic 6656.00MB)
[DEBUG] stable-diffusion.cpp:3806 - 12583936 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3898 - decode_first_stage completed, taking 82.95s
[INFO]  stable-diffusion.cpp:3912 - txt2img completed in 920.75s, use 6800.05MB of memory: peak params memory 1858.69MB, peak runtime memory 6666.14MB

bssrdf · 2023-12-28T20:58:26Z

bssrdf
Dec 28, 2023

@Green-Sky, could you share this model (artiusV21_v21_nsfw-ggml-model-q8_0.bin) somewhere? Thanks.

2 replies

Green-Sky Dec 29, 2023

the q8_0.bin specifically? the safetensor can be aquired here: https://civitai.com/models/27739/artius-v21 and https://civitai.com/models/15975/artius-v21-nsfw

bssrdf Dec 29, 2023

Never mind, I downloaded it from the link you provided. Thanks.

Support for SD2.x has been added, welcome to try it out #41

Uh oh!

leejet Sep 3, 2023 Maintainer

Replies: 3 comments · 14 replies

Uh oh!

Uh oh!

leejet Sep 3, 2023 Maintainer Author

Uh oh!

Uh oh!

Uh oh!

leejet Sep 3, 2023 Maintainer Author

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

leejet Sep 4, 2023 Maintainer Author

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

leejet
Sep 3, 2023
Maintainer

Replies: 3 comments 14 replies

leejet Sep 3, 2023
Maintainer Author

leejet Sep 3, 2023
Maintainer Author

leejet Sep 4, 2023
Maintainer Author