The reason row_flow_v3 is not working may be related to sdpa.
I specify a kernel other than math explicitly.

The reason row_flow_v3 is not working may be related to sdpa. I specify a kernel other than math explicitly.

nunif/nunif/modules/attention.py

Line 83 in e830a9a

with use_flash_attention(use_flash):

Looked at that so without sdpa there is no way to use flow methods (tried all of them, all of them freezes windows :) unlike giving a cuda error like other problems) , on the other hand, all of the previous methods -forward, backward, grid_sample and forward_fill- work. Of course forward_fill seems to be the best among them.

set use_flash=False uses math kernel as other models such as depth anything.
also If you use forward_fill, dev branch has a fixed version of it.

set use_flash=False uses math kernel as other models such as depth anything. also If you use forward_fill, dev branch has a fixed version of it.

regarding flash attention and sdpa this worked :

    def use_flash_attention(flag):
        if flag:
            return sdpa_kernel([SDPBackend.FLASH_ATTENTION, SDPBackend.EFFICIENT_ATTENTION, SDPBackend.MATH])
        else:
            return sdpa_kernel([SDPBackend.MATH])

to

    def use_flash_attention(flag):
        if flag:
           return sdpa_kernel([SDPBackend.MATH])

Is this the right way to do it ? It turned out to be a bit faster around %3 but the results seem identical to me not very different, anyway with this everything is working with zluda now.

Now, about that "fixed version" of forward_fill , is this across multiple files or is this wholly in a file , I don't want to just use dev branch as base if possible. EDIT : Found it, thanks.

I have never used AMD cards so I don't know anything.
If AMD doesn't seem willing to support native windows or lower models in rocm, I may support drectml on my end.

Would like know to how it would stack up against zluda here. With stable diffusion stuff it is 3-4x sometimes faster than directml. Still it would be an improvement.

On the other zoed thing, it is really much better than anyv2 just like quarter speed than it. I changed zoed width to 256 and it nearly took it upto half speed of anyv2. Lower values seem faster but now the effect is similar to any it doesnt worth it. Do we have other options that might make it faster ?

zoed_any_n is faster than zoed_n, less vram usage and better results.
If it doesn't work with batch-size 1, there is not enough vram.
With nvidia cards, it uses vram efficient sdpa, but you have disabled it with the patch above.

there is nothing I can do for zluda.
every problem that happens only on zluda are bugs in zluda. So you should report them to zluda developers, not to me.

there is nothing I can do for zluda. every problem that happens only on zluda are bugs in zluda. So you should report them to zluda developers, not to me.

Of course not. Was just asking your opinion , I am Not a programmer myself , this is just a hobby for me. Thanks a lot for creating this app. Going to take a look at that flow sdpa part when I have time later today. Thanks again.

Doesn't work for me, running iw3z.bat after install does nothing. pip install av >= 12.2.0, < 14.0.0 --quiet throws The system cannot find the path specified. error. Installed av manually in the venv, still no dice.

Change it into this and try again ::

pip install "av>=12.2.0,<14.0.0" --quiet

Forgot this, I uploaded an earlier file, going to upload the fixed one.

Yes, that worked, now GUI successfully starts, thank you.
By the way, is curl installed along with git on windows? I had to download it separately and add it to path, but that could be an issue on my end due to my setup.
And also I guess python version 3.10/3.11 is a hard requirement? Conversion doesn't start on 3.12 but I didn't extensively tested it atm since I rolled everything out on WSL for the time being.

Regarding curl : "If you have version 1803 or later of Windows 10, curl is installed by default."
That is actually why I am using it and tar (which also supposed to be added by default after windows 10) to install.

3.12 has some problems yes 10 is the most recommended regarding ai stuff, 11 works most of the time. But for the conversion not starting it , zluda is creating a database for the first time you run a model be it a stable diffusion model, audio model or upscaler or in this case depth estimation , it adds them to its database and after this one time setup for example you can do one sd 1.5 model and all sd1.5 and sdxl models work from now on. You need to do one flux generation to add that. Upscalers are also seems to be a different kind , so is this depth models.

If you have version 1803 or later of Windows 10, curl is installed by default.

I'm on W11 23H2, no idea why it was not installed. But anyway, it still would be good to mention that it is a requirement.

zluda is creating a database for the first time you run a model

I waited for about 5 minutes, gui was stuck at Loading ZoeD_K Model with no GPU/CPU activity. It worked on when I ran it with CPU. I will test this again later on python 3.10.

If you have version 1803 or later of Windows 10, curl is installed by default.

I'm on W11 23H2, no idea why it was not installed. But anyway, it still would be good to mention that it is a requirement.

zluda is creating a database for the first time you run a model

I waited for about 5 minutes, gui was stuck at Loading ZoeD_K Model with no GPU/CPU activity. It worked on when I ran it with CPU. I will test this again later on python 3.10.

Wait more, it takes a while. try with smallest model first like anyv2s to make sure this works. There would be no cpu-gpu activity it is doing shader compilation I think -not sure- which uses gpu but not wholly actually I am usually trying new models in comfyui when I am playing a small game at the same time, doesn't effect it very much. TLDR : wait.

Git: Download from https://git-scm.com/download/win. During installation don't forget to check the box for "Use Git from the Windows Command line and also from 3rd-party-software" to add Git to your system's PATH.
Python (3.10.11 3.11 also works, but 3.10 is used by most popular nodes atm): Install the latest release from python.org. Don't Use Windows Store Version. If you have that installed, uninstall and please install from python.org. During installation remember to check the box for "Add Python to PATH when you are at the "Customize Python" screen.
Visual C++ Runtime: Download vc_redist.x64.exe and install it.
Install HIP SDK 5.7.1 from HERE the correct version, "Windows 10 & 11 5.7.1 HIP SDK"
To see system variables : Right click My Computer - Properties - Advanced System Settings (on the menu right side) - Environment Variable. Add the system variable HIP_PATH, value: C:\Program Files\AMD\ROCm\5.7\ (This is the default folder, if you have installed it on another drive, change if necessary)
Check the variables on the lower part (System Variables), there should be a variable called: HIP_PATH.
Also check the variables on the lower part (System Variables), there should be a variable called: "Path". Double-click it and click "New" add this: C:\Program Files\AMD\ROCm\5.7\bin
If you have an AMD GPU below 6800 (6700,6600 etc.), download the recommended library files for your gpu from Brknsoul Repository
Go to folder "C:\Program Files\AMD\ROCm\5.7\bin\rocblas", there would be a "library" folder, backup the files inside to somewhere else.
Open your downloaded optimized library archive and put them inside the library folder (overwriting if necessary): "C:\Program Files\AMD\ROCm\5.7\bin\rocblas\library"
Reboot your system.

Setup (Windows-Only)

Open a cmd prompt. (Powershell doesn't work, you have to use command prompt.)

[ You open command prompt via typing "cmd" in start / run OR the easier way going into the drive or directory you want to install nunif/iw3 to on explorer, click on the address bar and type "cmd" press enter, this would open a commandline window on the directory you are in on explorer at the moment. ]

git clone https://github.com/patientx/nunif-amd

cd nunif-amd

installzludaforamd.bat

to start for later use (or create a shortcut to) :

iw3z.bat

First you have that python error which I've never seen , "ImportError: Use of the python311.dll module conflicts with this version of Python." And because of that error you don't have venv properly setup so the rest of the install goes down all together.

Second, you don't seem to have curl or tar which are installed by default after windows 10 22h2 I think, so they are on both w10 and 11 by default. So for the first problem you have to manually download and "patch" the zluda files. If you can solve your python problem.

@patientx
This might not be my business, but,
Windows environments can be very complicated. Sometimes offical python, windows store python, and anaconda are all installed. Sometimes a software has built-in python or mingw curl and it is appended to PATH.

So, my nunif-windows-package avoids those problems by installing Embeddable Python with isolation mode and restricting PATH.
Maybe building the environment for AMD on top of nunif-windows-package would make troubleshooting easier.

@patientx This might not be my business, but, Windows environments can be very complicated. Sometimes offical python, windows store python, and anaconda are all installed. Sometimes a software has built-in python or mingw curl and it is appended to PATH.

So, my nunif-windows-package avoids those problems by installing Embeddable Python with isolation mode and restricting PATH. Maybe building the environment for AMD on top of nunif-windows-package would make troubleshooting easier.

That would work of course. I am using it on top of the default setup your app normally provides
-albeit removed some packages that are not needed or incompatible with zluda, so there are a few potential problems-

Thanks for the help, and sorry for the inconvenience but after hours of trying I can't get it to work, finally I'll have to buy an Nvidia graphics card and be able to enjoy the IW3 program, which is an authentic work of art.
Greetings

It responds with cuDNN error: cuDNN_STATUS_INTERNAL_ERROR. Also, no matching distribution is found for truststore when I try to install the necessary packages.

I just realized I switched to python 3.9 while trying to get this working. Will switch back to 3.10 and let you know how it goes.

Nope, still the same error. (Without the truststore issue.) Any ideas why the above error might occur?

Same on my end

this is fixed in the latest version, just git pull or do fresh install. (git pull should work though)

do a fresh install maybe, it works here with everything is fixed, latest added models also work. if these don't work for you I have another idea for newest gpu's such as yours. But first try fresh install. Also delete zluda.db in "C:\Users\yourusername\AppData\Local\ZLUDA\ComputeCache" for last resort.

working fine after fresh install, thanks again its really appreciated

im getting that same error too and i just installed it minutes ago from the fork. deleting zluda.db did nothing. rog ally xtreme. also, when installing, do i start installzludaforamd.bat and then installerforwindows.bat? because the installerforwindows.bat wasnt in the github instructions but i did it anyway in that order. doing it in the opposite order caused the program to not start. Edit: I saw that stable diffusion amd used 6.2 so i had to install 5.7 which it installed in a diff folder and didnt overwrite my 6.2. that fixed the issue.

but the installerforwindows.bat is not in the tutorial so it should be included so it wont throw ppl off and i have a 78om apu so i had to download the rocm libraries for 1103 specifically for 5.7 and do the replacing in that step.

Why would you use installerforwindows ? We are using zluda for amd only. I didn't mention using it anywhere either, it is for the original app not my fork. Only use installzludaforamd.bat. If you use hip 6.2 -you can use it- you than have to change the patched zluda to one that is made for 6.2 and also you need find change the libs for 6.2 as well...

see i didnt know we werent supposed to use that, you should indicate that in the tutorial to specifically not use it. that's how a user informed ppl to install it. installforwindows.bat is not even in the default nunif program.

I didn't know we could do that with the hip either because it is not stated in the tutorial. tutorial specifically states 5.7, so i assumed that's the only compatible version like with many other ai things.

We also need a progress bar for installzludaforamd.bat

But it works now, thx.
Edit: now it stays on loading Distill_Any_B forever when trying to convert an image. cmd window is empty.

rocm.gfx1103.AMD.780M.phoenix.V5.0.for.hip.sdk.6.2.4.7z so this is the file i need to replace that folder in program files. inside is a library folder AND a rocblas.dll. No tutorial says what to do with the rocblas.dll. but there's also a rocblas.dll in the ROCm/6.2/bin folder. So i commented out the orginal rocblas.dll and put in this one, or should i have kept the original rocblas.dll (i renamed the original to rocblas.dll.bak)??

also, this was what was in this section already. your tutorial says to use double slants but it was already in as single slants. should I change it to double slants??

edit: so i ignored the dll and kept the original, put in the double slants, deleted the HIP_PATH_62, and restarted. now instead of hanging, iw3 silent crashes.....

so i'm lost......

1. what to do with the rocblas.dll that the tutorial doesn't mention?
2. should i keep both hip_path and hip_path_62?
3. should i use the double slants or the single slants?

Edit: this post says i do copy the dll to the bin folder which your tutorial excludes: vosen/ZLUDA#59 (comment)

also, they shouldve added the appropriate directories within that dll file in the zip so we can just drag and drop, so it's also an error on their part explaining where things go. seeeeeeeeeeEEEEEEEEEEEEEEEEEEEEEE

i'd also like to add that with my cpu, for a 1080p 10 min clip, it took 24hrs. 38 with tta checked. now it takes 4.5hrs with tta checked and 3.1hrs with tta unchecked when selecting the graphics card. nice! 780m apu rog ally xtreme

I haven't tried everything in depth in this app so tell me what does it supposed to do and with what setting I can test if it works or not ? Added the code (two parts) into my fork and we have to use "any" anyv2 models with foreground larger or equal to 1 ? Is that it ? If that is the case it runs well on gpu ? Dunno about the results if good or bad but it seems to run fully on gpu.

that's what im getting on gpu but dont worry about it because im not getting good results with it on cpu

hip path, check hip paths. System enviromental variables. As pointed on readme on github.

I've done that. It's still not working.

try to install it outside of user folder, aka c:\ d:\ etc aka non-system.

didnt work

updated BIOS , trying again!

no, still driver timeout - cannot get through a full conversion

try a different driver maybe something lower, other than that I don't have a definitive answer for you.

i've overclocked the gpu and its working great

It results in identical left-right views on any model besides "forward_fill." And when I run it on the "forward_fill" model, the output is identical left-right views at the back of the projection.

Found out I have gfx90c, not gfx1012. It worked. (I wish there were an easier way to tell which graphics card I own.)

its working.... seems a lot more stable.... thanks again

when's the update bat coming out XD

too much change to update, please install from zero

when's the update bat coming out XD

too much change to update, please install from zero

no it isnt. look like the install for amd takes care of 98% of it now. all that's needed is for you to auto add then environmental variables through the .bat if they dont exist. and install the proper hip sdk if it doesnt exist. But since i preinstalled this already i dont need to add it again. the only thing we should do is get the modded files for older gpus. if topaz adds those automatically so can you :)

when's the update bat coming out XD

too much change to update, please install from zero

no it isnt. look like the install for amd takes care of 98% of it now. all that's needed is for you to auto add then environmental variables through the .bat if they dont exist. and install the proper hip sdk if it doesnt exist. But since i preinstalled this already i dont need to add it again. the only thing we should do is get the modded files for older gpus. if topaz adds those automatically so can you :)

What does this mean , if you think it was easy then you should have done it instead of asking here ... I had to reset the github because there were some changes that had to be manually carried over and many not needed anymore so I choose to do it all over again from the start. And what does "topaz" supposed to mean ?

also, it's still not working for me as youve presented it. i still have to take my zluda folder from stable diffusion amd and replace iw3 zluda folder in order for the problem to not silent crash. this is what's in my sd amd zluda folder:

this is what's in your zluda folder:

i take that back, it's not even working that way anymore. it silent crashes when running iw3z.bat.

was the previous method working , I mean 5.7.1 and zluda 3.8.4 ,for you ? To be honest I am not sure your gpu would work without problems with 6.2.x and latest zluda. If you confirm previous method was working, please uninstall hip 6.2 , make sure the directory inside program files\amd\rocm is also completely deleted and make sure no path values left from 6.2 (env command on start , run) then install 5.7.1 , with your lib files for your gpu. I can give you a simple bat that would install those older torch versions with the old zluda -3.8.4-

Already went back to 5.7.1 using files saved to my computer from before. Still works, but could use the bat file for further reference.

@Echo off
cls
echo.
echo :: Activating virtual environment
Call "venv\Scripts\activate.bat"
echo :: Uninstalling previous torch packages
pip uninstall torch torchvision torchaudio -y --quiet
rmdir /S /Q "venv\Lib\site-packages\torch" 2>nul
echo :: Installing torch 2.3 and torchvision 0.18
pip install torch==2.3.0 torchvision==0.18.0 --index-url https://download.pytorch.org/whl/cu118 --quiet
:: pip install -r requirements.txt --quiet
:: pip install -r requirements-gui.txt --quiet
rmdir /S /Q zluda 2>nul
%SystemRoot%\system32\curl.exe -sL --ssl-no-revoke https://github.com/lshqqytiger/ZLUDA/releases/download/rel.c0804ca624963aab420cb418412b1c7fbae3454b/ZLUDA-windows-rocm5-amd64.zip > zluda.zip
%SystemRoot%\system32\tar.exe -xf zluda.zip
del zluda.zip
copy zluda\cublas.dll venv\Lib\site-packages\torch\lib\cublas64_11.dll /y >NUL
copy zluda\cusparse.dll venv\Lib\site-packages\torch\lib\cusparse64_11.dll /y >NUL
copy venv\Lib\site-packages\torch\lib\nvrtc64_112_0.dll venv\Lib\site-packages\torch\lib\nvrtc_cuda.dll /y >NUL
copy zluda\nvrtc.dll venv\Lib\site-packages\torch\lib\nvrtc64_112_0.dll /y >NUL
@echo :: ZLUDA is patched. (Zluda 3.8.4)
echo.
pause

save this as oldtorch.bat or whatever you want, this would patch torch 2.3 and put old zluda 3.8.4 on it, if you need old zluda with the patches now deleted in the new one tell me I would give you that one too.

I'll need that too. (Again for reference.) Thanks very much!

zluda for everything -audio fixes , onnx fix etc , some of these isn't necessary for nunif but it would be useful for other projects-

# ------------------- Hide ROCm/HIP -------------------
import os

os.environ.pop("ROCM_HOME", None)
os.environ.pop("HIP_HOME", None)
os.environ.pop("ROCM_VERSION", None)

paths = os.environ["PATH"].split(";")
paths_no_rocm = [p for p in paths if "rocm" not in p.lower()]
os.environ["PATH"] = ";".join(paths_no_rocm)
# ------------------- End ROCm/HIP Hiding -------------

# Fix for cublasLt errors on newer ZLUDA (if no hipblaslt)
os.environ['DISABLE_ADDMM_CUDA_LT'] = '1'

import torch

# ------------------- ZLUDA Detection -------------------
zluda_device_name = torch.cuda.get_device_name() if torch.cuda.is_available() else ""
is_zluda = zluda_device_name.endswith("[ZLUDA]")
# ------------------- End Detection --------------------

# ------------------- Top-K Fallback Patch -------------------
if is_zluda:
    _topk = torch.topk

    def safe_topk(input: torch.Tensor, *args, **kwargs):
        device = input.device
        values, indices = _topk(input.cpu(), *args, **kwargs)
        return torch.return_types.topk((values.to(device), indices.to(device),))

    torch.topk = safe_topk
# ------------------- End Top-K Patch -------------------

# ------------------- ONNX Runtime Patch -------------------
try:
    import onnxruntime as ort

    if is_zluda:
        print("\n***----------------------ZLUDA-----------------------------***")
        print("  ::  Patching ONNX Runtime for ZLUDA — disabling CUDA EP.")

        # Store original get_available_providers
        original_get_available_providers = ort.get_available_providers

        def filtered_providers():
            return [ep for ep in original_get_available_providers() if ep != "CUDAExecutionProvider"]

        # Patch ONLY the _pybind_state version (used during session creation)
        ort.capi._pybind_state.get_available_providers = filtered_providers

        # Wrap InferenceSession to force CPU provider when CUDA is explicitly requested
        OriginalSession = ort.InferenceSession

        class SafeInferenceSession(OriginalSession):
            def __init__(self, *args, providers=None, **kwargs):
                if providers and "CUDAExecutionProvider" in providers:
                    print("  ::  Forcing ONNX to use CPUExecutionProvider instead of CUDA.")
                    providers = ["CPUExecutionProvider"]
                super().__init__(*args, providers=providers, **kwargs)

        ort.InferenceSession = SafeInferenceSession
except ImportError:
    print("  ::  ONNX Runtime not installed — skipping patch.")
except Exception as e:
    print("  ::  Failed to patch ONNX Runtime:", e)
# ------------------- End ONNX Patch -------------------

# ------------------- ZLUDA Backend Patch -------------------
if is_zluda:
    print("  ::  ZLUDA detected, disabling non-supported functions.      ")
    torch.backends.cudnn.enabled = False

    if hasattr(torch.backends.cuda, "enable_flash_sdp"):
        torch.backends.cuda.enable_flash_sdp(False)
    if hasattr(torch.backends.cuda, "enable_math_sdp"):
        torch.backends.cuda.enable_math_sdp(True)
    if hasattr(torch.backends.cuda, "enable_mem_efficient_sdp"):
        torch.backends.cuda.enable_mem_efficient_sdp(False)
    print("  ::  CuDNN, flash_sdp, mem_efficient_sdp disabled.           ")
 
if is_zluda:
    print(f"  ::  Using ZLUDA with device: {zluda_device_name}")
    print("***--------------------------------------------------------***\n")
else:
    print(f"  ::  CUDA device detected: {zluda_device_name or 'None'}")
    print("***--------------------------------------------------------***\n")
# ------------------- End Zluda detection -------------------

save it as zluda.py , and import it in your project , or copy paste it inside zluda.py if there is already one ...

I mean use the iw3z bat from inside the archive instead of the one already present in the folder.

struggling with this part:

-- That's it , installation is complete, to run it correctly we have to do something specific.

remember the hip 6.5 directory we just extracted , like mine was "d:\sd\65" now with this information in hand , create this batch file

(***) Change comfyui folder locations and hip path location according to where you cloned comfyui and where you EXTRACTED THAT HIP 6.5 DEV archive, NOT the hip that is already installed on your pc, so not "c:\program files\amd\rocm\hip\6.2" that is an installed program , we are not installing stuff we extract this into a folder and by using the "set HIP_PATH" command below in the batch file, we make comfy and torch's look first into that folder then system path's. So yes, this also means you can have hip 6.2 installed and can use it with my comfyui-zluda AND at the same time have this fork installed and usable in its own environment.

starter batch file example :::

@echo off
:: Set base directory (CHANGE THE PATH TO THE DIRECTORY YOU HAVE INSTALLED COMFYUI TO)
set COMFYUI_PATH=D:\sd\ComfyUI
cd /d "%COMFYUI_PATH%"

:: Activate virtual environment
call "%COMFYUI_PATH%\venv\Scripts\activate"

:: AMD ROCm configuration (CHANGE HIP_PATH and TRITON_OVERRIDE_ARCH to what you have)
set HIP_PATH=D:\65
set TRITON_OVERRIDE_ARCH=gfx1030
set PYTORCH_HIP_ALLOC_CONF=expandable_segments:True

:: Cache directories (using ComfyUI path)
set TRITON_CACHE_DIR=%COMFYUI_PATH%.triton
set MIOPEN_CACHE_DIR=%COMFYUI_PATH%.miopen

:: ComfyUI paths
set COMFYUI_MODEL_PATH=%COMFYUI_PATH%\models

:: MIOpen settings
set MIOPEN_FIND_MODE=2
set MIOPEN_LOG_LEVEL=3

:: Performance optimizations
set TF_ENABLE_ONEDNN_OPTS=0

:: Add ROCm to PATH (PLEASE CHANGE THEM ACCORDING TO WHERE YOU PUT HIP6.5 ZIP CONTENTS)
set PATH=D:\65\rocm;D:\65\bin;D:\65\cmake;D:\65\include;D:\65\lib;%COMFYUI_PATH%;%COMFYUI_MODEL_PATH%;%PATH%

:: Update ComfyUI and dependencies
echo Updating ComfyUI...
git pull --quiet 2>nul || echo Git pull failed or not a git repository
pip install -U comfyui-frontend-package comfyui-workflow-templates av comfyui-embedded-docs --quiet --no-warn-script-location

:: Launch ComfyUI
echo Starting ComfyUI...
python main.py --use-sage-attention --lowvram --reserve-vram 0
Change the COMFYUI_PATH and HIP_PATH according to where you put them , also change the TRITON_OVERRIDE_ARCH to your gpu's corresponding value.

(sage-attention doesn't work with sd1.5 , in that case use --use-quad-cross-attention instead of sage-attention)

save these in a text file, save it as anynameyouwant.bat in desktop for example , that's it. That is how you run this. It updates and also handles the frontend updates comfy frequently pushes.

Everything I have thrown to it worked for me, and almost always was faster then zluda with all the attention stuff. So with only quad-cross-attention it already runs better than what we have with zluda atm. Hopefully when amd officially gets these to windows -these are not official builds as you can see- we would get much wider support for gpu's and that type of technologies.

Added comfyui-rock-installer.bat and example comfyui-starter.bat in the zip attached. Once you clone the repo with git clone https://github.com/comfyanonymous/ComfyUI.git , copy both of these from inside the zip to inside that comfyui folder and run comfyui-rock-installer.bat from there.

you don't need that, the installer I sent auto downloads and extracts hip 6.5 also the custom starter bat puts it into path so it works without those steps. you just need to have visual studio build tools , git , python ... and than install it via the files I sent.

all working great! thanks so much for the support!