Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@WANGSSSSSSS
Copy link

What does this PR do?

Fixes bug in DDIMInverseSampler
The current sampler has not aligned the timestep with prediction, this caused two problem:

  1. the previous step may be underfloor, this is easy to validate, and we can print the previous step and current step.
#     1. get previous step value (=t+1)
        prev_timestep = timestep
        timestep = min(timestep - self.config.num_train_timesteps // self.num_inference_steps, self.num_train_timesteps - 1)
        print("prev_t:",prev_timestep.item(), end=" ")
        print("curr_t:",timestep.item(), end="\n")

the print results looks like:
image

  1. the next problem is the noise is predicted at the current step, but using the alpha and beta of the previous step.
    as the above code snippet suggests, the current timestep is overwritten by the code line, thus delivers wrong result.
timestep = min(timestep - self.config.num_train_timesteps // self.num_inference_steps, self.num_train_timesteps - 1)
  1. I think the previous name convention is not suitable in inverse problem, maybe next_timestep and timestep better.
  2. the results visualization
    the code book link: colab
    I use pix2pixZero demo to demonstrate:
    the misaligned sampler:
    image
    this pr:
    Screenshot_20231117_105104

Fixes # (issue)

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

fix misalignment bug of inverse problem
change name convention
@patrickvonplaten
Copy link
Contributor

Can you post a reproducible code snippet here?

@patrickvonplaten
Copy link
Contributor

Also note that there was a very similar PR recently: #5145

@WANGSSSSSSS
Copy link
Author

Can you post a reproducible code snippet here?

I guess it is hard to write a code snippt without access to the timesteps defined in step function (where the error occurs), so i copy the step function in current diffusers repo to reproduce, is this make sense? so i provide the codebook here:
https://colab.research.google.com/drive/18WWCFydYmkBRBPSraCvezLIiaCBTEvdD?usp=sharing

@WANGSSSSSSS
Copy link
Author

Also note that there was a very similar PR recently: #5145

the error still not be fixed, this is my codebook: https://colab.research.google.com/drive/18WWCFydYmkBRBPSraCvezLIiaCBTEvdD?usp=sharing
image

image

@github-actions
Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

@github-actions github-actions bot added the stale Issues that haven't received updates label Dec 26, 2023
@github-actions github-actions bot closed this Jan 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale Issues that haven't received updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants