Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Basic dataset pipelines #139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
cloneofsimo merged 10 commits into from
Jan 16, 2023
Merged

Basic dataset pipelines #139

cloneofsimo merged 10 commits into from
Jan 16, 2023

Conversation

@cloneofsimo
Copy link
Owner

@cloneofsimo cloneofsimo commented Jan 15, 2023 •
edited
Loading

  • Basic stuff : Preprocessor that handles SR, BLIP for autocaption.

Further tricks are planned to go to other repo. Might as well remove these in the future as well, but I'll have these for quick CLI sake

  • SR with SwinIR
  • BLIP autocaption
  • CLIPSeg
  • Face Detection
  • Support Mask in dataset
  • Magic Saving CLI

@cloneofsimo
Copy link
Owner Author

cloneofsimo commented Jan 16, 2023

image

Simple CLI will now take any directory to create BLIP captioned + SwinIR upscaled + CLIPSeged mask.

@cloneofsimo
Copy link
Owner Author

cloneofsimo commented Jan 16, 2023

Example output :

image

@cloneofsimo
Copy link
Owner Author

cloneofsimo commented Jan 16, 2023

Usage :

lora_ppim ./data/data_cat ./data/data_catpp, --target_prompts="cat"

@Dango233
Copy link

Dango233 commented Jan 16, 2023 •
edited
Loading

Aw...we built the same stuff pretty much at the same time
Masking is helping with the final quality massively.

@cloneofsimo
Copy link
Owner Author

cloneofsimo commented Jan 16, 2023

By we do you mean stability AI? BTW I've actually developed masking quite a long time ago haha #96

@Dango233
Copy link

Dango233 commented Jan 16, 2023

I mean you and me lol. I was exploring the data pipeline for fine-tuning last weekend.

And then I saw your PR lol

@cloneofsimo
Copy link
Owner Author

cloneofsimo commented Jan 16, 2023

Oh yes, lots of room for improvement here. Would gladly accept any PRs 🤗

@cloneofsimo cloneofsimo marked this pull request as ready for review January 16, 2023 19:03
@cloneofsimo cloneofsimo merged commit 06fa37a into develop Jan 16, 2023
@cloneofsimo cloneofsimo deleted the feat/data-pipeline branch February 2, 2023 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@cloneofsimo @Dango233