This project aims to build an interesting dialogue system utilizing the characteristics of StreamDiffusion.

Users are not talking to a common Chatbot in English, but in a kanji-like fake language, where responses are rendered with diffusion-based models.

We build this system based on StreamDiffusionIO, a modified version of StreamDiffusion that supports rendering text streams into image streams.

🔥 Mar 05, 2024 | Kanji Streaming is reposted by enpitsu (original author of Fake Kanji Generation) on X(twitter)!

⬆️ Mar 04, 2024 | We update the demo and it now allows to chat with mistralai/Mixtral-8x7B-Instruct-v0.1 with HF InferenceClient, which also significantly saves GPU memory usage (from ~18.5G ➡️ ~5G). Also checkout the demo deployed on Huggingface Space!

🔥 Mar 01, 2024 | Kanji Streaming is reposted by AK on X(twitter)!

🚀 Feb 29, 2024 | Kanji Streaming is released!

git clone https://github.com/AgainstEntropy/kanji.git
cd kanji

conda create -n kanji python=3.10
conda activate kanji
pip install -r requirements.txt

pip install StreamDiffusionIO

To pull the source code of StreamDiffusionIO, one can either do

git submodule update --init --recursive

or

git clone https://github.com/AgainstEntropy/StreamDiffusionIO.git

Then install StreamDiffusionIO in editable mode

pip install --editable StreamDiffusionIO/

• Llama-2
• stable-diffusion-v1-5
  • LoRA for kanji generation: This LoRA enables sd-1.5 to generate kanji at resolution of 128 with text condition.
  • LCM-LoRA for kanji generation: This LCM-LoRA turns the LDM into a LCM.

Run git submodule update --init --recursive to pull the code in demo folder.

Modify the arguments (e.g., model paths and conda installation path) in the launching scripts to match your case before running.

cd demo
sh run-app-mixtral.sh

cd demo
sh run-kanji-local_llama.sh

Check out the guide on reproducing Kanji generation model used in this project.

• Kanji Generation by enpitsu
• StreamDiffusion
• Latent Consistency Models