Stable Diffusion: 练习特定人物模子
奇闻趣事 2023-05-04 12:32www.bnfh.cn奇闻趣事
Stable Diffusion: 训练特定人物模型
本次训练全程在colab上进行
1，连接云端硬盘
from google.colab import drive
drive.mount('/content/drive')2，启动 StableDiffusion Web UI
%cd /content
%env TF_CPP_MIN_LOG_LEVEL=1
!apt -y update -qq
!get http://launchpadlibrarian./367274644/libgoogle-perftools-dev_2.5-2.2ubuntu3_amd64.deb
!get http://launchpad./ubuntu/+source/google-perftools/2.5-2.2ubuntu3/+build/14795286/+files/google-perftools_2.5-2.2ubuntu3_all.deb
!get http://launchpad./ubuntu/+source/google-perftools/2.5-2.2ubuntu3/+build/14795286/+files/libtcmalloc-minimal4_2.5-2.2ubuntu3_amd64.deb
!get http://launchpad./ubuntu/+source/google-perftools/2.5-2.2ubuntu3/+build/14795286/+files/libgoogle-perftools4_2.5-2.2ubuntu3_amd64.deb
!apt install -qq libunind8-dev
!dpkg -i .deb
%env LD_PRELOAD=libtcmalloc.so
!rm .deb
!apt -y install -qq aria2
!pip install -q torch==1.13.1+cu116 torchvision==0.14.1+cu116 torchaudio==0.13.1 torchtext==0.14.1 torchdata==0.5.1 --extra-index-url http://donload.pytorch./hl/cu116 -U
!pip install -q xformers==0.0.16 triton==2.0.0 -U
!git clone -b v1.9 http://github./camenduru/stable-diffusion-ebui
!git clone http://huggingface.co/embed/negative /content/stable-diffusion-ebui/embeddings/negative
!git clone http://huggingface.co/embed/lora /content/stable-diffusion-ebui/models/Lora/positive
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M http://huggingface.co/embed/upscale/resolve/main/4x-UltraSharp.pth -d /content/stable-diffusion-ebui/models/ESRGAN -o 4x-UltraSharp.pth
!get http://ra.githubusercontent./camenduru/stable-diffusion-ebui-scripts/main/run_n_times.py -O /content/stable-diffusion-ebui/scripts/run_n_times.py
!git clone -b v1.9 http://github./camenduru/deforum-for-automatic1111-ebui /content/stable-diffusion-ebui/extensions/deforum-for-automatic1111-ebui
!git clone -b v1.9 http://github./camenduru/stable-diffusion-ebui-images-broser /content/stable-diffusion-ebui/extensions/stable-diffusion-ebui-images-broser
!git clone -b v1.9 http://github./camenduru/stable-diffusion-ebui-huggingface /content/stable-diffusion-ebui/extensions/stable-diffusion-ebui-huggingface
!git clone -b v1.9 http://github./camenduru/sd-civitai-broser /content/stable-diffusion-ebui/extensions/sd-civitai-broser
!git clone -b v1.9 http://github./camenduru/sd-ebui-additional-orks /content/stable-diffusion-ebui/extensions/sd-ebui-additional-orks
!git clone -b v1.9 http://github./camenduru/sd-ebui-tunnels /content/stable-diffusion-ebui/extensions/sd-ebui-tunnels
!git clone -b v1.9 http://github./camenduru/batchlinks-ebui /content/stable-diffusion-ebui/extensions/batchlinks-ebui
!git clone -b v1.9 http://github./camenduru/stable-diffusion-ebui-catppuin /content/stable-diffusion-ebui/extensions/stable-diffusion-ebui-catppuin
!git clone -b v1.9 http://github./camenduru/a1111-sd-ebui-locon /content/stable-diffusion-ebui/extensions/a1111-sd-ebui-locon
!git clone -b v1.9 http://github./camenduru/stable-diffusion-ebui-rembg /content/stable-diffusion-ebui/extensions/stable-diffusion-ebui-rembg
!git clone -b v1.9 http://github./camenduru/stable-diffusion-ebui-to-shot /content/stable-diffusion-ebui/extensions/stable-diffusion-ebui-to-shot
!git clone -b v1.9 http://github./camenduru/sd_ebui_stealth_pnginfo /content/stable-diffusion-ebui/extensions/sd_ebui_stealth_pnginfo
%cd /content/stable-diffusion-ebui
!git reset --hard
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M http://huggingface.co/ckpt/stable-diffusion-2-1/resolve/main/v2-1_768-ema-pruned.ckpt -d /content/stable-diffusion-ebui/models/Stable-diffusion -o v2-1_768-ema-pruned.ckpt
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M http://huggingface.co/ckpt/stable-diffusion-2-1/ra/main/v2-inference-v.yaml -d /content/stable-diffusion-ebui/models/Stable-diffusion -o v2-1_768-ema-pruned.yaml
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M http://huggingface.co/joachimsallstrom/Double-Exposure-Embedding/resolve/main/dblx.pt -d /content/stable-diffusion-ebui/embeddings -o dblx.pt
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M http://huggingface.co/Conflictx/AnimeScreencap/resolve/main/AnimeScreenCap.pt -d /content/stable-diffusion-ebui/embeddings -o AnimeScreenCap.pt
!sed -i -e '''/ prepare_environment()/a os.system(f"""sed -i -e ''"s/dict()))/dict())).cuda()/g"'' /content/stable-diffusion-ebui/repositories/stable-diffusion-stability-ai/ldm/util.py""")''' /content/stable-diffusion-ebui/launch.py
!sed -i -e 's/fastapi==0.90.1/fastapi==0.89.1/g' /content/stable-diffusion-ebui/requirements_versions.txt
!mkdir /content/stable-diffusion-ebui/extensions/deforum-for-automatic1111-ebui/models
!sed -i -e 's/"sd_model_checkpoint",/"sd_model_checkpoint,sd_vae,CLIP_s_at_last_layers",/g' /content/stable-diffusion-ebui/modules/shared.py
!python launch.py --listen --xformers --enable-insecure-extension-aess --theme dark --gradio-queue --multiple3，上传原始照片
我们上传10张朱丽叶基德曼的照片
4，开始训练
Train Tab: 已经集成了Textual Inversion，可以用来训练现有神经网络，创造出新的概念，比如特定的人物、特定的风格等等。
4.1 Preprocess Image 图片预处理
4.2 Create Embedding 创建模型
创建embedding给模型取个名字，这个非常重要，训练完成后，将会使用这个名字，作为触发的token。
Number of vectors per token:每个token的vector个数，如果图片较少，这里的值可以设置的大一些。
4.3 Train 训练模型
主要参数，见下图标识
Prompt模板内容
a photo of [name]关于参数具体的取值需要积累经验，根据不同的图片数量、质量，进行调优。
训练完成后，在/content/stable-diffusion-ebui/textual_inversion目录以及/content/stable-diffusion-ebui/embeddings目录生成了.pt文件，非常小，一般只有几十kb。可以复制保存，随时使用。
5，使用
a photo of modelofxyzmodelofxyz为之前你给模型取的名字
使用 Script Prompt S/R分析一下哪种CFG以及哪个阶段的模型效果最好
总的看来，较早阶段/较小CFG，人物饱和度较低，反之饱和度高。
下面开始使用Prompt，尝试不同的各种场景描述
a photo of modelofxyz-2000, alking at London streetCFG:4
a photo of modelofxyz-2000, earing Chinese Tang clothingCFG:5
a photo of modelofxyz-2000, earing fashion long dressa photo of modelofxyz-2000, (earing EVA spacesuit:1.1)其它