Update README.md
Browse files
README.md
CHANGED
|
@@ -16,24 +16,6 @@ Intended use-case is live-preview during generative model inference.
|
|
| 16 |
|
| 17 |
Shapes/Channels/etc are inferred from the base VAE, so no configuration changes are needed between different models.
|
| 18 |
|
| 19 |
-
## Models
|
| 20 |
-
|
| 21 |
-
Which MicroDecoder model can be used for which model types?
|
| 22 |
-
|
| 23 |
-
```json
|
| 24 |
-
'sd': ['sd', 'sdxs'],
|
| 25 |
-
'sdxl': ['sdxl', 'ldm', 'pixartalpha', 'pixartsigma', 'hunyuandit', 'omnigen', 'auraflow', 'bria', 'kolors', 'lumina'],
|
| 26 |
-
'f1': ['f1', 'h1', 'zimage', 'lumina2', 'chroma', 'longcat', 'omnigen2', 'flite', 'ovis', 'kandinsky5', 'glmimage', 'cogview3', 'cogview4', 'ultraflux', 'boogu', 'step1x_edit'],
|
| 27 |
-
'f2': ['f2', 'f3', 'ernieimage', 'lens', 'ideogram4', 'lladaimage', 'mossimage', 'sefi'],
|
| 28 |
-
'sd3': ['sd3'],
|
| 29 |
-
'wan21': ['wanai', 'qwen', 'chrono', 'cosmos', 'anima', 'fibo', 'joy', 'krea2', 'mingimage', 'nucleusimage'],
|
| 30 |
-
'minimaxh3': ['minimaxh3'],
|
| 31 |
-
'qwen21': ['qwen21'],
|
| 32 |
-
'dcae': ['sana'],
|
| 33 |
-
'mage': ['mageflow'],
|
| 34 |
-
'hy': ['hunyuanimage'],
|
| 35 |
-
```
|
| 36 |
-
|
| 37 |
## Code
|
| 38 |
|
| 39 |
- Model definition [here](https://github.com/vladmandic/sdnext/blob/dev/modules/vae/sd_vae_micro_model.py)
|
|
@@ -76,4 +58,69 @@ Encode ━━━━━━━━━━━━━━━━━━━━ 400/400 100%
|
|
| 76 |
Train ━━━━━━━━━━━━━━━━━━━━ 400/400 100% 0:00:00 0:06:33 Train(psnr=21.029 ssim=0.640 lpips=2.784 l1=0.089 lab=0.156 grad=0.073 sat=0.124 fft=0.071 total=4.536) Validate(psnr=30.607 ssim=0.927 lpips=1.242 l1=0.020 lab=0.046 grad=0.046 sat=0.057 fft=0.032 total=1.964)
|
| 77 |
Complete: Time=393.89 Epoch/Sec=1.02 PSNR=30.61 dB @ Epoch=400
|
| 78 |
Save: filename="microdecoder-minimaxh3.safetensors"
|
| 79 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
Shapes/Channels/etc are inferred from the base VAE, so no configuration changes are needed between different models.
|
| 18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
## Code
|
| 20 |
|
| 21 |
- Model definition [here](https://github.com/vladmandic/sdnext/blob/dev/modules/vae/sd_vae_micro_model.py)
|
|
|
|
| 58 |
Train ━━━━━━━━━━━━━━━━━━━━ 400/400 100% 0:00:00 0:06:33 Train(psnr=21.029 ssim=0.640 lpips=2.784 l1=0.089 lab=0.156 grad=0.073 sat=0.124 fft=0.071 total=4.536) Validate(psnr=30.607 ssim=0.927 lpips=1.242 l1=0.020 lab=0.046 grad=0.046 sat=0.057 fft=0.032 total=1.964)
|
| 59 |
Complete: Time=393.89 Epoch/Sec=1.02 PSNR=30.61 dB @ Epoch=400
|
| 60 |
Save: filename="microdecoder-minimaxh3.safetensors"
|
| 61 |
+
```
|
| 62 |
+
|
| 63 |
+
## Models
|
| 64 |
+
|
| 65 |
+
MicroDecoder includes following pre-trained models: `sd, sdxl, f1, f2, sd3, wan21, wan22, minimaxh3, qwen21, dcae, mage, cog, hy`
|
| 66 |
+
And each VAE model can be used for different base models as long as they share the same VAE architecture
|
| 67 |
+
|
| 68 |
+
### VAE to Model Map
|
| 69 |
+
|
| 70 |
+
Which MicroDecoder model can be used for which model types?
|
| 71 |
+
|
| 72 |
+
```json
|
| 73 |
+
'sd': ['sd',
|
| 74 |
+
'sdxs'],
|
| 75 |
+
'sdxl': ['sdxl',
|
| 76 |
+
'ldm',
|
| 77 |
+
'pixartalpha',
|
| 78 |
+
'pixartsigma',
|
| 79 |
+
'hunyuandit',
|
| 80 |
+
'omnigen',
|
| 81 |
+
'auraflow',
|
| 82 |
+
'bria',
|
| 83 |
+
'kolors',
|
| 84 |
+
'lumina'],
|
| 85 |
+
'f1': ['f1',
|
| 86 |
+
'h1',
|
| 87 |
+
'zimage',
|
| 88 |
+
'lumina2',
|
| 89 |
+
'chroma',
|
| 90 |
+
'longcat',
|
| 91 |
+
'omnigen2',
|
| 92 |
+
'flite',
|
| 93 |
+
'ovis',
|
| 94 |
+
'kandinsky5',
|
| 95 |
+
'ultraflux',
|
| 96 |
+
'boogu',
|
| 97 |
+
'step1x_edit'],
|
| 98 |
+
'f2': ['f2',
|
| 99 |
+
'f3',
|
| 100 |
+
'ernieimage',
|
| 101 |
+
'lens',
|
| 102 |
+
'ideogram4',
|
| 103 |
+
'lladaimage',
|
| 104 |
+
'mossimage',
|
| 105 |
+
'sefi',
|
| 106 |
+
'fibo2'],
|
| 107 |
+
'sd3': ['sd3'],
|
| 108 |
+
'wan21': ['wanai',
|
| 109 |
+
'qwen',
|
| 110 |
+
'chrono',
|
| 111 |
+
'cosmos',
|
| 112 |
+
'anima',
|
| 113 |
+
'joy',
|
| 114 |
+
'krea2',
|
| 115 |
+
'mingimage',
|
| 116 |
+
'nucleusimage'],
|
| 117 |
+
'wan22': ['wan22',
|
| 118 |
+
'fibo'],
|
| 119 |
+
'minimaxh3': ['minimaxh3'],
|
| 120 |
+
'qwen21': ['qwen21'],
|
| 121 |
+
'dcae': ['sana'],
|
| 122 |
+
'mage': ['mageflow'],
|
| 123 |
+
'cog': ['cogview3',
|
| 124 |
+
'cogview4'],
|
| 125 |
+
'hy': ['hunyuanimage'],
|
| 126 |
+
```
|