Small versions of known models: same vLLM loading characteristics as the base, fewer layers/experts, identical hidden dims and hyperparameters
-
inference-optimization/DeepSeek-V4-Pro-3.7B-A3.4B-small
Text Generation • 4B • Updated -
inference-optimization/Phi-3.5-MoE-1.0B-A0.7B-small
Text Generation • 1.0B • Updated -
inference-optimization/DeepSeek-V3-3.4B-A3.0B-small
Text Generation • 3B • Updated • 4 -
inference-optimization/Llama-3.2-0.5B-Instruct-small
Text Generation • 0.5B • Updated • 6