A native Apple-silicon conversion of inclusionAI/Ling-3.0-flash, quantized with the stock MLX-LM converter and packaged for MLX-LM and oMLX.
## About this conversion This repository contains an **8-bit affine MLX conversion** of Ling 3.0 Flash. Ling is a 124B-parameter hybrid-linear mixture-of-experts model with approximately 5.1B active parameters per token. The conversion preserves the upstream tokenizer and chat template and includes an MLX architecture adapter for the model's KDA, MLA, and sparse-MoE layers. | Item | Value | | --- | --- | | Base model | [`inclusionAI/Ling-3.0-flash`](https://huggingface.co/inclusionAI/Ling-3.0-flash) | | Format | MLX safetensors | | Quantization | 8-bit affine, group size 64 | | Converter | Stock `mlx_lm.convert` from MLX-LM 0.31.3 | | Effective weight precision | 8.500 bits per weight | | Repository size | Approximately 132 GB (123 GiB) | | Maximum configured context | 262,144 tokens | | Architecture | `bailing_hybrid` | > [!IMPORTANT] > This model includes the custom architecture file `bailing_hybrid.py`. Review the file and enable **Trust Remote Code** before loading it. In oMLX, enable Trust Remote Code in the model settings. With MLX-LM, pass `--trust-remote-code`. ## Apple-silicon validation The converted weights were loaded strictly with both stock MLX-LM and the MLX runtime bundled with oMLX. End-to-end generation was also tested through the oMLX OpenAI-compatible API on an Apple-silicon Mac Studio. One short local smoke test produced: | Measurement | Result | | --- | ---: | | Prefill | 24.15 tokens/s | | Decode | 65.79 tokens/s | | Reported peak memory | 132.36 GB | | oMLX API | HTTP 200, valid completion | These figures confirm that the model runs; they are not a controlled benchmark. Prompt length, sampler settings, macOS memory pressure, thermal state, and MLX/oMLX versions can materially change performance. ## Quick start with MLX-LM Install recent MLX-LM and Hugging Face tooling: ```bash python -m pip install -U mlx-lm huggingface_hub ``` Run a chat prompt directly from the Hub: ```bash mlx_lm.generate \ --model TensorFold/Ling-3.0-flash-MLX-8bit \ --trust-remote-code \ --prompt "Explain why hybrid linear attention is useful." \ --max-tokens 512 \ --temp 0.6 \ --top-p 0.95 \ --top-k 20 ``` Thinking mode is enabled by the upstream chat template by default. It can be disabled with: ```bash mlx_lm.generate \ --model TensorFold/Ling-3.0-flash-MLX-8bit \ --trust-remote-code \ --chat-template-config '{"enable_thinking": false}' \ --prompt "Write a short hello-world program in Swift." ``` To download the repository first: ```bash hf download TensorFold/Ling-3.0-flash-MLX-8bit \ --local-dir ~/.omlx/models/TensorFold/Ling-3.0-flash-MLX-8bit ``` ## Using it with oMLX 1. Place the model at `~/.omlx/models/TensorFold/Ling-3.0-flash-MLX-8bit`. 2. Refresh the oMLX model registry. 3. Open the model settings and enable **Trust Remote Code**. 4. Load `Ling-3.0-flash-MLX-8bit` and use the normal chat or OpenAI-compatible endpoint. Example request: ```bash curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OMLX_API_KEY" \ -d '{ "model": "Ling-3.0-flash-MLX-8bit", "messages": [{"role": "user", "content": "Say hello from MLX."}], "temperature": 0.6, "top_p": 0.95, "max_tokens": 128 }' ``` For long prompts, begin with a conservative context limit and increase it while watching memory pressure. The advertised 256K context is a model capability, not a promise that every host can prefill that context within its available unified memory. ## Architecture Ling 3.0 Flash is a native hybrid-linear MoE model. It alternates Kimi Delta Attention (KDA) and gated Multi-head Latent Attention (MLA) in a 5:1 ratio and uses highly sparse routed experts. | Architecture detail | Upstream value | | --- | ---: | | Total / active parameters | 124B / 5.1B | | Transformer layers | 35 KDA + 7 gated MLA | | Dense layers | 2 | | Routed / shared experts | 512 / 1 | | Active routed experts | 8 | | Attention heads | 32 | | Hidden size | 2,560 | | Expert intermediate size | 768 | | Dense intermediate size | 6,144 | | Vocabulary size | 157,184 | | Context training schedule | 8K → 32K → 256K | The included MLX adapter uses MLX-LM's native primitives for delta attention, absorbed MLA projections, RoPE, and quantized SwitchGLU experts. The auxiliary MTP training head is intentionally excluded from ordinary causal generation; this release does **not** claim MTP/DSpark speculative decoding support. ## Upstream model highlights InclusionAI describes Ling 3.0 Flash as a next-generation hybrid reasoning model designed for efficient long-context inference and agentic workloads. The upstream release reports strong results across software-engineering agents, tool use, deep research, general knowledge, mathematical reasoning, instruction following, and long-context understanding. The upstream defaults are: ```text thinking: enabled temperature: 0.6 top_p: 0.95 top_k: 20 ``` For benchmark methodology, scores, limitations, and framework-specific SGLang/vLLM deployment instructions, see the [original InclusionAI model card](https://huggingface.co/inclusionAI/Ling-3.0-flash). ## Conversion notes - Source weights: BF16 upstream checkpoint. - Conversion command: stock `mlx_lm.convert` with `--quantize --q-bits 8 --q-group-size 64 --q-mode affine`. - The upstream `chat_template.jinja` is included unchanged. - All source tensors required for base causal generation were consumed by a strict load before conversion. - The 42 base transformer layers are retained. The separate depth-5 MTP training/speculation stack is not part of this runtime adapter. This is a community conversion, not an official InclusionAI release. Because the architecture adapter is new, validate quality and numerical behavior on your own representative workload before production use. ## License and attribution The upstream model is released under the **MIT License**. This conversion preserves that license and is derived from [`inclusionAI/Ling-3.0-flash`](https://huggingface.co/inclusionAI/Ling-3.0-flash). All model design, training, and benchmark credit belongs to InclusionAI and the original contributors. The MLX conversion and compatibility adapter are provided by [TensorFold](https://huggingface.co/TensorFold). ## Choose for your Mac [64GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-64gb-macs-6a9fefda17932216ec9ab457) · [128GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-128gb-macs-6a9ff0abd31bc9abbe7922d7) · [256GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-256gb-macs-6a9ff0ef9fed7c5bdca15e9b) Published peak memory: **132.36 GB**; estimated starting tier: **256GB**, leaving about **123 GB** nominal headroom. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request. ### Runtime and evidence The exact tested oMLX application version is not recorded here; a library version is not an app version. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results. ### Quick start and demo prompt ```bash hf download TensorFold/Ling-3.0-flash-MLX-8bit --local-dir ./models/Ling-3.0-flash-MLX-8bit ``` Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading. Try this in a new chat with a 128-token output limit: ```text Explain why the sky looks blue in three short sentences. ``` This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available. [Follow TensorFold for new Apple Silicon releases and fixes.](https://huggingface.co/TensorFold)