Text Generation
MLX
Safetensors
bailing_hybrid
mlx-lm
omlx
mixture-of-experts
quantized
8-bit precision
conversational
custom_code
Instructions to use TensorFold/Ling-3.0-flash-MLX-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use TensorFold/Ling-3.0-flash-MLX-8bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("TensorFold/Ling-3.0-flash-MLX-8bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use TensorFold/Ling-3.0-flash-MLX-8bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "TensorFold/Ling-3.0-flash-MLX-8bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "TensorFold/Ling-3.0-flash-MLX-8bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use TensorFold/Ling-3.0-flash-MLX-8bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "TensorFold/Ling-3.0-flash-MLX-8bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "TensorFold/Ling-3.0-flash-MLX-8bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TensorFold/Ling-3.0-flash-MLX-8bit", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use TensorFold/Ling-3.0-flash-MLX-8bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "TensorFold/Ling-3.0-flash-MLX-8bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default TensorFold/Ling-3.0-flash-MLX-8bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use TensorFold/Ling-3.0-flash-MLX-8bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "TensorFold/Ling-3.0-flash-MLX-8bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "TensorFold/Ling-3.0-flash-MLX-8bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Rebrand model card to TensorFold
Browse files
README.md
CHANGED
|
@@ -13,6 +13,11 @@ tags:
|
|
| 13 |
- 8-bit
|
| 14 |
---
|
| 15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
<p align="center">
|
| 18 |
<img src="https://mdn.alipayobjects.com/huamei_qa8qxu/afts/img/A*4QxcQrBlTiAAAAAAQXAAAAgAemJ7AQ/original" width="100" alt="Ling logo">
|
|
@@ -92,7 +97,7 @@ Run a chat prompt directly from the Hub:
|
|
| 92 |
|
| 93 |
```bash
|
| 94 |
mlx_lm.generate \
|
| 95 |
-
--model
|
| 96 |
--trust-remote-code \
|
| 97 |
--prompt "Explain why hybrid linear attention is useful." \
|
| 98 |
--max-tokens 512 \
|
|
@@ -107,7 +112,7 @@ Thinking mode is enabled by the upstream chat template by default. It can be dis
|
|
| 107 |
|
| 108 |
```bash
|
| 109 |
mlx_lm.generate \
|
| 110 |
-
--model
|
| 111 |
--trust-remote-code \
|
| 112 |
--chat-template-config '{"enable_thinking": false}' \
|
| 113 |
--prompt "Write a short hello-world program in Swift."
|
|
@@ -118,15 +123,15 @@ To download the repository first:
|
|
| 118 |
|
| 119 |
|
| 120 |
```bash
|
| 121 |
-
hf download
|
| 122 |
-
--local-dir ~/.omlx/models/
|
| 123 |
```
|
| 124 |
|
| 125 |
|
| 126 |
## Using it with oMLX
|
| 127 |
|
| 128 |
|
| 129 |
-
1. Place the model at `~/.omlx/models/
|
| 130 |
2. Refresh the oMLX model registry.
|
| 131 |
3. Open the model settings and enable **Trust Remote Code**.
|
| 132 |
4. Load `Ling-3.0-flash-MLX-8bit` and use the normal chat or OpenAI-compatible endpoint.
|
|
@@ -215,14 +220,14 @@ This is a community conversion, not an official InclusionAI release. Because the
|
|
| 215 |
The upstream model is released under the **MIT License**. This conversion preserves that license and is derived from [`inclusionAI/Ling-3.0-flash`](https://huggingface.co/inclusionAI/Ling-3.0-flash).
|
| 216 |
|
| 217 |
|
| 218 |
-
All model design, training, and benchmark credit belongs to InclusionAI and the original contributors. The MLX conversion and compatibility adapter are provided by [
|
| 219 |
|
| 220 |
|
| 221 |
|
| 222 |
-
<!--
|
| 223 |
## Choose for your Mac
|
| 224 |
|
| 225 |
-
[64GB Macs](https://huggingface.co/collections/
|
| 226 |
|
| 227 |
Published peak memory: **132.36 GB**; estimated starting tier: **256GB**, leaving about **123 GB** nominal headroom. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request.
|
| 228 |
|
|
@@ -233,7 +238,7 @@ The exact tested oMLX application version is not recorded here; a library versio
|
|
| 233 |
### Quick start and demo prompt
|
| 234 |
|
| 235 |
```bash
|
| 236 |
-
hf download
|
| 237 |
```
|
| 238 |
|
| 239 |
Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading.
|
|
@@ -246,5 +251,5 @@ Explain why the sky looks blue in three short sentences.
|
|
| 246 |
|
| 247 |
This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available.
|
| 248 |
|
| 249 |
-
[Follow
|
| 250 |
-
<!--
|
|
|
|
| 13 |
- 8-bit
|
| 14 |
---
|
| 15 |
|
| 16 |
+
<p align="center">
|
| 17 |
+
<a href="https://tensorfold.dev">
|
| 18 |
+
<img src="https://huggingface.co/spaces/TensorFold/README/resolve/main/tensorfold-logo.png" alt="TensorFold" width="160">
|
| 19 |
+
</a>
|
| 20 |
+
</p>
|
| 21 |
|
| 22 |
<p align="center">
|
| 23 |
<img src="https://mdn.alipayobjects.com/huamei_qa8qxu/afts/img/A*4QxcQrBlTiAAAAAAQXAAAAgAemJ7AQ/original" width="100" alt="Ling logo">
|
|
|
|
| 97 |
|
| 98 |
```bash
|
| 99 |
mlx_lm.generate \
|
| 100 |
+
--model TensorFold/Ling-3.0-flash-MLX-8bit \
|
| 101 |
--trust-remote-code \
|
| 102 |
--prompt "Explain why hybrid linear attention is useful." \
|
| 103 |
--max-tokens 512 \
|
|
|
|
| 112 |
|
| 113 |
```bash
|
| 114 |
mlx_lm.generate \
|
| 115 |
+
--model TensorFold/Ling-3.0-flash-MLX-8bit \
|
| 116 |
--trust-remote-code \
|
| 117 |
--chat-template-config '{"enable_thinking": false}' \
|
| 118 |
--prompt "Write a short hello-world program in Swift."
|
|
|
|
| 123 |
|
| 124 |
|
| 125 |
```bash
|
| 126 |
+
hf download TensorFold/Ling-3.0-flash-MLX-8bit \
|
| 127 |
+
--local-dir ~/.omlx/models/TensorFold/Ling-3.0-flash-MLX-8bit
|
| 128 |
```
|
| 129 |
|
| 130 |
|
| 131 |
## Using it with oMLX
|
| 132 |
|
| 133 |
|
| 134 |
+
1. Place the model at `~/.omlx/models/TensorFold/Ling-3.0-flash-MLX-8bit`.
|
| 135 |
2. Refresh the oMLX model registry.
|
| 136 |
3. Open the model settings and enable **Trust Remote Code**.
|
| 137 |
4. Load `Ling-3.0-flash-MLX-8bit` and use the normal chat or OpenAI-compatible endpoint.
|
|
|
|
| 220 |
The upstream model is released under the **MIT License**. This conversion preserves that license and is derived from [`inclusionAI/Ling-3.0-flash`](https://huggingface.co/inclusionAI/Ling-3.0-flash).
|
| 221 |
|
| 222 |
|
| 223 |
+
All model design, training, and benchmark credit belongs to InclusionAI and the original contributors. The MLX conversion and compatibility adapter are provided by [TensorFold](https://huggingface.co/TensorFold).
|
| 224 |
|
| 225 |
|
| 226 |
|
| 227 |
+
<!-- TensorFold-chooser-start -->
|
| 228 |
## Choose for your Mac
|
| 229 |
|
| 230 |
+
[64GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-64gb-macs-6a9fefda17932216ec9ab457) 路 [128GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-128gb-macs-6a9ff0abd31bc9abbe7922d7) 路 [256GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-256gb-macs-6a9ff0ef9fed7c5bdca15e9b)
|
| 231 |
|
| 232 |
Published peak memory: **132.36 GB**; estimated starting tier: **256GB**, leaving about **123 GB** nominal headroom. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request.
|
| 233 |
|
|
|
|
| 238 |
### Quick start and demo prompt
|
| 239 |
|
| 240 |
```bash
|
| 241 |
+
hf download TensorFold/Ling-3.0-flash-MLX-8bit --local-dir ./models/Ling-3.0-flash-MLX-8bit
|
| 242 |
```
|
| 243 |
|
| 244 |
Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading.
|
|
|
|
| 251 |
|
| 252 |
This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available.
|
| 253 |
|
| 254 |
+
[Follow TensorFold for new Apple Silicon releases and fixes.](https://huggingface.co/TensorFold)
|
| 255 |
+
<!-- TensorFold-chooser-end -->
|