Upload README.md with huggingface_hub

README.md

@@ -11,6 +11,8 @@ tags:
 - edit-prediction
 - next-edit-suggestion
 - code
+- autocomplete
+- fim
 license: apache-2.0
 language:
 - en
@@ -20,13 +22,13 @@ pipeline_tag: text-generation
 
 # bouroo/zeta-2.1-OptiQ-5
 
-A 5.0-bits-per-weight target, mixed-precision MLX conversion of [zed-industries/zeta-2.1](https://huggingface.co/zed-industries/zeta-2.1), produced with [OptiQ](https://mlx-optiq.com/) using calibration-driven KL-divergence sensitivity analysis. Sensitive tensors are kept at 8-bit; robust tensors are quantized to 4-bit. A higher target BPW keeps more tensors at 8-bit (larger, higher fidelity); a lower target pushes more to 4-bit (smaller).
+A 5.0-bits-per-weight target, mixed-precision MLX conversion of [zed-industries/zeta-2.1](https://huggingface.co/zed-industries/zeta-2.1), tuned for **code edit-prediction / autocomplete**. Sensitive tensors stay 8-bit; robust tensors drop to 4-bit.
 
 > "4-bit" / "8-bit" here are per-tensor choices from a mixed-precision allocation, not a global bit-width. Achieved BPW is the size-weighted average.
 
 ## Variants
 
-This model is part of a target-BPW family. Pick the size/quality trade-off you want:
+Pick the size/quality trade-off:
 
 | Variant | Target BPW | Achieved BPW | 8-bit tensors | 4-bit tensors | Size |
 |---|---|---|---|---|---|
@@ -44,34 +46,50 @@ This model is part of a target-BPW family. Pick the size/quality trade-off you w
 | Candidate bits | 4, 8 |
 | Tensors 8-bit (sensitive) | 104 |
 | Tensors 4-bit (robust) | 121 |
-| Total quantized tensors | 225 |
 | Group size | 64 |
 | Reference signal | bf16 |
 | Calibration samples | 8 (OptiQ mix) |
 | Size on disk | 5.75 GB |
 
-Per-tensor bit allocation is recorded in `optiq_metadata.json` and embedded in `config.json` under `quantization`. All three variants share the **same** sensitivity analysis (one calibration sweep); only the allocation target differs.
+Per-tensor allocation is in `optiq_metadata.json` and `config.json` under `quantization`. A `generation_config.json` ships autocomplete-optimized defaults (`temperature` 0.2, `top_p` 0.95, `max_new_tokens` 128, `eos_token_id` 2).
 
 ## About the base model
 
-[Zeta 2.1](https://huggingface.co/zed-industries/zeta-2.1) is a **code edit prediction model** (next-edit suggestion) finetuned from `ByteDance-Seed/Seed-Coder-8B-Base`. Given code context, edit history and an editable region around the cursor, it predicts the rewritten content of the region. It is an 8B-parameter dense Llama-architecture model (32 layers, 4096 hidden, GQA with 8 KV heads), trained in BF16 with a 32k context window.
+[Zeta 2.1](https://huggingface.co/zed-industries/zeta-2.1) is a **code edit-prediction model** (next-edit suggestion) finetuned from `ByteDance-Seed/Seed-Coder-8B-Base` — an 8B dense Llama (32 layers, GQA x8 KV heads, 32k context, BF16). Given context + an editable region, it predicts the rewritten region.
 
-### Prompt format
+## Prompt format (edit-prediction / FIM)
 
-The model uses **SPM (suffix-prefix-middle)** style prompting with numbered multi-region markers for editable regions.
+This is a **completion model** (no chat template). It uses Zeta's SPM-style format with markers. Minimal insertion prompt (empty region):
 
 ```
-<[fim-suffix]> code after editable region <[fim-prefix]>
-<filename>related/file.py related file content <filename>
-edit_history
---- a/some_file.py
-+++ b/some_file.py
--old
-+new
-<filename>path/to/target_file.py code before editable region <|marker_1|> code needs to<|user_cursor|> rewritten <|marker_2|> <[fim-middle]>
+<[fim-suffix]>{code after cursor}
+<[fim-prefix]><filename>{file_path}
+{code before cursor}<|marker_1|><|marker_2|>
+<[fim-middle]>
 ```
 
-Expected output (no backticks): `<|marker_1|> revised content of the editable region <|marker_2|>`.
+For an **edit** (rewrite an existing region), wrap the current region content with the markers and put `<|user_cursor|>` where the cursor lands:
+
+```
+<[fim-suffix]>{code after region}
+<[fim-prefix]><filename>{path}
+{related files / edit_history, each prefixed with <filename>}
+<filename>{path}
+{before}<|marker_1|>{current part A}<|user_cursor|>{current part B}<|marker_2|>{after}
+<[fim-middle]>
+```
+
+The model generates `<|marker_1|>{predicted part A}<|user_cursor|>{part B}<|marker_2|>` and **stops at EOS** (`<[end_of_sentence]>`, id 2). Stop on `<|marker_2|>` / EOS.
+
+Minimal `mlx_lm` example:
+
+```python
+from mlx_lm import load, generate
+model, tokenizer = load("bouroo/zeta-2.1-OptiQ-5")
+prompt = "<[fim-suffix]>{suffix}\n<[fim-prefix]><filename>app.py\n{prefix}<|marker_1|><|marker_2|>\n<[fim-middle]>"
+out = generate(model, tokenizer, prompt=prompt, max_tokens=128, temp=0.2, top_p=0.95)
+# out starts with <|marker_1|>, stop at <|marker_2|>
+```
 
 ## Usage
 
@@ -80,7 +98,7 @@ Expected output (no backticks): `<|marker_1|> revised content of the editable re
 ```python
 from mlx_lm import load, generate
 model, tokenizer = load("bouroo/zeta-2.1-OptiQ-5")
-print(generate(model, tokenizer, prompt="def fibonacci(n):\n    ", max_tokens=128))
+out = generate(model, tokenizer, prompt=fim_prompt, max_tokens=128, temp=0.2, top_p=0.95)
 ```
 
 ### OptiQ serve
@@ -92,14 +110,24 @@ optiq serve --model bouroo/zeta-2.1-OptiQ-5
 
 ### LM Studio
 
-LM Studio supports MLX models on Apple Silicon:
-
 ```bash
 lms get https://huggingface.co/bouroo/zeta-2.1-OptiQ-5
-lms load bouroo/zeta-2.1-OptiQ-5
+lms load zeta-2.1-optiq-5   # LM Studio normalizes the key (no namespace); run `lms ls` to confirm
 ```
 
-Or search `bouroo/zeta-2.1-OptiQ-5` in the LM Studio in-app Hugging Face browser.
+### Edit-prediction / autocomplete client
+
+This repo ships a working client under [`examples/`](./tree/main/examples):
+
+- `examples/zeta_fim.py` — builds the exact FIM prompt, calls a local LM Studio server (`/v1/completions`), parses the markers, prints the predicted edit.
+- `examples/continue-config.json` — Continue.dev config pointing at the LM Studio server with a custom FIM template.
+
+> **LM Studio's built-in autocomplete is simple FIM and cannot format Zeta's multi-marker edit-prediction prompt.** For real autocomplete/edit-prediction, use the `examples/` client or the Zed editor against the server; the built-in autocomplete UI is not suitable.
+
+
+## Verification
+
+Confirmed with `mlx_lm` (load + edit-prediction generation) and LM Studio (`lms get` + `lms load`).
 
 ---