onw AMD 0.1.1: FastFlowLM v1.0.7, NPU check reasons, diag.py
Browse files- CHANGELOG.md +8 -0
- CHANGELOG_en.md +8 -0
- README.md +12 -1
- README_en.md +1 -1
- diag.py +183 -0
- onw/__init__.py +1 -1
- onw/i18n.json +26 -0
- onw/lemonade.py +31 -17
- onw/manager.py +43 -0
- onw/web/manage.html +9 -0
- pyproject.toml +1 -1
CHANGELOG.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
| 1 |
# 更新履歴
|
| 2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
## v0.1.0
|
| 4 |
|
| 5 |
onw(Intel NPU 版 v0.11.3)をもとにした AMD Ryzen AI NPU 版の最初の版。
|
|
|
|
| 1 |
# 更新履歴
|
| 2 |
|
| 3 |
+
## v0.1.1
|
| 4 |
+
|
| 5 |
+
- FastFlowLM を v1.0.7 に更新(v1.0.6 は Linux でカーネル 6.17 未満を一律に不合格にしていた。次のダウンロード時に自動で入れ替わります)。
|
| 6 |
+
- FastFlowLM の NPU チェック(`flm validate`)で止まったとき、理由を「モデル」タブに日本語で表示(ドライバーの版、入れ方への
|
| 7 |
+
リンク、「もう一度チェック」、`flm validate` の詳しい結果)。ダウンロードもその理由を出して止める。
|
| 8 |
+
- 診断スクリプト `diag.py` を追加(NPU・ドライバー・ファームウェア・`/dev/accel`・memlock・FastFlowLM の状態をまとめて
|
| 9 |
+
ファイルに保存。`--try-model` で小さいモデルを実際に NPU で動かして確かめる)。
|
| 10 |
+
|
| 11 |
## v0.1.0
|
| 12 |
|
| 13 |
onw(Intel NPU 版 v0.11.3)をもとにした AMD Ryzen AI NPU 版の最初の版。
|
CHANGELOG_en.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
| 1 |
# Changelog
|
| 2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
## v0.1.0
|
| 4 |
|
| 5 |
First release of the AMD Ryzen AI NPU edition, based on onw (Intel NPU edition) v0.11.3.
|
|
|
|
| 1 |
# Changelog
|
| 2 |
|
| 3 |
+
## v0.1.1
|
| 4 |
+
|
| 5 |
+
- FastFlowLM updated to v1.0.7 (v1.0.6 failed every Linux kernel older than 6.17; it is replaced at the next download).
|
| 6 |
+
- When FastFlowLM's NPU check (`flm validate`) fails, the Models tab says why (driver version, a link to the driver
|
| 7 |
+
guide, "Check again", flm's own report); downloads stop with that reason.
|
| 8 |
+
- New `diag.py`: NPU, driver, firmware, `/dev/accel`, memlock and FastFlowLM state in one file; `--try-model` runs a
|
| 9 |
+
small model on the NPU to tell a wrong check from a real failure.
|
| 10 |
+
|
| 11 |
## v0.1.0
|
| 12 |
|
| 13 |
First release of the AMD Ryzen AI NPU edition, based on onw (Intel NPU edition) v0.11.3.
|
README.md
CHANGED
|
@@ -24,7 +24,7 @@ UI と仕組み(タスクトレイ常駐、モデル管理ウィンドウ、Op
|
|
| 24 |
仕組みは [Lemonade Server](https://github.com/lemonade-sdk/lemonade) から移植しています。
|
| 25 |
|
| 26 |
- **NPU で動かす機能だけ**です。CPU・GPU(iGPU)・NPU+GPU のハイブリッドでは動かしません。
|
| 27 |
-
- NPU エンジンは FastFlowLM(`flm` v1.0.
|
| 28 |
チャットモデル(Qwen3.5/3.6、Gemma 4、gpt-oss、Llama 3.x、Phi-4-mini、LFM2 など)です。
|
| 29 |
|
| 30 |
## 必要なもの
|
|
@@ -68,6 +68,17 @@ curl -fsSL https://huggingface.co/ryugyosoft/onwAMD/resolve/main/install.sh | ba
|
|
| 68 |
- 思考モード(reasoning のラベルがあるモデル):チャット画面の「思考モード」、API の
|
| 69 |
`chat_template_kwargs: {"enable_thinking": true}` か `reasoning_effort`。思考は `reasoning_content` に分けて返します。
|
| 70 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
### `onw` コマンド
|
| 72 |
|
| 73 |
```bash
|
|
|
|
| 24 |
仕組みは [Lemonade Server](https://github.com/lemonade-sdk/lemonade) から移植しています。
|
| 25 |
|
| 26 |
- **NPU で動かす機能だけ**です。CPU・GPU(iGPU)・NPU+GPU のハイブリッドでは動かしません。
|
| 27 |
+
- NPU エンジンは FastFlowLM(`flm` v1.0.7、約 40 MB)だけで、初回に自動でダウンロードします。モデルは `flm list` に出る
|
| 28 |
チャットモデル(Qwen3.5/3.6、Gemma 4、gpt-oss、Llama 3.x、Phi-4-mini、LFM2 など)です。
|
| 29 |
|
| 30 |
## 必要なもの
|
|
|
|
| 68 |
- 思考モード(reasoning のラベルがあるモデル):チャット画面の「思考モード」、API の
|
| 69 |
`chat_template_kwargs: {"enable_thinking": true}` か `reasoning_effort`。思考は `reasoning_content` に分けて返します。
|
| 70 |
|
| 71 |
+
### うまく動かないとき
|
| 72 |
+
|
| 73 |
+
「モデル」タブに FastFlowLM の NPU チェックの結果(理由)が出ます。それでも分からないときは診断スクリプトを実行し、
|
| 74 |
+
保存される `onw-amd-diag.txt` を送ってください(読み取りのみ。`--try-model` を付けると小さいモデルで実際に試します)。
|
| 75 |
+
|
| 76 |
+
```bash
|
| 77 |
+
curl -fsSL https://huggingface.co/ryugyosoft/onwAMD/resolve/main/diag.py | python3 - --try-model
|
| 78 |
+
```
|
| 79 |
+
|
| 80 |
+
Windows(PowerShell)は `irm https://huggingface.co/ryugyosoft/onwAMD/resolve/main/diag.py | python - --try-model`。
|
| 81 |
+
|
| 82 |
### `onw` コマンド
|
| 83 |
|
| 84 |
```bash
|
README_en.md
CHANGED
|
@@ -9,7 +9,7 @@ chat UI, check page, self-update) and replaces the inference with the NPU-only e
|
|
| 9 |
[Lemonade Server](https://github.com/lemonade-sdk/lemonade).
|
| 10 |
|
| 11 |
- **NPU only.** Nothing runs on the CPU, the GPU (iGPU) or as an NPU+GPU hybrid.
|
| 12 |
-
- The one engine is FastFlowLM (`flm` v1.0.
|
| 13 |
`flm list` offers (Qwen3.5 / 3.6, Gemma 4, gpt-oss, Llama 3.x, Phi-4-mini, LFM2, ...).
|
| 14 |
|
| 15 |
## Requirements
|
|
|
|
| 9 |
[Lemonade Server](https://github.com/lemonade-sdk/lemonade).
|
| 10 |
|
| 11 |
- **NPU only.** Nothing runs on the CPU, the GPU (iGPU) or as an NPU+GPU hybrid.
|
| 12 |
+
- The one engine is FastFlowLM (`flm` v1.0.7, about 40 MB), downloaded on first use. The models are the chat models
|
| 13 |
`flm list` offers (Qwen3.5 / 3.6, Gemma 4, gpt-oss, Llama 3.x, Phi-4-mini, LFM2, ...).
|
| 14 |
|
| 15 |
## Requirements
|
diag.py
ADDED
|
@@ -0,0 +1,183 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""onw AMD diagnostics: why FastFlowLM's NPU check fails on this PC. Standard library only; reads, changes nothing
|
| 2 |
+
(except --try-model, which pulls one small model into onw's model folder and runs it once).
|
| 3 |
+
|
| 4 |
+
curl -fsSL https://huggingface.co/ryugyosoft/onwAMD/resolve/main/diag.py | python3 - (Linux)
|
| 5 |
+
curl -fsSL https://huggingface.co/ryugyosoft/onwAMD/resolve/main/diag.py | python3 - --try-model (+ a real run)
|
| 6 |
+
irm https://huggingface.co/ryugyosoft/onwAMD/resolve/main/diag.py | python - (Windows)
|
| 7 |
+
|
| 8 |
+
The report is printed and saved as onw-amd-diag.txt in the home folder: send that file.
|
| 9 |
+
"""
|
| 10 |
+
import glob, json, os, platform, shutil, subprocess, sys, time, urllib.request
|
| 11 |
+
|
| 12 |
+
WIN = os.name == "nt"
|
| 13 |
+
OUT = []
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
def say(*a):
|
| 17 |
+
line = " ".join(str(x) for x in a)
|
| 18 |
+
print(line, flush=True)
|
| 19 |
+
OUT.append(line)
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def section(t):
|
| 23 |
+
say("\n==== " + t)
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
def run(cmd, timeout=60, env=None):
|
| 27 |
+
try:
|
| 28 |
+
r = subprocess.run(cmd, capture_output=True, text=True, encoding="utf8", errors="replace", timeout=timeout,
|
| 29 |
+
env=env)
|
| 30 |
+
return r.returncode, r.stdout, r.stderr
|
| 31 |
+
except Exception as e:
|
| 32 |
+
return None, "", f"{type(e).__name__}: {e}"
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def show(cmd, timeout=60, env=None, limit=3000):
|
| 36 |
+
rc, out, err = run(cmd, timeout, env)
|
| 37 |
+
say(f"$ {' '.join(cmd)} (exit {rc})")
|
| 38 |
+
for s in (out, err):
|
| 39 |
+
s = s.strip()
|
| 40 |
+
if s:
|
| 41 |
+
say(s[-limit:])
|
| 42 |
+
return rc, out, err
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
def config_dir():
|
| 46 |
+
base = os.environ.get("APPDATA") if WIN else os.path.join(os.path.expanduser("~"), ".config")
|
| 47 |
+
return os.path.join(base or os.path.expanduser("~"), "onw-amd")
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
def main():
|
| 51 |
+
try_model = "--try-model" in sys.argv
|
| 52 |
+
section("system")
|
| 53 |
+
say("python", sys.version.split()[0], "|", platform.platform())
|
| 54 |
+
if not WIN:
|
| 55 |
+
try:
|
| 56 |
+
say(next(l for l in open("/etc/os-release") if l.startswith("PRETTY_NAME")).strip())
|
| 57 |
+
except Exception:
|
| 58 |
+
pass
|
| 59 |
+
say("kernel", platform.release())
|
| 60 |
+
try:
|
| 61 |
+
say("cpu", next(l.split(":", 1)[1].strip() for l in open("/proc/cpuinfo") if l.startswith("model name")))
|
| 62 |
+
except Exception:
|
| 63 |
+
pass
|
| 64 |
+
|
| 65 |
+
if WIN:
|
| 66 |
+
section("NPU (Windows)")
|
| 67 |
+
show(["powershell", "-NoProfile", "-Command",
|
| 68 |
+
"Get-CimInstance Win32_PnPEntity | Where-Object { $_.PNPDeviceID -match 'VEN_1022&DEV_(17F0|1502)' } | "
|
| 69 |
+
"Select-Object Name,PNPDeviceID,Status | Format-List; "
|
| 70 |
+
"Get-CimInstance Win32_PnPSignedDriver | Where-Object { $_.DeviceName -match 'NPU Compute' } | "
|
| 71 |
+
"Select-Object DeviceName,DriverVersion,DriverDate | Format-List"])
|
| 72 |
+
else:
|
| 73 |
+
section("NPU (Linux): PCI")
|
| 74 |
+
for dev in sorted(glob.glob("/sys/bus/pci/devices/*")):
|
| 75 |
+
rd = lambda f: (open(os.path.join(dev, f)).read().strip() if os.path.exists(os.path.join(dev, f)) else "?")
|
| 76 |
+
if rd("vendor") == "0x1022" and rd("class").startswith("0x1180"):
|
| 77 |
+
drv = os.path.basename(os.path.realpath(os.path.join(dev, "driver"))) if os.path.exists(
|
| 78 |
+
os.path.join(dev, "driver")) else "(no driver bound)"
|
| 79 |
+
say(os.path.basename(dev), "device", rd("device"), "rev", rd("revision"), "driver", drv)
|
| 80 |
+
section("NPU (Linux): driver, firmware, device node")
|
| 81 |
+
for p in ("/sys/module/amdxdna/version", "/sys/module/amdxdna/srcversion"):
|
| 82 |
+
if os.path.exists(p):
|
| 83 |
+
say(p, open(p).read().strip())
|
| 84 |
+
say("amdxdna module loaded:", os.path.isdir("/sys/module/amdxdna"))
|
| 85 |
+
show(["modinfo", "amdxdna"], limit=1200)
|
| 86 |
+
fw = sorted(glob.glob("/lib/firmware/amdnpu/**/*", recursive=True))
|
| 87 |
+
say("firmware /lib/firmware/amdnpu:", len(fw), "files")
|
| 88 |
+
for f in fw[:40]:
|
| 89 |
+
say(" ", f)
|
| 90 |
+
nodes = glob.glob("/dev/accel/*")
|
| 91 |
+
say("device nodes:", nodes or "none")
|
| 92 |
+
for n in nodes:
|
| 93 |
+
st = os.stat(n)
|
| 94 |
+
say(" ", n, oct(st.st_mode & 0o777), "uid", st.st_uid, "gid", st.st_gid,
|
| 95 |
+
"| readable", os.access(n, os.R_OK), "writable", os.access(n, os.W_OK))
|
| 96 |
+
show(["id"])
|
| 97 |
+
import resource
|
| 98 |
+
soft, hard = resource.getrlimit(resource.RLIMIT_MEMLOCK)
|
| 99 |
+
fmt = lambda v: "unlimited" if v == resource.RLIM_INFINITY else f"{v // 1024} KiB"
|
| 100 |
+
say("memlock (ulimit -l): soft", fmt(soft), "hard", fmt(hard))
|
| 101 |
+
show(["sh", "-c", "grep -rh memlock /etc/security/limits.conf /etc/security/limits.d/ 2>/dev/null | grep -v '^#'"])
|
| 102 |
+
show(["sh", "-c", "journalctl -k --no-pager -g 'amdxdna|amdnpu' -n 40 2>/dev/null || dmesg 2>&1 | grep -i -E 'amdxdna|amdnpu' | tail -40"])
|
| 103 |
+
|
| 104 |
+
section("FastFlowLM")
|
| 105 |
+
cfg = config_dir()
|
| 106 |
+
exe_name = "flm.exe" if WIN else "flm"
|
| 107 |
+
found = [p for p in glob.glob(os.path.join(cfg, "bin", "flm", "npu", "**", exe_name), recursive=True)]
|
| 108 |
+
say("onw config folder:", cfg, "exists" if os.path.isdir(cfg) else "MISSING")
|
| 109 |
+
say("flm installed by onw:", found or "none")
|
| 110 |
+
say("flm on PATH:", shutil.which("flm") or "none")
|
| 111 |
+
try:
|
| 112 |
+
say("onw FastFlowLM version.txt:", open(os.path.join(cfg, "bin", "flm", "npu", "version.txt")).read().strip())
|
| 113 |
+
except OSError:
|
| 114 |
+
pass
|
| 115 |
+
flm = found[0] if found else shutil.which("flm")
|
| 116 |
+
try:
|
| 117 |
+
conf = json.load(open(os.path.join(cfg, "config.json"), encoding="utf8"))
|
| 118 |
+
except Exception:
|
| 119 |
+
conf = {}
|
| 120 |
+
models = conf.get("models_dir") or os.path.join(cfg, "flm")
|
| 121 |
+
env = {**os.environ, "FLM_MODEL_PATH": models}
|
| 122 |
+
say("FLM_MODEL_PATH onw uses:", models, "| in the environment:", os.environ.get("FLM_MODEL_PATH", "(unset)"))
|
| 123 |
+
if flm:
|
| 124 |
+
if not WIN:
|
| 125 |
+
say("executable:", os.access(flm, os.X_OK))
|
| 126 |
+
rc, out, err = run(["ldd", flm])
|
| 127 |
+
missing = [l.strip() for l in out.splitlines() if "not found" in l]
|
| 128 |
+
say("ldd: missing libraries:", missing or "none")
|
| 129 |
+
show([flm, "version"], env=env)
|
| 130 |
+
show([flm, "validate", "--json"], env=env)
|
| 131 |
+
show([flm, "validate"], env=env)
|
| 132 |
+
show([flm, "list", "--filter", "installed", "--quiet", "--json"], env=env, limit=1500)
|
| 133 |
+
|
| 134 |
+
section("onw logs")
|
| 135 |
+
for name in ("download.log", "server.log"):
|
| 136 |
+
p = os.path.join(cfg, name)
|
| 137 |
+
if os.path.exists(p):
|
| 138 |
+
say(f"--- {name} (last lines)")
|
| 139 |
+
say("".join(open(p, encoding="utf8", errors="replace").readlines()[-40:]).rstrip())
|
| 140 |
+
|
| 141 |
+
if try_model and flm:
|
| 142 |
+
section("try a model anyway (qwen3:0.6b, about 0.7 GB): is the NPU check wrong, or does the NPU really fail?")
|
| 143 |
+
show([flm, "pull", "qwen3:0.6b"], timeout=1800, env=env, limit=800)
|
| 144 |
+
log = os.path.join(os.path.expanduser("~"), "onw-amd-diag-serve.log")
|
| 145 |
+
with open(log, "w") as lf:
|
| 146 |
+
p = subprocess.Popen([flm, "serve", "qwen3:0.6b", "--port", "8099", "--host", "127.0.0.1"], env=env,
|
| 147 |
+
stdout=lf, stderr=subprocess.STDOUT)
|
| 148 |
+
try:
|
| 149 |
+
ok = False
|
| 150 |
+
for _ in range(180):
|
| 151 |
+
if p.poll() is not None:
|
| 152 |
+
break
|
| 153 |
+
try:
|
| 154 |
+
urllib.request.urlopen("http://127.0.0.1:8099/api/tags", timeout=2)
|
| 155 |
+
ok = True
|
| 156 |
+
break
|
| 157 |
+
except OSError:
|
| 158 |
+
time.sleep(1)
|
| 159 |
+
say("flm serve came up:", ok, "| exit code" if p.poll() is not None else "", p.poll() if p.poll() is not None else "")
|
| 160 |
+
if ok:
|
| 161 |
+
body = {"model": "qwen3:0.6b", "max_tokens": 40, "think": False,
|
| 162 |
+
"messages": [{"role": "user", "content": "Say hello in one short sentence."}]}
|
| 163 |
+
try:
|
| 164 |
+
j = json.loads(urllib.request.urlopen(urllib.request.Request(
|
| 165 |
+
"http://127.0.0.1:8099/v1/chat/completions", json.dumps(body).encode(),
|
| 166 |
+
{"Content-Type": "application/json"}), timeout=300).read())
|
| 167 |
+
say("answer:", j["choices"][0]["message"].get("content"))
|
| 168 |
+
say("usage:", json.dumps(j.get("usage")))
|
| 169 |
+
except Exception as e:
|
| 170 |
+
say("request failed:", e)
|
| 171 |
+
finally:
|
| 172 |
+
p.kill()
|
| 173 |
+
say("--- flm serve log (last lines)")
|
| 174 |
+
say("".join(open(log, errors="replace").readlines()[-40:]).rstrip())
|
| 175 |
+
|
| 176 |
+
path = os.path.join(os.path.expanduser("~"), "onw-amd-diag.txt")
|
| 177 |
+
with open(path, "w", encoding="utf8") as f:
|
| 178 |
+
f.write("\n".join(OUT) + "\n")
|
| 179 |
+
print(f"\nSaved: {path} (send this file)")
|
| 180 |
+
|
| 181 |
+
|
| 182 |
+
if __name__ == "__main__":
|
| 183 |
+
main()
|
onw/__init__.py
CHANGED
|
@@ -2,4 +2,4 @@
|
|
| 2 |
|
| 3 |
LLMs entirely on the AMD Ryzen AI NPU (XDNA2), through FastFlowLM, the NPU-only engine of Lemonade Server.
|
| 4 |
"""
|
| 5 |
-
__version__ = "0.1.
|
|
|
|
| 2 |
|
| 3 |
LLMs entirely on the AMD Ryzen AI NPU (XDNA2), through FastFlowLM, the NPU-only engine of Lemonade Server.
|
| 4 |
"""
|
| 5 |
+
__version__ = "0.1.1"
|
onw/i18n.json
CHANGED
|
@@ -5,9 +5,18 @@
|
|
| 5 |
"2ターン会話: 1ターン目": "Two-turn chat: turn 1",
|
| 6 |
"2ターン会話: 2ターン目": "Two-turn chat: turn 2",
|
| 7 |
"API キー(任意)": "API key (optional)",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
"LAN からの接続を許可する(API キー推奨)": "Allow connections from the LAN (API key recommended)",
|
|
|
|
| 9 |
"NPU でロード中…": "Loading on the NPU…",
|
|
|
|
|
|
|
| 10 |
"NPU エンジン": "NPU engine",
|
|
|
|
|
|
|
| 11 |
"OpenAI API のURLをコピー": "Copy the OpenAI API URL",
|
| 12 |
"URL をコピーしました": "URL copied",
|
| 13 |
"auto(OS に合わせる)": "auto (follow the OS)",
|
|
@@ -41,6 +50,7 @@
|
|
| 41 |
"そのモデルはありません": "No such model",
|
| 42 |
"その都市の人口は?一文で。": "What is that city's population? One sentence.",
|
| 43 |
"なし": "No",
|
|
|
|
| 44 |
"アンロード": "Unload",
|
| 45 |
"アンロード(サーバー停止)": "Unload (stop the server)",
|
| 46 |
"エラー: {0}": "Error: {0}",
|
|
@@ -65,6 +75,7 @@
|
|
| 65 |
"ツール呼び出し": "tool calling",
|
| 66 |
"テキスト生成": "Text generation",
|
| 67 |
"デバイス": "Device",
|
|
|
|
| 68 |
"パッケージを更新中(数分かかることがあります)": "Updating packages (may take a few minutes)",
|
| 69 |
"フォルダが見つかりません: {0}": "Folder not found: {0}",
|
| 70 |
"フォルダのパスを入力してください": "Enter the folder path",
|
|
@@ -78,6 +89,7 @@
|
|
| 78 |
"メッセージを入力するか、画像を添付・貼り付け・ドロップしてください。": "Type a message, or attach, paste or drop an image.",
|
| 79 |
"メッセージ(Enterで送信、Shift+Enterで改行)": "Message (Enter to send, Shift+Enter for a new line)",
|
| 80 |
"メモリ 約{0} GB": "memory ~{0} GB",
|
|
|
|
| 81 |
"メモリ目安": "Memory (est.)",
|
| 82 |
"モデル": "Models",
|
| 83 |
"モデルの一覧は、FastFlowLM(約 40 MB)を入れると出ます": "The models are listed once FastFlowLM (about 40 MB) is installed",
|
|
@@ -183,6 +195,7 @@
|
|
| 183 |
"言語": "Language",
|
| 184 |
"設定": "Settings",
|
| 185 |
"設定は保存しました。サーバーを新しい設定で今すぐ再起動しますか?(キャンセルしても、次にモデルを読み込むときから反映されます)": "Settings saved. Restart the server with the new settings now? (Cancel keeps them for the next model load.)",
|
|
|
|
| 186 |
"読み込み中…": "Loading…",
|
| 187 |
"起動時と 1 日ごとに更新を確認する": "Check for updates at startup and daily",
|
| 188 |
"起動時に前回ロードしたモデルを自動でロードする": "Load the last model automatically at startup",
|
|
@@ -199,9 +212,18 @@
|
|
| 199 |
"2ターン会話: 1ターン目": "两轮对话:第 1 轮",
|
| 200 |
"2ターン会話: 2ターン目": "两轮对话:第 2 轮",
|
| 201 |
"API キー(任意)": "API 密钥(可选)",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 202 |
"LAN からの接続を許可する(API キー推奨)": "允许局域网连接(建议设置 API 密钥)",
|
|
|
|
| 203 |
"NPU でロード中…": "正在 NPU 上加载…",
|
|
|
|
|
|
|
| 204 |
"NPU エンジン": "NPU 引擎",
|
|
|
|
|
|
|
| 205 |
"OpenAI API のURLをコピー": "复制 OpenAI API 地址",
|
| 206 |
"URL をコピーしました": "已复制 URL",
|
| 207 |
"auto(OS に合わせる)": "auto(跟随系统)",
|
|
@@ -235,6 +257,7 @@
|
|
| 235 |
"そのモデルはありません": "没有该模型",
|
| 236 |
"その都市の人口は?一文で。": "那座城市的人口是多少?用一句话回答。",
|
| 237 |
"なし": "不支持",
|
|
|
|
| 238 |
"アンロード": "卸载",
|
| 239 |
"アンロード(サーバー停止)": "卸载(停止服务器)",
|
| 240 |
"エラー: {0}": "错误:{0}",
|
|
@@ -259,6 +282,7 @@
|
|
| 259 |
"ツール呼び出し": "工具调用",
|
| 260 |
"テキスト生成": "文本生成",
|
| 261 |
"デバイス": "设备",
|
|
|
|
| 262 |
"パッケージを更新中(数分かかることがあります)": "正在更新软件包(可能需要几分钟)",
|
| 263 |
"フォルダが見つかりません: {0}": "找不到文件夹:{0}",
|
| 264 |
"フォルダのパスを入力してください": "请输入文件夹路径",
|
|
@@ -272,6 +296,7 @@
|
|
| 272 |
"メッセージを入力するか、画像を添付・貼り付け・ドロップしてください。": "输入消息,或添加、粘贴、拖放图片。",
|
| 273 |
"メッセージ(Enterで送信、Shift+Enterで改行)": "消息(Enter 发送,Shift+Enter 换行)",
|
| 274 |
"メモリ 約{0} GB": "内存约 {0} GB",
|
|
|
|
| 275 |
"メモリ目安": "内存估计",
|
| 276 |
"モデル": "模型",
|
| 277 |
"モデルの一覧は、FastFlowLM(約 40 MB)を入れると出ます": "安装 FastFlowLM(约 40 MB)后会列出模型",
|
|
@@ -377,6 +402,7 @@
|
|
| 377 |
"言語": "语言",
|
| 378 |
"設定": "设置",
|
| 379 |
"設定は保存しました。サーバーを新しい設定で今すぐ再起動しますか?(キャンセルしても、次にモデルを読み込むときから反映されます)": "设置已保存。现在用新设置重启服务器?(取消后,下次加载模型时生效)",
|
|
|
|
| 380 |
"読み込み中…": "加载中…",
|
| 381 |
"起動時と 1 日ごとに更新を確認する": "启动时及每天检查更新",
|
| 382 |
"起動時に前回ロードしたモデルを自動でロードする": "启动时自动加载上次的模型",
|
|
|
|
| 5 |
"2ターン会話: 1ターン目": "Two-turn chat: turn 1",
|
| 6 |
"2ターン会話: 2ターン目": "Two-turn chat: turn 2",
|
| 7 |
"API キー(任意)": "API key (optional)",
|
| 8 |
+
"FastFlowLM から AMD の NPU が見えません。デバイスマネージャーに「NPU Compute Accelerator Device」があるか、NPU ドライバーが入っているか確認してください。": "FastFlowLM cannot see the AMD NPU. Check that \"NPU Compute Accelerator Device\" is in the Device Manager and that the NPU driver is installed.",
|
| 9 |
+
"FastFlowLM の NPU チェックで止まりました": "FastFlowLM's NPU check stopped here",
|
| 10 |
+
"FastFlowLM の NPU チェックで止まりました: {0}": "FastFlowLM's NPU check stopped here: {0}",
|
| 11 |
+
"FastFlowLM の NPU チェックを実行できませんでした。": "FastFlowLM's NPU check could not run.",
|
| 12 |
"LAN からの接続を許可する(API キー推奨)": "Allow connections from the LAN (API key recommended)",
|
| 13 |
+
"Linux カーネルが FastFlowLM に対応していません。": "The Linux kernel is not supported by FastFlowLM.",
|
| 14 |
"NPU でロード中…": "Loading on the NPU…",
|
| 15 |
+
"NPU のチェックに失敗しました(ドライバー {1})。NPU ドライバーを更新してから、もう一度試してください。": "The NPU check failed (driver {1}). Update the NPU driver and try again.",
|
| 16 |
+
"NPU のファームウェアが FastFlowLM に対応していません。NPU ドライバーを更新すると直ることがあります。": "The NPU firmware is not supported by FastFlowLM. Updating the NPU driver may fix it.",
|
| 17 |
"NPU エンジン": "NPU engine",
|
| 18 |
+
"NPU ドライバーが古いです(この PC は {1}、FastFlowLM は {0} 以降が必要)。Windows Update か PC メーカー/AMD のサイトで NPU ドライバーを更新してください。": "The NPU driver is too old (this PC: {1}, FastFlowLM needs {0} or newer). Update the NPU driver with Windows Update or from the PC maker's / AMD's site.",
|
| 19 |
+
"NPU ドライバー(amdxdna)が古いです(この PC は {1})。linux-firmware と amdxdna ドライバーを新しくしてください(ディストリビューションの更新、または AMD の案内)。": "The NPU driver (amdxdna) is too old (this PC: {1}). Update linux-firmware and the amdxdna driver (your distribution's updates, or AMD's guide).",
|
| 20 |
"OpenAI API のURLをコピー": "Copy the OpenAI API URL",
|
| 21 |
"URL をコピーしました": "URL copied",
|
| 22 |
"auto(OS に合わせる)": "auto (follow the OS)",
|
|
|
|
| 50 |
"そのモデルはありません": "No such model",
|
| 51 |
"その都市の人口は?一文で。": "What is that city's population? One sentence.",
|
| 52 |
"なし": "No",
|
| 53 |
+
"もう一度チェック": "Check again",
|
| 54 |
"アンロード": "Unload",
|
| 55 |
"アンロード(サーバー停止)": "Unload (stop the server)",
|
| 56 |
"エラー: {0}": "Error: {0}",
|
|
|
|
| 75 |
"ツール呼び出し": "tool calling",
|
| 76 |
"テキスト生成": "Text generation",
|
| 77 |
"デバイス": "Device",
|
| 78 |
+
"ドライバーの入れ方": "How to install the driver",
|
| 79 |
"パッケージを更新中(数分かかることがあります)": "Updating packages (may take a few minutes)",
|
| 80 |
"フォルダが見つかりません: {0}": "Folder not found: {0}",
|
| 81 |
"フォルダのパスを入力してください": "Enter the folder path",
|
|
|
|
| 89 |
"メッセージを入力するか、画像を添付・貼り付け・ドロップしてください。": "Type a message, or attach, paste or drop an image.",
|
| 90 |
"メッセージ(Enterで送信、Shift+Enterで改行)": "Message (Enter to send, Shift+Enter for a new line)",
|
| 91 |
"メモリ 約{0} GB": "memory ~{0} GB",
|
| 92 |
+
"メモリのロック上限(memlock)が足りません(Linux)。": "The memory lock limit (memlock) is too low (Linux).",
|
| 93 |
"メモリ目安": "Memory (est.)",
|
| 94 |
"モデル": "Models",
|
| 95 |
"モデルの一覧は、FastFlowLM(約 40 MB)を入れると出ます": "The models are listed once FastFlowLM (about 40 MB) is installed",
|
|
|
|
| 195 |
"言語": "Language",
|
| 196 |
"設定": "Settings",
|
| 197 |
"設定は保存しました。サーバーを新しい設定で今すぐ再起動しますか?(キャンセルしても、次にモデルを読み込むときから反映されます)": "Settings saved. Restart the server with the new settings now? (Cancel keeps them for the next model load.)",
|
| 198 |
+
"詳しい結果(flm validate)": "Details (flm validate)",
|
| 199 |
"読み込み中…": "Loading…",
|
| 200 |
"起動時と 1 日ごとに更新を確認する": "Check for updates at startup and daily",
|
| 201 |
"起動時に前回ロードしたモデルを自動でロードする": "Load the last model automatically at startup",
|
|
|
|
| 212 |
"2ターン会話: 1ターン目": "两轮对话:第 1 轮",
|
| 213 |
"2ターン会話: 2ターン目": "两轮对话:第 2 轮",
|
| 214 |
"API キー(任意)": "API 密钥(可选)",
|
| 215 |
+
"FastFlowLM から AMD の NPU が見えません。デバイスマネージャーに「NPU Compute Accelerator Device」があるか、NPU ドライバーが入っているか確認し��ください。": "FastFlowLM 找不到 AMD NPU。请确认设备管理器中有“NPU Compute Accelerator Device”,并已安装 NPU 驱动。",
|
| 216 |
+
"FastFlowLM の NPU チェックで止まりました": "FastFlowLM 的 NPU 检查未通过",
|
| 217 |
+
"FastFlowLM の NPU チェックで止まりました: {0}": "FastFlowLM 的 NPU 检查未通过:{0}",
|
| 218 |
+
"FastFlowLM の NPU チェックを実行できませんでした。": "无法运行 FastFlowLM 的 NPU 检查。",
|
| 219 |
"LAN からの接続を許可する(API キー推奨)": "允许局域网连接(建议设置 API 密钥)",
|
| 220 |
+
"Linux カーネルが FastFlowLM に対応していません。": "Linux 内核不受 FastFlowLM 支持。",
|
| 221 |
"NPU でロード中…": "正在 NPU 上加载…",
|
| 222 |
+
"NPU のチェックに失敗しました(ドライバー {1})。NPU ドライバーを更新してから、もう一度試してください。": "NPU 检查失败(驱动 {1})。请更新 NPU 驱动后再试。",
|
| 223 |
+
"NPU のファームウェアが FastFlowLM に対応していません。NPU ドライバーを更新すると直ることがあります。": "NPU 固件不受 FastFlowLM 支持。更新 NPU 驱动可能会解决。",
|
| 224 |
"NPU エンジン": "NPU 引擎",
|
| 225 |
+
"NPU ドライバーが古いです(この PC は {1}、FastFlowLM は {0} 以降が必要)。Windows Update か PC メーカー/AMD のサイトで NPU ドライバーを更新してください。": "NPU 驱动过旧(本机为 {1},FastFlowLM 需要 {0} 或更新)。请通过 Windows 更新或电脑厂商/AMD 网站更新 NPU 驱动。",
|
| 226 |
+
"NPU ドライバー(amdxdna)が古いです(この PC は {1})。linux-firmware と amdxdna ドライバーを新しくしてください(ディストリビューションの更新、または AMD の案内)。": "NPU 驱动(amdxdna)过旧(本机为 {1})。请更新 linux-firmware 和 amdxdna 驱动(发行版更新或 AMD 的说明)。",
|
| 227 |
"OpenAI API のURLをコピー": "复制 OpenAI API 地址",
|
| 228 |
"URL をコピーしました": "已复制 URL",
|
| 229 |
"auto(OS に合わせる)": "auto(跟随系统)",
|
|
|
|
| 257 |
"そのモデルはありません": "没有该模型",
|
| 258 |
"その都市の人口は?一文で。": "那座城市的人口是多少?用一句话回答。",
|
| 259 |
"なし": "不支持",
|
| 260 |
+
"もう一度チェック": "重新检查",
|
| 261 |
"アンロード": "卸载",
|
| 262 |
"アンロード(サーバー停止)": "卸载(停止服务器)",
|
| 263 |
"エラー: {0}": "错误:{0}",
|
|
|
|
| 282 |
"ツール呼び出し": "工具调用",
|
| 283 |
"テキスト生成": "文本生成",
|
| 284 |
"デバイス": "设备",
|
| 285 |
+
"ドライバーの入れ方": "如何安装驱动",
|
| 286 |
"パッケージを更新中(数分かかることがあります)": "正在更新软件包(可能需要几分钟)",
|
| 287 |
"フォルダが見つかりません: {0}": "找不到文件夹:{0}",
|
| 288 |
"フォルダのパスを入力してください": "请输入文件夹路径",
|
|
|
|
| 296 |
"メッセージを入力するか、画像を添付・貼り付け・ドロップしてください。": "输入消息,或添加、粘贴、拖放图片。",
|
| 297 |
"メッセージ(Enterで送信、Shift+Enterで改行)": "消息(Enter 发送,Shift+Enter 换行)",
|
| 298 |
"メモリ 約{0} GB": "内存约 {0} GB",
|
| 299 |
+
"メモリのロック上限(memlock)が足りません(Linux)。": "内存锁定上限(memlock)不足(Linux)。",
|
| 300 |
"メモリ目安": "内存估计",
|
| 301 |
"モデル": "模型",
|
| 302 |
"モデルの一覧は、FastFlowLM(約 40 MB)を入れると出ます": "安装 FastFlowLM(约 40 MB)后会列出模型",
|
|
|
|
| 402 |
"言語": "语言",
|
| 403 |
"設定": "设置",
|
| 404 |
"設定は保存しました。サーバーを新しい設定で今すぐ再起動しますか?(キャンセルしても、次にモデルを読み込むときから反映されます)": "设置已保存。现在用新设置重启服务器?(取消后,下次加载模型时生效)",
|
| 405 |
+
"詳しい結果(flm validate)": "详细结果(flm validate)",
|
| 406 |
"読み込み中…": "加载中…",
|
| 407 |
"起動時と 1 日ごとに更新を確認する": "启动时及每天检查更新",
|
| 408 |
"起動時に前回ロードしたモデルを自動でロードする": "启动时自动加载上次的模型",
|
onw/lemonade.py
CHANGED
|
@@ -6,7 +6,7 @@ OpenAI-compatible API on localhost. onw does the same with the one engine that i
|
|
| 6 |
Linux. (lemonade: src/cpp/server/backends/fastflowlm/fastflowlm_server.cpp, fastflowlm_models.cpp)
|
| 7 |
|
| 8 |
FastFlowLM is downloaded on first use from its GitHub release into <config>/bin/flm/npu, pinned to the version in
|
| 9 |
-
Lemonade's src/cpp/resources/backend_versions.json.
|
| 10 |
|
| 11 |
npu() the NPU: {"arch": "XDNA2" | None, "name", "driver"} (lemonade: system_info.cpp)
|
| 12 |
installed_version() FastFlowLM's installed version, or None; install(progress) downloads it
|
|
@@ -15,9 +15,9 @@ Lemonade's src/cpp/resources/backend_versions.json.
|
|
| 15 |
flm_models(), flm_pull_command(), flm_remove()
|
| 16 |
serve_command(...) the command line that serves one model on a port, and the URL that answers once it is ready
|
| 17 |
"""
|
| 18 |
-
import json, os, shutil, subprocess, sys, tarfile, time, urllib.request, zipfile
|
| 19 |
|
| 20 |
-
VERSION = "v1.0.
|
| 21 |
ENGINE = "FastFlowLM"
|
| 22 |
WINDOWS = os.name == "nt"
|
| 23 |
NO_WINDOW = {"creationflags": subprocess.CREATE_NO_WINDOW} if WINDOWS else {}
|
|
@@ -188,24 +188,38 @@ def _json_in(text):
|
|
| 188 |
return json.loads(text[s:e + 1]) if s >= 0 and e > s else {}
|
| 189 |
|
| 190 |
|
| 191 |
-
|
| 192 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 193 |
try:
|
| 194 |
r = _flm(["validate", "--json"], cfg, timeout=60)
|
| 195 |
j = _json_in(r.stdout)
|
| 196 |
except Exception as e:
|
| 197 |
-
return False, str(e)
|
| 198 |
-
if j.get("ready"):
|
| 199 |
-
return True, ""
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
|
| 206 |
-
|
| 207 |
-
|
| 208 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 209 |
|
| 210 |
|
| 211 |
def flm_name(checkpoint):
|
|
|
|
| 6 |
Linux. (lemonade: src/cpp/server/backends/fastflowlm/fastflowlm_server.cpp, fastflowlm_models.cpp)
|
| 7 |
|
| 8 |
FastFlowLM is downloaded on first use from its GitHub release into <config>/bin/flm/npu, pinned to the version in
|
| 9 |
+
Lemonade's src/cpp/resources/backend_versions.json (one release newer, see VERSION).
|
| 10 |
|
| 11 |
npu() the NPU: {"arch": "XDNA2" | None, "name", "driver"} (lemonade: system_info.cpp)
|
| 12 |
installed_version() FastFlowLM's installed version, or None; install(progress) downloads it
|
|
|
|
| 15 |
flm_models(), flm_pull_command(), flm_remove()
|
| 16 |
serve_command(...) the command line that serves one model on a port, and the URL that answers once it is ready
|
| 17 |
"""
|
| 18 |
+
import json, os, re, shutil, subprocess, sys, tarfile, time, urllib.request, zipfile
|
| 19 |
|
| 20 |
+
VERSION = "v1.0.7" # lemonade pins v1.0.6; v1.0.7 drops the Linux kernel >= 6.17 check in `flm validate`
|
| 21 |
ENGINE = "FastFlowLM"
|
| 22 |
WINDOWS = os.name == "nt"
|
| 23 |
NO_WINDOW = {"creationflags": subprocess.CREATE_NO_WINDOW} if WINDOWS else {}
|
|
|
|
| 188 |
return json.loads(text[s:e + 1]) if s >= 0 and e > s else {}
|
| 189 |
|
| 190 |
|
| 191 |
+
MIN_DRIVER = "32.0.203.311" # Windows NPU driver FastFlowLM asks for (its README)
|
| 192 |
+
CHECKS = (("amd_device_found", "no_device", "No AMD NPU device found."),
|
| 193 |
+
("npu_driver_ok", "driver_old", f"NPU driver version is too old (FastFlowLM needs {MIN_DRIVER} or newer)."),
|
| 194 |
+
("all_fw_ok", "firmware", "NPU firmware is incompatible."),
|
| 195 |
+
("kernel_ok", "kernel", "Kernel version is incompatible."),
|
| 196 |
+
("memlock_ok", "memlock", "Memlock limits are too low."))
|
| 197 |
+
|
| 198 |
+
|
| 199 |
+
def flm_check(cfg=None):
|
| 200 |
+
"""`flm validate --json` (lemonade: run_flm_validate): {"ok", "codes": [no_device | driver_old | firmware | kernel
|
| 201 |
+
| memlock | failed | error], "message": English text, "detail": flm's own output}."""
|
| 202 |
try:
|
| 203 |
r = _flm(["validate", "--json"], cfg, timeout=60)
|
| 204 |
j = _json_in(r.stdout)
|
| 205 |
except Exception as e:
|
| 206 |
+
return {"ok": False, "codes": ["error"], "message": str(e), "detail": ""}
|
| 207 |
+
if j.get("ready") or (not j and r.returncode == 0):
|
| 208 |
+
return {"ok": True, "codes": [], "message": "", "detail": ""}
|
| 209 |
+
bad = [(code, msg) for key, code, msg in CHECKS if key in j and not j[key]] or [("failed", "NPU validation failed.")]
|
| 210 |
+
try: # the plain report names the driver version flm saw
|
| 211 |
+
detail = _flm(["validate"], cfg, timeout=60).stdout
|
| 212 |
+
detail = "\n".join(l for l in (re.sub(r"\x1b\[[0-9;]*m", "", x).strip() for x in detail.splitlines()) if l)
|
| 213 |
+
except Exception:
|
| 214 |
+
detail = json.dumps(j)
|
| 215 |
+
return {"ok": False, "codes": [c for c, _ in bad], "message": " ".join(m for _, m in bad) + f" ({DRIVER_URL})",
|
| 216 |
+
"detail": detail[-600:]}
|
| 217 |
+
|
| 218 |
+
|
| 219 |
+
def flm_validate(cfg=None):
|
| 220 |
+
"""(ok, message) from `flm validate --json`."""
|
| 221 |
+
c = flm_check(cfg)
|
| 222 |
+
return c["ok"], c["message"]
|
| 223 |
|
| 224 |
|
| 225 |
def flm_name(checkpoint):
|
onw/manager.py
CHANGED
|
@@ -20,6 +20,16 @@ from .i18n import T
|
|
| 20 |
WEB = os.path.join(os.path.dirname(os.path.abspath(__file__)), "web")
|
| 21 |
INFO = lambda: os.path.join(app.config_dir(), "manager.json")
|
| 22 |
LAST_SEEN = [time.time()] # when the page last asked for the state (a browser-only window keeps us alive)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
NOTES = {"vision": "画像も読めます", "reasoning": "考えてから答えます", "coding": "コード向け", "tool-calling": "ツール呼び出し"}
|
| 24 |
|
| 25 |
|
|
@@ -39,6 +49,7 @@ class Manager:
|
|
| 39 |
self.lock = threading.Lock()
|
| 40 |
self.dl = None # {"key", "proc", "status", "frac", "text", "error", "done", "log"}
|
| 41 |
self.npu = None
|
|
|
|
| 42 |
self.ram = app.total_ram_gb()
|
| 43 |
self.busy = "" # a message about a long action (loading / deleting) for the page
|
| 44 |
self.busy_err = False
|
|
@@ -52,6 +63,26 @@ class Manager:
|
|
| 52 |
def _npu_info(self):
|
| 53 |
lemonade.npu()
|
| 54 |
self.npu = lemonade.npu_text()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
# ---------------------------------------------------------------- state
|
| 57 |
def _engine(self):
|
|
@@ -90,6 +121,8 @@ class Manager:
|
|
| 90 |
"engine_dl": {"frac": eng_dl["frac"], "text": eng_dl["text"], "error": eng_dl.get("error"),
|
| 91 |
"done": bool(eng_dl.get("done"))} if eng_dl else None,
|
| 92 |
"flm_error": app._FLM_CACHE.get("error", ""),
|
|
|
|
|
|
|
| 93 |
"server": {"status": status, "model": info and info.get("model"), "path": info and info.get("path"),
|
| 94 |
"error": app.failure_reason() if status == "failed" else "",
|
| 95 |
"title": app.model_title(info["model"]) if info else None, "port": info and info.get("port"),
|
|
@@ -130,6 +163,10 @@ class Manager:
|
|
| 130 |
return {"error": T("そのモデルはありません")}
|
| 131 |
if lemonade.npu().get("arch") != "XDNA2":
|
| 132 |
return {"error": T("この PC には XDNA2 の NPU が見つかりません(Ryzen AI 300 シリーズ以降と NPU ドライバーが必要です)")}
|
|
|
|
|
|
|
|
|
|
|
|
|
| 133 |
info = app.server_info()
|
| 134 |
if info and info.get("model") == key:
|
| 135 |
return {"error": T("このモデルはロード中です。先にアンロードしてください。")}
|
|
@@ -192,6 +229,8 @@ class Manager:
|
|
| 192 |
else:
|
| 193 |
dl["frac"] = 1.0
|
| 194 |
app.flm_catalog(refresh=True) # flm pulled a model / FastFlowLM was installed
|
|
|
|
|
|
|
| 195 |
if not dl["key"].startswith("engine:"):
|
| 196 |
with app._CFG_LOCK:
|
| 197 |
if not app.load_config().get("model"):
|
|
@@ -363,6 +402,10 @@ class Manager:
|
|
| 363 |
webbrowser.open(f"http://localhost:{cfg['port']}/{what}")
|
| 364 |
elif what == "flm":
|
| 365 |
webbrowser.open("https://github.com/ROCm/FastFlowLM")
|
|
|
|
|
|
|
|
|
|
|
|
|
| 366 |
return {}
|
| 367 |
|
| 368 |
|
|
|
|
| 20 |
WEB = os.path.join(os.path.dirname(os.path.abspath(__file__)), "web")
|
| 21 |
INFO = lambda: os.path.join(app.config_dir(), "manager.json")
|
| 22 |
LAST_SEEN = [time.time()] # when the page last asked for the state (a browser-only window keeps us alive)
|
| 23 |
+
CHECK_TEXT = { # flm validate's findings ({0}: driver FastFlowLM needs, {1}: driver installed)
|
| 24 |
+
"driver_old": "NPU ドライバーが古いです(この PC は {1}、FastFlowLM は {0} 以降が必要)。Windows Update か PC メーカー/AMD のサイトで NPU ドライバーを更新してください。",
|
| 25 |
+
"no_device": "FastFlowLM から AMD の NPU が見えません。デバイスマネージャーに「NPU Compute Accelerator Device」があるか、NPU ドライバーが入っているか確認してください。",
|
| 26 |
+
"firmware": "NPU のファームウェアが FastFlowLM に対応していません。NPU ドライバーを更新すると直ることがあります。",
|
| 27 |
+
"driver_old_linux": "NPU ドライバー(amdxdna)が古いです(この PC は {1})。linux-firmware と amdxdna ドライバーを新しくしてください(ディストリビューションの更新、または AMD の案内)。",
|
| 28 |
+
"kernel": "Linux カーネルが FastFlowLM に対応していません。",
|
| 29 |
+
"memlock": "メモリのロック上限(memlock)が足りません(Linux)。",
|
| 30 |
+
"failed": "NPU のチェックに失敗しました(ドライバー {1})。NPU ドライバーを更新してから、もう一度試してください。",
|
| 31 |
+
"error": "FastFlowLM の NPU チェックを実行できませんでした。",
|
| 32 |
+
}
|
| 33 |
NOTES = {"vision": "画像も読めます", "reasoning": "考えてから答えます", "coding": "コード向け", "tool-calling": "ツール呼び出し"}
|
| 34 |
|
| 35 |
|
|
|
|
| 49 |
self.lock = threading.Lock()
|
| 50 |
self.dl = None # {"key", "proc", "status", "frac", "text", "error", "done", "log"}
|
| 51 |
self.npu = None
|
| 52 |
+
self.check = None # lemonade.flm_check() + "text" (the reasons in the UI language)
|
| 53 |
self.ram = app.total_ram_gb()
|
| 54 |
self.busy = "" # a message about a long action (loading / deleting) for the page
|
| 55 |
self.busy_err = False
|
|
|
|
| 63 |
def _npu_info(self):
|
| 64 |
lemonade.npu()
|
| 65 |
self.npu = lemonade.npu_text()
|
| 66 |
+
self._check_npu()
|
| 67 |
+
|
| 68 |
+
def _check_npu(self):
|
| 69 |
+
"""FastFlowLM's own NPU check (`flm validate`), once FastFlowLM is there: why models cannot run, in words."""
|
| 70 |
+
if not lemonade.binary():
|
| 71 |
+
self.check = None
|
| 72 |
+
return
|
| 73 |
+
c = lemonade.flm_check()
|
| 74 |
+
n = lemonade.npu()
|
| 75 |
+
linux = lambda code: code + "_linux" if not lemonade.WINDOWS and code + "_linux" in CHECK_TEXT else code
|
| 76 |
+
c["text"] = [T(CHECK_TEXT.get(linux(code), CHECK_TEXT["failed"]), lemonade.MIN_DRIVER, n.get("driver") or "?")
|
| 77 |
+
for code in c["codes"]]
|
| 78 |
+
self.check = c
|
| 79 |
+
|
| 80 |
+
def _check_error(self):
|
| 81 |
+
"""The NPU check's failure as one message for the page, or None."""
|
| 82 |
+
c = self.check
|
| 83 |
+
if c and not c["ok"]:
|
| 84 |
+
return T("FastFlowLM の NPU チェックで止まりました: {0}", " ".join(c["text"]))
|
| 85 |
+
return None
|
| 86 |
|
| 87 |
# ---------------------------------------------------------------- state
|
| 88 |
def _engine(self):
|
|
|
|
| 121 |
"engine_dl": {"frac": eng_dl["frac"], "text": eng_dl["text"], "error": eng_dl.get("error"),
|
| 122 |
"done": bool(eng_dl.get("done"))} if eng_dl else None,
|
| 123 |
"flm_error": app._FLM_CACHE.get("error", ""),
|
| 124 |
+
"npu_check": {"ok": self.check["ok"], "text": self.check["text"], "detail": self.check["detail"]}
|
| 125 |
+
if self.check else None,
|
| 126 |
"server": {"status": status, "model": info and info.get("model"), "path": info and info.get("path"),
|
| 127 |
"error": app.failure_reason() if status == "failed" else "",
|
| 128 |
"title": app.model_title(info["model"]) if info else None, "port": info and info.get("port"),
|
|
|
|
| 163 |
return {"error": T("そのモデルはありません")}
|
| 164 |
if lemonade.npu().get("arch") != "XDNA2":
|
| 165 |
return {"error": T("この PC には XDNA2 の NPU が見つかりません(Ryzen AI 300 シリーズ以降と NPU ドライバーが必要です)")}
|
| 166 |
+
if self.check is None and lemonade.binary():
|
| 167 |
+
self._check_npu()
|
| 168 |
+
if self._check_error():
|
| 169 |
+
return {"error": self._check_error()}
|
| 170 |
info = app.server_info()
|
| 171 |
if info and info.get("model") == key:
|
| 172 |
return {"error": T("このモデルはロード中です。先にアンロードしてください。")}
|
|
|
|
| 229 |
else:
|
| 230 |
dl["frac"] = 1.0
|
| 231 |
app.flm_catalog(refresh=True) # flm pulled a model / FastFlowLM was installed
|
| 232 |
+
if dl["key"] == "engine":
|
| 233 |
+
threading.Thread(target=self._check_npu, daemon=True).start()
|
| 234 |
if not dl["key"].startswith("engine:"):
|
| 235 |
with app._CFG_LOCK:
|
| 236 |
if not app.load_config().get("model"):
|
|
|
|
| 402 |
webbrowser.open(f"http://localhost:{cfg['port']}/{what}")
|
| 403 |
elif what == "flm":
|
| 404 |
webbrowser.open("https://github.com/ROCm/FastFlowLM")
|
| 405 |
+
elif what == "driver":
|
| 406 |
+
webbrowser.open(lemonade.DRIVER_URL)
|
| 407 |
+
elif what == "recheck":
|
| 408 |
+
self._check_npu()
|
| 409 |
return {}
|
| 410 |
|
| 411 |
|
onw/web/manage.html
CHANGED
|
@@ -164,6 +164,7 @@
|
|
| 164 |
|
| 165 |
<section class="page" id="page-models">
|
| 166 |
<div class="banner small" id="npuwarn" style="display:none">この PC には XDNA2 の NPU が見つかりません(Ryzen AI 300 シリーズ以降と NPU ドライバーが必要です)</div>
|
|
|
|
| 167 |
<div class="banner small" id="dirwarn" style="display:none"></div>
|
| 168 |
<div class="card" style="padding:6px 10px">
|
| 169 |
<div class="row small" id="engines"></div>
|
|
@@ -311,6 +312,14 @@
|
|
| 311 |
$("b-load").disabled = same || !pick.value || st === "loading";
|
| 312 |
// models table
|
| 313 |
$("npuwarn").style.display = s.npu && !s.npu_ok ? "" : "none";
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 314 |
$("seng").textContent = "FastFlowLM " + (s.engine.installed || T("未インストール"));
|
| 315 |
const ed = s.engine_dl, e = s.engine;
|
| 316 |
let est;
|
|
|
|
| 164 |
|
| 165 |
<section class="page" id="page-models">
|
| 166 |
<div class="banner small" id="npuwarn" style="display:none">この PC には XDNA2 の NPU が見つかりません(Ryzen AI 300 シリーズ以降と NPU ドライバーが必要です)</div>
|
| 167 |
+
<div class="banner small" id="checkwarn" style="display:none"></div>
|
| 168 |
<div class="banner small" id="dirwarn" style="display:none"></div>
|
| 169 |
<div class="card" style="padding:6px 10px">
|
| 170 |
<div class="row small" id="engines"></div>
|
|
|
|
| 312 |
$("b-load").disabled = same || !pick.value || st === "loading";
|
| 313 |
// models table
|
| 314 |
$("npuwarn").style.display = s.npu && !s.npu_ok ? "" : "none";
|
| 315 |
+
const ck = s.npu_check, ckHtml = ck && !ck.ok
|
| 316 |
+
? `<b>${T("FastFlowLM の NPU チェックで止まりました")}</b><br>${ck.text.map(esc).join("<br>")}` +
|
| 317 |
+
`<br><button class="ghost" onclick="post('open',{what:'driver'})">${T("ドライバーの入れ方")}</button>` +
|
| 318 |
+
`<button class="ghost" onclick="post('open',{what:'recheck'})">${T("もう一度チェック")}</button>` +
|
| 319 |
+
(ck.detail ? `<details><summary>${T("詳しい結果(flm validate)")}</summary><pre class="mono" style="white-space:pre-wrap;margin:4px 0 0">${esc(ck.detail)}</pre></details>` : "")
|
| 320 |
+
: "";
|
| 321 |
+
if ($("checkwarn").innerHTML !== ckHtml) $("checkwarn").innerHTML = ckHtml;
|
| 322 |
+
$("checkwarn").style.display = ckHtml ? "" : "none";
|
| 323 |
$("seng").textContent = "FastFlowLM " + (s.engine.installed || T("未インストール"));
|
| 324 |
const ed = s.engine_dl, e = s.engine;
|
| 325 |
let est;
|
pyproject.toml
CHANGED
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
| 4 |
|
| 5 |
[project]
|
| 6 |
name = "onw-amd"
|
| 7 |
-
version = "0.1.
|
| 8 |
description = "Ore no NPU ga konna ni ugoku wake nai - LLMs entirely on the AMD Ryzen AI NPU (FastFlowLM, as in Lemonade Server)"
|
| 9 |
readme = "README.md"
|
| 10 |
license = {text = "Apache-2.0"}
|
|
|
|
| 4 |
|
| 5 |
[project]
|
| 6 |
name = "onw-amd"
|
| 7 |
+
version = "0.1.1"
|
| 8 |
description = "Ore no NPU ga konna ni ugoku wake nai - LLMs entirely on the AMD Ryzen AI NPU (FastFlowLM, as in Lemonade Server)"
|
| 9 |
readme = "README.md"
|
| 10 |
license = {text = "Apache-2.0"}
|