Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf LiquidAI/LFM2.5-Encoder-350M-GGUF:# Run inference directly in the terminal:
llama cli -hf LiquidAI/LFM2.5-Encoder-350M-GGUF:Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf LiquidAI/LFM2.5-Encoder-350M-GGUF:# Run inference directly in the terminal:
./llama-cli -hf LiquidAI/LFM2.5-Encoder-350M-GGUF:Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf LiquidAI/LFM2.5-Encoder-350M-GGUF:# Run inference directly in the terminal:
./build/bin/llama-cli -hf LiquidAI/LFM2.5-Encoder-350M-GGUF:Use Docker
docker model run hf.co/LiquidAI/LFM2.5-Encoder-350M-GGUF:
LFM2.5-Encoder-350M
LFM2.5-Encoder-350M is a multilingual bidirectional encoder built on the LFM2 architecture — a larger encoder for maximum downstream quality. It is a masked language model with full bidirectional attention, designed to be fine-tuned into task-specific models (classification, token classification, retrieval, reranking, and semantic similarity) across 15 languages, and to run efficiently on-device.
- Highly capable for its size. On par with the best similarly sized encoders and well ahead of our own retrieval siblings.
- General-purpose. 8k context, strong across NLI, paraphrase, sentiment, and multilingual tasks.
- Fast and on-device. Matches or beats ModernBERT throughput, with a long-context edge on CPU; runs in the browser on WebGPU.
Find more information about LFM2.5-Encoder-350M in our blog post.
🏃 How to run
Example usage with llama.cpp:
Start llama-server with per-token embeddings
hf download LiquidAI/LFM2.5-Encoder-350M-GGUF LFM2.5-Encoder-350M-F16.gguf --local-dir .
llama-server -m LFM2.5-Encoder-350M-F16.gguf --embeddings --pooling none
Run masked-token prediction — the mask position's logits come from the per-token hidden states and the tied embedding matrix read from the GGUF (fill-mask.py in this repo)
❯ uv run fill-mask.py LFM2.5-Encoder-350M-F16.gguf "The capital of France is [MASK]."
top-5 at [MASK]:
# 1 11.42 ' Paris'
# 2 10.43 'Paris'
# 3 9.65 ' Nice'
# 4 8.94 ' Strasbourg'
# 5 8.62 ' Lyon'
The same server also serves per-token embeddings directly:
curl -s http://localhost:8080/embedding -d '{"content": "hello world"}'
Find more details in the original model card: https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M
- Downloads last month
- 392
4-bit
8-bit
16-bit
Model tree for LiquidAI/LFM2.5-Encoder-350M-GGUF
Base model
LiquidAI/LFM2.5-350M-Base
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf LiquidAI/LFM2.5-Encoder-350M-GGUF:# Run inference directly in the terminal: llama cli -hf LiquidAI/LFM2.5-Encoder-350M-GGUF: