Instructions to use ornith-ai/Ornith-1.5-397B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ornith-ai/Ornith-1.5-397B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ornith-ai/Ornith-1.5-397B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ornith-ai/Ornith-1.5-397B")
model = AutoModelForMultimodalLM.from_pretrained("ornith-ai/Ornith-1.5-397B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ornith-ai/Ornith-1.5-397B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ornith-ai/Ornith-1.5-397B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ornith-ai/Ornith-1.5-397B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ornith-ai/Ornith-1.5-397B

SGLang

How to use ornith-ai/Ornith-1.5-397B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ornith-ai/Ornith-1.5-397B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ornith-ai/Ornith-1.5-397B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ornith-ai/Ornith-1.5-397B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ornith-ai/Ornith-1.5-397B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ornith-ai/Ornith-1.5-397B with Docker Model Runner:
```
docker model run hf.co/ornith-ai/Ornith-1.5-397B
```

chirpchirp commited on Aug 19

Commit

22e1dbf

verified ·

1 Parent(s): ba38f48

Add files using upload-large-folder tool

Browse files

Files changed (8) hide show

model-00003-of-00122.safetensors +1 -1
model-00047-of-00122.safetensors +1 -1
model-00048-of-00122.safetensors +1 -1
model-00065-of-00122.safetensors +1 -1
model-00074-of-00122.safetensors +1 -1
model-00093-of-00122.safetensors +1 -1
model-00110-of-00122.safetensors +1 -1
model-00118-of-00122.safetensors +1 -1

model-00003-of-00122.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:862860d6ee9ef659aa049d8e80c3b2c861db294d6a4f33c40b7e13a85de6cf1d
 size 4560382680

 version https://git-lfs.github.com/spec/v1
+oid sha256:3494c559efb660bac16cd9d4126a689c982f503836cae711acb29ebdced47384
 size 4560382680

model-00047-of-00122.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:768a5891cd2e23d12edd2e6488b8862bb24c9ac04ce4d5f146c9f6856d363711
 size 4534070104

 version https://git-lfs.github.com/spec/v1
+oid sha256:7423ac95e2bf59aed715b8fc962a6f4e85a08e5a7e438b85b4dfd46a981c27bc
 size 4534070104

model-00048-of-00122.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8d64105a168d469ae35d58659e8f3aace460358bfdb46cee7a7f1a2f58b93c65
 size 8589934736

 version https://git-lfs.github.com/spec/v1
+oid sha256:3b7be0b8b020e80a63fceebcf644e031d461e40d1521660ca186fb7b4f5975c5
 size 8589934736

model-00065-of-00122.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:42074905c48cec85c147c3260f17fde49b9df8397468d775f5cc5020d471ca07
 size 4560382696

 version https://git-lfs.github.com/spec/v1
+oid sha256:f859649312fb4599d177f7a17d08ed2dc4cb2919e7fb5f0ca1c6e1933baa33b2
 size 4560382696

model-00074-of-00122.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:62395226f4d918812576f9b4dbe69249b6aef8ad48539324ecf86e1fe0b496de
 size 8589934736

 version https://git-lfs.github.com/spec/v1
+oid sha256:446a2eeda72f5735501cf1c60fd49a80dd642b6f65f64024034ad735c13d716c
 size 8589934736

model-00093-of-00122.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3f314bd79afc4200e9dff48810d685cd8c1af98cef285f17bec30dc9758a2e43
 size 4560382696

 version https://git-lfs.github.com/spec/v1
+oid sha256:c8e22bd9797bc93158251ddf9ed323853bd31ab875fe480527510404098d61d3
 size 4560382696

model-00110-of-00122.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d3739f305980df67948f3f079d6abc7857031a0ae62a813e8d39397772b893a7
 size 8589934736

 version https://git-lfs.github.com/spec/v1
+oid sha256:747c3191fc40eba6a8e7642ec948c9efd698aec900677f3bd7422069afb8e4cb
 size 8589934736

model-00118-of-00122.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5437c7086fee7f009faf012dc8297fd0ce6cac5e7f0d3d18e7a0ae1a2548435e
 size 8589934736

 version https://git-lfs.github.com/spec/v1
+oid sha256:eef2e603671f055186ac0b3f2a5ad254038eaa0c1fa114294394e64795c07be2
 size 8589934736