Instructions to use NotASI/Qwen2-0.5B-Math with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use NotASI/Qwen2-0.5B-Math with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="NotASI/Qwen2-0.5B-Math")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("NotASI/Qwen2-0.5B-Math")
model = AutoModelForCausalLM.from_pretrained("NotASI/Qwen2-0.5B-Math", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use NotASI/Qwen2-0.5B-Math with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NotASI/Qwen2-0.5B-Math"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NotASI/Qwen2-0.5B-Math",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/NotASI/Qwen2-0.5B-Math

SGLang

How to use NotASI/Qwen2-0.5B-Math with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "NotASI/Qwen2-0.5B-Math" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NotASI/Qwen2-0.5B-Math",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "NotASI/Qwen2-0.5B-Math" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NotASI/Qwen2-0.5B-Math",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use NotASI/Qwen2-0.5B-Math with Docker Model Runner:
```
docker model run hf.co/NotASI/Qwen2-0.5B-Math
```

Qwen2-0.5B-Math

Commit History

Adding `safetensors` variant of this model (#1)

9107fe0
verified

NotAIlol

SFconvertbot commited on Jul 15, 2024

Update README.md

76728cd
verified

NotAIlol commited on Jul 9, 2024

Update README.md

db15e4b
verified

NotAIlol commited on Jul 8, 2024

Update README.md

f6bcec1
verified

NotAIlol commited on Jul 8, 2024

Trained with Unsloth

c870d6d
verified

NotAIlol commited on Jul 8, 2024

Upload tokenizer

633b690
verified

NotAIlol commited on Jul 8, 2024

Upload README.md with huggingface_hub

a211b0a
verified

NotAIlol commited on Jul 8, 2024

initial commit

8f5de3f
verified

NotAIlol commited on Jul 8, 2024

Commit History

Adding `safetensors` variant of this model (#1) 9107fe0 verified

Update README.md 76728cd verified

Update README.md db15e4b verified

Update README.md f6bcec1 verified

Trained with Unsloth c870d6d verified

Upload tokenizer 633b690 verified

Upload README.md with huggingface_hub a211b0a verified

initial commit 8f5de3f verified

Adding `safetensors` variant of this model (#1)

9107fe0
verified

Update README.md

76728cd
verified

Update README.md

db15e4b
verified

Update README.md

f6bcec1
verified

Trained with Unsloth

c870d6d
verified

Upload tokenizer

633b690
verified

Upload README.md with huggingface_hub

a211b0a
verified

initial commit

8f5de3f
verified