Instructions to use AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP / model-manifest.json

AutomatosX

AXQ development pack from Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0

d305817 verified 6 days ago

Raw

History Blame Contribute Delete

434 kB

	{
	"schema_version": "ax.native_model.v1",
	"model_family": "qwen3_5",
	"tensor_format": "safetensors",
	"layer_count": 64,
	"hidden_size": 5120,
	"intermediate_size": 17408,
	"attention_head_count": 24,
	"attention_head_dim": 256,
	"kv_head_count": 4,
	"vocab_size": 248320,
	"tie_word_embeddings": false,
	"rope_theta": 10000000,
	"no_rope_layer_interval": 0,
	"intermediate_size_mlp": 0,
	"attn_output_gate": true,
	"partial_rotary_factor": 0.25,
	"rms_norm_eps": 0.000001,
	"moe_norm_topk_prob": true,
	"hidden_size_per_layer_input": 0,
	"linear_attention": {
	"full_attention_interval": 4,
	"num_value_heads": 48,
	"num_key_heads": 16,
	"key_head_dim": 128,
	"value_head_dim": 128,
	"conv_kernel_dim": 4
	},
	"think_start_token_id": 248068,
	"think_end_token_id": 248069,
	"dropped_tensors": {
	"count": 1009,
	"media_role_hits": 0,
	"names_sample": [
	"language_model.model.embed_tokens.biases",
	"language_model.model.embed_tokens.scales",
	"language_model.model.layers.0.linear_attn.in_proj_a.biases",
	"language_model.model.layers.0.linear_attn.in_proj_a.scales",
	"language_model.model.layers.0.linear_attn.in_proj_b.biases",
	"language_model.model.layers.0.linear_attn.in_proj_b.scales",
	"language_model.model.layers.0.linear_attn.in_proj_qkv.biases",
	"language_model.model.layers.0.linear_attn.in_proj_qkv.scales",
	"language_model.model.layers.0.linear_attn.in_proj_z.biases",
	"language_model.model.layers.0.linear_attn.in_proj_z.scales",
	"language_model.model.layers.0.linear_attn.out_proj.biases",
	"language_model.model.layers.0.linear_attn.out_proj.scales",
	"language_model.model.layers.0.mlp.down_proj.biases",
	"language_model.model.layers.0.mlp.down_proj.scales",
	"language_model.model.layers.0.mlp.gate_proj.biases",
	"language_model.model.layers.0.mlp.gate_proj.scales"
	]
	},
	"tensors": [
	{
	"name": "language_model.model.norm.weight",
	"role": "final_norm",
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2604080296,
	"length_bytes": 10240
	},
	{
	"name": "language_model.lm_head.weight",
	"role": "lm_head",
	"dtype": "bf16",
	"shape": [
	248320,
	5120
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 7336,
	"length_bytes": 2542796800
	},
	{
	"name": "model.visual.blocks.0.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 37128,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.0.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 39432,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.0.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 2693640,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.0.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 2700552,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.0.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 10663176,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.0.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 10671784,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.0.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 20588200,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.0.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 20590504,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.0.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 30506920,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.0.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 30509224,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.0.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 30511528,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.0.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 30513832,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.1.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 30516136,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.1.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 30518440,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.1.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 33172648,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.1.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 33179560,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.1.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 41142184,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.1.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 41150792,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.1.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 51067208,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.1.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 51069512,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.1.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 60985928,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.1.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 60988232,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.1.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 60990536,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.1.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 60992840,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.10.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 60995144,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.10.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 60997448,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.10.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 63651656,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.10.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 63658568,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.10.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 71621192,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.10.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 71629800,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.10.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 81546216,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.10.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 81548520,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.10.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 91464936,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.10.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 91467240,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.10.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 91469544,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.10.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 91471848,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.11.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 91474152,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.11.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 91476456,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.11.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 94130664,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.11.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 94137576,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.11.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 102100200,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.11.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 102108808,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.11.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 112025224,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.11.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 112027528,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.11.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 121943944,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.11.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 121946248,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.11.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 121948552,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.11.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 121950856,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.12.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 121953160,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.12.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 121955464,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.12.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 124609672,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.12.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 124616584,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.12.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 132579208,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.12.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 132587816,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.12.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 142504232,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.12.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 142506536,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.12.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 152422952,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.12.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 152425256,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.12.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 152427560,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.12.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 152429864,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.13.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 152432168,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.13.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 152434472,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.13.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 155088680,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.13.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 155095592,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.13.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 163058216,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.13.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 163066824,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.13.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 172983240,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.13.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 172985544,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.13.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 182901960,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.13.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 182904264,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.13.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 182906568,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.13.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 182908872,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.14.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 182911176,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.14.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 182913480,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.14.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 185567688,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.14.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 185574600,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.14.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 193537224,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.14.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 193545832,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.14.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 203462248,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.14.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 203464552,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.14.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 213380968,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.14.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 213383272,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.14.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 213385576,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.14.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 213387880,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.15.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 213390184,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.15.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 213392488,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.15.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 216046696,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.15.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 216053608,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.15.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 224016232,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.15.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 224024840,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.15.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 233941256,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.15.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 233943560,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.15.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 243859976,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.15.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 243862280,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.15.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 243864584,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.15.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 243866888,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.16.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 243869192,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.16.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 243871496,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.16.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 246525704,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.16.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 246532616,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.16.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 254495240,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.16.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 254503848,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.16.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 264420264,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.16.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 264422568,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.16.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 274338984,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.16.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 274341288,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.16.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 274343592,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.16.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 274345896,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.17.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 274348200,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.17.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 274350504,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.17.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 277004712,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.17.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 277011624,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.17.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 284974248,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.17.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 284982856,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.17.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 294899272,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.17.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 294901576,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.17.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 304817992,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.17.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 304820296,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.17.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 304822600,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.17.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 304824904,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.18.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 304827208,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.18.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 304829512,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.18.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 307483720,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.18.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 307490632,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.18.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 315453256,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.18.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 315461864,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.18.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 325378280,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.18.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 325380584,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.18.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 335297000,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.18.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 335299304,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.18.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 335301608,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.18.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 335303912,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.19.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 335306216,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.19.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 335308520,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.19.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 337962728,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.19.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 337969640,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.19.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 345932264,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.19.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 345940872,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.19.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 355857288,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.19.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 355859592,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.19.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 365776008,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.19.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 365778312,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.19.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 365780616,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.19.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 365782920,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.2.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 365785224,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.2.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 365787528,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.2.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 368441736,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.2.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 368448648,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.2.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 376411272,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.2.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 376419880,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.2.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 386336296,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.2.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 386338600,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.2.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 396255016,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.2.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 396257320,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.2.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 396259624,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.2.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 396261928,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.20.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 396264232,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.20.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 396266536,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.20.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 398920744,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.20.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 398927656,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.20.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 406890280,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.20.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 406898888,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.20.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 416815304,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.20.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 416817608,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.20.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 426734024,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.20.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 426736328,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.20.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 426738632,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.20.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 426740936,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.21.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 426743240,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.21.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 426745544,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.21.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 429399752,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.21.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 429406664,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.21.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 437369288,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.21.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 437377896,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.21.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 447294312,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.21.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 447296616,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.21.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 457213032,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.21.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 457215336,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.21.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 457217640,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.21.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 457219944,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.22.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 457222248,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.22.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 457224552,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.22.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 459878760,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.22.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 459885672,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.22.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 467848296,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.22.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 467856904,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.22.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 477773320,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.22.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 477775624,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.22.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 487692040,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.22.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 487694344,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.22.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 487696648,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.22.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 487698952,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.23.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 487701256,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.23.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 487703560,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.23.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 490357768,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.23.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 490364680,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.23.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 498327304,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.23.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 498335912,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.23.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 508252328,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.23.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 508254632,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.23.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 518171048,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.23.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 518173352,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.23.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 518175656,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.23.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 518177960,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.24.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 518180264,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.24.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 518182568,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.24.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 520836776,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.24.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 520843688,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.24.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 528806312,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.24.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 528814920,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.24.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 538731336,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.24.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 538733640,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.24.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 548650056,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.24.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 548652360,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.24.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 548654664,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.24.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 548656968,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.25.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 548659272,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.25.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 548661576,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.25.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 551315784,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.25.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 551322696,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.25.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 559285320,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.25.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 559293928,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.25.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 569210344,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.25.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 569212648,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.25.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 579129064,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.25.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 579131368,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.25.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 579133672,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.25.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 579135976,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.26.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 579138280,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.26.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 579140584,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.26.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 581794792,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.26.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 581801704,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.26.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 589764328,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.26.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 589772936,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.26.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 599689352,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.26.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 599691656,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.26.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 609608072,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.26.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 609610376,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.26.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 609612680,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.26.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 609614984,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.3.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 609617288,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.3.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 609619592,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.3.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 612273800,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.3.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 612280712,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.3.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 620243336,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.3.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 620251944,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.3.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 630168360,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.3.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 630170664,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.3.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 640087080,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.3.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 640089384,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.3.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 640091688,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.3.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 640093992,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.4.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 640096296,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.4.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 640098600,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.4.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 642752808,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.4.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 642759720,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.4.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 650722344,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.4.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 650730952,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.4.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 660647368,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.4.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 660649672,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.4.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 670566088,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.4.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 670568392,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.4.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 670570696,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.4.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 670573000,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.5.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 670575304,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.5.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 670577608,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.5.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 673231816,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.5.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 673238728,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.5.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 681201352,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.5.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 681209960,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.5.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 691126376,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.5.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 691128680,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.5.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 701045096,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.5.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 701047400,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.5.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 701049704,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.5.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 701052008,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.6.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 701054312,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.6.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 701056616,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.6.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 703710824,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.6.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 703717736,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.6.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 711680360,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.6.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 711688968,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.6.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 721605384,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.6.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 721607688,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.6.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 731524104,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.6.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 731526408,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.6.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 731528712,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.6.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 731531016,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.7.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 731533320,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.7.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 731535624,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.7.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 734189832,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.7.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 734196744,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.7.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 742159368,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.7.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 742167976,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.7.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 752084392,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.7.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 752086696,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.7.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 762003112,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.7.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 762005416,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.7.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 762007720,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.7.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 762010024,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.8.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 762012328,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.8.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 762014632,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.8.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 764668840,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.8.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 764675752,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.8.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 772638376,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.8.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 772646984,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.8.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 782563400,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.8.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 782565704,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.8.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 792482120,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.8.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 792484424,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.8.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 792486728,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.8.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 792489032,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.9.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 792491336,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.9.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 792493640,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.9.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 795147848,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.9.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 795154760,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.9.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 803117384,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.9.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 803125992,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.9.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 813042408,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.9.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 813044712,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.9.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 822961128,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.9.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 822963432,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.9.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 822965736,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.9.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 822968040,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.merger.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4608
	],
	"file": "vision.safetensors",
	"offset_bytes": 822970344,
	"length_bytes": 9216
	},
	{
	"name": "model.visual.merger.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4608,
	4608
	],
	"file": "vision.safetensors",
	"offset_bytes": 822979560,
	"length_bytes": 42467328
	},
	{
	"name": "model.visual.merger.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "vision.safetensors",
	"offset_bytes": 865446888,
	"length_bytes": 10240
	},
	{
	"name": "model.visual.merger.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	5120,
	4608
	],
	"file": "vision.safetensors",
	"offset_bytes": 865457128,
	"length_bytes": 47185920
	},
	{
	"name": "model.visual.merger.norm.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 912643048,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.merger.norm.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 912645352,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.patch_embed.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 912647656,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.patch_embed.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	3,
	2,
	16,
	16
	],
	"file": "vision.safetensors",
	"offset_bytes": 912649960,
	"length_bytes": 3538944
	},
	{
	"name": "model.visual.pos_embed.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	2304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 916188904,
	"length_bytes": 5308416
	},
	{
	"name": "language_model.model.embed_tokens.weight",
	"role": "token_embedding",
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 8
	},
	"shape": [
	248320,
	1280
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3232195958,
	"length_bytes": 1271398400
	},
	{
	"name": "language_model.model.layers.0.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 0,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5097934998,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.0.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 0,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2304797142,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.0.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4685672246,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.0.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5269046198,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.0.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2423658198,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.0.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 0,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1748160086,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.0.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 0,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5011033238,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.0.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 0,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3004124918,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.0.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4745965366,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.0.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 891351766,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.0.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4984818838,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.0.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5122250390,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.0.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 0,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4807174966,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.0.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4575376758,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.1.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 1,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1142068022,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.1.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 1,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5030079638,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.1.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3169224822,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.1.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2809389974,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.1.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2133399926,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.1.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 1,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4605359478,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.1.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 1,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5185851222,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.1.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 1,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3033124694,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.1.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5029956758,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.1.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2095675766,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.1.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2257273302,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.1.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2283487702,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.1.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 1,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2649086774,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.1.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5032071318,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.2.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 2,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4503594358,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.2.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 2,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4503604598,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.2.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2764810134,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.2.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2207138262,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.2.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4513117558,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.2.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 2,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5216944982,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.2.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 2,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3092793366,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.2.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 2,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4638813910,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.2.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5313610678,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.2.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2125740406,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.2.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1841876662,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.2.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4730236726,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.2.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 2,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2891719670,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.2.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4557682038,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.3.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3214460022,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.3.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 3,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2719942806,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.3.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 3,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2649076438,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.3.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3137439862,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.3.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 3,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1144070038,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.3.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2595050198,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.3.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 3,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5122249878,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.3.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1774374582,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.3.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2503939798,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.3.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2959560438,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.3.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 481438742,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.4.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 4,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3232170102,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.4.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 4,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2343924438,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.4.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1159839734,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.4.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2894505206,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.4.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5141286742,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.4.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 4,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2649086678,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.4.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 4,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2886067094,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.4.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 4,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2664831030,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.4.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 526018582,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.4.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2719943318,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.4.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2859852694,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.4.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2720066198,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.4.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 4,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1368971350,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.4.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2630439638,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.5.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 5,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2299216342,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.5.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 5,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2191399382,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.5.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4759195446,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.5.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 955730646,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.5.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2018025846,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.5.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 5,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3137439766,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.5.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 5,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2423576278,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.5.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 5,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5137994486,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.5.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2328072918,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.5.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 660367382,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.5.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2682914966,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.5.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2649087030,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.5.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 5,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2310377942,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.5.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2347211478,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.6.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 6,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1922168598,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.6.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 6,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1922158262,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.6.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5224481718,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.6.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1877593782,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.6.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5047799958,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.6.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 6,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4875988022,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.6.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 6,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 680611862,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.6.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 6,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2013274390,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.6.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2084872566,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.6.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2095798646,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.6.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2099526006,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.6.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3153496182,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.6.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 6,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2013274134,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.6.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2310378198,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.7.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1777651382,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.7.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 7,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1660627542,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.7.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 7,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1614086486,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.7.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1732431446,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.7.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 7,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1614076246,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.7.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1799278262,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.7.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 7,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1668190294,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.7.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2857231254,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.7.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1563925846,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.7.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4884876438,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.7.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1513708886,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.8.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 8,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2944640246,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.8.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 8,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1428732246,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.8.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3033140150,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.8.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1287839734,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.8.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1685900886,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.8.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 8,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1332419574,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.8.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 8,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5313733558,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.8.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 8,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2177964406,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.8.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1015542582,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.8.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1382078806,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.8.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1442832726,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.8.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2191409622,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.8.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 8,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2013273878,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.8.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2477070038,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.9.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 9,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1142078262,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.9.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 9,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5122239638,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.9.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1086362422,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.9.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2550485718,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.9.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5313815478,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.9.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 9,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1144054582,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.9.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 9,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1513626966,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.9.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 9,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1144080278,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.9.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4807036726,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.9.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1276575734,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.9.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1748160182,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.9.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1144111094,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.9.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 9,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2735810198,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.9.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1668190806,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.10.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 10,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1009823542,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.10.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 10,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1005865686,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.10.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1968709398,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.10.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 730654838,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.10.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1922178838,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.10.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 10,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1922168502,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.10.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 10,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1086280502,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.10.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 10,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1009808086,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.10.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1009833782,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.10.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1015680822,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.10.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1060066102,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.10.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1019746102,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.10.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 10,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3232180342,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.10.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1041060662,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.11.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2666827926,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.11.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 11,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2953989366,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.11.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 11,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 857917654,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.11.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 859894486,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.11.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 11,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1668164694,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.11.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 907561686,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.11.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 11,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 857927894,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.11.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 891474646,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.11.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1384167766,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.11.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1218822134,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.11.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 780912854,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.12.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 12,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 680601622,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.12.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 12,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1017769782,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.12.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 615802902,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.12.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 686090358,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.12.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1614096982,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.12.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 12,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2886149014,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.12.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 12,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1558273366,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.12.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 12,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 682659862,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.12.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2648953558,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.12.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 682675318,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.12.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1332419670,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.12.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4623085270,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.12.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 12,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1614096726,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.12.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2062590326,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.13.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 13,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 465684246,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.13.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 13,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 465674006,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.13.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2368198358,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.13.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 421109526,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.13.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 359833366,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.13.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 13,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 780789878,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.13.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 13,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4836174902,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.13.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 13,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3079686070,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.13.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 526141462,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.13.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 780789974,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.13.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 544286742,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.13.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5011115158,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.13.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 13,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 481438486,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.13.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 465709846,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.14.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 325482838,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.14.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 325472598,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.14.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 280907606,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.14.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4640780086,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.14.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 225202006,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.14.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 343188054,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.14.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1270923254,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.14.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 343188150,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.14.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 343203606,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.14.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 343326486,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.14.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2738595734,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.14.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2328195798,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.14.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 343187798,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.14.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 327459158,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.15.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 15,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 181620566,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.15.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 15,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 325472086,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.15.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 15,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4884866198,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.15.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 15,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4605359574,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.15.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 15,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 176039254,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.15.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 15,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 188174166,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.15.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 15,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 176049494,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.15.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 15,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 178671446,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.15.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 15,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1469047126,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.15.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 15,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 131474774,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.15.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 15,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3092875286,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.16.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 16,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 100426998,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.16.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 16,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 100416758,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.16.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 50281718,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.16.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4929440918,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.16.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5717238,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.16.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 16,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 120098038,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.16.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 16,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 120251734,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.16.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 16,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5137979030,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.16.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 120128854,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.16.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 680478742,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.16.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3006910294,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.16.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 842189014,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.16.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 16,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4621103574,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.16.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 104369398,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.17.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 17,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2175123718,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.17.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 17,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4230794630,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.17.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5093936134,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.17.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4755400518,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.17.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2177146374,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.17.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 17,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5271251974,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.17.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 17,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 64758,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.17.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 17,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4922200998,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.17.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4678195686,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.17.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2756106790,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.17.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4809960502,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.17.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4209495430,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.17.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 17,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4108215942,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.17.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3866807942,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.18.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 18,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5265671174,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.18.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 18,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4678185446,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.18.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4582369766,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.18.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1300963174,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.18.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4486031846,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.18.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 18,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3162186214,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.18.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 18,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3268165286,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.18.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 18,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3363233446,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.18.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4478372326,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.18.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4755262278,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.18.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4895986598,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.18.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2006542150,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.18.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 18,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4680627782,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.18.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3707022982,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.19.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4392177030,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.19.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 19,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2223687174,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.19.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 19,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4345625990,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.19.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4320722310,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.19.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 19,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4250588550,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.19.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3618164678,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.19.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 19,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4838631590,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.19.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4336450950,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.19.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2411535558,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.19.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2082584742,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.19.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5199807494,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.20.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 20,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4846168742,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.20.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 20,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4108216198,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.20.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3998432902,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.20.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3485100742,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.20.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4532562406,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.20.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 20,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1428139366,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.20.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 20,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4159278214,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.20.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 20,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3699767334,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.20.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4674785766,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.20.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4143959174,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.20.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1874021190,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.20.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4410214790,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.20.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 20,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4799964998,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.20.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3538512582,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.21.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 21,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1566922470,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.21.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 21,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3377826246,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.21.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5049371654,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.21.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1696110310,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.21.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5004807174,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.21.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 21,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2827065222,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.21.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 21,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3885828742,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.21.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 21,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3664039878,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.21.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4159155334,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.21.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3377702854,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.21.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3667972134,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.21.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3749293702,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.21.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 21,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3618164422,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.21.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3557533382,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.22.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 22,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3694186534,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.22.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 22,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2916209638,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.22.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4693970758,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.22.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3108108518,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.22.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3037247718,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.22.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 22,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3167756870,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.22.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 22,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3383422406,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.22.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 22,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2297088006,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.22.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3262456486,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.22.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3357514406,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.22.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3723079302,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.22.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4822902950,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.22.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 22,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3162175718,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.22.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1832651078,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.23.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3885926022,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.23.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 23,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2965797350,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.23.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 23,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4674775526,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.23.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1531491782,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.23.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 23,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4046929542,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.23.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3920445062,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.23.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 23,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2963487718,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.23.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4343004550,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.23.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2827065318,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.23.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2916301798,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.23.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3268247206,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.24.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 24,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2668482854,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.24.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 24,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1547220422,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.24.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2678118950,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.24.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2607206694,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.24.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2543964454,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.24.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 24,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1564925382,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.24.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 24,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2916219878,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.24.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 24,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2756229670,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.24.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1802032966,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.24.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4686198598,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.24.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2784139142,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.24.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2761841286,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.24.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 24,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4115763078,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.24.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2724649510,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.25.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 25,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2302986438,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.25.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 25,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3162175974,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.25.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3573272262,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.25.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4052510342,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.25.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3212321446,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.25.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 25,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4433807750,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.25.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 25,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4693858118,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.25.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 25,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2532792518,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.25.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2372265158,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.25.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1451404742,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.25.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2506578118,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.25.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2456100038,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.25.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 25,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3477548486,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.25.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2324623558,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.26.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 26,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2027851590,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.26.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 26,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2027841350,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.26.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3822161542,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.26.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4270587270,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.26.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1956407110,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.26.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 26,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2082584646,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.26.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 26,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2372137158,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.26.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 26,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2127149222,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.26.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2171724038,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.26.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3809586822,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.26.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4977922054,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.26.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2306928838,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.26.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 26,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2032449350,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.26.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2471828678,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.27.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3472960966,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.27.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 27,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1821509446,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.27.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 27,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1821499206,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.27.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1857964870,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.27.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 27,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1804777286,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.27.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1904167750,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.27.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 27,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1848379718,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.27.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1802155846,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.27.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3428396486,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.27.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1757468390,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.27.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1645852390,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.28.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 28,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1529515462,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.28.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 28,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3366510342,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.28.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3951902342,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.28.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1479380422,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.28.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2871645158,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.28.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 28,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1564925478,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.28.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 28,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2030073670,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.28.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 28,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4678661606,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.28.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1566948070,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.28.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1567070950,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.28.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1569175270,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.28.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4234859910,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.28.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 28,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2668493094,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.28.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1549196742,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.29.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 29,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1406829926,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.29.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 29,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1406819686,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.29.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3167756966,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.29.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1232984934,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.29.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1351098214,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.29.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 29,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1564925574,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.29.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 29,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3866726022,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.29.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 29,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2302986342,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.29.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2406154438,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.29.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2029950790,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.29.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5298152646,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.29.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1435676102,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.29.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 29,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2780846726,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.29.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1412410726,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.30.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 30,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1279653734,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.30.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 30,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2221710854,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.30.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2231224326,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.30.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3383832006,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.30.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1160506214,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.30.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 30,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1802032870,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.30.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 30,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3888562822,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.30.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 30,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3377702662,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.30.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1695987430,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.30.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1277549414,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.30.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3888660102,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.30.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1279663974,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.30.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 30,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4846178982,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.30.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5276853446,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.31.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1138879334,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.31.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 31,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3377825734,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.31.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 31,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1131649126,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.31.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2487572678,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.31.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 31,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1131623526,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.31.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5144071174,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.31.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 31,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1131659366,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.31.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1133953638,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.31.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4849456038,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.31.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1087059046,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.31.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1025782886,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.32.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 32,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 950144774,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.32.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 32,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3363223206,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.32.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 905580294,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.32.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4927771654,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.32.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 849874694,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.32.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 32,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5276853350,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.32.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 32,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1848380230,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.32.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 32,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 965883910,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.32.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 965899366,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.32.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 966022246,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.32.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 988427366,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.32.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 966145126,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.32.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 32,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 965883654,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.32.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 950155014,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.33.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 33,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 784707142,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.33.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 33,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4392166790,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.33.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 734556742,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.33.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3312949926,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.33.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2032449606,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.33.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 33,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 800446022,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.33.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 33,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1757386470,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.33.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 33,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 800446118,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.33.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 800461574,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.33.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 816328454,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.33.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 823660294,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.33.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2281359366,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.33.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 33,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3034953702,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.33.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 784717382,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.34.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 34,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 697282694,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.34.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 34,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 697272454,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.34.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 647137414,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.34.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4159360134,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.34.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1595389670,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.34.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 34,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 699259270,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.34.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 34,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 734147142,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.34.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 34,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 699259366,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.34.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 699274822,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.34.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4232770950,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.34.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 707932742,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.34.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2740378150,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.34.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 34,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 699259014,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.34.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5249942534,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.35.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 609126534,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.35.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 35,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 545867910,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.35.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 35,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 545857670,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.35.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 547835014,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.35.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 35,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2171713798,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.35.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 615680134,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.35.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 35,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 545868422,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.35.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2960866278,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.35.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 451158150,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.35.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 501293190,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.35.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 563563654,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.36.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 36,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 395672518,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.36.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 36,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3377677062,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.36.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2127149318,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.36.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 351108038,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.36.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4433807846,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.36.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 36,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 439935014,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.36.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 36,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 439935110,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.36.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 36,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4808484934,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.36.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1950713670,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.36.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2029827910,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.36.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2372388038,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.36.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 420929574,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.36.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 36,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1138879078,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.36.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 399615014,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.37.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 37,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3573262022,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.37.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 37,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1138868838,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.37.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 195019622,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.37.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 245154662,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.37.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4630211046,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.37.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 37,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 309379942,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.37.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 37,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 334314438,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.37.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 37,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3377702758,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.37.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2676029990,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.37.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3312811686,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.37.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2340352198,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.37.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 315308998,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.37.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 37,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2175164678,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.37.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 293651302,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.38.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 38,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 120692486,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.38.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 38,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 120682246,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.38.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 70547206,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.38.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2980886502,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.38.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4347602310,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.38.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 38,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 138397446,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.38.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 38,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3096885478,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.38.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 38,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 395682758,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.38.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 138397542,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.38.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 138535782,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.38.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 157664102,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.38.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 138658662,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.38.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 38,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4209495174,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.38.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 122668806,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.39.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 15824646,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.39.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 39,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1395662694,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.39.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 39,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 11226886,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.39.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 800599814,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.39.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 39,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2761831046,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.39.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 22378246,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.39.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 39,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3705041030,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.39.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 13203206,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.39.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2729026866,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.39.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3765022342,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.39.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5299077042,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.40.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 40,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4876885234,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.40.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 40,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4404006770,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.40.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4745045042,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.40.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3630534962,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.40.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3059661554,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.40.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 40,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3310631346,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.40.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 40,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4007164306,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.40.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 40,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4456594386,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.40.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1512875666,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.40.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4791672882,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.40.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5206860722,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.40.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4435284850,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.40.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 40,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5074725042,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.40.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4684654642,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.41.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 41,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1545059986,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.41.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 41,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4642693682,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.41.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4521493042,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.41.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3962583954,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.41.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2189819826,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.41.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 41,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4360811602,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.41.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 41,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4313766386,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.41.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 41,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4791795762,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.41.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4366389938,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.41.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5233090482,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.41.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5048167506,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.41.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4817042578,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.41.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 41,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4182617330,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.41.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4668926002,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.42.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 42,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5180599218,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.42.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 42,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4451013490,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.42.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4700480562,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.42.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4269079026,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.42.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4314199026,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.42.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 42,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4388278034,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.42.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 42,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3927767954,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.42.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 42,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3306507090,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.42.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4404017010,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.42.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4640604722,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.42.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4482163762,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.42.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4461847602,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.42.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 42,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3721640754,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.42.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4419556210,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.43.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4477576242,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.43.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 43,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2667422514,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.43.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 43,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4071497714,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.43.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3603664946,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.43.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 43,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4007246834,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.43.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4191075826,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.43.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 43,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3848990866,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.43.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3927849874,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.43.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2412532242,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.43.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5080295858,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.43.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4012835314,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.44.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 44,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3904820114,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.44.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 44,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3721630002,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.44.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3118725874,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.44.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3677065522,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.44.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3530218290,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.44.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 44,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5074381906,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.44.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 44,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1520545426,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.44.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 44,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3800980530,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.44.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3800995986,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.44.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3848991378,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.44.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3163290354,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.44.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3908762514,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.44.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 44,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1975111922,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.44.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3721641010,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.45.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 45,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3785241650,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.45.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 45,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3300926290,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.45.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3234069234,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.45.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4936725490,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.45.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3189504754,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.45.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 45,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3326360082,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.45.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 45,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4634952242,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.45.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 45,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3331946098,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.45.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3331961554,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.45.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4313643506,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.45.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3476606674,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.45.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3875984274,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.45.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 45,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3630534706,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.45.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3310631442,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.46.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 46,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2976752946,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.46.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 46,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2100516338,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.46.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4992454130,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.46.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2932188466,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.46.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4093160690,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.46.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 46,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3295345394,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.46.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 46,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4629299762,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.46.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 46,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3024932242,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.46.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4368494258,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.46.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3030518514,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.46.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3930798994,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.46.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3035884274,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.46.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 46,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3875984018,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.46.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3005271346,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.47.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2773591346,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.47.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 47,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4007246322,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.47.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 47,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2666899762,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.47.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3580368690,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.47.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 47,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5206840242,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.47.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2875827506,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.47.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 47,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2667094322,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.47.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2679219506,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.47.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2616757042,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.47.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2472323954,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.47.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4573594162,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.48.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 48,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4073489394,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.48.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 48,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1977078258,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.48.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5130438578,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.48.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2367967762,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.48.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5254512562,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.48.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 48,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2464633618,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.48.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 48,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3113073394,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.48.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 48,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2560723570,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.48.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2466615154,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.48.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2516888434,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.48.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4642703922,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.48.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2522589554,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.48.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 48,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2555152754,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.48.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4388278130,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.49.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 49,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2098540018,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.49.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 49,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2098529778,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.49.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4224499186,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.49.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2053965298,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.49.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2234384306,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.49.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 49,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2169667314,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.49.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 49,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4791590962,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.49.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 49,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2169667410,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.49.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2549459314,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.49.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2189696946,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.49.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2291267602,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.49.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1671585426,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.49.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 49,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2150661618,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.49.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3514489554,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.50.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 50,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4461837362,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.50.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 50,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2175238066,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.50.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1860682386,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.50.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2682168626,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.50.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3804426386,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.50.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 50,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5048167410,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.50.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 50,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1975030002,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.50.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 50,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1905246866,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.50.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1931789042,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.50.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1931911922,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.50.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1948815602,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.50.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2986265906,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.50.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 50,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2464633362,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.50.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1842987666,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.51.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2918425906,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.51.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 51,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5195690418,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.51.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 51,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1600796306,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.51.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4075465714,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.51.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 51,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1600786066,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.51.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1734172306,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.51.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 51,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3721640242,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.51.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1836418706,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.51.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1545070226,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.51.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2100526578,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.51.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2776212786,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.52.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 52,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1512998546,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.52.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 52,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1505328786,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.52.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1399488146,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.52.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1460764306,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.52.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1350991506,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.52.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 52,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2287806386,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.52.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 52,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4700398642,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.52.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 52,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2616756946,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.52.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1520627346,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.52.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3310447026,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.52.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1645371026,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.52.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3849114258,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.52.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 52,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4093160434,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.52.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5238783922,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.53.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 53,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1309795026,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.53.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 53,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4869338354,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.53.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1215095506,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.53.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1265230546,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.53.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1164960466,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.53.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 53,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1311771602,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.53.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 53,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4869166834,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.53.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 53,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1311771698,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.53.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1345228946,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.53.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4182945266,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.53.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1319003794,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.53.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2150661874,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.53.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 53,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1311771346,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.53.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3785251890,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.54.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 54,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1120759250,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.54.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 54,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1120749010,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.54.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1600806546,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.54.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4137725170,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.54.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1065035730,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.54.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 54,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4451023730,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.54.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 54,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1159307986,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.54.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 54,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4372549298,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.54.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3059477234,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.54.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1151914706,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.54.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2334209042,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.54.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4372549394,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.54.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 54,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1151899090,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.54.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1136170450,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.55.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4414968690,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.55.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 55,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5206839730,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.55.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 55,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5180588978,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.55.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1001138130,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.55.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 55,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1345218706,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.55.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1028007890,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.55.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 55,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4007148434,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.55.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1017194450,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.55.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3428765394,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.55.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 956573650,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.55.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 863512530,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.56.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 56,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1765629586,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.56.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 56,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 822788050,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.56.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 677904850,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.56.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1689280146,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.56.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 627769810,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.56.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 56,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3020999986,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.56.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 56,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 852289490,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.56.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 56,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4007246226,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.56.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 801440210,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.56.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 801624530,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.56.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 826075090,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.56.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 803782610,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.56.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 56,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3603664690,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.56.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 780141010,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.57.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 57,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 581594738,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.57.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 57,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 581584498,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.57.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3376991954,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.57.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2572192466,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.57.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 520308338,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.57.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 57,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 586848114,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.57.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 57,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 611959250,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.57.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 57,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4869151378,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.57.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2053780978,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.57.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 589492690,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.57.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1810204306,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.57.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 589677010,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.57.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 57,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 586847858,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.57.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4853422738,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.58.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 58,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2287796146,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.58.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 58,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 504950706,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.58.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 452849586,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.58.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 728039890,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.58.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 402714546,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.58.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 58,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 506927122,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.58.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 58,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 507119218,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.58.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 58,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 506927026,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.58.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 506927218,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.58.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3514305234,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.58.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 924788690,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.58.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4799347858,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.58.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 58,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3024932338,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.58.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1794475666,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.59.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1522777746,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.59.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 59,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 376171954,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.59.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 59,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 376161714,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.59.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 381415346,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.59.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 59,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 376151474,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.59.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3750492210,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.59.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 59,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1345218194,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.59.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 586848210,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.59.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 275881394,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.59.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 331586994,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.59.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2827986226,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.60.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 60,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1595205266,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.60.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 60,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3300916050,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.60.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 163607378,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.60.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1990538738,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.60.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3332084434,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.60.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 60,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 227832658,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.60.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 60,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4869256434,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.60.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 60,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3530218194,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.60.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 227848114,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.60.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4891969010,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.60.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 238525874,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.60.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 612041170,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.60.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 60,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4876895474,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.60.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 212104018,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.61.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 61,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5206850482,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.61.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 61,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 111495666,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.61.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 22366706,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.61.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 66931186,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.61.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4892161010,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.61.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 61,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4360729586,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.61.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 61,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4360729682,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.61.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 61,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 129200882,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.61.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2291083282,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.61.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2666910002,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.61.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1905246962,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.61.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 131167058,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.61.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 61,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 129200626,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.61.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 113471986,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.62.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 62,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2984958568,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.62.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 62,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2814818312,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.62.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2739451304,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.62.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2934823528,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.62.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2663913384,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.62.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 62,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2822572552,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.62.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 62,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4190993906,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.62.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 62,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2814817704,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.62.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2816802312,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.62.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2918910568,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.62.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2990882408,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.62.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2919094888,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.62.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 62,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2663913128,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.62.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2799089064,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.63.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	1024,
	960
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2654410408,
	"length_bytes": 3932160
	},
	{
	"name": "language_model.model.layers.63.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 63,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2641302696,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.63.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 63,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2641292456,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.63.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2717980584,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.63.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 63,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2609824936,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.63.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2609835176,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.63.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 63,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2814817800,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.63.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	1024,
	960
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2644743848,
	"length_bytes": 3932160
	},
	{
	"name": "language_model.model.layers.63.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2553945256,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.63.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2829781608,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.63.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 32,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2874346088,
	"length_bytes": 44564480
	}
	]
	}