Spaces:
Running
Running
Commit History
-
Add routing debug log to trace provider resolution.β
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>65739aa
-
Add Cerebras and Silicon Flow provider supportβ
- Cerebras: qwen/qwen-3-235b-a22b-instruct-2507 (Qwen 3 235B) - Silicon Flow: Qwen3.6-35B/27B, Qwen3.5-35B/27B (VLM multimodal), Gemma-4-26B/31B reasoning models - Both use OpenAI-compatible /chat/completions via OpenAIChatTransport - API keys: CEREBRAS_API_KEY, SILICON_API_KEY Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>43ea069 -
Extend session visibility in admin dashboard with configurable retentionβ
- Add retention_seconds param to SessionTracker (default 30min) - Sessions now stay visible 30min after last request instead of 2min - Add SESSION_RETENTION_MINUTES env var for configuration - Show Active/Idle status in admin UI (<5min vs >5min idle) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>6339a53
-
Optimize auto routing: Zen unlimited, smarter fallback skippingβ
- Zen provider gets 9999 req/min scoped limiter (no rate limit wait) - Silent skip for blocked NIM providers (no failure penalty) - Zen blocked check uses debug instead of warning level - Minor ruff fixes (quoted types, ternary style) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>9358a6f -
Track sessions via X-Session-ID header for accurate admin dashboardβ
Claude Code --session-id <uuid> sends X-Session-ID on every request. Use this header as the session identifier instead of inferring from client IP (gateway_<ip>), which is unreliable when multiple sessions share an IP. Fall back to gateway IP for non-session-ID requests. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>948c8f9 -
Speed up NIM provider with failure tracking and faster timeoutsβ
- Add ModelHealthTracker for per-model failure tracking (30s TTL) - Faster timeouts: connect=10s, first_chunk=30s, fallback=20s - Auto model pre-checks health before attempting requests - Record failures on timeout/rate-limit for smart fallback - Skip unhealthy models in both router and services Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>a5ea640
-
Speed optimizations and enhanced auto-model fallback routingβ
- Connection pool tuning: increased max_connections (100β200), max_keepalive (20β50), reduced keepalive_expiry (30sβ15s) - NIM concurrency: increased max_concurrency for better throughput - Faster timeouts: connect (30β15s), first_chunk (60β45s), fallback (60β30s) - Retry backoff: reduced base_delay (1β0.5s), max_delay (60β30s) - New trivial request fast-path for hi/ok/status checks - Auto model now routes to all 7 NIM models by default - Pre-flight rate limit check before trying each candidate - Timeout/transient errors trigger automatic fallback to next model - Import cleanup for httpx/asyncio at module level Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>ebba9d6 -
Track sessions by gateway client IPβ
- Add _get_client_ip() to detect gateway/proxy requests - Track sessions only when X-Forwarded-For, X-Real-IP, X-Client-IP, or Via headers present - Each unique client IP behind a gateway = one session - Direct connections are not tracked Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>f56589d -
Fix: use sync version of track_request to avoid SyntaxErrorβ
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>b5bd2a8 -
Fix: sessions tracking now works correctly in dashboard and JSONβ
- Added `track_request` calls in services.py for both single-model and fallback paths - Sessions now show in `/admin` and `/api/admin/sessions` - Added `/` root endpoint to show active sessions count Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>ef123a8 -
Performance optimizations for proxy speed and shared sessionsβ
- Per-provider rate limiting: Zen gets unlimited, NVIDIA NIM gets 40/min - Higher Zen concurrency: 4x max_concurrency for fast minimax model - Connection pool tuning: Keepalive connections for faster reuse - Session tracker: Fair resource sharing across Claude Code instances - Smart auto-routing: Prioritize Zen (no limits), then by load Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>cc3287d -
Deploy claude-code-nvidia proxy to Hugging Face Spacesβ
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>0157ac7