Linker1907 Claude Sonnet 4.6 commited on
Commit
4f8462a
·
1 Parent(s): e7c9085

Prepare for HF Space deployment

Browse files

Update demo.launch() to use Space defaults (no hardcoded port),
add Space README with YAML front matter.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Files changed (2) hide show
  1. README.md +27 -109
  2. app.py +1 -1
README.md CHANGED
@@ -1,118 +1,36 @@
1
- # HF Leaderboard Watcher — daily email digest
 
 
 
 
 
 
 
 
 
 
2
 
3
- Polls official benchmark leaderboards on the [Hugging Face Hub](https://huggingface.co/docs/hub/leaderboard-data-guide),
4
- diffs against local state, and emails you a digest of new model entries.
5
 
6
- Stdlib only and packaged as a [uv single-file script](https://docs.astral.sh/uv/guides/scripts/) (PEP 723 inline metadata). uv handles the Python interpreter — no virtualenv, no `pip install`.
7
 
8
- ## Files
9
 
10
- - `watch.py` the watcher.
11
- - `config.json` created on first run; fill in your SMTP details.
12
- - `state.json` auto; tracks already-seen `(benchmark, model)` pairs.
13
- - `watcher.log` every run appends here.
14
- - `cron.log` captured stdout/stderr from cron runs.
 
15
 
16
- ## 1. Configure SMTP
17
 
18
- Edit `config.json` (or set env vars in your crontab see below):
 
19
 
20
- ```json
21
- {
22
- "smtp_host": "smtp.gmail.com",
23
- "smtp_port": 465,
24
- "smtp_user": "you@gmail.com",
25
- "smtp_password": "<app password>",
26
- "email_from": "you@gmail.com",
27
- "email_to": "you@gmail.com"
28
- }
29
- ```
30
-
31
- For **Gmail**, generate an App Password at
32
- <https://myaccount.google.com/apppasswords> (requires 2FA).
33
- Other SMTP providers (Fastmail, Proton Bridge, SES, Mailgun, etc.) work
34
- the same way — port 465 = SMTPS, anything else = STARTTLS.
35
-
36
- Test it:
37
-
38
- ```sh
39
- ~/leaderboard-watcher/watch.py --test-email
40
- # or explicitly:
41
- uv run --script ~/leaderboard-watcher/watch.py --test-email
42
- ```
43
-
44
- ## 2. Cron job (already installed)
45
-
46
- ```
47
- 0 9 * * * cd /Users/nathan/leaderboard-watcher && /Users/nathan/.cargo/bin/uv run --script watch.py >> cron.log 2>&1
48
- ```
49
-
50
- This runs every day at **09:00** local time.
51
-
52
- > **macOS gotcha**: `cron` needs **Full Disk Access**. Go to System
53
- > Settings → Privacy & Security → Full Disk Access → click `+` → press
54
- > ⌘⇧G → enter `/usr/sbin/cron` → enable. (You already have another
55
- > cron job running, so this is probably already done.)
56
-
57
- Useful commands:
58
-
59
- ```sh
60
- crontab -l # see installed jobs
61
- crontab -e # edit
62
- tail -f ~/leaderboard-watcher/watcher.log # tail the run log
63
- tail -f ~/leaderboard-watcher/cron.log # tail cron's stdout/stderr
64
- ```
65
-
66
- To change cadence, edit the `0 9 * * *` part:
67
- - `0 9 * * 1-5` = weekdays at 9am
68
- - `0 9,18 * * *` = 9am and 6pm
69
- - `*/30 * * * *` = every 30 minutes
70
-
71
- ## 3. CLI flags
72
-
73
- The shebang line (`#!/usr/bin/env -S uv run --script`) makes the file directly executable:
74
-
75
- ```sh
76
- ./watch.py # run one cycle (what cron calls)
77
- ./watch.py --list # print all official benchmark IDs
78
- ./watch.py --reset # delete state.json (next run reseeds silently)
79
- ./watch.py --test-email # send a test email and exit
80
- ```
81
-
82
- Equivalent explicit form: `uv run --script watch.py [...]`.
83
-
84
- ## How it works
85
-
86
- 1. `GET /api/datasets?filter=benchmark:official` to discover ~22 benchmarks.
87
- (Override with `"benchmarks": ["SWE-bench/SWE-bench_Verified", "cais/hle"]` in `config.json` to watch a fixed list.)
88
- 2. `GET /api/datasets/{id}/leaderboard` per benchmark.
89
- 3. New `modelId`s relative to `state.json` are collected.
90
- 4. If there are any, an HTML+text digest email is sent grouped by benchmark
91
- with rank, score, verified flag, and links to each model card.
92
- 5. State is updated so each new model is reported only once.
93
-
94
- The first run is **silent** (it just seeds state). Set
95
- `silent_first_run: false` if you want to receive the entire current state
96
- on first run instead.
97
-
98
- By default no email is sent on days with zero new entries
99
- (`skip_email_when_empty: true`).
100
-
101
- ## Gated benchmarks
102
-
103
- `cais/hle` and similar gated datasets need an HF token. The script
104
- auto-uses your `~/.cache/huggingface/token` (created by `hf auth login`),
105
- or you can set `HF_TOKEN` in the environment. **Cron has no access to
106
- your shell environment**, so if you rely on env vars, set them inside the
107
- crontab line itself, e.g.:
108
-
109
- ```cron
110
- 0 9 * * * SMTP_PASSWORD=xxxxx HF_TOKEN=hf_yyyy cd /Users/nathan/leaderboard-watcher && /Users/nathan/.cargo/bin/uv run --script watch.py >> cron.log 2>&1
111
- ```
112
-
113
- ## Uninstall
114
 
115
- ```sh
116
- crontab -l | grep -v leaderboard-watcher | crontab -
117
- rm -rf ~/leaderboard-watcher
118
  ```
 
1
+ ---
2
+ title: HF Hub Benchmark Dashboard
3
+ emoji: 🏆
4
+ colorFrom: purple
5
+ colorTo: indigo
6
+ sdk: gradio
7
+ sdk_version: 5.0.0
8
+ app_file: app.py
9
+ pinned: false
10
+ short_description: Live dashboard for all official benchmark leaderboards on the HF Hub
11
+ ---
12
 
13
+ # HF Hub Benchmark Dashboard
 
14
 
15
+ Live dashboard for every official benchmark leaderboard published on the Hugging Face Hub.
16
 
17
+ ## Features
18
 
19
+ - Sidebar navigation by category (Knowledge, Math, Code, Agents, Vision, …)
20
+ - Per-benchmark leaderboard with rank medals, model links, and provider chips
21
+ - Inference provider enrichment from [router.huggingface.co](https://router.huggingface.co): price/1M tokens (in & out), TTFT, throughput, context length
22
+ - License and parameter count per model (from the HF models API)
23
+ - Filter to only show models with live inference providers
24
+ - 6-hour cache with manual refresh
25
 
26
+ ## HF Token
27
 
28
+ Set `HF_TOKEN` as a Space secret to access gated benchmarks (e.g. HLE).
29
+ Public benchmarks work without a token.
30
 
31
+ ## Local run
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
 
33
+ ```bash
34
+ pip install gradio>=5.0.0
35
+ python app.py
36
  ```
app.py CHANGED
@@ -845,4 +845,4 @@ def build_app() -> gr.Blocks:
845
 
846
  if __name__ == "__main__":
847
  demo = build_app()
848
- demo.launch(server_name="0.0.0.0", server_port=7861)
 
845
 
846
  if __name__ == "__main__":
847
  demo = build_app()
848
+ demo.launch()