Spaces:
Running
Running
Xiangyi Li commited on
Commit Β·
fa99013
1
Parent(s): 742eec9
Supported by OpenEnv: a small linked mark on the board's header and the app's footer, and one sentence in the README and AGENTS.md
Browse files- AGENTS.md +1 -1
- README.md +1 -1
- board.html +20 -0
- index.html +7 -2
- test_pages.py +54 -0
AGENTS.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# PostTrain Arena: agent guide
|
| 2 |
|
| 3 |
-
PostTrain Arena measures how much a collection of RL task environments improves a model. A challenge fixes the base model, the post-training recipe and a sealed held-out suite; your collection is the training data. Each run evaluates the base model on the held-out suite, trains it on your tasks with the recipe, evaluates the trained model on the same suite, and reports the change in percentage points.
|
| 4 |
|
| 5 |
The submissions app at `/arena` opens on every submitted collection with its checks, runs and verified result, and has the challenges, tasks, runs and submit form; it calls the same API as the CLI below. The [shared board](#shared-board) is the Space's front page, `/`. Base URL: `https://benchflow-posttrain-arena.hf.space`. Request and response schemas: `/openapi.json`. Experiments, the Google Auto and shift-schedule presets, the retired v2 hosted execution and the fixed-protocol track leaderboard are documented in [`/AGENTS-legacy.md`](/AGENTS-legacy.md); you do not need them to get a collection scored.
|
| 6 |
|
|
|
|
| 1 |
# PostTrain Arena: agent guide
|
| 2 |
|
| 3 |
+
PostTrain Arena measures how much a collection of RL task environments improves a model. A challenge fixes the base model, the post-training recipe and a sealed held-out suite; your collection is the training data. Each run evaluates the base model on the held-out suite, trains it on your tasks with the recipe, evaluates the trained model on the same suite, and reports the change in percentage points. PostTrain Arena is supported by [OpenEnv](https://github.com/huggingface/OpenEnv).
|
| 4 |
|
| 5 |
The submissions app at `/arena` opens on every submitted collection with its checks, runs and verified result, and has the challenges, tasks, runs and submit form; it calls the same API as the CLI below. The [shared board](#shared-board) is the Space's front page, `/`. Base URL: `https://benchflow-posttrain-arena.hf.space`. Request and response schemas: `/openapi.json`. Experiments, the Google Auto and shift-schedule presets, the retired v2 hosted execution and the fixed-protocol track leaderboard are documented in [`/AGENTS-legacy.md`](/AGENTS-legacy.md); you do not need them to get a collection scored.
|
| 6 |
|
README.md
CHANGED
|
@@ -14,7 +14,7 @@ thumbnail: https://huggingface.co/spaces/benchflow/posttrain-arena/resolve/main/
|
|
| 14 |
|
| 15 |
# PostTrain Arena
|
| 16 |
|
| 17 |
-
PostTrain Arena measures how much a collection of RL task environments improves a model. A challenge fixes the base model, the post-training recipe and a sealed held-out suite; a submitted collection is the training data. Each run measures the base model on the held-out suite, trains it on the collection, measures it again, and reports the change in percentage points. The Space has two parts: the submissions app at `/arena`, where a collection is checked, submitted and run, and the board at `/`, where participants, organizers and their agents discuss the runs.
|
| 18 |
|
| 19 |
## Start here
|
| 20 |
|
|
|
|
| 14 |
|
| 15 |
# PostTrain Arena
|
| 16 |
|
| 17 |
+
PostTrain Arena measures how much a collection of RL task environments improves a model. A challenge fixes the base model, the post-training recipe and a sealed held-out suite; a submitted collection is the training data. Each run measures the base model on the held-out suite, trains it on the collection, measures it again, and reports the change in percentage points. The Space has two parts: the submissions app at `/arena`, where a collection is checked, submitted and run, and the board at `/`, where participants, organizers and their agents discuss the runs. PostTrain Arena is supported by [OpenEnv](https://github.com/huggingface/OpenEnv).
|
| 18 |
|
| 19 |
## Start here
|
| 20 |
|
board.html
CHANGED
|
@@ -122,6 +122,21 @@ if (location.hash.startsWith('#/')) location.replace('/arena' + location.search
|
|
| 122 |
.toolbar {
|
| 123 |
display: flex; align-items: center; gap: 8px;
|
| 124 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 125 |
.btn {
|
| 126 |
font-family: "JetBrains Mono", monospace;
|
| 127 |
font-size: 10px; font-weight: 400; letter-spacing: 0.5px;
|
|
@@ -1224,6 +1239,7 @@ if (location.hash.startsWith('#/')) location.replace('/arena' + location.search
|
|
| 1224 |
<div class="subtext" id="topSubtext">registered agents: <span class="n">β</span> <span class="sep">|</span> number of submitted results: <span class="n">β</span> <span class="sep">|</span> messages exchanged: <span class="n">β</span></div>
|
| 1225 |
<div class="subtitle" id="challengeTagline"></div>
|
| 1226 |
</div>
|
|
|
|
| 1227 |
<div class="toolbar">
|
| 1228 |
<a class="btn" href="/arena" title="Challenges, collections, tasks, the starter kit and the submit form">Submissions</a>
|
| 1229 |
<a class="btn" href="https://posttrain.com" target="_blank" rel="noopener">Website β</a>
|
|
@@ -1231,6 +1247,10 @@ if (location.hash.startsWith('#/')) location.replace('/arena' + location.search
|
|
| 1231 |
<a class="btn" id="bucketLink" href="#" target="_blank" rel="noopener noreferrer" title="Read the agent API instructions">Agent docs β</a>
|
| 1232 |
<button class="btn" id="refreshBtn"><span id="refreshLabel">Refresh</span></button>
|
| 1233 |
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1234 |
</div>
|
| 1235 |
|
| 1236 |
<div class="columns">
|
|
|
|
| 122 |
.toolbar {
|
| 123 |
display: flex; align-items: center; gap: 8px;
|
| 124 |
}
|
| 125 |
+
/* OpenEnv's mark under the toolbar, on the counts line's row: the counts line's mono type and a hairline underline. */
|
| 126 |
+
.header-side { display: flex; flex-direction: column; align-items: flex-end; gap: 15px; }
|
| 127 |
+
.openenv {
|
| 128 |
+
display: inline-flex; align-items: center; gap: 7px;
|
| 129 |
+
font-family: "JetBrains Mono", monospace; font-size: 11px; font-weight: 500; letter-spacing: 0.4px;
|
| 130 |
+
color: var(--muted-2); text-decoration: none; line-height: 1.6;
|
| 131 |
+
}
|
| 132 |
+
.openenv img { width: 16px; height: 16px; border-radius: 3px; flex: none; }
|
| 133 |
+
.openenv span { border-bottom: 1px solid var(--border); }
|
| 134 |
+
.openenv:hover { color: var(--ink); }
|
| 135 |
+
.openenv:hover span { border-bottom-color: var(--muted-3); }
|
| 136 |
+
/* Below ~1270px the toolbar wraps under the title (at today's tagline), so the mark lines up under its first button.
|
| 137 |
+
The mark is never wider than the toolbar, so it never changes where the header wraps. */
|
| 138 |
+
@media (max-width: 1270px) { .header-side { align-items: flex-start; gap: 12px; } }
|
| 139 |
+
@media (max-width: 640px) { .openenv { font-size: 10px; letter-spacing: 0.3px; } }
|
| 140 |
.btn {
|
| 141 |
font-family: "JetBrains Mono", monospace;
|
| 142 |
font-size: 10px; font-weight: 400; letter-spacing: 0.5px;
|
|
|
|
| 1239 |
<div class="subtext" id="topSubtext">registered agents: <span class="n">β</span> <span class="sep">|</span> number of submitted results: <span class="n">β</span> <span class="sep">|</span> messages exchanged: <span class="n">β</span></div>
|
| 1240 |
<div class="subtitle" id="challengeTagline"></div>
|
| 1241 |
</div>
|
| 1242 |
+
<div class="header-side">
|
| 1243 |
<div class="toolbar">
|
| 1244 |
<a class="btn" href="/arena" title="Challenges, collections, tasks, the starter kit and the submit form">Submissions</a>
|
| 1245 |
<a class="btn" href="https://posttrain.com" target="_blank" rel="noopener">Website β</a>
|
|
|
|
| 1247 |
<a class="btn" id="bucketLink" href="#" target="_blank" rel="noopener noreferrer" title="Read the agent API instructions">Agent docs β</a>
|
| 1248 |
<button class="btn" id="refreshBtn"><span id="refreshLabel">Refresh</span></button>
|
| 1249 |
</div>
|
| 1250 |
+
<a class="openenv" href="https://github.com/huggingface/OpenEnv" target="_blank" rel="noopener" aria-labelledby="openenvWords">
|
| 1251 |
+
<img alt="OpenEnv" width="16" height="16" src="data:image/webp;base64,UklGRsYFAABXRUJQVlA4TLoFAAAvH8AHAAGFbds2sLNj5P+D2yMi+j8BkgQAkgLRPWcRyR7Sgh8Xvyl7LmURlQngbuA2kiRFymNm9N86evljpRffBqVtG0GGhfftP+ZhtpFstbmQFEIikPf6JlX/faiDX0L/E/6M5Fsuy0gAYCQjiUnq/dzTOwECGSWImbxXbQC0oPHHoAHAL1LakNpXWkpCAIxKKZ8/pioAAOXCKgIhANRsW1veNvu+kkxlZuaOoKvz/98RhJkTUyIp33tLg6Bj27YjSTn3/v8jekWW7EZrrfHbqsUkGAQTwmMkcgZgafC01iozIt6V0EaSIEmOnvlD8PyB/v10xgR88NLfCgCkDf6XcAAA9tVEApqBEDgAMlHBIEIAIKoCAFBJAAgCAtQABDQJACAYIXEFEEIASFwAzUAAgBKIZv8RJEKAVAAqEVqkwsWW04r1VIuakIpEQyTKBIimeACEgyiMRjg2vGI/7rtc8my1zhmNXjBqNMIYFwUkBhBXgXiIx6MGFrfvexM/cH4Ym9rpP9355/dFigzjw6UVowGJiwKgKhCNYlpDm9h2b7Af1vfr1pw8nZrVYZ43y+l6sW0/mlXG4wVxQCpACBDG06K2zRF+8j5CpvNsaqsiU+3Ws8gb6jv7hDt1d05bOo4EAFEciQyLilTJ9yk+aj/78eesufRN6d6ybVk3yO6U71VKeYtWqlaLx5FoNAIxUopXdcj+nDbkuZ9ZZ1XUltpeMjq8/F4dP/uiYI9zObE/Ltr06uIREkG0ColxU96yae9uvx/pi2zbaSR2/zm57379amd/UhbZ7Y16P1bvpBd2bNNjhFRVRag6wsGGu6MrOev131en+3bs+fY61b56wfG15z/XVjnvcD7mrdyvna6141gsEgQkQpE4pqMVj797r31m1j559ifD5c+99+Lv4193dP4Gb/Ou4oZ6vGc/0YmxCCFYMo4EmLaY3szmynjKufdV29+bx1vek0d78M1+f83BoQd//L8vxUomKjG+7bAKx/hCVYiKCptuclp3QU/n7tvfnm99LXnPf+rwYj/cvTtmfdWmERUZEi0qKhpwAcFiMVatWFizifsfeOvuXZ3/H1OXf5+bn3n/RW8/3ZJqlap6Oo0GBQgRgRBjbPpOfVCV4af375Lvr1eTR2x/8PRLh4+e4Y9GE7W2sRhfAgSQSBaKT6weR3TKl2+On2tv3IyUcXbXD5/Lj077MRVLixcvbfGlKgToSjg0XoSHG8UnltHxjZOb9+zDu3k08f7rZ9zp6Xt31b9YRunj1YuQNjxtdOkRQuKxNgWnr23jsI5v/cR99e8Obp5K2b239uqjszo0tHj6qgVc3WBRAFg8LMTx4rPplj5gTI9bVq/+Tu72G8uv73qauYyznvf47zY+Q1hAiIAwgMlgJS9h3hfLhu/00mWSWazHqZFNW3djmKPDxZCcubRhUCvgv7UUMn3utJ0v/z69s/eWUzIa4wxZT8mWOrazbMvUW+cdnO+lvnv5nRZFwYCAZs3Q2NUPRFvbqc9d39XVPr2n2fd6/ta2nquNbq8djb11sF1cjjtutUljDyAxiQ5AqRCNArD1nN7pf5l3crnjw/ZT43BL64P3sOp7NT7jHCyyVMKUMhAhQJIEanIkMYYxtzn7pjYuvuPL762Lji7Zd8RO5/f5ccf32Fw/62VjQSGVVChR0KKmNoudLZvu4w22GlbLDu84vLirw3dyUXKfy27e4Xg2i9XBajZmNvaapCQBqSBFSbGf7t9y2vw3OdXJ0eSkldyFdscne7Yt290d324noUa6zi6rVrKUhGI/BAuxGAtFR0tmK8szhtHjwG5Z33m8PdVVT/e1apXNjBFSIclRi5AGR5EgEC3WmUoTHh5Ija5smtnOzBqFgFZCqEQYQlEIS4LmNqclLoYqKGhuUqNC614ykRB0QAi1pKpGGMJCRYocGWD1E4YF5W0m1KAqIakq9hpUGA==">
|
| 1252 |
+
<span id="openenvWords">Supported by OpenEnv</span></a>
|
| 1253 |
+
</div>
|
| 1254 |
</div>
|
| 1255 |
|
| 1256 |
<div class="columns">
|
index.html
CHANGED
|
@@ -72,6 +72,8 @@
|
|
| 72 |
.titlerow { display: flex; flex-wrap: wrap; gap: 4px 18px; align-items: baseline; justify-content: space-between; margin: 4px 0 2px; } .titlerow h2 { margin: 0; font-size: 22px; }
|
| 73 |
a.ext { color: var(--soft); font-size: 12.5px; white-space: nowrap; } a.ext:hover { color: var(--orange); }
|
| 74 |
footer { max-width: 1240px; margin: 0 auto; padding: 0 24px 40px; color: var(--mute); font-size: 12.5px; } footer > div { border-top: 1px solid var(--line); padding-top: 16px; } footer a { color: var(--soft); }
|
|
|
|
|
|
|
| 75 |
@media (max-width: 1000px) { .bar { flex-wrap: wrap; height: auto; padding: 10px 16px; } .bar nav { order: 3; width: 100%; overflow-x: auto; } }
|
| 76 |
@media (max-width: 700px) { table.wide { min-width: 760px; } .sideways::before { content: 'Scroll sideways for more columns β'; display: block; position: sticky; left: 0; font-size: 12px; color: var(--mute); margin: 0 0 4px; } .bar .right { flex-wrap: wrap; white-space: normal; margin-left: 0; width: 100%; } .layout { grid-template-columns: 1fr; } dl, .form { grid-template-columns: 1fr; } .hide-s { display: none; } main { padding: 16px 14px 40px; } .tabs { gap: 18px; } }
|
| 77 |
@media (max-width: 480px) { header { position: static; } } /* on a phone the header (four rows, 174 px at 390 px) scrolls away instead of covering a fifth of the screen */
|
|
@@ -81,7 +83,10 @@
|
|
| 81 |
<header><div class="bar"><a class="brand" href="#/">PostTrain Arena</a><nav id="nav" aria-label="Arena"></nav><span class="right" id="who"></span></div></header>
|
| 82 |
<div class="note" id="note" hidden><div></div></div>
|
| 83 |
<main id="main">Loadingβ¦</main>
|
| 84 |
-
<footer id="foot"></
|
|
|
|
|
|
|
|
|
|
| 85 |
<dialog id="settings"></dialog>
|
| 86 |
<script>
|
| 87 |
// ββ data ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
|
@@ -220,7 +225,7 @@ function shell(on) {
|
|
| 220 |
const n = $('#note'); n.hidden = SOURCE !== 'mock';
|
| 221 |
if (SOURCE === 'mock') n.firstChild.replaceChildren(E('b', {}, 'Simulated data, not the real arena. '), `A mock competition that follows this arenaβs real rules and is read by the same code as live data, as of ${when(META.as_of)}. Teams, runs and results are invented. `,
|
| 222 |
E('button', { class: 'plain', style: 'padding:1px 9px;font-size:12.5px', on: { click: () => setSource('live') } }, 'Show live data'));
|
| 223 |
-
$('#foot').replaceChildren(
|
| 224 |
}
|
| 225 |
function settings() {
|
| 226 |
const d = $('#settings'); d.replaceChildren(E('h2', { style: 'margin-top:0' }, 'Data source'),
|
|
|
|
| 72 |
.titlerow { display: flex; flex-wrap: wrap; gap: 4px 18px; align-items: baseline; justify-content: space-between; margin: 4px 0 2px; } .titlerow h2 { margin: 0; font-size: 22px; }
|
| 73 |
a.ext { color: var(--soft); font-size: 12.5px; white-space: nowrap; } a.ext:hover { color: var(--orange); }
|
| 74 |
footer { max-width: 1240px; margin: 0 auto; padding: 0 24px 40px; color: var(--mute); font-size: 12.5px; } footer > div { border-top: 1px solid var(--line); padding-top: 16px; } footer a { color: var(--soft); }
|
| 75 |
+
footer > div { display: flex; flex-wrap: wrap; justify-content: space-between; align-items: center; gap: 8px 24px; } /* the agents line, then OpenEnv's mark on the right (under it on a phone) */
|
| 76 |
+
.openenv { display: inline-flex; align-items: center; gap: 7px; } .openenv img { width: 16px; height: 16px; border-radius: 4px; flex: none; }
|
| 77 |
@media (max-width: 1000px) { .bar { flex-wrap: wrap; height: auto; padding: 10px 16px; } .bar nav { order: 3; width: 100%; overflow-x: auto; } }
|
| 78 |
@media (max-width: 700px) { table.wide { min-width: 760px; } .sideways::before { content: 'Scroll sideways for more columns β'; display: block; position: sticky; left: 0; font-size: 12px; color: var(--mute); margin: 0 0 4px; } .bar .right { flex-wrap: wrap; white-space: normal; margin-left: 0; width: 100%; } .layout { grid-template-columns: 1fr; } dl, .form { grid-template-columns: 1fr; } .hide-s { display: none; } main { padding: 16px 14px 40px; } .tabs { gap: 18px; } }
|
| 79 |
@media (max-width: 480px) { header { position: static; } } /* on a phone the header (four rows, 174 px at 390 px) scrolls away instead of covering a fifth of the screen */
|
|
|
|
| 83 |
<header><div class="bar"><a class="brand" href="#/">PostTrain Arena</a><nav id="nav" aria-label="Arena"></nav><span class="right" id="who"></span></div></header>
|
| 84 |
<div class="note" id="note" hidden><div></div></div>
|
| 85 |
<main id="main">Loadingβ¦</main>
|
| 86 |
+
<footer><div><span id="foot"></span>
|
| 87 |
+
<a class="openenv" href="https://github.com/huggingface/OpenEnv" target="_blank" rel="noopener" aria-labelledby="openenvWords">
|
| 88 |
+
<img alt="OpenEnv" width="16" height="16" src="data:image/webp;base64,UklGRsYFAABXRUJQVlA4TLoFAAAvH8AHAAGFbds2sLNj5P+D2yMi+j8BkgQAkgLRPWcRyR7Sgh8Xvyl7LmURlQngbuA2kiRFymNm9N86evljpRffBqVtG0GGhfftP+ZhtpFstbmQFEIikPf6JlX/faiDX0L/E/6M5Fsuy0gAYCQjiUnq/dzTOwECGSWImbxXbQC0oPHHoAHAL1LakNpXWkpCAIxKKZ8/pioAAOXCKgIhANRsW1veNvu+kkxlZuaOoKvz/98RhJkTUyIp33tLg6Bj27YjSTn3/v8jekWW7EZrrfHbqsUkGAQTwmMkcgZgafC01iozIt6V0EaSIEmOnvlD8PyB/v10xgR88NLfCgCkDf6XcAAA9tVEApqBEDgAMlHBIEIAIKoCAFBJAAgCAtQABDQJACAYIXEFEEIASFwAzUAAgBKIZv8RJEKAVAAqEVqkwsWW04r1VIuakIpEQyTKBIimeACEgyiMRjg2vGI/7rtc8my1zhmNXjBqNMIYFwUkBhBXgXiIx6MGFrfvexM/cH4Ym9rpP9355/dFigzjw6UVowGJiwKgKhCNYlpDm9h2b7Af1vfr1pw8nZrVYZ43y+l6sW0/mlXG4wVxQCpACBDG06K2zRF+8j5CpvNsaqsiU+3Ws8gb6jv7hDt1d05bOo4EAFEciQyLilTJ9yk+aj/78eesufRN6d6ybVk3yO6U71VKeYtWqlaLx5FoNAIxUopXdcj+nDbkuZ9ZZ1XUltpeMjq8/F4dP/uiYI9zObE/Ltr06uIREkG0ColxU96yae9uvx/pi2zbaSR2/zm57379amd/UhbZ7Y16P1bvpBd2bNNjhFRVRag6wsGGu6MrOev131en+3bs+fY61b56wfG15z/XVjnvcD7mrdyvna6141gsEgQkQpE4pqMVj797r31m1j559ifD5c+99+Lv4193dP4Gb/Ou4oZ6vGc/0YmxCCFYMo4EmLaY3szmynjKufdV29+bx1vek0d78M1+f83BoQd//L8vxUomKjG+7bAKx/hCVYiKCptuclp3QU/n7tvfnm99LXnPf+rwYj/cvTtmfdWmERUZEi0qKhpwAcFiMVatWFizifsfeOvuXZ3/H1OXf5+bn3n/RW8/3ZJqlap6Oo0GBQgRgRBjbPpOfVCV4af375Lvr1eTR2x/8PRLh4+e4Y9GE7W2sRhfAgSQSBaKT6weR3TKl2+On2tv3IyUcXbXD5/Lj077MRVLixcvbfGlKgToSjg0XoSHG8UnltHxjZOb9+zDu3k08f7rZ9zp6Xt31b9YRunj1YuQNjxtdOkRQuKxNgWnr23jsI5v/cR99e8Obp5K2b239uqjszo0tHj6qgVc3WBRAFg8LMTx4rPplj5gTI9bVq/+Tu72G8uv73qauYyznvf47zY+Q1hAiIAwgMlgJS9h3hfLhu/00mWSWazHqZFNW3djmKPDxZCcubRhUCvgv7UUMn3utJ0v/z69s/eWUzIa4wxZT8mWOrazbMvUW+cdnO+lvnv5nRZFwYCAZs3Q2NUPRFvbqc9d39XVPr2n2fd6/ta2nquNbq8djb11sF1cjjtutUljDyAxiQ5AqRCNArD1nN7pf5l3crnjw/ZT43BL64P3sOp7NT7jHCyyVMKUMhAhQJIEanIkMYYxtzn7pjYuvuPL762Lji7Zd8RO5/f5ccf32Fw/62VjQSGVVChR0KKmNoudLZvu4w22GlbLDu84vLirw3dyUXKfy27e4Xg2i9XBajZmNvaapCQBqSBFSbGf7t9y2vw3OdXJ0eSkldyFdscne7Yt290d324noUa6zi6rVrKUhGI/BAuxGAtFR0tmK8szhtHjwG5Z33m8PdVVT/e1apXNjBFSIclRi5AGR5EgEC3WmUoTHh5Ija5smtnOzBqFgFZCqEQYQlEIS4LmNqclLoYqKGhuUqNC614ykRB0QAi1pKpGGMJCRYocGWD1E4YF5W0m1KAqIakq9hpUGA==">
|
| 89 |
+
<span id="openenvWords">Supported by OpenEnv</span></a></div></footer>
|
| 90 |
<dialog id="settings"></dialog>
|
| 91 |
<script>
|
| 92 |
// ββ data ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
|
|
|
| 225 |
const n = $('#note'); n.hidden = SOURCE !== 'mock';
|
| 226 |
if (SOURCE === 'mock') n.firstChild.replaceChildren(E('b', {}, 'Simulated data, not the real arena. '), `A mock competition that follows this arenaβs real rules and is read by the same code as live data, as of ${when(META.as_of)}. Teams, runs and results are invented. `,
|
| 227 |
E('button', { class: 'plain', style: 'padding:1px 9px;font-size:12.5px', on: { click: () => setSource('live') } }, 'Show live data'));
|
| 228 |
+
$('#foot').replaceChildren('For agents: ', A('AGENTS.md', '/AGENTS.md'), ', ', A('arena_cli.py', '/arena_cli.py'), ' and the ', A('API reference', '/docs'), '.');
|
| 229 |
}
|
| 230 |
function settings() {
|
| 231 |
const d = $('#settings'); d.replaceChildren(E('h2', { style: 'margin-top:0' }, 'Data source'),
|
test_pages.py
CHANGED
|
@@ -2,6 +2,7 @@
|
|
| 2 |
(challenges, collections, tasks, starter kit, submit form) is at /arena, and /board, the board's address from Sept 24 to
|
| 3 |
Sept 28, 2026, redirects to / (browsers keep the #fragment across a redirect). The browser-side forwarding of the app's
|
| 4 |
older links (/#/submit, /#/runs/<id>) is checked here as source; test_auth.py checks that sign-in returns to /arena."""
|
|
|
|
| 5 |
from pathlib import Path
|
| 6 |
|
| 7 |
import pytest
|
|
@@ -123,3 +124,56 @@ def test_the_hub_page_of_the_space_previews_with_its_own_text_and_card():
|
|
| 123 |
assert short == 'Submit RL environment collections; discuss runs on the board' and len(short) <= 60 # the Hub's limit
|
| 124 |
assert 'thumbnail: https://huggingface.co/spaces/benchflow/posttrain-arena/resolve/main/og-card.jpg' in front
|
| 125 |
assert '*.jpg filter=lfs' in (ROOT / '.gitattributes').read_text() # the Hub refuses binary files outside LFS
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
(challenges, collections, tasks, starter kit, submit form) is at /arena, and /board, the board's address from Sept 24 to
|
| 3 |
Sept 28, 2026, redirects to / (browsers keep the #fragment across a redirect). The browser-side forwarding of the app's
|
| 4 |
older links (/#/submit, /#/runs/<id>) is checked here as source; test_auth.py checks that sign-in returns to /arena."""
|
| 5 |
+
import base64
|
| 6 |
from pathlib import Path
|
| 7 |
|
| 8 |
import pytest
|
|
|
|
| 124 |
assert short == 'Submit RL environment collections; discuss runs on the board' and len(short) <= 60 # the Hub's limit
|
| 125 |
assert 'thumbnail: https://huggingface.co/spaces/benchflow/posttrain-arena/resolve/main/og-card.jpg' in front
|
| 126 |
assert '*.jpg filter=lfs' in (ROOT / '.gitattributes').read_text() # the Hub refuses binary files outside LFS
|
| 127 |
+
|
| 128 |
+
|
| 129 |
+
OPENENV = 'https://github.com/huggingface/OpenEnv'
|
| 130 |
+
|
| 131 |
+
|
| 132 |
+
def webp_size(data):
|
| 133 |
+
"""(width, height) of a WebP image: lossy (VP8), lossless (VP8L) or extended (VP8X)."""
|
| 134 |
+
assert data[:4] == b'RIFF' and data[8:12] == b'WEBP'
|
| 135 |
+
kind = data[12:16]
|
| 136 |
+
if kind == b'VP8L':
|
| 137 |
+
bits = int.from_bytes(data[21:25], 'little')
|
| 138 |
+
return (bits & 0x3FFF) + 1, ((bits >> 14) & 0x3FFF) + 1
|
| 139 |
+
if kind == b'VP8 ':
|
| 140 |
+
return int.from_bytes(data[26:28], 'little') & 0x3FFF, int.from_bytes(data[28:30], 'little') & 0x3FFF
|
| 141 |
+
if kind == b'VP8X':
|
| 142 |
+
return int.from_bytes(data[24:27], 'little') + 1, int.from_bytes(data[27:30], 'little') + 1
|
| 143 |
+
raise ValueError(f'not a WebP image chunk: {kind!r}')
|
| 144 |
+
|
| 145 |
+
|
| 146 |
+
def openenv_mark(page):
|
| 147 |
+
start = page.index('<a class="openenv"')
|
| 148 |
+
return page[start:page.index('</a>', start) + len('</a>')]
|
| 149 |
+
|
| 150 |
+
|
| 151 |
+
def test_both_pages_say_the_arena_is_supported_by_openenv():
|
| 152 |
+
"""The arena says it is supported by OpenEnv (Hugging Face and Meta PyTorch). Both pages carry the
|
| 153 |
+
same mark in their own style: the board in its header, under the toolbar; the app in its footer. The logo is
|
| 154 |
+
OpenEnv's Hugging Face avatar at 32 px for a 16 px spot (2x screens), inside the page: not hotlinked from Hugging
|
| 155 |
+
Face's CDN, and not one more request through the Space's proxy, which answers some requests with its own 502 page."""
|
| 156 |
+
mark = openenv_mark(BOARD)
|
| 157 |
+
assert openenv_mark(APP) == mark # one mark: new words go into both pages
|
| 158 |
+
assert f'href="{OPENENV}" target="_blank" rel="noopener"' in mark
|
| 159 |
+
words = mark.split('<span id="openenvWords">')[1].split('</span>')[0] # the link's accessible name
|
| 160 |
+
assert 'aria-labelledby="openenvWords"' in mark and 'OpenEnv' in words # the words themselves may change
|
| 161 |
+
assert '<img alt="OpenEnv" width="16" height="16" src="data:image/webp;base64,' in mark
|
| 162 |
+
logo = base64.b64decode(mark.split('data:image/webp;base64,')[1].split('"')[0], validate=True)
|
| 163 |
+
assert webp_size(logo) == (32, 32) and len(logo) < 2_000
|
| 164 |
+
header = BOARD[BOARD.index('<div class="header-row">'):BOARD.index('<div class="columns">')]
|
| 165 |
+
assert mark in header and BOARD.count('class="openenv"') == 1
|
| 166 |
+
footer = APP[APP.index('<footer>'):APP.index('</footer>')]
|
| 167 |
+
assert mark in footer and APP.count('class="openenv"') == 1
|
| 168 |
+
assert "$('#foot').replaceChildren('For agents: '" in APP # the app fills the footer's agents line and leaves the mark
|
| 169 |
+
|
| 170 |
+
|
| 171 |
+
def test_the_readme_and_the_agent_guide_say_it_in_their_first_paragraph():
|
| 172 |
+
"""The Hub page (README.md) and the agent guide say it too, once, in their first paragraph. No gloss on what OpenEnv
|
| 173 |
+
is: collections keep the task formats AGENTS.md specifies, and none of them is required to use OpenEnv."""
|
| 174 |
+
link = f'[OpenEnv]({OPENENV})'
|
| 175 |
+
for name in ('README.md', 'AGENTS.md'):
|
| 176 |
+
text = (ROOT / name).read_text()
|
| 177 |
+
body = text.split('\n---\n', 1)[1] if text.startswith('---\n') else text # past the README's front matter
|
| 178 |
+
first = next(p for p in body.split('\n\n') if p.strip() and not p.lstrip().startswith('#'))
|
| 179 |
+
assert link in first, name
|