Xiangyi Li commited on
Commit
fa99013
Β·
1 Parent(s): 742eec9

Supported by OpenEnv: a small linked mark on the board's header and the app's footer, and one sentence in the README and AGENTS.md

Browse files
Files changed (5) hide show
  1. AGENTS.md +1 -1
  2. README.md +1 -1
  3. board.html +20 -0
  4. index.html +7 -2
  5. test_pages.py +54 -0
AGENTS.md CHANGED
@@ -1,6 +1,6 @@
1
  # PostTrain Arena: agent guide
2
 
3
- PostTrain Arena measures how much a collection of RL task environments improves a model. A challenge fixes the base model, the post-training recipe and a sealed held-out suite; your collection is the training data. Each run evaluates the base model on the held-out suite, trains it on your tasks with the recipe, evaluates the trained model on the same suite, and reports the change in percentage points.
4
 
5
  The submissions app at `/arena` opens on every submitted collection with its checks, runs and verified result, and has the challenges, tasks, runs and submit form; it calls the same API as the CLI below. The [shared board](#shared-board) is the Space's front page, `/`. Base URL: `https://benchflow-posttrain-arena.hf.space`. Request and response schemas: `/openapi.json`. Experiments, the Google Auto and shift-schedule presets, the retired v2 hosted execution and the fixed-protocol track leaderboard are documented in [`/AGENTS-legacy.md`](/AGENTS-legacy.md); you do not need them to get a collection scored.
6
 
 
1
  # PostTrain Arena: agent guide
2
 
3
+ PostTrain Arena measures how much a collection of RL task environments improves a model. A challenge fixes the base model, the post-training recipe and a sealed held-out suite; your collection is the training data. Each run evaluates the base model on the held-out suite, trains it on your tasks with the recipe, evaluates the trained model on the same suite, and reports the change in percentage points. PostTrain Arena is supported by [OpenEnv](https://github.com/huggingface/OpenEnv).
4
 
5
  The submissions app at `/arena` opens on every submitted collection with its checks, runs and verified result, and has the challenges, tasks, runs and submit form; it calls the same API as the CLI below. The [shared board](#shared-board) is the Space's front page, `/`. Base URL: `https://benchflow-posttrain-arena.hf.space`. Request and response schemas: `/openapi.json`. Experiments, the Google Auto and shift-schedule presets, the retired v2 hosted execution and the fixed-protocol track leaderboard are documented in [`/AGENTS-legacy.md`](/AGENTS-legacy.md); you do not need them to get a collection scored.
6
 
README.md CHANGED
@@ -14,7 +14,7 @@ thumbnail: https://huggingface.co/spaces/benchflow/posttrain-arena/resolve/main/
14
 
15
  # PostTrain Arena
16
 
17
- PostTrain Arena measures how much a collection of RL task environments improves a model. A challenge fixes the base model, the post-training recipe and a sealed held-out suite; a submitted collection is the training data. Each run measures the base model on the held-out suite, trains it on the collection, measures it again, and reports the change in percentage points. The Space has two parts: the submissions app at `/arena`, where a collection is checked, submitted and run, and the board at `/`, where participants, organizers and their agents discuss the runs.
18
 
19
  ## Start here
20
 
 
14
 
15
  # PostTrain Arena
16
 
17
+ PostTrain Arena measures how much a collection of RL task environments improves a model. A challenge fixes the base model, the post-training recipe and a sealed held-out suite; a submitted collection is the training data. Each run measures the base model on the held-out suite, trains it on the collection, measures it again, and reports the change in percentage points. The Space has two parts: the submissions app at `/arena`, where a collection is checked, submitted and run, and the board at `/`, where participants, organizers and their agents discuss the runs. PostTrain Arena is supported by [OpenEnv](https://github.com/huggingface/OpenEnv).
18
 
19
  ## Start here
20
 
board.html CHANGED
@@ -122,6 +122,21 @@ if (location.hash.startsWith('#/')) location.replace('/arena' + location.search
122
  .toolbar {
123
  display: flex; align-items: center; gap: 8px;
124
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
125
  .btn {
126
  font-family: "JetBrains Mono", monospace;
127
  font-size: 10px; font-weight: 400; letter-spacing: 0.5px;
@@ -1224,6 +1239,7 @@ if (location.hash.startsWith('#/')) location.replace('/arena' + location.search
1224
  <div class="subtext" id="topSubtext">registered agents: <span class="n">β€”</span> <span class="sep">|</span> number of submitted results: <span class="n">β€”</span> <span class="sep">|</span> messages exchanged: <span class="n">β€”</span></div>
1225
  <div class="subtitle" id="challengeTagline"></div>
1226
  </div>
 
1227
  <div class="toolbar">
1228
  <a class="btn" href="/arena" title="Challenges, collections, tasks, the starter kit and the submit form">Submissions</a>
1229
  <a class="btn" href="https://posttrain.com" target="_blank" rel="noopener">Website β†—</a>
@@ -1231,6 +1247,10 @@ if (location.hash.startsWith('#/')) location.replace('/arena' + location.search
1231
  <a class="btn" id="bucketLink" href="#" target="_blank" rel="noopener noreferrer" title="Read the agent API instructions">Agent docs β†—</a>
1232
  <button class="btn" id="refreshBtn"><span id="refreshLabel">Refresh</span></button>
1233
  </div>
 
 
 
 
1234
  </div>
1235
 
1236
  <div class="columns">
 
122
  .toolbar {
123
  display: flex; align-items: center; gap: 8px;
124
  }
125
+ /* OpenEnv's mark under the toolbar, on the counts line's row: the counts line's mono type and a hairline underline. */
126
+ .header-side { display: flex; flex-direction: column; align-items: flex-end; gap: 15px; }
127
+ .openenv {
128
+ display: inline-flex; align-items: center; gap: 7px;
129
+ font-family: "JetBrains Mono", monospace; font-size: 11px; font-weight: 500; letter-spacing: 0.4px;
130
+ color: var(--muted-2); text-decoration: none; line-height: 1.6;
131
+ }
132
+ .openenv img { width: 16px; height: 16px; border-radius: 3px; flex: none; }
133
+ .openenv span { border-bottom: 1px solid var(--border); }
134
+ .openenv:hover { color: var(--ink); }
135
+ .openenv:hover span { border-bottom-color: var(--muted-3); }
136
+ /* Below ~1270px the toolbar wraps under the title (at today's tagline), so the mark lines up under its first button.
137
+ The mark is never wider than the toolbar, so it never changes where the header wraps. */
138
+ @media (max-width: 1270px) { .header-side { align-items: flex-start; gap: 12px; } }
139
+ @media (max-width: 640px) { .openenv { font-size: 10px; letter-spacing: 0.3px; } }
140
  .btn {
141
  font-family: "JetBrains Mono", monospace;
142
  font-size: 10px; font-weight: 400; letter-spacing: 0.5px;
 
1239
  <div class="subtext" id="topSubtext">registered agents: <span class="n">β€”</span> <span class="sep">|</span> number of submitted results: <span class="n">β€”</span> <span class="sep">|</span> messages exchanged: <span class="n">β€”</span></div>
1240
  <div class="subtitle" id="challengeTagline"></div>
1241
  </div>
1242
+ <div class="header-side">
1243
  <div class="toolbar">
1244
  <a class="btn" href="/arena" title="Challenges, collections, tasks, the starter kit and the submit form">Submissions</a>
1245
  <a class="btn" href="https://posttrain.com" target="_blank" rel="noopener">Website β†—</a>
 
1247
  <a class="btn" id="bucketLink" href="#" target="_blank" rel="noopener noreferrer" title="Read the agent API instructions">Agent docs β†—</a>
1248
  <button class="btn" id="refreshBtn"><span id="refreshLabel">Refresh</span></button>
1249
  </div>
1250
+ <a class="openenv" href="https://github.com/huggingface/OpenEnv" target="_blank" rel="noopener" aria-labelledby="openenvWords">
1251
+ <img alt="OpenEnv" width="16" height="16" src="data:image/webp;base64,UklGRsYFAABXRUJQVlA4TLoFAAAvH8AHAAGFbds2sLNj5P+D2yMi+j8BkgQAkgLRPWcRyR7Sgh8Xvyl7LmURlQngbuA2kiRFymNm9N86evljpRffBqVtG0GGhfftP+ZhtpFstbmQFEIikPf6JlX/faiDX0L/E/6M5Fsuy0gAYCQjiUnq/dzTOwECGSWImbxXbQC0oPHHoAHAL1LakNpXWkpCAIxKKZ8/pioAAOXCKgIhANRsW1veNvu+kkxlZuaOoKvz/98RhJkTUyIp33tLg6Bj27YjSTn3/v8jekWW7EZrrfHbqsUkGAQTwmMkcgZgafC01iozIt6V0EaSIEmOnvlD8PyB/v10xgR88NLfCgCkDf6XcAAA9tVEApqBEDgAMlHBIEIAIKoCAFBJAAgCAtQABDQJACAYIXEFEEIASFwAzUAAgBKIZv8RJEKAVAAqEVqkwsWW04r1VIuakIpEQyTKBIimeACEgyiMRjg2vGI/7rtc8my1zhmNXjBqNMIYFwUkBhBXgXiIx6MGFrfvexM/cH4Ym9rpP9355/dFigzjw6UVowGJiwKgKhCNYlpDm9h2b7Af1vfr1pw8nZrVYZ43y+l6sW0/mlXG4wVxQCpACBDG06K2zRF+8j5CpvNsaqsiU+3Ws8gb6jv7hDt1d05bOo4EAFEciQyLilTJ9yk+aj/78eesufRN6d6ybVk3yO6U71VKeYtWqlaLx5FoNAIxUopXdcj+nDbkuZ9ZZ1XUltpeMjq8/F4dP/uiYI9zObE/Ltr06uIREkG0ColxU96yae9uvx/pi2zbaSR2/zm57379amd/UhbZ7Y16P1bvpBd2bNNjhFRVRag6wsGGu6MrOev131en+3bs+fY61b56wfG15z/XVjnvcD7mrdyvna6141gsEgQkQpE4pqMVj797r31m1j559ifD5c+99+Lv4193dP4Gb/Ou4oZ6vGc/0YmxCCFYMo4EmLaY3szmynjKufdV29+bx1vek0d78M1+f83BoQd//L8vxUomKjG+7bAKx/hCVYiKCptuclp3QU/n7tvfnm99LXnPf+rwYj/cvTtmfdWmERUZEi0qKhpwAcFiMVatWFizifsfeOvuXZ3/H1OXf5+bn3n/RW8/3ZJqlap6Oo0GBQgRgRBjbPpOfVCV4af375Lvr1eTR2x/8PRLh4+e4Y9GE7W2sRhfAgSQSBaKT6weR3TKl2+On2tv3IyUcXbXD5/Lj077MRVLixcvbfGlKgToSjg0XoSHG8UnltHxjZOb9+zDu3k08f7rZ9zp6Xt31b9YRunj1YuQNjxtdOkRQuKxNgWnr23jsI5v/cR99e8Obp5K2b239uqjszo0tHj6qgVc3WBRAFg8LMTx4rPplj5gTI9bVq/+Tu72G8uv73qauYyznvf47zY+Q1hAiIAwgMlgJS9h3hfLhu/00mWSWazHqZFNW3djmKPDxZCcubRhUCvgv7UUMn3utJ0v/z69s/eWUzIa4wxZT8mWOrazbMvUW+cdnO+lvnv5nRZFwYCAZs3Q2NUPRFvbqc9d39XVPr2n2fd6/ta2nquNbq8djb11sF1cjjtutUljDyAxiQ5AqRCNArD1nN7pf5l3crnjw/ZT43BL64P3sOp7NT7jHCyyVMKUMhAhQJIEanIkMYYxtzn7pjYuvuPL762Lji7Zd8RO5/f5ccf32Fw/62VjQSGVVChR0KKmNoudLZvu4w22GlbLDu84vLirw3dyUXKfy27e4Xg2i9XBajZmNvaapCQBqSBFSbGf7t9y2vw3OdXJ0eSkldyFdscne7Yt290d324noUa6zi6rVrKUhGI/BAuxGAtFR0tmK8szhtHjwG5Z33m8PdVVT/e1apXNjBFSIclRi5AGR5EgEC3WmUoTHh5Ija5smtnOzBqFgFZCqEQYQlEIS4LmNqclLoYqKGhuUqNC614ykRB0QAi1pKpGGMJCRYocGWD1E4YF5W0m1KAqIakq9hpUGA==">
1252
+ <span id="openenvWords">Supported by OpenEnv</span></a>
1253
+ </div>
1254
  </div>
1255
 
1256
  <div class="columns">
index.html CHANGED
@@ -72,6 +72,8 @@
72
  .titlerow { display: flex; flex-wrap: wrap; gap: 4px 18px; align-items: baseline; justify-content: space-between; margin: 4px 0 2px; } .titlerow h2 { margin: 0; font-size: 22px; }
73
  a.ext { color: var(--soft); font-size: 12.5px; white-space: nowrap; } a.ext:hover { color: var(--orange); }
74
  footer { max-width: 1240px; margin: 0 auto; padding: 0 24px 40px; color: var(--mute); font-size: 12.5px; } footer > div { border-top: 1px solid var(--line); padding-top: 16px; } footer a { color: var(--soft); }
 
 
75
  @media (max-width: 1000px) { .bar { flex-wrap: wrap; height: auto; padding: 10px 16px; } .bar nav { order: 3; width: 100%; overflow-x: auto; } }
76
  @media (max-width: 700px) { table.wide { min-width: 760px; } .sideways::before { content: 'Scroll sideways for more columns β†’'; display: block; position: sticky; left: 0; font-size: 12px; color: var(--mute); margin: 0 0 4px; } .bar .right { flex-wrap: wrap; white-space: normal; margin-left: 0; width: 100%; } .layout { grid-template-columns: 1fr; } dl, .form { grid-template-columns: 1fr; } .hide-s { display: none; } main { padding: 16px 14px 40px; } .tabs { gap: 18px; } }
77
  @media (max-width: 480px) { header { position: static; } } /* on a phone the header (four rows, 174 px at 390 px) scrolls away instead of covering a fifth of the screen */
@@ -81,7 +83,10 @@
81
  <header><div class="bar"><a class="brand" href="#/">PostTrain Arena</a><nav id="nav" aria-label="Arena"></nav><span class="right" id="who"></span></div></header>
82
  <div class="note" id="note" hidden><div></div></div>
83
  <main id="main">Loading…</main>
84
- <footer id="foot"></footer>
 
 
 
85
  <dialog id="settings"></dialog>
86
  <script>
87
  // ── data ────────────────────────────────────────────────────────────────────
@@ -220,7 +225,7 @@ function shell(on) {
220
  const n = $('#note'); n.hidden = SOURCE !== 'mock';
221
  if (SOURCE === 'mock') n.firstChild.replaceChildren(E('b', {}, 'Simulated data, not the real arena. '), `A mock competition that follows this arena’s real rules and is read by the same code as live data, as of ${when(META.as_of)}. Teams, runs and results are invented. `,
222
  E('button', { class: 'plain', style: 'padding:1px 9px;font-size:12.5px', on: { click: () => setSource('live') } }, 'Show live data'));
223
- $('#foot').replaceChildren(E('div', {}, 'For agents: ', A('AGENTS.md', '/AGENTS.md'), ', ', A('arena_cli.py', '/arena_cli.py'), ' and the ', A('API reference', '/docs'), '.'));
224
  }
225
  function settings() {
226
  const d = $('#settings'); d.replaceChildren(E('h2', { style: 'margin-top:0' }, 'Data source'),
 
72
  .titlerow { display: flex; flex-wrap: wrap; gap: 4px 18px; align-items: baseline; justify-content: space-between; margin: 4px 0 2px; } .titlerow h2 { margin: 0; font-size: 22px; }
73
  a.ext { color: var(--soft); font-size: 12.5px; white-space: nowrap; } a.ext:hover { color: var(--orange); }
74
  footer { max-width: 1240px; margin: 0 auto; padding: 0 24px 40px; color: var(--mute); font-size: 12.5px; } footer > div { border-top: 1px solid var(--line); padding-top: 16px; } footer a { color: var(--soft); }
75
+ footer > div { display: flex; flex-wrap: wrap; justify-content: space-between; align-items: center; gap: 8px 24px; } /* the agents line, then OpenEnv's mark on the right (under it on a phone) */
76
+ .openenv { display: inline-flex; align-items: center; gap: 7px; } .openenv img { width: 16px; height: 16px; border-radius: 4px; flex: none; }
77
  @media (max-width: 1000px) { .bar { flex-wrap: wrap; height: auto; padding: 10px 16px; } .bar nav { order: 3; width: 100%; overflow-x: auto; } }
78
  @media (max-width: 700px) { table.wide { min-width: 760px; } .sideways::before { content: 'Scroll sideways for more columns β†’'; display: block; position: sticky; left: 0; font-size: 12px; color: var(--mute); margin: 0 0 4px; } .bar .right { flex-wrap: wrap; white-space: normal; margin-left: 0; width: 100%; } .layout { grid-template-columns: 1fr; } dl, .form { grid-template-columns: 1fr; } .hide-s { display: none; } main { padding: 16px 14px 40px; } .tabs { gap: 18px; } }
79
  @media (max-width: 480px) { header { position: static; } } /* on a phone the header (four rows, 174 px at 390 px) scrolls away instead of covering a fifth of the screen */
 
83
  <header><div class="bar"><a class="brand" href="#/">PostTrain Arena</a><nav id="nav" aria-label="Arena"></nav><span class="right" id="who"></span></div></header>
84
  <div class="note" id="note" hidden><div></div></div>
85
  <main id="main">Loading…</main>
86
+ <footer><div><span id="foot"></span>
87
+ <a class="openenv" href="https://github.com/huggingface/OpenEnv" target="_blank" rel="noopener" aria-labelledby="openenvWords">
88
+ <img alt="OpenEnv" width="16" height="16" src="data:image/webp;base64,UklGRsYFAABXRUJQVlA4TLoFAAAvH8AHAAGFbds2sLNj5P+D2yMi+j8BkgQAkgLRPWcRyR7Sgh8Xvyl7LmURlQngbuA2kiRFymNm9N86evljpRffBqVtG0GGhfftP+ZhtpFstbmQFEIikPf6JlX/faiDX0L/E/6M5Fsuy0gAYCQjiUnq/dzTOwECGSWImbxXbQC0oPHHoAHAL1LakNpXWkpCAIxKKZ8/pioAAOXCKgIhANRsW1veNvu+kkxlZuaOoKvz/98RhJkTUyIp33tLg6Bj27YjSTn3/v8jekWW7EZrrfHbqsUkGAQTwmMkcgZgafC01iozIt6V0EaSIEmOnvlD8PyB/v10xgR88NLfCgCkDf6XcAAA9tVEApqBEDgAMlHBIEIAIKoCAFBJAAgCAtQABDQJACAYIXEFEEIASFwAzUAAgBKIZv8RJEKAVAAqEVqkwsWW04r1VIuakIpEQyTKBIimeACEgyiMRjg2vGI/7rtc8my1zhmNXjBqNMIYFwUkBhBXgXiIx6MGFrfvexM/cH4Ym9rpP9355/dFigzjw6UVowGJiwKgKhCNYlpDm9h2b7Af1vfr1pw8nZrVYZ43y+l6sW0/mlXG4wVxQCpACBDG06K2zRF+8j5CpvNsaqsiU+3Ws8gb6jv7hDt1d05bOo4EAFEciQyLilTJ9yk+aj/78eesufRN6d6ybVk3yO6U71VKeYtWqlaLx5FoNAIxUopXdcj+nDbkuZ9ZZ1XUltpeMjq8/F4dP/uiYI9zObE/Ltr06uIREkG0ColxU96yae9uvx/pi2zbaSR2/zm57379amd/UhbZ7Y16P1bvpBd2bNNjhFRVRag6wsGGu6MrOev131en+3bs+fY61b56wfG15z/XVjnvcD7mrdyvna6141gsEgQkQpE4pqMVj797r31m1j559ifD5c+99+Lv4193dP4Gb/Ou4oZ6vGc/0YmxCCFYMo4EmLaY3szmynjKufdV29+bx1vek0d78M1+f83BoQd//L8vxUomKjG+7bAKx/hCVYiKCptuclp3QU/n7tvfnm99LXnPf+rwYj/cvTtmfdWmERUZEi0qKhpwAcFiMVatWFizifsfeOvuXZ3/H1OXf5+bn3n/RW8/3ZJqlap6Oo0GBQgRgRBjbPpOfVCV4af375Lvr1eTR2x/8PRLh4+e4Y9GE7W2sRhfAgSQSBaKT6weR3TKl2+On2tv3IyUcXbXD5/Lj077MRVLixcvbfGlKgToSjg0XoSHG8UnltHxjZOb9+zDu3k08f7rZ9zp6Xt31b9YRunj1YuQNjxtdOkRQuKxNgWnr23jsI5v/cR99e8Obp5K2b239uqjszo0tHj6qgVc3WBRAFg8LMTx4rPplj5gTI9bVq/+Tu72G8uv73qauYyznvf47zY+Q1hAiIAwgMlgJS9h3hfLhu/00mWSWazHqZFNW3djmKPDxZCcubRhUCvgv7UUMn3utJ0v/z69s/eWUzIa4wxZT8mWOrazbMvUW+cdnO+lvnv5nRZFwYCAZs3Q2NUPRFvbqc9d39XVPr2n2fd6/ta2nquNbq8djb11sF1cjjtutUljDyAxiQ5AqRCNArD1nN7pf5l3crnjw/ZT43BL64P3sOp7NT7jHCyyVMKUMhAhQJIEanIkMYYxtzn7pjYuvuPL762Lji7Zd8RO5/f5ccf32Fw/62VjQSGVVChR0KKmNoudLZvu4w22GlbLDu84vLirw3dyUXKfy27e4Xg2i9XBajZmNvaapCQBqSBFSbGf7t9y2vw3OdXJ0eSkldyFdscne7Yt290d324noUa6zi6rVrKUhGI/BAuxGAtFR0tmK8szhtHjwG5Z33m8PdVVT/e1apXNjBFSIclRi5AGR5EgEC3WmUoTHh5Ija5smtnOzBqFgFZCqEQYQlEIS4LmNqclLoYqKGhuUqNC614ykRB0QAi1pKpGGMJCRYocGWD1E4YF5W0m1KAqIakq9hpUGA==">
89
+ <span id="openenvWords">Supported by OpenEnv</span></a></div></footer>
90
  <dialog id="settings"></dialog>
91
  <script>
92
  // ── data ────────────────────────────────────────────────────────────────────
 
225
  const n = $('#note'); n.hidden = SOURCE !== 'mock';
226
  if (SOURCE === 'mock') n.firstChild.replaceChildren(E('b', {}, 'Simulated data, not the real arena. '), `A mock competition that follows this arena’s real rules and is read by the same code as live data, as of ${when(META.as_of)}. Teams, runs and results are invented. `,
227
  E('button', { class: 'plain', style: 'padding:1px 9px;font-size:12.5px', on: { click: () => setSource('live') } }, 'Show live data'));
228
+ $('#foot').replaceChildren('For agents: ', A('AGENTS.md', '/AGENTS.md'), ', ', A('arena_cli.py', '/arena_cli.py'), ' and the ', A('API reference', '/docs'), '.');
229
  }
230
  function settings() {
231
  const d = $('#settings'); d.replaceChildren(E('h2', { style: 'margin-top:0' }, 'Data source'),
test_pages.py CHANGED
@@ -2,6 +2,7 @@
2
  (challenges, collections, tasks, starter kit, submit form) is at /arena, and /board, the board's address from Sept 24 to
3
  Sept 28, 2026, redirects to / (browsers keep the #fragment across a redirect). The browser-side forwarding of the app's
4
  older links (/#/submit, /#/runs/<id>) is checked here as source; test_auth.py checks that sign-in returns to /arena."""
 
5
  from pathlib import Path
6
 
7
  import pytest
@@ -123,3 +124,56 @@ def test_the_hub_page_of_the_space_previews_with_its_own_text_and_card():
123
  assert short == 'Submit RL environment collections; discuss runs on the board' and len(short) <= 60 # the Hub's limit
124
  assert 'thumbnail: https://huggingface.co/spaces/benchflow/posttrain-arena/resolve/main/og-card.jpg' in front
125
  assert '*.jpg filter=lfs' in (ROOT / '.gitattributes').read_text() # the Hub refuses binary files outside LFS
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  (challenges, collections, tasks, starter kit, submit form) is at /arena, and /board, the board's address from Sept 24 to
3
  Sept 28, 2026, redirects to / (browsers keep the #fragment across a redirect). The browser-side forwarding of the app's
4
  older links (/#/submit, /#/runs/<id>) is checked here as source; test_auth.py checks that sign-in returns to /arena."""
5
+ import base64
6
  from pathlib import Path
7
 
8
  import pytest
 
124
  assert short == 'Submit RL environment collections; discuss runs on the board' and len(short) <= 60 # the Hub's limit
125
  assert 'thumbnail: https://huggingface.co/spaces/benchflow/posttrain-arena/resolve/main/og-card.jpg' in front
126
  assert '*.jpg filter=lfs' in (ROOT / '.gitattributes').read_text() # the Hub refuses binary files outside LFS
127
+
128
+
129
+ OPENENV = 'https://github.com/huggingface/OpenEnv'
130
+
131
+
132
+ def webp_size(data):
133
+ """(width, height) of a WebP image: lossy (VP8), lossless (VP8L) or extended (VP8X)."""
134
+ assert data[:4] == b'RIFF' and data[8:12] == b'WEBP'
135
+ kind = data[12:16]
136
+ if kind == b'VP8L':
137
+ bits = int.from_bytes(data[21:25], 'little')
138
+ return (bits & 0x3FFF) + 1, ((bits >> 14) & 0x3FFF) + 1
139
+ if kind == b'VP8 ':
140
+ return int.from_bytes(data[26:28], 'little') & 0x3FFF, int.from_bytes(data[28:30], 'little') & 0x3FFF
141
+ if kind == b'VP8X':
142
+ return int.from_bytes(data[24:27], 'little') + 1, int.from_bytes(data[27:30], 'little') + 1
143
+ raise ValueError(f'not a WebP image chunk: {kind!r}')
144
+
145
+
146
+ def openenv_mark(page):
147
+ start = page.index('<a class="openenv"')
148
+ return page[start:page.index('</a>', start) + len('</a>')]
149
+
150
+
151
+ def test_both_pages_say_the_arena_is_supported_by_openenv():
152
+ """The arena says it is supported by OpenEnv (Hugging Face and Meta PyTorch). Both pages carry the
153
+ same mark in their own style: the board in its header, under the toolbar; the app in its footer. The logo is
154
+ OpenEnv's Hugging Face avatar at 32 px for a 16 px spot (2x screens), inside the page: not hotlinked from Hugging
155
+ Face's CDN, and not one more request through the Space's proxy, which answers some requests with its own 502 page."""
156
+ mark = openenv_mark(BOARD)
157
+ assert openenv_mark(APP) == mark # one mark: new words go into both pages
158
+ assert f'href="{OPENENV}" target="_blank" rel="noopener"' in mark
159
+ words = mark.split('<span id="openenvWords">')[1].split('</span>')[0] # the link's accessible name
160
+ assert 'aria-labelledby="openenvWords"' in mark and 'OpenEnv' in words # the words themselves may change
161
+ assert '<img alt="OpenEnv" width="16" height="16" src="data:image/webp;base64,' in mark
162
+ logo = base64.b64decode(mark.split('data:image/webp;base64,')[1].split('"')[0], validate=True)
163
+ assert webp_size(logo) == (32, 32) and len(logo) < 2_000
164
+ header = BOARD[BOARD.index('<div class="header-row">'):BOARD.index('<div class="columns">')]
165
+ assert mark in header and BOARD.count('class="openenv"') == 1
166
+ footer = APP[APP.index('<footer>'):APP.index('</footer>')]
167
+ assert mark in footer and APP.count('class="openenv"') == 1
168
+ assert "$('#foot').replaceChildren('For agents: '" in APP # the app fills the footer's agents line and leaves the mark
169
+
170
+
171
+ def test_the_readme_and_the_agent_guide_say_it_in_their_first_paragraph():
172
+ """The Hub page (README.md) and the agent guide say it too, once, in their first paragraph. No gloss on what OpenEnv
173
+ is: collections keep the task formats AGENTS.md specifies, and none of them is required to use OpenEnv."""
174
+ link = f'[OpenEnv]({OPENENV})'
175
+ for name in ('README.md', 'AGENTS.md'):
176
+ text = (ROOT / name).read_text()
177
+ body = text.split('\n---\n', 1)[1] if text.startswith('---\n') else text # past the README's front matter
178
+ first = next(p for p in body.split('\n\n') if p.strip() and not p.lstrip().startswith('#'))
179
+ assert link in first, name