Deploy HF RL Explorer

DESIGN.md

@@ -242,3 +242,33 @@ This public tooling directory contains only the Explorer. The dashboard applicat
 its assets, membership checks, tests and deployment tooling are maintained separately
 in the private `FineEnvs/RL-Explorer-admin` Space. The Explorer reads shared settings
 and moderation requests from the data bucket without importing the admin app.
+
+## Search indexing and social previews
+
+The Space card uses the committed `web/social/rl-explorer.png` thumbnail, so a
+sharing crawler does not have to start the application. HTML pages also publish
+Open Graph and Twitter metadata with 1200×630 previews. Page-specific images fit
+long names into the card; public task links retain their own canonical address
+both in the initial HTML and after browser navigation.
+
+`/sitemap.xml` links to shards of up to 10,000 public task URLs. Dataset tasks come
+from local indexes or the immutable catalog snapshot, with MiMo entries deduplicated.
+OpenEnv task coordinates come from advertised split counts on checked public
+Spaces. Their sitemaps expand ranges one shard at a time, without downloading tasks
+or creating millions of URLs in memory. These counts describe published entries,
+not individually tested episodes or Google-indexed pages.
+
+Task HTML includes public task text or structured input facts. Public row datasets
+and OpenEnv Task API records can be read anonymously on demand, with four concurrent
+reads and bounded caches. These reads do not build an index, wake a Space, run an
+episode, or forward a visitor's token. The existing answer-withholding rules apply.
+Invalid tasks return 404; temporary upstream failures return 503 with Retry-After.
+Private, gated and hidden content is excluded from sitemap generation.
+
+Public read APIs may be fetched to render the interactive pages, but API responses
+carry X-Robots-Tag: noindex. Rollouts, account pages and comparisons remain excluded.
+Submit `https://fineenvs-rl-explorer.hf.space/sitemap.xml` for the matching URL-prefix
+property in Google Search Console to monitor discovery and indexing. Robots.txt
+also advertises it. Google chooses which pages to index; no application setting
+can guarantee indexing or rankings. See Google's sitemap guidance:
+https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap

README.md

@@ -1,5 +1,5 @@
 ---
-title: HF RL Explorer
+title: RL Environments on Hugging Face
 emoji: 🤗
 colorFrom: yellow
 colorTo: gray
@@ -7,7 +7,8 @@ sdk: docker
 app_port: 7860
 pinned: false
 license: apache-2.0
-short_description: Browse every RL environment on the Hub and run agents on it
+short_description: Explore RL environments, browse tasks and run agent rollouts
+thumbnail: https://huggingface.co/spaces/FineEnvs/RL-Explorer/resolve/main/web/social/rl-explorer.png
 hf_oauth: true
 hf_oauth_expiration_minutes: 1440
 hf_oauth_scopes:
@@ -29,10 +30,17 @@ tags:
   - mcp
 ---
 
-# HF RL Explorer
+# RL environments on the Hugging Face Hub
 
-Every reinforcement-learning environment on the Hugging Face Hub, in one place: what each task asks, how it is
-graded, what it runs in, and an agent run on it, graded, that you can watch step by step.
+**HF RL Explorer** helps you discover reinforcement learning environments on the Hugging Face Hub.
+Explore public datasets and checked environment Spaces across OpenEnv, Harbor, MiMo, NeMo Gym and Verifiers.
+Inspect individual tasks, tools and reward functions, then run supported agent rollouts and compare results.
+
+[Open the RL environment explorer](https://fineenvs-rl-explorer.hf.space/) ·
+[Explore FineEnvs environments](https://fineenvs-rl-explorer.hf.space/?owner=FineEnvs) ·
+[Public task sitemap](https://fineenvs-rl-explorer.hf.space/sitemap.xml)
+
+![HF RL Explorer — RL environments on the Hugging Face Hub](web/social/rl-explorer.png)
 
 - **Harbor datasets**: every task folder indexed (instruction, tests, image, grader, multi-step tasks, resources);
   run one with OpenCode, Terminus 2, mini-SWE-agent or Pi on an HF Sandbox, graded by the task's own tests.

app/dockerfile.py

@@ -13,7 +13,7 @@ import re
 import shlex
 from dataclasses import dataclass, field
 
-IGNORED = {"CMD", "ENTRYPOINT", "EXPOSE", "LABEL", "HEALTHCHECK", "VOLUME", "STOPSIGNAL", "MAINTAINER", "USER", "SHELL"}
+IGNORED = {"CMD", "ENTRYPOINT", "EXPOSE", "LABEL", "HEALTHCHECK", "VOLUME", "STOPSIGNAL", "MAINTAINER"}
 
 
 @dataclass

app/http.py

@@ -79,6 +79,7 @@ def install(app: FastAPI) -> None:
             # APIs can contain a visitor's private dataset, run, or account. A shared
             # proxy or the browser's cache must never reuse them for another visitor.
             resp.headers["Cache-Control"] = "private, no-store"
+            resp.headers["X-Robots-Tag"] = "noindex"
         else:
             resp.headers.setdefault("Cache-Control", "no-cache")
         return resp

app/main.py

@@ -809,16 +809,21 @@ def sitemap_tasks(n: int, request: Request):
     return seo.sitemap_tasks(request, n)
 
 
+@app.get("/sitemap-space-tasks-{n}.xml", include_in_schema=False)
+def sitemap_space_tasks(n: int, request: Request):
+    return seo.sitemap_space_tasks(request, n)
+
+
 @app.get("/og.png", include_in_schema=False)
 def og_site():
     return seo.og_image("/")
 
 
 @app.get("/og/{path:path}", include_in_schema=False)
-def og_page(path: str):
+def og_page(path: str, request: Request):
     if not path.endswith(".png") or len(path) > 600:
         raise HTTPException(404, "no such image")
-    return seo.og_image("/" + path[:-4])
+    return seo.og_image("/" + path[:-4], request.query_params)
 
 
 @app.exception_handler(404)

app/seo.py

@@ -4,9 +4,9 @@ data (a Dataset for an environment, a SoftwareApplication for a Space, a task as
 in its body, the page's content as plain HTML, for crawlers that don't run JavaScript (the app replaces it as it
 starts). Plus robots.txt, a sitemap of every environment and every indexed task, and a preview image per page.
 
-Cheap and safe by design: only what is already known is used (the catalog's listing of public environments, indexes
-already built, the MiMo release's local index). A crawler never starts indexing a dataset, wakes a Space, or reads
-anything private; task text comes from the same withheld views the pages show.
+Catalog pages use public indexes. Row and Space task pages may make bounded anonymous
+reads of the same withheld task views shown in the UI. Crawlers never start indexing,
+wake a Space, run an episode or receive a visitor's credentials.
 """
 
 from __future__ import annotations
@@ -19,22 +19,22 @@ import threading
 import time
 from functools import lru_cache
 from typing import Any
-from urllib.parse import quote
+from urllib.parse import quote, urlencode
 
 from fastapi import Request
 from fastapi.responses import HTMLResponse, PlainTextResponse, Response
 
-from . import catalog, config
+from . import catalog, config, seo_tasks, snapshot, space_checks, spaces_live
 
 SITE = "HF RL Explorer"
-TAGLINE = "Explore RL environments on Hugging Face"
-DESCRIPTION = ("Explore reinforcement learning environments on Hugging Face across OpenEnv, Harbor, Verifiers, NeMo Gym and "
-               "more: what each task asks, how it's graded, what it runs in, and run an agent on it.")
+TAGLINE = "RL environments on the Hugging Face Hub"
+DESCRIPTION = ("Explore reinforcement learning environments and tasks on the Hugging Face Hub. Browse OpenEnv, Harbor, "
+               "MiMo, NeMo Gym and Verifiers, inspect rewards, and run supported agent rollouts.")
 KIND = {"harbor": "Harbor dataset", "verifiers": "Verifiers environment", "nemo-gym": "NeMo Gym dataset", "rows": "RL dataset",
         "mimo": "MiMo RL release", "openenv": "OpenEnv Space", "space": "environment Space"}
 MIMO = "XiaomiMiMo/MiMo-V2.6-RL-oss"
 NOINDEX = re.compile(r"^/(run/|runs$|compare/)")
-SITEMAP_CHUNK = 40_000
+SITEMAP_CHUNK = 10_000
 
 
 # ── where we are ─────────────────────────────────────────────────────────────
@@ -92,6 +92,15 @@ def listing() -> tuple[dict[str, dict[str, Any]], list[dict[str, Any]]]:
     return by, rows
 
 
+def public_rows(rows):
+    hidden = set(catalog.hidden())
+    return [r for r in rows if r["key"] not in hidden and not any(r.get(k) for k in ("private", "gated", "restricted"))]
+
+
+def discoverable(r):
+    return r.get("kind") == "dataset" or space_checks.browseable(space_checks.inventory().get(r["id"]))
+
+
 def index_rows(spec: str) -> list[dict[str, Any]]:
     """An environment's tasks, if they are already known here (never built for a crawler): [{path, title, brief, category}]."""
     if spec == MIMO:
@@ -100,7 +109,10 @@ def index_rows(spec: str) -> list[dict[str, Any]]:
         p = catalog._index_path(spec)
         mtime = p.stat().st_mtime
     except (OSError, ValueError):
-        return []
+        try:
+            return _snapshot_rows(spec, snapshot.get().name)
+        except snapshot.SnapshotError:
+            return []
     return _harbor_rows(spec, mtime)
 
 
@@ -114,9 +126,17 @@ def _mimo_rows() -> list[dict[str, Any]]:
 @lru_cache(maxsize=32)
 def _harbor_rows(spec: str, mtime: float) -> list[dict[str, Any]]:
     idx = catalog._read_index(spec)
+    if not idx or (idx.get("info") or {}).get("restricted"):
+        return []
     return [{"path": t["path"], "title": t.get("title"), "brief": t.get("brief"), "category": t.get("category")} for t in (idx or {}).get("tasks") or []]
 
 
+@lru_cache(maxsize=32)
+def _snapshot_rows(spec: str, revision: str) -> list[dict[str, Any]]:
+    with snapshot.use() as (_, conn):
+        return [dict(r) for r in conn.execute("SELECT ref AS path, title, brief, category FROM tasks WHERE env = ? ORDER BY ref", (spec,))]
+
+
 def display_name(row: dict[str, Any] | None, spec: str) -> str:
     """What to call an environment: its card's heading when that names it, else the repository name with the heading."""
     repo = spec.split("/")[-1]
@@ -145,9 +165,10 @@ def page(request: Request, *, title: str, description: str, path: str, body: str
     base = base_url(request)
     url = base + path
     full = title if SITE in title else f"{title} · {SITE}"
-    img = base + (image or "/og.png")
+    img = base + (image or "/social/rl-explorer.png")
     head = "\n".join([
         f'<link rel="canonical" href="{esc(url)}">',
+        f'<meta name="rlx-page-url" content="{esc(url)}">',
         '<meta name="robots" content="noindex, follow">' if noindex else '<meta name="robots" content="index, follow, max-image-preview:large, max-snippet:-1">',
         f'<meta property="og:site_name" content="{SITE}">',
         f'<meta property="og:type" content="{og_type}">',
@@ -155,6 +176,8 @@ def page(request: Request, *, title: str, description: str, path: str, body: str
         f'<meta property="og:description" content="{esc(description)}">',
         f'<meta property="og:url" content="{esc(url)}">',
         f'<meta property="og:image" content="{esc(img)}">',
+        '<meta property="og:image:type" content="image/png">',
+        '<meta property="og:locale" content="en_US">',
         '<meta property="og:image:width" content="1200">',
         '<meta property="og:image:height" content="630">',
         f'<meta property="og:image:alt" content="{esc(full)}">',
@@ -162,6 +185,7 @@ def page(request: Request, *, title: str, description: str, path: str, body: str
         f'<meta name="twitter:title" content="{esc(full)}">',
         f'<meta name="twitter:description" content="{esc(description)}">',
         f'<meta name="twitter:image" content="{esc(img)}">',
+        f'<meta name="twitter:image:alt" content="{esc(full)}">',
         *[f'<script type="application/ld+json">{_ld(x)}</script>' for x in jsonld or []],
     ])
     doc = template()
@@ -170,7 +194,7 @@ def page(request: Request, *, title: str, description: str, path: str, body: str
     doc = doc.replace("</head>", f"{head}\n</head>", 1)
     if body:   # the page as plain HTML until the app starts (crawlers that don't run JavaScript read this)
         doc = re.sub(r'<main id="view">(.*?)</main>', lambda m: f'<main id="view">{m.group(1)}<div class="wrap page ssr">{body}</div></main>', doc, count=1, flags=re.S)
-    return HTMLResponse(doc, status_code=status, headers={"Cache-Control": "no-cache"})
+    return HTMLResponse(doc, status_code=status, headers={"Cache-Control": "no-cache", **({"Retry-After": "60"} if status == 503 else {})})
 
 
 def _ld(x: dict[str, Any]) -> str:
@@ -191,18 +215,15 @@ def crumbs(base: str, items: list[tuple[str, str]]) -> tuple[str, dict[str, Any]
 def home(request: Request) -> HTMLResponse:
     base = base_url(request)
     by, rows = listing()
+    rows = [r for r in public_rows(rows) if discoverable(r)]
     top = sorted(rows, key=lambda r: (r.get("trending") or 0) * 1e9 + (r.get("downloads") or 0), reverse=True)[:120]
-    ds = sum(1 for r in rows if r["kind"] == "dataset")
-    sp = sum(1 for r in rows if r["kind"] == "space")
-    tasks = sum((r.get("indexed") or {}).get("tasks") or 0 for r in rows)
-    desc = (f"{ds:,} RL environment datasets and {sp:,} environment Spaces on Hugging Face, across OpenEnv, Harbor, Verifiers, "
-            f"NeMo Gym and more: see what each task asks and how it's graded, then run an agent on it." if rows else DESCRIPTION)
+    desc = DESCRIPTION
     body = (f"<header class=\"tp-head\"><h1>{TAGLINE}</h1><p class=\"lede\">{esc(desc)}</p></header>"
             + _env_list(top) + '<p><a href="/community">Community rollouts</a> · <a href="/d/XiaomiMiMo/MiMo-V2.6-RL-oss">MiMo-V2.6 RL</a></p>')
     ld = [{"@type": "WebSite", "name": SITE, "alternateName": TAGLINE, "url": base + "/", "description": desc,
            "potentialAction": {"@type": "SearchAction", "target": {"@type": "EntryPoint", "urlTemplate": base + "/?q={search_term_string}"},
                                "query-input": "required name=search_term_string"},
-           "publisher": {"@type": "Organization", "name": "Hugging Face", "url": "https://huggingface.co"}},
+           "publisher": {"@type": "Organization", "name": "FineEnvs", "url": "https://huggingface.co/FineEnvs"}},
           {"@type": "CollectionPage", "name": TAGLINE, "url": base + "/", "about": "Reinforcement learning environments",
            "mainEntity": {"@type": "ItemList", "numberOfItems": len(top), "itemListElement": [
                {"@type": "ListItem", "position": i + 1, "url": base + _href(r), "name": r.get("heading") or r["id"]} for i, r in enumerate(top[:50])]}}]
@@ -221,10 +242,10 @@ def _env_list(rows: list[dict[str, Any]]) -> str:
 def environment(request: Request, spec: str) -> HTMLResponse:
     base = base_url(request)
     by, _ = listing()
-    r = by.get(spec)
+    r = next((r for r in public_rows(list(by.values())) if r["key"] == spec), None)
     name = display_name(r, spec)
     kind = kind_of(r, spec)
-    tasks = index_rows(spec) if r or spec == MIMO else []
+    tasks = index_rows(spec) if r else []
     n = len(tasks) or ((r or {}).get("indexed") or {}).get("tasks")
     brief = clip((r or {}).get("brief") or "", 300)
     desc = clip(f"{name}: {kind} on Hugging Face{f' with {n:,} tasks' if n else ''}. "
@@ -243,16 +264,24 @@ def environment(request: Request, spec: str) -> HTMLResponse:
            "includedInDataCatalog": {"@type": "DataCatalog", "name": SITE, "url": base + "/"},
            "distribution": [{"@type": "DataDownload", "encodingFormat": "application/octet-stream", "contentUrl": f"https://huggingface.co/datasets/{spec}"}]},
           cld]
-    return page(request, title=f"{name} · {kind}", description=desc, path=path, body=body, jsonld=ld, image=f"/og{path}.png" if (r or spec == MIMO) else None, noindex=not r and spec != MIMO)
+    return page(request, title=f"{name} · {kind}", description=desc, path=path, body=body, jsonld=ld, image=f"/og{path}.png" if r else None, noindex=not r)
 
 
 def task(request: Request, spec: str, ref: str) -> HTMLResponse:
     base = base_url(request)
     by, _ = listing()
-    r = by.get(spec)
+    r = next((r for r in public_rows(list(by.values())) if r["key"] == spec), None)
     env_name = display_name(r, spec)
-    known = r is not None or spec == MIMO
+    known = r is not None
     row = next((t for t in index_rows(spec) if t["path"] == ref), None) if known else None
+    status = 200
+    if known and not row and re.fullmatch(r"(?:[^/]+/){1,2}\d{1,10}", ref):
+        try:
+            row = seo_tasks.row(spec, ref)
+        except seo_tasks.Unavailable as exc:
+            status = exc.status
+    elif known and not row:
+        status = 404
     split, _, n = ref.rpartition("/")
     # a row of a dataset read as rows (no index here): "Row 3 (train)" rather than a bare "3"
     fallback = f"Row {n} ({split})" if split and n.isdigit() and "/" not in split else ref.rsplit("/", 1)[-1]
@@ -262,18 +291,25 @@ def task(request: Request, spec: str, ref: str) -> HTMLResponse:
     path = f"/t/{enc(spec)}/{enc(ref)}"
     cr, cld = crumbs(base, [("Environments", "/"), (env_name, f"/d/{enc(spec)}"), (title, "")])
     body = (cr + f"<header class=\"tp-head\"><h1>{esc(title)}</h1><p class=\"lede\">{esc(desc)}</p></header>"
+            + (f'<section><h2>The task</h2><p class="seo-prompt">{esc(row.get("brief") or "")}</p></section>' if row else "")
             + f'<p>Part of <a href="/d/{enc(spec)}">{esc(spec)}</a>.</p>')
+    if row and type(row.get("total")) is int and n.isdigit():
+        body += '<nav aria-label="Other tasks">' + " · ".join(
+            f'<a href="/t/{enc(spec)}/{enc(split)}/{i}">{label}</a>'
+            for i, label in ((int(n) - 1, "Previous task"), (int(n) + 1, "Next task")) if 0 <= i < row["total"]) + '</nav>'
     ld = [{"@type": "CreativeWork", "name": title, "description": desc, "url": base + path, "learningResourceType": "RL environment task",
            "isPartOf": {"@type": "Dataset", "name": env_name, "url": f"{base}/d/{enc(spec)}", "sameAs": f"https://huggingface.co/datasets/{spec}"},
            "about": "reinforcement learning", **({"genre": row["category"]} if row and row.get("category") else {})}, cld]
     return page(request, title=f"{title} · {env_name}", description=desc, path=path, body=body, jsonld=ld, image=f"/og{path}.png" if known else None,
-                noindex=not row, og_type="article")
+                noindex=not row and status != 503, og_type="article", status=status)
 
 
 def space(request: Request, spec: str) -> HTMLResponse:
     base = base_url(request)
     by, _ = listing()
-    r = by.get(f"space:{spec}")
+    r = next((r for r in public_rows(list(by.values())) if r["key"] == f"space:{spec}"), None)
+    if "task" in request.query_params:
+        return space_task(request, spec, r)
     name = display_name(r, spec)
     kind = kind_of(r, spec) if r else "environment Space"
     desc = clip(f"{name}: an {kind} on Hugging Face. {clip((r or {}).get('brief') or '', 220) or 'See it live: its app, a playground with rewards, its tasks, and MCP for coding agents.'}", 300)
@@ -286,7 +322,49 @@ def space(request: Request, spec: str) -> HTMLResponse:
            "offers": {"@type": "Offer", "price": "0", "priceCurrency": "USD"},
            "author": {"@type": "Organization", "name": spec.split("/")[0], "url": f"https://huggingface.co/{spec.split('/')[0]}"},
            "keywords": ["reinforcement learning", "RL environment", *(["OpenEnv"] if (r or {}).get("openenv") else []), *((r or {}).get("tags") or [])[:10]]}, cld]
-    return page(request, title=f"{name} · {kind}", description=desc, path=path, body=body, jsonld=ld, image=f"/og{path}.png" if r else None, noindex=not r)
+    if r and discoverable(r):
+        ranges = seo_tasks.ranges((spaces_live.last_seen(spec) or {}).get("task_api"))
+        body += "".join(f'<section><h2>{esc(split)} tasks</h2><ul>' + "".join(
+            f'<li><a href="{esc(seo_tasks.space_path(spec, env, split, i))}">Task {i + 1}</a></li>' for i in range(min(n, 12)))
+            + "</ul></section>" for env, split, n in ranges)
+    return page(request, title=f"{name} · {kind}", description=desc, path=path, body=body, jsonld=ld, image=f"/og{path}.png" if r else None, noindex=not r or not discoverable(r))
+
+
+def space_task(request: Request, spec: str, environment: dict | None) -> HTMLResponse:
+    q = request.query_params
+    env, split, raw = q.get("env", ""), q.get("split", ""), q.get("task", "")
+    if not environment or not re.fullmatch(r"\d{1,10}", raw) or not env or not split or len(env) > 80 or len(split) > 200:
+        return not_found(request)
+    index = int(raw)
+    path = seo_tasks.space_path(spec, env, split, index)
+    status, row = 200, None
+    try:
+        row = seo_tasks.space(spec, env, split, index)
+    except seo_tasks.Unavailable as exc:
+        status = exc.status
+    except Exception:
+        status = 503
+    if status == 404:
+        return not_found(request)
+    title = clip((row or {}).get("title") or f"Task {index + 1}", 110)
+    desc = clip(f"{title}: {split} task in {spec}, an RL environment on the Hugging Face Hub. {(row or {}).get('brief') or ''}", 300)
+    cr, cld = crumbs(base_url(request), [("Environments", "/"), (spec, f"/s/{enc(spec)}"), (title, "")])
+    body = cr + f'<header class="tp-head"><h1>{esc(title)}</h1><p class="lede">{esc(desc)}</p></header>'
+    if row:
+        body += f'<section><h2>The task</h2><p class="seo-prompt">{esc(row["brief"])}</p></section>'
+        if row.get("fields"):
+            body += '<h2>Task details</h2><dl>' + "".join(
+                f'<dt>{esc(k.replace("_", " "))}</dt><dd>{esc(str(v)[:1000])}</dd>' for k, v in row["fields"].items()) + '</dl>'
+        body += '<nav aria-label="Other tasks">' + " · ".join(
+            f'<a href="{esc(seo_tasks.space_path(spec, env, split, i))}">{label}</a>'
+            for i, label in ((index - 1, "Previous task"), (index + 1, "Next task")) if 0 <= i < row["total"]) + '</nav>'
+    else:
+        body += '<p>The task server is temporarily unavailable. Please try again shortly.</p>'
+    ld = [{"@type": "CreativeWork", "name": title, "description": desc, "url": base_url(request) + path,
+           "isPartOf": {"@type": "SoftwareApplication", "name": spec, "url": base_url(request) + f"/s/{enc(spec)}"}}, cld]
+    return page(request, title=f"{title} · {spec}", description=desc, path=path, body=body, jsonld=ld,
+                image=f"/og/s/{enc(spec)}.png?" + urlencode({"env": env, "split": split, "task": index}),
+                og_type="article", status=status)
 
 
 def simple(request: Request, path: str) -> HTMLResponse:
@@ -310,7 +388,8 @@ def robots(request: Request) -> PlainTextResponse:
     base = base_url(request)
     return PlainTextResponse("\n".join([
         "User-agent: *", "Allow: /", "Disallow: /api/", "Disallow: /mcp/", "Disallow: /capture/", "Disallow: /run/", "Disallow: /runs",
-        "Disallow: /compare/", "Disallow: /login", "Disallow: /logout", "", f"Sitemap: {base}/sitemap.xml", ""]))
+        "Allow: /api/env/", "Allow: /api/spaces/", "Allow: /api/search", "Allow: /api/environments",
+        "Disallow: /api/environments/mine", "Disallow: /compare/", "Disallow: /login", "Disallow: /logout", "", f"Sitemap: {base}/sitemap.xml", ""]))
 
 
 _sitemap: dict[str, Any] = {"at": 0.0, "pages": [], "tasks": []}
@@ -318,29 +397,45 @@ _sitemap: dict[str, Any] = {"at": 0.0, "pages": [], "tasks": []}
 
 def _entries() -> tuple[list[tuple[str, str | None]], list[tuple[str, str | None]]]:
     with _lock:
-        if time.time() - _sitemap["at"] < 3600 and _sitemap["pages"]:
+        if time.time() - _sitemap["at"] < 300 and _sitemap["pages"]:
             return _sitemap["pages"], _sitemap["tasks"]
     by, rows = listing()
+    rows = public_rows(rows)
     pages: list[tuple[str, str | None]] = [("/", None), ("/community", None)]
-    # every dataset; a Space when someone liked it, it runs, or it's featured (thousands are near-copies from hackathons)
+    # Every public dataset and only Spaces with evidence of a supported API.
     pages += [(_href(r), (r.get("updated") or "")[:10] or None) for r in rows
-              if r["kind"] == "dataset" or (r.get("likes") or 0) > 0 or r.get("stage") == "RUNNING" or r.get("collection")]
-    if f"{MIMO}" not in by:
-        pages.append((f"/d/{enc(MIMO)}", None))
+              if discoverable(r)]
     tasks: list[tuple[str, str | None]] = []
-    for spec in [MIMO, *[r["id"] for r in rows if r["kind"] == "dataset" and (r.get("indexed") or {}).get("tasks")]]:
+    for spec in dict.fromkeys(r["id"] for r in rows if r["kind"] == "dataset"):
         try:
             tasks += [(f"/t/{enc(spec)}/{enc(t['path'])}", None) for t in index_rows(spec)]
         except Exception:  # noqa: BLE001 - one index unreadable leaves the rest
             continue
     with _lock:
+        tasks = list(dict.fromkeys(tasks))
         _sitemap.update(at=time.time(), pages=pages, tasks=tasks)
     return pages, tasks
 
 
+def _space_ranges():
+    """Represent task coordinates as ranges; do not allocate millions of URLs or fetch any tasks."""
+    records = space_checks.inventory()
+    _, rows = listing()
+    out = []
+    for r in sorted(public_rows(rows), key=lambda r: r["key"]):
+        if r.get("kind") != "space" or not space_checks.browseable(records.get(r["id"])):
+            continue
+        rec = records[r["id"]]
+        ranges = rec.get("task_splits") or seo_tasks.ranges((spaces_live.last_seen(r["id"]) or {}).get("task_api"))
+        for env, split, n in ranges:
+            if type(n) is int and 0 < n <= 1_000_000_000:
+                out.append((r["id"], env, split, n))
+    return out
+
+
 def _urlset(base: str, items: list[tuple[str, str | None]]) -> Response:
     xml = ['<?xml version="1.0" encoding="UTF-8"?>', '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">']
-    xml += [f"<url><loc>{esc(base + p)}</loc>{f'<lastmod>{esc(m)}</lastmod>' if m else ''}</url>" for p, m in items]
+    xml += [f"<url><loc>{esc(base + p)}</loc>{f'<lastmod>{esc(m)}</lastmod>' if m else ''}</url>" for p, m in items if len((base + p).encode()) <= 2048]
     xml.append("</urlset>")
     return Response("\n".join(xml), media_type="application/xml")
 
@@ -349,6 +444,8 @@ def sitemap_index(request: Request) -> Response:
     base = base_url(request)
     _, tasks = _entries()
     parts = ["/sitemap-pages.xml", *[f"/sitemap-tasks-{i + 1}.xml" for i in range((len(tasks) + SITEMAP_CHUNK - 1) // SITEMAP_CHUNK)]]
+    total = sum(r[3] for r in _space_ranges())
+    parts += [f"/sitemap-space-tasks-{i + 1}.xml" for i in range(min(49000, (total + SITEMAP_CHUNK - 1) // SITEMAP_CHUNK))]
     xml = ['<?xml version="1.0" encoding="UTF-8"?>', '<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">']
     xml += [f"<sitemap><loc>{esc(base + p)}</loc></sitemap>" for p in parts]
     xml.append("</sitemapindex>")
@@ -368,6 +465,23 @@ def sitemap_tasks(request: Request, n: int) -> Response:
     return _urlset(base_url(request), chunk)
 
 
+def sitemap_space_tasks(request: Request, n: int) -> Response:
+    if not 1 <= n <= 49000:
+        return Response("not found", status_code=404)
+    offset, remaining, urls = (n - 1) * SITEMAP_CHUNK, SITEMAP_CHUNK, []
+    for spec, env, split, count in _space_ranges():
+        if offset >= count:
+            offset -= count
+            continue
+        stop = min(count, offset + remaining)
+        urls.extend((seo_tasks.space_path(spec, env, split, i), None) for i in range(offset, stop))
+        remaining -= stop - offset
+        offset = 0
+        if not remaining:
+            break
+    return _urlset(base_url(request), urls) if urls else Response("not found", status_code=404)
+
+
 # ── preview images ───────────────────────────────────────────────────────────
 # 1200×630, plain: the site's name, what the page is, its title and a line of facts. Drawn once per page and kept.
 W, H = 1200, 630
@@ -377,27 +491,41 @@ W, H = 1200, 630
 def card_png(kind: str, title: str, sub: str, facts: str) -> bytes:
     from PIL import Image, ImageDraw, ImageFont
 
-    img = Image.new("RGB", (W, H), "#ffffff")
+    img = Image.new("RGB", (W, H), "#fffdf7")
     d = ImageDraw.Draw(img)
     font = lambda s: ImageFont.load_default(size=s)  # noqa: E731 - Pillow's own font, so no system fonts are needed
-    d.rectangle([0, 0, W, 6], fill="#111827")
-    d.text((72, 70), SITE, font=font(34), fill="#6b7280")
-    d.text((72, 150), kind.upper(), font=font(28), fill="#9ca3af")
-    y = 200
-    for line in _wrap(d, title, font(66), W - 144)[:3]:
-        d.text((72, y), line, font=font(66), fill="#111827")
-        y += 82
+    d.rectangle([0, 0, W, 12], fill="#ffcd36")
+    d.rounded_rectangle([56, 48, 198, 90], radius=10, fill="#ffdc62")
+    d.text((73, 58), "FINEENVS", font=font(23), fill="#352c0e")
+    d.text((220, 57), SITE, font=font(26), fill="#54504a")
+    d.text((56, 142), _ellipsize(d, kind.upper(), font(23), 1088), font=font(23), fill="#827256")
+    lines = _wrap(d, title, font(65), 1088)
+    size = 65 if len(lines) <= 3 else 54
+    lines = _wrap(d, title, font(size), 1088)[:3]
+    for i, line in enumerate(lines):
+        d.text((56, 194 + i * 77), _ellipsize(d, line, font(size), 1088), font=font(size), fill="#1c1b19")
     if sub:
-        d.text((72, y + 16), clip(sub, 70), font=font(32), fill="#4b5563")
-    if facts:
-        d.text((72, H - 104), facts, font=font(30), fill="#374151")
-    d.text((W - 72, H - 104), "RL environments on Hugging Face", font=font(28), fill="#9ca3af", anchor="ra")
+        d.text((56, 452), _ellipsize(d, sub, font(27), 1088), font=font(27), fill="#665e51")
+    d.line([56, 516, 1144, 516], fill="#e5dfd1", width=2)
+    d.text((56, 550), _ellipsize(d, facts or "Discover tasks. Inspect rewards. Run agents.", font(24), 680), font=font(24), fill="#524d42")
+    d.text((1144, 550), "Hugging Face Hub", font=font(24), fill="#827256", anchor="ra")
     out = io.BytesIO()
     img.save(out, "PNG", optimize=True)
     return out.getvalue()
 
 
+def _ellipsize(d, text, font, width):
+    text = str(text)
+    if d.textlength(text, font=font) <= width:
+        return text
+    while text and d.textlength(text + "…", font=font) > width:
+        text = text[:-1]
+    return text + "…"
+
+
 def _wrap(d, text: str, font, width: int) -> list[str]:
+    if "\n" in text:
+        return [line for part in text.splitlines() for line in _wrap(d, part, font, width)]
     words, lines, cur = str(text).split(), [], ""
     for w in words:
         nxt = f"{cur} {w}".strip()
@@ -415,9 +543,14 @@ def _wrap(d, text: str, font, width: int) -> list[str]:
     return lines or [""]
 
 
-def og_image(path: str) -> Response:
+def og_image(path: str, query=None) -> Response:
     """The preview image of a page, from its path (`/d/org/name`, `/t/org/name/ref`, `/s/org/name`, or the site's)."""
+    if path == "/":
+        return Response(card_png("Reinforcement learning", "Explore RL environments\non the Hugging Face Hub",
+                                 "OpenEnv  ·  Harbor  ·  MiMo  ·  NeMo Gym  ·  Verifiers", ""),
+                        media_type="image/png", headers={"Cache-Control": "public, max-age=86400"})
     by, _ = listing()
+    by = {r["key"]: r for r in public_rows(list(by.values()))}
     parts = [p for p in path.strip("/").split("/") if p]
     if parts and not (len(parts) >= 3 and parts[0] in ("d", "s", "t")):
         return Response(status_code=404)
@@ -425,14 +558,30 @@ def og_image(path: str) -> Response:
     if len(parts) >= 3 and parts[0] in ("d", "s", "t"):
         spec = f"{parts[1]}/{parts[2]}"
         r = by.get(spec if parts[0] != "s" else f"space:{spec}")
-        if r is None and spec != MIMO:   # nothing listed there: no image (its page points at the site's), no work done
+        if r is None:
             return Response(status_code=404)
         name = display_name(r, spec)
         k = kind_of(r, spec)
         if parts[0] == "t" and (r or spec == MIMO):
             ref = "/".join(parts[3:])
             row = next((t for t in index_rows(spec) if t["path"] == ref), None)
+            if not row and re.fullmatch(r"(?:[^/]+/){1,2}\d{1,10}", ref):
+                try:
+                    row = seo_tasks.row(spec, ref)
+                except seo_tasks.Unavailable as exc:
+                    return Response(status_code=exc.status)
+            if not row:
+                return Response(status_code=404)
             kind, title, sub = f"A task in {name}", clip((row or {}).get("title") or ref, 140), spec
+        elif parts[0] == "s" and query and "task" in query:
+            raw, env, split = query.get("task", ""), query.get("env", ""), query.get("split", "")
+            if not re.fullmatch(r"\d{1,10}", raw) or not env or not split or len(env) > 80 or len(split) > 200:
+                return Response(status_code=404)
+            try:
+                row = seo_tasks.space(spec, env, split, int(raw))
+            except Exception:
+                return Response(status_code=503, headers={"Retry-After": "60"})
+            kind, title, sub = f"{split} task", clip(row["title"], 140), spec
         else:
             n = ((r or {}).get("indexed") or {}).get("tasks") or (len(index_rows(spec)) if spec == MIMO else None)
             kind, title, sub = k, name, spec

app/seo_tasks.py

@@ -0,0 +1,115 @@
+"""Anonymous, bounded task reads for server-rendered public task pages.
+
+No episode, index build, credentials or model call. Only advertised Task API
+coordinates and public row datasets may be read. Answers use the UI's withholding.
+"""
+from __future__ import annotations
+
+import re
+import threading
+from urllib.parse import urlencode
+
+from . import catalog, spaces_live
+
+_reads = threading.BoundedSemaphore(4)
+
+
+class Unavailable(Exception):
+    def __init__(self, status=503):
+        self.status = status
+
+
+def read(key, fn):
+    def fetch():
+        if not _reads.acquire(blocking=False):
+            raise Unavailable()
+        try:
+            return fn()
+        finally:
+            _reads.release()
+    return catalog._cached(("seo-task", *key), 600, fetch)
+
+
+def space_path(spec, env, split, index):
+    return f"/s/{spec}?" + urlencode({"env": env, "split": split, "task": index})
+
+
+def ranges(task_api):
+    if not isinstance(task_api, dict):
+        return []
+    result, seen = [], set()
+    for e in (task_api.get("environments") or [task_api])[:16]:
+        name = e.get("env")
+        if not isinstance(name, str) or not re.fullmatch(r"[\w.-]{1,80}", name) or name in (".", ".."):
+            continue
+        for s in e.get("splits", [])[:40]:
+            split, n = s.get("name"), s.get("num_tasks")
+            if not isinstance(split, str) or not 0 < len(split) <= 200 or type(n) is not int or not 0 < n <= 1_000_000_000:
+                continue
+            if (name, split) not in seen:
+                result.append([name, split, n])
+                seen.add((name, split))
+    return result
+
+
+def space(spec, env, split, index):
+    """Validate against the last public probe before making a read-only task request."""
+    known = ranges((spaces_live.last_seen(spec) or {}).get("task_api"))
+    if not known:
+        known = ranges(spaces_live.probe(spec).get("task_api"))
+    match = next((r for r in known if r[0] == env and r[1] == split), None)
+    if not match or not 0 <= index < match[2]:
+        raise Unavailable(404)
+
+    def fetch():
+        try:
+            raw = spaces_live.task(spec, split, index, env).get("task")
+        except spaces_live.SpaceError as exc:
+            raise Unavailable(404 if exc.status == 404 else 503) from None
+        if not isinstance(raw, dict):
+            raise Unavailable(404)
+        # Explicit text fields only; do not serialize arbitrary task metadata.
+        title = next((raw[k] for k in ("task_name", "title", "task_id", "id") if isinstance(raw.get(k), str)), f"Task {index + 1}")
+        prompt = next((raw[k] for k in ("prompt", "instruction", "description", "question") if isinstance(raw.get(k), str)), "")
+        fields = {k: raw[k] for k in ("category", "difficulty", "language", "language_name", "family", "mime", "duration_seconds",
+                                     "sampling_rate", "n_frames", "provider", "sequence_id", "offline_ready", "media_ready")
+                  if isinstance(raw.get(k), (str, int, float, bool))}
+        return {"title": title[:400], "brief": prompt[:16000], "total": match[2], "fields": fields}
+    return read(("space", spec, env, split, index), fetch)
+
+
+def row(spec, ref):
+    from .envs import rows
+
+    try:
+        config, split, index = rows.parse_ref(ref)
+    except (ValueError, LookupError):
+        raise Unavailable(404) from None
+    if index < 0:
+        raise Unavailable(404)
+
+    def fetch():
+        try:
+            view = rows.task(spec, config, split, index, token=None)
+        except (LookupError, PermissionError):
+            raise Unavailable(404) from None
+        except Exception:
+            raise Unavailable() from None
+        if view.get("restricted"):
+            raise Unavailable(404)
+        texts = []
+        for section in view.get("sections", []):
+            if section.get("id") not in ("task", "prompt", "messages", "instruction"):
+                continue
+            body = section.get("body")
+            if isinstance(body, str):
+                texts.append(body)
+            elif section.get("kind") == "messages" and isinstance(body, list):
+                texts.extend(m["content"] for m in body if isinstance(m, dict)
+                             and m.get("role") in ("system", "user") and isinstance(m.get("content"), str))
+            elif section.get("kind") == "blocks" and isinstance(body, list):
+                texts.extend(b["text"] for b in body if isinstance(b, dict)
+                             and b.get("type") in ("markdown", "custom", "note") and isinstance(b.get("text"), str))
+        return {"title": str(view.get("title") or ref)[:400], "brief": "\n\n".join(texts)[:16000],
+                "path": ref, "total": view.get("total")}
+    return read(("row", spec, ref), fetch)

app/space_checks.py

@@ -219,6 +219,17 @@ def assess(api, health, metadata, schema, tools):
             "failed": [k for k, ok in checks.items() if not ok], "tools": len(tools or [])}
 
 
+def _task_splits(task_api):
+    from .seo_tasks import ranges
+
+    out = []
+    for row in ranges(task_api):
+        if len(json.dumps(out + [row]).encode()) > 12000:
+            break
+        out.append(row)
+    return out
+
+
 def check(spec):
     from . import spaces_live as live
     rec = {"schema": 1, "id": catalog.check_spec(spec), "checked_at": time.time(), "status": "Check unavailable"}
@@ -249,6 +260,7 @@ def check(spec):
             from . import space_tasks
             task_api = space_tasks.discover(hub, paths)
             rec["task_catalog"] = space_tasks.summary(task_api)
+            rec["task_splits"] = _task_splits(task_api)
             if rec["status"] == PASS:
                 rec["version"] = declared_version(spec, hub.get("tags"))
             elif rec["version"]["value"] == "Unknown":
@@ -273,6 +285,7 @@ def observe(spec, info):
             rec["interface"] = environment_interface({"info": {"version": info.get("openapi_version")}, "paths": info.get("endpoint_methods")})
             from . import space_tasks
             rec["task_catalog"] = space_tasks.summary(info.get("task_api"))
+            rec["task_splits"] = _task_splits(info.get("task_api"))
         info["openenv_verified"] = verified(rec)
         save(rec)
     except Exception:

web/index.html

@@ -3,7 +3,7 @@
 <head>
   <meta charset="utf-8">
   <meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
-  <title>HF RL Explorer: Explore RL environments on Hugging Face</title>
+  <title>HF RL Explorer: RL environments on the Hugging Face Hub</title>
   <link rel="icon" href="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" type="image/svg+xml">
   <meta name="description" content="Explore reinforcement learning environments on Hugging Face across OpenEnv, Harbor, Verifiers, NeMo Gym and more: what each task asks, how it's graded, what it runs in, and run an agent on it.">
   <meta name="application-name" content="HF RL Explorer">

web/js/seo-meta.js

@@ -0,0 +1,17 @@
+export const SITE = "HF RL Explorer";
+export const DESCRIPTION = "Explore reinforcement learning environments and tasks on the Hugging Face Hub. Browse OpenEnv, Harbor, MiMo, NeMo Gym and Verifiers, inspect rewards, and run supported agent rollouts.";
+
+// A Space task is a distinct resource. Ignore filters and tracking parameters,
+// but preserve its environment, split and numeric task index in a stable order.
+export function canonicalPath(url) {
+  const u = new URL(url), q = u.searchParams;
+  if (/^\/s\/[^/]+\/[^/]+$/.test(u.pathname) && /^\d{1,10}$/.test(q.get("task") || "") && q.get("env") && q.get("split")) {
+    return u.pathname + "?" + new URLSearchParams({ env: q.get("env"), split: q.get("split"), task: String(Number(q.get("task"))) });
+  }
+  return u.pathname;
+}
+
+export function imagePath(url) {
+  const path = canonicalPath(url), [name, query] = path.split("?");
+  return /^\/(d|t|s)\//.test(name) ? `/og${name}.png${query ? `?${query}` : ""}` : "/social/rl-explorer.png";
+}

web/js/space.js

@@ -782,14 +782,17 @@ function render(me) {
   if (!s) return;
   const [org, name] = spec.split("/");
   const sections = sectionsOf(me);
+  const linkedIndex = new URLSearchParams(location.search).get("task");
+  const taskView = /^\d{1,10}$/.test(linkedIndex || "");
   const kind = me.L?.openenv_verified ? "OpenEnv Space" : s.declared_openenv || s.openenv || s.manifest ? "Unverified Space" : s.framework === "ors" ? "ORS Space" : "environment Space";   // as the server-rendered title says
-  setMeta({ title: `${s.heading || name} · ${kind}`, description: `${spec}: an RL environment Space on Hugging Face. ${describe(me)} See it live: its app, a playground with rewards, its tasks, and MCP for coding agents.` });
+  if (!new URLSearchParams(location.search).has("task")) setMeta({ title: `${s.heading || name} · ${kind}`, description: `${spec}: an RL environment Space on Hugging Face. ${describe(me)} See it live: its app, a playground with rewards, its tasks, and MCP for coding agents.` });
   me.viewer?.destroy();
   me.viewer = null;
   el.innerHTML = `<div class="wrap page${me.rendered ? "" : " fade-in"}">
     <nav class="crumbs" aria-label="Breadcrumb"><a href="/">Environments</a>${icon("chevronRight", 13)}<a href="/?k=space">Spaces</a>${icon("chevronRight", 13)}<span>${esc(name)}</span></nav>
     <header class="tp-head ds-head">
-      <h1>${ownerLink(org, "/")}${esc(name)}</h1>
+      <h1>${taskView ? `Task ${Number(linkedIndex) + 1}` : `${ownerLink(org, "/")}${esc(name)}`}</h1>
+      ${taskView ? `<p><a href="/s/${enc(spec)}">All tasks in ${esc(spec)}</a></p>` : ""}
       <p class="lede" id="sp-lede">${esc(describe(me))}</p>
       <div class="facts sp-facts"><div class="sp-fi" id="sp-facts">${factsLine(me)}</div></div>
       <div class="facts sp-links" id="sp-links">${linksLine(me)}</div>
@@ -1371,16 +1374,27 @@ async function openTask(me, i) {
       <button class="icon-btn sm" type="button" data-task-move="-1" aria-label="Previous task in this page" ${i === 0 ? "disabled" : ""}>${icon("chevronRight", 15, "flip")}</button>
       <button class="icon-btn sm" type="button" data-task-move="1" aria-label="Next task in this page" ${i + 1 === me.tasks.length ? "disabled" : ""}>${icon("chevronRight", 15)}</button>
       <button class="icon-btn sm" type="button" id="tk-close" aria-label="Close task">${icon("x", 15)}</button></div>
-    <div class="tk-actions">${copyBtn(link.href, "Copy task link")}
+    <div class="tk-actions"><a class="btn sm" href="${esc(link.href)}">Open task page</a>${copyBtn(link.href, "Copy task link")}
       ${rel ? `<a class="btn sm" href="/t/${enc(hub)}/${enc(rel)}" title="Open it in the explorer, where it runs on your account">${icon("database", 13)}Open in the explorer</a>` : ""}
       ${canPlay ? `<button class="btn sm primary" type="button" data-play-task="${esc(idx)}">${icon("play", 13)}Start this task</button>` : ""}</div>
     <div class="tk-detail-b">${taskContent(t)}<p class="fine" id="tk-full-status" role="status">Loading the full task…</p></div>`;
   d.scrollIntoView({ block: "nearest", behavior: "smooth" });
+  const updateTaskMeta = (task) => {
+    const q = new URLSearchParams(location.search);
+    if (q.get("task") === String(idx) && q.get("env") === me.taskPage.env && q.get("split") === me.taskPage.split) {
+      const prompt = [task.prompt, task.instruction, task.description, task.question].find((v) => typeof v === "string") || "";
+      setMeta({ title: `${title} · ${me.spec}`, description: `${title}: ${me.taskPage.split} task in ${me.spec}, an RL environment on the Hugging Face Hub. ${prompt}` });
+      const heading = $(".tp-head h1", me.el);
+      if (heading) heading.textContent = title;
+    }
+  };
+  updateTaskMeta(t);
   try {
     const full = await api(`/api/spaces/${enc(me.spec)}/task?${new URLSearchParams({ env: me.taskPage.env, split: me.taskPage.split, index: idx })}`);
     if (page !== me || requestKey !== me.taskRequest || me.selectedTask !== i || !d.isConnected || d.hidden) return;
     if (full.task && typeof full.task === "object" && !Array.isArray(full.task)) {
       $(".tk-detail-b", d).innerHTML = taskContent(full.task);
+      updateTaskMeta(full.task);
     } else $("#tk-full-status", d).textContent = "Showing the task record returned by the list endpoint.";
   } catch (error) {
     if (page === me && requestKey === me.taskRequest && me.selectedTask === i && d.isConnected && !d.hidden)

web/js/util.js

@@ -1,14 +1,35 @@
 // Shared helpers: DOM, formatting, API, markdown, modal, loading states.
 import { icon, FILE_ICON } from "./icons.js";
+import { SITE, DESCRIPTION, canonicalPath, imagePath } from "./seo-meta.js";
 
 export const $ = (s, el = document) => el.querySelector(s);
 // the page's title, description and canonical address as you move around (the server writes them for a first load)
-export function setMeta({ title, description } = {}) {
-  const site = "HF RL Explorer";
-  document.title = !title ? `${site}: Explore RL environments on Hugging Face` : title.includes(site) ? title : `${title} · ${site}`;
-  if (description) document.querySelector('meta[name="description"]')?.setAttribute("content", description.slice(0, 300));
+export function setMeta({ title, description, indexable } = {}) {
+  const url = location.origin + canonicalPath(location.href);
+  const rendered = document.querySelector('meta[name="rlx-page-url"]');
+  const changed = rendered?.content !== url;
+  if (changed) {
+    document.querySelectorAll('script[type="application/ld+json"]').forEach((s) => s.remove());
+    if (rendered) rendered.content = url;
+  }
+  if (title || changed) document.title = !title ? `${SITE}: RL environments on the Hugging Face Hub` : title.includes(SITE) ? title : `${title} · ${SITE}`;
+  const desc = (description || (changed ? DESCRIPTION : document.querySelector('meta[name="description"]')?.content) || DESCRIPTION).slice(0, 300);
+  const set = (selector, value) => document.querySelector(selector)?.setAttribute("content", value);
+  set('meta[name="description"]', desc);
+  for (const key of ["og:title", "twitter:title"]) set(`meta[${key.startsWith("og:") ? "property" : "name"}="${key}"]`, document.title);
+  set('meta[property="og:description"]', desc);
+  set('meta[name="twitter:description"]', desc);
+  set('meta[property="og:url"]', url);
+  if (changed) {
+    const image = location.origin + imagePath(location.href);
+    set('meta[property="og:image"]', image);
+    set('meta[name="twitter:image"]', image);
+    set('meta[property="og:type"]', /^\/t\//.test(location.pathname) || location.search.includes("task=") ? "article" : "website");
+  }
+  if (changed || indexable !== undefined) set('meta[name="robots"]', indexable === false || /^\/(runs|run\/|compare\/)/.test(location.pathname)
+    ? "noindex, follow" : "index, follow, max-image-preview:large, max-snippet:-1");
   const canon = document.querySelector('link[rel="canonical"]');
-  if (canon) canon.href = location.origin + location.pathname;
+  if (canon) canon.href = url;
 }
 
 // go to another page of the app: a real path (/d/org/name, /t/...), so every page has its own address for search

web/rlx.css

@@ -395,6 +395,7 @@ body[data-view="home"] .top-search { visibility: hidden; }
 html:not(.js) #view > .boot { display: none; }
 .ssr ul { padding-left: 18px; display: grid; gap: 6px; }
 .ssr .crumbs { margin-bottom: 12px; }
+.seo-prompt { white-space: pre-wrap; overflow-wrap: anywhere; max-width: 85ch; line-height: 1.65; }
 
 /* ── RL framework rows (renderers/rl.js): NeMo Gym, Verifiers, verl / SkyRL ───────────────────────────────────────
    A conversation as the policy gets it and the tools it may call. Nothing scrolls inside the page: long parts fold. */