Serve the PostTrain console at /dashboard, read-only, with the example projects.

The console (apps/console of benchflow-ai/pta-dash at 74fb9a9) replaces the previous dashboard, which stays at /dashboard/previous. It serves the example projects built from published post-training programs; there is no workspace and no write API here (POSTTRAIN_READONLY=1), and the Docker build generates the example database. The submission board, /board, the relay and every existing API are unchanged.

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.woff2 filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -1,2 +1,7 @@
 __pycache__/
 *.pyc
+
+# the console builds its example database at image build time
+viewer/data/*.sqlite
+viewer/data/*.sqlite-*
+viewer/data/*.building

Dockerfile

@@ -3,6 +3,7 @@ RUN useradd -m -u 1000 user
 WORKDIR /app
 COPY --chown=user:user . /app
 RUN pip install --no-cache-dir fastapi==0.141.1 uvicorn==0.53.0 websockets==15.0.1 huggingface_hub==1.32.0 tomli-w==1.2.0 './pipeline[hf]'
+RUN python -m viewer.build && chown -R user:user /app/viewer/data   # the console's example database
 USER user
 ENV HOME=/home/user
 EXPOSE 7860

app.py

@@ -123,8 +123,8 @@ class Selection(BaseModel):
     rank: int = 16
 @app.get('/')
 def home(): return FileResponse(ROOT/'index.html')   # the submission board: what participants use
-@app.get('/dashboard', include_in_schema=False)
-def dashboard_page(): return FileResponse(ROOT/'dashboard.html')   # the organizers' and researchers' dashboard
+@app.get('/dashboard/previous', include_in_schema=False)
+def dashboard_page(): return FileResponse(ROOT/'dashboard.html')   # the previous organizers' dashboard, kept reachable
 DASHBOARD_API = {'title': 'PostTrain API', 'version': '2.0.0', 'description': 'The routes the PostTrain dashboard reads: projects, training runs and their records, evaluations, datasets, jobs, the model registry, deployments, inference, usage, reports and Ari\'s findings. Every route is a read-only GET.'}
 def dashboard_routes():
     """The routes the dashboard reads (results_api, run_api, the run records under /api/app/fwruns, sign-in state and the build), and no others."""
@@ -151,6 +151,15 @@ def dashboard_openapi():
 def dashboard_docs():
     from fastapi.openapi.docs import get_swagger_ui_html
     return get_swagger_ui_html(openapi_url='/dashboard/openapi.json', title='PostTrain API')
+# The PostTrain console at /dashboard (API under /api/v3), read-only here: the example projects built from
+# published post-training programs, no workspace and no write API (viewer/README in benchflow-ai/pta-dash apps/console).
+# Registered after /dashboard/openapi.json, /dashboard/docs and /dashboard/previous so its page catch-all doesn't shadow them.
+os.environ.setdefault('POSTTRAIN_READONLY', '1')
+import viewer.server as viewer_server
+import viewer.api_ui as viewer_ui
+app.include_router(viewer_server.router)
+app.include_router(viewer_ui.router)
+viewer_server.mount_static(app)
 @app.get('/board', include_in_schema=False)
 def board_page(): return FileResponse(ROOT/'board.html')
 @app.get('/icon.svg', include_in_schema=False)

viewer/DESIGN.md

@@ -0,0 +1,22 @@
+# Viewer design
+
+A console for post-training teams: the runs they train, the data and environments those runs learn from, and the held-out evals that say whether it worked. It is built from what those teams have to decide, not from another product's layout.
+
+## The questions, in the order people ask them
+
+1. **What is running, and does anything need me?** Project overview: runs with their progress, training curve and held-out change; a list of failing checks (from the runs' own metrics and task validation); recent restarts, notices and completions.
+2. **Is this run learning?** Run page, first screen: the training curve with restarts and notices marked, and one small chart per held-out benchmark (score ± SE by step) beside it.
+3. **Is it healthy?** The checks panel explains in one sentence each why a signal is fine or not (entropy, gradient spikes, trainer–sampler KL, truncation, groups with no learning signal, infrastructure errors, staleness). Below it, the signals themselves as small charts, each named in words with the framework's own tag underneath.
+4. **Where does the signal come from?** "By environment": batch share, pass rate first → last step, change, infrastructure errors, per data source.
+5. **What is the model actually doing?** Steps expand to their stored groups; a group shows every attempt at one task as a row of cells coloured by outcome; any cell opens the rollout: where it sits (run, step, task, attempt i/n), why its advantage is what it is, the score with the rule that produced it, and the transcript with collapsible tool calls and search.
+6. **Did the evals move beyond noise?** Evals page: a matrix of benchmarks × models with standard errors, curves during training, and per-eval pages with the distribution of per-task results and a task-by-task comparison with any other eval (gained, lost, still solved, still unsolved).
+7. **Are the environments and data any good?** Environments list every task with its validation result (oracle, no-op, rerun agreement, hack probe, overlap with evals) and its pass rate under the base and the latest policy, so too-easy and too-hard tasks are visible before they waste a step. Datasets show composition, processing funnel, sample rows and the runs that used them.
+8. **What did it cost, and what was learned?** Jobs (every attempt of every workload), usage by day and run, and reports written as claims with verdicts.
+
+## Rules the pages follow
+
+- Every number has its denominator (tasks × attempts, stored of total) and every score its standard error.
+- Infrastructure failures are shown and excluded, never scored as zero.
+- Colour only encodes meaning: outcome and status (always with an icon or label), train vs held-out, and run identity when runs are compared. Everything else is ink on white with hairlines.
+- Data comes from one SQLite protocol (`PROTOCOL.md`) whether it is the demo world or our own runs, and published numbers are labelled apart from simulated ones.
+- Every object links to the objects it came from: run → inputs → tasks → attempts; eval → model → run that produced it.

viewer/PROTOCOL.md

@@ -0,0 +1,57 @@
+# Viewer data protocol
+
+The viewer reads one SQLite database at a time. `data/demo.sqlite` is built by `python -m viewer.build` from public post-training recipes; `data/live.sqlite` is built from our own runs. Both follow this protocol, and the API and pages read them through the same code, so anything the demo shows, a live source can fill.
+
+Every record answers one question a post-training team asks. The tables below say which.
+
+## Scope
+
+- **org**: a team (a lab). **project**: one model program inside it, such as "MiMo-V2.6 RL" or "Nemotron 3 Nano post-training". Every other record belongs to a project.
+
+## Models
+
+- **models**: base models, intermediate checkpoints that were promoted, teachers, reward models and judges. `parent_id` + `run_id` give lineage: which run turned which model into which. Answers "what did each stage produce, and from what?"
+- **checkpoints**: every saved step of a run, with its eval summary. Answers "which step do we ship?"
+
+## Data
+
+- **datasets**: SFT, preference, RL prompt pools and eval sets, versioned (`parent_id` points at the version it was derived from). `processing` is the funnel from raw to final (`[{step, rows_in, rows_out, note}]`).
+- **dataset_sources**: the composition: each upstream source with category, rows, tokens, whether it is synthetic and which model generated it, and its license.
+- **dataset_rows**: a sample of rows for reading.
+
+## Environments
+
+- **environments**: an RL environment: domain, harness (the agent loop), tools, sandbox, grader, reward kind (binary, partial, scalar).
+- **graders**: how a rollout becomes a reward: components with weights and the rule for each, and the formula that combines them.
+- **tasks**: every task of an environment with its validation result (oracle score, no-op score, agreement across reruns), status (`ok`, `flaky`, `invalid`, `leaky`, `hackable`, `too_easy`, `too_hard`, `excluded`) with a reason, and pass rates for the base model and the latest policy.
+
+## Runs
+
+- **runs**: one training run of any kind (`sft`, `dpo`, `rl`, `distill`, `rm`). Holds the recipe (`algorithm`, `framework`, `hyperparams`, raw `config`), the models in and out, state, progress, compute and cost, and `provenance` (`published`, `simulated` or `mixed`).
+- **run_inputs**: datasets and environments a run trained on, with their weight in the mix.
+- **metrics**: every logged scalar, `(run, tag, step) → value`. Tag names are the framework's own (`actor/entropy_loss`, `dynsam/avg@n`, `train/loss`); **metric_defs** gives each tag a label, a description, a unit, which direction is better, and whether it is pinned.
+- **run_steps**: per-step facts derived from rollouts: prompts, rollouts, how many were stored, mean reward, pass rate, groups where every attempt passed / failed / differed, infrastructure errors, truncations.
+- **run_events**: restarts, notices, checkpoints, data changes, config changes and alerts, placed on the run's timeline.
+
+## Rollouts
+
+- **rollouts**: one attempt at one task by one policy version: step, group and sample index, task and environment, harness, reward, advantage, score components, outcome (`passed`, `failed`, `partial`, `infra_error`, `timeout`, `truncated`), stop reason, turns, tool calls, tokens (in, out, cached), time by phase, staleness (policy versions between sampling and training) and flags (`possible_leak`, `reward_hack_suspect`). Training rollouts carry `run_id`; evaluation rollouts carry `eval_id`. Large runs store a sample; `run_steps.rollouts_stored` says how many.
+- **transcripts**: the messages of a rollout when a source stores them. When absent, the demo source renders a transcript from the rollout's own fields and seed, so it always agrees with the stored reward, turns and stop reason.
+
+## Evals
+
+- **benchmarks**: a held-out suite: version, harness, metric (`avg@3`, `pass@1`), tasks and attempts per task.
+- **evals**: one model (or run step) on one benchmark: score with its standard error, tasks × attempts, infrastructure errors excluded from the score, the recorded command and config, status.
+- **eval_tasks**: per-task results of an eval, so two evals can be compared task by task (gained, lost, still passing, still failing).
+
+## Operations
+
+- **jobs**: every workload (train, rollout workers, eval, data processing, validation, serving) with cluster, GPUs, state, cost and the tail of its log.
+- **clusters**, **usage** (spend per day and category), **deployments** (a model served as an endpoint).
+- **reports**: written findings about runs, as claims with a verdict (`upheld`, `rejected`, `open`) and evidence links.
+
+## Rules
+
+- Numbers carry their denominators: a pass rate comes with tasks × attempts; a score with its standard error.
+- Infrastructure failures are never scored as zero: they are counted separately and excluded.
+- Every published number keeps its source URL; simulated records say so through `provenance`.

viewer/api_ui.py

@@ -0,0 +1,457 @@
+"""Read endpoints the website needs to do work, not only to look at it: where a project stands stage
+by stage, what a launch can use, and the state of the jobs behind queued and running runs.
+
+Writes stay in api_write.py; these only read. Runs, datasets and the rest are read from the source
+that holds the project (`?source=`); jobs, runners and compute targets always live in the workspace.
+"""
+import shlex
+import time
+
+from fastapi import APIRouter, HTTPException
+
+from . import api_write, db, workspace
+
+router = APIRouter(prefix="/api/v3")
+ONLINE = 90          # seconds since a runner's last heartbeat, the same window as api_write.list_compute
+ACTIVE_RUN = {"queued", "starting", "running", "stopping", "stalled"}
+ACTIVE_JOB = ("queued", "starting", "running")
+STAGES = ["data", "environments", "sft", "preference", "rl", "eval", "deploy"]
+RUN_KIND = {"sft": "sft", "preference": "dpo", "rl": "rl"}
+FEEDS = {"data": "sft", "environments": "rl"}   # an input counts as skipped only once the stage it feeds is passed
+
+
+def conn_for(source):
+    try:
+        return db.connect(source or "workspace")
+    except (KeyError, FileNotFoundError):
+        raise HTTPException(404, f"Data source '{source}' is not available.")
+
+
+def project_or_404(c, org, project):
+    p = db.one(c, "SELECT p.*, o.slug AS org_slug, o.name AS org_name FROM projects p JOIN orgs o ON o.id=p.org_id "
+                  "WHERE o.slug=? AND p.slug=?", (org, project))
+    if not p:
+        raise HTTPException(404, "No such project.")
+    return p
+
+
+def q(v):
+    """Shell-quote a value for a CLI line; <placeholders> stay as they are."""
+    s = str(v)
+    return s if s.startswith("<") and s.endswith(">") else shlex.quote(s)
+
+
+def compute_state(org):
+    """The org's compute targets (plus the built-in `local`) with runners online and jobs waiting, and its runners."""
+    c = workspace.connect()
+    o = db.one(c, "SELECT id FROM orgs WHERE slug=?", (org,))
+    if not o:
+        return {"targets": [{"name": "local", "kind": "local", "config": {}, "builtin": True, "runners": [], "queued": 0, "active": 0}],
+                "runners": []}
+    listed = api_write.list_compute(org)
+    targets, runners = listed["targets"], listed["runners"]
+    counts = {}
+    for row in c.execute("SELECT j.target, j.status, count(*) FROM jobs j JOIN projects p ON p.id=j.project_id "
+                         "WHERE p.org_id=? AND j.status IN ('queued','starting','running') GROUP BY j.target, j.status", (o["id"],)):
+        counts.setdefault(row[0], {})[row[1]] = row[2]
+    if not any(t["name"] == "local" for t in targets):
+        now = time.time()
+        targets.insert(0, {"name": "local", "kind": "local", "config": {}, "builtin": True,
+                           "runners": [r["name"] for r in runners if "local" in (r.get("targets") or []) and now - (r["last_seen"] or 0) < ONLINE]})
+    for t in targets:
+        n = counts.get(t["name"], {})
+        t["queued"] = n.get("queued", 0)
+        t["active"] = n.get("starting", 0) + n.get("running", 0)
+    return {"targets": targets, "runners": runners}
+
+
+# ------------------------------------------------------------------ environment readiness
+
+NOT_USABLE = ("invalid", "leaky", "flaky", "hackable", "excluded")
+
+
+def env_readiness(c, env_ids):
+    """PRD 4.2, from the stored task results: an environment is ready when validation ran, at least 16
+    tasks are usable, a difficulty profile exists, and at least one usable task is learnable."""
+    if not env_ids:
+        return {}
+    q = ",".join("?" for _ in env_ids)
+    bad = ",".join(f"'{x}'" for x in NOT_USABLE)
+    out = {}
+    for r in db.rows(c, f"SELECT env_id, count(*) AS n, sum(oracle_score IS NOT NULL) AS validated, sum(status NOT IN ({bad})) AS usable, "
+                        f"sum(base_pass IS NOT NULL) AS profiled, sum(status NOT IN ({bad}) AND base_pass > 0.02 AND base_pass < 0.98) AS learnable, "
+                        f"sum(status='flaky') AS flaky FROM tasks WHERE env_id IN ({q}) GROUP BY env_id", env_ids):
+        usable, learnable = r["usable"] or 0, r["learnable"] or 0
+        reason = ("not validated" if not r["validated"] else f"{usable} usable tasks (needs 16)" if usable < 16
+                  else "no difficulty profile" if not r["profiled"] else "no learnable task" if not learnable else None)
+        out[r["env_id"]] = {"tasks": r["n"], "usable": usable, "learnable": learnable, "profiled": r["profiled"] or 0,
+                            "flaky": r["flaky"] or 0, "ready": reason is None, "reason": reason}
+    for e in env_ids:
+        out.setdefault(e, {"tasks": 0, "usable": 0, "learnable": 0, "profiled": 0, "flaky": 0, "ready": False, "reason": "no tasks"})
+    return out
+
+
+# ------------------------------------------------------------------ stages
+
+@router.get("/p/{org}/{project}/stages")
+def stages(org: str, project: str, source: str = "workspace"):
+    """The Overview's Stages table (PRD 6.3): each stage's status (not started, in progress, blocked, done,
+    skipped), what exists, and the next action with its CLI line. Done criteria follow PRD section 4 as far as
+    the workspace records them; promotion, suites and acknowledgements are approximated (see `approx`)."""
+    c = conn_for(source)
+    p = project_or_404(c, org, project)
+    pid = p["id"]
+    datasets = db.rows(c, "SELECT id, name, kind, rows, tokens, created_at FROM datasets WHERE project_id=? ORDER BY created_at DESC", (pid,))
+    envs = db.rows(c, "SELECT id, name, domain, task_count, created_at FROM environments WHERE project_id=? ORDER BY created_at DESC", (pid,))
+    ready = env_readiness(c, [e["id"] for e in envs])
+    runs = db.rows(c, "SELECT r.id, r.name, r.kind, r.status, r.status_reason, r.framework, r.algorithm, r.steps_done, r.steps_planned, "
+                      "r.started_at, r.updated_at, r.ended_at, r.base_model_id, r.output_model_id, m.name AS output_model, b.name AS base_model "
+                      "FROM runs r LEFT JOIN models m ON m.id=r.output_model_id LEFT JOIN models b ON b.id=r.base_model_id "
+                      "WHERE r.project_id=? ORDER BY coalesce(r.updated_at, r.started_at) DESC", (pid,))
+    evals = db.rows(c, "SELECT e.id, e.status, e.score, e.stderr, e.step, e.n_tasks, e.k, e.started_at, e.model_id, e.benchmark_id, "
+                       "b.name AS benchmark, b.metric, m.name AS model_name, r.name AS run_name FROM evals e JOIN benchmarks b ON b.id=e.benchmark_id "
+                       "LEFT JOIN models m ON m.id=e.model_id LEFT JOIN runs r ON r.id=e.run_id WHERE e.project_id=? "
+                       "ORDER BY e.started_at DESC", (pid,))
+    deps = db.rows(c, "SELECT d.id, d.name, d.status, d.endpoint, d.created_at, m.name AS model_name FROM deployments d "
+                      "LEFT JOIN models m ON m.id=d.model_id WHERE d.project_id=? ORDER BY d.created_at DESC", (pid,))
+    models = db.rows(c, "SELECT id, name, kind, hf_repo, created_at FROM models WHERE project_id=? ORDER BY created_at DESC", (pid,))
+    benches = [r["name"] for r in db.rows(c, "SELECT name FROM benchmarks WHERE project_id=? ORDER BY name", (pid,))]
+    evaluated = {}                                  # model id -> benchmark ids with a completed eval
+    for e in evals:
+        if e["status"] == "completed" and e["model_id"]:
+            evaluated.setdefault(e["model_id"], set()).add(e["benchmark_id"])
+
+    comp = compute_state(org) if source == "workspace" else {"targets": [], "runners": []}
+    served = [t["name"] for t in comp["targets"] if t.get("runners")]
+    target = served[0] if served else next((t["name"] for t in comp["targets"] if not t.get("builtin")), "local")
+    base = next((m["hf_repo"] or m["name"] for m in models if m["kind"] == "base"), "<model>")
+    bench = benches[0] if benches else "gsm8k"
+
+    def run_obj(r):
+        return {"type": "run", "id": r["id"], "name": r["name"], "status": r["status"], "reason": r["status_reason"],
+                "framework": r["framework"], "algorithm": r["algorithm"], "steps_done": r["steps_done"], "steps_planned": r["steps_planned"],
+                "output_model": r["output_model"], "evaluated": bool(r["output_model_id"] and evaluated.get(r["output_model_id"]))}
+
+    def act(label, cli, kind, arg=None):
+        return {"label": label, "cli": cli, "kind": kind, "arg": arg}
+
+    rows = {}
+    # data
+    by_kind = {}
+    for d in datasets:
+        by_kind.setdefault(d["kind"], []).append(d)
+    rows["data"] = {"status": "done" if datasets else "not_started",
+                    "objects": [{"type": "dataset", "id": d["id"], "name": d["name"], "kind": d["kind"], "rows": d["rows"]} for d in datasets],
+                    "next": None if datasets else act("Add dataset", "posttrain data add ./train.jsonl --kind sft", "add", "dataset")}
+    # environments
+    env_objs = [{"type": "environment", "id": e["id"], "name": e["name"], "domain": e["domain"], **ready[e["id"]]} for e in envs]
+    unready = [e for e in env_objs if not e["ready"]]
+    rows["environments"] = {
+        "status": "not_started" if not envs else "in_progress" if unready else "done", "objects": env_objs,
+        "next": act("Add environment", "posttrain env add ./my-env", "add", "environment") if not envs
+        else act("Validate", f"posttrain env validate {q(unready[0]['name'])} --on {q(target)}", "env", unready[0]["id"]) if unready else None}
+
+    # training stages: done when a model trained in the stage has an eval (PRD: "a model promoted from an ... run has a quick eval")
+    NEEDS = {"sft": ("sft", "needs an SFT dataset"), "preference": ("preference", "needs a preference dataset")}
+    LAUNCH = {"sft": "Launch SFT", "preference": "Launch preference", "rl": "Launch RL"}
+    CLI_STAGE = {"sft": "sft", "preference": "dpo", "rl": "rl"}
+
+    def latest_output(kinds):
+        return next((r["output_model"] for r in runs if r["kind"] in kinds and r["status"] == "completed" and r["output_model"]), None)
+    start_from = {"sft": base, "preference": latest_output(("sft",)) or base, "rl": latest_output(("dpo", "sft")) or base}
+    for st, kind in RUN_KIND.items():
+        rs = [r for r in runs if r["kind"] == kind]
+        active = [r for r in rs if r["status"] in ACTIVE_RUN]
+        trained = [r for r in rs if r["status"] == "completed" and r["output_model_id"]]
+        done = [r for r in trained if evaluated.get(r["output_model_id"])]
+        unevaluated = [r for r in trained if not evaluated.get(r["output_model_id"])]
+        if st == "rl":
+            missing = None if envs or by_kind.get("rl") else "needs an environment or an RL dataset"
+            ranked = sorted(envs, key=lambda e: not ready[e["id"]]["ready"])
+            inputs = f"--env {q(','.join(e['name'] for e in (([e for e in ranked if ready[e['id']]['ready']] or ranked)[:3])))}" if envs else f"--data {q(by_kind['rl'][0]['name'])}" if by_kind.get("rl") else "--env <environment>"
+        else:
+            dk, msg = NEEDS[st]
+            missing = None if by_kind.get(dk) else msg
+            inputs = f"--data {q(by_kind[dk][0]['name'])}" if by_kind.get(dk) else f"--data <{'pairs' if st == 'preference' else 'dataset'}>"
+        launch_cli = f"posttrain train {CLI_STAGE[st]} --base {q(start_from[st])} {inputs} --on {q(target)}"
+        if done:
+            status, nxt = "done", None
+        elif active:
+            status, nxt = "in_progress", act("Watch", f"posttrain runs watch {q(active[0]['name'])}", "run", active[0]["id"])
+        elif unevaluated:
+            m = unevaluated[0]["output_model"]
+            status, nxt = "in_progress", act("Run eval", f"posttrain eval {q(m)} --bench {q(bench)} --on {q(target)}", "eval", m)
+        elif missing:
+            status = "not_started"
+            nxt = act("Add dataset" if st != "rl" else ("Add environment"), f"posttrain data add ./{'pairs' if st == 'preference' else 'train'}.jsonl --kind {NEEDS[st][0]}"
+                      if st != "rl" else "posttrain env add ./my-env", "add", "dataset" if st != "rl" else "environment")
+            nxt["needs"] = missing
+        else:
+            status, nxt = "not_started", act(LAUNCH[st], launch_cli, "launch", CLI_STAGE[st])
+        rows[st] = {"status": status, "objects": [run_obj(r) for r in rs], "next": nxt, "launch": act(LAUNCH[st], launch_cli, "launch", CLI_STAGE[st])}
+
+    # eval: a trained model evaluated next to the model it started from (PRD: a release eval compared with the baseline)
+    eval_runs = [r for r in runs if r["kind"] == "eval"]
+    trained = [r for r in runs if r["kind"] in ("sft", "dpo", "rl") and r["status"] == "completed" and r["output_model_id"]]
+    compared = [r for r in trained if evaluated.get(r["output_model_id"]) and evaluated.get(r["base_model_id"])
+                and evaluated[r["output_model_id"]] & evaluated[r["base_model_id"]]]
+    newest = trained[0]["output_model"] if trained else None
+    missing_baseline = next((r for r in trained if evaluated.get(r["output_model_id"]) and not evaluated.get(r["base_model_id"])), None)
+    ev_objs = [run_obj(r) for r in eval_runs if r["status"] in ACTIVE_RUN or r["status"] == "failed"] + [
+        {"type": "eval", "id": e["id"], "name": e["benchmark"], "status": e["status"], "score": e["score"], "stderr": e["stderr"],
+         "metric": e["metric"], "model": e["run_name"] or e["model_name"], "step": e["step"], "n_tasks": e["n_tasks"], "k": e["k"]}
+        for e in evals]
+    if compared:
+        ev_status, ev_next = "done", None
+    elif any(r["status"] in ACTIVE_RUN for r in eval_runs) or any(e["status"] == "running" for e in evals):
+        ev_status, ev_next = "in_progress", act("Watch", "posttrain evals", "page", "/evals")
+    elif missing_baseline:
+        ev_status = "in_progress"
+        ev_next = act("Run baseline eval", f"posttrain eval {q(missing_baseline['base_model'] or base)} --bench {q(bench)} --on {q(target)}",
+                      "eval", missing_baseline["base_model"] or base)
+    elif not trained:
+        ev_status, ev_next = "not_started", {"needs": "needs a trained model", "label": "Run eval", "cli": f"posttrain eval {q(base)} --bench {q(bench)} --on {q(target)}",
+                                             "kind": "eval", "arg": None if base == "<model>" else base}
+    else:
+        ev_status, ev_next = "not_started", act("Run eval", f"posttrain eval {q(newest)} --bench {q(bench)} --on {q(target)}", "eval", newest)
+    rows["eval"] = {"status": ev_status, "objects": ev_objs, "next": ev_next}
+
+    live_deps = [d for d in deps if (d["status"] or "") not in ("failed", "stopped", "deleted")]
+    rows["deploy"] = {"status": "done" if live_deps else "not_started",
+                      "objects": [{"type": "deployment", "id": d["id"], "name": d["name"], "status": d["status"], "model": d["model_name"],
+                                   "endpoint": d["endpoint"]} for d in deps],
+                      "next": None if live_deps else {"needs": "needs the Eval stage"} if ev_status != "done"
+                      else act("Deploy", f"posttrain models deploy {q(newest or '<model>')} --on {q(target)}", "page", "/models")}
+
+    # a stage nobody did while a later one went ahead is skipped (RL straight from a base model skips SFT and preference)
+    reached = max((i for i, s in enumerate(STAGES) if rows[s]["status"] in ("done", "in_progress")), default=-1)
+    for s in STAGES:
+        if rows[s]["status"] == "not_started" and STAGES.index(FEEDS.get(s, s)) < reached:
+            rows[s]["status"] = "skipped"
+            if rows[s].get("launch"):
+                rows[s]["next"] = rows[s]["launch"]
+    nxt = next((s for s in STAGES if rows[s]["status"] in ("not_started", "in_progress") and rows[s]["next"] and rows[s]["next"].get("label")), None)
+    out = [{"key": s, "status": rows[s]["status"], "objects": rows[s]["objects"][:6], "total": len(rows[s]["objects"]), "next": rows[s]["next"]}
+           for s in STAGES]
+
+    # runs that wait for a runner nobody is serving
+    online = {t["name"]: t.get("runners") or [] for t in comp["targets"]}
+    waiting = []
+    if source == "workspace":
+        for j in db.rows(c, "SELECT j.id, j.run_id, j.target, j.created_at, r.name AS run_name FROM jobs j JOIN runs r ON r.id=j.run_id "
+                            "WHERE j.project_id=? AND j.status='queued' ORDER BY j.created_at", (pid,)):
+            if not online.get(j["target"]):
+                waiting.append(j)
+    counts = {"datasets": len(datasets), "environments": len(envs), "runs": len(runs), "evals": len(evals), "models": len(models),
+              "deployments": len(deps)}
+    return {"project": p, "stages": out, "next": nxt, "empty": not any(v for k, v in counts.items() if k != "models"), "counts": counts, "waiting": waiting,
+            "active": any(s["status"] == "in_progress" for s in out) or bool(waiting),
+            "approx": {"promoted": "a completed run's output model", "quick eval": "any completed eval of that model",
+                       "release eval": "a trained model and the model it started from evaluated on a common benchmark"},
+            "compute": [{"name": t["name"], "kind": t["kind"], "runners": t.get("runners") or [], "builtin": bool(t.get("builtin"))}
+                        for t in comp["targets"]]}
+
+
+# ------------------------------------------------------------------ launch
+
+def P(key, label, default, typ="float", help="", **kw):
+    return {"key": key, "label": label, "default": default, "type": typ, "help": help, **kw}
+
+
+def METHOD(default):
+    return [P("method", "Method", default, "choice", "LoRA trains a small adapter and needs far less memory; full updates every weight.",
+              choices=["lora", "full"]),
+            P("lora.r", "LoRA rank", 16, "int", "Adapter rank; 8 to 64 is typical.", when=["method", "lora"])]
+
+
+RL_KEYS = [P("prompts_per_step", "Prompts per step", 64, "int", "Each prompt gets a group of completions."),
+           P("group_size", "Group size", 8, "int", "Completions per prompt; each is scored against its group."),
+           P("lr", "Learning rate", 1e-6), P("max_tokens", "Max tokens", 8192, "int", "Per completion; longer ones are cut off and count as truncated."),
+           P("temperature", "Temperature", 1.0, help="Sampling temperature for training attempts.")]
+
+# The launch form's key settings (PRD 6.4) for each recipe and stage, under the recipes' own setting
+# names (PRD section 4; posttrain.recipes checks them and rejects names it doesn't know). Defaults and help
+# come from posttrain.recipes when this server can import it; `lr_by_method` holds the PRD's rates.
+RECIPES = [
+    {"id": "trl", "label": "TRL", "framework": {"sft": "trl_sft", "dpo": "trl_dpo"}, "stages": {"sft": "SFT", "dpo": "DPO"},
+     "about": "Hugging Face TRL on one node: full fine-tuning or LoRA.",
+     "params": {
+         "sft": [*METHOD("lora"), P("lr", "Learning rate", None, lr_by_method={"lora": 1e-4, "full": 1e-5}),
+                 P("epochs", "Epochs", 1, help="Passes over the data when steps is blank."),
+                 P("batch_size", "Batch size", 32, "int", "Examples per optimizer step, across GPUs."),
+                 P("max_seq_len", "Max length", 8192, "int", "Tokens per example; longer ones are truncated.")],
+         "dpo": [*METHOD("full"), P("beta", "β", 0.1, help="How far the policy may move from the reference; 0.01 to 0.5 is usual."),
+                 P("lr", "Learning rate", None, lr_by_method={"lora": 5e-6, "full": 5e-7}),
+                 P("epochs", "Epochs", 1, help="Passes over the pairs when steps is blank.")]}},
+    {"id": "trl-grpo", "label": "TRL GRPO", "framework": {"rl": "trl_grpo"}, "stages": {"rl": "GRPO"},
+     "about": "TRL's GRPO trainer on one node, with built-in verifiable rewards.",
+     "params": {"rl": [*METHOD("lora"), *[dict(x, lr_by_method={"lora": 1e-5, "full": 1e-6}, default=None) if x["key"] == "lr" else x for x in RL_KEYS]]}},
+    {"id": "prime-rl", "label": "prime-rl", "framework": {"rl": "prime_rl"}, "stages": {"rl": "GRPO"},
+     "about": "Prime Intellect's asynchronous RL trainer on your GPUs, with verifiers environments.", "params": {"rl": RL_KEYS}},
+    {"id": "verl", "label": "verl", "framework": {"rl": "verl"}, "stages": {"rl": "GRPO"},
+     "about": "ByteDance's verl: FSDP or Megatron training with vLLM or SGLang sampling; scales past one node.",
+     "params": {"rl": [*RL_KEYS, P("kl_coef", "KL coefficient", 0.0)]}},
+    {"id": "prime-hosted", "label": "Prime hosted RL", "framework": {"rl": "prime_rl"}, "stages": {"rl": "GRPO"},
+     "about": "Prime Intellect runs the training (LoRA) on its GPUs; the target is a Prime target in hosted mode.",
+     "note": "The model must be one Prime serves (prime train models), and each environment must be on the Prime Environments Hub.",
+     "params": {"rl": [*[dict(x, default=1e-5) if x["key"] == "lr" else x for x in RL_KEYS]]}},
+    {"id": "lm-eval", "label": "lm-evaluation-harness", "framework": {"eval": "lm_eval"}, "stages": {"eval": None},
+     "about": "EleutherAI's harness: standard benchmarks by task name (gsm8k, ifeval, mmlu, …).",
+     "params": {"eval": [P("limit", "Examples per task", None, "int", "Blank runs every example; a small number makes a smoke test."),
+                         P("num_fewshot", "Few-shot examples", None, "int", "Blank uses each task's default."),
+                         P("apply_chat_template", "Chat template", False, "bool", "Format prompts with the model's chat template (instruct models)."),
+                         P("batch_size", "Batch size", "auto", "str", "An integer, or auto.")]}},
+]
+STAGE_RECIPES = {"sft": ["trl"], "dpo": ["trl"], "rl": ["trl-grpo", "prime-rl", "verl", "prime-hosted"], "eval": ["lm-eval"]}
+DEFAULT_RECIPE = {"sft": "trl", "dpo": "trl", "rl": "trl-grpo", "eval": "lm-eval"}
+STRUCTURAL = {"base", "data", "env", "steps", "name", "model", "benchmarks"}   # set by the form's own fields
+
+
+def _typed(v):
+    return "bool" if isinstance(v, bool) else "int" if isinstance(v, int) else "float" if isinstance(v, float) else "str"
+
+
+def _metric_defs(framework):
+    try:
+        from .build.signals import FRAMEWORK_TAGS, SIGNALS
+    except Exception:
+        return []
+    out = []
+    for signal, tag in FRAMEWORK_TAGS.get(framework, {}).items():
+        label, unit, fmt, better, grp, desc = SIGNALS[signal]
+        out.append({"tag": tag, "label": label, "description": desc, "unit": unit, "format": fmt, "grp": grp,
+                    "better": better, "pinned": 0, "signal": signal})
+    return out
+
+
+def _recipe_source(stage, recipe):
+    """({setting: (default, help)}, framework) from posttrain.recipes, or (None, None) when this server lacks it."""
+    try:
+        from posttrain import recipes as R
+        table = R.params(stage, recipe)
+        mod = R.REGISTRY.get(recipe)
+        fw = getattr(mod, "FRAMEWORK", None)
+        fw = (fw.get(stage) or fw.get(recipe)) if isinstance(fw, dict) else fw
+        return table, fw
+    except Exception:
+        return None, None
+
+
+def recipe_catalog():
+    """RECIPES with each stage's settings checked against posttrain.recipes where this server has it:
+    defaults and help from the recipe's own table, settings it doesn't know dropped, the rest listed."""
+    out = []
+    for r in RECIPES:
+        entry = {k: v for k, v in r.items() if k != "params"}
+        entry["params"], entry["more"], entry["checked"], entry["metric_defs"] = {}, {}, {}, {}
+        for stage, keys in r["params"].items():
+            table, fw = _recipe_source(stage, r["id"])
+            fw = fw or r["framework"].get(stage)
+            entry["framework"] = {**entry["framework"], stage: fw}
+            entry["metric_defs"][stage] = _metric_defs(fw)
+            if not table:
+                entry["params"][stage], entry["more"][stage], entry["checked"][stage] = keys, [], False
+                continue
+            shown = []
+            for prm in keys:
+                if prm["key"] in table:
+                    default, help_ = table[prm["key"]]
+                    shown.append({**prm, "default": prm["default"] if default is None else default, "help": prm["help"] or help_})
+            listed = {x["key"] for x in shown} | STRUCTURAL
+            entry["params"][stage] = shown
+            entry["more"][stage] = [{"key": k, "default": v[0], "type": _typed(v[0]), "help": v[1]} for k, v in table.items() if k not in listed]
+            entry["checked"][stage] = True
+        out.append(entry)
+    return out
+
+
+@router.get("/p/{org}/{project}/launch")
+def launch_options(org: str, project: str, source: str = "workspace"):
+    """What a run launched from the website can use: models and checkpoints, datasets, environments,
+    benchmarks, the org's compute targets with the runners serving them, and the recipes."""
+    c = conn_for(source)
+    p = project_or_404(c, org, project)
+    pid = p["id"]
+    models = db.rows(c, "SELECT m.id, m.name, m.kind, m.hf_repo, m.stage, m.status, m.run_id, m.created_at, r.name AS run_name, "
+                        "r.kind AS run_kind, r.status AS run_status FROM models m LEFT JOIN runs r ON r.id=m.run_id "
+                        "WHERE m.project_id=? AND coalesce(m.kind,'') NOT IN ('external','judge','reward') ORDER BY m.created_at DESC", (pid,))
+    made_by = {r["output_model_id"]: r for r in db.rows(c, "SELECT id, name, kind, status, output_model_id FROM runs WHERE project_id=? "
+                                                         "AND output_model_id IS NOT NULL", (pid,))}
+    for m in models:
+        r = made_by.get(m["id"])
+        if r and not m["run_id"]:
+            m.update(run_id=r["id"], run_name=r["name"], run_kind=r["kind"], run_status=r["status"])
+    ckpts = db.rows(c, "SELECT k.id, k.run_id, k.step, k.path, k.created_at, r.name AS run_name, r.kind AS run_kind FROM checkpoints k "
+                       "JOIN runs r ON r.id=k.run_id WHERE r.project_id=? ORDER BY k.created_at DESC LIMIT 60", (pid,))
+    datasets = db.rows(c, "SELECT id, name, kind, rows, tokens, version, hf_repo FROM datasets WHERE project_id=? ORDER BY created_at DESC", (pid,))
+    envs = db.rows(c, "SELECT id, name, domain, task_count, reward_kind FROM environments WHERE project_id=? ORDER BY name", (pid,))
+    readiness = env_readiness(c, [e["id"] for e in envs])
+    for e in envs:
+        e.update(readiness[e["id"]])
+    has_eval = {r[0] for r in c.execute("SELECT DISTINCT model_id FROM evals WHERE project_id=? AND status='completed' AND model_id IS NOT NULL", (pid,))}
+    for m in models:
+        m["evaluated"] = m["id"] in has_eval
+    benches = db.rows(c, "SELECT id, name, metric, n_tasks, k, harness FROM benchmarks WHERE project_id=? ORDER BY name", (pid,))
+    runs = db.rows(c, "SELECT id, name, kind, status, group_name FROM runs WHERE project_id=? ORDER BY started_at DESC LIMIT 500", (pid,))
+    comp = compute_state(org) if source == "workspace" else {"targets": [], "runners": []}
+    return {"project": p, "models": models, "checkpoints": ckpts, "datasets": datasets, "environments": envs,
+            "benchmarks": benches, "runs": runs, "compute": comp, "recipes": recipe_catalog(), "default_recipe": DEFAULT_RECIPE,
+            "stage_recipes": STAGE_RECIPES}
+
+
+# ------------------------------------------------------------------ jobs behind runs
+
+def _runner_view(r, now):
+    return {"id": r["id"], "name": r["name"], "hostname": r["hostname"], "last_seen": r["last_seen"], "version": r["version"],
+            "online": now - (r["last_seen"] or 0) < ONLINE} if r else None
+
+
+@router.get("/runs/{run_id}/jobs")
+def run_jobs(run_id: str):
+    """The jobs behind a workspace run: where each waits or runs, the runner that has it, what is
+    ahead of it in the queue, and how much log it has written."""
+    c = workspace.connect()
+    r = db.one(c, "SELECT r.id, r.name, r.status, r.status_reason, r.steps_done, r.steps_planned, r.started_at, r.updated_at, "
+                  "r.ended_at, r.last_seen, p.org_id, o.slug AS org FROM runs r JOIN projects p ON p.id=r.project_id JOIN orgs o ON o.id=p.org_id "
+                  "WHERE r.id=?", (run_id,))
+    if not r:
+        raise HTTPException(404, f"No run {run_id} in the workspace.")
+    now = time.time()
+    jobs = db.rows(c, "SELECT id, name, kind, status, target, spec, runner_id, external_id, created_at, claimed_at, started_at, "
+                      "ended_at, cancel, message, exit FROM jobs WHERE run_id=? ORDER BY created_at", (run_id,))
+    runners = db.rows(c, "SELECT id, name, hostname, targets, version, last_seen FROM runners WHERE org_id=?", (r["org_id"],))
+    targets = {t["name"]: t for t in db.rows(c, "SELECT name, kind, config FROM compute_targets WHERE org_id=?", (r["org_id"],))}
+    by_id = {x["id"]: x for x in runners}
+    for j in jobs:
+        j["runner"] = _runner_view(by_id.get(j["runner_id"]), now)
+        j["runners_online"] = [x["name"] for x in runners if j["target"] in (x.get("targets") or []) and now - (x["last_seen"] or 0) < ONLINE]
+        t = targets.get(j["target"])
+        j["target_kind"] = t["kind"] if t else ("local" if j["target"] == "local" else None)
+        j["target_registered"] = bool(t) or j["target"] == "local"
+        j["ahead"] = c.execute("SELECT count(*) FROM jobs j2 JOIN projects p ON p.id=j2.project_id WHERE p.org_id=? AND j2.status='queued' "
+                               "AND j2.target=? AND j2.created_at < ?", (r["org_id"], j["target"], j["created_at"] or now)).fetchone()[0] \
+            if j["status"] == "queued" else 0
+    n, last_seq, last_t = c.execute("SELECT count(*), max(seq), max(t) FROM logs WHERE run_id=?", (run_id,)).fetchone()
+    ev = db.one(c, "SELECT t, kind, title, body FROM run_events WHERE run_id=? ORDER BY t DESC LIMIT 1", (run_id,))
+    return {"run": {k: v for k, v in r.items() if k != "org_id"}, "jobs": jobs, "now": now,
+            "logs": {"lines": n, "last_seq": last_seq or 0, "last_t": last_t}, "last_event": ev}
+
+
+@router.get("/orgs/{org}/jobs")
+def org_jobs(org: str, status: str = "queued,starting,running", limit: int = 200):
+    """Jobs across the org's projects (queued, starting and running by default), with where they run."""
+    c = workspace.connect()
+    wanted = [s for s in status.split(",") if s]
+    if not wanted:
+        return []
+    now = time.time()
+    rows = db.rows(c, f"SELECT j.id, j.name, j.kind, j.status, j.target, j.runner_id, j.created_at, j.claimed_at, j.started_at, "
+                      f"j.ended_at, j.cancel, j.message, j.run_id, r.name AS run_name, r.kind AS run_kind, p.slug AS project, "
+                      f"p.name AS project_name FROM jobs j JOIN projects p ON p.id=j.project_id JOIN orgs o ON o.id=p.org_id "
+                      f"LEFT JOIN runs r ON r.id=j.run_id WHERE o.slug=? AND j.status IN ({','.join('?' for _ in wanted)}) "
+                      f"ORDER BY j.created_at DESC LIMIT ?", (org, *wanted, limit))
+    runners = {x["id"]: x for x in db.rows(c, "SELECT r.id, r.name, r.hostname, r.version, r.last_seen FROM runners r JOIN orgs o "
+                                              "ON o.id=r.org_id WHERE o.slug=?", (org,))}
+    for j in rows:
+        j["runner"] = _runner_view(runners.get(j["runner_id"]), now)
+    return rows

viewer/api_write.py

@@ -0,0 +1,1028 @@
+"""Write API for the workspace: what the CLI, the SDK, trainers and runners call.
+
+Every call needs `Authorization: Bearer <token>` (see `posttrain login`), except when the server
+runs with POSTTRAIN_OPEN=1 (a single-user local server; `posttrain server` sets it on localhost).
+Reads go through the same endpoints as every other source, with `?source=workspace`.
+"""
+import json
+import os
+import re
+import secrets
+import time
+
+from fastapi import APIRouter, Body, Header, HTTPException, Request
+
+from . import db, workspace
+from . import stages as S
+
+router = APIRouter(prefix="/api/v3")
+TERMINAL = {"completed", "failed", "stopped"}
+KINDS = {"sft", "dpo", "rl", "distill", "rm", "eval"}
+
+
+def new_id(prefix):
+    return f"{prefix}_{secrets.token_hex(6)}"
+
+
+def user(authorization: str = Header(default="")):
+    token = authorization.split(" ", 1)[1].strip() if authorization.lower().startswith("bearer ") else ""
+    u = workspace.check_token(token) if token else None
+    if u:
+        return u
+    if os.environ.get("POSTTRAIN_OPEN") == "1":
+        return {"id": "local", "name": "local"}
+    raise HTTPException(401, "Missing or invalid API token. Run `posttrain login` or pass Authorization: Bearer <token>.")
+
+
+def auth(request: Request, run_id=None, allow_run_token=False, allow_runner=False):
+    """The caller; run-scoped tokens may only touch their own run, runner tokens only runner/job/run endpoints."""
+    u = user(request.headers.get("authorization", ""))
+    scope = u.get("scope") or ""
+    if scope.startswith("run:"):
+        if not (allow_run_token and run_id and scope == f"run:{run_id}"):
+            raise HTTPException(403, "This token belongs to one run and can only report for that run.")
+    elif scope.startswith("runner:") and not (allow_runner or allow_run_token):
+        raise HTTPException(403, "Runner tokens can only claim, watch and report runs.")
+    return u
+
+
+def eval_caller(request):
+    """The caller of an eval write, and the run its token is scoped to (an eval job holds its run's token), if any."""
+    u = user(request.headers.get("authorization", ""))
+    scope = u.get("scope") or ""
+    if scope.startswith("run:"):
+        return u, scope[4:]
+    auth(request)  # user tokens (and open mode); runner tokens are refused as elsewhere
+    return u, None
+
+
+def dumps(v):
+    return json.dumps(v, separators=(",", ":")) if isinstance(v, (dict, list)) else v
+
+
+def project_row(conn, org, project):
+    p = db.one(conn, "SELECT p.* FROM projects p JOIN orgs o ON o.id=p.org_id WHERE o.slug=? AND p.slug=?", (org, project))
+    if not p:
+        raise HTTPException(404, f"No project {org}/{project} in the workspace. Create it with `posttrain init`.")
+    return p
+
+
+def run_row(conn, run_id):
+    r = db.one(conn, "SELECT * FROM runs WHERE id=?", (run_id,))
+    if not r:
+        raise HTTPException(404, f"No run {run_id}.")
+    return r
+
+
+def insert(conn, table, rec):
+    rec = {k: dumps(v) for k, v in rec.items()}
+    conn.execute(f"INSERT OR REPLACE INTO {table} ({','.join(rec)}) VALUES ({','.join('?' for _ in rec)})", tuple(rec.values()))
+
+
+HF_REPO = re.compile(r"[A-Za-z0-9][\w.-]*/[\w.-]+")
+
+
+def resolve_model(conn, pid, ref, kind="base"):
+    """A model id for a model id, name or HF repo; a checkpoint this project saved (its path, or RUN:STEP);
+    or a run (its output model). Anything else is registered as a new model, so lineage is kept."""
+    if not ref:
+        return None
+    m = db.one(conn, "SELECT id FROM models WHERE project_id=? AND (id=? OR name=? OR hf_repo=?)", (pid, ref, ref, ref))
+    if m:
+        return m["id"]
+    run_ref, _, step = ref.rpartition(":") if re.search(r":\d+$", ref) else (ref, "", "")
+    c = db.one(conn, "SELECT c.* FROM checkpoints c JOIN runs r ON r.id=c.run_id WHERE r.project_id=? AND "
+                     "(c.path=? OR ((r.id=? OR r.name=?) AND c.step=?)) ORDER BY c.created_at DESC LIMIT 1",
+               (pid, ref, run_ref, run_ref, int(step) if step else -1))
+    if c:
+        if c["model_id"]:
+            return c["model_id"]
+        r = run_row(conn, c["run_id"])
+        mid = new_id("model")
+        insert(conn, "models", {"id": mid, "project_id": pid, "name": f"{r['name']} step {c['step']}", "kind": "checkpoint",
+                                "hf_repo": None, "parent_id": r["base_model_id"], "run_id": r["id"], "step": c["step"],
+                                "stage": r["stage"], "created_at": time.time(), "status": "available", "notes": c["path"], "source": ""})
+        conn.execute("UPDATE checkpoints SET model_id=? WHERE id=?", (mid, c["id"]))
+        return mid
+    r = db.one(conn, "SELECT output_model_id FROM runs WHERE project_id=? AND (id=? OR name=?) AND output_model_id IS NOT NULL "
+                     "ORDER BY started_at DESC LIMIT 1", (pid, ref, ref))
+    if r:
+        return r["output_model_id"]
+    hub = bool(HF_REPO.fullmatch(ref)) and not ref.startswith((".", "~"))
+    mid = new_id("model")
+    insert(conn, "models", {"id": mid, "project_id": pid, "name": ref, "kind": kind if hub else "checkpoint",
+                            "hf_repo": ref if hub else None, "notes": "" if hub else ref,
+                            "created_at": time.time(), "status": "available"})
+    return mid
+
+
+def resolve_input(conn, pid, kind, ref):
+    table = "datasets" if kind == "dataset" else "environments"
+    r = db.one(conn, f"SELECT id FROM {table} WHERE project_id=? AND (id=? OR name=?)", (pid, ref, ref))
+    if not r:
+        raise HTTPException(404, f"No {kind} named {ref!r} in this project. Add it first (`posttrain {'data' if kind == 'dataset' else 'env'} add`).")
+    return r["id"]
+
+
+EVENT_BODY_MAX = 1500
+
+
+def event(conn, run_id, kind, title, body="", severity="info", step=None, t=None):
+    body = body or ""
+    if len(body) > EVENT_BODY_MAX:   # whole configs or tracebacks belong in Config and Logs, not in the activity feed
+        body = body[:EVENT_BODY_MAX].rstrip() + f"… ({len(body) - EVENT_BODY_MAX:,} more characters in the run's Logs or Config)"
+    insert(conn, "run_events", {"run_id": run_id, "t": t or time.time(), "step": step, "kind": kind, "severity": severity,
+                                "title": title, "body": body})
+
+
+def summarize_reason(conn, run_id, reason):
+    """A one-line reason for the run header and the activity feed. A multi-line reason (a traceback, the last lines
+    of a log) goes to the run's Logs in full, and the summary keeps its last meaningful line."""
+    reason = (reason or "").strip()
+    if "\n" not in reason and len(reason) <= 300:
+        return reason
+    lines = [ln.rstrip() for ln in reason.splitlines() if ln.strip()]
+    seq = conn.execute("SELECT coalesce(max(seq), 0) FROM logs WHERE run_id=?", (run_id,)).fetchone()[0]
+    conn.executemany("INSERT INTO logs (run_id, seq, t, stream, text) VALUES (?,?,?,?,?)",
+                     [(run_id, seq + i + 1, time.time(), "stderr", ln[:4000]) for i, ln in enumerate(lines[-200:])])
+    last = next((ln.strip() for ln in reversed(lines) if not ln.strip().startswith(("File ", "^", "~"))), lines[-1].strip())
+    first = lines[0].strip().rstrip(":")
+    # wrap's reasons start with a summary ("exited with code 3"); a bare traceback starts with "Traceback"
+    head = last if first.startswith("Traceback") or first == last else f"{first}: {last}"
+    return (head[:280] + ("…" if len(head) > 280 else "")) + " (full output in the Logs tab)"
+
+
+def seen(conn, r):
+    """Any report from the job (metrics, logs, steps...) proves it is alive, not only heartbeats."""
+    now = time.time()
+    conn.execute("UPDATE runs SET last_seen=?, updated_at=? WHERE id=?", (now, now, r["id"]))
+    if r["status"] == "stalled":
+        conn.execute("UPDATE runs SET status='running', status_reason='' WHERE id=? AND status='stalled'", (r["id"],))
+        event(conn, r["id"], "notice", "Reporting", "Reports resumed.")
+
+
+# ------------------------------------------------------------------ identity and tokens
+
+@router.get("/whoami")
+def whoami(request: Request):
+    u = auth(request)
+    return {"user": u, "open": os.environ.get("POSTTRAIN_OPEN") == "1", "workspace": str(workspace.path())}
+
+
+@router.post("/tokens")
+def create_token(request: Request, payload: dict = Body(default={})):
+    """A user token, or with {"runner": true, "targets": [...]} a runner token."""
+    u = auth(request)
+    scope = f"runner:{','.join(payload.get('targets', []))}" if payload.get("runner") else None
+    expires = None if payload.get("expires") == "never" or scope else time.time() + 90 * 86400
+    return {"token": workspace.issue_token(payload.get("name", "cli"), payload.get("user") or u["name"], payload.get("email", ""), scope, expires),
+            "expires": expires, "scope": scope}
+
+
+@router.post("/tokens/revoke")
+def revoke_token(request: Request, payload: dict = Body(...)):
+    auth(request)
+    return {"revoked": workspace.revoke_token(payload["prefix"])}
+
+
+@router.get("/tokens")
+def tokens(request: Request):
+    auth(request)
+    return workspace.list_tokens()
+
+
+# ------------------------------------------------------------------ projects and inputs
+
+def _settings_payload(payload):
+    """Validated project settings from a request (PRD 3: base model, planned stages, quick and release suites, budget)."""
+    out, suites = {}, {}
+    try:
+        if payload.get("base_model"):
+            out["base_model"] = str(payload["base_model"]).strip()
+        if payload.get("stages") is not None:
+            out["stages"] = S.normalize_stages(payload["stages"])
+        budget = payload.get("budget_usd", payload.get("budget"))
+        if budget not in (None, ""):
+            try:
+                out["budget_usd"] = float(budget)
+            except (TypeError, ValueError):
+                raise S.Problem(422, f"budget must be a number of US dollars, not {budget!r}")
+            if out["budget_usd"] < 0:
+                raise S.Problem(422, "budget must be 0 or more")
+        for name in ("quick", "release"):
+            if payload.get(name) not in (None, "", []):
+                suites[name] = [b["spec"] for b in S.parse_benches(payload[name])]
+        for name, benches in (payload.get("suites") or {}).items():
+            if benches not in (None, "", []):
+                if not S.SUITE_NAME.fullmatch(str(name)):
+                    raise S.Problem(422, f"suite name {name!r}: lowercase letters, digits, - and _")
+                suites[name] = [b["spec"] for b in S.parse_benches(benches)]
+    except S.Problem as e:
+        raise HTTPException(e.status, str(e))
+    return out, suites
+
+
+def _apply_settings(conn, pid, settings, suites, user):
+    changed = S.put_settings(conn, pid, settings, user)
+    if settings.get("base_model"):
+        resolve_model(conn, pid, settings["base_model"], kind="base")     # the base model is a model of the project
+    for name, benches in suites.items():
+        S.set_suite(conn, pid, name, benches, user)
+    if changed or suites:
+        S.audit(conn, pid, user, "settings", pid, {"changed": changed, "suites": suites})
+
+
+def settings_response(conn, org, project, p):
+    s = S.get_settings(conn, p["id"])
+    su = S.get_suites(conn, p["id"])
+    return {"schema": "posttrain.v1.project_settings", "id": p["id"], "url": f"/dashboard/{org}/{project}/settings",
+            "project": f"{org}/{project}", "name": p.get("name"), **s,
+            "suites": {k: {"benchmarks": [b["spec"] for b in v["benchmarks"]], "defined": v["defined"], "inherits": v["inherits"]}
+                       for k, v in su.items()},
+            "aliases": S.aliases(conn, p["id"])}
+
+
+@router.post("/projects")
+def create_project(request: Request, payload: dict = Body(...)):
+    """Create (or join) a project. Optional settings (PRD 3, 5.2 `init`): base_model, stages (["sft", "dpo", "rl"] or
+    "sft,dpo,rl"; "none" plans none), quick and release (benchmark lists, or suites: {name: [...]}), budget_usd."""
+    u = auth(request)
+    org, slug = payload.get("org"), payload.get("slug")
+    if not org or not slug:
+        raise HTTPException(422, "org and slug are required.")
+    settings, suites = _settings_payload(payload)
+
+    def fn(conn):
+        o = db.one(conn, "SELECT id FROM orgs WHERE slug=?", (org,))
+        oid = o["id"] if o else new_id("org")
+        if not o:
+            insert(conn, "orgs", {"id": oid, "slug": org, "name": payload.get("org_name") or org, "about": "", "url": ""})
+        p = db.one(conn, "SELECT id FROM projects WHERE org_id=? AND slug=?", (oid, slug))
+        created = not p
+        pid = p["id"] if p else new_id("proj")
+        if created:
+            insert(conn, "projects", {"id": pid, "org_id": oid, "slug": slug, "name": payload.get("name") or slug,
+                                      "summary": payload.get("summary", ""), "created_at": time.time(), "sources": [],
+                                      "data_note": f"Created by {u['name']}.", "pins": []})
+            S.audit(conn, pid, u["name"], "create_project", pid, {"org": org, "slug": slug})
+        _apply_settings(conn, pid, settings, suites, u["name"])
+        row = db.one(conn, "SELECT * FROM projects WHERE id=?", (pid,))
+        return {"project_id": pid, "created": created, "settings": settings_response(conn, org, slug, row)}
+    res = workspace.write(fn)
+    return {**res, "id": res["project_id"], "schema": "posttrain.v1.project", "project": f"{org}/{slug}", "url": f"/dashboard/{org}/{slug}"}
+
+
+@router.post("/p/{org}/{project}/settings")
+def update_settings(org: str, project: str, request: Request, payload: dict = Body(...)):
+    """Change a project's base model, planned stages, budget or suites (only the keys given)."""
+    u = auth(request)
+    settings, suites = _settings_payload(payload)
+
+    def fn(conn):
+        p = project_row(conn, org, project)
+        _apply_settings(conn, p["id"], settings, suites, u["name"])
+        return settings_response(conn, org, project, p)
+    return workspace.write(fn)
+
+
+def _write(fn):
+    """workspace.write, with the rules' Problems (viewer/stages.py) answered as HTTP errors."""
+    try:
+        return workspace.write(fn)
+    except S.Problem as e:
+        raise HTTPException(e.status, str(e))
+
+
+@router.post("/p/{org}/{project}/suites")
+def set_suite(org: str, project: str, request: Request, payload: dict = Body(...)):
+    """{"name": "release", "benchmarks": ["arc_easy?limit=200", "env:arith-heldout?k=4"]} (or "a,b"). Replaces the suite."""
+    u = auth(request)
+
+    def fn(conn):
+        p = project_row(conn, org, project)
+        parsed, warnings = S.set_suite(conn, p["id"], payload.get("name"), payload.get("benchmarks") or payload.get("bench") or [], u["name"])
+        S.audit(conn, p["id"], u["name"], "suite", payload.get("name"), {"benchmarks": [b["spec"] for b in parsed]})
+        su = S.get_suites(conn, p["id"])[str(payload.get("name")).strip().lower()]
+        return {**S.suite_object(org, project, p["id"], su), "warnings": warnings}
+    return _write(fn)
+
+
+def _stage_table(conn, org, project, p):
+    from . import api_ui          # lazy: api_ui imports this module
+    return S.stage_table(conn, org, project, p, "workspace", api_ui.compute_state(org))
+
+
+@router.post("/p/{org}/{project}/stages/{stage}/skip")
+def skip_stage(org: str, project: str, stage: str, request: Request, payload: dict = Body(default={})):
+    """Mark a stage skipped (`posttrain status --skip STAGE`), recorded with the user, the time and an optional note."""
+    u = auth(request)
+    if stage not in S.STAGES:
+        raise HTTPException(422, f"stage must be one of {', '.join(S.STAGES)}")
+
+    def fn(conn):
+        p = project_row(conn, org, project)
+        note = str(payload.get("note") or "").strip()
+        conn.execute("INSERT OR REPLACE INTO stage_state (project_id, stage, skipped, note, by, at) VALUES (?,?,?,?,?,?)",
+                     (p["id"], stage, 1, note, u["name"], time.time()))
+        S.audit(conn, p["id"], u["name"], "skip", stage, {"note": note})
+        return _stage_table(conn, org, project, p)
+    return _write(fn)
+
+
+@router.post("/p/{org}/{project}/stages/{stage}/unskip")
+def unskip_stage(org: str, project: str, stage: str, request: Request):
+    u = auth(request)
+    if stage not in S.STAGES:
+        raise HTTPException(422, f"stage must be one of {', '.join(S.STAGES)}")
+
+    def fn(conn):
+        p = project_row(conn, org, project)
+        conn.execute("INSERT OR REPLACE INTO stage_state (project_id, stage, skipped, note, by, at) VALUES (?,?,?,?,?,?)",
+                     (p["id"], stage, 0, "", u["name"], time.time()))
+        S.audit(conn, p["id"], u["name"], "unskip", stage, {})
+        return _stage_table(conn, org, project, p)
+    return _write(fn)
+
+
+@router.post("/p/{org}/{project}/acks")
+def acknowledge(org: str, project: str, request: Request, payload: dict = Body(...)):
+    """Acknowledge a regression: {"model", "suite", "benchmark", "note"} (note required), recorded with the user."""
+    u = auth(request)
+
+    def fn(conn):
+        p = project_row(conn, org, project)
+        return S.acknowledge(conn, p["id"], org, project, payload.get("model"), payload.get("suite") or "release",
+                             payload.get("benchmark") or payload.get("bench"), payload.get("note"), u["name"])
+    return _write(fn)
+
+
+@router.post("/p/{org}/{project}/models/promote")
+def promote(org: str, project: str, request: Request, payload: dict = Body(...)):
+    """Promote a checkpoint: {"ref": "RUN[:STEP]", "name", "notes", "replace", "dry_run"} (PRD 4.7)."""
+    u = auth(request)
+
+    def fn(conn):
+        p = project_row(conn, org, project)
+        return S.promote(conn, p["id"], org, project, payload.get("ref") or payload.get("run"), payload.get("name") or payload.get("as"),
+                         u["name"], notes=payload.get("notes"), replace=bool(payload.get("replace")), dry_run=bool(payload.get("dry_run")))
+    return _write(fn)
+
+
+@router.post("/p/{org}/{project}/aliases")
+def set_alias(org: str, project: str, request: Request, payload: dict = Body(...)):
+    """Point an alias at a model: {"alias": "production", "model": NAME}. `production` needs a model that passes the
+    model-level deploy checks (promoted, release eval complete, regressions acknowledged)."""
+    u = auth(request)
+    alias = str(payload.get("alias") or "").strip()
+    if not S.SUITE_NAME.fullmatch(alias):
+        raise HTTPException(422, "alias: lowercase letters, digits, - and _")
+
+    def fn(conn):
+        p = project_row(conn, org, project)
+        sel = S.resolve_ref(conn, p["id"], payload.get("model"))
+        if not sel or not sel["model"]:
+            raise S.Problem(404, f"No model {payload.get('model')!r} in this project.")
+        if alias == "production":
+            ok, why = S.model_gate(conn, p["id"], sel)
+            if not ok:
+                raise S.Problem(409, f"production can't point at {sel['name']}: {why}")
+        S.set_alias(conn, p["id"], alias, sel["model"]["id"], u["name"])
+        return {"schema": "posttrain.v1.alias", "id": f"{p['id']}:{alias}", "url": f"/dashboard/{org}/{project}/models",
+                "alias": alias, "model": sel["name"], "model_id": sel["model"]["id"], "set_by": u["name"]}
+    return _write(fn)
+
+
+DEPLOYMENT_STATUSES = {"starting", "healthy", "failed", "stopped", "pushed"}
+
+
+def _dep_row(conn, dep_id):
+    d = db.one(conn, "SELECT d.*, m.name AS model_name, p.slug AS project_slug, o.slug AS org_slug FROM deployments d "
+                     "LEFT JOIN models m ON m.id=d.model_id JOIN projects p ON p.id=d.project_id JOIN orgs o ON o.id=p.org_id "
+                     "WHERE d.id=?", (dep_id,))
+    if not d:
+        raise S.Problem(404, f"No deployment {dep_id}.")
+    return d
+
+
+def _deployed(conn, d, user):
+    """A deployment that became healthy (or a Hub push): record the weights' new home, and point production at the
+    model when it passes the model-level deploy checks. Returns what happened to production."""
+    sel = S.resolve_ref(conn, d["project_id"], d["model_id"]) if d.get("model_id") else None
+    if d.get("kind") == "hub" and (d.get("endpoint") or "").startswith("hf://"):
+        repo = d["endpoint"][len("hf://"):]
+        conn.execute("UPDATE models SET hf_repo=coalesce(hf_repo, ?) WHERE id=?", (repo, d["model_id"]))
+        conn.execute("UPDATE promotions SET pushed_to=? WHERE model_id=?", (d["endpoint"], d["model_id"]))
+    if not sel or not sel["model"]:
+        return {"set": False, "reason": "unknown model"}
+    ok, why = S.model_gate(conn, d["project_id"], sel)
+    if not ok:
+        return {"set": False, "model": sel["name"], "reason": why}
+    S.set_alias(conn, d["project_id"], "production", sel["model"]["id"], user)
+    return {"set": True, "model": sel["name"]}
+
+
+@router.post("/p/{org}/{project}/deployments")
+def create_deployment(org: str, project: str, request: Request, payload: dict = Body(...)):
+    """Record a deployment the CLI starts (kind vllm) or a Hub push (kind hub, status pushed): {"name", "model",
+    "target", "kind", "gpus", "gpu", "serving", "command", "checks", "endpoint", "status", "handle", "message"}."""
+    u = auth(request)
+    status = payload.get("status") or "starting"
+    if status not in DEPLOYMENT_STATUSES:
+        raise HTTPException(422, f"status must be one of {', '.join(sorted(DEPLOYMENT_STATUSES))}")
+
+    def fn(conn):
+        p = project_row(conn, org, project)
+        sel = S.resolve_ref(conn, p["id"], payload.get("model"))
+        if not sel or not sel["model"]:
+            raise S.Problem(404, f"No model {payload.get('model')!r} in this project.")
+        name = S.slug(payload.get("name") or sel["name"])
+        live = db.one(conn, "SELECT id, status FROM deployments WHERE project_id=? AND name=? AND status IN ('starting','healthy','serving')",
+                      (p["id"], name))
+        if live and payload.get("kind", "vllm") != "hub":
+            raise S.Problem(409, f"deployment {name} is {live['status']}; stop it first (posttrain deployments stop {name}) or pick another --name")
+        did = new_id("dep")
+        now = time.time()
+        insert(conn, "deployments", {"id": did, "project_id": p["id"], "model_id": sel["model"]["id"], "name": name, "status": status,
+                                     "endpoint": payload.get("endpoint"), "gpu": payload.get("gpu") or "GPU", "replicas": 1,
+                                     "created_at": now, "kind": payload.get("kind") or "vllm", "target": payload.get("target"),
+                                     "handle": payload.get("handle"), "serving": payload.get("serving"), "smoke": payload.get("smoke"),
+                                     "message": payload.get("message") or "", "command": payload.get("command"),
+                                     "checks": payload.get("checks"), "created_by": u["name"], "updated_at": now})
+        S.audit(conn, p["id"], u["name"], "deploy", did, {"name": name, "model": sel["name"], "target": payload.get("target"),
+                                                          "kind": payload.get("kind") or "vllm", "status": status})
+        d = _dep_row(conn, did)
+        production = _deployed(conn, d, u["name"]) if status in ("healthy", "pushed") else None
+        return {**S.deployment_object(org, project, _dep_row(conn, did)), "production": production}
+    return _write(fn)
+
+
+@router.post("/deployments/{dep_id}/status")
+def deployment_status(dep_id: str, request: Request, payload: dict = Body(...)):
+    """Update a deployment: {"status": starting|healthy|failed|stopped|pushed, "endpoint", "handle", "smoke", "message"}."""
+    u = auth(request)
+    status = payload.get("status")
+    if status is not None and status not in DEPLOYMENT_STATUSES:
+        raise HTTPException(422, f"status must be one of {', '.join(sorted(DEPLOYMENT_STATUSES))}")
+
+    def fn(conn):
+        d = _dep_row(conn, dep_id)
+        sets = {k: dumps(payload[k]) for k in ("status", "endpoint", "handle", "smoke", "message", "serving", "command") if k in payload}
+        sets["updated_at"] = time.time()
+        conn.execute(f"UPDATE deployments SET {','.join(k + '=?' for k in sets)} WHERE id=?", (*sets.values(), d["id"]))
+        d = _dep_row(conn, d["id"])
+        production = _deployed(conn, d, u["name"]) if status in ("healthy", "pushed") else None
+        if status:
+            S.audit(conn, d["project_id"], u["name"], f"deployment_{status}", d["id"], {"message": payload.get("message")})
+        return {**S.deployment_object(d["org_slug"], d["project_slug"], _dep_row(conn, d["id"])), "production": production}
+    return _write(fn)
+
+
+@router.post("/deployments/{dep_id}/stop")
+def deployment_stop(dep_id: str, request: Request, payload: dict = Body(default={})):
+    """Mark a deployment stopped (the CLI has ended the server process), recorded with the user."""
+    u = auth(request)
+
+    def fn(conn):
+        d = _dep_row(conn, dep_id)
+        now = time.time()
+        conn.execute("UPDATE deployments SET status='stopped', stopped_at=?, stopped_by=?, updated_at=?, message=? WHERE id=?",
+                     (now, u["name"], now, payload.get("message") or f"stopped by {u['name']}", d["id"]))
+        S.audit(conn, d["project_id"], u["name"], "deployment_stop", d["id"], {"name": d["name"]})
+        return S.deployment_object(d["org_slug"], d["project_slug"], _dep_row(conn, d["id"]))
+    return _write(fn)
+
+
+@router.post("/p/{org}/{project}/datasets")
+def add_dataset(org: str, project: str, request: Request, payload: dict = Body(...)):
+    auth(request)
+
+    def fn(conn):
+        p = project_row(conn, org, project)
+        existing = db.one(conn, "SELECT id FROM datasets WHERE project_id=? AND name=? AND coalesce(version,'')=?",
+                          (p["id"], payload["name"], payload.get("version", "")))
+        did = existing["id"] if existing else new_id("ds")
+        if existing:
+            for t in ("dataset_sources", "dataset_rows"):
+                conn.execute(f"DELETE FROM {t} WHERE dataset_id=?", (did,))
+        insert(conn, "datasets", {"id": did, "project_id": p["id"], "name": payload["name"], "kind": payload.get("kind", "sft"),
+                                  "version": payload.get("version", ""), "parent_id": payload.get("parent_id"),
+                                  "rows": payload.get("rows"), "tokens": payload.get("tokens"), "license": payload.get("license"),
+                                  "hf_repo": payload.get("hf_repo"), "description": payload.get("description", ""),
+                                  "created_at": time.time(), "processing": payload.get("processing", []),
+                                  "fields": payload.get("fields"), "source": payload.get("source", ""), "provenance": "published"})
+        for s in payload.get("sources", []):
+            insert(conn, "dataset_sources", {"dataset_id": did, "name": s.get("name"), "category": s.get("category"),
+                                             "rows": s.get("rows"), "tokens": s.get("tokens"), "synthetic": s.get("synthetic"),
+                                             "generator": s.get("generator"), "license": s.get("license"), "url": s.get("url")})
+        for i, r in enumerate(payload.get("samples", [])[:200]):
+            insert(conn, "dataset_rows", {"dataset_id": did, "idx": i, "source": r.get("source"), "category": r.get("category"),
+                                          "data": r.get("data", r), "tokens": r.get("tokens")})
+        return {"dataset_id": did, "updated": bool(existing)}
+    return workspace.write(fn)
+
+
+@router.post("/p/{org}/{project}/environments")
+def add_environment(org: str, project: str, request: Request, payload: dict = Body(...)):
+    auth(request)
+
+    def fn(conn):
+        p = project_row(conn, org, project)
+        existing = db.one(conn, "SELECT id FROM environments WHERE project_id=? AND name=?", (p["id"], payload["name"]))
+        eid = existing["id"] if existing else new_id("env")
+        gid = None
+        if payload.get("grader"):
+            g = payload["grader"]
+            gid = new_id("grader")
+            insert(conn, "graders", {"id": gid, "project_id": p["id"], "name": g.get("name", payload["name"] + "-grader"),
+                                     "kind": g.get("kind", "unit_tests"), "description": g.get("description", ""),
+                                     "components": g.get("components", []), "formula": g.get("formula", "")})
+        if existing:
+            conn.execute("DELETE FROM tasks WHERE env_id=?", (eid,))
+        tasks = payload.get("tasks", [])
+        insert(conn, "environments", {"id": eid, "project_id": p["id"], "name": payload["name"], "domain": payload.get("domain", "other"),
+                                      "version": payload.get("version", ""), "description": payload.get("description", ""),
+                                      "harness": payload.get("harness"), "tools": payload.get("tools", []), "grader_id": gid,
+                                      "reward_kind": payload.get("reward_kind", "binary"), "sandbox": payload.get("sandbox"),
+                                      "task_count": payload.get("task_count") or len(tasks), "created_at": time.time(),
+                                      "source": payload.get("source", ""), "provenance": "published", "checks": payload.get("checks")})
+        for t in tasks[:5000]:
+            insert(conn, "tasks", {"id": t.get("id") or new_id("task"), "env_id": eid, "name": t["name"],
+                                   "instruction": t.get("instruction", ""), "difficulty": t.get("difficulty"), "tags": t.get("tags", []),
+                                   "status": t.get("status", "ok"), "status_reason": t.get("status_reason", ""),
+                                   "oracle_score": t.get("oracle_score"), "noop_score": t.get("noop_score"),
+                                   "reruns": t.get("reruns"), "rerun_agree": t.get("rerun_agree"), "base_pass": t.get("base_pass"),
+                                   "latest_pass": t.get("latest_pass"), "attempts": t.get("attempts")})
+        return {"environment_id": eid, "updated": bool(existing), "tasks": min(len(tasks), 5000)}
+    return workspace.write(fn)
+
+
+@router.post("/p/{org}/{project}/models")
+def add_model(org: str, project: str, request: Request, payload: dict = Body(...)):
+    auth(request)
+
+    def fn(conn):
+        p = project_row(conn, org, project)
+        mid = new_id("model")
+        parent = resolve_model(conn, p["id"], payload.get("parent")) if payload.get("parent") else None
+        insert(conn, "models", {"id": mid, "project_id": p["id"], "name": payload["name"], "kind": payload.get("kind", "checkpoint"),
+                                "hf_repo": payload.get("hf_repo"), "arch": payload.get("arch"), "params_total": payload.get("params_total"),
+                                "params_active": payload.get("params_active"), "context_len": payload.get("context_len"),
+                                "parent_id": parent, "run_id": payload.get("run_id"), "step": payload.get("step"),
+                                "stage": payload.get("stage"), "created_at": time.time(), "status": payload.get("status", "available"),
+                                "notes": payload.get("notes", ""), "source": payload.get("source", "")})
+        return {"model_id": mid}
+    return workspace.write(fn)
+
+
+# ------------------------------------------------------------------ runs
+
+@router.post("/p/{org}/{project}/runs")
+def create_run(org: str, project: str, request: Request, payload: dict = Body(...)):
+    """Create a run. launch="runner" queues a job for a runner on `target`; otherwise the caller
+    (the CLI or a trainer via the SDK) executes it and reports progress."""
+    u = auth(request)
+    kind = payload.get("kind", "sft")
+    if kind not in KINDS:
+        raise HTTPException(422, f"kind must be one of {sorted(KINDS)}")
+
+    def fn(conn):
+        p = project_row(conn, org, project)
+        run_id = payload.get("id") or new_id("run")
+        base = resolve_model(conn, p["id"], payload.get("base_model"))
+        queued = payload.get("launch") == "runner"
+        now = time.time()
+        insert(conn, "runs", {
+            "id": run_id, "project_id": p["id"], "name": payload.get("name") or run_id, "kind": kind,
+            "stage": payload.get("stage") or {"sft": "SFT", "dpo": "Preference", "rl": "RL", "distill": "Distillation", "eval": "Eval"}.get(kind),
+            "algorithm": payload.get("algorithm"), "framework": payload.get("framework"),
+            "status": "queued" if queued else payload.get("status", "running"), "status_reason": "",
+            "base_model_id": base, "output_model_id": None, "started_at": now, "ended_at": None, "updated_at": now,
+            "steps_planned": payload.get("steps_planned"), "steps_done": 0, "primary_metric": payload.get("primary_metric"),
+            "gpu": payload.get("gpu"), "gpus": payload.get("gpus"), "cost_usd": None, "cost_rate": payload.get("cost_rate"),
+            "owner": u["name"], "tags": payload.get("tags", []), "code_ref": payload.get("code_ref", ""),
+            "config": payload.get("config", ""), "config_format": payload.get("config_format", "toml"),
+            "hyperparams": payload.get("hyperparams", {}), "parent_run_id": payload.get("parent_run_id"),
+            "group_name": payload.get("group"), "description": payload.get("description", ""), "source": payload.get("target", ""),
+            "provenance": "published"})
+        for inp in payload.get("inputs", []):
+            ref_id = resolve_input(conn, p["id"], inp.get("kind", "dataset"), inp["ref"])
+            insert(conn, "run_inputs", {"run_id": run_id, "kind": inp.get("kind", "dataset"), "ref_id": ref_id, "weight": inp.get("weight", 1.0)})
+        if payload.get("metric_defs"):
+            for d in payload["metric_defs"]:
+                insert(conn, "metric_defs", dict({"project_id": p["id"], "label": d.get("tag"), "description": "", "unit": "", "format": "num3",
+                                                  "grp": d.get("tag", "").split("/")[0], "better": "none", "pinned": 0, "signal": None}, **d))
+        job_id = None
+        if queued or payload.get("target"):
+            job_id = new_id("job")
+            insert(conn, "jobs", {"id": job_id, "project_id": p["id"], "run_id": run_id, "eval_id": None,
+                                  "name": f"{payload.get('name') or run_id}", "kind": "train" if kind != "eval" else "eval",
+                                  "status": "queued" if queued else "running", "cluster_id": None, "gpu": payload.get("gpu"),
+                                  "gpus": payload.get("gpus"), "nodes": payload.get("nodes"), "started_at": None if queued else now,
+                                  "ended_at": None, "cost_usd": None, "exit": None, "log_tail": "", "target": payload.get("target", "local"),
+                                  "spec": payload.get("spec", {}), "runner_id": None, "external_id": payload.get("external_id"),
+                                  "created_at": now, "claimed_at": None, "cancel": 0, "message": ""})
+        event(conn, run_id, "start" if not queued else "config", "Run created" if not queued else "Queued",
+              f"{kind.upper()} on {payload.get('target') or 'the caller'}" + (f", waiting for a runner" if queued else ""))
+        return {"run_id": run_id, "job_id": job_id, "url": f"/dashboard/{org}/{project}/runs/{run_id}"}
+    res = workspace.write(fn)
+    res["run_token"] = workspace.issue_token(f"run {res['run_id']}", u["name"], scope=f"run:{res['run_id']}", expires=time.time() + 7 * 86400 * 4)
+    return res
+
+
+@router.post("/runs/{run_id}/status")
+def run_status(run_id: str, request: Request, payload: dict = Body(...)):
+    auth(request, run_id, allow_run_token=True)
+    status = payload.get("status")
+    if status not in {"queued", "starting", "running", "stopping", "stalled", "completed", "failed", "stopped"}:
+        raise HTTPException(422, "status must be queued, starting, running, stopping, stalled, completed, failed or stopped")
+
+    def fn(conn):
+        r = run_row(conn, run_id)
+        now = time.time()
+        if r["status"] in TERMINAL and status != r["status"]:
+            # e.g. a runner starting a run that was canceled while it was being picked up: keep the end, tell the job to stop
+            return {"run_id": run_id, "status": r["status"], "ignored": True, "stop": True}
+        if r["status"] == "stopping" and status in ("queued", "starting", "running", "stalled"):
+            # a stop was requested: the job reporting that it runs does not undo it
+            conn.execute("UPDATE runs SET last_seen=?, updated_at=? WHERE id=?", (now, now, run_id))
+            return {"run_id": run_id, "status": "stopping", "ignored": True, "stop": True}
+        reason = summarize_reason(conn, run_id, payload.get("reason", ""))
+        sets = {"status": status, "status_reason": reason, "updated_at": now, "last_seen": now}
+        if status in TERMINAL:
+            sets["ended_at"] = now
+        if status == "running" and r["status"] == "queued":
+            sets["started_at"] = now
+        if payload.get("cost_usd") is not None:
+            sets["cost_usd"] = payload["cost_usd"]
+        if payload.get("output_model"):
+            mid = new_id("model")
+            insert(conn, "models", {"id": mid, "project_id": r["project_id"], "name": payload["output_model"], "kind": "checkpoint",
+                                    "hf_repo": payload.get("output_repo"), "parent_id": r["base_model_id"], "run_id": run_id,
+                                    "step": r["steps_done"], "stage": r["stage"], "created_at": now, "status": "available",
+                                    "notes": "", "source": ""})
+            sets["output_model_id"] = mid
+        conn.execute(f"UPDATE runs SET {','.join(k + '=?' for k in sets)} WHERE id=?", (*sets.values(), run_id))
+        if status in TERMINAL or status == "running":
+            conn.execute("UPDATE jobs SET status=?, ended_at=CASE WHEN ? THEN ? ELSE ended_at END, "
+                         "started_at=coalesce(started_at, ?) WHERE run_id=? AND status NOT IN ('completed','failed','stopped')",
+                         (status, status in TERMINAL, now, now, run_id))
+        if status in TERMINAL:
+            event(conn, run_id, "end", {"completed": "Run completed", "failed": "Run failed", "stopped": "Run stopped"}[status],
+                  reason, "error" if status == "failed" else "info", r["steps_done"])
+        elif status == "running" and r["status"] != "running":
+            event(conn, run_id, "start", "Running", reason)
+        return {"run_id": run_id, "status": status}
+    return workspace.write(fn)
+
+
+@router.post("/runs/{run_id}/heartbeat")
+def run_heartbeat(run_id: str, request: Request, payload: dict = Body(default={})):
+    """Sent every 30 s by the job (posttrain wrap / the SDK). Returns whether someone asked to stop the run."""
+    auth(request, run_id, allow_run_token=True)
+
+    def fn(conn):
+        r = run_row(conn, run_id)
+        now = time.time()
+        status = r["status"]
+        if status in TERMINAL:
+            return {"stop": True, "status": status}
+        if status in ("queued", "starting", "stalled"):
+            status = "running"
+            event(conn, run_id, "notice", "Reporting" if r["status"] == "stalled" else "Running",
+                  "Reports resumed." if r["status"] == "stalled" else "The job started reporting.")
+        conn.execute("UPDATE runs SET last_seen=?, updated_at=?, status=CASE WHEN status IN ('queued','starting','stalled') THEN ? ELSE status END, "
+                     "started_at=CASE WHEN status='queued' THEN ? ELSE started_at END WHERE id=?", (now, now, status, now, run_id))
+        conn.execute("UPDATE jobs SET status='running', started_at=coalesce(started_at, ?) WHERE run_id=? AND status IN ('queued','starting')", (now, run_id))
+        stop = conn.execute("SELECT max(coalesce(cancel,0)) FROM jobs WHERE run_id=?", (run_id,)).fetchone()[0]
+        return {"stop": bool(stop) or r["status"] == "stopping", "status": status}
+    return workspace.write(fn)
+
+
+@router.post("/runs/{run_id}/metrics")
+def run_metrics(run_id: str, request: Request, payload: dict = Body(...)):
+    """{"points": [[tag, step, value], ...]} or {"step": n, "values": {tag: value}}."""
+    auth(request, run_id, allow_run_token=True)
+    pts = [tuple(x) for x in payload.get("points", [])]
+    if "values" in payload:
+        pts += [(k, payload["step"], v) for k, v in payload["values"].items()]
+
+    def fn(conn):
+        r = run_row(conn, run_id)
+        seen(conn, r)
+        good = [(run_id, str(t), int(s), float(v)) for t, s, v in pts if isinstance(v, (int, float)) and v == v]
+        conn.executemany("INSERT OR REPLACE INTO metrics (run_id, tag, step, value) VALUES (?,?,?,?)", good)
+        if good:
+            top = max(s for _, _, s, _ in good)
+            conn.execute("UPDATE runs SET steps_done=max(coalesce(steps_done,0), ?), updated_at=?, "
+                         "primary_metric=coalesce(primary_metric, ?) WHERE id=?", (top, time.time(), payload.get("primary") or good[0][1], run_id))
+        return {"stored": len(good)}
+    return workspace.write(fn)
+
+
+@router.post("/runs/{run_id}/steps")
+def run_steps(run_id: str, request: Request, payload: dict = Body(...)):
+    auth(request, run_id, allow_run_token=True)
+
+    def fn(conn):
+        seen(conn, run_row(conn, run_id))
+        for row in payload.get("rows", []):
+            insert(conn, "run_steps", dict({"run_id": run_id, "phase": "train"}, **row))
+        return {"stored": len(payload.get("rows", []))}
+    return workspace.write(fn)
+
+
+@router.post("/runs/{run_id}/rollouts")
+def run_rollouts(run_id: str, request: Request, payload: dict = Body(...)):
+    """{"rows": [protocol rollout fields...], "transcripts": {rollout_id: messages}}"""
+    auth(request, run_id, allow_run_token=True)
+
+    def fn(conn):
+        r = run_row(conn, run_id)
+        seen(conn, r)
+        ids = []
+        for row in payload.get("rows", [])[:5000]:
+            rid = row.get("id") or new_id("roll")
+            ids.append(rid)
+            insert(conn, "rollouts", dict({"id": rid, "run_id": run_id, "phase": "train", "model_id": r["base_model_id"],
+                                           "trained": 1, "seed": 0}, **row))
+        for rid, msgs in (payload.get("transcripts") or {}).items():
+            insert(conn, "transcripts", {"rollout_id": rid, "messages": json.dumps(msgs)})
+        return {"stored": len(ids), "ids": ids}
+    return workspace.write(fn)
+
+
+@router.post("/runs/{run_id}/events")
+def run_events(run_id: str, request: Request, payload: dict = Body(...)):
+    auth(request, run_id, allow_run_token=True)
+
+    def fn(conn):
+        seen(conn, run_row(conn, run_id))
+        for e in payload.get("events", []):
+            event(conn, run_id, e.get("kind", "notice"), e.get("title", ""), e.get("body", ""), e.get("severity", "info"),
+                  e.get("step"), e.get("t"))
+        return {"stored": len(payload.get("events", []))}
+    return workspace.write(fn)
+
+
+@router.post("/runs/{run_id}/checkpoints")
+def run_checkpoint(run_id: str, request: Request, payload: dict = Body(...)):
+    auth(request, run_id, allow_run_token=True)
+
+    def fn(conn):
+        seen(conn, run_row(conn, run_id))
+        cid = new_id("ckpt")
+        insert(conn, "checkpoints", {"id": cid, "run_id": run_id, "step": payload.get("step"), "model_id": None,
+                                     "path": payload.get("path", ""), "size_gb": payload.get("size_gb"), "created_at": time.time(), "kept": 1})
+        event(conn, run_id, "checkpoint", f"Checkpoint step {payload.get('step')}", payload.get("path", ""), step=payload.get("step"))
+        return {"checkpoint_id": cid}
+    return workspace.write(fn)
+
+
+@router.post("/runs/{run_id}/logs")
+def post_logs(run_id: str, request: Request, payload: dict = Body(...)):
+    auth(request, run_id, allow_run_token=True)
+
+    def fn(conn):
+        seen(conn, run_row(conn, run_id))
+        seq = conn.execute("SELECT coalesce(max(seq), 0) FROM logs WHERE run_id=?", (run_id,)).fetchone()[0]
+        lines = payload.get("lines", [])
+        conn.executemany("INSERT INTO logs (run_id, seq, t, stream, text) VALUES (?,?,?,?,?)",
+                         [(run_id, seq + i + 1, x.get("t") or time.time(), x.get("stream", "stdout"), str(x.get("text", ""))[:4000])
+                          for i, x in enumerate(lines)])
+        return {"stored": len(lines), "last_seq": seq + len(lines)}
+    return workspace.write(fn)
+
+
+@router.get("/runs/{run_id}/logs")
+def get_logs(run_id: str, after: int = 0, limit: int = 2000):
+    conn = workspace.connect()
+    return db.rows(conn, "SELECT seq, t, stream, text FROM logs WHERE run_id=? AND seq>? ORDER BY seq LIMIT ?", (run_id, after, limit))
+
+
+@router.post("/runs/{run_id}/cancel")
+def cancel_run(run_id: str, request: Request):
+    auth(request, run_id, allow_runner=True)
+
+    def fn(conn):
+        r = run_row(conn, run_id)
+        if r["status"] in TERMINAL:
+            return {"run_id": run_id, "status": r["status"]}
+        queued = db.rows(conn, "SELECT id FROM jobs WHERE run_id=? AND status='queued'", (run_id,))
+        claimed = db.rows(conn, "SELECT id FROM jobs WHERE run_id=? AND status IN ('starting','running')", (run_id,))
+        conn.execute("UPDATE jobs SET cancel=1 WHERE run_id=? AND status NOT IN ('completed','failed','stopped')", (run_id,))
+        if not claimed and (queued or r["status"] == "queued"):
+            conn.execute("UPDATE jobs SET status='stopped', ended_at=? WHERE run_id=? AND status='queued'", (time.time(), run_id))
+            conn.execute("UPDATE runs SET status='stopped', status_reason='Canceled before it started.', ended_at=?, updated_at=? WHERE id=?",
+                         (time.time(), time.time(), run_id))
+            event(conn, run_id, "end", "Run stopped", "Canceled before it started.")
+            return {"run_id": run_id, "status": "stopped"}
+        conn.execute("UPDATE runs SET status='stopping', updated_at=? WHERE id=?", (time.time(), run_id))
+        if not db.one(conn, "SELECT id FROM jobs WHERE run_id=?", (run_id,)):
+            # a run reported by the SDK or wrap without a job row: record the stop request on a job so heartbeats see it
+            conn.execute("INSERT INTO jobs (id, project_id, run_id, status, cancel, created_at) VALUES (?,?,?,?,?,?)",
+                         (new_id("job"), r["project_id"], run_id, "running", 1, time.time()))
+        event(conn, run_id, "notice", "Stop requested", "The job will see it at its next heartbeat (within 30 s), save a checkpoint and exit.", "warning", r["steps_done"])
+        return {"run_id": run_id, "status": "stopping"}
+    return workspace.write(fn)
+
+
+@router.get("/runs/{run_id}/cancel")
+def cancel_requested(run_id: str, request: Request):
+    """Polled by a CLI that executes a run directly."""
+    conn = workspace.connect()
+    j = conn.execute("SELECT max(coalesce(cancel,0)) FROM jobs WHERE run_id=?", (run_id,)).fetchone()[0]
+    st = conn.execute("SELECT status FROM runs WHERE id=?", (run_id,)).fetchone()
+    return {"cancel": bool(j) or bool(st and st[0] == "stopping")}
+
+
+# ------------------------------------------------------------------ evals
+
+@router.post("/p/{org}/{project}/evals")
+def create_eval(org: str, project: str, request: Request, payload: dict = Body(...)):
+    """Create (or complete) an eval: benchmark by name (registered if new), model by name or HF id.
+    An eval job may call this with its own run token; the eval is then attached to that run."""
+    u, scoped = eval_caller(request)
+    if scoped:
+        if payload.get("run_id") not in (None, "", scoped):
+            raise HTTPException(403, "This token belongs to one run and can only record evals for that run.")
+        payload = dict(payload, run_id=scoped)
+
+    def fn(conn):
+        p = project_row(conn, org, project)
+        if scoped and run_row(conn, scoped)["project_id"] != p["id"]:
+            raise HTTPException(403, "This token's run is in another project.")
+        b = db.one(conn, "SELECT id FROM benchmarks WHERE project_id=? AND name=?", (p["id"], payload["benchmark"]))
+        if b:
+            bid = b["id"]
+        else:
+            bid = new_id("bench")
+            insert(conn, "benchmarks", {"id": bid, "project_id": p["id"], "name": payload["benchmark"], "version": payload.get("version", ""),
+                                        "category": payload.get("category", ""), "metric": payload.get("metric", "avg@1"),
+                                        "harness": payload.get("harness"), "n_tasks": payload.get("n_tasks"), "k": payload.get("k", 1),
+                                        "description": payload.get("description", ""), "source": payload.get("source", "")})
+        mid = resolve_model(conn, p["id"], payload.get("model"), kind="checkpoint") if payload.get("model") else None
+        eid = new_id("eval")
+        insert(conn, "evals", {"id": eid, "project_id": p["id"], "benchmark_id": bid, "model_id": mid, "run_id": payload.get("run_id"),
+                               "step": payload.get("step"), "status": payload.get("status", "running"), "score": payload.get("score"),
+                               "stderr": payload.get("stderr"), "n_tasks": payload.get("n_tasks"), "k": payload.get("k", 1),
+                               "n_infra": payload.get("n_infra"), "started_at": time.time(), "ended_at": None,
+                               "cost_usd": None, "config": payload.get("config"), "command": payload.get("command", ""),
+                               "source": payload.get("target", ""), "provenance": "published"})
+        return {"eval_id": eid, "benchmark_id": bid}
+    return workspace.write(fn)
+
+
+@router.post("/evals/{eval_id}/results")
+def eval_results(eval_id: str, request: Request, payload: dict = Body(...)):
+    u, scoped = eval_caller(request)
+
+    def fn(conn):
+        e = db.one(conn, "SELECT * FROM evals WHERE id=?", (eval_id,))
+        if not e:
+            raise HTTPException(404, f"No eval {eval_id}.")
+        if scoped and e["run_id"] != scoped:
+            raise HTTPException(403, "This token belongs to one run and can only record results for that run's evals.")
+        tasks = payload.get("tasks", [])
+        for t in tasks:
+            insert(conn, "eval_tasks", {"eval_id": eval_id, "task_id": t.get("task_id"), "task_name": t["task_name"],
+                                        "attempts": t.get("attempts", 1), "passes": t.get("passes"), "infra": t.get("infra", 0),
+                                        "score": t.get("score"), "mean_turns": t.get("mean_turns"), "mean_tokens": t.get("mean_tokens")})
+        score, se = payload.get("score"), payload.get("stderr")
+        if score is None and tasks:
+            vals = [t["score"] for t in tasks if t.get("score") is not None]
+            if vals:
+                score = sum(vals) / len(vals)
+                if len(vals) > 1:
+                    sd = (sum((v - score) ** 2 for v in vals) / (len(vals) - 1)) ** 0.5
+                    se = sd / len(vals) ** 0.5
+        conn.execute("UPDATE evals SET status=?, score=?, stderr=?, n_tasks=coalesce(?, n_tasks), n_infra=coalesce(?, n_infra), ended_at=? WHERE id=?",
+                     (payload.get("status", "completed"), score, se, len(tasks) or None, payload.get("n_infra"), time.time(), eval_id))
+        conn.execute("UPDATE benchmarks SET n_tasks=coalesce(n_tasks, ?) WHERE id=?", (len(tasks) or None, e["benchmark_id"]))
+        return {"eval_id": eval_id, "score": score, "stderr": se}
+    return workspace.write(fn)
+
+
+# ------------------------------------------------------------------ compute targets and runners
+
+@router.get("/orgs/{org}/compute")
+def list_compute(org: str):
+    conn = workspace.connect()
+    targets = db.rows(conn, "SELECT c.* FROM compute_targets c JOIN orgs o ON o.id=c.org_id WHERE o.slug=? ORDER BY c.name", (org,))
+    runners = db.rows(conn, "SELECT r.* FROM runners r JOIN orgs o ON o.id=r.org_id WHERE o.slug=? ORDER BY r.last_seen DESC", (org,))
+    now = time.time()
+    for t in targets:
+        t["config"] = json.loads(t["config"]) if isinstance(t["config"], str) else t["config"]
+        t["runners"] = [r["name"] for r in runners if t["name"] in (r.get("targets") or []) and now - (r["last_seen"] or 0) < 90]
+    for r in runners:
+        r["online"] = now - (r["last_seen"] or 0) < 90
+    queued = db.rows(conn, "SELECT j.id, j.target, j.name, j.created_at, j.run_id FROM jobs j JOIN projects p ON p.id=j.project_id "
+                           "JOIN orgs o ON o.id=p.org_id WHERE o.slug=? AND j.status='queued' ORDER BY j.created_at", (org,))
+    return {"targets": targets, "runners": runners, "queued": queued}
+
+
+@router.post("/orgs/{org}/compute")
+def add_compute(org: str, request: Request, payload: dict = Body(...)):
+    """Register a compute target. Only non-secret settings are stored; credentials stay with the
+    machine that runs jobs (the CLI or a runner)."""
+    u = auth(request)
+    kind = payload.get("kind")
+    if kind not in {"local", "ssh", "slurm", "prime", "hf-jobs", "fireworks"}:
+        raise HTTPException(422, "kind must be local, ssh, slurm, prime, hf-jobs or fireworks")
+    cfg = {k: v for k, v in (payload.get("config") or {}).items() if not any(s in k.lower() for s in ("token", "secret", "password", "key"))}
+
+    def fn(conn):
+        o = db.one(conn, "SELECT id FROM orgs WHERE slug=?", (org,))
+        if not o:
+            raise HTTPException(404, f"No org {org}. Create a project first.")
+        existing = db.one(conn, "SELECT id FROM compute_targets WHERE org_id=? AND name=?", (o["id"], payload["name"]))
+        cid = existing["id"] if existing else new_id("target")
+        insert(conn, "compute_targets", {"id": cid, "org_id": o["id"], "name": payload["name"], "kind": kind, "config": cfg,
+                                         "created_at": time.time(), "created_by": u["name"]})
+        return {"target_id": cid, "updated": bool(existing)}
+    return workspace.write(fn)
+
+
+@router.post("/runners/register")
+def register_runner(request: Request, payload: dict = Body(...)):
+    auth(request, allow_runner=True)
+
+    def fn(conn):
+        o = db.one(conn, "SELECT id FROM orgs WHERE slug=?", (payload["org"],))
+        if not o:
+            raise HTTPException(404, f"No org {payload['org']}.")
+        rid = payload.get("id") or new_id("runner")
+        insert(conn, "runners", {"id": rid, "org_id": o["id"], "name": payload.get("name") or rid, "hostname": payload.get("hostname", ""),
+                                 "targets": payload.get("targets", []), "version": payload.get("version", ""),
+                                 "started_at": time.time(), "last_seen": time.time()})
+        return {"runner_id": rid}
+    return workspace.write(fn)
+
+
+@router.post("/runners/{runner_id}/heartbeat")
+def heartbeat(runner_id: str, request: Request, payload: dict = Body(default={})):
+    auth(request, allow_runner=True)
+
+    def fn(conn):
+        conn.execute("UPDATE runners SET last_seen=? WHERE id=?", (time.time(), runner_id))
+        running = payload.get("running", [])
+        if running:
+            q0 = ",".join("?" for _ in running)
+            conn.execute(f"UPDATE jobs SET claimed_at=? WHERE id IN ({q0}) AND runner_id=?", (time.time(), *running, runner_id))  # renew leases
+        cancel = []
+        if running:
+            q = ",".join("?" for _ in running)
+            cancel = [r[0] for r in conn.execute(f"SELECT id FROM jobs WHERE id IN ({q}) AND coalesce(cancel,0)=1", running)]
+        return {"cancel": cancel}
+    return workspace.write(fn)
+
+
+@router.post("/runners/{runner_id}/claim")
+def claim(runner_id: str, request: Request):
+    """The oldest queued job for one of this runner's targets, now assigned to it (a lease renewed by heartbeat)."""
+    auth(request, allow_runner=True)
+
+    def fn(conn):
+        r = db.one(conn, "SELECT * FROM runners WHERE id=?", (runner_id,))
+        if not r:
+            raise HTTPException(404, "Unknown runner; register again.")
+        targets = r.get("targets") or []
+        if not targets:
+            return {"job": None}
+        q = ",".join("?" for _ in targets)
+        j = db.one(conn, f"SELECT j.*, p.slug AS project_slug, o.slug AS org_slug FROM jobs j JOIN projects p ON p.id=j.project_id "
+                         f"JOIN orgs o ON o.id=p.org_id WHERE j.status='queued' AND coalesce(j.cancel,0)=0 AND j.target IN ({q}) AND o.id=? ORDER BY j.created_at LIMIT 1",
+                   (*targets, r["org_id"]))
+        if not j:
+            conn.execute("UPDATE runners SET last_seen=? WHERE id=?", (time.time(), runner_id))
+            return {"job": None}
+        now = time.time()
+        conn.execute("UPDATE jobs SET status='starting', runner_id=?, claimed_at=? WHERE id=? AND status='queued'", (runner_id, now, j["id"]))
+        j.update(status="starting", runner_id=runner_id, claimed_at=now)
+        conn.execute("UPDATE runs SET status='starting', updated_at=? WHERE id=? AND status='queued'", (now, j["run_id"]))
+        event(conn, j["run_id"], "notice", "Picked up by a runner", f"{r['name']} on {r['hostname']} will run it on {j['target']}.")
+        run = db.one(conn, "SELECT * FROM runs WHERE id=?", (j["run_id"],))
+        return {"job": j, "run": run, "run_token": None}
+    res = workspace.write(fn)
+    if res.get("job"):
+        res["run_token"] = workspace.issue_token(f"run {res['job']['run_id']}", "runner", scope=f"run:{res['job']['run_id']}",
+                                                 expires=time.time() + 7 * 86400 * 4)
+    return res
+
+
+@router.post("/jobs/{job_id}/status")
+def job_status(job_id: str, request: Request, payload: dict = Body(...)):
+    auth(request, allow_runner=True)
+
+    def fn(conn):
+        j = db.one(conn, "SELECT * FROM jobs WHERE id=?", (job_id,))
+        if not j:
+            raise HTTPException(404, f"No job {job_id}.")
+        sets = {k: payload[k] for k in ("status", "external_id", "message", "exit", "cost_usd", "log_tail") if k in payload}
+        if payload.get("status") == "running" and not j["started_at"]:
+            sets["started_at"] = time.time()
+        if payload.get("status") in TERMINAL:
+            sets["ended_at"] = time.time()
+        if sets:
+            conn.execute(f"UPDATE jobs SET {','.join(k + '=?' for k in sets)} WHERE id=?", (*sets.values(), job_id))
+        return {"job_id": job_id, **sets}
+    return workspace.write(fn)

viewer/build/LAB_MODULES.md

@@ -0,0 +1,33 @@
+# Writing a lab module for the demo source
+
+The demo database shows what a post-training team's whole program looks like in the viewer: models and their lineage, datasets, RL environments and tasks, training runs (SFT, preference, RL, distillation), held-out evals, jobs, spend and written findings. Each lab gets one module, `viewer/build/labs/<lab>.py`, exposing `build(w, now)`.
+
+## Rules
+
+1. **Published numbers are the skeleton.** Take every number you can from the lab's dossier (`~/benchflow/pta-work/ref/viewer-v3/<lab>/DOSSIER.md` and `recipe.json`) and from real public run data (`~/benchflow/pta-work/public-runs/<run>/metrics.jsonl`, `attempts.jsonl`, `traces/`). Record the source URL on every record (`source=`).
+2. **Simulate only what isn't published, and consistently.** Rollouts, per-task results and missing curves are simulated with the engine so they agree with the published numbers (for example `rl_run(..., pass_start=, pass_end=)` or `env_targets` matching a published curve; `eval_run(score=published)`). Mark provenance: `published` (everything real), `mixed` (real metrics or scores, simulated rollouts/tasks), `simulated`.
+3. **Never invent concepts the lab doesn't have.** No runs, datasets or environments that the sources don't describe, except clearly-marked continuations that follow the lab's own stated next step (say which sentence in the source motivates it, in the run description). Anonymized names stay anonymized.
+4. **Denominators.** Benchmarks need a task count and attempts per task (`n_tasks`, `k`). If the count isn't published, pick a plausible one, say so in the benchmark `description`, and keep it ≤ 1,000 stored tasks.
+5. **Size budget.** Keep a lab under ~40 MB of SQLite: store ≤ 2,000 tasks per environment, `store_groups` 2–6 per step, ≤ 300 steps per run (log every step for RL; SFT logs are downsampled automatically).
+6. **Nothing harmful.** For security/cyber data, keep counts, grader kinds and domain names only: no exploit steps, payloads, proof-of-concept instructions or crash-reproduction prompts in task names, instructions, dataset samples or descriptions. The transcript renderer already withholds the `cyber` domain.
+7. **Self-contained build.** The demo is built from the repo alone (it deploys to a Space). Any file a module reads must live in `viewer/build/inputs/<lab>/` (gzip files over ~200 KB, keep a lab's inputs under ~15 MB, add a `SOURCE.md` with each file's origin and license) and be read via `Path(__file__).resolve().parent.parent / "inputs" / "<lab>"`. Never copy secrets or API keys. Hard-coding published numbers in the module is fine.
+8. **Only touch your own files**: `viewer/build/labs/<lab>.py` (and an optional `viewer/build/inputs/<lab>/` with a `SOURCE.md` saying where each file came from). Read the engine, don't edit it; if you need a helper, write it inside your module. Don't start or stop servers on port 7880 and don't commit.
+
+## The engine
+
+- `viewer/build/kit.py`: `org`, `project`, `model`, `dataset`, `grader`, `environment` (+ `write_tasks`), `import_metrics`, `metric_defs`, `cluster`, `jobs_for_run`, `usage_for_run`, `report`, `deployment`, `ts`.
+- `viewer/build/training.py`: `rl_run` (simulated RL with per-step aggregates computed from simulated attempts), `sft_run`, `dpo_run`, `benchmark`, `eval_run` (per-task results that average to the published score; `raw=True` for Elo/index scores).
+- `viewer/build/signals.py`: canonical signals and each framework's tag names (`verl`, `nemo_rl`, `prime_rl`, `open_instruct`, `skyrl`, `trl_sft`, `trl_dpo`, `megatron_sft`). Real imported tags should get `metric_defs` rows with a `signal` so the run page's health view finds them (`kit.metric_defs(w, pid, framework, extra=[...])`).
+- `viewer/build/labs/banks.py`: task-name/instruction generators per domain (`bank_for(domain)`); write your own bank in your module when the lab publishes real task examples.
+- `viewer/build/labs/mimo.py`: the reference module (real published metrics imported, rollouts simulated to match).
+- `viewer/PROTOCOL.md`: what each table means. `viewer/server.py`: how pages read it.
+
+## Check your work
+
+```bash
+cd ~/benchflow/pta-work/pta-space-viewer
+python3 -m viewer.build --only <lab> --out /tmp/<lab>.sqlite      # builds just your lab
+VIEWER_DATA=/tmp python3 -c "..."                                  # or query the file with sqlite3
+```
+
+Then look at the numbers the pages will show: runs (status, steps, primary metric first → last), evals (score ± SE per benchmark and model), environments (task counts, pass rates), datasets (rows, tokens, sources). Every published number you used should appear unchanged.

viewer/build/__main__.py

@@ -0,0 +1,57 @@
+"""Build the demo database: python -m viewer.build [--out path] [--only mimo,...]"""
+import argparse
+import importlib
+import time
+from pathlib import Path
+
+from .. import db
+from .sim import World
+
+LABS = ["mimo", "nemotron", "marin", "olmo", "prime", "openthoughts", "agentica"]
+NOW = 1790395200.0  # 2026-09-26 04:00 UTC, the demo's clock
+
+
+def check_spend(conn):
+    """One number per fact: a project's run costs and its usage ledger must agree (within 1%), because the
+    home page shows one and the overview and Usage page the other."""
+    bad = []
+    for pid, name in conn.execute("SELECT id, name FROM projects"):
+        runs = conn.execute("SELECT coalesce(sum(cost_usd), 0) FROM runs WHERE project_id=?", (pid,)).fetchone()[0]
+        usage = conn.execute("SELECT coalesce(sum(cost_usd), 0) FROM usage WHERE project_id=?", (pid,)).fetchone()[0]
+        if (runs or usage) and abs(runs - usage) > 0.01 * max(runs, usage):
+            bad.append(f"{name}: runs ${runs:,.0f} vs usage ${usage:,.0f}")
+    if bad:
+        raise SystemExit("Run costs and usage disagree:\n  " + "\n  ".join(bad))
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--out", default=str(db.DATA / "demo.sqlite"))
+    ap.add_argument("--only", default="")
+    args = ap.parse_args()
+    out = Path(args.out)
+    out.parent.mkdir(parents=True, exist_ok=True)
+    tmp = out.with_suffix(".building")
+    w = World(tmp, NOW)
+    w.meta(source="demo", now=NOW, built_at=time.time(), version="3")
+    labs = [x for x in args.only.split(",") if x] or LABS
+    for name in labs:
+        t0 = time.time()
+        try:
+            mod = importlib.import_module(f".labs.{name}", __package__)
+        except ModuleNotFoundError as e:
+            if e.name and e.name.endswith(f"labs.{name}"):
+                print(f"{name}: no module yet, skipped")
+                continue
+            raise
+        mod.build(w, NOW)
+        w.conn.commit()
+        print(f"{name}: {time.time() - t0:.1f}s")
+    check_spend(w.conn)
+    w.close()
+    tmp.replace(out)
+    print(out, f"{out.stat().st_size / 1e6:.1f} MB", w.counts)
+
+
+if __name__ == "__main__":
+    main()

viewer/build/inputs/agentica/SOURCE.md

@@ -0,0 +1,13 @@
+# Inputs for `labs/agentica.py`
+
+Everything here was copied from public sources on 2026-09-26. The viewer build reads only these files. Published numbers that are not in these files (card and blog scores, hyperparameters) are written in `labs/agentica.py` next to their source URL. Strings that look like credentials would have been replaced with `[redacted]` while copying; none were found.
+
+| File | What it is | Where it came from | License |
+|---|---|---|---|
+| `wandb_runs.json.gz` | Full logged history (every key, every step) and run metadata of the four runs in the public W&B project: `deepswe-preview-part1` (bx0o5d9l, steps 1-170), `deepswe-preview-part2` (fmwxpge7, 83 steps resumed from part 1's `global_step_170`), `swe-14b-no-overlong-filter-fail` (tzaqgde3, steps 41-273), `swe-sft-rl-fail` (dax4at2n, steps 0-100). The history `_timestamp`s are the 2025-07-01 upload time, not the training time. | https://wandb.ai/mluo/deepswe (anonymous GraphQL `history` per run) | none stated (public W&B project) |
+| `eval_runs_7of16.json.gz` | Per-instance results of 7 of the 16 official DeepSWE-Preview SWE-bench Verified evaluation runs (runs 0, 1, 2, 5, 10, 11, 13; 500 instances each): reward, exit reason, agent steps, tokens, LLM / environment / scoring time, patch size. Summarised from the run files inside `deepswe.zip`. | https://drive.google.com/file/d/10LIwpJeaFuiX6Y-qEG2a4a335PEuQJeS (linked from the DeepSWE blog) | released with the MIT-licensed DeepSWE project; no separate license stated |
+| `eval_trajectories.json.gz` | 112 of those trajectories (16 instances, chosen two per "solved in k of 7 runs" level, × 7 runs): the agent's steps converted to the viewer's message format (system prompt cut at 1,500 characters, issue at 1,800; thoughts, tool arguments and observations cut at 500; at most 30 steps kept, the rest summarised in a note) with reward, exit reason, token and time totals. | same `deepswe.zip` | as above |
+| `swebv_instances.json.gz` | The 500 SWE-bench Verified instance ids with repo, issue title and problem-statement length. | https://huggingface.co/datasets/R2E-Gym/SWE-Bench-Verified | not stated |
+| `r2e_gym_subset.json.gz` | For all 4,578 R2E-Gym-Subset rows: repo, commit, non-test files and lines changed, relevant files; real problem statements for 11 rows downloaded whole. | https://huggingface.co/datasets/R2E-Gym/R2E-Gym-Subset | apache-2.0 |
+
+The DeepSWE-Preview training curves are the same W&B series that `~/benchflow/pta-work/public-runs/agentica-deepswe-preview-qwen3-32b/` imported; here they are copied with every logged key. No training rollouts were released, so the run's rollouts in the demo are simulated to match each step's logged pass rate.

viewer/build/inputs/agentica/eval_runs_7of16.json.gz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9902393838a784d8602c56643b29b48bb96107100d7c6cd2ff21a1a621c7ade
+size 227893

viewer/build/inputs/agentica/eval_trajectories.json.gz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97106c146f75e00a28d5a3f8095b84274ed187ac6d58d9e71ca2b8238a866a7b
+size 643016

viewer/build/inputs/agentica/r2e_gym_subset.json.gz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ff5dbf29d1f08533a6c249539c4f7c973c4db8c3813e5dd9dfbd5a0b7e2b6a0
+size 65635

viewer/build/inputs/agentica/swebv_instances.json.gz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:14f8c1be623803c392b481f35a4ac46e3e567bad0c8a38ec8d60915e0a9d7bd4
+size 19828

viewer/build/inputs/agentica/wandb_runs.json.gz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a22b794efafc896ff2fdb660fde298ce438b00a8b5daba0b45518c7060928542
+size 184638

viewer/build/inputs/marin/SOURCE.md

@@ -0,0 +1,49 @@
+# Inputs for `labs/marin.py`
+
+Every file here was copied from our research workspace (`~/benchflow/pta-work`) on 2026-09-26. Nothing here contains credentials. All published numbers not read from these files are hard-coded in `labs/marin.py` with their source URL.
+
+## `recipe.json.gz`
+
+The Marin dossier's machine-readable recipe (`ref/viewer-v3/marin/recipe.json`, schema `ref/viewer-v3/SCHEMA.md`), compiled on 2026-09-26 from public sources: `marin-community/marin` issues and docs (repo at commit `f97c9c5d08be`, Apache-2.0), Hugging Face model and dataset cards (`marin-community`, `open-athena`, `laion`, `penfever`), and Marin's public reports on `storage.googleapis.com/marin-public`. Every entry carries its own source URL.
+
+One change from the original: the example-task instruction of three environments (the TaskTrove safety source and two Nemotron RL Ultra jailbreak domains) is replaced by `[withheld: safety/jailbreak prompt text is not copied]`, so no harmful prompt text enters the repository. Gzipped.
+
+The module reads its environments (TaskTrove Clean kept sources with grader modes, validation verdicts and example tasks; Nemotron RL Ultra domain agents), the Datakit SFT source registry, the RLVR1 example prompts and the 26-benchmark eval-policy panel (15 models each) from it.
+
+## `runs/<id>/run.json` and `runs/<id>/metrics.jsonl`
+
+Exact copies of `public-runs/<id>/`, written by our importers `importers/train_marin.py` (training curves) and `importers/rollouts_snowball_v104.py` (v104). `metrics.jsonl` has one row per logged step with the framework's own tags, no smoothing; `run.json` has the metadata and provenance. Known label problems in `run.json` are corrected in the module and noted in the run descriptions (v104 is RLOO-N at lr 8e-6, not GRPO at 4e-6; the #7785 arms' method label; pymethods2test-large's reported 0.74 vs its curve).
+
+| Folder | Upstream file(s) | License |
+|---|---|---|
+| `runs/marin-a3-inferredbugs/` | https://huggingface.co/laion/a3-rl-DCAgent_inferredbugs-sandboxes-verifier-55-8B/blob/main/training_logs/20260525_030311_metrics_table.csv | not stated |
+| `runs/marin-a3-llm-verifier-freelancer/` | https://huggingface.co/laion/a3-rl-DCAgent_llm-verifier-freelancer-70-8B/blob/main/training_logs/20260525_084909_metrics_table.csv | not stated |
+| `runs/marin-a3-nemotron-agent-calendar/` | https://huggingface.co/laion/a3-rl-laion_nemotron-gym-agent-calendar-80-8B/blob/main/training_logs/20260602_174231_metrics_table.csv | Apache-2.0 |
+| `runs/marin-a3-nl2bash/` | https://huggingface.co/laion/a3-rl-DCAgent2_nl2bash-tasks-cleaned-oracle-40-8B/blob/main/training_logs/20260526_134647_metrics_table.csv | not stated |
+| `runs/marin-a3-pymethods2test-large/` | https://huggingface.co/laion/a3-rl-DCAgent_exp_rpt_pymethods2test-large-80-8B/blob/main/training_logs/20260605_113517_metrics_table.csv (cut after step 80) | Apache-2.0 |
+| `runs/marin-q3c-tt-x3-kl0p001/` | https://huggingface.co/laion/tt-x3_kl-kl0p001-76-30B/blob/main/training_logs/metrics.csv | Apache-2.0 |
+| `runs/marin-q3c-tt-x5-gradnorm0p45/` | https://huggingface.co/laion/tt-x5_gradnorm-gn0p45-30-30B/blob/main/training_logs/metrics.csv | Apache-2.0 |
+| `runs/marin-q3c-tt-x10-fsdp2/` | https://huggingface.co/laion/tt-x10-fsdp2-fa2-117-30B/blob/main/training_logs/metrics.csv | Apache-2.0 |
+| `runs/marin-q3c-tt-x15-megatron/` | https://huggingface.co/laion/tt-x15-megatron-51-30B/blob/main/training_logs/metrics.csv | Apache-2.0 |
+| `runs/marin-q3c-cal-if-rloo-lr2/` | https://huggingface.co/penfever/qwen3coder-calendar-if-v49-lr2-step18/blob/main/training_logs/finelog.log and https://huggingface.co/datasets/penfever/qwen3coder-iris-rl-data-sweep-artifacts/blob/main/top-ten-checkpoint-fixed-validation.csv | not stated |
+| `runs/marin-q3c-cal-agent-rloo-lr2/` | https://huggingface.co/penfever/qwen3coder-calendar-agent-v49-lr2-step12/blob/main/training_logs/finelog.log and the same validation CSV | not stated |
+| `runs/marin-q3c-cal-agent-rloo-lr4/` | https://huggingface.co/penfever/qwen3coder-calendar-agent-v49-lr4-step9/blob/main/training_logs/finelog.log and the same validation CSV | not stated |
+| `runs/marin-snowball-e6-rlvr-math/` | https://storage.googleapis.com/marin-public/benjaminfeuer/passk-pass1-comparison/2026.08.31/run-metrics/e6-original.csv, https://huggingface.co/datasets/penfever/snowball-67b-a2b-math-rl-artifacts/blob/main/html-report/reward_curves.csv, https://huggingface.co/datasets/penfever/snowball-67b-a2b-math-rl-artifacts/blob/main/MATH_EVALS.md | not stated |
+| `runs/marin-snowball-e11-deepscaler-dapo/` | .../run-metrics/e11-deepscaler-dapo.csv plus the same reward_curves.csv and MATH_EVALS.md | not stated |
+| `runs/marin-snowball-e12-deepscaler-grpo/` | .../run-metrics/e12-deepscaler-grpo.csv plus the same reward_curves.csv and MATH_EVALS.md | not stated |
+| `runs/marin-snowball-v104-rlvr1-traces/` | `open-athena/Snowball-67B-A2B-Mixed-RLVR-Experiment-Artifacts@0ed714e5afbe11c13dfb227d6b13ac911a27d253:01-provenance-and-history/history/v104-termination-20260917` (https://huggingface.co/datasets/open-athena/Snowball-67B-A2B-Mixed-RLVR-Experiment-Artifacts) | not stated on the card; prompts come from nvidia/Nemotron-RL-Ultra-Training-Blends (components CC BY-SA 4.0, CC BY 4.0, ODC-BY 1.0, MIT, Apache-2.0) |
+
+## `runs/marin-snowball-v104-rlvr1-traces/attempts.jsonl.gz`
+
+`public-runs/marin-snowball-v104-rlvr1-traces/attempts.jsonl`, gzipped, unchanged: 1,952 attempts (1,152 training rollouts = 36 random complete 16-rollout groups from each of steps 1 and 15; 800 holdout samples = 100 prompts at steps 0, 2, …, 14) with the published rewards, stop reasons, token counts and domains. Same origin and license as the row above.
+
+## `runs/marin-snowball-v104-rlvr1-traces/transcripts.json.gz`
+
+Derived from `public-runs/marin-snowball-v104-rlvr1-traces/traces/*.json` (46 MB, not copied) by a one-off script with these rules:
+
+- Rollouts of the four `jailbreak_*` domains are skipped entirely (no prompt or response text).
+- Messages are converted to the viewer's format (system, user, assistant with `reasoning` and `tool_calls`, tool results, a final note with the last 300 characters of verifier output) and cut: system 200 characters, first user message 900 (later ones 400), assistant text 700, reasoning 350, tool arguments 200, tool results 250; cut text ends with "… [cut]".
+- A transcript longer than 12 messages keeps the first 6 and last 5 with a note in between.
+- `prompts` holds each task's first user message (up to 2,000 characters), keyed `train:<task>` or `eval:<task>`, because the importer's task ids (`rlvr1-<row index>`) restart per file and collide between training and holdout.
+
+Result: 1,864 transcripts and 164 prompts, about 4.0 MB uncompressed. The module stores full transcripts for the 800 holdout samples and for 12 of the 36 stored groups per training step, and a short pointer note for the other real training rollouts.

viewer/build/inputs/marin/recipe.json.gz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7df33e6fd7306457d96eb0acbc6668965070d71b9a63d4bf72db6a9c51958d18
+size 53623

viewer/build/inputs/marin/runs/marin-a3-inferredbugs/metrics.jsonl

@@ -0,0 +1,78 @@
+{"step":1,"async/staleness_mean":0.0,"generate/avg_num_tokens":3554.5293,"generate/avg_tokens_non_zero_rewards":5487.5524,"generate/avg_tokens_zero_rewards":2210.3742,"generate/max_num_tokens":27297,"generate/std_num_tokens":4153.5765,"loss/avg_final_rewards":0.4102,"loss/avg_raw_advantages":0.0484,"loss/avg_raw_advantages_abs":0.2364,"policy/policy_entropy":0.0552,"policy/policy_loss":-0.0059,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0288,"reward/avg_pass_at_8":0.6562,"reward/avg_raw_reward":0.4102,"timing/step":8293.4295,"trainer/epoch":0}
+{"step":2,"async/staleness_mean":1.0,"generate/avg_num_tokens":3575.0723,"generate/avg_tokens_non_zero_rewards":6572.3139,"generate/avg_tokens_zero_rewards":2480.08,"generate/max_num_tokens":30777,"generate/std_num_tokens":5688.8025,"loss/avg_final_rewards":0.2676,"loss/avg_raw_advantages":0.0765,"loss/avg_raw_advantages_abs":0.1962,"policy/policy_entropy":0.0396,"policy/policy_loss":-0.0083,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0419,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.2676,"timing/step":664.4752,"trainer/epoch":0}
+{"step":3,"async/staleness_mean":1.4531,"generate/avg_num_tokens":2257.6348,"generate/avg_tokens_non_zero_rewards":5324.374,"generate/avg_tokens_zero_rewards":1203.1916,"generate/max_num_tokens":30351,"generate/std_num_tokens":3596.3146,"loss/avg_final_rewards":0.2559,"loss/avg_raw_advantages":0.1299,"loss/avg_raw_advantages_abs":0.2503,"policy/policy_entropy":0.0375,"policy/policy_loss":-0.0294,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0446,"reward/avg_pass_at_8":0.5156,"reward/avg_raw_reward":0.2559,"timing/step":6178.9035,"trainer/epoch":0}
+{"step":4,"async/staleness_mean":1.5625,"generate/avg_num_tokens":2162.7051,"generate/avg_tokens_non_zero_rewards":6591.2113,"generate/avg_tokens_zero_rewards":1449.7256,"generate/max_num_tokens":30986,"generate/std_num_tokens":4168.7167,"loss/avg_final_rewards":0.1387,"loss/avg_raw_advantages":0.0369,"loss/avg_raw_advantages_abs":0.2575,"policy/policy_entropy":0.0219,"policy/policy_loss":-0.0087,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.016,"reward/avg_pass_at_8":0.375,"reward/avg_raw_reward":0.1387,"timing/step":1267.1686,"trainer/epoch":0}
+{"step":5,"async/staleness_mean":1.3906,"generate/avg_num_tokens":1752.9648,"generate/avg_tokens_non_zero_rewards":7083.6949,"generate/avg_tokens_zero_rewards":1058.6755,"generate/max_num_tokens":22939,"generate/std_num_tokens":3735.264,"loss/avg_final_rewards":0.1152,"loss/avg_raw_advantages":0.1295,"loss/avg_raw_advantages_abs":0.2618,"policy/policy_entropy":0.0152,"policy/policy_loss":-0.0298,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0113,"reward/avg_pass_at_8":0.4062,"reward/avg_raw_reward":0.1152,"timing/step":5785.6711,"trainer/epoch":0}
+{"step":6,"async/staleness_mean":1.1719,"generate/avg_num_tokens":1250.6895,"generate/avg_tokens_non_zero_rewards":6531.6818,"generate/avg_tokens_zero_rewards":754.1859,"generate/max_num_tokens":27177,"generate/std_num_tokens":3653.1449,"loss/avg_final_rewards":0.0859,"loss/avg_raw_advantages":0.0677,"loss/avg_raw_advantages_abs":0.2001,"policy/policy_entropy":0.0082,"policy/policy_loss":-0.0176,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0091,"reward/avg_pass_at_8":0.25,"reward/avg_raw_reward":0.0859,"timing/step":1901.8618,"trainer/epoch":0}
+{"step":7,"async/staleness_mean":0.1562,"generate/avg_num_tokens":2757.6699,"generate/avg_tokens_non_zero_rewards":5131.4013,"generate/avg_tokens_zero_rewards":1755.4278,"generate/max_num_tokens":30895,"generate/std_num_tokens":4025.9714,"loss/avg_final_rewards":0.2969,"loss/avg_raw_advantages":0.0821,"loss/avg_raw_advantages_abs":0.2883,"policy/policy_entropy":0.0327,"policy/policy_loss":-0.0375,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0242,"reward/avg_pass_at_8":0.6333,"reward/avg_raw_reward":0.2969,"timing/step":8205.9669,"trainer/epoch":0}
+{"step":8,"async/staleness_mean":1.0,"generate/avg_num_tokens":2169.2109,"generate/avg_tokens_non_zero_rewards":7972.4857,"generate/avg_tokens_zero_rewards":1250.1403,"generate/max_num_tokens":30300,"generate/std_num_tokens":4891.1346,"loss/avg_final_rewards":0.1367,"loss/avg_raw_advantages":0.0692,"loss/avg_raw_advantages_abs":0.1725,"policy/policy_entropy":0.0148,"policy/policy_loss":-0.0241,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0109,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.1367,"timing/step":1319.1071,"trainer/epoch":0}
+{"step":9,"async/staleness_mean":1.4844,"generate/avg_num_tokens":3028.4199,"generate/avg_tokens_non_zero_rewards":5770.2286,"generate/avg_tokens_zero_rewards":1996.5565,"generate/max_num_tokens":29148,"generate/std_num_tokens":4542.8655,"loss/avg_final_rewards":0.2734,"loss/avg_raw_advantages":0.0444,"loss/avg_raw_advantages_abs":0.1845,"policy/policy_entropy":0.0292,"policy/policy_loss":-0.0213,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0121,"reward/avg_pass_at_8":0.5156,"reward/avg_raw_reward":0.2734,"timing/step":5804.1709,"trainer/epoch":0}
+{"step":10,"async/staleness_mean":1.5,"generate/avg_num_tokens":2115.5254,"generate/avg_tokens_non_zero_rewards":7133.6164,"generate/avg_tokens_zero_rewards":1281.082,"generate/max_num_tokens":28527,"generate/std_num_tokens":4442.122,"loss/avg_final_rewards":0.1426,"loss/avg_raw_advantages":0.0836,"loss/avg_raw_advantages_abs":0.1262,"policy/policy_entropy":0.0158,"policy/policy_loss":-0.0213,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0079,"reward/avg_pass_at_8":0.4531,"reward/avg_raw_reward":0.1426,"timing/step":1543.1181,"trainer/epoch":0}
+{"step":11,"async/staleness_mean":1.3594,"generate/avg_num_tokens":2746.5098,"generate/avg_tokens_non_zero_rewards":5941.9194,"generate/avg_tokens_zero_rewards":1725.2964,"generate/max_num_tokens":24484,"generate/std_num_tokens":4396.7582,"loss/avg_final_rewards":0.2422,"loss/avg_raw_advantages":0.0111,"loss/avg_raw_advantages_abs":0.1144,"policy/policy_entropy":0.0227,"policy/policy_loss":-0.0056,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0111,"reward/avg_pass_at_8":0.5,"reward/avg_raw_reward":0.2422,"timing/step":5597.2787,"trainer/epoch":0}
+{"step":12,"async/staleness_mean":1.6094,"generate/avg_num_tokens":1974.1328,"generate/avg_tokens_non_zero_rewards":6117.1944,"generate/avg_tokens_zero_rewards":866.5817,"generate/max_num_tokens":21058,"generate/std_num_tokens":3670.2518,"loss/avg_final_rewards":0.2109,"loss/avg_raw_advantages":0.1302,"loss/avg_raw_advantages_abs":0.2538,"policy/policy_entropy":0.0208,"policy/policy_loss":-0.0273,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.014,"reward/avg_pass_at_8":0.4531,"reward/avg_raw_reward":0.2109,"timing/step":2062.4401,"trainer/epoch":0}
+{"step":13,"async/staleness_mean":1.625,"generate/avg_num_tokens":2930.0977,"generate/avg_tokens_non_zero_rewards":6136.312,"generate/avg_tokens_zero_rewards":1894.4987,"generate/max_num_tokens":29462,"generate/std_num_tokens":4844.6221,"loss/avg_final_rewards":0.2441,"loss/avg_raw_advantages":0.0868,"loss/avg_raw_advantages_abs":0.1834,"policy/policy_entropy":0.0232,"policy/policy_loss":-0.0514,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.014,"reward/avg_pass_at_8":0.5,"reward/avg_raw_reward":0.2441,"timing/step":4021.5429,"trainer/epoch":0}
+{"step":14,"async/staleness_mean":1.875,"generate/avg_num_tokens":2880.4043,"generate/avg_tokens_non_zero_rewards":6461.1908,"generate/avg_tokens_zero_rewards":1649.2152,"generate/max_num_tokens":27936,"generate/std_num_tokens":4381.1046,"loss/avg_final_rewards":0.2559,"loss/avg_raw_advantages":0.0729,"loss/avg_raw_advantages_abs":0.2213,"policy/policy_entropy":0.0232,"policy/policy_loss":-0.0343,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.016,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.2559,"timing/step":3677.0767,"trainer/epoch":0}
+{"step":15,"async/staleness_mean":0.0,"generate/avg_num_tokens":3899.7969,"generate/avg_tokens_non_zero_rewards":7074.6913,"generate/avg_tokens_zero_rewards":2596.6033,"generate/max_num_tokens":26927,"generate/std_num_tokens":5093.0614,"loss/avg_final_rewards":0.291,"loss/avg_raw_advantages":0.0352,"loss/avg_raw_advantages_abs":0.151,"policy/policy_entropy":0.0258,"policy/policy_loss":-0.0172,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0088,"reward/avg_pass_at_8":0.4688,"reward/avg_raw_reward":0.291,"timing/step":8365.709,"trainer/epoch":0}
+{"step":16,"async/staleness_mean":1.0,"generate/avg_num_tokens":3354.5391,"generate/avg_tokens_non_zero_rewards":8819.5865,"generate/avg_tokens_zero_rewards":1961.4877,"generate/max_num_tokens":30311,"generate/std_num_tokens":5355.9926,"loss/avg_final_rewards":0.2031,"loss/avg_raw_advantages":0.0644,"loss/avg_raw_advantages_abs":0.1826,"policy/policy_entropy":0.0184,"policy/policy_loss":-0.02,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.015,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.2031,"timing/step":1673.4662,"trainer/epoch":0}
+{"step":17,"async/staleness_mean":1.6719,"generate/avg_num_tokens":3884.9746,"generate/avg_tokens_non_zero_rewards":5600.6923,"generate/avg_tokens_zero_rewards":2938.7303,"generate/max_num_tokens":27180,"generate/std_num_tokens":4337.686,"loss/avg_final_rewards":0.3555,"loss/avg_raw_advantages":0.0129,"loss/avg_raw_advantages_abs":0.1233,"policy/policy_entropy":0.0369,"policy/policy_loss":-0.0086,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0208,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.3555,"timing/step":5026.4629,"trainer/epoch":0}
+{"step":18,"async/staleness_mean":1.9531,"generate/avg_num_tokens":3891.8926,"generate/avg_tokens_non_zero_rewards":7625.4897,"generate/avg_tokens_zero_rewards":2416.7657,"generate/max_num_tokens":30131,"generate/std_num_tokens":5490.3941,"loss/avg_final_rewards":0.2832,"loss/avg_raw_advantages":0.023,"loss/avg_raw_advantages_abs":0.2999,"policy/policy_entropy":0.0211,"policy/policy_loss":-0.0187,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0128,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.2832,"timing/step":1340.1342,"trainer/epoch":0}
+{"step":19,"async/staleness_mean":1.7969,"generate/avg_num_tokens":3556.9355,"generate/avg_tokens_non_zero_rewards":6212.4203,"generate/avg_tokens_zero_rewards":2577.1043,"generate/max_num_tokens":31194,"generate/std_num_tokens":5555.2022,"loss/avg_final_rewards":0.2695,"loss/avg_raw_advantages":0.0373,"loss/avg_raw_advantages_abs":0.1216,"policy/policy_entropy":0.0214,"policy/policy_loss":-0.0168,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0171,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.2695,"timing/step":3726.9289,"trainer/epoch":0}
+{"step":20,"async/staleness_mean":1.9531,"generate/avg_num_tokens":4239.8027,"generate/avg_tokens_non_zero_rewards":5839.0,"generate/avg_tokens_zero_rewards":3136.7261,"generate/max_num_tokens":27132,"generate/std_num_tokens":5137.5811,"loss/avg_final_rewards":0.4082,"loss/avg_raw_advantages":0.0127,"loss/avg_raw_advantages_abs":0.1571,"policy/policy_entropy":0.0319,"policy/policy_loss":-0.0089,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0168,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.4082,"timing/step":3066.7613,"trainer/epoch":0}
+{"step":21,"async/staleness_mean":1.5312,"generate/avg_num_tokens":3199.2871,"generate/avg_tokens_non_zero_rewards":5553.7681,"generate/avg_tokens_zero_rewards":2330.5214,"generate/max_num_tokens":30770,"generate/std_num_tokens":4916.8226,"loss/avg_final_rewards":0.2695,"loss/avg_raw_advantages":0.021,"loss/avg_raw_advantages_abs":0.1296,"policy/policy_entropy":0.0252,"policy/policy_loss":-0.0058,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0133,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.2695,"timing/step":2831.6951,"trainer/epoch":0}
+{"step":22,"async/staleness_mean":1.9219,"generate/avg_num_tokens":3256.2715,"generate/avg_tokens_non_zero_rewards":5572.2785,"generate/avg_tokens_zero_rewards":2222.5734,"generate/max_num_tokens":20250,"generate/std_num_tokens":4011.3658,"loss/avg_final_rewards":0.3086,"loss/avg_raw_advantages":0.0222,"loss/avg_raw_advantages_abs":0.1556,"policy/policy_entropy":0.0298,"policy/policy_loss":-0.008,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0123,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.3086,"timing/step":3108.7729,"trainer/epoch":0}
+{"step":23,"async/staleness_mean":1.8438,"generate/avg_num_tokens":3942.6367,"generate/avg_tokens_non_zero_rewards":6538.8837,"generate/avg_tokens_zero_rewards":2629.2412,"generate/max_num_tokens":31316,"generate/std_num_tokens":5134.6975,"loss/avg_final_rewards":0.3359,"loss/avg_raw_advantages":0.0132,"loss/avg_raw_advantages_abs":0.1686,"policy/policy_entropy":0.0287,"policy/policy_loss":-0.0107,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0668,"reward/avg_pass_at_8":0.5469,"reward/avg_raw_reward":0.3359,"timing/step":2508.6885,"trainer/epoch":0}
+{"step":24,"async/staleness_mean":2.0781,"generate/avg_num_tokens":3206.0605,"generate/avg_tokens_non_zero_rewards":5638.4586,"generate/avg_tokens_zero_rewards":2130.3239,"generate/max_num_tokens":31119,"generate/std_num_tokens":4114.0874,"loss/avg_final_rewards":0.3066,"loss/avg_raw_advantages":0.0066,"loss/avg_raw_advantages_abs":0.1306,"policy/policy_entropy":0.0305,"policy/policy_loss":-0.0006,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0121,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.3066,"timing/step":2153.3934,"trainer/epoch":0}
+{"step":25,"async/staleness_mean":2.0312,"generate/avg_num_tokens":4323.1875,"generate/avg_tokens_non_zero_rewards":6594.0226,"generate/avg_tokens_zero_rewards":3123.3731,"generate/max_num_tokens":28049,"generate/std_num_tokens":4973.1141,"loss/avg_final_rewards":0.3457,"loss/avg_raw_advantages":0.0054,"loss/avg_raw_advantages_abs":0.1836,"policy/policy_entropy":0.0244,"policy/policy_loss":-0.0106,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0156,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.3457,"timing/step":3221.5509,"trainer/epoch":0}
+{"step":26,"async/staleness_mean":2.0,"generate/avg_num_tokens":3369.8633,"generate/avg_tokens_non_zero_rewards":6680.9032,"generate/avg_tokens_zero_rewards":1480.7423,"generate/max_num_tokens":23175,"generate/std_num_tokens":4426.3121,"loss/avg_final_rewards":0.3633,"loss/avg_raw_advantages":0.0363,"loss/avg_raw_advantages_abs":0.191,"policy/policy_entropy":0.0255,"policy/policy_loss":-0.0125,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.014,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.3633,"timing/step":3009.1833,"trainer/epoch":0}
+{"step":27,"async/staleness_mean":0.3906,"generate/avg_num_tokens":4734.0098,"generate/avg_tokens_non_zero_rewards":5301.7187,"generate/avg_tokens_zero_rewards":3730.5459,"generate/max_num_tokens":20084,"generate/std_num_tokens":3683.1818,"loss/avg_final_rewards":0.6387,"loss/avg_raw_advantages":0.0168,"loss/avg_raw_advantages_abs":0.1982,"policy/policy_entropy":0.0468,"policy/policy_loss":-0.0076,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0172,"reward/avg_pass_at_8":0.8,"reward/avg_raw_reward":0.6387,"timing/step":8253.2289,"trainer/epoch":0}
+{"step":28,"async/staleness_mean":1.0,"generate/avg_num_tokens":3503.1445,"generate/avg_tokens_non_zero_rewards":7824.5556,"generate/avg_tokens_zero_rewards":2467.2615,"generate/max_num_tokens":30411,"generate/std_num_tokens":5444.356,"loss/avg_final_rewards":0.1934,"loss/avg_raw_advantages":0.0426,"loss/avg_raw_advantages_abs":0.1969,"policy/policy_entropy":0.0153,"policy/policy_loss":-0.0174,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0089,"reward/avg_pass_at_8":0.4219,"reward/avg_raw_reward":0.1934,"timing/step":877.8049,"trainer/epoch":0}
+{"step":29,"async/staleness_mean":1.7812,"generate/avg_num_tokens":3965.5879,"generate/avg_tokens_non_zero_rewards":4949.3244,"generate/avg_tokens_zero_rewards":3194.3659,"generate/max_num_tokens":29557,"generate/std_num_tokens":4568.2977,"loss/avg_final_rewards":0.4395,"loss/avg_raw_advantages":0.0079,"loss/avg_raw_advantages_abs":0.1816,"policy/policy_entropy":0.0326,"policy/policy_loss":-0.0088,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0167,"reward/avg_pass_at_8":0.6719,"reward/avg_raw_reward":0.4395,"timing/step":2373.1954,"trainer/epoch":0}
+{"step":30,"async/staleness_mean":2.1562,"generate/avg_num_tokens":4404.9922,"generate/avg_tokens_non_zero_rewards":6360.2297,"generate/avg_tokens_zero_rewards":2908.2241,"generate/max_num_tokens":29058,"generate/std_num_tokens":4579.7426,"loss/avg_final_rewards":0.4336,"loss/avg_raw_advantages":0.0075,"loss/avg_raw_advantages_abs":0.1682,"policy/policy_entropy":0.033,"policy/policy_loss":-0.0065,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0156,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.4336,"timing/step":3572.8006,"trainer/epoch":0}
+{"step":31,"async/staleness_mean":2.5781,"generate/avg_num_tokens":3999.6875,"generate/avg_tokens_non_zero_rewards":6186.3742,"generate/avg_tokens_zero_rewards":3050.2857,"generate/max_num_tokens":29749,"generate/std_num_tokens":4909.8164,"loss/avg_final_rewards":0.3027,"loss/avg_raw_advantages":0.0018,"loss/avg_raw_advantages_abs":0.1616,"policy/policy_entropy":0.0305,"policy/policy_loss":-0.0124,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0204,"reward/avg_pass_at_8":0.6094,"reward/avg_raw_reward":0.3027,"timing/step":1794.4302,"trainer/epoch":0}
+{"step":32,"async/staleness_mean":2.3594,"generate/avg_num_tokens":3965.4336,"generate/avg_tokens_non_zero_rewards":6360.0714,"generate/avg_tokens_zero_rewards":2644.7545,"generate/max_num_tokens":24908,"generate/std_num_tokens":4427.5009,"loss/avg_final_rewards":0.3555,"loss/avg_raw_advantages":0.0451,"loss/avg_raw_advantages_abs":0.1816,"policy/policy_entropy":0.0315,"policy/policy_loss":-0.0219,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0165,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.3555,"timing/step":2352.6975,"trainer/epoch":0}
+{"step":33,"async/staleness_mean":1.9688,"generate/avg_num_tokens":3756.4199,"generate/avg_tokens_non_zero_rewards":5020.2679,"generate/avg_tokens_zero_rewards":2884.6568,"generate/max_num_tokens":29683,"generate/std_num_tokens":4355.1836,"loss/avg_final_rewards":0.4082,"loss/avg_raw_advantages":0.013,"loss/avg_raw_advantages_abs":0.1548,"policy/policy_entropy":0.0302,"policy/policy_loss":-0.0111,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.023,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.4082,"timing/step":3022.2582,"trainer/epoch":0}
+{"step":34,"async/staleness_mean":2.1094,"generate/avg_num_tokens":3504.6309,"generate/avg_tokens_non_zero_rewards":7160.2857,"generate/avg_tokens_zero_rewards":2221.7757,"generate/max_num_tokens":26231,"generate/std_num_tokens":4896.6027,"loss/avg_final_rewards":0.2598,"loss/avg_raw_advantages":0.0351,"loss/avg_raw_advantages_abs":0.136,"policy/policy_entropy":0.0225,"policy/policy_loss":-0.0161,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0101,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.2598,"timing/step":2524.0268,"trainer/epoch":0}
+{"step":35,"async/staleness_mean":2.0469,"generate/avg_num_tokens":4162.2695,"generate/avg_tokens_non_zero_rewards":6298.2949,"generate/avg_tokens_zero_rewards":2591.0237,"generate/max_num_tokens":31360,"generate/std_num_tokens":4591.9243,"loss/avg_final_rewards":0.4238,"loss/avg_raw_advantages":0.0075,"loss/avg_raw_advantages_abs":0.1742,"policy/policy_entropy":0.0235,"policy/policy_loss":-0.006,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0409,"reward/avg_pass_at_8":0.6719,"reward/avg_raw_reward":0.4238,"timing/step":2347.1455,"trainer/epoch":0}
+{"step":36,"async/staleness_mean":1.9219,"generate/avg_num_tokens":3721.7012,"generate/avg_tokens_non_zero_rewards":4800.5648,"generate/avg_tokens_zero_rewards":3068.9718,"generate/max_num_tokens":29714,"generate/std_num_tokens":4245.8268,"loss/avg_final_rewards":0.377,"loss/avg_raw_advantages":0.0102,"loss/avg_raw_advantages_abs":0.2038,"policy/policy_entropy":0.033,"policy/policy_loss":-0.0055,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0469,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.377,"timing/step":2721.9797,"trainer/epoch":0}
+{"step":37,"async/staleness_mean":2.1094,"generate/avg_num_tokens":3798.9453,"generate/avg_tokens_non_zero_rewards":5469.6588,"generate/avg_tokens_zero_rewards":2968.4737,"generate/max_num_tokens":29671,"generate/std_num_tokens":4825.2589,"loss/avg_final_rewards":0.332,"loss/avg_raw_advantages":0.0144,"loss/avg_raw_advantages_abs":0.1735,"policy/policy_entropy":0.0274,"policy/policy_loss":-0.0106,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0177,"reward/avg_pass_at_8":0.5156,"reward/avg_raw_reward":0.332,"timing/step":2876.2119,"trainer/epoch":0}
+{"step":38,"async/staleness_mean":1.9062,"generate/avg_num_tokens":3871.7402,"generate/avg_tokens_non_zero_rewards":6783.8218,"generate/avg_tokens_zero_rewards":2372.6213,"generate/max_num_tokens":26739,"generate/std_num_tokens":4882.4938,"loss/avg_final_rewards":0.3398,"loss/avg_raw_advantages":-0.0027,"loss/avg_raw_advantages_abs":0.1698,"policy/policy_entropy":0.024,"policy/policy_loss":-0.0043,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0127,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.3398,"timing/step":2213.9343,"trainer/epoch":0}
+{"step":39,"async/staleness_mean":1.8281,"generate/avg_num_tokens":4060.5625,"generate/avg_tokens_non_zero_rewards":5228.2353,"generate/avg_tokens_zero_rewards":3562.9192,"generate/max_num_tokens":28012,"generate/std_num_tokens":5343.7181,"loss/avg_final_rewards":0.2988,"loss/avg_raw_advantages":0.0237,"loss/avg_raw_advantages_abs":0.1214,"policy/policy_entropy":0.0233,"policy/policy_loss":-0.008,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0157,"reward/avg_pass_at_8":0.4375,"reward/avg_raw_reward":0.2988,"timing/step":2673.1147,"trainer/epoch":0}
+{"step":40,"async/staleness_mean":1.9688,"generate/avg_num_tokens":4187.6387,"generate/avg_tokens_non_zero_rewards":5793.3686,"generate/avg_tokens_zero_rewards":2814.6232,"generate/max_num_tokens":18121,"generate/std_num_tokens":4439.6933,"loss/avg_final_rewards":0.4609,"loss/avg_raw_advantages":0.0037,"loss/avg_raw_advantages_abs":0.118,"policy/policy_entropy":0.0303,"policy/policy_loss":-0.0032,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.012,"reward/avg_pass_at_8":0.625,"reward/avg_raw_reward":0.4609,"timing/step":2590.4576,"trainer/epoch":0}
+{"step":41,"async/staleness_mean":0.7344,"generate/avg_num_tokens":5014.9102,"generate/avg_tokens_non_zero_rewards":5363.7867,"generate/avg_tokens_zero_rewards":4573.4115,"generate/max_num_tokens":21764,"generate/std_num_tokens":3448.4712,"loss/avg_final_rewards":0.5586,"loss/avg_raw_advantages":0.0045,"loss/avg_raw_advantages_abs":0.16,"policy/policy_entropy":0.0359,"policy/policy_loss":-0.0062,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0191,"reward/avg_pass_at_8":0.75,"reward/avg_raw_reward":0.5586,"timing/step":7641.2515,"trainer/epoch":0}
+{"step":42,"async/staleness_mean":1.0,"generate/avg_num_tokens":4234.4883,"generate/avg_tokens_non_zero_rewards":7709.5957,"generate/avg_tokens_zero_rewards":2913.7601,"generate/max_num_tokens":25390,"generate/std_num_tokens":5166.8672,"loss/avg_final_rewards":0.2754,"loss/avg_raw_advantages":0.0715,"loss/avg_raw_advantages_abs":0.2396,"policy/policy_entropy":0.0205,"policy/policy_loss":-0.0386,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.1278,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.2754,"timing/step":809.0107,"trainer/epoch":0}
+{"step":43,"async/staleness_mean":1.9062,"generate/avg_num_tokens":4277.3184,"generate/avg_tokens_non_zero_rewards":6012.0857,"generate/avg_tokens_zero_rewards":3376.4748,"generate/max_num_tokens":35754,"generate/std_num_tokens":5085.0484,"loss/avg_final_rewards":0.3418,"loss/avg_raw_advantages":-0.0014,"loss/avg_raw_advantages_abs":0.1355,"policy/policy_entropy":0.0249,"policy/policy_loss":-0.0062,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0144,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.3418,"timing/step":2843.7962,"trainer/epoch":0}
+{"step":44,"async/staleness_mean":2.4219,"generate/avg_num_tokens":4201.5156,"generate/avg_tokens_non_zero_rewards":6196.9589,"generate/avg_tokens_zero_rewards":2710.041,"generate/max_num_tokens":25645,"generate/std_num_tokens":4505.7438,"loss/avg_final_rewards":0.4277,"loss/avg_raw_advantages":0.02,"loss/avg_raw_advantages_abs":0.196,"policy/policy_entropy":0.0289,"policy/policy_loss":-0.0125,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0205,"reward/avg_pass_at_8":0.6719,"reward/avg_raw_reward":0.4277,"timing/step":2403.379,"trainer/epoch":0}
+{"step":45,"async/staleness_mean":2.3594,"generate/avg_num_tokens":3554.498,"generate/avg_tokens_non_zero_rewards":5824.5833,"generate/avg_tokens_zero_rewards":2323.7289,"generate/max_num_tokens":28004,"generate/std_num_tokens":4192.7437,"loss/avg_final_rewards":0.3516,"loss/avg_raw_advantages":0.0091,"loss/avg_raw_advantages_abs":0.1841,"policy/policy_entropy":0.0278,"policy/policy_loss":-0.0127,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0174,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.3516,"timing/step":2111.06,"trainer/epoch":0}
+{"step":46,"async/staleness_mean":2.0625,"generate/avg_num_tokens":4247.9727,"generate/avg_tokens_non_zero_rewards":5589.4333,"generate/avg_tokens_zero_rewards":3315.1689,"generate/max_num_tokens":30401,"generate/std_num_tokens":4779.4237,"loss/avg_final_rewards":0.4102,"loss/avg_raw_advantages":0.0081,"loss/avg_raw_advantages_abs":0.1125,"policy/policy_entropy":0.0265,"policy/policy_loss":-0.0027,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0149,"reward/avg_pass_at_8":0.625,"reward/avg_raw_reward":0.4102,"timing/step":2808.4339,"trainer/epoch":0}
+{"step":47,"async/staleness_mean":2.2188,"generate/avg_num_tokens":4706.6543,"generate/avg_tokens_non_zero_rewards":6220.3009,"generate/avg_tokens_zero_rewards":3602.1014,"generate/max_num_tokens":29987,"generate/std_num_tokens":5076.1121,"loss/avg_final_rewards":0.4219,"loss/avg_raw_advantages":-0.0061,"loss/avg_raw_advantages_abs":0.2368,"policy/policy_entropy":0.0294,"policy/policy_loss":-0.0063,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0195,"reward/avg_pass_at_8":0.6719,"reward/avg_raw_reward":0.4219,"timing/step":2333.4236,"trainer/epoch":0}
+{"step":48,"async/staleness_mean":2.1719,"generate/avg_num_tokens":4437.375,"generate/avg_tokens_non_zero_rewards":6230.3756,"generate/avg_tokens_zero_rewards":3160.087,"generate/max_num_tokens":24808,"generate/std_num_tokens":4496.5138,"loss/avg_final_rewards":0.416,"loss/avg_raw_advantages":0.0075,"loss/avg_raw_advantages_abs":0.1863,"policy/policy_entropy":0.0275,"policy/policy_loss":-0.0004,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0128,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.416,"timing/step":2473.6369,"trainer/epoch":0}
+{"step":49,"async/staleness_mean":2.2969,"generate/avg_num_tokens":5040.9453,"generate/avg_tokens_non_zero_rewards":5835.8168,"generate/avg_tokens_zero_rewards":4522.9968,"generate/max_num_tokens":25559,"generate/std_num_tokens":4828.1681,"loss/avg_final_rewards":0.3945,"loss/avg_raw_advantages":0.0117,"loss/avg_raw_advantages_abs":0.1464,"policy/policy_entropy":0.033,"policy/policy_loss":-0.0052,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0169,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.3945,"timing/step":2328.2427,"trainer/epoch":0}
+{"step":50,"async/staleness_mean":2.4844,"generate/avg_num_tokens":4544.0625,"generate/avg_tokens_non_zero_rewards":6245.1566,"generate/avg_tokens_zero_rewards":3471.3981,"generate/max_num_tokens":25117,"generate/std_num_tokens":3843.4054,"loss/avg_final_rewards":0.3867,"loss/avg_raw_advantages":-0.0041,"loss/avg_raw_advantages_abs":0.2038,"policy/policy_entropy":0.0299,"policy/policy_loss":-0.002,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0175,"reward/avg_pass_at_8":0.6562,"reward/avg_raw_reward":0.3867,"timing/step":2052.8109,"trainer/epoch":0}
+{"step":51,"async/staleness_mean":2.4219,"generate/avg_num_tokens":4180.918,"generate/avg_tokens_non_zero_rewards":5545.2165,"generate/avg_tokens_zero_rewards":3348.6101,"generate/max_num_tokens":23252,"generate/std_num_tokens":4819.6171,"loss/avg_final_rewards":0.3789,"loss/avg_raw_advantages":0.0055,"loss/avg_raw_advantages_abs":0.1441,"policy/policy_entropy":0.0275,"policy/policy_loss":-0.0066,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0158,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.3789,"timing/step":2260.6657,"trainer/epoch":0}
+{"step":52,"async/staleness_mean":2.5938,"generate/avg_num_tokens":4455.75,"generate/avg_tokens_non_zero_rewards":6957.9505,"generate/avg_tokens_zero_rewards":2825.2839,"generate/max_num_tokens":30540,"generate/std_num_tokens":4759.6593,"loss/avg_final_rewards":0.3945,"loss/avg_raw_advantages":0.0314,"loss/avg_raw_advantages_abs":0.1364,"policy/policy_entropy":0.0211,"policy/policy_loss":-0.0143,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0096,"reward/avg_pass_at_8":0.625,"reward/avg_raw_reward":0.3945,"timing/step":5334.2377,"trainer/epoch":0}
+{"step":53,"async/staleness_mean":2.2344,"generate/avg_num_tokens":2974.4414,"generate/avg_tokens_non_zero_rewards":5231.4716,"generate/avg_tokens_zero_rewards":1792.1875,"generate/max_num_tokens":23029,"generate/std_num_tokens":3988.3975,"loss/avg_final_rewards":0.3438,"loss/avg_raw_advantages":0.1127,"loss/avg_raw_advantages_abs":0.1811,"policy/policy_entropy":0.0228,"policy/policy_loss":-0.0441,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0375,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.3438,"timing/step":3496.4626,"trainer/epoch":0}
+{"step":54,"async/staleness_mean":2.1719,"generate/avg_num_tokens":3025.7383,"generate/avg_tokens_non_zero_rewards":4947.9773,"generate/avg_tokens_zero_rewards":2018.8512,"generate/max_num_tokens":17298,"generate/std_num_tokens":3230.3986,"loss/avg_final_rewards":0.3438,"loss/avg_raw_advantages":0.109,"loss/avg_raw_advantages_abs":0.2328,"policy/policy_entropy":0.0263,"policy/policy_loss":-0.0476,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.018,"reward/avg_pass_at_8":0.6094,"reward/avg_raw_reward":0.3438,"timing/step":1967.8768,"trainer/epoch":0}
+{"step":55,"async/staleness_mean":0.2188,"generate/avg_num_tokens":4983.5684,"generate/avg_tokens_non_zero_rewards":5594.0812,"generate/avg_tokens_zero_rewards":4061.8137,"generate/max_num_tokens":21234,"generate/std_num_tokens":3444.2816,"loss/avg_final_rewards":0.6016,"loss/avg_raw_advantages":0.0189,"loss/avg_raw_advantages_abs":0.1898,"policy/policy_entropy":0.0339,"policy/policy_loss":-0.0177,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0176,"reward/avg_pass_at_8":0.7541,"reward/avg_raw_reward":0.6016,"timing/step":7700.0928,"trainer/epoch":0}
+{"step":56,"async/staleness_mean":1.0,"generate/avg_num_tokens":4016.0664,"generate/avg_tokens_non_zero_rewards":7264.2264,"generate/avg_tokens_zero_rewards":2553.0142,"generate/max_num_tokens":25297,"generate/std_num_tokens":4964.2252,"loss/avg_final_rewards":0.3105,"loss/avg_raw_advantages":0.0383,"loss/avg_raw_advantages_abs":0.2742,"policy/policy_entropy":0.0224,"policy/policy_loss":-0.0164,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0151,"reward/avg_pass_at_8":0.5469,"reward/avg_raw_reward":0.3105,"timing/step":698.2019,"trainer/epoch":0}
+{"step":57,"async/staleness_mean":1.9688,"generate/avg_num_tokens":3494.8184,"generate/avg_tokens_non_zero_rewards":5344.1636,"generate/avg_tokens_zero_rewards":2615.4467,"generate/max_num_tokens":25477,"generate/std_num_tokens":4190.2245,"loss/avg_final_rewards":0.3223,"loss/avg_raw_advantages":0.0173,"loss/avg_raw_advantages_abs":0.2263,"policy/policy_entropy":0.025,"policy/policy_loss":-0.015,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0175,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.3223,"timing/step":1666.5993,"trainer/epoch":0}
+{"step":58,"async/staleness_mean":2.2344,"generate/avg_num_tokens":4374.5781,"generate/avg_tokens_non_zero_rewards":5539.7117,"generate/avg_tokens_zero_rewards":3033.2059,"generate/max_num_tokens":17396,"generate/std_num_tokens":3565.8817,"loss/avg_final_rewards":0.5352,"loss/avg_raw_advantages":0.0235,"loss/avg_raw_advantages_abs":0.1949,"policy/policy_entropy":0.027,"policy/policy_loss":-0.013,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0221,"reward/avg_pass_at_8":0.7812,"reward/avg_raw_reward":0.5352,"timing/step":2546.9027,"trainer/epoch":0}
+{"step":59,"async/staleness_mean":2.2812,"generate/avg_num_tokens":4491.2891,"generate/avg_tokens_non_zero_rewards":5043.9627,"generate/avg_tokens_zero_rewards":3999.797,"generate/max_num_tokens":25211,"generate/std_num_tokens":4037.2616,"loss/avg_final_rewards":0.4707,"loss/avg_raw_advantages":0.0002,"loss/avg_raw_advantages_abs":0.1642,"policy/policy_entropy":0.0279,"policy/policy_loss":-0.0039,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0164,"reward/avg_pass_at_8":0.6562,"reward/avg_raw_reward":0.4707,"timing/step":2529.1779,"trainer/epoch":0}
+{"step":60,"async/staleness_mean":2.6406,"generate/avg_num_tokens":4325.4512,"generate/avg_tokens_non_zero_rewards":6503.0047,"generate/avg_tokens_zero_rewards":2749.1077,"generate/max_num_tokens":25313,"generate/std_num_tokens":4579.467,"loss/avg_final_rewards":0.4199,"loss/avg_raw_advantages":0.0244,"loss/avg_raw_advantages_abs":0.2086,"policy/policy_entropy":0.0233,"policy/policy_loss":-0.0161,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0104,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.4199,"timing/step":1310.3803,"trainer/epoch":0}
+{"step":61,"async/staleness_mean":2.4375,"generate/avg_num_tokens":4239.7832,"generate/avg_tokens_non_zero_rewards":5941.0693,"generate/avg_tokens_zero_rewards":2841.2171,"generate/max_num_tokens":26838,"generate/std_num_tokens":4123.1173,"loss/avg_final_rewards":0.4512,"loss/avg_raw_advantages":0.0195,"loss/avg_raw_advantages_abs":0.2248,"policy/policy_entropy":0.0245,"policy/policy_loss":-0.0112,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0205,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.4512,"timing/step":2339.0794,"trainer/epoch":0}
+{"step":62,"async/staleness_mean":2.5781,"generate/avg_num_tokens":4891.2207,"generate/avg_tokens_non_zero_rewards":6417.1683,"generate/avg_tokens_zero_rewards":3896.8935,"generate/max_num_tokens":27581,"generate/std_num_tokens":4785.9904,"loss/avg_final_rewards":0.3945,"loss/avg_raw_advantages":0.0074,"loss/avg_raw_advantages_abs":0.1881,"policy/policy_entropy":0.0247,"policy/policy_loss":-0.0039,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0143,"reward/avg_pass_at_8":0.6094,"reward/avg_raw_reward":0.3945,"timing/step":2045.6361,"trainer/epoch":0}
+{"step":63,"async/staleness_mean":2.2031,"generate/avg_num_tokens":4822.709,"generate/avg_tokens_non_zero_rewards":5594.5819,"generate/avg_tokens_zero_rewards":3739.1878,"generate/max_num_tokens":22463,"generate/std_num_tokens":3575.863,"loss/avg_final_rewards":0.584,"loss/avg_raw_advantages":0.0156,"loss/avg_raw_advantages_abs":0.1904,"policy/policy_entropy":0.0285,"policy/policy_loss":-0.0065,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0374,"reward/avg_pass_at_8":0.8281,"reward/avg_raw_reward":0.584,"timing/step":1623.7142,"trainer/epoch":0}
+{"step":64,"async/staleness_mean":2.7188,"generate/avg_num_tokens":4897.4883,"generate/avg_tokens_non_zero_rewards":5614.4464,"generate/avg_tokens_zero_rewards":4298.7384,"generate/max_num_tokens":29490,"generate/std_num_tokens":4482.8336,"loss/avg_final_rewards":0.4551,"loss/avg_raw_advantages":-0.0007,"loss/avg_raw_advantages_abs":0.1731,"policy/policy_entropy":0.0293,"policy/policy_loss":-0.0029,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0201,"reward/avg_pass_at_8":0.6875,"reward/avg_raw_reward":0.4551,"timing/step":1801.7146,"trainer/epoch":0}
+{"step":65,"async/staleness_mean":2.6406,"generate/avg_num_tokens":5770.957,"generate/avg_tokens_non_zero_rewards":6332.9202,"generate/avg_tokens_zero_rewards":5282.8285,"generate/max_num_tokens":22508,"generate/std_num_tokens":4287.7432,"loss/avg_final_rewards":0.4648,"loss/avg_raw_advantages":0.0155,"loss/avg_raw_advantages_abs":0.2291,"policy/policy_entropy":0.0293,"policy/policy_loss":-0.0071,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0197,"reward/avg_pass_at_8":0.7031,"reward/avg_raw_reward":0.4648,"timing/step":1732.397,"trainer/epoch":0}
+{"step":66,"async/staleness_mean":2.6562,"generate/avg_num_tokens":5634.877,"generate/avg_tokens_non_zero_rewards":6787.384,"generate/avg_tokens_zero_rewards":4641.6255,"generate/max_num_tokens":31352,"generate/std_num_tokens":5143.29,"loss/avg_final_rewards":0.4629,"loss/avg_raw_advantages":0.019,"loss/avg_raw_advantages_abs":0.2278,"policy/policy_entropy":0.0269,"policy/policy_loss":-0.0128,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0192,"reward/avg_pass_at_8":0.6562,"reward/avg_raw_reward":0.4629,"timing/step":1991.3162,"trainer/epoch":0}
+{"step":67,"async/staleness_mean":2.7031,"generate/avg_num_tokens":5362.1816,"generate/avg_tokens_non_zero_rewards":6330.2122,"generate/avg_tokens_zero_rewards":4212.1282,"generate/max_num_tokens":22071,"generate/std_num_tokens":3636.0517,"loss/avg_final_rewards":0.543,"loss/avg_raw_advantages":-0.0005,"loss/avg_raw_advantages_abs":0.1837,"policy/policy_entropy":0.0279,"policy/policy_loss":-0.0014,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0179,"reward/avg_pass_at_8":0.75,"reward/avg_raw_reward":0.543,"timing/step":1343.1399,"trainer/epoch":0}
+{"step":68,"async/staleness_mean":2.7812,"generate/avg_num_tokens":4970.2012,"generate/avg_tokens_non_zero_rewards":5580.6046,"generate/avg_tokens_zero_rewards":4325.4779,"generate/max_num_tokens":25827,"generate/std_num_tokens":4095.0748,"loss/avg_final_rewards":0.5137,"loss/avg_raw_advantages":0.0012,"loss/avg_raw_advantages_abs":0.1885,"policy/policy_entropy":0.0268,"policy/policy_loss":-0.0039,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0191,"reward/avg_pass_at_8":0.7344,"reward/avg_raw_reward":0.5137,"timing/step":1569.2032,"trainer/epoch":0}
+{"step":69,"async/staleness_mean":2.7656,"generate/avg_num_tokens":4856.9043,"generate/avg_tokens_non_zero_rewards":6299.4016,"generate/avg_tokens_zero_rewards":3543.5858,"generate/max_num_tokens":28538,"generate/std_num_tokens":3972.354,"loss/avg_final_rewards":0.4766,"loss/avg_raw_advantages":0.0233,"loss/avg_raw_advantages_abs":0.2329,"policy/policy_entropy":0.0252,"policy/policy_loss":-0.0181,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0355,"reward/avg_pass_at_8":0.7344,"reward/avg_raw_reward":0.4766,"timing/step":2174.8383,"trainer/epoch":0}
+{"step":70,"async/staleness_mean":2.5781,"generate/avg_num_tokens":4174.5391,"generate/avg_tokens_non_zero_rewards":5174.5363,"generate/avg_tokens_zero_rewards":3235.1477,"generate/max_num_tokens":23443,"generate/std_num_tokens":4084.7715,"loss/avg_final_rewards":0.4844,"loss/avg_raw_advantages":0.0041,"loss/avg_raw_advantages_abs":0.1119,"policy/policy_entropy":0.0253,"policy/policy_loss":-0.0044,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0136,"reward/avg_pass_at_8":0.6562,"reward/avg_raw_reward":0.4844,"timing/step":1889.4222,"trainer/epoch":0}
+{"step":71,"async/staleness_mean":2.2812,"generate/avg_num_tokens":4969.4277,"generate/avg_tokens_non_zero_rewards":6109.6913,"generate/avg_tokens_zero_rewards":4039.4255,"generate/max_num_tokens":29103,"generate/std_num_tokens":4638.401,"loss/avg_final_rewards":0.4492,"loss/avg_raw_advantages":0.008,"loss/avg_raw_advantages_abs":0.1585,"policy/policy_entropy":0.0263,"policy/policy_loss":-0.0073,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0281,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.4492,"timing/step":1848.6629,"trainer/epoch":0}
+{"step":72,"async/staleness_mean":2.625,"generate/avg_num_tokens":4861.5605,"generate/avg_tokens_non_zero_rewards":6199.4355,"generate/avg_tokens_zero_rewards":3604.7689,"generate/max_num_tokens":19869,"generate/std_num_tokens":4409.6282,"loss/avg_final_rewards":0.4844,"loss/avg_raw_advantages":0.0137,"loss/avg_raw_advantages_abs":0.1836,"policy/policy_entropy":0.0231,"policy/policy_loss":-0.0125,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0517,"reward/avg_pass_at_8":0.7188,"reward/avg_raw_reward":0.4844,"timing/step":1362.1325,"trainer/epoch":0}
+{"step":73,"async/staleness_mean":2.7031,"generate/avg_num_tokens":4400.0762,"generate/avg_tokens_non_zero_rewards":5518.6064,"generate/avg_tokens_zero_rewards":3341.0875,"generate/max_num_tokens":23078,"generate/std_num_tokens":4122.3915,"loss/avg_final_rewards":0.4863,"loss/avg_raw_advantages":0.0024,"loss/avg_raw_advantages_abs":0.2172,"policy/policy_entropy":0.025,"policy/policy_loss":-0.0054,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0687,"reward/avg_pass_at_8":0.6875,"reward/avg_raw_reward":0.4863,"timing/step":2094.5654,"trainer/epoch":0}
+{"step":74,"async/staleness_mean":2.5625,"generate/avg_num_tokens":4532.9746,"generate/avg_tokens_non_zero_rewards":5339.6484,"generate/avg_tokens_zero_rewards":3295.0099,"generate/max_num_tokens":26198,"generate/std_num_tokens":3759.0977,"loss/avg_final_rewards":0.6055,"loss/avg_raw_advantages":0.0214,"loss/avg_raw_advantages_abs":0.1527,"policy/policy_entropy":0.0276,"policy/policy_loss":-0.0083,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0191,"reward/avg_pass_at_8":0.8438,"reward/avg_raw_reward":0.6055,"timing/step":1415.8977,"trainer/epoch":0}
+{"step":75,"async/staleness_mean":0.5312,"generate/avg_num_tokens":4576.875,"generate/avg_tokens_non_zero_rewards":4728.5101,"generate/avg_tokens_zero_rewards":4369.0787,"generate/max_num_tokens":17535,"generate/std_num_tokens":3160.5729,"loss/avg_final_rewards":0.5781,"loss/avg_raw_advantages":0.0119,"loss/avg_raw_advantages_abs":0.1715,"policy/policy_entropy":0.0328,"policy/policy_loss":-0.0082,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0306,"reward/avg_pass_at_8":0.7931,"reward/avg_raw_reward":0.5781,"timing/step":6652.8217,"trainer/epoch":0}
+{"step":76,"async/staleness_mean":0.9844,"generate/avg_num_tokens":4948.4961,"generate/avg_tokens_non_zero_rewards":6104.6826,"generate/avg_tokens_zero_rewards":4005.5071,"generate/max_num_tokens":29321,"generate/std_num_tokens":4568.6688,"loss/avg_final_rewards":0.4492,"loss/avg_raw_advantages":0.027,"loss/avg_raw_advantages_abs":0.1829,"policy/policy_entropy":0.0254,"policy/policy_loss":-0.0218,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0182,"reward/avg_pass_at_8":0.6719,"reward/avg_raw_reward":0.4492,"timing/step":1520.4627,"trainer/epoch":0}
+{"step":77,"async/staleness_mean":1.875,"generate/avg_num_tokens":3918.4082,"generate/avg_tokens_non_zero_rewards":6218.3667,"generate/avg_tokens_zero_rewards":2671.4428,"generate/max_num_tokens":25717,"generate/std_num_tokens":4431.6038,"loss/avg_final_rewards":0.3516,"loss/avg_raw_advantages":0.0006,"loss/avg_raw_advantages_abs":0.1057,"policy/policy_entropy":0.0157,"policy/policy_loss":-0.0085,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0128,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.3516,"timing/step":582.7078,"trainer/epoch":0}
+{"step":78,"async/staleness_mean":2.4219,"generate/avg_num_tokens":4184.1621,"generate/avg_tokens_non_zero_rewards":5520.4978,"generate/avg_tokens_zero_rewards":3102.8163,"generate/max_num_tokens":16019,"generate/std_num_tokens":3591.7024,"loss/avg_final_rewards":0.4473,"loss/avg_raw_advantages":0.0128,"loss/avg_raw_advantages_abs":0.2054,"policy/policy_entropy":0.0255,"policy/policy_loss":-0.0059,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0179,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.4473,"timing/step":2340.5418,"trainer/epoch":0}

viewer/build/inputs/marin/runs/marin-a3-inferredbugs/run.json

@@ -0,0 +1,27 @@
+{
+ "id": "marin-a3-inferredbugs",
+ "title": "A3 RLOO on inferredbugs-sandboxes-verifier (Qwen3-8B agent)",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://huggingface.co/laion/a3-rl-DCAgent_inferredbugs-sandboxes-verifier-55-8B/tree/main/training_logs",
+ "license": "unknown",
+ "model": "laion/a3-rl-DCAgent_inferredbugs-sandboxes-verifier-55-8B",
+ "base_model": "laion/GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink",
+ "method": "RLOO-N (SkyRL, binary verifier reward)",
+ "dataset": "DCAgent/inferredbugs-sandboxes-verifier",
+ "eval_suite": null,
+ "kind": "training",
+ "state": "finished",
+ "started_at": null,
+ "updated_at": "2026-05-25T01:27:54Z",
+ "attempts": 0,
+ "note": "Marin A3 sweep (issue #6187): one RLOO-N run per training dataset from the same Qwen3-8B-derived SFT agent, here DCAgent/inferredbugs-sandboxes-verifier; EMA-best step 55 at reward 0.602, per the issue. Our copy has every logged step of the final lineage (7 resumed job segments, 2 superseded rows dropped). The issue concluded this binary-reward setup was uninformative about dataset utility.",
+ "metrics_map": {
+  "reward": "reward/avg_raw_reward",
+  "loss": "policy/policy_loss",
+  "entropy": "policy/policy_entropy",
+  "grad_norm": "policy/raw_grad_norm",
+  "response_length": "generate/avg_num_tokens"
+ }
+}

viewer/build/inputs/marin/runs/marin-a3-llm-verifier-freelancer/metrics.jsonl

@@ -0,0 +1,76 @@
+{"step":1,"async/staleness_mean":0.0,"generate/avg_num_tokens":6833.9395,"generate/avg_tokens_non_zero_rewards":7155.1346,"generate/avg_tokens_zero_rewards":5442.0938,"generate/max_num_tokens":30654,"generate/std_num_tokens":5046.9818,"loss/avg_final_rewards":0.5664,"loss/avg_raw_advantages":0.0248,"loss/avg_raw_advantages_abs":0.2355,"policy/policy_entropy":0.2043,"policy/policy_loss":-0.0058,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0234,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.5664,"timing/step":3854.9161,"trainer/epoch":0}
+{"step":2,"async/staleness_mean":0.9062,"generate/avg_num_tokens":8661.0078,"generate/avg_tokens_non_zero_rewards":8783.2251,"generate/avg_tokens_zero_rewards":8266.0744,"generate/max_num_tokens":32047,"generate/std_num_tokens":5953.0178,"loss/avg_final_rewards":0.4891,"loss/avg_raw_advantages":0.0114,"loss/avg_raw_advantages_abs":0.2438,"policy/policy_entropy":0.1978,"policy/policy_loss":-0.0068,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0227,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.4891,"timing/step":3100.7366,"trainer/epoch":0}
+{"step":3,"async/staleness_mean":1.8594,"generate/avg_num_tokens":9751.8027,"generate/avg_tokens_non_zero_rewards":11110.7092,"generate/avg_tokens_zero_rewards":6279.0417,"generate/max_num_tokens":31830,"generate/std_num_tokens":7140.833,"loss/avg_final_rewards":0.4268,"loss/avg_raw_advantages":0.0158,"loss/avg_raw_advantages_abs":0.2256,"policy/policy_entropy":0.1712,"policy/policy_loss":-0.01,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0204,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.4268,"timing/step":2035.4234,"trainer/epoch":0}
+{"step":4,"async/staleness_mean":2.6875,"generate/avg_num_tokens":11438.873,"generate/avg_tokens_non_zero_rewards":13812.7345,"generate/avg_tokens_zero_rewards":6787.2023,"generate/max_num_tokens":32985,"generate/std_num_tokens":8297.216,"loss/avg_final_rewards":0.3528,"loss/avg_raw_advantages":0.0135,"loss/avg_raw_advantages_abs":0.1855,"policy/policy_entropy":0.1521,"policy/policy_loss":-0.0034,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.02,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.3528,"timing/step":824.0228,"trainer/epoch":0}
+{"step":5,"async/staleness_mean":3.0312,"generate/avg_num_tokens":8473.5078,"generate/avg_tokens_non_zero_rewards":11101.1261,"generate/avg_tokens_zero_rewards":3585.257,"generate/max_num_tokens":32045,"generate/std_num_tokens":7384.1156,"loss/avg_final_rewards":0.3561,"loss/avg_raw_advantages":0.0113,"loss/avg_raw_advantages_abs":0.1719,"policy/policy_entropy":0.1423,"policy/policy_loss":-0.0105,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0154,"reward/avg_pass_at_8":0.9375,"reward/avg_raw_reward":0.3561,"timing/step":2368.7394,"trainer/epoch":0}
+{"step":6,"async/staleness_mean":2.9219,"generate/avg_num_tokens":9709.6738,"generate/avg_tokens_non_zero_rewards":11000.4118,"generate/avg_tokens_zero_rewards":4646.0096,"generate/max_num_tokens":31767,"generate/std_num_tokens":7106.1134,"loss/avg_final_rewards":0.4579,"loss/avg_raw_advantages":0.0209,"loss/avg_raw_advantages_abs":0.1416,"policy/policy_entropy":0.1599,"policy/policy_loss":-0.0159,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0136,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.4579,"timing/step":2055.0597,"trainer/epoch":0}
+{"step":7,"async/staleness_mean":2.9844,"generate/avg_num_tokens":8835.0137,"generate/avg_tokens_non_zero_rewards":10083.2451,"generate/avg_tokens_zero_rewards":3938.1058,"generate/max_num_tokens":32048,"generate/std_num_tokens":6535.2771,"loss/avg_final_rewards":0.4908,"loss/avg_raw_advantages":0.0172,"loss/avg_raw_advantages_abs":0.1832,"policy/policy_entropy":0.1648,"policy/policy_loss":-0.0101,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0164,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.4908,"timing/step":2973.8084,"trainer/epoch":0}
+{"step":8,"async/staleness_mean":2.5156,"generate/avg_num_tokens":8257.9219,"generate/avg_tokens_non_zero_rewards":11010.7934,"generate/avg_tokens_zero_rewards":1551.2617,"generate/max_num_tokens":32120,"generate/std_num_tokens":7447.5951,"loss/avg_final_rewards":0.4457,"loss/avg_raw_advantages":0.0088,"loss/avg_raw_advantages_abs":0.1721,"policy/policy_entropy":0.1292,"policy/policy_loss":-0.0226,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0138,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.4457,"timing/step":2149.1646,"trainer/epoch":0}
+{"step":9,"async/staleness_mean":2.7188,"generate/avg_num_tokens":9825.5918,"generate/avg_tokens_non_zero_rewards":11938.4278,"generate/avg_tokens_zero_rewards":2692.5128,"generate/max_num_tokens":32118,"generate/std_num_tokens":7441.7294,"loss/avg_final_rewards":0.4697,"loss/avg_raw_advantages":0.0195,"loss/avg_raw_advantages_abs":0.1718,"policy/policy_entropy":0.1446,"policy/policy_loss":-0.0223,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0271,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.4697,"timing/step":1945.6135,"trainer/epoch":0}
+{"step":10,"async/staleness_mean":0.3281,"generate/avg_num_tokens":8510.8438,"generate/avg_tokens_non_zero_rewards":9187.2851,"generate/avg_tokens_zero_rewards":3002.6786,"generate/max_num_tokens":31705,"generate/std_num_tokens":5658.3599,"loss/avg_final_rewards":0.5799,"loss/avg_raw_advantages":0.0365,"loss/avg_raw_advantages_abs":0.1551,"policy/policy_entropy":0.1728,"policy/policy_loss":-0.022,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0187,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.5799,"timing/step":7833.5253,"trainer/epoch":0}
+{"step":11,"async/staleness_mean":1.0,"generate/avg_num_tokens":10939.0723,"generate/avg_tokens_non_zero_rewards":14077.518,"generate/avg_tokens_zero_rewards":3435.9007,"generate/max_num_tokens":32122,"generate/std_num_tokens":8655.8075,"loss/avg_final_rewards":0.4057,"loss/avg_raw_advantages":-0.0024,"loss/avg_raw_advantages_abs":0.1477,"policy/policy_entropy":0.1192,"policy/policy_loss":-0.0136,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0121,"reward/avg_pass_at_8":0.9375,"reward/avg_raw_reward":0.4057,"timing/step":1125.0477,"trainer/epoch":0}
+{"step":12,"async/staleness_mean":1.875,"generate/avg_num_tokens":10134.1504,"generate/avg_tokens_non_zero_rewards":12561.2746,"generate/avg_tokens_zero_rewards":2698.6746,"generate/max_num_tokens":32120,"generate/std_num_tokens":8078.7739,"loss/avg_final_rewards":0.4517,"loss/avg_raw_advantages":0.0026,"loss/avg_raw_advantages_abs":0.171,"policy/policy_entropy":0.1254,"policy/policy_loss":-0.0125,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0203,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.4517,"timing/step":1694.0212,"trainer/epoch":0}
+{"step":13,"async/staleness_mean":2.3906,"generate/avg_num_tokens":9665.498,"generate/avg_tokens_non_zero_rewards":10825.1273,"generate/avg_tokens_zero_rewards":3403.5,"generate/max_num_tokens":31846,"generate/std_num_tokens":6770.7501,"loss/avg_final_rewards":0.529,"loss/avg_raw_advantages":0.0169,"loss/avg_raw_advantages_abs":0.1458,"policy/policy_entropy":0.1541,"policy/policy_loss":-0.0092,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0123,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.529,"timing/step":2774.1276,"trainer/epoch":0}
+{"step":14,"async/staleness_mean":2.5469,"generate/avg_num_tokens":9017.6797,"generate/avg_tokens_non_zero_rewards":10972.3399,"generate/avg_tokens_zero_rewards":1255.9709,"generate/max_num_tokens":32118,"generate/std_num_tokens":6784.8966,"loss/avg_final_rewards":0.5264,"loss/avg_raw_advantages":0.0298,"loss/avg_raw_advantages_abs":0.1343,"policy/policy_entropy":0.1406,"policy/policy_loss":-0.0188,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0116,"reward/avg_pass_at_8":0.9375,"reward/avg_raw_reward":0.5264,"timing/step":2628.9733,"trainer/epoch":0}
+{"step":15,"async/staleness_mean":2.8906,"generate/avg_num_tokens":10018.084,"generate/avg_tokens_non_zero_rewards":11944.8235,"generate/avg_tokens_zero_rewards":3792.0083,"generate/max_num_tokens":32108,"generate/std_num_tokens":7642.9612,"loss/avg_final_rewards":0.4393,"loss/avg_raw_advantages":0.0123,"loss/avg_raw_advantages_abs":0.159,"policy/policy_entropy":0.1332,"policy/policy_loss":-0.0102,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0122,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.4393,"timing/step":1612.1218,"trainer/epoch":0}
+{"step":16,"async/staleness_mean":0.7188,"generate/avg_num_tokens":7710.1855,"generate/avg_tokens_non_zero_rewards":8252.7751,"generate/avg_tokens_zero_rewards":3108.2222,"generate/max_num_tokens":31754,"generate/std_num_tokens":5364.2242,"loss/avg_final_rewards":0.5849,"loss/avg_raw_advantages":0.0239,"loss/avg_raw_advantages_abs":0.1484,"policy/policy_entropy":0.1659,"policy/policy_loss":-0.0155,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0151,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.5849,"timing/step":6444.7011,"trainer/epoch":0}
+{"step":17,"async/staleness_mean":0.9844,"generate/avg_num_tokens":10174.5684,"generate/avg_tokens_non_zero_rewards":12229.3085,"generate/avg_tokens_zero_rewards":2665.4273,"generate/max_num_tokens":32050,"generate/std_num_tokens":7609.0707,"loss/avg_final_rewards":0.4872,"loss/avg_raw_advantages":0.0266,"loss/avg_raw_advantages_abs":0.1505,"policy/policy_entropy":0.1356,"policy/policy_loss":-0.021,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0127,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.4872,"timing/step":2344.092,"trainer/epoch":0}
+{"step":18,"async/staleness_mean":1.7656,"generate/avg_num_tokens":10939.5273,"generate/avg_tokens_non_zero_rewards":13794.5484,"generate/avg_tokens_zero_rewards":3353.3286,"generate/max_num_tokens":32046,"generate/std_num_tokens":8679.4954,"loss/avg_final_rewards":0.4208,"loss/avg_raw_advantages":0.007,"loss/avg_raw_advantages_abs":0.1411,"policy/policy_entropy":0.1162,"policy/policy_loss":-0.0068,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0107,"reward/avg_pass_at_8":0.9531,"reward/avg_raw_reward":0.4208,"timing/step":1400.3478,"trainer/epoch":0}
+{"step":19,"async/staleness_mean":2.3438,"generate/avg_num_tokens":8532.4668,"generate/avg_tokens_non_zero_rewards":9477.3466,"generate/avg_tokens_zero_rewards":3785.8353,"generate/max_num_tokens":32044,"generate/std_num_tokens":6449.129,"loss/avg_final_rewards":0.5238,"loss/avg_raw_advantages":0.0355,"loss/avg_raw_advantages_abs":0.1371,"policy/policy_entropy":0.1502,"policy/policy_loss":-0.017,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0128,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.5238,"timing/step":1864.9276,"trainer/epoch":0}
+{"step":20,"async/staleness_mean":2.8281,"generate/avg_num_tokens":8662.2715,"generate/avg_tokens_non_zero_rewards":9761.8886,"generate/avg_tokens_zero_rewards":3506.2889,"generate/max_num_tokens":32045,"generate/std_num_tokens":6046.8415,"loss/avg_final_rewards":0.5943,"loss/avg_raw_advantages":0.0077,"loss/avg_raw_advantages_abs":0.1446,"policy/policy_entropy":0.1525,"policy/policy_loss":-0.0092,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0141,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.5943,"timing/step":2429.2105,"trainer/epoch":0}
+{"step":21,"async/staleness_mean":2.6562,"generate/avg_num_tokens":9512.9219,"generate/avg_tokens_non_zero_rewards":11186.8321,"generate/avg_tokens_zero_rewards":2165.3368,"generate/max_num_tokens":32046,"generate/std_num_tokens":7091.9621,"loss/avg_final_rewards":0.505,"loss/avg_raw_advantages":0.0152,"loss/avg_raw_advantages_abs":0.1465,"policy/policy_entropy":0.1382,"policy/policy_loss":-0.0121,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0122,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.505,"timing/step":2222.8783,"trainer/epoch":0}
+{"step":22,"async/staleness_mean":0.7812,"generate/avg_num_tokens":8607.3203,"generate/avg_tokens_non_zero_rewards":9263.9739,"generate/avg_tokens_zero_rewards":2920.4528,"generate/max_num_tokens":31752,"generate/std_num_tokens":5678.662,"loss/avg_final_rewards":0.6205,"loss/avg_raw_advantages":0.0245,"loss/avg_raw_advantages_abs":0.1402,"policy/policy_entropy":0.1559,"policy/policy_loss":-0.0138,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0156,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.6205,"timing/step":7128.5454,"trainer/epoch":0}
+{"step":23,"async/staleness_mean":1.0,"generate/avg_num_tokens":10841.8223,"generate/avg_tokens_non_zero_rewards":13884.7411,"generate/avg_tokens_zero_rewards":3140.0897,"generate/max_num_tokens":32120,"generate/std_num_tokens":8445.9015,"loss/avg_final_rewards":0.4345,"loss/avg_raw_advantages":0.0334,"loss/avg_raw_advantages_abs":0.1464,"policy/policy_entropy":0.1132,"policy/policy_loss":-0.0233,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0133,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.4345,"timing/step":2101.9452,"trainer/epoch":0}
+{"step":24,"async/staleness_mean":1.9062,"generate/avg_num_tokens":10107.7227,"generate/avg_tokens_non_zero_rewards":12953.8747,"generate/avg_tokens_zero_rewards":3429.4967,"generate/max_num_tokens":31852,"generate/std_num_tokens":8266.8672,"loss/avg_final_rewards":0.4389,"loss/avg_raw_advantages":0.0125,"loss/avg_raw_advantages_abs":0.168,"policy/policy_entropy":0.1111,"policy/policy_loss":-0.0134,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0127,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.4389,"timing/step":1092.8984,"trainer/epoch":0}
+{"step":25,"async/staleness_mean":2.2812,"generate/avg_num_tokens":9390.4082,"generate/avg_tokens_non_zero_rewards":10635.257,"generate/avg_tokens_zero_rewards":3047.6071,"generate/max_num_tokens":31893,"generate/std_num_tokens":7153.5074,"loss/avg_final_rewards":0.5253,"loss/avg_raw_advantages":0.012,"loss/avg_raw_advantages_abs":0.1503,"policy/policy_entropy":0.1425,"policy/policy_loss":-0.0075,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0187,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.5253,"timing/step":2718.5878,"trainer/epoch":0}
+{"step":26,"async/staleness_mean":2.5625,"generate/avg_num_tokens":9182.2207,"generate/avg_tokens_non_zero_rewards":9713.4084,"generate/avg_tokens_zero_rewards":2362.9189,"generate/max_num_tokens":31707,"generate/std_num_tokens":6016.6304,"loss/avg_final_rewards":0.6262,"loss/avg_raw_advantages":0.0158,"loss/avg_raw_advantages_abs":0.124,"policy/policy_entropy":0.1515,"policy/policy_loss":-0.0056,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0119,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.6262,"timing/step":2192.0123,"trainer/epoch":0}
+{"step":27,"async/staleness_mean":2.8438,"generate/avg_num_tokens":10177.2734,"generate/avg_tokens_non_zero_rewards":11794.7915,"generate/avg_tokens_zero_rewards":2592.9111,"generate/max_num_tokens":32115,"generate/std_num_tokens":7898.6955,"loss/avg_final_rewards":0.5667,"loss/avg_raw_advantages":0.0169,"loss/avg_raw_advantages_abs":0.1284,"policy/policy_entropy":0.1255,"policy/policy_loss":-0.0154,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0121,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.5667,"timing/step":2114.1214,"trainer/epoch":0}
+{"step":28,"async/staleness_mean":0.75,"generate/avg_num_tokens":8692.0527,"generate/avg_tokens_non_zero_rewards":9277.3557,"generate/avg_tokens_zero_rewards":4666.9692,"generate/max_num_tokens":31733,"generate/std_num_tokens":6704.6075,"loss/avg_final_rewards":0.5886,"loss/avg_raw_advantages":0.0166,"loss/avg_raw_advantages_abs":0.1246,"policy/policy_entropy":0.149,"policy/policy_loss":-0.0179,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0137,"reward/avg_pass_at_8":0.9815,"reward/avg_raw_reward":0.5886,"timing/step":8105.2142,"trainer/epoch":0}
+{"step":29,"async/staleness_mean":1.0,"generate/avg_num_tokens":10905.0195,"generate/avg_tokens_non_zero_rewards":14689.5271,"generate/avg_tokens_zero_rewards":2654.323,"generate/max_num_tokens":31884,"generate/std_num_tokens":9216.2628,"loss/avg_final_rewards":0.4064,"loss/avg_raw_advantages":0.0296,"loss/avg_raw_advantages_abs":0.1358,"policy/policy_entropy":0.0998,"policy/policy_loss":-0.0205,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.011,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.4064,"timing/step":1155.1303,"trainer/epoch":0}
+{"step":30,"async/staleness_mean":1.9531,"generate/avg_num_tokens":11282.5879,"generate/avg_tokens_non_zero_rewards":14253.5969,"generate/avg_tokens_zero_rewards":2552.3923,"generate/max_num_tokens":32120,"generate/std_num_tokens":8607.3032,"loss/avg_final_rewards":0.448,"loss/avg_raw_advantages":0.0173,"loss/avg_raw_advantages_abs":0.1516,"policy/policy_entropy":0.1042,"policy/policy_loss":-0.0177,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0104,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.448,"timing/step":1069.2842,"trainer/epoch":0}
+{"step":31,"async/staleness_mean":2.4062,"generate/avg_num_tokens":9116.4531,"generate/avg_tokens_non_zero_rewards":10364.7674,"generate/avg_tokens_zero_rewards":3637.0105,"generate/max_num_tokens":32047,"generate/std_num_tokens":6662.5562,"loss/avg_final_rewards":0.5596,"loss/avg_raw_advantages":0.0125,"loss/avg_raw_advantages_abs":0.1174,"policy/policy_entropy":0.1297,"policy/policy_loss":-0.0064,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0106,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.5596,"timing/step":3380.0364,"trainer/epoch":0}
+{"step":32,"async/staleness_mean":2.5312,"generate/avg_num_tokens":9350.9492,"generate/avg_tokens_non_zero_rewards":11628.3231,"generate/avg_tokens_zero_rewards":2070.8197,"generate/max_num_tokens":32051,"generate/std_num_tokens":7606.569,"loss/avg_final_rewards":0.516,"loss/avg_raw_advantages":0.0084,"loss/avg_raw_advantages_abs":0.146,"policy/policy_entropy":0.1175,"policy/policy_loss":-0.0091,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0139,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.516,"timing/step":2638.9178,"trainer/epoch":0}
+{"step":33,"async/staleness_mean":2.5,"generate/avg_num_tokens":11750.9355,"generate/avg_tokens_non_zero_rewards":13641.2512,"generate/avg_tokens_zero_rewards":3765.3163,"generate/max_num_tokens":32123,"generate/std_num_tokens":8456.3621,"loss/avg_final_rewards":0.4489,"loss/avg_raw_advantages":0.0123,"loss/avg_raw_advantages_abs":0.1183,"policy/policy_entropy":0.1132,"policy/policy_loss":-0.0066,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0119,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.4489,"timing/step":1628.9191,"trainer/epoch":0}
+{"step":34,"async/staleness_mean":2.5,"generate/avg_num_tokens":9573.7402,"generate/avg_tokens_non_zero_rewards":10717.4671,"generate/avg_tokens_zero_rewards":2469.7465,"generate/max_num_tokens":32045,"generate/std_num_tokens":6242.7726,"loss/avg_final_rewards":0.5954,"loss/avg_raw_advantages":0.019,"loss/avg_raw_advantages_abs":0.1491,"policy/policy_entropy":0.133,"policy/policy_loss":-0.0101,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0128,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.5954,"timing/step":2030.3738,"trainer/epoch":0}
+{"step":35,"async/staleness_mean":2.4844,"generate/avg_num_tokens":10894.6309,"generate/avg_tokens_non_zero_rewards":12434.633,"generate/avg_tokens_zero_rewards":2059.8816,"generate/max_num_tokens":32051,"generate/std_num_tokens":7367.9947,"loss/avg_final_rewards":0.5028,"loss/avg_raw_advantages":0.0081,"loss/avg_raw_advantages_abs":0.114,"policy/policy_entropy":0.1195,"policy/policy_loss":-0.0086,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0127,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.5028,"timing/step":3126.3372,"trainer/epoch":0}
+{"step":36,"async/staleness_mean":2.5781,"generate/avg_num_tokens":10317.5273,"generate/avg_tokens_non_zero_rewards":12661.5225,"generate/avg_tokens_zero_rewards":1946.1161,"generate/max_num_tokens":32119,"generate/std_num_tokens":7794.2575,"loss/avg_final_rewards":0.5109,"loss/avg_raw_advantages":0.0014,"loss/avg_raw_advantages_abs":0.1617,"policy/policy_entropy":0.1122,"policy/policy_loss":-0.0106,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0126,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.5109,"timing/step":2236.2336,"trainer/epoch":0}
+{"step":37,"async/staleness_mean":2.3125,"generate/avg_num_tokens":10482.5938,"generate/avg_tokens_non_zero_rewards":11641.9167,"generate/avg_tokens_zero_rewards":5934.4808,"generate/max_num_tokens":32051,"generate/std_num_tokens":7190.2295,"loss/avg_final_rewards":0.4886,"loss/avg_raw_advantages":0.004,"loss/avg_raw_advantages_abs":0.135,"policy/policy_entropy":0.1224,"policy/policy_loss":-0.0074,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0126,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.4886,"timing/step":1792.8176,"trainer/epoch":0}
+{"step":38,"async/staleness_mean":2.3438,"generate/avg_num_tokens":9999.0859,"generate/avg_tokens_non_zero_rewards":11378.8105,"generate/avg_tokens_zero_rewards":1832.6081,"generate/max_num_tokens":31882,"generate/std_num_tokens":7055.9958,"loss/avg_final_rewards":0.5558,"loss/avg_raw_advantages":0.0107,"loss/avg_raw_advantages_abs":0.1305,"policy/policy_entropy":0.1208,"policy/policy_loss":-0.0049,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0121,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.5558,"timing/step":2562.2797,"trainer/epoch":0}
+{"step":39,"async/staleness_mean":2.3438,"generate/avg_num_tokens":9061.4551,"generate/avg_tokens_non_zero_rewards":10640.0995,"generate/avg_tokens_zero_rewards":3292.2273,"generate/max_num_tokens":32045,"generate/std_num_tokens":7388.3659,"loss/avg_final_rewards":0.4996,"loss/avg_raw_advantages":0.0168,"loss/avg_raw_advantages_abs":0.1453,"policy/policy_entropy":0.1188,"policy/policy_loss":-0.0131,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0149,"reward/avg_pass_at_8":0.9531,"reward/avg_raw_reward":0.4996,"timing/step":2320.4125,"trainer/epoch":0}
+{"step":40,"async/staleness_mean":2.5625,"generate/avg_num_tokens":10218.9258,"generate/avg_tokens_non_zero_rewards":11969.8435,"generate/avg_tokens_zero_rewards":3266.2524,"generate/max_num_tokens":32049,"generate/std_num_tokens":6845.5778,"loss/avg_final_rewards":0.5163,"loss/avg_raw_advantages":0.0291,"loss/avg_raw_advantages_abs":0.1419,"policy/policy_entropy":0.1153,"policy/policy_loss":-0.016,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0126,"reward/avg_pass_at_8":0.9531,"reward/avg_raw_reward":0.5163,"timing/step":1652.7905,"trainer/epoch":0}
+{"step":41,"async/staleness_mean":2.7344,"generate/avg_num_tokens":9904.8164,"generate/avg_tokens_non_zero_rewards":11499.066,"generate/avg_tokens_zero_rewards":2223.4318,"generate/max_num_tokens":32042,"generate/std_num_tokens":6720.7832,"loss/avg_final_rewards":0.5634,"loss/avg_raw_advantages":0.0143,"loss/avg_raw_advantages_abs":0.1337,"policy/policy_entropy":0.1218,"policy/policy_loss":-0.0089,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0132,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.5634,"timing/step":2326.2753,"trainer/epoch":0}
+{"step":42,"async/staleness_mean":2.5625,"generate/avg_num_tokens":9639.502,"generate/avg_tokens_non_zero_rewards":11166.766,"generate/avg_tokens_zero_rewards":2380.7079,"generate/max_num_tokens":32049,"generate/std_num_tokens":6990.9528,"loss/avg_final_rewards":0.576,"loss/avg_raw_advantages":0.0291,"loss/avg_raw_advantages_abs":0.1305,"policy/policy_entropy":0.1126,"policy/policy_loss":-0.0184,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0121,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.576,"timing/step":1945.1063,"trainer/epoch":0}
+{"step":43,"async/staleness_mean":0.4375,"generate/avg_num_tokens":8526.0098,"generate/avg_tokens_non_zero_rewards":9153.4397,"generate/avg_tokens_zero_rewards":2460.8542,"generate/max_num_tokens":31657,"generate/std_num_tokens":5366.2312,"loss/avg_final_rewards":0.6858,"loss/avg_raw_advantages":0.0365,"loss/avg_raw_advantages_abs":0.1366,"policy/policy_entropy":0.1384,"policy/policy_loss":-0.0328,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0155,"reward/avg_pass_at_8":0.9821,"reward/avg_raw_reward":0.6858,"timing/step":7503.4743,"trainer/epoch":0}
+{"step":44,"async/staleness_mean":1.0,"generate/avg_num_tokens":10857.1738,"generate/avg_tokens_non_zero_rewards":13542.7188,"generate/avg_tokens_zero_rewards":2800.5391,"generate/max_num_tokens":31842,"generate/std_num_tokens":8448.3655,"loss/avg_final_rewards":0.4057,"loss/avg_raw_advantages":0.0253,"loss/avg_raw_advantages_abs":0.1327,"policy/policy_entropy":0.0992,"policy/policy_loss":-0.0222,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0117,"reward/avg_pass_at_8":0.9531,"reward/avg_raw_reward":0.4057,"timing/step":2178.0809,"trainer/epoch":0}
+{"step":45,"async/staleness_mean":1.9219,"generate/avg_num_tokens":8753.6836,"generate/avg_tokens_non_zero_rewards":11951.7628,"generate/avg_tokens_zero_rewards":2804.1844,"generate/max_num_tokens":31838,"generate/std_num_tokens":8045.577,"loss/avg_final_rewards":0.4379,"loss/avg_raw_advantages":0.0202,"loss/avg_raw_advantages_abs":0.1272,"policy/policy_entropy":0.089,"policy/policy_loss":-0.016,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0108,"reward/avg_pass_at_8":0.9531,"reward/avg_raw_reward":0.4379,"timing/step":1175.6698,"trainer/epoch":0}
+{"step":46,"async/staleness_mean":2.1875,"generate/avg_num_tokens":9020.1895,"generate/avg_tokens_non_zero_rewards":11089.6061,"generate/avg_tokens_zero_rewards":1955.6293,"generate/max_num_tokens":33138,"generate/std_num_tokens":7640.7865,"loss/avg_final_rewards":0.5028,"loss/avg_raw_advantages":0.0222,"loss/avg_raw_advantages_abs":0.1315,"policy/policy_entropy":0.106,"policy/policy_loss":-0.018,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0289,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.5028,"timing/step":3217.8506,"trainer/epoch":0}
+{"step":47,"async/staleness_mean":2.3125,"generate/avg_num_tokens":9114.3848,"generate/avg_tokens_non_zero_rewards":11057.5164,"generate/avg_tokens_zero_rewards":2406.3565,"generate/max_num_tokens":32280,"generate/std_num_tokens":6726.3824,"loss/avg_final_rewards":0.5133,"loss/avg_raw_advantages":0.0356,"loss/avg_raw_advantages_abs":0.1508,"policy/policy_entropy":0.1057,"policy/policy_loss":-0.0267,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0148,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.5133,"timing/step":2750.6137,"trainer/epoch":0}
+{"step":48,"async/staleness_mean":2.4062,"generate/avg_num_tokens":8759.4766,"generate/avg_tokens_non_zero_rewards":12008.6229,"generate/avg_tokens_zero_rewards":1206.2662,"generate/max_num_tokens":31994,"generate/std_num_tokens":7475.912,"loss/avg_final_rewards":0.4376,"loss/avg_raw_advantages":0.02,"loss/avg_raw_advantages_abs":0.13,"policy/policy_entropy":0.0896,"policy/policy_loss":-0.0175,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0294,"reward/avg_pass_at_8":0.9531,"reward/avg_raw_reward":0.4376,"timing/step":2385.7088,"trainer/epoch":0}
+{"step":49,"async/staleness_mean":2.3594,"generate/avg_num_tokens":8316.1094,"generate/avg_tokens_non_zero_rewards":10984.5378,"generate/avg_tokens_zero_rewards":1363.162,"generate/max_num_tokens":31760,"generate/std_num_tokens":6980.733,"loss/avg_final_rewards":0.4956,"loss/avg_raw_advantages":0.0331,"loss/avg_raw_advantages_abs":0.1298,"policy/policy_entropy":0.0933,"policy/policy_loss":-0.0203,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0125,"reward/avg_pass_at_8":0.9375,"reward/avg_raw_reward":0.4956,"timing/step":2394.7512,"trainer/epoch":0}
+{"step":50,"async/staleness_mean":2.1562,"generate/avg_num_tokens":9415.2988,"generate/avg_tokens_non_zero_rewards":10773.6524,"generate/avg_tokens_zero_rewards":694.2754,"generate/max_num_tokens":32046,"generate/std_num_tokens":6691.1476,"loss/avg_final_rewards":0.5997,"loss/avg_raw_advantages":0.0347,"loss/avg_raw_advantages_abs":0.1401,"policy/policy_entropy":0.1122,"policy/policy_loss":-0.034,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0142,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.5997,"timing/step":2754.4871,"trainer/epoch":0}
+{"step":51,"async/staleness_mean":2.625,"generate/avg_num_tokens":7991.6914,"generate/avg_tokens_non_zero_rewards":10300.8103,"generate/avg_tokens_zero_rewards":2033.1958,"generate/max_num_tokens":32118,"generate/std_num_tokens":6981.3409,"loss/avg_final_rewards":0.486,"loss/avg_raw_advantages":0.0258,"loss/avg_raw_advantages_abs":0.1455,"policy/policy_entropy":0.0989,"policy/policy_loss":-0.0285,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0146,"reward/avg_pass_at_8":0.9531,"reward/avg_raw_reward":0.486,"timing/step":1850.0278,"trainer/epoch":0}
+{"step":52,"async/staleness_mean":0.4531,"generate/avg_num_tokens":8983.584,"generate/avg_tokens_non_zero_rewards":9595.1301,"generate/avg_tokens_zero_rewards":2313.4651,"generate/max_num_tokens":31648,"generate/std_num_tokens":5431.2092,"loss/avg_final_rewards":0.6782,"loss/avg_raw_advantages":0.0182,"loss/avg_raw_advantages_abs":0.1253,"policy/policy_entropy":0.1224,"policy/policy_loss":-0.0118,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0141,"reward/avg_pass_at_8":0.9836,"reward/avg_raw_reward":0.6782,"timing/step":7710.4512,"trainer/epoch":0}
+{"step":53,"async/staleness_mean":1.0,"generate/avg_num_tokens":11968.8965,"generate/avg_tokens_non_zero_rewards":14394.2147,"generate/avg_tokens_zero_rewards":4842.1923,"generate/max_num_tokens":32049,"generate/std_num_tokens":8483.5468,"loss/avg_final_rewards":0.4545,"loss/avg_raw_advantages":0.0165,"loss/avg_raw_advantages_abs":0.1415,"policy/policy_entropy":0.0901,"policy/policy_loss":-0.0122,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0123,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.4545,"timing/step":1194.0669,"trainer/epoch":0}
+{"step":54,"async/staleness_mean":1.8906,"generate/avg_num_tokens":11108.5215,"generate/avg_tokens_non_zero_rewards":12862.0496,"generate/avg_tokens_zero_rewards":2774.3371,"generate/max_num_tokens":32232,"generate/std_num_tokens":8304.5944,"loss/avg_final_rewards":0.5139,"loss/avg_raw_advantages":0.0056,"loss/avg_raw_advantages_abs":0.1438,"policy/policy_entropy":0.0946,"policy/policy_loss":-0.0085,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0126,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.5139,"timing/step":1531.9723,"trainer/epoch":0}
+{"step":55,"async/staleness_mean":2.3594,"generate/avg_num_tokens":10867.6074,"generate/avg_tokens_non_zero_rewards":11586.3348,"generate/avg_tokens_zero_rewards":3586.587,"generate/max_num_tokens":31860,"generate/std_num_tokens":6673.593,"loss/avg_final_rewards":0.576,"loss/avg_raw_advantages":0.0091,"loss/avg_raw_advantages_abs":0.1225,"policy/policy_entropy":0.1094,"policy/policy_loss":-0.0041,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0136,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.576,"timing/step":2104.6037,"trainer/epoch":0}
+{"step":56,"async/staleness_mean":2.6562,"generate/avg_num_tokens":10019.8652,"generate/avg_tokens_non_zero_rewards":10413.0448,"generate/avg_tokens_zero_rewards":5731.4651,"generate/max_num_tokens":32047,"generate/std_num_tokens":6491.9299,"loss/avg_final_rewards":0.6383,"loss/avg_raw_advantages":0.0059,"loss/avg_raw_advantages_abs":0.1323,"policy/policy_entropy":0.1111,"policy/policy_loss":-0.0103,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0152,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.6383,"timing/step":2776.3766,"trainer/epoch":0}
+{"step":57,"async/staleness_mean":2.8281,"generate/avg_num_tokens":10692.1504,"generate/avg_tokens_non_zero_rewards":12498.0024,"generate/avg_tokens_zero_rewards":2765.4105,"generate/max_num_tokens":32109,"generate/std_num_tokens":8081.0706,"loss/avg_final_rewards":0.5351,"loss/avg_raw_advantages":0.0017,"loss/avg_raw_advantages_abs":0.1368,"policy/policy_entropy":0.0919,"policy/policy_loss":-0.0086,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0132,"reward/avg_pass_at_8":0.9531,"reward/avg_raw_reward":0.5351,"timing/step":1932.6384,"trainer/epoch":0}
+{"step":58,"async/staleness_mean":2.4844,"generate/avg_num_tokens":10424.6172,"generate/avg_tokens_non_zero_rewards":11960.4939,"generate/avg_tokens_zero_rewards":4325.8447,"generate/max_num_tokens":31812,"generate/std_num_tokens":7610.1572,"loss/avg_final_rewards":0.518,"loss/avg_raw_advantages":0.0066,"loss/avg_raw_advantages_abs":0.1312,"policy/policy_entropy":0.0981,"policy/policy_loss":-0.0111,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0177,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.518,"timing/step":1760.3623,"trainer/epoch":0}
+{"step":59,"async/staleness_mean":2.4688,"generate/avg_num_tokens":10010.4961,"generate/avg_tokens_non_zero_rewards":10718.9843,"generate/avg_tokens_zero_rewards":5222.8333,"generate/max_num_tokens":31864,"generate/std_num_tokens":7115.95,"loss/avg_final_rewards":0.6093,"loss/avg_raw_advantages":0.0099,"loss/avg_raw_advantages_abs":0.1241,"policy/policy_entropy":0.1053,"policy/policy_loss":-0.0027,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0133,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.6093,"timing/step":1933.2122,"trainer/epoch":0}
+{"step":60,"async/staleness_mean":2.6094,"generate/avg_num_tokens":10168.5332,"generate/avg_tokens_non_zero_rewards":11320.7862,"generate/avg_tokens_zero_rewards":1956.4444,"generate/max_num_tokens":32334,"generate/std_num_tokens":6849.4232,"loss/avg_final_rewards":0.6181,"loss/avg_raw_advantages":0.027,"loss/avg_raw_advantages_abs":0.1435,"policy/policy_entropy":0.1025,"policy/policy_loss":-0.0131,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0156,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.6181,"timing/step":2221.5922,"trainer/epoch":0}
+{"step":61,"async/staleness_mean":2.5156,"generate/avg_num_tokens":10184.793,"generate/avg_tokens_non_zero_rewards":11908.068,"generate/avg_tokens_zero_rewards":3084.9,"generate/max_num_tokens":32049,"generate/std_num_tokens":7351.8065,"loss/avg_final_rewards":0.5326,"loss/avg_raw_advantages":0.009,"loss/avg_raw_advantages_abs":0.1271,"policy/policy_entropy":0.0961,"policy/policy_loss":-0.0029,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.014,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.5326,"timing/step":2152.2381,"trainer/epoch":0}
+{"step":62,"async/staleness_mean":2.5,"generate/avg_num_tokens":9331.7812,"generate/avg_tokens_non_zero_rewards":10914.7356,"generate/avg_tokens_zero_rewards":2472.3125,"generate/max_num_tokens":32049,"generate/std_num_tokens":6865.2744,"loss/avg_final_rewards":0.5387,"loss/avg_raw_advantages":0.0108,"loss/avg_raw_advantages_abs":0.1101,"policy/policy_entropy":0.0966,"policy/policy_loss":-0.0087,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0132,"reward/avg_pass_at_8":0.9531,"reward/avg_raw_reward":0.5387,"timing/step":1654.4657,"trainer/epoch":0}
+{"step":63,"async/staleness_mean":2.625,"generate/avg_num_tokens":9486.832,"generate/avg_tokens_non_zero_rewards":10367.2965,"generate/avg_tokens_zero_rewards":2854.0,"generate/max_num_tokens":32121,"generate/std_num_tokens":6102.5569,"loss/avg_final_rewards":0.6796,"loss/avg_raw_advantages":0.02,"loss/avg_raw_advantages_abs":0.1069,"policy/policy_entropy":0.1018,"policy/policy_loss":-0.0147,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0134,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.6796,"timing/step":1965.9876,"trainer/epoch":0}
+{"step":64,"async/staleness_mean":2.5156,"generate/avg_num_tokens":10798.4805,"generate/avg_tokens_non_zero_rewards":12088.3382,"generate/avg_tokens_zero_rewards":5349.4898,"generate/max_num_tokens":31906,"generate/std_num_tokens":7584.6937,"loss/avg_final_rewards":0.5364,"loss/avg_raw_advantages":0.0113,"loss/avg_raw_advantages_abs":0.1162,"policy/policy_entropy":0.0945,"policy/policy_loss":-0.009,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0119,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.5364,"timing/step":1857.1893,"trainer/epoch":0}
+{"step":65,"async/staleness_mean":2.4688,"generate/avg_num_tokens":10813.1543,"generate/avg_tokens_non_zero_rewards":11583.4165,"generate/avg_tokens_zero_rewards":3850.5882,"generate/max_num_tokens":31906,"generate/std_num_tokens":6913.4062,"loss/avg_final_rewards":0.6414,"loss/avg_raw_advantages":0.0123,"loss/avg_raw_advantages_abs":0.1119,"policy/policy_entropy":0.1004,"policy/policy_loss":-0.0049,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.012,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.6414,"timing/step":2337.8249,"trainer/epoch":0}
+{"step":66,"async/staleness_mean":2.6719,"generate/avg_num_tokens":10605.4082,"generate/avg_tokens_non_zero_rewards":11567.5558,"generate/avg_tokens_zero_rewards":2610.8364,"generate/max_num_tokens":31877,"generate/std_num_tokens":6789.7385,"loss/avg_final_rewards":0.5981,"loss/avg_raw_advantages":0.0008,"loss/avg_raw_advantages_abs":0.1207,"policy/policy_entropy":0.1001,"policy/policy_loss":-0.0028,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0129,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.5981,"timing/step":1795.6286,"trainer/epoch":0}
+{"step":67,"async/staleness_mean":2.5312,"generate/avg_num_tokens":10565.9805,"generate/avg_tokens_non_zero_rewards":11555.4614,"generate/avg_tokens_zero_rewards":4519.1528,"generate/max_num_tokens":31830,"generate/std_num_tokens":7442.9398,"loss/avg_final_rewards":0.5891,"loss/avg_raw_advantages":0.003,"loss/avg_raw_advantages_abs":0.1114,"policy/policy_entropy":0.0957,"policy/policy_loss":-0.0034,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0131,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.5891,"timing/step":2330.4669,"trainer/epoch":0}
+{"step":68,"async/staleness_mean":2.625,"generate/avg_num_tokens":9743.6543,"generate/avg_tokens_non_zero_rewards":11561.9899,"generate/avg_tokens_zero_rewards":3395.4298,"generate/max_num_tokens":31921,"generate/std_num_tokens":7782.6672,"loss/avg_final_rewards":0.5301,"loss/avg_raw_advantages":0.0179,"loss/avg_raw_advantages_abs":0.1379,"policy/policy_entropy":0.089,"policy/policy_loss":-0.0099,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0148,"reward/avg_pass_at_8":0.9531,"reward/avg_raw_reward":0.5301,"timing/step":2086.6562,"trainer/epoch":0}
+{"step":69,"async/staleness_mean":2.2969,"generate/avg_num_tokens":11047.3789,"generate/avg_tokens_non_zero_rewards":11709.6925,"generate/avg_tokens_zero_rewards":4494.7021,"generate/max_num_tokens":31892,"generate/std_num_tokens":6973.144,"loss/avg_final_rewards":0.6016,"loss/avg_raw_advantages":0.0066,"loss/avg_raw_advantages_abs":0.1147,"policy/policy_entropy":0.0962,"policy/policy_loss":-0.0042,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0125,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.6016,"timing/step":1763.9431,"trainer/epoch":0}
+{"step":70,"async/staleness_mean":0.8281,"generate/avg_num_tokens":9673.7988,"generate/avg_tokens_non_zero_rewards":9948.0247,"generate/avg_tokens_zero_rewards":4547.8846,"generate/max_num_tokens":32047,"generate/std_num_tokens":5122.4505,"loss/avg_final_rewards":0.7178,"loss/avg_raw_advantages":0.0134,"loss/avg_raw_advantages_abs":0.0889,"policy/policy_entropy":0.1091,"policy/policy_loss":-0.0062,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.013,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.7178,"timing/step":6252.1126,"trainer/epoch":0}
+{"step":71,"async/staleness_mean":0.9844,"generate/avg_num_tokens":12553.4453,"generate/avg_tokens_non_zero_rewards":13870.9911,"generate/avg_tokens_zero_rewards":3330.625,"generate/max_num_tokens":32049,"generate/std_num_tokens":8285.0009,"loss/avg_final_rewards":0.5413,"loss/avg_raw_advantages":0.0144,"loss/avg_raw_advantages_abs":0.1347,"policy/policy_entropy":0.091,"policy/policy_loss":-0.0105,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0135,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.5413,"timing/step":2571.0828,"trainer/epoch":0}
+{"step":72,"async/staleness_mean":1.8125,"generate/avg_num_tokens":12188.6406,"generate/avg_tokens_non_zero_rewards":15205.9711,"generate/avg_tokens_zero_rewards":3502.3864,"generate/max_num_tokens":32041,"generate/std_num_tokens":9010.8414,"loss/avg_final_rewards":0.451,"loss/avg_raw_advantages":0.0119,"loss/avg_raw_advantages_abs":0.1232,"policy/policy_entropy":0.0745,"policy/policy_loss":-0.0091,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0104,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.451,"timing/step":904.4599,"trainer/epoch":0}
+{"step":73,"async/staleness_mean":2.1875,"generate/avg_num_tokens":10409.3262,"generate/avg_tokens_non_zero_rewards":11811.1733,"generate/avg_tokens_zero_rewards":3367.1059,"generate/max_num_tokens":32118,"generate/std_num_tokens":7098.8033,"loss/avg_final_rewards":0.5814,"loss/avg_raw_advantages":0.0013,"loss/avg_raw_advantages_abs":0.1175,"policy/policy_entropy":0.089,"policy/policy_loss":-0.0017,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0153,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.5814,"timing/step":2376.5707,"trainer/epoch":0}
+{"step":74,"async/staleness_mean":2.6406,"generate/avg_num_tokens":10459.8105,"generate/avg_tokens_non_zero_rewards":11643.9531,"generate/avg_tokens_zero_rewards":2170.8125,"generate/max_num_tokens":31867,"generate/std_num_tokens":5955.108,"loss/avg_final_rewards":0.6191,"loss/avg_raw_advantages":0.0084,"loss/avg_raw_advantages_abs":0.1129,"policy/policy_entropy":0.094,"policy/policy_loss":-0.0039,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0141,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.6191,"timing/step":2432.5072,"trainer/epoch":0}
+{"step":75,"async/staleness_mean":2.4531,"generate/avg_num_tokens":10980.2188,"generate/avg_tokens_non_zero_rewards":11777.0306,"generate/avg_tokens_zero_rewards":4359.4364,"generate/max_num_tokens":32716,"generate/std_num_tokens":6644.6176,"loss/avg_final_rewards":0.6147,"loss/avg_raw_advantages":0.0106,"loss/avg_raw_advantages_abs":0.1136,"policy/policy_entropy":0.0935,"policy/policy_loss":-0.0054,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0136,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.6147,"timing/step":2139.2163,"trainer/epoch":0}
+{"step":76,"async/staleness_mean":1.1406,"generate/avg_num_tokens":9838.6426,"generate/avg_tokens_non_zero_rewards":10797.2354,"generate/avg_tokens_zero_rewards":3360.8788,"generate/max_num_tokens":31829,"generate/std_num_tokens":5633.872,"loss/avg_final_rewards":0.6303,"loss/avg_raw_advantages":0.0354,"loss/avg_raw_advantages_abs":0.1315,"policy/policy_entropy":0.0946,"policy/policy_loss":-0.0305,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0225,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.6303,"timing/step":7783.8267,"trainer/epoch":0}

viewer/build/inputs/marin/runs/marin-a3-llm-verifier-freelancer/run.json

@@ -0,0 +1,27 @@
+{
+ "id": "marin-a3-llm-verifier-freelancer",
+ "title": "A3 RLOO on llm-verifier-freelancer (Qwen3-8B agent)",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://huggingface.co/laion/a3-rl-DCAgent_llm-verifier-freelancer-70-8B/tree/main/training_logs",
+ "license": "unknown",
+ "model": "laion/a3-rl-DCAgent_llm-verifier-freelancer-70-8B",
+ "base_model": "laion/GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink",
+ "method": "RLOO-N (SkyRL, binary verifier reward)",
+ "dataset": "DCAgent/llm-verifier-freelancer",
+ "eval_suite": null,
+ "kind": "training",
+ "state": "finished",
+ "started_at": null,
+ "updated_at": "2026-05-25T07:12:34Z",
+ "attempts": 0,
+ "note": "Marin A3 sweep (issue #6187): one RLOO-N run per training dataset from the same Qwen3-8B-derived SFT agent, here DCAgent/llm-verifier-freelancer; the highest A3 reward (0.718 at the EMA-best step 70, per the issue). Our copy has every logged step of the final lineage (11 resumed job segments, 12 superseded rows dropped). The issue concluded this binary-reward setup was uninformative about dataset utility.",
+ "metrics_map": {
+  "reward": "reward/avg_raw_reward",
+  "loss": "policy/policy_loss",
+  "entropy": "policy/policy_entropy",
+  "grad_norm": "policy/raw_grad_norm",
+  "response_length": "generate/avg_num_tokens"
+ }
+}

viewer/build/inputs/marin/runs/marin-a3-nemotron-agent-calendar/metrics.jsonl

@@ -0,0 +1,80 @@
+{"step":1,"async/staleness_mean":0.0,"generate/avg_num_tokens":1431.8281,"generate/avg_tokens_non_zero_rewards":2889.6108,"generate/avg_tokens_zero_rewards":726.1768,"generate/max_num_tokens":11273,"generate/std_num_tokens":1901.8064,"loss/avg_final_rewards":0.3262,"loss/avg_raw_advantages":0.0775,"loss/avg_raw_advantages_abs":0.3587,"policy/policy_entropy":0.0623,"policy/policy_loss":-0.0593,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0237,"reward/avg_pass_at_8":0.8281,"reward/avg_raw_reward":0.3262,"timing/step":16272.8114,"trainer/epoch":0}
+{"step":2,"async/staleness_mean":0.9844,"generate/avg_num_tokens":614.2637,"generate/avg_tokens_non_zero_rewards":2778.7671,"generate/avg_tokens_zero_rewards":254.3349,"generate/max_num_tokens":14488,"generate/std_num_tokens":1513.2025,"loss/avg_final_rewards":0.1426,"loss/avg_raw_advantages":0.2496,"loss/avg_raw_advantages_abs":0.3492,"policy/policy_entropy":0.0271,"policy/policy_loss":-0.0559,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.028,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.1426,"timing/step":12716.0999,"trainer/epoch":0}
+{"step":3,"async/staleness_mean":0.2969,"generate/avg_num_tokens":2085.2129,"generate/avg_tokens_non_zero_rewards":2333.9798,"generate/avg_tokens_zero_rewards":1235.9741,"generate/max_num_tokens":11796,"generate/std_num_tokens":1504.9847,"loss/avg_final_rewards":0.7734,"loss/avg_raw_advantages":0.0113,"loss/avg_raw_advantages_abs":0.1682,"policy/policy_entropy":0.1068,"policy/policy_loss":-0.0151,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0261,"reward/avg_pass_at_8":0.9194,"reward/avg_raw_reward":0.7734,"timing/step":1385.6931,"trainer/epoch":0}
+{"step":4,"async/staleness_mean":0.9531,"generate/avg_num_tokens":2736.7129,"generate/avg_tokens_non_zero_rewards":2478.469,"generate/avg_tokens_zero_rewards":3915.6522,"generate/max_num_tokens":16638,"generate/std_num_tokens":1706.5579,"loss/avg_final_rewards":0.8203,"loss/avg_raw_advantages":-0.0196,"loss/avg_raw_advantages_abs":0.2291,"policy/policy_entropy":0.1264,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0289,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.8203,"timing/step":421.796,"trainer/epoch":0}
+{"step":5,"async/staleness_mean":1.6094,"generate/avg_num_tokens":2976.0801,"generate/avg_tokens_non_zero_rewards":2705.5583,"generate/avg_tokens_zero_rewards":3616.7895,"generate/max_num_tokens":14808,"generate/std_num_tokens":1868.1502,"loss/avg_final_rewards":0.7031,"loss/avg_raw_advantages":-0.0001,"loss/avg_raw_advantages_abs":0.2735,"policy/policy_entropy":0.1246,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.028,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.7031,"timing/step":454.4222,"trainer/epoch":0}
+{"step":6,"async/staleness_mean":2.1562,"generate/avg_num_tokens":2689.2773,"generate/avg_tokens_non_zero_rewards":2421.3426,"generate/avg_tokens_zero_rewards":3614.2348,"generate/max_num_tokens":12730,"generate/std_num_tokens":1537.6723,"loss/avg_final_rewards":0.7754,"loss/avg_raw_advantages":-0.0164,"loss/avg_raw_advantages_abs":0.2754,"policy/policy_entropy":0.1243,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0306,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.7754,"timing/step":356.6975,"trainer/epoch":0}
+{"step":7,"async/staleness_mean":2.7656,"generate/avg_num_tokens":3006.9551,"generate/avg_tokens_non_zero_rewards":2735.0862,"generate/avg_tokens_zero_rewards":3814.1318,"generate/max_num_tokens":15163,"generate/std_num_tokens":2099.7744,"loss/avg_final_rewards":0.748,"loss/avg_raw_advantages":-0.0351,"loss/avg_raw_advantages_abs":0.3179,"policy/policy_entropy":0.1262,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0334,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.748,"timing/step":453.3526,"trainer/epoch":0}
+{"step":8,"async/staleness_mean":3.0312,"generate/avg_num_tokens":3005.291,"generate/avg_tokens_non_zero_rewards":2802.7748,"generate/avg_tokens_zero_rewards":3762.8519,"generate/max_num_tokens":28052,"generate/std_num_tokens":2190.8773,"loss/avg_final_rewards":0.7891,"loss/avg_raw_advantages":-0.0292,"loss/avg_raw_advantages_abs":0.2875,"policy/policy_entropy":0.1201,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0299,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.7891,"timing/step":369.1187,"trainer/epoch":0}
+{"step":9,"async/staleness_mean":3.2656,"generate/avg_num_tokens":2968.4883,"generate/avg_tokens_non_zero_rewards":2685.375,"generate/avg_tokens_zero_rewards":3893.325,"generate/max_num_tokens":14971,"generate/std_num_tokens":1731.3402,"loss/avg_final_rewards":0.7656,"loss/avg_raw_advantages":-0.0382,"loss/avg_raw_advantages_abs":0.278,"policy/policy_entropy":0.1161,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0284,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.7656,"timing/step":289.7455,"trainer/epoch":0}
+{"step":10,"async/staleness_mean":3.7031,"generate/avg_num_tokens":3184.5234,"generate/avg_tokens_non_zero_rewards":2926.0157,"generate/avg_tokens_zero_rewards":3936.3664,"generate/max_num_tokens":23313,"generate/std_num_tokens":2232.7037,"loss/avg_final_rewards":0.7441,"loss/avg_raw_advantages":-0.0322,"loss/avg_raw_advantages_abs":0.27,"policy/policy_entropy":0.1188,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0296,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.7441,"timing/step":322.2241,"trainer/epoch":0}
+{"step":11,"async/staleness_mean":4.0625,"generate/avg_num_tokens":3158.5684,"generate/avg_tokens_non_zero_rewards":2915.6842,"generate/avg_tokens_zero_rewards":4016.1858,"generate/max_num_tokens":18903,"generate/std_num_tokens":2125.1072,"loss/avg_final_rewards":0.7793,"loss/avg_raw_advantages":-0.0314,"loss/avg_raw_advantages_abs":0.3072,"policy/policy_entropy":0.1181,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0303,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.7793,"timing/step":251.7501,"trainer/epoch":0}
+{"step":12,"async/staleness_mean":3.7344,"generate/avg_num_tokens":2897.3184,"generate/avg_tokens_non_zero_rewards":2694.0557,"generate/avg_tokens_zero_rewards":3583.547,"generate/max_num_tokens":16805,"generate/std_num_tokens":1754.4908,"loss/avg_final_rewards":0.7715,"loss/avg_raw_advantages":-0.0268,"loss/avg_raw_advantages_abs":0.2844,"policy/policy_entropy":0.1199,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0312,"reward/avg_pass_at_8":0.9688,"reward/avg_raw_reward":0.7715,"timing/step":293.1536,"trainer/epoch":0}
+{"step":13,"async/staleness_mean":4.0,"generate/avg_num_tokens":2865.752,"generate/avg_tokens_non_zero_rewards":2647.5686,"generate/avg_tokens_zero_rewards":3653.964,"generate/max_num_tokens":17502,"generate/std_num_tokens":1861.9091,"loss/avg_final_rewards":0.7832,"loss/avg_raw_advantages":-0.0135,"loss/avg_raw_advantages_abs":0.2793,"policy/policy_entropy":0.1162,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0317,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.7832,"timing/step":295.1542,"trainer/epoch":0}
+{"step":14,"async/staleness_mean":4.5312,"generate/avg_num_tokens":3130.1191,"generate/avg_tokens_non_zero_rewards":2818.9857,"generate/avg_tokens_zero_rewards":4531.8925,"generate/max_num_tokens":25853,"generate/std_num_tokens":2346.123,"loss/avg_final_rewards":0.8184,"loss/avg_raw_advantages":-0.04,"loss/avg_raw_advantages_abs":0.2752,"policy/policy_entropy":0.1144,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0286,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.8184,"timing/step":236.553,"trainer/epoch":0}
+{"step":15,"async/staleness_mean":4.5312,"generate/avg_num_tokens":3037.168,"generate/avg_tokens_non_zero_rewards":2717.8701,"generate/avg_tokens_zero_rewards":4289.7981,"generate/max_num_tokens":23975,"generate/std_num_tokens":2117.6372,"loss/avg_final_rewards":0.7969,"loss/avg_raw_advantages":-0.0415,"loss/avg_raw_advantages_abs":0.3254,"policy/policy_entropy":0.1174,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0311,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.7969,"timing/step":226.9706,"trainer/epoch":0}
+{"step":16,"async/staleness_mean":3.8906,"generate/avg_num_tokens":2836.6016,"generate/avg_tokens_non_zero_rewards":2672.5718,"generate/avg_tokens_zero_rewards":3823.0274,"generate/max_num_tokens":15261,"generate/std_num_tokens":1599.6131,"loss/avg_final_rewards":0.8574,"loss/avg_raw_advantages":-0.0401,"loss/avg_raw_advantages_abs":0.2081,"policy/policy_entropy":0.1147,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0239,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.8574,"timing/step":201.3474,"trainer/epoch":0}
+{"step":17,"async/staleness_mean":4.0625,"generate/avg_num_tokens":2856.3184,"generate/avg_tokens_non_zero_rewards":2694.0571,"generate/avg_tokens_zero_rewards":3816.7297,"generate/max_num_tokens":16407,"generate/std_num_tokens":1656.0729,"loss/avg_final_rewards":0.8555,"loss/avg_raw_advantages":-0.0268,"loss/avg_raw_advantages_abs":0.2471,"policy/policy_entropy":0.1143,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0291,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.8555,"timing/step":210.9901,"trainer/epoch":0}
+{"step":18,"async/staleness_mean":4.6562,"generate/avg_num_tokens":3046.6074,"generate/avg_tokens_non_zero_rewards":2938.9504,"generate/avg_tokens_zero_rewards":3444.6422,"generate/max_num_tokens":19486,"generate/std_num_tokens":1865.0103,"loss/avg_final_rewards":0.7871,"loss/avg_raw_advantages":-0.0125,"loss/avg_raw_advantages_abs":0.2417,"policy/policy_entropy":0.1117,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0314,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.7871,"timing/step":215.0986,"trainer/epoch":0}
+{"step":19,"async/staleness_mean":4.5156,"generate/avg_num_tokens":2960.2129,"generate/avg_tokens_non_zero_rewards":2813.3377,"generate/avg_tokens_zero_rewards":4232.2075,"generate/max_num_tokens":16390,"generate/std_num_tokens":1803.631,"loss/avg_final_rewards":0.8965,"loss/avg_raw_advantages":-0.0253,"loss/avg_raw_advantages_abs":0.2026,"policy/policy_entropy":0.1102,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0229,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.8965,"timing/step":211.8662,"trainer/epoch":0}
+{"step":20,"async/staleness_mean":4.25,"generate/avg_num_tokens":2770.2656,"generate/avg_tokens_non_zero_rewards":2650.6307,"generate/avg_tokens_zero_rewards":3900.6939,"generate/max_num_tokens":17942,"generate/std_num_tokens":1758.7398,"loss/avg_final_rewards":0.9043,"loss/avg_raw_advantages":-0.0214,"loss/avg_raw_advantages_abs":0.1757,"policy/policy_entropy":0.1095,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0243,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9043,"timing/step":202.3078,"trainer/epoch":0}
+{"step":21,"async/staleness_mean":4.5312,"generate/avg_num_tokens":2691.3633,"generate/avg_tokens_non_zero_rewards":2551.3468,"generate/avg_tokens_zero_rewards":3605.5882,"generate/max_num_tokens":10583,"generate/std_num_tokens":1312.5686,"loss/avg_final_rewards":0.8672,"loss/avg_raw_advantages":-0.0232,"loss/avg_raw_advantages_abs":0.1864,"policy/policy_entropy":0.1084,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0254,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.8672,"timing/step":188.4747,"trainer/epoch":0}
+{"step":22,"async/staleness_mean":4.5781,"generate/avg_num_tokens":2696.9902,"generate/avg_tokens_non_zero_rewards":2578.5987,"generate/avg_tokens_zero_rewards":3572.3115,"generate/max_num_tokens":12061,"generate/std_num_tokens":1342.3068,"loss/avg_final_rewards":0.8809,"loss/avg_raw_advantages":-0.0192,"loss/avg_raw_advantages_abs":0.1849,"policy/policy_entropy":0.1114,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0262,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.8809,"timing/step":190.1527,"trainer/epoch":0}
+{"step":23,"async/staleness_mean":4.2969,"generate/avg_num_tokens":2664.4746,"generate/avg_tokens_non_zero_rewards":2559.6116,"generate/avg_tokens_zero_rewards":3398.5156,"generate/max_num_tokens":9214,"generate/std_num_tokens":1283.4053,"loss/avg_final_rewards":0.875,"loss/avg_raw_advantages":-0.0141,"loss/avg_raw_advantages_abs":0.1952,"policy/policy_entropy":0.1107,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0278,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.875,"timing/step":182.7698,"trainer/epoch":0}
+{"step":24,"async/staleness_mean":4.9062,"generate/avg_num_tokens":2722.1816,"generate/avg_tokens_non_zero_rewards":2625.0022,"generate/avg_tokens_zero_rewards":3640.4286,"generate/max_num_tokens":13495,"generate/std_num_tokens":1452.6242,"loss/avg_final_rewards":0.9043,"loss/avg_raw_advantages":-0.019,"loss/avg_raw_advantages_abs":0.1909,"policy/policy_entropy":0.1116,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0271,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9043,"timing/step":181.0263,"trainer/epoch":0}
+{"step":25,"async/staleness_mean":4.129,"generate/avg_num_tokens":2633.0,"generate/avg_tokens_non_zero_rewards":2519.7539,"generate/avg_tokens_zero_rewards":3666.0816,"generate/max_num_tokens":13173,"generate/std_num_tokens":1296.4858,"loss/avg_final_rewards":0.9012,"loss/avg_raw_advantages":-0.0241,"loss/avg_raw_advantages_abs":0.1707,"policy/policy_entropy":0.1071,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9012,"timing/step":185.2818,"trainer/epoch":0}
+{"step":26,"async/staleness_mean":5.1406,"generate/avg_num_tokens":3091.373,"generate/avg_tokens_non_zero_rewards":2982.3311,"generate/avg_tokens_zero_rewards":3979.2857,"generate/max_num_tokens":14234,"generate/std_num_tokens":1712.4445,"loss/avg_final_rewards":0.8906,"loss/avg_raw_advantages":-0.0212,"loss/avg_raw_advantages_abs":0.2161,"policy/policy_entropy":0.1093,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.037,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.8906,"timing/step":209.3612,"trainer/epoch":0}
+{"step":27,"async/staleness_mean":4.3281,"generate/avg_num_tokens":2723.832,"generate/avg_tokens_non_zero_rewards":2614.4815,"generate/avg_tokens_zero_rewards":3670.8491,"generate/max_num_tokens":13130,"generate/std_num_tokens":1463.1991,"loss/avg_final_rewards":0.8965,"loss/avg_raw_advantages":-0.021,"loss/avg_raw_advantages_abs":0.1762,"policy/policy_entropy":0.1093,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0255,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.8965,"timing/step":190.8597,"trainer/epoch":0}
+{"step":28,"async/staleness_mean":4.9365,"generate/avg_num_tokens":2846.3234,"generate/avg_tokens_non_zero_rewards":2815.3597,"generate/avg_tokens_zero_rewards":3237.1351,"generate/max_num_tokens":12936,"generate/std_num_tokens":1420.929,"loss/avg_final_rewards":0.9266,"loss/avg_raw_advantages":-0.0085,"loss/avg_raw_advantages_abs":0.105,"policy/policy_entropy":0.1077,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9266,"timing/step":193.0694,"trainer/epoch":0}
+{"step":29,"async/staleness_mean":4.7619,"generate/avg_num_tokens":2635.8472,"generate/avg_tokens_non_zero_rewards":2533.13,"generate/avg_tokens_zero_rewards":3568.52,"generate/max_num_tokens":18514,"generate/std_num_tokens":1375.3518,"loss/avg_final_rewards":0.9008,"loss/avg_raw_advantages":-0.0207,"loss/avg_raw_advantages_abs":0.1426,"policy/policy_entropy":0.1085,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9008,"timing/step":176.5434,"trainer/epoch":0}
+{"step":30,"async/staleness_mean":5.254,"generate/avg_num_tokens":2849.5853,"generate/avg_tokens_non_zero_rewards":2753.7478,"generate/avg_tokens_zero_rewards":3760.0417,"generate/max_num_tokens":11625,"generate/std_num_tokens":1346.0454,"loss/avg_final_rewards":0.9048,"loss/avg_raw_advantages":-0.0177,"loss/avg_raw_advantages_abs":0.1905,"policy/policy_entropy":0.1064,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9048,"timing/step":192.0073,"trainer/epoch":0}
+{"step":31,"async/staleness_mean":4.8254,"generate/avg_num_tokens":2590.8393,"generate/avg_tokens_non_zero_rewards":2545.0174,"generate/avg_tokens_zero_rewards":3058.2222,"generate/max_num_tokens":10878,"generate/std_num_tokens":1164.3999,"loss/avg_final_rewards":0.9107,"loss/avg_raw_advantages":-0.0107,"loss/avg_raw_advantages_abs":0.141,"policy/policy_entropy":0.1101,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9107,"timing/step":173.492,"trainer/epoch":0}
+{"step":32,"async/staleness_mean":4.8254,"generate/avg_num_tokens":2799.3294,"generate/avg_tokens_non_zero_rewards":2676.5465,"generate/avg_tokens_zero_rewards":3866.5962,"generate/max_num_tokens":11311,"generate/std_num_tokens":1269.2805,"loss/avg_final_rewards":0.8968,"loss/avg_raw_advantages":-0.0271,"loss/avg_raw_advantages_abs":0.2102,"policy/policy_entropy":0.1086,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.8968,"timing/step":198.3175,"trainer/epoch":0}
+{"step":33,"async/staleness_mean":4.2344,"generate/avg_num_tokens":2602.0664,"generate/avg_tokens_non_zero_rewards":2530.3838,"generate/avg_tokens_zero_rewards":3753.7667,"generate/max_num_tokens":11238,"generate/std_num_tokens":1306.177,"loss/avg_final_rewards":0.9414,"loss/avg_raw_advantages":-0.0158,"loss/avg_raw_advantages_abs":0.1187,"policy/policy_entropy":0.1119,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0685,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9414,"timing/step":177.8609,"trainer/epoch":0}
+{"step":34,"async/staleness_mean":4.2698,"generate/avg_num_tokens":2713.0813,"generate/avg_tokens_non_zero_rewards":2644.9394,"generate/avg_tokens_zero_rewards":3462.6429,"generate/max_num_tokens":11070,"generate/std_num_tokens":1278.3998,"loss/avg_final_rewards":0.9167,"loss/avg_raw_advantages":-0.0117,"loss/avg_raw_advantages_abs":0.1655,"policy/policy_entropy":0.1095,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9167,"timing/step":187.63,"trainer/epoch":0}
+{"step":35,"async/staleness_mean":5.0161,"generate/avg_num_tokens":2792.2036,"generate/avg_tokens_non_zero_rewards":2721.2575,"generate/avg_tokens_zero_rewards":3894.2333,"generate/max_num_tokens":15150,"generate/std_num_tokens":1595.4949,"loss/avg_final_rewards":0.9395,"loss/avg_raw_advantages":-0.0037,"loss/avg_raw_advantages_abs":0.1416,"policy/policy_entropy":0.111,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9395,"timing/step":192.0157,"trainer/epoch":0}
+{"step":36,"async/staleness_mean":4.5469,"generate/avg_num_tokens":2547.2129,"generate/avg_tokens_non_zero_rewards":2503.2521,"generate/avg_tokens_zero_rewards":3206.625,"generate/max_num_tokens":17277,"generate/std_num_tokens":1415.5193,"loss/avg_final_rewards":0.9375,"loss/avg_raw_advantages":-0.01,"loss/avg_raw_advantages_abs":0.1143,"policy/policy_entropy":0.1088,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.043,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9375,"timing/step":190.8551,"trainer/epoch":0}
+{"step":37,"async/staleness_mean":5.6333,"generate/avg_num_tokens":2878.9,"generate/avg_tokens_non_zero_rewards":2834.3428,"generate/avg_tokens_zero_rewards":3806.5,"generate/max_num_tokens":14504,"generate/std_num_tokens":1430.6157,"loss/avg_final_rewards":0.9542,"loss/avg_raw_advantages":-0.0079,"loss/avg_raw_advantages_abs":0.0894,"policy/policy_entropy":0.107,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9542,"timing/step":179.0071,"trainer/epoch":0}
+{"step":38,"async/staleness_mean":5.2131,"generate/avg_num_tokens":2764.8115,"generate/avg_tokens_non_zero_rewards":2712.4091,"generate/avg_tokens_zero_rewards":3245.1667,"generate/max_num_tokens":9187,"generate/std_num_tokens":1240.2673,"loss/avg_final_rewards":0.9016,"loss/avg_raw_advantages":-0.0064,"loss/avg_raw_advantages_abs":0.1739,"policy/policy_entropy":0.11,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9016,"timing/step":174.8333,"trainer/epoch":0}
+{"step":39,"async/staleness_mean":5.1875,"generate/avg_num_tokens":2784.2344,"generate/avg_tokens_non_zero_rewards":2724.0524,"generate/avg_tokens_zero_rewards":3604.4286,"generate/max_num_tokens":11538,"generate/std_num_tokens":1391.8281,"loss/avg_final_rewards":0.9316,"loss/avg_raw_advantages":-0.0165,"loss/avg_raw_advantages_abs":0.1368,"policy/policy_entropy":0.1105,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0402,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9316,"timing/step":193.34,"trainer/epoch":0}
+{"step":40,"async/staleness_mean":5.4603,"generate/avg_num_tokens":2781.9107,"generate/avg_tokens_non_zero_rewards":2664.9104,"generate/avg_tokens_zero_rewards":4349.7143,"generate/max_num_tokens":23281,"generate/std_num_tokens":1790.8724,"loss/avg_final_rewards":0.9306,"loss/avg_raw_advantages":-0.0269,"loss/avg_raw_advantages_abs":0.1582,"policy/policy_entropy":0.1096,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9306,"timing/step":190.5524,"trainer/epoch":0}
+{"step":41,"async/staleness_mean":4.8906,"generate/avg_num_tokens":2870.9629,"generate/avg_tokens_non_zero_rewards":2837.7256,"generate/avg_tokens_zero_rewards":3688.6,"generate/max_num_tokens":19338,"generate/std_num_tokens":1423.2774,"loss/avg_final_rewards":0.9609,"loss/avg_raw_advantages":-0.0051,"loss/avg_raw_advantages_abs":0.0823,"policy/policy_entropy":0.1056,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0329,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9609,"timing/step":189.0223,"trainer/epoch":0}
+{"step":42,"async/staleness_mean":5.3492,"generate/avg_num_tokens":2922.2956,"generate/avg_tokens_non_zero_rewards":2855.1073,"generate/avg_tokens_zero_rewards":3746.2368,"generate/max_num_tokens":12895,"generate/std_num_tokens":1337.5946,"loss/avg_final_rewards":0.9246,"loss/avg_raw_advantages":-0.0155,"loss/avg_raw_advantages_abs":0.1321,"policy/policy_entropy":0.1098,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9246,"timing/step":248.2852,"trainer/epoch":0}
+{"step":43,"async/staleness_mean":4.8548,"generate/avg_num_tokens":2856.1794,"generate/avg_tokens_non_zero_rewards":2756.965,"generate/avg_tokens_zero_rewards":4018.7692,"generate/max_num_tokens":10806,"generate/std_num_tokens":1469.6077,"loss/avg_final_rewards":0.9214,"loss/avg_raw_advantages":-0.0233,"loss/avg_raw_advantages_abs":0.1463,"policy/policy_entropy":0.1079,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9214,"timing/step":209.3624,"trainer/epoch":0}
+{"step":44,"async/staleness_mean":4.0476,"generate/avg_num_tokens":2714.9643,"generate/avg_tokens_non_zero_rewards":2676.3822,"generate/avg_tokens_zero_rewards":3648.65,"generate/max_num_tokens":13880,"generate/std_num_tokens":1304.622,"loss/avg_final_rewards":0.9603,"loss/avg_raw_advantages":-0.0094,"loss/avg_raw_advantages_abs":0.086,"policy/policy_entropy":0.1103,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9603,"timing/step":271.8882,"trainer/epoch":0}
+{"step":45,"async/staleness_mean":4.381,"generate/avg_num_tokens":2944.752,"generate/avg_tokens_non_zero_rewards":2866.9834,"generate/avg_tokens_zero_rewards":4648.5909,"generate/max_num_tokens":16462,"generate/std_num_tokens":1309.1103,"loss/avg_final_rewards":0.9563,"loss/avg_raw_advantages":-0.0171,"loss/avg_raw_advantages_abs":0.1029,"policy/policy_entropy":0.1059,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9563,"timing/step":280.8006,"trainer/epoch":0}
+{"step":46,"async/staleness_mean":4.5484,"generate/avg_num_tokens":2756.6694,"generate/avg_tokens_non_zero_rewards":2692.0579,"generate/avg_tokens_zero_rewards":3760.3,"generate/max_num_tokens":12071,"generate/std_num_tokens":1244.5635,"loss/avg_final_rewards":0.9395,"loss/avg_raw_advantages":-0.0117,"loss/avg_raw_advantages_abs":0.1218,"policy/policy_entropy":0.1081,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9395,"timing/step":222.0341,"trainer/epoch":0}
+{"step":47,"async/staleness_mean":4.3651,"generate/avg_num_tokens":2945.1944,"generate/avg_tokens_non_zero_rewards":2893.2955,"generate/avg_tokens_zero_rewards":3600.2432,"generate/max_num_tokens":12644,"generate/std_num_tokens":1448.864,"loss/avg_final_rewards":0.9266,"loss/avg_raw_advantages":-0.0111,"loss/avg_raw_advantages_abs":0.0915,"policy/policy_entropy":0.1063,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":0.9841,"reward/avg_raw_reward":0.9266,"timing/step":283.4806,"trainer/epoch":0}
+{"step":48,"async/staleness_mean":4.8594,"generate/avg_num_tokens":3077.9453,"generate/avg_tokens_non_zero_rewards":3027.75,"generate/avg_tokens_zero_rewards":3830.875,"generate/max_num_tokens":14566,"generate/std_num_tokens":1617.056,"loss/avg_final_rewards":0.9375,"loss/avg_raw_advantages":-0.0134,"loss/avg_raw_advantages_abs":0.1268,"policy/policy_entropy":0.1068,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0641,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9375,"timing/step":354.9404,"trainer/epoch":0}
+{"step":49,"async/staleness_mean":3.7188,"generate/avg_num_tokens":2821.5469,"generate/avg_tokens_non_zero_rewards":2779.9917,"generate/avg_tokens_zero_rewards":3489.2,"generate/max_num_tokens":24244,"generate/std_num_tokens":1617.4158,"loss/avg_final_rewards":0.9414,"loss/avg_raw_advantages":-0.0114,"loss/avg_raw_advantages_abs":0.1108,"policy/policy_entropy":0.1077,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0233,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9414,"timing/step":197.8306,"trainer/epoch":0}
+{"step":50,"async/staleness_mean":4.4286,"generate/avg_num_tokens":3010.8155,"generate/avg_tokens_non_zero_rewards":2991.4758,"generate/avg_tokens_zero_rewards":3327.5862,"generate/max_num_tokens":18278,"generate/std_num_tokens":1543.572,"loss/avg_final_rewards":0.9425,"loss/avg_raw_advantages":-0.0056,"loss/avg_raw_advantages_abs":0.1075,"policy/policy_entropy":0.1065,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9425,"timing/step":256.6199,"trainer/epoch":0}
+{"step":51,"async/staleness_mean":5.4754,"generate/avg_num_tokens":3193.7377,"generate/avg_tokens_non_zero_rewards":3157.676,"generate/avg_tokens_zero_rewards":3861.6,"generate/max_num_tokens":9998,"generate/std_num_tokens":1336.5231,"loss/avg_final_rewards":0.9488,"loss/avg_raw_advantages":-0.0148,"loss/avg_raw_advantages_abs":0.0983,"policy/policy_entropy":0.1143,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9488,"timing/step":469.5261,"trainer/epoch":0}
+{"step":52,"async/staleness_mean":7.3492,"generate/avg_num_tokens":3425.1369,"generate/avg_tokens_non_zero_rewards":3476.6468,"generate/avg_tokens_zero_rewards":2967.6078,"generate/max_num_tokens":23285,"generate/std_num_tokens":1951.9608,"loss/avg_final_rewards":0.8988,"loss/avg_raw_advantages":-0.0139,"loss/avg_raw_advantages_abs":0.1411,"policy/policy_entropy":0.1084,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.8988,"timing/step":1611.2371,"trainer/epoch":0}
+{"step":53,"async/staleness_mean":0.0,"generate/avg_num_tokens":2381.2148,"generate/avg_tokens_non_zero_rewards":2366.4783,"generate/avg_tokens_zero_rewards":3624.0,"generate/max_num_tokens":6838,"generate/std_num_tokens":947.8529,"loss/avg_final_rewards":0.9883,"loss/avg_raw_advantages":-0.0053,"loss/avg_raw_advantages_abs":0.0295,"policy/policy_entropy":0.1075,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0519,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9883,"timing/step":1581.4156,"trainer/epoch":1}
+{"step":54,"async/staleness_mean":0.875,"generate/avg_num_tokens":2330.7402,"generate/avg_tokens_non_zero_rewards":2324.7067,"generate/avg_tokens_zero_rewards":3097.0,"generate/max_num_tokens":9577,"generate/std_num_tokens":969.3405,"loss/avg_final_rewards":0.9922,"loss/avg_raw_advantages":-0.0016,"loss/avg_raw_advantages_abs":0.0191,"policy/policy_entropy":0.1062,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.009,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9922,"timing/step":729.4275,"trainer/epoch":1}
+{"step":55,"async/staleness_mean":1.6562,"generate/avg_num_tokens":2704.9648,"generate/avg_tokens_non_zero_rewards":2697.7662,"generate/avg_tokens_zero_rewards":3926.3333,"generate/max_num_tokens":12609,"generate/std_num_tokens":1248.1393,"loss/avg_final_rewards":0.9941,"loss/avg_raw_advantages":-0.0017,"loss/avg_raw_advantages_abs":0.0153,"policy/policy_entropy":0.1068,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.009,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9941,"timing/step":684.5591,"trainer/epoch":1}
+{"step":56,"async/staleness_mean":1.9844,"generate/avg_num_tokens":2774.0977,"generate/avg_tokens_non_zero_rewards":2748.4112,"generate/avg_tokens_zero_rewards":3944.0,"generate/max_num_tokens":11344,"generate/std_num_tokens":1423.8899,"loss/avg_final_rewards":0.9785,"loss/avg_raw_advantages":-0.0058,"loss/avg_raw_advantages_abs":0.0413,"policy/policy_entropy":0.1071,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0109,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9785,"timing/step":839.3294,"trainer/epoch":1}
+{"step":57,"async/staleness_mean":2.5938,"generate/avg_num_tokens":2707.8516,"generate/avg_tokens_non_zero_rewards":2783.6818,"generate/avg_tokens_zero_rewards":1397.0714,"generate/max_num_tokens":11134,"generate/std_num_tokens":1335.8597,"loss/avg_final_rewards":0.9453,"loss/avg_raw_advantages":-0.0053,"loss/avg_raw_advantages_abs":0.0501,"policy/policy_entropy":0.1042,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0173,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9453,"timing/step":861.4917,"trainer/epoch":1}
+{"step":58,"async/staleness_mean":3.25,"generate/avg_num_tokens":2563.6445,"generate/avg_tokens_non_zero_rewards":2843.4382,"generate/avg_tokens_zero_rewards":705.3134,"generate/max_num_tokens":9047,"generate/std_num_tokens":1457.9575,"loss/avg_final_rewards":0.8691,"loss/avg_raw_advantages":0.0028,"loss/avg_raw_advantages_abs":0.0628,"policy/policy_entropy":0.0988,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0205,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.8691,"timing/step":639.2276,"trainer/epoch":1}
+{"step":59,"async/staleness_mean":3.1719,"generate/avg_num_tokens":2639.1445,"generate/avg_tokens_non_zero_rewards":2656.652,"generate/avg_tokens_zero_rewards":2400.5429,"generate/max_num_tokens":30113,"generate/std_num_tokens":2147.6442,"loss/avg_final_rewards":0.9316,"loss/avg_raw_advantages":-0.0399,"loss/avg_raw_advantages_abs":0.0778,"policy/policy_entropy":0.1018,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0169,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9316,"timing/step":624.4459,"trainer/epoch":1}
+{"step":60,"async/staleness_mean":3.3281,"generate/avg_num_tokens":2546.6992,"generate/avg_tokens_non_zero_rewards":2721.448,"generate/avg_tokens_zero_rewards":539.2195,"generate/max_num_tokens":8830,"generate/std_num_tokens":1398.6611,"loss/avg_final_rewards":0.9199,"loss/avg_raw_advantages":0.0061,"loss/avg_raw_advantages_abs":0.0363,"policy/policy_entropy":0.1022,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0217,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9199,"timing/step":651.3771,"trainer/epoch":1}
+{"step":61,"async/staleness_mean":3.625,"generate/avg_num_tokens":2845.9648,"generate/avg_tokens_non_zero_rewards":2992.8376,"generate/avg_tokens_zero_rewards":1283.7727,"generate/max_num_tokens":22700,"generate/std_num_tokens":1768.4694,"loss/avg_final_rewards":0.9141,"loss/avg_raw_advantages":-0.0046,"loss/avg_raw_advantages_abs":0.0427,"policy/policy_entropy":0.1015,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0149,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9141,"timing/step":776.8216,"trainer/epoch":1}
+{"step":62,"async/staleness_mean":3.1875,"generate/avg_num_tokens":2664.6875,"generate/avg_tokens_non_zero_rewards":2777.6955,"generate/avg_tokens_zero_rewards":552.3077,"generate/max_num_tokens":12053,"generate/std_num_tokens":1443.9594,"loss/avg_final_rewards":0.9492,"loss/avg_raw_advantages":0.0033,"loss/avg_raw_advantages_abs":0.0243,"policy/policy_entropy":0.1054,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0148,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9492,"timing/step":839.203,"trainer/epoch":1}
+{"step":63,"async/staleness_mean":2.8438,"generate/avg_num_tokens":2621.5898,"generate/avg_tokens_non_zero_rewards":2721.7835,"generate/avg_tokens_zero_rewards":821.8148,"generate/max_num_tokens":9347,"generate/std_num_tokens":1257.4092,"loss/avg_final_rewards":0.9473,"loss/avg_raw_advantages":0.0016,"loss/avg_raw_advantages_abs":0.0318,"policy/policy_entropy":0.1049,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0243,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9473,"timing/step":463.7961,"trainer/epoch":1}
+{"step":64,"async/staleness_mean":3.3906,"generate/avg_num_tokens":2631.5742,"generate/avg_tokens_non_zero_rewards":2770.8408,"generate/avg_tokens_zero_rewards":1031.7073,"generate/max_num_tokens":10004,"generate/std_num_tokens":1444.8114,"loss/avg_final_rewards":0.9199,"loss/avg_raw_advantages":-0.0043,"loss/avg_raw_advantages_abs":0.0509,"policy/policy_entropy":0.1037,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0119,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9199,"timing/step":677.503,"trainer/epoch":1}
+{"step":65,"async/staleness_mean":3.9531,"generate/avg_num_tokens":2877.4531,"generate/avg_tokens_non_zero_rewards":3008.7457,"generate/avg_tokens_zero_rewards":1608.2917,"generate/max_num_tokens":14795,"generate/std_num_tokens":1673.452,"loss/avg_final_rewards":0.9062,"loss/avg_raw_advantages":-0.0005,"loss/avg_raw_advantages_abs":0.0823,"policy/policy_entropy":0.1057,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0218,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9062,"timing/step":590.4898,"trainer/epoch":1}
+{"step":66,"async/staleness_mean":3.8438,"generate/avg_num_tokens":2806.2852,"generate/avg_tokens_non_zero_rewards":2922.8112,"generate/avg_tokens_zero_rewards":934.1,"generate/max_num_tokens":12837,"generate/std_num_tokens":1487.4971,"loss/avg_final_rewards":0.9414,"loss/avg_raw_advantages":0.0006,"loss/avg_raw_advantages_abs":0.0396,"policy/policy_entropy":0.1053,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0126,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9414,"timing/step":559.1384,"trainer/epoch":1}
+{"step":67,"async/staleness_mean":3.875,"generate/avg_num_tokens":2812.5215,"generate/avg_tokens_non_zero_rewards":2960.1459,"generate/avg_tokens_zero_rewards":1022.1026,"generate/max_num_tokens":11669,"generate/std_num_tokens":1560.2387,"loss/avg_final_rewards":0.9238,"loss/avg_raw_advantages":0.0003,"loss/avg_raw_advantages_abs":0.0256,"policy/policy_entropy":0.1005,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0106,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.9238,"timing/step":765.5756,"trainer/epoch":1}
+{"step":68,"async/staleness_mean":3.375,"generate/avg_num_tokens":2612.9922,"generate/avg_tokens_non_zero_rewards":2763.431,"generate/avg_tokens_zero_rewards":498.0,"generate/max_num_tokens":9393,"generate/std_num_tokens":1315.2754,"loss/avg_final_rewards":0.9336,"loss/avg_raw_advantages":-0.002,"loss/avg_raw_advantages_abs":0.0208,"policy/policy_entropy":0.1015,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.009,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9336,"timing/step":751.558,"trainer/epoch":1}
+{"step":69,"async/staleness_mean":3.7031,"generate/avg_num_tokens":2728.7969,"generate/avg_tokens_non_zero_rewards":2841.2257,"generate/avg_tokens_zero_rewards":856.2759,"generate/max_num_tokens":10693,"generate/std_num_tokens":1381.2743,"loss/avg_final_rewards":0.9434,"loss/avg_raw_advantages":-0.0041,"loss/avg_raw_advantages_abs":0.0314,"policy/policy_entropy":0.1077,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.008,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9434,"timing/step":596.0836,"trainer/epoch":1}
+{"step":70,"async/staleness_mean":3.5156,"generate/avg_num_tokens":2916.2012,"generate/avg_tokens_non_zero_rewards":3104.4258,"generate/avg_tokens_zero_rewards":695.15,"generate/max_num_tokens":14016,"generate/std_num_tokens":1841.5559,"loss/avg_final_rewards":0.9219,"loss/avg_raw_advantages":-0.0024,"loss/avg_raw_advantages_abs":0.0277,"policy/policy_entropy":0.1043,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0112,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9219,"timing/step":571.6014,"trainer/epoch":1}
+{"step":71,"async/staleness_mean":3.3906,"generate/avg_num_tokens":2742.6836,"generate/avg_tokens_non_zero_rewards":2872.2635,"generate/avg_tokens_zero_rewards":660.7667,"generate/max_num_tokens":27804,"generate/std_num_tokens":1741.732,"loss/avg_final_rewards":0.9414,"loss/avg_raw_advantages":0.0053,"loss/avg_raw_advantages_abs":0.0322,"policy/policy_entropy":0.106,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0291,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9414,"timing/step":701.936,"trainer/epoch":1}
+{"step":72,"async/staleness_mean":4.25,"generate/avg_num_tokens":2518.4102,"generate/avg_tokens_non_zero_rewards":2719.7987,"generate/avg_tokens_zero_rewards":657.58,"generate/max_num_tokens":10577,"generate/std_num_tokens":1418.1654,"loss/avg_final_rewards":0.9023,"loss/avg_raw_advantages":0.0003,"loss/avg_raw_advantages_abs":0.0131,"policy/policy_entropy":0.0997,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0069,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.9023,"timing/step":518.1089,"trainer/epoch":1}
+{"step":73,"async/staleness_mean":4.2344,"generate/avg_num_tokens":2859.4844,"generate/avg_tokens_non_zero_rewards":3043.2833,"generate/avg_tokens_zero_rewards":630.3333,"generate/max_num_tokens":12331,"generate/std_num_tokens":1680.893,"loss/avg_final_rewards":0.9238,"loss/avg_raw_advantages":0.0026,"loss/avg_raw_advantages_abs":0.0294,"policy/policy_entropy":0.1025,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0148,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9238,"timing/step":589.3744,"trainer/epoch":1}
+{"step":74,"async/staleness_mean":3.8594,"generate/avg_num_tokens":2868.1055,"generate/avg_tokens_non_zero_rewards":2998.4584,"generate/avg_tokens_zero_rewards":1446.3488,"generate/max_num_tokens":12107,"generate/std_num_tokens":1580.0912,"loss/avg_final_rewards":0.916,"loss/avg_raw_advantages":0.0017,"loss/avg_raw_advantages_abs":0.038,"policy/policy_entropy":0.1019,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0126,"reward/avg_pass_at_8":0.9844,"reward/avg_raw_reward":0.916,"timing/step":698.1936,"trainer/epoch":1}
+{"step":75,"async/staleness_mean":1.375,"generate/avg_num_tokens":2515.0352,"generate/avg_tokens_non_zero_rewards":2533.1736,"generate/avg_tokens_zero_rewards":675.8,"generate/max_num_tokens":10097,"generate/std_num_tokens":1195.7212,"loss/avg_final_rewards":0.9902,"loss/avg_raw_advantages":-0.001,"loss/avg_raw_advantages_abs":0.0042,"policy/policy_entropy":0.1075,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0057,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9902,"timing/step":1309.7496,"trainer/epoch":1}
+{"step":76,"async/staleness_mean":0.8281,"generate/avg_num_tokens":2758.6719,"generate/avg_tokens_non_zero_rewards":2760.4344,"generate/avg_tokens_zero_rewards":1858.0,"generate/max_num_tokens":13970,"generate/std_num_tokens":1186.4283,"loss/avg_final_rewards":0.998,"loss/avg_raw_advantages":0.0004,"loss/avg_raw_advantages_abs":0.0031,"policy/policy_entropy":0.1076,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0047,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.998,"timing/step":731.729,"trainer/epoch":1}
+{"step":77,"async/staleness_mean":1.5312,"generate/avg_num_tokens":2914.1328,"generate/avg_tokens_non_zero_rewards":2911.6673,"generate/avg_tokens_zero_rewards":3092.0,"generate/max_num_tokens":13460,"generate/std_num_tokens":1433.5917,"loss/avg_final_rewards":0.9863,"loss/avg_raw_advantages":-0.0004,"loss/avg_raw_advantages_abs":0.0245,"policy/policy_entropy":0.1113,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0112,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9863,"timing/step":622.7508,"trainer/epoch":1}
+{"step":78,"async/staleness_mean":1.7969,"generate/avg_num_tokens":2782.416,"generate/avg_tokens_non_zero_rewards":2771.4862,"generate/avg_tokens_zero_rewards":4170.5,"generate/max_num_tokens":12210,"generate/std_num_tokens":1349.6586,"loss/avg_final_rewards":0.9922,"loss/avg_raw_advantages":-0.0051,"loss/avg_raw_advantages_abs":0.0183,"policy/policy_entropy":0.1109,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0085,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9922,"timing/step":560.5876,"trainer/epoch":1}
+{"step":79,"async/staleness_mean":2.25,"generate/avg_num_tokens":2749.4746,"generate/avg_tokens_non_zero_rewards":2746.5992,"generate/avg_tokens_zero_rewards":3237.3333,"generate/max_num_tokens":8954,"generate/std_num_tokens":1306.7353,"loss/avg_final_rewards":0.9941,"loss/avg_raw_advantages":0.0002,"loss/avg_raw_advantages_abs":0.014,"policy/policy_entropy":0.114,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0068,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9941,"timing/step":461.4439,"trainer/epoch":1}
+{"step":80,"async/staleness_mean":2.7969,"generate/avg_num_tokens":2860.7715,"generate/avg_tokens_non_zero_rewards":2862.1807,"generate/avg_tokens_zero_rewards":2810.6429,"generate/max_num_tokens":9394,"generate/std_num_tokens":1326.79,"loss/avg_final_rewards":0.9727,"loss/avg_raw_advantages":0.0006,"loss/avg_raw_advantages_abs":0.0289,"policy/policy_entropy":0.1129,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0119,"reward/avg_pass_at_8":1.0,"reward/avg_raw_reward":0.9727,"timing/step":397.2111,"trainer/epoch":1}

viewer/build/inputs/marin/runs/marin-a3-nemotron-agent-calendar/run.json

@@ -0,0 +1,27 @@
+{
+ "id": "marin-a3-nemotron-agent-calendar",
+ "title": "A3 RLOO on nemotron-gym-agent-calendar (Qwen3-8B agent)",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://huggingface.co/laion/a3-rl-laion_nemotron-gym-agent-calendar-80-8B/tree/main/training_logs",
+ "license": "apache-2.0",
+ "model": "laion/a3-rl-laion_nemotron-gym-agent-calendar-80-8B",
+ "base_model": "laion/GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink",
+ "method": "RLOO-N (SkyRL, binary verifier reward)",
+ "dataset": "laion/nemotron-gym-agent-calendar",
+ "eval_suite": null,
+ "kind": "training",
+ "state": "finished",
+ "started_at": null,
+ "updated_at": "2026-09-04T16:21:32Z",
+ "attempts": 0,
+ "note": "Marin A3 sweep (issue #6187): one RLOO-N run per training dataset from the same Qwen3-8B-derived SFT agent, here laion/nemotron-gym-agent-calendar; reward 0.973 and pass@8 1.0 at step 80, per the issue. Our copy has every logged step of the final lineage (3 resumed job segments, 2 superseded rows dropped). The issue concluded this binary-reward setup was uninformative about dataset utility.",
+ "metrics_map": {
+  "reward": "reward/avg_raw_reward",
+  "loss": "policy/policy_loss",
+  "entropy": "policy/policy_entropy",
+  "grad_norm": "policy/raw_grad_norm",
+  "response_length": "generate/avg_num_tokens"
+ }
+}

viewer/build/inputs/marin/runs/marin-a3-nl2bash/metrics.jsonl

@@ -0,0 +1,48 @@
+{"step":1,"async/staleness_mean":0.0,"generate/avg_num_tokens":3352.2402,"generate/avg_tokens_non_zero_rewards":2985.2197,"generate/avg_tokens_zero_rewards":3479.7316,"generate/max_num_tokens":18150,"generate/std_num_tokens":1982.5026,"loss/avg_final_rewards":0.2578,"loss/avg_raw_advantages":-0.0167,"loss/avg_raw_advantages_abs":0.1818,"policy/policy_entropy":0.1654,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0319,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.2578,"timing/step":1216.34,"trainer/epoch":0}
+{"step":2,"async/staleness_mean":0.8281,"generate/avg_num_tokens":3726.5098,"generate/avg_tokens_non_zero_rewards":3381.7315,"generate/avg_tokens_zero_rewards":3868.0303,"generate/max_num_tokens":21220,"generate/std_num_tokens":2508.9219,"loss/avg_final_rewards":0.291,"loss/avg_raw_advantages":-0.0166,"loss/avg_raw_advantages_abs":0.2525,"policy/policy_entropy":0.1678,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0338,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.291,"timing/step":661.7763,"trainer/epoch":0}
+{"step":3,"async/staleness_mean":1.1875,"generate/avg_num_tokens":3550.0,"generate/avg_tokens_non_zero_rewards":2737.7194,"generate/avg_tokens_zero_rewards":3852.6997,"generate/max_num_tokens":21754,"generate/std_num_tokens":2318.228,"loss/avg_final_rewards":0.2715,"loss/avg_raw_advantages":-0.0207,"loss/avg_raw_advantages_abs":0.211,"policy/policy_entropy":0.163,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0386,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.2715,"timing/step":707.2637,"trainer/epoch":0}
+{"step":4,"async/staleness_mean":1.5312,"generate/avg_num_tokens":3137.4629,"generate/avg_tokens_non_zero_rewards":2889.3333,"generate/avg_tokens_zero_rewards":3268.5642,"generate/max_num_tokens":12387,"generate/std_num_tokens":1813.1676,"loss/avg_final_rewards":0.3457,"loss/avg_raw_advantages":0.0105,"loss/avg_raw_advantages_abs":0.2118,"policy/policy_entropy":0.1633,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0289,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.3457,"timing/step":672.5336,"trainer/epoch":0}
+{"step":5,"async/staleness_mean":1.5625,"generate/avg_num_tokens":3214.2305,"generate/avg_tokens_non_zero_rewards":2715.2885,"generate/avg_tokens_zero_rewards":3432.868,"generate/max_num_tokens":23624,"generate/std_num_tokens":2069.6596,"loss/avg_final_rewards":0.3047,"loss/avg_raw_advantages":-0.0074,"loss/avg_raw_advantages_abs":0.1808,"policy/policy_entropy":0.1649,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0324,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.3047,"timing/step":912.8212,"trainer/epoch":0}
+{"step":6,"async/staleness_mean":2.0469,"generate/avg_num_tokens":3736.7402,"generate/avg_tokens_non_zero_rewards":3083.0563,"generate/avg_tokens_zero_rewards":4033.8693,"generate/max_num_tokens":29460,"generate/std_num_tokens":2935.8412,"loss/avg_final_rewards":0.3125,"loss/avg_raw_advantages":0.0021,"loss/avg_raw_advantages_abs":0.172,"policy/policy_entropy":0.1657,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0301,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.3125,"timing/step":731.2619,"trainer/epoch":0}
+{"step":7,"async/staleness_mean":2.1094,"generate/avg_num_tokens":3666.4707,"generate/avg_tokens_non_zero_rewards":3125.0391,"generate/avg_tokens_zero_rewards":3846.9479,"generate/max_num_tokens":31791,"generate/std_num_tokens":2969.1608,"loss/avg_final_rewards":0.25,"loss/avg_raw_advantages":-0.0123,"loss/avg_raw_advantages_abs":0.2135,"policy/policy_entropy":0.167,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0392,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.25,"timing/step":842.0906,"trainer/epoch":0}
+{"step":8,"async/staleness_mean":1.7969,"generate/avg_num_tokens":3067.0254,"generate/avg_tokens_non_zero_rewards":3047.8589,"generate/avg_tokens_zero_rewards":3075.9771,"generate/max_num_tokens":25018,"generate/std_num_tokens":2106.2528,"loss/avg_final_rewards":0.3184,"loss/avg_raw_advantages":-0.0036,"loss/avg_raw_advantages_abs":0.2085,"policy/policy_entropy":0.1645,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0326,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.3184,"timing/step":967.5325,"trainer/epoch":0}
+{"step":9,"async/staleness_mean":2.8125,"generate/avg_num_tokens":3860.4453,"generate/avg_tokens_non_zero_rewards":3139.155,"generate/avg_tokens_zero_rewards":4103.3864,"generate/max_num_tokens":29565,"generate/std_num_tokens":3376.5289,"loss/avg_final_rewards":0.252,"loss/avg_raw_advantages":-0.0101,"loss/avg_raw_advantages_abs":0.1668,"policy/policy_entropy":0.1688,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0262,"reward/avg_pass_at_8":0.4688,"reward/avg_raw_reward":0.252,"timing/step":696.2709,"trainer/epoch":0}
+{"step":10,"async/staleness_mean":2.9531,"generate/avg_num_tokens":3999.9629,"generate/avg_tokens_non_zero_rewards":3180.4966,"generate/avg_tokens_zero_rewards":4336.3278,"generate/max_num_tokens":18644,"generate/std_num_tokens":2863.4872,"loss/avg_final_rewards":0.291,"loss/avg_raw_advantages":-0.0047,"loss/avg_raw_advantages_abs":0.1412,"policy/policy_entropy":0.1711,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0273,"reward/avg_pass_at_8":0.5156,"reward/avg_raw_reward":0.291,"timing/step":621.677,"trainer/epoch":0}
+{"step":11,"async/staleness_mean":3.0312,"generate/avg_num_tokens":3999.1641,"generate/avg_tokens_non_zero_rewards":2960.3374,"generate/avg_tokens_zero_rewards":4484.3467,"generate/max_num_tokens":31797,"generate/std_num_tokens":3286.2161,"loss/avg_final_rewards":0.3184,"loss/avg_raw_advantages":-0.0152,"loss/avg_raw_advantages_abs":0.1742,"policy/policy_entropy":0.1695,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0279,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.3184,"timing/step":664.4641,"trainer/epoch":0}
+{"step":12,"async/staleness_mean":2.4375,"generate/avg_num_tokens":3714.834,"generate/avg_tokens_non_zero_rewards":2952.5714,"generate/avg_tokens_zero_rewards":4160.8638,"generate/max_num_tokens":21755,"generate/std_num_tokens":2515.9931,"loss/avg_final_rewards":0.3691,"loss/avg_raw_advantages":-0.0163,"loss/avg_raw_advantages_abs":0.1796,"policy/policy_entropy":0.1702,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0272,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.3691,"timing/step":821.7749,"trainer/epoch":0}
+{"step":13,"async/staleness_mean":1.9531,"generate/avg_num_tokens":3352.8047,"generate/avg_tokens_non_zero_rewards":2851.5652,"generate/avg_tokens_zero_rewards":3633.9878,"generate/max_num_tokens":31802,"generate/std_num_tokens":2432.2051,"loss/avg_final_rewards":0.3594,"loss/avg_raw_advantages":-0.0078,"loss/avg_raw_advantages_abs":0.177,"policy/policy_entropy":0.1637,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0295,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.3594,"timing/step":679.5294,"trainer/epoch":0}
+{"step":14,"async/staleness_mean":2.2188,"generate/avg_num_tokens":3381.3848,"generate/avg_tokens_non_zero_rewards":2674.2604,"generate/avg_tokens_zero_rewards":3729.793,"generate/max_num_tokens":31807,"generate/std_num_tokens":3159.6383,"loss/avg_final_rewards":0.3301,"loss/avg_raw_advantages":-0.0037,"loss/avg_raw_advantages_abs":0.1685,"policy/policy_entropy":0.1646,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0283,"reward/avg_pass_at_8":0.5469,"reward/avg_raw_reward":0.3301,"timing/step":1026.1547,"trainer/epoch":0}
+{"step":15,"async/staleness_mean":2.4531,"generate/avg_num_tokens":4004.1074,"generate/avg_tokens_non_zero_rewards":2946.5819,"generate/avg_tokens_zero_rewards":4562.8597,"generate/max_num_tokens":31811,"generate/std_num_tokens":3190.4199,"loss/avg_final_rewards":0.3457,"loss/avg_raw_advantages":-0.0139,"loss/avg_raw_advantages_abs":0.1807,"policy/policy_entropy":0.1752,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0313,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.3457,"timing/step":1006.1819,"trainer/epoch":0}
+{"step":16,"async/staleness_mean":2.5625,"generate/avg_num_tokens":3649.4199,"generate/avg_tokens_non_zero_rewards":2577.3217,"generate/avg_tokens_zero_rewards":4064.8943,"generate/max_num_tokens":31857,"generate/std_num_tokens":3637.0503,"loss/avg_final_rewards":0.2793,"loss/avg_raw_advantages":-0.0074,"loss/avg_raw_advantages_abs":0.0982,"policy/policy_entropy":0.1653,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0251,"reward/avg_pass_at_8":0.4062,"reward/avg_raw_reward":0.2793,"timing/step":1278.6512,"trainer/epoch":0}
+{"step":17,"async/staleness_mean":2.5781,"generate/avg_num_tokens":3307.5,"generate/avg_tokens_non_zero_rewards":3224.385,"generate/avg_tokens_zero_rewards":3355.3231,"generate/max_num_tokens":14549,"generate/std_num_tokens":1869.9983,"loss/avg_final_rewards":0.3652,"loss/avg_raw_advantages":-0.0102,"loss/avg_raw_advantages_abs":0.1643,"policy/policy_entropy":0.1709,"policy/policy_loss":-0.0008,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0233,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.3652,"timing/step":1166.6575,"trainer/epoch":0}
+{"step":18,"async/staleness_mean":3.0938,"generate/avg_num_tokens":3185.6543,"generate/avg_tokens_non_zero_rewards":3065.5391,"generate/avg_tokens_zero_rewards":3225.6927,"generate/max_num_tokens":27509,"generate/std_num_tokens":2931.8951,"loss/avg_final_rewards":0.25,"loss/avg_raw_advantages":0.027,"loss/avg_raw_advantages_abs":0.2172,"policy/policy_entropy":0.1516,"policy/policy_loss":-0.0368,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0291,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.25,"timing/step":598.3133,"trainer/epoch":0}
+{"step":19,"async/staleness_mean":3.125,"generate/avg_num_tokens":3407.1797,"generate/avg_tokens_non_zero_rewards":2795.6562,"generate/avg_tokens_zero_rewards":3685.1449,"generate/max_num_tokens":31796,"generate/std_num_tokens":2731.5702,"loss/avg_final_rewards":0.3125,"loss/avg_raw_advantages":0.0033,"loss/avg_raw_advantages_abs":0.1421,"policy/policy_entropy":0.1585,"policy/policy_loss":-0.0223,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0255,"reward/avg_pass_at_8":0.5156,"reward/avg_raw_reward":0.3125,"timing/step":874.4677,"trainer/epoch":0}
+{"step":20,"async/staleness_mean":3.0938,"generate/avg_num_tokens":3191.4785,"generate/avg_tokens_non_zero_rewards":3127.3125,"generate/avg_tokens_zero_rewards":3212.8672,"generate/max_num_tokens":16154,"generate/std_num_tokens":2152.0271,"loss/avg_final_rewards":0.25,"loss/avg_raw_advantages":0.0168,"loss/avg_raw_advantages_abs":0.1292,"policy/policy_entropy":0.1588,"policy/policy_loss":-0.0084,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0369,"reward/avg_pass_at_8":0.4688,"reward/avg_raw_reward":0.25,"timing/step":1144.1321,"trainer/epoch":0}
+{"step":21,"async/staleness_mean":3.2857,"generate/avg_num_tokens":3254.998,"generate/avg_tokens_non_zero_rewards":2910.2583,"generate/avg_tokens_zero_rewards":3402.4646,"generate/max_num_tokens":31800,"generate/std_num_tokens":2612.288,"loss/avg_final_rewards":0.2996,"loss/avg_raw_advantages":0.0123,"loss/avg_raw_advantages_abs":0.1575,"policy/policy_entropy":0.1591,"policy/policy_loss":-0.0164,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":0.5079,"reward/avg_raw_reward":0.2996,"timing/step":1202.3364,"trainer/epoch":0}
+{"step":22,"async/staleness_mean":3.3281,"generate/avg_num_tokens":2967.5527,"generate/avg_tokens_non_zero_rewards":3486.7418,"generate/avg_tokens_zero_rewards":2681.2121,"generate/max_num_tokens":18393,"generate/std_num_tokens":2216.7916,"loss/avg_final_rewards":0.3555,"loss/avg_raw_advantages":0.0428,"loss/avg_raw_advantages_abs":0.2496,"policy/policy_entropy":0.1389,"policy/policy_loss":-0.0343,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.041,"reward/avg_pass_at_8":0.6562,"reward/avg_raw_reward":0.3555,"timing/step":191.685,"trainer/epoch":0}
+{"step":23,"async/staleness_mean":5.1406,"generate/avg_num_tokens":4687.584,"generate/avg_tokens_non_zero_rewards":4047.3274,"generate/avg_tokens_zero_rewards":4868.9098,"generate/max_num_tokens":31856,"generate/std_num_tokens":4223.1637,"loss/avg_final_rewards":0.2207,"loss/avg_raw_advantages":0.0029,"loss/avg_raw_advantages_abs":0.1345,"policy/policy_entropy":0.1661,"policy/policy_loss":-0.0131,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0216,"reward/avg_pass_at_8":0.4375,"reward/avg_raw_reward":0.2207,"timing/step":731.0408,"trainer/epoch":0}
+{"step":24,"async/staleness_mean":5.3651,"generate/avg_num_tokens":8120.4286,"generate/avg_tokens_non_zero_rewards":6356.3905,"generate/avg_tokens_zero_rewards":8584.6491,"generate/max_num_tokens":31822,"generate/std_num_tokens":8224.9844,"loss/avg_final_rewards":0.2083,"loss/avg_raw_advantages":-0.0051,"loss/avg_raw_advantages_abs":0.1786,"policy/policy_entropy":0.1651,"policy/policy_loss":-0.0116,"policy/ppo_clip_ratio":0.0,"reward/avg_pass_at_8":0.4762,"reward/avg_raw_reward":0.2083,"timing/step":4309.9466,"trainer/epoch":0}
+{"step":25,"async/staleness_mean":0.0,"generate/avg_num_tokens":2936.2734,"generate/avg_tokens_non_zero_rewards":2846.2981,"generate/avg_tokens_zero_rewards":2997.8355,"generate/max_num_tokens":22699,"generate/std_num_tokens":2173.5412,"loss/avg_final_rewards":0.4062,"loss/avg_raw_advantages":-0.0247,"loss/avg_raw_advantages_abs":0.2101,"policy/policy_entropy":0.1565,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0363,"reward/avg_pass_at_8":0.625,"reward/avg_raw_reward":0.4062,"timing/step":1335.3915,"trainer/epoch":1}
+{"step":26,"async/staleness_mean":0.7812,"generate/avg_num_tokens":3038.418,"generate/avg_tokens_non_zero_rewards":2761.0186,"generate/avg_tokens_zero_rewards":3239.229,"generate/max_num_tokens":16213,"generate/std_num_tokens":2018.5572,"loss/avg_final_rewards":0.4199,"loss/avg_raw_advantages":-0.0028,"loss/avg_raw_advantages_abs":0.1708,"policy/policy_entropy":0.1622,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0334,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.4199,"timing/step":855.1144,"trainer/epoch":1}
+{"step":27,"async/staleness_mean":1.3125,"generate/avg_num_tokens":3356.1113,"generate/avg_tokens_non_zero_rewards":2782.2455,"generate/avg_tokens_zero_rewards":3788.476,"generate/max_num_tokens":23835,"generate/std_num_tokens":2371.8118,"loss/avg_final_rewards":0.4297,"loss/avg_raw_advantages":-0.0023,"loss/avg_raw_advantages_abs":0.155,"policy/policy_entropy":0.1603,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0258,"reward/avg_pass_at_8":0.6094,"reward/avg_raw_reward":0.4297,"timing/step":1080.1866,"trainer/epoch":1}
+{"step":28,"async/staleness_mean":1.7969,"generate/avg_num_tokens":3113.0625,"generate/avg_tokens_non_zero_rewards":2736.2213,"generate/avg_tokens_zero_rewards":3432.7653,"generate/max_num_tokens":12512,"generate/std_num_tokens":1519.7836,"loss/avg_final_rewards":0.459,"loss/avg_raw_advantages":0.0022,"loss/avg_raw_advantages_abs":0.1806,"policy/policy_entropy":0.1607,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.029,"reward/avg_pass_at_8":0.6719,"reward/avg_raw_reward":0.459,"timing/step":1060.8113,"trainer/epoch":1}
+{"step":29,"async/staleness_mean":1.7344,"generate/avg_num_tokens":2415.0977,"generate/avg_tokens_non_zero_rewards":2673.164,"generate/avg_tokens_zero_rewards":2264.0929,"generate/max_num_tokens":10450,"generate/std_num_tokens":1586.9879,"loss/avg_final_rewards":0.3691,"loss/avg_raw_advantages":0.0347,"loss/avg_raw_advantages_abs":0.2096,"policy/policy_entropy":0.1327,"policy/policy_loss":-0.0424,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0258,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.3691,"timing/step":1504.4548,"trainer/epoch":1}
+{"step":30,"async/staleness_mean":2.375,"generate/avg_num_tokens":2830.498,"generate/avg_tokens_non_zero_rewards":3100.0791,"generate/avg_tokens_zero_rewards":2730.0375,"generate/max_num_tokens":31793,"generate/std_num_tokens":2256.1524,"loss/avg_final_rewards":0.2715,"loss/avg_raw_advantages":0.0223,"loss/avg_raw_advantages_abs":0.1885,"policy/policy_entropy":0.1401,"policy/policy_loss":-0.022,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0276,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.2715,"timing/step":1405.3238,"trainer/epoch":1}
+{"step":31,"async/staleness_mean":2.75,"generate/avg_num_tokens":3558.3574,"generate/avg_tokens_non_zero_rewards":2757.8198,"generate/avg_tokens_zero_rewards":3963.3353,"generate/max_num_tokens":21395,"generate/std_num_tokens":2960.4903,"loss/avg_final_rewards":0.3359,"loss/avg_raw_advantages":0.0217,"loss/avg_raw_advantages_abs":0.1805,"policy/policy_entropy":0.1491,"policy/policy_loss":-0.0246,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0286,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.3359,"timing/step":1063.248,"trainer/epoch":1}
+{"step":32,"async/staleness_mean":2.75,"generate/avg_num_tokens":3455.5938,"generate/avg_tokens_non_zero_rewards":3406.0,"generate/avg_tokens_zero_rewards":3475.377,"generate/max_num_tokens":31860,"generate/std_num_tokens":3578.9994,"loss/avg_final_rewards":0.2852,"loss/avg_raw_advantages":0.04,"loss/avg_raw_advantages_abs":0.1608,"policy/policy_entropy":0.1445,"policy/policy_loss":-0.0321,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0241,"reward/avg_pass_at_8":0.5469,"reward/avg_raw_reward":0.2852,"timing/step":1066.546,"trainer/epoch":1}
+{"step":33,"async/staleness_mean":3.4844,"generate/avg_num_tokens":3966.6543,"generate/avg_tokens_non_zero_rewards":3122.9412,"generate/avg_tokens_zero_rewards":4386.0439,"generate/max_num_tokens":31837,"generate/std_num_tokens":4143.3029,"loss/avg_final_rewards":0.332,"loss/avg_raw_advantages":0.0029,"loss/avg_raw_advantages_abs":0.1575,"policy/policy_entropy":0.1508,"policy/policy_loss":-0.0059,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0269,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.332,"timing/step":1072.7697,"trainer/epoch":1}
+{"step":34,"async/staleness_mean":3.5312,"generate/avg_num_tokens":3689.877,"generate/avg_tokens_non_zero_rewards":3468.7487,"generate/avg_tokens_zero_rewards":3817.1108,"generate/max_num_tokens":31787,"generate/std_num_tokens":2794.5776,"loss/avg_final_rewards":0.3652,"loss/avg_raw_advantages":-0.0098,"loss/avg_raw_advantages_abs":0.2416,"policy/policy_entropy":0.1519,"policy/policy_loss":-0.01,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0267,"reward/avg_pass_at_8":0.625,"reward/avg_raw_reward":0.3652,"timing/step":807.2692,"trainer/epoch":1}
+{"step":35,"async/staleness_mean":3.6094,"generate/avg_num_tokens":3523.7188,"generate/avg_tokens_non_zero_rewards":3091.564,"generate/avg_tokens_zero_rewards":3742.3382,"generate/max_num_tokens":31893,"generate/std_num_tokens":3531.3037,"loss/avg_final_rewards":0.3359,"loss/avg_raw_advantages":0.0209,"loss/avg_raw_advantages_abs":0.1373,"policy/policy_entropy":0.146,"policy/policy_loss":-0.0167,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0241,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.3359,"timing/step":1104.5501,"trainer/epoch":1}
+{"step":36,"async/staleness_mean":3.3594,"generate/avg_num_tokens":2614.9492,"generate/avg_tokens_non_zero_rewards":3124.878,"generate/avg_tokens_zero_rewards":2453.7121,"generate/max_num_tokens":31790,"generate/std_num_tokens":2523.1787,"loss/avg_final_rewards":0.2402,"loss/avg_raw_advantages":0.0515,"loss/avg_raw_advantages_abs":0.1496,"policy/policy_entropy":0.1157,"policy/policy_loss":-0.0454,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0232,"reward/avg_pass_at_8":0.4688,"reward/avg_raw_reward":0.2402,"timing/step":579.0805,"trainer/epoch":1}
+{"step":37,"async/staleness_mean":3.875,"generate/avg_num_tokens":4085.3691,"generate/avg_tokens_non_zero_rewards":3542.0213,"generate/avg_tokens_zero_rewards":4291.8706,"generate/max_num_tokens":31869,"generate/std_num_tokens":3237.1246,"loss/avg_final_rewards":0.2754,"loss/avg_raw_advantages":0.0112,"loss/avg_raw_advantages_abs":0.1345,"policy/policy_entropy":0.1495,"policy/policy_loss":-0.0259,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0229,"reward/avg_pass_at_8":0.4688,"reward/avg_raw_reward":0.2754,"timing/step":919.7587,"trainer/epoch":1}
+{"step":38,"async/staleness_mean":2.9688,"generate/avg_num_tokens":3382.25,"generate/avg_tokens_non_zero_rewards":3006.8256,"generate/avg_tokens_zero_rewards":3613.1893,"generate/max_num_tokens":31806,"generate/std_num_tokens":3421.0889,"loss/avg_final_rewards":0.3809,"loss/avg_raw_advantages":0.0223,"loss/avg_raw_advantages_abs":0.1881,"policy/policy_entropy":0.1386,"policy/policy_loss":-0.0193,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0284,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.3809,"timing/step":1279.6712,"trainer/epoch":1}
+{"step":39,"async/staleness_mean":3.1406,"generate/avg_num_tokens":3678.4531,"generate/avg_tokens_non_zero_rewards":2720.9455,"generate/avg_tokens_zero_rewards":4133.7522,"generate/max_num_tokens":31879,"generate/std_num_tokens":2850.2191,"loss/avg_final_rewards":0.3223,"loss/avg_raw_advantages":-0.0032,"loss/avg_raw_advantages_abs":0.0884,"policy/policy_entropy":0.1538,"policy/policy_loss":-0.0053,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0229,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.3223,"timing/step":1041.2257,"trainer/epoch":1}
+{"step":40,"async/staleness_mean":3.0469,"generate/avg_num_tokens":3621.1445,"generate/avg_tokens_non_zero_rewards":3113.7897,"generate/avg_tokens_zero_rewards":3985.4866,"generate/max_num_tokens":31806,"generate/std_num_tokens":3049.3881,"loss/avg_final_rewards":0.418,"loss/avg_raw_advantages":0.0059,"loss/avg_raw_advantages_abs":0.1633,"policy/policy_entropy":0.1498,"policy/policy_loss":-0.0022,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0286,"reward/avg_pass_at_8":0.625,"reward/avg_raw_reward":0.418,"timing/step":1218.6808,"trainer/epoch":1}
+{"step":41,"async/staleness_mean":0.2188,"generate/avg_num_tokens":3203.291,"generate/avg_tokens_non_zero_rewards":3123.5,"generate/avg_tokens_zero_rewards":3245.0863,"generate/max_num_tokens":13902,"generate/std_num_tokens":1793.951,"loss/avg_final_rewards":0.3438,"loss/avg_raw_advantages":-0.0068,"loss/avg_raw_advantages_abs":0.1501,"policy/policy_entropy":0.1505,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0267,"reward/avg_pass_at_8":0.5333,"reward/avg_raw_reward":0.3438,"timing/step":3662.3125,"trainer/epoch":1}
+{"step":42,"async/staleness_mean":0.9062,"generate/avg_num_tokens":3331.1836,"generate/avg_tokens_non_zero_rewards":2943.854,"generate/avg_tokens_zero_rewards":3637.2552,"generate/max_num_tokens":11623,"generate/std_num_tokens":1556.8658,"loss/avg_final_rewards":0.4414,"loss/avg_raw_advantages":-0.0133,"loss/avg_raw_advantages_abs":0.1948,"policy/policy_entropy":0.1504,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0308,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.4414,"timing/step":1534.3597,"trainer/epoch":1}
+{"step":43,"async/staleness_mean":0.1719,"generate/avg_num_tokens":3190.0996,"generate/avg_tokens_non_zero_rewards":3053.0224,"generate/avg_tokens_zero_rewards":3295.872,"generate/max_num_tokens":11164,"generate/std_num_tokens":1502.4466,"loss/avg_final_rewards":0.4355,"loss/avg_raw_advantages":-0.0071,"loss/avg_raw_advantages_abs":0.1941,"policy/policy_entropy":0.1458,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0276,"reward/avg_pass_at_8":0.6667,"reward/avg_raw_reward":0.4355,"timing/step":4401.7553,"trainer/epoch":1}
+{"step":44,"async/staleness_mean":0.9375,"generate/avg_num_tokens":3215.6934,"generate/avg_tokens_non_zero_rewards":2875.5211,"generate/avg_tokens_zero_rewards":3416.4161,"generate/max_num_tokens":12180,"generate/std_num_tokens":1578.4323,"loss/avg_final_rewards":0.3711,"loss/avg_raw_advantages":-0.007,"loss/avg_raw_advantages_abs":0.1815,"policy/policy_entropy":0.1453,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0269,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.3711,"timing/step":1523.7192,"trainer/epoch":1}
+{"step":45,"async/staleness_mean":1.5938,"generate/avg_num_tokens":3762.9023,"generate/avg_tokens_non_zero_rewards":3104.2857,"generate/avg_tokens_zero_rewards":4195.5858,"generate/max_num_tokens":31796,"generate/std_num_tokens":3033.3303,"loss/avg_final_rewards":0.3965,"loss/avg_raw_advantages":-0.0017,"loss/avg_raw_advantages_abs":0.1568,"policy/policy_entropy":0.1536,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0283,"reward/avg_pass_at_8":0.6094,"reward/avg_raw_reward":0.3965,"timing/step":1062.3058,"trainer/epoch":1}
+{"step":46,"async/staleness_mean":2.2656,"generate/avg_num_tokens":3646.7539,"generate/avg_tokens_non_zero_rewards":3598.0478,"generate/avg_tokens_zero_rewards":3686.4787,"generate/max_num_tokens":17881,"generate/std_num_tokens":2052.2221,"loss/avg_final_rewards":0.4492,"loss/avg_raw_advantages":-0.0053,"loss/avg_raw_advantages_abs":0.2006,"policy/policy_entropy":0.15,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0297,"reward/avg_pass_at_8":0.6719,"reward/avg_raw_reward":0.4492,"timing/step":470.1386,"trainer/epoch":1}
+{"step":47,"async/staleness_mean":3.5469,"generate/avg_num_tokens":6660.0742,"generate/avg_tokens_non_zero_rewards":5136.8,"generate/avg_tokens_zero_rewards":6984.9431,"generate/max_num_tokens":31832,"generate/std_num_tokens":5486.724,"loss/avg_final_rewards":0.1758,"loss/avg_raw_advantages":-0.0159,"loss/avg_raw_advantages_abs":0.1279,"policy/policy_entropy":0.167,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0228,"reward/avg_pass_at_8":0.3594,"reward/avg_raw_reward":0.1758,"timing/step":948.8113,"trainer/epoch":1}
+{"step":48,"async/staleness_mean":4.4531,"generate/avg_num_tokens":9621.7012,"generate/avg_tokens_non_zero_rewards":7109.9741,"generate/avg_tokens_zero_rewards":10357.4596,"generate/max_num_tokens":31815,"generate/std_num_tokens":9164.0238,"loss/avg_final_rewards":0.2266,"loss/avg_raw_advantages":-0.0114,"loss/avg_raw_advantages_abs":0.0914,"policy/policy_entropy":0.1578,"policy/policy_loss":-0.0013,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0216,"reward/avg_pass_at_8":0.4062,"reward/avg_raw_reward":0.2266,"timing/step":1736.6484,"trainer/epoch":1}

viewer/build/inputs/marin/runs/marin-a3-nl2bash/run.json

@@ -0,0 +1,27 @@
+{
+ "id": "marin-a3-nl2bash",
+ "title": "A3 RLOO on nl2bash-tasks-cleaned-oracle (Qwen3-8B agent)",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://huggingface.co/laion/a3-rl-DCAgent2_nl2bash-tasks-cleaned-oracle-40-8B/tree/main/training_logs",
+ "license": "unknown",
+ "model": "laion/a3-rl-DCAgent2_nl2bash-tasks-cleaned-oracle-40-8B",
+ "base_model": "laion/GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink",
+ "method": "RLOO-N (SkyRL, binary verifier reward)",
+ "dataset": "DCAgent2/nl2bash-tasks-cleaned-oracle",
+ "eval_suite": null,
+ "kind": "training",
+ "state": "finished",
+ "started_at": null,
+ "updated_at": "2026-05-26T11:57:42Z",
+ "attempts": 0,
+ "note": "Marin A3 sweep (issue #6187): one RLOO-N run per training dataset from the same Qwen3-8B-derived SFT agent, here DCAgent2/nl2bash-tasks-cleaned-oracle; EMA-best step 40 at reward 0.418 (peak 0.449 at step 46), per the issue. Our copy has every logged step of the final lineage (3 resumed job segments). The issue concluded this binary-reward setup was uninformative about dataset utility.",
+ "metrics_map": {
+  "reward": "reward/avg_raw_reward",
+  "loss": "policy/policy_loss",
+  "entropy": "policy/policy_entropy",
+  "grad_norm": "policy/raw_grad_norm",
+  "response_length": "generate/avg_num_tokens"
+ }
+}

viewer/build/inputs/marin/runs/marin-a3-pymethods2test-large/metrics.jsonl

@@ -0,0 +1,80 @@
+{"step":1,"async/staleness_mean":0.0,"generate/avg_num_tokens":4192.293,"generate/avg_tokens_non_zero_rewards":4001.5203,"generate/avg_tokens_zero_rewards":4829.2797,"generate/max_num_tokens":24079,"generate/std_num_tokens":2506.7135,"loss/avg_final_rewards":0.7695,"loss/avg_raw_advantages":-0.0021,"loss/avg_raw_advantages_abs":0.0903,"policy/policy_entropy":0.1167,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.012,"reward/avg_pass_at_8":0.8281,"reward/avg_raw_reward":0.7695,"timing/step":2875.5985,"trainer/epoch":0}
+{"step":2,"async/staleness_mean":0.7969,"generate/avg_num_tokens":5257.0879,"generate/avg_tokens_non_zero_rewards":4815.84,"generate/avg_tokens_zero_rewards":5603.0139,"generate/max_num_tokens":31833,"generate/std_num_tokens":4823.3388,"loss/avg_final_rewards":0.4395,"loss/avg_raw_advantages":0.0098,"loss/avg_raw_advantages_abs":0.1284,"policy/policy_entropy":0.1334,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0208,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.4395,"timing/step":2058.8732,"trainer/epoch":0}
+{"step":3,"async/staleness_mean":1.5469,"generate/avg_num_tokens":5455.3965,"generate/avg_tokens_non_zero_rewards":5425.0375,"generate/avg_tokens_zero_rewards":5469.196,"generate/max_num_tokens":31760,"generate/std_num_tokens":5367.2198,"loss/avg_final_rewards":0.3125,"loss/avg_raw_advantages":0.013,"loss/avg_raw_advantages_abs":0.1173,"policy/policy_entropy":0.1368,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0244,"reward/avg_pass_at_8":0.4219,"reward/avg_raw_reward":0.3125,"timing/step":832.7235,"trainer/epoch":0}
+{"step":4,"async/staleness_mean":1.6562,"generate/avg_num_tokens":5456.8906,"generate/avg_tokens_non_zero_rewards":5023.9922,"generate/avg_tokens_zero_rewards":5893.1843,"generate/max_num_tokens":31773,"generate/std_num_tokens":4390.4548,"loss/avg_final_rewards":0.502,"loss/avg_raw_advantages":-0.0034,"loss/avg_raw_advantages_abs":0.0818,"policy/policy_entropy":0.1275,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0125,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.502,"timing/step":1506.9729,"trainer/epoch":0}
+{"step":5,"async/staleness_mean":1.6875,"generate/avg_num_tokens":4792.168,"generate/avg_tokens_non_zero_rewards":4960.31,"generate/avg_tokens_zero_rewards":4656.1095,"generate/max_num_tokens":21648,"generate/std_num_tokens":3443.3735,"loss/avg_final_rewards":0.4473,"loss/avg_raw_advantages":0.002,"loss/avg_raw_advantages_abs":0.1262,"policy/policy_entropy":0.1283,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.015,"reward/avg_pass_at_8":0.5469,"reward/avg_raw_reward":0.4473,"timing/step":1502.3635,"trainer/epoch":0}
+{"step":6,"async/staleness_mean":2.1875,"generate/avg_num_tokens":5843.9004,"generate/avg_tokens_non_zero_rewards":5913.6589,"generate/avg_tokens_zero_rewards":5793.8054,"generate/max_num_tokens":31837,"generate/std_num_tokens":4968.0944,"loss/avg_final_rewards":0.418,"loss/avg_raw_advantages":0.0195,"loss/avg_raw_advantages_abs":0.1118,"policy/policy_entropy":0.129,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0133,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.418,"timing/step":1135.6277,"trainer/epoch":0}
+{"step":7,"async/staleness_mean":1.9688,"generate/avg_num_tokens":4991.7988,"generate/avg_tokens_non_zero_rewards":4943.9369,"generate/avg_tokens_zero_rewards":5028.4379,"generate/max_num_tokens":31826,"generate/std_num_tokens":4014.433,"loss/avg_final_rewards":0.4336,"loss/avg_raw_advantages":0.0158,"loss/avg_raw_advantages_abs":0.1432,"policy/policy_entropy":0.136,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0145,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.4336,"timing/step":1248.2931,"trainer/epoch":0}
+{"step":8,"async/staleness_mean":1.8906,"generate/avg_num_tokens":5594.0117,"generate/avg_tokens_non_zero_rewards":5319.9113,"generate/avg_tokens_zero_rewards":5851.5,"generate/max_num_tokens":31801,"generate/std_num_tokens":4759.5927,"loss/avg_final_rewards":0.4844,"loss/avg_raw_advantages":-0.0092,"loss/avg_raw_advantages_abs":0.1119,"policy/policy_entropy":0.1317,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0122,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.4844,"timing/step":1727.4666,"trainer/epoch":0}
+{"step":9,"async/staleness_mean":1.8281,"generate/avg_num_tokens":5525.2188,"generate/avg_tokens_non_zero_rewards":5002.9954,"generate/avg_tokens_zero_rewards":5906.3007,"generate/max_num_tokens":31777,"generate/std_num_tokens":4894.2365,"loss/avg_final_rewards":0.4219,"loss/avg_raw_advantages":0.0035,"loss/avg_raw_advantages_abs":0.0801,"policy/policy_entropy":0.1326,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0099,"reward/avg_pass_at_8":0.5156,"reward/avg_raw_reward":0.4219,"timing/step":1368.8331,"trainer/epoch":0}
+{"step":10,"async/staleness_mean":1.75,"generate/avg_num_tokens":5405.0176,"generate/avg_tokens_non_zero_rewards":4519.9746,"generate/avg_tokens_zero_rewards":5958.5206,"generate/max_num_tokens":31815,"generate/std_num_tokens":4909.1097,"loss/avg_final_rewards":0.3848,"loss/avg_raw_advantages":-0.0014,"loss/avg_raw_advantages_abs":0.0918,"policy/policy_entropy":0.1359,"policy/policy_loss":-0.0003,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0133,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.3848,"timing/step":1439.6074,"trainer/epoch":0}
+{"step":11,"async/staleness_mean":1.9062,"generate/avg_num_tokens":4465.0586,"generate/avg_tokens_non_zero_rewards":4697.9737,"generate/avg_tokens_zero_rewards":4213.2073,"generate/max_num_tokens":31776,"generate/std_num_tokens":4382.1066,"loss/avg_final_rewards":0.5195,"loss/avg_raw_advantages":-0.0057,"loss/avg_raw_advantages_abs":0.1603,"policy/policy_entropy":0.1193,"policy/policy_loss":-0.0036,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0164,"reward/avg_pass_at_8":0.6875,"reward/avg_raw_reward":0.5195,"timing/step":1156.4299,"trainer/epoch":0}
+{"step":12,"async/staleness_mean":2.0938,"generate/avg_num_tokens":5363.0527,"generate/avg_tokens_non_zero_rewards":4521.8653,"generate/avg_tokens_zero_rewards":5871.9843,"generate/max_num_tokens":31759,"generate/std_num_tokens":4307.6574,"loss/avg_final_rewards":0.377,"loss/avg_raw_advantages":-0.0043,"loss/avg_raw_advantages_abs":0.0705,"policy/policy_entropy":0.131,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0118,"reward/avg_pass_at_8":0.4375,"reward/avg_raw_reward":0.377,"timing/step":1280.5093,"trainer/epoch":0}
+{"step":13,"async/staleness_mean":1.8438,"generate/avg_num_tokens":5225.9648,"generate/avg_tokens_non_zero_rewards":4635.1852,"generate/avg_tokens_zero_rewards":5759.6431,"generate/max_num_tokens":31599,"generate/std_num_tokens":4392.6399,"loss/avg_final_rewards":0.4746,"loss/avg_raw_advantages":-0.0052,"loss/avg_raw_advantages_abs":0.0701,"policy/policy_entropy":0.1226,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0125,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.4746,"timing/step":1231.3013,"trainer/epoch":0}
+{"step":14,"async/staleness_mean":2.0625,"generate/avg_num_tokens":5098.8379,"generate/avg_tokens_non_zero_rewards":4982.8538,"generate/avg_tokens_zero_rewards":5218.504,"generate/max_num_tokens":31827,"generate/std_num_tokens":4200.4394,"loss/avg_final_rewards":0.5078,"loss/avg_raw_advantages":0.0053,"loss/avg_raw_advantages_abs":0.1294,"policy/policy_entropy":0.1213,"policy/policy_loss":-0.0029,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.015,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.5078,"timing/step":1253.8114,"trainer/epoch":0}
+{"step":15,"async/staleness_mean":2.2031,"generate/avg_num_tokens":5303.2578,"generate/avg_tokens_non_zero_rewards":4787.0615,"generate/avg_tokens_zero_rewards":5620.7918,"generate/max_num_tokens":31595,"generate/std_num_tokens":4592.2823,"loss/avg_final_rewards":0.3809,"loss/avg_raw_advantages":-0.0098,"loss/avg_raw_advantages_abs":0.0731,"policy/policy_entropy":0.1345,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0103,"reward/avg_pass_at_8":0.4688,"reward/avg_raw_reward":0.3809,"timing/step":1160.9981,"trainer/epoch":0}
+{"step":16,"async/staleness_mean":2.125,"generate/avg_num_tokens":5503.4707,"generate/avg_tokens_non_zero_rewards":4714.2294,"generate/avg_tokens_zero_rewards":6152.2776,"generate/max_num_tokens":31683,"generate/std_num_tokens":4822.9636,"loss/avg_final_rewards":0.4512,"loss/avg_raw_advantages":-0.0046,"loss/avg_raw_advantages_abs":0.0667,"policy/policy_entropy":0.1228,"policy/policy_loss":-0.0006,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0114,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.4512,"timing/step":1198.5693,"trainer/epoch":0}
+{"step":17,"async/staleness_mean":2.1406,"generate/avg_num_tokens":4977.7129,"generate/avg_tokens_non_zero_rewards":4686.6981,"generate/avg_tokens_zero_rewards":5183.3633,"generate/max_num_tokens":31791,"generate/std_num_tokens":4497.3766,"loss/avg_final_rewards":0.4141,"loss/avg_raw_advantages":0.0027,"loss/avg_raw_advantages_abs":0.1366,"policy/policy_entropy":0.1268,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0162,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.4141,"timing/step":1324.8522,"trainer/epoch":0}
+{"step":18,"async/staleness_mean":2.3594,"generate/avg_num_tokens":4896.666,"generate/avg_tokens_non_zero_rewards":4869.7656,"generate/avg_tokens_zero_rewards":4912.8062,"generate/max_num_tokens":31713,"generate/std_num_tokens":3970.0096,"loss/avg_final_rewards":0.375,"loss/avg_raw_advantages":0.0047,"loss/avg_raw_advantages_abs":0.1314,"policy/policy_entropy":0.1218,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0168,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.375,"timing/step":1196.595,"trainer/epoch":0}
+{"step":19,"async/staleness_mean":2.2656,"generate/avg_num_tokens":5069.6445,"generate/avg_tokens_non_zero_rewards":5041.7042,"generate/avg_tokens_zero_rewards":5089.5485,"generate/max_num_tokens":31746,"generate/std_num_tokens":4326.0408,"loss/avg_final_rewards":0.416,"loss/avg_raw_advantages":0.0035,"loss/avg_raw_advantages_abs":0.0926,"policy/policy_entropy":0.1361,"policy/policy_loss":-0.0053,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0128,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.416,"timing/step":2034.4556,"trainer/epoch":0}
+{"step":20,"async/staleness_mean":1.8281,"generate/avg_num_tokens":4947.582,"generate/avg_tokens_non_zero_rewards":4791.9533,"generate/avg_tokens_zero_rewards":5059.3423,"generate/max_num_tokens":31736,"generate/std_num_tokens":3768.1089,"loss/avg_final_rewards":0.418,"loss/avg_raw_advantages":-0.01,"loss/avg_raw_advantages_abs":0.1096,"policy/policy_entropy":0.1296,"policy/policy_loss":-0.0027,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0157,"reward/avg_pass_at_8":0.5469,"reward/avg_raw_reward":0.418,"timing/step":1215.6439,"trainer/epoch":0}
+{"step":21,"async/staleness_mean":0.1094,"generate/avg_num_tokens":4881.0703,"generate/avg_tokens_non_zero_rewards":4619.1404,"generate/avg_tokens_zero_rewards":5209.9251,"generate/max_num_tokens":21809,"generate/std_num_tokens":3151.7318,"loss/avg_final_rewards":0.5566,"loss/avg_raw_advantages":0.0038,"loss/avg_raw_advantages_abs":0.0888,"policy/policy_entropy":0.1267,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.01,"reward/avg_pass_at_8":0.623,"reward/avg_raw_reward":0.5566,"timing/step":4128.171,"trainer/epoch":0}
+{"step":22,"async/staleness_mean":0.9531,"generate/avg_num_tokens":5270.2227,"generate/avg_tokens_non_zero_rewards":4899.0374,"generate/avg_tokens_zero_rewards":5368.2889,"generate/max_num_tokens":31764,"generate/std_num_tokens":5057.6621,"loss/avg_final_rewards":0.209,"loss/avg_raw_advantages":0.0119,"loss/avg_raw_advantages_abs":0.0484,"policy/policy_entropy":0.132,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0122,"reward/avg_pass_at_8":0.3125,"reward/avg_raw_reward":0.209,"timing/step":973.4908,"trainer/epoch":0}
+{"step":23,"async/staleness_mean":1.7344,"generate/avg_num_tokens":5478.166,"generate/avg_tokens_non_zero_rewards":5498.642,"generate/avg_tokens_zero_rewards":5467.4405,"generate/max_num_tokens":31775,"generate/std_num_tokens":5485.7275,"loss/avg_final_rewards":0.3438,"loss/avg_raw_advantages":-0.0068,"loss/avg_raw_advantages_abs":0.152,"policy/policy_entropy":0.1245,"policy/policy_loss":-0.0016,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0152,"reward/avg_pass_at_8":0.5469,"reward/avg_raw_reward":0.3438,"timing/step":784.2654,"trainer/epoch":0}
+{"step":24,"async/staleness_mean":1.6406,"generate/avg_num_tokens":4716.4082,"generate/avg_tokens_non_zero_rewards":4747.716,"generate/avg_tokens_zero_rewards":4684.8549,"generate/max_num_tokens":31777,"generate/std_num_tokens":4097.8926,"loss/avg_final_rewards":0.502,"loss/avg_raw_advantages":0.0004,"loss/avg_raw_advantages_abs":0.1541,"policy/policy_entropy":0.114,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0134,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.502,"timing/step":1437.2086,"trainer/epoch":0}
+{"step":25,"async/staleness_mean":1.4375,"generate/avg_num_tokens":4553.209,"generate/avg_tokens_non_zero_rewards":4195.6736,"generate/avg_tokens_zero_rewards":5012.8973,"generate/max_num_tokens":31787,"generate/std_num_tokens":3536.9931,"loss/avg_final_rewards":0.5625,"loss/avg_raw_advantages":-0.0032,"loss/avg_raw_advantages_abs":0.0728,"policy/policy_entropy":0.118,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0145,"reward/avg_pass_at_8":0.6562,"reward/avg_raw_reward":0.5625,"timing/step":999.0684,"trainer/epoch":0}
+{"step":26,"async/staleness_mean":2.2812,"generate/avg_num_tokens":5104.5664,"generate/avg_tokens_non_zero_rewards":4828.5526,"generate/avg_tokens_zero_rewards":5326.1549,"generate/max_num_tokens":31736,"generate/std_num_tokens":4746.2648,"loss/avg_final_rewards":0.4453,"loss/avg_raw_advantages":0.001,"loss/avg_raw_advantages_abs":0.0947,"policy/policy_entropy":0.118,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0112,"reward/avg_pass_at_8":0.5469,"reward/avg_raw_reward":0.4453,"timing/step":878.2187,"trainer/epoch":0}
+{"step":27,"async/staleness_mean":2.2656,"generate/avg_num_tokens":4773.3457,"generate/avg_tokens_non_zero_rewards":4676.4102,"generate/avg_tokens_zero_rewards":4870.2812,"generate/max_num_tokens":31628,"generate/std_num_tokens":3592.542,"loss/avg_final_rewards":0.5,"loss/avg_raw_advantages":-0.0004,"loss/avg_raw_advantages_abs":0.0708,"policy/policy_entropy":0.129,"policy/policy_loss":-0.0011,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0115,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.5,"timing/step":932.2836,"trainer/epoch":0}
+{"step":28,"async/staleness_mean":2.1094,"generate/avg_num_tokens":5473.4512,"generate/avg_tokens_non_zero_rewards":5122.3896,"generate/avg_tokens_zero_rewards":5805.8251,"generate/max_num_tokens":31813,"generate/std_num_tokens":4291.0941,"loss/avg_final_rewards":0.4863,"loss/avg_raw_advantages":0.0135,"loss/avg_raw_advantages_abs":0.1157,"policy/policy_entropy":0.1245,"policy/policy_loss":-0.0042,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0123,"reward/avg_pass_at_8":0.6094,"reward/avg_raw_reward":0.4863,"timing/step":1119.2529,"trainer/epoch":0}
+{"step":29,"async/staleness_mean":2.5469,"generate/avg_num_tokens":5148.4551,"generate/avg_tokens_non_zero_rewards":4924.1006,"generate/avg_tokens_zero_rewards":5269.0541,"generate/max_num_tokens":31727,"generate/std_num_tokens":4432.698,"loss/avg_final_rewards":0.3496,"loss/avg_raw_advantages":0.0199,"loss/avg_raw_advantages_abs":0.1139,"policy/policy_entropy":0.1332,"policy/policy_loss":-0.0042,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0133,"reward/avg_pass_at_8":0.5,"reward/avg_raw_reward":0.3496,"timing/step":1315.1914,"trainer/epoch":0}
+{"step":30,"async/staleness_mean":2.0938,"generate/avg_num_tokens":5256.7539,"generate/avg_tokens_non_zero_rewards":4689.2149,"generate/avg_tokens_zero_rewards":5712.3838,"generate/max_num_tokens":31823,"generate/std_num_tokens":4880.1149,"loss/avg_final_rewards":0.4453,"loss/avg_raw_advantages":-0.0006,"loss/avg_raw_advantages_abs":0.1078,"policy/policy_entropy":0.1216,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0131,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.4453,"timing/step":1126.1993,"trainer/epoch":0}
+{"step":31,"async/staleness_mean":2.1562,"generate/avg_num_tokens":5447.8008,"generate/avg_tokens_non_zero_rewards":4882.6652,"generate/avg_tokens_zero_rewards":5876.9931,"generate/max_num_tokens":31794,"generate/std_num_tokens":5373.6525,"loss/avg_final_rewards":0.4316,"loss/avg_raw_advantages":0.0119,"loss/avg_raw_advantages_abs":0.1249,"policy/policy_entropy":0.1187,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0128,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.4316,"timing/step":1135.5162,"trainer/epoch":0}
+{"step":32,"async/staleness_mean":2.1562,"generate/avg_num_tokens":5149.9277,"generate/avg_tokens_non_zero_rewards":4759.6182,"generate/avg_tokens_zero_rewards":5443.9966,"generate/max_num_tokens":31786,"generate/std_num_tokens":4760.0625,"loss/avg_final_rewards":0.4297,"loss/avg_raw_advantages":0.0053,"loss/avg_raw_advantages_abs":0.1298,"policy/policy_entropy":0.1244,"policy/policy_loss":-0.0042,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0193,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.4297,"timing/step":1412.6928,"trainer/epoch":0}
+{"step":33,"async/staleness_mean":2.0156,"generate/avg_num_tokens":4476.2832,"generate/avg_tokens_non_zero_rewards":4271.9141,"generate/avg_tokens_zero_rewards":4605.1529,"generate/max_num_tokens":31642,"generate/std_num_tokens":3530.384,"loss/avg_final_rewards":0.3867,"loss/avg_raw_advantages":-0.0015,"loss/avg_raw_advantages_abs":0.0917,"policy/policy_entropy":0.1288,"policy/policy_loss":-0.0003,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.013,"reward/avg_pass_at_8":0.5,"reward/avg_raw_reward":0.3867,"timing/step":1360.6217,"trainer/epoch":0}
+{"step":34,"async/staleness_mean":1.8438,"generate/avg_num_tokens":5241.541,"generate/avg_tokens_non_zero_rewards":4209.3825,"generate/avg_tokens_zero_rewards":5815.6596,"generate/max_num_tokens":31785,"generate/std_num_tokens":4688.0521,"loss/avg_final_rewards":0.3574,"loss/avg_raw_advantages":-0.0011,"loss/avg_raw_advantages_abs":0.0807,"policy/policy_entropy":0.1295,"policy/policy_loss":-0.002,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0098,"reward/avg_pass_at_8":0.4375,"reward/avg_raw_reward":0.3574,"timing/step":1222.0475,"trainer/epoch":0}
+{"step":35,"async/staleness_mean":1.9375,"generate/avg_num_tokens":4957.7051,"generate/avg_tokens_non_zero_rewards":4424.7014,"generate/avg_tokens_zero_rewards":5362.4948,"generate/max_num_tokens":31815,"generate/std_num_tokens":4909.2384,"loss/avg_final_rewards":0.4316,"loss/avg_raw_advantages":-0.0126,"loss/avg_raw_advantages_abs":0.151,"policy/policy_entropy":0.1185,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0168,"reward/avg_pass_at_8":0.6094,"reward/avg_raw_reward":0.4316,"timing/step":1206.7027,"trainer/epoch":0}
+{"step":36,"async/staleness_mean":2.2188,"generate/avg_num_tokens":5280.5957,"generate/avg_tokens_non_zero_rewards":5239.2991,"generate/avg_tokens_zero_rewards":5310.2517,"generate/max_num_tokens":31783,"generate/std_num_tokens":4369.8009,"loss/avg_final_rewards":0.418,"loss/avg_raw_advantages":0.0091,"loss/avg_raw_advantages_abs":0.1401,"policy/policy_entropy":0.1373,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0154,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.418,"timing/step":1319.1682,"trainer/epoch":0}
+{"step":37,"async/staleness_mean":1.875,"generate/avg_num_tokens":4702.6836,"generate/avg_tokens_non_zero_rewards":4466.1733,"generate/avg_tokens_zero_rewards":4981.4638,"generate/max_num_tokens":31828,"generate/std_num_tokens":3856.0146,"loss/avg_final_rewards":0.541,"loss/avg_raw_advantages":-0.0118,"loss/avg_raw_advantages_abs":0.1465,"policy/policy_entropy":0.1245,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0155,"reward/avg_pass_at_8":0.6719,"reward/avg_raw_reward":0.541,"timing/step":1191.7158,"trainer/epoch":0}
+{"step":38,"async/staleness_mean":1.9062,"generate/avg_num_tokens":4688.2715,"generate/avg_tokens_non_zero_rewards":4292.636,"generate/avg_tokens_zero_rewards":5136.6583,"generate/max_num_tokens":31757,"generate/std_num_tokens":4297.222,"loss/avg_final_rewards":0.5312,"loss/avg_raw_advantages":0.0092,"loss/avg_raw_advantages_abs":0.0646,"policy/policy_entropy":0.1179,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0114,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.5312,"timing/step":1120.233,"trainer/epoch":0}
+{"step":39,"async/staleness_mean":2.25,"generate/avg_num_tokens":5307.8359,"generate/avg_tokens_non_zero_rewards":5155.7417,"generate/avg_tokens_zero_rewards":5442.0368,"generate/max_num_tokens":31780,"generate/std_num_tokens":5012.0495,"loss/avg_final_rewards":0.4688,"loss/avg_raw_advantages":0.0029,"loss/avg_raw_advantages_abs":0.0985,"policy/policy_entropy":0.117,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0123,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.4688,"timing/step":1066.0668,"trainer/epoch":0}
+{"step":40,"async/staleness_mean":1.875,"generate/avg_num_tokens":4436.6641,"generate/avg_tokens_non_zero_rewards":4166.7167,"generate/avg_tokens_zero_rewards":4662.1039,"generate/max_num_tokens":31753,"generate/std_num_tokens":3999.5304,"loss/avg_final_rewards":0.4551,"loss/avg_raw_advantages":-0.0054,"loss/avg_raw_advantages_abs":0.0715,"policy/policy_entropy":0.1236,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0129,"reward/avg_pass_at_8":0.5469,"reward/avg_raw_reward":0.4551,"timing/step":1481.8178,"trainer/epoch":0}
+{"step":41,"async/staleness_mean":1.8438,"generate/avg_num_tokens":4542.9355,"generate/avg_tokens_non_zero_rewards":4035.5451,"generate/avg_tokens_zero_rewards":5004.8881,"generate/max_num_tokens":31718,"generate/std_num_tokens":4393.3637,"loss/avg_final_rewards":0.4766,"loss/avg_raw_advantages":0.0045,"loss/avg_raw_advantages_abs":0.1199,"policy/policy_entropy":0.1242,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0259,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.4766,"timing/step":1157.2234,"trainer/epoch":0}
+{"step":42,"async/staleness_mean":2.0156,"generate/avg_num_tokens":5080.5547,"generate/avg_tokens_non_zero_rewards":5164.2338,"generate/avg_tokens_zero_rewards":5026.4727,"generate/max_num_tokens":31811,"generate/std_num_tokens":4388.8504,"loss/avg_final_rewards":0.3926,"loss/avg_raw_advantages":0.0045,"loss/avg_raw_advantages_abs":0.0991,"policy/policy_entropy":0.1305,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0115,"reward/avg_pass_at_8":0.4688,"reward/avg_raw_reward":0.3926,"timing/step":1086.2825,"trainer/epoch":0}
+{"step":43,"async/staleness_mean":1.6875,"generate/avg_num_tokens":4376.7949,"generate/avg_tokens_non_zero_rewards":4072.7086,"generate/avg_tokens_zero_rewards":4738.0598,"generate/max_num_tokens":31719,"generate/std_num_tokens":4099.6547,"loss/avg_final_rewards":0.543,"loss/avg_raw_advantages":0.004,"loss/avg_raw_advantages_abs":0.0827,"policy/policy_entropy":0.1201,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0099,"reward/avg_pass_at_8":0.6094,"reward/avg_raw_reward":0.543,"timing/step":1205.4114,"trainer/epoch":0}
+{"step":44,"async/staleness_mean":1.7969,"generate/avg_num_tokens":5107.6914,"generate/avg_tokens_non_zero_rewards":4460.8762,"generate/avg_tokens_zero_rewards":5529.1645,"generate/max_num_tokens":31857,"generate/std_num_tokens":4652.3757,"loss/avg_final_rewards":0.3945,"loss/avg_raw_advantages":0.0099,"loss/avg_raw_advantages_abs":0.1185,"policy/policy_entropy":0.1249,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0144,"reward/avg_pass_at_8":0.5156,"reward/avg_raw_reward":0.3945,"timing/step":980.7355,"trainer/epoch":0}
+{"step":45,"async/staleness_mean":2.3594,"generate/avg_num_tokens":4768.9492,"generate/avg_tokens_non_zero_rewards":4444.9615,"generate/avg_tokens_zero_rewards":5103.2222,"generate/max_num_tokens":31722,"generate/std_num_tokens":3864.159,"loss/avg_final_rewards":0.5078,"loss/avg_raw_advantages":-0.0016,"loss/avg_raw_advantages_abs":0.1191,"policy/policy_entropy":0.1323,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0142,"reward/avg_pass_at_8":0.6094,"reward/avg_raw_reward":0.5078,"timing/step":1139.241,"trainer/epoch":0}
+{"step":46,"async/staleness_mean":2.1406,"generate/avg_num_tokens":4766.8594,"generate/avg_tokens_non_zero_rewards":4459.6325,"generate/avg_tokens_zero_rewards":5025.4604,"generate/max_num_tokens":31694,"generate/std_num_tokens":3772.8036,"loss/avg_final_rewards":0.457,"loss/avg_raw_advantages":0.0014,"loss/avg_raw_advantages_abs":0.0708,"policy/policy_entropy":0.1301,"policy/policy_loss":-0.0017,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0122,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.457,"timing/step":996.6627,"trainer/epoch":0}
+{"step":47,"async/staleness_mean":2.2812,"generate/avg_num_tokens":4696.0488,"generate/avg_tokens_non_zero_rewards":4467.2452,"generate/avg_tokens_zero_rewards":4933.9681,"generate/max_num_tokens":31741,"generate/std_num_tokens":3822.6237,"loss/avg_final_rewards":0.5098,"loss/avg_raw_advantages":0.0086,"loss/avg_raw_advantages_abs":0.1125,"policy/policy_entropy":0.1265,"policy/policy_loss":-0.0018,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0132,"reward/avg_pass_at_8":0.5938,"reward/avg_raw_reward":0.5098,"timing/step":1240.2406,"trainer/epoch":0}
+{"step":48,"async/staleness_mean":2.125,"generate/avg_num_tokens":4586.2266,"generate/avg_tokens_non_zero_rewards":4308.5578,"generate/avg_tokens_zero_rewards":4762.7636,"generate/max_num_tokens":31792,"generate/std_num_tokens":3798.6281,"loss/avg_final_rewards":0.3887,"loss/avg_raw_advantages":-0.0035,"loss/avg_raw_advantages_abs":0.0731,"policy/policy_entropy":0.1356,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0134,"reward/avg_pass_at_8":0.5469,"reward/avg_raw_reward":0.3887,"timing/step":1163.4312,"trainer/epoch":0}
+{"step":49,"async/staleness_mean":1.9375,"generate/avg_num_tokens":4732.1387,"generate/avg_tokens_non_zero_rewards":3860.716,"generate/avg_tokens_zero_rewards":5519.3346,"generate/max_num_tokens":31737,"generate/std_num_tokens":4044.3982,"loss/avg_final_rewards":0.4746,"loss/avg_raw_advantages":0.0099,"loss/avg_raw_advantages_abs":0.1,"policy/policy_entropy":0.1271,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0118,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.4746,"timing/step":1178.3931,"trainer/epoch":0}
+{"step":50,"async/staleness_mean":1.9844,"generate/avg_num_tokens":4752.7832,"generate/avg_tokens_non_zero_rewards":3984.8378,"generate/avg_tokens_zero_rewards":5187.2477,"generate/max_num_tokens":31754,"generate/std_num_tokens":4564.5581,"loss/avg_final_rewards":0.3613,"loss/avg_raw_advantages":0.0104,"loss/avg_raw_advantages_abs":0.0732,"policy/policy_entropy":0.1246,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0142,"reward/avg_pass_at_8":0.4531,"reward/avg_raw_reward":0.3613,"timing/step":1369.6342,"trainer/epoch":0}
+{"step":51,"async/staleness_mean":2.0,"generate/avg_num_tokens":4994.0371,"generate/avg_tokens_non_zero_rewards":4122.0166,"generate/avg_tokens_zero_rewards":5470.8822,"generate/max_num_tokens":31799,"generate/std_num_tokens":4809.8043,"loss/avg_final_rewards":0.3535,"loss/avg_raw_advantages":0.007,"loss/avg_raw_advantages_abs":0.088,"policy/policy_entropy":0.1301,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0112,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.3535,"timing/step":1317.2383,"trainer/epoch":0}
+{"step":52,"async/staleness_mean":1.8594,"generate/avg_num_tokens":4473.2402,"generate/avg_tokens_non_zero_rewards":4444.8734,"generate/avg_tokens_zero_rewards":4496.1943,"generate/max_num_tokens":31718,"generate/std_num_tokens":4204.3242,"loss/avg_final_rewards":0.4473,"loss/avg_raw_advantages":-0.0028,"loss/avg_raw_advantages_abs":0.1468,"policy/policy_entropy":0.1284,"policy/policy_loss":-0.0016,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.015,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.4473,"timing/step":1203.2679,"trainer/epoch":0}
+{"step":53,"async/staleness_mean":0.5156,"generate/avg_num_tokens":5199.4434,"generate/avg_tokens_non_zero_rewards":4615.7731,"generate/avg_tokens_zero_rewards":5706.427,"generate/max_num_tokens":19708,"generate/std_num_tokens":3413.6197,"loss/avg_final_rewards":0.4648,"loss/avg_raw_advantages":0.0013,"loss/avg_raw_advantages_abs":0.1057,"policy/policy_entropy":0.137,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0141,"reward/avg_pass_at_8":0.5556,"reward/avg_raw_reward":0.4648,"timing/step":4341.8808,"trainer/epoch":0}
+{"step":54,"async/staleness_mean":1.0,"generate/avg_num_tokens":4652.5859,"generate/avg_tokens_non_zero_rewards":4194.3092,"generate/avg_tokens_zero_rewards":4846.0806,"generate/max_num_tokens":31805,"generate/std_num_tokens":4709.4531,"loss/avg_final_rewards":0.2969,"loss/avg_raw_advantages":-0.0034,"loss/avg_raw_advantages_abs":0.1147,"policy/policy_entropy":0.1323,"policy/policy_loss":-0.0011,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0166,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.2969,"timing/step":865.4975,"trainer/epoch":0}
+{"step":55,"async/staleness_mean":1.6875,"generate/avg_num_tokens":4469.4453,"generate/avg_tokens_non_zero_rewards":4121.8427,"generate/avg_tokens_zero_rewards":4654.6946,"generate/max_num_tokens":31781,"generate/std_num_tokens":4099.3488,"loss/avg_final_rewards":0.3477,"loss/avg_raw_advantages":0.0009,"loss/avg_raw_advantages_abs":0.0838,"policy/policy_entropy":0.1244,"policy/policy_loss":-0.0033,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0124,"reward/avg_pass_at_8":0.4375,"reward/avg_raw_reward":0.3477,"timing/step":891.5335,"trainer/epoch":0}
+{"step":56,"async/staleness_mean":1.8281,"generate/avg_num_tokens":4797.8691,"generate/avg_tokens_non_zero_rewards":4535.0826,"generate/avg_tokens_zero_rewards":5033.4037,"generate/max_num_tokens":31839,"generate/std_num_tokens":4085.0137,"loss/avg_final_rewards":0.4727,"loss/avg_raw_advantages":-0.004,"loss/avg_raw_advantages_abs":0.0812,"policy/policy_entropy":0.1286,"policy/policy_loss":-0.0022,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0124,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.4727,"timing/step":1874.4506,"trainer/epoch":0}
+{"step":57,"async/staleness_mean":1.9531,"generate/avg_num_tokens":4748.166,"generate/avg_tokens_non_zero_rewards":4372.1198,"generate/avg_tokens_zero_rewards":5024.7831,"generate/max_num_tokens":31734,"generate/std_num_tokens":4589.3974,"loss/avg_final_rewards":0.4238,"loss/avg_raw_advantages":-0.019,"loss/avg_raw_advantages_abs":0.1613,"policy/policy_entropy":0.1205,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0158,"reward/avg_pass_at_8":0.5781,"reward/avg_raw_reward":0.4238,"timing/step":1421.3428,"trainer/epoch":0}
+{"step":58,"async/staleness_mean":2.0156,"generate/avg_num_tokens":4414.9258,"generate/avg_tokens_non_zero_rewards":4027.8325,"generate/avg_tokens_zero_rewards":4681.9307,"generate/max_num_tokens":31744,"generate/std_num_tokens":4331.3861,"loss/avg_final_rewards":0.4082,"loss/avg_raw_advantages":0.02,"loss/avg_raw_advantages_abs":0.0943,"policy/policy_entropy":0.1266,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0086,"reward/avg_pass_at_8":0.5,"reward/avg_raw_reward":0.4082,"timing/step":1211.3842,"trainer/epoch":0}
+{"step":59,"async/staleness_mean":1.5469,"generate/avg_num_tokens":4602.1758,"generate/avg_tokens_non_zero_rewards":4262.5259,"generate/avg_tokens_zero_rewards":4981.124,"generate/max_num_tokens":31779,"generate/std_num_tokens":3628.5089,"loss/avg_final_rewards":0.5273,"loss/avg_raw_advantages":-0.0044,"loss/avg_raw_advantages_abs":0.0806,"policy/policy_entropy":0.1288,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0116,"reward/avg_pass_at_8":0.625,"reward/avg_raw_reward":0.5273,"timing/step":1271.1443,"trainer/epoch":0}
+{"step":60,"async/staleness_mean":2.125,"generate/avg_num_tokens":5046.2031,"generate/avg_tokens_non_zero_rewards":4003.8878,"generate/avg_tokens_zero_rewards":5742.2117,"generate/max_num_tokens":31794,"generate/std_num_tokens":4683.5018,"loss/avg_final_rewards":0.4004,"loss/avg_raw_advantages":-0.0136,"loss/avg_raw_advantages_abs":0.1153,"policy/policy_entropy":0.1261,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0149,"reward/avg_pass_at_8":0.5156,"reward/avg_raw_reward":0.4004,"timing/step":1231.3773,"trainer/epoch":0}
+{"step":61,"async/staleness_mean":2.0312,"generate/avg_num_tokens":4474.1406,"generate/avg_tokens_non_zero_rewards":4349.8018,"generate/avg_tokens_zero_rewards":4573.1754,"generate/max_num_tokens":31577,"generate/std_num_tokens":4069.0396,"loss/avg_final_rewards":0.4434,"loss/avg_raw_advantages":0.0248,"loss/avg_raw_advantages_abs":0.1317,"policy/policy_entropy":0.1278,"policy/policy_loss":-0.0033,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0172,"reward/avg_pass_at_8":0.6094,"reward/avg_raw_reward":0.4434,"timing/step":941.3057,"trainer/epoch":0}
+{"step":62,"async/staleness_mean":1.9688,"generate/avg_num_tokens":4016.3633,"generate/avg_tokens_non_zero_rewards":3720.6367,"generate/avg_tokens_zero_rewards":4367.6966,"generate/max_num_tokens":31692,"generate/std_num_tokens":3405.5035,"loss/avg_final_rewards":0.543,"loss/avg_raw_advantages":-0.0101,"loss/avg_raw_advantages_abs":0.087,"policy/policy_entropy":0.122,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0129,"reward/avg_pass_at_8":0.6406,"reward/avg_raw_reward":0.543,"timing/step":906.4568,"trainer/epoch":0}
+{"step":63,"async/staleness_mean":2.2344,"generate/avg_num_tokens":4799.0352,"generate/avg_tokens_non_zero_rewards":4425.4542,"generate/avg_tokens_zero_rewards":5128.6654,"generate/max_num_tokens":31672,"generate/std_num_tokens":4411.6362,"loss/avg_final_rewards":0.4688,"loss/avg_raw_advantages":0.0048,"loss/avg_raw_advantages_abs":0.0778,"policy/policy_entropy":0.1212,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0132,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.4688,"timing/step":1291.3037,"trainer/epoch":0}
+{"step":64,"async/staleness_mean":2.2188,"generate/avg_num_tokens":4530.0508,"generate/avg_tokens_non_zero_rewards":4097.4039,"generate/avg_tokens_zero_rewards":4814.2816,"generate/max_num_tokens":31839,"generate/std_num_tokens":3885.326,"loss/avg_final_rewards":0.3965,"loss/avg_raw_advantages":0.0035,"loss/avg_raw_advantages_abs":0.0729,"policy/policy_entropy":0.1266,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0113,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.3965,"timing/step":1254.7905,"trainer/epoch":0}
+{"step":65,"async/staleness_mean":1.9219,"generate/avg_num_tokens":4867.0156,"generate/avg_tokens_non_zero_rewards":4570.0317,"generate/avg_tokens_zero_rewards":5092.5601,"generate/max_num_tokens":31738,"generate/std_num_tokens":5141.2786,"loss/avg_final_rewards":0.4316,"loss/avg_raw_advantages":-0.0013,"loss/avg_raw_advantages_abs":0.08,"policy/policy_entropy":0.1177,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0109,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.4316,"timing/step":1262.3508,"trainer/epoch":0}
+{"step":66,"async/staleness_mean":1.9531,"generate/avg_num_tokens":4835.3867,"generate/avg_tokens_non_zero_rewards":4683.6935,"generate/avg_tokens_zero_rewards":4931.8307,"generate/max_num_tokens":31768,"generate/std_num_tokens":4588.801,"loss/avg_final_rewards":0.3887,"loss/avg_raw_advantages":0.0186,"loss/avg_raw_advantages_abs":0.1044,"policy/policy_entropy":0.129,"policy/policy_loss":-0.0029,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0383,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.3887,"timing/step":1226.6652,"trainer/epoch":0}
+{"step":67,"async/staleness_mean":2.1406,"generate/avg_num_tokens":4859.6602,"generate/avg_tokens_non_zero_rewards":4760.3577,"generate/avg_tokens_zero_rewards":4951.4962,"generate/max_num_tokens":31778,"generate/std_num_tokens":4213.4389,"loss/avg_final_rewards":0.4805,"loss/avg_raw_advantages":0.0169,"loss/avg_raw_advantages_abs":0.1201,"policy/policy_entropy":0.1157,"policy/policy_loss":-0.002,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0143,"reward/avg_pass_at_8":0.625,"reward/avg_raw_reward":0.4805,"timing/step":1502.8795,"trainer/epoch":0}
+{"step":68,"async/staleness_mean":2.0625,"generate/avg_num_tokens":4823.1309,"generate/avg_tokens_non_zero_rewards":3819.7772,"generate/avg_tokens_zero_rewards":5476.929,"generate/max_num_tokens":31747,"generate/std_num_tokens":4682.8852,"loss/avg_final_rewards":0.3945,"loss/avg_raw_advantages":-0.003,"loss/avg_raw_advantages_abs":0.0765,"policy/policy_entropy":0.1258,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0111,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.3945,"timing/step":1347.7896,"trainer/epoch":0}
+{"step":69,"async/staleness_mean":2.0625,"generate/avg_num_tokens":4802.7988,"generate/avg_tokens_non_zero_rewards":4466.5288,"generate/avg_tokens_zero_rewards":5032.8783,"generate/max_num_tokens":31776,"generate/std_num_tokens":4001.0803,"loss/avg_final_rewards":0.4062,"loss/avg_raw_advantages":0.0129,"loss/avg_raw_advantages_abs":0.0779,"policy/policy_entropy":0.1312,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0104,"reward/avg_pass_at_8":0.5312,"reward/avg_raw_reward":0.4062,"timing/step":1173.5764,"trainer/epoch":0}
+{"step":70,"async/staleness_mean":1.9844,"generate/avg_num_tokens":4320.1016,"generate/avg_tokens_non_zero_rewards":3786.6414,"generate/avg_tokens_zero_rewards":4833.1226,"generate/max_num_tokens":31673,"generate/std_num_tokens":4150.9558,"loss/avg_final_rewards":0.4902,"loss/avg_raw_advantages":0.0049,"loss/avg_raw_advantages_abs":0.1045,"policy/policy_entropy":0.1204,"policy/policy_loss":-0.0007,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0155,"reward/avg_pass_at_8":0.6094,"reward/avg_raw_reward":0.4902,"timing/step":982.2804,"trainer/epoch":0}
+{"step":71,"async/staleness_mean":1.9844,"generate/avg_num_tokens":4748.3926,"generate/avg_tokens_non_zero_rewards":4197.352,"generate/avg_tokens_zero_rewards":5044.5976,"generate/max_num_tokens":31719,"generate/std_num_tokens":4557.159,"loss/avg_final_rewards":0.3496,"loss/avg_raw_advantages":-0.006,"loss/avg_raw_advantages_abs":0.0866,"policy/policy_entropy":0.1196,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0111,"reward/avg_pass_at_8":0.4375,"reward/avg_raw_reward":0.3496,"timing/step":1222.2327,"trainer/epoch":0}
+{"step":72,"async/staleness_mean":2.1562,"generate/avg_num_tokens":4384.8965,"generate/avg_tokens_non_zero_rewards":4009.738,"generate/avg_tokens_zero_rewards":4688.47,"generate/max_num_tokens":31676,"generate/std_num_tokens":3724.2524,"loss/avg_final_rewards":0.4473,"loss/avg_raw_advantages":-0.0023,"loss/avg_raw_advantages_abs":0.1007,"policy/policy_entropy":0.1237,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0121,"reward/avg_pass_at_8":0.5156,"reward/avg_raw_reward":0.4473,"timing/step":1228.542,"trainer/epoch":0}
+{"step":73,"async/staleness_mean":2.1719,"generate/avg_num_tokens":4876.4766,"generate/avg_tokens_non_zero_rewards":4516.1187,"generate/avg_tokens_zero_rewards":5145.8225,"generate/max_num_tokens":31688,"generate/std_num_tokens":4021.3811,"loss/avg_final_rewards":0.4277,"loss/avg_raw_advantages":-0.0084,"loss/avg_raw_advantages_abs":0.0792,"policy/policy_entropy":0.1229,"policy/policy_loss":-0.0017,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0115,"reward/avg_pass_at_8":0.5156,"reward/avg_raw_reward":0.4277,"timing/step":1018.1799,"trainer/epoch":0}
+{"step":74,"async/staleness_mean":2.3281,"generate/avg_num_tokens":4804.1309,"generate/avg_tokens_non_zero_rewards":4542.4787,"generate/avg_tokens_zero_rewards":4955.9537,"generate/max_num_tokens":31743,"generate/std_num_tokens":4592.2684,"loss/avg_final_rewards":0.3672,"loss/avg_raw_advantages":0.0062,"loss/avg_raw_advantages_abs":0.1369,"policy/policy_entropy":0.1199,"policy/policy_loss":-0.0033,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.017,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.3672,"timing/step":1390.0971,"trainer/epoch":0}
+{"step":75,"async/staleness_mean":2.5156,"generate/avg_num_tokens":4587.1035,"generate/avg_tokens_non_zero_rewards":4325.1543,"generate/avg_tokens_zero_rewards":4739.0988,"generate/max_num_tokens":31723,"generate/std_num_tokens":4230.4597,"loss/avg_final_rewards":0.3672,"loss/avg_raw_advantages":0.0006,"loss/avg_raw_advantages_abs":0.0935,"policy/policy_entropy":0.1223,"policy/policy_loss":-0.0042,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0152,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.3672,"timing/step":1209.4139,"trainer/epoch":0}
+{"step":76,"async/staleness_mean":3.0938,"generate/avg_num_tokens":4055.0,"generate/avg_tokens_non_zero_rewards":4114.375,"generate/avg_tokens_zero_rewards":4021.6921,"generate/max_num_tokens":31784,"generate/std_num_tokens":3911.3755,"loss/avg_final_rewards":0.3594,"loss/avg_raw_advantages":0.0034,"loss/avg_raw_advantages_abs":0.1224,"policy/policy_entropy":0.1159,"policy/policy_loss":-0.0039,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0131,"reward/avg_pass_at_8":0.4844,"reward/avg_raw_reward":0.3594,"timing/step":2079.0748,"trainer/epoch":0}
+{"step":77,"async/staleness_mean":1.125,"generate/avg_num_tokens":4684.8711,"generate/avg_tokens_non_zero_rewards":4627.4862,"generate/avg_tokens_zero_rewards":4716.2508,"generate/max_num_tokens":29309,"generate/std_num_tokens":3669.1501,"loss/avg_final_rewards":0.3535,"loss/avg_raw_advantages":-0.0044,"loss/avg_raw_advantages_abs":0.0207,"policy/policy_entropy":0.1248,"policy/policy_loss":-0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0043,"reward/avg_pass_at_8":0.3962,"reward/avg_raw_reward":0.3535,"timing/step":4232.7558,"trainer/epoch":0}
+{"step":78,"async/staleness_mean":1.0,"generate/avg_num_tokens":5120.0469,"generate/avg_tokens_non_zero_rewards":6577.6081,"generate/avg_tokens_zero_rewards":4873.7922,"generate/max_num_tokens":26429,"generate/std_num_tokens":4703.8288,"loss/avg_final_rewards":0.1445,"loss/avg_raw_advantages":0.0011,"loss/avg_raw_advantages_abs":0.1266,"policy/policy_entropy":0.1302,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0133,"reward/avg_pass_at_8":0.2969,"reward/avg_raw_reward":0.1445,"timing/step":1225.2097,"trainer/epoch":0}
+{"step":79,"async/staleness_mean":0.0,"generate/avg_num_tokens":3803.4238,"generate/avg_tokens_non_zero_rewards":3557.0632,"generate/avg_tokens_zero_rewards":4512.6439,"generate/max_num_tokens":23523,"generate/std_num_tokens":2552.2554,"loss/avg_final_rewards":0.7422,"loss/avg_raw_advantages":-0.0062,"loss/avg_raw_advantages_abs":0.083,"policy/policy_entropy":0.114,"policy/policy_loss":0.0,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.1022,"reward/avg_pass_at_8":0.7969,"reward/avg_raw_reward":0.7422,"timing/step":2217.0772,"trainer/epoch":1}
+{"step":80,"async/staleness_mean":0.75,"generate/avg_num_tokens":4761.9473,"generate/avg_tokens_non_zero_rewards":4151.458,"generate/avg_tokens_zero_rewards":5534.5133,"generate/max_num_tokens":31689,"generate/std_num_tokens":3837.8046,"loss/avg_final_rewards":0.5586,"loss/avg_raw_advantages":0.0217,"loss/avg_raw_advantages_abs":0.098,"policy/policy_entropy":0.1188,"policy/policy_loss":-0.0006,"policy/ppo_clip_ratio":0.0,"policy/raw_grad_norm":0.0114,"reward/avg_pass_at_8":0.6719,"reward/avg_raw_reward":0.5586,"timing/step":2250.5857,"trainer/epoch":1}

viewer/build/inputs/marin/runs/marin-a3-pymethods2test-large/run.json

@@ -0,0 +1,27 @@
+{
+ "id": "marin-a3-pymethods2test-large",
+ "title": "A3 RLOO on exp_rpt_pymethods2test-large (Qwen3-8B agent)",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://huggingface.co/laion/a3-rl-DCAgent_exp_rpt_pymethods2test-large-80-8B/tree/main/training_logs",
+ "license": "apache-2.0",
+ "model": "laion/a3-rl-DCAgent_exp_rpt_pymethods2test-large-80-8B",
+ "base_model": "laion/GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink",
+ "method": "RLOO-N (SkyRL, binary verifier reward)",
+ "dataset": "DCAgent/exp_rpt_pymethods2test-large",
+ "eval_suite": null,
+ "kind": "training",
+ "state": "finished",
+ "started_at": null,
+ "updated_at": "2026-09-04T16:21:14Z",
+ "attempts": 0,
+ "note": "Marin A3 sweep (issue #6187): one RLOO-N run per training dataset from the same Qwen3-8B-derived SFT agent, here DCAgent/exp_rpt_pymethods2test-large; the issue calls it the paper hero dataset (0.74 reward, 0.83 pass@8 at step 80). Our copy has every logged step of the final lineage (7 resumed job segments, 2 superseded rows dropped), cut at step 80 because the model card says later steps are not part of the legitimate run. The issue concluded this binary-reward setup was uninformative about dataset utility.",
+ "metrics_map": {
+  "reward": "reward/avg_raw_reward",
+  "loss": "policy/policy_loss",
+  "entropy": "policy/policy_entropy",
+  "grad_norm": "policy/raw_grad_norm",
+  "response_length": "generate/avg_num_tokens"
+ }
+}

viewer/build/inputs/marin/runs/marin-q3c-cal-agent-rloo-lr2/metrics.jsonl

@@ -0,0 +1,19 @@
+{"step":0,"val/pass_at_1":0.34375,"val/avg_score":0.341015625,"val/passed":44,"val/turn_cap_rate":0.0546875}
+{"step":1,"async/staleness_mean":0.0,"generate/avg_num_tokens":3169.005859375,"generate/avg_tokens_non_zero_rewards":3047.5511363636365,"generate/avg_tokens_zero_rewards":3232.625,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":16680,"generate/std_num_tokens":2301.8342443880583,"loss/avg_final_rewards":0.34208986163139343,"loss/avg_raw_advantages":-0.020328659564256668,"loss/avg_raw_advantages_abs":0.13818322122097015,"policy/policy_entropy":0.33798813761677593,"policy/policy_loss":0.0005956882123427931,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.020572011387230305,"policy/raw_grad_norm":0.144287109375,"reward/avg_pass_at_8":0.46875,"reward/avg_raw_reward":0.34208984374999996,"timing/step":1197.5906174411066,"trainer/epoch":0}
+{"step":2,"async/staleness_mean":1.0,"generate/avg_num_tokens":3110.166015625,"generate/avg_tokens_non_zero_rewards":3008.1951219512193,"generate/avg_tokens_zero_rewards":3204.46992481203,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":15321,"generate/std_num_tokens":2193.8297238401246,"loss/avg_final_rewards":0.4786132574081421,"loss/avg_raw_advantages":-0.02599477395415306,"loss/avg_raw_advantages_abs":0.15201282501220703,"policy/policy_entropy":0.23885099159087986,"policy/policy_loss":0.0007072385169522022,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.024321202359715244,"policy/raw_grad_norm":0.1138916015625,"reward/avg_pass_at_8":0.625,"reward/avg_raw_reward":0.4786132812499999,"timing/step":836.4066986110993,"trainer/epoch":0}
+{"step":3,"async/staleness_mean":1.9375,"generate/avg_num_tokens":3501.435546875,"generate/avg_tokens_non_zero_rewards":3145.368715083799,"generate/avg_tokens_zero_rewards":3692.834834834835,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":23355,"generate/std_num_tokens":2772.1238536379096,"loss/avg_final_rewards":0.34707027673721313,"loss/avg_raw_advantages":-0.024716880172491074,"loss/avg_raw_advantages_abs":0.17512236535549164,"policy/policy_entropy":0.1889580829301849,"policy/policy_loss":0.001008056457976636,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.036330787639599293,"policy/raw_grad_norm":0.236328125,"reward/avg_pass_at_8":0.546875,"reward/avg_raw_reward":0.3470703125,"timing/step":913.7465231099632,"trainer/epoch":0,"val/pass_at_1":0.359375,"val/avg_score":0.35234375,"val/passed":46,"val/turn_cap_rate":0.140625}
+{"step":4,"async/staleness_mean":1.75,"generate/avg_num_tokens":3881.056640625,"generate/avg_tokens_non_zero_rewards":3660.344680851064,"generate/avg_tokens_zero_rewards":4068.303249097473,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":18340,"generate/std_num_tokens":3362.801656821283,"loss/avg_final_rewards":0.4541015625,"loss/avg_raw_advantages":-0.02971545234322548,"loss/avg_raw_advantages_abs":0.12146639823913574,"policy/policy_entropy":0.11838510842062533,"policy/policy_loss":0.0006702039972878993,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.02975204315953306,"policy/raw_grad_norm":0.0953369140625,"reward/avg_pass_at_8":0.546875,"reward/avg_raw_reward":0.4541015625,"timing/step":889.9785334360786,"trainer/epoch":0}
+{"step":5,"async/staleness_mean":1.75,"generate/avg_num_tokens":5361.0859375,"generate/avg_tokens_non_zero_rewards":5343.271889400921,"generate/avg_tokens_zero_rewards":5374.189830508474,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":23187,"generate/std_num_tokens":4496.000063471946,"loss/avg_final_rewards":0.4136718511581421,"loss/avg_raw_advantages":-0.059510089457035065,"loss/avg_raw_advantages_abs":0.16911280155181885,"policy/policy_entropy":0.09194204691448249,"policy/policy_loss":0.0009682493318905472,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.022816467586380895,"policy/raw_grad_norm":0.13134765625,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.413671875,"timing/step":996.7525708028115,"trainer/epoch":0}
+{"step":6,"async/staleness_mean":1.5,"generate/avg_num_tokens":4458.87109375,"generate/avg_tokens_non_zero_rewards":4671.420560747663,"generate/avg_tokens_zero_rewards":4306.234899328859,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":19872,"generate/std_num_tokens":3870.8484321845617,"loss/avg_final_rewards":0.40693360567092896,"loss/avg_raw_advantages":-0.05148852616548538,"loss/avg_raw_advantages_abs":0.1130906492471695,"policy/policy_entropy":0.0621955449169036,"policy/policy_loss":0.0005404666471804376,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.02080482004384976,"policy/raw_grad_norm":0.0806884765625,"reward/avg_pass_at_8":0.46875,"reward/avg_raw_reward":0.40693359375,"timing/step":949.7337679369375,"trainer/epoch":0,"val/pass_at_1":0.359375,"val/avg_score":0.355078125,"val/passed":46,"val/turn_cap_rate":0.0859375}
+{"step":7,"async/staleness_mean":1.6875,"generate/avg_num_tokens":4579.322265625,"generate/avg_tokens_non_zero_rewards":4203.926829268293,"generate/avg_tokens_zero_rewards":4829.99348534202,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":20889,"generate/std_num_tokens":4123.921052034155,"loss/avg_final_rewards":0.38720703125,"loss/avg_raw_advantages":-0.05643348768353462,"loss/avg_raw_advantages_abs":0.10127931088209152,"policy/policy_entropy":0.05551820420078002,"policy/policy_loss":0.0003398132812435506,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.022684094990836456,"policy/raw_grad_norm":0.04833984375,"reward/avg_pass_at_8":0.421875,"reward/avg_raw_reward":0.38720703125,"timing/step":979.0248667930719,"trainer/epoch":0}
+{"step":8,"async/staleness_mean":1.671875,"generate/avg_num_tokens":3923.87109375,"generate/avg_tokens_non_zero_rewards":3488.4464285714284,"generate/avg_tokens_zero_rewards":4262.534722222223,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":22216,"generate/std_num_tokens":3455.0974664854866,"loss/avg_final_rewards":0.4271484315395355,"loss/avg_raw_advantages":-0.03237992152571678,"loss/avg_raw_advantages_abs":0.07515560835599899,"policy/policy_entropy":0.0504411113797687,"policy/policy_loss":0.00031748439323564526,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.020362420014862437,"policy/raw_grad_norm":0.0975341796875,"reward/avg_pass_at_8":0.484375,"reward/avg_raw_reward":0.4271484375,"timing/step":921.379292229889,"trainer/epoch":0}
+{"step":9,"async/staleness_mean":1.984375,"generate/avg_num_tokens":3512.30078125,"generate/avg_tokens_non_zero_rewards":3417.864077669903,"generate/avg_tokens_zero_rewards":3656.0492610837437,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":15068,"generate/std_num_tokens":3268.448468310215,"loss/avg_final_rewards":0.595703125,"loss/avg_raw_advantages":-0.01284950040280819,"loss/avg_raw_advantages_abs":0.06535952538251877,"policy/policy_entropy":0.050410433934303,"policy/policy_loss":0.00029762777739961166,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.012124416214646772,"policy/raw_grad_norm":0.0872802734375,"reward/avg_pass_at_8":0.640625,"reward/avg_raw_reward":0.595703125,"timing/step":877.7770353299566,"trainer/epoch":0,"val/pass_at_1":0.4375,"val/avg_score":0.435546875,"val/passed":56,"val/turn_cap_rate":0.0390625}
+{"step":10,"async/staleness_mean":1.984375,"generate/avg_num_tokens":3033.501953125,"generate/avg_tokens_non_zero_rewards":2860.9162995594716,"generate/avg_tokens_zero_rewards":3170.964912280702,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":18054,"generate/std_num_tokens":2950.8963812918246,"loss/avg_final_rewards":0.43867185711860657,"loss/avg_raw_advantages":-0.04082788899540901,"loss/avg_raw_advantages_abs":0.08856792747974396,"policy/policy_entropy":0.04837490466888994,"policy/policy_loss":0.00027923968082177453,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.010592285847451421,"policy/raw_grad_norm":0.077880859375,"reward/avg_pass_at_8":0.46875,"reward/avg_raw_reward":0.438671875,"timing/step":861.4091469449922,"trainer/epoch":0}
+{"step":11,"async/staleness_mean":2.0,"generate/avg_num_tokens":2887.61328125,"generate/avg_tokens_non_zero_rewards":2516.802197802198,"generate/avg_tokens_zero_rewards":3311.175732217573,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":16128,"generate/std_num_tokens":3032.386360269517,"loss/avg_final_rewards":0.5287109613418579,"loss/avg_raw_advantages":-0.03458466753363609,"loss/avg_raw_advantages_abs":0.07965916395187378,"policy/policy_entropy":0.047808086790610105,"policy/policy_loss":0.0002547272488300223,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.00924236840364756,"policy/raw_grad_norm":0.1026611328125,"reward/avg_pass_at_8":0.578125,"reward/avg_raw_reward":0.5287109375000001,"timing/step":865.8828692450188,"trainer/epoch":0}
+{"step":12,"async/staleness_mean":1.53125,"generate/avg_num_tokens":2202.90625,"generate/avg_tokens_non_zero_rewards":1955.3356401384083,"generate/avg_tokens_zero_rewards":2523.748878923767,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":15027,"generate/std_num_tokens":2255.361813363809,"loss/avg_final_rewards":0.561718761920929,"loss/avg_raw_advantages":-0.007593624293804169,"loss/avg_raw_advantages_abs":0.05278369411826134,"policy/policy_entropy":0.043530179289518856,"policy/policy_loss":0.00019258220800111303,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.006955705969630799,"policy/raw_grad_norm":0.0836181640625,"reward/avg_pass_at_8":0.59375,"reward/avg_raw_reward":0.56171875,"timing/step":812.169394148048,"trainer/epoch":0,"val/pass_at_1":0.4609375,"val/avg_score":0.460546875,"val/passed":59,"val/turn_cap_rate":0.0078125}
+{"step":13,"async/staleness_mean":1.578125,"generate/avg_num_tokens":2165.09765625,"generate/avg_tokens_non_zero_rewards":2050.3623188405795,"generate/avg_tokens_zero_rewards":2299.2796610169494,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":17296,"generate/std_num_tokens":2136.5049278832726,"loss/avg_final_rewards":0.5366211533546448,"loss/avg_raw_advantages":-0.04266875982284546,"loss/avg_raw_advantages_abs":0.08567553758621216,"policy/policy_entropy":0.03967576705326792,"policy/policy_loss":0.00019057253030041466,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.008383233656786615,"policy/raw_grad_norm":0.0908203125,"reward/avg_pass_at_8":0.578125,"reward/avg_raw_reward":0.53662109375,"timing/step":814.6181545020081,"trainer/epoch":0}
+{"step":14,"async/staleness_mean":1.578125,"generate/avg_num_tokens":1745.55078125,"generate/avg_tokens_non_zero_rewards":1616.888030888031,"generate/avg_tokens_zero_rewards":1877.2648221343873,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":16855,"generate/std_num_tokens":1625.8323124968222,"loss/avg_final_rewards":0.5048827528953552,"loss/avg_raw_advantages":-0.009111667983233929,"loss/avg_raw_advantages_abs":0.03608344867825508,"policy/policy_entropy":0.03488813224248588,"policy/policy_loss":0.0001153757903011865,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.004651798823942954,"policy/raw_grad_norm":0.077392578125,"reward/avg_pass_at_8":0.53125,"reward/avg_raw_reward":0.5048828125,"timing/step":790.6542232320644,"trainer/epoch":0}
+{"step":15,"async/staleness_mean":1.890625,"generate/avg_num_tokens":1611.859375,"generate/avg_tokens_non_zero_rewards":1367.7348484848485,"generate/avg_tokens_zero_rewards":1871.733870967742,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":16654,"generate/std_num_tokens":1608.4713943114932,"loss/avg_final_rewards":0.5148437023162842,"loss/avg_raw_advantages":-0.03163832798600197,"loss/avg_raw_advantages_abs":0.10191994160413742,"policy/policy_entropy":0.03233787130739074,"policy/policy_loss":0.00011077359067712678,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.004108040203391283,"policy/raw_grad_norm":0.076904296875,"reward/avg_pass_at_8":0.546875,"reward/avg_raw_reward":0.51484375,"timing/step":780.7512752939947,"trainer/epoch":0,"val/pass_at_1":0.4453125,"val/avg_score":0.441015625,"val/passed":57,"val/turn_cap_rate":0.0859375}
+{"step":16,"async/staleness_mean":1.734375,"generate/avg_num_tokens":1595.240234375,"generate/avg_tokens_non_zero_rewards":1433.0642570281125,"generate/avg_tokens_zero_rewards":1748.783269961977,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":13935,"generate/std_num_tokens":1060.3658543266188,"loss/avg_final_rewards":0.4862304627895355,"loss/avg_raw_advantages":-0.0023246118798851967,"loss/avg_raw_advantages_abs":0.029960589483380318,"policy/policy_entropy":0.03520431253127754,"policy/policy_loss":0.00010947280679829419,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.0019810999947367236,"policy/raw_grad_norm":0.0634765625,"reward/avg_pass_at_8":0.53125,"reward/avg_raw_reward":0.48623046875,"timing/step":773.9093815900851,"trainer/epoch":0}
+{"step":17,"async/staleness_mean":1.734375,"generate/avg_num_tokens":2764.517578125,"generate/avg_tokens_non_zero_rewards":1756.5095785440612,"generate/avg_tokens_zero_rewards":3812.6852589641435,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":18960,"generate/std_num_tokens":3766.739370339553,"loss/avg_final_rewards":0.5052734613418579,"loss/avg_raw_advantages":-0.007282214239239693,"loss/avg_raw_advantages_abs":0.040688592940568924,"policy/policy_entropy":0.0449187800695654,"policy/policy_loss":8.809234077489236e-05,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.003578624970941746,"policy/raw_grad_norm":0.0391845703125,"reward/avg_pass_at_8":0.53125,"reward/avg_raw_reward":0.5052734375,"timing/step":926.391248482978,"trainer/epoch":0}
+{"step":18,"async/staleness_mean":1.703125,"generate/avg_num_tokens":2957.24609375,"generate/avg_tokens_non_zero_rewards":2089.0084388185655,"generate/avg_tokens_zero_rewards":3705.509090909091,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":19754,"generate/std_num_tokens":3817.2444236934994,"loss/avg_final_rewards":0.45839840173721313,"loss/avg_raw_advantages":-0.0050569092854857445,"loss/avg_raw_advantages_abs":0.0917782336473465,"policy/policy_entropy":0.045910545653896406,"policy/policy_loss":0.00015974731468304526,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.002598824180495285,"policy/raw_grad_norm":0.05810546875,"reward/avg_pass_at_8":0.5,"reward/avg_raw_reward":0.45839843750000003,"timing/step":965.5253673281986,"trainer/epoch":0,"val/pass_at_1":0.4296875,"val/avg_score":0.421875,"val/passed":55,"val/turn_cap_rate":0.15625}

viewer/build/inputs/marin/runs/marin-q3c-cal-agent-rloo-lr2/run.json

@@ -0,0 +1,29 @@
+{
+ "id": "marin-q3c-cal-agent-rloo-lr2",
+ "title": "Qwen3-Coder-30B-A3B RLOO on calendar agent (lr 2e-6)",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://huggingface.co/penfever/qwen3coder-calendar-agent-v49-lr2-step12/blob/main/training_logs/finelog.log",
+ "license": "unknown",
+ "model": "penfever/qwen3coder-calendar-agent-v49-lr2-step12",
+ "base_model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
+ "method": "RLOO-N (MarinSkyRL, asynchronous)",
+ "dataset": "open-thoughts/TaskTrove::laion__nemotron-gym-agent-calendar-v2",
+ "eval_suite": "fixed 128-task holdout (Harbor, pass@1 at temperature 0)",
+ "kind": "training",
+ "state": "finished",
+ "started_at": "2026-09-02T21:21:18",
+ "updated_at": "2026-09-07T00:42:30Z",
+ "attempts": 0,
+ "note": "Marin Qwen3-Coder agentic data-source sweep (issue #8942), calendar agent source, recipe: asynchronous shaped RLOO-N, group 8, sequence-mean loss, DAPO disabled, lr 2e-6, max staleness 2. Training metrics are the per-step dicts the run mirrored to its log (1 attempt); val/* is the fixed 128-task holdout at checkpoints, with step 0 = the untrained base model on the same holdout. Marin found the holdout overlaps the training source, so the scores select configurations but are not clean generalization estimates.",
+ "metrics_map": {
+  "reward": "reward/avg_raw_reward",
+  "loss": "policy/policy_loss",
+  "entropy": "policy/policy_entropy",
+  "lr": "policy/policy_lr",
+  "grad_norm": "policy/raw_grad_norm",
+  "response_length": "generate/avg_num_tokens",
+  "eval:holdout_pass@1": "val/pass_at_1"
+ }
+}

viewer/build/inputs/marin/runs/marin-q3c-cal-agent-rloo-lr4/metrics.jsonl

@@ -0,0 +1,15 @@
+{"step":0,"val/pass_at_1":0.34375,"val/avg_score":0.341015625,"val/passed":44,"val/turn_cap_rate":0.0546875}
+{"step":1,"async/staleness_mean":0.0,"generate/avg_num_tokens":3258.30078125,"generate/avg_tokens_non_zero_rewards":3009.4545454545455,"generate/avg_tokens_zero_rewards":3401.483076923077,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":15883,"generate/std_num_tokens":2412.092858973694,"loss/avg_final_rewards":0.36328125,"loss/avg_raw_advantages":-0.01877858117222786,"loss/avg_raw_advantages_abs":0.08647201210260391,"policy/policy_entropy":0.3429131949087605,"policy/policy_loss":0.0003610016692618956,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.02055206497425388,"policy/raw_grad_norm":0.097900390625,"reward/avg_pass_at_8":0.453125,"reward/avg_raw_reward":0.36328125,"timing/step":1314.7745120348409,"trainer/epoch":0}
+{"step":2,"async/staleness_mean":1.0,"generate/avg_num_tokens":3266.564453125,"generate/avg_tokens_non_zero_rewards":3232.5172413793102,"generate/avg_tokens_zero_rewards":3288.9320388349515,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":15455,"generate/std_num_tokens":2303.0115346342263,"loss/avg_final_rewards":0.3946288824081421,"loss/avg_raw_advantages":-0.03380077704787254,"loss/avg_raw_advantages_abs":0.1611316204071045,"policy/policy_entropy":0.1798987213987857,"policy/policy_loss":0.000882684250427701,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.033205747065949254,"policy/raw_grad_norm":0.135986328125,"reward/avg_pass_at_8":0.53125,"reward/avg_raw_reward":0.39462890625,"timing/step":861.3972484134138,"trainer/epoch":0}
+{"step":3,"async/staleness_mean":2.0,"generate/avg_num_tokens":3446.9453125,"generate/avg_tokens_non_zero_rewards":3192.1979695431473,"generate/avg_tokens_zero_rewards":3606.263492063492,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":14918,"generate/std_num_tokens":2464.671014933439,"loss/avg_final_rewards":0.3822265863418579,"loss/avg_raw_advantages":-0.012645360082387924,"loss/avg_raw_advantages_abs":0.15931321680545807,"policy/policy_entropy":0.1437556112650782,"policy/policy_loss":0.0009703533769425121,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.039731398748699576,"policy/raw_grad_norm":0.1630859375,"reward/avg_pass_at_8":0.53125,"reward/avg_raw_reward":0.3822265625,"timing/step":905.4366959445179,"trainer/epoch":0,"val/pass_at_1":0.4453125,"val/avg_score":0.444921875,"val/passed":57,"val/turn_cap_rate":0.0078125}
+{"step":4,"async/staleness_mean":1.984375,"generate/avg_num_tokens":2052.61328125,"generate/avg_tokens_non_zero_rewards":1818.5573770491803,"generate/avg_tokens_zero_rewards":2265.708955223881,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":17062,"generate/std_num_tokens":1431.3124777777766,"loss/avg_final_rewards":0.4754882752895355,"loss/avg_raw_advantages":-0.01460456196218729,"loss/avg_raw_advantages_abs":0.06350195407867432,"policy/policy_entropy":0.06603187214932404,"policy/policy_loss":0.00045276512628333876,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.00948260566292447,"policy/raw_grad_norm":0.07373046875,"reward/avg_pass_at_8":0.53125,"reward/avg_raw_reward":0.47548828125000003,"timing/step":780.4206026559696,"trainer/epoch":0}
+{"step":5,"async/staleness_mean":1.96875,"generate/avg_num_tokens":2265.8828125,"generate/avg_tokens_non_zero_rewards":2036.1479591836735,"generate/avg_tokens_zero_rewards":2408.3765822784812,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":13682,"generate/std_num_tokens":1643.0199953187089,"loss/avg_final_rewards":0.38115233182907104,"loss/avg_raw_advantages":-0.014263740740716457,"loss/avg_raw_advantages_abs":0.05251633748412132,"policy/policy_entropy":0.04388834274141118,"policy/policy_loss":0.00012059892196703004,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.01027956759207882,"policy/raw_grad_norm":0.06280517578125,"reward/avg_pass_at_8":0.40625,"reward/avg_raw_reward":0.38115234375,"timing/step":812.6122711673379,"trainer/epoch":0}
+{"step":6,"async/staleness_mean":1.96875,"generate/avg_num_tokens":1283.65234375,"generate/avg_tokens_non_zero_rewards":1190.7259786476868,"generate/avg_tokens_zero_rewards":1396.6926406926407,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":7890,"generate/std_num_tokens":505.5879083219671,"loss/avg_final_rewards":0.548632800579071,"loss/avg_raw_advantages":-0.0007544859545305371,"loss/avg_raw_advantages_abs":0.0035512458998709917,"policy/policy_entropy":0.02852481407171581,"policy/policy_loss":-1.5519672160735354e-05,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.0009133670209848788,"policy/raw_grad_norm":0.0172119140625,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.5486328125,"timing/step":756.4714208105579,"trainer/epoch":0,"val/pass_at_1":0.4609375,"val/avg_score":0.4609375,"val/passed":59,"val/turn_cap_rate":0}
+{"step":7,"async/staleness_mean":1.984375,"generate/avg_num_tokens":1248.970703125,"generate/avg_tokens_non_zero_rewards":1144.6309963099632,"generate/avg_tokens_zero_rewards":1366.298755186722,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":8793,"generate/std_num_tokens":486.3455773686732,"loss/avg_final_rewards":0.5291992425918579,"loss/avg_raw_advantages":0.0022934735752642155,"loss/avg_raw_advantages_abs":0.01935821957886219,"policy/policy_entropy":0.024887905921787024,"policy/policy_loss":4.0540489862905815e-06,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.0009020073212013813,"policy/raw_grad_norm":0.03582763671875,"reward/avg_pass_at_8":0.546875,"reward/avg_raw_reward":0.52919921875,"timing/step":754.9154837308452,"trainer/epoch":0}
+{"step":8,"async/staleness_mean":1.828125,"generate/avg_num_tokens":1270.369140625,"generate/avg_tokens_non_zero_rewards":1198.4738955823293,"generate/avg_tokens_zero_rewards":1338.4372623574145,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":2780,"generate/std_num_tokens":412.39438807205414,"loss/avg_final_rewards":0.486328125,"loss/avg_raw_advantages":-0.001359981601126492,"loss/avg_raw_advantages_abs":0.030472196638584137,"policy/policy_entropy":0.020796455290110316,"policy/policy_loss":7.066901889629662e-05,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.0013078707961540204,"policy/raw_grad_norm":0.1002197265625,"reward/avg_pass_at_8":0.5,"reward/avg_raw_reward":0.486328125,"timing/step":762.5740822041407,"trainer/epoch":0}
+{"step":9,"async/staleness_mean":1.8125,"generate/avg_num_tokens":1307.755859375,"generate/avg_tokens_non_zero_rewards":1231.0196078431372,"generate/avg_tokens_zero_rewards":1421.7427184466019,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":4242,"generate/std_num_tokens":474.48180715094884,"loss/avg_final_rewards":0.59765625,"loss/avg_raw_advantages":0.0024578634183853865,"loss/avg_raw_advantages_abs":0.024736523628234863,"policy/policy_entropy":0.018660222129256,"policy/policy_loss":5.2358314860612154e-05,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.0003075657068620785,"policy/raw_grad_norm":0.03485107421875,"reward/avg_pass_at_8":0.609375,"reward/avg_raw_reward":0.59765625,"timing/step":764.1869538594037,"trainer/epoch":0,"val/pass_at_1":0.4609375,"val/avg_score":0.4609375,"val/passed":59,"val/turn_cap_rate":0}
+{"step":10,"async/staleness_mean":1.8125,"generate/avg_num_tokens":1315.171875,"generate/avg_tokens_non_zero_rewards":1181.7900355871886,"generate/avg_tokens_zero_rewards":1477.4242424242425,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":5891,"generate/std_num_tokens":481.48725660341665,"loss/avg_final_rewards":0.548828125,"loss/avg_raw_advantages":-0.0004911343567073345,"loss/avg_raw_advantages_abs":0.011280394159257412,"policy/policy_entropy":0.02154614836035762,"policy/policy_loss":3.325467696413398e-05,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.00047591524344170466,"policy/raw_grad_norm":0.02557373046875,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.548828125,"timing/step":747.0789433037862,"trainer/epoch":0}
+{"step":11,"async/staleness_mean":1.984375,"generate/avg_num_tokens":1425.4296875,"generate/avg_tokens_non_zero_rewards":1387.2847222222222,"generate/avg_tokens_zero_rewards":1474.4732142857142,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":3994,"generate/std_num_tokens":470.2107810864212,"loss/avg_final_rewards":0.5624023675918579,"loss/avg_raw_advantages":-0.0012276413617655635,"loss/avg_raw_advantages_abs":0.014891757629811764,"policy/policy_entropy":0.019480210423353128,"policy/policy_loss":0.0001193937359857955,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.0008961566391008091,"policy/raw_grad_norm":0.072509765625,"reward/avg_pass_at_8":0.578125,"reward/avg_raw_reward":0.56240234375,"timing/step":763.7978097638115,"trainer/epoch":0}
+{"step":12,"async/staleness_mean":1.96875,"generate/avg_num_tokens":1408.634765625,"generate/avg_tokens_non_zero_rewards":1316.4825174825176,"generate/avg_tokens_zero_rewards":1525.2522123893805,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":12411,"generate/std_num_tokens":675.449128215424,"loss/avg_final_rewards":0.5584961175918579,"loss/avg_raw_advantages":-0.016240207478404045,"loss/avg_raw_advantages_abs":0.024128973484039307,"policy/policy_entropy":0.018671778838324826,"policy/policy_loss":3.617058973759413e-05,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.0004153201936105688,"policy/raw_grad_norm":0.0103607177734375,"reward/avg_pass_at_8":0.5625,"reward/avg_raw_reward":0.55849609375,"timing/step":744.7515858206898,"trainer/epoch":0,"val/pass_at_1":0.4453125,"val/avg_score":0.44453125,"val/passed":57,"val/turn_cap_rate":0.015625}
+{"step":13,"async/staleness_mean":1.953125,"generate/avg_num_tokens":1548.177734375,"generate/avg_tokens_non_zero_rewards":1449.823275862069,"generate/avg_tokens_zero_rewards":1629.6714285714286,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":12336,"generate/std_num_tokens":711.2155346797952,"loss/avg_final_rewards":0.4530273377895355,"loss/avg_raw_advantages":-0.0006657633348368108,"loss/avg_raw_advantages_abs":0.0008905019494704902,"policy/policy_entropy":0.016578597751504276,"policy/policy_loss":9.77545823843684e-07,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.00014028424195089428,"policy/raw_grad_norm":0.00021600723266601562,"reward/avg_pass_at_8":0.453125,"reward/avg_raw_reward":0.45302734375,"timing/step":753.2786856554449,"trainer/epoch":0}
+{"step":14,"async/staleness_mean":1.9375,"generate/avg_num_tokens":1741.0703125,"generate/avg_tokens_non_zero_rewards":1448.3484162895927,"generate/avg_tokens_zero_rewards":1963.3780068728522,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":16036,"generate/std_num_tokens":1635.1196269003997,"loss/avg_final_rewards":0.4305664002895355,"loss/avg_raw_advantages":-0.03447751700878143,"loss/avg_raw_advantages_abs":0.053240541368722916,"policy/policy_entropy":0.01576114430645248,"policy/policy_loss":8.285757849080255e-05,"policy/policy_lr":3.999999989900971e-06,"policy/ppo_clip_ratio":0.0011577582673680809,"policy/raw_grad_norm":0.0352783203125,"reward/avg_pass_at_8":0.453125,"reward/avg_raw_reward":0.43056640625,"timing/step":817.7870323034003,"trainer/epoch":0}

viewer/build/inputs/marin/runs/marin-q3c-cal-agent-rloo-lr4/run.json

@@ -0,0 +1,29 @@
+{
+ "id": "marin-q3c-cal-agent-rloo-lr4",
+ "title": "Qwen3-Coder-30B-A3B RLOO on calendar agent (lr 4e-6)",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://huggingface.co/penfever/qwen3coder-calendar-agent-v49-lr4-step9/blob/main/training_logs/finelog.log",
+ "license": "unknown",
+ "model": "penfever/qwen3coder-calendar-agent-v49-lr4-step9",
+ "base_model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
+ "method": "RLOO-N (MarinSkyRL, asynchronous)",
+ "dataset": "open-thoughts/TaskTrove::laion__nemotron-gym-agent-calendar-v2",
+ "eval_suite": "fixed 128-task holdout (Harbor, pass@1 at temperature 0)",
+ "kind": "training",
+ "state": "finished",
+ "started_at": "2026-09-02T21:27:18",
+ "updated_at": "2026-09-07T00:42:25Z",
+ "attempts": 0,
+ "note": "Marin Qwen3-Coder agentic data-source sweep (issue #8942), calendar agent source, recipe: asynchronous shaped RLOO-N, group 8, sequence-mean loss, DAPO disabled, lr 4e-6, max staleness 2. Training metrics are the per-step dicts the run mirrored to its log (2 attempts, 2 superseded rows dropped); val/* is the fixed 128-task holdout at checkpoints, with step 0 = the untrained base model on the same holdout. Marin found the holdout overlaps the training source, so the scores select configurations but are not clean generalization estimates.",
+ "metrics_map": {
+  "reward": "reward/avg_raw_reward",
+  "loss": "policy/policy_loss",
+  "entropy": "policy/policy_entropy",
+  "lr": "policy/policy_lr",
+  "grad_norm": "policy/raw_grad_norm",
+  "response_length": "generate/avg_num_tokens",
+  "eval:holdout_pass@1": "val/pass_at_1"
+ }
+}

viewer/build/inputs/marin/runs/marin-q3c-cal-if-rloo-lr2/metrics.jsonl

@@ -0,0 +1,19 @@
+{"step":0,"val/pass_at_1":0.21875,"val/avg_score":0.21875,"val/passed":28,"val/turn_cap_rate":0.0234375}
+{"step":1,"async/staleness_mean":0.0,"generate/avg_num_tokens":2922.529296875,"generate/avg_tokens_non_zero_rewards":2628.1650485436894,"generate/avg_tokens_zero_rewards":2996.6601466992665,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":16976,"generate/std_num_tokens":1699.099749890863,"loss/avg_final_rewards":0.201171875,"loss/avg_raw_advantages":-0.015550758689641953,"loss/avg_raw_advantages_abs":0.11316736787557602,"policy/policy_entropy":0.3233463108772412,"policy/policy_loss":0.00047632897621951997,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.008489559246299905,"policy/raw_grad_norm":0.125244140625,"reward/avg_pass_at_8":0.296875,"reward/avg_raw_reward":0.201171875,"timing/step":1247.5515871401876,"trainer/epoch":0}
+{"step":2,"async/staleness_mean":1.0,"generate/avg_num_tokens":3196.943359375,"generate/avg_tokens_non_zero_rewards":2677.0419161676646,"generate/avg_tokens_zero_rewards":3448.6057971014493,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":23248,"generate/std_num_tokens":2582.2836525111284,"loss/avg_final_rewards":0.326171875,"loss/avg_raw_advantages":-0.05848308652639389,"loss/avg_raw_advantages_abs":0.14437760412693024,"policy/policy_entropy":0.22240215388592333,"policy/policy_loss":0.0006850036152172834,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.014397729029042239,"policy/raw_grad_norm":0.1220703125,"reward/avg_pass_at_8":0.421875,"reward/avg_raw_reward":0.326171875,"timing/step":899.112334751524,"trainer/epoch":0}
+{"step":3,"async/staleness_mean":1.984375,"generate/avg_num_tokens":3433.09375,"generate/avg_tokens_non_zero_rewards":3004.1635220125786,"generate/avg_tokens_zero_rewards":3626.2946175637394,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":15908,"generate/std_num_tokens":2426.7649639859414,"loss/avg_final_rewards":0.310546875,"loss/avg_raw_advantages":-0.027315860614180565,"loss/avg_raw_advantages_abs":0.10836391150951385,"policy/policy_entropy":0.1688442649319768,"policy/policy_loss":0.0007813010888639838,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.018787344419251895,"policy/raw_grad_norm":0.096435546875,"reward/avg_pass_at_8":0.390625,"reward/avg_raw_reward":0.310546875,"timing/step":907.9915293175727,"trainer/epoch":0,"val/pass_at_1":0.296875,"val/avg_score":0.296875,"val/passed":38,"val/turn_cap_rate":0.0234375}
+{"step":4,"async/staleness_mean":1.65625,"generate/avg_num_tokens":2338.025390625,"generate/avg_tokens_non_zero_rewards":2220.559585492228,"generate/avg_tokens_zero_rewards":2409.094043887147,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":16983,"generate/std_num_tokens":1620.7816586148151,"loss/avg_final_rewards":0.376953125,"loss/avg_raw_advantages":-0.00914244819432497,"loss/avg_raw_advantages_abs":0.05165892094373703,"policy/policy_entropy":0.08989189460407943,"policy/policy_loss":0.00025026549701578915,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.004808870648048469,"policy/raw_grad_norm":0.086181640625,"reward/avg_pass_at_8":0.4375,"reward/avg_raw_reward":0.376953125,"timing/step":804.7161148218438,"trainer/epoch":0}
+{"step":5,"async/staleness_mean":1.65625,"generate/avg_num_tokens":2521.55078125,"generate/avg_tokens_non_zero_rewards":1991.0967741935483,"generate/avg_tokens_zero_rewards":2691.077319587629,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":12800,"generate/std_num_tokens":1698.88877093676,"loss/avg_final_rewards":0.2421875,"loss/avg_raw_advantages":-0.0002022742119152099,"loss/avg_raw_advantages_abs":0.007534603122621775,"policy/policy_entropy":0.06430556328268722,"policy/policy_loss":2.057745587080717e-05,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.00040736651135375723,"policy/raw_grad_norm":0.03472900390625,"reward/avg_pass_at_8":0.25,"reward/avg_raw_reward":0.2421875,"timing/step":829.9793928796425,"trainer/epoch":0}
+{"step":6,"async/staleness_mean":1.671875,"generate/avg_num_tokens":2351.5078125,"generate/avg_tokens_non_zero_rewards":2334.445497630332,"generate/avg_tokens_zero_rewards":2363.468438538206,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":13835,"generate/std_num_tokens":1844.949622432538,"loss/avg_final_rewards":0.412109375,"loss/avg_raw_advantages":-0.022126171737909317,"loss/avg_raw_advantages_abs":0.0651857927441597,"policy/policy_entropy":0.056418789899908006,"policy/policy_loss":0.0003275337512604892,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.0050245242255186895,"policy/raw_grad_norm":0.0999755859375,"reward/avg_pass_at_8":0.453125,"reward/avg_raw_reward":0.412109375,"timing/step":814.493153183721,"trainer/epoch":0,"val/pass_at_1":0.3046875,"val/avg_score":0.3046875,"val/passed":39,"val/turn_cap_rate":0.09375}
+{"step":7,"async/staleness_mean":1.96875,"generate/avg_num_tokens":2665.869140625,"generate/avg_tokens_non_zero_rewards":2442.811111111111,"generate/avg_tokens_zero_rewards":2786.8042168674697,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":13313,"generate/std_num_tokens":2343.36020684992,"loss/avg_final_rewards":0.3515625,"loss/avg_raw_advantages":-0.009583254344761372,"loss/avg_raw_advantages_abs":0.03796368092298508,"policy/policy_entropy":0.05372556674410589,"policy/policy_loss":0.00018093397375196218,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.0023538639416074147,"policy/raw_grad_norm":0.0587158203125,"reward/avg_pass_at_8":0.375,"reward/avg_raw_reward":0.3515625,"timing/step":837.664447597228,"trainer/epoch":0}
+{"step":8,"async/staleness_mean":1.96875,"generate/avg_num_tokens":3034.578125,"generate/avg_tokens_non_zero_rewards":2631.346153846154,"generate/avg_tokens_zero_rewards":3211.275280898876,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":17092,"generate/std_num_tokens":2852.41176287545,"loss/avg_final_rewards":0.3046875,"loss/avg_raw_advantages":-0.019987449049949646,"loss/avg_raw_advantages_abs":0.03976084291934967,"policy/policy_entropy":0.049168585654115304,"policy/policy_loss":0.0002641555620357394,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.0022013890393282054,"policy/raw_grad_norm":0.058349609375,"reward/avg_pass_at_8":0.328125,"reward/avg_raw_reward":0.3046875,"timing/step":899.393081064336,"trainer/epoch":0}
+{"step":9,"async/staleness_mean":1.9375,"generate/avg_num_tokens":2632.48828125,"generate/avg_tokens_non_zero_rewards":2642.7224880382773,"generate/avg_tokens_zero_rewards":2625.4290429042903,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":14082,"generate/std_num_tokens":2362.6197613203485,"loss/avg_final_rewards":0.408203125,"loss/avg_raw_advantages":-0.006909183692187071,"loss/avg_raw_advantages_abs":0.04882024601101875,"policy/policy_entropy":0.04640811731223948,"policy/policy_loss":0.0001382353948429227,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.0018512427323003067,"policy/raw_grad_norm":0.044189453125,"reward/avg_pass_at_8":0.4375,"reward/avg_raw_reward":0.408203125,"timing/step":869.2640024060383,"trainer/epoch":0,"val/pass_at_1":0.296875,"val/avg_score":0.296875,"val/passed":38,"val/turn_cap_rate":0.0546875}
+{"step":10,"async/staleness_mean":1.96875,"generate/avg_num_tokens":3102.42578125,"generate/avg_tokens_non_zero_rewards":2488.9593023255816,"generate/avg_tokens_zero_rewards":3412.7676470588235,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":16033,"generate/std_num_tokens":2765.429942690481,"loss/avg_final_rewards":0.3359375,"loss/avg_raw_advantages":-0.013159617781639099,"loss/avg_raw_advantages_abs":0.024752289056777954,"policy/policy_entropy":0.044459557422669604,"policy/policy_loss":9.228897397406399e-05,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.0012809775325877126,"policy/raw_grad_norm":0.030731201171875,"reward/avg_pass_at_8":0.34375,"reward/avg_raw_reward":0.3359375,"timing/step":889.8330702623352,"trainer/epoch":0}
+{"step":11,"async/staleness_mean":1.96875,"generate/avg_num_tokens":3238.13671875,"generate/avg_tokens_non_zero_rewards":2270.303225806452,"generate/avg_tokens_zero_rewards":3658.344537815126,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":19247,"generate/std_num_tokens":3393.55023192828,"loss/avg_final_rewards":0.302734375,"loss/avg_raw_advantages":-0.026760205626487732,"loss/avg_raw_advantages_abs":0.06007860228419304,"policy/policy_entropy":0.04191497477586381,"policy/policy_loss":0.00012307526776567101,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.0017760449400157086,"policy/raw_grad_norm":0.04730224609375,"reward/avg_pass_at_8":0.328125,"reward/avg_raw_reward":0.302734375,"timing/step":947.7256898907945,"trainer/epoch":0}
+{"step":12,"async/staleness_mean":1.984375,"generate/avg_num_tokens":2615.6796875,"generate/avg_tokens_non_zero_rewards":2025.364,"generate/avg_tokens_zero_rewards":3178.9580152671756,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":15463,"generate/std_num_tokens":2653.739819615405,"loss/avg_final_rewards":0.48828125,"loss/avg_raw_advantages":-0.022558946162462234,"loss/avg_raw_advantages_abs":0.06458038836717606,"policy/policy_entropy":0.04119368302053772,"policy/policy_loss":0.00016294774832203984,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.0025851095797406742,"policy/raw_grad_norm":0.0633544921875,"reward/avg_pass_at_8":0.53125,"reward/avg_raw_reward":0.48828125,"timing/step":873.0298640858382,"trainer/epoch":0,"val/pass_at_1":0.296875,"val/avg_score":0.296875,"val/passed":38,"val/turn_cap_rate":0.0703125}
+{"step":13,"async/staleness_mean":1.6875,"generate/avg_num_tokens":2355.857421875,"generate/avg_tokens_non_zero_rewards":1540.9831460674156,"generate/avg_tokens_zero_rewards":2790.131736526946,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":17332,"generate/std_num_tokens":2416.81613040527,"loss/avg_final_rewards":0.34765625,"loss/avg_raw_advantages":-0.001377171603962779,"loss/avg_raw_advantages_abs":0.012774012982845306,"policy/policy_entropy":0.04099735175259411,"policy/policy_loss":0.00016055197920650244,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.0010651398997651995,"policy/raw_grad_norm":0.0892333984375,"reward/avg_pass_at_8":0.359375,"reward/avg_raw_reward":0.34765625,"timing/step":859.3556041838601,"trainer/epoch":0}
+{"step":14,"async/staleness_mean":1.6875,"generate/avg_num_tokens":2572.556640625,"generate/avg_tokens_non_zero_rewards":1660.6912751677853,"generate/avg_tokens_zero_rewards":2946.848484848485,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":15063,"generate/std_num_tokens":2528.232650040209,"loss/avg_final_rewards":0.291015625,"loss/avg_raw_advantages":-0.00018373012426309288,"loss/avg_raw_advantages_abs":0.0015214678132906556,"policy/policy_entropy":0.037931958577246405,"policy/policy_loss":2.402684185653925e-05,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.00042363112515886314,"policy/raw_grad_norm":0.0394287109375,"reward/avg_pass_at_8":0.296875,"reward/avg_raw_reward":0.291015625,"timing/step":877.4724658243358,"trainer/epoch":0}
+{"step":15,"async/staleness_mean":1.703125,"generate/avg_num_tokens":2467.619140625,"generate/avg_tokens_non_zero_rewards":1419.9806451612903,"generate/avg_tokens_zero_rewards":2922.4761904761904,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":20581,"generate/std_num_tokens":2670.6801278289604,"loss/avg_final_rewards":0.302734375,"loss/avg_raw_advantages":-0.0032388262916356325,"loss/avg_raw_advantages_abs":0.016803132370114326,"policy/policy_entropy":0.03846031984721776,"policy/policy_loss":0.0001407616655342281,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.0013565665376518155,"policy/raw_grad_norm":0.056884765625,"reward/avg_pass_at_8":0.328125,"reward/avg_raw_reward":0.302734375,"timing/step":866.9504282530397,"trainer/epoch":0,"val/pass_at_1":0.2890625,"val/avg_score":0.2890625,"val/passed":37,"val/turn_cap_rate":0.125}
+{"step":16,"async/staleness_mean":1.953125,"generate/avg_num_tokens":1988.51171875,"generate/avg_tokens_non_zero_rewards":1306.1990049751244,"generate/avg_tokens_zero_rewards":2429.491961414791,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":16664,"generate/std_num_tokens":2385.2919206540887,"loss/avg_final_rewards":0.392578125,"loss/avg_raw_advantages":-0.020614368841052055,"loss/avg_raw_advantages_abs":0.03734636306762695,"policy/policy_entropy":0.03604317925055511,"policy/policy_loss":0.00011708831880241632,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.0021453153890433896,"policy/raw_grad_norm":0.064208984375,"reward/avg_pass_at_8":0.40625,"reward/avg_raw_reward":0.392578125,"timing/step":829.8454401083291,"trainer/epoch":0}
+{"step":17,"async/staleness_mean":1.953125,"generate/avg_num_tokens":1831.455078125,"generate/avg_tokens_non_zero_rewards":1629.86875,"generate/avg_tokens_zero_rewards":1923.0852272727273,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":13342,"generate/std_num_tokens":1539.3687150747137,"loss/avg_final_rewards":0.3125,"loss/avg_raw_advantages":-0.01566789299249649,"loss/avg_raw_advantages_abs":0.02782127447426319,"policy/policy_entropy":0.03298378288309323,"policy/policy_loss":0.00010075257159769535,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.00047483830030614627,"policy/raw_grad_norm":0.05096435546875,"reward/avg_pass_at_8":0.328125,"reward/avg_raw_reward":0.3125,"timing/step":799.2809429997578,"trainer/epoch":0}
+{"step":18,"async/staleness_mean":1.921875,"generate/avg_num_tokens":1623.966796875,"generate/avg_tokens_non_zero_rewards":1263.1387559808613,"generate/avg_tokens_zero_rewards":1872.854785478548,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":19269,"generate/std_num_tokens":1441.1131605379494,"loss/avg_final_rewards":0.408203125,"loss/avg_raw_advantages":-0.015875305980443954,"loss/avg_raw_advantages_abs":0.02556450478732586,"policy/policy_entropy":0.03677595766203012,"policy/policy_loss":0.0001014711451716721,"policy/policy_lr":1.9999999949504854e-06,"policy/ppo_clip_ratio":0.0012722727988148108,"policy/raw_grad_norm":0.04461669921875,"reward/avg_pass_at_8":0.4375,"reward/avg_raw_reward":0.408203125,"timing/step":783.585354629904,"trainer/epoch":0,"val/pass_at_1":0.3203125,"val/avg_score":0.3203125,"val/passed":41,"val/turn_cap_rate":0.09375}

viewer/build/inputs/marin/runs/marin-q3c-cal-if-rloo-lr2/run.json

@@ -0,0 +1,29 @@
+{
+ "id": "marin-q3c-cal-if-rloo-lr2",
+ "title": "Qwen3-Coder-30B-A3B RLOO on calendar instruction-following (lr 2e-6)",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://huggingface.co/penfever/qwen3coder-calendar-if-v49-lr2-step18/blob/main/training_logs/finelog.log",
+ "license": "unknown",
+ "model": "penfever/qwen3coder-calendar-if-v49-lr2-step18",
+ "base_model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
+ "method": "RLOO-N (MarinSkyRL, asynchronous)",
+ "dataset": "open-thoughts/TaskTrove::laion__nemotron-gym-instruction-following-calendar-v3",
+ "eval_suite": "fixed 128-task holdout (Harbor, pass@1 at temperature 0)",
+ "kind": "training",
+ "state": "finished",
+ "started_at": "2026-09-02T21:21:02",
+ "updated_at": "2026-09-07T01:24:46Z",
+ "attempts": 0,
+ "note": "Marin Qwen3-Coder agentic data-source sweep (issue #8942), calendar instruction-following source, recipe: asynchronous RLOO-N, group 8, sequence-mean loss, no DAPO, no reward shaping, lr 2e-6, max staleness 2. Training metrics are the per-step dicts the run mirrored to its log (1 attempt); val/* is the fixed 128-task holdout at checkpoints, with step 0 = the untrained base model on the same holdout. Marin found the holdout overlaps the training source, so the scores select configurations but are not clean generalization estimates.",
+ "metrics_map": {
+  "reward": "reward/avg_raw_reward",
+  "loss": "policy/policy_loss",
+  "entropy": "policy/policy_entropy",
+  "lr": "policy/policy_lr",
+  "grad_norm": "policy/raw_grad_norm",
+  "response_length": "generate/avg_num_tokens",
+  "eval:holdout_pass@1": "val/pass_at_1"
+ }
+}

viewer/build/inputs/marin/runs/marin-q3c-tt-x10-fsdp2/metrics.jsonl

@@ -0,0 +1,131 @@
+{"step":1,"reward":0.212890625,"avg_pass_at_8":0.3125,"entropy":0.2841771899256855,"grad_norm":0.0657958984375,"tokens":11881.736328125,"tis_log_ratio_mean":0.038364515829016455}
+{"step":2,"reward":0.087890625,"avg_pass_at_8":0.234375,"entropy":0.22218329785391688,"grad_norm":0.0667724609375,"tokens":12495.234375,"tis_log_ratio_mean":0.03849506180267781}
+{"step":3,"reward":0.1328125,"avg_pass_at_8":0.234375,"entropy":0.16953804343938828,"grad_norm":0.060791015625,"tokens":12662.1484375,"tis_log_ratio_mean":0.09063192247413099}
+{"step":4,"reward":0.087890625,"avg_pass_at_8":0.1875,"entropy":0.13060367346042767,"grad_norm":0.03411865234375,"tokens":11763.46875,"tis_log_ratio_mean":0.14461265184218064}
+{"step":5,"reward":0.15625,"avg_pass_at_8":0.28125,"entropy":0.09753880253992975,"grad_norm":0.0433349609375,"tokens":12134.107421875,"tis_log_ratio_mean":0.10195304278749973}
+{"step":6,"reward":0.220703125,"avg_pass_at_8":0.359375,"entropy":0.09256732041831128,"grad_norm":0.0457763671875,"tokens":12589.193359375,"tis_log_ratio_mean":0.10138594143791124}
+{"step":7,"reward":0.19921875,"avg_pass_at_8":0.3125,"entropy":0.07342620013514534,"grad_norm":0.05181884765625,"tokens":12317.18359375,"tis_log_ratio_mean":0.05984293416258879}
+{"step":8,"reward":0.15625,"avg_pass_at_8":0.296875,"entropy":0.0664285022940021,"grad_norm":0.04241943359375,"tokens":12594.4140625,"tis_log_ratio_mean":0.03522976358362939}
+{"step":9,"reward":0.142578125,"avg_pass_at_8":0.28125,"entropy":0.05864239588845521,"grad_norm":0.04180908203125,"tokens":12068.154296875,"tis_log_ratio_mean":0.027953602751949802}
+{"step":10,"reward":0.16796875,"avg_pass_at_8":0.3125,"entropy":0.0568410100240726,"grad_norm":0.04449462890625,"tokens":12847.015625,"tis_log_ratio_mean":0.024628574421512894}
+{"step":11,"reward":0.15234375,"avg_pass_at_8":0.296875,"entropy":0.053056443139212206,"grad_norm":0.0452880859375,"tokens":12536.08203125,"tis_log_ratio_mean":0.022202180218300782}
+{"step":12,"reward":0.12890625,"avg_pass_at_8":0.3125,"entropy":0.051567029440775514,"grad_norm":0.04400634765625,"tokens":12282.0859375,"tis_log_ratio_mean":0.020878274139249697}
+{"step":13,"reward":0.19921875,"avg_pass_at_8":0.375,"entropy":0.048805676226038486,"grad_norm":0.05438232421875,"tokens":12786.90234375,"tis_log_ratio_mean":0.01941614526003832}
+{"step":14,"reward":0.19921875,"avg_pass_at_8":0.296875,"entropy":0.04660979693289846,"grad_norm":0.03485107421875,"tokens":12327.34375,"tis_log_ratio_mean":0.018360236805165187}
+{"step":15,"reward":0.193359375,"avg_pass_at_8":0.34375,"entropy":0.04514222368015908,"grad_norm":0.05419921875,"tokens":12222.05078125,"tis_log_ratio_mean":0.017846548245870508}
+{"step":16,"reward":0.154296875,"avg_pass_at_8":0.28125,"entropy":0.042855857784161344,"grad_norm":0.04254150390625,"tokens":13332.435546875,"tis_log_ratio_mean":0.016569633662584238}
+{"step":17,"reward":0.25,"avg_pass_at_8":0.375,"entropy":0.04131283948663622,"grad_norm":0.0418701171875,"tokens":12548.912109375,"tis_log_ratio_mean":0.016559631236304995}
+{"step":18,"reward":0.203125,"avg_pass_at_8":0.328125,"entropy":0.03891567382379435,"grad_norm":0.05523681640625,"tokens":12550.705078125,"tis_log_ratio_mean":0.015734463297121692}
+{"step":19,"reward":0.134765625,"avg_pass_at_8":0.296875,"entropy":0.04044313328631688,"grad_norm":0.0438232421875,"tokens":12908.900390625,"tis_log_ratio_mean":0.015913555289444048}
+{"step":20,"reward":0.1796875,"avg_pass_at_8":0.3125,"entropy":0.03743738950288389,"grad_norm":0.03448486328125,"tokens":13096.10546875,"tis_log_ratio_mean":0.015082018857356161}
+{"step":21,"reward":0.2421875,"avg_pass_at_8":0.328125,"entropy":0.03658617738983594,"grad_norm":0.03643798828125,"tokens":12645.8125,"tis_log_ratio_mean":0.01441338995937258}
+{"step":22,"reward":0.158203125,"avg_pass_at_8":0.359375,"entropy":0.03569843286823016,"grad_norm":0.04046630859375,"tokens":12698.052734375,"tis_log_ratio_mean":0.014125670662906487}
+{"step":23,"reward":0.193359375,"avg_pass_at_8":0.359375,"entropy":0.03394435811787844,"grad_norm":0.04620361328125,"tokens":12710.998046875,"tis_log_ratio_mean":0.0132341858552536}
+{"step":24,"reward":0.158203125,"avg_pass_at_8":0.25,"entropy":0.03395247203297913,"grad_norm":0.028564453125,"tokens":12332.021484375,"tis_log_ratio_mean":0.01289007104060147}
+{"step":25,"reward":0.16015625,"avg_pass_at_8":0.265625,"entropy":0.03358004772599088,"grad_norm":0.03521728515625,"tokens":12002.28125,"tis_log_ratio_mean":0.012765458937792573}
+{"step":26,"reward":0.138671875,"avg_pass_at_8":0.28125,"entropy":0.033016397661413066,"grad_norm":0.03424072265625,"tokens":12812.439453125,"tis_log_ratio_mean":0.012117673104512505}
+{"step":27,"reward":0.228515625,"avg_pass_at_8":0.359375,"entropy":0.031657723418902606,"grad_norm":0.0445556640625,"tokens":12256.541015625,"tis_log_ratio_mean":0.011358821244357387}
+{"step":28,"reward":0.1796875,"avg_pass_at_8":0.3125,"entropy":0.03244724746036809,"grad_norm":0.0372314453125,"tokens":12348.115234375,"tis_log_ratio_mean":0.01094988789918716}
+{"step":29,"reward":0.19140625,"avg_pass_at_8":0.328125,"entropy":0.03243779667536728,"grad_norm":0.0343017578125,"tokens":13087.103515625,"tis_log_ratio_mean":0.011190733315743273}
+{"step":30,"reward":0.208984375,"avg_pass_at_8":0.3125,"entropy":0.02962814005149994,"grad_norm":0.0364990234375,"tokens":12577.455078125,"tis_log_ratio_mean":0.010096439207700314}
+{"step":31,"reward":0.20703125,"avg_pass_at_8":0.328125,"entropy":0.0309488231287105,"grad_norm":0.041259765625,"tokens":12363.822265625,"tis_log_ratio_mean":0.010903368882281939}
+{"step":32,"reward":0.181640625,"avg_pass_at_8":0.265625,"entropy":0.02925875797518529,"grad_norm":0.02789306640625,"tokens":12903.921875,"tis_log_ratio_mean":0.01017761023103958}
+{"step":33,"reward":0.203125,"avg_pass_at_8":0.34375,"entropy":0.030067210784181952,"grad_norm":0.0631103515625,"tokens":12612.9140625,"tis_log_ratio_mean":0.010808022936544148}
+{"step":34,"reward":0.1640625,"avg_pass_at_8":0.296875,"entropy":0.02850124496035278,"grad_norm":0.03546142578125,"tokens":13280.998046875,"tis_log_ratio_mean":0.010313762260921067}
+{"step":35,"reward":0.208984375,"avg_pass_at_8":0.34375,"entropy":0.03092884058423806,"grad_norm":0.03924560546875,"tokens":12773.3984375,"tis_log_ratio_mean":0.010966236462991219}
+{"step":36,"reward":0.26953125,"avg_pass_at_8":0.359375,"entropy":0.032130426552612334,"grad_norm":0.03912353515625,"tokens":13144.921875,"tis_log_ratio_mean":0.011368247018253896}
+{"step":37,"reward":0.205078125,"avg_pass_at_8":0.296875,"entropy":0.02857135566591751,"grad_norm":0.03680419921875,"tokens":12472.890625,"tis_log_ratio_mean":0.010153336690564174}
+{"step":38,"reward":0.3046875,"avg_pass_at_8":0.421875,"entropy":0.030453038096311502,"grad_norm":0.0467529296875,"tokens":12788.86328125,"tis_log_ratio_mean":0.010748489661636995}
+{"step":39,"reward":0.25390625,"avg_pass_at_8":0.328125,"entropy":0.030632267269538715,"grad_norm":0.037109375,"tokens":12523.921875,"tis_log_ratio_mean":0.011051126886741258}
+{"step":40,"reward":0.1953125,"avg_pass_at_8":0.296875,"entropy":0.030233249824959785,"grad_norm":0.03253173828125,"tokens":13367.421875,"tis_log_ratio_mean":0.010567939076281618}
+{"step":41,"reward":0.279296875,"avg_pass_at_8":0.4375,"entropy":0.02989778813207522,"grad_norm":0.03314208984375,"tokens":12535.595703125,"tis_log_ratio_mean":0.010793829402246047}
+{"step":42,"reward":0.193359375,"avg_pass_at_8":0.34375,"entropy":0.02982971494202502,"grad_norm":0.03607177734375,"tokens":12638.52734375,"tis_log_ratio_mean":0.010689664333767723}
+{"step":43,"reward":0.228515625,"avg_pass_at_8":0.328125,"entropy":0.03052262075652834,"grad_norm":0.03955078125,"tokens":12657.53125,"tis_log_ratio_mean":0.010411221523099812}
+{"step":44,"reward":0.1484375,"avg_pass_at_8":0.265625,"entropy":0.029005808028159663,"grad_norm":0.03387451171875,"tokens":12091.994140625,"tis_log_ratio_mean":0.01026588813692797}
+{"step":45,"reward":0.20703125,"avg_pass_at_8":0.296875,"entropy":0.028449535981053486,"grad_norm":0.027496337890625,"tokens":13019.103515625,"tis_log_ratio_mean":0.009740504043293186}
+{"step":46,"reward":0.203125,"avg_pass_at_8":0.296875,"entropy":0.03021659243677277,"grad_norm":0.0467529296875,"tokens":13155.5078125,"tis_log_ratio_mean":0.010293671719409758}
+{"step":47,"reward":0.240234375,"avg_pass_at_8":0.390625,"entropy":0.030966883699875325,"grad_norm":0.03729248046875,"tokens":12550.833984375,"tis_log_ratio_mean":0.010248874641547445}
+{"step":48,"reward":0.220703125,"avg_pass_at_8":0.390625,"entropy":0.02938420452119317,"grad_norm":0.044677734375,"tokens":13721.046875,"tis_log_ratio_mean":0.009808252561924746}
+{"step":49,"reward":0.259765625,"avg_pass_at_8":0.40625,"entropy":0.027371705378754996,"grad_norm":0.030853271484375,"tokens":12779.29296875,"tis_log_ratio_mean":0.009135908836469753}
+{"step":50,"reward":0.201171875,"avg_pass_at_8":0.375,"entropy":0.031289247184759006,"grad_norm":0.03594970703125,"tokens":13823.228515625,"tis_log_ratio_mean":0.010198387328273384}
+{"step":51,"reward":0.294921875,"avg_pass_at_8":0.421875,"entropy":0.028124889126047492,"grad_norm":0.04296875,"tokens":13054.509765625,"tis_log_ratio_mean":0.009511162501439685}
+{"step":52,"reward":0.181640625,"avg_pass_at_8":0.265625,"entropy":0.028007985587464646,"grad_norm":0.0322265625,"tokens":13133.361328125,"tis_log_ratio_mean":0.009548205816827249}
+{"step":53,"reward":0.22265625,"avg_pass_at_8":0.375,"entropy":0.029032153950538486,"grad_norm":0.03997802734375,"tokens":12777.341796875,"tis_log_ratio_mean":0.009956562164006755}
+{"step":54,"reward":0.2421875,"avg_pass_at_8":0.34375,"entropy":0.02751605203957297,"grad_norm":0.0350341796875,"tokens":13278.8359375,"tis_log_ratio_mean":0.009461242421821225}
+{"step":55,"reward":0.287109375,"avg_pass_at_8":0.421875,"entropy":0.027387330548663158,"grad_norm":0.0325927734375,"tokens":13895.283203125,"tis_log_ratio_mean":0.009503483528533252}
+{"step":56,"reward":0.169921875,"avg_pass_at_8":0.328125,"entropy":0.02758093578449916,"grad_norm":0.035888671875,"tokens":12757.833984375,"tis_log_ratio_mean":0.0096796070538403}
+{"step":57,"reward":0.205078125,"avg_pass_at_8":0.328125,"entropy":0.027183092141058296,"grad_norm":0.033203125,"tokens":12691.109375,"tis_log_ratio_mean":0.009581870112015167}
+{"step":58,"reward":0.2109375,"avg_pass_at_8":0.296875,"entropy":0.027204874844755977,"grad_norm":0.03155517578125,"tokens":12914.486328125,"tis_log_ratio_mean":0.009408349818841089}
+{"step":59,"reward":0.16015625,"avg_pass_at_8":0.265625,"entropy":0.026098899441421963,"grad_norm":0.022918701171875,"tokens":12813.265625,"tis_log_ratio_mean":0.009095330358832143}
+{"step":60,"reward":0.23828125,"avg_pass_at_8":0.359375,"entropy":0.026620148346410133,"grad_norm":0.032958984375,"tokens":12866.810546875,"tis_log_ratio_mean":0.00890859115315834}
+{"step":61,"reward":0.169921875,"avg_pass_at_8":0.28125,"entropy":0.027887248012120835,"grad_norm":0.030975341796875,"tokens":11839.19140625,"tis_log_ratio_mean":0.009386354246089468}
+{"step":62,"reward":0.259765625,"avg_pass_at_8":0.390625,"entropy":0.027947976093855686,"grad_norm":0.04302978515625,"tokens":11692.138671875,"tis_log_ratio_mean":0.009609241533325985}
+{"step":63,"reward":0.201171875,"avg_pass_at_8":0.3125,"entropy":0.02903481676185038,"grad_norm":0.02935791015625,"tokens":11817.130859375,"tis_log_ratio_mean":0.009614631126169115}
+{"step":64,"reward":0.189453125,"avg_pass_at_8":0.34375,"entropy":0.027791766842710786,"grad_norm":0.0382080078125,"tokens":10968.287109375,"tis_log_ratio_mean":0.00962878475911566}
+{"step":65,"reward":0.259765625,"avg_pass_at_8":0.375,"entropy":0.029161494036088698,"grad_norm":0.029510498046875,"tokens":11231.85546875,"tis_log_ratio_mean":0.009696890971099492}
+{"step":66,"reward":0.197265625,"avg_pass_at_8":0.328125,"entropy":0.029056368875899352,"grad_norm":0.05419921875,"tokens":10758.220703125,"tis_log_ratio_mean":0.010006335822254186}
+{"step":67,"reward":0.275390625,"avg_pass_at_8":0.40625,"entropy":0.029272589934407733,"grad_norm":0.0487060546875,"tokens":11061.978515625,"tis_log_ratio_mean":0.010036646322987508}
+{"step":68,"reward":0.26953125,"avg_pass_at_8":0.46875,"entropy":0.029200913282693364,"grad_norm":0.04052734375,"tokens":11505.048828125,"tis_log_ratio_mean":0.01006036519902409}
+{"step":69,"reward":0.22265625,"avg_pass_at_8":0.28125,"entropy":0.027148673427291214,"grad_norm":0.024871826171875,"tokens":11390.96875,"tis_log_ratio_mean":0.009945427700586151}
+{"step":70,"reward":0.171875,"avg_pass_at_8":0.25,"entropy":0.02843844138260465,"grad_norm":0.03619384765625,"tokens":10790.607421875,"tis_log_ratio_mean":0.010516655500396155}
+{"step":71,"reward":0.28515625,"avg_pass_at_8":0.375,"entropy":0.0306850428605685,"grad_norm":0.04302978515625,"tokens":10056.451171875,"tis_log_ratio_mean":0.010788282568682916}
+{"step":72,"reward":0.232421875,"avg_pass_at_8":0.328125,"entropy":0.02988124210969545,"grad_norm":0.0428466796875,"tokens":10550.240234375,"tis_log_ratio_mean":0.010693700241972692}
+{"step":73,"reward":0.232421875,"avg_pass_at_8":0.34375,"entropy":0.031069430653587915,"grad_norm":0.076416015625,"tokens":8945.734375,"tis_log_ratio_mean":0.010977997284498997}
+{"step":74,"reward":0.236328125,"avg_pass_at_8":0.34375,"entropy":0.030875944896251895,"grad_norm":0.05029296875,"tokens":8423.21875,"tis_log_ratio_mean":0.011183002883626614}
+{"step":75,"reward":0.24609375,"avg_pass_at_8":0.359375,"entropy":0.03141133471217472,"grad_norm":0.03704833984375,"tokens":9757.416015625,"tis_log_ratio_mean":0.011101966993010137}
+{"step":76,"reward":0.15625,"avg_pass_at_8":0.3125,"entropy":0.03016999464307446,"grad_norm":0.051513671875,"tokens":9065.568359375,"tis_log_ratio_mean":0.009808475118916249}
+{"step":77,"reward":0.21875,"avg_pass_at_8":0.359375,"entropy":0.030204133305232972,"grad_norm":0.0450439453125,"tokens":8089.38671875,"tis_log_ratio_mean":0.009717524277220946}
+{"step":78,"reward":0.244140625,"avg_pass_at_8":0.375,"entropy":0.03067694982746616,"grad_norm":0.0452880859375,"tokens":8266.53125,"tis_log_ratio_mean":0.009917856954416493}
+{"step":79,"reward":0.154296875,"avg_pass_at_8":0.25,"entropy":0.029473103117197752,"grad_norm":0.04559326171875,"tokens":8943.947265625,"tis_log_ratio_mean":0.009989923401008127}
+{"step":80,"reward":0.17578125,"avg_pass_at_8":0.234375,"entropy":0.030776130486628972,"grad_norm":0.056396484375,"tokens":8618.27734375,"tis_log_ratio_mean":0.010737046046415344}
+{"step":81,"reward":0.16796875,"avg_pass_at_8":0.25,"entropy":0.030294847732875496,"grad_norm":0.04229736328125,"tokens":8587.7890625,"tis_log_ratio_mean":0.011067064486269373}
+{"step":82,"reward":0.294921875,"avg_pass_at_8":0.421875,"entropy":0.030869852038449608,"grad_norm":0.048583984375,"tokens":9556.98828125,"tis_log_ratio_mean":0.011962324206251651}
+{"step":83,"reward":0.302734375,"avg_pass_at_8":0.421875,"entropy":0.02902125303808134,"grad_norm":0.04351806640625,"tokens":9309.201171875,"tis_log_ratio_mean":0.01167204885132378}
+{"step":84,"reward":0.220703125,"avg_pass_at_8":0.359375,"entropy":0.03206025120744016,"grad_norm":0.0452880859375,"tokens":8866.61328125,"tis_log_ratio_mean":0.012416040139214601}
+{"step":85,"reward":0.20703125,"avg_pass_at_8":0.328125,"entropy":0.02951743867015466,"grad_norm":0.04656982421875,"tokens":8920.86328125,"tis_log_ratio_mean":0.01213949977682205}
+{"step":86,"reward":0.1953125,"avg_pass_at_8":0.34375,"entropy":0.030636480951216072,"grad_norm":0.049560546875,"tokens":8653.85546875,"tis_log_ratio_mean":0.012725080690870527}
+{"step":87,"reward":0.220703125,"avg_pass_at_8":0.3125,"entropy":0.028880212877993472,"grad_norm":0.04705810546875,"tokens":8024.61328125,"tis_log_ratio_mean":0.00992833813870675}
+{"step":88,"reward":0.193359375,"avg_pass_at_8":0.3125,"entropy":0.030465519274002872,"grad_norm":0.0540771484375,"tokens":8904.87109375,"tis_log_ratio_mean":0.011851582581584807}
+{"step":89,"reward":0.154296875,"avg_pass_at_8":0.25,"entropy":0.030161284681526013,"grad_norm":0.04217529296875,"tokens":8971.01171875,"tis_log_ratio_mean":0.012485835279221646}
+{"step":90,"reward":0.224609375,"avg_pass_at_8":0.3125,"entropy":0.031217851268593222,"grad_norm":0.03680419921875,"tokens":9764.037109375,"tis_log_ratio_mean":0.013571985386079177}
+{"step":91,"reward":0.283203125,"avg_pass_at_8":0.40625,"entropy":0.03072588321811054,"grad_norm":0.04400634765625,"tokens":8531.015625,"tis_log_ratio_mean":0.013256323050882202}
+{"step":92,"reward":0.12109375,"avg_pass_at_8":0.21875,"entropy":0.030266602989286184,"grad_norm":0.04656982421875,"tokens":9535.240234375,"tis_log_ratio_mean":0.013386551494477317}
+{"step":93,"reward":0.236328125,"avg_pass_at_8":0.359375,"entropy":0.03157717621070333,"grad_norm":0.03814697265625,"tokens":8697.197265625,"tis_log_ratio_mean":0.01356871890311595}
+{"step":94,"reward":0.205078125,"avg_pass_at_8":0.296875,"entropy":0.028351451794151217,"grad_norm":0.03875732421875,"tokens":10050.4296875,"tis_log_ratio_mean":0.012061884459399153}
+{"step":95,"reward":0.28125,"avg_pass_at_8":0.375,"entropy":0.029586737306090072,"grad_norm":0.0404052734375,"tokens":9361.25,"tis_log_ratio_mean":0.012686615977145266}
+{"step":96,"reward":0.26171875,"avg_pass_at_8":0.375,"entropy":0.028367519276798703,"grad_norm":0.040771484375,"tokens":8762.1484375,"tis_log_ratio_mean":0.012179035584267695}
+{"step":97,"reward":0.142578125,"avg_pass_at_8":0.3125,"entropy":0.028749061428243294,"grad_norm":0.051513671875,"tokens":8839.486328125,"tis_log_ratio_mean":0.012086589234968415}
+{"step":98,"reward":0.25390625,"avg_pass_at_8":0.40625,"entropy":0.027298774002701975,"grad_norm":0.04180908203125,"tokens":9277.3125,"tis_log_ratio_mean":0.01207058996806154}
+{"step":99,"reward":0.1953125,"avg_pass_at_8":0.3125,"entropy":0.02528718973917421,"grad_norm":0.03314208984375,"tokens":9087.712890625,"tis_log_ratio_mean":0.012222951369039947}
+{"step":100,"reward":0.212890625,"avg_pass_at_8":0.375,"entropy":0.02481368707231013,"grad_norm":0.04888916015625,"tokens":9057.140625,"tis_log_ratio_mean":0.012854296277510002}
+{"step":101,"reward":0.22265625,"avg_pass_at_8":0.375,"entropy":0.024845084742992185,"grad_norm":0.03515625,"tokens":8838.267578125,"tis_log_ratio_mean":0.012943789715791354}
+{"step":102,"reward":0.1796875,"avg_pass_at_8":0.328125,"entropy":0.023100286729459185,"grad_norm":0.03948974609375,"tokens":8778.7890625,"tis_log_ratio_mean":0.01172922586920322}
+{"step":103,"reward":0.1484375,"avg_pass_at_8":0.25,"entropy":0.02291044098819839,"grad_norm":0.0291748046875,"tokens":9222.0625,"tis_log_ratio_mean":0.010485118804353988}
+{"step":104,"reward":0.185546875,"avg_pass_at_8":0.34375,"entropy":0.021337949663575273,"grad_norm":0.03851318359375,"tokens":9168.283203125,"tis_log_ratio_mean":0.00943335782721988}
+{"step":105,"reward":0.1875,"avg_pass_at_8":0.34375,"entropy":0.020512918639724376,"grad_norm":0.03253173828125,"tokens":8979.609375,"tis_log_ratio_mean":0.008232188210968161}
+{"step":106,"reward":0.15234375,"avg_pass_at_8":0.25,"entropy":0.020784744549018797,"grad_norm":0.04254150390625,"tokens":9041.8828125,"tis_log_ratio_mean":0.007534047388617182}
+{"step":107,"reward":0.212890625,"avg_pass_at_8":0.375,"entropy":0.020715639744594228,"grad_norm":0.03704833984375,"tokens":9587.263671875,"tis_log_ratio_mean":0.007625855079822941}
+{"step":108,"reward":0.1953125,"avg_pass_at_8":0.328125,"entropy":0.02319042348972289,"grad_norm":0.03924560546875,"tokens":9752.072265625,"tis_log_ratio_mean":0.008176897834346164}
+{"step":109,"reward":0.244140625,"avg_pass_at_8":0.375,"entropy":0.02238455408223672,"grad_norm":0.0357666015625,"tokens":9610.7421875,"tis_log_ratio_mean":0.008344108191522537}
+{"step":110,"reward":0.19921875,"avg_pass_at_8":0.359375,"entropy":0.02211651918332791,"grad_norm":0.04931640625,"tokens":10630.197265625,"tis_log_ratio_mean":0.008396779932809295}
+{"step":111,"reward":0.21484375,"avg_pass_at_8":0.328125,"entropy":0.023869622134952806,"grad_norm":0.04449462890625,"tokens":9603.0234375,"tis_log_ratio_mean":0.009957528061931953}
+{"step":112,"reward":0.173828125,"avg_pass_at_8":0.265625,"entropy":0.02290310490934644,"grad_norm":0.034423828125,"tokens":10343.5,"tis_log_ratio_mean":0.009010585992655251}
+{"step":113,"reward":0.18359375,"avg_pass_at_8":0.296875,"entropy":0.02170805272180587,"grad_norm":0.03399658203125,"tokens":10329.326171875,"tis_log_ratio_mean":0.00844007851992501}
+{"step":114,"reward":0.31640625,"avg_pass_at_8":0.421875,"entropy":0.022855756091303192,"grad_norm":0.0362548828125,"tokens":10306.609375,"tis_log_ratio_mean":0.008500853640725836}
+{"step":115,"reward":0.1171875,"avg_pass_at_8":0.265625,"entropy":0.0217873677611351,"grad_norm":0.06939697265625,"tokens":10864.84375,"tis_log_ratio_mean":0.008366413821931928}
+{"step":116,"reward":0.33203125,"avg_pass_at_8":0.421875,"entropy":0.021444061640067957,"grad_norm":0.030609130859375,"tokens":9638.8046875,"tis_log_ratio_mean":0.008688700883794809}
+{"step":117,"reward":0.279296875,"avg_pass_at_8":0.390625,"entropy":0.021509501690161414,"grad_norm":0.04095458984375,"tokens":10569.3359375,"tis_log_ratio_mean":0.009384592554852134}
+{"step":118,"reward":0.21484375,"avg_pass_at_8":0.328125,"entropy":0.020106458068767097,"grad_norm":0.030548095703125,"tokens":9987.4140625,"tis_log_ratio_mean":0.009645613168686396}
+{"step":119,"reward":0.271484375,"avg_pass_at_8":0.40625,"entropy":0.019727750004676636,"grad_norm":0.031005859375,"tokens":11005.328125,"tis_log_ratio_mean":0.009348023195343558}
+{"step":120,"reward":0.20703125,"avg_pass_at_8":0.359375,"entropy":0.018726605470874347,"grad_norm":0.0355224609375,"tokens":11316.013671875,"tis_log_ratio_mean":0.0090948855431634}
+{"step":121,"reward":0.171875,"avg_pass_at_8":0.25,"entropy":0.018549661894212477,"grad_norm":0.023162841796875,"tokens":11738.21875,"tis_log_ratio_mean":0.00890883397005382}
+{"step":122,"reward":0.208984375,"avg_pass_at_8":0.296875,"entropy":0.017353089358948637,"grad_norm":0.022674560546875,"tokens":11436.73046875,"tis_log_ratio_mean":0.008552061837690417}
+{"step":123,"reward":0.18359375,"avg_pass_at_8":0.3125,"entropy":0.016328347330272663,"grad_norm":0.028900146484375,"tokens":11837.998046875,"tis_log_ratio_mean":0.007505549854613491}
+{"step":124,"reward":0.171875,"avg_pass_at_8":0.265625,"entropy":0.01668986696313368,"grad_norm":0.023895263671875,"tokens":12577.373046875,"tis_log_ratio_mean":0.007292221354873618}
+{"step":125,"reward":0.212890625,"avg_pass_at_8":0.359375,"entropy":0.01560962187795667,"grad_norm":0.02288818359375,"tokens":12170.55078125,"tis_log_ratio_mean":0.0067350550343689974}
+{"step":126,"reward":0.248046875,"avg_pass_at_8":0.40625,"entropy":0.015482696348044556,"grad_norm":0.027252197265625,"tokens":12163.478515625,"tis_log_ratio_mean":0.006610192500374978}
+{"step":127,"reward":0.181640625,"avg_pass_at_8":0.328125,"entropy":0.014299407914222684,"grad_norm":0.02288818359375,"tokens":12578.37109375,"tis_log_ratio_mean":0.005982442955428269}
+{"step":128,"reward":0.166015625,"avg_pass_at_8":0.375,"entropy":0.014562470947566908,"grad_norm":0.0274658203125,"tokens":12808.64453125,"tis_log_ratio_mean":0.00599842362862546}
+{"step":129,"reward":0.21484375,"avg_pass_at_8":0.40625,"entropy":0.013737520610447973,"grad_norm":0.03387451171875,"tokens":12796.728515625,"tis_log_ratio_mean":0.005632446040181094}
+{"step":130,"reward":0.146484375,"avg_pass_at_8":0.25,"entropy":0.013513566274923505,"grad_norm":0.02020263671875,"tokens":12773.41015625,"tis_log_ratio_mean":0.00586021634444478}
+{"step":131,"reward":0.166015625,"avg_pass_at_8":0.296875,"entropy":0.012879661524493713,"grad_norm":0.0299072265625,"tokens":13502.77734375,"tis_log_ratio_mean":0.0059986837823089445}

viewer/build/inputs/marin/runs/marin-q3c-tt-x10-fsdp2/run.json

@@ -0,0 +1,26 @@
+{
+ "id": "marin-q3c-tt-x10-fsdp2",
+ "title": "TaskTrove RL, FSDP2 backend (Qwen3-Coder-30B-A3B)",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://huggingface.co/laion/tt-x10-fsdp2-fa2-117-30B/tree/main/training_logs",
+ "license": "apache-2.0",
+ "model": "laion/tt-x10-fsdp2-fa2-117-30B",
+ "base_model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
+ "method": "GRPO (SkyRL + Terminus-2, pass-ratio shaped verifier reward)",
+ "dataset": "DCAgent/exp_rpt_multifile",
+ "eval_suite": null,
+ "kind": "training",
+ "state": "finished",
+ "started_at": null,
+ "updated_at": "2026-09-04T16:22:37Z",
+ "attempts": 0,
+ "note": "Marin TaskTrove RL hyperparameter ablation (issue #7785) on Qwen3-Coder-30B-A3B-Instruct with DCAgent/exp_rpt_multifile tasks and the Terminus-2 harness; arm: X10b training backend: FSDP2 + FlashAttention 2, lr 8e-6, temperature 1.2. Our copy is the per-step curve Marin stitched from W&B (its trailing-EMA column left out).",
+ "metrics_map": {
+  "reward": "reward",
+  "entropy": "entropy",
+  "grad_norm": "grad_norm",
+  "response_length": "tokens"
+ }
+}

viewer/build/inputs/marin/runs/marin-q3c-tt-x15-megatron/metrics.jsonl

@@ -0,0 +1,101 @@
+{"step":1,"reward":0.208984375,"avg_pass_at_8":0.40625,"entropy":0.28380009054671973,"grad_norm":0.20195640623569489,"tokens":11957.6953125,"tis_log_ratio_mean":0.03879948745452566}
+{"step":2,"reward":0.115234375,"avg_pass_at_8":0.265625,"entropy":0.18324961440521292,"grad_norm":0.1760411113500595,"tokens":12894.6796875,"tis_log_ratio_mean":0.08047457538486924}
+{"step":3,"reward":0.140625,"avg_pass_at_8":0.28125,"entropy":0.15693244856083766,"grad_norm":0.0973483994603157,"tokens":12006.486328125,"tis_log_ratio_mean":0.040257213353470434}
+{"step":4,"reward":0.13671875,"avg_pass_at_8":0.234375,"entropy":0.13055996870389208,"grad_norm":0.08374636620283127,"tokens":11356.103515625,"tis_log_ratio_mean":0.031027401058963733}
+{"step":5,"reward":0.12109375,"avg_pass_at_8":0.171875,"entropy":0.10914250410860404,"grad_norm":0.0662873163819313,"tokens":10752.634765625,"tis_log_ratio_mean":0.026213129080133513}
+{"step":6,"reward":0.201171875,"avg_pass_at_8":0.359375,"entropy":0.0992232505523134,"grad_norm":0.055875904858112335,"tokens":10838.822265625,"tis_log_ratio_mean":0.040491714818926994}
+{"step":7,"reward":0.220703125,"avg_pass_at_8":0.359375,"entropy":0.08605229249224067,"grad_norm":0.052110519260168076,"tokens":10235.841796875,"tis_log_ratio_mean":0.03386010333633749}
+{"step":8,"reward":0.150390625,"avg_pass_at_8":0.34375,"entropy":0.0795121548435418,"grad_norm":0.0495506152510643,"tokens":10783.67578125,"tis_log_ratio_mean":0.02351990335591836}
+{"step":9,"reward":0.158203125,"avg_pass_at_8":0.265625,"entropy":0.06820493974373676,"grad_norm":0.039372142404317856,"tokens":11311.109375,"tis_log_ratio_mean":0.024784139131952543}
+{"step":10,"reward":0.189453125,"avg_pass_at_8":0.296875,"entropy":0.06720394286094233,"grad_norm":0.032950326800346375,"tokens":10775.3671875,"tis_log_ratio_mean":0.02816865162094473}
+{"step":11,"reward":0.138671875,"avg_pass_at_8":0.234375,"entropy":0.059770767868030816,"grad_norm":0.03556487336754799,"tokens":10556.32421875,"tis_log_ratio_mean":0.023838659817556618}
+{"step":12,"reward":0.1328125,"avg_pass_at_8":0.25,"entropy":0.057798539637587965,"grad_norm":0.030953196808695793,"tokens":10504.240234375,"tis_log_ratio_mean":0.028407111112755956}
+{"step":13,"reward":0.1640625,"avg_pass_at_8":0.328125,"entropy":0.05396210770413745,"grad_norm":0.038232702761888504,"tokens":10968.626953125,"tis_log_ratio_mean":0.02425438290083548}
+{"step":14,"reward":0.171875,"avg_pass_at_8":0.328125,"entropy":0.05111169814335881,"grad_norm":0.04912637919187546,"tokens":10553.677734375,"tis_log_ratio_mean":0.01996984917059308}
+{"step":15,"reward":0.119140625,"avg_pass_at_8":0.25,"entropy":0.04949884647066938,"grad_norm":0.029686804860830307,"tokens":10711.8125,"tis_log_ratio_mean":0.022283174423137098}
+{"step":16,"reward":0.142578125,"avg_pass_at_8":0.25,"entropy":0.0478212947564316,"grad_norm":0.03400680795311928,"tokens":12657.220703125,"tis_log_ratio_mean":0.021065440352685982}
+{"step":17,"reward":0.232421875,"avg_pass_at_8":0.421875,"entropy":0.047510785767372,"grad_norm":0.0400405079126358,"tokens":11585.21484375,"tis_log_ratio_mean":0.01808502074527496}
+{"step":18,"reward":0.177734375,"avg_pass_at_8":0.328125,"entropy":0.046791273438429926,"grad_norm":0.035172123461961746,"tokens":11229.455078125,"tis_log_ratio_mean":0.020748747818288393}
+{"step":19,"reward":0.1484375,"avg_pass_at_8":0.34375,"entropy":0.04391411290271208,"grad_norm":0.03131328895688057,"tokens":11505.431640625,"tis_log_ratio_mean":0.018782067854772322}
+{"step":20,"reward":0.146484375,"avg_pass_at_8":0.234375,"entropy":0.037673404531233246,"grad_norm":0.027123216539621353,"tokens":12338.1328125,"tis_log_ratio_mean":0.01719331451022299}
+{"step":21,"reward":0.193359375,"avg_pass_at_8":0.328125,"entropy":0.035175977704057004,"grad_norm":0.028621729463338852,"tokens":12146.90625,"tis_log_ratio_mean":0.018719438132393407}
+{"step":22,"reward":0.14453125,"avg_pass_at_8":0.296875,"entropy":0.03437250474235043,"grad_norm":0.02626977488398552,"tokens":11635.396484375,"tis_log_ratio_mean":0.016605473649178748}
+{"step":23,"reward":0.201171875,"avg_pass_at_8":0.359375,"entropy":0.03006787772028474,"grad_norm":0.02914234809577465,"tokens":12961.05859375,"tis_log_ratio_mean":0.014169360549203702}
+{"step":24,"reward":0.16015625,"avg_pass_at_8":0.296875,"entropy":0.03030111043517536,"grad_norm":0.025229381397366524,"tokens":11709.1875,"tis_log_ratio_mean":0.016277045840979554}
+{"step":25,"reward":0.15625,"avg_pass_at_8":0.234375,"entropy":0.034945957182571874,"grad_norm":0.02751891501247883,"tokens":13984.34375,"tis_log_ratio_mean":0.019390785493669682}
+{"step":26,"reward":0.146484375,"avg_pass_at_8":0.25,"entropy":0.027067872822954087,"grad_norm":0.023381037637591362,"tokens":14325.361328125,"tis_log_ratio_mean":0.013600780663182377}
+{"step":27,"reward":0.1953125,"avg_pass_at_8":0.34375,"entropy":0.02622751688977587,"grad_norm":0.025417208671569824,"tokens":13526.552734375,"tis_log_ratio_mean":0.013845963384483184}
+{"step":28,"reward":0.22265625,"avg_pass_at_8":0.40625,"entropy":0.02686344806352281,"grad_norm":0.026991698890924454,"tokens":12715.685546875,"tis_log_ratio_mean":0.013580091088442714}
+{"step":29,"reward":0.18359375,"avg_pass_at_8":0.3125,"entropy":0.027546874021936674,"grad_norm":0.027726875618100166,"tokens":13227.87890625,"tis_log_ratio_mean":0.015239666638080962}
+{"step":30,"reward":0.177734375,"avg_pass_at_8":0.296875,"entropy":0.026558556372037856,"grad_norm":0.026201946660876274,"tokens":13108.8203125,"tis_log_ratio_mean":0.014960829257688602}
+{"step":31,"reward":0.275390625,"avg_pass_at_8":0.453125,"entropy":0.02491572662984254,"grad_norm":0.03018501028418541,"tokens":14046.927734375,"tis_log_ratio_mean":0.01332841400471807}
+{"step":32,"reward":0.18359375,"avg_pass_at_8":0.3125,"entropy":0.023597900850290898,"grad_norm":0.030108008533716202,"tokens":15153.35546875,"tis_log_ratio_mean":0.01289054514199961}
+{"step":33,"reward":0.205078125,"avg_pass_at_8":0.375,"entropy":0.022782319501857273,"grad_norm":0.026987791061401367,"tokens":13900.43359375,"tis_log_ratio_mean":0.013242687222373206}
+{"step":34,"reward":0.150390625,"avg_pass_at_8":0.265625,"entropy":0.02198731954217692,"grad_norm":0.025769177824258804,"tokens":14393.548828125,"tis_log_ratio_mean":0.012399274958966089}
+{"step":35,"reward":0.212890625,"avg_pass_at_8":0.328125,"entropy":0.022274382772593526,"grad_norm":0.022737322375178337,"tokens":15026.42578125,"tis_log_ratio_mean":0.011683876431561657}
+{"step":36,"reward":0.162109375,"avg_pass_at_8":0.328125,"entropy":0.02151524323198828,"grad_norm":0.026578430086374283,"tokens":15314.123046875,"tis_log_ratio_mean":0.011316415322653484}
+{"step":37,"reward":0.216796875,"avg_pass_at_8":0.34375,"entropy":0.020070513177415705,"grad_norm":0.031216923147439957,"tokens":14620.84375,"tis_log_ratio_mean":0.010617028958222363}
+{"step":38,"reward":0.271484375,"avg_pass_at_8":0.453125,"entropy":0.019342027647326177,"grad_norm":0.030968215316534042,"tokens":14905.142578125,"tis_log_ratio_mean":0.010256622203087318}
+{"step":39,"reward":0.224609375,"avg_pass_at_8":0.390625,"entropy":0.019655574318676372,"grad_norm":0.02511611580848694,"tokens":15209.525390625,"tis_log_ratio_mean":0.010830225144673022}
+{"step":40,"reward":0.240234375,"avg_pass_at_8":0.375,"entropy":0.019524238567100838,"grad_norm":0.024492397904396057,"tokens":15218.685546875,"tis_log_ratio_mean":0.010586443410829816}
+{"step":41,"reward":0.18359375,"avg_pass_at_8":0.296875,"entropy":0.01956303241149726,"grad_norm":0.02695755660533905,"tokens":15683.1796875,"tis_log_ratio_mean":0.010300105025635276}
+{"step":42,"reward":0.134765625,"avg_pass_at_8":0.28125,"entropy":0.019306482887714083,"grad_norm":0.0217769555747509,"tokens":15135.251953125,"tis_log_ratio_mean":0.010191034092258633}
+{"step":43,"reward":0.21484375,"avg_pass_at_8":0.296875,"entropy":0.01947490777092753,"grad_norm":0.024594906717538834,"tokens":14987.998046875,"tis_log_ratio_mean":0.010296048583768425}
+{"step":44,"reward":0.14453125,"avg_pass_at_8":0.25,"entropy":0.020008591887744842,"grad_norm":0.021484268829226494,"tokens":13937.09765625,"tis_log_ratio_mean":0.01109534449824423}
+{"step":45,"reward":0.08203125,"avg_pass_at_8":0.125,"entropy":0.018497942000976764,"grad_norm":0.017940938472747803,"tokens":15377.01953125,"tis_log_ratio_mean":0.010187213998506195}
+{"step":46,"reward":0.16015625,"avg_pass_at_8":0.265625,"entropy":0.0180558448219017,"grad_norm":0.023389659821987152,"tokens":15197.205078125,"tis_log_ratio_mean":0.009801067089938442}
+{"step":47,"reward":0.228515625,"avg_pass_at_8":0.421875,"entropy":0.017513232594865258,"grad_norm":0.02789674699306488,"tokens":15983.25,"tis_log_ratio_mean":0.009791816185497737}
+{"step":48,"reward":0.23046875,"avg_pass_at_8":0.390625,"entropy":0.01702760207672327,"grad_norm":0.02901029773056507,"tokens":15627.125,"tis_log_ratio_mean":0.009561383883010421}
+{"step":49,"reward":0.20703125,"avg_pass_at_8":0.3125,"entropy":0.017318538779818482,"grad_norm":0.020607849583029747,"tokens":14943.44921875,"tis_log_ratio_mean":0.010215631610662967}
+{"step":50,"reward":0.189453125,"avg_pass_at_8":0.28125,"entropy":0.018237074678836507,"grad_norm":0.024501772597432137,"tokens":14884.583984375,"tis_log_ratio_mean":0.010509410354643478}
+{"step":51,"reward":0.244140625,"avg_pass_at_8":0.421875,"entropy":0.019424339329816576,"grad_norm":0.04047199711203575,"tokens":14745.19921875,"tis_log_ratio_mean":0.010038193337095436}
+{"step":52,"reward":0.150390625,"avg_pass_at_8":0.265625,"entropy":0.06501297822433116,"grad_norm":0.03430159389972687,"tokens":13412.27734375,"tis_log_ratio_mean":0.04482977876250516}
+{"step":53,"reward":0.19921875,"avg_pass_at_8":0.375,"entropy":0.029081797672915854,"grad_norm":0.0413086824119091,"tokens":13819.90625,"tis_log_ratio_mean":0.04973612193316512}
+{"step":54,"reward":0.240234375,"avg_pass_at_8":0.359375,"entropy":0.021708972970372997,"grad_norm":0.021884219720959663,"tokens":13132.80859375,"tis_log_ratio_mean":0.06473713517698343}
+{"step":55,"reward":0.126953125,"avg_pass_at_8":0.265625,"entropy":0.01730329568090383,"grad_norm":0.0220363549888134,"tokens":13841.787109375,"tis_log_ratio_mean":0.012940252210682957}
+{"step":56,"reward":0.134765625,"avg_pass_at_8":0.265625,"entropy":0.015859588795137824,"grad_norm":0.01752507872879505,"tokens":14798.53515625,"tis_log_ratio_mean":0.009623677470699477}
+{"step":57,"reward":0.1328125,"avg_pass_at_8":0.25,"entropy":0.015599822476360714,"grad_norm":0.01599693112075329,"tokens":13839.533203125,"tis_log_ratio_mean":0.009627907483263698}
+{"step":58,"reward":0.083984375,"avg_pass_at_8":0.203125,"entropy":0.014008130634465488,"grad_norm":0.015376968309283257,"tokens":12856.60546875,"tis_log_ratio_mean":0.00812156598749425}
+{"step":59,"reward":0.05078125,"avg_pass_at_8":0.15625,"entropy":0.013305543288879562,"grad_norm":0.013282292522490025,"tokens":12804.419921875,"tis_log_ratio_mean":0.007885103695116413}
+{"step":60,"reward":0.080078125,"avg_pass_at_8":0.1875,"entropy":0.013582145929831313,"grad_norm":0.014613457955420017,"tokens":13667.490234375,"tis_log_ratio_mean":0.008013823374312778}
+{"step":61,"reward":0.091796875,"avg_pass_at_8":0.203125,"entropy":0.014827042250544764,"grad_norm":0.013647147454321384,"tokens":13353.837890625,"tis_log_ratio_mean":0.008521898822436924}
+{"step":62,"reward":0.142578125,"avg_pass_at_8":0.234375,"entropy":0.014874127516122826,"grad_norm":0.014650963246822357,"tokens":13697.2421875,"tis_log_ratio_mean":0.008342467674992804}
+{"step":63,"reward":0.09375,"avg_pass_at_8":0.265625,"entropy":0.014775633096178353,"grad_norm":0.013611107133328915,"tokens":13763.412109375,"tis_log_ratio_mean":0.008059417741606012}
+{"step":64,"reward":0.111328125,"avg_pass_at_8":0.21875,"entropy":0.015684132975366083,"grad_norm":0.01736508123576641,"tokens":13141.935546875,"tis_log_ratio_mean":0.008590416489823838}
+{"step":65,"reward":0.095703125,"avg_pass_at_8":0.1875,"entropy":0.014850527240923839,"grad_norm":0.01192401722073555,"tokens":13143.5390625,"tis_log_ratio_mean":0.008769816534368147}
+{"step":66,"reward":0.1171875,"avg_pass_at_8":0.328125,"entropy":0.0131793683858632,"grad_norm":0.019544003531336784,"tokens":13002.341796875,"tis_log_ratio_mean":0.007673890992009547}
+{"step":67,"reward":0.095703125,"avg_pass_at_8":0.21875,"entropy":0.011721476694219746,"grad_norm":0.01832517422735691,"tokens":12023.30859375,"tis_log_ratio_mean":0.007435109231664683}
+{"step":68,"reward":0.10546875,"avg_pass_at_8":0.28125,"entropy":0.013029860050664865,"grad_norm":0.014709926210343838,"tokens":13366.576171875,"tis_log_ratio_mean":0.008103573467451497}
+{"step":69,"reward":0.095703125,"avg_pass_at_8":0.1875,"entropy":0.013776918771327473,"grad_norm":0.0132818054407835,"tokens":14198.638671875,"tis_log_ratio_mean":0.008552698302082717}
+{"step":70,"reward":0.08203125,"avg_pass_at_8":0.21875,"entropy":0.013082287365250522,"grad_norm":0.013494989834725857,"tokens":13854.41796875,"tis_log_ratio_mean":0.00775652703941887}
+{"step":71,"reward":0.12890625,"avg_pass_at_8":0.25,"entropy":0.01257556774362456,"grad_norm":0.014938576146960258,"tokens":14066.126953125,"tis_log_ratio_mean":0.007207598023342143}
+{"step":72,"reward":0.06640625,"avg_pass_at_8":0.234375,"entropy":0.012152429171692347,"grad_norm":0.013156929984688759,"tokens":14561.115234375,"tis_log_ratio_mean":0.007443324982887134}
+{"step":73,"reward":0.095703125,"avg_pass_at_8":0.25,"entropy":0.01166447547984717,"grad_norm":0.01610151119530201,"tokens":13685.494140625,"tis_log_ratio_mean":0.006867967626931204}
+{"step":74,"reward":0.1328125,"avg_pass_at_8":0.25,"entropy":0.011587170273742231,"grad_norm":0.017074616625905037,"tokens":13497.849609375,"tis_log_ratio_mean":0.007014049957888346}
+{"step":75,"reward":0.078125,"avg_pass_at_8":0.203125,"entropy":0.010882317621508264,"grad_norm":0.017167704179883003,"tokens":13300.873046875,"tis_log_ratio_mean":0.006594639553441084}
+{"step":76,"reward":0.095703125,"avg_pass_at_8":0.15625,"entropy":0.011490128739751526,"grad_norm":0.013622788712382317,"tokens":14515.5234375,"tis_log_ratio_mean":0.006766945659364865}
+{"step":77,"reward":0.083984375,"avg_pass_at_8":0.203125,"entropy":0.011007365490513621,"grad_norm":0.014603457413613796,"tokens":12801.55078125,"tis_log_ratio_mean":0.006185150935380079}
+{"step":78,"reward":0.029296875,"avg_pass_at_8":0.078125,"entropy":0.010783515261209686,"grad_norm":0.010431727394461632,"tokens":13792.759765625,"tis_log_ratio_mean":0.007173382323344413}
+{"step":79,"reward":0.025390625,"avg_pass_at_8":0.078125,"entropy":0.009709339030450792,"grad_norm":0.012834833934903145,"tokens":10749.4140625,"tis_log_ratio_mean":0.005043381663199398}
+{"step":80,"reward":0.01171875,"avg_pass_at_8":0.09375,"entropy":0.005842892507644137,"grad_norm":0.013382860459387302,"tokens":9077.505859375,"tis_log_ratio_mean":0.0031812634838388476}
+{"step":81,"reward":0.009765625,"avg_pass_at_8":0.046875,"entropy":0.004876266244536964,"grad_norm":0.0124436030164361,"tokens":8178.64453125,"tis_log_ratio_mean":0.003160090742767352}
+{"step":82,"reward":0.0078125,"avg_pass_at_8":0.046875,"entropy":0.004734864871352329,"grad_norm":0.010083426721394062,"tokens":8307.58203125,"tis_log_ratio_mean":0.002627384877541772}
+{"step":83,"reward":0.017578125,"avg_pass_at_8":0.03125,"entropy":0.004628976187632361,"grad_norm":0.00922521110624075,"tokens":8142.046875,"tis_log_ratio_mean":0.002568632257862191}
+{"step":84,"reward":0.0,"avg_pass_at_8":0.0,"entropy":0.004764419196362724,"grad_norm":0.008718364872038364,"tokens":8368.6328125,"tis_log_ratio_mean":0.0031707382072454493}
+{"step":85,"reward":0.005859375,"avg_pass_at_8":0.046875,"entropy":0.0045769202670271625,"grad_norm":0.01027734950184822,"tokens":8037.625,"tis_log_ratio_mean":0.0025995669832354906}
+{"step":86,"reward":0.0,"avg_pass_at_8":0.0,"entropy":0.005000907969588297,"grad_norm":0.00927400216460228,"tokens":8412.037109375,"tis_log_ratio_mean":0.0026737586770195776}
+{"step":87,"reward":0.01953125,"avg_pass_at_8":0.046875,"entropy":0.0052621469667428755,"grad_norm":0.009100525639951229,"tokens":8270.173828125,"tis_log_ratio_mean":0.002948913346926929}
+{"step":88,"reward":0.01171875,"avg_pass_at_8":0.046875,"entropy":0.005899567382584792,"grad_norm":0.009215905331075191,"tokens":8804.10546875,"tis_log_ratio_mean":0.003219244339106808}
+{"step":89,"reward":0.03125,"avg_pass_at_8":0.125,"entropy":0.006737047274327779,"grad_norm":0.00983156356960535,"tokens":9199.109375,"tis_log_ratio_mean":0.0034877278890235175}
+{"step":90,"reward":0.017578125,"avg_pass_at_8":0.046875,"entropy":0.007792984229126887,"grad_norm":0.008721471764147282,"tokens":8973.79296875,"tis_log_ratio_mean":0.004256129757322924}
+{"step":91,"reward":0.029296875,"avg_pass_at_8":0.109375,"entropy":0.007726218957031961,"grad_norm":0.010015777312219143,"tokens":8897.896484375,"tis_log_ratio_mean":0.003926137936559826}
+{"step":92,"reward":0.009765625,"avg_pass_at_8":0.046875,"entropy":0.007923210539956926,"grad_norm":0.008899352513253689,"tokens":8843.322265625,"tis_log_ratio_mean":0.004391970237520582}
+{"step":93,"reward":0.021484375,"avg_pass_at_8":0.109375,"entropy":0.007826762632248574,"grad_norm":0.00953613966703415,"tokens":8713.794921875,"tis_log_ratio_mean":0.005461695624944696}
+{"step":94,"reward":0.03515625,"avg_pass_at_8":0.0625,"entropy":0.0063756551271580975,"grad_norm":0.009874533861875534,"tokens":8322.857421875,"tis_log_ratio_mean":0.003692385547765298}
+{"step":95,"reward":0.0078125,"avg_pass_at_8":0.03125,"entropy":0.006243172675567621,"grad_norm":0.00907274428755045,"tokens":8002.880859375,"tis_log_ratio_mean":0.0033519496157623507}
+{"step":96,"reward":0.01953125,"avg_pass_at_8":0.046875,"entropy":0.005606744040960621,"grad_norm":0.008685912936925888,"tokens":7674.302734375,"tis_log_ratio_mean":0.0034769170592880982}
+{"step":97,"reward":0.015625,"avg_pass_at_8":0.0625,"entropy":0.005897639412069111,"grad_norm":0.007625251077115536,"tokens":7617.8046875,"tis_log_ratio_mean":0.0028614974348784017}
+{"step":98,"reward":0.01953125,"avg_pass_at_8":0.046875,"entropy":0.005992065715872741,"grad_norm":0.006407018285244703,"tokens":7630.611328125,"tis_log_ratio_mean":0.002722906548569881}
+{"step":99,"reward":0.0078125,"avg_pass_at_8":0.0625,"entropy":0.006267066774853447,"grad_norm":0.006610449869185686,"tokens":7877.609375,"tis_log_ratio_mean":0.0031985354089556495}
+{"step":100,"reward":0.005859375,"avg_pass_at_8":0.03125,"entropy":0.005822780703056196,"grad_norm":0.007009792607277632,"tokens":7668.25390625,"tis_log_ratio_mean":0.002807933654366934}
+{"step":101,"reward":0.041015625,"avg_pass_at_8":0.078125,"entropy":0.005608550804936385,"grad_norm":0.008033922873437405,"tokens":7613.501953125,"tis_log_ratio_mean":0.0026493279283386073}

viewer/build/inputs/marin/runs/marin-q3c-tt-x15-megatron/run.json

@@ -0,0 +1,26 @@
+{
+ "id": "marin-q3c-tt-x15-megatron",
+ "title": "TaskTrove RL, Megatron backend, collapsed (Qwen3-Coder-30B-A3B)",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://huggingface.co/laion/tt-x15-megatron-51-30B/tree/main/training_logs",
+ "license": "apache-2.0",
+ "model": "laion/tt-x15-megatron-51-30B",
+ "base_model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
+ "method": "GRPO (SkyRL + Terminus-2, pass-ratio shaped verifier reward)",
+ "dataset": "DCAgent/exp_rpt_multifile",
+ "eval_suite": null,
+ "kind": "training",
+ "state": "finished",
+ "started_at": null,
+ "updated_at": "2026-09-04T16:22:40Z",
+ "attempts": 0,
+ "note": "Marin TaskTrove RL hyperparameter ablation (issue #7785) on Qwen3-Coder-30B-A3B-Instruct with DCAgent/exp_rpt_multifile tasks and the Terminus-2 harness; arm: X15 Megatron backend with lr 8e-6, lower clip 0.3 and temperature 1.2, an attempt to prevent an earlier Megatron arm's late collapse; Marin marks it collapsed (reward fell to about 0 at steps 84 to 86). Our copy is the per-step curve Marin stitched from W&B (its trailing-EMA column left out).",
+ "metrics_map": {
+  "reward": "reward",
+  "entropy": "entropy",
+  "grad_norm": "grad_norm",
+  "response_length": "tokens"
+ }
+}

viewer/build/inputs/marin/runs/marin-q3c-tt-x3-kl0p001/metrics.jsonl

@@ -0,0 +1,75 @@
+{"step":1,"async/staleness_mean":0.0,"generate/avg_num_tokens":11520.150390625,"generate/avg_tokens_non_zero_rewards":10283.037037037036,"generate/avg_tokens_zero_rewards":11752.647331786542,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24133,"generate/std_num_tokens":3174.1665316649714,"loss/avg_final_rewards":0.158203125,"loss/avg_raw_advantages":-0.004655978176742792,"loss/avg_raw_advantages_abs":0.1155080571770668,"policy/policy_entropy":0.12057892512530088,"policy/policy_kl":0.011378814902855083,"policy/policy_loss":1.479408282989425e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.0065977229714917485,"policy/raw_grad_norm":0.0009613037109375,"reward/avg_pass_at_8":0.3125,"reward/avg_raw_reward":0.158203125,"reward/policy_ref_kl":0.011411887593567371,"timing/step":5360.391353664978,"trainer/epoch":0}
+{"step":2,"async/staleness_mean":0.96875,"generate/avg_num_tokens":12130.35546875,"generate/avg_tokens_non_zero_rewards":12249.4375,"generate/avg_tokens_zero_rewards":12113.34375,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":29231,"generate/std_num_tokens":3616.652141080127,"loss/avg_final_rewards":0.125,"loss/avg_raw_advantages":-0.0023999004624783993,"loss/avg_raw_advantages_abs":0.04737769812345505,"policy/policy_entropy":0.1314482360612601,"policy/policy_kl":0.010211955945123918,"policy/policy_loss":2.088674577294114e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.0032432157549919793,"policy/raw_grad_norm":0.001171112060546875,"reward/avg_pass_at_8":0.1875,"reward/avg_raw_reward":0.125,"reward/policy_ref_kl":0.01015262771397829,"timing/step":2362.740868670022,"trainer/epoch":0}
+{"step":3,"async/staleness_mean":1.515625,"generate/avg_num_tokens":11949.158203125,"generate/avg_tokens_non_zero_rewards":10437.015384615384,"generate/avg_tokens_zero_rewards":12169.044742729306,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":23772,"generate/std_num_tokens":3307.467880662244,"loss/avg_final_rewards":0.126953125,"loss/avg_raw_advantages":-0.0015606010565534234,"loss/avg_raw_advantages_abs":0.07942964136600494,"policy/policy_entropy":0.13389885774813592,"policy/policy_kl":0.00930634442192968,"policy/policy_loss":7.357469868907174e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.004376971557576326,"policy/raw_grad_norm":0.0009164810180664062,"reward/avg_pass_at_8":0.234375,"reward/avg_raw_reward":0.126953125,"reward/policy_ref_kl":0.009207329712808132,"timing/step":2865.7047030749964,"trainer/epoch":0}
+{"step":4,"async/staleness_mean":0.625,"generate/avg_num_tokens":12177.072265625,"generate/avg_tokens_non_zero_rewards":11653.55357142857,"generate/avg_tokens_zero_rewards":12323.6575,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":30226,"generate/std_num_tokens":3760.3423205639874,"loss/avg_final_rewards":0.21875,"loss/avg_raw_advantages":-0.002428284380584955,"loss/avg_raw_advantages_abs":0.12967264652252197,"policy/policy_entropy":0.14373120106756687,"policy/policy_kl":0.009528914582915604,"policy/policy_loss":6.66695314066601e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008088960392342415,"policy/raw_grad_norm":0.0007610321044921875,"reward/avg_pass_at_8":0.3269230769230769,"reward/avg_raw_reward":0.21875,"reward/policy_ref_kl":0.009471283294260502,"timing/step":4825.538106569991,"trainer/epoch":0}
+{"step":5,"async/staleness_mean":0.984375,"generate/avg_num_tokens":13186.36328125,"generate/avg_tokens_non_zero_rewards":12721.5,"generate/avg_tokens_zero_rewards":13236.67316017316,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26507,"generate/std_num_tokens":4120.369872664101,"loss/avg_final_rewards":0.09765625,"loss/avg_raw_advantages":-0.0022999641951173544,"loss/avg_raw_advantages_abs":0.10895732045173645,"policy/policy_entropy":0.14602772565558553,"policy/policy_kl":0.009254388263798319,"policy/policy_loss":1.5198289133877552e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.004555513134619105,"policy/raw_grad_norm":0.0009555816650390625,"reward/avg_pass_at_8":0.234375,"reward/avg_raw_reward":0.09765625,"reward/policy_ref_kl":0.009296870790421963,"timing/step":2336.7251445800066,"trainer/epoch":0}
+{"step":7,"async/staleness_mean":0.59375,"generate/avg_num_tokens":12402.6796875,"generate/avg_tokens_non_zero_rewards":11121.279411764706,"generate/avg_tokens_zero_rewards":12598.93018018018,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24931,"generate/std_num_tokens":3682.4973617471746,"loss/avg_final_rewards":0.1328125,"loss/avg_raw_advantages":-9.09359150682576e-05,"loss/avg_raw_advantages_abs":0.09268297255039215,"policy/policy_entropy":0.1535300884861499,"policy/policy_kl":0.009366340120323002,"policy/policy_loss":7.232201930662541e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006652169173321454,"policy/raw_grad_norm":0.00067138671875,"reward/avg_pass_at_8":0.2830188679245283,"reward/avg_raw_reward":0.1328125,"reward/policy_ref_kl":0.009449854493141174,"timing/step":5680.3754920979845,"trainer/epoch":0}
+{"step":8,"async/staleness_mean":1.0,"generate/avg_num_tokens":12703.255859375,"generate/avg_tokens_non_zero_rewards":11361.797101449276,"generate/avg_tokens_zero_rewards":13198.232620320856,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25080,"generate/std_num_tokens":4218.916626682181,"loss/avg_final_rewards":0.26953125,"loss/avg_raw_advantages":-0.00150024495087564,"loss/avg_raw_advantages_abs":0.1288275569677353,"policy/policy_entropy":0.1594758138526231,"policy/policy_kl":0.009739218265167437,"policy/policy_loss":6.705167088227881e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007151501808039029,"policy/raw_grad_norm":0.0007829666137695312,"reward/avg_pass_at_8":0.453125,"reward/avg_raw_reward":0.26953125,"reward/policy_ref_kl":0.00981323141604662,"timing/step":2529.278858130012,"trainer/epoch":0}
+{"step":9,"async/staleness_mean":1.5,"generate/avg_num_tokens":12213.91796875,"generate/avg_tokens_non_zero_rewards":10949.69,"generate/avg_tokens_zero_rewards":12520.769417475729,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27445,"generate/std_num_tokens":3986.414058279963,"loss/avg_final_rewards":0.1953125,"loss/avg_raw_advantages":-0.0021864010486751795,"loss/avg_raw_advantages_abs":0.12060174345970154,"policy/policy_entropy":0.164930478669703,"policy/policy_kl":0.010383207249105908,"policy/policy_loss":1.6173328774016227e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00635735374999058,"policy/raw_grad_norm":0.0007534027099609375,"reward/avg_pass_at_8":0.34375,"reward/avg_raw_reward":0.1953125,"reward/policy_ref_kl":0.010384080931544304,"timing/step":2593.0344647429883,"trainer/epoch":0}
+{"step":10,"async/staleness_mean":1.796875,"generate/avg_num_tokens":12446.64453125,"generate/avg_tokens_non_zero_rewards":11086.1125,"generate/avg_tokens_zero_rewards":12698.594907407407,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26786,"generate/std_num_tokens":4062.766223321337,"loss/avg_final_rewards":0.15625,"loss/avg_raw_advantages":0.0003447741037234664,"loss/avg_raw_advantages_abs":0.09824231266975403,"policy/policy_entropy":0.170739711727947,"policy/policy_kl":0.01077629512292333,"policy/policy_loss":1.3042485917935664e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006561215333931614,"policy/raw_grad_norm":0.0006866455078125,"reward/avg_pass_at_8":0.296875,"reward/avg_raw_reward":0.15625,"reward/policy_ref_kl":0.010884546674787998,"timing/step":2607.6530492189922,"trainer/epoch":0}
+{"step":11,"async/staleness_mean":1.734375,"generate/avg_num_tokens":12828.693359375,"generate/avg_tokens_non_zero_rewards":10499.039603960397,"generate/avg_tokens_zero_rewards":13401.187347931873,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25531,"generate/std_num_tokens":4329.152854227765,"loss/avg_final_rewards":0.197265625,"loss/avg_raw_advantages":-0.0028160614892840385,"loss/avg_raw_advantages_abs":0.09344282001256943,"policy/policy_entropy":0.1708759746979922,"policy/policy_kl":0.01105841840035282,"policy/policy_loss":1.873289072307216e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.0067140512428522925,"policy/raw_grad_norm":0.0007772445678710938,"reward/avg_pass_at_8":0.34375,"reward/avg_raw_reward":0.197265625,"reward/policy_ref_kl":0.011041730642318726,"timing/step":3293.791356369009,"trainer/epoch":0}
+{"step":12,"async/staleness_mean":0.59375,"generate/avg_num_tokens":14282.462890625,"generate/avg_tokens_non_zero_rewards":12157.8125,"generate/avg_tokens_zero_rewards":14772.766826923076,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":28449,"generate/std_num_tokens":5050.1083271362,"loss/avg_final_rewards":0.1875,"loss/avg_raw_advantages":-0.0003190449206158519,"loss/avg_raw_advantages_abs":0.12273643165826797,"policy/policy_entropy":0.1737018742132932,"policy/policy_kl":0.01136889161716681,"policy/policy_loss":2.716034607885831e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.009474808177401428,"policy/raw_grad_norm":0.0006437301635742188,"reward/avg_pass_at_8":0.2916666666666667,"reward/avg_raw_reward":0.1875,"reward/policy_ref_kl":0.011390856467187405,"timing/step":3895.7628284870007,"trainer/epoch":0}
+{"step":13,"async/staleness_mean":0.984375,"generate/avg_num_tokens":13343.310546875,"generate/avg_tokens_non_zero_rewards":11512.985294117647,"generate/avg_tokens_zero_rewards":13623.63063063063,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26996,"generate/std_num_tokens":4827.6946660208605,"loss/avg_final_rewards":0.1328125,"loss/avg_raw_advantages":0.002555670216679573,"loss/avg_raw_advantages_abs":0.09375318884849548,"policy/policy_entropy":0.1716453975532204,"policy/policy_kl":0.011846716020954773,"policy/policy_loss":-1.5221991134239943e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00539899025716295,"policy/raw_grad_norm":0.0007801055908203125,"reward/avg_pass_at_8":0.25,"reward/avg_raw_reward":0.1328125,"reward/policy_ref_kl":0.011785060167312622,"timing/step":2333.01166681599,"trainer/epoch":0}
+{"step":14,"async/staleness_mean":1.265625,"generate/avg_num_tokens":13112.001953125,"generate/avg_tokens_non_zero_rewards":12019.342592592593,"generate/avg_tokens_zero_rewards":13404.09900990099,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27977,"generate/std_num_tokens":4519.398179868566,"loss/avg_final_rewards":0.2109375,"loss/avg_raw_advantages":-0.00024403379939030856,"loss/avg_raw_advantages_abs":0.1029505655169487,"policy/policy_entropy":0.180516378255561,"policy/policy_kl":0.012922705325763673,"policy/policy_loss":2.644551653219196e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00883393854564929,"policy/raw_grad_norm":0.00063323974609375,"reward/avg_pass_at_8":0.359375,"reward/avg_raw_reward":0.2109375,"reward/policy_ref_kl":0.012995418161153793,"timing/step":2226.93960427001,"trainer/epoch":0}
+{"step":15,"async/staleness_mean":2.953125,"generate/avg_num_tokens":590.986328125,"generate/avg_tokens_non_zero_rewards":10815.166666666666,"generate/avg_tokens_zero_rewards":469.7509881422925,"generate/failed_trajectory_fraction":0.953125,"generate/max_num_tokens":25927,"generate/std_num_tokens":2886.1717551462016,"loss/avg_final_rewards":0.01171875,"loss/avg_raw_advantages":-0.030360322445631027,"loss/avg_raw_advantages_abs":0.15290628373622894,"policy/policy_entropy":0.00837914140720386,"policy/policy_kl":0.0006221270996320527,"policy/policy_loss":2.948053270301898e-05,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.000583669519983232,"policy/raw_grad_norm":0.004241943359375,"reward/avg_pass_at_8":0.015625,"reward/avg_raw_reward":0.01171875,"reward/policy_ref_kl":0.0006309784948825836,"timing/step":587.5361071719963,"trainer/epoch":0}
+{"step":16,"async/staleness_mean":1.921875,"generate/avg_num_tokens":9451.427734375,"generate/avg_tokens_non_zero_rewards":12487.6,"generate/avg_tokens_zero_rewards":9158.86295503212,"generate/failed_trajectory_fraction":0.296875,"generate/max_num_tokens":26843,"generate/std_num_tokens":7714.8902051545865,"loss/avg_final_rewards":0.087890625,"loss/avg_raw_advantages":-0.012057358399033546,"loss/avg_raw_advantages_abs":0.07266092300415039,"policy/policy_entropy":0.13196935725864023,"policy/policy_kl":0.010434669879032299,"policy/policy_loss":1.0351788397144901e-05,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.002872511637633579,"policy/raw_grad_norm":0.0009679794311523438,"reward/avg_pass_at_8":0.25,"reward/avg_raw_reward":0.087890625,"reward/policy_ref_kl":0.010366416536271572,"timing/step":3355.813335865998,"trainer/epoch":0}
+{"step":17,"async/staleness_mean":1.296875,"generate/avg_num_tokens":13019.73046875,"generate/avg_tokens_non_zero_rewards":11584.86170212766,"generate/avg_tokens_zero_rewards":13342.404306220096,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":28744,"generate/std_num_tokens":4835.879246176739,"loss/avg_final_rewards":0.18359375,"loss/avg_raw_advantages":-0.00376077345572412,"loss/avg_raw_advantages_abs":0.11722420156002045,"policy/policy_entropy":0.18961325683631003,"policy/policy_kl":0.015988921353709884,"policy/policy_loss":4.4630642364040796e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007356617323239334,"policy/raw_grad_norm":0.0007162094116210938,"reward/avg_pass_at_8":0.328125,"reward/avg_raw_reward":0.18359375,"reward/policy_ref_kl":0.016019770875573158,"timing/step":2311.427478128986,"trainer/epoch":0}
+{"step":18,"async/staleness_mean":1.328125,"generate/avg_num_tokens":13198.310546875,"generate/avg_tokens_non_zero_rewards":11591.757281553399,"generate/avg_tokens_zero_rewards":13602.894865525672,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27395,"generate/std_num_tokens":4799.945513530077,"loss/avg_final_rewards":0.201171875,"loss/avg_raw_advantages":-0.009775816462934017,"loss/avg_raw_advantages_abs":0.12495778501033783,"policy/policy_entropy":0.19386467593722045,"policy/policy_kl":0.016832266192068346,"policy/policy_loss":1.0172352958193187e-05,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007168849582740222,"policy/raw_grad_norm":0.000789642333984375,"reward/avg_pass_at_8":0.34375,"reward/avg_raw_reward":0.201171875,"reward/policy_ref_kl":0.016779445111751556,"timing/step":2265.080029439967,"trainer/epoch":0}
+{"step":19,"async/staleness_mean":3.90625,"generate/avg_num_tokens":632.615234375,"generate/avg_tokens_non_zero_rewards":16561.333333333332,"generate/avg_tokens_zero_rewards":538.7328094302554,"generate/failed_trajectory_fraction":0.953125,"generate/max_num_tokens":24127,"generate/std_num_tokens":2979.4973953518634,"loss/avg_final_rewards":0.005859375,"loss/avg_raw_advantages":0.021190010011196136,"loss/avg_raw_advantages_abs":0.19794359803199768,"policy/policy_entropy":0.008327799660037272,"policy/policy_kl":0.0007430063042193069,"policy/policy_loss":-1.9086663996858988e-05,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00026922848701360635,"policy/raw_grad_norm":0.0093536376953125,"reward/avg_pass_at_8":0.015625,"reward/avg_raw_reward":0.005859375,"reward/policy_ref_kl":0.0007349271327257156,"timing/step":587.8864458530443,"trainer/epoch":0}
+{"step":20,"async/staleness_mean":3.296875,"generate/avg_num_tokens":6384.888671875,"generate/avg_tokens_non_zero_rewards":14596.142857142857,"generate/avg_tokens_zero_rewards":6271.069306930693,"generate/failed_trajectory_fraction":0.484375,"generate/max_num_tokens":24740,"generate/std_num_tokens":6940.398236482214,"loss/avg_final_rewards":0.013671875,"loss/avg_raw_advantages":0.0022123847156763077,"loss/avg_raw_advantages_abs":0.04962588846683502,"policy/policy_entropy":0.10471462336136028,"policy/policy_kl":0.01040853738959413,"policy/policy_loss":3.0178705401340267e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.0010135387833543064,"policy/raw_grad_norm":0.0012359619140625,"reward/avg_pass_at_8":0.0625,"reward/avg_raw_reward":0.013671875,"reward/policy_ref_kl":0.010325618088245392,"timing/step":1954.3406890810002,"trainer/epoch":0}
+{"step":21,"async/staleness_mean":1.484375,"generate/avg_num_tokens":10269.890625,"generate/avg_tokens_non_zero_rewards":11812.981481481482,"generate/avg_tokens_zero_rewards":10087.954148471616,"generate/failed_trajectory_fraction":0.203125,"generate/max_num_tokens":30591,"generate/std_num_tokens":7119.678675415212,"loss/avg_final_rewards":0.10546875,"loss/avg_raw_advantages":-0.002249879762530327,"loss/avg_raw_advantages_abs":0.10369190573692322,"policy/policy_entropy":0.16701123374514282,"policy/policy_kl":0.01808081350463908,"policy/policy_loss":4.3307347041832145e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.005469236296448798,"policy/raw_grad_norm":0.0007600784301757812,"reward/avg_pass_at_8":0.2978723404255319,"reward/avg_raw_reward":0.10546875,"reward/policy_ref_kl":0.01795804314315319,"timing/step":3634.414853741997,"trainer/epoch":0}
+{"step":22,"async/staleness_mean":0.546875,"generate/avg_num_tokens":13763.82421875,"generate/avg_tokens_non_zero_rewards":11697.275,"generate/avg_tokens_zero_rewards":14146.518518518518,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27322,"generate/std_num_tokens":5428.976571835474,"loss/avg_final_rewards":0.15625,"loss/avg_raw_advantages":-0.0057094041258096695,"loss/avg_raw_advantages_abs":0.1270723193883896,"policy/policy_entropy":0.21655491250567138,"policy/policy_kl":0.025198107643518597,"policy/policy_loss":3.4922253746572096e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008068953055044403,"policy/raw_grad_norm":0.000537872314453125,"reward/avg_pass_at_8":0.4186046511627907,"reward/avg_raw_reward":0.15625,"reward/policy_ref_kl":0.024703320115804672,"timing/step":3774.4446516280295,"trainer/epoch":0}
+{"step":23,"async/staleness_mean":1.0,"generate/avg_num_tokens":14015.69140625,"generate/avg_tokens_non_zero_rewards":12113.91111111111,"generate/avg_tokens_zero_rewards":14198.946466809422,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":28913,"generate/std_num_tokens":5617.070049753021,"loss/avg_final_rewards":0.087890625,"loss/avg_raw_advantages":-0.0041635241359472275,"loss/avg_raw_advantages_abs":0.0721747949719429,"policy/policy_entropy":0.20869401982054114,"policy/policy_kl":0.024472877295920625,"policy/policy_loss":7.48560976404633e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.004209239258671005,"policy/raw_grad_norm":0.0007152557373046875,"reward/avg_pass_at_8":0.234375,"reward/avg_raw_reward":0.087890625,"reward/policy_ref_kl":0.02427729032933712,"timing/step":2003.162217552017,"trainer/epoch":0}
+{"step":24,"async/staleness_mean":1.453125,"generate/avg_num_tokens":12446.9375,"generate/avg_tokens_non_zero_rewards":11864.898876404495,"generate/avg_tokens_zero_rewards":12569.39952718676,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26296,"generate/std_num_tokens":4987.050998351255,"loss/avg_final_rewards":0.173828125,"loss/avg_raw_advantages":-0.009328571148216724,"loss/avg_raw_advantages_abs":0.10279203206300735,"policy/policy_entropy":0.2139452830888331,"policy/policy_kl":0.028673187305685133,"policy/policy_loss":7.571979363518722e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00677915826872777,"policy/raw_grad_norm":0.0006856918334960938,"reward/avg_pass_at_8":0.296875,"reward/avg_raw_reward":0.173828125,"reward/policy_ref_kl":0.02846628427505493,"timing/step":2076.8665304880124,"trainer/epoch":0}
+{"step":25,"async/staleness_mean":2.6875,"generate/avg_num_tokens":4133.359375,"generate/avg_tokens_non_zero_rewards":11742.413793103447,"generate/avg_tokens_zero_rewards":3676.5010351966876,"generate/failed_trajectory_fraction":0.6875,"generate/max_num_tokens":26633,"generate/std_num_tokens":6848.312755519574,"loss/avg_final_rewards":0.056640625,"loss/avg_raw_advantages":-0.011311296373605728,"loss/avg_raw_advantages_abs":0.12002793699502945,"policy/policy_entropy":0.06487899896455929,"policy/policy_kl":0.00890194817839074,"policy/policy_loss":7.974907830998745e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.002421893369501049,"policy/raw_grad_norm":0.0008993148803710938,"reward/avg_pass_at_8":0.125,"reward/avg_raw_reward":0.056640625,"reward/policy_ref_kl":0.00881329644471407,"timing/step":917.5565133430064,"trainer/epoch":0}
+{"step":26,"async/staleness_mean":3.375,"generate/avg_num_tokens":3930.609375,"generate/avg_tokens_non_zero_rewards":10710.90322580645,"generate/avg_tokens_zero_rewards":3493.6257796257796,"generate/failed_trajectory_fraction":0.6875,"generate/max_num_tokens":26772,"generate/std_num_tokens":6533.840164308017,"loss/avg_final_rewards":0.060546875,"loss/avg_raw_advantages":-0.007055288180708885,"loss/avg_raw_advantages_abs":0.17655104398727417,"policy/policy_entropy":0.0710617910081055,"policy/policy_kl":0.010610319579427596,"policy/policy_loss":3.2946167749514643e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.0035983943544124486,"policy/raw_grad_norm":0.0009326934814453125,"reward/avg_pass_at_8":0.140625,"reward/avg_raw_reward":0.060546875,"reward/policy_ref_kl":0.010565480217337608,"timing/step":1894.8930446990416,"trainer/epoch":0}
+{"step":27,"async/staleness_mean":2.765625,"generate/avg_num_tokens":8663.666015625,"generate/avg_tokens_non_zero_rewards":10858.094594594595,"generate/avg_tokens_zero_rewards":8292.917808219177,"generate/failed_trajectory_fraction":0.3125,"generate/max_num_tokens":30360,"generate/std_num_tokens":7412.7607887683325,"loss/avg_final_rewards":0.14453125,"loss/avg_raw_advantages":-0.006476950366050005,"loss/avg_raw_advantages_abs":0.1568218171596527,"policy/policy_entropy":0.1615357584087178,"policy/policy_kl":0.025188544481352437,"policy/policy_loss":3.1341099955284335e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007956358026149246,"policy/raw_grad_norm":0.0006771087646484375,"reward/avg_pass_at_8":0.3125,"reward/avg_raw_reward":0.14453125,"reward/policy_ref_kl":0.02528686635196209,"timing/step":2178.5039152220124,"trainer/epoch":0}
+{"step":28,"async/staleness_mean":1.359375,"generate/avg_num_tokens":12314.18359375,"generate/avg_tokens_non_zero_rewards":11241.701298701299,"generate/avg_tokens_zero_rewards":12504.025287356322,"generate/failed_trajectory_fraction":0.03125,"generate/max_num_tokens":30251,"generate/std_num_tokens":5726.558147123548,"loss/avg_final_rewards":0.150390625,"loss/avg_raw_advantages":0.002246356103569269,"loss/avg_raw_advantages_abs":0.09129797667264938,"policy/policy_entropy":0.2339239837601781,"policy/policy_kl":0.03710899030556902,"policy/policy_loss":5.26072614803752e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.005974647071525396,"policy/raw_grad_norm":0.0006799697875976562,"reward/avg_pass_at_8":0.265625,"reward/avg_raw_reward":0.150390625,"reward/policy_ref_kl":0.03703828901052475,"timing/step":2103.8544816499925,"trainer/epoch":0}
+{"step":29,"async/staleness_mean":2.984375,"generate/avg_num_tokens":7605.59765625,"generate/avg_tokens_non_zero_rewards":10105.636363636364,"generate/avg_tokens_zero_rewards":7304.71772428884,"generate/failed_trajectory_fraction":0.390625,"generate/max_num_tokens":25779,"generate/std_num_tokens":7097.482312070299,"loss/avg_final_rewards":0.107421875,"loss/avg_raw_advantages":-0.008276039734482765,"loss/avg_raw_advantages_abs":0.10945608466863632,"policy/policy_entropy":0.14744931901805103,"policy/policy_kl":0.02452511538285762,"policy/policy_loss":2.515417154569377e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.005753857200943457,"policy/raw_grad_norm":0.0007276535034179688,"reward/avg_pass_at_8":0.203125,"reward/avg_raw_reward":0.107421875,"reward/policy_ref_kl":0.0244667436927557,"timing/step":1164.8339345050044,"trainer/epoch":0}
+{"step":30,"async/staleness_mean":3.34375,"generate/avg_num_tokens":4849.12890625,"generate/avg_tokens_non_zero_rewards":7665.785714285715,"generate/avg_tokens_zero_rewards":4597.427659574468,"generate/failed_trajectory_fraction":0.546875,"generate/max_num_tokens":30211,"generate/std_num_tokens":6599.964371869418,"loss/avg_final_rewards":0.08203125,"loss/avg_raw_advantages":-0.014133543707430363,"loss/avg_raw_advantages_abs":0.09120868891477585,"policy/policy_entropy":0.11049468000419438,"policy/policy_kl":0.02142431735410355,"policy/policy_loss":7.849502878798376e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.004834163813484338,"policy/raw_grad_norm":0.000812530517578125,"reward/avg_pass_at_8":0.14893617021276595,"reward/avg_raw_reward":0.08203125,"reward/policy_ref_kl":0.021464448422193527,"timing/step":3113.939232982986,"trainer/epoch":0}
+{"step":31,"async/staleness_mean":1.0,"generate/avg_num_tokens":12553.37890625,"generate/avg_tokens_non_zero_rewards":11423.037037037036,"generate/avg_tokens_zero_rewards":12855.549504950495,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":30553,"generate/std_num_tokens":5926.709789313338,"loss/avg_final_rewards":0.2109375,"loss/avg_raw_advantages":-0.0023064645938575268,"loss/avg_raw_advantages_abs":0.15468695759773254,"policy/policy_entropy":0.2470123863313347,"policy/policy_kl":0.045240708568599075,"policy/policy_loss":3.7584804637447178e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.010124025011464255,"policy/raw_grad_norm":0.0005474090576171875,"reward/avg_pass_at_8":0.453125,"reward/avg_raw_reward":0.2109375,"reward/policy_ref_kl":0.04534436762332916,"timing/step":1859.0860955840326,"trainer/epoch":0}
+{"step":32,"async/staleness_mean":1.640625,"generate/avg_num_tokens":12385.154296875,"generate/avg_tokens_non_zero_rewards":9789.247191011236,"generate/avg_tokens_zero_rewards":12931.338061465722,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27142,"generate/std_num_tokens":5450.404686238499,"loss/avg_final_rewards":0.173828125,"loss/avg_raw_advantages":-0.007225672248750925,"loss/avg_raw_advantages_abs":0.11963548511266708,"policy/policy_entropy":0.2563481607940048,"policy/policy_kl":0.048126303299795836,"policy/policy_loss":2.5971306669703154e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007300604949705303,"policy/raw_grad_norm":0.0006608963012695312,"reward/avg_pass_at_8":0.390625,"reward/avg_raw_reward":0.173828125,"reward/policy_ref_kl":0.04764966666698456,"timing/step":1658.742349572014,"trainer/epoch":0}
+{"step":33,"async/staleness_mean":1.734375,"generate/avg_num_tokens":11339.63671875,"generate/avg_tokens_non_zero_rewards":9053.0,"generate/avg_tokens_zero_rewards":11880.917874396135,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":30291,"generate/std_num_tokens":5046.122778777334,"loss/avg_final_rewards":0.19140625,"loss/avg_raw_advantages":-0.004310745280236006,"loss/avg_raw_advantages_abs":0.08660412579774857,"policy/policy_entropy":0.25972675322555006,"policy/policy_kl":0.052274918009061366,"policy/policy_loss":1.9894275666842987e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008022326192076434,"policy/raw_grad_norm":0.0006084442138671875,"reward/avg_pass_at_8":0.328125,"reward/avg_raw_reward":0.19140625,"reward/policy_ref_kl":0.05244266986846924,"timing/step":1460.841757357004,"trainer/epoch":0}
+{"step":34,"async/staleness_mean":2.515625,"generate/avg_num_tokens":6605.802734375,"generate/avg_tokens_non_zero_rewards":8866.559322033898,"generate/avg_tokens_zero_rewards":6311.355408388521,"generate/failed_trajectory_fraction":0.359375,"generate/max_num_tokens":25426,"generate/std_num_tokens":6288.047211382135,"loss/avg_final_rewards":0.115234375,"loss/avg_raw_advantages":-0.006769534200429916,"loss/avg_raw_advantages_abs":0.09783092886209488,"policy/policy_entropy":0.16492588038090616,"policy/policy_kl":0.03658945977804251,"policy/policy_loss":4.084705963691704e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.0058314965285717335,"policy/raw_grad_norm":0.0008029937744140625,"reward/avg_pass_at_8":0.171875,"reward/avg_raw_reward":0.115234375,"reward/policy_ref_kl":0.03652539104223251,"timing/step":1626.586485509004,"trainer/epoch":0}
+{"step":35,"async/staleness_mean":1.3125,"generate/avg_num_tokens":10772.728515625,"generate/avg_tokens_non_zero_rewards":10070.07142857143,"generate/avg_tokens_zero_rewards":10835.51914893617,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27402,"generate/std_num_tokens":5332.699882942093,"loss/avg_final_rewards":0.08203125,"loss/avg_raw_advantages":0.0020064988639205694,"loss/avg_raw_advantages_abs":0.08542323857545853,"policy/policy_entropy":0.26560914400033653,"policy/policy_kl":0.05814935587113723,"policy/policy_loss":-1.3922782393649413e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00522930055467441,"policy/raw_grad_norm":0.0004572868347167969,"reward/avg_pass_at_8":0.30612244897959184,"reward/avg_raw_reward":0.08203125,"reward/policy_ref_kl":0.057939305901527405,"timing/step":2943.430134943046,"trainer/epoch":0}
+{"step":36,"async/staleness_mean":1.0,"generate/avg_num_tokens":10979.017578125,"generate/avg_tokens_non_zero_rewards":9649.625,"generate/avg_tokens_zero_rewards":11142.276315789473,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":29905,"generate/std_num_tokens":5155.885330445843,"loss/avg_final_rewards":0.109375,"loss/avg_raw_advantages":-0.000782620336394757,"loss/avg_raw_advantages_abs":0.0885033831000328,"policy/policy_entropy":0.26378958486020565,"policy/policy_kl":0.05984377895947546,"policy/policy_loss":1.229606130692673e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.005809887343275477,"policy/raw_grad_norm":0.0006837844848632812,"reward/avg_pass_at_8":0.21875,"reward/avg_raw_reward":0.109375,"reward/policy_ref_kl":0.05981138348579407,"timing/step":1281.4162046539714,"trainer/epoch":0}
+{"step":37,"async/staleness_mean":1.890625,"generate/avg_num_tokens":10529.58984375,"generate/avg_tokens_non_zero_rewards":8747.039682539682,"generate/avg_tokens_zero_rewards":11111.458549222798,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26880,"generate/std_num_tokens":4967.80999399729,"loss/avg_final_rewards":0.24609375,"loss/avg_raw_advantages":-0.0020701943431049585,"loss/avg_raw_advantages_abs":0.09944380074739456,"policy/policy_entropy":0.26699011330492795,"policy/policy_kl":0.061075665114913136,"policy/policy_loss":2.2677656765779375e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.009871083094822097,"policy/raw_grad_norm":0.0005292892456054688,"reward/avg_pass_at_8":0.40625,"reward/avg_raw_reward":0.24609375,"reward/policy_ref_kl":0.060871198773384094,"timing/step":1624.791639958974,"trainer/epoch":0}
+{"step":38,"async/staleness_mean":1.640625,"generate/avg_num_tokens":11207.474609375,"generate/avg_tokens_non_zero_rewards":9598.333333333334,"generate/avg_tokens_zero_rewards":11509.888631090487,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26103,"generate/std_num_tokens":4723.91228214023,"loss/avg_final_rewards":0.158203125,"loss/avg_raw_advantages":-0.0021967708598822355,"loss/avg_raw_advantages_abs":0.10900045931339264,"policy/policy_entropy":0.2735761175863445,"policy/policy_kl":0.062205269699916244,"policy/policy_loss":2.1436307164890422e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007634607282852812,"policy/raw_grad_norm":0.0006685256958007812,"reward/avg_pass_at_8":0.296875,"reward/avg_raw_reward":0.158203125,"reward/policy_ref_kl":0.061775900423526764,"timing/step":1554.5684656830272,"trainer/epoch":0}
+{"step":39,"async/staleness_mean":2.25,"generate/avg_num_tokens":8405.46484375,"generate/avg_tokens_non_zero_rewards":9203.6,"generate/avg_tokens_zero_rewards":8299.517699115044,"generate/failed_trajectory_fraction":0.203125,"generate/max_num_tokens":29782,"generate/std_num_tokens":6065.421396210471,"loss/avg_final_rewards":0.1171875,"loss/avg_raw_advantages":-0.007282145321369171,"loss/avg_raw_advantages_abs":0.10211862623691559,"policy/policy_entropy":0.21169885131530464,"policy/policy_kl":0.05152540860581212,"policy/policy_loss":3.563284842300618e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.005000196675609914,"policy/raw_grad_norm":0.0008573532104492188,"reward/avg_pass_at_8":0.21875,"reward/avg_raw_reward":0.1171875,"reward/policy_ref_kl":0.05160372704267502,"timing/step":1517.0903278860496,"trainer/epoch":0}
+{"step":40,"async/staleness_mean":1.71875,"generate/avg_num_tokens":10264.150390625,"generate/avg_tokens_non_zero_rewards":8853.0,"generate/avg_tokens_zero_rewards":10545.058548009367,"generate/failed_trajectory_fraction":0.03125,"generate/max_num_tokens":27401,"generate/std_num_tokens":5438.816656218592,"loss/avg_final_rewards":0.166015625,"loss/avg_raw_advantages":-0.009717313572764397,"loss/avg_raw_advantages_abs":0.10701550543308258,"policy/policy_entropy":0.2645056543406099,"policy/policy_kl":0.06515088659944013,"policy/policy_loss":4.3287500375299715e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007282450075763336,"policy/raw_grad_norm":0.00079345703125,"reward/avg_pass_at_8":0.3125,"reward/avg_raw_reward":0.166015625,"reward/policy_ref_kl":0.06514370441436768,"timing/step":1715.9820904320222,"trainer/epoch":0}
+{"step":41,"async/staleness_mean":1.53125,"generate/avg_num_tokens":9977.291015625,"generate/avg_tokens_non_zero_rewards":8668.717391304348,"generate/avg_tokens_zero_rewards":10106.463519313305,"generate/failed_trajectory_fraction":0.0625,"generate/max_num_tokens":27727,"generate/std_num_tokens":5589.559986182994,"loss/avg_final_rewards":0.08984375,"loss/avg_raw_advantages":-0.00047264707973226905,"loss/avg_raw_advantages_abs":0.08974139392375946,"policy/policy_entropy":0.26719074440188706,"policy/policy_kl":0.06607147789327428,"policy/policy_loss":4.72841698240245e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.004578336624945223,"policy/raw_grad_norm":0.000789642333984375,"reward/avg_pass_at_8":0.32,"reward/avg_raw_reward":0.08984375,"reward/policy_ref_kl":0.06571701169013977,"timing/step":2935.441768702003,"trainer/epoch":0}
+{"step":42,"async/staleness_mean":1.0,"generate/avg_num_tokens":10536.76171875,"generate/avg_tokens_non_zero_rewards":8503.575,"generate/avg_tokens_zero_rewards":10913.277777777777,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":28091,"generate/std_num_tokens":5414.634889463074,"loss/avg_final_rewards":0.15625,"loss/avg_raw_advantages":-0.005558960139751434,"loss/avg_raw_advantages_abs":0.11117872595787048,"policy/policy_entropy":0.29194846423342824,"policy/policy_kl":0.07430687558371574,"policy/policy_loss":2.8779867022876715e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008355631571248523,"policy/raw_grad_norm":0.0007457733154296875,"reward/avg_pass_at_8":0.3125,"reward/avg_raw_reward":0.15625,"reward/policy_ref_kl":0.07432179152965546,"timing/step":1176.8887087280164,"trainer/epoch":0}
+{"step":43,"async/staleness_mean":1.921875,"generate/avg_num_tokens":10297.57421875,"generate/avg_tokens_non_zero_rewards":8563.033613445377,"generate/avg_tokens_zero_rewards":10822.791348600509,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26511,"generate/std_num_tokens":4946.911663078701,"loss/avg_final_rewards":0.232421875,"loss/avg_raw_advantages":-0.004421906545758247,"loss/avg_raw_advantages_abs":0.14453831315040588,"policy/policy_entropy":0.29095891979523003,"policy/policy_kl":0.07546567072859034,"policy/policy_loss":9.062713566265757e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.010480910853402747,"policy/raw_grad_norm":0.0005674362182617188,"reward/avg_pass_at_8":0.40625,"reward/avg_raw_reward":0.232421875,"reward/policy_ref_kl":0.07562603801488876,"timing/step":1340.407153931039,"trainer/epoch":0}
+{"step":44,"async/staleness_mean":2.0,"generate/avg_num_tokens":10058.21484375,"generate/avg_tokens_non_zero_rewards":8509.36923076923,"generate/avg_tokens_zero_rewards":10585.30890052356,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26969,"generate/std_num_tokens":4889.106827932472,"loss/avg_final_rewards":0.25390625,"loss/avg_raw_advantages":-0.0014505119761452079,"loss/avg_raw_advantages_abs":0.10298856347799301,"policy/policy_entropy":0.2833636768627912,"policy/policy_kl":0.07643870066385716,"policy/policy_loss":9.137919185775445e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.009378637018016889,"policy/raw_grad_norm":0.0006361007690429688,"reward/avg_pass_at_8":0.359375,"reward/avg_raw_reward":0.25390625,"reward/policy_ref_kl":0.07653485238552094,"timing/step":1289.3652989440016,"trainer/epoch":0}
+{"step":45,"async/staleness_mean":1.828125,"generate/avg_num_tokens":10600.720703125,"generate/avg_tokens_non_zero_rewards":8160.1612903225805,"generate/avg_tokens_zero_rewards":11142.420047732698,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26308,"generate/std_num_tokens":5573.588983969632,"loss/avg_final_rewards":0.181640625,"loss/avg_raw_advantages":0.002410867949947715,"loss/avg_raw_advantages_abs":0.11823154985904694,"policy/policy_entropy":0.28867044299840927,"policy/policy_kl":0.07763769297162071,"policy/policy_loss":-1.4232071166020432e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00873736634002853,"policy/raw_grad_norm":0.000507354736328125,"reward/avg_pass_at_8":0.359375,"reward/avg_raw_reward":0.181640625,"reward/policy_ref_kl":0.07759134471416473,"timing/step":1540.5377486840007,"trainer/epoch":0}
+{"step":46,"async/staleness_mean":1.78125,"generate/avg_num_tokens":9900.9140625,"generate/avg_tokens_non_zero_rewards":8777.052631578947,"generate/avg_tokens_zero_rewards":10096.816513761469,"generate/failed_trajectory_fraction":0.015625,"generate/max_num_tokens":28203,"generate/std_num_tokens":4972.940664042101,"loss/avg_final_rewards":0.1484375,"loss/avg_raw_advantages":-0.009025480598211288,"loss/avg_raw_advantages_abs":0.09734418988227844,"policy/policy_entropy":0.298495233990252,"policy/policy_kl":0.08124065725132823,"policy/policy_loss":2.027448758212813e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.005788561053122976,"policy/raw_grad_norm":0.0006284713745117188,"reward/avg_pass_at_8":0.328125,"reward/avg_raw_reward":0.1484375,"reward/policy_ref_kl":0.08133600652217865,"timing/step":1573.1666124730255,"trainer/epoch":0}
+{"step":47,"async/staleness_mean":1.8125,"generate/avg_num_tokens":11181.4921875,"generate/avg_tokens_non_zero_rewards":10087.78947368421,"generate/avg_tokens_zero_rewards":11318.505494505494,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":30154,"generate/std_num_tokens":5712.809224051483,"loss/avg_final_rewards":0.111328125,"loss/avg_raw_advantages":-0.0036313156597316265,"loss/avg_raw_advantages_abs":0.07884193956851959,"policy/policy_entropy":0.2901087701320648,"policy/policy_kl":0.07752084656385705,"policy/policy_loss":1.3416152704337492e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006538600405292527,"policy/raw_grad_norm":0.0005884170532226562,"reward/avg_pass_at_8":0.265625,"reward/avg_raw_reward":0.111328125,"reward/policy_ref_kl":0.07736596465110779,"timing/step":1551.934252355015,"trainer/epoch":0}
+{"step":48,"async/staleness_mean":1.421875,"generate/avg_num_tokens":9596.197265625,"generate/avg_tokens_non_zero_rewards":7493.898876404494,"generate/avg_tokens_zero_rewards":10038.524822695035,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27621,"generate/std_num_tokens":4987.533391389666,"loss/avg_final_rewards":0.173828125,"loss/avg_raw_advantages":-0.004873102530837059,"loss/avg_raw_advantages_abs":0.10169121623039246,"policy/policy_entropy":0.3064133394509554,"policy/policy_kl":0.08607757720164955,"policy/policy_loss":1.3583721454324404e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.010139810141481576,"policy/raw_grad_norm":0.0005970001220703125,"reward/avg_pass_at_8":0.27450980392156865,"reward/avg_raw_reward":0.173828125,"reward/policy_ref_kl":0.08597256243228912,"timing/step":2194.1954475599923,"trainer/epoch":0}
+{"step":49,"async/staleness_mean":1.0,"generate/avg_num_tokens":10685.990234375,"generate/avg_tokens_non_zero_rewards":8742.588235294117,"generate/avg_tokens_zero_rewards":10983.628378378378,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":28891,"generate/std_num_tokens":5382.0666056051605,"loss/avg_final_rewards":0.1328125,"loss/avg_raw_advantages":-0.003280358389019966,"loss/avg_raw_advantages_abs":0.09629471600055695,"policy/policy_entropy":0.3008341323584318,"policy/policy_kl":0.0807173497742042,"policy/policy_loss":1.5879632577764369e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007545516091340687,"policy/raw_grad_norm":0.0005970001220703125,"reward/avg_pass_at_8":0.28125,"reward/avg_raw_reward":0.1328125,"reward/policy_ref_kl":0.08056585490703583,"timing/step":1745.5779216710362,"trainer/epoch":0}
+{"step":50,"async/staleness_mean":1.640625,"generate/avg_num_tokens":10817.05078125,"generate/avg_tokens_non_zero_rewards":8993.477477477478,"generate/avg_tokens_zero_rewards":11321.83042394015,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":30593,"generate/std_num_tokens":5582.4411671006455,"loss/avg_final_rewards":0.216796875,"loss/avg_raw_advantages":0.003819285659119487,"loss/avg_raw_advantages_abs":0.1059006080031395,"policy/policy_entropy":0.29654152737930417,"policy/policy_kl":0.08111756335711107,"policy/policy_loss":-1.0974637518756936e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.010399492329270288,"policy/raw_grad_norm":0.0004949569702148438,"reward/avg_pass_at_8":0.359375,"reward/avg_raw_reward":0.216796875,"reward/policy_ref_kl":0.08103735744953156,"timing/step":1492.108971470967,"trainer/epoch":0}
+{"step":51,"async/staleness_mean":1.78125,"generate/avg_num_tokens":9212.404296875,"generate/avg_tokens_non_zero_rewards":7869.661971830986,"generate/avg_tokens_zero_rewards":9428.58276643991,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":29746,"generate/std_num_tokens":4732.702635178805,"loss/avg_final_rewards":0.138671875,"loss/avg_raw_advantages":-0.0010487546678632498,"loss/avg_raw_advantages_abs":0.10335904359817505,"policy/policy_entropy":0.3097125142812729,"policy/policy_kl":0.08953588572330773,"policy/policy_loss":-5.385828671933268e-09,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00730696719892876,"policy/raw_grad_norm":0.0005626678466796875,"reward/avg_pass_at_8":0.28125,"reward/avg_raw_reward":0.138671875,"reward/policy_ref_kl":0.08960030972957611,"timing/step":1142.907168699021,"trainer/epoch":0}
+{"step":52,"async/staleness_mean":1.9375,"generate/avg_num_tokens":10636.751953125,"generate/avg_tokens_non_zero_rewards":8785.070175438597,"generate/avg_tokens_zero_rewards":10868.72087912088,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27896,"generate/std_num_tokens":5002.867257472921,"loss/avg_final_rewards":0.111328125,"loss/avg_raw_advantages":-0.008851923979818821,"loss/avg_raw_advantages_abs":0.08511700481176376,"policy/policy_entropy":0.3066310998983681,"policy/policy_kl":0.08284365886356682,"policy/policy_loss":2.6972222002541457e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006021968355526042,"policy/raw_grad_norm":0.0007982254028320312,"reward/avg_pass_at_8":0.21875,"reward/avg_raw_reward":0.111328125,"reward/policy_ref_kl":0.08285485208034515,"timing/step":1686.3284518159926,"trainer/epoch":0}
+{"step":53,"async/staleness_mean":1.5,"generate/avg_num_tokens":9240.810546875,"generate/avg_tokens_non_zero_rewards":8683.781609195403,"generate/avg_tokens_zero_rewards":9354.837647058823,"generate/failed_trajectory_fraction":0.046875,"generate/max_num_tokens":26390,"generate/std_num_tokens":5028.114342869192,"loss/avg_final_rewards":0.169921875,"loss/avg_raw_advantages":0.0007180176908150315,"loss/avg_raw_advantages_abs":0.08501230180263519,"policy/policy_entropy":0.3085846765898168,"policy/policy_kl":0.08571053645573556,"policy/policy_loss":-2.7430048277210517e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008424479451605293,"policy/raw_grad_norm":0.00045299530029296875,"reward/avg_pass_at_8":0.3018867924528302,"reward/avg_raw_reward":0.169921875,"reward/policy_ref_kl":0.08584316074848175,"timing/step":2707.7360814409913,"trainer/epoch":0}
+{"step":54,"async/staleness_mean":1.0,"generate/avg_num_tokens":10869.625,"generate/avg_tokens_non_zero_rewards":7907.75,"generate/avg_tokens_zero_rewards":11385.915137614678,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":30345,"generate/std_num_tokens":5610.1162222670355,"loss/avg_final_rewards":0.1484375,"loss/avg_raw_advantages":-0.0004573550831992179,"loss/avg_raw_advantages_abs":0.10319145023822784,"policy/policy_entropy":0.31978797167539597,"policy/policy_kl":0.08748819341417402,"policy/policy_loss":3.086834325927157e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007280157577042701,"policy/raw_grad_norm":0.00063323974609375,"reward/avg_pass_at_8":0.296875,"reward/avg_raw_reward":0.1484375,"reward/policy_ref_kl":0.08748757839202881,"timing/step":1417.1502681659767,"trainer/epoch":0}
+{"step":55,"async/staleness_mean":1.765625,"generate/avg_num_tokens":9611.40234375,"generate/avg_tokens_non_zero_rewards":7751.168674698795,"generate/avg_tokens_zero_rewards":9971.307692307691,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25938,"generate/std_num_tokens":5014.403397790784,"loss/avg_final_rewards":0.162109375,"loss/avg_raw_advantages":-0.003854760667309165,"loss/avg_raw_advantages_abs":0.09948883950710297,"policy/policy_entropy":0.31978206569328904,"policy/policy_kl":0.09096269321162254,"policy/policy_loss":1.069530568997834e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007452300071236095,"policy/raw_grad_norm":0.0005474090576171875,"reward/avg_pass_at_8":0.328125,"reward/avg_raw_reward":0.162109375,"reward/policy_ref_kl":0.09101472049951553,"timing/step":1333.1662747240043,"trainer/epoch":0}
+{"step":56,"async/staleness_mean":1.9375,"generate/avg_num_tokens":9777.513671875,"generate/avg_tokens_non_zero_rewards":8637.564102564103,"generate/avg_tokens_zero_rewards":9982.389400921658,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27295,"generate/std_num_tokens":4822.379741466533,"loss/avg_final_rewards":0.15234375,"loss/avg_raw_advantages":-0.0023286757059395313,"loss/avg_raw_advantages_abs":0.11182728409767151,"policy/policy_entropy":0.3255767119117081,"policy/policy_kl":0.08985532412771136,"policy/policy_loss":1.1031983149223379e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007804233415299677,"policy/raw_grad_norm":0.0005512237548828125,"reward/avg_pass_at_8":0.296875,"reward/avg_raw_reward":0.15234375,"reward/policy_ref_kl":0.0899093821644783,"timing/step":1450.312160749978,"trainer/epoch":0}
+{"step":57,"async/staleness_mean":1.84375,"generate/avg_num_tokens":10273.80859375,"generate/avg_tokens_non_zero_rewards":8863.728971962617,"generate/avg_tokens_zero_rewards":10646.348148148149,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27926,"generate/std_num_tokens":5138.054283747754,"loss/avg_final_rewards":0.208984375,"loss/avg_raw_advantages":0.002836314495652914,"loss/avg_raw_advantages_abs":0.11798383295536041,"policy/policy_entropy":0.3284221440553665,"policy/policy_kl":0.09197159064933658,"policy/policy_loss":-7.414994129817387e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008859603390192206,"policy/raw_grad_norm":0.000518798828125,"reward/avg_pass_at_8":0.359375,"reward/avg_raw_reward":0.208984375,"reward/policy_ref_kl":0.09176859259605408,"timing/step":1485.4764534829883,"trainer/epoch":0}
+{"step":58,"async/staleness_mean":1.765625,"generate/avg_num_tokens":10117.720703125,"generate/avg_tokens_non_zero_rewards":7939.894117647059,"generate/avg_tokens_zero_rewards":10551.245901639344,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":29805,"generate/std_num_tokens":5339.251392082286,"loss/avg_final_rewards":0.166015625,"loss/avg_raw_advantages":-0.0033337173517793417,"loss/avg_raw_advantages_abs":0.09456609189510345,"policy/policy_entropy":0.3271936837118119,"policy/policy_kl":0.09136124106589705,"policy/policy_loss":1.2400030691139818e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006373711056767206,"policy/raw_grad_norm":0.0005178451538085938,"reward/avg_pass_at_8":0.34375,"reward/avg_raw_reward":0.166015625,"reward/policy_ref_kl":0.09111484885215759,"timing/step":1331.5946923530428,"trainer/epoch":0}
+{"step":59,"async/staleness_mean":1.796875,"generate/avg_num_tokens":10487.126953125,"generate/avg_tokens_non_zero_rewards":9100.972222222223,"generate/avg_tokens_zero_rewards":10713.952272727272,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":28703,"generate/std_num_tokens":5093.622063337373,"loss/avg_final_rewards":0.140625,"loss/avg_raw_advantages":-0.004352542106062174,"loss/avg_raw_advantages_abs":0.11413611471652985,"policy/policy_entropy":0.3185904249548912,"policy/policy_kl":0.08968210138846189,"policy/policy_loss":2.0084965655087217e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007056174211811594,"policy/raw_grad_norm":0.0006666183471679688,"reward/avg_pass_at_8":0.28125,"reward/avg_raw_reward":0.140625,"reward/policy_ref_kl":0.08973187208175659,"timing/step":1525.6629140709992,"trainer/epoch":0}
+{"step":60,"async/staleness_mean":1.734375,"generate/avg_num_tokens":10568.05078125,"generate/avg_tokens_non_zero_rewards":8820.26168224299,"generate/avg_tokens_zero_rewards":11029.812345679013,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":30370,"generate/std_num_tokens":5390.531208727486,"loss/avg_final_rewards":0.208984375,"loss/avg_raw_advantages":-0.013820058666169643,"loss/avg_raw_advantages_abs":0.15290656685829163,"policy/policy_entropy":0.3355519655160606,"policy/policy_kl":0.09337997727561742,"policy/policy_loss":2.983205753537277e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.011954482946748612,"policy/raw_grad_norm":0.0006046295166015625,"reward/avg_pass_at_8":0.421875,"reward/avg_raw_reward":0.208984375,"reward/policy_ref_kl":0.09311745315790176,"timing/step":1703.9064403590164,"trainer/epoch":0}
+{"step":61,"async/staleness_mean":1.34375,"generate/avg_num_tokens":10107.814453125,"generate/avg_tokens_non_zero_rewards":9226.115384615385,"generate/avg_tokens_zero_rewards":10207.484782608695,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27619,"generate/std_num_tokens":5402.306737738911,"loss/avg_final_rewards":0.1015625,"loss/avg_raw_advantages":0.0023994946386665106,"loss/avg_raw_advantages_abs":0.08774221688508987,"policy/policy_entropy":0.33941745944321156,"policy/policy_kl":0.09455709846224636,"policy/policy_loss":-3.020650041207773e-08,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006432388067878492,"policy/raw_grad_norm":0.0005197525024414062,"reward/avg_pass_at_8":0.2916666666666667,"reward/avg_raw_reward":0.1015625,"reward/policy_ref_kl":0.0943375676870346,"timing/step":2638.451030005992,"trainer/epoch":0}
+{"step":62,"async/staleness_mean":0.71875,"generate/avg_num_tokens":10593.521484375,"generate/avg_tokens_non_zero_rewards":8387.910256410256,"generate/avg_tokens_zero_rewards":10989.921658986175,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":30539,"generate/std_num_tokens":5503.567906254739,"loss/avg_final_rewards":0.15234375,"loss/avg_raw_advantages":-0.003986469469964504,"loss/avg_raw_advantages_abs":0.08826606720685959,"policy/policy_entropy":0.34060708060860634,"policy/policy_kl":0.0924252487020567,"policy/policy_loss":1.417391068514462e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008016829119696922,"policy/raw_grad_norm":0.0005979537963867188,"reward/avg_pass_at_8":0.2777777777777778,"reward/avg_raw_reward":0.15234375,"reward/policy_ref_kl":0.09239897131919861,"timing/step":2985.9529779309523,"trainer/epoch":0}
+{"step":63,"async/staleness_mean":1.0,"generate/avg_num_tokens":11527.576171875,"generate/avg_tokens_non_zero_rewards":9544.456790123457,"generate/avg_tokens_zero_rewards":11900.273781902551,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27873,"generate/std_num_tokens":5982.690258778161,"loss/avg_final_rewards":0.158203125,"loss/avg_raw_advantages":-0.008965528570115566,"loss/avg_raw_advantages_abs":0.12020907551050186,"policy/policy_entropy":0.3500221772119403,"policy/policy_kl":0.09282010642345995,"policy/policy_loss":2.637731736143678e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00969673465260712,"policy/raw_grad_norm":0.0006561279296875,"reward/avg_pass_at_8":0.3125,"reward/avg_raw_reward":0.158203125,"reward/policy_ref_kl":0.09269394725561142,"timing/step":1277.2689383979887,"trainer/epoch":0}
+{"step":64,"async/staleness_mean":1.9375,"generate/avg_num_tokens":10973.4921875,"generate/avg_tokens_non_zero_rewards":10628.828571428572,"generate/avg_tokens_zero_rewards":11103.20430107527,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":30246,"generate/std_num_tokens":5424.169229298065,"loss/avg_final_rewards":0.2734375,"loss/avg_raw_advantages":0.002175062196329236,"loss/avg_raw_advantages_abs":0.20116810500621796,"policy/policy_entropy":0.34684486081823707,"policy/policy_kl":0.09444799763150513,"policy/policy_loss":7.679060232135271e-08,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.01593458792012825,"policy/raw_grad_norm":0.0004687309265136719,"reward/avg_pass_at_8":0.5,"reward/avg_raw_reward":0.2734375,"reward/policy_ref_kl":0.0942552387714386,"timing/step":1403.5921690840041,"trainer/epoch":0}
+{"step":65,"async/staleness_mean":1.953125,"generate/avg_num_tokens":9798.30078125,"generate/avg_tokens_non_zero_rewards":9523.988636363636,"generate/avg_tokens_zero_rewards":9855.233490566037,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26037,"generate/std_num_tokens":4810.207675577728,"loss/avg_final_rewards":0.171875,"loss/avg_raw_advantages":-0.00014144131273496896,"loss/avg_raw_advantages_abs":0.1151258647441864,"policy/policy_entropy":0.3597131953574717,"policy/policy_kl":0.09977425576653332,"policy/policy_loss":5.356295567082725e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008647990531244432,"policy/raw_grad_norm":0.000553131103515625,"reward/avg_pass_at_8":0.328125,"reward/avg_raw_reward":0.171875,"reward/policy_ref_kl":0.09958639740943909,"timing/step":1209.0861944250064,"trainer/epoch":0}
+{"step":66,"async/staleness_mean":1.90625,"generate/avg_num_tokens":10493.5234375,"generate/avg_tokens_non_zero_rewards":8840.578947368422,"generate/avg_tokens_zero_rewards":10781.651376146789,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":29457,"generate/std_num_tokens":4825.382862424694,"loss/avg_final_rewards":0.1484375,"loss/avg_raw_advantages":-0.007219565100967884,"loss/avg_raw_advantages_abs":0.14186859130859375,"policy/policy_entropy":0.3472798904404044,"policy/policy_kl":0.09657934552524239,"policy/policy_loss":2.312569620244176e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.009847354898738558,"policy/raw_grad_norm":0.0005817413330078125,"reward/avg_pass_at_8":0.359375,"reward/avg_raw_reward":0.1484375,"reward/policy_ref_kl":0.0962652713060379,"timing/step":1507.7622687380062,"trainer/epoch":0}
+{"step":67,"async/staleness_mean":1.671875,"generate/avg_num_tokens":10204.669921875,"generate/avg_tokens_non_zero_rewards":8814.088235294117,"generate/avg_tokens_zero_rewards":10417.641891891892,"generate/failed_trajectory_fraction":0.017578125,"generate/max_num_tokens":27104,"generate/std_num_tokens":5024.6995714795285,"loss/avg_final_rewards":0.1328125,"loss/avg_raw_advantages":0.002797940280288458,"loss/avg_raw_advantages_abs":0.0874834731221199,"policy/policy_entropy":0.3441550927236676,"policy/policy_kl":0.09555600339081138,"policy/policy_loss":-1.4268640953218892e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006901480337546673,"policy/raw_grad_norm":0.0006389617919921875,"reward/avg_pass_at_8":0.25,"reward/avg_raw_reward":0.1328125,"reward/policy_ref_kl":0.09542868286371231,"timing/step":1753.9922892030445,"trainer/epoch":0}
+{"step":68,"async/staleness_mean":1.28125,"generate/avg_num_tokens":9952.908203125,"generate/avg_tokens_non_zero_rewards":10115.307692307691,"generate/avg_tokens_zero_rewards":9923.721198156682,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27273,"generate/std_num_tokens":4757.378276713468,"loss/avg_final_rewards":0.15234375,"loss/avg_raw_advantages":-0.0028747457545250654,"loss/avg_raw_advantages_abs":0.12735013663768768,"policy/policy_entropy":0.36331964284181595,"policy/policy_kl":0.10026614053640515,"policy/policy_loss":5.572444106149987e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008835620532408939,"policy/raw_grad_norm":0.0005092620849609375,"reward/avg_pass_at_8":0.3958333333333333,"reward/avg_raw_reward":0.15234375,"reward/policy_ref_kl":0.10013977438211441,"timing/step":3008.0987080579507,"trainer/epoch":0}
+{"step":69,"async/staleness_mean":1.0,"generate/avg_num_tokens":10636.716796875,"generate/avg_tokens_non_zero_rewards":9431.893805309735,"generate/avg_tokens_zero_rewards":10977.932330827067,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27546,"generate/std_num_tokens":5162.446486544593,"loss/avg_final_rewards":0.220703125,"loss/avg_raw_advantages":-0.0080715361982584,"loss/avg_raw_advantages_abs":0.14897999167442322,"policy/policy_entropy":0.36579873878508806,"policy/policy_kl":0.09825965412892401,"policy/policy_loss":4.12654564740933e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.01208015432166576,"policy/raw_grad_norm":0.0006504058837890625,"reward/avg_pass_at_8":0.375,"reward/avg_raw_reward":0.220703125,"reward/policy_ref_kl":0.09793112426996231,"timing/step":1484.0231077329954,"trainer/epoch":0}
+{"step":70,"async/staleness_mean":1.796875,"generate/avg_num_tokens":10542.96484375,"generate/avg_tokens_non_zero_rewards":9105.492307692308,"generate/avg_tokens_zero_rewards":10751.993288590604,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26902,"generate/std_num_tokens":4776.69326639926,"loss/avg_final_rewards":0.126953125,"loss/avg_raw_advantages":-0.000555152480956167,"loss/avg_raw_advantages_abs":0.08110947161912918,"policy/policy_entropy":0.37231709295883775,"policy/policy_kl":0.1008207107661292,"policy/policy_loss":3.4711339935711294e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.005745726210079738,"policy/raw_grad_norm":0.0006341934204101562,"reward/avg_pass_at_8":0.25,"reward/avg_raw_reward":0.126953125,"reward/policy_ref_kl":0.10062971711158752,"timing/step":1629.6555689849774,"trainer/epoch":0}
+{"step":71,"async/staleness_mean":1.6875,"generate/avg_num_tokens":10125.859375,"generate/avg_tokens_non_zero_rewards":8726.63,"generate/avg_tokens_zero_rewards":10465.478155339806,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27586,"generate/std_num_tokens":4559.04763129794,"loss/avg_final_rewards":0.1953125,"loss/avg_raw_advantages":0.0031303002033382654,"loss/avg_raw_advantages_abs":0.1446843296289444,"policy/policy_entropy":0.3700702306814492,"policy/policy_kl":0.10148334677796811,"policy/policy_loss":1.4780505352973705e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.011877982949044963,"policy/raw_grad_norm":0.0004940032958984375,"reward/avg_pass_at_8":0.421875,"reward/avg_raw_reward":0.1953125,"reward/policy_ref_kl":0.10134249925613403,"timing/step":1743.012564742996,"trainer/epoch":0}
+{"step":72,"async/staleness_mean":2.25,"generate/avg_num_tokens":7939.7421875,"generate/avg_tokens_non_zero_rewards":8348.45,"generate/avg_tokens_zero_rewards":7864.055555555556,"generate/failed_trajectory_fraction":0.234375,"generate/max_num_tokens":26377,"generate/std_num_tokens":6389.370781564799,"loss/avg_final_rewards":0.15625,"loss/avg_raw_advantages":-0.010004510171711445,"loss/avg_raw_advantages_abs":0.1339646577835083,"policy/policy_entropy":0.288550419267267,"policy/policy_kl":0.0774159679422155,"policy/policy_loss":2.331978716796357e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.009204388855323486,"policy/raw_grad_norm":0.0006589889526367188,"reward/avg_pass_at_8":0.296875,"reward/avg_raw_reward":0.15625,"reward/policy_ref_kl":0.07740311324596405,"timing/step":1510.6720685529872,"trainer/epoch":0}
+{"step":73,"async/staleness_mean":1.734375,"generate/avg_num_tokens":10401.306640625,"generate/avg_tokens_non_zero_rewards":8600.776315789473,"generate/avg_tokens_zero_rewards":10715.160550458715,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":28289,"generate/std_num_tokens":4609.855109499745,"loss/avg_final_rewards":0.1484375,"loss/avg_raw_advantages":-0.010888329707086086,"loss/avg_raw_advantages_abs":0.0975562259554863,"policy/policy_entropy":0.3721249010413885,"policy/policy_kl":0.10344237240497023,"policy/policy_loss":3.4217064559527444e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007225164272313123,"policy/raw_grad_norm":0.000637054443359375,"reward/avg_pass_at_8":0.3125,"reward/avg_raw_reward":0.1484375,"reward/policy_ref_kl":0.10350901633501053,"timing/step":1667.215378700057,"trainer/epoch":0}
+{"step":74,"async/staleness_mean":1.6875,"generate/avg_num_tokens":10855.0703125,"generate/avg_tokens_non_zero_rewards":9874.043956043955,"generate/avg_tokens_zero_rewards":11067.121140142517,"generate/failed_trajectory_fraction":0.015625,"generate/max_num_tokens":28029,"generate/std_num_tokens":5162.87936285254,"loss/avg_final_rewards":0.177734375,"loss/avg_raw_advantages":-0.0060962773859500885,"loss/avg_raw_advantages_abs":0.14381326735019684,"policy/policy_entropy":0.3851857129484415,"policy/policy_kl":0.10112944734282792,"policy/policy_loss":1.934143245563291e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.010585801082015678,"policy/raw_grad_norm":0.0006465911865234375,"reward/avg_pass_at_8":0.40625,"reward/avg_raw_reward":0.177734375,"reward/policy_ref_kl":0.10120263695716858,"timing/step":1876.7345566349686,"trainer/epoch":0}
+{"step":75,"async/staleness_mean":2.015625,"generate/avg_num_tokens":9523.41796875,"generate/avg_tokens_non_zero_rewards":10488.574468085106,"generate/avg_tokens_zero_rewards":9425.864516129031,"generate/failed_trajectory_fraction":0.125,"generate/max_num_tokens":26207,"generate/std_num_tokens":6027.9584108258305,"loss/avg_final_rewards":0.091796875,"loss/avg_raw_advantages":-0.005430603865534067,"loss/avg_raw_advantages_abs":0.1059478148818016,"policy/policy_entropy":0.34921422926709056,"policy/policy_kl":0.09285718202590942,"policy/policy_loss":2.9216338575110967e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006882215475343401,"policy/raw_grad_norm":0.001132965087890625,"reward/avg_pass_at_8":0.203125,"reward/avg_raw_reward":0.091796875,"reward/policy_ref_kl":0.09273342788219452,"timing/step":1711.0288707740256,"trainer/epoch":0}
+{"step":76,"async/staleness_mean":2.15625,"generate/avg_num_tokens":9774.830078125,"generate/avg_tokens_non_zero_rewards":8967.0,"generate/avg_tokens_zero_rewards":9911.312785388129,"generate/failed_trajectory_fraction":0.140625,"generate/max_num_tokens":28625,"generate/std_num_tokens":6425.753194576565,"loss/avg_final_rewards":0.14453125,"loss/avg_raw_advantages":-0.0015279221115633845,"loss/avg_raw_advantages_abs":0.09564267843961716,"policy/policy_entropy":0.3453949235845357,"policy/policy_kl":0.09004482871387154,"policy/policy_loss":1.2386992764845672e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006416692681796121,"policy/raw_grad_norm":0.0007715225219726562,"reward/avg_pass_at_8":0.265625,"reward/avg_raw_reward":0.14453125,"reward/policy_ref_kl":0.08992964029312134,"timing/step":1737.6250440620352,"trainer/epoch":0}

viewer/build/inputs/marin/runs/marin-q3c-tt-x3-kl0p001/run.json

@@ -0,0 +1,29 @@
+{
+ "id": "marin-q3c-tt-x3-kl0p001",
+ "title": "TaskTrove RL, KL 0.001 (Qwen3-Coder-30B-A3B)",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://huggingface.co/laion/tt-x3_kl-kl0p001-76-30B/tree/main/training_logs",
+ "license": "apache-2.0",
+ "model": "laion/tt-x3_kl-kl0p001-76-30B",
+ "base_model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
+ "method": "GRPO (SkyRL + Terminus-2, pass-ratio shaped verifier reward)",
+ "dataset": "DCAgent/exp_rpt_multifile",
+ "eval_suite": null,
+ "kind": "training",
+ "state": "finished",
+ "started_at": null,
+ "updated_at": "2026-09-04T16:23:02Z",
+ "attempts": 0,
+ "note": "Marin TaskTrove RL hyperparameter ablation (issue #7785) on Qwen3-Coder-30B-A3B-Instruct with DCAgent/exp_rpt_multifile tasks and the Terminus-2 harness; arm: X3 KL arm: KL coefficient 0.001 with a reference model. Our copy has every logged step of the final lineage (15 log segments).",
+ "metrics_map": {
+  "reward": "reward/avg_raw_reward",
+  "loss": "policy/policy_loss",
+  "entropy": "policy/policy_entropy",
+  "lr": "policy/policy_lr",
+  "grad_norm": "policy/raw_grad_norm",
+  "response_length": "generate/avg_num_tokens",
+  "kl": "policy/policy_kl"
+ }
+}

viewer/build/inputs/marin/runs/marin-q3c-tt-x5-gradnorm0p45/metrics.jsonl

@@ -0,0 +1,68 @@
+{"step":1,"async/staleness_mean":0.0,"generate/avg_num_tokens":11791.3984375,"generate/avg_tokens_non_zero_rewards":10858.223684210527,"generate/avg_tokens_zero_rewards":11954.061926605504,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":23222,"generate/std_num_tokens":3710.802324569285,"loss/avg_final_rewards":0.1484375,"loss/avg_raw_advantages":0.0018616283778101206,"loss/avg_raw_advantages_abs":0.0788487046957016,"policy/policy_entropy":0.12270080274902284,"policy/policy_loss":-1.3846631503611206e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.0039631352465221426,"policy/raw_grad_norm":0.00193023681640625,"reward/avg_pass_at_8":0.265625,"reward/avg_raw_reward":0.1484375,"timing/step":5539.47597769089,"trainer/epoch":0}
+{"step":2,"async/staleness_mean":1.0,"generate/avg_num_tokens":12140.3828125,"generate/avg_tokens_non_zero_rewards":11862.845070422536,"generate/avg_tokens_zero_rewards":12185.065759637188,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26957,"generate/std_num_tokens":3823.8181637662033,"loss/avg_final_rewards":0.138671875,"loss/avg_raw_advantages":-0.0014097518287599087,"loss/avg_raw_advantages_abs":0.11524277925491333,"policy/policy_entropy":0.1276655530091375,"policy/policy_loss":6.059068411445878e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.0052097445632171,"policy/raw_grad_norm":0.00159454345703125,"reward/avg_pass_at_8":0.296875,"reward/avg_raw_reward":0.138671875,"timing/step":2612.8492291190196,"trainer/epoch":0}
+{"step":3,"async/staleness_mean":1.734375,"generate/avg_num_tokens":12078.0390625,"generate/avg_tokens_non_zero_rewards":11067.16923076923,"generate/avg_tokens_zero_rewards":12225.03355704698,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24160,"generate/std_num_tokens":3585.8093210257807,"loss/avg_final_rewards":0.126953125,"loss/avg_raw_advantages":-0.00024399122048635036,"loss/avg_raw_advantages_abs":0.10163431614637375,"policy/policy_entropy":0.13537711673416197,"policy/policy_loss":1.66076262075876e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.004986432578334643,"policy/raw_grad_norm":0.002044677734375,"reward/avg_pass_at_8":0.25,"reward/avg_raw_reward":0.126953125,"timing/step":2766.697756807087,"trainer/epoch":0}
+{"step":4,"async/staleness_mean":1.859375,"generate/avg_num_tokens":11823.015625,"generate/avg_tokens_non_zero_rewards":11380.34693877551,"generate/avg_tokens_zero_rewards":11927.80193236715,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24404,"generate/std_num_tokens":3535.6687646097603,"loss/avg_final_rewards":0.19140625,"loss/avg_raw_advantages":0.00661264406517148,"loss/avg_raw_advantages_abs":0.14293375611305237,"policy/policy_entropy":0.14035974472062662,"policy/policy_loss":-7.031288298264826e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007570894845230214,"policy/raw_grad_norm":0.0012493133544921875,"reward/avg_pass_at_8":0.4375,"reward/avg_raw_reward":0.19140625,"timing/step":2686.3743848078884,"trainer/epoch":0}
+{"step":5,"async/staleness_mean":2.046875,"generate/avg_num_tokens":12326.57421875,"generate/avg_tokens_non_zero_rewards":11862.416666666666,"generate/avg_tokens_zero_rewards":12388.188053097345,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25414,"generate/std_num_tokens":3628.3863334982093,"loss/avg_final_rewards":0.1171875,"loss/avg_raw_advantages":-0.0026458406355232,"loss/avg_raw_advantages_abs":0.07605545967817307,"policy/policy_entropy":0.14718504570191726,"policy/policy_loss":1.0213674599413025e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.003797877667238936,"policy/raw_grad_norm":0.001953125,"reward/avg_pass_at_8":0.21875,"reward/avg_raw_reward":0.1171875,"timing/step":3229.0331079550087,"trainer/epoch":0}
+{"step":6,"async/staleness_mean":2.015625,"generate/avg_num_tokens":12621.51171875,"generate/avg_tokens_non_zero_rewards":12952.682539682539,"generate/avg_tokens_zero_rewards":12575.044543429844,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24664,"generate/std_num_tokens":3639.944873683724,"loss/avg_final_rewards":0.123046875,"loss/avg_raw_advantages":0.003352612489834428,"loss/avg_raw_advantages_abs":0.094528928399086,"policy/policy_entropy":0.1504530839738436,"policy/policy_loss":-6.657595577053144e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.004997343969989743,"policy/raw_grad_norm":0.0013885498046875,"reward/avg_pass_at_8":0.28125,"reward/avg_raw_reward":0.123046875,"timing/step":3105.6845996279735,"trainer/epoch":0}
+{"step":7,"async/staleness_mean":1.84375,"generate/avg_num_tokens":12322.51953125,"generate/avg_tokens_non_zero_rewards":11112.201612903225,"generate/avg_tokens_zero_rewards":12709.322164948453,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25078,"generate/std_num_tokens":4023.680210107692,"loss/avg_final_rewards":0.2421875,"loss/avg_raw_advantages":-0.002092145849019289,"loss/avg_raw_advantages_abs":0.13969583809375763,"policy/policy_entropy":0.1563977369805798,"policy/policy_loss":7.391564977865528e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007780624531733338,"policy/raw_grad_norm":0.001434326171875,"reward/avg_pass_at_8":0.421875,"reward/avg_raw_reward":0.2421875,"timing/step":2946.508635859005,"trainer/epoch":0}
+{"step":8,"async/staleness_mean":1.90625,"generate/avg_num_tokens":11752.962890625,"generate/avg_tokens_non_zero_rewards":10402.192857142858,"generate/avg_tokens_zero_rewards":12261.317204301075,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":23837,"generate/std_num_tokens":3630.953788019378,"loss/avg_final_rewards":0.2734375,"loss/avg_raw_advantages":-0.005398645531386137,"loss/avg_raw_advantages_abs":0.15247876942157745,"policy/policy_entropy":0.1665698119904846,"policy/policy_loss":1.4565656627496537e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.009989192425564397,"policy/raw_grad_norm":0.00139617919921875,"reward/avg_pass_at_8":0.453125,"reward/avg_raw_reward":0.2734375,"timing/step":3158.2748083001934,"trainer/epoch":0}
+{"step":9,"async/staleness_mean":1.9375,"generate/avg_num_tokens":11817.62890625,"generate/avg_tokens_non_zero_rewards":9741.08888888889,"generate/avg_tokens_zero_rewards":12260.492890995261,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":23100,"generate/std_num_tokens":3738.939095720627,"loss/avg_final_rewards":0.17578125,"loss/avg_raw_advantages":-0.0038736003916710615,"loss/avg_raw_advantages_abs":0.086604043841362,"policy/policy_entropy":0.16816551377996802,"policy/policy_loss":2.9437300455015247e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.005893173031381593,"policy/raw_grad_norm":0.0016326904296875,"reward/avg_pass_at_8":0.28125,"reward/avg_raw_reward":0.17578125,"timing/step":3357.954769005999,"trainer/epoch":0}
+{"step":10,"async/staleness_mean":0.828125,"generate/avg_num_tokens":12138.19140625,"generate/avg_tokens_non_zero_rewards":10555.372093023256,"generate/avg_tokens_zero_rewards":12457.727699530516,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24681,"generate/std_num_tokens":3891.8769951521977,"loss/avg_final_rewards":0.16796875,"loss/avg_raw_advantages":-0.002505233511328697,"loss/avg_raw_advantages_abs":0.11009347438812256,"policy/policy_entropy":0.175362812820822,"policy/policy_loss":1.8996596251596998e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007943858899125189,"policy/raw_grad_norm":0.0012531280517578125,"reward/avg_pass_at_8":0.34,"reward/avg_raw_reward":0.16796875,"timing/step":8233.293633271009,"trainer/epoch":0}
+{"step":11,"async/staleness_mean":1.0,"generate/avg_num_tokens":12493.68359375,"generate/avg_tokens_non_zero_rewards":10565.48051948052,"generate/avg_tokens_zero_rewards":12834.997701149425,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26500,"generate/std_num_tokens":4070.1660829572834,"loss/avg_final_rewards":0.150390625,"loss/avg_raw_advantages":-0.0007450665580108762,"loss/avg_raw_advantages_abs":0.1090240329504013,"policy/policy_entropy":0.1830622258130461,"policy/policy_loss":-7.142266547077725e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007385002267710661,"policy/raw_grad_norm":0.001190185546875,"reward/avg_pass_at_8":0.359375,"reward/avg_raw_reward":0.150390625,"timing/step":3904.2820108069573,"trainer/epoch":0}
+{"step":12,"async/staleness_mean":1.625,"generate/avg_num_tokens":11847.392578125,"generate/avg_tokens_non_zero_rewards":9899.443037974683,"generate/avg_tokens_zero_rewards":12202.792147806005,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26786,"generate/std_num_tokens":4081.134515041962,"loss/avg_final_rewards":0.154296875,"loss/avg_raw_advantages":-0.00043515273137018085,"loss/avg_raw_advantages_abs":0.09914527833461761,"policy/policy_entropy":0.18060476693790406,"policy/policy_loss":3.07914053365721e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006132250740847667,"policy/raw_grad_norm":0.001361846923828125,"reward/avg_pass_at_8":0.3125,"reward/avg_raw_reward":0.154296875,"timing/step":4274.771966025,"trainer/epoch":0}
+{"step":13,"async/staleness_mean":0.96875,"generate/avg_num_tokens":11537.6328125,"generate/avg_tokens_non_zero_rewards":10357.143835616438,"generate/avg_tokens_zero_rewards":12008.53825136612,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26204,"generate/std_num_tokens":3918.1012997191433,"loss/avg_final_rewards":0.28515625,"loss/avg_raw_advantages":0.0015004277229309082,"loss/avg_raw_advantages_abs":0.1437944918870926,"policy/policy_entropy":0.18902051635086536,"policy/policy_loss":-5.92013716271822e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008760957843151118,"policy/raw_grad_norm":0.0012569427490234375,"reward/avg_pass_at_8":0.42857142857142855,"reward/avg_raw_reward":0.28515625,"timing/step":8295.136788924923,"trainer/epoch":0}
+{"step":14,"async/staleness_mean":1.0,"generate/avg_num_tokens":11937.884765625,"generate/avg_tokens_non_zero_rewards":11125.27380952381,"generate/avg_tokens_zero_rewards":12097.369158878504,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24149,"generate/std_num_tokens":3882.3090418572638,"loss/avg_final_rewards":0.1640625,"loss/avg_raw_advantages":0.0012504333863034844,"loss/avg_raw_advantages_abs":0.1251000463962555,"policy/policy_entropy":0.19222709105815738,"policy/policy_loss":7.823198941991905e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007415477591166564,"policy/raw_grad_norm":0.001102447509765625,"reward/avg_pass_at_8":0.34375,"reward/avg_raw_reward":0.1640625,"timing/step":4205.859765880043,"trainer/epoch":0}
+{"step":15,"async/staleness_mean":1.703125,"generate/avg_num_tokens":10819.3515625,"generate/avg_tokens_non_zero_rewards":9531.09677419355,"generate/avg_tokens_zero_rewards":11105.28878281623,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25148,"generate/std_num_tokens":3713.4985022756923,"loss/avg_final_rewards":0.181640625,"loss/avg_raw_advantages":-0.001738061779178679,"loss/avg_raw_advantages_abs":0.10060781985521317,"policy/policy_entropy":0.19566651748027653,"policy/policy_loss":1.7838118608892728e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006150492391498119,"policy/raw_grad_norm":0.001346588134765625,"reward/avg_pass_at_8":0.359375,"reward/avg_raw_reward":0.181640625,"timing/step":4588.221793039003,"trainer/epoch":0}
+{"step":16,"async/staleness_mean":1.640625,"generate/avg_num_tokens":11313.685546875,"generate/avg_tokens_non_zero_rewards":9162.0,"generate/avg_tokens_zero_rewards":11760.26179245283,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24972,"generate/std_num_tokens":4199.191746162719,"loss/avg_final_rewards":0.171875,"loss/avg_raw_advantages":6.0792273870902136e-05,"loss/avg_raw_advantages_abs":0.08020862936973572,"policy/policy_entropy":0.19867552362848073,"policy/policy_loss":1.1189188064975042e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.005750729101237084,"policy/raw_grad_norm":0.00128936767578125,"reward/avg_pass_at_8":0.28125,"reward/avg_raw_reward":0.171875,"timing/step":4810.580587374046,"trainer/epoch":0}
+{"step":17,"async/staleness_mean":1.5625,"generate/avg_num_tokens":10844.0546875,"generate/avg_tokens_non_zero_rewards":10470.67415730337,"generate/avg_tokens_zero_rewards":10922.614657210403,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26203,"generate/std_num_tokens":3881.7259879397434,"loss/avg_final_rewards":0.173828125,"loss/avg_raw_advantages":-0.0055550821125507355,"loss/avg_raw_advantages_abs":0.09101925045251846,"policy/policy_entropy":0.20081515645142645,"policy/policy_loss":6.020457654187794e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.004961596513567201,"policy/raw_grad_norm":0.00159454345703125,"reward/avg_pass_at_8":0.28125,"reward/avg_raw_reward":0.173828125,"timing/step":3503.288930212846,"trainer/epoch":0}
+{"step":18,"async/staleness_mean":1.703125,"generate/avg_num_tokens":11015.08203125,"generate/avg_tokens_non_zero_rewards":11637.46052631579,"generate/avg_tokens_zero_rewards":10906.594036697248,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26538,"generate/std_num_tokens":3953.248603935866,"loss/avg_final_rewards":0.1484375,"loss/avg_raw_advantages":0.003047340316697955,"loss/avg_raw_advantages_abs":0.07736817747354507,"policy/policy_entropy":0.2135052295634523,"policy/policy_loss":-1.043862162930509e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.0047767863920853415,"policy/raw_grad_norm":0.0013294219970703125,"reward/avg_pass_at_8":0.265625,"reward/avg_raw_reward":0.1484375,"timing/step":3697.590688298922,"trainer/epoch":0}
+{"step":19,"async/staleness_mean":1.09375,"generate/avg_num_tokens":9763.001953125,"generate/avg_tokens_non_zero_rewards":9869.875,"generate/avg_tokens_zero_rewards":9743.210648148148,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25622,"generate/std_num_tokens":3749.3249639825913,"loss/avg_final_rewards":0.15625,"loss/avg_raw_advantages":0.004701052326709032,"loss/avg_raw_advantages_abs":0.14787836372852325,"policy/policy_entropy":0.21245330886449665,"policy/policy_loss":-7.139672781875106e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008342701584297174,"policy/raw_grad_norm":0.00106048583984375,"reward/avg_pass_at_8":0.3958333333333333,"reward/avg_raw_reward":0.15625,"timing/step":6247.273198880022,"trainer/epoch":0}
+{"step":20,"async/staleness_mean":0.984375,"generate/avg_num_tokens":10311.333984375,"generate/avg_tokens_non_zero_rewards":8615.714285714286,"generate/avg_tokens_zero_rewards":10490.784017278618,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27576,"generate/std_num_tokens":4202.477838627773,"loss/avg_final_rewards":0.095703125,"loss/avg_raw_advantages":-0.004628123715519905,"loss/avg_raw_advantages_abs":0.07581286877393723,"policy/policy_entropy":0.2198514414485544,"policy/policy_loss":2.5055335868273687e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00390106179202121,"policy/raw_grad_norm":0.002239227294921875,"reward/avg_pass_at_8":0.203125,"reward/avg_raw_reward":0.095703125,"timing/step":3683.694798693061,"trainer/epoch":0}
+{"step":21,"async/staleness_mean":1.625,"generate/avg_num_tokens":9981.158203125,"generate/avg_tokens_non_zero_rewards":8986.023529411765,"generate/avg_tokens_zero_rewards":10179.252927400468,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26296,"generate/std_num_tokens":4006.4444556418907,"loss/avg_final_rewards":0.166015625,"loss/avg_raw_advantages":-0.0009517140570096672,"loss/avg_raw_advantages_abs":0.09436691552400589,"policy/policy_entropy":0.22331321274396032,"policy/policy_loss":1.492436542349651e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006505488994662301,"policy/raw_grad_norm":0.0011425018310546875,"reward/avg_pass_at_8":0.296875,"reward/avg_raw_reward":0.166015625,"timing/step":2688.6806287160143,"trainer/epoch":0}
+{"step":22,"async/staleness_mean":1.875,"generate/avg_num_tokens":9693.916015625,"generate/avg_tokens_non_zero_rewards":9522.76923076923,"generate/avg_tokens_zero_rewards":9718.803131991051,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26021,"generate/std_num_tokens":3676.9179846705174,"loss/avg_final_rewards":0.126953125,"loss/avg_raw_advantages":-0.0027303651440888643,"loss/avg_raw_advantages_abs":0.06795687973499298,"policy/policy_entropy":0.22961094707716256,"policy/policy_loss":1.7855831586643944e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.005025325771384814,"policy/raw_grad_norm":0.0013065338134765625,"reward/avg_pass_at_8":0.21875,"reward/avg_raw_reward":0.126953125,"timing/step":2892.5090383028146,"trainer/epoch":0}
+{"step":23,"async/staleness_mean":1.875,"generate/avg_num_tokens":9015.10546875,"generate/avg_tokens_non_zero_rewards":8034.220338983051,"generate/avg_tokens_zero_rewards":9308.873096446701,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":30242,"generate/std_num_tokens":4029.328156793182,"loss/avg_final_rewards":0.23046875,"loss/avg_raw_advantages":-0.0029741472098976374,"loss/avg_raw_advantages_abs":0.13171382248401642,"policy/policy_entropy":0.23563165520317852,"policy/policy_loss":2.2168108451126045e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008840009284540429,"policy/raw_grad_norm":0.0009489059448242188,"reward/avg_pass_at_8":0.390625,"reward/avg_raw_reward":0.23046875,"timing/step":2417.879588205833,"trainer/epoch":0}
+{"step":24,"async/staleness_mean":1.765625,"generate/avg_num_tokens":9006.51953125,"generate/avg_tokens_non_zero_rewards":7572.977011494253,"generate/avg_tokens_zero_rewards":9299.974117647058,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25581,"generate/std_num_tokens":3849.12844279712,"loss/avg_final_rewards":0.169921875,"loss/avg_raw_advantages":-0.0067780502140522,"loss/avg_raw_advantages_abs":0.13192960619926453,"policy/policy_entropy":0.23873403831385076,"policy/policy_loss":1.587046671858161e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008033234577851545,"policy/raw_grad_norm":0.003902435302734375,"reward/avg_pass_at_8":0.34375,"reward/avg_raw_reward":0.169921875,"timing/step":2591.1554496511817,"trainer/epoch":0}
+{"step":25,"async/staleness_mean":1.484375,"generate/avg_num_tokens":9003.896484375,"generate/avg_tokens_non_zero_rewards":7411.025974025974,"generate/avg_tokens_zero_rewards":9285.852873563219,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":29941,"generate/std_num_tokens":4072.509812256459,"loss/avg_final_rewards":0.150390625,"loss/avg_raw_advantages":-0.004779601935297251,"loss/avg_raw_advantages_abs":0.1207507848739624,"policy/policy_entropy":0.2452373158885166,"policy/policy_loss":6.715754450326017e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008138476143358275,"policy/raw_grad_norm":0.00102996826171875,"reward/avg_pass_at_8":0.30357142857142855,"reward/avg_raw_reward":0.150390625,"timing/step":4561.494081897195,"trainer/epoch":0}
+{"step":26,"async/staleness_mean":1.0,"generate/avg_num_tokens":8622.01171875,"generate/avg_tokens_non_zero_rewards":7850.939759036145,"generate/avg_tokens_zero_rewards":8771.193473193473,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25129,"generate/std_num_tokens":3760.8618760390445,"loss/avg_final_rewards":0.162109375,"loss/avg_raw_advantages":0.0017682735342532396,"loss/avg_raw_advantages_abs":0.1285792887210846,"policy/policy_entropy":0.2520464884582907,"policy/policy_loss":2.3826420658679126e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007328983873776451,"policy/raw_grad_norm":0.000949859619140625,"reward/avg_pass_at_8":0.375,"reward/avg_raw_reward":0.162109375,"timing/step":2017.5475478242151,"trainer/epoch":0}
+{"step":27,"async/staleness_mean":1.765625,"generate/avg_num_tokens":9189.955078125,"generate/avg_tokens_non_zero_rewards":8092.058823529412,"generate/avg_tokens_zero_rewards":9358.101351351352,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24839,"generate/std_num_tokens":4323.662929605105,"loss/avg_final_rewards":0.1328125,"loss/avg_raw_advantages":0.004346746951341629,"loss/avg_raw_advantages_abs":0.1056930422782898,"policy/policy_entropy":0.2566796252503991,"policy/policy_loss":-1.22847141881266e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00545892059471953,"policy/raw_grad_norm":0.001018524169921875,"reward/avg_pass_at_8":0.3125,"reward/avg_raw_reward":0.1328125,"timing/step":2277.8071907367557,"trainer/epoch":0}
+{"step":28,"async/staleness_mean":2.125,"generate/avg_num_tokens":8719.71484375,"generate/avg_tokens_non_zero_rewards":7651.050359712231,"generate/avg_tokens_zero_rewards":9117.957104557641,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25411,"generate/std_num_tokens":4251.726803230767,"loss/avg_final_rewards":0.271484375,"loss/avg_raw_advantages":0.010601941496133804,"loss/avg_raw_advantages_abs":0.10304071754217148,"policy/policy_entropy":0.248472306644544,"policy/policy_loss":-1.442182700372996e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.009371816738166672,"policy/raw_grad_norm":0.0008325576782226562,"reward/avg_pass_at_8":0.421875,"reward/avg_raw_reward":0.271484375,"timing/step":2055.8702136264183,"trainer/epoch":0}
+{"step":29,"async/staleness_mean":2.3125,"generate/avg_num_tokens":8530.779296875,"generate/avg_tokens_non_zero_rewards":7675.91011235955,"generate/avg_tokens_zero_rewards":8710.645390070922,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26162,"generate/std_num_tokens":3814.1119998991253,"loss/avg_final_rewards":0.173828125,"loss/avg_raw_advantages":0.006155760493129492,"loss/avg_raw_advantages_abs":0.11663798987865448,"policy/policy_entropy":0.2504065647954121,"policy/policy_loss":-9.027127170213589e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008392806101255701,"policy/raw_grad_norm":0.0007762908935546875,"reward/avg_pass_at_8":0.359375,"reward/avg_raw_reward":0.173828125,"timing/step":2059.2828750237823,"trainer/epoch":0}
+{"step":30,"async/staleness_mean":2.1875,"generate/avg_num_tokens":8033.111328125,"generate/avg_tokens_non_zero_rewards":6752.803571428572,"generate/avg_tokens_zero_rewards":8391.5975,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27098,"generate/std_num_tokens":3348.273771852882,"loss/avg_final_rewards":0.21875,"loss/avg_raw_advantages":-0.008964852429926395,"loss/avg_raw_advantages_abs":0.13920991122722626,"policy/policy_entropy":0.25663610687479377,"policy/policy_loss":1.1815737046561026e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.009418263862244203,"policy/raw_grad_norm":0.0008630752563476562,"reward/avg_pass_at_8":0.421875,"reward/avg_raw_reward":0.21875,"timing/step":1692.8483859929256,"trainer/epoch":0}
+{"step":31,"async/staleness_mean":2.296875,"generate/avg_num_tokens":8216.111328125,"generate/avg_tokens_non_zero_rewards":7300.9358974358975,"generate/avg_tokens_zero_rewards":8380.589861751152,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25948,"generate/std_num_tokens":3551.055464426622,"loss/avg_final_rewards":0.15234375,"loss/avg_raw_advantages":0.001577701885253191,"loss/avg_raw_advantages_abs":0.0884578675031662,"policy/policy_entropy":0.2625905995955691,"policy/policy_loss":9.056617145120072e-08,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.005948080246525933,"policy/raw_grad_norm":0.0010395050048828125,"reward/avg_pass_at_8":0.265625,"reward/avg_raw_reward":0.15234375,"timing/step":1969.9152718950063,"trainer/epoch":0}
+{"step":32,"async/staleness_mean":2.375,"generate/avg_num_tokens":8439.556640625,"generate/avg_tokens_non_zero_rewards":6737.115384615385,"generate/avg_tokens_zero_rewards":8745.52534562212,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":23928,"generate/std_num_tokens":3889.620730917809,"loss/avg_final_rewards":0.15234375,"loss/avg_raw_advantages":-0.006564537063241005,"loss/avg_raw_advantages_abs":0.08425632864236832,"policy/policy_entropy":0.26623984973412007,"policy/policy_loss":1.042172939946795e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006512630303404876,"policy/raw_grad_norm":0.0010509490966796875,"reward/avg_pass_at_8":0.265625,"reward/avg_raw_reward":0.15234375,"timing/step":2227.8955586119555,"trainer/epoch":0}
+{"step":33,"async/staleness_mean":2.375,"generate/avg_num_tokens":8452.41015625,"generate/avg_tokens_non_zero_rewards":7604.746987951808,"generate/avg_tokens_zero_rewards":8616.410256410256,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24504,"generate/std_num_tokens":3941.260987007756,"loss/avg_final_rewards":0.162109375,"loss/avg_raw_advantages":0.0006342987762764096,"loss/avg_raw_advantages_abs":0.1227927878499031,"policy/policy_entropy":0.25510654237587005,"policy/policy_loss":-1.1258239673850312e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008736123881590174,"policy/raw_grad_norm":0.0008411407470703125,"reward/avg_pass_at_8":0.34375,"reward/avg_raw_reward":0.162109375,"timing/step":1880.4757572351955,"trainer/epoch":0}
+{"step":34,"async/staleness_mean":1.96875,"generate/avg_num_tokens":8150.865234375,"generate/avg_tokens_non_zero_rewards":7106.529411764706,"generate/avg_tokens_zero_rewards":8310.808558558558,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24051,"generate/std_num_tokens":3794.4292549091097,"loss/avg_final_rewards":0.1328125,"loss/avg_raw_advantages":0.0008692203555256128,"loss/avg_raw_advantages_abs":0.11227869242429733,"policy/policy_entropy":0.26446314831264317,"policy/policy_loss":3.019232650558479e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.0066033733482981916,"policy/raw_grad_norm":0.0009622573852539062,"reward/avg_pass_at_8":0.3157894736842105,"reward/avg_raw_reward":0.1328125,"timing/step":3583.6257864767686,"trainer/epoch":0}
+{"step":35,"async/staleness_mean":0.96875,"generate/avg_num_tokens":8175.40234375,"generate/avg_tokens_non_zero_rewards":6532.51,"generate/avg_tokens_zero_rewards":8574.162621359223,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25152,"generate/std_num_tokens":4166.211938000545,"loss/avg_final_rewards":0.1953125,"loss/avg_raw_advantages":-0.001226629363372922,"loss/avg_raw_advantages_abs":0.1164650246500969,"policy/policy_entropy":0.2674451186321676,"policy/policy_loss":1.8878955643231166e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008993270268547349,"policy/raw_grad_norm":0.0007581710815429688,"reward/avg_pass_at_8":0.375,"reward/avg_raw_reward":0.1953125,"timing/step":2448.022929954808,"trainer/epoch":0}
+{"step":36,"async/staleness_mean":1.765625,"generate/avg_num_tokens":8569.685546875,"generate/avg_tokens_non_zero_rewards":7602.6760563380285,"generate/avg_tokens_zero_rewards":8725.371882086169,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25172,"generate/std_num_tokens":3781.879460100309,"loss/avg_final_rewards":0.138671875,"loss/avg_raw_advantages":-0.0037242623511701822,"loss/avg_raw_advantages_abs":0.08868999034166336,"policy/policy_entropy":0.26500918704550713,"policy/policy_loss":1.3630032675848724e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006329319606265926,"policy/raw_grad_norm":0.00099945068359375,"reward/avg_pass_at_8":0.265625,"reward/avg_raw_reward":0.138671875,"timing/step":2023.737669208087,"trainer/epoch":0}
+{"step":37,"async/staleness_mean":1.390625,"generate/avg_num_tokens":8359.568359375,"generate/avg_tokens_non_zero_rewards":7296.132075471698,"generate/avg_tokens_zero_rewards":8637.214285714286,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25107,"generate/std_num_tokens":3658.1980978962847,"loss/avg_final_rewards":0.20703125,"loss/avg_raw_advantages":-0.00382994394749403,"loss/avg_raw_advantages_abs":0.11880567669868469,"policy/policy_entropy":0.2647894473047927,"policy/policy_loss":8.986008026568015e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00881478434894234,"policy/raw_grad_norm":0.0008716583251953125,"reward/avg_pass_at_8":0.375,"reward/avg_raw_reward":0.20703125,"timing/step":4794.51497527305,"trainer/epoch":0}
+{"step":38,"async/staleness_mean":1.0,"generate/avg_num_tokens":8729.275390625,"generate/avg_tokens_non_zero_rewards":7117.242424242424,"generate/avg_tokens_zero_rewards":8967.82735426009,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24924,"generate/std_num_tokens":4008.999531369314,"loss/avg_final_rewards":0.12890625,"loss/avg_raw_advantages":-0.0035560934338718653,"loss/avg_raw_advantages_abs":0.09100429713726044,"policy/policy_entropy":0.2688232328509912,"policy/policy_loss":9.768856159553252e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006665974592579005,"policy/raw_grad_norm":0.0009002685546875,"reward/avg_pass_at_8":0.28125,"reward/avg_raw_reward":0.12890625,"timing/step":2481.1378593281843,"trainer/epoch":0}
+{"step":39,"async/staleness_mean":1.84375,"generate/avg_num_tokens":8429.80859375,"generate/avg_tokens_non_zero_rewards":7771.643835616438,"generate/avg_tokens_zero_rewards":8539.25284738041,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24909,"generate/std_num_tokens":3936.226032308885,"loss/avg_final_rewards":0.142578125,"loss/avg_raw_advantages":-0.0018265643157064915,"loss/avg_raw_advantages_abs":0.11231062561273575,"policy/policy_entropy":0.2751005010213703,"policy/policy_loss":1.818506660811181e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006773055443773046,"policy/raw_grad_norm":0.0010776519775390625,"reward/avg_pass_at_8":0.28125,"reward/avg_raw_reward":0.142578125,"timing/step":2332.7048763069324,"trainer/epoch":0}
+{"step":40,"async/staleness_mean":2.140625,"generate/avg_num_tokens":8772.287109375,"generate/avg_tokens_non_zero_rewards":7428.3125,"generate/avg_tokens_zero_rewards":9082.435096153846,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25015,"generate/std_num_tokens":4171.385065050646,"loss/avg_final_rewards":0.1875,"loss/avg_raw_advantages":0.0044614048674702644,"loss/avg_raw_advantages_abs":0.11403487622737885,"policy/policy_entropy":0.27879614918492734,"policy/policy_loss":-3.570694087073889e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007280434857420914,"policy/raw_grad_norm":0.0009517669677734375,"reward/avg_pass_at_8":0.375,"reward/avg_raw_reward":0.1875,"timing/step":2158.4758325950243,"trainer/epoch":0}
+{"step":41,"async/staleness_mean":2.21875,"generate/avg_num_tokens":8716.140625,"generate/avg_tokens_non_zero_rewards":6865.609756097561,"generate/avg_tokens_zero_rewards":9301.269922879177,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":24313,"generate/std_num_tokens":3921.0140130304126,"loss/avg_final_rewards":0.240234375,"loss/avg_raw_advantages":-0.005121576599776745,"loss/avg_raw_advantages_abs":0.10941809415817261,"policy/policy_entropy":0.2802881811512634,"policy/policy_loss":1.3286443572013695e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.009372069907840341,"policy/raw_grad_norm":0.0009822845458984375,"reward/avg_pass_at_8":0.359375,"reward/avg_raw_reward":0.240234375,"timing/step":2380.049254124984,"trainer/epoch":0}
+{"step":42,"async/staleness_mean":2.328125,"generate/avg_num_tokens":8675.287109375,"generate/avg_tokens_non_zero_rewards":7392.808,"generate/avg_tokens_zero_rewards":9089.524547803618,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":23602,"generate/std_num_tokens":3907.880252510155,"loss/avg_final_rewards":0.244140625,"loss/avg_raw_advantages":0.00016087625408545136,"loss/avg_raw_advantages_abs":0.11920905113220215,"policy/policy_entropy":0.2811605960596353,"policy/policy_loss":7.394799261817298e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008903771042241715,"policy/raw_grad_norm":0.0009660720825195312,"reward/avg_pass_at_8":0.390625,"reward/avg_raw_reward":0.244140625,"timing/step":2235.372112269979,"trainer/epoch":0}
+{"step":43,"async/staleness_mean":1.578125,"generate/avg_num_tokens":8146.0,"generate/avg_tokens_non_zero_rewards":6813.911764705882,"generate/avg_tokens_zero_rewards":8477.397560975609,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26114,"generate/std_num_tokens":4020.4304996285846,"loss/avg_final_rewards":0.19921875,"loss/avg_raw_advantages":-0.007802317850291729,"loss/avg_raw_advantages_abs":0.1580718606710434,"policy/policy_entropy":0.2890364667400718,"policy/policy_loss":1.2024663895715548e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.011198079480891465,"policy/raw_grad_norm":0.0008573532104492188,"reward/avg_pass_at_8":0.38461538461538464,"reward/avg_raw_reward":0.19921875,"timing/step":5446.255247236928,"trainer/epoch":0}
+{"step":44,"async/staleness_mean":1.0,"generate/avg_num_tokens":9437.998046875,"generate/avg_tokens_non_zero_rewards":8071.714285714285,"generate/avg_tokens_zero_rewards":9654.3778280543,"generate/failed_trajectory_fraction":0.001953125,"generate/max_num_tokens":27001,"generate/std_num_tokens":4802.998744883821,"loss/avg_final_rewards":0.13671875,"loss/avg_raw_advantages":-0.0052741169929504395,"loss/avg_raw_advantages_abs":0.1189226359128952,"policy/policy_entropy":0.2879157244460657,"policy/policy_loss":1.529355378337982e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007491457441574312,"policy/raw_grad_norm":0.0013790130615234375,"reward/avg_pass_at_8":0.3125,"reward/avg_raw_reward":0.13671875,"timing/step":2913.559509156039,"trainer/epoch":0}
+{"step":45,"async/staleness_mean":1.75,"generate/avg_num_tokens":9041.490234375,"generate/avg_tokens_non_zero_rewards":6774.735294117647,"generate/avg_tokens_zero_rewards":9605.414634146342,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26182,"generate/std_num_tokens":4520.857839544768,"loss/avg_final_rewards":0.19921875,"loss/avg_raw_advantages":-0.0035168039612472057,"loss/avg_raw_advantages_abs":0.09181530028581619,"policy/policy_entropy":0.2909987417515367,"policy/policy_loss":6.327150376961299e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007422954747198673,"policy/raw_grad_norm":0.0009603500366210938,"reward/avg_pass_at_8":0.359375,"reward/avg_raw_reward":0.19921875,"timing/step":3084.2673707830254,"trainer/epoch":0}
+{"step":46,"async/staleness_mean":0.875,"generate/avg_num_tokens":8259.595703125,"generate/avg_tokens_non_zero_rewards":7605.38679245283,"generate/avg_tokens_zero_rewards":8430.399014778324,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26492,"generate/std_num_tokens":4263.425029601015,"loss/avg_final_rewards":0.20703125,"loss/avg_raw_advantages":-0.013118461705744267,"loss/avg_raw_advantages_abs":0.09052544832229614,"policy/policy_entropy":0.3056734409183264,"policy/policy_loss":1.9694319668417393e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00974274522423002,"policy/raw_grad_norm":0.0011081695556640625,"reward/avg_pass_at_8":0.29411764705882354,"reward/avg_raw_reward":0.20703125,"timing/step":7070.034014225006,"trainer/epoch":0}
+{"step":47,"async/staleness_mean":1.0,"generate/avg_num_tokens":10387.423828125,"generate/avg_tokens_non_zero_rewards":9463.77108433735,"generate/avg_tokens_zero_rewards":10566.125874125873,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26375,"generate/std_num_tokens":5227.461121371239,"loss/avg_final_rewards":0.162109375,"loss/avg_raw_advantages":-0.00417260592803359,"loss/avg_raw_advantages_abs":0.04630599170923233,"policy/policy_entropy":0.30557956150732934,"policy/policy_loss":2.33683931583073e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.003882210367009975,"policy/raw_grad_norm":0.001712799072265625,"reward/avg_pass_at_8":0.21875,"reward/avg_raw_reward":0.162109375,"timing/step":2772.290991407819,"trainer/epoch":0}
+{"step":48,"async/staleness_mean":1.875,"generate/avg_num_tokens":9421.37890625,"generate/avg_tokens_non_zero_rewards":7876.908163265306,"generate/avg_tokens_zero_rewards":9786.978260869566,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26099,"generate/std_num_tokens":4368.647168964187,"loss/avg_final_rewards":0.19140625,"loss/avg_raw_advantages":-0.0039308215491473675,"loss/avg_raw_advantages_abs":0.11343784630298615,"policy/policy_entropy":0.3074710522778332,"policy/policy_loss":1.2229538608465873e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.01040839788493031,"policy/raw_grad_norm":0.0009698867797851562,"reward/avg_pass_at_8":0.375,"reward/avg_raw_reward":0.19140625,"timing/step":3181.0612665340304,"trainer/epoch":0}
+{"step":49,"async/staleness_mean":1.890625,"generate/avg_num_tokens":9219.716796875,"generate/avg_tokens_non_zero_rewards":7454.241379310345,"generate/avg_tokens_zero_rewards":9581.12,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26033,"generate/std_num_tokens":4783.0605199363845,"loss/avg_final_rewards":0.169921875,"loss/avg_raw_advantages":0.002498070942237973,"loss/avg_raw_advantages_abs":0.11906687170267105,"policy/policy_entropy":0.3036602227948606,"policy/policy_loss":-5.342363067484257e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00947845229984523,"policy/raw_grad_norm":0.0009307861328125,"reward/avg_pass_at_8":0.359375,"reward/avg_raw_reward":0.169921875,"timing/step":3107.249698700849,"trainer/epoch":0}
+{"step":50,"async/staleness_mean":2.09375,"generate/avg_num_tokens":9547.40625,"generate/avg_tokens_non_zero_rewards":8267.58,"generate/avg_tokens_zero_rewards":9685.915584415585,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26049,"generate/std_num_tokens":5002.093986814891,"loss/avg_final_rewards":0.09765625,"loss/avg_raw_advantages":0.002024407498538494,"loss/avg_raw_advantages_abs":0.0485578291118145,"policy/policy_entropy":0.31965480255894363,"policy/policy_loss":-8.268469571248716e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.003482727381197037,"policy/raw_grad_norm":0.0012912750244140625,"reward/avg_pass_at_8":0.1875,"reward/avg_raw_reward":0.09765625,"timing/step":3427.9984126063064,"trainer/epoch":0}
+{"step":51,"async/staleness_mean":2.0,"generate/avg_num_tokens":9990.958984375,"generate/avg_tokens_non_zero_rewards":7699.271428571428,"generate/avg_tokens_zero_rewards":10353.89592760181,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26250,"generate/std_num_tokens":4775.0150793566445,"loss/avg_final_rewards":0.13671875,"loss/avg_raw_advantages":-0.0047527397982776165,"loss/avg_raw_advantages_abs":0.1103021427989006,"policy/policy_entropy":0.32656489266082644,"policy/policy_loss":5.014291417637651e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.00707310474535916,"policy/raw_grad_norm":0.0012569427490234375,"reward/avg_pass_at_8":0.296875,"reward/avg_raw_reward":0.13671875,"timing/step":3744.2121266839094,"trainer/epoch":0}
+{"step":52,"async/staleness_mean":1.015625,"generate/avg_num_tokens":10499.109375,"generate/avg_tokens_non_zero_rewards":7592.34,"generate/avg_tokens_zero_rewards":11204.635922330097,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26608,"generate/std_num_tokens":5624.68945529492,"loss/avg_final_rewards":0.1953125,"loss/avg_raw_advantages":-0.0010208401363343,"loss/avg_raw_advantages_abs":0.09705197811126709,"policy/policy_entropy":0.3335581294959411,"policy/policy_loss":5.072883269008344e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008387434737869626,"policy/raw_grad_norm":0.001224517822265625,"reward/avg_pass_at_8":0.3333333333333333,"reward/avg_raw_reward":0.1953125,"timing/step":8279.12804713007,"trainer/epoch":0}
+{"step":53,"async/staleness_mean":1.0,"generate/avg_num_tokens":10057.123046875,"generate/avg_tokens_non_zero_rewards":8042.0,"generate/avg_tokens_zero_rewards":10397.577625570777,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26227,"generate/std_num_tokens":5258.414567543752,"loss/avg_final_rewards":0.14453125,"loss/avg_raw_advantages":-0.006202935706824064,"loss/avg_raw_advantages_abs":0.0868324413895607,"policy/policy_entropy":0.3381909884046763,"policy/policy_loss":1.8669722265940436e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.007297491781628196,"policy/raw_grad_norm":0.001514434814453125,"reward/avg_pass_at_8":0.25,"reward/avg_raw_reward":0.14453125,"timing/step":2664.5530663589016,"trainer/epoch":0}
+{"step":54,"async/staleness_mean":1.875,"generate/avg_num_tokens":9678.37109375,"generate/avg_tokens_non_zero_rewards":7591.064516129032,"generate/avg_tokens_zero_rewards":10141.663484486873,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":26388,"generate/std_num_tokens":4784.194531518804,"loss/avg_final_rewards":0.181640625,"loss/avg_raw_advantages":-0.005927966441959143,"loss/avg_raw_advantages_abs":0.09328899532556534,"policy/policy_entropy":0.3633612405974418,"policy/policy_loss":2.8145531665302315e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008472242967854982,"policy/raw_grad_norm":0.0013751983642578125,"reward/avg_pass_at_8":0.296875,"reward/avg_raw_reward":0.181640625,"timing/step":3650.185230393894,"trainer/epoch":0}
+{"step":55,"async/staleness_mean":2.171875,"generate/avg_num_tokens":10411.666015625,"generate/avg_tokens_non_zero_rewards":9310.28125,"generate/avg_tokens_zero_rewards":10665.83173076923,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":25941,"generate/std_num_tokens":5185.13700018814,"loss/avg_final_rewards":0.1875,"loss/avg_raw_advantages":0.005737764295190573,"loss/avg_raw_advantages_abs":0.1463140994310379,"policy/policy_entropy":0.3668097753543407,"policy/policy_loss":-1.0202376614643072e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.009982707390008727,"policy/raw_grad_norm":0.0009632110595703125,"reward/avg_pass_at_8":0.40625,"reward/avg_raw_reward":0.1875,"timing/step":4766.921925783157,"trainer/epoch":0}
+{"step":56,"async/staleness_mean":1.890625,"generate/avg_num_tokens":10620.34765625,"generate/avg_tokens_non_zero_rewards":8445.822222222223,"generate/avg_tokens_zero_rewards":11084.109004739337,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":29809,"generate/std_num_tokens":5523.311386487628,"loss/avg_final_rewards":0.17578125,"loss/avg_raw_advantages":-0.007907772436738014,"loss/avg_raw_advantages_abs":0.09605777263641357,"policy/policy_entropy":0.3636972000822425,"policy/policy_loss":3.490397723737715e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.005557403340844758,"policy/raw_grad_norm":0.0018749237060546875,"reward/avg_pass_at_8":0.328125,"reward/avg_raw_reward":0.17578125,"timing/step":3964.189185673371,"trainer/epoch":0}
+{"step":57,"async/staleness_mean":1.875,"generate/avg_num_tokens":11039.845703125,"generate/avg_tokens_non_zero_rewards":9345.072164948453,"generate/avg_tokens_zero_rewards":11435.973493975904,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27418,"generate/std_num_tokens":5405.577233575115,"loss/avg_final_rewards":0.189453125,"loss/avg_raw_advantages":0.007507964037358761,"loss/avg_raw_advantages_abs":0.1060580164194107,"policy/policy_entropy":0.38697593309916556,"policy/policy_loss":-2.3567315619033025e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.009201306208979076,"policy/raw_grad_norm":0.001186370849609375,"reward/avg_pass_at_8":0.34375,"reward/avg_raw_reward":0.189453125,"timing/step":4748.439351412002,"trainer/epoch":0}
+{"step":58,"async/staleness_mean":0.609375,"generate/avg_num_tokens":11945.6953125,"generate/avg_tokens_non_zero_rewards":11228.893203883496,"generate/avg_tokens_zero_rewards":12126.210268948655,"generate/failed_trajectory_fraction":0.00390625,"generate/max_num_tokens":27493,"generate/std_num_tokens":6150.510504800844,"loss/avg_final_rewards":0.201171875,"loss/avg_raw_advantages":0.0025003906339406967,"loss/avg_raw_advantages_abs":0.16300296783447266,"policy/policy_entropy":0.4177480931393802,"policy/policy_loss":3.671130777149756e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.014197787017110386,"policy/raw_grad_norm":0.00124359130859375,"reward/avg_pass_at_8":0.4166666666666667,"reward/avg_raw_reward":0.201171875,"timing/step":9875.224191051908,"trainer/epoch":0}
+{"step":59,"async/staleness_mean":0.984375,"generate/avg_num_tokens":11575.060546875,"generate/avg_tokens_non_zero_rewards":9524.396825396825,"generate/avg_tokens_zero_rewards":11862.792873051225,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":29703,"generate/std_num_tokens":5956.860779794144,"loss/avg_final_rewards":0.123046875,"loss/avg_raw_advantages":-0.0035006857942789793,"loss/avg_raw_advantages_abs":0.08761342614889145,"policy/policy_entropy":0.41764762648381293,"policy/policy_loss":2.287662400846102e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.006892739470004017,"policy/raw_grad_norm":0.001705169677734375,"reward/avg_pass_at_8":0.265625,"reward/avg_raw_reward":0.123046875,"timing/step":5967.962672992144,"trainer/epoch":0}
+{"step":60,"async/staleness_mean":1.546875,"generate/avg_num_tokens":11873.125,"generate/avg_tokens_non_zero_rewards":11056.25,"generate/avg_tokens_zero_rewards":11981.559734513274,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":29500,"generate/std_num_tokens":5626.0215739010455,"loss/avg_final_rewards":0.1171875,"loss/avg_raw_advantages":0.003636080538854003,"loss/avg_raw_advantages_abs":0.11055222153663635,"policy/policy_entropy":0.43152059568092227,"policy/policy_loss":-9.484258391978528e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008591869624069659,"policy/raw_grad_norm":0.0012264251708984375,"reward/avg_pass_at_8":0.3125,"reward/avg_raw_reward":0.1171875,"timing/step":5322.306719806045,"trainer/epoch":0}
+{"step":61,"async/staleness_mean":1.609375,"generate/avg_num_tokens":12350.1328125,"generate/avg_tokens_non_zero_rewards":11411.978571428572,"generate/avg_tokens_zero_rewards":12703.201612903225,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":30514,"generate/std_num_tokens":6199.974247830961,"loss/avg_final_rewards":0.2734375,"loss/avg_raw_advantages":-0.005648551508784294,"loss/avg_raw_advantages_abs":0.14002735912799835,"policy/policy_entropy":0.4602184642571956,"policy/policy_loss":2.7435726206448408e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.013399694073996216,"policy/raw_grad_norm":0.001880645751953125,"reward/avg_pass_at_8":0.453125,"reward/avg_raw_reward":0.2734375,"timing/step":5991.592536879703,"trainer/epoch":0}
+{"step":62,"async/staleness_mean":1.546875,"generate/avg_num_tokens":11827.33203125,"generate/avg_tokens_non_zero_rewards":10418.213235294117,"generate/avg_tokens_zero_rewards":12337.01329787234,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27735,"generate/std_num_tokens":5753.3824639488375,"loss/avg_final_rewards":0.265625,"loss/avg_raw_advantages":-0.003736194223165512,"loss/avg_raw_advantages_abs":0.1392456740140915,"policy/policy_entropy":0.47746485751122236,"policy/policy_loss":9.162238097104591e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.011443705617693922,"policy/raw_grad_norm":0.0017681121826171875,"reward/avg_pass_at_8":0.46875,"reward/avg_raw_reward":0.265625,"timing/step":5385.837114124093,"trainer/epoch":0}
+{"step":63,"async/staleness_mean":1.59375,"generate/avg_num_tokens":12837.427734375,"generate/avg_tokens_non_zero_rewards":11478.38596491228,"generate/avg_tokens_zero_rewards":13007.681318681318,"generate/failed_trajectory_fraction":0.001953125,"generate/max_num_tokens":29197,"generate/std_num_tokens":6078.297916133897,"loss/avg_final_rewards":0.111328125,"loss/avg_raw_advantages":-0.009539480321109295,"loss/avg_raw_advantages_abs":0.09877653419971466,"policy/policy_entropy":0.5093152353074402,"policy/policy_loss":3.6863803991593613e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.008908780561796448,"policy/raw_grad_norm":0.002593994140625,"reward/avg_pass_at_8":0.265625,"reward/avg_raw_reward":0.111328125,"timing/step":6230.082083825022,"trainer/epoch":0}
+{"step":64,"async/staleness_mean":0.53125,"generate/avg_num_tokens":13422.99609375,"generate/avg_tokens_non_zero_rewards":12933.420289855072,"generate/avg_tokens_zero_rewards":13499.250564334086,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":30092,"generate/std_num_tokens":6395.515129026726,"loss/avg_final_rewards":0.134765625,"loss/avg_raw_advantages":-0.0019149556756019592,"loss/avg_raw_advantages_abs":0.11301226168870926,"policy/policy_entropy":0.5293293185532093,"policy/policy_loss":9.82752151656996e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.009566483376602264,"policy/raw_grad_norm":0.001873016357421875,"reward/avg_pass_at_8":0.30434782608695654,"reward/avg_raw_reward":0.134765625,"timing/step":10154.383708074005,"trainer/epoch":0}
+{"step":65,"async/staleness_mean":0.984375,"generate/avg_num_tokens":12917.7109375,"generate/avg_tokens_non_zero_rewards":11311.372340425532,"generate/avg_tokens_zero_rewards":13278.944976076555,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":30576,"generate/std_num_tokens":6387.428287149228,"loss/avg_final_rewards":0.18359375,"loss/avg_raw_advantages":-0.0013121777446940541,"loss/avg_raw_advantages_abs":0.12830360233783722,"policy/policy_entropy":0.5610155819449574,"policy/policy_loss":7.220311992739425e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.011286421251043066,"policy/raw_grad_norm":0.002407073974609375,"reward/avg_pass_at_8":0.390625,"reward/avg_raw_reward":0.18359375,"timing/step":7163.567044896998,"trainer/epoch":0}
+{"step":66,"async/staleness_mean":1.375,"generate/avg_num_tokens":13958.44140625,"generate/avg_tokens_non_zero_rewards":13873.722222222223,"generate/avg_tokens_zero_rewards":13976.509478672986,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":28741,"generate/std_num_tokens":6516.526721812627,"loss/avg_final_rewards":0.17578125,"loss/avg_raw_advantages":0.004518852569162846,"loss/avg_raw_advantages_abs":0.14305809140205383,"policy/policy_entropy":0.6256877051200718,"policy/policy_loss":-1.136031368531576e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.01238623321569321,"policy/raw_grad_norm":0.002162933349609375,"reward/avg_pass_at_8":0.34375,"reward/avg_raw_reward":0.17578125,"timing/step":6706.461922962997,"trainer/epoch":0}
+{"step":67,"async/staleness_mean":1.359375,"generate/avg_num_tokens":14313.8203125,"generate/avg_tokens_non_zero_rewards":12228.776315789473,"generate/avg_tokens_zero_rewards":14677.268348623853,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":29142,"generate/std_num_tokens":6390.396745536904,"loss/avg_final_rewards":0.1484375,"loss/avg_raw_advantages":-0.009015744552016258,"loss/avg_raw_advantages_abs":0.13885138928890228,"policy/policy_entropy":0.6556471716612577,"policy/policy_loss":3.5683096690775074e-06,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.01236404105929978,"policy/raw_grad_norm":0.003376007080078125,"reward/avg_pass_at_8":0.34375,"reward/avg_raw_reward":0.1484375,"timing/step":7437.145370946993,"trainer/epoch":0}
+{"step":68,"async/staleness_mean":1.28125,"generate/avg_num_tokens":14532.927734375,"generate/avg_tokens_non_zero_rewards":12126.197674418605,"generate/avg_tokens_zero_rewards":15018.793427230046,"generate/failed_trajectory_fraction":0.0,"generate/max_num_tokens":27192,"generate/std_num_tokens":6333.561667123961,"loss/avg_final_rewards":0.16796875,"loss/avg_raw_advantages":-0.0012404834851622581,"loss/avg_raw_advantages_abs":0.12768308818340302,"policy/policy_entropy":0.6911203351337463,"policy/policy_loss":7.268206303479019e-07,"policy/policy_lr":7.999999979801942e-06,"policy/ppo_clip_ratio":0.01262365348975436,"policy/raw_grad_norm":0.003765106201171875,"reward/avg_pass_at_8":0.390625,"reward/avg_raw_reward":0.16796875,"timing/step":7510.405918068995,"trainer/epoch":0}

viewer/build/inputs/marin/runs/marin-q3c-tt-x5-gradnorm0p45/run.json

@@ -0,0 +1,28 @@
+{
+ "id": "marin-q3c-tt-x5-gradnorm0p45",
+ "title": "TaskTrove RL, max grad norm 0.45 (Qwen3-Coder-30B-A3B)",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://huggingface.co/laion/tt-x5_gradnorm-gn0p45-30-30B/tree/main/training_logs",
+ "license": "apache-2.0",
+ "model": "laion/tt-x5_gradnorm-gn0p45-30-30B",
+ "base_model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
+ "method": "GRPO (SkyRL + Terminus-2, pass-ratio shaped verifier reward)",
+ "dataset": "DCAgent/exp_rpt_multifile",
+ "eval_suite": null,
+ "kind": "training",
+ "state": "finished",
+ "started_at": null,
+ "updated_at": "2026-09-04T16:23:13Z",
+ "attempts": 0,
+ "note": "Marin TaskTrove RL hyperparameter ablation (issue #7785) on Qwen3-Coder-30B-A3B-Instruct with DCAgent/exp_rpt_multifile tasks and the Terminus-2 harness; arm: X5 gradient-clipping arm: max grad norm 0.45. Our copy has every logged step of the final lineage (13 log segments, 9 superseded rows dropped).",
+ "metrics_map": {
+  "reward": "reward/avg_raw_reward",
+  "loss": "policy/policy_loss",
+  "entropy": "policy/policy_entropy",
+  "lr": "policy/policy_lr",
+  "grad_norm": "policy/raw_grad_norm",
+  "response_length": "generate/avg_num_tokens"
+ }
+}

viewer/build/inputs/marin/runs/marin-snowball-e11-deepscaler-dapo/metrics.jsonl

@@ -0,0 +1,51 @@
+{"step":0,"heldout/aime24":17.67,"heldout/aime24_se":1.06,"heldout/math500":64.0,"heldout/math500_se":2.15,"heldout/olympiadbench":12.67,"heldout/olympiadbench_se":1.03}
+{"step":1,"train/pass_at_1":0.053466796875,"train/pass_at_16":0.4296875,"reward/avg_raw_reward":-0.874453125}
+{"step":2,"train/pass_at_1":0.092529296875,"train/pass_at_16":0.5390625,"reward/avg_raw_reward":-0.816494141}
+{"step":3,"train/pass_at_1":0.11279296875,"train/pass_at_16":0.52734375,"reward/avg_raw_reward":-0.750654297}
+{"step":4,"train/pass_at_1":0.177001953125,"train/pass_at_16":0.5625,"reward/avg_raw_reward":-0.677050781}
+{"step":5,"train/pass_at_1":0.226806640625,"train/pass_at_16":0.65234375,"reward/avg_raw_reward":-0.577412109}
+{"step":6,"train/pass_at_1":0.212890625,"train/pass_at_16":0.6875,"reward/avg_raw_reward":-0.465449219,"heldout/aime24":16.33,"heldout/aime24_se":1.29,"heldout/math500":69.0,"heldout/math500_se":2.07,"heldout/olympiadbench":16.67,"heldout/olympiadbench_se":1.63}
+{"step":7,"train/pass_at_1":0.2626953125,"train/pass_at_16":0.6796875,"reward/avg_raw_reward":-0.415107422}
+{"step":8,"train/pass_at_1":0.296875,"train/pass_at_16":0.7421875,"reward/avg_raw_reward":-0.377226562}
+{"step":9,"train/pass_at_1":0.2607421875,"train/pass_at_16":0.76171875,"reward/avg_raw_reward":-0.455751953}
+{"step":10,"train/pass_at_1":0.319580078125,"train/pass_at_16":0.82421875,"reward/avg_raw_reward":-0.280556641}
+{"step":11,"train/pass_at_1":0.345458984375,"train/pass_at_16":0.8515625,"reward/avg_raw_reward":-0.246933594}
+{"step":12,"train/pass_at_1":0.33642578125,"train/pass_at_16":0.9140625,"reward/avg_raw_reward":-0.242255859,"heldout/aime24":17.67,"heldout/aime24_se":0.95,"heldout/math500":74.6,"heldout/math500_se":1.95,"heldout/olympiadbench":16.67,"heldout/olympiadbench_se":1.33}
+{"step":13,"train/pass_at_1":0.3720703125,"train/pass_at_16":0.94140625,"reward/avg_raw_reward":-0.157167969}
+{"step":14,"train/pass_at_1":0.376953125,"train/pass_at_16":0.9296875,"reward/avg_raw_reward":-0.17484375}
+{"step":15,"train/pass_at_1":0.37060546875,"train/pass_at_16":0.95703125}
+{"step":16,"train/pass_at_1":0.394287109375,"train/pass_at_16":0.98828125}
+{"step":17,"train/pass_at_1":0.397705078125,"train/pass_at_16":0.984375}
+{"step":18,"train/pass_at_1":0.39111328125,"train/pass_at_16":0.99609375,"heldout/aime24":19.67,"heldout/aime24_se":0.88,"heldout/math500":72.2,"heldout/math500_se":2.0,"heldout/olympiadbench":20.0,"heldout/olympiadbench_se":1.05}
+{"step":19,"train/pass_at_1":0.43701171875,"train/pass_at_16":0.9765625}
+{"step":20,"train/pass_at_1":0.40771484375,"train/pass_at_16":0.9453125}
+{"step":21,"train/pass_at_1":0.44677734375,"train/pass_at_16":0.96484375}
+{"step":22,"train/pass_at_1":0.443359375,"train/pass_at_16":0.9765625}
+{"step":23,"train/pass_at_1":0.430908203125,"train/pass_at_16":0.96875}
+{"step":24,"train/pass_at_1":0.4208984375,"train/pass_at_16":0.95703125,"heldout/aime24":20.0,"heldout/aime24_se":1.76,"heldout/math500":72.8,"heldout/math500_se":1.99,"heldout/olympiadbench":19.33,"heldout/olympiadbench_se":0.79}
+{"step":25,"train/pass_at_1":0.4580078125,"train/pass_at_16":0.96875}
+{"step":26,"train/pass_at_1":0.43798828125,"train/pass_at_16":0.94921875}
+{"step":27,"train/pass_at_1":0.458740234375,"train/pass_at_16":1}
+{"step":28,"train/pass_at_1":0.487060546875,"train/pass_at_16":1}
+{"step":29,"train/pass_at_1":0.47265625,"train/pass_at_16":1}
+{"step":30,"train/pass_at_1":0.47509765625,"train/pass_at_16":1}
+{"step":31,"train/pass_at_1":0.46630859375,"train/pass_at_16":1}
+{"step":32,"train/pass_at_1":0.4833984375,"train/pass_at_16":1,"heldout/aime24":20.33,"heldout/aime24_se":1.85,"heldout/math500":73.4,"heldout/math500_se":1.98,"heldout/olympiadbench":16.0,"heldout/olympiadbench_se":1.4}
+{"step":33,"train/pass_at_1":0.478271484375,"train/pass_at_16":1}
+{"step":34,"train/pass_at_1":0.503662109375,"train/pass_at_16":1}
+{"step":35,"train/pass_at_1":0.5234375,"train/pass_at_16":1}
+{"step":36,"train/pass_at_1":0.536376953125,"train/pass_at_16":1}
+{"step":37,"train/pass_at_1":0.553955078125,"train/pass_at_16":1}
+{"step":38,"train/pass_at_1":0.518310546875,"train/pass_at_16":1}
+{"step":39,"train/pass_at_1":0.520751953125,"train/pass_at_16":1}
+{"step":40,"train/pass_at_1":0.54248046875,"train/pass_at_16":1,"heldout/aime24":20.0,"heldout/aime24_se":1.94,"heldout/math500":68.0,"heldout/math500_se":2.09,"heldout/olympiadbench":15.33,"heldout/olympiadbench_se":1.71}
+{"step":41,"train/pass_at_1":0.49072265625,"train/pass_at_16":1}
+{"step":42,"train/pass_at_1":0.526123046875,"train/pass_at_16":1}
+{"step":43,"train/pass_at_1":0.504638671875,"train/pass_at_16":1}
+{"step":44,"train/pass_at_1":0.5263671875,"train/pass_at_16":1}
+{"step":45,"train/pass_at_1":0.533203125,"train/pass_at_16":1}
+{"step":46,"train/pass_at_1":0.5390625,"train/pass_at_16":1}
+{"step":47,"train/pass_at_1":0.533447265625,"train/pass_at_16":1}
+{"step":48,"train/pass_at_1":0.52978515625,"train/pass_at_16":1,"heldout/aime24":20.33,"heldout/aime24_se":1.45,"heldout/math500":70.0,"heldout/math500_se":2.05,"heldout/olympiadbench":14.67,"heldout/olympiadbench_se":1.26}
+{"step":49,"train/pass_at_1":0.525146484375,"train/pass_at_16":1}
+{"step":50,"train/pass_at_1":0.51416015625,"train/pass_at_16":1}

viewer/build/inputs/marin/runs/marin-snowball-e11-deepscaler-dapo/run.json

@@ -0,0 +1,26 @@
+{
+ "id": "marin-snowball-e11-deepscaler-dapo",
+ "title": "Snowball 67B-A2B math RL, E11: DeepScaleR + DAPO",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://storage.googleapis.com/marin-public/benjaminfeuer/snowball-67b-a2b-math-rl/2026.08.27.1/index.html",
+ "license": "unknown",
+ "model": "Snowball 67B-A2B, RL arm E11",
+ "base_model": "grug-67b-a2b-sft-s2-thinking-step630 (Marin Snowball 67B-A2B, 2T Thinking SFT)",
+ "method": "DAPO",
+ "dataset": "DeepScaleR",
+ "eval_suite": "AIME24, MATH-500, OlympiadBench (held-out math, evalchemy long-generation)",
+ "kind": "training",
+ "state": "finished",
+ "started_at": null,
+ "updated_at": "2026-08-31T11:03:10Z",
+ "attempts": 0,
+ "note": "Marin math RLVR on its Snowball 67B-A2B MoE (issue #7786), E11: DeepScaleR + DAPO; recipe: DAPO objective, MuonH lr 1e-4, on the hero-v3g recipe body. train/pass_at_1 and train/pass_at_16 are per-step training-batch values as Marin exported them from W&B (pass@1 source: environment/acc); reward/avg_raw_reward is the raw ±1 verifier reward for the steps the report plots. heldout/* are AIME24 / MATH-500 / OlympiadBench percentages (± SE in *_se) from MATH_EVALS.md at checkpoints; step 0 is the SFT base the RL started from.",
+ "metrics_map": {
+  "reward": "train/pass_at_1",
+  "eval:aime24": "heldout/aime24",
+  "eval:math500": "heldout/math500",
+  "eval:olympiadbench": "heldout/olympiadbench"
+ }
+}

viewer/build/inputs/marin/runs/marin-snowball-e12-deepscaler-grpo/metrics.jsonl

@@ -0,0 +1,25 @@
+{"step":0,"heldout/aime24":17.67,"heldout/aime24_se":1.06,"heldout/math500":64.0,"heldout/math500_se":2.15,"heldout/olympiadbench":12.67,"heldout/olympiadbench_se":1.03}
+{"step":1,"train/pass_at_1":0.04541015625,"train/pass_at_16":0.2890625}
+{"step":2,"train/pass_at_1":0.072021484375,"train/pass_at_16":0.4296875}
+{"step":3,"train/pass_at_1":0.1181640625,"train/pass_at_16":0.4921875}
+{"step":4,"train/pass_at_1":0.16748046875,"train/pass_at_16":0.5390625}
+{"step":5,"train/pass_at_1":0.182373046875,"train/pass_at_16":0.578125}
+{"step":6,"train/pass_at_1":0.201904296875,"train/pass_at_16":0.5546875}
+{"step":7,"train/pass_at_1":0.254638671875,"train/pass_at_16":0.60546875}
+{"step":8,"train/pass_at_1":0.311767578125,"train/pass_at_16":0.70703125,"heldout/aime24":20.0,"heldout/aime24_se":1.05,"heldout/math500":71.0,"heldout/math500_se":2.03,"heldout/olympiadbench":15.0,"heldout/olympiadbench_se":1.27}
+{"step":9,"train/pass_at_1":0.32373046875,"train/pass_at_16":0.73046875}
+{"step":10,"train/pass_at_1":0.36328125,"train/pass_at_16":0.76171875}
+{"step":11,"train/pass_at_1":0.373291015625,"train/pass_at_16":0.7265625}
+{"step":12,"train/pass_at_1":0.32666015625,"train/pass_at_16":0.703125}
+{"step":13,"train/pass_at_1":0.351318359375,"train/pass_at_16":0.71484375}
+{"step":14,"train/pass_at_1":0.354736328125,"train/pass_at_16":0.73828125}
+{"step":15,"train/pass_at_1":0.39208984375,"train/pass_at_16":0.7265625}
+{"step":16,"train/pass_at_1":0.377197265625,"train/pass_at_16":0.76953125,"heldout/aime24":16.67,"heldout/aime24_se":0.94,"heldout/math500":70.2,"heldout/math500_se":2.05,"heldout/olympiadbench":18.0,"heldout/olympiadbench_se":1.26}
+{"step":17,"train/pass_at_1":0.31396484375,"train/pass_at_16":0.69921875}
+{"step":18,"train/pass_at_1":0.342529296875,"train/pass_at_16":0.7265625}
+{"step":19,"train/pass_at_1":0.374267578125,"train/pass_at_16":0.76953125}
+{"step":20,"train/pass_at_1":0.389892578125,"train/pass_at_16":0.76953125}
+{"step":21,"train/pass_at_1":0.395263671875,"train/pass_at_16":0.76171875}
+{"step":22,"train/pass_at_1":0.388427734375,"train/pass_at_16":0.7578125}
+{"step":23,"train/pass_at_1":0.40087890625,"train/pass_at_16":0.74609375}
+{"step":24,"train/pass_at_1":0.400390625,"train/pass_at_16":0.74609375,"heldout/aime24":16.0,"heldout/aime24_se":1.32,"heldout/math500":71.8,"heldout/math500_se":2.01,"heldout/olympiadbench":16.33,"heldout/olympiadbench_se":0.99}

viewer/build/inputs/marin/runs/marin-snowball-e12-deepscaler-grpo/run.json

@@ -0,0 +1,26 @@
+{
+ "id": "marin-snowball-e12-deepscaler-grpo",
+ "title": "Snowball 67B-A2B math RL, E12: DeepScaleR + GRPO (control for E11)",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://storage.googleapis.com/marin-public/benjaminfeuer/snowball-67b-a2b-math-rl/2026.08.27.1/index.html",
+ "license": "unknown",
+ "model": "Snowball 67B-A2B, RL arm E12",
+ "base_model": "grug-67b-a2b-sft-s2-thinking-step630 (Marin Snowball 67B-A2B, 2T Thinking SFT)",
+ "method": "GRPO",
+ "dataset": "DeepScaleR",
+ "eval_suite": "AIME24, MATH-500, OlympiadBench (held-out math, evalchemy long-generation)",
+ "kind": "training",
+ "state": "finished",
+ "started_at": null,
+ "updated_at": "2026-08-31T11:03:10Z",
+ "attempts": 0,
+ "note": "Marin math RLVR on its Snowball 67B-A2B MoE (issue #7786), E12: DeepScaleR + GRPO (control for E11); recipe: GRPO objective (the E11 recipe without DAPO), 10k context. train/pass_at_1 and train/pass_at_16 are per-step training-batch values as Marin exported them from W&B (pass@1 source: environment/acc). heldout/* are AIME24 / MATH-500 / OlympiadBench percentages (± SE in *_se) from MATH_EVALS.md at checkpoints; step 0 is the SFT base the RL started from.",
+ "metrics_map": {
+  "reward": "train/pass_at_1",
+  "eval:aime24": "heldout/aime24",
+  "eval:math500": "heldout/math500",
+  "eval:olympiadbench": "heldout/olympiadbench"
+ }
+}

viewer/build/inputs/marin/runs/marin-snowball-e6-rlvr-math/metrics.jsonl

@@ -0,0 +1,21 @@
+{"step":0,"heldout/aime24":17.67,"heldout/aime24_se":1.06,"heldout/math500":64.0,"heldout/math500_se":2.15,"heldout/olympiadbench":12.67,"heldout/olympiadbench_se":1.03}
+{"step":1,"train/pass_at_1":0.1103515625,"train/pass_at_16":0.58203125,"reward/avg_raw_reward":-0.779296875}
+{"step":2,"train/pass_at_1":0.18603515625,"train/pass_at_16":0.44921875,"reward/avg_raw_reward":-0.6279296875}
+{"step":3,"train/pass_at_1":0.230224609375,"train/pass_at_16":0.52734375,"reward/avg_raw_reward":-0.53955078125}
+{"step":4,"train/pass_at_1":0.293701171875,"train/pass_at_16":0.5625,"reward/avg_raw_reward":-0.41259765625}
+{"step":5,"train/pass_at_1":0.256103515625,"train/pass_at_16":0.5078125,"reward/avg_raw_reward":-0.48779296875,"heldout/aime24":20.33,"heldout/aime24_se":2.08,"heldout/math500":71.8,"heldout/math500_se":2.01,"heldout/olympiadbench":16.33,"heldout/olympiadbench_se":1.1}
+{"step":6,"train/pass_at_1":0.267822265625,"train/pass_at_16":0.57421875,"reward/avg_raw_reward":-0.46435546875}
+{"step":7,"train/pass_at_1":0.28564453125,"train/pass_at_16":0.58984375,"reward/avg_raw_reward":-0.4287109375}
+{"step":8,"train/pass_at_1":0.289794921875,"train/pass_at_16":0.59765625,"reward/avg_raw_reward":-0.42041015625}
+{"step":9,"train/pass_at_1":0.314453125,"train/pass_at_16":0.6015625,"reward/avg_raw_reward":-0.37109375}
+{"step":10,"train/pass_at_1":0.34765625,"train/pass_at_16":0.66015625,"reward/avg_raw_reward":-0.3046875,"heldout/aime24":25.33,"heldout/aime24_se":1.43,"heldout/math500":74.0,"heldout/math500_se":1.96,"heldout/olympiadbench":19.33,"heldout/olympiadbench_se":1.03}
+{"step":11,"train/pass_at_1":0.3427734375,"train/pass_at_16":0.63671875,"reward/avg_raw_reward":-0.314453125}
+{"step":12,"train/pass_at_1":0.33837890625,"train/pass_at_16":0.63671875,"reward/avg_raw_reward":-0.3232421875}
+{"step":13,"train/pass_at_1":0.36669921875,"train/pass_at_16":0.6484375,"reward/avg_raw_reward":-0.2666015625}
+{"step":14,"train/pass_at_1":0.39013671875,"train/pass_at_16":0.61328125,"reward/avg_raw_reward":-0.2197265625}
+{"step":15,"train/pass_at_1":0.329345703125,"train/pass_at_16":0.5703125,"reward/avg_raw_reward":-0.34130859375,"heldout/aime24":27.33,"heldout/aime24_se":1.87,"heldout/math500":78.4,"heldout/math500_se":1.84,"heldout/olympiadbench":19.67,"heldout/olympiadbench_se":1.2}
+{"step":16,"train/pass_at_1":0.33544921875,"train/pass_at_16":0.56640625,"reward/avg_raw_reward":-0.3291015625}
+{"step":17,"train/pass_at_1":0.3466796875,"train/pass_at_16":0.6015625,"reward/avg_raw_reward":-0.306640625}
+{"step":18,"train/pass_at_1":0.38623046875,"train/pass_at_16":0.64453125,"reward/avg_raw_reward":-0.2275390625}
+{"step":19,"train/pass_at_1":0.355712890625,"train/pass_at_16":0.59765625,"reward/avg_raw_reward":-0.28857421875}
+{"step":20,"train/pass_at_1":0.41162109375,"train/pass_at_16":0.6328125,"reward/avg_raw_reward":-0.1767578125,"heldout/aime24":26.0,"heldout/aime24_se":2.2,"heldout/math500":76.8,"heldout/math500_se":1.89,"heldout/olympiadbench":22.67,"heldout/olympiadbench_se":1.03}

viewer/build/inputs/marin/runs/marin-snowball-e6-rlvr-math/run.json

@@ -0,0 +1,26 @@
+{
+ "id": "marin-snowball-e6-rlvr-math",
+ "title": "Snowball 67B-A2B math RL, E6: RLVR-MATH + unregularized GRPO",
+ "source": "public",
+ "org": "Marin",
+ "project": "marin",
+ "url": "https://storage.googleapis.com/marin-public/benjaminfeuer/snowball-67b-a2b-math-rl/2026.08.27.1/index.html",
+ "license": "unknown",
+ "model": "Snowball 67B-A2B, RL arm E6",
+ "base_model": "grug-67b-a2b-sft-s2-thinking-step630 (Marin Snowball 67B-A2B, 2T Thinking SFT)",
+ "method": "unregularized GRPO",
+ "dataset": "RLVR-MATH",
+ "eval_suite": "AIME24, MATH-500, OlympiadBench (held-out math, evalchemy long-generation)",
+ "kind": "training",
+ "state": "finished",
+ "started_at": null,
+ "updated_at": "2026-08-31T11:03:10Z",
+ "attempts": 0,
+ "note": "Marin math RLVR on its Snowball 67B-A2B MoE (issue #7786), E6: RLVR-MATH + unregularized GRPO; recipe: unregularized GRPO, AdamW 1e-5, mutable router bias, 8192-token window. train/pass_at_1 and train/pass_at_16 are per-step training-batch values as Marin exported them from W&B (pass@1 source: derived from reward/avg_raw_reward (binary -1/+1 verifier)); reward/avg_raw_reward is the raw ±1 verifier reward for the steps the report plots. heldout/* are AIME24 / MATH-500 / OlympiadBench percentages (± SE in *_se) from MATH_EVALS.md at checkpoints; step 0 is the SFT base the RL started from. The E6 original checkpoints were evaluated as router-bias-repaired exports, which the report treats as exceptional.",
+ "metrics_map": {
+  "reward": "train/pass_at_1",
+  "eval:aime24": "heldout/aime24",
+  "eval:math500": "heldout/math500",
+  "eval:olympiadbench": "heldout/olympiadbench"
+ }
+}