RyeCatcher/bioinfo-public
0 Bytes
Justin Johnson
RyeCatcher
AI & ML interests
Im a builder
Organizations
Posts 1
Post
106
I spent a week pointing 123 automated reproductions at ICML 2026 submissions, checking anchored theoretical claims against the papers' own pinned LaTeX.
The interesting part was not the ones that reproduced. It was the handful that did not.
One paper claims its convergence rate matches a cited result under bounded variance. Its own text says the opposite at four separate sites, including a table column headed "unbounded variance" with a check in that row, and its own conclusion. Six other people reproduced that paper and called the claim verified. So did the organizers' reference logbook, which called it inconclusive.
Another paper's bandit proposition states a regret bound of O(T^2/3). Its own comparison table cites that same proposition for O(T^4/5), and a later dynamic bound disagrees with the table a second time. Four other people reproduced it. None of them caught it.
A third asserts its method converges without a decaying learning rate, unlike two baselines. All three theorems carry the identical Robbins-Monro condition word for word, and the paper's own experiment section says a constant stepsize does not converge to zero.
The habit that found these was boring: pin the source, quote line numbers, and refuse to call something falsified when the paper is merely silent. We withdrew one of our own falsifications this week after an independent clean-room run reached our facts and refused our label, and that one had already been accepted.
Logbooks: @RyeCatcher
The interesting part was not the ones that reproduced. It was the handful that did not.
One paper claims its convergence rate matches a cited result under bounded variance. Its own text says the opposite at four separate sites, including a table column headed "unbounded variance" with a check in that row, and its own conclusion. Six other people reproduced that paper and called the claim verified. So did the organizers' reference logbook, which called it inconclusive.
Another paper's bandit proposition states a regret bound of O(T^2/3). Its own comparison table cites that same proposition for O(T^4/5), and a later dynamic bound disagrees with the table a second time. Four other people reproduced it. None of them caught it.
A third asserts its method converges without a decaying learning rate, unlike two baselines. All three theorems carry the identical Robbins-Monro condition word for word, and the paper's own experiment section says a constant stepsize does not converge to zero.
The habit that found these was boring: pin the source, quote line numbers, and refuse to call something falsified when the paper is merely silent. We withdrew one of our own falsifications this week after an independent clean-room run reached our facts and refused our label, and that one had already been accepted.
Logbooks: @RyeCatcher
spaces 163
Running
Reproduction: Quantifying Error Propagation and Model Collapse in Diffusion Models
🎯
Explore research logs, code, and traces in one view
Running
Reproduction: ScaLoRA: Optimally Scaled Low-Rank Adaptation for Efficient High-Rank Fine-Tuning
🎯
Explore code, traces, and workspace in an interactive logbook
Running
Reproduction: A Sketch-and-Project Analysis of Subsampled Natural Gradient Algorithms
🎯
Explore and manage research logs with a web‑based logbook
Running
Reproduction: On Structured State-Space Duality
🎯
Explore code logs, traces, and workspace interactively
Running
Reproduction: Modeling Covariate Transition for Efficient Estimation of Longitudinal Treatment Effects in Randomized Experiments
🎯
Explore experiment logs, traces, and workspace online
Running
Reproduction: Robust and Consistent Ski Rental with Distributional Advice
🎯
Explore experiment logs, code, and traces
datasets 6
RyeCatcher/repro-consistent-adversarial-attacks-traces
Traces • Updated • 1 • 45
RyeCatcher/repro-balancing-learning-rates-traces
Traces • Updated • 1 • 38
RyeCatcher/repro-convergence-rate-of-the-last-iterate-of-stochastic-proximal-algorithms-traces
Traces • Updated • 1 • 35
RyeCatcher/dspy-optimizer-shootout-results
Preview • Updated • 25
RyeCatcher/z-image-sdxl-benchmark
Viewer • Updated • 80 • 34
RyeCatcher/gpu-benchmark-5000x5000
Viewer • Updated • 1 • 11