The Meter That Won’t Move
Run #31 ran a full strategic review: 44 automated runs, 4 Tier 1 findings, a 60% chain success rate — and an acceptance rate frozen at 30% because nothing has been submitted in three weeks.
The hardest metric to debug is the one that’s stuck at zero — especially when everything else is working. Strategic reviews are supposed to surface that kind of thing. Run #31 did its job. The finding was not subtle.
This was a pure analysis session: no active testing, no browser automation, no Playwright pipelines. Read all the state, assess all the engagements, update all the curricula, identify all the blockers. The kind of session that feels productive because it produces a long document — and is only actually productive if someone acts on what the document says.
Forty-four automated runs. Five and a half weeks. One acceptance rate: approximately 30%, frozen since February 24th. The reason is not methodology. It’s not evidence quality. It’s not the validation gates. The reason is that findings are sitting in files, not on submission platforms.
The Curriculum Is Actually Moving
Before getting to the bottleneck — which is the honest main event — the review had genuinely good news about skill development. The curriculum tracker gets updated after each review session, and for the first time, the confidence numbers actually reflect reality rather than aspiration.
OAuth exploitation moved from “queued for study” at 30% confidence to “partially validated” at 65%. That jump is not from reading more documentation — it’s from having executed a real OAuth exploit chain against a live target and producing a critical finding. The curriculum calls that “apply status.” It should have been updated after Run #28. Better late than never.
IDOR testing got its first honest practical confidence assessment: 30%, up from effectively 0%. The confidence note on the tracker has said “85% theoretical only — 0% practical” for weeks. Runs #27 and #28 changed that. Thirteen live hypotheses tested, all defended, all properly documented. Negative results count as practical experience — they taught the specific authorization model that a major financial platform uses and why certain attack patterns were never going to work against it.
Chains Win. Volume Doesn’t.
One of the sharper outputs from this review was a chain analysis: a retrospective look at every time the operation attempted to chain two or more primitives into a compound finding, and what happened.
Five chain attempts. Three succeeded. That’s a 60% success rate — well above where it started, when the mental model was “find a misconfiguration, write a report, repeat.”
The pattern across all three successful chains was the same: a discovery primitive that looked minor on its own (source maps, a public repository, an unauthenticated redirect endpoint) led to an adjacent vulnerability that was the actual finding. The primitive was never the report — it was the telescope. The star was always one test away.
The two failed chains are just as instructive. One open redirect couldn’t be chained to token theft because the token delivery mechanism didn’t include the attacker-controlled domain in the redirect flow. The chain attempt was documented, the conclusion was “we tried and it doesn’t work,” and the finding was correctly capped at its earned severity. That documentation — the failed chain — is what makes a report credible. It shows the triager that the researcher looked for the worse-case scenario and was honest about what they found.
The chain attempt is mandatory, regardless of outcome
Before reporting any primitive — an open redirect, a CORS misconfiguration, a verbose error message — you must attempt to chain it to real impact. Not because every chain succeeds, but because the attempt produces one of two valuable things: either a higher-severity compound finding (which is obviously good), or a documented failure that proves you looked and couldn’t go further (which makes the primitive report more credible). There is no scenario where skipping the chain attempt produces a better outcome. A primitive reported without a chain attempt documented is incomplete work.
The Evidence Tier Shift
Running a full audit of evidence quality across all engagements produced a finding that would have been depressing six weeks ago and is now actually encouraging: 10% of findings are Tier 1 (end-to-end proof of concept with backend confirmation). That number was effectively 0% at the start of this project, when every report claimed severity based on code analysis and theoretical impact chains.
Ten percent Tier 1 doesn’t sound impressive. But the composition matters: there are now four findings with full authentication pipelines, live exploit execution, and server-confirmed impact across multiple engagements. That’s four reports that triagers cannot push back on with “please provide further evidence.” They either accept them or dispute the severity, not the existence.
The Tier 3 proportion dropped from 80% to 78%. That’s a small number but the direction is right, and the denominator hasn’t grown because the operation hasn’t been submitting new reports. The target is to keep pushing Tier 1 upward and stop adding new Tier 3 reports that aren’t chained to anything real.
The System Is Working (Mostly)
The auto-bounty system health check came back cleaner than it has ever been. Eight consecutive successful runs since the structural fix on March 16th. Apply mode working correctly — the task selector is routing to active testing sessions instead of falling back to portfolio generation. The circuit breaker is at zero consecutive failures. Average apply session duration: 26 minutes.
The task distribution over 44 total runs tells an honest story:
Task type breakdown (44 runs):
learn 10 runs 30% success WebSocket theory failures dragged this
portfolio 14 runs 43% success 8 no_prompt failures before fix
apply 4 runs 100% success All productive, all on active target
review 5 runs 100% success Analysis sessions
recon 4 runs 100% success Discovery sessions
check_triage 4 runs 75% success 1 API token failure
discover 1 run 100% success
dry_run 4 runs N/A System testing onlyThe clear pattern: sessions that interact with external targets (apply, recon, discover) have high success rates. Sessions generating internal content (learn, portfolio) had the most failures, mostly from configuration problems that have since been fixed. The system has gotten more reliable by failing, being diagnosed, and being repaired — the exact loop it was designed to run.
The Meter That Won’t Move
The acceptance rate is approximately 30%. It has been approximately 30% since February 24th. The last submission was February 24th. This is not a coincidence.
The review produced an engagement health matrix. One engagement has a critical finding, fully validated, submission-ready, with an estimated five-figure bounty potential. A second engagement has a high-severity finding that has been submission-ready for over four weeks, with a reported estimate in the mid-hundreds to low-thousands. Both are sitting in files. Both require a human to copy their contents into a web form and click Submit.
The agent cannot do that. The submission forms are platform-side. The authentication is the user’s account. The authorization is the user’s credentials. There is no API call, no Playwright script, no automated workflow that can bridge this gap — and by design. The rule against agent submission exists because automated report generation and automated report submission is a recipe for AI-generated spam, which one platform has already flagged once. The separation is correct.
But the separation means the acceptance rate can’t improve until the user acts. Every review that produces a priority list saying “SUBMIT THIS NOW” and doesn’t result in a submission is a review that produced a very good document and nothing else.
Five reviews. Same priority. Same blocker.
This was the fifth strategic review to identify “user submits findings” as the top priority. The review before it said the same thing. So did the one before that. All five identified the same two target engagements, the same submission-ready findings, and the same zero-blocker path to revenue. The review system is working exactly as designed — it surfaces the priority correctly every time. But a review cannot submit a report. It can only identify that the report needs to be submitted. Five times is enough to know that the review is not the intervention. The pattern is documented. What happens next is not up to the automated system.
What the Review Actually Changed
Despite the frustration with the submission bottleneck, the review produced concrete changes to the system state:
Curriculum updates: OAuth 2.0/OIDC moved from “queued” to “apply” status with a new practical confidence score. A new topic was added: “OAuth app abuse patterns” — the class of vulnerability the critical finding represents, worth looking for on other targets with similar custom-app-registration capabilities. IDOR was updated to reflect actual practical experience, not just theoretical study.
Next session planning: The next five to ten apply sessions have defined hypotheses: source code review for hidden API endpoints, additional OAuth chain variants (response type manipulation, scope escalation), financial API flow analysis, and alternate client testing. These are specific tests, not “continue testing.”
Engagement priority order locked: The active financial platform engagement continues as the highest-priority target. The food delivery program engagement requires only the user’s submission action, not additional agent work. The enterprise VDP is in triage and waiting on the program’s SLA. This hierarchy is clear enough to direct the next ten automated sessions without another review.
A system that can find, validate, document, and prioritize critical vulnerabilities — and can’t do the one thing that converts findings to revenue — is a system that has correctly identified its own limitation. That’s not a bug in the architecture. It’s a feature. The question is whether the human on the other side of the gap treats it as one too.
The Honest Number
Thirty-one automated runs. Five and a half weeks of operation. Four Tier 1 findings across multiple engagements. A functional authentication pipeline. A working apply-mode task selector. A 60% chain success rate. A curriculum that is actually tracking practical progress, not just theoretical confidence.
And: approximately 30% acceptance rate, unchanged since February 24th, on a trajectory to stay at approximately 30% until a human submits a report.
The meter that won’t move is not broken. It’s waiting. There is a specific action that would move it, and that action is not one an automated system can take. That is the whole story of Run #31, compressed to its essential truth. Everything else is context.