Commit Graph

3 Commits

Author SHA1 Message Date
Feng Ruohang 378f281bef fix(batch): leave no record for a canceled job nobody owns after discovery
Canceling a job ID this node does not own recorded a canceled owner so
that startup discovery, which may still admit the manifest of that ID,
would not run a job that was canceled before the node had loaded it.
The record was kept for any ID, at any time, and never removed: a caller
with admin:CancelBatchJob could grow the ownership table by one entry per
cancellation of an unknown ID, and the retry scheduler walks that table
every second.

Once discovery has finished, nothing admits a job this node does not own:
new submissions use new IDs, so the record has nothing left to fence.
canceler now records a canceled owner for an unknown ID only while
discovery is still running. Canceling an owned job, and the fencing of a
cancellation during discovery, are unchanged, and the cancel handler
still removes the manifest.

TestBatchCancelUnknownJobAfterDiscovery cancels an unknown ID during
discovery and requires its admission to be refused, then cancels a
thousand unknown IDs after discovery and requires the ownership table
not to grow. Before this change it held a thousand records.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-07 16:21:14 +08:00
Feng Ruohang 41b71a8bc1 fix(batch): recover incomplete startup discovery without rerunning jobs
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-07 02:35:15 +08:00
Feng Ruohang bdd17b0f47 fix(batch): pause transient authorization failures with owned retries
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-07 00:42:02 +08:00