fix(batch): leave no record for a canceled job nobody owns after discovery

Canceling a job ID this node does not own recorded a canceled owner so
that startup discovery, which may still admit the manifest of that ID,
would not run a job that was canceled before the node had loaded it.
The record was kept for any ID, at any time, and never removed: a caller
with admin:CancelBatchJob could grow the ownership table by one entry per
cancellation of an unknown ID, and the retry scheduler walks that table
every second.

Once discovery has finished, nothing admits a job this node does not own:
new submissions use new IDs, so the record has nothing left to fence.
canceler now records a canceled owner for an unknown ID only while
discovery is still running. Canceling an owned job, and the fencing of a
cancellation during discovery, are unchanged, and the cancel handler
still removes the manifest.

TestBatchCancelUnknownJobAfterDiscovery cancels an unknown ID during
discovery and requires its admission to be refused, then cancels a
thousand unknown IDs after discovery and requires the ownership table
not to grow. Before this change it held a thousand records.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
This commit is contained in:
Feng Ruohang
2026-11-07 16:08:52 +08:00
parent 5fb983d3d2
commit 378f281bef
2 changed files with 43 additions and 4 deletions
+10 -4
View File
@@ -2520,15 +2520,21 @@ func (j *BatchJobPool) canceler(jobID string, cancel bool) error {
if !cancel {
return nil
} // Worker completion uses its generation token.
if j.jobOwners == nil {
j.jobOwners = make(map[string]*batchJobOwner)
}
if owner := j.jobOwners[jobID]; owner != nil {
if owner.cancel != nil {
owner.cancel()
}
j.jobOwners[jobID] = &batchJobOwner{state: "canceled"}
} else {
return nil
}
// Only startup discovery admits a manifest this node does not own yet.
// A cancellation before that admission leaves a record that fences it;
// once discovery has finished, nothing admits the ID again (submissions
// use new IDs), so an unknown ID leaves no record to accumulate.
if !j.discoveryDone {
if j.jobOwners == nil {
j.jobOwners = make(map[string]*batchJobOwner)
}
j.jobOwners[jobID] = &batchJobOwner{state: "canceled"}
}
return nil