Skip to content
The Data EngineeringThe Data EngineeringBeta
About
Main content

Backfill scheduling

How does backfill fill the holes around reservations, and why does it make users declare honest run times?

Backfill fills the hole that and reservations create. A large job is waiting for n nodes, only k < n are free, and the gap is pure idle. runs a small job in the gap if it can prove the job will finish before the reservation starts. The proof needs a declared run-time bound (a walltime limit).

Look at what that does. The scheduler needs the one number it cannot obtain. Rather than estimating it, it constructs an incentive:

  • declare a short and you are scheduled into gaps sooner;
  • declare it too short and the job is killed at the limit.

So telling the truth becomes the best strategy, and the information appears by itself. The system solves an information problem by mechanism design (designing the rules so that participants reveal the truth) instead of measurement, the same move as chargeback (billing teams for their usage) and quotas.

The gap goes to whoever can prove they fitreservation at 30 min · 20 nodes free

Boundary. The incentive works only if the penalty is real. Where limits are advisory, declared drift up to the maximum, and backfill stops working.