Why nothing ran last night
Almost always one of five things: the window never opened, the kill switch is on, no runner was online, the cards were not ready, or a cap had already been reached.
Start on the night page. It shows the last run and, when that run stopped early, the reason it stopped in the product’s own words. If there is a reason on the screen, find it in the first table below and stop reading; the rest of this page is for when there is nothing there to read.
Two different failures look the same from the board. Either the run never started, or it started and your particular card was passed over. They have different causes and different fixes, so they are separated here. A run that halted says so; a card that was skipped is listed with its own reason.
The run never started. These are checked in this order, and the first one that matches stops everything. The sentence on the right is the one the product shows.
- Nothing is configured
- “No night run is configured.” There is no window, no caps and no schedule yet, so there is nothing for the clock to open. This is the state a new account is in.
- The kill switch
- “The kill switch is on. Nothing will be dispatched.” It is checked before anything else and the night page shows a banner while it is on. It stays on until somebody turns it off, which is the point of it.
- The window is closed
- “Outside the night run window.” Three versions of this are worth knowing apart: no days are scheduled, which is a window that can never open; the window starts and ends at the same time, which is the same thing written differently; and between windows, which names the time the next one opens and means the configuration is fine.
- It already finished
- “This window’s run has already finished.” The window is still open and the run inside it is done. If it finished because the work ran out, that is not a failure: a window from 18:00 to 08:00 says agents may work for fourteen hours, never that the work will take that long.
- The spend cap
- “Spent $X of the $Y cap for this window.” The cap is measured against what was actually spent, so a run can reach it and stop hours before the window closes.
- Too many failures in a row
- “N cards failed in a row. Something is wrong that more attempts will not fix.” A deliberate stop rather than a crash. Something shared is broken, the model key or the repository or the build, and the run declines to spend the rest of the window proving it.
- Not enough time left
- “Only N minutes left in the window, not enough to finish a card, so nothing new is started.” Near the end of a window nothing new goes out. If this is the reason every night, the window is shorter than one card takes.
The run started and your card did not go. Then the card is listed as skipped, with one of these. The first four are properties of the card and will not change on their own; the last three are capacity, and the card is likely to go out later the same night.
- Autonomy is off
- Switched off for that card specifically.
- No risk grade
- How much autonomy it may have is undecided, so it gets none. Grade it and it becomes eligible.
- Above tonight's ceiling
- The card's grade is higher than the maximum this window allows. Either the ceiling moves or the card waits for a person.
- Not ready
- The card is missing something an agent needs. The blockers are: no description, no acceptance criteria, no repository linked, no risk grade, and no title. Each one is shown on the card itself.
- Over the per-card cap
- The forecast for that one card is above the per-card limit. The forecast is points times a rate and is deliberately rough: it exists to catch a card that is clearly too big.
- No idle coding runner
- Every runner is busy, or there are none. This is the one to check first if nothing at all moved.
- At the concurrency limit
- The window's limit on cards in flight is reached. The card waits for one to finish.
And the case with no message at all, which is the confusing one. A card is only a candidate if it is in the ready column and has no attempt already running. A card anywhere else is not skipped, it is not considered, so it produces no line and no reason. If a card you expected to run is not in the skipped list either, this is why: check which column it is in.
One thing that is easy to miss about runners. A runner reports in every minute or so, and one that has not reported for ninety seconds is counted as offline even though its process may still be alive. A machine that went to sleep, lost its network, or was never restarted after a deploy reads exactly like a machine that was switched off. If the runner looks online to you and offline to us, that gap is usually the reason.
If none of the above matches, the useful thing to send us is the night page for the date in question and the code of one card you expected to run. Those two together say which of the two failures it was, which is most of the answer.
This did not answer it? Write to us. A person reads every message.