A pause that held the order open

A flow can contain a wait. You provision a machine, you give the agent a moment, then you send the next command. Until this week that wait was a sleep inside the request that started the flow, which meant a customer placing an order sat looking at a spinner for the duration of the pause.

The old upper bound was five minutes. Somebody could therefore build a perfectly reasonable looking flow whose effect was a five minute order form.

Now that a run can stop and be continued by a worker, the pause does not have to happen inside the request any more. The upper bound moved to thirty days. The interesting question is not the ceiling, though. It is where the line sits between a pause the request can absorb and a pause that should suspend the run.

The threshold is measured, not chosen

Thirty seconds.

The question we asked was not "how long may a pause be", which has no answer, but "how much pause does the request this flow runs in already tolerate", which does. That request is the order.

Measured against what already happens in it: an action node waits sixty seconds for a command by default and may be configured up to three hundred, and a notification with email costs about half a second per recipient. A pause that adds another half minute on top of that is defensible. A pause that adds a full minute is not.

There is a trap in this that we deliberately walked into rather than around. The comfortable choice would have been to set the threshold at the old upper bound of three hundred seconds, because then every existing graph behaves exactly as it did before and nobody is surprised. That is also wrong: it preserves the entire class of problem that motivated the change. Graphs with a pause between thirty one and three hundred seconds now do behave differently. Their order finishes earlier and the rest of the flow follows later. That is the improvement, not the cost of it.

The ceiling of thirty days is a product rule and not a technical limit. The wait machinery would happily hold a run for a year. What cannot hold for a year is an organisation in which nobody remembers why a run is open.

Why it is a threshold and not a checkbox

The obvious alternative is a "wait durably" toggle on the node. We did not build one, and the reason is a question you cannot answer.

If there were a checkbox, when would you want it off? The honest answer is never. Durable waiting is an implementation detail: the writer of a flow is expressing "pause here for this long" and has no opinion about whether that costs a database row. A setting whose correct value is always the same is not a setting, it is a thing you forgot to decide.

So the writer types a number of seconds and the threshold decides. Below the line the walk simply sleeps, inside the request, costing nothing. Above it, the run writes its continuation to the queue with a wake time and the caller returns immediately.

The step that eventually appears in the run log carries a durable flag and the actually measured elapsed time rather than the requested one, because those are not the same number and the difference is the only way to see that the queue is running late.

Flows that call flows, and two limits checked twice each

The second half of this work is a subflow node: a flow that runs another flow and waits for it.

That creates two ways to build something that never terminates, and they look similar enough that it is tempting to check them together. They are not the same thing. A cycle is infinite. A chain that is too deep is finite but too long. If you answer both with one function, a cycle gets reported as "too deep", and the writer goes looking for a long chain when theirs is two flows long.

So there are two questions, and each is asked twice.

They are asked once when a flow is saved, statically, against the other flows in the organisation. That check has to be there because it is the only place where a good error message is possible: it can name the flow that closes the loop.

They are asked again during the run. That check has to be there because flows are edited independently. Flow A is saved and is fine. Somebody then edits flow C, which now points back at A. Nothing revalidated A. The static check is a courtesy to the writer; the dynamic one is the actual guarantee.

Two implementation details are worth naming. Asking "can this flow reach itself in at least one step" answers both the direct case and the indirect one in a single query, and the "at least one" is the entire definition: a path of zero steps would make every flow reach itself. And the depth walk keeps its own path and returns infinity when it meets a flow already on it, so it does not hang on exactly the input the cycle check exists for. That matters because it means the two checks can be run in either order, which means nobody has to remember an ordering rule.

Depth is stored as a column on the child run, which is against our usual line that a derivable value is not a column. The foreign key to the parent nulls out when the parent is deleted, so after a deletion the chain cannot be walked back to recompute it. A derived value whose source can disappear is not derivable.

The fallback clock is a day, not five minutes

When a child run finishes it sets the wake time on the parent's wait row to now, and the parent is picked up within ten seconds. That is the mechanism.

There is also a fallback clock, for the case where that does not happen: a container that fell over halfway through finishing the child, a row that was touched by hand. It is twenty four hours.

Five minutes would be the instinctive choice and it is the wrong one. A child that is sitting on an approval may legitimately wait thirty days. With a short clock the parent would wake thousands of times over that period to observe that nothing has changed, and each of those is an HTTP request from the gateway to the web container. A day bounds the damage of a missed signal to a day and costs at most thirty pointless wake ups over the longest wait the system permits.

An approval that nobody is told about is a stalled run

An approval request can now send an email, and the category it is sent under is its own rather than the general flow category. That was a deliberate deviation from the plan.

The general flow category goes to everybody who has the automation module. An approval request should go to the people who can actually decide it, which is a different and smaller set. So there is a second sieve on the category: the notification is only delivered to members who hold the deciding permission.

Measured on our test organisation: the general flow category resolved to three recipients and the approval category to two. Two numbers that differ is the whole proof that the sieve is doing something.

A click that selected nothing

The measurement lesson from this round is about the browser test rather than the product.

Our browser helper has a click method that dispatches a mouse press and a mouse release. On an ordinary button that works. On a node in the flow canvas it does nothing at all: the properties panel stayed on the trigger node and the dropdown we wanted to change did not exist, which reads exactly like a missing feature.

The canvas starts a drag on pointer down and calls setPointerCapture on the node. Without a preceding mouse move event, and without the buttons field set on the press, the browser does not produce a click that the capture path recognises. Adding those two makes the selection immediate.

There is a second detail in the same helper. Do not click the middle of a node. The middle of a node with ports sits between the port dots, and those have their own pointer handler for dragging a wire out. Click the top left of the header instead.

Both of these look like product bugs when you hit them, and neither is. That is the general shape of the problem: an input simulation that is almost right fails in a way that is indistinguishable from the thing you were trying to test being broken.

What we would write down

1. 01A threshold beats a checkbox when the correct value of the checkbox is always the same. Ask when somebody would turn it off; if there is no answer, do not build the switch.
2. 02Measure the threshold against the budget it has to fit in, not against the feature you are limiting.
3. 03Do not set the new threshold at the old limit just because nothing then changes. Nothing changing is the problem you were solving.
4. 04A cycle and a chain that is too long are different errors. One message for both sends people to the wrong place.
5. 05Validate on save for the message and at runtime for the guarantee. Neighbouring records change without revalidating yours.
6. 06A search that terminates on its own path can be run in any order relative to the check it supports, which removes an ordering rule nobody would remember.
7. 07A derived value whose source can be deleted is not derived. Store it.
8. 08Size a fallback clock against the longest legitimate wait, not against how quickly you would like to notice.
9. 09Route a notification by who can act on it, not by which module it belongs to.
10. 10A simulated input that is almost right fails like a broken feature. Check how the component actually receives events before believing the result.

For clarity about what this is: the merge verification ran 16 of 16 steps without a failure line, the web suite was 4583 green, the package, gateway and tools suites 7286 green, the walk tests 75 green, the subflow checks 13 green, the end to end run against the development database 58 green, the recipient measurement 4 green, and the browser test 31 green with a real gateway worker, in which a ninety second pause resumed after ninety five seconds, a parent and child run were linked, exactly one approval email was sent, and a chain of four and a recursive reference were both refused with the offending chain named in the message. The editor test was 9 green and the accessibility sweep found nothing across six pages. Not proven: the attempt exhaustion and stale claim branches, the twenty four hour fallback clock, a subflow inside a loop, a vanished child run, the email to a non owner through a real send, and nothing at all against a real hypervisor.

From the platform work on Provibr. How the walk learned to stop in the first place is in every self call was in tail position, and the shape of the graph itself is in a flow is a graph that has to be finite.