Skip to main content

Loop

Runs the nodes connected to it repeatedly: over batches of items, a fixed number of times, until a condition holds, or once per page of a data source.

Overview​

A workflow normally passes all of its items through every node at once. That works until the data no longer fits in one run: a table with a million rows can be imported, but pushing every row through the rest of the workflow at the same time runs out of memory.

The Loop node splits the work into iterations. Each iteration runs the nodes inside the loop (the loop body) on part of the data, keeps only what you send back to the loop, and frees the rest before the next iteration starts. You build the body as if it ran over everything at once.

Use it to:

  • Process more rows than fit in a single run, in batches
  • Run one iteration per item, by using a batch size of 1
  • Repeat a step a fixed number of times
  • Repeat a step until a condition holds, such as polling an API until a job is done
  • Page through a data source that is too large to read in one go, one page per iteration

Connections​

HandleKindMeaning
Trigger (top)InputThe items the loop iterates over or that start it
Loop (loop, right)OutputFires once per iteration, with that iteration's items or its index. Connect it to the first body node
Accumulator (bottom)InputConnect body nodes back here. Everything they send is collected when the iteration ends, across all iterations
Continue (continue)OutputFires once, after the last iteration, with every accumulated item
Errors (errors)OutputFires once, after the last iteration, with the body's uncaught error items (only if any exist)

The loop body is every node reachable from the Loop output. Connections back to the Accumulator are return connections and do not add nodes to the body. Continue fires even when nothing was accumulated, so nodes after the loop always run once the loop finishes.

Adding a Loop with the + on an existing connection splits that connection: the node below it, and everything reachable from that node, become the body, reached from the Loop output. Where they cannot all be in the body — one of them is connected to from elsewhere, or belongs to another loop — they stay outside it and the Continue output reaches them instead.

A body does not need a return connection. An iteration ends when every node that ran in it has finished; branches that received no items simply do not run in that iteration.

Parameters​

ParameterDescriptionRequired
LoopOver batches of items, A fixed number of times, Until a condition holds, or Page through a data sourceYes
Batch sizeNumber of trigger items per iteration (default 1000)For Over batches of items
IterationsHow many times to run the body (default 1)For A fixed number of times
Stop whenExpression checked after each iteration; the loop ends when it is trueFor Until a condition holds
Max iterationsSafety limit so a loop that never ends by itself cannot run forever (default 1000)For Until a condition holds and Page through a data source

The four modes​

LoopWhat happens
Over batches of itemsThe trigger items are split into batches of Batch size; one iteration per batch, ending when the items run out
A fixed number of timesThe body runs Iterations times (a "for" loop)
Until a condition holdsThe body runs, then Stop when is checked; it repeats until that is true or Max iterations is reached (a "do-while")
Page through a data sourceThe node at the start of the loop reads one page per iteration; the loop ends when that node has no more rows, or at Max iterations

Over batches of items is the only mode that hands the trigger items to the body: the Loop output carries that batch's items. Use a batch size of 1 to run one iteration per item. With no trigger items the loop runs no iterations, and Continue fires with no items.

In the other three modes the loop does not pass the trigger items through. The Loop output emits exactly one item per iteration, containing only the iteration index:

{ "index": 0 }

Body nodes read their data from the node above the loop instead, for example {{$("Import Table").allItems}}, which the loop keeps available for the whole run. These modes run their iterations whether or not the loop received any trigger items. In Page through a data source the body reads the page from the node at the start of the loop, which is inside the body.

Stop when is checked after an iteration, never before the first one, so the body always runs at least once. It only accepts an expression.

Examples​

Process a million rows in batches​

Import a large table, add a field to every row, and export the result without holding every intermediate item in memory:

[Import Table] → [Loop] ──loop──→ [Edit Item] ─┐
↑ │
└──────── accumulator ───────┘
[Loop] ──continue──→ [Export to CSV]
  1. Import Table reads the rows
  2. Loop: Loop Over batches of items, Batch size 1000
  3. Edit Item (inside the loop) adds or changes fields, for example total = {{$item.data.price * $item.data.quantity}}
  4. Connect Edit Item back to the loop's Accumulator input
  5. Connect Continue to Export to CSV

Each iteration handles 1,000 rows. After the last one, Export to CSV receives every edited row, in order. Expressions after the loop can still pair items with the rows they came from, for example {{$("Import Table").item.data.id}}.

Only accumulate what you need after the loop. If the body writes rows somewhere itself (for example with Insert Rows), you don't need an accumulator at all.

Page through a table that is too large to read at once​

Import Table can read one page of rows per run instead of the whole table. Put it at the start of a paging loop and the loop keeps going until the table runs out:

[Manual Trigger] → [Loop] ──loop──→ [Import Table] → [Insert Rows]

[Loop] ──continue──→ [Send Email]
  1. Loop: Loop Page through a data source, Max iterations 1000
  2. Import Table (the first node inside the loop): turn paging on and set the page size to 5000 rows
  3. Insert Rows writes that page to its destination
  4. Connect Continue to Send Email, which reports that the import finished

Insert Rows writes each page itself, so nothing needs to go back to the Accumulator. Connect it there only if the nodes after the loop need those rows.

Each iteration reads the next 5,000 rows, writes them, and frees them before the next one starts, so a table of any size moves through the workflow in constant memory. Import Table reads page 0 in the first iteration, page 1 in the second, and so on; the loop ends after the iteration where it reports no more rows. A table with no rows at all still runs one iteration, and Continue fires afterwards.

The Loop output hands the body only { "index": 0 }, { "index": 1 } and so on, as in the other repeating modes. The page itself comes from Import Table, so the body reads {{$item}} as usual for the rows of the current page.

A paging loop reads its pages from the single node on its Loop output. If that node is not set to read a page at a time, the loop says so when it runs and stops after the first iteration.

Run the body a fixed number of times​

Create three copies of a report:

  1. Loop: Loop A fixed number of times, Iterations 3
  2. Inside the loop, use {{$item.index + 1}} to number each copy, for example a file name of report-{{$item.index + 1}}.pdf
  3. The rows to put in each copy come from the node above the loop, for example {{$("Import Table").allItems}}

The Loop output hands the body only { "index": 0 }, { "index": 1 } and { "index": 2 }, so the body runs once per iteration no matter how many items the loop received.

Poll a job until it reports done​

A do-while loop that stops on a result, with a cap on the attempts:

[Manual Trigger] → [Loop] ──loop──→ [API Request] ─┐
↑ │
└────────── accumulator ──────┘
  1. Loop: Loop Until a condition holds
  2. Stop when: {{$("API Request").item.data.status === "done"}}
  3. Max iterations: 5

The loop calls the API and stops as soon as the job reports done, or after five attempts. Max iterations is the attempt limit, so the loop needs nothing to count with.

Stop when is the one place outside the body that can read body nodes, so it can check the result of the iteration that just finished.

Handle one item per iteration​

Give every row its own iteration, so one failure only affects that row:

  1. Loop: Loop Over batches of items, Batch size 1
  2. The body receives a single item per iteration, so {{$item}} is that row

This is much slower than larger batches, so use it only when each row really needs its own iteration.

Expressions inside a loop​

Nodes inside the loop body can reference:

  • The current item and the current iteration's items, as usual ($item, $items). In every mode but Over batches of items the loop's own output is its index item, so a node connected straight to Loop reads {{$item.index}} as the 0-based iteration number
  • Other body nodes that already ran in the same iteration, for example {{$("Edit Item").item.data.total}}
  • Nodes before the loop, for example {{$("Import Table").item.data.id}}. Items pair through the loop with the trigger items they came from. In the other modes the body reads whole outputs instead, for example {{$("Import Table").allItems}}

They cannot see data from earlier iterations: a body node that has not run yet in the current iteration returns nothing, never the previous iteration's output. What the body sends to the Accumulator is the way to carry results past the iteration they were produced in.

Nodes outside the loop cannot reference body nodes, and the editor reports an error if an expression does. Use the Continue output instead. The loop's own Stop when is the exception.

The iteration index​

Over batches of items hands the body the iteration's batch of trigger items, so {{$item}} is a row of that batch. The other modes emit a single item holding only the index of the iteration, so {{$item.index}} is the 0-based iteration number and {{$item.index + 1}} counts from 1.

Stop when runs on the loop node itself rather than in the body, so it has no item of its own. It reads the iteration that just finished through a body node, for example {{$("API Request").item.data.status}}. To stop after a set number of repeats, use Max iterations rather than a condition.

See Expressions for more.

Errors​

When a node inside the loop fails on an item, the error item goes to that node's own error output if it is connected. Otherwise the error is uncaught, and the loop decides what happens to it:

  • Errors output connected: the loop collects the uncaught error items from every iteration and sends them on Errors once the loop finishes, up to a hundred per iteration and a thousand for the whole loop. The workflow's Error Catcher ignores them.
  • Errors output not connected: the errors stay uncaught. An enclosing loop with a connected Errors output collects them; otherwise the workflow's Error Catcher does.

Error items are saved to run history whichever iteration they happen in, and the loop keeps its own copy of them once the iteration that produced them is cleaned up. The error count the loop reports is the number of errors that happened, however many items Errors carries. Items on Errors pair with the trigger items they came from. A failed body node still counts toward the run's status, even when the loop collects its errors.

By default the loop keeps going after uncaught errors. Turn off Continue on error to end the loop after the first iteration that has one; Continue still fires with everything accumulated so far. Errors handled by a node's own connected error output never end the loop.

The Error Catcher node cannot be placed inside a loop.

If Stop when itself throws an error, the loop node fails and the nodes after it do not run.

Settings​

SettingDescriptionDefault
Continue on errorKeep iterating after an uncaught error in the bodyOn
Display error outputShow the Errors handle on the canvasOff

Run history and memory are managed for you: the first hundred iterations are saved to run history, and in the editor an iteration that is cleaned up keeps a few sample items per node output so you can still inspect it. Later iterations are not saved individually, but the loop's own Errors output keeps a sample of everything that failed and the loop always reports how many errors there were in total.

In the editor​

  • The loop rectangle. Nodes inside the loop are drawn inside a shaded rectangle. The rectangle follows the connections: connecting a node from the Loop output (or from another body node) puts it inside, and removing that connection takes it out. Drag the rectangle to move the whole body. Nested loops are drawn as nested rectangles.
  • Return connections. Connections to the Accumulator are drawn as dashed lines that run along the bottom of the rectangle back to the loop node.
  • Connection rules. The editor refuses connections that would break the loop, and explains why: body nodes can only connect to other body nodes or to their loop's Accumulator; nodes outside can only enter the body through the Loop output; Continue and Errors must connect to nodes outside the loop; and body nodes cannot connect back to the loop's trigger input.
  • Running the workflow. A run from the trigger runs every iteration, just like a live run. To check the first results before processing everything, start the loop from the loop node itself and step through it.
  • Running step by step. The loop node offers two moves, and never both at once. Initialize loop creates the first iteration without running anything; on a loop that has run before the same control is called Reinitialize, and starts it again from its first iteration. Run this iteration runs the body of the iteration the loop stands at. Advance iteration moves the loop on to the next iteration once the body has run, without running anything; from the last iteration it finishes the loop and publishes its Continue items (in Until a condition holds the condition is checked at this point, so advancing may finish the loop earlier than expected). Run to completion is offered at the start of an iteration: it runs every remaining iteration and then the nodes after Continue. At the end of an iteration, advance first and run to completion from there. Stop cancels the running iteration and ends the loop; the nodes after it do not run. Reinitialize is available whenever nothing in the body is running. Every control is always shown: one that does not apply is greyed out, and hovering over it says why.
  • Running body nodes by hand. Inside an iteration you can run body nodes one at a time, as you would outside a loop. The loop reads what ran: once every node that should run in the iteration has, the iteration counts as done and the loop offers to advance; while some have run and others have not, the loop reads as partly run and offers only Reinitialize. What the body sent to the Accumulator is collected when you advance, so nodes run by hand contribute to Continue exactly as driven ones do, and the accumulated count moves when the iteration ends.
  • The status icon. The loop node's status dot says where the loop stands rather than how its last run ended: a play dot for an iteration waiting to be run, the spinning ring while a body node runs, an ellipsis dot for an iteration run part-way by hand, a skip-next dot for an iteration that is done and waiting to advance, and the usual check, warning, cross or stop dot once the loop has finished — the green check means the loop has finished and fired Continue. Hovering over the dot gives the detail: Not run, Iteration 3 · ready, Iteration 3 · running, Iteration 3 · partly run, Iteration 3 · done, Completed · 12 iterations, with the error count and, for a loop that ran more than once, which run it describes. The nodes inside the loop show what ran.
  • Inspecting iterations. Earlier iterations are cleaned up to save memory; for them, the editor shows the sample of items kept for that iteration together with the original item counts.
  • A loop that runs more than once. If more than one node connects to the loop's trigger input, each arrival starts its own run of the loop, and those runs proceed at the same time without waiting for each other. Each keeps its own iterations, accumulated items and errors, and fires Continue on its own. The loop node describes one of them at a time — the newest that is still going, or whichever run you pick in the iteration selector — and says so: "run 2 of 2, 1 more running". Stopping the node stops all of them.

Tips​

  • Prefer larger batches. Each iteration has a fixed cost, so a batch size of 1 over many items is much slower than a batch size of 1,000. Use batch size 1 only when each item really needs its own iteration
  • Accumulate only what later nodes need. Everything sent to the Accumulator is kept in memory until the loop ends
  • Nodes inside the body process each iteration's items as usual, so a node's own batch size setting still applies within an iteration
  • Keep Max iterations close to the number of repeats you expect, so a mistake in Stop when ends quickly. In Page through a data source it is only a safety net: leave room for every page the source can return
  • Loops cannot shorten a run's time limit: iterations run one after another within the same run
  • A loop fed by two different nodes runs twice at the same time, so it holds one iteration's data per run rather than one in total. Feed a loop from a single node when memory is tight