TUTORIAL

How to Process Lists and Batches in AI Workflows: The Foreach Step

Apr 10, 2026 8 min read

Most workflow tutorials show you a clean, linear pipeline: a trigger fires, a few steps run, a result comes out the other end. That works fine when you are processing one thing at a time. But real business processes rarely work that way.

You get a directory with 12 new files. A webhook delivers a payload with 40 order records. A database query returns 200 rows that each need validation. The moment your workflow needs to handle a list of things instead of one thing, you need iteration — and that is exactly what the foreach step provides.

This guide covers how the foreach step works in Orckai workflows, when to use it, when not to, and how to handle the partial failures that inevitably happen when you process batches.

The Batch Processing Problem

Without a foreach step, handling lists in a workflow means one of three things:

Build a separate workflow per item — which means you need something external to split the list and trigger each one. You have moved the problem, not solved it.
Write a code step that loops internally — which works for simple transformations but falls apart when you need AI agent calls, MCP tool invocations, or condition branches inside the loop.
Process the entire list as a single blob — which means one failure kills everything, you get no per-item visibility, and your LLM calls become expensive because you are cramming the entire dataset into a single prompt.

The foreach step eliminates these workarounds. It takes an array, iterates through each element, runs a full sub-pipeline per item, and collects the results — including which items succeeded and which failed.

How the Foreach Step Works

A foreach step has four configuration properties:

collection — The array to iterate over. This is a reference to the output of a previous step, using the standard {{stepName.path.to.array}} variable interpolation syntax.
itemVariable — The name you give each element during iteration. Defaults to item. Inside sub-steps, you reference the current element as {{item}} (or whatever name you choose).
steps — The sub-pipeline that runs once per element. This can include any step type: agent, code, condition, transform, MCP tool, or action.
continueOnError — Whether to keep processing remaining items when one fails. Defaults to true, which is the right default for batch processing — you usually want all results, not a halt on the first failure.

Here is what the configuration looks like in JSON:

{
  "type": "foreach",
  "name": "process_files",
  "config": {
    "collection": "{{detect_new_files.files}}",
    "itemVariable": "file",
    "continueOnError": true,
    "steps": [
      {
        "type": "code",
        "name": "validate_format",
        "config": {
          "code": "const ext = inputs.file.name.split('.').pop(); return { valid: ['csv', 'json', 'xml'].includes(ext), filename: inputs.file.name };"
        },
        "outputVariable": "validation"
      },
      {
        "type": "condition",
        "name": "check_valid",
        "config": {
          "field": "validation.valid",
          "operator": "equals",
          "value": true
        },
        "onFalse": "stop"
      },
      {
        "type": "action",
        "name": "download_file",
        "config": {
          "action": "utility",
          "toolName": "sftp_download",
          "parameters": {
            "connection_name": "{{sftp_connection}}",
            "remote_path": "{{file.path}}"
          }
        },
        "outputVariable": "downloaded"
      }
    ]
  }
}

When the foreach completes, the output has a consistent structure:

{
  "items": [
    { "index": 0, "status": "completed", "item": { ... }, "outputs": { "validation": { ... }, "downloaded": { ... } } },
    { "index": 1, "status": "failed", "item": { ... } },
    { "index": 2, "status": "completed", "item": { ... }, "outputs": { "validation": { ... }, "downloaded": { ... } } }
  ],
  "total": 12,
  "succeeded": 10,
  "failed": 2
}

Downstream steps can reference {{process_files.succeeded}}, {{process_files.failed}}, or iterate over the items array to build summaries, send alerts, or decide what to do next.

Example 1: Validate and Download Files from SFTP

The scenario: A monitoring step detects 12 new files in a remote SFTP directory. Each file needs to be validated (correct format, correct naming convention, non-zero size) and then downloaded to local storage. Some files might be malformed, and you do not want one bad file to block the other 11.

The workflow:

Schedule trigger — runs every 15 minutes
MCP tool step — connects to the SFTP server and lists new files since the last check, returning an array of file metadata objects
Foreach step (continueOnError: true) — iterates over the file list with three sub-steps:
- Code step — validates the file extension, naming convention, and reported size
- Condition step — checks validation result; if invalid, onFalse: "stop" skips this item
- Action step — downloads the validated file via SFTP
Code step — builds a summary from the foreach output: which files passed, which failed, and why
Action step — sends an email: "SFTP Batch Complete: 10 downloaded, 2 failed" with the failure details

Why foreach matters here: Without it, the first invalid file would kill the entire batch. With continueOnError: true, the workflow processes all 12 files, captures two failures with their reasons, and delivers a complete report. The operations team sees exactly what went wrong without digging through logs.

Example 2: Process a Webhook Payload with Multiple Records

The scenario: An external system sends a webhook with an array of new orders. Each order needs to be validated against business rules, and orders that exceed a dollar threshold or contain flagged SKUs need a ServiceNow ticket created for manual review.

The workflow:

Webhook trigger — receives the payload, authenticated via X-Webhook-Secret header
Transform step — extracts the orders array from the webhook body
Foreach step (continueOnError: true, itemVariable: "order") — iterates over orders:
- Code step — validates required fields (customer ID, line items, shipping address) and checks the order total against the review threshold
- Condition step — if needs_review is false, onFalse: "stop" skips the ticket creation and moves to the next order
- Agent step — generates a human-readable summary of why this order was flagged (high value, flagged SKU, new customer, etc.)
- Action step — creates a ServiceNow incident with the order details and the AI-generated summary
Code step — aggregates: "40 orders processed, 6 flagged for review, 34 passed"
Action step — posts the summary to a Slack channel

Why foreach matters here: The webhook delivers the entire batch at once. The foreach step lets you apply per-order logic — including an AI agent that generates contextual explanations — without processing the entire array in a single LLM call. Each order gets individual attention, and the output tells you exactly which ones created tickets.

Sub-Step Conditions: Filtering Items Without Stopping the Loop

A common pattern inside foreach is using a condition step with onFalse: "stop". This is not the same as stopping the entire workflow. Inside a foreach, "stop" means stop processing the current item and move to the next one. The foreach continues.

This gives you filtering behavior. You can check whether the current item meets your criteria, and if it does not, skip it cleanly. The item shows up in the output as status: "completed" (because skipping via a condition is not an error), but the downstream sub-steps within that iteration do not run.

Some practical filtering patterns:

Amount threshold — "If {{order.total}} is less than $500, skip." Only high-value orders get the expensive AI review step.
File type gate — "If the file extension is not in the allowed list, skip." Only valid formats get downloaded.
Duplicate check — "If {{record.id}} already exists in the database, skip." A code step queries the DB first, and the condition step acts on the result.
Status filter — "If {{incident.status}} is 'resolved', skip." Only open incidents get the escalation logic.

The key insight is that condition steps inside foreach work at the item level, not the workflow level. You are building a per-item pipeline with its own branching logic.

Error Handling Strategies

The continueOnError flag is the most important design decision in a foreach step. It determines whether your workflow treats the batch as a collection of independent items or a dependent chain.

continueOnError: true — Independent batch processing

Use this when each item is independent and a failure on one should not affect the others. This is the common case for file processing, record validation, notification delivery, and report generation.

// continueOnError: true (default)
// Item 3 fails? Items 4-12 still process.
// Output: { total: 12, succeeded: 11, failed: 1 }

The real value here is that you do not lose work. If you are processing 50 invoices and one has a malformed field, the other 49 still get processed. The failure is captured in the output, and a downstream step can handle it — send an alert, log it, retry it later.

continueOnError: false — Dependent chain processing

Use this when items have dependencies or when a single failure means the entire batch is unreliable. Examples: sequential database migrations, ordered transaction processing, or deployment steps where a failure in step 3 means steps 4 through 10 should not run.

// continueOnError: false (opt-in for fail-fast)
// Item 3 fails? Items 4-12 are skipped.
// Output: { total: 12, succeeded: 2, failed: 1 }
// (items 4-12 never ran)

In this mode, the foreach stops at the first failure and reports exactly where things went wrong. The output includes the successful items, the failed item, and the count of items that were never attempted.

Choosing the right mode

Ask yourself: "If item 5 out of 20 fails, do I want items 6 through 20 to still run?" If the answer is yes, use continueOnError: true. If the answer is "it depends on why it failed," you probably want true with a downstream condition step that inspects the failure count and decides whether to proceed with the rest of the workflow.

When Not to Use Foreach

Foreach is not always the right tool. There are two common situations where it is the wrong choice.

Storage change triggers already loop for you

If your workflow uses a storage change trigger, the trigger itself fires once per file. Drop 10 files into a monitored folder and you get 10 independent workflow executions — each one processing a single file. The trigger IS the loop.

Wrapping a storage change trigger in a foreach is redundant. Each execution already has exactly one file. There is no array to iterate over. Use foreach when a single trigger produces an array (like a webhook payload or a query result), not when the trigger mechanism itself handles the fan-out.

High-volume parallel processing

Foreach is sequential. Item 1 completes before item 2 starts. For most business workflows — processing 10 to 100 items where each sub-pipeline takes a few seconds — this is fine. A foreach over 50 items with a 2-second sub-pipeline takes under 2 minutes.

But if you need to process thousands of items, or if each sub-pipeline involves a long-running operation (large file downloads, complex AI analysis), sequential processing becomes a bottleneck. In that case, use a code step to split the array and trigger multiple workflow executions via the API, running them in parallel. Each execution handles a subset of the list.

The rule of thumb: foreach for tens to low hundreds of items. Parallel workflow executions for thousands.

Combining Foreach with Workflow Variables

Foreach sub-steps can reference workflow-level variables just like any other step. This means you do not need to hardcode configuration values inside the foreach pipeline.

// Workflow-level global variables (set in the workflow builder Settings step)
{
  "globalVariables": {
    "sftp_host": "sftp.example.com",
    "alert_email": "ops-team@company.com",
    "review_threshold": 5000
  }
}

// Inside the foreach sub-steps, reference them directly by key:
// {{sftp_host}} in the SFTP download action
// {{review_threshold}} in a condition step value
// {{alert_email}} in the summary email action

This keeps your foreach pipeline clean and reusable. Change the SFTP host or the review threshold in one place, and every iteration picks up the new value. It also makes it easy to clone a workflow for a different environment — swap the variables, keep the foreach logic identical.

A practical pattern is to set variables at the workflow level for anything that applies to all items (connection strings, thresholds, recipient lists), and use the {{item}} variable for anything specific to the current element (file name, order ID, record data).

Summary

The foreach step handles a problem that every non-trivial workflow eventually hits: processing a list of things, one at a time, with proper error handling and visibility into what happened to each item.

The key points:

Foreach iterates sequentially over an array, running a full sub-pipeline per item. It is not parallel.
continueOnError determines whether one failure stops the batch or just records it. It defaults to true, which is the right choice for most batch processing. Set it to false only when items depend on each other.
Condition steps inside foreach filter at the item level. onFalse: "stop" skips the current item, not the entire workflow.
The output structure always tells you what succeeded and what failed, with counts and per-item details.
Do not use foreach with storage change triggers — the trigger already handles per-file fan-out.
For high-volume work, split the array and trigger parallel workflow executions instead.

Most business workflows process tens to hundreds of items at a time: files from SFTP, records from a webhook, rows from a database query, incidents from a monitoring system. Foreach is designed for exactly that scale — giving you per-item control, per-item error handling, and a clean summary when the batch is done.

For real-world workflows that use foreach, see 5 Business Operations You Should Automate with AI Workflows. Or jump into the workflow builder and try it out — the Quick Start templates include foreach-based pipelines you can deploy in one click.