For AI agents: the complete documentation index is at llms.txt. Remove the trailing slash from a page URL and append .md to read its Markdown version. For the docs home, use index.md.

Fan-Out

A step can split its work into many child runs that execute in parallel, then continue once all of them have finished. Use it to research a list of companies, process the rows of an import, or render the chunks of a video.

Start child runs

Return fan_out with a child for each run to start, and the step to continue with. As on the other pages, Base is the declarative base from Defining workflows, and research_company stands in for your own code:

from sqlalchemy import String, Text
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column

from reflex_workflow import Workflow, child, fan_out, step


class Lookup(Base, Workflow):
    __tablename__ = "lookup"

    id: Mapped[int] = mapped_column(primary_key=True)
    key: Mapped[str] = mapped_column(String(255), unique=True)
    company: Mapped[str] = mapped_column(String(255))
    summary: Mapped[str | None] = mapped_column(Text, default=None)

    @step(retries=3)
    async def research(self):
        self.summary = await research_company(self.company)


class Research(Base, Workflow):
    __tablename__ = "research"

    id: Mapped[int] = mapped_column(primary_key=True)
    companies: Mapped[list[str]] = mapped_column(JSONB)
    found: Mapped[int] = mapped_column(default=0)

    @step
    async def research_all(self):
        return fan_out(
            (
                child(
                    Lookup(key=f"{self.id}:{company}", company=company), Lookup.research
                )
                for company in self.companies
            ),
            then=Research.report,
        )

    @step
    async def report(self):
        lookups = await self.children(Lookup).all()
        self.found = sum(lookup.summary is not None for lookup in lookups)
Expand

child pairs a new row with the step it starts at. The children can belong to any workflow, including the parent's own, and one fan-out can mix several.

Read the children's results

Each child is a run in its own table, so its results are in its own columns. In the then step, self.children(Lookup) addresses the children of this run in the Lookup table, and returns the same kind of handle as Workflow.by.

How children run

Each child is an ordinary run:

  • Whichever worker is free claims it, so the children run in parallel across all of your workers.
  • It retries on its own schedule, and records its own last_error. A child that keeps failing doesn't make the others repeat their work.
  • You can limit how many children run at once, for example per customer or per provider.

While the children run, the parent is a row with nothing scheduled. It holds no worker and no connection.

When the parent continues

The then step runs once the last child finishes, whether that child succeeded, gave up after its retries, or was cancelled. One item that nobody can process doesn't hold up the whole batch. Check each child's columns, or its last_error, in the then step to see which ones succeeded.

If there are no children to start, then runs straight away.

Avoid starting a child twice

The children are inserted in the same transaction that commits the parent's step. If the parent's step is overtaken, for example by run or a worker that took over its claim, none of its children start.

A child that would break a unique constraint isn't started, and the parent doesn't wait for it. Give each child a key derived from the parent and the item, like f"{self.id}:{company}" above, and running the fan-out step again can't start a second run for the same item.