Tree Workflow API

Static and status trees use the shared text display connectors while preserving task execution and reverse cleanup order. Framework error records use one complete diagnostic and compact Python traceback evidence by default.

Design model

lclang.workflow follows the same separation as Modules and Frames. A workflow is a strong reusable format: an immutable tree of valid task IDs, typed dataclass boundaries, explicit variable flows, ordered contexts, and ordered children. Workflow.execute supplies flexibility only inside that format: actions are ordinary async Python, contexts may own arbitrary dismissible resources, and status managers may add run-specific detail.

The definition never owns execution state. One execution borrows a shared Frame, derives short-lived task Frames, records argument and output values, and creates a fresh status tree. Reusing a Workflow is therefore analogous to reusing a Module; each call is analogous to creating a new per-run Frame.

For runtime-sized batches, use async with workflow.execute_in_task(...) from an action. It provides an isolated Frame, native results, detached status attachment and explicit resource lifetime.

define_workflow(title, root_task, lcl_mixin=None) accepts a dict[str, Any] of host values and callables, for example lcl_mixin={"service": service}. The mapping is copied shallowly when defined; objects remain shared by reference. Qualified names and trailing ! masking markers follow normal Frame rules. Invalid names and conflicting scopes fail at definition time.

Direct execute mixes these bindings into the borrowed shared Frame before any task starts. They override existing bindings and remain in that Frame afterward. Configuration expressions, context tasks, and actions resolve them through the same Frame hierarchy. Existing cached dependants remain snapshots; mixins do not invalidate them. Task outputs may subsequently replace the supplied bindings.

For to_cli, these bindings instead seed the command's host preset before configuration evaluation and logger setup. Explicit to_cli(preset=...) values, configuration definitions, and CLI overrides retain their normal precedence over these defaults. Only inferred external task variables remain CLI parameters; extra host helpers do not become parameters. Dynamic using targets retain the loader's existing scope and do not receive command host presets.

Variables and mappings

See variable types and boxes and mapping shapes and best practices for supported annotations, conversion costs and complete examples of each mapping shape.

Variables accept mutually exclusive default= and default_factory= keywords. Omission differs from an explicit default=None; an explicitly supplied default_factory must be callable. Defaults are lower priority than all effective configuration, CLI, preset and host bindings. They are available to both mapping resolution and LCL expressions during execution. A missing quote may finally use its dataclass field default; a failed existing definition never falls back. Factories accept no arguments, may return an awaitable, and run only on first use, sharing one result or failure per execution. Fixed default values retain identity; use factories for independent mutable values. Help displays factory metadata without invoking factories. Reuse a fresh Frame when new dependency values are needed: cached expressions remain snapshots.

Mappings recursively resolve dataclass fields, including nested whole-record quotes. Containers, callbacks, and concrete referenced records retain their identity. Mapping templates are not mutated. Cyclic templates are rejected; reusing an acyclic nested template is supported. Output extraction validates the complete structure and target set before publishing one atomic update.

variable.field("name", FieldType).quote reads a declared dataclass field. Projections may be chained, remain read-only, and retain the original variable's dependency and masking metadata. Missing fields, incompatible declared types, and null intermediate values report their mapping and variable paths. CLI inference, static trees and verbose logs follow the same recursive structure. Verbose mapping logs expand a field when its declared annotation is a dataclass and its value is a dataclass instance. Other fields are rendered as single typed values. This includes LCL functions declared through callback Protocol or Callable annotations. Generic dataclass annotations retain nested field paths; masked fields are checked before their values are read. Ordinary failures in verbose mapping rendering or DEBUG emission discard that record. They do not prevent actions or contexts from running, change publication or status, or replace execution and cleanup failures. Contexts still unwind in their declared order. Native control signals continue to propagate. Runtime validation checks record structure and reference contracts, not every ordinary value or container element. FrameProxy.as_record() remains shallow; recursive conversion is specific to workflow mappings.

Create variables through the subscribed factory:

from lclang.workflow import define_variable

source = define_variable[int]("source", "Required source value")
secret = define_variable[str]("service.token", "Service token", is_masked=True)
result = define_variable[float]("result")

The subscription accepts Python type annotations, including list[str], Mapping[str, int], Callable[[str], int], unions, and generic dataclasses such as Record[int]. The resulting .quote retains that exact static type.

An entire argument or output mapping may also be a dataclass variable's .quote. Its declared type must exactly match the callable's corresponding annotation (including generic arguments); non-dataclass variables cannot be whole mappings. The keyword remains outputs_mapping for output publication. These rules apply to actions and context tasks alike.

args_mapping=var.quote reads just var.name from the task Frame. A stored dataclass is passed through; a scope proxy is materialized using as_record with the declared dataclass type. Conversion is recursive, preserves constructor defaults, and reports missing required fields or failed field expressions. Runtime checks validate the dataclass class, not its individual field types.

outputs_mapping=var.quote publishes the returned dataclass under var.name. If that name already denotes a scope in the destination Frame, its direct dataclass fields instead become var.name.field bindings in one mixin update. This is shallow publication, not recursive asdict conversion: nested values retain their identity, and unrelated existing scope fields remain unchanged. Action outputs target the shared Frame; context outputs target the task Frame. Whole-variable masking also masks every field published into a scope. Whole mappings participate in static trees, verbose logs, and CLI input inference as the named variable; existing per-field mappings remain supported. The existing output target is resolved to determine whether it is a proxy; resolution errors use the normal workflow error path without publishing fields.

import asyncio
import logging
from dataclasses import dataclass
from datetime import date

from lclang.lang import define_frame
from lclang.workflow import (
    ExecutionStatus,
    ExecutionStatusManager,
    TaskContext,
    TaskID,
    WorkflowExecutionContext,
    define_task,
    define_variable,
    define_workflow,
)


@dataclass
class Batch[T]:
    items: list[T]


async def copy_batch(
    context: TaskContext,
    args: Batch[str],
    status_mgr: ExecutionStatusManager,
) -> Batch[str]:
    return args


async def main() -> None:
    source = define_variable[Batch[str]]("source")
    target = define_variable[Batch[str]]("target")
    task = define_task(
        "copy", "Copy batch", task_action=copy_batch,
        args_mapping=source.quote, outputs_mapping=target.quote,
    )
    workflow = define_workflow("Batch copy", task)
    async with define_frame(
        preset={"source.items": ["a", "b"], "target.items": []},
    ) as frame:
        result = await workflow.execute(WorkflowExecutionContext(
            False, date(2026, 9, 21), False, logging.getLogger("batch-example"), frame,
        ))
        assert result.execution_status.status is ExecutionStatus.SUCCESS
        assert result.task_args[TaskID("copy")] == Batch(["a", "b"])
        assert await frame.get("target.items") == ["a", "b"]


asyncio.run(main())

The qualified input creates a source scope. The whole argument mapping reads its items child into Batch[str], and the action returns that record. Since target is also a scope, publication updates target.items. Without the initial target.items binding, publication would instead store the record at target. In both cases the task result retains the returned dataclass.

TaskVar.quote is statically typed as the represented value but returns the variable marker at runtime. It can therefore occupy a normally typed dataclass field in a definition mapping. Nested argument fields containing a marker are resolved from the current task Frame. Other fields retain their literal value, including values supplied by dataclass defaults.

Output publication is opt-in. With no output mapping, nothing is published. Within a mapping, only fields containing TaskVar.quote are copied from the returned dataclass into a Frame; other fields are retained in task_outputs but remain unmapped. Masked variables use the same exact-name redaction model as Modules, Presets, configuration, CLI bindings, and Frame mixins.

Grouped arguments and projections

import asyncio
import logging
from dataclasses import dataclass
from datetime import date

from lclang.lang import define_frame
from lclang.workflow import (
    ExecutionStatus,
    ExecutionStatusManager,
    TaskContext,
    TaskID,
    WorkflowExecutionContext,
    define_task,
    define_variable,
    define_workflow,
)


@dataclass
class FormatOptions:
    encoding: str


@dataclass
class ConvertArgs:
    options: FormatOptions


async def convert(
    context: TaskContext,
    args: ConvertArgs,
    status_mgr: ExecutionStatusManager,
) -> ConvertArgs:
    return args


async def main() -> None:
    source = define_variable[FormatOptions]("source")
    target = define_variable[str]("target.encoding")
    task = define_task(
        "convert", "Resolve grouped options", task_action=convert,
        args_mapping=ConvertArgs(FormatOptions(source.field("encoding", str).quote)),
        outputs_mapping=ConvertArgs(FormatOptions(target.quote)),
    )
    workflow = define_workflow("Grouped conversion", task)
    async with define_frame(preset={"source.encoding": "utf-8"}) as frame:
        result = await workflow.execute(WorkflowExecutionContext(
            False, date(2026, 9, 22), False, logging.getLogger("grouped"), frame,
        ))
        assert result.execution_status.status is ExecutionStatus.SUCCESS
        assert await frame.get("target.encoding") == "utf-8"
        assert result.task_args[TaskID("convert")] == ConvertArgs(FormatOptions("utf-8"))


asyncio.run(main())

The projection materializes the source record from its scope, then reads its encoding field. The nested argument template becomes a separate record; the returned nested field publishes to the explicitly selected target. CLI analysis recognizes the projection's original source variable as the input, while the output remains an assigned value.

Tasks and contexts

An action has the exact async form:

from lclang.workflow import ExecutionStatusManager, TaskContext


async def action(
    context: TaskContext,
    args: Args,
    status_mgr: ExecutionStatusManager,
) -> Outputs:
    ...

Args and Outputs are dataclasses with plain annotated fields. lclang checks the callable boundary and dataclass shape but deliberately does not impose runtime field validation. Application code remains responsible for semantic types received from external systems.

Each TaskNode may have ordered context tasks and ordered child tasks. Context factories accept the same three arguments and return an async context manager. They enter left-to-right before the action and exit right-to-left after all children. Their mapped resources enter the owning task Frame. Child task Frames derive from their parent task Frame, so these resources remain visible throughout the owned subtree and do not leak to outside sibling branches. Mapped action outputs enter the shared Frame and are visible to later parent-first, depth-first tasks.

Each derived task-node Frame contains __task_id_branch__, the dot-connected root-to-node ID path. A context-task lifecycle log appends its own ID, but its shared task Frame continues to expose the owning task-node branch.

FailureCoveringContextTask is the standard template for recovery contexts. Subclasses implement acquire, handle_exception, and optionally release. If handling fails, the original business failure and handling failure are retained together. Release still runs, and a release failure joins the pending failures under LCL535911. Cancellation and other control signals keep their native identity with ordinary failures retained in their cause chain. Iterator termination signals also propagate through execution and recovery boundaries, including native groups that contain them. They are not treated as successful workflow execution or as failures covered by a recovery handler. Successful handling and cleanup suppress the ordinary exception and mark the context FAILURE_COVERED. Covered failures remain visible and still stop later siblings; covering means the failure was wrapped up, not that execution can be resumed safely.

Execution and results

Omitting child tasks

During its action, a task may call synchronous context.skip_children() to omit all its children and their descendants, including their context tasks. No status nodes (not even SKIPPED), lifecycle logs, or argument/output entries are created for that subtree. The call is idempotent and irreversible for this execution; it does not return early from the action or change its status. Normal output validation/publication still occurs, entered contexts unwind in reverse order, and successful completion permits later parent-level siblings.

The omission takes effect immediately, even if the action subsequently fails, output publication fails, cleanup fails, or an exception is covered. Those failures retain their normal reporting and propagation, but omitted children are never added back as skipped status nodes. Cancellation still propagates. Calls before or after the action's active lifetime raise LclStateError; context entry and exit are outside that lifetime. A task with no children may call the method without additional effects. Each execution owns fresh control state, including concurrent executions of the same workflow.

Static to_lines() still shows the complete definition tree, and CLI parameter inference and required-input checks still consider all declared tasks.

import asyncio
import logging
from dataclasses import dataclass
from datetime import date

from lclang.lang import define_frame
from lclang.workflow import (
    ExecutionStatus,
    ExecutionStatusManager,
    TaskContext,
    WorkflowExecutionContext,
    define_task,
    define_workflow,
)


@dataclass
class Options:
    run_children: bool


async def choose(
    context: TaskContext,
    args: Options,
    status_mgr: ExecutionStatusManager,
) -> Options:
    if not args.run_children:
        context.skip_children()
    return args


async def main() -> None:
    child = define_task(
        "child", "Optional child", task_action=choose, args_mapping=Options(True),
    )
    root = define_task(
        "root", "Choose children", task_action=choose,
        args_mapping=Options(False), children=[child],
    )
    async with define_frame() as frame:
        result = await define_workflow("Conditional work", root).execute(
            WorkflowExecutionContext(
                False, date(2026, 9, 21), False, logging.getLogger("skip-example"), frame,
            )
        )
    assert result.execution_status.status is ExecutionStatus.SUCCESS
    assert result.execution_status.sub_tasks[0].sub_tasks == []
    assert set(result.task_args) == {"root"}
    assert result.task_outputs == {"root": Options(False)}


asyncio.run(main())

The root action chooses not to enter its child, then returns its normal output. The root succeeds and retains that output; the child has no status node or materialized values. The immutable definition still contains the child for other executions that choose to run it.

Result values

await workflow.execute(context) returns WorkflowExecutionResult with the finalized execution_status, the borrowed execution_frame, and dictionaries of successfully materialized action arguments and returned outputs. A key is added to task_args after argument construction and to task_outputs after a valid action return. Structural, unreached, and failed stages are absent.

Raised exceptions become WorkflowException under the shared Frame name __exception__. The value retains the ordinary exception and originating task or context-task ID. Explicit FAILURE, ERROR, or FAILURE_COVERED statuses also stop execution and receive a deterministic synthetic exception. All unexecuted declared branches appear as SKIPPED.

Severity is ERROR, FAILURE, FAILURE_COVERED, then PENDING. Clean work retains the existing success/all-skipped aggregation rules. Process-control BaseException values unwind task contexts and Frames, then propagate.

Static trees and CLI conversion

Dataclass parameter fields

An external definite dataclass exposes its root description and constructor fields in help, including generic specialization and nested paths. Field help uses metadata={"help": "..."}; missing help uses NO HELP MESSAGE PROVIDED. Containers, unions, explicit boxes and recursive type boundaries remain leaves. init=False, ClassVar and InitVar are not configurable paths. Explicit leaf declarations retain their help and must agree with inferred annotations. Root masks propagate to known fields in help, overrides and audit logs.

Constructor defaults are display metadata, not injected Frame bindings. None displays as a real default; factories display as <factory> without running. Child required/optional labels describe construction without enclosing defaults. Presence checks apply to the root; once a scope exists, missing required fields retain the shared constructor's normal workflow error. An absent dataclass whose constructor fields all have defaults can be constructed in both CLI and direct execution. Mutable factory defaults remain independent. Existing bindings that fail never fall back to constructor defaults.

Inferred paths work in to_cli(preset=...) and -o; unknown paths remain usage errors. Existing CLI value parsing is unchanged; field types do not introduce coercion. Whole-object bindings and child paths remain mutually exclusive. Use flatten_to_dict(instance, prefix="csv") for field presets. Root objects and factory results are not deep-merged with partial child overrides.

import asyncio
from dataclasses import dataclass, field

from lclang.utils import flatten_to_dict
from lclang.workflow import (
    ExecutionStatusManager, TaskContext, define_task, define_variable, define_workflow,
)


@dataclass
class CsvOptions:
    encoding: str = field(default="utf-8", metadata={"help": "Input encoding"})
    delimiter: str = field(default=",", metadata={"help": "Input separator"})


async def convert(
    context: TaskContext, args: CsvOptions, status_mgr: ExecutionStatusManager,
) -> CsvOptions:
    assert args == CsvOptions(encoding="gbk", delimiter=";")
    return args


csv = define_variable[CsvOptions]("csv", "CSV conversion options")
workflow = define_workflow("CSV", define_task(
    "convert", "Convert", task_action=convert, args_mapping=csv.quote,
))
command = workflow.to_cli(
    "convert", "Convert CSV", preset=flatten_to_dict(CsvOptions(encoding="gbk"), "csv"),
)
assert {item.name for item in command.parameter_docs} == {"csv", "csv.encoding", "csv.delimiter"}
assert asyncio.run(command.run(["python", "tool.py", "-o", "csv.delimiter", ";"])) == 0

One declared variable supplies the whole record. Its fields provide additional help rows and accepted paths. The flattened preset supplies the encoding; the explicit override replaces only the delimiter before record construction.

workflow.to_lines() renders the definition without executing it. Task and context lines show IDs, titles, argument flows (field <- $variable), output flows (field -> $variable), literals, defaults, and unmapped fields. Context entry lines nest above child tasks and matching exit lines appear below them in reverse order.

workflow.to_cli(name, summary, preset=None) infers external variables by the same scope-aware traversal. A context output is visible to later contexts, the owning action, and all descendant tasks; an action output is visible globally after that action. Variables first read without a visible assignment become CLI parameters: they are optional with a variable default, applicable field defaults, or values from workflow lcl_mixin, explicit to_cli(preset=...), or the CLI entrance. Explicit presets override workflow host defaults, which override entrance bindings; variable defaults are below all of those sources. an explicit None is a default too. Reading before a later assignment still requires an input. Variables assigned before their first read, and write-only variables, are omitted and cannot be directly overridden. Context assignments do not hide inputs needed outside the owning subtree. Missing descriptions render as NO HELP MESSAGE PROVIDED. Help groups parameters by scope and shows optional defaults without loading configuration; see parameter help.

Every reached task and context task logs Workflow task started: [path] at INFO, with its title and (dryrun) when applicable. A real exception produces Workflow task error: [path] STATUS with its diagnostic and original traceback. Finalization still logs Workflow task completed: [path] STATUS; its severity follows the finalized status. Default lifecycle layouts retain timestamp, level, process and thread, without framework logger names or implementation locations. Custom logger.format layouts remain authoritative.

Log paths include dynamic parent calls. WorkflowExecutionContext.log_task_prefix and TaskContext.log_task_stack are keyword-only tuples of nonempty strings, defaulting to empty tuples. An empty task log stack falls back to task_id_stack. Execution populates each callback's full log stack, including context-task IDs. Local task_id_stack, __task_id_branch__, TaskID values and result dictionary keys retain their existing meaning. Separate calls and executions own their paths.

Completion records include action_ms for an executed action, or enter_ms and exit_ms for executed context callbacks, in milliseconds with three decimal places. A monotonic clock measures only callback invocation and awaiting: mapping, publication, declared child traversal and extra Frame cleanup are excluded. Context resource holding time is excluded. I/O and dynamic workflow calls awaited inside an action count toward that action. Unstarted callbacks have no duration.

An explicit stopping status produces a single-line WARN headed Workflow task stopped: [path] STATUS, with the reason and the remaining task traversal being skipped. WorkflowStatusStop still reaches context exit so the context can finish its work. It does not produce an exception tree or traceback unless a separate failure is attached. Handling and cleanup failures retain ERROR diagnostics and every member of a combined failure.

Verbose mode adds one aligned, sorted args mapping DEBUG record after materialization and one outputs mapping record after completion when an output exists. Runtime types remain visible for masked values. Sources and targets are labelled as LCL variables, defaults, literals, or unused outputs.

The finalized workflow tree is one severity-aware multi-line record headed Workflow completed: [command.path] STATUS: and includes the existing tree root. Exit codes are success 0, failure 1, error 2, and covered failure 3.

lclang.cli.scan_commands(module, name, description) imports a package and its submodules deterministically, collects public module-level Command objects, deduplicates re-exports by identity, and rejects distinct duplicate names.

Context scope migration and cleanup failures

Ancestor context resources now shadow same-named external values in descendant tasks. Outside sibling branches keep their own lookup environment. Ordinary action outputs still publish to the shared workflow Frame. A context publishes its acquisition-time resource, not a result only determined during exit.

Context exit remains reverse ordered after the whole owned subtree finishes. An ordinary business exception and cleanup exceptions are retained together in an exception group. Cancellation remains cancellation, with cleanup failures attached as its cause; cleanup continues through the remaining outer contexts and task Frames.

Business event logs

TaskContext.log_event(event, *, level=logging.INFO, record=None, fields=(), masked_fields=(), **values) emits an application event through the execution logger and its existing configuration. Messages start with [task.path] | message, including a bounded, newline-escaped message and dry-run status. The default event layout omits the logger name but keeps qualified_module_name.py:lineno:func_name for the application caller. Record metadata also supplies lclang_event, lclang_task_branch, and lclang_dryrun.

Each event is one log record: its title is followed by four-space-indented field: value lines with aligned colons. Empty events contain only the title. String newlines remain escaped within their value. Console and file handlers retain the multiline body; timestamps and severity belong to the logger formatter.

record is an optional dataclass instance. Only direct fields explicitly named in fields are selected, in declaration order of that selection; additional values follow in keyword order. Unknown fields, repeated selections, or names shared by the selection and keyword values raise LclValidationError. A non-dataclass record raises LclValidationError. No dictionary indexing, methods, or implicit whole record expansion occurs.

Disabled levels return before inspecting records or formatting values. Explicit masked_fields are checked before reading selected fields or calling repr; ordinary Python values do not acquire masks by object identity. Unmasked values use safe_repr with its default 200-character limit. Standard and lclang loggers attribute the call to the application's log_event line.

See errors and diagnostic codes for exception fields, specific codes, cause chains, and executable troubleshooting examples. Ordinary library validation uses LclValidationError; native callback and operation failures retain their original exception in __cause__.

The exception guide describes each failure family, grouped execution and cleanup failures, actual diagnostic output, and application handling with except* and native control signals.

Logging normal context unwinding

Use a field-free WARN event when a stopping task causes a context to skip its normal success actions. Put the reason and the actual skipped business action in the message. Calling logger.exception would report this expected exit as a new exception. A failed handler or cleanup operation still needs its error diagnostic.

import asyncio
import io
import logging
from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager
from dataclasses import dataclass
from datetime import date

from lclang.error import WorkflowStatusStop
from lclang.lang import define_frame
from lclang.logger import LclExceptionFormatter
from lclang.workflow import (
    ExecutionStatus, ExecutionStatusManager, TaskContext, WorkflowExecutionContext,
    define_context_task, define_task, define_workflow,
)


@dataclass
class Value:
    count: int


notifications: list[str] = []
cleanup: list[str] = []


@asynccontextmanager
async def run_scope(
    context: TaskContext, args: Value, status_mgr: ExecutionStatusManager,
) -> AsyncGenerator[Value]:
    try:
        yield args
    except WorkflowStatusStop as stopped:
        context.log_event(
            f"{stopped.message}; skipped: success notification", level=logging.WARNING,
        )
        raise
    else:
        notifications.append("success")
    finally:
        cleanup.append("closed")


async def stop(
    context: TaskContext, args: Value, status_mgr: ExecutionStatusManager,
) -> Value:
    status_mgr.update(ExecutionStatus.ERROR, "upstream task failed")
    return args


async def main() -> None:
    output = io.StringIO()
    logger = logging.Logger("context-warning", level=logging.WARNING)
    handler = logging.StreamHandler(output)
    handler.setFormatter(LclExceptionFormatter("%(levelname)s %(message)s"))
    logger.addHandler(handler)
    workflow = define_workflow("Run", define_task(
        "root", "Run", task_action=stop, args_mapping=Value(1),
        context_tasks=(define_context_task("scope", "Run scope", run_scope, Value(1)),),
    ))
    async with define_frame() as frame:
        result = await workflow.execute(WorkflowExecutionContext(
            False, date(2026, 10, 13), False, logger, frame,
        ))
    assert result.execution_status.status is ExecutionStatus.ERROR
    assert notifications == [] and cleanup == ["closed"]
    lines = output.getvalue().splitlines()
    assert "WARN [root.scope] | task 'root' ended with ERROR; skipped: success notification" in lines
    assert not any("Python traceback:" in line for line in lines)


asyncio.run(main())

The action returns its value but leaves an ERROR status. Traversal stops and the context logs why it omits the success notification. Its finally block still runs; re-raising preserves the failed workflow result.

Log path validation uses LCL533131 for wrong tuple/component types and LCL533211 for empty components. Empty tuples preserve the default path behavior.

import asyncio
import logging
from dataclasses import replace
from datetime import date

from lclang.error import LclValidationError
from lclang.lang import define_frame
from lclang.workflow import WorkflowExecutionContext


async def main() -> None:
    async with define_frame() as frame:
        context = WorkflowExecutionContext(
            False, date(2026, 10, 13), False, logging.Logger("paths"), frame,
        )
        for value, code in ((["batch"], "LCL533131"), (("",), "LCL533211")):
            try:
                replace(context, log_task_prefix=value)
            except LclValidationError as error:
                assert error.code == code
            else:
                raise AssertionError("invalid log prefix accepted")
        assert context.log_task_prefix == ()


asyncio.run(main())