# Workflow Orchestration Versus Simple Scheduled Scripts

**Decision evidence dossier — research snapshot: October 10, 2026**

## Decision answer and uncertainty

### Bottom line

For the bounded daily process studied here — **source export → immutable raw preservation → ingestion → dbt/warehouse transformation → business report** — I would **not adopt a workflow orchestrator merely because the process has multiple steps**.

For a small data team, the strongest default is:

> **First define the business-period and completion contract. Then implement the smallest system that can durably enforce it.**

For one daily pipeline, that can credibly be a **simple runner plus durable state, locking/idempotency, explicit period/version checks, reconciliation, and a managed/native scheduler**. Once those requirements expand into frequent cross-workflow dependencies, waits, backfills, parallel branches, per-task recovery, multiple operators, and incident handover, an orchestrator becomes progressively easier to justify.

The important distinction is therefore not “cron versus Airflow.” It is:

**stateless time-triggered execution**  
versus  
**stateful business-process execution**.

A cron expression or cloud scheduler answers approximately *when should something be invoked?* It does not by itself prove that the correct source period arrived, the correct version flowed through every stage, the report is current, or the business outcome completed. Google Cloud Scheduler, for example, explicitly provides at-least-once delivery and warns that a scheduled execution can be delivered more than once; Kubernetes similarly describes CronJob scheduling as approximate and says two Jobs or no Job can occur in some circumstances. Both therefore push idempotency into the workload design. citeturn18view1turn24view5

An orchestrator such as Airflow does provide materially more lifecycle machinery: task dependencies, per-task state, data intervals, retries, timeouts, catchup/backfill, and persistent workflow metadata. citeturn23view2turn24view1turn24view2turn24view3 But it does **not** remove the need for a business completion rule. Airflow's own documentation gives a revealing counterexample: a DAG can be marked successful even when an intermediate task failed if a successful leaf task uses an `all_done` trigger rule. citeturn23view1

That produces the decision rule I would use:

| Situation | Preferred starting point | Why |
|---|---|---|
| One bounded daily chain, one accountable technical owner, sequential dependencies, occasional replay | **Managed/native scheduler + simple stateful runner** | Lowest additional control-plane burden while meeting the requirements if state/reconciliation are implemented properly |
| Same chain, but existing platform already has reliable native job/dependency automation | **Native platform automation** | Reuse the control plane you already operate; avoid introducing a second scheduler |
| Frequent late inputs, waits, partial reruns, backfills, fan-out/fan-in, several workflows or operators, incident handover | **Workflow orchestrator** | Persistent workflow/task state and first-class recovery increasingly replace custom coordination code |
| Very small volume and low consequence of lateness | **Manual checklist/runbook may still win** | Automation has upkeep; eliminating or standardizing the problematic step can be more valuable than automating it |
| Strict/audited completion requirements but only one pipeline | **Either stateful runner or orchestrator, evaluated on recovery evidence rather than pipeline count** | Criticality can justify orchestration even before scale does |
| Source can reliably emit a readiness event with period/version | **Prefer event/readiness-driven start over blindly fixed-time start** | Removes an unnecessary race between “06:30 arrived” and “the correct data is ready” |

The last three rows are **decision heuristics**, not results of comparative trials. I found no controlled public study establishing a universal break-even number of jobs, tasks, incidents, or engineers at which an orchestrator becomes economically superior.

### The main uncertainty

The public evidence is asymmetric. Product documentation is strong on **what specific implementations are designed to do**, but weak on comparative operational outcomes. The empirical data-pipeline literature I found does not demonstrate that introducing an orchestrator reduces incident rates or total cost for a small team's bounded daily pipeline. One relevant empirical study developed a 41-factor data-pipeline-quality taxonomy and found significant problems around data types, cleaning, ingestion and integration — evidence that scheduling/orchestration is only one part of pipeline reliability, not a general cure for it. citeturn23view8

Accordingly, this dossier treats:

**Documented behavior** as evidence of platform capability;  
**architecture comparisons** as reasoned interpretation;  
**the worked process and economics** as explicitly synthetic; and  
**the supplied tests** as firsthand execution only of the synthetic fixture, **not** of Airflow, Kubernetes, Cloud Scheduler, dbt, a warehouse, or a production source.

## Bounded process and time-period contract

### The recognizable business process

The reference process is a **daily sales/operations report**. The source system produces an export for business date \(D\). The data team must preserve that source, ingest it, transform it in the warehouse using dbt, validate it, publish the resulting report, and declare the period complete by a business deadline.

The key change from a typical “run this every morning” design is that **the scheduled time is not the identity of the work**. The identity is the business period and source version.

| Element | Reference design |
|---|---|
| Business owner | Consumer of the daily sales/operations report |
| Technical owner | Data/analytics engineering owner or on-call |
| Trigger | Daily calendar schedule at **06:30 Europe/Prague**, plus manual/reconciliation triggers |
| Expected work identity | `pipeline_id + business_date + source_version` |
| Business date | \(D =\) scheduled local calendar date minus one calendar day |
| Source input | Export manifest declaring \(D\), source version, generation time and checksum plus payload |
| Raw preservation | Immutable/path-versioned copy keyed by \(D\), version and checksum |
| Ingestion | Loads only the preserved input for the declared \(D\)/version |
| Transformation | dbt/warehouse processing explicitly scoped to \(D\)/version |
| Validation | Technical tests plus explicit period/version/freshness assertions |
| Publication | Report/snapshot labelled with \(D\)/version |
| Business completion | **Correct \(D\)/version accepted by 08:30 Europe/Prague** |
| Failure | Anything that prevents that acceptance, even if every process happened to return exit code 0 |

dbt's `run_results.json` is useful evidence of a **completed dbt invocation** and records timing/status information for executed nodes, but that artifact describes dbt execution; it does not itself establish that the business report contains the required period/version or met a business deadline. citeturn18view2 That distinction is central here.

### Simplest manual baseline

The minimum reliable baseline is not “five cron entries.” It is an operator following one small, explicit checklist:

The operator identifies \(D\), verifies that the source manifest says \(D\), preserves the file and checksum, runs ingestion for \(D\)/version, runs the warehouse/dbt transformation for the same identity, checks validation results, verifies the published report identifies \(D\)/version, and records **ACCEPTED** in a shared ledger.

That baseline is intentionally boring. It separates the **process contract** from the implementation. If the team cannot unambiguously perform and verify the process manually, adding a workflow engine tends to automate ambiguity rather than resolve it.

Before automation I would therefore challenge the process in this order:

**Remove** unnecessary handoffs or duplicate exports → **standardize** one period/version identifier → **make source readiness observable** → **define completion and failure** → only then choose the automation technology.

### Inline time and period contract

The required time contract should be part of the system specification, not tribal knowledge.

| Contract field | Proposed value | Semantics |
|---|---|---|
| `business_calendar_zone` | `Europe/Prague` | IANA geographical zone, not merely `CET` or an offset |
| `business_date` | explicit local calendar date \(D\) | Independent run parameter; never reconstructed later from “now” |
| Reporting interval | \([D\ 00{:}00,\ D+1\ 00{:}00)\) in `Europe/Prague` | A **calendar-day period**, not “24 elapsed hours” |
| Normal scheduled invocation | 06:30 local on \(D+1\) | Local civil/calendar schedule |
| Source normal expectation | 06:15 local | Synthetic operating assumption |
| Late-source alert | 07:15 local | Synthetic policy; waiting may continue |
| Business completion deadline | 08:30 local | Period must be accepted, not merely started |
| Hard automated-recovery cutoff | 12:00 local | Synthetic policy; beyond it require explicit operator decision |
| Retries | elapsed duration, e.g. 5 min → 10 min → 20 min | Measure in UTC/monotonic elapsed time; do not use civil-clock arithmetic |
| Backfill | explicit set/range of business dates | Never implement as repeated `now - 24h` |
| Weekend policy | daily, including weekends | Change only through an explicit business-calendar rule |
| Holiday policy | daily by default | If business-day-only, reference a versioned business calendar |
| Source revisions | new `source_version`, never silently overwrite accepted input | Reprocessing must be explicit |
| Timestamp storage | source/schedule/acceptance UTC instants **plus** relevant IANA zone and business date | Separates an instant from business-calendar meaning |

This distinction matters because calendar schedules and elapsed intervals are different abstractions. Airflow explicitly distinguishes delta-based timetables, which run after an elapsed delta and need not align to hour/day boundaries, from cron/data-interval schedules aligned to cron trigger points. citeturn25view0turn25view1 Google Cloud Scheduler likewise says its timezone-based scheduler runs on **wall-clock time**, warns that daylight-saving transitions can make executions anomalous, and recommends UTC when a very specific cadence is required. citeturn24view0

Consequently:

> “Every day at 06:30 Europe/Prague” and “every 24 hours” are not the same contract.

The IANA timezone database is built around geographical timezone identifiers and civil-time rules; it also explicitly warns that future timezone predictions can become wrong after governments change rules. citeturn23view6turn25view2 The practical implication is to store `Europe/Prague` or `America/New_York`, not bake today's UTC offset permanently into future business schedules, and to treat timezone-rule updates as a freshness trigger.

Czech and US deployments do not require different architecture advice on this evidence. The material difference is simply that a US business process might use a zone such as `America/New_York` while the Czech process uses `Europe/Prague`; their DST transitions occur under their respective civil-time rules. The same period/version contract is appropriate to both. IANA also cautions that timezone abbreviations can be ambiguous in practice, further supporting geographical zone identifiers for business rules. citeturn24view8

One should **not infer a universal DST policy from the word “cron.”** Different systems define edge cases differently. RFC 5545, for example, has specific rules for interpreting repeated and nonexistent local calendar times, while Cloud Scheduler documents its own wall-clock behavior and potential DST anomalies. Those are implementation-specific semantics, not a universal cron guarantee. citeturn23view7turn24view0

A particularly simple risk reduction is to avoid scheduling this daily job inside the likely clock-transition window at all. The synthetic design uses 06:30 local, while still testing DST-crossing **periods and backfills**, because the reporting day itself can contain 23 or 25 elapsed hours.

## Same requirements across the alternatives

The fair comparison is to give every approach **the same completion requirements**. A deliberately weak shell script should not be compared with a fully engineered orchestrator and then declared inherently unreliable. Conversely, once the script gains durable state, locking, retries, reconciliation and history, that engineering effort must count against its apparent simplicity.

### Capability comparison

| Requirement | Host-local cron-style scheduled script | Managed/native scheduler + runner | Representative orchestrator: Airflow 3.3.2 |
|---|---|---|---|
| Invoke at a schedule | Scheduler does this | First-class scheduler capability | First-class timetable/scheduler capability |
| Explicit timezone | Implementation-specific; must inspect actual scheduler | Cloud Scheduler supports timezone selection and defaults to UTC | Cron timetables accept timezone; other timetable types support elapsed-delta semantics |
| Dependency graph | Code/runbook | Usually application code unless native product adds workflow semantics | Explicit DAG dependencies |
| Same-period correlation | Must implement | Must implement | Data intervals help correlate task instances, but source version remains application logic |
| Durable business state | Custom DB/ledger | Custom DB/ledger | Workflow/task state stored in metadata DB; business acceptance still custom |
| Duplicate handling | Custom idempotency | Required: Cloud Scheduler can deliver a scheduled invocation more than once | Side-effect idempotency still required even with task state/retries |
| Overlap control | Lock/lease | Lock/lease plus scheduler-specific facilities | Concurrency/task controls plus application-level side-effect protection |
| Step retry | Shell/application logic | Application logic plus trigger-level retry where supported | Task retry is first class |
| Wait for late data | Polling/application state | Polling/application state | Sensors/deferred patterns and timeouts |
| Timeout | Application logic | Scheduler/request timeout plus application business timeout | Task `execution_timeout`; sensors have overall timeout semantics |
| Partial recovery | State-machine logic | State-machine logic | Clear/re-run selected tasks; task history/state available |
| Backfill | Loop over explicit dates | Explicit replay loop | First-class date-range backfill |
| Missed-run reconciliation | Startup/period scanner | Period scanner recommended | Catchup or backfill, but catchup is not on by default |
| Observable whole-process state | Custom dashboard/SQL/logs | Custom ledger + provider logs | UI/API plus DAG/task states, but business acceptance needs explicit design |
| Control-plane burden | OS/host + runtime + state store | Application/state store; scheduler operated by provider | Scheduler, DAG processor, API server, DAG bundle and metadata DB at minimum |
| Platform upgrades | Runtime/OS/dependencies | Application dependencies; provider manages scheduler service | Airflow upgrades can include metadata DB migrations |
| Handover to another operator | Depends heavily on runbook/ledger quality | Better if state ledger is explicit | Usually easier because task/run history and workflow UI are first-class |
| Exit/migration | Simple executable/state schema potentially portable | Scheduler-specific trigger configuration plus portable runner | DAG/platform semantics can create greater migration surface |

Cloud Scheduler is a useful representative of the middle category. It sends a scheduled unit of work to a target, provides configurable retry behavior, explicitly promises at-least-once delivery, and exposes the original schedule time for deduplication. citeturn18view0turn18view1 That is significantly more reliable as a **trigger service** than “hope one laptop is awake,” while still leaving cross-step business state with the application.

Kubernetes CronJob illustrates why “cron-format scheduler” must not be treated as a universal reliability specification. Kubernetes supports an explicit `.spec.timeZone`, delayed-start deadlines and concurrency policies, but its documentation calls Job creation approximate, says duplication or absence can occur, and recommends idempotent Jobs. citeturn24view6turn24view7 It can also stop catching up after more than 100 missed schedules in the documented circumstances. citeturn24view5

Airflow provides a different abstraction. Tasks in one DAG run share a data interval, upstream success normally gates downstream execution, and task instances carry explicit lifecycle states such as scheduled, queued, running, success, failed, upstream-failed and up-for-retry. citeturn23view2 Its scheduler monitors dependencies and triggers eligible task instances. citeturn25view4 For a pipeline where operators frequently need to answer “which step for which period failed?”, this is genuine leverage rather than cosmetic UI.

Airflow's catchup/backfill machinery is also materially different from a raw timer. Catchup can generate a run for each missed data interval, while backfill can explicitly process a historical date range with reprocessing and concurrency options. But **catchup defaults to false**, and Airflow cautions that code which uses “now” rather than the interval may not be written safely for catchup. citeturn24view1turn24view2 An orchestrator therefore makes correct period-oriented execution easier; it does not make badly periodized application code correct automatically.

### The complexity the orchestrator introduces

For the same tiny pipeline, Airflow is not “just cron with arrows.” A minimal Airflow 3.3.2 installation has a scheduler/executor, DAG processor, DAG bundle, API server and metadata database. citeturn23view3 Its production documentation says the default SQLite setup is for testing and warns that production deployment needs an external supported database such as PostgreSQL or MySQL. citeturn23view5

Upgrades can require schema migrations, and Airflow recommends backing up the metadata database because a failed migration can leave a half-migrated state. citeturn23view4 Managed Airflow products can shift some of that work to a provider, but they do not remove DAG testing, runtime compatibility, workflow ownership, source/warehouse permissions or incident responsibility.

This is the strongest counterweight to an automatic “use an orchestrator” recommendation: the orchestration control plane is **another production system whose health and lifecycle become part of the reporting pipeline**.

### A key design insight: a good script starts resembling a small orchestrator

To meet the stated requirements, the simple implementation needs at least:

```text
durable period ledger
+ step states
+ idempotency keys
+ lock/lease
+ exact source-version checks
+ retries classified by error
+ deadline/timeout handling
+ logs/metrics
+ reconciliation loop
+ backfill command
+ operator runbook
```

That does not mean “therefore Airflow.” It means the decision should be made on **which machinery the team prefers to own**.

For one stable sequential workflow, a few hundred lines of explicit state-machine code plus a relational table can be easier to reason about than an orchestration platform. For dozens of workflows, repeatedly rebuilding those same primitives becomes questionable. The evidence does not supply a universal numerical boundary between those cases.

## Failure, recovery and acceptance

### Dependency state, timeout and recovery contract

The following state model is proposed architecture, not observed production behavior. Times are synthetic policy inputs.

| State | Required correlation | Timeout / trigger | Recovery or next transition |
|---|---|---|---|
| `DUE` | `pipeline_id, D` | 06:30 scheduled trigger or reconciliation finds missing period | Look for source manifest |
| `WAITING_SOURCE` | exact \(D\), expected source | Alert 07:15; keep checking until hard cutoff 12:00 | On source arrival, checksum and preserve raw; no transformation on substitute/stale data |
| `RAW_PRESERVED` | \(D\), `source_version`, checksum | No short business timeout; must remain recoverable | Resume ingestion from preserved object |
| `INGESTING` | same \(D\)/version; lease + attempt ID | Synthetic 15-minute lease | Retry safely after lease expiry; do not duplicate accepted side effects |
| `INGESTED` | same \(D\)/version recorded in staging metadata | Downstream immediately eligible | Transform only declared period/version |
| `TRANSFORMING` | \(D\)/version passed explicitly | Bounded task timeout | Retry transient failure or move to operator-required state |
| `TRANSFORMED` | transformation output declares \(D\)/version | Immediate validation | Run data/period/freshness tests |
| `VALIDATED` | required tests passed | Immediate publication | Publish report version |
| `REPORT_PUBLISHED` | report metadata = \(D\)/version | Acceptance deadline 08:30 | Final acceptance gate |
| `ACCEPTED` | all contract predicates true | Terminal | No implicit replacement by later source revision |
| `RETRYABLE_FAILED` | attempt/error class retained | Exponential elapsed-time backoff | Resume failed step, not entire workflow unless required |
| `BLOCKED_AUTH` | failing resource and credential owner identified | Do not blindly retry forever | Rotate/restore permission, then replay from durable state |
| `TIMED_OUT` | state/attempt retained | Business or technical timeout exceeded | Reconcile external side effects before retry |
| `RECOVERY_NEEDED` | evidence of wrong/stale period, inconsistent output, or uncertain side effect | Immediate operator attention | Determine authoritative state, compensate or replay |
| `SUPERSEDED` | old and new source versions both retained | Explicit revision event | New controlled run; never silently relabel old output |

Airflow can represent substantial portions of this mechanically: dependencies, task-instance states, retries, execution timeouts and waiting sensors are documented capabilities. citeturn23view2turn24view3 But `source_version`, business acceptance, source-revision policy and reconciliation with external side effects remain domain concerns.

### Worked failure: source arrives after the fixed schedule

At **06:30 Europe/Prague**, the scheduler triggers the run for business date `2026-10-09`. The source export has not yet arrived.

A weak implementation executes:

```text
SELECT latest available sales
→ latest available happens to be 2026-10-08
→ dbt succeeds
→ report publish succeeds
→ cron exits 0
```

Technically, the automation “worked.” Business-wise, it failed.

The reference design instead executes:

```text
expected_business_date = 2026-10-09
source_manifest(2026-10-09) = absent

state = WAITING_SOURCE
report = not publishable
business_completion = false
```

At 06:52 the manifest for `2026-10-09 / v1` appears:

```text
preserve raw payload + SHA-256
→ ingest 2026-10-09/v1
→ transform 2026-10-09/v1
→ validate period/version
→ publish report 2026-10-09/v1
→ write ACCEPTED
```

The important control is **not the retry**. It is refusing to replace the missing expected period with whatever data happens to exist.

### Worked failure: downstream succeeds on yesterday's data

Suppose ingestion correctly processes \(D\), but a transformation accidentally queries a warehouse table still containing \(D-1\). The SQL succeeds and dbt reports the node as successful.

The completion predicate must therefore contain something like:

\[
\text{accepted}(D,v)=
S \land R \land I \land T \land V \land P \land M
\]

where:

- \(S\) = source manifest identifies \(D,v\)
- \(R\) = raw source \(D,v\) is preserved and checksummed
- \(I\) = ingestion finished for \(D,v\)
- \(T\) = transformation consumed the declared \(D,v\)
- \(V\) = required validations passed
- \(P\) = report for \(D,v\) was published
- \(M\) = acceptance occurred within the business deadline

A task/process exit code is only one input to those predicates.

This also challenges the idea that an orchestrator's green workflow indicator is automatically the business truth. Airflow explicitly derives DAG-run status from terminal leaf states and warns that certain trigger-rule designs can yield DAG `success` despite a failure in the middle. citeturn23view1 The reference DAG should therefore end in an **explicit acceptance task** whose own inputs verify the business predicates, rather than treating generic DAG completion as sufficient evidence.

### Worked failure: scheduler/host interruption

Assume a host-local scheduler is unavailable when 06:30 passes.

The robust script design does **not** depend on assumptions about whether that particular cron implementation catches up. At process startup, and periodically thereafter, it performs reconciliation:

```text
expected periods through current business deadline
MINUS
periods whose durable status = ACCEPTED
=
periods requiring inspection/recovery
```

If `2026-10-09` is missing, the runner creates/reclaims the durable work item and continues from its last confirmed state.

Airflow has native catchup/backfill concepts, but catchup is disabled by default, so the operator must make the intended recovery semantics explicit. citeturn24view1turn24view2 Kubernetes has its own delayed-start and missed-schedule rules. citeturn24view7turn24view5 This variation reinforces the broader point: **cron syntax is not a recovery contract**.

### Worked failure: replay crosses daylight-saving time

For `Europe/Prague`, define the business period as local-calendar midnight to next local-calendar midnight. The synthetic fixture evaluates:

```text
2026-03-28 -> 24 elapsed hours
2026-03-29 -> 23 elapsed hours
2026-03-30 -> 24 elapsed hours
```

and the autumn transition day as:

```text
2026-10-25 -> 25 elapsed hours
```

A backfill covering these days therefore creates **one business run per calendar date**, rather than chopping history into fixed 24-hour UTC windows.

The fixture also classifies `Europe/Prague 2026-03-29 02:30` as nonexistent and `Europe/Prague 2026-10-25 02:30` as ambiguous. A corresponding US test classifies `America/New_York 2026-03-08 02:30` as nonexistent.

These are **actually executed synthetic tests**, not provider tests. They ran under Python 3.13.5 using the runtime's IANA tzdb 2025b. Eleven tests passed. They do not demonstrate how any external scheduler behaves.

The portable asset contains the code, fixtures, proposed SQL state schema, test cases and captured test run:

**[Download the synthetic period/recovery test assets](sandbox:/mnt/data/workflow_orchestration_dossier_assets.zip)**

The package explicitly labels unperformed cases such as real credential expiry, warehouse outages, scheduler control-plane outages and source-version revisions.

### Acceptance cases

| Case | Expected invariant | Script/native implementation | Orchestrator implementation |
|---|---|---|---|
| Source arrives late | Do not use earlier period | Reconciliation/waiting state | Sensor/wait task plus exact-period check |
| Duplicate trigger | One logical side effect | Unique idempotency key / durable state | Same; task state alone does not make external API writes idempotent |
| Previous attempt died after raw save | Do not download/recreate input unnecessarily | Resume from `RAW_PRESERVED` | Retry/clear downstream task after verifying prior side effect |
| Transformation uses stale date | Reject apparently successful run | Acceptance predicate fails | Explicit terminal validator fails |
| Permission expires | Do not retry endlessly | Classify as operator-required | Configure retry policy/terminal failure; operator repairs credential |
| Host/scheduler unavailable | Recover missing business period | Reconciliation scanner | Catchup/backfill/retrigger under explicit policy |
| Backfill crosses DST | One run per declared business date | Iterate dates, derive UTC boundaries separately | Use data intervals/backfill designed around explicit periods |
| Source releases revised `v2` after `v1` accepted | Never mutate historical meaning silently | Create controlled reprocess/supersession | New run/reprocessing policy; source version remains domain metadata |
| Report task times out after remote publish | Determine whether publication actually happened before retry | Query remote/report state, then reconcile | Same requirement; orchestrator cannot infer an external side effect it did not observe |

That last row is one of the most important lifecycle cases: **timeout does not imply “nothing happened.”** Recovery must first reconcile the external system before replaying a non-idempotent action.

An optional AI system may help classify logs, summarize an incident, or propose a recovery action. It should **not be the authority deciding that period \(D\) is complete** unless the completion condition itself genuinely requires probabilistic judgment. For this pipeline, period/version/deadline acceptance is deterministic and should remain so.

## Practical decision, pilot economics and exit criteria

### What I would approve for the small-team case

I would first approve a bounded pilot of **managed/native scheduling plus a stateful runner**, not a new self-managed orchestrator, provided the team can demonstrate all of the following:

| Approval evidence | Minimum bar |
|---|---|
| Period contract | Explicit `business_date`, timezone, source version and deadline |
| Durable state | Survives process/host restart |
| Idempotency | Duplicate invocation demonstrably causes no harmful duplicate effect |
| Lock/overlap control | Two invocations for the same \(D\)/version cannot corrupt results |
| Late source | Runner waits/fails visibly; cannot silently use stale period |
| Partial recovery | Restart resumes from known durable step |
| Backfill | Explicit `--business-date` or bounded date-range replay |
| Reconciliation | Can detect an expected-but-unaccepted period independently of the scheduler |
| Permissions | Service identities and credential owners documented |
| Observability | Owner can answer “what period/version is waiting/failed/accepted?” without SSH archaeology |
| Handover | Another engineer can recover the pipeline using the runbook |
| Exit path | Runner logic not tightly coupled to the scheduler so it can later become an orchestrator task |

This is not an argument for “DIY forever.” It is deliberately an architecture that can migrate cleanly:

```text
today:
managed scheduler
    ↓
stateful runner(D)

later:
Airflow/other orchestrator task
    ↓
same runner(D)
```

The scheduler should supply the **trigger**, while the executable owns the **business-period logic**. That separation lowers both migration and rollback risk.

### When I would skip the pilot and choose orchestration now

I would lean directly toward an orchestrator where several of these are already true:

The pipeline routinely waits on multiple upstreams; independent branches converge; source arrival is variable; operators frequently replay individual stages; historical backfills are normal rather than exceptional; several people share incidents; task-level state needs to survive handovers; there are many pipelines implementing the same home-grown retry/state framework; or the organization already has an orchestrator and competent owners, making its marginal operational burden low.

Airflow directly supports dependencies, stateful task instances, timeouts and date-range backfills, so these are not speculative benefits of the orchestration category, although competing orchestrators must be assessed separately. citeturn23view2turn24view3turn24view2

Conversely, I would defer orchestration when the dominant problem is still that nobody can define which source file constitutes Tuesday's report. An orchestrator cannot compensate for an undefined period contract.

### Transparent synthetic economics

No public source located provides defensible universal ROI inputs for this exact decision, so the useful model is a **break-even equation**, not a fake business case.

Let:

\[
B = \text{extra orchestrator implementation hours}
\]

\[
A = \text{amortization period in years}
\]

\[
O = \text{extra annual orchestration-platform operating hours}
\]

\[
C = \text{extra annual cash cost converted to equivalent team hours}
\]

\[
S = \text{average engineer hours saved per recovery/backfill event}
\]

Then the annual break-even event count is:

\[
N_{\text{break-even}} =
\frac{B/A + O + C}{S}
\]

A deliberately synthetic illustration:

- incremental orchestration implementation effort \(B=40\) hours
- amortization \(A=2\) years
- additional platform upkeep \(O=36\) hours/year
- cash subscription/infrastructure delta omitted, \(C=0\)
- operator time saved \(S=1.5\) hours per incident/backfill

Then:

\[
N =
\frac{40/2+36}{1.5}
=
37.3\text{ events/year}
\]

or roughly **3.1 recovery/backfill events per month**.

That is **not a forecast or benchmark**. Every numerical input is hypothetical. Its purpose is to make the approval question testable: instrument the pilot and replace \(B,O,C,S\) with the team's observed values.

It also reveals an often-missed trade-off. If the company already operates Airflow well, \(B\) and \(O\) can be close to marginal rather than greenfield costs, strongly favoring orchestration. If nobody currently owns Airflow, those terms can dominate.

### Rollback and exit criteria

The simple-runner pilot should migrate to an orchestrator when its custom coordination layer starts becoming a product of its own: numerous state transitions, dependency polling, increasingly sophisticated rerun UI, multiple pipelines duplicating the same code, or sustained operator effort that can credibly be removed by the platform.

The orchestrator should itself be reconsidered when control-plane operations, upgrades and incidents exceed the coordination burden it replaces. Airflow's required components and metadata database, plus documented migration requirements, make this a real rather than theoretical cost. citeturn23view3turn23view4turn23view5

The rollback-friendly design is to keep ingestion/transformation/publication logic callable using explicit parameters:

```text
pipeline run --business-date 2026-10-09 --source-version v1
```

rather than embedding all business logic inside scheduler/orchestrator callbacks.

## Claim-to-source ledger, contradictions and unresolved gaps

### Primary-source ledger

| Claim / decision use | Primary source and exact scope | Direct URL / DOI | Source date/version | Inspected | Evidence type | Assessment / editorial treatment |
|---|---|---|---|---|---|---|
| Cloud Scheduler is at-least-once and can execute one scheduled occurrence multiple times; target should tolerate duplicates | **About Cloud Scheduler**, “at least once” and request deduplication sections | `https://cloud.google.com/scheduler/docs/overview` | Last updated 2026-10-07 | 2026-10-10 | Provider documentation | **Supported. Keep.** Applies to Google Cloud Scheduler specifically, not all schedulers. citeturn18view1 |
| Cloud Scheduler retries unacknowledged failures with configurable exponential backoff; retries can overlap/skip next scheduled execution under documented settings | **Retry jobs**, retry settings | `https://cloud.google.com/scheduler/docs/configuring/retry-jobs` | Last updated 2026-10-07 | 2026-10-10 | Provider documentation | **Supported. Qualify by provider/configuration.** citeturn18view0 |
| Local wall-clock scheduling and UTC/elapsed cadence are different; Cloud Scheduler warns of DST anomalies | **Cron job format and time zone**, “Time zone” | `https://cloud.google.com/scheduler/docs/configuring/cron-job-schedules` | Last updated 2026-10-07 | 2026-10-10 | Provider documentation | **Supported. Keep with provider qualification.** Strong evidence for article claim about defining which business time matters. citeturn24view0 |
| Kubernetes CronJob execution is approximate; duplicate or absent Jobs are possible; Jobs should be idempotent | **CronJob**, “Job creation” | `https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/` | Current Kubernetes docs; feature page states CronJob stable since v1.21 | 2026-10-10 | Provider/project documentation | **Supported. Keep.** Never generalize as all cron implementations. citeturn24view4turn24view5 |
| Kubernetes supports named timezone, delayed-start deadline and overlap policies | **CronJob**, “Time zones”, “Deadline for delayed Job start”, “Concurrency policy” | `https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/` | Timezone feature stable since v1.27 | 2026-10-10 | Provider/project documentation | **Supported. Keep as representative scheduler behavior.** citeturn24view6turn24view7 |
| Airflow represents dependencies and persistent task-instance lifecycle states | **Tasks**, “Relationships” and “Task Instances” | `https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html` | Airflow 3.3.2 | 2026-10-10 | Project documentation | **Supported. Keep.** Representative orchestrator capability, not universal. citeturn23view2 |
| Airflow provides data intervals; DAG success is not necessarily equivalent to every intermediate task succeeding | **Dag Runs**, “Dag Run Status” and “Data Interval” | `https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dag-run.html` | Airflow 3.3.2 | 2026-10-10 | Project documentation | **Supported. Important qualification.** Justifies explicit business acceptance state. citeturn23view1 |
| Airflow catchup can instantiate missed intervals; default catchup is false; explicit backfill supports historical ranges | **Dag Runs**, “Catchup” and “Backfill” | `https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dag-run.html` | Airflow 3.3.2 | 2026-10-10 | Project documentation | **Supported. Keep.** Corrects simplistic “orchestrator automatically catches everything up” claim. citeturn24view1turn24view2 |
| Airflow supports task execution timeouts and sensor-wide timeout behavior | **Tasks**, “Timeouts” | `https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html` | Airflow 3.3.2 | 2026-10-10 | Project documentation | **Supported. Keep.** Application-level business deadlines remain separate. citeturn24view3 |
| Cron/calendar and elapsed-delta timetables are separate schedule models | **Timetables**, `DeltaTriggerTimetable`, `CronTriggerTimetable`, `DeltaDataIntervalTimetable`, `CronDataIntervalTimetable` | `https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/timetable.html` | Airflow 3.3.2 | 2026-10-10 | Project documentation | **Supported. Keep.** Good concrete demonstration that “daily” needs semantic definition. citeturn25view0turn25view1 |
| Airflow itself has a nontrivial control plane | **Architecture Overview**, required components | `https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/overview.html` | Airflow 3.3.2 | 2026-10-10 | Project documentation | **Supported. Keep as counterevidence to automatic orchestration adoption.** citeturn23view3 |
| Production Airflow needs more than default SQLite; upgrades can involve DB migration and backup/recovery considerations | **Production Deployment** and **Upgrading Airflow** | `https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/production-deployment.html`; `https://airflow.apache.org/docs/apache-airflow/stable/installation/upgrading.html` | Airflow 3.3.2 | 2026-10-10 | Project documentation | **Supported. Keep.** Evidence of lifecycle upkeep, not proof that Airflow is uneconomic. citeturn23view4turn23view5 |
| IANA geographical zones model civil-time rules; future conversions can become wrong after government rule changes | **Theory and pragmatics of the tz code and data**, “Scope” and “Accuracy” | `https://data.iana.org/time-zones/tzdb/theory.html` | Current tzdb theory page | 2026-10-10 | Standards/data-project documentation | **Supported. Keep.** Drives named-zone and freshness policy. citeturn23view6turn25view2 |
| Repeated/nonexistent local-time treatment is implementation/standard specific rather than inherent to “cron” | **RFC 5545**, DATE-TIME with TZID | `https://www.rfc-editor.org/rfc/rfc5545.html` | September 2009 | 2026-10-10 | Internet standard | **Supported as counterexample. Qualify.** RFC 5545 describes iCalendar semantics, not cron scheduler semantics. citeturn23view7 |
| dbt `run_results.json` records status/timing for executed nodes of completed invocations | **Run results JSON file** | `https://docs.getdbt.com/reference/artifacts/run-results-json` | Last updated 2026-09-13 | 2026-10-10 | Provider/project documentation | **Supported. Keep but rewrite business claim:** dbt success is evidence for transformation execution, not full report acceptance. citeturn18view2 |
| Pipeline-quality problems extend beyond orchestration/scheduling | Foidl et al., abstract: taxonomy plus GitHub/Stack Overflow mining; incorrect data types, cleaning, ingestion/integration findings | `https://arxiv.org/abs/2309.07067`; journal DOI `10.1016/j.jss.2023.111855` | arXiv v1 2023-09-13; Journal of Systems & Software publication associated with DOI | 2026-10-10 | Empirical research / primary preprint | **Supported but limited. Keep as counterevidence.** It is not a cron-vs-orchestrator experiment and cannot establish orchestration effects. citeturn23view8 |

### Requested draft-claim audit

| Draft target | Status | Evidence judgment | Recommended treatment |
|---|---|---|---|
| **“Automatizace běží v jiném čase, než čekáte. Který čas má pro firmu platit?”** | **Supported, but too vague as written** | Cloud Scheduler directly documents wall-clock timezone scheduling and DST anomalies; Airflow explicitly distinguishes delta and cron/calendar semantics; IANA makes future civil-time rules mutable. citeturn24view0turn25view0turn25view2 | **Rewrite:** “A schedule is only unambiguous after you decide whether the business means 06:30 Europe/Prague, every 24 elapsed hours, or a UTC instant — and define what happens around DST and business-calendar exceptions.” |
| **“Jedna automatizace čeká na druhou. Kdo hlídá, že celý postup doběhl?”** | **Supported, with an important qualification** | Airflow can explicitly model dependencies and task state, but even orchestrator-level DAG success can misrepresent business completion under certain trigger-rule structures. citeturn23view2turn23view1 | **Rewrite:** “When one step waits for another, correlate both to the same business period/version and define a terminal acceptance condition. A green task — or even a green workflow under the wrong rule — is not automatically proof the business process completed.” |

### Contradictions and counterevidence

There is no contradiction in the sources about the fundamental distinction between schedule time and workflow state. There **is** a significant contradiction to simplistic recommendations.

**“Cron is unreliable; orchestrators are reliable.”** Unsupported as a categorical claim. Kubernetes CronJob and Cloud Scheduler document duplicate/retry/missed-execution nuances, but an engineered simple runner can compensate through idempotency and reconciliation. Meanwhile Airflow has its own scheduler, metadata database, deployment requirements and upgrade failure modes. citeturn24view5turn18view1turn23view3turn23view4

**“Managed scheduler means exactly once.”** Contradicted for Google Cloud Scheduler, which explicitly says at-least-once delivery and possible multiple deliveries. citeturn18view1

**“An orchestrator means missed dates are automatically backfilled.”** Contradicted for default Airflow behavior: scheduler catchup is false by default, though explicit catchup and backfill are available. citeturn24view1turn24view2

**“Workflow success means the intended report is correct.”** Contradicted as a general inference. Airflow warns of DAG-success edge cases based on leaf-node trigger rules; dbt's run artifact reports execution status, not the independently defined business acceptance contract. citeturn23view1turn18view2

**“DST can be solved once by converting the schedule to UTC.”** Conditional at best. UTC removes local repeated/skipped clock times for the scheduler, but if the business requirement remains “06:30 Prague time,” its UTC instant can change as civil-time offsets change. IANA also warns future rules may be changed by governments. citeturn24view0turn25view2

**“An orchestrator addresses most data-pipeline reliability failures.”** Not established. The empirical pipeline-quality research found substantial issues in data types, cleaning, ingestion and integration; it did not demonstrate causal benefits from orchestration. citeturn23view8

### Unresolved evidence gaps and inaccessible sources

I did **not** find an original controlled study comparing, for this exact bounded workload, the incident rate, mean recovery time, total engineering effort or cost of a durable scheduled script against an orchestrator. Claims about universal ROI or a magic “number of jobs” threshold should therefore be removed rather than filled with anecdote.

The Journal of Systems & Software publisher page for Foidl et al. was inaccessible during research; the authors' accessible arXiv primary preprint was used instead. The publisher DOI is retained in the ledger so the substitution is explicit rather than silent.

The Open Group's current POSIX `crontab` page returned access restrictions during inspection. I therefore did **not** use it to claim specific Unix cron catch-up, DST or missed-run guarantees.

The freedesktop.org systemd timer manual pages likewise returned access restrictions during inspection. Consequently, systemd-specific persistence/catch-up semantics were not used as evidence. The dossier deliberately says a host-local design must define its own recovery contract rather than assuming a particular local timer's behavior.

No real Google Cloud Scheduler, Kubernetes, Airflow, dbt, warehouse or source-system experiment was executed. Provider behavior in this dossier is **documented behavior**, not firsthand validation.

The synthetic tests used the container's tzdb **2025b**, not necessarily the most recent tzdb release available on October 10, 2026. The fixture therefore demonstrates the mechanics of the proposed contract under that installed rule set; it should not be represented as an independent validation of the latest 2026 timezone database.

## Candidate article angles, reusable assets and freshness triggers

The strongest reusable article angle is not “Airflow versus cron.” It is:

**“Your scheduler is not your business process.”**  
A timer tells code to start. The business cares whether the right period/version was accepted by the deadline. The worked stale-data failure gives this angle a concrete, non-vendor-specific backbone.

A second distinct angle is:

**“Every day is not always 24 hours.”**  
Use the local-calendar versus elapsed-time distinction to explain why `06:30 Europe/Prague`, `every 24 hours`, and `06:30 UTC` are three different requirements. Cloud Scheduler's documented wall-clock/DST behavior, Airflow's delta-versus-cron timetables and IANA's civil-time model provide unusually clean primary evidence. citeturn24view0turn25view0turn25view2

A third is:

**“A simple script stops being simple when reliability matters — but that still doesn't automatically mean Airflow.”**  
Walk through the exact additions: durable state, lock, idempotency, retry classification, reconciliation and backfill. The decision becomes whether those few primitives are cheaper to own directly or whether the team is recreating an orchestration platform.

A fourth is:

**“Green does not mean done.”**  
Use the stale-period example plus Airflow's own DAG-run status caveat and dbt's execution artifact to show the difference between technical completion and business acceptance. citeturn23view1turn18view2

A fifth is:

**“Fix the workflow before automating it.”**  
The sequence would be remove → standardize → identify period/version → define exceptions → automate. This is an editorial/process-design argument rather than an empirical claim and should be labeled accordingly.

A sixth, aimed at decision-makers, is:

**“When should a small data team actually buy orchestration?”**  
Use a capability threshold rather than a job-count threshold: frequent waits, backfills, branching, shared operational ownership and incident handover are the signals. The dossier deliberately does **not** invent a numerical industry benchmark where none was found.

### Reusable assets

The generated package is portable and contains:

`period_contract.py` — calendar-day boundaries and DST classification;  
`synthetic_pipeline.py` — synthetic period/version state machine and idempotency model;  
`state_schema.sql` — proposed durable state and attempt tables;  
`fixtures.json` — the complete worked time/period contract;  
`acceptance_cases.csv` — proposed cases with executed/unexecuted labels;  
`test_dossier.py` — eleven actually executed standard-library tests;  
`TEST_RUN.txt` — runtime, tzdb version and captured test result; and  
`README.md` — scope and explicit non-claims.

**[Download the complete portable asset](sandbox:/mnt/data/workflow_orchestration_dossier_assets.zip)**

The test result was **11/11 passing** under Python 3.13.5 and tzdb 2025b. This is firsthand execution only of the synthetic model. It is not a benchmark, production result, vendor test or customer outcome.

### Freshness triggers

This dossier should be rechecked when any of the following changes:

| Trigger | What to revalidate |
|---|---|
| IANA tzdb release or Czech/US time-law change | Future local schedules, DST acceptance cases and UTC projections |
| Cloud Scheduler behavior/docs change | Delivery guarantee, retry limits, timezone/DST semantics and deduplication identifiers |
| Kubernetes upgrade | CronJob timezone, missed-run, concurrency and scheduling guarantees |
| Airflow upgrade | Task/DAG states, timetable semantics, catchup/backfill defaults, architecture and migration requirements |
| dbt major/version-track change | Artifacts, testing/freshness semantics and retry behavior |
| Source SLA/versioning changes | Waiting timeout, source-readiness contract and deadline feasibility |
| Business deadline/calendar changes | Business date, weekend/holiday treatment and acceptance deadline |
| Pipeline gets a second upstream or fan-out branch | Reconsider orchestrator leverage |
| Manual recovery/backfill frequency rises materially | Replace the synthetic break-even variables with observed incident data |
| Another team assumes ownership | Test whether the custom runner's state/runbook remain operable without original author knowledge |
| Existing company orchestration platform becomes available | Recalculate marginal orchestration cost; greenfield Airflow burden may cease to be relevant |

The durable recommendation is therefore narrower than “use cron” or “use orchestration”: **make business time, period identity, completion and recovery explicit; then choose the least complex control plane that demonstrably enforces that contract.** For the single bounded daily workflow here, a managed trigger plus a durable, reconciling runner is a defensible first choice. An orchestrator earns its additional surface area when workflow-state management becomes recurring operational work rather than an occasional edge case.