Most pipeline-as-code conversations start with syntax. YAML is tedious; a general-purpose language has types, functions, imports, and fewer opportunities to accidentally turn a string into an array.

All true. None of it is the interesting part.

A YAML file can describe a good dependency graph. A TypeScript file can be an opaque program that starts containers, makes HTTP requests, consults the current time, and decides what its next piece of work is only after it has run three earlier pieces. Changing braces into indentation is not the architectural choice.

The question we cared about was more basic: what can the platform know before execution starts?

Canopy’s answer is deliberately two-layered. canopy.pipeline.ts is TypeScript, because that is a useful authoring surface. The thing Canopy plans and executes is a Canopy Pipeline Graph, or CPG, because that is a bounded contract the platform can inspect.

The pipeline is code. The execution is a graph.

That distinction sounds small until you try to make a CI system explain what it is about to do, decide what it can skip, restrict an untrusted pull request, or preserve a meaningful record of how a deployment came to exist.

The file that does not run your pipeline

Here is the beginning of this repository’s real root canopy.pipeline.ts. It is not a toy example. The first two steps are intentionally mundane: the first writes a proof into its workspace; the second reads it. They exist to exercise actual workspace chaining between isolated nodes.

import { definePipeline } from '@canopy/ci';

export default definePipeline(({ step }) => {
  const src = step.source();
  const build = step('build', {
    image: BUN_ALPINE_IMAGE,
    run: 'echo "hello from canopy.pipeline.ts" && echo "workspace-passing-proof-$(date +%s)" > proof.txt',
    inputs: [src],
  });
  step('test', {
    image: BUN_ALPINE_IMAGE,
    run: 'test -f proof.txt && echo "workspace chaining works: $(cat proof.txt)"',
    inputs: [build],
  });
});

At a glance, step() looks a little like a function that runs a command. It is not. It returns an inert { id } handle. Passing that handle in inputs declares an edge. No container starts. No command runs. No network connection is made.

definePipeline() itself is even less dramatic. It captures the builder function and returns a tagged definition. Later, the generation service calls compilePipeline(definition, context). That one synchronous call supplies the real repository ID, ref, commit, trigger event, trust level, and locked plugin versions, then collects the declarations into data.

flowchart TD
  source["canopy.pipeline.ts"] --> definition["definePipeline(builder)<br/>captures builder only"]
  definition --> compile["Hermetic generator calls<br/>compilePipeline(definition, context)"]
  compile --> cpg["Canopy Pipeline Graph<br/>nodes, edges, operation declarations"]
  cpg --> validate["Validation"]
  cpg --> identity["Content identity<br/>and digests"]
  cpg --> plan["Planning<br/>trust, policy, cache prediction"]
  validate --> execute["Host execution layer"]
  identity --> execute
  plan --> execute
  execute --> state["Recorded node state,<br/>workspaces, artifacts, deployment"]

The generator runs the source in a sandbox with no network and writes a graph JSON file through a declared output path. Canopy runs generation twice and compares the resulting graph digests. It also freezes the clock and seeds randomness from the commit as a determinism aid. Those measures do not turn arbitrary JavaScript into a security boundary. The sandbox is the boundary. They do make accidental time- and randomness-shaped graph drift easier to detect before a plan becomes execution.

There is a tradeoff here. A pipeline author cannot write await ci.run(...) and let the order in which JavaScript promises settle become the scheduler. That is intentional. The SDK’s author.ts is explicit about rejecting that model: program execution order should not secretly become pipeline execution order.

This makes canopy.pipeline.ts neither YAML with braces nor an arbitrary Node.js script. It is also not an async workflow, a shell script wearing an SDK costume, magic, or an AI system. It is TypeScript used to construct a declarative graph.

A graph made from ordinary TypeScript

The root pipeline does more than its small build-to-test proof. It declares independent checks from the same source checkout: an OpenAPI merge unit test, API-spec coverage, an SDK drift check, and Go build, vet, and test work for Forest. The latter two carry facts that a scheduler and a reviewer should not have to recover from a shell script.

step('sdk-drift', {
  image: OPENAPI_DRIFT_TEST_IMAGE,
  run: `
    set -eu
    python3 -m venv cortex/venv
    # install dependencies and verify generated SDK types
  `,
  inputs: [src],
  egress: [...NPM_EGRESS, ...PIP_EGRESS],
  resources: { memoryMb: 8192, cpus: 2 },
  timeoutSeconds: 2400,
});

step('forest-go-test', {
  image: GO_TEST_IMAGE,
  run: `cd forest && go build ./... && go vet ./... && go test ./... -timeout 120s`,
  inputs: [src],
  egress: ['proxy.golang.org', 'sum.golang.org'],
  resources: { memoryMb: 2048, cpus: 2 },
  timeoutSeconds: 600,
});

The resulting shape is a real consequence of those calls, not an illustration pasted beside them:

flowchart LR
  source["source"] --> build["build"] --> test["test"]
  source --> merge["spec-merge-unit"]
  source --> coverage["spec-coverage"]
  source --> drift["sdk-drift"]
  source --> forest["forest-go-test"]

The independent branches are visible before any one of them begins. test cannot start until build has succeeded, because its input says so. The other four checks can be considered independently. In a conventional job-and-step system, that relationship is often partly structural and partly hidden in an executor’s control flow. Here it is the primary object.

The CPG currently has six operation kinds: source, exec, file, platform, gate, and dynamic. An ExecOp declares a digest-pinned image, command, optional environment values, secret references, egress hosts, cache mounts, resource limits, timeout, and retry policy. A secret reference contains a name and source, never the secret value. A source operation carries the repository, ref, and commit bound by the generation context. File operations describe copy, mkdir, and remove actions. Gate operations name an approval-token class.

Platform operations are separate because a deployment or cache invalidation is not just another shell command. Today the model includes deploy, rollback, cache invalidation, promote, scale, and image build actions. step.buildImage() creates a build_image platform node without accepting an author-selected environment or builder image; those are platform-owned context. canopy.environment(id).deploy(...) creates a first-class deploy node. The distinction gives the planner somewhere specific to apply platform policy instead of attempting to infer intent from a command string.

Not every type is equally complete at runtime. Gate approval has a resolution path. Deploy, rollback, cache invalidation, image build, and Forest-managed scale have backing services. Dynamic subgraphs are explicitly represented as the one sanctioned runtime escape hatch, but are not executed yet. Promotion is rejected at planning time where it lacks the required backing information. That is less glamorous than pretending every enum member is finished, and more useful.

What becomes possible when the work is data

The raw graph uses node IDs for edges. That keeps ordinary graph work ordinary: Canopy can reject dangling inputs and self-references, topologically sort a DAG, report an actual cycle path, find a critical path from recorded durations, and identify dead branches relative to its meaningful terminal nodes.

Content identity is a second layer. For each node, digestGraph() walks the graph in topological order. It hashes canonical JSON for the node’s platform and operation, plus the already-derived digests of its inputs. Object keys are sorted so insertion order does not decide identity. A downstream node’s identity changes when the thing producing one of its inputs changes, rather than merely because an upstream node happens to have a different local ID.

Two fields are deliberately excluded from an execution result’s digest: resource limits and mutable cache mounts. Giving a node more memory does not change what it is trying to produce, and a package-manager scratch directory is not a proof of output content. Including either would fragment the result cache for reasons unrelated to the result.

That distinction supports real planning today. Before scheduling containers, Host validates the authored graph, derives node digests, checks cache namespaces for predicted hits, and calls the graph solver’s prune() function. A direct cache hit avoids executing that node. If all dependents of an upstream node are already prunable, that upstream work can disappear too. The executor seeds those pruned nodes as successful cache hits and only queues nodes whose explicit inputs have succeeded.

It also gives validation a useful place to live. The Stage-2 validator currently rejects duplicate IDs, missing or self-referential inputs, graphs beyond their node or declared aggregate resource ceilings, unpinned execution images, malformed egress entries, malformed source operations, empty file operations, unknown platform actions, and incomplete gate or dynamic operations. The important property is timing: these failures happen while Canopy has a graph, not after a half-started collection of containers has done some unrelated work.

Planning adds decisions that cannot safely belong to the authoring API. For an untrusted pull request, Host strips platform operations and secret references even if the author wrote them. It evaluates applicable promotion policy before execution. It resolves plugin attribution against an integrity-checked canopy.plugins.lock and verifies a plugin-created node’s requested image, egress, and secrets against that plugin’s permission manifest. It can show the explicit set of egress hosts, secret references, platform operations, cache predictions, and stripped nodes in a plan.

None of this means TypeScript itself is safe. Nor does a graph make a command safe. The graph makes its requested capabilities legible enough to validate and constrain before the command gets a runtime.

Where execution actually begins

Once a graph has been planned, the Host executor stores a run row for every node and advances from persisted node state. It is not holding a JavaScript promise chain in memory. A node becomes ready when every input node has succeeded; independent branches continue after an unrelated branch fails, while transitive dependents of the failed node are marked skipped.

The operation declarations are then handed to the right execution path. A source node performs the real checkout. An exec node runs in a sandboxed container. File actions are materialized through the same sandboxed workspace machinery. Platform nodes dispatch to existing Host services. Workspace output can be captured and chained to downstream nodes, with multi-parent inputs merged in declared order; Forest-backed execution can use shared object storage for that workspace hand-off.

This is also where the limits of inspectability are honest. The graph can say that a command needs registry.npmjs.org, 8 GB of memory, and forty minutes. It cannot prove that the command will not compile a different binary when a dependency’s own inputs are floating. It cannot make a non-deterministic test deterministic. It cannot know the contents of an arbitrary command’s output without running it. A graph is a contract for requested work, not a proof that the work is morally tidy.

Still, it is a much better starting point than “some CI command ran.” Host can retain the requested graph alongside node state, logs, digests, cache outcomes, workspace/artifact lineage, and later deployment state. That is a natural fit for a system concerned with current state, sequence, provenance, artifacts, deployments, verification, recovery, and engineering history.

Today, graph compilation and validation are broadly in the build path, while graph execution is early access: gate mode is live for allowlisted environments and full graph driver mode is enabled cautiously, one environment at a time. The data model is deliberately ahead of universal rollout. That is not a promise that every possible optimization already happens. It is a commitment that the platform will have an inspectable object to improve rather than a pile of imperative history to reverse-engineer later.

The boundary moved for a reason

This was not the first shape of Canopy’s CI work. Earlier repository history contains conventional GitLab CI jobs and steps. They were useful for running checks, but the execution model and the representation were intertwined: the CI system discovered the work through a provider-specific configuration and job runner.

The first @canopy/ci commit established the CPG as a small, pure data package with validation, graph solving, and canonical digests. The next authoring-surface commit added definePipeline() and compilePipeline() only after preserving that separation. Later changes made the boundary more explicit rather than less: platform image builds became a dedicated operation with planner injection when appropriate; plugins gained a deliberately narrow step.plugin() capability so a plugin cannot choose its own attribution or escalate into source, platform, or nested-plugin authority; author-declared build egress became graph data too.

That progression is a useful correction to a tempting design. It is easy to make code-first CI mean “write arbitrary code and call a runner.” It feels expressive at first. It also asks the platform to execute a program before it knows what exists, then reconstruct dependencies, permissions, and identity from side effects and control flow. We wanted programmability without giving up inspectability, so the boundary moved: TypeScript is free to compose declarations; only the CPG is allowed to describe executable work.

There are costs. Runtime conditionals do not fit cleanly. Dynamic graphs are intentionally rare and still unfinished. The API must be designed with care because every new convenience becomes a new operation or a field the planner needs to understand. We think those are good costs to pay in CI, where “what will happen next?” is usually more valuable than unrestricted cleverness.

The value is not that a .ts extension has better autocomplete. The value is that before execution, Canopy can see a finite graph of source state, declared work, dependencies, runtime requirements, requested privileges, and platform actions. That lets it validate and plan today. It gives it a sound basis for richer scheduling, policy, cache explanation, and provenance work later.

That is the part we wanted to build.