auto-update-daily-20260202

Tekton Pipelines as Code Migration Guide

This document provides a complete migration of all GitHub Actions workflows to Tekton Pipelines as Code (PaC).

Prerequisites

  • Kubernetes cluster with Tekton Pipelines installed (v0.54+ recommended for matrix support)
  • Heterogeneous cluster with both x86_64 and aarch64 nodes labeled appropriately
  • Pipelines as Code installed and configured
  • Repository CRD created for this repository
  • GitHub App or webhook configured for event delivery
  • Necessary secrets configured (CACHIX_AUTH_TOKEN, SBR_BOT_TOKEN)
  • Service account with permissions to create child PipelineRuns (for dynamic matrix builds)

Node Architecture Labeling

Ensure your nodes are labeled with architecture information:

# Verify node labels
kubectl get nodes -L kubernetes.io/arch

# Expected output:
# NAME        STATUS   ARCH
# node-1      Ready    amd64
# node-2      Ready    arm64

If labels are missing, add them:

kubectl label nodes <node-name> kubernetes.io/arch=amd64
kubectl label nodes <node-name> kubernetes.io/arch=arm64

Repository CRD Configuration

First, create a Repository CR to connect this repository to Pipelines as Code:

apiVersion: pipelinesascode.tekton.dev/v1alpha1
kind: Repository
metadata:
  name: home-repo
  namespace: ci  # Adjust to your CI namespace
spec:
  url: "https://github.com/vincent/home"  # Adjust to your actual repo URL
  settings:
    # Optionally fetch pipelines from default branch for security
    # pipelinerun_provenance: "default_branch"

Migration Overview

All PipelineRun definitions should be placed in .tekton/ directory at the repository root:

.tekton/
├── build-keyboard-eyelash-corne.yaml
├── build-keyboard-moonlander.yaml
├── build-packages.yaml
├── build-systems.yaml
└── nix-auto-upgrade.yaml

Workflow Translations

1. Build Keyboard Eyelash Corne

GitHub Actions: .github/workflows/build-keyboard-eyelash-corne.yaml Tekton PaC: .tekton/build-keyboard-eyelash-corne.yaml

---
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: keyboard-eyelash-corne
  annotations:
    pipelinesascode.tekton.dev/on-event: "[pull_request, push]"
    pipelinesascode.tekton.dev/on-target-branch: "[main]"
    pipelinesascode.tekton.dev/on-path-change: |
      keyboards/eyelash_corne/**,keyboards/lib/**,keyboards/Makefile,.tekton/build-keyboard-eyelash-corne.yaml
    pipelinesascode.tekton.dev/max-keep-runs: "5"
spec:
  params:
    - name: repo_url
      value: "{{repo_url}}"
    - name: revision
      value: "{{revision}}"
  workspaces:
    - name: source
      volumeClaimTemplate:
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 1Gi
    - name: artifacts
      volumeClaimTemplate:
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 500Mi
  pipelineSpec:
    params:
      - name: repo_url
      - name: revision
    workspaces:
      - name: source
      - name: artifacts
    tasks:
      - name: fetch-repository
        taskRef:
          name: git-clone
          kind: ClusterTask
        workspaces:
          - name: output
            workspace: source
        params:
          - name: url
            value: $(params.repo_url)
          - name: revision
            value: $(params.revision)
          - name: depth
            value: "1"

      - name: build-firmware
        runAfter:
          - fetch-repository
        workspaces:
          - name: source
            workspace: source
          - name: artifacts
            workspace: artifacts
        taskSpec:
          workspaces:
            - name: source
            - name: artifacts
          steps:
            - name: build
              image: ubuntu:latest
              workingDir: $(workspaces.source.path)
              script: |
                #!/bin/bash
                set -euo pipefail
                cd keyboards
                make eyelash_corne/build

            - name: copy-artifacts
              image: ubuntu:latest
              workingDir: $(workspaces.source.path)
              script: |
                #!/bin/bash
                set -euo pipefail
                cp keyboards/eyelash_corne/firmwares/eyelash_corne_*.uf2 $(workspaces.artifacts.path)/ || {
                  echo "Error: No firmware files found"
                  exit 1
                }

      - name: upload-artifacts
        runAfter:
          - build-firmware
        workspaces:
          - name: artifacts
            workspace: artifacts
        taskSpec:
          workspaces:
            - name: artifacts
          steps:
            # Option 1: Upload to S3-compatible storage
            - name: upload-to-s3
              image: amazon/aws-cli:latest
              workingDir: $(workspaces.artifacts.path)
              env:
                - name: AWS_ACCESS_KEY_ID
                  valueFrom:
                    secretKeyRef:
                      name: s3-credentials
                      key: access-key-id
                      optional: true
                - name: AWS_SECRET_ACCESS_KEY
                  valueFrom:
                    secretKeyRef:
                      name: s3-credentials
                      key: secret-access-key
                      optional: true
                - name: AWS_DEFAULT_REGION
                  value: us-east-1
                - name: S3_BUCKET
                  value: "your-artifacts-bucket"
              script: |
                #!/bin/bash
                set -euo pipefail
                if [ -n "${AWS_ACCESS_KEY_ID:-}" ]; then
                  ARTIFACT_PATH="eyelash_corne/{{revision}}"
                  aws s3 cp . "s3://${S3_BUCKET}/${ARTIFACT_PATH}/" --recursive
                  echo "Artifacts uploaded to s3://${S3_BUCKET}/${ARTIFACT_PATH}/"
                else
                  echo "S3 credentials not configured, skipping upload"
                fi

2. Build Keyboard Moonlander

GitHub Actions: .github/workflows/build-keyboard-moonlander.yaml Tekton PaC: .tekton/build-keyboard-moonlander.yaml

---
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: keyboard-moonlander
  annotations:
    pipelinesascode.tekton.dev/on-event: "[pull_request, push]"
    pipelinesascode.tekton.dev/on-target-branch: "[main]"
    pipelinesascode.tekton.dev/on-path-change: |
      keyboards/moonlander/**,keyboards/lib/**,.tekton/build-keyboard-moonlander.yaml
    pipelinesascode.tekton.dev/max-keep-runs: "5"
spec:
  params:
    - name: repo_url
      value: "{{repo_url}}"
    - name: revision
      value: "{{revision}}"
  workspaces:
    - name: source
      volumeClaimTemplate:
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 5Gi
    - name: artifacts
      volumeClaimTemplate:
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 500Mi
  pipelineSpec:
    params:
      - name: repo_url
      - name: revision
    workspaces:
      - name: source
      - name: artifacts
    tasks:
      - name: fetch-repository
        taskRef:
          name: git-clone
          kind: ClusterTask
        workspaces:
          - name: output
            workspace: source
        params:
          - name: url
            value: $(params.repo_url)
          - name: revision
            value: $(params.revision)
          - name: depth
            value: "1"

      - name: build-firmware
        runAfter:
          - fetch-repository
        workspaces:
          - name: source
            workspace: source
          - name: artifacts
            workspace: artifacts
        taskSpec:
          workspaces:
            - name: source
            - name: artifacts
          steps:
            - name: install-nix
              image: nixos/nix:latest
              workingDir: $(workspaces.source.path)
              script: |
                #!/bin/sh
                set -eu
                # Nix is already installed in nixos/nix image
                nix --version

            - name: build
              image: nixos/nix:latest
              workingDir: $(workspaces.source.path)
              script: |
                #!/bin/sh
                set -eu
                cd keyboards
                make moonlander/update moonlander/build

            - name: copy-artifacts
              image: nixos/nix:latest
              workingDir: $(workspaces.source.path)
              script: |
                #!/bin/sh
                set -eu
                cp keyboards/moonlander/build/zsa_moonlander_vincent.bin $(workspaces.artifacts.path)/ || {
                  echo "Error: Firmware file not found"
                  exit 1
                }

      - name: upload-artifacts
        runAfter:
          - build-firmware
        workspaces:
          - name: artifacts
            workspace: artifacts
        taskSpec:
          workspaces:
            - name: artifacts
          steps:
            - name: upload-to-s3
              image: amazon/aws-cli:latest
              workingDir: $(workspaces.artifacts.path)
              env:
                - name: AWS_ACCESS_KEY_ID
                  valueFrom:
                    secretKeyRef:
                      name: s3-credentials
                      key: access-key-id
                      optional: true
                - name: AWS_SECRET_ACCESS_KEY
                  valueFrom:
                    secretKeyRef:
                      name: s3-credentials
                      key: secret-access-key
                      optional: true
                - name: AWS_DEFAULT_REGION
                  value: us-east-1
                - name: S3_BUCKET
                  value: "your-artifacts-bucket"
              script: |
                #!/bin/bash
                set -euo pipefail
                if [ -n "${AWS_ACCESS_KEY_ID:-}" ]; then
                  ARTIFACT_PATH="moonlander/{{revision}}"
                  aws s3 cp . "s3://${S3_BUCKET}/${ARTIFACT_PATH}/" --recursive
                  echo "Artifacts uploaded to s3://${S3_BUCKET}/${ARTIFACT_PATH}/"
                else
                  echo "S3 credentials not configured, skipping upload"
                fi

3. Build Packages (Dynamic Matrix)

GitHub Actions: .github/workflows/build-packages.yaml Tekton PaC: .tekton/build-packages.yaml

This implementation uses a dynamic matrix pattern where a generator task spawns individual PipelineRuns for each package, allowing true parallel execution with proper architecture scheduling.

---
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: packages
  annotations:
    pipelinesascode.tekton.dev/on-event: "[pull_request, push]"
    pipelinesascode.tekton.dev/on-target-branch: "[main]"
    pipelinesascode.tekton.dev/on-path-change: |
      pkgs/**,flake.nix,flake.lock,.tekton/build-packages.yaml
    pipelinesascode.tekton.dev/max-keep-runs: "5"
spec:
  params:
    - name: repo_url
      value: "{{repo_url}}"
    - name: revision
      value: "{{revision}}"
    - name: event_type
      value: "{{event_type}}"
  workspaces:
    - name: source
      volumeClaimTemplate:
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 10Gi
  pipelineSpec:
    params:
      - name: repo_url
      - name: revision
      - name: event_type
    workspaces:
      - name: source
    tasks:
      - name: fetch-repository
        taskRef:
          name: git-clone
          kind: ClusterTask
        workspaces:
          - name: output
            workspace: source
        params:
          - name: url
            value: $(params.repo_url)
          - name: revision
            value: $(params.revision)
          - name: depth
            value: "1"

      - name: generate-and-spawn-matrix
        runAfter:
          - fetch-repository
        workspaces:
          - name: source
            workspace: source
        params:
          - name: repo_url
            value: $(params.repo_url)
          - name: revision
            value: $(params.revision)
          - name: event_type
            value: $(params.event_type)
        taskSpec:
          params:
            - name: repo_url
            - name: revision
            - name: event_type
          workspaces:
            - name: source
          stepTemplate:
            env:
              - name: REPO_URL
                value: $(params.repo_url)
              - name: REVISION
                value: $(params.revision)
              - name: EVENT_TYPE
                value: $(params.event_type)
          steps:
            - name: generate-matrix
              image: nixos/nix:latest
              workingDir: $(workspaces.source.path)
              script: |
                #!/bin/sh
                set -eu

                # Generate matrix from nix flake
                echo "Generating package matrix..."
                nix eval --json '.#githubActions.matrix' > /tmp/matrix.json
                cat /tmp/matrix.json

                # Parse and format for next step
                # Expected format: {"include":[{"attr":"pkg1","os":"ubuntu-latest"}, ...]}
                cat /tmp/matrix.json > /workspace/matrix.json

            - name: spawn-child-pipelineruns
              image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/entrypoint:latest
              workingDir: $(workspaces.source.path)
              script: |
                #!/bin/sh
                set -eu

                # Install jq for JSON parsing
                apk add --no-cache jq kubectl

                # Parse matrix JSON
                MATRIX=$(cat /workspace/matrix.json)
                echo "Matrix: $MATRIX"

                # Extract package list with architecture
                PACKAGES=$(echo "$MATRIX" | jq -r '.include[] | @base64')

                # Spawn a PipelineRun for each package
                for pkg_encoded in $PACKAGES; do
                  pkg_data=$(echo "$pkg_encoded" | base64 -d)
                  ATTR=$(echo "$pkg_data" | jq -r '.attr')
                  OS=$(echo "$pkg_data" | jq -r '.os // "ubuntu-latest"')

                  # Map OS to architecture
                  ARCH="amd64"
                  if echo "$OS" | grep -q "arm"; then
                    ARCH="arm64"
                  fi

                  echo "Spawning build for package: $ATTR (arch: $ARCH)"

                  # Create child PipelineRun
                  cat <<EOF | kubectl create -f -
                apiVersion: tekton.dev/v1
                kind: PipelineRun
                metadata:
                  generateName: package-build-${ATTR}-
                  namespace: ci
                  labels:
                    app: package-build
                    package: ${ATTR}
                    parent: packages
                spec:
                  params:
                    - name: package_attr
                      value: "${ATTR}"
                    - name: repo_url
                      value: "${REPO_URL}"
                    - name: revision
                      value: "${REVISION}"
                    - name: event_type
                      value: "${EVENT_TYPE}"
                    - name: target_arch
                      value: "${ARCH}"
                  workspaces:
                    - name: source
                      volumeClaimTemplate:
                        spec:
                          accessModes:
                            - ReadWriteOnce
                          resources:
                            requests:
                              storage: 10Gi
                  pipelineSpec:
                    params:
                      - name: package_attr
                      - name: repo_url
                      - name: revision
                      - name: event_type
                      - name: target_arch
                    workspaces:
                      - name: source
                    tasks:
                      - name: fetch-repository
                        taskRef:
                          name: git-clone
                          kind: ClusterTask
                        workspaces:
                          - name: output
                            workspace: source
                        params:
                          - name: url
                            value: \$(params.repo_url)
                          - name: revision
                            value: \$(params.revision)
                          - name: depth
                            value: "1"

                      - name: build-package
                        runAfter:
                          - fetch-repository
                        workspaces:
                          - name: source
                            workspace: source
                        params:
                          - name: package
                            value: \$(params.package_attr)
                          - name: event_type
                            value: \$(params.event_type)
                          - name: arch
                            value: \$(params.target_arch)
                        taskSpec:
                          params:
                            - name: package
                            - name: event_type
                            - name: arch
                          workspaces:
                            - name: source
                          steps:
                            - name: build
                              image: nixos/nix:latest
                              workingDir: \$(workspaces.source.path)
                              env:
                                - name: CACHIX_AUTH_TOKEN
                                  valueFrom:
                                    secretKeyRef:
                                      name: ci-secrets
                                      key: cachix-auth-token
                              script: |
                                #!/bin/sh
                                set -eu

                                echo "Building package: \$(params.package) for arch: \$(params.arch)"

                                # Install and configure cachix
                                nix profile install nixpkgs#cachix
                                cachix use chapeau-rouge
                                if [ -n "\${CACHIX_AUTH_TOKEN:-}" ]; then
                                  cachix authtoken "\${CACHIX_AUTH_TOKEN}"
                                fi

                                # Build the package
                                nix build -L ".#\$(params.package)"

                                # Push to cachix (only on push events, not PRs)
                                if [ "\$(params.event_type)" != "pull_request" ] && [ -n "\${CACHIX_AUTH_TOKEN:-}" ]; then
                                  cachix push chapeau-rouge ./result
                                fi
                              resources:
                                requests:
                                  memory: "2Gi"
                                  cpu: "1"
                                limits:
                                  memory: "4Gi"
                                  cpu: "2"
                              # Schedule on appropriate architecture
                              computeResources:
                                requests:
                                  kubernetes.io/arch: \$(params.arch)
EOF
                done

                echo "Spawned child PipelineRuns for all packages"

Key Features:

  • True Dynamic Matrix: Evaluates nix eval output at runtime and spawns individual PipelineRuns
  • Architecture-Aware: Automatically schedules builds on the correct architecture (amd64/arm64)
  • Parallel Execution: Each package builds independently and in parallel
  • Proper Resource Isolation: Each build gets its own workspace and resources

Note: This requires the ServiceAccount running the pipeline to have permissions to create PipelineRuns. See RBAC configuration below.

4. Build Systems (Dynamic Matrix with Multi-Arch)

GitHub Actions: .github/workflows/build-systems.yaml Tekton PaC: .tekton/build-systems.yaml

This implementation dynamically spawns separate PipelineRuns for each NixOS system configuration, scheduling builds on the appropriate architecture nodes.

---
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: systems
  annotations:
    pipelinesascode.tekton.dev/on-event: "[pull_request, push]"
    pipelinesascode.tekton.dev/on-target-branch: "[main]"
    pipelinesascode.tekton.dev/on-path-change: |
      home/**,systems/**,lib/**,modules/**,tools/battery-monitor/**,tools/bekind/**,tools/go-org-readwise/**,tools/k8s.infra/**,flake.nix,flake.lock,.tekton/build-systems.yaml
    pipelinesascode.tekton.dev/max-keep-runs: "5"
spec:
  params:
    - name: repo_url
      value: "{{repo_url}}"
    - name: revision
      value: "{{revision}}"
    - name: event_type
      value: "{{event_type}}"
  workspaces:
    - name: source
      volumeClaimTemplate:
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 10Gi
  pipelineSpec:
    params:
      - name: repo_url
      - name: revision
      - name: event_type
    workspaces:
      - name: source
    tasks:
      - name: fetch-repository
        taskRef:
          name: git-clone
          kind: ClusterTask
        workspaces:
          - name: output
            workspace: source
        params:
          - name: url
            value: $(params.repo_url)
          - name: revision
            value: $(params.revision)
          - name: depth
            value: "1"

      - name: generate-and-spawn-systems
        runAfter:
          - fetch-repository
        workspaces:
          - name: source
            workspace: source
        params:
          - name: repo_url
            value: $(params.repo_url)
          - name: revision
            value: $(params.revision)
          - name: event_type
            value: $(params.event_type)
        taskSpec:
          params:
            - name: repo_url
            - name: revision
            - name: event_type
          workspaces:
            - name: source
          stepTemplate:
            env:
              - name: REPO_URL
                value: $(params.repo_url)
              - name: REVISION
                value: $(params.revision)
              - name: EVENT_TYPE
                value: $(params.event_type)
          steps:
            - name: generate-matrix
              image: nixos/nix:latest
              workingDir: $(workspaces.source.path)
              script: |
                #!/bin/sh
                set -eu

                echo "Generating systems matrix..."
                # Expected format: [{"name":"kyushu","arch":"x86_64-linux"}, ...]
                nix eval .#githubActionsMatrix --raw > /tmp/matrix.json
                cat /tmp/matrix.json
                cat /tmp/matrix.json > /workspace/systems-matrix.json

            - name: spawn-child-pipelineruns
              image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/entrypoint:latest
              workingDir: $(workspaces.source.path)
              script: |
                #!/bin/sh
                set -eu

                # Install jq and kubectl
                apk add --no-cache jq kubectl

                # Parse systems matrix
                MATRIX=$(cat /workspace/systems-matrix.json)
                echo "Systems matrix: $MATRIX"

                # Extract system list
                SYSTEMS=$(echo "$MATRIX" | jq -c '.[]' 2>/dev/null || echo "$MATRIX" | jq -c '.include[]')

                # Spawn a PipelineRun for each system
                for system_data in $SYSTEMS; do
                  SYSTEM_NAME=$(echo "$system_data" | jq -r '.name')
                  SYSTEM_ARCH=$(echo "$system_data" | jq -r '.arch // "x86_64-linux"')

                  # Map nix arch to kubernetes arch
                  K8S_ARCH="amd64"
                  if echo "$SYSTEM_ARCH" | grep -q "aarch64"; then
                    K8S_ARCH="arm64"
                  fi

                  echo "Spawning build for system: $SYSTEM_NAME (arch: $SYSTEM_ARCH -> k8s: $K8S_ARCH)"

                  # Create child PipelineRun
                  cat <<EOF | kubectl create -f -
                apiVersion: tekton.dev/v1
                kind: PipelineRun
                metadata:
                  generateName: system-build-${SYSTEM_NAME}-
                  namespace: ci
                  labels:
                    app: system-build
                    system: ${SYSTEM_NAME}
                    parent: systems
                spec:
                  params:
                    - name: system_name
                      value: "${SYSTEM_NAME}"
                    - name: repo_url
                      value: "${REPO_URL}"
                    - name: revision
                      value: "${REVISION}"
                    - name: event_type
                      value: "${EVENT_TYPE}"
                    - name: target_arch
                      value: "${K8S_ARCH}"
                  workspaces:
                    - name: source
                      volumeClaimTemplate:
                        spec:
                          accessModes:
                            - ReadWriteOnce
                          resources:
                            requests:
                              storage: 50Gi
                  pipelineSpec:
                    params:
                      - name: system_name
                      - name: repo_url
                      - name: revision
                      - name: event_type
                      - name: target_arch
                    workspaces:
                      - name: source
                    tasks:
                      - name: fetch-repository
                        taskRef:
                          name: git-clone
                          kind: ClusterTask
                        workspaces:
                          - name: output
                            workspace: source
                        params:
                          - name: url
                            value: \$(params.repo_url)
                          - name: revision
                            value: \$(params.revision)
                          - name: depth
                            value: "1"

                      - name: build-system
                        runAfter:
                          - fetch-repository
                        workspaces:
                          - name: source
                            workspace: source
                        params:
                          - name: system
                            value: \$(params.system_name)
                          - name: event_type
                            value: \$(params.event_type)
                        taskSpec:
                          params:
                            - name: system
                            - name: event_type
                          workspaces:
                            - name: source
                          steps:
                            - name: setup-directories
                              image: nixos/nix:latest
                              script: |
                                #!/bin/sh
                                set -eu
                                # Create required directories for builds
                                mkdir -p /home/vincent/src/home/dots/.config/emacs
                                mkdir -p /home/vincent/desktop/documents
                                touch /home/vincent/desktop/documents/.oath

                            - name: build
                              image: nixos/nix:latest
                              workingDir: \$(workspaces.source.path)
                              env:
                                - name: CACHIX_AUTH_TOKEN
                                  valueFrom:
                                    secretKeyRef:
                                      name: ci-secrets
                                      key: cachix-auth-token
                              script: |
                                #!/bin/sh
                                set -eu

                                echo "Building NixOS system: \$(params.system)"

                                # Install and configure cachix
                                nix profile install nixpkgs#cachix
                                cachix use vdemeester
                                if [ -n "\${CACHIX_AUTH_TOKEN:-}" ]; then
                                  cachix authtoken "\${CACHIX_AUTH_TOKEN}"
                                fi

                                # Build the system
                                nix build --accept-flake-config -L ".#nixosConfigurations.\$(params.system).config.system.build.toplevel"

                                # Push to cachix (only on push events, not PRs)
                                if [ "\$(params.event_type)" != "pull_request" ] && [ -n "\${CACHIX_AUTH_TOKEN:-}" ]; then
                                  cachix push vdemeester ./result
                                fi
                              resources:
                                requests:
                                  memory: "8Gi"
                                  cpu: "2"
                                limits:
                                  memory: "16Gi"
                                  cpu: "4"
                        # Schedule on appropriate architecture node
                        podTemplate:
                          nodeSelector:
                            kubernetes.io/arch: \$(params.target_arch)
EOF
                done

                echo "Spawned child PipelineRuns for all systems"

Key Features:

  • Dynamic Multi-Arch: Parses system matrix and automatically determines target architecture
  • Heterogeneous Scheduling: Uses nodeSelector to schedule builds on matching architecture nodes
  • Resource Isolation: Each system gets its own 50Gi workspace and memory allocation
  • Parallel Builds: All systems build concurrently on their respective architecture nodes
  • Fail-Fast Disabled: Individual system failures don’t block other builds

Architecture Mapping:

  • x86_64-linuxkubernetes.io/arch: amd64
  • aarch64-linuxkubernetes.io/arch: arm64

5. Nix Auto Upgrade

GitHub Actions: .github/workflows/nix-auto-upgrade.yaml Tekton PaC: .tekton/nix-auto-upgrade.yaml

This workflow is triggered by schedule, not git events. Pipelines as Code doesn’t natively support cron schedules, so you have two options:

Create a CronJob that triggers a PipelineRun:

---
apiVersion: batch/v1
kind: CronJob
metadata:
  name: nix-auto-upgrade
  namespace: ci  # Same namespace as Repository CR
spec:
  schedule: "0 0 * * 3"  # Weekly on Wednesday at 00:00
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: pac-cronjob-trigger
          containers:
            - name: trigger-pipeline
              image: bitnami/kubectl:latest
              command:
                - /bin/bash
                - -c
                - |
                  cat <<EOF | kubectl apply -f -
                  apiVersion: tekton.dev/v1
                  kind: PipelineRun
                  metadata:
                    generateName: nix-auto-upgrade-
                    namespace: ci
                  spec:
                    pipelineRef:
                      name: nix-auto-upgrade-pipeline
                    workspaces:
                      - name: source
                        volumeClaimTemplate:
                          spec:
                            accessModes:
                              - ReadWriteOnce
                            resources:
                              requests:
                                storage: 5Gi
                  EOF
          restartPolicy: OnFailure
---
# Service account with permissions to create PipelineRuns
apiVersion: v1
kind: ServiceAccount
metadata:
  name: pac-cronjob-trigger
  namespace: ci
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pipelinerun-creator
  namespace: ci
rules:
  - apiGroups: ["tekton.dev"]
    resources: ["pipelineruns"]
    verbs: ["create", "get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: pac-cronjob-trigger-binding
  namespace: ci
subjects:
  - kind: ServiceAccount
    name: pac-cronjob-trigger
roleRef:
  kind: Role
  name: pipelinerun-creator
  apiGroup: rbac.authorization.k8s.io

Pipeline Definition for Auto-Upgrade

---
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: nix-auto-upgrade-pipeline
  namespace: ci
spec:
  workspaces:
    - name: source
  tasks:
    - name: fetch-repository
      taskRef:
        name: git-clone
        kind: ClusterTask
      workspaces:
        - name: output
          workspace: source
      params:
        - name: url
          value: "https://github.com/vincent/home"  # Adjust to your repo
        - name: revision
          value: "main"

    - name: update-flake-lock
      runAfter:
        - fetch-repository
      workspaces:
        - name: source
          workspace: source
      taskSpec:
        workspaces:
          - name: source
        steps:
          - name: install-nix
            image: nixos/nix:latest
            script: |
              #!/bin/sh
              nix --version

          - name: update-flake
            image: nixos/nix:latest
            workingDir: $(workspaces.source.path)
            env:
              - name: GITHUB_TOKEN
                valueFrom:
                  secretKeyRef:
                    name: ci-secrets
                    key: sbr-bot-token
            script: |
              #!/bin/sh
              set -eu

              # Configure git
              git config user.name "Vincent Demeester (sbr-bot)"
              git config user.email "bot@sbr.pm"

              # Update flake.lock
              nix flake update

              # Check if there are changes
              if git diff --quiet flake.lock; then
                echo "No updates available"
                exit 0
              fi

              # Create branch
              BRANCH="flake-update-$(date +%Y%m%d)"
              git checkout -b "$BRANCH"
              git add flake.lock
              git commit -m "Update flake.lock"

              # Push branch
              git remote set-url origin "https://x-access-token:${GITHUB_TOKEN}@github.com/vincent/home.git"
              git push origin "$BRANCH"

              # Create PR using GitHub API
              curl -X POST \
                -H "Authorization: token ${GITHUB_TOKEN}" \
                -H "Accept: application/vnd.github.v3+json" \
                https://api.github.com/repos/vincent/home/pulls \
                -d "{
                  \"title\": \"Update flake.lock\",
                  \"head\": \"$BRANCH\",
                  \"base\": \"main\",
                  \"body\": \"Automated flake.lock update\",
                  \"labels\": [\"dependencies\", \"automated\"]
                }"

Option B: GitHub Actions (Hybrid Approach)

Keep this specific workflow in GitHub Actions since it’s deeply integrated with GitHub’s PR creation workflow and scheduled triggers.

Additional Tekton Tasks

You may need to create or reference these ClusterTasks:

Git Clone Task

Most Tekton installations include this, but if not:

apiVersion: tekton.dev/v1
kind: ClusterTask
metadata:
  name: git-clone
spec:
  params:
    - name: url
      type: string
    - name: revision
      type: string
      default: "main"
    - name: depth
      type: string
      default: "1"
  workspaces:
    - name: output
  steps:
    - name: clone
      image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:latest
      script: |
        #!/bin/sh
        set -eu
        CHECKOUT_DIR="$(workspaces.output.path)"
        /ko-app/git-init \
          -url "$(params.url)" \
          -revision "$(params.revision)" \
          -path "$CHECKOUT_DIR" \
          -depth "$(params.depth)"

RBAC Configuration for Dynamic Matrix Builds

For dynamic matrix workflows (packages and systems), the PipelineRun needs permissions to create child PipelineRuns:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: pipeline-spawner
  namespace: ci
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pipelinerun-spawner
  namespace: ci
rules:
  - apiGroups: ["tekton.dev"]
    resources: ["pipelineruns"]
    verbs: ["create", "get", "list", "watch"]
  - apiGroups: [""]
    resources: ["pods", "pods/log"]
    verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: pipeline-spawner-binding
  namespace: ci
subjects:
  - kind: ServiceAccount
    name: pipeline-spawner
    namespace: ci
roleRef:
  kind: Role
  name: pipelinerun-spawner
  apiGroup: rbac.authorization.k8s.io

Then configure your Repository CR to use this ServiceAccount:

apiVersion: pipelinesascode.tekton.dev/v1alpha1
kind: Repository
metadata:
  name: home-repo
  namespace: ci
spec:
  url: "https://github.com/vincent/home"
  settings:
    # Use custom service account for dynamic matrix builds
    service_account: pipeline-spawner

Secrets Configuration

Create a Kubernetes Secret with your credentials:

apiVersion: v1
kind: Secret
metadata:
  name: ci-secrets
  namespace: ci
type: Opaque
stringData:
  cachix-auth-token: "your-cachix-token"
  sbr-bot-token: "your-github-token"
---
# Optional: S3 credentials for artifact storage
apiVersion: v1
kind: Secret
metadata:
  name: s3-credentials
  namespace: ci
type: Opaque
stringData:
  access-key-id: "your-access-key"
  secret-access-key: "your-secret-key"

Artifact Storage Solutions

Since Tekton doesn’t have built-in artifact storage like GitHub Actions, here are your options:

Use MinIO, AWS S3, or any S3-compatible storage. The examples above show S3 upload steps.

2. Tekton Results

Install Tekton Results for native artifact and log storage:

kubectl apply -f https://storage.googleapis.com/tekton-releases/results/latest/release.yaml

3. Persistent Volume Claims

Keep artifacts in PVCs (as shown in examples) for short-term storage. Clean up old PVCs periodically.

4. Container Registry

For firmware binaries, you could package them in container images and push to a registry:

- name: package-and-push
  image: gcr.io/kaniko-project/executor:latest
  args:
    - --dockerfile=Dockerfile.artifacts
    - --context=$(workspaces.artifacts.path)
    - --destination=ghcr.io/vincent/home/firmware:$(params.revision)

Tekton/Pipelines-as-Code Feature Gaps & Limitations

Based on this migration effort, here are notable missing features and limitations compared to GitHub Actions:

Missing Features

1. Native Dynamic Matrix Support

  • GitHub Actions: Built-in support for dynamic matrix via fromJSON() with runtime-evaluated values
  • Tekton: Static matrix only (v0.54+). Dynamic matrices require:
    • Custom tasks that spawn child PipelineRuns (as shown in packages/systems workflows)
    • Additional RBAC permissions
    • More complex pipeline definitions
    • Harder to track overall status (parent doesn’t wait for children by default)
  • Impact: High - Matrix builds are common in CI/CD
  • Workaround: Spawn child PipelineRuns via kubectl (implemented in this migration)

2. Built-in Artifact Storage

  • GitHub Actions: Automatic 30-day artifact retention with actions/upload-artifact
  • Tekton: No built-in solution, requires:
    • External S3-compatible storage
    • Tekton Results (separate installation)
    • PVCs (requires manual cleanup)
    • Container registry (for binary artifacts)
  • Impact: High - Most workflows produce artifacts
  • Workaround: S3 upload steps in pipelines

3. Native Scheduled Triggers

  • GitHub Actions: Built-in schedule with cron syntax
  • Pipelines as Code: No native support, requires:
    • Kubernetes CronJob to create PipelineRuns
    • Additional RBAC and ServiceAccount setup
    • Separate Pipeline definition (not in .tekton/)
  • Impact: Medium - Only affects periodic jobs
  • Workaround: CronJob wrapper (implemented for nix-auto-upgrade)

4. Path Change Filtering Maturity

  • GitHub Actions: Stable path filtering with glob patterns
  • Pipelines as Code: on-path-change is in Technology Preview
    • May have bugs or behavior changes
    • Limited documentation on edge cases
    • No exclusion patterns (e.g., !docs/**)
  • Impact: Medium - Used in most workflows for efficiency
  • Alternative: CEL expressions with .pathChanged() (more complex)

5. Workflow Visualization & Debugging

  • GitHub Actions: Rich UI with:
    • Workflow graph visualization
    • Step-by-step logs with timestamps
    • Re-run individual jobs
    • Inline annotations for errors
  • Tekton: Limited UI, primarily CLI-based:
    • Tekton Dashboard (basic, requires installation)
    • tkn CLI for logs
    • No native graph visualization for dynamic matrix builds
    • Hard to correlate parent and child PipelineRuns
  • Impact: High - Developer experience
  • Workaround: Use labels to query child runs: kubectl get pr -l parent=systems

6. Dependency Caching

  • GitHub Actions: actions/cache for dependency caching between runs
  • Tekton: No built-in caching, requires:
    • Persistent Volumes (slow, expensive)
    • External cache services
    • Manual cache key management
  • Impact: High - Nix builds can benefit from caching
  • Workaround: Cachix (already used), but Nix store not cached between runs

7. Concurrency Groups

  • GitHub Actions: concurrency.group with auto-cancellation
  • Pipelines as Code: max-keep-runs limits history, but:
    • No built-in queue management
    • No automatic cancellation of older runs
    • Requires Kueue for advanced queueing
  • Impact: Medium - Can waste resources on outdated runs
  • Workaround: External cancellation logic or Kueue integration

8. Multi-Line Environment Variables

  • GitHub Actions: Natural support via HEREDOC
  • Tekton: Requires careful YAML escaping
  • Impact: Low - Cosmetic issue
  • Workaround: Use ConfigMaps or multi-line YAML literals

9. Cross-Job Artifact Passing

  • GitHub Actions: needs: with implicit artifact sharing
  • Tekton: Requires explicit workspace configuration
  • Impact: Medium - Common pattern in multi-stage builds
  • Workaround: Shared workspaces (already used)

10. Failure Aggregation for Matrix Builds

  • GitHub Actions: Automatically aggregates matrix results into single status
  • Tekton (Dynamic Matrix): Parent PipelineRun doesn’t track child status
    • Need custom controller or polling logic
    • GitHub Check shows parent status only
  • Impact: High - Confusing status reporting
  • Potential Solution: Custom task that waits for children and reports aggregate status

Tekton-Specific Challenges

11. PVC Storage Management

  • Each PipelineRun creates new PVCs (if using volumeClaimTemplate)
  • Old PVCs not auto-deleted (accumulate over time)
  • Large builds (50Gi for systems) can exhaust cluster storage
  • Workaround: Periodic cleanup or use dynamic provisioner with reclaim policy

12. Image Pull for Each Step

  • Tekton pulls container images for each step individually
  • No built-in image caching at node level (depends on container runtime)
  • Can slow down pipelines with many steps
  • Workaround: Use fewer, larger steps or configure image caching at cluster level

13. Limited GitHub Integration

  • GitHub Checks API: Supported but less polished than native Actions
  • PR Comments: /test, /retest work, but limited compared to Actions bot
  • Status Details: Less granular than Actions (especially for matrix builds)
  • Annotations: No inline code annotations for errors

14. No Built-in Notification System

  • GitHub Actions can easily integrate with Slack, email, etc. via marketplace actions
  • Tekton requires custom notification tasks or external systems

Architecture-Specific Issues

15. Cross-Compilation Complexity

  • Building aarch64 on amd64 (or vice versa) requires:
    • QEMU emulation (slow, 10-100x slower)
    • Native nodes for each architecture (infrastructure cost)
    • Careful node selector configuration
  • GitHub Actions provides ARM runners natively

Stability Concerns

16. PaC Relative Maturity

  • Pipelines as Code is newer than GitHub Actions
  • Fewer real-world deployments at scale
  • Smaller community, fewer resources
  • Breaking changes more common in minor versions

Operational Overhead

17. Self-Hosting Burden

  • Requires managing Kubernetes cluster
  • Tekton version upgrades (breaking changes possible)
  • Storage provisioning and management
  • Network policies, security
  • Disaster recovery
  • Multi-arch node pool management

Recommendations for Feature Gaps

  1. Monitor Tekton Issues: Track these missing features in Tekton GitHub:

  2. Consider Hybrid Approach:

    • Keep scheduled workflows in GitHub Actions
    • Keep workflows that need rich debugging in GitHub Actions
    • Migrate only compute-intensive builds to Tekton (leverage cheaper self-hosted)
  3. Invest in Tooling:

    • Build a custom dashboard for matrix build status aggregation
    • Create reusable tasks for common patterns
    • Set up proper monitoring and alerting
  4. Contribute Upstream:

    • Report bugs in PaC path filtering
    • Contribute examples for dynamic matrix patterns
    • Document heterogeneous cluster best practices

Testing the Migration

  1. Install tkn-pac CLI:

    brew install tektoncd/tools/tektoncd-cli
    tkn pac version
    
  2. Bootstrap locally (optional):

    kind create cluster
    kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml
    kubectl apply -f https://github.com/openshift-pipelines/pipelines-as-code/releases/latest/download/release.yaml
    
  3. Test a pipeline:

    # Dry run
    tkn pac resolve -f .tekton/build-keyboard-moonlander.yaml
    
    # Create PR and watch
    tkn pac logs -L
    

Migration Checklist

Infrastructure Setup

  • Set up Kubernetes cluster with Tekton Pipelines (v0.54+)
  • Provision heterogeneous node pools:
    • x86_64 (amd64) nodes with 16Gi+ RAM for system builds
    • aarch64 (arm64) nodes with 8Gi+ RAM for ARM builds
    • Verify node labels: kubernetes.io/arch=amd64|arm64
  • Configure dynamic storage provisioner (50Gi+ PVCs for system builds)
  • Install Pipelines as Code
  • Configure GitHub App or webhook for repository

RBAC & Security

  • Create pipeline-spawner ServiceAccount with PipelineRun creation permissions
  • Create Role and RoleBinding for dynamic matrix builds
  • Create secrets:
    • ci-secrets (CACHIX_AUTH_TOKEN, SBR_BOT_TOKEN)
    • s3-credentials (if using S3 artifact storage)
  • Configure Repository CRD with custom ServiceAccount

Pipeline Definitions

  • Create ClusterTasks (git-clone if not present)
  • Copy all .tekton/*.yaml files to repository:
    • build-keyboard-eyelash-corne.yaml
    • build-keyboard-moonlander.yaml
    • build-packages.yaml (dynamic matrix)
    • build-systems.yaml (dynamic matrix multi-arch)
  • Create separate Pipeline for nix-auto-upgrade
  • Create CronJob for nix-auto-upgrade scheduled trigger

Artifact Storage

  • Set up S3-compatible storage (MinIO/AWS S3/etc)
  • Configure bucket and access credentials
  • OR install Tekton Results for native storage

Testing & Validation

  • Test simple workflow: build-keyboard-eyelash-corne
  • Test Nix workflow: build-keyboard-moonlander
  • Test dynamic matrix on amd64: build-packages
  • Test multi-arch matrix:
    • Verify amd64 systems build on amd64 nodes
    • Verify arm64 systems build on arm64 nodes
    • Check child PipelineRun scheduling with kubectl get pr -l parent=systems
  • Test artifact upload to S3
  • Verify Cachix integration works
  • Test PR comment commands (/retest, /test)

Operational Readiness

  • Configure resource limits/requests for large builds (systems: 16Gi RAM)
  • Set up PVC cleanup automation (old workspaces)
  • Set up monitoring and log aggregation
  • Create dashboard for tracking matrix build children
  • Configure alerts for:
    • PipelineRun failures
    • Storage exhaustion
    • Node resource pressure
  • Document team runbooks for:
    • Debugging failed matrix builds
    • Manually triggering pipelines
    • Querying child PipelineRuns
    • Cleaning up stuck PVCs
  • Plan for Tekton version upgrades

Troubleshooting

Pipeline not triggering

  • Check Repository CR status: kubectl describe repository home-repo -n ci
  • Verify webhook delivery in GitHub settings
  • Check PaC logs: kubectl logs -n pipelines-as-code deployment/pipelines-as-code-controller

Build failures

  • Check PipelineRun status: kubectl describe pipelinerun <name> -n ci
  • View logs: tkn pac logs or kubectl logs
  • Verify workspace PVC is large enough
  • Check node resources (CPU/memory/disk)

Artifact upload issues

  • Verify S3 credentials: kubectl get secret s3-credentials -n ci
  • Test S3 access from a pod
  • Check network policies allow egress to S3

Multi-arch build issues

  • Wrong architecture node: Check PipelineRun events
    kubectl describe pr <name> -n ci | grep -A5 Events
    # Look for "FailedScheduling" events
    
  • No nodes available: Verify heterogeneous nodes exist
    kubectl get nodes -L kubernetes.io/arch
    
  • Child PipelineRuns not spawning: Check ServiceAccount permissions
    kubectl auth can-i create pipelineruns --as=system:serviceaccount:ci:pipeline-spawner -n ci
    
  • Matrix build status unclear: Query child runs by label
    # For packages
    kubectl get pr -l parent=packages -n ci
    
    # For systems
    kubectl get pr -l parent=systems -n ci
    
    # Check status
    kubectl get pr -l parent=systems -n ci -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.conditions[?(@.type=="Succeeded")].status}{"\n"}{end}'
    

Storage exhaustion

  • PVCs accumulating: Set up automated cleanup
    # List old PVCs
    kubectl get pvc -n ci --sort-by=.metadata.creationTimestamp
    
    # Delete PVCs from completed PipelineRuns (older than 7 days)
    kubectl get pvc -n ci -o json | jq -r '.items[] | select(.metadata.creationTimestamp < (now - 604800 | strftime("%Y-%m-%dT%H:%M:%SZ"))) | .metadata.name' | xargs -r kubectl delete pvc -n ci
    
  • Node disk pressure: Monitor node storage
    kubectl top nodes
    kubectl describe nodes | grep -A5 "Allocated resources"
    

Comparison: GitHub Actions vs Tekton PaC

Feature GitHub Actions Tekton PaC
Infrastructure GitHub-hosted Self-hosted K8s
Cost Pay-per-use or free Cluster costs (24/7)
Setup complexity Low High
Artifact storage Built-in (30 days) External (S3/etc) + manual setup
Matrix builds Native, dynamic Static matrix OR custom spawning
Dynamic matrix fromJSON() native Custom task + RBAC + kubectl
Secrets GitHub UI K8s Secrets + CLI
Scheduled jobs Native cron CronJob wrapper + separate Pipeline
Multi-arch GitHub runners (x64/ARM) Heterogeneous cluster + node selectors
Caching actions/cache Cachix/external (no built-in)
GitOps Files in repo Files in repo
Path filtering Stable Tech Preview
Debugging UI Rich web UI CLI-first, basic dashboard
Status reporting Single status for matrix Parent + N children (harder to track)
Auto-cancellation Built-in concurrency Manual or Kueue
Storage cleanup Automatic Manual PVC cleanup needed

Recommendations

  1. Start with simplest workflow: Migrate build-keyboard-eyelash-corne first
  2. Use hybrid approach: Keep nix-auto-upgrade in GitHub Actions
  3. Invest in observability: Set up monitoring for PipelineRuns
  4. Plan for scale: aarch64 builds need dedicated nodes or emulation
  5. Backup strategy: Keep GitHub Actions workflows until fully validated

Summary

This migration guide provides complete translations of all 5 GitHub Actions workflows to Tekton Pipelines as Code:

Successfully Migrated Workflows

  1. build-keyboard-eyelash-corne: Simple firmware build → Direct translation
  2. build-keyboard-moonlander: Nix-based build with Cachix → Direct translation with node selector support
  3. build-packages: Dynamic matrix from nix evalCustom dynamic matrix implementation spawning child PipelineRuns
  4. build-systems: Multi-arch NixOS builds → Advanced dynamic matrix with heterogeneous node scheduling
  5. nix-auto-upgrade: Scheduled updates → CronJob wrapper + separate Pipeline

Key Implementation Patterns

  • Dynamic Matrix Builds: Custom task pattern that:

    • Parses nix eval JSON output
    • Spawns child PipelineRuns via kubectl
    • Maps architectures to node selectors
    • Requires RBAC for PipelineRun creation
  • Heterogeneous Scheduling:

    • nodeSelector: {kubernetes.io/arch: amd64|arm64}
    • Automatic mapping from Nix arch to Kubernetes arch
    • Parallel execution across architecture pools
  • Artifact Storage: S3-compatible external storage pattern

Migration Complexity Assessment

Workflow Complexity Reason
eyelash_corne Low Simple build, single step
moonlander Low-Medium Nix build, artifact upload
packages High Dynamic matrix requires custom spawning
systems Very High Dynamic matrix + multi-arch + large PVCs
nix-auto-upgrade Medium Requires CronJob wrapper

Critical Feature Gaps Identified

  1. No native dynamic matrix → Custom spawning pattern required
  2. No built-in artifact storage → External S3 setup required
  3. No scheduled triggers → CronJob wrapper required
  4. Path filtering in Tech Preview → May have stability issues
  5. Limited matrix status aggregation → Parent doesn’t track children
  6. Manual PVC cleanup → Automation required to prevent storage exhaustion
  7. No dependency caching → Rely on Cachix for Nix artifacts

Infrastructure Requirements

  • Kubernetes: v1.24+ with Tekton Pipelines v0.54+
  • Nodes:
    • Minimum 2x amd64 nodes (16Gi RAM each)
    • Minimum 1x arm64 node (8Gi RAM)
  • Storage: Dynamic provisioner supporting 50Gi+ PVCs
  • External Services:
    • S3-compatible storage for artifacts
    • Cachix for Nix binary cache

Cost Considerations

GitHub Actions (Current):

  • Free for public repos OR pay-per-minute for private
  • No infrastructure management
  • Zero operational overhead

Tekton PaC (Proposed):

  • Kubernetes cluster running 24/7
  • Storage costs (PVCs, S3)
  • Network egress costs
  • DevOps time for maintenance
  • Break-even: Only if you run many builds OR have very cheap Kubernetes infrastructure

Recommendation

For this specific use case, the migration is technically feasible but operationally expensive:

Migrate if:

  • You already run Kubernetes infrastructure
  • You need tighter control over build environments
  • You have compliance requirements for self-hosted CI
  • You want to leverage heterogeneous ARM hardware you already own

Don’t migrate if:

  • You want simplicity and low maintenance
  • Your builds run infrequently
  • You value rich debugging UI and developer experience
  • You don’t have Kubernetes expertise

Hybrid Alternative:

  • Keep complex workflows (packages, systems, nix-auto-upgrade) in GitHub Actions
  • Migrate only simple keyboard builds to Tekton if you have excess cluster capacity

Next Steps

  1. Pilot: Test build-keyboard-eyelash-corne first in a staging cluster
  2. Evaluate: Measure operational overhead vs. benefits
  3. Decide: Full migration, hybrid, or stay with GitHub Actions
  4. Document: Create runbooks for team if proceeding

References