Files
qinglong/deploy/kubernetes/ql3-worker

QingLong 3.0 Worker deployment baseline

The Worker is an outbound-only, headless process. It opens one shared TLS 1.3 client authority to the Cluster Worker ingress and owns one local execution journal. It does not expose an HTTP Service, database connection or Kubernetes API credential.

The committed base intentionally references, but does not create, ql3-worker-config, ql3-worker-identity or the mutable credential target. The separate credential-bootstrap asset creates an empty, labelled ql3-worker-credential exactly once so the delivery authority needs only exact-name get and update, never broad Secret creation in the Worker namespace. Copy the ConfigMap and TLS identity examples into a private overlay, replace every placeholder and use a separately issued Worker ID, certificate, ql3w token and PVC for every deployed Worker. Do not put the ql3w token in the TLS identity Secret.

Never scale this Deployment above one replica. A replica is not a stateless copy: the Worker ID, Session, credential, journal, logs, receipts and running POSIX processes form one authority. Horizontal capacity is added by deploying another independently named Worker with another identity and volume.

Build the image from the repository root:

node scripts/ql3-worker-image-lock.cjs
docker build \
  --file deploy/containers/ql3-worker/Dockerfile \
  --tag qinglong3-worker:3.0.0-alpha.2 \
  --build-arg SOURCE_REVISION="$(git rev-parse HEAD)" \
  .

Production rollout must take the Worker @sha256: reference from the same verified cluster or all release set as control, control-ai and admin. A version/source tag or a successful control image alone is not Worker release authority. Render the private Worker overlay, run the offline deployment-lock post-renderer with --required-images=worker, audit the locked manifest, and apply only that locked output. See docs/operations/ql3-release-set-deployment.md.

The builder and runtime dependency roots have separate npm v3 locks generated from the reviewed workspace pnpm-lock.yaml. The runtime image contains only runtime-core, local-process, worker-runtime and their 24 external runtime packages. The bundled durable launcher is copied explicitly and made executable; no QingLong 2.x server, UI, SQLite or PostgreSQL package is present.

Each Worker identity needs two namespaces: one Worker namespace containing only that identity's Deployment, PVC, ConfigMap, TLS Secret and credential target, and a separate staging namespace containing only immutable credential delivery stages. Never deploy either into the shared qinglong3-system namespace. A single Worker namespace is not sufficient because the stage authority must not be able to list the TLS private-key Secret.

Before applying, create both namespaces and materialize the private ConfigMap/TLS Secret. The committed names are qinglong3-worker and qinglong3-worker-credential-staging; a private overlay must replace both for every independently named Worker. It must also replace qinglong.io/worker-identity-generation with a stable, non-secret generation identifier derived by the operator's release process. Then apply either the edge base or the node resource overlay and the credential-admin RBAC. Render the private credential-bootstrap overlay and submit it with create-only semantics before the first delivery:

kubectl kustomize deploy/kubernetes/ql3-worker/base > /private/workstation/path/worker-rendered.yaml
# or render deploy/kubernetes/ql3-worker/overlays/node
# Materialize and audit worker-locked.yaml using the release-set procedure,
# then apply only the locked output:
kubectl apply -f /private/workstation/path/worker-locked.yaml
kubectl apply -k deploy/kubernetes/ql3-worker/credential-admin
kubectl kustomize deploy/kubernetes/ql3-worker/credential-bootstrap \
  | kubectl create -f -

An AlreadyExists response is a safety stop, not an instruction to apply, replace or delete the target. The normal base/GitOps reconciliation set must exclude this mutable Secret; if an external reconciler imports it, configure that reconciler to ignore the target data, labels and annotations owned by credential delivery. Never use server-side apply to continuously reconcile the bootstrap manifest.

The first Pod remains pending until credential delivery populates the prepared ql3-worker-credential. Publication is not complete after only replacing that Secret. WorkerCredentialKubernetesDeliveryAdapter stages immutable Secrets in the separate staging namespace, then reads the exact single-replica Recreate Deployment, verifies that it projects the dedicated Secret, and replaces the PodTemplate with delivery ID, credential ID, generation, token digest and publication digest annotations under the same resourceVersion fence. A crash between Secret and Deployment updates is replayed from the durable delivery ledger; the adapter returns success only after the PodTemplate converges. It owns no watcher, timer, controller loop or Worker-side Kubernetes credential.

Run a delivery through createWorkerCredentialKubernetesKubeConfigTokenRequestSession, using an external OIDC/client-certificate identity mapped to the dedicated qinglong:worker-credential-operators Kubernetes group. That group can only create the serviceaccounts/token subresource for the exact delivery ServiceAccount; it cannot use the delivery RBAC itself. The session requests a TokenRequest credential, not an automatically mounted or persisted ServiceAccount token:

kubectl -n qinglong3-worker-credential-staging create token \
  ql3-worker-credential-admin --duration=10m

The command above is an operator troubleshooting equivalent, not the product execution path: it prints the bearer token and therefore must never be used by automation. The production session receives only the issuer kubeconfig, constructs the delivery client in memory, validates the issued JWT subject and at-most-600-second lifetime, runs 8 required-allow and 20 required-deny SelfSubjectAccessReview checks before mutation, and invalidates the restricted client in finally. It never returns the token or adapter. The ServiceAccount has automountServiceAccountToken: false. In the staging namespace it can only get/list/create/delete Secrets. In the Worker namespace it can only get/update the exact prepared credential Secret and exact Worker Deployment. It cannot read the TLS Secret, list Worker Secrets, access Pods, create a target Secret, patch/delete workloads, mint a replacement token or inspect cluster-scoped resources. Stage create and list cannot safely be narrowed by resourceNames, which is why the empty staging namespace is a required isolation boundary.

The issuer kubeconfig remains external operator authority and must use strong, short-lived authentication; do not bind the token-issuer Role to the delivery ServiceAccount, Worker Pod, control Pod or a broad system group. Approval and durable product command binding are separate management-plane gates and are not implied by possession of this Kubernetes role.

The projected TLS Secret, credential Secret and ConfigMap are not consumed directly. Kubernetes projects them through symlinks and group-readable modes, while the Worker credential boundary requires direct files, a 0700 parent and 0400 private material. The non-root init container creates its own private/ subdirectory inside the fsGroup-writable tmpfs, copies the bounded inputs there and initializes 0700 recovery directories on the PVC before the main process starts. It never relies on changing the root-owned mount-point mode.

CA/key/certificate rotation uses the same Recreate boundary but remains an explicit deployment operation: update ql3-worker-identity, advance qinglong.io/worker-identity-generation in the private overlay and wait for the old Pod to terminate before accepting the replacement Session. Never reuse the credential adapter as TLS-key authority; separating the Secrets prevents its least-privilege API client from reading the Worker private key.

The PVC is mandatory. Replacing it with emptyDir can erase the only evidence needed to decide whether a process was spawned or completed after a restart. The Pod uses Recreate and a 360-second termination grace so the node profile's five-minute drain can keep heartbeating and settle completion evidence. An incomplete drain deliberately does not exit successfully.

The process entrypoint itself retains one ref'ed lifecycle handle while it waits for SIGINT/SIGTERM. A pending JavaScript Promise alone does not keep Node alive; removing this handle makes an otherwise active Worker exit with code 0 and puts an Always-restart Pod into CrashLoopBackOff. The handle is cleared with signal authority after proof-bearing stop, does not poll Kubernetes and does not replace the application's single execution cadence.

There is no synthetic Kubernetes readiness or liveness probe. A running PID does not prove that startup reconciliation completed or that the Worker Session is schedulable, and an automatic liveness kill can destroy the only drain owner. Operational readiness is the current durable Worker Session observed through the Cluster control plane.

The manual ql3-worker-kubernetes-rollout-live workflow and qinglong/worker-kubernetes-rollout-live-contract@v2 prove the current Worker image through three distinct Pod/Session generations, per-Session heartbeat, credential and mTLS identity Recreate, startup reconciliation, shared PVC and graceful offline transition. This single-node K3s/local-path evidence is not a claim about multi-node CSI detach/attach or physical node loss.

For routers or other small non-Kubernetes devices, run the same ql3-worker binary with QL3_WORKER_CAPACITY_PROFILE=edge, one concurrent run, the 2-second cadence defaults and private persistent directories. Do not install the Cluster PostgreSQL, control-plane, admin or UI closure on that device.