diff --git a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md index 62510c6d..6d3811c3 100644 --- a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md +++ b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md @@ -6,7 +6,7 @@ - 目标版本：QingLong 3.x - 作者：QingLong Maintainers - 创建日期：2026-07-17 -- 最后更新：2026-08-13 +- 最后更新：2026-08-14 - 讨论范围：架构与演进路线，不包含最终 UI 视觉方案 最新增量证据（2026-08-13）： @@ -29,7 +29,7 @@ - D-305/ADR-0393（已接受）：Plugin Package Manifest 的逻辑 Secret requirement 获得按 resource generation 固定的不可变 binding。binding 精确覆盖 Manifest requirements，只保存同 Project、显式 version 的 `qlsecret://` 引用与 `approved-action-execution|local-owner-confirmation` authority evidence digest，不保存 Secret 明文；required 不可为空，optional 可显式为 `null`。发布由当前 active installation head、lock、generation 与 Manifest digest 联合 fencing，相同事实幂等、不同事实冲突；domain-separated digest、最多 64 项和 64 KiB 单行预算同时约束 Local 与 Cluster。SQLite 追加 `0091` ledger 与 capability v46，PostgreSQL 追加 `pg-0059`、capability v58，并只向 package executor 授予 `SELECT, INSERT`。不新增 package、daemon、timer、watcher、连接、缓存或集群 workload，低配路由设备只承担一个有界表和三个索引。D-305 不冒充 Secret 已进入执行路径：现有 materialization 拒绝仍保留，D-306 再完成用户授权、Secret resolution、runtime consumption 与 lifecycle/rebinding 语义。core 509/509、SQLite 232/232、PostgreSQL 316 pass/1 条件 skip；完整 18-package clean build/test 退出 0，backend 1,188 pass/2 skip，五项边界审计零 finding，workspace 仍无 single-source/shallow-source package。PostgreSQL 18.4 arm64 HA 125 项 gate 全绿、timeline `1→2`，报告 SHA-256 为 `acf0fea7ca7699989dfe70f5dd0061cdf5fb1968c691094331fea06ce01b96dc`。 - D-306A/ADR-0394（已接受）：Package Task source 以 `package-secret` placeholder 引用逻辑 requirement，materialization 只用当前 generation 的 D-305 binding 编译为已有、固定 version 的 Task `SecretRef`；Package source 直接携带 SecretRef、缺失 binding、未批准 `secret.use`、跨 binding 引用和 optional/required 漂移全部失败关闭。binding 快照不含明文并进入 materialized revision digest，Local/Cluster 启动发布复用既有 repository/pool，Task dispatch、Local 短时解密和 Cluster offer/lease-fenced delivery 不另造协议。SQLite/PostgreSQL INSERT trigger 同时防止直接写库绕过；Local 只读 readiness 继续不加载 DDL。Local contract v47、Cluster v59；不新增 package、表、索引、连接、daemon、watcher、timer、cache 或 workload。完整 18-package clean build/test 退出 0；backend 1,188 pass/2 条件 skip/0 fail；五项 package/dependency/edge/service-manager/local-image 审计零 finding，workspace 仍无 single-source/shallow-source package，两个有序 migration ledger 精确为 PostgreSQL 61、SQLite 95 个 source；PostgreSQL 18.4 arm64 HA 125 项 gate 全绿、timeline `1→2`，报告 SHA-256 为 `f9107e8e54892a788779758f0573ac8d6a80f6d086516a1f5f5bbacb59bbb4be`。D-306A 不冒充产品闭环：Local bind/rebind 命令、Cluster Approved Action/API 与新 generation rotation/revocation 编排属于 D-306B。 - D-306B1/ADR-0395（已接受）：当前 active、尚未绑定 Package generation 的首次 Secret binding 已形成 Local 与 Cluster 产品闭环，且不允许原地 rebind。共享 content-free plan 由服务端从 installation/proposal/lock/Manifest/generation 重建；Local 使用短生命周期 `ql3-package`、Owner human confirmation 与单 SQLite transaction，Cluster 使用既有 package-management HTTPS/CLI、package-manager separation-of-duty Approval 和短生命周期 package-executor。三节点 K3s `v1.34.3+k3s1` arm64 现场门已在真实 PostgreSQL `18.4` 上完成两个 management Pod 跨节点部署、正式 client `plan→跨副本 replay→propose→双人 decide→inspect`、真实 executor Job 与只读 Kubernetes Secret projection。management/executor 均无 Secret API 读取权和 ServiceAccount token；management 不挂载 Package value，executor 只验证投影元数据；最终恰好一条 immutable binding，Approval consumed、execution succeeded，数据库敏感值扫描为 0。16/16 gate 的 owner-private、低敏报告通过独立 exact-shape 审计，SHA-256 为 `aaabb5ebea77c50bce671f91dd3051671fd20875c11a8f787fe8933f29dbfa4d`。完整 18-package clean build/test、backend 与七项边界审计，以及 PostgreSQL 18.4 physical HA 125 gate/timeline `1→2` 证据继续有效；没有新增 workspace package、migration、表、索引、依赖或常驻 workload。B2 rebind/rotation/revocation 必须通过新 Package generation 独立推进。 -- D-306B2/ADR-0396（进行中）：Secret rebind/rotation/revocation 不更新历史 binding，而是作为下一 Package generation 的 activation 前置事实。共享 transition plan v1 同时绑定上一 active target、可选的上一 binding、durable install history 的最后尝试 generation、新 target、可选下一 binding plan、逐 requirement 与 SecretRef 差异及独立 digest；上一 active Manifest 没有 Secret requirement 时 binding 可空，但 target/lock/generation lineage 不可省略。失败 install 也永久消耗 generation，重试必须使用 `lastAttemptGeneration + 1`，active lineage 继续由 `previousActiveLockDigest` 指回旧代。SQLite capability v49（0097/0098）已完成 immutable transition receipt ledger、Local Owner staged `plan→execute`、单事务 binding/audit/receipt 和 activation prerequisite。PostgreSQL capability v62（pg-0063）具有 receipt ledger，Cluster management 与 package-executor 也已完成 separation-of-duty 产品编排：executor 在一个 SERIALIZABLE transaction 中复验 current staged head、上一 active lineage、durable 最大 generation 与可选上一 binding，并原子提交可选目标 binding 和 immutable receipt；数据库 trigger 和最小角色 readiness 防止绕过，recovery/直接 activation 缺 receipt 均失败关闭。Cluster startup recovery 现把 binding/transition receipt 编译为 content-blind Kubernetes active pointer v3：只保存 source Secret 名、不可逆 SHA-256 key/path、`0440`、binding/receipt/projection digest 和逻辑 assignment，不保存 SecretRef 或明文；同一次 ConfigMap `resourceVersion` CAS 同时切换 Package generation 与投影声明。rotate 只投影下一代 exact key；revoke 生成显式空 projection，Pod volume renderer 对空项返回不挂载，避免 Kubernetes 空 `items` 被解释为投影全部 key。无 Secret generation 继续发布兼容 v2；publisher 不获得 Secret `get/list`，不新增 watcher/controller。响应丢失通过 durable pointer inspect 精确收敛，projection source 不可用或 digest 漂移时旧 active pointer 保持不变。真实三节点 K3s `v1.34.3+k3s1` 已完成两个受限 actor 同一 resourceVersion 的 v3 rotation 竞争：1 成功/1 冲突、最终恰好 1 pointer、1 个 exact projection item、Secret API read denied，projection digest `22add8965accf3f736935167963b9dbdeab8fba05f739f24d39dec941aae9680`、transition receipt digest `355b89ecd54422af33fa573780c8b70ed41da4df226ec301bdd5b6c71de609e1`；临时容器/网络已清理。实现没有新增 workspace package，Secret projection/renderer 内聚在既有 `cluster-admin/plugin-package/secret-binding`，18-package boundary 仍为 `singleSourcePackages=[]`、`shallowSourcePackages=[]`，cluster dependency 与 edge import 审计无 finding。上一阶段真实 PostgreSQL 18.4 的 63 条 migration、原子提交、exact replay、错误状态/lineage 拒绝，以及 physical HA 125 gate/timeline `1→2` 证据继续有效。当前增量的完整 18-package/PG/HA 回归、真实 Kubernetes revoke 工作负载、升级后业务 Pod rollout/失败回滚和低配物理证据仍待完成。 +- D-306B2/ADR-0396（进行中）：Secret rebind/rotation/revocation 不更新历史 binding，而是作为下一 Package generation 的 activation 前置事实。共享 transition plan v1 同时绑定上一 active target、可选的上一 binding、durable install history 的最后尝试 generation、新 target、可选下一 binding plan、逐 requirement 与 SecretRef 差异及独立 digest；上一 active Manifest 没有 Secret requirement 时 binding 可空，但 target/lock/generation lineage 不可省略。失败 install 也永久消耗 generation，重试必须使用 `lastAttemptGeneration + 1`，active lineage 继续由 `previousActiveLockDigest` 指回旧代。SQLite capability v49（0097/0098）已完成 immutable transition receipt ledger、Local Owner staged `plan→execute`、单事务 binding/audit/receipt 和 activation prerequisite。PostgreSQL capability v62（pg-0063）具有 receipt ledger，Cluster management 与 package-executor 也已完成 separation-of-duty 产品编排：executor 在一个 SERIALIZABLE transaction 中复验 current staged head、上一 active lineage、durable 最大 generation 与可选上一 binding，并原子提交可选目标 binding 和 immutable receipt；数据库 trigger 和最小角色 readiness 防止绕过，recovery/直接 activation 缺 receipt 均失败关闭。Cluster startup recovery 现把 binding/transition receipt 编译为 content-blind Kubernetes active pointer v3：只保存 source Secret 名、不可逆 SHA-256 key/path、`0440`、binding/receipt/projection digest 和逻辑 assignment，不保存 SecretRef 或明文；同一次 ConfigMap `resourceVersion` CAS 同时切换 Package generation 与投影声明。rotate 只投影下一代 exact key；revoke 生成显式空 projection，Pod volume renderer 对空项返回不挂载，避免 Kubernetes 空 `items` 被解释为投影全部 key。无 Secret generation 继续发布兼容 v2；publisher 不获得 Secret `get/list`，不新增 watcher/controller。响应丢失通过 durable pointer inspect 精确收敛，projection source 不可用或 digest 漂移时旧 active pointer 保持不变。真实三节点 K3s `v1.34.3+k3s1` 已完成两个受限 actor 同一 resourceVersion 的 v3 rotation 竞争：1 成功/1 冲突、最终恰好 1 pointer、1 个 exact projection item、Secret API read denied，projection digest `22add8965accf3f736935167963b9dbdeab8fba05f739f24d39dec941aae9680`、transition receipt digest `355b89ecd54422af33fa573780c8b70ed41da4df226ec301bdd5b6c71de609e1`；随后生产 renderer 驱动 2 副本 workload 分布到 2 个节点，源 Secret 同时保留目标 key 与 decoy key，而 Pod 只看到 exact path、文件模式 `0440`。第三个同权限 actor 以 generation 3 transition receipt `252e8cd1d0f8a2b63f8c0af6247861c08e5b7ae8886c0d65cd90537be3e9f9ac` 发布显式空 projection；Deployment 滚动产生全新 Pod UID，两个新 Pod 均无 Secret volume/mount 和投影根目录，源 Secret 保留，actor 的 Secret `get` 仍为 403；临时容器/网络已清理。实现没有新增 workspace package，Secret projection/renderer 内聚在既有 `cluster-admin/plugin-package/secret-binding`，18-package boundary 仍为 `singleSourcePackages=[]`、`shallowSourcePackages=[]`，cluster dependency 与 edge import 审计无 finding。阶段提交 `9d7431c2` 后，完整 18-package 串行 build/test 退出 0，backend 1,192 pass/2 skip/0 fail；PostgreSQL 18.4 physical HA 125 gate、timeline `1→2` 通过，报告 SHA-256 为 `2f0d1107cc6d447868bfaeb3650284acd803924c6cd3cdee978b3eb5882eb26c`；Edge 本机观测的模块加载 RSS 增量约 6.0 MiB、1 万行输出峰值增量约 3.9 MiB，但尚无固定物理低配门。该 K3s workload 仍由 live harness 显式应用，不能冒充生产 rollout controller；现有 opt-in Package executor CronJob 的无 `items` optional Secret mount 也尚未收敛为 action-scoped exact projection。生产升级失败自动回滚、executor 精确投影重构和低配物理证据仍待完成。 - D-302/ADR-0390（已接受） Cluster operator context 增加无网络、无 mutation 的内建 `ql3-cluster-admin context validate` 预检。它先复用 owner-private context reader，再让每个 entry 经过与真实请求相同的 production HTTPS/Kubernetes configuration preparation，验证精确 route、hostname、CA、 diff --git a/scripts/ql3-plugin-package-kubernetes-k3s-live-contract.cjs b/scripts/ql3-plugin-package-kubernetes-k3s-live-contract.cjs index 0829db3d..18b5a1a4 100644 --- a/scripts/ql3-plugin-package-kubernetes-k3s-live-contract.cjs +++ b/scripts/ql3-plugin-package-kubernetes-k3s-live-contract.cjs @@ -21,6 +21,11 @@ const RESULT_SCHEMA = 'qinglong/plugin-package-kubernetes-live-actor-result@v1'; const NAMESPACE = 'ql3-plugin-package-live'; const SERVICE_ACCOUNT = 'ql3-plugin-package-recovery-live'; const IMAGE = 'qinglong3-cluster-admin:ql3-k3s-kubernetes-live'; +const WORKLOAD = 'ql3-plugin-package-workload-live'; +const SECRET_ROOT = '/var/run/secrets/qinglong3/plugin-package-values'; +const WORKLOAD_REPLICAS = 2; +const SECRET_MARKER = 'ql3-live-exact-projection'; +let renderWorkloadVolume = null; function run(binary, args, options = {}) { const result = spawnSync(binary, args, { @@ -171,10 +176,233 @@ async function actorResult(fixture, actor) { return value; } +function activePointer(fixture) { + const pointers = fixture.kubectlJson([ + '-n', + NAMESPACE, + 'get', + 'configmaps', + '-l', + 'qinglong.io/plugin-package-active=v3', + ]).items; + assert.equal(pointers.length, 1); + const pointer = JSON.parse(pointers[0].data['active.json']); + assert.equal( + pointer.schema, + 'qinglong/plugin-package-kubernetes-active-pointer@v3', + ); + return Object.freeze({ configMap: pointers[0], pointer }); +} + +function sourceSecret(projection) { + assert.equal(projection.items.length, 1); + const projected = projection.items[0].key; + const decoy = projected === 'f'.repeat(64) ? 'e'.repeat(64) : 'f'.repeat(64); + return Object.freeze({ + decoy, + document: { + apiVersion: 'v1', + kind: 'Secret', + type: 'Opaque', + immutable: true, + metadata: { + name: projection.sourceSecretName, + namespace: NAMESPACE, + labels: { + 'app.kubernetes.io/managed-by': 'qinglong3-live-gate', + 'qinglong.io/live-gate-role': 'projection-source', + }, + }, + data: { + [projected]: Buffer.from(SECRET_MARKER, 'utf8').toString('base64'), + [decoy]: Buffer.from('ql3-live-decoy', 'utf8').toString('base64'), + }, + }, + }); +} + +function workloadDeployment(active) { + const projection = active.pointer.secretProjection; + assert.equal(typeof renderWorkloadVolume, 'function'); + const rendered = renderWorkloadVolume(projection); + const labels = { + 'app.kubernetes.io/name': WORKLOAD, + 'app.kubernetes.io/component': 'plugin-package-workload', + }; + return { + apiVersion: 'apps/v1', + kind: 'Deployment', + metadata: { + name: WORKLOAD, + namespace: NAMESPACE, + labels, + }, + spec: { + replicas: WORKLOAD_REPLICAS, + revisionHistoryLimit: 2, + progressDeadlineSeconds: 60, + strategy: { + type: 'RollingUpdate', + rollingUpdate: { maxUnavailable: 0, maxSurge: 1 }, + }, + selector: { matchLabels: labels }, + template: { + metadata: { + labels, + annotations: { + 'qinglong.io/plugin-package-generation-digest': + active.pointer.intent.resourceGeneration.generationDigest, + 'qinglong.io/plugin-package-lock-digest': + active.pointer.intent.lockDigest, + 'qinglong.io/plugin-package-secret-projection-digest': + projection?.projectionDigest ?? 'none', + }, + }, + spec: { + automountServiceAccountToken: false, + securityContext: { + runAsNonRoot: true, + runAsUser: 10001, + runAsGroup: 10001, + fsGroup: 10001, + seccompProfile: { type: 'RuntimeDefault' }, + }, + affinity: { + podAntiAffinity: { + requiredDuringSchedulingIgnoredDuringExecution: [ + { + labelSelector: { matchLabels: labels }, + topologyKey: 'kubernetes.io/hostname', + }, + ], + }, + }, + containers: [ + { + name: 'workload', + image: IMAGE, + imagePullPolicy: 'Never', + command: [ + 'node', + '-e', + 'setInterval(() => {}, 2147483647)', + ], + securityContext: { + allowPrivilegeEscalation: false, + readOnlyRootFilesystem: true, + capabilities: { drop: ['ALL'] }, + }, + resources: { + requests: { cpu: '10m', memory: '32Mi' }, + limits: { cpu: '250m', memory: '128Mi' }, + }, + ...(rendered + ? { volumeMounts: [rendered.volumeMount] } + : {}), + }, + ], + ...(rendered ? { volumes: [rendered.volume] } : {}), + }, + }, + }, + }; +} + +async function workloadReady(fixture, active, minimumGeneration = 1) { + const expectedLockDigest = active.pointer.intent.lockDigest; + const expectedProjectionDigest = + active.pointer.secretProjection?.projectionDigest ?? 'none'; + const observed = await waitFor( + `Deployment/${WORKLOAD} rollout`, + 120_000, + () => { + const deployment = fixture.kubectlJson([ + '-n', + NAMESPACE, + 'get', + 'deployment', + WORKLOAD, + ]); + const pods = fixture.kubectlJson([ + '-n', + NAMESPACE, + 'get', + 'pods', + '-l', + `app.kubernetes.io/name=${WORKLOAD}`, + ]).items; + const currentPods = pods.filter( + (pod) => + pod.metadata?.annotations?.[ + 'qinglong.io/plugin-package-lock-digest' + ] === expectedLockDigest && + pod.metadata?.annotations?.[ + 'qinglong.io/plugin-package-secret-projection-digest' + ] === expectedProjectionDigest && + pod.status?.phase === 'Running' && + pod.status?.conditions?.some( + (condition) => + condition.type === 'Ready' && condition.status === 'True', + ), + ); + const generation = deployment.metadata?.generation ?? 0; + const ready = + generation >= minimumGeneration && + deployment.status?.observedGeneration === generation && + deployment.status?.updatedReplicas === WORKLOAD_REPLICAS && + deployment.status?.readyReplicas === WORKLOAD_REPLICAS && + deployment.status?.availableReplicas === WORKLOAD_REPLICAS && + currentPods.length === WORKLOAD_REPLICAS && + new Set(currentPods.map((pod) => pod.spec?.nodeName)).size === + WORKLOAD_REPLICAS; + return ready + ? { ready: true, value: { deployment, pods: currentPods } } + : { + ready: false, + fact: JSON.stringify({ + generation, + observedGeneration: deployment.status?.observedGeneration, + updatedReplicas: deployment.status?.updatedReplicas, + readyReplicas: deployment.status?.readyReplicas, + availableReplicas: deployment.status?.availableReplicas, + currentPods: currentPods.length, + }), + }; + }, + ); + return observed.value; +} + +function inspectWorkloadPod(fixture, pod, expectedPath) { + const source = expectedPath + ? ` + const fs = require('node:fs'); + const root = ${JSON.stringify(SECRET_ROOT)}; + const expected = ${JSON.stringify(expectedPath)}; + const files = fs.readdirSync(root).filter((name) => !name.startsWith('..')).sort(); + const value = fs.readFileSync(root + '/' + expected, 'utf8'); + const mode = fs.statSync(root + '/' + expected).mode & 0o777; + process.stdout.write(JSON.stringify({ files, valueMatches: value === ${JSON.stringify(SECRET_MARKER)}, mode })); + ` + : ` + const fs = require('node:fs'); + process.stdout.write(JSON.stringify({ rootAbsent: !fs.existsSync(${JSON.stringify(SECRET_ROOT)}) })); + `; + return JSON.parse( + fixture.kubectl( + ['-n', NAMESPACE, 'exec', pod.metadata.name, '--', 'node', '-e', source], + { capture: true, quiet: true }, + ).stdout, + ); +} + async function main() { if (process.env.QL3_PLUGIN_PACKAGE_K3S_LIVE !== '1') { throw new Error('refusing to run without QL3_PLUGIN_PACKAGE_K3S_LIVE=1'); } + ({ + pluginPackageKubernetesProjectedSecretWorkloadVolume: renderWorkloadVolume, + } = require('../packages/ql3-cluster-admin/dist/plugin-package/recovery/pluginPackageKubernetesActivation.js')); const fixture = new K3sDockerLiveFixture({ prefix: 'ql3-plugin-v3-live', kubectl: @@ -256,15 +484,63 @@ async function main() { crossNamespaceRead: 403, }); assert.deepEqual(loser.rbac, winner.rbac); - const pointers = fixture.kubectlJson([ + const rotated = activePointer(fixture); + const projectedPath = rotated.pointer.secretProjection.items[0].path; + const secret = sourceSecret(rotated.pointer.secretProjection); + fixture.apply(secret.document); + fixture.apply(workloadDeployment(rotated)); + const mounted = await workloadReady(fixture, rotated); + const mountedInspection = mounted.pods.map((pod) => + inspectWorkloadPod(fixture, pod, projectedPath), + ); + for (const inspection of mountedInspection) { + assert.deepEqual(inspection.files, [projectedPath]); + assert.equal(inspection.valueMatches, true); + assert.equal(inspection.mode, 0o440); + assert.equal(inspection.files.includes(secret.decoy), false); + } + + fixture.apply(actorPod('c')); + const revoker = await actorResult(fixture, 'c'); + assert.equal(revoker.mode, 'revoke'); + assert.equal(revoker.cas.status, 'fulfilled'); + assert.equal(revoker.final.pointerSchema.endsWith('@v3'), true); + assert.equal(revoker.final.projectionItemCount, 0); + assert.equal(revoker.final.projectedWorkloadVolume, false); + assert.deepEqual(revoker.rbac, winner.rbac); + const revoked = activePointer(fixture); + assert.equal(revoked.pointer.secretProjection.items.length, 0); + assert.equal( + revoked.pointer.secretProjection.transitionReceiptDigest, + revoker.final.transitionReceiptDigest, + ); + fixture.apply(workloadDeployment(revoked)); + const unmounted = await workloadReady( + fixture, + revoked, + (mounted.deployment.metadata?.generation ?? 1) + 1, + ); + assert.equal( + unmounted.pods.some((pod) => + mounted.pods.some((old) => old.metadata.uid === pod.metadata.uid), + ), + false, + ); + const revokedInspection = unmounted.pods.map((pod) => + inspectWorkloadPod(fixture, pod, null), + ); + assert.equal( + revokedInspection.every((inspection) => inspection.rootAbsent === true), + true, + ); + const retainedSource = fixture.kubectlJson([ '-n', NAMESPACE, 'get', - 'configmaps', - '-l', - 'qinglong.io/plugin-package-active=v3', - ]).items; - assert.equal(pointers.length, 1); + 'secret', + rotated.pointer.secretProjection.sourceSecretName, + ]); + assert.equal(Object.keys(retainedSource.data).length, 2); process.stdout.write( `${JSON.stringify( { @@ -283,7 +559,15 @@ async function main() { transitionReceiptDigest: winner.final.transitionReceiptDigest, exactProjectionItems: winner.final.projectionItemCount, secretApiReadDenied: winner.rbac.readSecret === 403, - activePointers: pointers.length, + activePointers: 1, + workloadReplicas: WORKLOAD_REPLICAS, + workloadNodes: new Set( + unmounted.pods.map((pod) => pod.spec.nodeName), + ).size, + revokeProjectionItems: + revoked.pointer.secretProjection.items.length, + revokeTransitionReceiptDigest: + revoker.final.transitionReceiptDigest, }, gates: { realThreeNodeKubernetes: true, @@ -291,6 +575,13 @@ async function main() { v3TransitionReceiptBound: true, exactSecretProjectionRendered: true, secretApiReadDenied: true, + exactItemMountedByRealWorkload: true, + unprojectedSecretKeyAbsent: true, + workloadReplicasOnDistinctNodes: true, + revokeReceiptBound: true, + revokeRolledNewPods: true, + revokedWorkloadHasNoSecretMount: true, + sourceSecretRetainedButInaccessible: true, passed: true, }, elapsedMs: Date.now() - startedAt, diff --git a/scripts/ql3-plugin-package-kubernetes-live-actor.cjs b/scripts/ql3-plugin-package-kubernetes-live-actor.cjs index 455b5037..4c641fb2 100644 --- a/scripts/ql3-plugin-package-kubernetes-live-actor.cjs +++ b/scripts/ql3-plugin-package-kubernetes-live-actor.cjs @@ -37,7 +37,7 @@ const TOKEN_FILE = '/var/run/secrets/kubernetes.io/serviceaccount/token'; const RESULT_SCHEMA = 'qinglong/plugin-package-kubernetes-live-actor-result@v1'; const NAMESPACE = process.env.QL3_LIVE_NAMESPACE; const ACTOR = process.env.QL3_LIVE_ACTOR; -const PEER = ACTOR === 'a' ? 'b' : 'a'; +const PEER = ACTOR === 'a' ? 'b' : ACTOR === 'b' ? 'a' : null; const INITIAL_LOCK_DIGEST = 'a'.repeat(64); const CANDIDATES = Object.freeze({ a: Object.freeze({ @@ -56,6 +56,14 @@ const CANDIDATES = Object.freeze({ contentDigest: '9'.repeat(64), intentDigest: '0'.repeat(64), }), + c: Object.freeze({ + installationId: 'install-live-revoke', + lockDigest: 'f'.repeat(64), + stageReceiptDigest: 'a'.repeat(64), + stageEvidenceDigest: 'b'.repeat(64), + contentDigest: 'c'.repeat(64), + intentDigest: 'd'.repeat(64), + }), }); function fail(message) { @@ -171,10 +179,10 @@ function manifest(version, secrets) { }); } -function transitionEvidence(initial, candidate) { +function transitionEvidence(initial, candidate, actor = ACTOR) { const secretRef = createSecretRef({ projectId: 'default', - name: `live-token-${ACTOR}`, + name: `live-token-${actor}`, version: 2, }); const previousManifest = manifest('1.0.0', []); @@ -211,6 +219,57 @@ function transitionEvidence(initial, candidate) { return Object.freeze({ secretRef, binding, receipt }); } +function revokeEvidence(activeCandidate, activeTransition, candidate) { + const plan = createPluginPackageSecretBindingTransitionPlan({ + previousTarget: createPluginPackageSecretBindingTarget( + activeCandidate.resourceGeneration, + manifest('2.0.0', [ + Object.freeze({ name: 'TOKEN', required: true }), + ]), + ), + previousBinding: activeTransition.binding, + previousAttemptGeneration: 2, + nextGeneration: candidate.resourceGeneration, + nextManifest: manifest('3.0.0', []), + assignments: [], + plannedAtMs: 300, + }); + const receipt = createPluginPackageSecretBindingTransitionReceipt({ + transitionPlan: plan, + authority: Object.freeze({ + kind: 'approved-action-execution', + evidenceDigest: candidate.stageEvidenceDigest, + }), + binding: null, + committedAtMs: 400, + }); + return Object.freeze({ binding: null, receipt }); +} + +function activeTargetName() { + return ( + 'ql3p-' + + require('node:crypto') + .createHash('sha256') + .update( + Buffer.from('qinglong/plugin-package-kubernetes-target@v1\0', 'utf8'), + ) + .update( + require('node:crypto') + .createHash('sha256') + .update('qinglong/plugin-package-kubernetes-cluster@v1\0', 'utf8') + .update('ql3-plugin-package-live-cluster', 'utf8') + .digest('hex'), + 'utf8', + ) + .update('\0', 'utf8') + .update(NAMESPACE, 'utf8') + .update('\0default\0live-cas-package', 'utf8') + .digest('hex') + .slice(0, 52) + ); +} + function exactEvidence(intent) { return Object.freeze({ lockDigest: intent.lockDigest, @@ -252,8 +311,8 @@ async function main() { if (!/^[a-z0-9](?:[-a-z0-9]{0,61}[a-z0-9])?$/.test(NAMESPACE ?? '')) { fail('QL3_LIVE_NAMESPACE is invalid'); } - if (ACTOR !== 'a' && ACTOR !== 'b') { - fail('QL3_LIVE_ACTOR must be a or b'); + if (ACTOR !== 'a' && ACTOR !== 'b' && ACTOR !== 'c') { + fail('QL3_LIVE_ACTOR must be a, b or c'); } const token = fs.readFileSync(TOKEN_FILE); assert.ok(token.length >= 32 && token.length <= 16 * 1024); @@ -286,29 +345,34 @@ async function main() { replaceCalls += 1; replaceResourceVersion = request.body.metadata.resourceVersion; assert.match(replaceResourceVersion ?? '', /^[1-9][0-9]*$/); - const ownReadyName = `ql3-live-cas-ready-${ACTOR}`; - const peerReadyName = `ql3-live-cas-ready-${PEER}`; - await rawApi.createNamespacedConfigMap({ - namespace: NAMESPACE, - body: readyConfigMap(ownReadyName), - fieldManager: 'qinglong-plugin-package-live-gate', - fieldValidation: 'Strict', - }); - await waitFor(`peer CAS barrier ConfigMap/${peerReadyName}`, async () => { - try { - const peer = await rawApi.readNamespacedConfigMap({ - namespace: NAMESPACE, - name: peerReadyName, - }); - return { - ready: peer.data?.actor === PEER, - fact: `peer actor=${String(peer.data?.actor)}`, - }; - } catch (error) { - if (apiStatus(error) === 404) return { ready: false }; - throw error; - } - }); + if (PEER !== null) { + const ownReadyName = `ql3-live-cas-ready-${ACTOR}`; + const peerReadyName = `ql3-live-cas-ready-${PEER}`; + await rawApi.createNamespacedConfigMap({ + namespace: NAMESPACE, + body: readyConfigMap(ownReadyName), + fieldManager: 'qinglong-plugin-package-live-gate', + fieldValidation: 'Strict', + }); + await waitFor( + `peer CAS barrier ConfigMap/${peerReadyName}`, + async () => { + try { + const peer = await rawApi.readNamespacedConfigMap({ + namespace: NAMESPACE, + name: peerReadyName, + }); + return { + ready: peer.data?.actor === PEER, + fact: `peer actor=${String(peer.data?.actor)}`, + }; + } catch (error) { + if (apiStatus(error) === 404) return { ready: false }; + throw error; + } + }, + ); + } return rawApi.replaceNamespacedConfigMap(request); }, }; @@ -329,6 +393,117 @@ async function main() { }, ); + if (ACTOR === 'c') { + const initial = activationIntent(); + const activeConfigMap = await rawApi.readNamespacedConfigMap({ + namespace: NAMESPACE, + name: activeTargetName(), + }); + const activePointer = JSON.parse(activeConfigMap.data?.['active.json']); + const winnerActor = ['a', 'b'].find( + (actor) => CANDIDATES[actor].lockDigest === activePointer.intent.lockDigest, + ); + assert.ok(winnerActor); + const activeCandidate = activationIntent({ + ...CANDIDATES[winnerActor], + targetGeneration: 2, + previousActiveLockDigest: INITIAL_LOCK_DIGEST, + }); + const activeTransition = transitionEvidence( + initial, + activeCandidate, + winnerActor, + ); + assert.equal( + activePointer.secretProjection.bindingDigest, + activeTransition.binding.bindingDigest, + ); + const candidate = activationIntent({ + ...CANDIDATES.c, + targetGeneration: 3, + previousActiveLockDigest: activeCandidate.lockDigest, + }); + const transition = revokeEvidence( + activeCandidate, + activeTransition, + candidate, + ); + const publisher = createPublisher({ + sourceSecretName: 'ql3-cluster-plugin-package-values', + bindings: { find: async () => null }, + transitions: { find: async () => transition.receipt }, + }); + const receipt = await publisher.publish(candidate); + assert.equal(replaceCalls, 1); + const active = await publisher.findActiveDeployment( + 'default', + 'live-cas-package', + ); + assert.ok(active); + assert.equal(active.resourceGeneration.lockDigest, candidate.lockDigest); + assert.equal(active.secretProjection.items.length, 0); + assert.equal( + pluginPackageKubernetesProjectedSecretWorkloadVolume( + active.secretProjection, + ), + null, + ); + const rbac = Object.freeze({ + listConfigMaps: await expectForbidden(() => + rawApi.listNamespacedConfigMap({ namespace: NAMESPACE }), + ), + deleteConfigMap: await expectForbidden(() => + rawApi.deleteNamespacedConfigMap({ + namespace: NAMESPACE, + name: activeTargetName(), + }), + ), + readSecret: await expectForbidden(() => + rawApi.readNamespacedSecret({ + namespace: NAMESPACE, + name: 'forbidden-secret', + }), + ), + crossNamespaceRead: await expectForbidden(() => + rawApi.readNamespacedConfigMap({ + namespace: 'default', + name: 'kube-root-ca.crt', + }), + ), + }); + const result = JSON.stringify({ + schema: RESULT_SCHEMA, + actor: ACTOR, + mode: 'revoke', + serviceAccountTokenMounted: true, + responseLoss: null, + cas: { + status: 'fulfilled', + receipt, + attemptedResourceVersion: replaceResourceVersion, + replaceCalls, + }, + final: { + resourceVersion: activeConfigMap.metadata.resourceVersion, + lockDigest: candidate.lockDigest, + generation: receipt.generation, + pointerSchema: 'qinglong/plugin-package-kubernetes-active-pointer@v3', + projectionDigest: active.secretProjection.projectionDigest, + transitionReceiptDigest: + active.secretProjection.transitionReceiptDigest, + projectionItemCount: active.secretProjection.items.length, + projectedWorkloadVolume: false, + }, + rbac, + }); + fs.writeFileSync('/dev/termination-log', result, { + encoding: 'utf8', + flag: 'w', + }); + process.stdout.write(`${result}\n`); + return; + } + const initial = activationIntent(); const initialPublisher = createPublisher(); let responseLoss = null; @@ -386,26 +561,7 @@ async function main() { } assert.equal(replaceCalls, 1); - const targetName = - 'ql3p-' + - require('node:crypto') - .createHash('sha256') - .update( - Buffer.from('qinglong/plugin-package-kubernetes-target@v1\0', 'utf8'), - ) - .update( - require('node:crypto') - .createHash('sha256') - .update('qinglong/plugin-package-kubernetes-cluster@v1\0', 'utf8') - .update('ql3-plugin-package-live-cluster', 'utf8') - .digest('hex'), - 'utf8', - ) - .update('\0', 'utf8') - .update(NAMESPACE, 'utf8') - .update('\0default\0live-cas-package', 'utf8') - .digest('hex') - .slice(0, 52); + const targetName = activeTargetName(); const finalConfigMap = await rawApi.readNamespacedConfigMap({ namespace: NAMESPACE, name: targetName, @@ -458,6 +614,7 @@ async function main() { const result = JSON.stringify({ schema: RESULT_SCHEMA, actor: ACTOR, + mode: 'rotate', serviceAccountTokenMounted: true, responseLoss, cas: { diff --git a/test/back/ql3PluginPackageKubernetesLiveContract.test.cjs b/test/back/ql3PluginPackageKubernetesLiveContract.test.cjs index 54fb5a49..2fc9b050 100644 --- a/test/back/ql3PluginPackageKubernetesLiveContract.test.cjs +++ b/test/back/ql3PluginPackageKubernetesLiveContract.test.cjs @@ -16,8 +16,13 @@ const actorFile = path.join( root, 'scripts/ql3-plugin-package-kubernetes-live-actor.cjs', ); +const k3sHostFile = path.join( + root, + 'scripts/ql3-plugin-package-kubernetes-k3s-live-contract.cjs', +); const hostSource = fs.readFileSync(hostFile, 'utf8'); const actorSource = fs.readFileSync(actorFile, 'utf8'); +const k3sHostSource = fs.readFileSync(k3sHostFile, 'utf8'); const packageJson = JSON.parse( fs.readFileSync(path.join(root, 'package.json'), 'utf8'), ); @@ -108,3 +113,32 @@ test('CI runs the dedicated live gate without reusing another cluster', () => { }); assert.equal(liveStep.env.QL3_REUSE_KIND_CLUSTER, undefined); }); + +test('three-node K3s gate proves exact workload projection and receipt-bound revoke', () => { + const result = spawnSync(process.execPath, [k3sHostFile], { + cwd: root, + encoding: 'utf8', + }); + assert.equal(result.status, 1); + assert.match( + result.stderr, + /refusing to run without QL3_PLUGIN_PACKAGE_K3S_LIVE=1/, + ); + assert.equal( + packageJson.scripts['test:plugin-package-kubernetes-k3s-live:ql3'], + 'pnpm --filter @qinglong/cluster-admin build && node scripts/ql3-plugin-package-kubernetes-k3s-live-contract.cjs', + ); + assert.match(k3sHostSource, /fixture\.apply\(actorPod\('c'\)\)/); + assert.match(k3sHostSource, /maxUnavailable: 0, maxSurge: 1/); + assert.match(k3sHostSource, /requiredDuringSchedulingIgnoredDuringExecution/); + assert.match(k3sHostSource, /sourceSecret\(rotated\.pointer\.secretProjection\)/); + assert.match(k3sHostSource, /assert\.deepEqual\(inspection\.files, \[projectedPath\]\)/); + assert.match(k3sHostSource, /revokedWorkloadHasNoSecretMount: true/); + assert.match(k3sHostSource, /sourceSecretRetainedButInaccessible: true/); + assert.match(actorSource, /mode: 'revoke'/); + assert.match(actorSource, /active\.secretProjection\.items\.length, 0/); + assert.match( + actorSource, + /pluginPackageKubernetesProjectedSecretWorkloadVolume\([\s\S]*active\.secretProjection/, + ); +});