test(ql3): prove kubernetes secret revoke rollout

This commit is contained in:
whyour
2026-08-14 01:08:18 +08:00
parent 9d7431c2cd
commit 53c150c4ec
4 changed files with 539 additions and 57 deletions
+2 -2
View File
@@ -6,7 +6,7 @@
- 目标版本:QingLong 3.x
- 作者:QingLong Maintainers
- 创建日期:2026-07-17
- 最后更新:2026-08-13
- 最后更新:2026-08-14
- 讨论范围:架构与演进路线,不包含最终 UI 视觉方案
最新增量证据(2026-08-13):
@@ -29,7 +29,7 @@
- D-305/ADR-0393(已接受):Plugin Package Manifest 的逻辑 Secret requirement 获得按 resource generation 固定的不可变 binding。binding 精确覆盖 Manifest requirements,只保存同 Project、显式 version 的 `qlsecret://` 引用与 `approved-action-execution|local-owner-confirmation` authority evidence digest,不保存 Secret 明文;required 不可为空,optional 可显式为 `null`。发布由当前 active installation head、lock、generation 与 Manifest digest 联合 fencing,相同事实幂等、不同事实冲突;domain-separated digest、最多 64 项和 64 KiB 单行预算同时约束 Local 与 Cluster。SQLite 追加 `0091` ledger 与 capability v46PostgreSQL 追加 `pg-0059`、capability v58,并只向 package executor 授予 `SELECT, INSERT`。不新增 package、daemon、timer、watcher、连接、缓存或集群 workload,低配路由设备只承担一个有界表和三个索引。D-305 不冒充 Secret 已进入执行路径:现有 materialization 拒绝仍保留,D-306 再完成用户授权、Secret resolution、runtime consumption 与 lifecycle/rebinding 语义。core 509/509、SQLite 232/232、PostgreSQL 316 pass/1 条件 skip;完整 18-package clean build/test 退出 0backend 1,188 pass/2 skip,五项边界审计零 findingworkspace 仍无 single-source/shallow-source package。PostgreSQL 18.4 arm64 HA 125 项 gate 全绿、timeline `1→2`,报告 SHA-256 为 `acf0fea7ca7699989dfe70f5dd0061cdf5fb1968c691094331fea06ce01b96dc`
- D-306A/ADR-0394(已接受):Package Task source 以 `package-secret` placeholder 引用逻辑 requirementmaterialization 只用当前 generation 的 D-305 binding 编译为已有、固定 version 的 Task `SecretRef`Package source 直接携带 SecretRef、缺失 binding、未批准 `secret.use`、跨 binding 引用和 optional/required 漂移全部失败关闭。binding 快照不含明文并进入 materialized revision digestLocal/Cluster 启动发布复用既有 repository/poolTask dispatch、Local 短时解密和 Cluster offer/lease-fenced delivery 不另造协议。SQLite/PostgreSQL INSERT trigger 同时防止直接写库绕过;Local 只读 readiness 继续不加载 DDL。Local contract v47、Cluster v59;不新增 package、表、索引、连接、daemon、watcher、timer、cache 或 workload。完整 18-package clean build/test 退出 0backend 1,188 pass/2 条件 skip/0 fail;五项 package/dependency/edge/service-manager/local-image 审计零 findingworkspace 仍无 single-source/shallow-source package,两个有序 migration ledger 精确为 PostgreSQL 61、SQLite 95 个 sourcePostgreSQL 18.4 arm64 HA 125 项 gate 全绿、timeline `1→2`,报告 SHA-256 为 `f9107e8e54892a788779758f0573ac8d6a80f6d086516a1f5f5bbacb59bbb4be`。D-306A 不冒充产品闭环:Local bind/rebind 命令、Cluster Approved Action/API 与新 generation rotation/revocation 编排属于 D-306B。
- D-306B1/ADR-0395(已接受):当前 active、尚未绑定 Package generation 的首次 Secret binding 已形成 Local 与 Cluster 产品闭环,且不允许原地 rebind。共享 content-free plan 由服务端从 installation/proposal/lock/Manifest/generation 重建;Local 使用短生命周期 `ql3-package`、Owner human confirmation 与单 SQLite transactionCluster 使用既有 package-management HTTPS/CLI、package-manager separation-of-duty Approval 和短生命周期 package-executor。三节点 K3s `v1.34.3+k3s1` arm64 现场门已在真实 PostgreSQL `18.4` 上完成两个 management Pod 跨节点部署、正式 client `plan→跨副本 replay→propose→双人 decide→inspect`、真实 executor Job 与只读 Kubernetes Secret projection。management/executor 均无 Secret API 读取权和 ServiceAccount tokenmanagement 不挂载 Package valueexecutor 只验证投影元数据;最终恰好一条 immutable bindingApproval consumed、execution succeeded,数据库敏感值扫描为 0。16/16 gate 的 owner-private、低敏报告通过独立 exact-shape 审计,SHA-256 为 `aaabb5ebea77c50bce671f91dd3051671fd20875c11a8f787fe8933f29dbfa4d`。完整 18-package clean build/test、backend 与七项边界审计,以及 PostgreSQL 18.4 physical HA 125 gate/timeline `1→2` 证据继续有效;没有新增 workspace package、migration、表、索引、依赖或常驻 workload。B2 rebind/rotation/revocation 必须通过新 Package generation 独立推进。
- D-306B2/ADR-0396(进行中):Secret rebind/rotation/revocation 不更新历史 binding,而是作为下一 Package generation 的 activation 前置事实。共享 transition plan v1 同时绑定上一 active target、可选的上一 binding、durable install history 的最后尝试 generation、新 target、可选下一 binding plan、逐 requirement 与 SecretRef 差异及独立 digest;上一 active Manifest 没有 Secret requirement 时 binding 可空,但 target/lock/generation lineage 不可省略。失败 install 也永久消耗 generation,重试必须使用 `lastAttemptGeneration + 1`active lineage 继续由 `previousActiveLockDigest` 指回旧代。SQLite capability v490097/0098)已完成 immutable transition receipt ledger、Local Owner staged `plan→execute`、单事务 binding/audit/receipt 和 activation prerequisite。PostgreSQL capability v62pg-0063)具有 receipt ledgerCluster management 与 package-executor 也已完成 separation-of-duty 产品编排:executor 在一个 SERIALIZABLE transaction 中复验 current staged head、上一 active lineage、durable 最大 generation 与可选上一 binding,并原子提交可选目标 binding 和 immutable receipt;数据库 trigger 和最小角色 readiness 防止绕过,recovery/直接 activation 缺 receipt 均失败关闭。Cluster startup recovery 现把 binding/transition receipt 编译为 content-blind Kubernetes active pointer v3:只保存 source Secret 名、不可逆 SHA-256 key/path、`0440`、binding/receipt/projection digest 和逻辑 assignment,不保存 SecretRef 或明文;同一次 ConfigMap `resourceVersion` CAS 同时切换 Package generation 与投影声明。rotate 只投影下一代 exact keyrevoke 生成显式空 projectionPod volume renderer 对空项返回不挂载,避免 Kubernetes 空 `items` 被解释为投影全部 key。无 Secret generation 继续发布兼容 v2publisher 不获得 Secret `get/list`,不新增 watcher/controller。响应丢失通过 durable pointer inspect 精确收敛,projection source 不可用或 digest 漂移时旧 active pointer 保持不变。真实三节点 K3s `v1.34.3+k3s1` 已完成两个受限 actor 同一 resourceVersion 的 v3 rotation 竞争:1 成功/1 冲突、最终恰好 1 pointer、1 个 exact projection item、Secret API read deniedprojection digest `22add8965accf3f736935167963b9dbdeab8fba05f739f24d39dec941aae9680`、transition receipt digest `355b89ecd54422af33fa573780c8b70ed41da4df226ec301bdd5b6c71de609e1`;临时容器/网络已清理。实现没有新增 workspace packageSecret projection/renderer 内聚在既有 `cluster-admin/plugin-package/secret-binding`18-package boundary 仍为 `singleSourcePackages=[]``shallowSourcePackages=[]`cluster dependency 与 edge import 审计无 finding。上一阶段真实 PostgreSQL 18.4 的 63 条 migration、原子提交、exact replay、错误状态/lineage 拒绝,以及 physical HA 125 gate/timeline `1→2` 证据继续有效。当前增量的完整 18-package/PG/HA 回归、真实 Kubernetes revoke 工作负载、升级后业务 Pod rollout/失败回滚和低配物理证据仍待完成。
- D-306B2/ADR-0396(进行中):Secret rebind/rotation/revocation 不更新历史 binding,而是作为下一 Package generation 的 activation 前置事实。共享 transition plan v1 同时绑定上一 active target、可选的上一 binding、durable install history 的最后尝试 generation、新 target、可选下一 binding plan、逐 requirement 与 SecretRef 差异及独立 digest;上一 active Manifest 没有 Secret requirement 时 binding 可空,但 target/lock/generation lineage 不可省略。失败 install 也永久消耗 generation,重试必须使用 `lastAttemptGeneration + 1`active lineage 继续由 `previousActiveLockDigest` 指回旧代。SQLite capability v490097/0098)已完成 immutable transition receipt ledger、Local Owner staged `plan→execute`、单事务 binding/audit/receipt 和 activation prerequisite。PostgreSQL capability v62pg-0063)具有 receipt ledgerCluster management 与 package-executor 也已完成 separation-of-duty 产品编排:executor 在一个 SERIALIZABLE transaction 中复验 current staged head、上一 active lineage、durable 最大 generation 与可选上一 binding,并原子提交可选目标 binding 和 immutable receipt;数据库 trigger 和最小角色 readiness 防止绕过,recovery/直接 activation 缺 receipt 均失败关闭。Cluster startup recovery 现把 binding/transition receipt 编译为 content-blind Kubernetes active pointer v3:只保存 source Secret 名、不可逆 SHA-256 key/path、`0440`、binding/receipt/projection digest 和逻辑 assignment,不保存 SecretRef 或明文;同一次 ConfigMap `resourceVersion` CAS 同时切换 Package generation 与投影声明。rotate 只投影下一代 exact keyrevoke 生成显式空 projectionPod volume renderer 对空项返回不挂载,避免 Kubernetes 空 `items` 被解释为投影全部 key。无 Secret generation 继续发布兼容 v2publisher 不获得 Secret `get/list`,不新增 watcher/controller。响应丢失通过 durable pointer inspect 精确收敛,projection source 不可用或 digest 漂移时旧 active pointer 保持不变。真实三节点 K3s `v1.34.3+k3s1` 已完成两个受限 actor 同一 resourceVersion 的 v3 rotation 竞争:1 成功/1 冲突、最终恰好 1 pointer、1 个 exact projection item、Secret API read deniedprojection digest `22add8965accf3f736935167963b9dbdeab8fba05f739f24d39dec941aae9680`、transition receipt digest `355b89ecd54422af33fa573780c8b70ed41da4df226ec301bdd5b6c71de609e1`随后生产 renderer 驱动 2 副本 workload 分布到 2 个节点,源 Secret 同时保留目标 key 与 decoy key,而 Pod 只看到 exact path、文件模式 `0440`。第三个同权限 actor 以 generation 3 transition receipt `252e8cd1d0f8a2b63f8c0af6247861c08e5b7ae8886c0d65cd90537be3e9f9ac` 发布显式空 projectionDeployment 滚动产生全新 Pod UID,两个新 Pod 均无 Secret volume/mount 和投影根目录,源 Secret 保留,actor 的 Secret `get` 仍为 403临时容器/网络已清理。实现没有新增 workspace packageSecret projection/renderer 内聚在既有 `cluster-admin/plugin-package/secret-binding`18-package boundary 仍为 `singleSourcePackages=[]``shallowSourcePackages=[]`cluster dependency 与 edge import 审计无 finding。阶段提交 `9d7431c2` 后,完整 18-package 串行 build/test 退出 0backend 1,192 pass/2 skip/0 failPostgreSQL 18.4 physical HA 125 gatetimeline `1→2` 通过,报告 SHA-256 为 `2f0d1107cc6d447868bfaeb3650284acd803924c6cd3cdee978b3eb5882eb26c`;Edge 本机观测的模块加载 RSS 增量约 6.0 MiB、1 万行输出峰值增量约 3.9 MiB,但尚无固定物理低配门。该 K3s workload 仍由 live harness 显式应用,不能冒充生产 rollout controller;现有 opt-in Package executor CronJob 的无 `items` optional Secret mount 也尚未收敛为 action-scoped exact projection。生产升级失败自动回滚、executor 精确投影重构和低配物理证据仍待完成。
- D-302/ADR-0390(已接受)
Cluster operator context 增加无网络、无 mutation 的内建 `ql3-cluster-admin context validate` 预检。它先复用 owner-private context
reader,再让每个 entry 经过与真实请求相同的 production HTTPS/Kubernetes configuration preparation,验证精确 route、hostname、CA、
@@ -21,6 +21,11 @@ const RESULT_SCHEMA = 'qinglong/plugin-package-kubernetes-live-actor-result@v1';
const NAMESPACE = 'ql3-plugin-package-live';
const SERVICE_ACCOUNT = 'ql3-plugin-package-recovery-live';
const IMAGE = 'qinglong3-cluster-admin:ql3-k3s-kubernetes-live';
const WORKLOAD = 'ql3-plugin-package-workload-live';
const SECRET_ROOT = '/var/run/secrets/qinglong3/plugin-package-values';
const WORKLOAD_REPLICAS = 2;
const SECRET_MARKER = 'ql3-live-exact-projection';
let renderWorkloadVolume = null;
function run(binary, args, options = {}) {
const result = spawnSync(binary, args, {
@@ -171,10 +176,233 @@ async function actorResult(fixture, actor) {
return value;
}
function activePointer(fixture) {
const pointers = fixture.kubectlJson([
'-n',
NAMESPACE,
'get',
'configmaps',
'-l',
'qinglong.io/plugin-package-active=v3',
]).items;
assert.equal(pointers.length, 1);
const pointer = JSON.parse(pointers[0].data['active.json']);
assert.equal(
pointer.schema,
'qinglong/plugin-package-kubernetes-active-pointer@v3',
);
return Object.freeze({ configMap: pointers[0], pointer });
}
function sourceSecret(projection) {
assert.equal(projection.items.length, 1);
const projected = projection.items[0].key;
const decoy = projected === 'f'.repeat(64) ? 'e'.repeat(64) : 'f'.repeat(64);
return Object.freeze({
decoy,
document: {
apiVersion: 'v1',
kind: 'Secret',
type: 'Opaque',
immutable: true,
metadata: {
name: projection.sourceSecretName,
namespace: NAMESPACE,
labels: {
'app.kubernetes.io/managed-by': 'qinglong3-live-gate',
'qinglong.io/live-gate-role': 'projection-source',
},
},
data: {
[projected]: Buffer.from(SECRET_MARKER, 'utf8').toString('base64'),
[decoy]: Buffer.from('ql3-live-decoy', 'utf8').toString('base64'),
},
},
});
}
function workloadDeployment(active) {
const projection = active.pointer.secretProjection;
assert.equal(typeof renderWorkloadVolume, 'function');
const rendered = renderWorkloadVolume(projection);
const labels = {
'app.kubernetes.io/name': WORKLOAD,
'app.kubernetes.io/component': 'plugin-package-workload',
};
return {
apiVersion: 'apps/v1',
kind: 'Deployment',
metadata: {
name: WORKLOAD,
namespace: NAMESPACE,
labels,
},
spec: {
replicas: WORKLOAD_REPLICAS,
revisionHistoryLimit: 2,
progressDeadlineSeconds: 60,
strategy: {
type: 'RollingUpdate',
rollingUpdate: { maxUnavailable: 0, maxSurge: 1 },
},
selector: { matchLabels: labels },
template: {
metadata: {
labels,
annotations: {
'qinglong.io/plugin-package-generation-digest':
active.pointer.intent.resourceGeneration.generationDigest,
'qinglong.io/plugin-package-lock-digest':
active.pointer.intent.lockDigest,
'qinglong.io/plugin-package-secret-projection-digest':
projection?.projectionDigest ?? 'none',
},
},
spec: {
automountServiceAccountToken: false,
securityContext: {
runAsNonRoot: true,
runAsUser: 10001,
runAsGroup: 10001,
fsGroup: 10001,
seccompProfile: { type: 'RuntimeDefault' },
},
affinity: {
podAntiAffinity: {
requiredDuringSchedulingIgnoredDuringExecution: [
{
labelSelector: { matchLabels: labels },
topologyKey: 'kubernetes.io/hostname',
},
],
},
},
containers: [
{
name: 'workload',
image: IMAGE,
imagePullPolicy: 'Never',
command: [
'node',
'-e',
'setInterval(() => {}, 2147483647)',
],
securityContext: {
allowPrivilegeEscalation: false,
readOnlyRootFilesystem: true,
capabilities: { drop: ['ALL'] },
},
resources: {
requests: { cpu: '10m', memory: '32Mi' },
limits: { cpu: '250m', memory: '128Mi' },
},
...(rendered
? { volumeMounts: [rendered.volumeMount] }
: {}),
},
],
...(rendered ? { volumes: [rendered.volume] } : {}),
},
},
},
};
}
async function workloadReady(fixture, active, minimumGeneration = 1) {
const expectedLockDigest = active.pointer.intent.lockDigest;
const expectedProjectionDigest =
active.pointer.secretProjection?.projectionDigest ?? 'none';
const observed = await waitFor(
`Deployment/${WORKLOAD} rollout`,
120_000,
() => {
const deployment = fixture.kubectlJson([
'-n',
NAMESPACE,
'get',
'deployment',
WORKLOAD,
]);
const pods = fixture.kubectlJson([
'-n',
NAMESPACE,
'get',
'pods',
'-l',
`app.kubernetes.io/name=${WORKLOAD}`,
]).items;
const currentPods = pods.filter(
(pod) =>
pod.metadata?.annotations?.[
'qinglong.io/plugin-package-lock-digest'
] === expectedLockDigest &&
pod.metadata?.annotations?.[
'qinglong.io/plugin-package-secret-projection-digest'
] === expectedProjectionDigest &&
pod.status?.phase === 'Running' &&
pod.status?.conditions?.some(
(condition) =>
condition.type === 'Ready' && condition.status === 'True',
),
);
const generation = deployment.metadata?.generation ?? 0;
const ready =
generation >= minimumGeneration &&
deployment.status?.observedGeneration === generation &&
deployment.status?.updatedReplicas === WORKLOAD_REPLICAS &&
deployment.status?.readyReplicas === WORKLOAD_REPLICAS &&
deployment.status?.availableReplicas === WORKLOAD_REPLICAS &&
currentPods.length === WORKLOAD_REPLICAS &&
new Set(currentPods.map((pod) => pod.spec?.nodeName)).size ===
WORKLOAD_REPLICAS;
return ready
? { ready: true, value: { deployment, pods: currentPods } }
: {
ready: false,
fact: JSON.stringify({
generation,
observedGeneration: deployment.status?.observedGeneration,
updatedReplicas: deployment.status?.updatedReplicas,
readyReplicas: deployment.status?.readyReplicas,
availableReplicas: deployment.status?.availableReplicas,
currentPods: currentPods.length,
}),
};
},
);
return observed.value;
}
function inspectWorkloadPod(fixture, pod, expectedPath) {
const source = expectedPath
? `
const fs = require('node:fs');
const root = ${JSON.stringify(SECRET_ROOT)};
const expected = ${JSON.stringify(expectedPath)};
const files = fs.readdirSync(root).filter((name) => !name.startsWith('..')).sort();
const value = fs.readFileSync(root + '/' + expected, 'utf8');
const mode = fs.statSync(root + '/' + expected).mode & 0o777;
process.stdout.write(JSON.stringify({ files, valueMatches: value === ${JSON.stringify(SECRET_MARKER)}, mode }));
`
: `
const fs = require('node:fs');
process.stdout.write(JSON.stringify({ rootAbsent: !fs.existsSync(${JSON.stringify(SECRET_ROOT)}) }));
`;
return JSON.parse(
fixture.kubectl(
['-n', NAMESPACE, 'exec', pod.metadata.name, '--', 'node', '-e', source],
{ capture: true, quiet: true },
).stdout,
);
}
async function main() {
if (process.env.QL3_PLUGIN_PACKAGE_K3S_LIVE !== '1') {
throw new Error('refusing to run without QL3_PLUGIN_PACKAGE_K3S_LIVE=1');
}
({
pluginPackageKubernetesProjectedSecretWorkloadVolume: renderWorkloadVolume,
} = require('../packages/ql3-cluster-admin/dist/plugin-package/recovery/pluginPackageKubernetesActivation.js'));
const fixture = new K3sDockerLiveFixture({
prefix: 'ql3-plugin-v3-live',
kubectl:
@@ -256,15 +484,63 @@ async function main() {
crossNamespaceRead: 403,
});
assert.deepEqual(loser.rbac, winner.rbac);
const pointers = fixture.kubectlJson([
const rotated = activePointer(fixture);
const projectedPath = rotated.pointer.secretProjection.items[0].path;
const secret = sourceSecret(rotated.pointer.secretProjection);
fixture.apply(secret.document);
fixture.apply(workloadDeployment(rotated));
const mounted = await workloadReady(fixture, rotated);
const mountedInspection = mounted.pods.map((pod) =>
inspectWorkloadPod(fixture, pod, projectedPath),
);
for (const inspection of mountedInspection) {
assert.deepEqual(inspection.files, [projectedPath]);
assert.equal(inspection.valueMatches, true);
assert.equal(inspection.mode, 0o440);
assert.equal(inspection.files.includes(secret.decoy), false);
}
fixture.apply(actorPod('c'));
const revoker = await actorResult(fixture, 'c');
assert.equal(revoker.mode, 'revoke');
assert.equal(revoker.cas.status, 'fulfilled');
assert.equal(revoker.final.pointerSchema.endsWith('@v3'), true);
assert.equal(revoker.final.projectionItemCount, 0);
assert.equal(revoker.final.projectedWorkloadVolume, false);
assert.deepEqual(revoker.rbac, winner.rbac);
const revoked = activePointer(fixture);
assert.equal(revoked.pointer.secretProjection.items.length, 0);
assert.equal(
revoked.pointer.secretProjection.transitionReceiptDigest,
revoker.final.transitionReceiptDigest,
);
fixture.apply(workloadDeployment(revoked));
const unmounted = await workloadReady(
fixture,
revoked,
(mounted.deployment.metadata?.generation ?? 1) + 1,
);
assert.equal(
unmounted.pods.some((pod) =>
mounted.pods.some((old) => old.metadata.uid === pod.metadata.uid),
),
false,
);
const revokedInspection = unmounted.pods.map((pod) =>
inspectWorkloadPod(fixture, pod, null),
);
assert.equal(
revokedInspection.every((inspection) => inspection.rootAbsent === true),
true,
);
const retainedSource = fixture.kubectlJson([
'-n',
NAMESPACE,
'get',
'configmaps',
'-l',
'qinglong.io/plugin-package-active=v3',
]).items;
assert.equal(pointers.length, 1);
'secret',
rotated.pointer.secretProjection.sourceSecretName,
]);
assert.equal(Object.keys(retainedSource.data).length, 2);
process.stdout.write(
`${JSON.stringify(
{
@@ -283,7 +559,15 @@ async function main() {
transitionReceiptDigest: winner.final.transitionReceiptDigest,
exactProjectionItems: winner.final.projectionItemCount,
secretApiReadDenied: winner.rbac.readSecret === 403,
activePointers: pointers.length,
activePointers: 1,
workloadReplicas: WORKLOAD_REPLICAS,
workloadNodes: new Set(
unmounted.pods.map((pod) => pod.spec.nodeName),
).size,
revokeProjectionItems:
revoked.pointer.secretProjection.items.length,
revokeTransitionReceiptDigest:
revoker.final.transitionReceiptDigest,
},
gates: {
realThreeNodeKubernetes: true,
@@ -291,6 +575,13 @@ async function main() {
v3TransitionReceiptBound: true,
exactSecretProjectionRendered: true,
secretApiReadDenied: true,
exactItemMountedByRealWorkload: true,
unprojectedSecretKeyAbsent: true,
workloadReplicasOnDistinctNodes: true,
revokeReceiptBound: true,
revokeRolledNewPods: true,
revokedWorkloadHasNoSecretMount: true,
sourceSecretRetainedButInaccessible: true,
passed: true,
},
elapsedMs: Date.now() - startedAt,
@@ -37,7 +37,7 @@ const TOKEN_FILE = '/var/run/secrets/kubernetes.io/serviceaccount/token';
const RESULT_SCHEMA = 'qinglong/plugin-package-kubernetes-live-actor-result@v1';
const NAMESPACE = process.env.QL3_LIVE_NAMESPACE;
const ACTOR = process.env.QL3_LIVE_ACTOR;
const PEER = ACTOR === 'a' ? 'b' : 'a';
const PEER = ACTOR === 'a' ? 'b' : ACTOR === 'b' ? 'a' : null;
const INITIAL_LOCK_DIGEST = 'a'.repeat(64);
const CANDIDATES = Object.freeze({
a: Object.freeze({
@@ -56,6 +56,14 @@ const CANDIDATES = Object.freeze({
contentDigest: '9'.repeat(64),
intentDigest: '0'.repeat(64),
}),
c: Object.freeze({
installationId: 'install-live-revoke',
lockDigest: 'f'.repeat(64),
stageReceiptDigest: 'a'.repeat(64),
stageEvidenceDigest: 'b'.repeat(64),
contentDigest: 'c'.repeat(64),
intentDigest: 'd'.repeat(64),
}),
});
function fail(message) {
@@ -171,10 +179,10 @@ function manifest(version, secrets) {
});
}
function transitionEvidence(initial, candidate) {
function transitionEvidence(initial, candidate, actor = ACTOR) {
const secretRef = createSecretRef({
projectId: 'default',
name: `live-token-${ACTOR}`,
name: `live-token-${actor}`,
version: 2,
});
const previousManifest = manifest('1.0.0', []);
@@ -211,6 +219,57 @@ function transitionEvidence(initial, candidate) {
return Object.freeze({ secretRef, binding, receipt });
}
function revokeEvidence(activeCandidate, activeTransition, candidate) {
const plan = createPluginPackageSecretBindingTransitionPlan({
previousTarget: createPluginPackageSecretBindingTarget(
activeCandidate.resourceGeneration,
manifest('2.0.0', [
Object.freeze({ name: 'TOKEN', required: true }),
]),
),
previousBinding: activeTransition.binding,
previousAttemptGeneration: 2,
nextGeneration: candidate.resourceGeneration,
nextManifest: manifest('3.0.0', []),
assignments: [],
plannedAtMs: 300,
});
const receipt = createPluginPackageSecretBindingTransitionReceipt({
transitionPlan: plan,
authority: Object.freeze({
kind: 'approved-action-execution',
evidenceDigest: candidate.stageEvidenceDigest,
}),
binding: null,
committedAtMs: 400,
});
return Object.freeze({ binding: null, receipt });
}
function activeTargetName() {
return (
'ql3p-' +
require('node:crypto')
.createHash('sha256')
.update(
Buffer.from('qinglong/plugin-package-kubernetes-target@v1\0', 'utf8'),
)
.update(
require('node:crypto')
.createHash('sha256')
.update('qinglong/plugin-package-kubernetes-cluster@v1\0', 'utf8')
.update('ql3-plugin-package-live-cluster', 'utf8')
.digest('hex'),
'utf8',
)
.update('\0', 'utf8')
.update(NAMESPACE, 'utf8')
.update('\0default\0live-cas-package', 'utf8')
.digest('hex')
.slice(0, 52)
);
}
function exactEvidence(intent) {
return Object.freeze({
lockDigest: intent.lockDigest,
@@ -252,8 +311,8 @@ async function main() {
if (!/^[a-z0-9](?:[-a-z0-9]{0,61}[a-z0-9])?$/.test(NAMESPACE ?? '')) {
fail('QL3_LIVE_NAMESPACE is invalid');
}
if (ACTOR !== 'a' && ACTOR !== 'b') {
fail('QL3_LIVE_ACTOR must be a or b');
if (ACTOR !== 'a' && ACTOR !== 'b' && ACTOR !== 'c') {
fail('QL3_LIVE_ACTOR must be a, b or c');
}
const token = fs.readFileSync(TOKEN_FILE);
assert.ok(token.length >= 32 && token.length <= 16 * 1024);
@@ -286,29 +345,34 @@ async function main() {
replaceCalls += 1;
replaceResourceVersion = request.body.metadata.resourceVersion;
assert.match(replaceResourceVersion ?? '', /^[1-9][0-9]*$/);
const ownReadyName = `ql3-live-cas-ready-${ACTOR}`;
const peerReadyName = `ql3-live-cas-ready-${PEER}`;
await rawApi.createNamespacedConfigMap({
namespace: NAMESPACE,
body: readyConfigMap(ownReadyName),
fieldManager: 'qinglong-plugin-package-live-gate',
fieldValidation: 'Strict',
});
await waitFor(`peer CAS barrier ConfigMap/${peerReadyName}`, async () => {
try {
const peer = await rawApi.readNamespacedConfigMap({
namespace: NAMESPACE,
name: peerReadyName,
});
return {
ready: peer.data?.actor === PEER,
fact: `peer actor=${String(peer.data?.actor)}`,
};
} catch (error) {
if (apiStatus(error) === 404) return { ready: false };
throw error;
}
});
if (PEER !== null) {
const ownReadyName = `ql3-live-cas-ready-${ACTOR}`;
const peerReadyName = `ql3-live-cas-ready-${PEER}`;
await rawApi.createNamespacedConfigMap({
namespace: NAMESPACE,
body: readyConfigMap(ownReadyName),
fieldManager: 'qinglong-plugin-package-live-gate',
fieldValidation: 'Strict',
});
await waitFor(
`peer CAS barrier ConfigMap/${peerReadyName}`,
async () => {
try {
const peer = await rawApi.readNamespacedConfigMap({
namespace: NAMESPACE,
name: peerReadyName,
});
return {
ready: peer.data?.actor === PEER,
fact: `peer actor=${String(peer.data?.actor)}`,
};
} catch (error) {
if (apiStatus(error) === 404) return { ready: false };
throw error;
}
},
);
}
return rawApi.replaceNamespacedConfigMap(request);
},
};
@@ -329,6 +393,117 @@ async function main() {
},
);
if (ACTOR === 'c') {
const initial = activationIntent();
const activeConfigMap = await rawApi.readNamespacedConfigMap({
namespace: NAMESPACE,
name: activeTargetName(),
});
const activePointer = JSON.parse(activeConfigMap.data?.['active.json']);
const winnerActor = ['a', 'b'].find(
(actor) => CANDIDATES[actor].lockDigest === activePointer.intent.lockDigest,
);
assert.ok(winnerActor);
const activeCandidate = activationIntent({
...CANDIDATES[winnerActor],
targetGeneration: 2,
previousActiveLockDigest: INITIAL_LOCK_DIGEST,
});
const activeTransition = transitionEvidence(
initial,
activeCandidate,
winnerActor,
);
assert.equal(
activePointer.secretProjection.bindingDigest,
activeTransition.binding.bindingDigest,
);
const candidate = activationIntent({
...CANDIDATES.c,
targetGeneration: 3,
previousActiveLockDigest: activeCandidate.lockDigest,
});
const transition = revokeEvidence(
activeCandidate,
activeTransition,
candidate,
);
const publisher = createPublisher({
sourceSecretName: 'ql3-cluster-plugin-package-values',
bindings: { find: async () => null },
transitions: { find: async () => transition.receipt },
});
const receipt = await publisher.publish(candidate);
assert.equal(replaceCalls, 1);
const active = await publisher.findActiveDeployment(
'default',
'live-cas-package',
);
assert.ok(active);
assert.equal(active.resourceGeneration.lockDigest, candidate.lockDigest);
assert.equal(active.secretProjection.items.length, 0);
assert.equal(
pluginPackageKubernetesProjectedSecretWorkloadVolume(
active.secretProjection,
),
null,
);
const rbac = Object.freeze({
listConfigMaps: await expectForbidden(() =>
rawApi.listNamespacedConfigMap({ namespace: NAMESPACE }),
),
deleteConfigMap: await expectForbidden(() =>
rawApi.deleteNamespacedConfigMap({
namespace: NAMESPACE,
name: activeTargetName(),
}),
),
readSecret: await expectForbidden(() =>
rawApi.readNamespacedSecret({
namespace: NAMESPACE,
name: 'forbidden-secret',
}),
),
crossNamespaceRead: await expectForbidden(() =>
rawApi.readNamespacedConfigMap({
namespace: 'default',
name: 'kube-root-ca.crt',
}),
),
});
const result = JSON.stringify({
schema: RESULT_SCHEMA,
actor: ACTOR,
mode: 'revoke',
serviceAccountTokenMounted: true,
responseLoss: null,
cas: {
status: 'fulfilled',
receipt,
attemptedResourceVersion: replaceResourceVersion,
replaceCalls,
},
final: {
resourceVersion: activeConfigMap.metadata.resourceVersion,
lockDigest: candidate.lockDigest,
generation: receipt.generation,
pointerSchema: 'qinglong/plugin-package-kubernetes-active-pointer@v3',
projectionDigest: active.secretProjection.projectionDigest,
transitionReceiptDigest:
active.secretProjection.transitionReceiptDigest,
projectionItemCount: active.secretProjection.items.length,
projectedWorkloadVolume: false,
},
rbac,
});
fs.writeFileSync('/dev/termination-log', result, {
encoding: 'utf8',
flag: 'w',
});
process.stdout.write(`${result}\n`);
return;
}
const initial = activationIntent();
const initialPublisher = createPublisher();
let responseLoss = null;
@@ -386,26 +561,7 @@ async function main() {
}
assert.equal(replaceCalls, 1);
const targetName =
'ql3p-' +
require('node:crypto')
.createHash('sha256')
.update(
Buffer.from('qinglong/plugin-package-kubernetes-target@v1\0', 'utf8'),
)
.update(
require('node:crypto')
.createHash('sha256')
.update('qinglong/plugin-package-kubernetes-cluster@v1\0', 'utf8')
.update('ql3-plugin-package-live-cluster', 'utf8')
.digest('hex'),
'utf8',
)
.update('\0', 'utf8')
.update(NAMESPACE, 'utf8')
.update('\0default\0live-cas-package', 'utf8')
.digest('hex')
.slice(0, 52);
const targetName = activeTargetName();
const finalConfigMap = await rawApi.readNamespacedConfigMap({
namespace: NAMESPACE,
name: targetName,
@@ -458,6 +614,7 @@ async function main() {
const result = JSON.stringify({
schema: RESULT_SCHEMA,
actor: ACTOR,
mode: 'rotate',
serviceAccountTokenMounted: true,
responseLoss,
cas: {
@@ -16,8 +16,13 @@ const actorFile = path.join(
root,
'scripts/ql3-plugin-package-kubernetes-live-actor.cjs',
);
const k3sHostFile = path.join(
root,
'scripts/ql3-plugin-package-kubernetes-k3s-live-contract.cjs',
);
const hostSource = fs.readFileSync(hostFile, 'utf8');
const actorSource = fs.readFileSync(actorFile, 'utf8');
const k3sHostSource = fs.readFileSync(k3sHostFile, 'utf8');
const packageJson = JSON.parse(
fs.readFileSync(path.join(root, 'package.json'), 'utf8'),
);
@@ -108,3 +113,32 @@ test('CI runs the dedicated live gate without reusing another cluster', () => {
});
assert.equal(liveStep.env.QL3_REUSE_KIND_CLUSTER, undefined);
});
test('three-node K3s gate proves exact workload projection and receipt-bound revoke', () => {
const result = spawnSync(process.execPath, [k3sHostFile], {
cwd: root,
encoding: 'utf8',
});
assert.equal(result.status, 1);
assert.match(
result.stderr,
/refusing to run without QL3_PLUGIN_PACKAGE_K3S_LIVE=1/,
);
assert.equal(
packageJson.scripts['test:plugin-package-kubernetes-k3s-live:ql3'],
'pnpm --filter @qinglong/cluster-admin build && node scripts/ql3-plugin-package-kubernetes-k3s-live-contract.cjs',
);
assert.match(k3sHostSource, /fixture\.apply\(actorPod\('c'\)\)/);
assert.match(k3sHostSource, /maxUnavailable: 0, maxSurge: 1/);
assert.match(k3sHostSource, /requiredDuringSchedulingIgnoredDuringExecution/);
assert.match(k3sHostSource, /sourceSecret\(rotated\.pointer\.secretProjection\)/);
assert.match(k3sHostSource, /assert\.deepEqual\(inspection\.files, \[projectedPath\]\)/);
assert.match(k3sHostSource, /revokedWorkloadHasNoSecretMount: true/);
assert.match(k3sHostSource, /sourceSecretRetainedButInaccessible: true/);
assert.match(actorSource, /mode: 'revoke'/);
assert.match(actorSource, /active\.secretProjection\.items\.length, 0/);
assert.match(
actorSource,
/pluginPackageKubernetesProjectedSecretWorkloadVolume\([\s\S]*active\.secretProjection/,
);
});