diff --git a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md index 2f85a245..4ed6992d 100644 --- a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md +++ b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md @@ -43,15 +43,25 @@ 和 receipt。item 使用 deferred parent FK 流式落库,最终 receipt 才关闭父引用,因此最多 100,000 Task/500,000 Trigger 不进入 JS 全集或 O(N×M) 扫描。 commit response-loss exact replay 会重验 Secret envelope、Task/Trigger durable head 与 schedule;Secret 占用、provenance 缺项和提交前 authority 漂移均回滚全部 DML。 第七切片在同一 Secret/Config application 子域补齐 Owner `prepared → apply → rollback`,仍不新增 package、dependency 或常驻进程。Owner 只持有私有 - ciphertext-only `materials.ndjson`,单行上限 64 KiB,Edge/Standalone 总量上限 4/16 MiB,且 keyring 必须位于 deployment root 之外;intent 之前的孤儿 + ciphertext-only `materials.ndjson`,单行上限 64 KiB,Edge/Standalone 总量上限 4/16 MiB;keyring、Owner Pepper 与 credential material 必须位于私有 + deployment root 内,并与 apply authority root、target database 相互隔离;intent 之前的孤儿 material 丢弃并重新生成,intent 之后只重放同一 ciphertext。编排先复验 stopped proof、同一 reviewer 的 5 分钟内 `local_console` 强认证并完成 write-before SQLite backup,随后单向推进 `reconciliation_secret_config_reviewed → reconciliation_secret_config_apply_prepared → reconciliation_secret_config_applied`。 material、backup、prepared head、数据库 commit、receipt、applied head 与 seal 的 response-loss 均精确恢复;rollback 只从 applied 恢复 exact 写前 snapshot, 再覆盖 restore、rollback receipt/head/seal 的 response-loss 并推进 `reconciliation_secret_config_rolled_back`。ENOSPC、权限错误或 backup 漂移发生在 head/DML 前, 适用于小型路由设备;该一次性 Owner authority 不进入常驻 Application 制品。 + 第八切片以 ADR-0492 增加兼容 v1/v2 的 completion schema v3:只有 `secret_config_application` evidence 才升级 receipt 版本,binding 只包含 + SecretConfig/decision/apply digest 与四条非敏感 authority path;`adapterCount` 由 Automation、Secret/Config、Run History 三类实际证明推导为 + `0|1|2|3`。Completion 将 Application 中原本正确失败关闭的 `secret_and_config=manual_external` 经 ready signed decision、exact apply intent/receipt、 + current target snapshot 和 applied head 收敛为 receipt 内的 `adapter_required`,但不会替 Identity/Policy/Audit、Unknown 或其他 manual 域背书。 + Automation 与 Secret/Config 连续写同一 SQLite 时,不再错误要求当前 target 等于 Automation 的历史写后快照,而是证明 + `Secret/Config backup.sha256 == Automation targetAfter.sha256`,再证明 current target 等于 Secret/Config `targetAfter`。lineage 只新增 + `reconciliation_secret_config_applied → reconciliation_completed` 一条合法边;receipt seal 与 completed head durable 后,才幂等删除两类数据库等量 + backup。Secret/Config 的 ciphertext material、intent 与 receipt 继续 `0400` 保留,backup/rollback root 封为 `0500`,不增加后台 GC、timer 或常驻内存。 + rolled-back、target drift、提前丢失 backup 与其余 manual 域继续失败关闭;完整迁移库当前仍可能因 Identity/Unknown 保持 manual,这不是 v3 可绕过的限制。 全部 evidence 不含原 Env name/value、目标 ciphertext/key ID 或 row body。v52 Local SQLite 完整测试为 `247/247`,publisher 定向回归 `6/6`;fresh Edge readiness 为 contract v52、104 migrations、89 required tables、SQLite 3.53.3、`DELETE` journal。Local Admin 为 `96/96`,Local Owner 为 - `297 total / 290 pass / 7 conditional skip / 0 fail`;18-package clean build 与逐包顺序测试单次退出 0,完整 backend 为 + `300 total / 293 pass / 7 conditional skip / 0 fail`;18-package clean build 与逐包顺序测试单次退出 0,完整 backend 为 `1567 total / 1565 pass / 2 conditional skip / 0 fail`。package boundary、精确 Cluster dependency/legacy boundary、122-module Edge import、本地镜像与 `14/14` Local artifact audit 全部 compatible;基础 Edge/Standalone 为 `2,635,529 / 2,635,607 bytes`、323 files、58 loaded modules,距 4 MiB 上限仍分别保留 `1,558,775 / 1,558,697 bytes`,且闭包只有 Local SQLite、runtime-core 与 SemVer,没有 Cluster/PostgreSQL 依赖。 @@ -60,7 +70,7 @@ 因而不重跑且不重新占有 PostgreSQL HA 证明;相邻已通过的 remote CI/HA 只作为基线。 D-385~D-388 的 `config.sh`/Keyv/SSH data-directory lineage 与 SQLite `Envs` 保持分离;当前无稳定生产 schema 的历史 `Configs` 表继续 sealed+manual, - 不猜字段。后续切片必须完成 completion 下一 schema 与跨领域完成后的备份回收、真实 Edge 空间证据和 Cluster Secret provider live gate。D-397 apply + 不猜字段。后续切片必须完成真实 Edge 空间证据和 Cluster Secret provider live gate。D-397 apply 只声明 sealed source retained 且 `physicalErasureGuaranteed=false`;明文销毁必须在 restart/ readiness、观察窗和 rollback retention 之后另行强认证。Cluster 必须使用 PostgreSQL SERIALIZABLE ledger、外部 KMS/Secret provider 与 HA evidence, 不复用 Local SQLite/POSIX authority,也不得把明文写入 PostgreSQL、ConfigMap、Pod env 或 Job command。 diff --git a/docs/adr/ADR-0491-bounded-secret-config-reconciliation-and-task-binding.md b/docs/adr/ADR-0491-bounded-secret-config-reconciliation-and-task-binding.md index 1ee57c54..ef0c6edd 100644 --- a/docs/adr/ADR-0491-bounded-secret-config-reconciliation-and-task-binding.md +++ b/docs/adr/ADR-0491-bounded-secret-config-reconciliation-and-task-binding.md @@ -1,6 +1,6 @@ # ADR-0491:有界 Secret/Config Reconciliation 与任务环境绑定 -- 状态:Proposed(D-397 已实现 Legacy Env inspection、私有有界 row plan、durable plan publication、独立 signed decision、逐项 Automation adoption provenance、Local SQLite 原子 application publisher 与 Owner prepared/apply/rollback 编排;跨领域 completion schema、真实 Edge 空间证据和 Cluster Secret provider live gate 尚未完成) +- 状态:Proposed(D-397 已实现 Legacy Env inspection、私有有界 row plan、durable plan publication、独立 signed decision、逐项 Automation adoption provenance、Local SQLite 原子 application publisher、Owner prepared/apply/rollback 编排与 ADR-0492 completion v3;真实 Edge 空间证据和 Cluster Secret provider live gate 尚未完成) - 日期:2026-08-23 - 决策:D-397 - 关联:ADR-0073、ADR-0074、ADR-0092、ADR-0094、ADR-0480、ADR-0482、ADR-0483、ADR-0484、ADR-0485、ADR-0486、ADR-0487、ADR-0488、ADR-0490 @@ -85,7 +85,7 @@ D-397 application 已在一个 `BEGIN IMMEDIATE` 事务内完成: 任一 Secret、Task、Trigger、dispatch、audit、ledger 或 fence 冲突都回滚整个事务。禁止先提交 Secret 再逐任务修补,也禁止在现有 Task revision 上原地改 JSON。目标已有同名/同源 Secret、非 Legacy Task、Plugin-owned Task 或用户在 stopped window 中产生的 revision 都按冲突处理,不自动覆盖或重命名。 -Owner 编排保留在既有 `local-owner-cli/deployment/reconciliation/application/secret-and-config/application/` 子域,不新增 package、常驻进程或依赖。它只把密文写入私有 `materials.ndjson`,单行不超过 64 KiB,Edge/Standalone 文件分别不超过 4/16 MiB;POSIX keyring 必须位于 deployment root 之外。编排在推进 head 前重新证明 stopped state、同一 reviewer 的 `local_console` 强认证与最多 5 分钟认证年龄,并先创建、校验 write-before SQLite v52 backup。空间不足、权限错误或 backup 漂移均发生在 prepared head 与任何 DML 之前,不得在低配设备上边写边赌。 +Owner 编排保留在既有 `local-owner-cli/deployment/reconciliation/application/secret-and-config/application/` 子域,不新增 package、常驻进程或依赖。它只把密文写入私有 `materials.ndjson`,单行不超过 64 KiB,Edge/Standalone 文件分别不超过 4/16 MiB;POSIX keyring、Owner Pepper 与 credential material 必须位于私有 deployment root 内,且与 apply authority root、target database 相互隔离。编排在推进 head 前重新证明 stopped state、同一 reviewer 的 `local_console` 强认证与最多 5 分钟认证年龄,并先创建、校验 write-before SQLite v52 backup。空间不足、权限错误或 backup 漂移均发生在 prepared head 与任何 DML 之前,不得在低配设备上边写边赌。 lineage 单向推进 `reconciliation_secret_config_reviewed → reconciliation_secret_config_apply_prepared → reconciliation_secret_config_applied`;回滚只允许从 applied 精确恢复写前 SQLite snapshot,再推进到 `reconciliation_secret_config_rolled_back`。material、backup、prepared head、数据库 commit、receipt、applied head、seal,以及 restore、rollback receipt/head/seal 的每个 response-loss 窗口都通过 immutable digest 与 durable target state 精确重放:intent 之前的孤儿密文会丢弃并重新生成,intent 之后只复用同一 ciphertext;数据库 commit 丢失响应时由 v52 publisher receipt 复验,不重复 DML。回滚保持原 SQLite 文件 identity 证明,任何 receipt/head/seal 漂移都失败关闭。 @@ -141,8 +141,8 @@ Cluster 不得把 Legacy Env 明文写入 PostgreSQL、ConfigMap、Job command ## 当前验证与后续门禁 -D-397 当前七切片已经实现:absent、unsupported、Edge over-budget、2.x 顺序、同名连接、disabled preservation、保留前缀、异常状态、effective overflow、candidate digest、content-free diagnostics、私有有界 row plan、目标 Secret 冲突、Automation adoption projection、no-effect/manual outcome、durable no-replace publication、terminal seal、head CAS、逐候选独立 signed decision、同一强认证 reviewer、decision/authorization byte bound、`skip → manual_required`、prepare/commit response-loss exact replay、只读 terminal verify、v51 逐 Task/Trigger adoption provenance、v52 Local SQLite 原子 application publisher,以及 Owner prepared/apply/rollback orchestration。v52 在一个 `BEGIN IMMEDIATE` 内复验 Project/RoleBinding、外部 authority、逐 Task/Trigger provenance、当前 head、Plugin ownership 与 Trigger 数量,流式写入加密 Secret、content-free audit、Task rev2、dispatch、Trigger rev2、schedule 和四类 application ledger;deferred parent FK 允许最多 100,000 Task/500,000 Trigger 逐项发布而不在 JS 堆保留全集。Owner 在写前固定 backup 与 stopped proof,以有界 ciphertext-only material 连接 reviewed decision 和 publisher,并覆盖 apply/rollback 全部 response-loss 窗口。commit response-loss 通过 durable receipt exact replay,并重新验证 Secret envelope、Task/Trigger head 与 schedule;目标占用、provenance 缺项、提交前 authority 漂移均回滚全部 DML,rollback 则恢复写前 snapshot。 +D-397 当前八切片已经实现:absent、unsupported、Edge over-budget、2.x 顺序、同名连接、disabled preservation、保留前缀、异常状态、effective overflow、candidate digest、content-free diagnostics、私有有界 row plan、目标 Secret 冲突、Automation adoption projection、no-effect/manual outcome、durable no-replace publication、terminal seal、head CAS、逐候选独立 signed decision、同一强认证 reviewer、decision/authorization byte bound、`skip → manual_required`、prepare/commit response-loss exact replay、只读 terminal verify、v51 逐 Task/Trigger adoption provenance、v52 Local SQLite 原子 application publisher、Owner prepared/apply/rollback orchestration,以及 ADR-0492 completion v3。v52 在一个 `BEGIN IMMEDIATE` 内复验 Project/RoleBinding、外部 authority、逐 Task/Trigger provenance、当前 head、Plugin ownership 与 Trigger 数量,流式写入加密 Secret、content-free audit、Task rev2、dispatch、Trigger rev2、schedule 和四类 application ledger;deferred parent FK 允许最多 100,000 Task/500,000 Trigger 逐项发布而不在 JS 堆保留全集。Owner 在写前固定 backup 与 stopped proof,以有界 ciphertext-only material 连接 reviewed decision 和 publisher,并覆盖 apply/rollback 全部 response-loss 窗口。completion v3 保留 v1/v2 exact shape,验证 signed decision/apply/current target/head,只有 completed head durable 后才幂等回收 Secret/Config backup;rolled-back、target drift 与其余 manual 域继续失败关闭。commit response-loss 通过 durable receipt exact replay,并重新验证 Secret envelope、Task/Trigger head 与 schedule;目标占用、provenance 缺项、提交前 authority 漂移均回滚全部 DML,rollback 则恢复写前 snapshot。 -本切片当前验证:Local SQLite `247/247`,其中 Secret/Config application publisher 定向回归 `6/6`;fresh Edge readiness 为 contract v52、104 migrations、89 required tables、SQLite 3.53.3、`DELETE` journal。Local Admin 为 `96/96`,Local Owner 为 `297 total / 290 pass / 7 conditional skip / 0 fail`;18-package clean build 与逐包顺序测试单次退出 0,完整 backend 为 `1567 total / 1565 pass / 2 conditional skip / 0 fail`。package boundary、精确 Cluster dependency/legacy boundary、122-module Edge import、本地镜像与 `14/14` Local artifact audit 全部 compatible;Local Admin 为 49 source / 48 nested / 1 root export,Local Owner 为 188/187/1,workspace 仍为 18 packages 且没有单文件或浅层 package。基础 Edge/Standalone 为 `2,635,529 / 2,635,607 bytes`、323 files、58 loaded modules,且没有 Cluster/PostgreSQL 闭包。本切片不改 PostgreSQL schema、连接、role、Pool、容器或 Kubernetes 拓扑,因此不重跑且不重新占有 PostgreSQL HA 证明。 +本切片当前验证:Local SQLite `247/247`,其中 Secret/Config application publisher 定向回归 `6/6`;fresh Edge readiness 为 contract v52、104 migrations、89 required tables、SQLite 3.53.3、`DELETE` journal。Local Admin 为 `96/96`,Local Owner 为 `300 total / 293 pass / 7 conditional skip / 0 fail`;18-package clean build 与逐包顺序测试单次退出 0,完整 backend 为 `1567 total / 1565 pass / 2 conditional skip / 0 fail`。package boundary、精确 Cluster dependency/legacy boundary、122-module Edge import、service-manager bridge import、本地镜像与 `14/14` Local artifact audit 全部 compatible;Local Admin 为 49 source / 48 nested / 1 root export,Local Owner 为 188/187/1,workspace 仍为 18 packages 且没有单文件或浅层 package。基础 Edge/Standalone 为 `2,635,529 / 2,635,607 bytes`、323 files、58 loaded modules,且没有 Cluster/PostgreSQL 闭包。本切片不改 PostgreSQL schema、连接、role、Pool、容器或 Kubernetes 拓扑,因此不重跑且不重新占有 PostgreSQL HA 证明。 -转为 Accepted 前仍必须完成:completion schema 演进及 completion 后 rollback material 回收、真实 Edge 空间预算证据与 Cluster Secret provider live gate;本切片的 Local Owner 编排或 PostgreSQL HA 证据不得冒充外部密钥托管或跨领域 completion 已完成。 +转为 Accepted 前仍必须完成:真实 Edge 空间预算证据与 Cluster Secret provider live gate;ADR-0492 已完成本机 completion schema 演进和 completed-head 后 rollback material 回收,但本切片的 Local Owner 编排或 PostgreSQL HA 证据不得冒充外部密钥托管,也不得把仍含 Identity/Unknown manual 域的迁移冒充跨领域 completion 已完成。 diff --git a/docs/adr/ADR-0492-compatible-secret-config-completion-v3.md b/docs/adr/ADR-0492-compatible-secret-config-completion-v3.md new file mode 100644 index 00000000..de91ed01 --- /dev/null +++ b/docs/adr/ADR-0492-compatible-secret-config-completion-v3.md @@ -0,0 +1,90 @@ +# ADR-0492:兼容 Secret/Config 的 Reconciliation Completion v3 + +- 状态:Accepted +- 日期:2026-08-24 +- 决策:D-397 +- 关联:ADR-0487、ADR-0488、ADR-0490、ADR-0491 + +## 背景 + +ADR-0488 的 completion v1 只识别 Application 与 Automation,ADR-0490 的 v2 再加入 Run History。ADR-0491 已经让 Secret/Config 形成 signed decision、原子 application receipt、写后 target snapshot、applied head 与可恢复 backup,但旧 completion 无法消费这些证明,也不能在全局完成后回收 Secret/Config 的数据库等量 rollback material。 + +Application plan 中 `secret_and_config` 的历史 action 是 `manual_external`。这是专用 adapter 获得授权前的正确失败关闭状态,不能改写 sealed plan 为 `adapter_required`。Completion 必须验证后续 plan、signed decision 和 apply 全链路,再在自己的 receipt 中把该域收敛为已证明的 adapter;它仍不能替其他 `manual_external` 域背书。 + +## 决策 + +### 1. 保持 v1/v2 精确兼容,新增 v3 + +`local.deployment.reconciliation.complete|complete.verify` 新增 schema v3。v1/v2 的输入 shape、receipt 语义和验证路径保持不变;只有 receipt 包含 `secret_config_application` 时版本才为 v3。v3 可同时携带 Automation、Secret/Config 与 Run History authority,`adapterCount` 由八域证据实际推导并扩大为 `0|1|2|3`,调用方不能自报。 + +Secret/Config completion binding 只包含 `secretConfigId`、`decisionId` 与 `expectedApplyDigest`。options 只携带 plan/decision/apply authority roots 和 target SQLite 路径,不重新携带 keyring、credential 或明文 material。所有 authority roots 必须互不重叠;Automation 与 Secret/Config 同时存在时必须指向同一个 target SQLite。 + +### 2. 证明链必须闭合 + +Completion 重新验证: + +1. sealed Application plan 的 `secret_and_config=manual_external`; +2. 同一 Application 上 ready、无 skip 的 signed Secret/Config decision; +3. exact apply intent、receipt、decision/SecretConfig identity、preparation digest 与 apply digest; +4. 当前 target snapshot 等于 Secret/Config `targetAfter`; +5. apply storage 是合法的 `applied` 或 completion 后 `completed` layout,且不存在 rollback receipt; +6. source head 是 `reconciliation_secret_config_applied` 且绑定 apply digest。 + +若 Automation 也先写入同一数据库,当前 target 不再等于 Automation 的旧 `targetAfter`。此时链式证明固定为 `Secret/Config backup.sha256 == Automation targetAfter.sha256`,然后再验证当前 target 等于 Secret/Config `targetAfter`;不得把合法的后续写入误判为 Automation drift,也不得跳过两阶段之间的 digest 连续性。 + +Rolled-back state、target drift、decision/apply 脱离、提前丢失 backup、unknown `Configs`、Identity/Policy/Audit、Unknown 或任意其他未终态域都继续拒绝全局 completion。Secret/Config 已 applied 不代表整个 reconciliation 已完成。 + +### 3. durable head 先于 rollback material 回收 + +Instance lineage 新增唯一合法边: + +```text +reconciliation_secret_config_applied → reconciliation_completed +``` + +既有边、generation、source digest 和 CAS 规则不变。Receipt 必须先 no-replace 发布并封存,随后 completed head durable,最后才可删除 Automation 与 Secret/Config backup。两类 storage 都接受 `applied|completed` 重放:head 尚未 completed 时 backup 缺失立即失败;head 已 completed 后允许任一 backup 已回收,并幂等收敛剩余 backup。 + +Secret/Config completed layout 删除的只有 `backup/before.sqlite`。加密 `materials.ndjson`、intent 与 apply receipt 保留为 `0400` audit evidence,root、backup root 与空 rollback work root 封为 `0500`。该操作不扫描数据库、不创建数据库等量副本、不引入 timer/GC,适合低容量 Edge;真正销毁 ciphertext 或 sealed Legacy source 仍是后续独立 retention ceremony。 + +### 4. 部署边界不变 + +实现全部位于既有 `@qinglong/local-owner-cli` 的 cutover、completion 与 Secret/Config application 子域,没有新增 workspace package、production dependency、SQL migration、daemon、listener、watcher、timer、Pool、容器或 Kubernetes workload。Edge/Standalone 常驻闭包不加载该一次性 Owner authority。 + +Cluster 不复用本机 receipt、POSIX storage 或 instance head。Cluster completion 仍需要 PostgreSQL SERIALIZABLE ledger、外部 Secret provider/KMS、HA timeline 与 promotion 后验证。 + +## 影响 + +- completion receipt 的消费者必须按 `schemaVersion` 解析,未知版本继续失败关闭。 +- `advanceLocalCutoverInstanceHead` 是 CRITICAL 共享状态机;本 ADR 只增加一条 source-state 边,不改任何旧 transition。 +- 完整迁移库当前仍可能因 `identity_policy_audit` 或 `unknown` 保持 manual;这是正确的全局围栏,不应为了演示 v3 成功而放宽。 +- v3 verify 要求 completed storage,不能在 head durable 之前充当修复命令。 + +## 被拒绝的替代方案 + +### 改写 sealed Application plan + +拒绝。历史 `manual_external` 是 adapter 授权前的事实;事后把它改成 `adapter_required` 会破坏签名、digest 与审计语义。 + +### 只检查 Secret/Config apply receipt + +拒绝。孤立 receipt 不能证明 signed decision、当前 target、source head、Automation 前序写入或 rollback 状态。 + +### completion 前删除 backup + +拒绝。receipt/head crash window 仍可能需要显式 rollback;提前删除会让响应丢失不可恢复。 + +### 为 Edge 增加后台 GC + +拒绝。一次性 completion 已能固定内存、幂等回收;常驻 timer 会扩大路由设备资源与生命周期表面。 + +## 验证 + +- completion v3 定向:`3/3`,覆盖 exact Secret/Config evidence、其他 manual 域失败关闭、rolled-back、target drift、v3 receipt、lineage 边和 completed-storage 幂等回收; +- completion v1/v2 兼容:no-effect v1、Automation rollback retention 与 Run History v2 均通过; +- Local Owner:受限沙箱 `300 total / 290 pass / 7 skip / 3 loopback EPERM`,对应两个 loopback 文件在沙箱外 `15/15`,有效结果 `300/293/7/0`; +- TypeScript package closure:8 个 Local 依赖包与 Local Owner 全部通过; +- 完整 backend:`1567 total / 1565 pass / 2 conditional skip / 0 fail`; +- 18-package clean build 与逐包顺序测试单次退出 0;package boundary、Cluster dependency、122-module Edge import、service-manager bridge import、本地镜像与 `14/14` Local artifact audit 全部 compatible; +- workspace 保持 18 packages,`singleSourcePackages=[]`、`shallowSourcePackages=[]`;基础 Edge/Standalone 为 `2,635,529 / 2,635,607 bytes`、323 files、58 loaded modules。 + +阶段提交后的远程 CI 仍需重新验证;D-397 的真实 Edge 空间证据和 Cluster Secret provider live gate 仍属于 ADR-0491 后续工作。 diff --git a/docs/adr/README.md b/docs/adr/README.md index 3f73b949..fdc889d8 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -494,7 +494,8 @@ | [ADR-0488](./ADR-0488-cross-domain-reconciliation-completion-fence.md) | 跨领域 Reconciliation 完成围栏与目标重启授权 | Accepted | | [ADR-0489](./ADR-0489-service-manager-completion-restart-lineage.md) | Service Manager 完成围栏重启谱系 | Accepted | | [ADR-0490](./ADR-0490-run-history-terminal-preservation.md) | Run History 终态保全与跨领域完成证明 | Accepted | -| [ADR-0491](./ADR-0491-bounded-secret-config-reconciliation-and-task-binding.md) | 有界 Secret/Config Reconciliation 与任务环境绑定 | Proposed(D-397 local apply/rollback complete;completion/Cluster gate pending) | +| [ADR-0491](./ADR-0491-bounded-secret-config-reconciliation-and-task-binding.md) | 有界 Secret/Config Reconciliation 与任务环境绑定 | Proposed(D-397 local apply/rollback/completion complete;Edge/Cluster gate pending) | +| [ADR-0492](./ADR-0492-compatible-secret-config-completion-v3.md) | 兼容 Secret/Config 的 Reconciliation Completion v3 | Accepted | ## 规则 diff --git a/packages/ql3-local-owner-cli/src/deployment/cutover/instanceLineage.ts b/packages/ql3-local-owner-cli/src/deployment/cutover/instanceLineage.ts index 552021c9..fbdb71f0 100644 --- a/packages/ql3-local-owner-cli/src/deployment/cutover/instanceLineage.ts +++ b/packages/ql3-local-owner-cli/src/deployment/cutover/instanceLineage.ts @@ -501,7 +501,8 @@ export function advanceLocalCutoverInstanceHead( current.state === 'reconciliation_secret_config_applied') || (state === 'reconciliation_completed' && (current.state === 'reconciliation_application_planned' || - current.state === 'reconciliation_automation_applied')) || + current.state === 'reconciliation_automation_applied' || + current.state === 'reconciliation_secret_config_applied')) || (state === 'rollback_prepared' && current.state === 'target_stopped') || (state === 'legacy_restart_requested' && current.state === 'rollback_prepared') || diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/secret-and-config/application/storage.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/secret-and-config/application/storage.ts index aa295f07..c32f29be 100644 --- a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/secret-and-config/application/storage.ts +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/secret-and-config/application/storage.ts @@ -570,6 +570,78 @@ export function validateLocalReconciliationSecretConfigAppliedStorage( validateBackup(selected, intent, uid, [0o400]); } +/** + * Collects the rollback copy only after the cross-domain completion head is + * durable. Encrypted material and receipts remain sealed as audit evidence. + * The partially collected layout is accepted so a response loss can replay. + */ +export function collectLocalReconciliationSecretConfigCompletedStorage( + selected: Readonly, + intent: Readonly, + uid: number, +): void { + directoryMode(selected.root, uid, [0o500], 'root'); + directoryMode(selected.backupRoot, uid, [0o700, 0o500], 'backup root'); + directoryMode( + selected.rollbackRoot, + uid, + [0o700, 0o500], + 'rollback work root', + ); + validateLocalReconciliationSecretConfigApplyCatalog(selected); + if (fs.readdirSync(selected.rollbackRoot).length !== 0) { + fail('rollback work root must be empty'); + } + readLocalReconciliationSecretConfigApplyIntent(selected, uid); + readLocalReconciliationSecretConfigApplyReceipt(selected, uid); + readLocalReconciliationSecretConfigMaterials( + selected, + intent.profile, + uid, + intent.material, + ); + if (fs.existsSync(selected.backup)) { + validateBackup(selected, intent, uid, [0o400]); + if ((fs.statSync(selected.backupRoot).mode & 0o777) !== 0o700) { + fs.chmodSync(selected.backupRoot, 0o700); + syncDirectory(selected.root); + } + unlinkIfPresent(selected.backup); + } + sealDirectory(selected.backupRoot, uid, 'backup root'); + sealDirectory(selected.rollbackRoot, uid, 'rollback work root'); + validateLocalReconciliationSecretConfigCompletedStorage( + selected, + intent, + uid, + ); +} + +export function validateLocalReconciliationSecretConfigCompletedStorage( + selected: Readonly, + intent: Readonly, + uid: number, +): void { + directoryMode(selected.root, uid, [0o500], 'root'); + directoryMode(selected.backupRoot, uid, [0o500], 'backup root'); + directoryMode(selected.rollbackRoot, uid, [0o500], 'rollback work root'); + validateLocalReconciliationSecretConfigApplyCatalog(selected); + if (fs.readdirSync(selected.backupRoot).length !== 0) { + fail('backup root must be empty'); + } + if (fs.readdirSync(selected.rollbackRoot).length !== 0) { + fail('rollback work root must be empty'); + } + readLocalReconciliationSecretConfigApplyIntent(selected, uid); + readLocalReconciliationSecretConfigApplyReceipt(selected, uid); + readLocalReconciliationSecretConfigMaterials( + selected, + intent.profile, + uid, + intent.material, + ); +} + export function prepareLocalReconciliationSecretConfigRollbackSource( selected: Readonly, intent: Readonly, diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/contract.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/contract.ts index b12dab5d..370999e5 100644 --- a/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/contract.ts +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/contract.ts @@ -17,11 +17,19 @@ export interface LocalReconciliationCompletionAutomationOptions { readonly targetDatabasePath: string; } +export interface LocalReconciliationCompletionSecretConfigOptions { + readonly secretConfigRoot: string; + readonly secretConfigDecisionRoot: string; + readonly secretConfigApplyRoot: string; + readonly targetDatabasePath: string; +} + export interface LocalReconciliationCompletionOptions { readonly deploymentRoot: string; readonly applicationRoot: string; readonly completionRoot: string; readonly automation: Readonly | null; + readonly secretConfig: Readonly | null; readonly runHistory: Readonly | null; readonly allowRootService: boolean; } @@ -37,13 +45,19 @@ export interface LocalReconciliationCompletionAutomationBinding { readonly expectedApplyDigest: string; } +export interface LocalReconciliationCompletionSecretConfigBinding { + readonly secretConfigId: string; + readonly decisionId: string; + readonly expectedApplyDigest: string; +} + export interface LocalReconciliationCompletionRunHistoryBinding { readonly preservationId: string; readonly expectedPreservationDigest: string; } export interface LocalReconciliationCompleteCommand { - readonly schemaVersion: 1 | 2; + readonly schemaVersion: 1 | 2 | 3; readonly operation: 'local.deployment.reconciliation.complete'; readonly options: Readonly; readonly request: Readonly<{ @@ -52,13 +66,14 @@ export interface LocalReconciliationCompleteCommand { expectedApplicationPlanDigest: string; expectedHeadDigest: string; automation: Readonly | null; + secretConfig: Readonly | null; runHistory: Readonly | null; completedAtMs: number; }>; } export interface LocalReconciliationCompletionVerifyCommand { - readonly schemaVersion: 1 | 2; + readonly schemaVersion: 1 | 2 | 3; readonly operation: 'local.deployment.reconciliation.complete.verify'; readonly options: Readonly; readonly request: Readonly<{ @@ -66,6 +81,7 @@ export interface LocalReconciliationCompletionVerifyCommand { applicationId: string; expectedCompletionDigest: string; automation: Readonly | null; + secretConfig: Readonly | null; runHistory: Readonly | null; }>; } @@ -81,7 +97,7 @@ export interface LocalReconciliationCompletionResult { readonly applicationId: string; readonly completionDigest: string; readonly domainCount: 8; - readonly adapterCount: 0 | 1 | 2; + readonly adapterCount: 0 | 1 | 2 | 3; readonly instanceHeadDigest: string; } @@ -180,9 +196,41 @@ function normalizeAutomationOptions( }); } +function normalizeSecretConfigOptions( + value: unknown, +): Readonly | null { + if (value === null) return null; + const selected = record(value, 'secret config options'); + exact( + selected, + [ + 'secretConfigApplyRoot', + 'secretConfigDecisionRoot', + 'secretConfigRoot', + 'targetDatabasePath', + ], + 'secret config options', + ); + return Object.freeze({ + secretConfigRoot: safePath(selected.secretConfigRoot, 'secretConfigRoot'), + secretConfigDecisionRoot: safePath( + selected.secretConfigDecisionRoot, + 'secretConfigDecisionRoot', + ), + secretConfigApplyRoot: safePath( + selected.secretConfigApplyRoot, + 'secretConfigApplyRoot', + ), + targetDatabasePath: safePath( + selected.targetDatabasePath, + 'targetDatabasePath', + ), + }); +} + function normalizeOptions( value: unknown, - schemaVersion: 1 | 2, + schemaVersion: 1 | 2 | 3, ): Readonly { const selected = record(value, 'options'); exact( @@ -195,6 +243,15 @@ function normalizeOptions( 'completionRoot', 'deploymentRoot', ] + : schemaVersion === 2 + ? [ + 'allowRootService', + 'applicationRoot', + 'automation', + 'completionRoot', + 'deploymentRoot', + 'runHistory', + ] : [ 'allowRootService', 'applicationRoot', @@ -202,6 +259,7 @@ function normalizeOptions( 'completionRoot', 'deploymentRoot', 'runHistory', + 'secretConfig', ], 'options', ); @@ -212,8 +270,14 @@ function normalizeOptions( fail('command identity is invalid'); } const automation = normalizeAutomationOptions(selected.automation); + const secretConfig = + schemaVersion === 3 + ? normalizeSecretConfigOptions(selected.secretConfig) + : null; const runHistory = schemaVersion === 1 + ? null + : schemaVersion === 3 && selected.runHistory === null ? null : normalizeRunHistoryOptions(selected.runHistory); const normalized = Object.freeze({ @@ -221,6 +285,7 @@ function normalizeOptions( applicationRoot: safePath(selected.applicationRoot, 'applicationRoot'), completionRoot: safePath(selected.completionRoot, 'completionRoot'), automation, + secretConfig, runHistory, allowRootService: selected.allowRootService, }) as Readonly; @@ -235,6 +300,13 @@ function normalizeOptions( automation.automationDecisionRoot, automation.automationApplyRoot, ]), + ...(secretConfig === null + ? [] + : [ + secretConfig.secretConfigRoot, + secretConfig.secretConfigDecisionRoot, + secretConfig.secretConfigApplyRoot, + ]), ...(runHistory === null ? [] : [runHistory.runHistoryRoot]), ]; for (let left = 0; left < roots.length; left += 1) { @@ -257,6 +329,23 @@ function normalizeOptions( ) { fail('targetDatabasePath overlaps an authority root'); } + if ( + secretConfig !== null && + roots.some( + (root) => + overlaps(root, secretConfig.targetDatabasePath) || + overlaps(secretConfig.targetDatabasePath, root), + ) + ) { + fail('targetDatabasePath overlaps an authority root'); + } + if ( + automation !== null && + secretConfig !== null && + automation.targetDatabasePath !== secretConfig.targetDatabasePath + ) { + fail('adapter targetDatabasePath values differ'); + } if ( runHistory !== null && roots.some( @@ -305,6 +394,30 @@ function normalizeAutomationBinding( }); } +function normalizeSecretConfigBinding( + value: unknown, +): Readonly | null { + if (value === null) return null; + const selected = record(value, 'secret config binding'); + exact( + selected, + ['decisionId', 'expectedApplyDigest', 'secretConfigId'], + 'secret config binding', + ); + return Object.freeze({ + secretConfigId: identifier( + selected.secretConfigId, + UUID_V4, + 'secretConfigId', + ), + decisionId: identifier(selected.decisionId, UUID_V7, 'decisionId'), + expectedApplyDigest: digest( + selected.expectedApplyDigest, + 'expectedApplyDigest', + ), + }); +} + function normalizeRunHistoryBinding( value: unknown, ): Readonly { @@ -335,7 +448,9 @@ function command(value: unknown, operation: string) { 'command', ); if ( - (selected.schemaVersion !== 1 && selected.schemaVersion !== 2) || + (selected.schemaVersion !== 1 && + selected.schemaVersion !== 2 && + selected.schemaVersion !== 3) || selected.operation !== operation ) { fail('command version or operation is invalid'); @@ -362,6 +477,16 @@ export function normalizeLocalReconciliationCompleteCommand( 'expectedApplicationPlanDigest', 'expectedHeadDigest', ] + : selected.schemaVersion === 2 + ? [ + 'applicationId', + 'automation', + 'completedAtMs', + 'completionId', + 'expectedApplicationPlanDigest', + 'expectedHeadDigest', + 'runHistory', + ] : [ 'applicationId', 'automation', @@ -370,6 +495,7 @@ export function normalizeLocalReconciliationCompleteCommand( 'expectedApplicationPlanDigest', 'expectedHeadDigest', 'runHistory', + 'secretConfig', ], 'request', ); @@ -403,8 +529,14 @@ export function normalizeLocalReconciliationCompleteCommand( 'expectedHeadDigest', ), automation: normalizeAutomationBinding(selected.request.automation), + secretConfig: + selected.schemaVersion === 3 + ? normalizeSecretConfigBinding(selected.request.secretConfig) + : null, runHistory: selected.schemaVersion === 1 + ? null + : selected.schemaVersion === 3 && selected.request.runHistory === null ? null : normalizeRunHistoryBinding(selected.request.runHistory), completedAtMs: selected.request.completedAtMs as number, @@ -428,12 +560,21 @@ export function normalizeLocalReconciliationCompletionVerifyCommand( 'completionId', 'expectedCompletionDigest', ] + : selected.schemaVersion === 2 + ? [ + 'applicationId', + 'automation', + 'completionId', + 'expectedCompletionDigest', + 'runHistory', + ] : [ 'applicationId', 'automation', 'completionId', 'expectedCompletionDigest', 'runHistory', + 'secretConfig', ], 'request', ); @@ -457,8 +598,14 @@ export function normalizeLocalReconciliationCompletionVerifyCommand( 'expectedCompletionDigest', ), automation: normalizeAutomationBinding(selected.request.automation), + secretConfig: + selected.schemaVersion === 3 + ? normalizeSecretConfigBinding(selected.request.secretConfig) + : null, runHistory: selected.schemaVersion === 1 + ? null + : selected.schemaVersion === 3 && selected.request.runHistory === null ? null : normalizeRunHistoryBinding(selected.request.runHistory), }), diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/coordinator.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/coordinator.ts index b83c44bb..6368d577 100644 --- a/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/coordinator.ts +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/coordinator.ts @@ -33,6 +33,20 @@ import type { LocalReconciliationAutomationApplyReceipt, } from '../application/automation/applyEvidence'; import { readLocalReconciliationAutomationDecisionTerminal } from '../application/automation/decisionCoordinator'; +import { readLocalReconciliationSecretConfigDecisionTerminal } from '../application/secret-and-config/decisionCoordinator'; +import type { + LocalReconciliationSecretConfigApplyIntent, + LocalReconciliationSecretConfigApplyReceipt, +} from '../application/secret-and-config/application/evidence'; +import { + collectLocalReconciliationSecretConfigCompletedStorage, + localReconciliationSecretConfigApplyPaths, + readLocalReconciliationSecretConfigApplyIntent, + readLocalReconciliationSecretConfigApplyReceipt, + validateLocalReconciliationSecretConfigAppliedStorage, + validateLocalReconciliationSecretConfigApplyCatalog, + validateLocalReconciliationSecretConfigCompletedStorage, +} from '../application/secret-and-config/application/storage'; import { readLocalReconciliationRunHistoryTerminal, type LocalReconciliationRunHistoryDependencies, @@ -66,6 +80,14 @@ interface AutomationProof { readonly intent: Readonly; readonly receipt: Readonly; readonly paths: ReturnType; + readonly storageState: 'applied' | 'completed'; +} + +interface SecretConfigProof { + readonly intent: Readonly; + readonly receipt: Readonly; + readonly paths: ReturnType; + readonly storageState: 'applied' | 'completed'; } interface RunHistoryProof { @@ -79,6 +101,7 @@ export interface LocalReconciliationCompletionDependencies readonly afterTerminalSealed?: () => void; readonly afterHeadAdvanced?: () => void; readonly afterBackupCollected?: () => void; + readonly afterSecretConfigBackupCollected?: () => void; } async function runHistoryProof( @@ -444,6 +467,134 @@ async function automationProof( ) { fail('automation apply evidence is detached'); } + if (command.request.secretConfig === null) { + const current = await ( + dependencies.inspectSnapshot ?? inspectLocalSqliteSnapshot + )({ + databasePath: options.targetDatabasePath, + profile: intent.profile, + }); + if (current.sha256 !== receipt.targetAfter.sha256) { + fail('automation target drifted after apply'); + } + } + const storageState = fs.existsSync(selected.backup) + ? ('applied' as const) + : ('completed' as const); + if (storageState === 'applied') { + validateLocalReconciliationAutomationAppliedStorage(selected, intent, uid); + } else { + validateLocalReconciliationAutomationCompletedStorage(selected, uid); + } + return Object.freeze({ + intent, + receipt, + paths: selected, + storageState, + }); +} + +async function secretConfigProof( + command: Readonly, + terminal: Readonly, + automation: Readonly | null, + uid: number, + dependencies: LocalReconciliationCompletionDependencies, +): Promise | null> { + const secretConfigDomain = terminal.plan.domains.find( + (domain) => domain.domain === 'secret_and_config', + ); + if (!secretConfigDomain) fail('secret and config domain is absent'); + if (secretConfigDomain.action === 'no_effect') { + if ( + command.options.secretConfig !== null || + command.request.secretConfig !== null + ) { + fail('no-effect completion must not carry secret config authority'); + } + return null; + } + if ( + secretConfigDomain.action === 'manual_external' && + command.options.secretConfig === null && + command.request.secretConfig === null + ) { + return null; + } + if ( + secretConfigDomain.action !== 'manual_external' || + command.options.secretConfig === null || + command.request.secretConfig === null + ) { + fail('secret and config domain is not terminally provable'); + } + const options = command.options.secretConfig; + const binding = command.request.secretConfig; + for (const [directory, label] of [ + [options.secretConfigRoot, 'secretConfigRoot'], + [options.secretConfigDecisionRoot, 'secretConfigDecisionRoot'], + [options.secretConfigApplyRoot, 'secretConfigApplyRoot'], + ] as const) { + validatePrivateDirectory(directory, uid, label); + } + const decision = await readLocalReconciliationSecretConfigDecisionTerminal( + { + deploymentRoot: command.options.deploymentRoot, + applicationRoot: command.options.applicationRoot, + secretConfigRoot: options.secretConfigRoot, + secretConfigDecisionRoot: options.secretConfigDecisionRoot, + allowRootService: command.options.allowRootService, + }, + binding.secretConfigId, + uid, + [ + 'reconciliation_secret_config_applied', + 'reconciliation_secret_config_rolled_back', + 'reconciliation_completed', + ], + ); + if ( + decision.receipt.decisionId !== binding.decisionId || + decision.receipt.outcome !== 'ready' || + decision.context.application.plan.applicationPlanDigest !== + terminal.plan.applicationPlanDigest + ) { + fail('secret config decision is detached from application authority'); + } + const selected = localReconciliationSecretConfigApplyPaths( + options.secretConfigApplyRoot, + binding.secretConfigId, + ); + validateLocalReconciliationSecretConfigApplyCatalog(selected); + const intent = readLocalReconciliationSecretConfigApplyIntent(selected, uid); + const receipt = readLocalReconciliationSecretConfigApplyReceipt( + selected, + uid, + ); + if ( + intent.command.options.deploymentRoot !== command.options.deploymentRoot || + intent.command.options.applicationRoot !== + command.options.applicationRoot || + intent.command.options.secretConfigRoot !== options.secretConfigRoot || + intent.command.options.secretConfigDecisionRoot !== + options.secretConfigDecisionRoot || + intent.command.options.secretConfigApplyRoot !== + options.secretConfigApplyRoot || + intent.command.options.targetDatabasePath !== options.targetDatabasePath || + intent.command.request.secretConfigId !== binding.secretConfigId || + intent.command.request.decisionId !== binding.decisionId || + intent.command.request.expectedDecisionDigest !== + decision.receipt.decisionDigest || + receipt.secretConfigId !== binding.secretConfigId || + receipt.decisionId !== binding.decisionId || + receipt.applyDigest !== binding.expectedApplyDigest || + receipt.preparationDigest !== intent.preparationDigest || + (automation !== null && + intent.backup.sha256 !== automation.receipt.targetAfter.sha256) || + fs.existsSync(selected.rollbackReceipt) + ) { + fail('secret config apply evidence is detached'); + } const current = await ( dependencies.inspectSnapshot ?? inspectLocalSqliteSnapshot )({ @@ -451,14 +602,36 @@ async function automationProof( profile: intent.profile, }); if (current.sha256 !== receipt.targetAfter.sha256) { - fail('automation target drifted after apply'); + fail('secret config target drifted after apply'); } - return Object.freeze({ intent, receipt, paths: selected }); + const storageState = fs.existsSync(selected.backup) + ? ('applied' as const) + : ('completed' as const); + if (storageState === 'applied') { + validateLocalReconciliationSecretConfigAppliedStorage( + selected, + intent, + uid, + ); + } else { + validateLocalReconciliationSecretConfigCompletedStorage( + selected, + intent, + uid, + ); + } + return Object.freeze({ + intent, + receipt, + paths: selected, + storageState, + }); } function domainEvidence( terminal: Readonly, automation: Readonly | null, + secretConfig: Readonly | null, runHistory: Readonly | null, ): readonly Readonly[] { return Object.freeze( @@ -483,6 +656,18 @@ function domainEvidence( evidenceDigest: automation.receipt.applyDigest, }); } + if ( + domain.domain === 'secret_and_config' && + domain.action === 'manual_external' && + secretConfig !== null + ) { + return Object.freeze({ + domain: domain.domain, + action: 'adapter_required' as const, + evidenceKind: 'secret_config_application' as const, + evidenceDigest: secretConfig.receipt.applyDigest, + }); + } if ( domain.domain === 'run_history' && domain.action === 'adapter_required' && @@ -571,13 +756,20 @@ function assertSourceHead( head: Readonly, expectedHeadDigest: string, automation: Readonly | null, + secretConfig: Readonly | null, ): void { const expectedState = - automation === null + secretConfig !== null + ? 'reconciliation_secret_config_applied' + : automation === null ? 'reconciliation_application_planned' : 'reconciliation_automation_applied'; const expectedSource = - automation === null ? undefined : automation.receipt.applyDigest; + secretConfig !== null + ? secretConfig.receipt.applyDigest + : automation === null + ? undefined + : automation.receipt.applyDigest; if ( head.headDigest !== expectedHeadDigest || head.state !== expectedState || @@ -618,13 +810,25 @@ export async function completeLocalReconciliation( uid, dependencies, ); + const secretConfig = await secretConfigProof( + command, + terminal, + automation, + uid, + dependencies, + ); const runHistory = await runHistoryProof( command, terminal, uid, dependencies, ); - const domains = domainEvidence(terminal, automation, runHistory); + const domains = domainEvidence( + terminal, + automation, + secretConfig, + runHistory, + ); const selected = ensureCompletionDirectory( command.options.completionRoot, command.request.completionId, @@ -654,13 +858,19 @@ export async function completeLocalReconciliation( fail('completion command is not an exact replay'); } } else { - assertSourceHead(head, command.request.expectedHeadDigest, automation); + assertSourceHead( + head, + command.request.expectedHeadDigest, + automation, + secretConfig, + ); const adapterCount = domains.filter( (domain) => domain.action === 'adapter_required', - ).length as 0 | 1 | 2; + ).length as 0 | 1 | 2 | 3; const latestEvidenceAtMs = Math.max( terminal.plan.committedAtMs, automation?.receipt.appliedAtMs ?? 0, + secretConfig?.receipt.appliedAtMs ?? 0, runHistory?.receipt.preservedAtMs ?? 0, ); if (command.request.completedAtMs < latestEvidenceAtMs) { @@ -705,13 +915,12 @@ export async function completeLocalReconciliation( uid, ); if (head.state !== 'reconciliation_completed') { - assertSourceHead(head, receipt.sourceHeadDigest, automation); - if (automation !== null) { - validateLocalReconciliationAutomationAppliedStorage( - automation.paths, - automation.intent, - uid, - ); + assertSourceHead(head, receipt.sourceHeadDigest, automation, secretConfig); + if (automation?.storageState === 'completed') { + fail('automation rollback backup was collected before completion'); + } + if (secretConfig?.storageState === 'completed') { + fail('secret config rollback backup was collected before completion'); } head = advanceCompletedHead(terminal, receipt, uid); } else if (head.sourceRecordDigest !== receipt.completionDigest) { @@ -726,6 +935,14 @@ export async function completeLocalReconciliation( ); dependencies.afterBackupCollected?.(); } + if (secretConfig !== null) { + collectLocalReconciliationSecretConfigCompletedStorage( + secretConfig.paths, + secretConfig.intent, + uid, + ); + dependencies.afterSecretConfigBackupCollected?.(); + } return result(command.operation, status, receipt, head); } @@ -777,6 +994,7 @@ export async function verifyLocalReconciliationCompletion( expectedApplicationPlanDigest: receipt.applicationPlanDigest, expectedHeadDigest: receipt.sourceHeadDigest, automation: command.request.automation, + secretConfig: command.request.secretConfig, runHistory: command.request.runHistory, completedAtMs: receipt.completedAtMs, }), @@ -787,13 +1005,25 @@ export async function verifyLocalReconciliationCompletion( uid, dependencies, ); + const secretConfig = await secretConfigProof( + syntheticCompleteCommand, + terminal, + automation, + uid, + dependencies, + ); const runHistory = await runHistoryProof( syntheticCompleteCommand, terminal, uid, dependencies, ); - const domains = domainEvidence(terminal, automation, runHistory); + const domains = domainEvidence( + terminal, + automation, + secretConfig, + runHistory, + ); validateReceiptBinding( receipt, terminal, @@ -818,6 +1048,13 @@ export async function verifyLocalReconciliationCompletion( uid, ); } + if (secretConfig !== null) { + validateLocalReconciliationSecretConfigCompletedStorage( + secretConfig.paths, + secretConfig.intent, + uid, + ); + } return result(command.operation, 'verified', receipt, head); } diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/evidence.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/evidence.ts index d622c10e..036da91b 100644 --- a/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/evidence.ts +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/evidence.ts @@ -16,13 +16,14 @@ export interface LocalReconciliationCompletionDomainEvidence { readonly evidenceKind: | 'application_summary' | 'automation_apply' - | 'run_history_preservation'; + | 'run_history_preservation' + | 'secret_config_application'; readonly evidenceDigest: string; } export interface LocalReconciliationCompletionReceipt { readonly schema: typeof RECEIPT_SCHEMA; - readonly schemaVersion: 1 | 2; + readonly schemaVersion: 1 | 2 | 3; readonly state: 'reconciliation_completed'; readonly completionId: string; readonly applicationId: string; @@ -34,7 +35,7 @@ export interface LocalReconciliationCompletionReceipt { readonly applicationPlanDigest: string; readonly sourceHeadDigest: string; readonly domains: readonly Readonly[]; - readonly adapterCount: 0 | 1 | 2; + readonly adapterCount: 0 | 1 | 2 | 3; readonly completedAtMs: number; readonly completionDigest: string; } @@ -68,7 +69,7 @@ function exact( function domainEvidence( value: unknown, expectedDomain: LocalReconciliationPlanDomain, - schemaVersion: 1 | 2, + schemaVersion: 1 | 2 | 3, ): Readonly { const selected = exact( value, @@ -83,13 +84,18 @@ function domainEvidence( selected.action === 'adapter_required' && selected.evidenceKind === 'automation_apply'; const runHistory = - schemaVersion === 2 && + schemaVersion >= 2 && expectedDomain === 'run_history' && selected.action === 'adapter_required' && selected.evidenceKind === 'run_history_preservation'; + const secretConfig = + schemaVersion === 3 && + expectedDomain === 'secret_and_config' && + selected.action === 'adapter_required' && + selected.evidenceKind === 'secret_config_application'; if ( selected.domain !== expectedDomain || - (!noEffect && !automation && !runHistory) || + (!noEffect && !automation && !runHistory && !secretConfig) || typeof selected.evidenceDigest !== 'string' || !DIGEST.test(selected.evidenceDigest) ) { @@ -110,8 +116,12 @@ export function buildLocalReconciliationCompletionReceipt( >, ): Readonly { const schemaVersion = input.domains.some( - (domain) => domain.evidenceKind === 'run_history_preservation', + (domain) => domain.evidenceKind === 'secret_config_application', ) + ? (3 as const) + : input.domains.some( + (domain) => domain.evidenceKind === 'run_history_preservation', + ) ? (2 as const) : (1 as const); const payload = Object.freeze({ @@ -154,7 +164,11 @@ export function normalizeLocalReconciliationCompletionReceipt( if (!Array.isArray(selected.domains) || selected.domains.length !== 8) { fail('receipt domain catalog is invalid'); } - if (selected.schemaVersion !== 1 && selected.schemaVersion !== 2) { + if ( + selected.schemaVersion !== 1 && + selected.schemaVersion !== 2 && + selected.schemaVersion !== 3 + ) { fail('receipt schema version is invalid'); } const schemaVersion = selected.schemaVersion; @@ -171,10 +185,23 @@ export function normalizeLocalReconciliationCompletionReceipt( const normalized = Object.freeze({ ...raw, domains }); if ( selected.schema !== RECEIPT_SCHEMA || - (schemaVersion === 2) !== - domains.some( + (schemaVersion === 1 && + domains.some((domain) => + ['run_history_preservation', 'secret_config_application'].includes( + domain.evidenceKind, + ), + )) || + (schemaVersion === 2 && + (!domains.some( (domain) => domain.evidenceKind === 'run_history_preservation', ) || + domains.some( + (domain) => domain.evidenceKind === 'secret_config_application', + ))) || + (schemaVersion === 3 && + !domains.some( + (domain) => domain.evidenceKind === 'secret_config_application', + )) || selected.state !== 'reconciliation_completed' || typeof selected.completionId !== 'string' || !UUID_V4.test(selected.completionId) || @@ -195,7 +222,7 @@ export function normalizeLocalReconciliationCompletionReceipt( ].every( (candidate) => typeof candidate === 'string' && DIGEST.test(candidate), ) || - ![0, 1, 2].includes(selected.adapterCount as number) || + ![0, 1, 2, 3].includes(selected.adapterCount as number) || selected.adapterCount !== adapterCount || !Number.isSafeInteger(selected.completedAtMs) || (selected.completedAtMs as number) < 0 || diff --git a/packages/ql3-local-owner-cli/test/reconciliationCapturePrepare.test.cjs b/packages/ql3-local-owner-cli/test/reconciliationCapturePrepare.test.cjs index 986656d7..021aeed8 100644 --- a/packages/ql3-local-owner-cli/test/reconciliationCapturePrepare.test.cjs +++ b/packages/ql3-local-owner-cli/test/reconciliationCapturePrepare.test.cjs @@ -54,6 +54,9 @@ const { const { createLocalDataDirectoryApplicationCommit, } = require('@qinglong/local-sqlite/data-directory-application-commit'); +const { + applyPreparedReconciliationSecretConfigApplication, +} = require('@qinglong/local-admin/reconciliation-secret-and-config-application'); const { advanceLocalCutoverInstanceHead, assertLocalCutoverTargetHead, @@ -71,6 +74,16 @@ const { const { targetStoppedEvidence, } = require('../dist/deployment/cutover/targetStopRecordEvidence.js'); +const { + collectLocalReconciliationSecretConfigCompletedStorage, + localReconciliationSecretConfigApplyPaths, + readLocalReconciliationSecretConfigApplyIntent, + validateLocalReconciliationSecretConfigCompletedStorage, +} = require('../dist/deployment/reconciliation/application/secret-and-config/application/storage.js'); +const { + buildLocalReconciliationCompletionReceipt, + normalizeLocalReconciliationCompletionReceipt, +} = require('../dist/deployment/reconciliation/completion/evidence.js'); function digest(value) { return crypto @@ -801,9 +814,27 @@ function automationReadyDatabaseInitializer() { }; } +function insertSecretConfigOwnerBinding(targetDatabasePath) { + const target = new DatabaseSync(targetDatabasePath); + target.exec(` + INSERT OR IGNORE INTO "QingLong3ProjectRoleBindings" ( + "project_id", "subject_type", "subject_id", "version", "state", + "role", "mutation_id", "changed_by_type", "changed_by_id", + "created_at_ms" + ) VALUES ( + 'default', 'user', 'review-owner', 1, 'active', 'owner', + 'secret-config-apply-owner-binding', 'user', 'review-owner', 1 + ); + PRAGMA wal_checkpoint(TRUNCATE); + PRAGMA journal_mode=DELETE; + `); + target.close(); +} + function secretConfigDatabaseInitializer({ active = false, configs = false, + ownerBinding = true, } = {}) { return ({ legacySourcePath, recoveryPath, targetDatabasePath }) => { const legacy = new DatabaseSync(legacySourcePath); @@ -850,20 +881,16 @@ function secretConfigDatabaseInitializer({ ); assert.equal(migration.status, 0, migration.stderr); fs.chmodSync(targetDatabasePath, 0o600); - const target = new DatabaseSync(targetDatabasePath); - target.exec(` - INSERT INTO "QingLong3ProjectRoleBindings" ( - "project_id", "subject_type", "subject_id", "version", "state", - "role", "mutation_id", "changed_by_type", "changed_by_id", - "created_at_ms" - ) VALUES ( - 'default', 'user', 'review-owner', 1, 'active', 'owner', - 'secret-config-apply-owner-binding', 'user', 'review-owner', 1 - ); - PRAGMA wal_checkpoint(TRUNCATE); - PRAGMA journal_mode=DELETE; - `); - target.close(); + if (ownerBinding) { + insertSecretConfigOwnerBinding(targetDatabasePath); + } else { + const target = new DatabaseSync(targetDatabasePath); + target.exec(` + PRAGMA wal_checkpoint(TRUNCATE); + PRAGMA journal_mode=DELETE; + `); + target.close(); + } }; } @@ -1255,6 +1282,7 @@ async function secretConfigPlanFixture(t, options = {}) { initializeDatabases: secretConfigDatabaseInitializer({ active: options.active === true, configs: options.configs === true, + ownerBinding: options.ownerBinding !== false, }), mutateTarget({ targetDatabasePath }) { const target = new DatabaseSync(targetDatabasePath); @@ -1552,6 +1580,7 @@ async function plannedSecretConfigDecisionFixture(t, options = {}) { reviewId: options.reviewId, applicationId: options.applicationId, secretConfigId: options.secretConfigId, + ownerBinding: options.ownerBinding, }); const planned = await planLocalReconciliationSecretConfig( state.secretConfigCommand, @@ -1823,6 +1852,124 @@ async function appliedAutomationFixture(t, options = {}) { }; } +async function appliedSecretConfigFixture(t, options = {}) { + const suffix = options.suffix ?? 'completion-secret-config'; + const state = await plannedSecretConfigDecisionFixture(t, { + suffix, + planId: options.planId ?? '00000000-0000-4000-8000-000000000521', + reviewId: options.reviewId ?? '00000000-0000-4000-8000-000000000522', + applicationId: + options.applicationId ?? '00000000-0000-4000-8000-000000000523', + secretConfigId: + options.secretConfigId ?? '00000000-0000-4000-8000-000000000524', + ownerBinding: false, + }); + const decisionId = + options.decisionId ?? '019b0000-0000-7000-8000-000000000521'; + const prepareCommand = secretConfigDecisionPrepareCommand(state, decisionId); + const prepared = await prepareLocalReconciliationSecretConfigDecision( + prepareCommand, + ); + const review = secretConfigDecisionFile( + state, + { result: prepared }, + [ + { + disposition: 'preserve_disabled', + reason: 'reviewed_disabled_preservation', + }, + ], + suffix, + ); + const decisionCommit = secretConfigDecisionCommitFixture( + state, + { result: prepared, commandOptions: prepareCommand.options }, + review.filePath, + ); + const decision = await commitLocalReconciliationSecretConfigDecision( + decisionCommit.command, + decisionCommit.dependencies, + ); + assert.equal(decision.outcome, 'ready'); + + const secretKeyringPath = path.join( + state.deploymentRoot, + `local-secret-keyring-${suffix}.json`, + ); + await provisionLocalSecretKeyring(secretKeyringPath); + const secretConfigApplyRoot = path.join( + path.dirname(state.captureRoot), + `secret-config-apply-${suffix}`, + ); + fs.mkdirSync(secretConfigApplyRoot, { mode: 0o700 }); + const applyOptions = { + ...prepareCommand.options, + secretConfigApplyRoot, + targetDatabasePath: state.targetDatabasePath, + secretKeyringPath, + ownerPepperKeyringDirectory: + state.command.options.ownerPepperKeyringDirectory, + credentialFilePath: state.command.options.credentialFilePath, + }; + const appliedAtMs = decisionCommit.command.request.committedAtMs + 1; + const applyCommand = { + schemaVersion: 1, + operation: 'local.deployment.reconciliation.secret-config.apply', + options: applyOptions, + request: { + decisionId, + secretConfigId: state.secretConfigId, + expectedDecisionDigest: decision.decisionDigest, + expectedHeadDigest: decision.instanceHeadDigest, + mutationId: options.mutationId ?? '00000000-0000-4000-8000-000000000525', + requestId: `secret-config-apply-${suffix}`, + appliedAtMs, + }, + }; + const applyDependencies = { + async openAuthenticationDatabase() { + return { async close() {} }; + }, + async authenticate(_database, authenticationOptions) { + const authenticatedAtMs = authenticationOptions.now(); + return { + principal: { + subject: { type: 'user', id: 'review-owner' }, + authenticationId: 'local_reconciliation_secret_config_apply:test', + authenticatedAtMs, + expiresAtMs: authenticatedAtMs + 60 * 60 * 1_000, + assurance: 'local_console', + }, + databaseFence: { + credentialId: 'review-owner', + credentialVersion: 1, + pepperKeyId: 'review-owner-v1', + pepperVersion: 1, + }, + async confirm() {}, + }; + }, + async applyApplication(input) { + insertSecretConfigOwnerBinding(state.targetDatabasePath); + return applyPreparedReconciliationSecretConfigApplication(input); + }, + }; + const applied = await applyLocalReconciliationSecretConfig( + applyCommand, + applyDependencies, + ); + return { + ...state, + decisionId, + decision, + secretConfigApplyRoot, + applyOptions, + applyCommand, + applyDependencies, + applied, + }; +} + function dockerReadSealedSqlite(assetsDirectory, mode) { const source = mode === 'main_only_immutable' @@ -4205,6 +4352,312 @@ test('Secret/Config apply publishes encrypted material atomically and recovers e assert.equal(databaseCloses, authentications); }); +test('completion v3 proves Secret/Config apply but preserves rollback authority while other domains remain manual', async (t) => { + const state = await appliedSecretConfigFixture(t, { + suffix: 'completion-v3', + }); + const completionRoot = path.join( + path.dirname(state.captureRoot), + 'completion-secret-config-v3', + ); + fs.mkdirSync(completionRoot, { mode: 0o700 }); + const secretConfig = { + secretConfigId: state.secretConfigId, + decisionId: state.decisionId, + expectedApplyDigest: state.applied.applyDigest, + }; + const command = { + schemaVersion: 3, + operation: 'local.deployment.reconciliation.complete', + options: { + deploymentRoot: state.deploymentRoot, + applicationRoot: state.applicationRoot, + completionRoot, + automation: null, + secretConfig: { + secretConfigRoot: state.secretConfigRoot, + secretConfigDecisionRoot: state.secretConfigDecisionRoot, + secretConfigApplyRoot: state.secretConfigApplyRoot, + targetDatabasePath: state.targetDatabasePath, + }, + runHistory: null, + allowRootService: rootAcknowledgement(), + }, + request: { + completionId: '00000000-0000-4000-8000-000000000526', + applicationId: state.application.applicationId, + expectedApplicationPlanDigest: state.application.applicationPlanDigest, + expectedHeadDigest: state.applied.instanceHeadDigest, + automation: null, + secretConfig, + runHistory: null, + completedAtMs: state.applyCommand.request.appliedAtMs + 1, + }, + }; + const applyRoot = path.join( + state.secretConfigApplyRoot, + state.secretConfigId, + ); + const backupRoot = path.join(applyRoot, 'backup'); + const backupPath = path.join(backupRoot, 'before.sqlite'); + const materialPath = path.join(applyRoot, 'materials.ndjson'); + assert.equal(fs.existsSync(backupPath), true); + + await assert.rejects( + completeLocalReconciliation(command), + /identity_policy_audit is not terminally reconciled/, + ); + assert.equal(fs.existsSync(backupPath), true); + assert.equal( + readLocalCutoverInstanceHead( + state.deploymentRoot, + state.captureCommand.request.instanceId, + state.uid, + ).state, + 'reconciliation_secret_config_applied', + ); + assert.equal( + fs.existsSync(path.join(completionRoot, command.request.completionId)), + false, + ); + + const domainNames = [ + 'schema_lineage', + 'automation', + 'secret_and_config', + 'run_history', + 'plugin_package', + 'ai_and_tool', + 'identity_policy_audit', + 'unknown', + ]; + const domains = domainNames.map((domain) => + domain === 'secret_and_config' + ? { + domain, + action: 'adapter_required', + evidenceKind: 'secret_config_application', + evidenceDigest: state.applied.applyDigest, + } + : { + domain, + action: 'no_effect', + evidenceKind: 'application_summary', + evidenceDigest: 'a'.repeat(64), + }, + ); + const receipt = buildLocalReconciliationCompletionReceipt({ + completionId: command.request.completionId, + applicationId: command.request.applicationId, + profile: state.captureCommand.request.profile, + instanceId: state.captureCommand.request.instanceId, + cutoverId: state.captureCommand.request.cutoverId, + generation: 1, + activationDigest: state.captureCommand.request.expectedActivationDigest, + applicationPlanDigest: state.application.applicationPlanDigest, + sourceHeadDigest: state.applied.instanceHeadDigest, + domains, + adapterCount: 1, + completedAtMs: command.request.completedAtMs, + }); + assert.equal(receipt.schemaVersion, 3); + assert.deepEqual( + normalizeLocalReconciliationCompletionReceipt(receipt), + receipt, + ); + + const currentHead = readLocalCutoverInstanceHead( + state.deploymentRoot, + state.captureCommand.request.instanceId, + state.uid, + ); + const durableHead = advanceLocalCutoverInstanceHead( + { + options: { deploymentRoot: state.deploymentRoot }, + request: { + cutoverId: state.captureCommand.request.cutoverId, + profile: state.captureCommand.request.profile, + instanceId: state.captureCommand.request.instanceId, + expectedActivationDigest: + state.captureCommand.request.expectedActivationDigest, + requestedAtMs: command.request.completedAtMs, + }, + }, + state.uid, + 'reconciliation_completed', + currentHead.generation, + receipt.completionDigest, + ); + assert.equal(durableHead.state, 'reconciliation_completed'); + const selected = localReconciliationSecretConfigApplyPaths( + state.secretConfigApplyRoot, + state.secretConfigId, + ); + const intent = readLocalReconciliationSecretConfigApplyIntent( + selected, + state.uid, + ); + collectLocalReconciliationSecretConfigCompletedStorage( + selected, + intent, + state.uid, + ); + collectLocalReconciliationSecretConfigCompletedStorage( + selected, + intent, + state.uid, + ); + validateLocalReconciliationSecretConfigCompletedStorage( + selected, + intent, + state.uid, + ); + assert.equal(fs.existsSync(backupPath), false); + assert.deepEqual(fs.readdirSync(backupRoot), []); + assert.deepEqual(fs.readdirSync(path.join(applyRoot, 'rollback-work')), []); + assert.equal(fs.statSync(backupRoot).mode & 0o777, 0o500); + assert.equal(fs.statSync(materialPath).mode & 0o777, 0o400); + assert.ok(fs.statSync(materialPath).size < 64 * 1024); + assert.deepEqual( + receipt.domains.find((domain) => domain.domain === 'secret_and_config'), + { + domain: 'secret_and_config', + action: 'adapter_required', + evidenceKind: 'secret_config_application', + evidenceDigest: state.applied.applyDigest, + }, + ); +}); + +test('completion v3 rejects rolled-back Secret/Config evidence', async (t) => { + const state = await appliedSecretConfigFixture(t, { + suffix: 'completion-rolled-back', + planId: '00000000-0000-4000-8000-000000000531', + reviewId: '00000000-0000-4000-8000-000000000532', + applicationId: '00000000-0000-4000-8000-000000000533', + secretConfigId: '00000000-0000-4000-8000-000000000534', + decisionId: '019b0000-0000-7000-8000-000000000531', + mutationId: '00000000-0000-4000-8000-000000000535', + }); + const rollback = await rollbackLocalReconciliationSecretConfigApply( + { + schemaVersion: 1, + operation: 'local.deployment.reconciliation.secret-config.apply.rollback', + options: state.applyOptions, + request: { + decisionId: state.decisionId, + secretConfigId: state.secretConfigId, + expectedApplyDigest: state.applied.applyDigest, + expectedHeadDigest: state.applied.instanceHeadDigest, + rolledBackAtMs: state.applyCommand.request.appliedAtMs + 1, + }, + }, + state.applyDependencies, + ); + const completionRoot = path.join( + path.dirname(state.captureRoot), + 'completion-secret-config-rolled-back', + ); + fs.mkdirSync(completionRoot, { mode: 0o700 }); + await assert.rejects( + completeLocalReconciliation({ + schemaVersion: 3, + operation: 'local.deployment.reconciliation.complete', + options: { + deploymentRoot: state.deploymentRoot, + applicationRoot: state.applicationRoot, + completionRoot, + automation: null, + secretConfig: { + secretConfigRoot: state.secretConfigRoot, + secretConfigDecisionRoot: state.secretConfigDecisionRoot, + secretConfigApplyRoot: state.secretConfigApplyRoot, + targetDatabasePath: state.targetDatabasePath, + }, + runHistory: null, + allowRootService: rootAcknowledgement(), + }, + request: { + completionId: '00000000-0000-4000-8000-000000000536', + applicationId: state.application.applicationId, + expectedApplicationPlanDigest: state.application.applicationPlanDigest, + expectedHeadDigest: rollback.instanceHeadDigest, + automation: null, + secretConfig: { + secretConfigId: state.secretConfigId, + decisionId: state.decisionId, + expectedApplyDigest: state.applied.applyDigest, + }, + runHistory: null, + completedAtMs: state.applyCommand.request.appliedAtMs + 2, + }, + }), + /secret config apply evidence is detached/, + ); +}); + +test('completion v3 rejects Secret/Config target drift without collecting rollback authority', async (t) => { + const state = await appliedSecretConfigFixture(t, { + suffix: 'completion-target-drift', + planId: '00000000-0000-4000-8000-000000000541', + reviewId: '00000000-0000-4000-8000-000000000542', + applicationId: '00000000-0000-4000-8000-000000000543', + secretConfigId: '00000000-0000-4000-8000-000000000544', + decisionId: '019b0000-0000-7000-8000-000000000541', + mutationId: '00000000-0000-4000-8000-000000000545', + }); + const target = new DatabaseSync(state.targetDatabasePath); + target.exec('PRAGMA user_version=77'); + target.close(); + const completionRoot = path.join( + path.dirname(state.captureRoot), + 'completion-secret-config-target-drift', + ); + fs.mkdirSync(completionRoot, { mode: 0o700 }); + const backupPath = path.join( + state.secretConfigApplyRoot, + state.secretConfigId, + 'backup', + 'before.sqlite', + ); + await assert.rejects( + completeLocalReconciliation({ + schemaVersion: 3, + operation: 'local.deployment.reconciliation.complete', + options: { + deploymentRoot: state.deploymentRoot, + applicationRoot: state.applicationRoot, + completionRoot, + automation: null, + secretConfig: { + secretConfigRoot: state.secretConfigRoot, + secretConfigDecisionRoot: state.secretConfigDecisionRoot, + secretConfigApplyRoot: state.secretConfigApplyRoot, + targetDatabasePath: state.targetDatabasePath, + }, + runHistory: null, + allowRootService: rootAcknowledgement(), + }, + request: { + completionId: '00000000-0000-4000-8000-000000000546', + applicationId: state.application.applicationId, + expectedApplicationPlanDigest: state.application.applicationPlanDigest, + expectedHeadDigest: state.applied.instanceHeadDigest, + automation: null, + secretConfig: { + secretConfigId: state.secretConfigId, + decisionId: state.decisionId, + expectedApplyDigest: state.applied.applyDigest, + }, + runHistory: null, + completedAtMs: state.applyCommand.request.appliedAtMs + 1, + }, + }), + /secret config target drifted after apply/, + ); + assert.equal(fs.existsSync(backupPath), true); +}); + test('Secret/Config decision rejects manual plans, invalid candidate choices and reviewer drift', async (t) => { const manual = await plannedSecretConfigDecisionFixture(t, { suffix: 'decision-manual-plan',