diff --git a/deploy/containers/ql3-local-application/Dockerfile b/deploy/containers/ql3-local-application/Dockerfile index 9052d450..b44a9d72 100644 --- a/deploy/containers/ql3-local-application/Dockerfile +++ b/deploy/containers/ql3-local-application/Dockerfile @@ -143,9 +143,9 @@ LABEL org.opencontainers.image.title="QingLong 3.0 Local Application" \ io.qinglong.profile="edge,standalone" \ io.qinglong.ai="excluded" \ io.qinglong.local.application-config="2,3,4" \ - io.qinglong.local.sqlite-contract-min="50" \ - io.qinglong.local.sqlite-contract-max="50" \ - io.qinglong.local.sqlite-write-contract="50" \ + io.qinglong.local.sqlite-contract-min="51" \ + io.qinglong.local.sqlite-contract-max="51" \ + io.qinglong.local.sqlite-write-contract="51" \ io.qinglong.local.compose-selection="1" ENV NODE_ENV=production diff --git a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md index 781e9051..5e4ef54a 100644 --- a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md +++ b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md @@ -32,10 +32,17 @@ prepared head/bundle/reviewer/time。Edge/Standalone decision 与 authorization 文件上限分别为 1/4 MiB，沿用 owner-only、sealed、no-replace 与 `fsync`； lineage 仅允许 `reconciliation_secret_config_planned → reconciliation_secret_config_decision_prepared → reconciliation_secret_config_reviewed`，prepare/commit 的 publication response-loss 可精确重放而不重复认证，terminal verify 只读复算 sealed decision、authorization、receipt 与 reviewed head。 - 全部 evidence 不含原 Env name/value、目标 ciphertext/key ID 或 row body。Local Admin 完整测试 `96/96`，Local Owner 完整测试 `295/288/7/0`； - 后端完整门 `1564/1562/2/0`，18-package clean build/test `2942/2920/22/0`；package boundary、Cluster dependency `62/62`、Edge import 与十四档 - Local artifact audit 全部 compatible，基础 Edge/Standalone 仍为 `2,611,978 / 2,612,056 bytes`、319 files、58 loaded modules，Owner-only authority - 没有进入低资源常驻制品。 + 第五切片补上此前聚合 adoption ledger 无法安全定位 Task/Trigger 的架构缺口：Local SQLite contract v51 新增 + `QingLong3LegacyAdoptionTasks` 与 `QingLong3LegacyAdoptionTriggers`，Automation publisher 在原子事务内流式写入逐项 identity/revision/mutation/content/item + digest。Secret/Config planner 不再扫描 `legacy-cron:*` 或相信聚合计数，而是流式复算 provenance，并验证当前 Task/Trigger head、Trigger schedule 与非 + Plugin ownership。旧记录缺少逐项 provenance 为 `missing`，current revision/schedule/ownership 漂移为 `drifted`，两者都强制 + `manual_required`；不得猜测或自动回填。聚合 adoption 记录仍只保留 Edge/Standalone 128/512 条有界 Map，Task/Trigger 逐行读取，无目标规模 Set。 + 全部 evidence 不含原 Env name/value、目标 ciphertext/key ID 或 row body。Local SQLite、Local Admin 与 Local Owner 完整测试分别为 `241/241`、 + `96/96` 与 `296 total / 289 pass / 7 conditional skip / 0 fail`；18-package clean build/test 在非沙箱环境全部通过。后端完整门在受限沙箱中为 + `1544 total / 1531 pass / 11 loopback EPERM / 2 conditional skip`，四个受影响文件随后在非沙箱环境 `38/38` 通过，未发现代码失败。package + boundary、72 项 Cluster dependency/legacy boundary、Edge import、本地镜像与十四档 Local artifact audit 全部 compatible；基础 Edge/Standalone 为 + `2,620,531 / 2,620,609 bytes`、321 files、58 loaded modules，Owner-only authority 没有进入低资源常驻制品。fresh Edge readiness 为 contract v51、 + 102 migrations、85 tables、SQLite 3.53.3、`DELETE` journal；PostgreSQL 18.6 arm64 physical HA 以 timeline `1 → 2`、146 gates 通过。 Local Admin 保持 48/47，Local Owner 因六个职责明确的 decision 嵌套文件增至 184/183，根目录仍只有一个 50 行 binary entry；workspace 仍为 18 packages、 `singleSourcePackages=[]`、`shallowSourcePackages=[]`，且只允许 exact Secret/Config row planner 导入 inspection subpath。 diff --git a/docs/adr/ADR-0491-bounded-secret-config-reconciliation-and-task-binding.md b/docs/adr/ADR-0491-bounded-secret-config-reconciliation-and-task-binding.md index 2f4d241e..601caa57 100644 --- a/docs/adr/ADR-0491-bounded-secret-config-reconciliation-and-task-binding.md +++ b/docs/adr/ADR-0491-bounded-secret-config-reconciliation-and-task-binding.md @@ -1,6 +1,6 @@ # ADR-0491：有界 Secret/Config Reconciliation 与任务环境绑定 -- 状态：Proposed（D-397 已实现 Legacy Env inspection、私有有界 row plan、durable plan publication 与独立 signed decision；原子 application 尚未完成） +- 状态：Proposed（D-397 已实现 Legacy Env inspection、私有有界 row plan、durable plan publication、独立 signed decision 与逐项 Automation adoption provenance；原子 application 尚未完成） - 日期：2026-08-23 - 决策：D-397 - 关联：ADR-0073、ADR-0074、ADR-0092、ADR-0094、ADR-0480、ADR-0482、ADR-0483、ADR-0484、ADR-0485、ADR-0486、ADR-0487、ADR-0488、ADR-0490 @@ -29,7 +29,9 @@ application plan Secret/Config 不消费 Automation decision 作为自身授权。D-391 把 `secret_and_config` facts 标为 `blocked`，其 review action 只能是 `manual_external|defer`；D-397 专用 adapter 只接收每条 fact 都精确选择 `manual_external` 的决策流，把它重新绑定到同一 sealed bundle、D-392 application plan 与当前 target snapshot，任何 `defer` 都继续失败关闭。D-397 使用独立的逐候选 signed decision：只有 `plan.outcome=ready`、候选非空且没有 manual/conflict 才能 prepare；后续 application 必须消费该终态授权。强认证 User、Project Policy、Secret custody 与 Task mutation authority 都要在写事务前及事务内重新验证。 -存在 active Env 时，Automation 必须已经完成，且至少一个经 `QingLong3LegacyAdoptions` 证明的 Legacy Task 可绑定；否则不得用“Secret 已保存”冒充行为迁移。只有停用 Env 的场景可以在 Automation `no_effect` 后做纯保全。 +存在 active Env 时，Automation 必须已经完成，且至少一个同时经 `QingLong3LegacyAdoptions` 聚合账本和逐项 provenance 证明的 Legacy Task 可绑定；否则不得用“Secret 已保存”冒充行为迁移。只有停用 Env 的场景可以在 Automation `no_effect` 后做纯保全。 + +聚合计数不能作为 Task/Trigger 写权限。Local v51 增加 `QingLong3LegacyAdoptionTasks` 与 `QingLong3LegacyAdoptionTriggers`，由 Automation publisher 在原 `BEGIN IMMEDIATE` 中逐项写入 adoption mutation、source digest、Task/Trigger identity、revision、mutation、content digest、ordinal 与 item digest。Task provenance 精确引用 revision 1；Trigger provenance 同时引用 Task provenance、Trigger revision 1 与原 adoption。父 adoption 外键延迟到事务提交检查，使 Edge/Standalone 可以流式发布最多 100,000 个 Task 与 500,000 个 Trigger，无需在内存保留全集。旧 pre-release 数据若只有聚合 ledger 而没有逐项 provenance，只能判定 `missing → manual_required`，不得按 `legacy-cron:*` 扫描、猜测或回填所有权。 ### 2. Legacy Env 的确定性语义 @@ -64,7 +66,7 @@ id ASC 实现逐行读取，不把整张 `Envs` 或全部停用值加载到内存；active value 的在途内存由 64 KiB 合同封顶，停用值通过第二次有界扫描逐个交付。它位于既有 `@qinglong/local-admin/src/legacy-adoption/secret-and-config/`，不新增 workspace package、production dependency、daemon、timer、watcher、listener、socket、数据库连接池或 `src` 根平铺文件。 -Local Owner 使用私有 NDJSON row plan 记录 header、逐行 content-free disposition、逐 candidate 目标冲突投影与 footer。Edge/Standalone plan 文件分别限制为 8 MiB/32 MiB，单行不超过 64 KiB；超过预算立即失败关闭。公开 plan/receipt 不保存原 Env name/value、目标 ciphertext、key ID 或原始 row body。active 与 disabled candidate 分别使用 `legacy-db-env-*` 和 `legacy-db-env-disabled-*` 命名空间；目标已经存在时只记录 envelope 元数据的组合摘要并进入 `review_skip_conflict`，不得读取明文、覆盖或自动改名。plan 绑定 application、D-391 review authorization、sealed bundle、target projection、Automation adoption ledger 的有界 content-free 投影与 prepared head，并产生可重新计算的 row-set、candidate-set、adoption-set、plan-file 和 receipt digest。 +Local Owner 使用私有 NDJSON row plan 记录 header、逐行 content-free disposition、逐 candidate 目标冲突投影与 footer。Edge/Standalone plan 文件分别限制为 8 MiB/32 MiB，单行不超过 64 KiB；超过预算立即失败关闭。公开 plan/receipt 不保存原 Env name/value、目标 ciphertext、key ID 或原始 row body。active 与 disabled candidate 分别使用 `legacy-db-env-*` 和 `legacy-db-env-disabled-*` 命名空间；目标已经存在时只记录 envelope 元数据的组合摘要并进入 `review_skip_conflict`，不得读取明文、覆盖或自动改名。plan 绑定 application、D-391 review authorization、sealed bundle、target projection、Automation adoption ledger 及逐项 provenance 的有界 content-free 投影与 prepared head，并产生可重新计算的 row-set、candidate-set、adoption-set、plan-file 和 receipt digest。planner 流式复算每个 provenance item digest，并要求当前 Task/Trigger head 仍精确指向记录的 revision、mutation 与 content digest，Trigger schedule 仍精确指向记录的 Trigger revision，且 Task 不属于 Plugin Package。计数缺项形成 `missing`，当前对象、schedule 或 ownership 漂移形成 `drifted`；两者都只能 `manual_required`。投影只在内存保留最多 128/512 条 adoption 聚合记录，不按目标 Task/Trigger 总量建立 Set。 durable publisher 固定写入 `//{plan.ndjson,receipt.json,staging/}`，使用 no-replace publication、`0400/0500` 权限、文件与目录 `fsync`，并覆盖 plan、receipt、terminal seal、head CAS 四个 response-loss 窗口。只有 Automation 无需 adapter 时的 `reconciliation_application_planned`，或 Automation 已完成时的 `reconciliation_automation_applied`，可以单向推进到 `reconciliation_secret_config_planned`；verify 只读复算 plan/receipt/seal/head 绑定，不修复漂移。active Env 若没有至少一条已采纳 Legacy Task ledger 记录仍为 manual；历史 `Configs` 计入 `unadaptedLegacyConfigCount` 并保持 manual。 @@ -135,6 +137,8 @@ Cluster 不得把 Legacy Env 明文写入 PostgreSQL、ConfigMap、Job command ## 当前验证与后续门禁 -D-397 当前四切片已经实现并测试：absent、unsupported、Edge over-budget、2.x 顺序、同名连接、disabled preservation、保留前缀、异常状态、effective overflow、candidate digest、content-free diagnostics、私有有界 row plan、目标 Secret 冲突、Automation adoption projection、no-effect/manual outcome、durable no-replace publication、terminal seal、head CAS、逐候选独立 signed decision、同一强认证 reviewer、decision/authorization byte bound、`skip → manual_required`、prepare/commit response-loss exact replay、只读 terminal verify 与 plan/receipt/authorization/head drift。调用方 visitor 的预算异常保持原始类型，不再被误报为 SQLite 读取失败。Local Admin 完整测试为 96/96；Local Owner 完整测试为 295/288/7/0；后端完整门为 1564/1562/2/0，18-package clean build/test 为 2942/2920/22/0。package boundary、Cluster dependency 62/62、Edge import 与十四档 Local artifact audit 全部 compatible；基础 Edge/Standalone 仍为 2,611,978 / 2,612,056 bytes、319 files、58 loaded modules，Owner-only authority 没有进入低资源常驻制品。workspace 保持 18 packages、`singleSourcePackages=[]`、`shallowSourcePackages=[]`。Local Admin 保持 48/47，Local Owner 随六个职责明确的 decision 嵌套文件增至 184/183；根目录仍只有一个 50 行 binary entry，没有新增平铺源文件。依赖隔离门只允许 exact Secret/Config row planner 导入 inspection subpath，并只允许 decision authorization/coordinator 导入各自最小密钥、principal 与强认证只读 authority；邻接文件继续被拒绝。 +D-397 当前五切片已经实现：absent、unsupported、Edge over-budget、2.x 顺序、同名连接、disabled preservation、保留前缀、异常状态、effective overflow、candidate digest、content-free diagnostics、私有有界 row plan、目标 Secret 冲突、Automation adoption projection、no-effect/manual outcome、durable no-replace publication、terminal seal、head CAS、逐候选独立 signed decision、同一强认证 reviewer、decision/authorization byte bound、`skip → manual_required`、prepare/commit response-loss exact replay、只读 terminal verify，以及 v51 逐 Task/Trigger adoption provenance。新 provenance 与 Automation Task/Trigger/audit/ledger 在同一事务发布并在 replay 时完整流式复算；Secret/Config plan 对 complete、missing、current-revision drift 和 Plugin ownership 失败关闭。 -转为 Accepted 前仍必须完成：原子 Secret/Task/Trigger/dispatch publisher、prepared/apply/rollback response-loss、completion schema 演进、18-package/boundary/artifact gates、真实 Edge 空间预算、PostgreSQL HA 与 Cluster Secret provider live gate。 +本切片验证已闭合：Local SQLite `241/241`、Local Admin `96/96`、Local Owner `296 total / 289 pass / 7 conditional skip / 0 fail`，Secret/Config 定向回归 `51/51`；18-package clean build/test 在非沙箱环境全部通过。后端完整门在受限沙箱中仅有 11 个 loopback `listen EPERM`，四个受影响文件在非沙箱环境 `38/38` 通过。package boundary 保持 18 packages、`singleSourcePackages=[]`、`shallowSourcePackages=[]`，72 项 Cluster dependency/legacy boundary、Edge import、本地镜像与十四档 artifact audit 全 compatible；基础 Edge/Standalone 为 `2,620,531 / 2,620,609 bytes`、321 files、58 modules。fresh Edge readiness 为 contract v51、102 migrations、85 tables、SQLite 3.53.3、`DELETE` journal。PostgreSQL 18.6 arm64 physical HA 以 146 gates、timeline `1 → 2` 通过；两次前置 `initdb` 超时均由本机 Docker 虚拟磁盘 100% 占满导致，释放可重建缓存后同一门成功。 + +转为 Accepted 前仍必须完成：原子 Secret/Task/Trigger/dispatch publisher、prepared/apply/rollback response-loss、completion schema 演进、在后续原子 application 切片上重新执行 18-package/boundary/artifact gates、真实 Edge 空间预算与 Cluster Secret provider live gate；本切片的 PostgreSQL HA 证据不得冒充后续 Cluster 密钥托管证明。 diff --git a/docs/ql3-package-boundaries.json b/docs/ql3-package-boundaries.json index 7a1eeadf..fa60f34f 100644 --- a/docs/ql3-package-boundaries.json +++ b/docs/ql3-package-boundaries.json @@ -12,7 +12,7 @@ }, { "kind": "ordered_ledger", - "maxDirectSourceFiles": 101, + "maxDirectSourceFiles": 103, "path": "packages/ql3-local-sqlite/src/migrations", "rationale": "SQLite migrations are an append-only version ledger whose ordering and discoverability are safer in one reviewed directory." } diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/secret-and-config/planReader.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/secret-and-config/planReader.ts index f8b057d2..2b2104b9 100644 --- a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/secret-and-config/planReader.ts +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/secret-and-config/planReader.ts @@ -405,7 +405,11 @@ export function createLocalReconciliationSecretConfigDecisionRequirementFactory( 'activeGroupCount', 'activeRowCount', 'adoptedLegacyTaskCount', + 'adoptedLegacyTriggerCount', + 'adoptionProvenanceTaskCount', + 'adoptionProvenanceTriggerCount', 'automationAdoptionRecordCount', + 'automationAdoptionProvenanceState', 'automationAdoptionSetDigest', 'bindingReadyCount', 'candidateSetDigest', @@ -441,6 +445,18 @@ export function createLocalReconciliationSecretConfigDecisionRequirementFactory( footer.eligiblePreservationCount !== receipt.eligiblePreservationCount || footer.targetConflictCount !== receipt.targetConflictCount || + footer.automationAdoptionRecordCount !== + receipt.automationAdoptionRecordCount || + footer.adoptedLegacyTaskCount !== + receipt.adoptedLegacyTaskCount || + footer.adoptedLegacyTriggerCount !== + receipt.adoptedLegacyTriggerCount || + footer.adoptionProvenanceTaskCount !== + receipt.adoptionProvenanceTaskCount || + footer.adoptionProvenanceTriggerCount !== + receipt.adoptionProvenanceTriggerCount || + footer.automationAdoptionProvenanceState !== + receipt.automationAdoptionProvenanceState || footer.outcome !== receipt.outcome || candidateCount !== receipt.eligibleBindingCount + diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/secret-and-config/rowPlan.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/secret-and-config/rowPlan.ts index 54b8a4d9..83c626c9 100644 --- a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/secret-and-config/rowPlan.ts +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/secret-and-config/rowPlan.ts @@ -8,6 +8,11 @@ import { type LegacyEnvironmentInventory, type LegacyEnvironmentRowInspection, } from '@qinglong/local-admin/reconciliation-secret-and-config-inspection'; +import { + LegacyAdoptionPublicationDigest, + legacyAdoptionTaskProvenanceDigest, + legacyAdoptionTriggerProvenanceDigest, +} from '@qinglong/local-sqlite/adoption-provenance'; import { LocalDeploymentConfigurationError } from '../../../foundation/error'; import { cutoverDigest } from '../../../cutover/targetEvidence'; @@ -26,6 +31,8 @@ const MAX_LINE_BYTES = 64 * 1024; const HASH_BUFFER_BYTES = 64 * 1024; const MAX_EDGE_AUTOMATION_ADOPTION_RECORDS = 128; const MAX_STANDALONE_AUTOMATION_ADOPTION_RECORDS = 512; +const MAX_AUTOMATION_ADOPTION_TASKS = 100_000; +const MAX_AUTOMATION_ADOPTION_TRIGGERS = 500_000; export const MAX_EDGE_LOCAL_RECONCILIATION_SECRET_CONFIG_PLAN_BYTES = 8 * 1024 * 1024; export const MAX_STANDALONE_LOCAL_RECONCILIATION_SECRET_CONFIG_PLAN_BYTES = @@ -104,6 +111,13 @@ export interface LocalReconciliationSecretConfigPlanSummary { readonly targetConflictCount: number; readonly automationAdoptionRecordCount: number; readonly adoptedLegacyTaskCount: number; + readonly adoptedLegacyTriggerCount: number; + readonly adoptionProvenanceTaskCount: number; + readonly adoptionProvenanceTriggerCount: number; + readonly automationAdoptionProvenanceState: + | 'complete' + | 'missing' + | 'drifted'; readonly unadaptedLegacyConfigCount: number; readonly outcome: 'ready' | 'manual_required' | 'no_effect'; } @@ -248,6 +262,10 @@ function targetAutomationAdoptionProjection( ): Readonly<{ recordCount: number; adoptedTaskCount: number; + adoptedTriggerCount: number; + provenanceTaskCount: number; + provenanceTriggerCount: number; + provenanceState: 'complete' | 'missing' | 'drifted'; setDigest: string; }> { const maximumRecords = @@ -257,8 +275,23 @@ function targetAutomationAdoptionProjection( const hash = createHash('sha256').update( 'qinglong3.local-reconciliation-secret-config-automation-adoption-set.v1\0', ); + const records = new Map< + string, + { + readonly expectedTaskCount: number; + readonly expectedTriggerCount: number; + readonly expectedPublicationDigest: string; + readonly publication: LegacyAdoptionPublicationDigest; + taskCount: number; + triggerCount: number; + } + >(); let recordCount = 0; let adoptedTaskCount = 0; + let adoptedTriggerCount = 0; + let provenanceTaskCount = 0; + let provenanceTriggerCount = 0; + let drifted = false; try { const rows = target .prepare( @@ -300,6 +333,15 @@ function targetAutomationAdoptionProjection( if (!Number.isSafeInteger(adoptedTaskCount)) { fail('target Automation adoption task count overflowed'); } + const selectedAdoptedTriggerCount = adoptionCount( + row, + 'adoptedTriggerCount', + 500_000, + ); + adoptedTriggerCount += selectedAdoptedTriggerCount; + if (!Number.isSafeInteger(adoptedTriggerCount)) { + fail('target Automation adoption trigger count overflowed'); + } const payload = Object.freeze({ mutationId: adoptionText(row, 'mutationId', UUID_V4_PATTERN), decisionId: adoptionText( @@ -327,11 +369,7 @@ function targetAutomationAdoptionProjection( ), rowCount, adoptedTaskCount: selectedAdoptedTaskCount, - adoptedTriggerCount: adoptionCount( - row, - 'adoptedTriggerCount', - 500_000, - ), + adoptedTriggerCount: selectedAdoptedTriggerCount, skippedCount, auditEventId: adoptionText(row, 'auditEventId', UUID_V4_PATTERN), createdAtMs: adoptionCount( @@ -343,15 +381,366 @@ function targetAutomationAdoptionProjection( if (payload.auditEventId !== payload.mutationId) { fail('target Automation adoption audit binding drifted'); } + if (records.has(payload.mutationId)) { + fail('target Automation adoption identity is duplicated'); + } + records.set(payload.mutationId, { + expectedTaskCount: selectedAdoptedTaskCount, + expectedTriggerCount: selectedAdoptedTriggerCount, + expectedPublicationDigest: payload.publicationDigest, + publication: new LegacyAdoptionPublicationDigest(payload.mutationId), + taskCount: 0, + triggerCount: 0, + }); hash.update('\0').update(JSON.stringify(payload)); } + + const tasks = target + .prepare( + `SELECT provenance."adoption_mutation_id" AS "adoptionMutationId", + provenance."row_ordinal" AS "rowOrdinal", + provenance."project_id" AS "projectId", + provenance."source_digest" AS "sourceDigest", + provenance."task_id" AS "taskId", + provenance."task_revision" AS "taskRevision", + provenance."task_mutation_id" AS "taskMutationId", + provenance."task_content_digest" AS "taskContentDigest", + provenance."trigger_count" AS "triggerCount", + provenance."item_digest" AS "itemDigest", + head."current_revision" AS "currentRevision", + revision."mutation_id" AS "storedMutationId", + revision."content_digest" AS "storedContentDigest", + ownership."package_name" AS "packageName" + FROM "QingLong3LegacyAdoptionTasks" AS provenance + JOIN "QingLong3LegacyAdoptions" AS adoption + ON adoption."mutation_id" = provenance."adoption_mutation_id" + AND adoption."project_id" = provenance."project_id" + LEFT JOIN "QingLong3TaskDefinitions" AS head + ON head."project_id" = provenance."project_id" + AND head."task_id" = provenance."task_id" + LEFT JOIN "QingLong3TaskDefinitionRevisions" AS revision + ON revision."project_id" = provenance."project_id" + AND revision."task_id" = provenance."task_id" + AND revision."revision" = provenance."task_revision" + LEFT JOIN "QingLong3PluginPackageTaskOwnerships" AS ownership + ON ownership."project_id" = provenance."project_id" + AND ownership."task_id" = provenance."task_id" + WHERE adoption."project_id" = ? + ORDER BY adoption."created_at_ms" ASC, + adoption."mutation_id" ASC, + provenance."row_ordinal" ASC`, + ) + .iterate(projectId) as Iterable>>; + for (const row of tasks) { + provenanceTaskCount += 1; + if (provenanceTaskCount > MAX_AUTOMATION_ADOPTION_TASKS) { + fail('target Automation adoption Task provenance exceeds budget'); + } + const adoptionMutationId = adoptionText( + row, + 'adoptionMutationId', + UUID_V4_PATTERN, + ); + const selected = records.get(adoptionMutationId); + const rowOrdinal = adoptionCount( + row, + 'rowOrdinal', + MAX_AUTOMATION_ADOPTION_TASKS, + ); + const triggerCount = adoptionCount( + row, + 'triggerCount', + MAX_AUTOMATION_ADOPTION_TRIGGERS, + ); + const payload = Object.freeze({ + adoptionMutationId, + rowOrdinal, + projectId: adoptionText(row, 'projectId'), + sourceDigest: adoptionText(row, 'sourceDigest', DIGEST_PATTERN), + taskId: adoptionText(row, 'taskId'), + taskRevision: adoptionCount(row, 'taskRevision', 1), + taskMutationId: adoptionText(row, 'taskMutationId', UUID_V4_PATTERN), + taskContentDigest: adoptionText( + row, + 'taskContentDigest', + DIGEST_PATTERN, + ), + triggerCount, + }); + const itemDigest = adoptionText(row, 'itemDigest', DIGEST_PATTERN); + selected && (selected.taskCount += 1); + const currentState = Object.freeze({ + currentRevision: row.currentRevision, + storedMutationId: row.storedMutationId, + storedContentDigest: row.storedContentDigest, + pluginOwned: row.packageName !== null, + }); + if ( + !selected || + payload.projectId !== projectId || + payload.rowOrdinal < 1 || + payload.taskRevision !== 1 || + legacyAdoptionTaskProvenanceDigest(payload) !== itemDigest || + currentState.currentRevision !== payload.taskRevision || + currentState.storedMutationId !== payload.taskMutationId || + currentState.storedContentDigest !== payload.taskContentDigest || + currentState.pluginOwned + ) { + drifted = true; + } + hash.update('\0task\0').update( + JSON.stringify({ + ...payload, + itemDigest, + currentState, + }), + ); + } + + const triggers = target + .prepare( + `SELECT provenance."adoption_mutation_id" AS "adoptionMutationId", + provenance."row_ordinal" AS "rowOrdinal", + provenance."trigger_ordinal" AS "triggerOrdinal", + provenance."project_id" AS "projectId", + provenance."task_id" AS "taskId", + provenance."task_revision" AS "taskRevision", + provenance."trigger_id" AS "triggerId", + provenance."trigger_revision" AS "triggerRevision", + provenance."trigger_mutation_id" AS "triggerMutationId", + provenance."trigger_content_digest" AS "triggerContentDigest", + provenance."item_digest" AS "itemDigest", + head."current_revision" AS "currentRevision", + revision."mutation_id" AS "storedMutationId", + revision."content_digest" AS "storedContentDigest", + schedule."trigger_revision" AS "scheduleRevision" + FROM "QingLong3LegacyAdoptionTriggers" AS provenance + JOIN "QingLong3LegacyAdoptions" AS adoption + ON adoption."mutation_id" = provenance."adoption_mutation_id" + AND adoption."project_id" = provenance."project_id" + LEFT JOIN "QingLong3Triggers" AS head + ON head."project_id" = provenance."project_id" + AND head."trigger_id" = provenance."trigger_id" + AND head."task_id" = provenance."task_id" + LEFT JOIN "QingLong3TriggerRevisions" AS revision + ON revision."project_id" = provenance."project_id" + AND revision."trigger_id" = provenance."trigger_id" + AND revision."revision" = provenance."trigger_revision" + LEFT JOIN "QingLong3LocalTriggerSchedules" AS schedule + ON schedule."project_id" = provenance."project_id" + AND schedule."trigger_id" = provenance."trigger_id" + WHERE adoption."project_id" = ? + ORDER BY adoption."created_at_ms" ASC, + adoption."mutation_id" ASC, + provenance."row_ordinal" ASC, + provenance."trigger_ordinal" ASC`, + ) + .iterate(projectId) as Iterable>>; + let previousTaskKey = ''; + let previousTriggerOrdinal = 0; + for (const row of triggers) { + provenanceTriggerCount += 1; + if (provenanceTriggerCount > MAX_AUTOMATION_ADOPTION_TRIGGERS) { + fail('target Automation adoption Trigger provenance exceeds budget'); + } + const adoptionMutationId = adoptionText( + row, + 'adoptionMutationId', + UUID_V4_PATTERN, + ); + const selected = records.get(adoptionMutationId); + const rowOrdinal = adoptionCount( + row, + 'rowOrdinal', + MAX_AUTOMATION_ADOPTION_TASKS, + ); + const triggerOrdinal = adoptionCount( + row, + 'triggerOrdinal', + MAX_AUTOMATION_ADOPTION_TRIGGERS, + ); + const taskKey = `${adoptionMutationId}\0${rowOrdinal}`; + if (taskKey !== previousTaskKey) { + previousTaskKey = taskKey; + previousTriggerOrdinal = 0; + } + previousTriggerOrdinal += 1; + const payload = Object.freeze({ + adoptionMutationId, + rowOrdinal, + triggerOrdinal, + projectId: adoptionText(row, 'projectId'), + taskId: adoptionText(row, 'taskId'), + taskRevision: adoptionCount(row, 'taskRevision', 1), + triggerId: adoptionText(row, 'triggerId'), + triggerRevision: adoptionCount(row, 'triggerRevision', 1), + triggerMutationId: adoptionText( + row, + 'triggerMutationId', + UUID_V4_PATTERN, + ), + triggerContentDigest: adoptionText( + row, + 'triggerContentDigest', + DIGEST_PATTERN, + ), + }); + const itemDigest = adoptionText(row, 'itemDigest', DIGEST_PATTERN); + selected && (selected.triggerCount += 1); + const currentState = Object.freeze({ + currentRevision: row.currentRevision, + storedMutationId: row.storedMutationId, + storedContentDigest: row.storedContentDigest, + scheduleRevision: row.scheduleRevision, + }); + if ( + !selected || + payload.projectId !== projectId || + payload.rowOrdinal < 1 || + payload.triggerOrdinal !== previousTriggerOrdinal || + payload.taskRevision !== 1 || + payload.triggerRevision !== 1 || + legacyAdoptionTriggerProvenanceDigest(payload) !== itemDigest || + currentState.currentRevision !== payload.triggerRevision || + currentState.storedMutationId !== payload.triggerMutationId || + currentState.storedContentDigest !== payload.triggerContentDigest || + currentState.scheduleRevision !== payload.triggerRevision + ) { + drifted = true; + } + hash.update('\0trigger\0').update( + JSON.stringify({ + ...payload, + itemDigest, + currentState, + }), + ); + } + + const publicationRows = target + .prepare( + `SELECT adoption."mutation_id" AS "adoptionMutationId", + task."row_ordinal" AS "rowOrdinal", + task."source_digest" AS "sourceDigest", + task."task_content_digest" AS "taskContentDigest", + task."trigger_count" AS "expectedTriggerCount", + task."item_digest" AS "taskItemDigest", + trigger."trigger_ordinal" AS "triggerOrdinal", + trigger."trigger_content_digest" AS "triggerContentDigest", + trigger."item_digest" AS "triggerItemDigest" + FROM "QingLong3LegacyAdoptions" AS adoption + LEFT JOIN "QingLong3LegacyAdoptionTasks" AS task + ON task."adoption_mutation_id" = adoption."mutation_id" + AND task."project_id" = adoption."project_id" + LEFT JOIN "QingLong3LegacyAdoptionTriggers" AS trigger + ON trigger."adoption_mutation_id" = task."adoption_mutation_id" + AND trigger."row_ordinal" = task."row_ordinal" + WHERE adoption."project_id" = ? + ORDER BY adoption."created_at_ms" ASC, + adoption."mutation_id" ASC, + task."row_ordinal" ASC, + trigger."trigger_ordinal" ASC`, + ) + .iterate(projectId) as Iterable>>; + let publicationTaskKey = ''; + let publicationExpectedTriggerCount = 0; + let publicationTriggerCount = 0; + for (const row of publicationRows) { + const adoptionMutationId = adoptionText( + row, + 'adoptionMutationId', + UUID_V4_PATTERN, + ); + const selected = records.get(adoptionMutationId); + if (!selected) { + drifted = true; + continue; + } + if (row.rowOrdinal === null) continue; + const rowOrdinal = adoptionCount( + row, + 'rowOrdinal', + MAX_AUTOMATION_ADOPTION_TASKS, + ); + const taskKey = `${adoptionMutationId}\0${rowOrdinal}`; + if (taskKey !== publicationTaskKey) { + if ( + publicationTaskKey !== '' && + publicationTriggerCount !== publicationExpectedTriggerCount + ) { + drifted = true; + } + publicationTaskKey = taskKey; + publicationTriggerCount = 0; + publicationExpectedTriggerCount = adoptionCount( + row, + 'expectedTriggerCount', + MAX_AUTOMATION_ADOPTION_TRIGGERS, + ); + selected.publication.appendTask({ + rowOrdinal, + sourceDigest: adoptionText(row, 'sourceDigest', DIGEST_PATTERN), + taskContentDigest: adoptionText( + row, + 'taskContentDigest', + DIGEST_PATTERN, + ), + itemDigest: adoptionText(row, 'taskItemDigest', DIGEST_PATTERN), + }); + } + if (row.triggerOrdinal === null) continue; + publicationTriggerCount += 1; + if ( + adoptionCount( + row, + 'triggerOrdinal', + MAX_AUTOMATION_ADOPTION_TRIGGERS, + ) !== publicationTriggerCount + ) { + drifted = true; + } + selected.publication.appendTrigger({ + triggerContentDigest: adoptionText( + row, + 'triggerContentDigest', + DIGEST_PATTERN, + ), + itemDigest: adoptionText(row, 'triggerItemDigest', DIGEST_PATTERN), + }); + } + if ( + publicationTaskKey !== '' && + publicationTriggerCount !== publicationExpectedTriggerCount + ) { + drifted = true; + } } catch (error) { if (error instanceof LocalDeploymentConfigurationError) throw error; return fail('target Automation adoption projection is unavailable', error); } + const missing = [...records.values()].some( + (record) => + record.taskCount !== record.expectedTaskCount || + record.triggerCount !== record.expectedTriggerCount, + ); + if (!missing) { + for (const record of records.values()) { + if (record.publication.digest() !== record.expectedPublicationDigest) { + drifted = true; + } + } + } return Object.freeze({ recordCount, adoptedTaskCount, + adoptedTriggerCount, + provenanceTaskCount, + provenanceTriggerCount, + provenanceState: drifted + ? ('drifted' as const) + : missing + ? ('missing' as const) + : ('complete' as const), setDigest: hash.digest('hex'), }); } @@ -578,6 +967,10 @@ export function writeLocalReconciliationSecretConfigPlan( targetConflictCount, automationAdoptionRecordCount: automationAdoption.recordCount, adoptedLegacyTaskCount: automationAdoption.adoptedTaskCount, + adoptedLegacyTriggerCount: automationAdoption.adoptedTriggerCount, + adoptionProvenanceTaskCount: automationAdoption.provenanceTaskCount, + adoptionProvenanceTriggerCount: automationAdoption.provenanceTriggerCount, + automationAdoptionProvenanceState: automationAdoption.provenanceState, unadaptedLegacyConfigCount: header.unadaptedLegacyConfigCount, outcome: (inventory.tableState === 'absent' || inventory.rowCount === 0) && @@ -586,6 +979,7 @@ export function writeLocalReconciliationSecretConfigPlan( : !inventory.mutationReady || targetConflictCount > 0 || header.unadaptedLegacyConfigCount > 0 || + automationAdoption.provenanceState !== 'complete' || (eligibleBindingCount > 0 && automationAdoption.adoptedTaskCount < 1) ? ('manual_required' as const) : ('ready' as const), @@ -651,6 +1045,11 @@ export function buildLocalReconciliationSecretConfigPlanReceipt( targetConflictCount: footer.targetConflictCount, automationAdoptionRecordCount: footer.automationAdoptionRecordCount, adoptedLegacyTaskCount: footer.adoptedLegacyTaskCount, + adoptedLegacyTriggerCount: footer.adoptedLegacyTriggerCount, + adoptionProvenanceTaskCount: footer.adoptionProvenanceTaskCount, + adoptionProvenanceTriggerCount: footer.adoptionProvenanceTriggerCount, + automationAdoptionProvenanceState: + footer.automationAdoptionProvenanceState, unadaptedLegacyConfigCount: footer.unadaptedLegacyConfigCount, outcome: footer.outcome, preparedAtMs: header.preparedAtMs, @@ -667,9 +1066,13 @@ export function normalizeLocalReconciliationSecretConfigPlanReceipt( 'activeGroupCount', 'activeRowCount', 'adoptedLegacyTaskCount', + 'adoptedLegacyTriggerCount', + 'adoptionProvenanceTaskCount', + 'adoptionProvenanceTriggerCount', 'applicationId', 'applicationPlanDigest', 'automationAdoptionRecordCount', + 'automationAdoptionProvenanceState', 'automationAdoptionSetDigest', 'bindingReadyCount', 'candidateSetDigest', @@ -736,6 +1139,9 @@ export function normalizeLocalReconciliationSecretConfigPlanReceipt( receipt.targetConflictCount, receipt.automationAdoptionRecordCount, receipt.adoptedLegacyTaskCount, + receipt.adoptedLegacyTriggerCount, + receipt.adoptionProvenanceTaskCount, + receipt.adoptionProvenanceTriggerCount, receipt.unadaptedLegacyConfigCount, receipt.planFileBytes, receipt.preparedAtMs, @@ -746,6 +1152,9 @@ export function normalizeLocalReconciliationSecretConfigPlanReceipt( !['ready', 'manual_required', 'no_effect'].includes( receipt.outcome as string, ) || + !['complete', 'missing', 'drifted'].includes( + receipt.automationAdoptionProvenanceState as string, + ) || cutoverDigest(payload) !== receiptDigest ) { fail('receipt drifted'); diff --git a/packages/ql3-local-owner-cli/test/adoptedDeploymentBundle.test.cjs b/packages/ql3-local-owner-cli/test/adoptedDeploymentBundle.test.cjs index be19e7ce..29314ad9 100644 --- a/packages/ql3-local-owner-cli/test/adoptedDeploymentBundle.test.cjs +++ b/packages/ql3-local-owner-cli/test/adoptedDeploymentBundle.test.cjs @@ -366,9 +366,9 @@ function adoptedDockerHarness(state, options = {}) { '/opt/qinglong/node_modules/@qinglong/local-application/dist/cli.js', ], Labels: { - 'io.qinglong.local.sqlite-contract-min': '50', - 'io.qinglong.local.sqlite-contract-max': '50', - 'io.qinglong.local.sqlite-write-contract': '50', + 'io.qinglong.local.sqlite-contract-min': '51', + 'io.qinglong.local.sqlite-contract-max': '51', + 'io.qinglong.local.sqlite-write-contract': '51', 'io.qinglong.local.application-config': '2,3,4', 'io.qinglong.local.compose-selection': '1', 'io.qinglong.ai': 'excluded', @@ -839,7 +839,7 @@ test('preflights adopted Compose identity mounts and rejects mount drift', async ); assert.equal(ready.status, 'ready'); assert.equal(ready.profile, 'edge'); - assert.equal(ready.sqlite.contractVersion, 50); + assert.equal(ready.sqlite.contractVersion, 51); await assert.rejects( preflightLocalDeploymentCompose(composePreflightCommand(state, 1), { runDocker: adoptedDockerHarness(state, { driftMount: true }).runDocker, diff --git a/packages/ql3-local-owner-cli/test/localDeployment.test.cjs b/packages/ql3-local-owner-cli/test/localDeployment.test.cjs index dd1505b3..7ecf530c 100644 --- a/packages/ql3-local-owner-cli/test/localDeployment.test.cjs +++ b/packages/ql3-local-owner-cli/test/localDeployment.test.cjs @@ -490,9 +490,9 @@ function composeDockerHarness( '/opt/qinglong/node_modules/@qinglong/local-application/dist/cli.js', ], Labels: { - 'io.qinglong.local.sqlite-contract-min': '50', - 'io.qinglong.local.sqlite-contract-max': '50', - 'io.qinglong.local.sqlite-write-contract': '50', + 'io.qinglong.local.sqlite-contract-min': '51', + 'io.qinglong.local.sqlite-contract-max': '51', + 'io.qinglong.local.sqlite-write-contract': '51', 'io.qinglong.local.application-config': '2', 'io.qinglong.local.compose-selection': '1', 'io.qinglong.ai': 'excluded', @@ -1199,9 +1199,9 @@ test('preflights exact local image, Compose merge and SQLite capability', async '/opt/qinglong/node_modules/@qinglong/local-application/dist/cli.js', ], Labels: { - 'io.qinglong.local.sqlite-contract-min': '50', - 'io.qinglong.local.sqlite-contract-max': '50', - 'io.qinglong.local.sqlite-write-contract': '50', + 'io.qinglong.local.sqlite-contract-min': '51', + 'io.qinglong.local.sqlite-contract-max': '51', + 'io.qinglong.local.sqlite-write-contract': '51', 'io.qinglong.local.application-config': '2', 'io.qinglong.local.compose-selection': '1', 'io.qinglong.ai': 'excluded', @@ -1259,7 +1259,7 @@ test('preflights exact local image, Compose merge and SQLite capability', async assert.equal(result.status, 'ready'); assert.equal(result.generation, 1); assert.equal(result.profile, 'edge'); - assert.equal(result.sqlite.contractVersion, 50); + assert.equal(result.sqlite.contractVersion, 51); assert.equal(result.image.architecture, 'arm64'); assert.equal(calls.length, 2); assert.deepEqual(calls[0].slice(0, 2), ['image', 'inspect']); @@ -1359,8 +1359,8 @@ test('applies one Compose generation and exactly replays its health receipt', as assert.equal(mode(receiptPath), 0o600); const receipt = JSON.parse(fs.readFileSync(receiptPath, 'utf8')); assert.deepEqual(receipt.sqlite, { - contractVersion: 50, - writeContractVersion: 50, + contractVersion: 51, + writeContractVersion: 51, writeObservation: 'unchanged', backup: null, }); @@ -1661,8 +1661,8 @@ test('rolls a failed Compose candidate forward to a healthy prior digest', async `${command.request.rolloutId}.sqlite`, ); assert.equal(mode(backupPath), 0o600); - assert.equal(receipt.sqlite.contractVersion, 50); - assert.equal(receipt.sqlite.writeContractVersion, 50); + assert.equal(receipt.sqlite.contractVersion, 51); + assert.equal(receipt.sqlite.writeContractVersion, 51); assert.equal(receipt.sqlite.writeObservation, 'changed'); assert.match(receipt.sqlite.backup.sha256, /^[0-9a-f]{64}$/); assert.equal(receipt.sqlite.backup.bytes > 0, true); diff --git a/packages/ql3-local-owner-cli/test/localReadiness.test.cjs b/packages/ql3-local-owner-cli/test/localReadiness.test.cjs index 9be3c3a0..2029a10c 100644 --- a/packages/ql3-local-owner-cli/test/localReadiness.test.cjs +++ b/packages/ql3-local-owner-cli/test/localReadiness.test.cjs @@ -34,8 +34,8 @@ test('inspects the exact fresh Profile schema without exposing its path', async assert.equal(result.status, 'ready'); assert.equal(result.profile, 'edge'); assert.equal(result.storage.contractName, 'local-control-core'); - assert.equal(result.storage.contractVersion, 50); - assert.equal(result.storage.migrationCount, 100); + assert.equal(result.storage.contractVersion, 51); + assert.equal(result.storage.migrationCount, 102); assert.equal(result.storage.journalMode, 'delete'); assert.equal(JSON.stringify(result).includes(state.directory), false); }); diff --git a/packages/ql3-local-owner-cli/test/reconciliationSecretConfigRowPlan.test.cjs b/packages/ql3-local-owner-cli/test/reconciliationSecretConfigRowPlan.test.cjs index e39261f5..a6a89cc2 100644 --- a/packages/ql3-local-owner-cli/test/reconciliationSecretConfigRowPlan.test.cjs +++ b/packages/ql3-local-owner-cli/test/reconciliationSecretConfigRowPlan.test.cjs @@ -4,6 +4,10 @@ const os = require('node:os'); const path = require('node:path'); const { DatabaseSync } = require('node:sqlite'); const { test } = require('node:test'); +const { + LegacyAdoptionPublicationDigest, + legacyAdoptionTaskProvenanceDigest, +} = require('@qinglong/local-sqlite/adoption-provenance'); const { buildLocalReconciliationSecretConfigPlanReceipt, @@ -78,11 +82,85 @@ function databases() { audit_event_id TEXT NOT NULL, created_at_ms INTEGER NOT NULL ); + CREATE TABLE "QingLong3TaskDefinitions" ( + project_id TEXT NOT NULL, + task_id TEXT NOT NULL, + current_revision INTEGER NOT NULL, + created_at_ms INTEGER NOT NULL, + updated_at_ms INTEGER NOT NULL, + PRIMARY KEY (project_id, task_id) + ); + CREATE TABLE "QingLong3TaskDefinitionRevisions" ( + project_id TEXT NOT NULL, + task_id TEXT NOT NULL, + revision INTEGER NOT NULL, + mutation_id TEXT NOT NULL, + content_digest TEXT NOT NULL, + PRIMARY KEY (project_id, task_id, revision) + ); + CREATE TABLE "QingLong3PluginPackageTaskOwnerships" ( + project_id TEXT NOT NULL, + task_id TEXT NOT NULL, + package_name TEXT NOT NULL, + PRIMARY KEY (project_id, task_id) + ); + CREATE TABLE "QingLong3Triggers" ( + project_id TEXT NOT NULL, + trigger_id TEXT NOT NULL, + task_id TEXT NOT NULL, + current_revision INTEGER NOT NULL, + PRIMARY KEY (project_id, trigger_id) + ); + CREATE TABLE "QingLong3TriggerRevisions" ( + project_id TEXT NOT NULL, + trigger_id TEXT NOT NULL, + revision INTEGER NOT NULL, + mutation_id TEXT NOT NULL, + content_digest TEXT NOT NULL, + PRIMARY KEY (project_id, trigger_id, revision) + ); + CREATE TABLE "QingLong3LocalTriggerSchedules" ( + project_id TEXT NOT NULL, + trigger_id TEXT NOT NULL, + trigger_revision INTEGER NOT NULL, + PRIMARY KEY (project_id, trigger_id) + ); + CREATE TABLE "QingLong3LegacyAdoptionTasks" ( + adoption_mutation_id TEXT NOT NULL, + row_ordinal INTEGER NOT NULL, + project_id TEXT NOT NULL, + source_digest TEXT NOT NULL, + task_id TEXT NOT NULL, + task_revision INTEGER NOT NULL, + task_mutation_id TEXT NOT NULL, + task_content_digest TEXT NOT NULL, + trigger_count INTEGER NOT NULL, + item_digest TEXT NOT NULL, + PRIMARY KEY (adoption_mutation_id, row_ordinal) + ); + CREATE TABLE "QingLong3LegacyAdoptionTriggers" ( + adoption_mutation_id TEXT NOT NULL, + row_ordinal INTEGER NOT NULL, + trigger_ordinal INTEGER NOT NULL, + project_id TEXT NOT NULL, + task_id TEXT NOT NULL, + task_revision INTEGER NOT NULL, + trigger_id TEXT NOT NULL, + trigger_revision INTEGER NOT NULL, + trigger_mutation_id TEXT NOT NULL, + trigger_content_digest TEXT NOT NULL, + item_digest TEXT NOT NULL, + PRIMARY KEY (adoption_mutation_id, row_ordinal, trigger_ordinal) + ); `); return { legacy, target }; } -function insertAutomationAdoption(target, adoptedTaskCount = 1) { +function insertAutomationAdoption( + target, + adoptedTaskCount = 1, + withProvenance = true, +) { const mutationId = '30000000-0000-4000-8000-000000000003'; target .prepare( @@ -106,6 +184,70 @@ function insertAutomationAdoption(target, adoptedTaskCount = 1) { mutationId, HEADER.preparedAtMs, ); + if (!withProvenance) return; + const publication = new LegacyAdoptionPublicationDigest(mutationId); + for (let rowOrdinal = 1; rowOrdinal <= adoptedTaskCount; rowOrdinal += 1) { + const taskId = `legacy-cron:${rowOrdinal}`; + const taskMutationId = `31000000-0000-4000-8000-${String(rowOrdinal).padStart(12, '0')}`; + const sourceDigest = String(rowOrdinal % 10).repeat(64); + const taskContentDigest = String((rowOrdinal + 1) % 10).repeat(64); + const payload = { + adoptionMutationId: mutationId, + rowOrdinal, + projectId: HEADER.projectId, + sourceDigest, + taskId, + taskRevision: 1, + taskMutationId, + taskContentDigest, + triggerCount: 0, + }; + const itemDigest = legacyAdoptionTaskProvenanceDigest(payload); + target + .prepare( + `INSERT INTO "QingLong3TaskDefinitions" VALUES (?, ?, 1, ?, ?)` + ) + .run( + HEADER.projectId, + taskId, + HEADER.preparedAtMs, + HEADER.preparedAtMs, + ); + target + .prepare( + `INSERT INTO "QingLong3TaskDefinitionRevisions" VALUES + (?, ?, 1, ?, ?)` + ) + .run(HEADER.projectId, taskId, taskMutationId, taskContentDigest); + target + .prepare( + `INSERT INTO "QingLong3LegacyAdoptionTasks" VALUES + (?, ?, ?, ?, ?, 1, ?, ?, 0, ?)` + ) + .run( + mutationId, + rowOrdinal, + HEADER.projectId, + sourceDigest, + taskId, + taskMutationId, + taskContentDigest, + itemDigest, + ); + publication.appendTask({ + rowOrdinal, + sourceDigest, + taskContentDigest, + itemDigest, + }); + } + target + .prepare( + `UPDATE "QingLong3LegacyAdoptions" + SET publication_digest = ? + WHERE mutation_id = ?`, + ) + .run(publication.digest(), mutationId); } function writePlan( @@ -167,6 +309,10 @@ test('writes a content-free Env plan with separate active and disabled candidate assert.equal(result.footer.targetConflictCount, 0); assert.equal(result.footer.automationAdoptionRecordCount, 1); assert.equal(result.footer.adoptedLegacyTaskCount, 1); + assert.equal(result.footer.adoptedLegacyTriggerCount, 0); + assert.equal(result.footer.adoptionProvenanceTaskCount, 1); + assert.equal(result.footer.adoptionProvenanceTriggerCount, 0); + assert.equal(result.footer.automationAdoptionProvenanceState, 'complete'); assert.match(result.footer.automationAdoptionSetDigest, /^[0-9a-f]{64}$/); const candidates = records.filter((record) => record.kind.endsWith('-candidate'), @@ -218,6 +364,60 @@ test('writes a content-free Env plan with separate active and disabled candidate }), /receipt drifted/, ); + + const tamperedPayload = { + adoptionMutationId: '30000000-0000-4000-8000-000000000003', + rowOrdinal: 1, + projectId: HEADER.projectId, + sourceDigest: '9'.repeat(64), + taskId: 'legacy-cron:1', + taskRevision: 1, + taskMutationId: '31000000-0000-4000-8000-000000000001', + taskContentDigest: '2'.repeat(64), + triggerCount: 0, + }; + target + .prepare( + `UPDATE "QingLong3LegacyAdoptionTasks" + SET source_digest = ?, item_digest = ? + WHERE adoption_mutation_id = ? AND row_ordinal = 1`, + ) + .run( + tamperedPayload.sourceDigest, + legacyAdoptionTaskProvenanceDigest(tamperedPayload), + tamperedPayload.adoptionMutationId, + ); + const resealedItem = writePlan(t, legacy, target); + assert.equal(resealedItem.result.footer.outcome, 'manual_required'); + assert.equal( + resealedItem.result.footer.automationAdoptionProvenanceState, + 'drifted', + ); + + const originalPayload = { ...tamperedPayload, sourceDigest: '1'.repeat(64) }; + target + .prepare( + `UPDATE "QingLong3LegacyAdoptionTasks" + SET source_digest = ?, item_digest = ? + WHERE adoption_mutation_id = ? AND row_ordinal = 1`, + ) + .run( + originalPayload.sourceDigest, + legacyAdoptionTaskProvenanceDigest(originalPayload), + originalPayload.adoptionMutationId, + ); + + target.exec( + `UPDATE "QingLong3TaskDefinitions" + SET current_revision = 2 + WHERE project_id = 'project-1' AND task_id = 'legacy-cron:1'`, + ); + const drifted = writePlan(t, legacy, target); + assert.equal(drifted.result.footer.outcome, 'manual_required'); + assert.equal( + drifted.result.footer.automationAdoptionProvenanceState, + 'drifted', + ); }); test('captures a target Secret collision without reading plaintext', (t) => { @@ -310,6 +510,23 @@ test('keeps active Env and historical Configs manual without adoption authority' assert.equal(withConfigs.result.footer.unadaptedLegacyConfigCount, 1); }); +test('keeps pre-provenance Automation adoption records manual', (t) => { + const { legacy, target } = databases(); + t.after(() => legacy.close()); + t.after(() => target.close()); + legacy.exec( + `INSERT INTO "Envs" VALUES + (1, 'TOKEN', 'private-value', 0, 1, 0, '2026-01-01')`, + ); + insertAutomationAdoption(target, 1, false); + + const planned = writePlan(t, legacy, target); + assert.equal(planned.result.footer.outcome, 'manual_required'); + assert.equal(planned.result.footer.automationAdoptionProvenanceState, 'missing'); + assert.equal(planned.result.footer.adoptionProvenanceTaskCount, 0); + assert.equal(planned.serialized.includes('private-value'), false); +}); + test('fails closed before exceeding the plan byte budget', (t) => { const { legacy, target } = databases(); t.after(() => legacy.close()); diff --git a/packages/ql3-local-sqlite/package.json b/packages/ql3-local-sqlite/package.json index 2691a0c2..55204a51 100644 --- a/packages/ql3-local-sqlite/package.json +++ b/packages/ql3-local-sqlite/package.json @@ -75,6 +75,11 @@ "require": "./dist/adoption/legacyAdoptionDatabase.js", "default": "./dist/adoption/legacyAdoptionDatabase.js" }, + "./adoption-provenance": { + "types": "./dist/adoption/legacyAdoptionProvenance.d.ts", + "require": "./dist/adoption/legacyAdoptionProvenance.js", + "default": "./dist/adoption/legacyAdoptionProvenance.js" + }, "./data-directory-adoption": { "types": "./dist/adoption/data-directory/dataDirectoryAdoptionDatabase.d.ts", "require": "./dist/adoption/data-directory/dataDirectoryAdoptionDatabase.js", diff --git a/packages/ql3-local-sqlite/src/adoption/legacyAdoptionDatabase.ts b/packages/ql3-local-sqlite/src/adoption/legacyAdoptionDatabase.ts index baf88d27..0d31b0f0 100644 --- a/packages/ql3-local-sqlite/src/adoption/legacyAdoptionDatabase.ts +++ b/packages/ql3-local-sqlite/src/adoption/legacyAdoptionDatabase.ts @@ -44,6 +44,11 @@ import { type LocalSqliteReadinessEvidence, } from '../readiness/readiness'; import { LocalSqliteSecurityAuthorityStore } from '../security/securityAuthorityStore'; +import { + LegacyAdoptionPublicationDigest, + legacyAdoptionTaskProvenanceDigest, + legacyAdoptionTriggerProvenanceDigest, +} from './legacyAdoptionProvenance'; export const MAX_LOCAL_LEGACY_ADOPTION_TASKS = 100_000; export const MAX_LOCAL_LEGACY_ADOPTION_TRIGGERS = 500_000; @@ -202,6 +207,86 @@ function deterministicMutationId( )}-${hex.slice(16, 20)}-${hex.slice(20)}`; } +function assertStoredProvenance( + client: DatabaseSync, + adoption: Readonly, +): void { + const rows = client + .prepare( + `SELECT task."row_ordinal" AS "rowOrdinal", + task."source_digest" AS "sourceDigest", + task."task_content_digest" AS "taskContentDigest", + task."trigger_count" AS "triggerCount", + task."item_digest" AS "taskItemDigest", + trigger."trigger_ordinal" AS "triggerOrdinal", + trigger."trigger_content_digest" AS "triggerContentDigest", + trigger."item_digest" AS "triggerItemDigest" + FROM "QingLong3LegacyAdoptionTasks" AS task + LEFT JOIN "QingLong3LegacyAdoptionTriggers" AS trigger + ON trigger."adoption_mutation_id" = task."adoption_mutation_id" + AND trigger."row_ordinal" = task."row_ordinal" + WHERE task."adoption_mutation_id" = ? + ORDER BY task."row_ordinal" ASC, trigger."trigger_ordinal" ASC`, + ) + .iterate(adoption.mutationId) as Iterable; + const publication = new LegacyAdoptionPublicationDigest( + adoption.mutationId, + ); + let taskCount = 0; + let triggerCount = 0; + let currentRowOrdinal = 0; + let expectedTriggerCount = 0; + let currentTriggerCount = 0; + for (const row of rows) { + const rowOrdinal = integer(row, 'rowOrdinal'); + if (rowOrdinal !== currentRowOrdinal) { + if ( + currentRowOrdinal !== 0 && + currentTriggerCount !== expectedTriggerCount + ) { + throw new LocalLegacyAdoptionConflictError(); + } + if (rowOrdinal <= currentRowOrdinal) { + throw new LocalLegacyAdoptionConflictError(); + } + currentRowOrdinal = rowOrdinal; + taskCount += 1; + currentTriggerCount = 0; + expectedTriggerCount = integer(row, 'triggerCount'); + publication.appendTask({ + rowOrdinal, + sourceDigest: text(row, 'sourceDigest'), + taskContentDigest: text(row, 'taskContentDigest'), + itemDigest: text(row, 'taskItemDigest'), + }); + } + if (row.triggerOrdinal === null) { + if (expectedTriggerCount !== 0) { + throw new LocalLegacyAdoptionConflictError(); + } + continue; + } + const triggerOrdinal = integer(row, 'triggerOrdinal'); + currentTriggerCount += 1; + triggerCount += 1; + if (triggerOrdinal !== currentTriggerCount) { + throw new LocalLegacyAdoptionConflictError(); + } + publication.appendTrigger({ + triggerContentDigest: text(row, 'triggerContentDigest'), + itemDigest: text(row, 'triggerItemDigest'), + }); + } + if ( + (currentRowOrdinal !== 0 && currentTriggerCount !== expectedTriggerCount) || + taskCount !== adoption.adoptedTaskCount || + triggerCount !== adoption.adoptedTriggerCount || + publication.digest() !== adoption.publicationDigest + ) { + throw new LocalLegacyAdoptionConflictError(); + } +} + function exactReplay( existing: LocalLegacyAdoptionRecord, command: PublishLocalLegacyAdoptionCommand, @@ -344,6 +429,7 @@ export class LocalSqliteLegacyAdoptionPublisher { if (!exactReplay(existing, input)) { throw new LocalLegacyAdoptionConflictError(); } + assertStoredProvenance(client, existing); await input.confirmExternalAuthority(); client.exec('COMMIT'); began = false; @@ -389,9 +475,9 @@ export class LocalSqliteLegacyAdoptionPublisher { const taskRegistry = createBuiltInTaskSpecSemanticRegistry(); const triggerRegistry = createBuiltInTriggerSpecSemanticRegistry(); const dispatch = new LocalSqliteDispatchDefinitionStore(client); - const publication = createHash('sha256') - .update('qinglong3.legacy-adoption-publication.v1\0') - .update(input.mutationId); + const publication = new LegacyAdoptionPublicationDigest( + input.mutationId, + ); let adoptedTaskCount = 0; let adoptedTriggerCount = 0; let previousRowOrdinal = 0; @@ -488,13 +574,46 @@ export class LocalSqliteLegacyAdoptionPublisher { compileLocalCommandTaskDefinition(task, taskRegistry), ); } - publication - .update('\0task\0') - .update(String(candidate.rowOrdinal)) - .update('\0') - .update(candidate.sourceDigest) - .update('\0') - .update(task.contentDigest); + const taskProvenance = Object.freeze({ + adoptionMutationId: input.mutationId, + rowOrdinal: candidate.rowOrdinal, + projectId: task.projectId, + sourceDigest: candidate.sourceDigest, + taskId: task.taskId, + taskRevision: task.revision, + taskMutationId: task.mutationId, + taskContentDigest: task.contentDigest, + triggerCount: candidate.triggers.length, + }); + const taskItemDigest = + legacyAdoptionTaskProvenanceDigest(taskProvenance); + client + .prepare( + `INSERT INTO "QingLong3LegacyAdoptionTasks" ( + "adoption_mutation_id", "row_ordinal", "project_id", + "source_digest", "task_id", "task_revision", + "task_mutation_id", "task_content_digest", + "trigger_count", "item_digest" + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + ) + .run( + taskProvenance.adoptionMutationId, + taskProvenance.rowOrdinal, + taskProvenance.projectId, + taskProvenance.sourceDigest, + taskProvenance.taskId, + taskProvenance.taskRevision, + taskProvenance.taskMutationId, + taskProvenance.taskContentDigest, + taskProvenance.triggerCount, + taskItemDigest, + ); + publication.appendTask({ + rowOrdinal: candidate.rowOrdinal, + sourceDigest: candidate.sourceDigest, + taskContentDigest: task.contentDigest, + itemDigest: taskItemDigest, + }); for (const [ triggerIndex, @@ -605,13 +724,54 @@ export class LocalSqliteLegacyAdoptionPublisher { trigger.revision, trigger.updatedAtMs, ); - publication.update('\0trigger\0').update(trigger.contentDigest); + const triggerOrdinal = triggerIndex + 1; + const triggerProvenance = Object.freeze({ + adoptionMutationId: input.mutationId, + rowOrdinal: candidate.rowOrdinal, + triggerOrdinal, + projectId: trigger.projectId, + taskId: trigger.taskId, + taskRevision: trigger.taskRevision, + triggerId: trigger.triggerId, + triggerRevision: trigger.revision, + triggerMutationId: trigger.mutationId, + triggerContentDigest: trigger.contentDigest, + }); + const triggerItemDigest = + legacyAdoptionTriggerProvenanceDigest(triggerProvenance); + client + .prepare( + `INSERT INTO "QingLong3LegacyAdoptionTriggers" ( + "adoption_mutation_id", "row_ordinal", + "trigger_ordinal", "project_id", "task_id", + "task_revision", "trigger_id", "trigger_revision", + "trigger_mutation_id", "trigger_content_digest", + "item_digest" + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + ) + .run( + triggerProvenance.adoptionMutationId, + triggerProvenance.rowOrdinal, + triggerProvenance.triggerOrdinal, + triggerProvenance.projectId, + triggerProvenance.taskId, + triggerProvenance.taskRevision, + triggerProvenance.triggerId, + triggerProvenance.triggerRevision, + triggerProvenance.triggerMutationId, + triggerProvenance.triggerContentDigest, + triggerItemDigest, + ); + publication.appendTrigger({ + triggerContentDigest: trigger.contentDigest, + itemDigest: triggerItemDigest, + }); } } if (adoptedTaskCount + input.skippedCount !== input.rowCount) { throw new LocalLegacyAdoptionConflictError(); } - const publicationDigest = publication.digest('hex'); + const publicationDigest = publication.digest(); insertAudit(client, audit); client .prepare( @@ -650,6 +810,7 @@ export class LocalSqliteLegacyAdoptionPublisher { ) .get(input.mutationId) as Row, ); + assertStoredProvenance(client, stored); await input.confirmExternalAuthority(); client.exec('COMMIT'); began = false; diff --git a/packages/ql3-local-sqlite/src/adoption/legacyAdoptionProvenance.ts b/packages/ql3-local-sqlite/src/adoption/legacyAdoptionProvenance.ts new file mode 100644 index 00000000..6bd89374 --- /dev/null +++ b/packages/ql3-local-sqlite/src/adoption/legacyAdoptionProvenance.ts @@ -0,0 +1,95 @@ +import { createHash, type Hash } from 'node:crypto'; + +export interface LegacyAdoptionTaskProvenancePayload { + readonly adoptionMutationId: string; + readonly rowOrdinal: number; + readonly projectId: string; + readonly sourceDigest: string; + readonly taskId: string; + readonly taskRevision: number; + readonly taskMutationId: string; + readonly taskContentDigest: string; + readonly triggerCount: number; +} + +export interface LegacyAdoptionTriggerProvenancePayload { + readonly adoptionMutationId: string; + readonly rowOrdinal: number; + readonly triggerOrdinal: number; + readonly projectId: string; + readonly taskId: string; + readonly taskRevision: number; + readonly triggerId: string; + readonly triggerRevision: number; + readonly triggerMutationId: string; + readonly triggerContentDigest: string; +} + +function itemDigest( + domain: 'task' | 'trigger', + payload: object, +): string { + return createHash('sha256') + .update(`qinglong3.legacy-adoption-${domain}-provenance.v1\0`) + .update(JSON.stringify(payload)) + .digest('hex'); +} + +export function legacyAdoptionTaskProvenanceDigest( + payload: Readonly, +): string { + return itemDigest('task', payload); +} + +export function legacyAdoptionTriggerProvenanceDigest( + payload: Readonly, +): string { + return itemDigest('trigger', payload); +} + +export class LegacyAdoptionPublicationDigest { + readonly #hash: Hash; + #sealed = false; + + constructor(mutationId: string) { + this.#hash = createHash('sha256') + .update('qinglong3.legacy-adoption-publication.v2\0') + .update(mutationId); + } + + appendTask(input: { + readonly rowOrdinal: number; + readonly sourceDigest: string; + readonly taskContentDigest: string; + readonly itemDigest: string; + }): void { + if (this.#sealed) throw new TypeError('Publication digest is sealed'); + this.#hash + .update('\0task\0') + .update(String(input.rowOrdinal)) + .update('\0') + .update(input.sourceDigest) + .update('\0') + .update(input.taskContentDigest) + .update('\0') + .update(input.itemDigest); + } + + appendTrigger(input: { + readonly triggerContentDigest: string; + readonly itemDigest: string; + }): void { + if (this.#sealed) throw new TypeError('Publication digest is sealed'); + this.#hash + .update('\0trigger\0') + .update(input.triggerContentDigest) + .update('\0') + .update(input.itemDigest); + } + + digest(): string { + if (this.#sealed) throw new TypeError('Publication digest is sealed'); + this.#sealed = true; + return this.#hash.digest('hex'); + } +} diff --git a/packages/ql3-local-sqlite/src/migration/migration.ts b/packages/ql3-local-sqlite/src/migration/migration.ts index a4f9d108..6fe652ae 100644 --- a/packages/ql3-local-sqlite/src/migration/migration.ts +++ b/packages/ql3-local-sqlite/src/migration/migration.ts @@ -110,6 +110,8 @@ import { local0097PluginPackageSecretBindingTransitionReceiptsMigration } from ' import { local0098CapabilityV49Migration } from '../migrations/0098-capability-v49'; import { local0099LegacyDataDirectoryAdoptionsMigration } from '../migrations/0099-legacy-data-directory-adoptions'; import { local0100CapabilityV50Migration } from '../migrations/0100-capability-v50'; +import { local0101LegacyAdoptionProvenanceMigration } from '../migrations/0101-legacy-adoption-provenance'; +import { local0102CapabilityV51Migration } from '../migrations/0102-capability-v51'; import type { LocalSqliteMigrationContext } from '../migrations/sqlMigration'; import { LOCAL_SQLITE_MIGRATION_STREAM_ID, @@ -232,6 +234,8 @@ export const localSqliteMigrationDefinition: MigrationStreamDefinition= 0`), uniqueIndex('ql3_legacy_adoptions_decision_uidx').on(table.decisionId), + uniqueIndex('ql3_legacy_adoptions_mutation_project_uidx').on( + table.mutationId, + table.projectId, + ), index('ql3_legacy_adoptions_project_time_idx').on( table.projectId, sql`${table.createdAtMs} desc`, @@ -3328,6 +3332,146 @@ export const legacyAdoptions = sqliteTable( ], ); +export const legacyAdoptionTasks = sqliteTable( + 'QingLong3LegacyAdoptionTasks', + { + adoptionMutationId: text('adoption_mutation_id').notNull(), + rowOrdinal: integer('row_ordinal').notNull(), + projectId: text('project_id').notNull(), + sourceDigest: text('source_digest').notNull(), + taskId: text('task_id').notNull(), + taskRevision: integer('task_revision').notNull(), + taskMutationId: text('task_mutation_id').notNull(), + taskContentDigest: text('task_content_digest').notNull(), + triggerCount: integer('trigger_count').notNull(), + itemDigest: text('item_digest').notNull(), + }, + (table) => [ + primaryKey({ columns: [table.adoptionMutationId, table.rowOrdinal] }), + foreignKey({ + columns: [table.adoptionMutationId, table.projectId], + foreignColumns: [legacyAdoptions.mutationId, legacyAdoptions.projectId], + }) + .onDelete('restrict') + .onUpdate('restrict'), + foreignKey({ + columns: [table.projectId, table.taskId, table.taskRevision], + foreignColumns: [ + taskDefinitionRevisions.projectId, + taskDefinitionRevisions.taskId, + taskDefinitionRevisions.revision, + ], + }) + .onDelete('restrict') + .onUpdate('restrict'), + foreignKey({ + columns: [table.taskMutationId], + foreignColumns: [taskDefinitionRevisions.mutationId], + }) + .onDelete('restrict') + .onUpdate('restrict'), + check( + 'ql3_legacy_adoption_task_identity_check', + sql`${table.rowOrdinal} between 1 and 100000 and length(${table.projectId}) between 1 and 128 and length(${table.taskId}) between 1 and 128 and ${table.taskRevision} = 1 and length(${table.taskMutationId}) = 36 and replace(${table.taskMutationId}, '-', '') not glob '*[^0-9a-f]*' and ${table.triggerCount} between 0 and 500000`, + ), + check( + 'ql3_legacy_adoption_task_digest_check', + sql`length(${table.sourceDigest}) = 64 and ${table.sourceDigest} not glob '*[^0-9a-f]*' and length(${table.taskContentDigest}) = 64 and ${table.taskContentDigest} not glob '*[^0-9a-f]*' and length(${table.itemDigest}) = 64 and ${table.itemDigest} not glob '*[^0-9a-f]*'`, + ), + uniqueIndex('ql3_legacy_adoption_tasks_project_task_uidx').on( + table.projectId, + table.taskId, + ), + uniqueIndex('ql3_legacy_adoption_tasks_mutation_uidx').on( + table.taskMutationId, + ), + uniqueIndex('ql3_legacy_adoption_tasks_item_uidx').on(table.itemDigest), + uniqueIndex('ql3_legacy_adoption_tasks_identity_uidx').on( + table.adoptionMutationId, + table.rowOrdinal, + table.projectId, + table.taskId, + table.taskRevision, + ), + ], +); + +export const legacyAdoptionTriggers = sqliteTable( + 'QingLong3LegacyAdoptionTriggers', + { + adoptionMutationId: text('adoption_mutation_id').notNull(), + rowOrdinal: integer('row_ordinal').notNull(), + triggerOrdinal: integer('trigger_ordinal').notNull(), + projectId: text('project_id').notNull(), + taskId: text('task_id').notNull(), + taskRevision: integer('task_revision').notNull(), + triggerId: text('trigger_id').notNull(), + triggerRevision: integer('trigger_revision').notNull(), + triggerMutationId: text('trigger_mutation_id').notNull(), + triggerContentDigest: text('trigger_content_digest').notNull(), + itemDigest: text('item_digest').notNull(), + }, + (table) => [ + primaryKey({ + columns: [ + table.adoptionMutationId, + table.rowOrdinal, + table.triggerOrdinal, + ], + }), + foreignKey({ + columns: [ + table.adoptionMutationId, + table.rowOrdinal, + table.projectId, + table.taskId, + table.taskRevision, + ], + foreignColumns: [ + legacyAdoptionTasks.adoptionMutationId, + legacyAdoptionTasks.rowOrdinal, + legacyAdoptionTasks.projectId, + legacyAdoptionTasks.taskId, + legacyAdoptionTasks.taskRevision, + ], + }) + .onDelete('restrict') + .onUpdate('restrict'), + foreignKey({ + columns: [table.projectId, table.triggerId, table.triggerRevision], + foreignColumns: [ + triggerRevisions.projectId, + triggerRevisions.triggerId, + triggerRevisions.revision, + ], + }) + .onDelete('restrict') + .onUpdate('restrict'), + foreignKey({ + columns: [table.triggerMutationId], + foreignColumns: [triggerRevisions.mutationId], + }) + .onDelete('restrict') + .onUpdate('restrict'), + check( + 'ql3_legacy_adoption_trigger_identity_check', + sql`${table.rowOrdinal} between 1 and 100000 and ${table.triggerOrdinal} between 1 and 500000 and length(${table.projectId}) between 1 and 128 and length(${table.taskId}) between 1 and 128 and ${table.taskRevision} = 1 and length(${table.triggerId}) between 1 and 128 and ${table.triggerRevision} = 1 and length(${table.triggerMutationId}) = 36 and replace(${table.triggerMutationId}, '-', '') not glob '*[^0-9a-f]*'`, + ), + check( + 'ql3_legacy_adoption_trigger_digest_check', + sql`length(${table.triggerContentDigest}) = 64 and ${table.triggerContentDigest} not glob '*[^0-9a-f]*' and length(${table.itemDigest}) = 64 and ${table.itemDigest} not glob '*[^0-9a-f]*'`, + ), + uniqueIndex('ql3_legacy_adoption_triggers_project_trigger_uidx').on( + table.projectId, + table.triggerId, + ), + uniqueIndex('ql3_legacy_adoption_triggers_mutation_uidx').on( + table.triggerMutationId, + ), + uniqueIndex('ql3_legacy_adoption_triggers_item_uidx').on(table.itemDigest), + ], +); + export const legacyDataDirectoryAdoptions = sqliteTable( 'QingLong3LegacyDataDirectoryAdoptions', { @@ -5180,6 +5324,8 @@ export const localSqliteSchema = Object.freeze({ toolExecutionResultRekeyHeads, toolResultKeyRetirementReceipts, legacyAdoptions, + legacyAdoptionTasks, + legacyAdoptionTriggers, legacyDataDirectoryAdoptions, legacyDataDirectoryAdoptionSecrets, localIdentitySubjects, diff --git a/packages/ql3-local-sqlite/test/adoptionPublisher.test.cjs b/packages/ql3-local-sqlite/test/adoptionPublisher.test.cjs index 04c777bb..417c6f7c 100644 --- a/packages/ql3-local-sqlite/test/adoptionPublisher.test.cjs +++ b/packages/ql3-local-sqlite/test/adoptionPublisher.test.cjs @@ -157,6 +157,45 @@ test('publishes tasks, execution facts, triggers, audit and ledger atomically', .get().count, 2, ); + assert.deepEqual( + client + .prepare( + `SELECT task."row_ordinal" AS rowOrdinal, + task."task_id" AS taskId, + task."task_revision" AS taskRevision, + task."trigger_count" AS triggerCount, + trigger."trigger_ordinal" AS triggerOrdinal, + trigger."trigger_id" AS triggerId, + trigger."trigger_revision" AS triggerRevision + FROM "QingLong3LegacyAdoptionTasks" AS task + JOIN "QingLong3LegacyAdoptionTriggers" AS trigger + ON trigger."adoption_mutation_id" = task."adoption_mutation_id" + AND trigger."row_ordinal" = task."row_ordinal" + ORDER BY task."row_ordinal", trigger."trigger_ordinal"`, + ) + .all() + .map((row) => ({ ...row })), + [ + { + rowOrdinal: 1, + taskId: 'legacy-cron:1', + taskRevision: 1, + triggerCount: 1, + triggerOrdinal: 1, + triggerId: 'legacy-cron:1:cron:1', + triggerRevision: 1, + }, + { + rowOrdinal: 2, + taskId: 'legacy-cron:2', + taskRevision: 1, + triggerCount: 1, + triggerOrdinal: 1, + triggerId: 'legacy-cron:2:cron:1', + triggerRevision: 1, + }, + ], + ); assert.deepEqual( { ...client @@ -191,6 +230,8 @@ test('rolls the complete publication back on a later candidate conflict', async 'QingLong3Triggers', 'QingLong3LocalTriggerSchedules', 'QingLong3LegacyAdoptions', + 'QingLong3LegacyAdoptionTasks', + 'QingLong3LegacyAdoptionTriggers', 'QingLong3SecurityAuditEvents', ]) { assert.equal( @@ -235,6 +276,8 @@ test('awaits the final external authority check and rolls back on rejection', as 'QingLong3Triggers', 'QingLong3LocalTriggerSchedules', 'QingLong3LegacyAdoptions', + 'QingLong3LegacyAdoptionTasks', + 'QingLong3LegacyAdoptionTriggers', 'QingLong3SecurityAuditEvents', ]) { assert.equal( @@ -264,3 +307,33 @@ test('rejects stale authorization fences before any adoption mutation', async (t ); await adoption.close(); }); + +test('rejects exact replay when durable provenance has drifted', async (t) => { + const databasePath = await preparedDatabase(t); + const input = command([candidate(1)]); + const adoption = await openLocalSqliteAdoptionDatabase({ + databasePath, + profile: 'edge', + }); + await adoption.publisher.publish(input); + await adoption.close(); + + const client = new DatabaseSync(databasePath); + client + .prepare( + `UPDATE "QingLong3LegacyAdoptionTasks" + SET "item_digest" = ? WHERE "adoption_mutation_id" = ?`, + ) + .run('f'.repeat(64), MUTATION_ID); + client.close(); + + const reopened = await openLocalSqliteAdoptionDatabase({ + databasePath, + profile: 'edge', + }); + await assert.rejects( + reopened.publisher.publish(input), + LocalLegacyAdoptionConflictError, + ); + await reopened.close(); +}); diff --git a/packages/ql3-local-sqlite/test/authenticationRead.test.cjs b/packages/ql3-local-sqlite/test/authenticationRead.test.cjs index 457969db..800e18a7 100644 --- a/packages/ql3-local-sqlite/test/authenticationRead.test.cjs +++ b/packages/ql3-local-sqlite/test/authenticationRead.test.cjs @@ -25,7 +25,7 @@ test('authentication projection opens the target read-only without journal or fi try { assert.equal(database.profile, 'edge'); assert.equal(database.readiness.contractName, 'local-control-core'); - assert.equal(database.readiness.contractVersion, 50); + assert.equal(database.readiness.contractVersion, 51); assert.equal(await database.apiCredentials.resolve('absent'), null); assert.equal(await database.ownerPepper.resolveKey('absent'), null); } finally { diff --git a/packages/ql3-local-sqlite/test/database.test.cjs b/packages/ql3-local-sqlite/test/database.test.cjs index 393b1724..1daf2f47 100644 --- a/packages/ql3-local-sqlite/test/database.test.cjs +++ b/packages/ql3-local-sqlite/test/database.test.cjs @@ -150,9 +150,11 @@ test('creates a reviewed edge database and opens runtime only after readiness', '0098-capability-v49', '0099-legacy-data-directory-adoptions', '0100-capability-v50', + '0101-legacy-adoption-provenance', + '0102-capability-v51', ]); assert.equal(migrated.readiness.contractName, 'local-control-core'); - assert.equal(migrated.readiness.contractVersion, 50); + assert.equal(migrated.readiness.contractVersion, 51); assert.equal(migrated.readiness.journalMode, 'delete'); assert.equal(fs.statSync(databasePath).mode & 0o777, 0o600); @@ -598,8 +600,8 @@ test('backfills v14 execution revisions with a verified independent digest', asy .get(), }, { - contract_version: 50, - migration_id: '0099-legacy-data-directory-adoptions', + contract_version: 51, + migration_id: '0101-legacy-adoption-provenance', }, ); } finally { @@ -786,19 +788,19 @@ test('excludes reviewed optional feature tables while preserving unknown table d const options = { databasePath, profile: 'edge' }; await migrateLocalSqlitePath(options); const client = new DatabaseSync(databasePath); - assert.equal((await auditLocalSqlitePath(options)).tableCount, 83); + assert.equal((await auditLocalSqlitePath(options)).tableCount, 85); client.exec( 'CREATE TABLE "ModelInvocationFeatureHead" (feature_id TEXT PRIMARY KEY)', ); client.close(); - assert.equal((await auditLocalSqlitePath(options)).tableCount, 83); + assert.equal((await auditLocalSqlitePath(options)).tableCount, 85); const unknownClient = new DatabaseSync(databasePath); unknownClient.exec('CREATE TABLE "UserExtensionData" (id TEXT PRIMARY KEY)'); unknownClient.close(); - assert.equal((await auditLocalSqlitePath(options)).tableCount, 84); + assert.equal((await auditLocalSqlitePath(options)).tableCount, 86); const triggerClient = new DatabaseSync(databasePath); triggerClient.exec(` diff --git a/packages/ql3-local-sqlite/test/pluginPackageWorkflowAdmissionRepository.test.cjs b/packages/ql3-local-sqlite/test/pluginPackageWorkflowAdmissionRepository.test.cjs index a5ed2b0e..e78ec44f 100644 --- a/packages/ql3-local-sqlite/test/pluginPackageWorkflowAdmissionRepository.test.cjs +++ b/packages/ql3-local-sqlite/test/pluginPackageWorkflowAdmissionRepository.test.cjs @@ -156,7 +156,7 @@ test('atomically admits one generation-bound Workflow Run and exactly replays it }, { runs: 1, steps: 2, events: 3, mutations: 2, admissions: 1 }, ); - assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 50); + assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 51); }); test('runs an optional authorization guard inside new and replay transactions', async (t) => { @@ -288,7 +288,7 @@ test('exactly replays immutable admission after the Workflow StepRun advances', }, { status: 'running', version: 5, eventSequence: 5 }, ); - assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 50); + assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 51); }); test('fails closed before writing when the exact installation is not active', async (t) => { diff --git a/packages/ql3-local-sqlite/test/pluginPackageWorkflowTaskAttemptAdmissionRepository.test.cjs b/packages/ql3-local-sqlite/test/pluginPackageWorkflowTaskAttemptAdmissionRepository.test.cjs index a3cd19e6..8420784c 100644 --- a/packages/ql3-local-sqlite/test/pluginPackageWorkflowTaskAttemptAdmissionRepository.test.cjs +++ b/packages/ql3-local-sqlite/test/pluginPackageWorkflowTaskAttemptAdmissionRepository.test.cjs @@ -231,7 +231,7 @@ test('atomically admits the exact reconciled local Task revision and replays it' stepAttemptCount: 0, }, ); - assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 50); + assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 51); }); test('bounds candidate paging before SQL and fences cancellation', async (t) => { diff --git a/packages/ql3-local-sqlite/test/rolloutSafety.test.cjs b/packages/ql3-local-sqlite/test/rolloutSafety.test.cjs index 22791bad..bf10d809 100644 --- a/packages/ql3-local-sqlite/test/rolloutSafety.test.cjs +++ b/packages/ql3-local-sqlite/test/rolloutSafety.test.cjs @@ -40,9 +40,9 @@ test('creates and exactly replays a reviewed rollout backup', async (t) => { await migrateLocalSqlitePath(state); const prepared = await createLocalSqliteRolloutBackup(state); assert.equal(prepared.status, 'prepared'); - assert.equal(prepared.contractVersion, 50); - assert.equal(prepared.writeContractVersion, 50); - assert.equal(LOCAL_SQLITE_WRITE_CONTRACT_VERSION, 50); + assert.equal(prepared.contractVersion, 51); + assert.equal(prepared.writeContractVersion, 51); + assert.equal(LOCAL_SQLITE_WRITE_CONTRACT_VERSION, 51); assert.match(prepared.sha256, /^[0-9a-f]{64}$/); assert.equal(prepared.bytes > 0, true); assert.equal(prepared.pageCount > 0, true); diff --git a/scripts/ql3-cluster-dependency-audit.cjs b/scripts/ql3-cluster-dependency-audit.cjs index dcd17cb6..7ed7c87d 100644 --- a/scripts/ql3-cluster-dependency-audit.cjs +++ b/scripts/ql3-cluster-dependency-audit.cjs @@ -2069,8 +2069,10 @@ function auditSourceImports(root, packagePath, findings) { !( path.relative(packageDirectory, filePath) === 'src/deployment/reconciliation/application/secret-and-config/rowPlan.ts' && - specifier === - '@qinglong/local-admin/reconciliation-secret-and-config-inspection' + [ + '@qinglong/local-admin/reconciliation-secret-and-config-inspection', + '@qinglong/local-sqlite/adoption-provenance', + ].includes(specifier) ) && !( [ diff --git a/scripts/ql3-cluster-oci-layout-audit.cjs b/scripts/ql3-cluster-oci-layout-audit.cjs index e9ce5a0c..fa699c72 100644 --- a/scripts/ql3-cluster-oci-layout-audit.cjs +++ b/scripts/ql3-cluster-oci-layout-audit.cjs @@ -225,9 +225,9 @@ function expectedImageConfig(architecture, revision, image) { 'io.qinglong.ai': 'excluded', 'io.qinglong.local.application-config': '2,3,4', 'io.qinglong.local.compose-selection': '1', - 'io.qinglong.local.sqlite-contract-max': '50', - 'io.qinglong.local.sqlite-contract-min': '50', - 'io.qinglong.local.sqlite-write-contract': '50', + 'io.qinglong.local.sqlite-contract-max': '51', + 'io.qinglong.local.sqlite-contract-min': '51', + 'io.qinglong.local.sqlite-write-contract': '51', 'io.qinglong.profile': 'edge,standalone', 'org.opencontainers.image.description': 'QingLong 3.0 AI-excluded Edge and Standalone runtime', diff --git a/scripts/ql3-local-image-audit.cjs b/scripts/ql3-local-image-audit.cjs index eaffdb48..320247b6 100644 --- a/scripts/ql3-local-image-audit.cjs +++ b/scripts/ql3-local-image-audit.cjs @@ -261,9 +261,9 @@ function auditDockerfile(contents, findings) { !contents.includes('io.qinglong.ai="excluded"') || !contents.includes('io.qinglong.profile="edge,standalone"') || !contents.includes('io.qinglong.local.application-config="2,3,4"') || - !contents.includes('io.qinglong.local.sqlite-contract-min="50"') || - !contents.includes('io.qinglong.local.sqlite-contract-max="50"') || - !contents.includes('io.qinglong.local.sqlite-write-contract="50"') || + !contents.includes('io.qinglong.local.sqlite-contract-min="51"') || + !contents.includes('io.qinglong.local.sqlite-contract-max="51"') || + !contents.includes('io.qinglong.local.sqlite-write-contract="51"') || !contents.includes('io.qinglong.local.compose-selection="1"') ) { addFinding(findings, 'RUNTIME_IDENTITY_OR_LABEL_DRIFT'); diff --git a/test/back/ql3ClusterDependencyAudit.test.cjs b/test/back/ql3ClusterDependencyAudit.test.cjs index 0d7aa5ff..40627ee1 100644 --- a/test/back/ql3ClusterDependencyAudit.test.cjs +++ b/test/back/ql3ClusterDependencyAudit.test.cjs @@ -2257,11 +2257,11 @@ test('confines reconciliation Secret and Config inspection to its exact row plan fs.mkdirSync(secretConfigDirectory, { recursive: true }); fs.writeFileSync( path.join(secretConfigDirectory, 'rowPlan.ts'), - "import { inspect } from '@qinglong/local-admin/reconciliation-secret-and-config-inspection';", + "import { inspect } from '@qinglong/local-admin/reconciliation-secret-and-config-inspection';\nimport { digest } from '@qinglong/local-sqlite/adoption-provenance';", ); fs.writeFileSync( path.join(secretConfigDirectory, 'neighbor.ts'), - "import { inspect } from '@qinglong/local-admin/reconciliation-secret-and-config-inspection';", + "import { inspect } from '@qinglong/local-admin/reconciliation-secret-and-config-inspection';\nimport { digest } from '@qinglong/local-sqlite/adoption-provenance';", ); t.after(() => fs.rmSync(root, { recursive: true, force: true })); @@ -2276,6 +2276,11 @@ test('confines reconciliation Secret and Config inspection to its exact row plan specifier: '@qinglong/local-admin/reconciliation-secret-and-config-inspection', }, + { + code: 'FORBIDDEN_LOCAL_ADOPTION_CLI_AUTHORITY_IMPORT', + file: 'packages/ql3-local-owner-cli/src/deployment/reconciliation/application/secret-and-config/neighbor.ts', + specifier: '@qinglong/local-sqlite/adoption-provenance', + }, ], ); }); diff --git a/test/back/ql3ClusterOciLayoutAudit.test.cjs b/test/back/ql3ClusterOciLayoutAudit.test.cjs index 207ac9a7..3fff8e2f 100644 --- a/test/back/ql3ClusterOciLayoutAudit.test.cjs +++ b/test/back/ql3ClusterOciLayoutAudit.test.cjs @@ -138,9 +138,9 @@ function createFixture(t, options = {}) { 'io.qinglong.ai': 'excluded', 'io.qinglong.local.application-config': '2,3,4', 'io.qinglong.local.compose-selection': '1', - 'io.qinglong.local.sqlite-contract-max': '50', - 'io.qinglong.local.sqlite-contract-min': '50', - 'io.qinglong.local.sqlite-write-contract': '50', + 'io.qinglong.local.sqlite-contract-max': '51', + 'io.qinglong.local.sqlite-contract-min': '51', + 'io.qinglong.local.sqlite-write-contract': '51', 'io.qinglong.profile': 'edge,standalone', } : {}), diff --git a/test/back/ql3LocalImageAudit.test.cjs b/test/back/ql3LocalImageAudit.test.cjs index aa09bc55..8d9d42dc 100644 --- a/test/back/ql3LocalImageAudit.test.cjs +++ b/test/back/ql3LocalImageAudit.test.cjs @@ -188,7 +188,7 @@ test('rejects removal of the SQLite rollout compatibility labels', () => { const dockerfilePath = path.join(current.target, 'Dockerfile'); const dockerfile = fs .readFileSync(dockerfilePath, 'utf8') - .replace(' io.qinglong.local.sqlite-write-contract="50" \\\n', ''); + .replace(' io.qinglong.local.sqlite-write-contract="51" \\\n', ''); fs.writeFileSync(dockerfilePath, dockerfile); const report = auditLocalImageContract(current.root); assert.equal(report.compatible, false); diff --git a/test/back/ql3PackageBoundaryAudit.test.cjs b/test/back/ql3PackageBoundaryAudit.test.cjs index a44f6f5a..6320f4b4 100644 --- a/test/back/ql3PackageBoundaryAudit.test.cjs +++ b/test/back/ql3PackageBoundaryAudit.test.cjs @@ -79,7 +79,7 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( }, { directory: 'packages/ql3-local-sqlite/src/migrations', - directSourceFiles: 101, + directSourceFiles: 103, reviewKind: 'ordered_ledger', }, ], @@ -540,10 +540,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( rootSourceFileRoles: localSqlite.rootSourceFileRoles, }, { - sourceFiles: 203, + sourceFiles: 206, rootSourceFiles: 1, rootSourceLines: 31, - nestedSourceFiles: 202, + nestedSourceFiles: 205, rootSourceFileRoles: { 'index.ts': 'public_export' }, }, );