diff --git a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md index 54d7f50e..a8edab22 100644 --- a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md +++ b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md @@ -6,10 +6,28 @@ - 目标版本：QingLong 3.x - 作者：QingLong Maintainers - 创建日期：2026-07-17 -- 最后更新：2026-08-22 +- 最后更新：2026-08-23 - 讨论范围：架构与演进路线，不包含最终 UI 视觉方案 -最新增量证据（2026-08-22）： +最新增量证据（2026-08-23）： + +- D-396/ADR-0490（进行中）：Run History 不再只有永久 `manual_external`，但也没有被错误实现为 Legacy 日志到 3.0 Run ledger 的回灌。 + 新的 Local adapter 以 ADR-0482 sealed capture bundle 作为 append-only 保全资产：Legacy history 必须逐事实选择 `retain_both`，Target history + 必须选择 `retain_target`；receipt 只绑定 signed review、application、bundle fingerprint、领域 inventory 与有界 fact counts，不保存表名、Run ID、 + 状态值、日志、路径或 row body。Target 只有在所有 Run 已 `succeeded|failed|cancelled|timed_out` 且有完成时间、没有 active Attempt、所有 + StepRun 已终态时才从 `blocked/historical_integrity_required` 提升为 `required/historical_preservation_required`；active、waiting、retry、lost、 + 缺列或 schema drift 继续失败关闭，人工 decision 不能升级。 + + 实现位于既有 `local-owner-cli/deployment/reconciliation/application/run-history/{contract,evidence,coordinator}`，新增短生命周期 + `reconciliation-run-history-preserve|verify`，不新增 instance state、package、dependency、binary、SQL、daemon 或 `src/` 根平铺。preservation + receipt no-replace 发布并封为 `0400/0500`，绑定 exact application/plan/head；跨领域 completion 以兼容 v1 的 schema v2 消费 + `run_history_preservation`，`adapterCount` 可为 `0|1|2`，因此 Automation 与 Run History 可以同时被证明而不产生 adapter 状态组合爆炸。 + 当前聚焦 Run History `2/2`、旧 completion v1 `2/2`，已覆盖 publication/seal response loss、active Run、review escalation、receipt tamper、v2 + completion 与 content-free CLI。完整 reconciliation 为 `48/46/2/0`，Local Owner 为 `273/266/7/0`，tracked backend 为 + `1561/1559/2/0`，18-package clean build/test 为 `2919/2897/22/0`；package/dependency/source boundary、service-manager bridge、Edge import + 与十四档 Local artifact audit 全部 compatible。workspace 保持 18 packages、`singleSourcePackages=[]`、`shallowSourcePackages=[]`，Local Owner + 为 `175/174/1`（source/nested/root binary entry）；基础 Edge/Standalone 制品仍为 `2,611,978 / 2,612,056 bytes`、319 files、58 modules， + 一次性 Owner authority 未进入低资源常驻 runtime。远程 CI 尚未验证，ADR-0490 保持 Proposed。 - D-393/ADR-0486/ADR-0487（进行中）：首个 Automation adapter 已先建立独立的有界逐行 plan fence，而没有提前取得 DML authority。既有 Legacy Crontab classifier 被复用于 exact sealed Legacy source；每行只记录 source/candidate digest、classification/reasons、proposed diff --git a/docs/adr/ADR-0490-run-history-terminal-preservation.md b/docs/adr/ADR-0490-run-history-terminal-preservation.md new file mode 100644 index 00000000..ad1822f8 --- /dev/null +++ b/docs/adr/ADR-0490-run-history-terminal-preservation.md @@ -0,0 +1,110 @@ +# ADR-0490：Run History 终态保全与跨领域完成证明 + +- 状态：Proposed（D-396 实现与本地完整门禁已完成，等待远程 CI） +- 日期：2026-08-23 +- 决策：D-396 +- 关联：ADR-0482、ADR-0483、ADR-0484、ADR-0485、ADR-0488、ADR-0489 + +## 背景 + +QingLong 2.x 的 `CrontabStats`、`Logs` 与 QingLong 3.0 的 `Runs`、`RunAttempts`、`RunEvents`、`StepRuns` 等对象都属于运行历史，但两代模型并不具备可证明的一一映射。把旧统计或日志行直接插入 3.0 Run ledger 会伪造 Run identity、event sequence、Attempt/Step 关系与执行来源；用 target 覆盖 Legacy 又会违反 append-only 历史边界。 + +D-391 因而把 Run History 全部标为 blocked，D-394 completion 也只能拒绝该领域。这个保守默认能避免数据损坏，却使一个已经停稳、只要求保留两侧历史的迁移永远不能形成完成证明。D-396 必须提供真正可完成、但不冒充历史转换的领域 adapter。 + +## 决策 + +### 1. Run History adapter 是保全证明，不是数据导入器 + +Legacy 与 target 的历史继续保存在 ADR-0482 已封存的 capture bundle 中。adapter 不执行 INSERT、UPDATE、DELETE、checkpoint、日志复制、格式转换或外部上传，也不把旧行回灌到 3.0 Run ledger。 + +合法裁决固定为： + +- Legacy Run History 的每个 required fact 必须选择 `retain_both / preserve_both`； +- Target Run History 的每个 required fact 必须选择 `retain_target / preserve_target`； +- `adopt_legacy`、`exclude_legacy`、`defer`、`manual_external` 或任意 blocked fact 都不能形成保全证明。 + +receipt 只绑定 bundle、review、application、领域 inventory 与 fact counts，不保存表名、Run ID、状态值、日志、路径或 row body。 + +### 2. 只有确定终态的 target history 可被提升为 required + +Legacy capture 已由 stopped source fence 固定，因此其 Run History 可进入 `historical_preservation_required`。Target 必须从 exact sealed SQLite 重新证明： + +- `Runs` 存在 `status` 与 `finished_at_ms`，所有 Run 只能是 `succeeded|failed|cancelled|timed_out`，且具备完成时间； +- 若存在 `RunAttempts`，不能有 `claimed|starting|running` Attempt； +- 若存在 `StepRuns`，所有 Step 只能是 `succeeded|failed|skipped|cancelled|timed_out`，且具备完成时间； +- 缺少 canonical 列、active/waiting/retry/lost Run、active Attempt、pending/waiting/running/lost Step 或查询漂移继续保持 `blocked / historical_integrity_required`。 + +该判断同时用于 diagnostics、review commit 和 adapter replay；operator 不能通过修改私有 decision file 把 active Run 提升为可保全。 + +### 3. 新增独立短生命周期 preserve/verify 命令 + +既有 `ql3-local-deploy` 增加： + +```text +local.deployment.reconciliation.run-history.preserve +local.deployment.reconciliation.run-history.verify +``` + +preserve 重新打开 sealed application、plan、bundle 与 owner-private decision file，重放全部 canonical facts，并要求 review authorization 的 decision file digest/count 完全一致。它只允许 Run History domain 为 `adapter_required`，并绑定 exact `reconciliation_application_planned` 或 `reconciliation_automation_applied` source head。 + +每个 preservation directory 最多只含 `receipt.json`，以 no-replace publication、file/directory fsync 和 exact replay 收敛，最终文件/目录权限为 `0400/0500`；catalog 上限 64。receipt 发布或封存后的 response loss 可由相同命令继续完成，不覆盖冲突证据。 + +preserve 不增加 instance head 状态。它本身不授予 restart；若在发布期间另一个 adapter 推进 head，该证明会因 source-head digest 脱离而不能被 completion 消费，调用方必须在新的终态 head 上使用新 preservation ID 重新证明。这避免了多个独立 adapter 状态的组合爆炸。 + +### 4. Completion schema v2 消费第二种 adapter evidence + +completion command/receipt 保留 v1 兼容语义；只有携带 Run History preservation 时使用 schema v2。固定八领域证据新增： + +```text +run_history + adapter_required + run_history_preservation +``` + +completion 重新验证 sealed preservation、原 decision stream、bundle fingerprint、application plan 与 source head；调用方只能提交 preservation ID 和 expected digest，不能自报 domain evidence。`adapterCount` 从 `0|1` 扩展为 `0|1|2`，允许 Automation apply 与 Run History preservation 同时作为已证明 adapter。 + +完成时间不得早于 application commit、Automation apply 或 history preservation 中任一最新证据。v1 receipt 仍按原规则读取和验证；v2 receipt 必须实际包含 Run History preservation，不能用版本号绕过领域证明。 + +### 5. 低资源与集群边界 + +实现内聚于既有 `@qinglong/local-owner-cli/src/deployment/reconciliation/application/run-history/`，不新增 workspace package、production dependency、binary、daemon、timer、watcher、listener、socket、SQL migration 或 `src/` 根平铺。 + +Edge/Standalone 每次最多打开一个 sealed SQLite handle，状态判定均为 `LIMIT 1`，decision file 继续使用 64 KiB 流式缓冲；长期新增资产只有一个小型 content-free receipt，不复制数据库或日志。 + +本 ADR 只定义 Local SQLite 保全。Cluster/PostgreSQL 的 Run History 必须使用数据库 snapshot、事务一致性、租约/运行收敛与 HA retention 的独立证明，不能复用本机路径或把 Local capture bundle 当成集群历史 authority。 + +## 被拒绝的替代方案 + +### 把 Legacy 统计和日志导入 3.0 Run ledger + +拒绝。没有 Run/Attempt/Event/Step identity 与顺序证明，导入会制造看似完整但语义虚假的运行记录。 + +### 只保存 source database 的 SHA-256 + +拒绝。整库 hash 会被其他 adapter 对无关表的合法写入破坏，也不能表达签名的 append-only 保留选择。D-396 绑定 sealed bundle 与 Run History domain inventory。 + +### 每增加一个 adapter 就新增 instance head 状态 + +拒绝。八领域会产生顺序与组合爆炸。无 DML 的 history proof 绑定当前 source head，最终由跨领域 completion 一次 CAS。 + +### 把 active/lost Run 当成历史 + +拒绝。`running|waiting_approval|retry_wait|lost` 都可能仍需执行、恢复或人工裁决，封存为完成历史会丢失运行 authority。 + +## 验收条件 + +1. terminal Legacy/Target history 从 blocked 精确提升为 required；active/inconclusive target history 始终 blocked。 +2. 只有 `legacy retain_both + target retain_target` 的完整 signed canonical decision stream 可发布 receipt。 +3. receipt 不包含名称、路径、Run ID、状态值、日志或 row body，且收敛为 `0400/0500`。 +4. receipt publication/seal response loss exact replay；替换、篡改、额外 catalog、stale head 与 decision drift 全部失败关闭。 +5. completion v2 重新验证 preservation 并生成固定八领域 evidence；completion v1 保持兼容。 +6. preserve/verify 无 SQLite DML、数据库复制、service、Docker、network 或后台对象。 +7. 聚焦、完整 Local Owner、tracked backend、18-package、package/dependency/source boundary、artifact/Edge import 与远程 CI 全部通过后，本 ADR 才能转为 Accepted。 + +## 当前证据 + +- 聚焦 Run History：`2/2`，覆盖终态 preserve/verify、两处 response-loss replay、v2 completion、CLI content-free、active Run blocked、人工越权拒绝与 sealed receipt 篡改。 +- 既有 completion v1 聚焦回归：`2/2`。 +- 完整 reconciliation：`48 total / 46 pass / 2 conditional Docker skip / 0 fail`；完整 Local Owner：`273 / 266 / 7 / 0`。 +- tracked backend：`1561 total / 1559 pass / 2 conditional skip / 0 fail`；18-package clean build/test：`2919 / 2897 / 22 / 0`。 +- package/dependency/source boundary、service-manager bridge import、Edge import 与十四档 Local artifact audit 全部 compatible；workspace 仍为 18 packages、`singleSourcePackages=[]`、`shallowSourcePackages=[]`，Local Owner 为 `175 source / 174 nested / 1 root binary entry`。 +- 基础 Edge/Standalone 制品保持 `2,611,978 / 2,612,056 bytes`、319 files、58 loaded modules；本阶段未把一次性 Owner authority 带入常驻小设备 runtime。 +- 远程 CI 尚未验证，因此本 ADR 仍为 Proposed。 diff --git a/packages/ql3-local-owner-cli/src/deployment/localDeployment.ts b/packages/ql3-local-owner-cli/src/deployment/localDeployment.ts index 57921220..392a6958 100644 --- a/packages/ql3-local-owner-cli/src/deployment/localDeployment.ts +++ b/packages/ql3-local-owner-cli/src/deployment/localDeployment.ts @@ -154,6 +154,13 @@ import { verifyLocalReconciliationAutomationApply, verifyLocalReconciliationAutomationApplyCommandFile, } from './reconciliation/application/automation/applyCoordinator'; +import { + preserveLocalReconciliationRunHistory, + preserveLocalReconciliationRunHistoryCommandFile, + readLocalReconciliationRunHistoryTerminal, + verifyLocalReconciliationRunHistory, + verifyLocalReconciliationRunHistoryCommandFile, +} from './reconciliation/application/run-history/coordinator'; import { completeLocalReconciliation, completeLocalReconciliationCommandFile, @@ -199,6 +206,11 @@ export { verifyLocalReconciliationAutomationApplyCommandFile, rollbackLocalReconciliationAutomationApply, rollbackLocalReconciliationAutomationApplyCommandFile, + preserveLocalReconciliationRunHistory, + preserveLocalReconciliationRunHistoryCommandFile, + readLocalReconciliationRunHistoryTerminal, + verifyLocalReconciliationRunHistory, + verifyLocalReconciliationRunHistoryCommandFile, completeLocalReconciliation, completeLocalReconciliationCommandFile, verifyLocalReconciliationCompletion, @@ -212,6 +224,8 @@ export { type LocalReconciliationCompletionAutomationBinding, type LocalReconciliationCompletionAutomationOptions, type LocalReconciliationCompletionOptions, + type LocalReconciliationCompletionRunHistoryBinding, + type LocalReconciliationCompletionRunHistoryOptions, type LocalReconciliationCompletionResult, type LocalReconciliationCompletionVerifyCommand, } from './reconciliation/completion/contract'; @@ -234,6 +248,23 @@ export { } from './reconciliation/application/automation/applyContract'; export { type LocalReconciliationAutomationApplyDependencies } from './reconciliation/application/automation/applyCoordinator'; +export { + normalizeLocalReconciliationRunHistoryPreserveCommand, + normalizeLocalReconciliationRunHistoryVerifyCommand, + type LocalReconciliationRunHistoryOptions, + type LocalReconciliationRunHistoryPreserveCommand, + type LocalReconciliationRunHistoryResult, + type LocalReconciliationRunHistoryVerifyCommand, +} from './reconciliation/application/run-history/contract'; +export { + type LocalReconciliationRunHistoryDependencies, + type LocalReconciliationRunHistoryTerminal, +} from './reconciliation/application/run-history/coordinator'; +export { + normalizeLocalReconciliationRunHistoryPreservationReceipt, + type LocalReconciliationRunHistoryPreservationReceipt, +} from './reconciliation/application/run-history/evidence'; + export { normalizeLocalReconciliationApplicationCommitCommand, normalizeLocalReconciliationApplicationPrepareCommand, diff --git a/packages/ql3-local-owner-cli/src/deployment/localDeploymentCli.ts b/packages/ql3-local-owner-cli/src/deployment/localDeploymentCli.ts index b7da246c..7cc50f0f 100644 --- a/packages/ql3-local-owner-cli/src/deployment/localDeploymentCli.ts +++ b/packages/ql3-local-owner-cli/src/deployment/localDeploymentCli.ts @@ -33,6 +33,8 @@ import { applyLocalReconciliationAutomationCommandFile, verifyLocalReconciliationAutomationApplyCommandFile, rollbackLocalReconciliationAutomationApplyCommandFile, + preserveLocalReconciliationRunHistoryCommandFile, + verifyLocalReconciliationRunHistoryCommandFile, completeLocalReconciliationCommandFile, verifyLocalReconciliationCompletionCommandFile, writeLocalReconciliationReviewDiagnosticsCommandFile, @@ -50,7 +52,7 @@ import { } from './localDeployment'; const USAGE = - 'Usage: ql3-local-deploy --command-file /absolute/private-command.json'; + 'Usage: ql3-local-deploy --command-file /absolute/private-command.json'; async function main(argv: readonly string[]): Promise { if (argv.length === 1 && (argv[0] === '--help' || argv[0] === '-h')) { @@ -100,6 +102,8 @@ async function main(argv: readonly string[]): Promise { argv[0] !== 'reconciliation-automation-apply' && argv[0] !== 'reconciliation-automation-apply-verify' && argv[0] !== 'reconciliation-automation-apply-rollback' && + argv[0] !== 'reconciliation-run-history-preserve' && + argv[0] !== 'reconciliation-run-history-verify' && argv[0] !== 'reconciliation-complete' && argv[0] !== 'reconciliation-complete-verify' && argv[0] !== 'compose-revision' && @@ -214,6 +218,10 @@ async function main(argv: readonly string[]): Promise { ? verifyLocalReconciliationAutomationApplyCommandFile(argv[2]!) : argv[0] === 'reconciliation-automation-apply-rollback' ? rollbackLocalReconciliationAutomationApplyCommandFile(argv[2]!) + : argv[0] === 'reconciliation-run-history-preserve' + ? preserveLocalReconciliationRunHistoryCommandFile(argv[2]!) + : argv[0] === 'reconciliation-run-history-verify' + ? verifyLocalReconciliationRunHistoryCommandFile(argv[2]!) : argv[0] === 'reconciliation-complete' ? completeLocalReconciliationCommandFile(argv[2]!) : argv[0] === 'reconciliation-complete-verify' diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/run-history/contract.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/run-history/contract.ts new file mode 100644 index 00000000..b0adc5fa --- /dev/null +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/run-history/contract.ts @@ -0,0 +1,289 @@ +import path from 'node:path'; + +import { currentIdentity } from '../../../foundation/contract'; +import { LocalDeploymentConfigurationError } from '../../../foundation/error'; + +const DIGEST = /^[0-9a-f]{64}$/; +const UUID_V4 = + /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/; +const SAFE_PATH = /^\/[A-Za-z0-9._/@-]+$/; + +export interface LocalReconciliationRunHistoryOptions { + readonly deploymentRoot: string; + readonly applicationRoot: string; + readonly runHistoryRoot: string; + readonly allowRootService: boolean; +} + +export interface LocalReconciliationRunHistoryPreserveCommand { + readonly schemaVersion: 1; + readonly operation: 'local.deployment.reconciliation.run-history.preserve'; + readonly options: Readonly; + readonly request: Readonly<{ + preservationId: string; + applicationId: string; + expectedApplicationPlanDigest: string; + expectedHeadDigest: string; + decisionFilePath: string; + preservedAtMs: number; + }>; +} + +export interface LocalReconciliationRunHistoryVerifyCommand { + readonly schemaVersion: 1; + readonly operation: 'local.deployment.reconciliation.run-history.verify'; + readonly options: Readonly; + readonly request: Readonly<{ + preservationId: string; + applicationId: string; + expectedPreservationDigest: string; + decisionFilePath: string; + }>; +} + +export interface LocalReconciliationRunHistoryResult { + readonly schemaVersion: 1; + readonly operation: + | LocalReconciliationRunHistoryPreserveCommand['operation'] + | LocalReconciliationRunHistoryVerifyCommand['operation']; + readonly status: 'preserved' | 'existing' | 'verified'; + readonly state: 'reconciliation_run_history_preserved'; + readonly preservationId: string; + readonly applicationId: string; + readonly preservationDigest: string; + readonly legacyFactCount: number; + readonly targetFactCount: number; +} + +function fail(message: string): never { + throw new LocalDeploymentConfigurationError( + `reconciliation run history ${message}`, + ); +} + +function record(value: unknown, label: string): Record { + if (!value || typeof value !== 'object' || Array.isArray(value)) { + fail(`${label} must be an object`); + } + return value as Record; +} + +function exact( + value: Record, + keys: readonly string[], + label: string, +): void { + const actual = Object.keys(value).sort(); + const expected = [...keys].sort(); + if ( + actual.length !== expected.length || + actual.some((key, index) => key !== expected[index]) + ) { + fail(`${label} shape is invalid`); + } +} + +function safePath(value: unknown, label: string): string { + if ( + typeof value !== 'string' || + !path.isAbsolute(value) || + path.parse(value).root === value || + path.normalize(value) !== value || + value.includes('\0') || + value.includes('//') || + !SAFE_PATH.test(value) || + Buffer.byteLength(value, 'utf8') > 4_096 + ) { + fail(`${label} must be a safe non-root absolute path`); + } + return value; +} + +function overlaps(left: string, right: string): boolean { + const relative = path.relative(left, right); + return ( + relative === '' || + (!relative.startsWith('..') && !path.isAbsolute(relative)) + ); +} + +function identifier(value: unknown, pattern: RegExp, label: string): string { + if (typeof value !== 'string' || !pattern.test(value)) + fail(`${label} is invalid`); + return value; +} + +function normalizeOptions( + value: unknown, +): Readonly { + const selected = record(value, 'options'); + exact( + selected, + ['allowRootService', 'applicationRoot', 'deploymentRoot', 'runHistoryRoot'], + 'options', + ); + if ( + typeof selected.allowRootService !== 'boolean' || + (currentIdentity().uid === 0) !== selected.allowRootService + ) { + fail('command identity is invalid'); + } + const roots = [ + safePath(selected.deploymentRoot, 'deploymentRoot'), + safePath(selected.applicationRoot, 'applicationRoot'), + safePath(selected.runHistoryRoot, 'runHistoryRoot'), + ]; + for (let left = 0; left < roots.length; left += 1) { + for (let right = left + 1; right < roots.length; right += 1) { + if ( + overlaps(roots[left]!, roots[right]!) || + overlaps(roots[right]!, roots[left]!) + ) { + fail('authority roots overlap'); + } + } + } + return Object.freeze({ + deploymentRoot: roots[0]!, + applicationRoot: roots[1]!, + runHistoryRoot: roots[2]!, + allowRootService: selected.allowRootService, + }); +} + +function command(value: unknown, operation: string) { + const selected = record(value, 'command'); + exact( + selected, + ['operation', 'options', 'request', 'schemaVersion'], + 'command', + ); + if (selected.schemaVersion !== 1 || selected.operation !== operation) { + fail('command version or operation is invalid'); + } + return Object.freeze({ + options: normalizeOptions(selected.options), + request: record(selected.request, 'request'), + }); +} + +function decisionFilePath( + value: unknown, + options: Readonly, +): string { + const selected = safePath(value, 'decisionFilePath'); + for (const root of [ + options.deploymentRoot, + options.applicationRoot, + options.runHistoryRoot, + ]) { + if (overlaps(root, selected) || overlaps(selected, root)) { + fail('decisionFilePath overlaps an authority root'); + } + } + return selected; +} + +export function normalizeLocalReconciliationRunHistoryPreserveCommand( + value: unknown, +): Readonly { + const selected = command( + value, + 'local.deployment.reconciliation.run-history.preserve', + ); + exact( + selected.request, + [ + 'applicationId', + 'decisionFilePath', + 'expectedApplicationPlanDigest', + 'expectedHeadDigest', + 'preservationId', + 'preservedAtMs', + ], + 'request', + ); + if ( + !Number.isSafeInteger(selected.request.preservedAtMs) || + (selected.request.preservedAtMs as number) < 0 + ) { + fail('preservedAtMs is invalid'); + } + return Object.freeze({ + schemaVersion: 1, + operation: 'local.deployment.reconciliation.run-history.preserve', + options: selected.options, + request: Object.freeze({ + preservationId: identifier( + selected.request.preservationId, + UUID_V4, + 'preservationId', + ), + applicationId: identifier( + selected.request.applicationId, + UUID_V4, + 'applicationId', + ), + expectedApplicationPlanDigest: identifier( + selected.request.expectedApplicationPlanDigest, + DIGEST, + 'expectedApplicationPlanDigest', + ), + expectedHeadDigest: identifier( + selected.request.expectedHeadDigest, + DIGEST, + 'expectedHeadDigest', + ), + decisionFilePath: decisionFilePath( + selected.request.decisionFilePath, + selected.options, + ), + preservedAtMs: selected.request.preservedAtMs as number, + }), + }); +} + +export function normalizeLocalReconciliationRunHistoryVerifyCommand( + value: unknown, +): Readonly { + const selected = command( + value, + 'local.deployment.reconciliation.run-history.verify', + ); + exact( + selected.request, + [ + 'applicationId', + 'decisionFilePath', + 'expectedPreservationDigest', + 'preservationId', + ], + 'request', + ); + return Object.freeze({ + schemaVersion: 1, + operation: 'local.deployment.reconciliation.run-history.verify', + options: selected.options, + request: Object.freeze({ + preservationId: identifier( + selected.request.preservationId, + UUID_V4, + 'preservationId', + ), + applicationId: identifier( + selected.request.applicationId, + UUID_V4, + 'applicationId', + ), + expectedPreservationDigest: identifier( + selected.request.expectedPreservationDigest, + DIGEST, + 'expectedPreservationDigest', + ), + decisionFilePath: decisionFilePath( + selected.request.decisionFilePath, + selected.options, + ), + }), + }); +} diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/run-history/coordinator.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/run-history/coordinator.ts new file mode 100644 index 00000000..cb696c9c --- /dev/null +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/run-history/coordinator.ts @@ -0,0 +1,623 @@ +import fs from 'node:fs'; +import path from 'node:path'; + +import { readPrivateLocalCommandFile } from '@qinglong/local-command-file'; + +import { currentIdentity } from '../../../foundation/contract'; +import { LocalDeploymentConfigurationError } from '../../../foundation/error'; +import { + preflightPublishedFile, + publishExactFile, + validatePrivateDirectory, +} from '../../../foundation/files'; +import { readLocalCutoverInstanceHead } from '../../../cutover/instanceLineage'; +import { readLocalReconciliationPlanTerminal } from '../../planning/preparation'; +import { + assertLocalReconciliationReviewDecisionMatchesFact, + withLocalReconciliationReviewDecisionFile, +} from '../../review/decisionFile'; +import { visitLocalReconciliationDiagnosticFacts } from '../../review/diagnostics'; +import { readLocalReconciliationReviewTerminalJson } from '../../review/terminalEvidence'; +import { + withLocalReconciliationSealedDatabase, + type LocalReconciliationSealedBundleReaderDependencies, +} from '../../sealed-bundle/reader'; +import { + readLocalReconciliationApplicationTerminal, + type LocalReconciliationApplicationTerminal, +} from '../coordinator'; +import { + normalizeLocalReconciliationRunHistoryPreserveCommand, + normalizeLocalReconciliationRunHistoryVerifyCommand, + type LocalReconciliationRunHistoryOptions, + type LocalReconciliationRunHistoryPreserveCommand, + type LocalReconciliationRunHistoryResult, + type LocalReconciliationRunHistoryVerifyCommand, +} from './contract'; +import { + buildLocalReconciliationRunHistoryPreservationReceipt, + localReconciliationRunHistoryReceiptContents, + normalizeLocalReconciliationRunHistoryPreservationReceipt, + type LocalReconciliationRunHistoryPreservationReceipt, +} from './evidence'; + +const MAX_PRESERVATIONS = 64; +const MAX_RECEIPT_BYTES = 64 * 1024; + +interface RunHistoryPaths { + readonly root: string; + readonly receipt: string; +} + +interface RunHistoryAuthority { + readonly application: Readonly; + readonly runHistoryInventoryDigest: string; + readonly bundleDigest: string; + readonly bundleFingerprintDigest: string; + readonly decisionFileDigest: string; + readonly legacyFactCount: number; + readonly targetFactCount: number; + confirmDecisionFileIdentity(): void; +} + +export interface LocalReconciliationRunHistoryDependencies + extends LocalReconciliationSealedBundleReaderDependencies { + readonly afterReceiptPublished?: () => void; + readonly afterTerminalSealed?: () => void; +} + +export interface LocalReconciliationRunHistoryTerminal { + readonly receipt: Readonly; + readonly application: Readonly; +} + +function fail(message: string, cause?: unknown): never { + throw new LocalDeploymentConfigurationError( + `reconciliation run history ${message}`, + { cause }, + ); +} + +function runHistoryPaths( + runHistoryRoot: string, + preservationId: string, +): Readonly { + const root = path.join(runHistoryRoot, preservationId); + return Object.freeze({ root, receipt: path.join(root, 'receipt.json') }); +} + +function validateDirectory( + directory: string, + uid: number, + modes: readonly number[], + label: string, +): number { + let stat: fs.Stats; + try { + stat = fs.lstatSync(directory); + } catch (error) { + return fail(`${label} is unavailable`, error); + } + const mode = stat.mode & 0o777; + if ( + !stat.isDirectory() || + stat.isSymbolicLink() || + stat.uid !== uid || + !modes.includes(mode) || + fs.realpathSync(directory) !== directory + ) { + fail(`${label} identity is invalid`); + } + return mode; +} + +function validateCatalog(selected: Readonly, sealed: boolean) { + const allowed = new Set([ + 'receipt.json', + ...(!sealed ? ['.receipt.json.ql3-deploy-stage'] : []), + ]); + for (const entry of fs.readdirSync(selected.root, { withFileTypes: true })) { + if (!allowed.has(entry.name) || entry.isSymbolicLink()) { + fail('preservation catalog contains unknown material'); + } + } +} + +function ensurePreservationDirectory( + runHistoryRoot: string, + preservationId: string, + uid: number, +): Readonly { + const selected = runHistoryPaths(runHistoryRoot, preservationId); + const entries = fs.readdirSync(runHistoryRoot, { withFileTypes: true }); + if (entries.some((entry) => !entry.isDirectory() || entry.isSymbolicLink())) { + fail('preservation root catalog contains drift'); + } + if (entries.length >= MAX_PRESERVATIONS && !fs.existsSync(selected.root)) { + fail('preservation retention limit is reached'); + } + try { + fs.mkdirSync(selected.root, { mode: 0o700 }); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== 'EEXIST') { + fail('preservation directory cannot be created', error); + } + } + const mode = validateDirectory( + selected.root, + uid, + [0o700, 0o500], + 'preservation directory', + ); + validateCatalog(selected, mode === 0o500); + return selected; +} + +function readReceipt( + selected: Readonly, + uid: number, + modes: readonly number[], +): Readonly { + let stat: fs.Stats; + try { + stat = fs.lstatSync(selected.receipt); + } catch (error) { + return fail('preservation receipt is unavailable', error); + } + if ( + !stat.isFile() || + stat.isSymbolicLink() || + stat.uid !== uid || + !modes.includes(stat.mode & 0o777) || + stat.nlink !== 1 || + stat.size < 2 || + stat.size > MAX_RECEIPT_BYTES + ) { + fail('preservation receipt identity is invalid'); + } + try { + return normalizeLocalReconciliationRunHistoryPreservationReceipt( + readLocalReconciliationReviewTerminalJson(selected.receipt, uid, modes), + ); + } catch (error) { + if (error instanceof LocalDeploymentConfigurationError) throw error; + return fail('preservation receipt cannot be read', error); + } +} + +function sealPreservation( + selected: Readonly, + uid: number, +): void { + validateDirectory( + selected.root, + uid, + [0o700, 0o500], + 'preservation directory', + ); + readReceipt(selected, uid, [0o600, 0o400]); + let descriptor = fs.openSync( + selected.receipt, + fs.constants.O_RDONLY | (fs.constants.O_NOFOLLOW ?? 0), + ); + try { + const stat = fs.fstatSync(descriptor); + if (!stat.isFile() || stat.uid !== uid || stat.nlink !== 1) { + fail('preservation receipt cannot be sealed'); + } + if ((stat.mode & 0o777) !== 0o400) { + fs.fchmodSync(descriptor, 0o400); + fs.fsyncSync(descriptor); + } + } finally { + fs.closeSync(descriptor); + } + descriptor = fs.openSync(selected.root, fs.constants.O_RDONLY); + try { + const stat = fs.fstatSync(descriptor); + if ((stat.mode & 0o777) !== 0o500) fs.fchmodSync(descriptor, 0o500); + fs.fsyncSync(descriptor); + } finally { + fs.closeSync(descriptor); + } + validateCatalog(selected, true); + readReceipt(selected, uid, [0o400]); +} + +function validateApplication( + application: Readonly, + options: Readonly, + applicationId: string, + applicationPlanDigest: string, +): void { + const runHistory = application.plan.domains.find( + (domain) => domain.domain === 'run_history', + ); + if ( + application.intent.command.options.deploymentRoot !== + options.deploymentRoot || + application.intent.command.options.applicationRoot !== + options.applicationRoot || + application.plan.applicationId !== applicationId || + application.plan.applicationPlanDigest !== applicationPlanDigest || + !runHistory || + runHistory.action !== 'adapter_required' + ) { + fail('preservation authority is detached from the application plan'); + } +} + +function assertSourceHead( + application: Readonly, + expectedHeadDigest: string, + uid: number, +): void { + const head = readLocalCutoverInstanceHead( + application.intent.command.options.deploymentRoot, + application.intent.instanceId, + uid, + ); + const automation = application.plan.domains.find( + (domain) => domain.domain === 'automation', + ); + const expectedState = + automation?.action === 'no_effect' + ? 'reconciliation_application_planned' + : automation?.action === 'adapter_required' + ? 'reconciliation_automation_applied' + : null; + if ( + expectedState === null || + head.headDigest !== expectedHeadDigest || + head.state !== expectedState || + head.instanceId !== application.intent.instanceId || + head.cutoverId !== application.intent.cutoverId || + head.generation !== application.intent.generation || + head.activationDigest !== application.intent.activationDigest + ) { + fail('preservation lost source head compare-and-swap'); + } +} + +async function deriveAuthority( + options: Readonly, + applicationId: string, + applicationPlanDigest: string, + decisionFilePath: string, + uid: number, + dependencies: LocalReconciliationRunHistoryDependencies, +): Promise> { + const application = await readLocalReconciliationApplicationTerminal( + options.applicationRoot, + applicationId, + uid, + ); + validateApplication( + application, + options, + applicationId, + applicationPlanDigest, + ); + const planTerminal = readLocalReconciliationPlanTerminal( + application.review.intent.command.options.planRoot, + application.review.intent.command.request.planId, + uid, + ); + const runHistory = planTerminal.plan.domains.find( + (domain) => domain.domain === 'run_history', + ); + if (!runHistory) fail('run history inventory is absent'); + let legacyFactCount = 0; + let targetFactCount = 0; + const reviewed = withLocalReconciliationReviewDecisionFile( + decisionFilePath, + { + reviewId: application.review.review.reviewId, + profile: application.plan.profile, + planDigest: planTerminal.plan.planDigest, + preparationDigest: application.review.intent.preparationDigest, + }, + (cursor) => { + for (const database of ['legacy', 'target'] as const) { + const opened = withLocalReconciliationSealedDatabase( + planTerminal.bundle, + database, + uid, + dependencies, + (client) => + visitLocalReconciliationDiagnosticFacts( + client, + database, + (fact) => { + if (fact.decisionRequirement === 'informational') return; + const decision = cursor.next(); + if (decision === null) { + fail('decision file omitted a canonical fact'); + } + assertLocalReconciliationReviewDecisionMatchesFact( + decision, + fact, + ); + if (fact.domain !== 'run_history') return; + if ( + fact.decisionRequirement !== 'required' || + fact.reason !== 'historical_preservation_required' || + (database === 'legacy' && + decision.disposition !== 'retain_both') || + (database === 'target' && + decision.disposition !== 'retain_target') + ) { + fail( + 'signed review did not authorize append-only preservation', + ); + } + if (database === 'legacy') legacyFactCount += 1; + else targetFactCount += 1; + }, + ), + ); + if (opened === null) { + fail('manual-required SQLite topology cannot be preserved'); + } + } + }, + ); + if ( + reviewed.evidence.fileDigest !== + application.review.authorization.decisionFileDigest || + reviewed.evidence.decisionCount !== + application.review.authorization.decisionCount || + legacyFactCount < 1 || + targetFactCount < 1 + ) { + fail('signed review authority drifted'); + } + return Object.freeze({ + application, + runHistoryInventoryDigest: runHistory.inventoryDigest, + bundleDigest: planTerminal.bundle.receipt.bundleDigest, + bundleFingerprintDigest: planTerminal.bundle.fingerprintDigest, + decisionFileDigest: reviewed.evidence.fileDigest, + legacyFactCount, + targetFactCount, + confirmDecisionFileIdentity: reviewed.confirmIdentity, + }); +} + +function validateReceiptBinding( + receipt: Readonly, + authority: Readonly, + preservationId: string, +): void { + const application = authority.application; + if ( + receipt.preservationId !== preservationId || + receipt.applicationId !== application.plan.applicationId || + receipt.profile !== application.plan.profile || + receipt.instanceId !== application.intent.instanceId || + receipt.cutoverId !== application.intent.cutoverId || + receipt.generation !== application.intent.generation || + receipt.activationDigest !== application.intent.activationDigest || + receipt.applicationPlanDigest !== application.plan.applicationPlanDigest || + receipt.reviewDigest !== application.review.review.reviewDigest || + receipt.reviewAuthorizationDigest !== + application.review.authorization.authorizationDigest || + receipt.reviewDecisionSetDigest !== + application.review.authorization.decisionSetDigest || + receipt.reviewDecisionFileDigest !== authority.decisionFileDigest || + receipt.bundleDigest !== authority.bundleDigest || + receipt.bundleFingerprintDigest !== authority.bundleFingerprintDigest || + receipt.runHistoryInventoryDigest !== authority.runHistoryInventoryDigest || + receipt.legacyFactCount !== authority.legacyFactCount || + receipt.targetFactCount !== authority.targetFactCount + ) { + fail('preservation receipt is detached from terminal authority'); + } +} + +function result( + operation: LocalReconciliationRunHistoryResult['operation'], + status: LocalReconciliationRunHistoryResult['status'], + receipt: Readonly, +): Readonly { + return Object.freeze({ + schemaVersion: 1, + operation, + status, + state: 'reconciliation_run_history_preserved', + preservationId: receipt.preservationId, + applicationId: receipt.applicationId, + preservationDigest: receipt.preservationDigest, + legacyFactCount: receipt.legacyFactCount, + targetFactCount: receipt.targetFactCount, + }); +} + +export async function readLocalReconciliationRunHistoryTerminal( + options: Readonly, + preservationId: string, + applicationId: string, + decisionFilePath: string, + uid: number, + dependencies: LocalReconciliationRunHistoryDependencies = {}, +): Promise> { + const selected = runHistoryPaths(options.runHistoryRoot, preservationId); + validateDirectory(selected.root, uid, [0o500], 'preservation directory'); + validateCatalog(selected, true); + const receipt = readReceipt(selected, uid, [0o400]); + if (receipt.applicationId !== applicationId) { + fail('preservation path binding drifted'); + } + const authority = await deriveAuthority( + options, + applicationId, + receipt.applicationPlanDigest, + decisionFilePath, + uid, + dependencies, + ); + validateReceiptBinding(receipt, authority, preservationId); + authority.confirmDecisionFileIdentity(); + return Object.freeze({ receipt, application: authority.application }); +} + +export async function preserveLocalReconciliationRunHistory( + value: unknown, + dependencies: LocalReconciliationRunHistoryDependencies = {}, +): Promise> { + const command = normalizeLocalReconciliationRunHistoryPreserveCommand(value); + const uid = currentIdentity().uid; + for (const [directory, label] of [ + [command.options.deploymentRoot, 'deploymentRoot'], + [command.options.applicationRoot, 'applicationRoot'], + [command.options.runHistoryRoot, 'runHistoryRoot'], + ] as const) { + validatePrivateDirectory(directory, uid, label); + } + const selected = ensurePreservationDirectory( + command.options.runHistoryRoot, + command.request.preservationId, + uid, + ); + const authority = await deriveAuthority( + command.options, + command.request.applicationId, + command.request.expectedApplicationPlanDigest, + command.request.decisionFilePath, + uid, + dependencies, + ); + let receipt: Readonly; + let status: 'preserved' | 'existing' = 'preserved'; + if (fs.existsSync(selected.receipt)) { + status = 'existing'; + receipt = readReceipt(selected, uid, [0o600, 0o400]); + validateReceiptBinding(receipt, authority, command.request.preservationId); + if ( + receipt.sourceHeadDigest !== command.request.expectedHeadDigest || + receipt.preservedAtMs !== command.request.preservedAtMs + ) { + fail('preservation command is not an exact replay'); + } + } else { + assertSourceHead( + authority.application, + command.request.expectedHeadDigest, + uid, + ); + if ( + command.request.preservedAtMs < authority.application.plan.committedAtMs + ) { + fail('preservation timestamp precedes application evidence'); + } + receipt = buildLocalReconciliationRunHistoryPreservationReceipt({ + preservationId: command.request.preservationId, + applicationId: command.request.applicationId, + profile: authority.application.plan.profile, + instanceId: authority.application.intent.instanceId, + cutoverId: authority.application.intent.cutoverId, + generation: authority.application.intent.generation, + activationDigest: authority.application.intent.activationDigest, + applicationPlanDigest: authority.application.plan.applicationPlanDigest, + sourceHeadDigest: command.request.expectedHeadDigest, + reviewDigest: authority.application.review.review.reviewDigest, + reviewAuthorizationDigest: + authority.application.review.authorization.authorizationDigest, + reviewDecisionSetDigest: + authority.application.review.authorization.decisionSetDigest, + reviewDecisionFileDigest: authority.decisionFileDigest, + bundleDigest: authority.bundleDigest, + bundleFingerprintDigest: authority.bundleFingerprintDigest, + runHistoryInventoryDigest: authority.runHistoryInventoryDigest, + legacyFactCount: authority.legacyFactCount, + targetFactCount: authority.targetFactCount, + preservedAtMs: command.request.preservedAtMs, + }); + const contents = localReconciliationRunHistoryReceiptContents(receipt); + preflightPublishedFile( + selected.receipt, + contents, + 0o600, + uid, + 'run history preservation receipt', + ); + authority.confirmDecisionFileIdentity(); + assertSourceHead( + authority.application, + command.request.expectedHeadDigest, + uid, + ); + publishExactFile( + selected.receipt, + contents, + 0o600, + uid, + 'run history preservation receipt', + ); + dependencies.afterReceiptPublished?.(); + } + sealPreservation(selected, uid); + dependencies.afterTerminalSealed?.(); + authority.confirmDecisionFileIdentity(); + assertSourceHead( + authority.application, + command.request.expectedHeadDigest, + uid, + ); + return result(command.operation, status, receipt); +} + +export async function verifyLocalReconciliationRunHistory( + value: unknown, + dependencies: LocalReconciliationRunHistoryDependencies = {}, +): Promise> { + const command = normalizeLocalReconciliationRunHistoryVerifyCommand(value); + const uid = currentIdentity().uid; + for (const [directory, label] of [ + [command.options.deploymentRoot, 'deploymentRoot'], + [command.options.applicationRoot, 'applicationRoot'], + [command.options.runHistoryRoot, 'runHistoryRoot'], + ] as const) { + validatePrivateDirectory(directory, uid, label); + } + const terminal = await readLocalReconciliationRunHistoryTerminal( + command.options, + command.request.preservationId, + command.request.applicationId, + command.request.decisionFilePath, + uid, + dependencies, + ); + if ( + terminal.receipt.preservationDigest !== + command.request.expectedPreservationDigest + ) { + fail('verify command is detached from preservation receipt'); + } + return result(command.operation, 'verified', terminal.receipt); +} + +export function preserveLocalReconciliationRunHistoryCommandFile( + filePath: string, + dependencies: LocalReconciliationRunHistoryDependencies = {}, +) { + return preserveLocalReconciliationRunHistory( + readPrivateLocalCommandFile(filePath), + dependencies, + ); +} + +export function verifyLocalReconciliationRunHistoryCommandFile( + filePath: string, + dependencies: LocalReconciliationRunHistoryDependencies = {}, +) { + return verifyLocalReconciliationRunHistory( + readPrivateLocalCommandFile(filePath), + dependencies, + ); +} + +export type { + LocalReconciliationRunHistoryPreserveCommand, + LocalReconciliationRunHistoryVerifyCommand, +}; diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/run-history/evidence.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/run-history/evidence.ts new file mode 100644 index 00000000..5aa8aa11 --- /dev/null +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/run-history/evidence.ts @@ -0,0 +1,158 @@ +import { LocalDeploymentConfigurationError } from '../../../foundation/error'; +import { cutoverDigest } from '../../../cutover/targetEvidence'; + +const RECEIPT_SCHEMA = + 'qinglong3-local-reconciliation-run-history-preservation-receipt'; +const DIGEST = /^[0-9a-f]{64}$/; +const UUID_V4 = + /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/; + +export interface LocalReconciliationRunHistoryPreservationReceipt { + readonly schema: typeof RECEIPT_SCHEMA; + readonly schemaVersion: 1; + readonly state: 'reconciliation_run_history_preserved'; + readonly preservationId: string; + readonly applicationId: string; + readonly profile: 'edge' | 'standalone'; + readonly instanceId: string; + readonly cutoverId: string; + readonly generation: number; + readonly activationDigest: string; + readonly applicationPlanDigest: string; + readonly sourceHeadDigest: string; + readonly reviewDigest: string; + readonly reviewAuthorizationDigest: string; + readonly reviewDecisionSetDigest: string; + readonly reviewDecisionFileDigest: string; + readonly bundleDigest: string; + readonly bundleFingerprintDigest: string; + readonly runHistoryInventoryDigest: string; + readonly legacyFactCount: number; + readonly targetFactCount: number; + readonly preservedAtMs: number; + readonly preservationDigest: string; +} + +function fail(message: string): never { + throw new LocalDeploymentConfigurationError( + `reconciliation run history evidence ${message}`, + ); +} + +function exact( + value: unknown, + keys: readonly string[], +): Record { + if (!value || typeof value !== 'object' || Array.isArray(value)) { + fail('receipt must be an object'); + } + const selected = value as Record; + const actual = Object.keys(selected).sort(); + const expected = [...keys].sort(); + if ( + actual.length !== expected.length || + actual.some((key, index) => key !== expected[index]) + ) { + fail('receipt shape is invalid'); + } + return selected; +} + +export function buildLocalReconciliationRunHistoryPreservationReceipt( + input: Omit< + LocalReconciliationRunHistoryPreservationReceipt, + 'schema' | 'schemaVersion' | 'state' | 'preservationDigest' + >, +): Readonly { + const payload = Object.freeze({ + schema: RECEIPT_SCHEMA, + schemaVersion: 1 as const, + state: 'reconciliation_run_history_preserved' as const, + ...input, + }); + return Object.freeze({ + ...payload, + preservationDigest: cutoverDigest(payload), + }); +} + +export function normalizeLocalReconciliationRunHistoryPreservationReceipt( + value: unknown, +): Readonly { + const selected = exact(value, [ + 'activationDigest', + 'applicationId', + 'applicationPlanDigest', + 'bundleDigest', + 'bundleFingerprintDigest', + 'cutoverId', + 'generation', + 'instanceId', + 'legacyFactCount', + 'preservationDigest', + 'preservationId', + 'preservedAtMs', + 'profile', + 'reviewAuthorizationDigest', + 'reviewDecisionFileDigest', + 'reviewDecisionSetDigest', + 'reviewDigest', + 'runHistoryInventoryDigest', + 'schema', + 'schemaVersion', + 'sourceHeadDigest', + 'state', + 'targetFactCount', + ]); + const { preservationDigest, ...payload } = selected; + if ( + selected.schema !== RECEIPT_SCHEMA || + selected.schemaVersion !== 1 || + selected.state !== 'reconciliation_run_history_preserved' || + typeof selected.preservationId !== 'string' || + !UUID_V4.test(selected.preservationId) || + typeof selected.applicationId !== 'string' || + !UUID_V4.test(selected.applicationId) || + (selected.profile !== 'edge' && selected.profile !== 'standalone') || + typeof selected.instanceId !== 'string' || + selected.instanceId.length < 1 || + typeof selected.cutoverId !== 'string' || + selected.cutoverId.length < 1 || + !Number.isSafeInteger(selected.generation) || + (selected.generation as number) < 1 || + ![ + selected.activationDigest, + selected.applicationPlanDigest, + selected.sourceHeadDigest, + selected.reviewDigest, + selected.reviewAuthorizationDigest, + selected.reviewDecisionSetDigest, + selected.reviewDecisionFileDigest, + selected.bundleDigest, + selected.bundleFingerprintDigest, + selected.runHistoryInventoryDigest, + preservationDigest, + ].every( + (candidate) => typeof candidate === 'string' && DIGEST.test(candidate), + ) || + !Number.isSafeInteger(selected.legacyFactCount) || + (selected.legacyFactCount as number) < 1 || + !Number.isSafeInteger(selected.targetFactCount) || + (selected.targetFactCount as number) < 1 || + !Number.isSafeInteger(selected.preservedAtMs) || + (selected.preservedAtMs as number) < 0 || + cutoverDigest(payload) !== preservationDigest + ) { + fail('receipt binding is invalid'); + } + return Object.freeze({ + ...payload, + preservationDigest, + }) as unknown as Readonly; +} + +export function localReconciliationRunHistoryReceiptContents( + receipt: Readonly, +): string { + return `${JSON.stringify(receipt, null, 2)}\n`; +} diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/contract.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/contract.ts index 4fd38c03..b12dab5d 100644 --- a/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/contract.ts +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/contract.ts @@ -22,17 +22,28 @@ export interface LocalReconciliationCompletionOptions { readonly applicationRoot: string; readonly completionRoot: string; readonly automation: Readonly | null; + readonly runHistory: Readonly | null; readonly allowRootService: boolean; } +export interface LocalReconciliationCompletionRunHistoryOptions { + readonly runHistoryRoot: string; + readonly decisionFilePath: string; +} + export interface LocalReconciliationCompletionAutomationBinding { readonly automationId: string; readonly decisionId: string; readonly expectedApplyDigest: string; } +export interface LocalReconciliationCompletionRunHistoryBinding { + readonly preservationId: string; + readonly expectedPreservationDigest: string; +} + export interface LocalReconciliationCompleteCommand { - readonly schemaVersion: 1; + readonly schemaVersion: 1 | 2; readonly operation: 'local.deployment.reconciliation.complete'; readonly options: Readonly; readonly request: Readonly<{ @@ -41,12 +52,13 @@ export interface LocalReconciliationCompleteCommand { expectedApplicationPlanDigest: string; expectedHeadDigest: string; automation: Readonly | null; + runHistory: Readonly | null; completedAtMs: number; }>; } export interface LocalReconciliationCompletionVerifyCommand { - readonly schemaVersion: 1; + readonly schemaVersion: 1 | 2; readonly operation: 'local.deployment.reconciliation.complete.verify'; readonly options: Readonly; readonly request: Readonly<{ @@ -54,6 +66,7 @@ export interface LocalReconciliationCompletionVerifyCommand { applicationId: string; expectedCompletionDigest: string; automation: Readonly | null; + runHistory: Readonly | null; }>; } @@ -68,7 +81,7 @@ export interface LocalReconciliationCompletionResult { readonly applicationId: string; readonly completionDigest: string; readonly domainCount: 8; - readonly adapterCount: 0 | 1; + readonly adapterCount: 0 | 1 | 2; readonly instanceHeadDigest: string; } @@ -169,17 +182,27 @@ function normalizeAutomationOptions( function normalizeOptions( value: unknown, + schemaVersion: 1 | 2, ): Readonly { const selected = record(value, 'options'); exact( selected, - [ - 'allowRootService', - 'applicationRoot', - 'automation', - 'completionRoot', - 'deploymentRoot', - ], + schemaVersion === 1 + ? [ + 'allowRootService', + 'applicationRoot', + 'automation', + 'completionRoot', + 'deploymentRoot', + ] + : [ + 'allowRootService', + 'applicationRoot', + 'automation', + 'completionRoot', + 'deploymentRoot', + 'runHistory', + ], 'options', ); if ( @@ -189,11 +212,16 @@ function normalizeOptions( fail('command identity is invalid'); } const automation = normalizeAutomationOptions(selected.automation); + const runHistory = + schemaVersion === 1 + ? null + : normalizeRunHistoryOptions(selected.runHistory); const normalized = Object.freeze({ deploymentRoot: safePath(selected.deploymentRoot, 'deploymentRoot'), applicationRoot: safePath(selected.applicationRoot, 'applicationRoot'), completionRoot: safePath(selected.completionRoot, 'completionRoot'), automation, + runHistory, allowRootService: selected.allowRootService, }) as Readonly; const roots = [ @@ -207,6 +235,7 @@ function normalizeOptions( automation.automationDecisionRoot, automation.automationApplyRoot, ]), + ...(runHistory === null ? [] : [runHistory.runHistoryRoot]), ]; for (let left = 0; left < roots.length; left += 1) { for (let right = left + 1; right < roots.length; right += 1) { @@ -228,9 +257,34 @@ function normalizeOptions( ) { fail('targetDatabasePath overlaps an authority root'); } + if ( + runHistory !== null && + roots.some( + (root) => + overlaps(root, runHistory.decisionFilePath) || + overlaps(runHistory.decisionFilePath, root), + ) + ) { + fail('decisionFilePath overlaps an authority root'); + } return normalized; } +function normalizeRunHistoryOptions( + value: unknown, +): Readonly { + const selected = record(value, 'run history options'); + exact( + selected, + ['decisionFilePath', 'runHistoryRoot'], + 'run history options', + ); + return Object.freeze({ + runHistoryRoot: safePath(selected.runHistoryRoot, 'runHistoryRoot'), + decisionFilePath: safePath(selected.decisionFilePath, 'decisionFilePath'), + }); +} + function normalizeAutomationBinding( value: unknown, ): Readonly | null { @@ -251,6 +305,28 @@ function normalizeAutomationBinding( }); } +function normalizeRunHistoryBinding( + value: unknown, +): Readonly { + const selected = record(value, 'run history binding'); + exact( + selected, + ['expectedPreservationDigest', 'preservationId'], + 'run history binding', + ); + return Object.freeze({ + preservationId: identifier( + selected.preservationId, + UUID_V4, + 'preservationId', + ), + expectedPreservationDigest: digest( + selected.expectedPreservationDigest, + 'expectedPreservationDigest', + ), + }); +} + function command(value: unknown, operation: string) { const selected = record(value, 'command'); exact( @@ -258,11 +334,15 @@ function command(value: unknown, operation: string) { ['operation', 'options', 'request', 'schemaVersion'], 'command', ); - if (selected.schemaVersion !== 1 || selected.operation !== operation) { + if ( + (selected.schemaVersion !== 1 && selected.schemaVersion !== 2) || + selected.operation !== operation + ) { fail('command version or operation is invalid'); } return Object.freeze({ - options: normalizeOptions(selected.options), + schemaVersion: selected.schemaVersion, + options: normalizeOptions(selected.options, selected.schemaVersion), request: record(selected.request, 'request'), }); } @@ -273,14 +353,24 @@ export function normalizeLocalReconciliationCompleteCommand( const selected = command(value, 'local.deployment.reconciliation.complete'); exact( selected.request, - [ - 'applicationId', - 'automation', - 'completedAtMs', - 'completionId', - 'expectedApplicationPlanDigest', - 'expectedHeadDigest', - ], + selected.schemaVersion === 1 + ? [ + 'applicationId', + 'automation', + 'completedAtMs', + 'completionId', + 'expectedApplicationPlanDigest', + 'expectedHeadDigest', + ] + : [ + 'applicationId', + 'automation', + 'completedAtMs', + 'completionId', + 'expectedApplicationPlanDigest', + 'expectedHeadDigest', + 'runHistory', + ], 'request', ); if ( @@ -290,7 +380,7 @@ export function normalizeLocalReconciliationCompleteCommand( fail('completedAtMs is invalid'); } return Object.freeze({ - schemaVersion: 1, + schemaVersion: selected.schemaVersion, operation: 'local.deployment.reconciliation.complete', options: selected.options, request: Object.freeze({ @@ -313,6 +403,10 @@ export function normalizeLocalReconciliationCompleteCommand( 'expectedHeadDigest', ), automation: normalizeAutomationBinding(selected.request.automation), + runHistory: + selected.schemaVersion === 1 + ? null + : normalizeRunHistoryBinding(selected.request.runHistory), completedAtMs: selected.request.completedAtMs as number, }), }); @@ -327,11 +421,24 @@ export function normalizeLocalReconciliationCompletionVerifyCommand( ); exact( selected.request, - ['applicationId', 'automation', 'completionId', 'expectedCompletionDigest'], + selected.schemaVersion === 1 + ? [ + 'applicationId', + 'automation', + 'completionId', + 'expectedCompletionDigest', + ] + : [ + 'applicationId', + 'automation', + 'completionId', + 'expectedCompletionDigest', + 'runHistory', + ], 'request', ); return Object.freeze({ - schemaVersion: 1, + schemaVersion: selected.schemaVersion, operation: 'local.deployment.reconciliation.complete.verify', options: selected.options, request: Object.freeze({ @@ -350,6 +457,10 @@ export function normalizeLocalReconciliationCompletionVerifyCommand( 'expectedCompletionDigest', ), automation: normalizeAutomationBinding(selected.request.automation), + runHistory: + selected.schemaVersion === 1 + ? null + : normalizeRunHistoryBinding(selected.request.runHistory), }), }); } diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/coordinator.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/coordinator.ts index 756770f9..b83c44bb 100644 --- a/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/coordinator.ts +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/coordinator.ts @@ -33,6 +33,11 @@ import type { LocalReconciliationAutomationApplyReceipt, } from '../application/automation/applyEvidence'; import { readLocalReconciliationAutomationDecisionTerminal } from '../application/automation/decisionCoordinator'; +import { + readLocalReconciliationRunHistoryTerminal, + type LocalReconciliationRunHistoryDependencies, +} from '../application/run-history/coordinator'; +import type { LocalReconciliationRunHistoryPreservationReceipt } from '../application/run-history/evidence'; import type { LocalReconciliationCompletionDomainEvidence } from './evidence'; import { buildLocalReconciliationCompletionReceipt, @@ -63,7 +68,12 @@ interface AutomationProof { readonly paths: ReturnType; } -export interface LocalReconciliationCompletionDependencies { +interface RunHistoryProof { + readonly receipt: Readonly; +} + +export interface LocalReconciliationCompletionDependencies + extends LocalReconciliationRunHistoryDependencies { readonly inspectSnapshot?: typeof inspectLocalSqliteSnapshot; readonly afterReceiptPublished?: () => void; readonly afterTerminalSealed?: () => void; @@ -71,6 +81,62 @@ export interface LocalReconciliationCompletionDependencies { readonly afterBackupCollected?: () => void; } +async function runHistoryProof( + command: Readonly, + terminal: Readonly, + uid: number, + dependencies: LocalReconciliationCompletionDependencies, +): Promise | null> { + const domain = terminal.plan.domains.find( + (selected) => selected.domain === 'run_history', + ); + if (!domain) fail('run history domain is absent'); + if (domain.action === 'no_effect') { + if ( + command.options.runHistory !== null || + command.request.runHistory !== null + ) { + fail('no-effect completion must not carry run history authority'); + } + return null; + } + if ( + domain.action !== 'adapter_required' || + command.options.runHistory === null || + command.request.runHistory === null + ) { + fail('run history domain is not terminally provable'); + } + validatePrivateDirectory( + command.options.runHistory.runHistoryRoot, + uid, + 'runHistoryRoot', + ); + const history = await readLocalReconciliationRunHistoryTerminal( + { + deploymentRoot: command.options.deploymentRoot, + applicationRoot: command.options.applicationRoot, + runHistoryRoot: command.options.runHistory.runHistoryRoot, + allowRootService: command.options.allowRootService, + }, + command.request.runHistory.preservationId, + command.request.applicationId, + command.options.runHistory.decisionFilePath, + uid, + dependencies, + ); + if ( + history.receipt.preservationDigest !== + command.request.runHistory.expectedPreservationDigest || + history.receipt.applicationPlanDigest !== + terminal.plan.applicationPlanDigest || + history.receipt.sourceHeadDigest !== command.request.expectedHeadDigest + ) { + fail('run history preservation evidence is detached'); + } + return Object.freeze({ receipt: history.receipt }); +} + function fail(message: string, cause?: unknown): never { throw new LocalDeploymentConfigurationError( `reconciliation completion ${message}`, @@ -393,6 +459,7 @@ async function automationProof( function domainEvidence( terminal: Readonly, automation: Readonly | null, + runHistory: Readonly | null, ): readonly Readonly[] { return Object.freeze( terminal.plan.domains.map((domain) => { @@ -416,6 +483,18 @@ function domainEvidence( evidenceDigest: automation.receipt.applyDigest, }); } + if ( + domain.domain === 'run_history' && + domain.action === 'adapter_required' && + runHistory !== null + ) { + return Object.freeze({ + domain: domain.domain, + action: 'adapter_required' as const, + evidenceKind: 'run_history_preservation' as const, + evidenceDigest: runHistory.receipt.preservationDigest, + }); + } return fail(`${domain.domain} is not terminally reconciled`); }), ); @@ -539,7 +618,13 @@ export async function completeLocalReconciliation( uid, dependencies, ); - const domains = domainEvidence(terminal, automation); + const runHistory = await runHistoryProof( + command, + terminal, + uid, + dependencies, + ); + const domains = domainEvidence(terminal, automation, runHistory); const selected = ensureCompletionDirectory( command.options.completionRoot, command.request.completionId, @@ -572,10 +657,11 @@ export async function completeLocalReconciliation( assertSourceHead(head, command.request.expectedHeadDigest, automation); const adapterCount = domains.filter( (domain) => domain.action === 'adapter_required', - ).length as 0 | 1; + ).length as 0 | 1 | 2; const latestEvidenceAtMs = Math.max( terminal.plan.committedAtMs, automation?.receipt.appliedAtMs ?? 0, + runHistory?.receipt.preservedAtMs ?? 0, ); if (command.request.completedAtMs < latestEvidenceAtMs) { fail('completion timestamp precedes terminal evidence'); @@ -682,7 +768,7 @@ export async function verifyLocalReconciliationCompletion( command.options.applicationRoot, ); const syntheticCompleteCommand = Object.freeze({ - schemaVersion: 1 as const, + schemaVersion: receipt.schemaVersion, operation: 'local.deployment.reconciliation.complete' as const, options: command.options, request: Object.freeze({ @@ -691,6 +777,7 @@ export async function verifyLocalReconciliationCompletion( expectedApplicationPlanDigest: receipt.applicationPlanDigest, expectedHeadDigest: receipt.sourceHeadDigest, automation: command.request.automation, + runHistory: command.request.runHistory, completedAtMs: receipt.completedAtMs, }), }); @@ -700,7 +787,13 @@ export async function verifyLocalReconciliationCompletion( uid, dependencies, ); - const domains = domainEvidence(terminal, automation); + const runHistory = await runHistoryProof( + syntheticCompleteCommand, + terminal, + uid, + dependencies, + ); + const domains = domainEvidence(terminal, automation, runHistory); validateReceiptBinding( receipt, terminal, diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/evidence.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/evidence.ts index 6b971e91..d622c10e 100644 --- a/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/evidence.ts +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/completion/evidence.ts @@ -13,13 +13,16 @@ const UUID_V4 = export interface LocalReconciliationCompletionDomainEvidence { readonly domain: LocalReconciliationPlanDomain; readonly action: 'no_effect' | 'adapter_required'; - readonly evidenceKind: 'application_summary' | 'automation_apply'; + readonly evidenceKind: + | 'application_summary' + | 'automation_apply' + | 'run_history_preservation'; readonly evidenceDigest: string; } export interface LocalReconciliationCompletionReceipt { readonly schema: typeof RECEIPT_SCHEMA; - readonly schemaVersion: 1; + readonly schemaVersion: 1 | 2; readonly state: 'reconciliation_completed'; readonly completionId: string; readonly applicationId: string; @@ -31,7 +34,7 @@ export interface LocalReconciliationCompletionReceipt { readonly applicationPlanDigest: string; readonly sourceHeadDigest: string; readonly domains: readonly Readonly[]; - readonly adapterCount: 0 | 1; + readonly adapterCount: 0 | 1 | 2; readonly completedAtMs: number; readonly completionDigest: string; } @@ -65,6 +68,7 @@ function exact( function domainEvidence( value: unknown, expectedDomain: LocalReconciliationPlanDomain, + schemaVersion: 1 | 2, ): Readonly { const selected = exact( value, @@ -78,9 +82,14 @@ function domainEvidence( expectedDomain === 'automation' && selected.action === 'adapter_required' && selected.evidenceKind === 'automation_apply'; + const runHistory = + schemaVersion === 2 && + expectedDomain === 'run_history' && + selected.action === 'adapter_required' && + selected.evidenceKind === 'run_history_preservation'; if ( selected.domain !== expectedDomain || - (!noEffect && !automation) || + (!noEffect && !automation && !runHistory) || typeof selected.evidenceDigest !== 'string' || !DIGEST.test(selected.evidenceDigest) ) { @@ -100,9 +109,14 @@ export function buildLocalReconciliationCompletionReceipt( 'schema' | 'schemaVersion' | 'state' | 'completionDigest' >, ): Readonly { + const schemaVersion = input.domains.some( + (domain) => domain.evidenceKind === 'run_history_preservation', + ) + ? (2 as const) + : (1 as const); const payload = Object.freeze({ schema: RECEIPT_SCHEMA, - schemaVersion: 1 as const, + schemaVersion, state: 'reconciliation_completed' as const, ...input, }); @@ -140,10 +154,14 @@ export function normalizeLocalReconciliationCompletionReceipt( if (!Array.isArray(selected.domains) || selected.domains.length !== 8) { fail('receipt domain catalog is invalid'); } + if (selected.schemaVersion !== 1 && selected.schemaVersion !== 2) { + fail('receipt schema version is invalid'); + } + const schemaVersion = selected.schemaVersion; const rawDomains = selected.domains as unknown[]; const domains = Object.freeze( LOCAL_RECONCILIATION_PLAN_DOMAINS.map((domain, index) => - domainEvidence(rawDomains[index], domain), + domainEvidence(rawDomains[index], domain, schemaVersion), ), ); const adapterCount = domains.filter( @@ -153,7 +171,10 @@ export function normalizeLocalReconciliationCompletionReceipt( const normalized = Object.freeze({ ...raw, domains }); if ( selected.schema !== RECEIPT_SCHEMA || - selected.schemaVersion !== 1 || + (schemaVersion === 2) !== + domains.some( + (domain) => domain.evidenceKind === 'run_history_preservation', + ) || selected.state !== 'reconciliation_completed' || typeof selected.completionId !== 'string' || !UUID_V4.test(selected.completionId) || @@ -174,7 +195,7 @@ export function normalizeLocalReconciliationCompletionReceipt( ].every( (candidate) => typeof candidate === 'string' && DIGEST.test(candidate), ) || - (selected.adapterCount !== 0 && selected.adapterCount !== 1) || + ![0, 1, 2].includes(selected.adapterCount as number) || selected.adapterCount !== adapterCount || !Number.isSafeInteger(selected.completedAtMs) || (selected.completedAtMs as number) < 0 || diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/review/diagnostics.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/review/diagnostics.ts index 2aaab0aa..81dfc7cc 100644 --- a/packages/ql3-local-owner-cli/src/deployment/reconciliation/review/diagnostics.ts +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/review/diagnostics.ts @@ -31,6 +31,7 @@ export type LocalReconciliationDiagnosticReason = | 'reviewable_fact' | 'unknown_schema' | 'secret_custody_required' + | 'historical_preservation_required' | 'historical_integrity_required' | 'identity_custody_required'; @@ -89,7 +90,83 @@ function quotedIdentifier(name: string): string { return `"${name.replaceAll('"', '""')}"`; } -function requirement(domain: LocalReconciliationPlanDomain): Readonly<{ +function targetRunHistoryIsTerminal(client: DatabaseSync): boolean { + const tables = new Map>(); + for (const tableName of ['Runs', 'RunAttempts', 'StepRuns'] as const) { + const table = client + .prepare( + `SELECT 1 AS present + FROM sqlite_schema + WHERE type = 'table' AND name = ? + LIMIT 1`, + ) + .get(tableName) as { readonly present?: unknown } | undefined; + if (table?.present !== 1) continue; + const columns = new Set(); + const rows = client + .prepare(`SELECT name FROM pragma_table_info(?) ORDER BY cid LIMIT 257`) + .iterate(tableName) as IterableIterator<{ readonly name?: unknown }>; + let count = 0; + for (const row of rows) { + count += 1; + if ( + count > 256 || + typeof row.name !== 'string' || + Buffer.byteLength(row.name, 'utf8') > MAX_NAME_BYTES + ) { + return false; + } + columns.add(row.name); + } + tables.set(tableName, columns); + } + const runs = tables.get('Runs'); + if (!runs?.has('status') || !runs.has('finished_at_ms')) return false; + const activeRun = client + .prepare( + `SELECT 1 AS present + FROM "Runs" + WHERE status NOT IN ('succeeded','failed','cancelled','timed_out') + OR finished_at_ms IS NULL + LIMIT 1`, + ) + .get() as { readonly present?: unknown } | undefined; + if (activeRun?.present === 1) return false; + const attempts = tables.get('RunAttempts'); + if (attempts !== undefined) { + if (!attempts.has('status')) return false; + const activeAttempt = client + .prepare( + `SELECT 1 AS present + FROM "RunAttempts" + WHERE status IN ('claimed','starting','running') + LIMIT 1`, + ) + .get() as { readonly present?: unknown } | undefined; + if (activeAttempt?.present === 1) return false; + } + const steps = tables.get('StepRuns'); + if (steps !== undefined) { + if (!steps.has('status') || !steps.has('finished_at_ms')) return false; + const activeStep = client + .prepare( + `SELECT 1 AS present + FROM "StepRuns" + WHERE status NOT IN ('succeeded','failed','skipped','cancelled','timed_out') + OR finished_at_ms IS NULL + LIMIT 1`, + ) + .get() as { readonly present?: unknown } | undefined; + if (activeStep?.present === 1) return false; + } + return true; +} + +function requirement( + client: DatabaseSync, + database: LocalReconciliationSealedDatabaseKind, + domain: LocalReconciliationPlanDomain, +): Readonly<{ decisionRequirement: LocalReconciliationDiagnosticDecisionRequirement; reason: LocalReconciliationDiagnosticReason; }> { @@ -106,6 +183,12 @@ function requirement(domain: LocalReconciliationPlanDomain): Readonly<{ }); } if (domain === 'run_history') { + if (database === 'legacy' || targetRunHistoryIsTerminal(client)) { + return Object.freeze({ + decisionRequirement: 'required' as const, + reason: 'historical_preservation_required' as const, + }); + } return Object.freeze({ decisionRequirement: 'blocked' as const, reason: 'historical_integrity_required' as const, @@ -161,8 +244,11 @@ function fact( name: string, tableName: string, count: string | null, + review: Readonly<{ + decisionRequirement: LocalReconciliationDiagnosticDecisionRequirement; + reason: LocalReconciliationDiagnosticReason; + }>, ): Readonly { - const review = requirement(domain); const payload = Object.freeze({ schema: FACT_SCHEMA, schemaVersion: 1 as const, @@ -190,6 +276,7 @@ function schemaFacts( records: readonly Readonly[]; complete: boolean; }> { + const review = requirement(client, database, domain); const rows = client .prepare( `SELECT type, name, tbl_name AS tableName @@ -240,6 +327,7 @@ function schemaFacts( row.name, row.tableName, null, + review, ), ); } @@ -256,6 +344,7 @@ function tableFacts( records: readonly Readonly[]; complete: boolean; }> { + const review = requirement(client, database, domain); const rows = client .prepare( `SELECT name, type @@ -302,6 +391,7 @@ function tableFacts( row.name, row.name, rowCount(client, domain, row.name, row.type), + review, ), ); } diff --git a/packages/ql3-local-owner-cli/test/reconciliationCapturePrepare.test.cjs b/packages/ql3-local-owner-cli/test/reconciliationCapturePrepare.test.cjs index 53f0adaf..e6195ee5 100644 --- a/packages/ql3-local-owner-cli/test/reconciliationCapturePrepare.test.cjs +++ b/packages/ql3-local-owner-cli/test/reconciliationCapturePrepare.test.cjs @@ -18,6 +18,7 @@ const { prepareLocalReconciliationCapture, prepareLocalReconciliationApplication, prepareLocalReconciliationAutomationDecision, + preserveLocalReconciliationRunHistory, readLocalReconciliationAutomationDecisionTerminal, rollbackLocalReconciliationAutomationApply, planLocalReconciliationAutomation, @@ -31,6 +32,7 @@ const { verifyLocalReconciliationCompletion, verifyLocalReconciliationPlan, verifyLocalReconciliationReview, + verifyLocalReconciliationRunHistory, writeLocalReconciliationReviewDiagnostics, } = require('../dist/deployment/localDeployment.js'); const { @@ -710,6 +712,50 @@ function automationDatabaseInitializer() { }; } +function runHistoryDatabaseInitializer() { + return ({ legacySourcePath, recoveryPath, targetDatabasePath }) => { + const legacy = new DatabaseSync(legacySourcePath); + legacy.exec(` + CREATE TABLE "CrontabStats" ( + id INTEGER PRIMARY KEY, + timestamp INTEGER NOT NULL, + status INTEGER NOT NULL + ); + INSERT INTO "CrontabStats" (id, timestamp, status) + VALUES (1, 1000, 0); + `); + legacy.close(); + fs.chmodSync(legacySourcePath, 0o600); + fs.copyFileSync(legacySourcePath, recoveryPath); + fs.chmodSync(recoveryPath, 0o600); + + const target = new DatabaseSync(targetDatabasePath); + target.exec(` + CREATE TABLE "Runs" ( + id TEXT PRIMARY KEY, + status TEXT NOT NULL, + finished_at_ms INTEGER + ); + INSERT INTO "Runs" (id, status, finished_at_ms) + VALUES ('baseline', 'succeeded', 1000); + `); + target.close(); + fs.chmodSync(targetDatabasePath, 0o600); + }; +} + +function mutateRunHistoryTarget({ targetDatabasePath }, status = 'failed') { + const target = new DatabaseSync(targetDatabasePath); + target + .prepare( + `INSERT INTO "Runs" (id, status, finished_at_ms) + VALUES (?, ?, ?)`, + ) + .run('captured', status, status === 'running' ? null : 2000); + target.close(); + return Object.freeze({}); +} + function automationReadyDatabaseInitializer() { const initializeLegacy = automationDatabaseInitializer(); return (paths) => { @@ -3244,6 +3290,303 @@ test('completion fence retains automation rollback backup while other domains re ); }); +test('run history preservation seals terminal histories and completes through v2 evidence', async (t) => { + const state = await reviewedApplicationFixture(t, { + planId: '00000000-0000-4000-8000-000000000501', + reviewId: '00000000-0000-4000-8000-000000000502', + applicationId: '00000000-0000-4000-8000-000000000503', + reviewSuffix: 'run-history-preservation', + createDefaultSidecars: false, + initializeDatabases: runHistoryDatabaseInitializer(), + mutateTarget(paths) { + return mutateRunHistoryTarget(paths); + }, + mutateDecisions(records) { + const selected = records.filter( + (record) => + record.kind === 'qinglong3-local-reconciliation-review-decision' && + record.database === 'legacy' && + record.domain === 'run_history', + ); + assert.ok(selected.length > 0); + for (const record of selected) { + record.disposition = 'retain_both'; + record.reason = 'preserve_both'; + } + }, + }); + const prepared = await prepareLocalReconciliationApplication( + state.prepareApplicationCommand, + ); + const application = await commitLocalReconciliationApplication( + applicationCommitCommand(state, prepared), + ); + assert.equal(application.outcome, 'adapter_required'); + const applicationPlan = JSON.parse( + fs.readFileSync( + path.join(state.applicationRoot, application.applicationId, 'plan.json'), + 'utf8', + ), + ); + assert.equal( + applicationPlan.domains.find((domain) => domain.domain === 'run_history') + .action, + 'adapter_required', + ); + + const runHistoryRoot = path.join( + path.dirname(state.captureRoot), + 'run-history-preservation-root', + ); + fs.mkdirSync(runHistoryRoot, { mode: 0o700 }); + const preservationCommand = { + schemaVersion: 1, + operation: 'local.deployment.reconciliation.run-history.preserve', + options: { + deploymentRoot: state.deploymentRoot, + applicationRoot: state.applicationRoot, + runHistoryRoot, + allowRootService: rootAcknowledgement(), + }, + request: { + preservationId: '00000000-0000-4000-8000-000000000504', + applicationId: application.applicationId, + expectedApplicationPlanDigest: application.applicationPlanDigest, + expectedHeadDigest: application.instanceHeadDigest, + decisionFilePath: state.reviewFile.filePath, + preservedAtMs: state.prepareApplicationCommand.request.preparedAtMs + 3, + }, + }; + for (const boundary of ['afterReceiptPublished', 'afterTerminalSealed']) { + await assert.rejects( + preserveLocalReconciliationRunHistory(preservationCommand, { + [boundary]() { + throw new Error(`run history ${boundary} response loss`); + }, + }), + new RegExp(`run history ${boundary} response loss`), + ); + } + const preserved = await preserveLocalReconciliationRunHistory( + preservationCommand, + ); + assert.equal(preserved.status, 'existing'); + assert.ok(preserved.legacyFactCount > 0); + assert.ok(preserved.targetFactCount > 0); + const preservationDirectory = path.join( + runHistoryRoot, + preservationCommand.request.preservationId, + ); + assert.deepEqual(fs.readdirSync(preservationDirectory), ['receipt.json']); + assert.equal(fs.statSync(preservationDirectory).mode & 0o777, 0o500); + assert.equal( + fs.statSync(path.join(preservationDirectory, 'receipt.json')).mode & 0o777, + 0o400, + ); + const preservationReceiptText = fs.readFileSync( + path.join(preservationDirectory, 'receipt.json'), + 'utf8', + ); + assert.equal(preservationReceiptText.includes('CrontabStats'), false); + assert.equal(preservationReceiptText.includes('captured'), false); + assert.equal( + preservationReceiptText.includes(state.reviewFile.filePath), + false, + ); + + const preservationVerifyCommand = { + schemaVersion: 1, + operation: 'local.deployment.reconciliation.run-history.verify', + options: preservationCommand.options, + request: { + preservationId: preservationCommand.request.preservationId, + applicationId: preservationCommand.request.applicationId, + expectedPreservationDigest: preserved.preservationDigest, + decisionFilePath: state.reviewFile.filePath, + }, + }; + const preservationVerified = await verifyLocalReconciliationRunHistory( + preservationVerifyCommand, + ); + assert.equal(preservationVerified.status, 'verified'); + + const completionRoot = path.join( + path.dirname(state.captureRoot), + 'completion-run-history', + ); + fs.mkdirSync(completionRoot, { mode: 0o700 }); + const runHistory = { + preservationId: preservationCommand.request.preservationId, + expectedPreservationDigest: preserved.preservationDigest, + }; + const completionCommand = { + schemaVersion: 2, + operation: 'local.deployment.reconciliation.complete', + options: { + deploymentRoot: state.deploymentRoot, + applicationRoot: state.applicationRoot, + completionRoot, + automation: null, + runHistory: { + runHistoryRoot, + decisionFilePath: state.reviewFile.filePath, + }, + allowRootService: rootAcknowledgement(), + }, + request: { + completionId: '00000000-0000-4000-8000-000000000505', + applicationId: application.applicationId, + expectedApplicationPlanDigest: application.applicationPlanDigest, + expectedHeadDigest: application.instanceHeadDigest, + automation: null, + runHistory, + completedAtMs: preservationCommand.request.preservedAtMs + 1, + }, + }; + const completed = await completeLocalReconciliation(completionCommand); + assert.equal(completed.adapterCount, 1); + const completionReceipt = JSON.parse( + fs.readFileSync( + path.join( + completionRoot, + completionCommand.request.completionId, + 'receipt.json', + ), + 'utf8', + ), + ); + assert.equal(completionReceipt.schemaVersion, 2); + assert.deepEqual( + completionReceipt.domains.find((domain) => domain.domain === 'run_history'), + { + domain: 'run_history', + action: 'adapter_required', + evidenceKind: 'run_history_preservation', + evidenceDigest: preserved.preservationDigest, + }, + ); + const completionVerified = await verifyLocalReconciliationCompletion({ + schemaVersion: 2, + operation: 'local.deployment.reconciliation.complete.verify', + options: completionCommand.options, + request: { + completionId: completionCommand.request.completionId, + applicationId: completionCommand.request.applicationId, + expectedCompletionDigest: completed.completionDigest, + automation: null, + runHistory, + }, + }); + assert.equal(completionVerified.status, 'verified'); + + const commandPath = path.join( + state.deploymentRoot, + 'run-history-verify.json', + ); + fs.writeFileSync( + commandPath, + `${JSON.stringify(preservationVerifyCommand)}\n`, + { mode: 0o600 }, + ); + const cli = spawnSync( + process.execPath, + [ + path.join(__dirname, '../dist/deployment/localDeploymentCli.js'), + 'reconciliation-run-history-verify', + '--command-file', + commandPath, + ], + { encoding: 'utf8' }, + ); + assert.equal(cli.status, 0, cli.stderr); + assert.equal(JSON.parse(cli.stdout).status, 'verified'); + assert.equal(cli.stdout.includes(runHistoryRoot), false); + assert.equal(cli.stdout.includes('CrontabStats'), false); + + const preservationReceiptPath = path.join( + preservationDirectory, + 'receipt.json', + ); + const tamperedReceipt = JSON.parse( + fs.readFileSync(preservationReceiptPath, 'utf8'), + ); + tamperedReceipt.legacyFactCount += 1; + fs.chmodSync(preservationDirectory, 0o700); + fs.chmodSync(preservationReceiptPath, 0o600); + fs.writeFileSync( + preservationReceiptPath, + `${JSON.stringify(tamperedReceipt, null, 2)}\n`, + ); + fs.chmodSync(preservationReceiptPath, 0o400); + fs.chmodSync(preservationDirectory, 0o500); + await assert.rejects( + verifyLocalReconciliationRunHistory(preservationVerifyCommand), + /receipt binding is invalid/, + ); +}); + +test('run history preservation rejects active target runs and review escalation', async (t) => { + const state = reviewCommitFixture(t, { + planId: '00000000-0000-4000-8000-000000000511', + reviewId: '00000000-0000-4000-8000-000000000512', + reviewSuffix: 'run-history-active', + createDefaultSidecars: false, + initializeDatabases: runHistoryDatabaseInitializer(), + mutateTarget(paths) { + return mutateRunHistoryTarget(paths, 'running'); + }, + }); + const pageCommand = diagnosticCommand(state, state.prepared, { + database: 'target', + domain: 'run_history', + factKind: 'table', + outputName: 'active-run-history.json', + }); + writeLocalReconciliationReviewDiagnostics(pageCommand); + const page = JSON.parse( + fs.readFileSync(pageCommand.request.outputPath, 'utf8'), + ); + assert.ok(page.records.length > 0); + assert.equal( + page.records.every( + (fact) => + fact.decisionRequirement === 'blocked' && + fact.reason === 'historical_integrity_required', + ), + true, + ); + const selected = state.reviewFile.records.find( + (record) => + record.kind === 'qinglong3-local-reconciliation-review-decision' && + record.database === 'target' && + record.domain === 'run_history', + ); + assert.ok(selected); + selected.disposition = 'retain_target'; + selected.reason = 'preserve_target'; + fs.writeFileSync( + state.reviewFile.filePath, + `${state.reviewFile.records + .map((record) => JSON.stringify(record)) + .join('\n')}\n`, + { mode: 0o600 }, + ); + await assert.rejects( + commitLocalReconciliationReview(state.command, state.dependencies), + /decision disposition is not allowed for canonical fact/, + ); + assert.equal( + fs.existsSync( + path.join( + state.reviewRoot, + state.reviewCommand.request.reviewId, + 'review.json', + ), + ), + false, + ); +}); + test('automation adapter builds a sealed row plan with bounded conflict evidence', async (t) => { const state = await reviewedApplicationFixture(t, { planId: '00000000-0000-4000-8000-000000000431', diff --git a/test/back/ql3PackageBoundaryAudit.test.cjs b/test/back/ql3PackageBoundaryAudit.test.cjs index 6083f3ef..5e9bb195 100644 --- a/test/back/ql3PackageBoundaryAudit.test.cjs +++ b/test/back/ql3PackageBoundaryAudit.test.cjs @@ -207,10 +207,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( rootSourceFileRoles: localOwnerCli.rootSourceFileRoles, }, { - sourceFiles: 172, + sourceFiles: 175, rootSourceFiles: 1, rootSourceLines: 50, - nestedSourceFiles: 171, + nestedSourceFiles: 174, rootSourceFileRoles: { 'cli.ts': 'binary_entry' }, }, );