diff --git a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md index c708165b..a057c7d3 100644 --- a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md +++ b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md @@ -6,11 +6,29 @@ - 目标版本：QingLong 3.x - 作者：QingLong Maintainers - 创建日期：2026-07-17 -- 最后更新：2026-08-21 +- 最后更新：2026-08-22 - 讨论范围：架构与演进路线，不包含最终 UI 视觉方案 -最新增量证据（2026-08-21）： +最新增量证据（2026-08-22）： +- D-393/ADR-0486（进行中）：首个 Automation adapter 已先建立独立的有界逐行 plan fence，而没有提前取得 DML authority。既有 + Legacy Crontab classifier 被复用于 exact sealed Legacy source；每行只记录 source/candidate digest、classification/reasons、proposed + Task ID、trigger count，以及 captured target 当前 Task revision/content digest 的 `absent|occupied` 冲突证据，不保存 command/spec、Task + name、reviewer、credential、Secret、路径或 target row body。planner 重新验证 D-392 application terminal、D-391 signed review 所绑定的原始 + decision file、canonical fact stream 和 Legacy `Crontabs` table 的 `adopt_legacy|retain_both`；表级选择不能覆盖 target，也不能自动改名。 + `local.deployment.reconciliation.automation.plan|verify` 以 + `reconciliation_application_planned → reconciliation_automation_planned` CAS 发布 NDJSON/receipt/seal，Edge/Standalone 上限分别为 8/32 MiB， + 双 readonly handle 的 SQLite cache 总计约 4/16 MiB，不建立随 target 规模增长的内存 Set。复用只经过 Local Admin 的窄化只读 + `adoption-inspection` 子路径，包根仍隐藏 candidate，依赖审计只对 D-393 `rowPlan.ts` 精确许可该入口。plan/receipt/seal/head 四个 response-loss + 窗口、collision→manual、空表、timezone/manual、字节预算、plan tamper 和 CLI content-free 均已覆盖；聚焦套件 + `40 total / 38 pass / 2 conditional Docker skip / 0 fail`，完整 Local Owner `262 total / 255 pass / 7 conditional skip / 0 fail`，tracked + backend `1541 total / 1539 pass / 2 conditional skip / 0 fail`，18-package clean build/逐包测试、六项相关架构/readiness 门、真实 Docker `2/2` + 与十四档 artifact audit 全通过。workspace 保持 18 packages、`singleSourcePackages=[]`、`shallowSourcePackages=[]`；Local Admin/Owner 分别为 + `46/45/1` 与 `161/160/1`（source/nested/root entry）。基础 Edge/Standalone 精确保持 `2,611,978 / 2,612,056 bytes`、319 files、58 modules， + GitNexus staged audit 为 `15 files / 56 symbols / 0 flows / LOW`；`next` 相对 `develop` 的 269-flow CRITICAL 差异继续作为 3.0 累计风险保留。因此 + ADR-0486 的行级计划切片已接受。D-393 整体仍在进行；下一切片必须继续完成 row-plan-bound signed decision、再次强认证、同 write fence + 写前 backup、幂等 transaction/replay、post-apply evidence 与 rollback，当前状态不代表 applied、restart-ready 或 reconciliation-complete。本切片无 + SQL/Cluster 变更，未重复占有 PostgreSQL HA 证明；进入 apply 写语义时必须重新选择数据库门禁。 - D-392/ADR-0485（已接受）：D-391 的 signed review 不能直接获得通用 DML authority；表级 `adopt_legacy/retain_both` 也不能证明 Automation 行级 command/trigger 兼容，更不能覆盖 Secret custody、append-only history、Plugin/AI 外部资产与 Identity/Policy 语义。 因此既有 Local Owner 新增 `reconciliation.application.prepare|commit|verify`，以 diff --git a/docs/adr/ADR-0486-bounded-automation-row-reconciliation-plan.md b/docs/adr/ADR-0486-bounded-automation-row-reconciliation-plan.md new file mode 100644 index 00000000..227bc707 --- /dev/null +++ b/docs/adr/ADR-0486-bounded-automation-row-reconciliation-plan.md @@ -0,0 +1,124 @@ +# ADR-0486：有界 Automation 行级 Reconciliation Plan + +- 状态：Accepted +- 日期：2026-08-21 +- 决策：D-393 +- 关联：ADR-0002、ADR-0482、ADR-0483、ADR-0484、ADR-0485 + +## 背景 + +ADR-0485 只把 signed schema/table review 汇总为八领域 application plan。`automation=adapter_required` 说明必须进入领域 adapter，却不能证明任一 Legacy `Crontabs` 行能够安全写为 3.0 `TaskDefinition/Trigger`。直接把表级 `adopt_legacy` 解释为批量 INSERT 会遗漏： + +- command/schedule/timezone 与 shell compatibility； +- Project 重绑定后的 Task ID 冲突； +- target current revision 的稳定内容身份； +- malformed、manual-only 与 security-review 行； +- Edge 上逐行材料的内存和文件预算； +- review file、sealed bundle 与 application head 的 exact binding。 + +既有 Legacy Crontab adoption 已实现有界分类与 canonical Task/Trigger candidate。D-393 应复用该语义，而不是建立第二套转换器；但首次 adoption 尚不知道 D-392 application、captured target 冲突和后续 backup/rollback，因此不能直接调用其 publication。复用只允许通过 `@qinglong/local-admin/adoption-inspection` 的窄化只读子路径，包根继续隐藏 candidate；依赖审计只对 D-393 `rowPlan.ts` 精确放行该子路径，不能借此取得 Local Admin 的其它写 authority。 + +## 决策 + +### 1. 先发布逐行计划，不取得 DML authority + +Local Owner 在既有 `deployment/reconciliation/application/automation/` 下提供： + +- `local.deployment.reconciliation.automation.plan`； +- `local.deployment.reconciliation.automation.verify`。 + +plan 只允许 exact `reconciliation_application_planned` head。它重新验证 application terminal、signed review 所绑定的原始 decision file、canonical sealed facts，以及 Legacy `Crontabs` table 的 `adopt_legacy|retain_both` 方向授权。没有 Automation adapter action、decision file digest/count 漂移、hot journal 或 unpaired sidecar 都失败关闭。 + +### 2. 逐行 NDJSON 为私有、内容最小化证据 + +每行重新使用既有 Legacy Crontab classifier，并记录： + +- row ordinal、source digest、classification/reasons； +- proposed Task ID、enabled、trigger count； +- Task/Trigger candidate digest，而不是 command/spec 正文； +- target 当前 Task 的 `absent`，或 current revision/content digest； +- `review_adopt | review_skip_conflict | manual_required` requirement； +- domain-separated row plan digest。 + +文件不保存 command、Task name、reviewer、credential、Secret、路径或 target row body。header 绑定 application/review/decision/bundle/head/Project/timezone，footer 绑定 inventory、row-set、计数和 outcome。外部 receipt 再绑定完整 plan file SHA-256 和字节数。 + +### 3. 冲突矩阵 + +| Legacy candidate | Target Task ID | 计划 requirement | 本阶段含义 | +|---|---|---|---| +| 可转换 | absent | `review_adopt` | 可进入后续签名行级裁决 | +| 可转换 | occupied | `review_skip_conflict` | 不覆盖 target；后续必须显式 skip/rename/manual | +| 不可转换 | 任意 | `manual_required` | 不获得自动写入资格 | + +表级 `retain_both` 不能自动改名，表级 `adopt_legacy` 也不能覆盖已存在 Task。后续 apply 必须消费 exact row plan digest 和独立签名裁决。 + +### 4. Edge/Standalone 预算 + +- plan 按 NDJSON 流式写入，单行最多 64 KiB； +- Edge 文件最多 8 MiB，Standalone 最多 32 MiB； +- Legacy 与 target 各只打开一个 readonly SQLite handle； +- Edge 每 handle cache 2 MiB，总 SQLite cache 约 4 MiB；Standalone 每 handle 8 MiB； +- 不把全量 Task ID 或 candidate 放入内存 Set/JSON array； +- 不新增 package、依赖、binary、daemon、timer 或 listener。 + +超出 byte/row/schema budget 时失败关闭，不以截断计划换取成功。 + +### 5. Crash、replay 与 seal + +plan 使用 owner-only stage、fsync、hard-link no-replace publication。plan、receipt、terminal seal 与 instance-head CAS 四个窗口都允许 exact replay： + +- 已发布 plan 会用重新派生的 exact bytes/digest 验证； +- 已发布 receipt 会验证 plan size/hash 和 command binding； +- terminal 文件为 `0400`、目录为 `0500`； +- head 只允许 `reconciliation_application_planned → reconciliation_automation_planned`； +- verify 只读，不打开 live target、不执行 DML、不修复 drift。 + +### 6. 尚未授予的能力 + +`reconciliation_automation_planned` 不等于 applied、rollback-ready 或 reconciliation-complete。它不允许启动 target、删除 Legacy、写 Task/Trigger、创建 Secret、复制日志或解释其它领域。 + +下一切片仍必须完成： + +1. exact row-plan-bound 的独立签名行级裁决与再次强认证； +2. 在同一 target write fence 内生成并验证写前 SQLite backup； +3. Project Policy fence、幂等 Task/Trigger transaction 与 response-loss replay； +4. post-apply exact evidence、无后续写保护和全库 rollback； +5. Edge 磁盘峰值及 Cluster/PostgreSQL 的独立并发/HA 语义。 + +## 被拒绝的替代方案 + +### 直接调用首次 Legacy adoption + +拒绝。它没有 D-392 application binding，也没有 captured target collision snapshot 与 rollback authority。 + +### 把 target Task ID 全量载入 Set + +拒绝。内存随 target 规模增长，低配路由设备的峰值不可控。当前实现以有界双 readonly handle 对每个 candidate 做索引查询。 + +### 在行计划保存 command/spec + +拒绝。人工裁决只需要 classification、reason、digest 与冲突证据；command 正文扩大泄露面和计划文件预算。apply 必须从 exact sealed source 重新派生 candidate。 + +### 自动为冲突 Task 改名 + +拒绝。改名会改变外部引用、Trigger ID 和用户可见 identity，必须成为明确行级决定，不能由表级 `retain_both` 推导。 + +## 验收条件 + +ADR 只有在以下证据全部通过后才可改为 Accepted： + +1. exact signed review 与 Crontabs table disposition 被重新验证； +2. 无冲突、冲突、manual/malformed、空表、timezone 与 byte budget 有测试； +3. plan/receipt/seal/head response-loss exact replay 与篡改拒绝有测试； +4. 新状态对既有 capture/plan/review/application/cutover 状态机无回归； +5. Local Owner、tracked backend、18-package、架构与 artifact gates 通过； +6. GitNexus `detect_changes` 仅报告预期 Automation/Cutover 影响。 + +## 验收证据（2026-08-21） + +- Automation 聚焦套件 `40 total / 38 pass / 2 conditional Docker skip / 0 fail`，覆盖无冲突、target collision、空表 `no_effect`、缺失 timezone 转 manual、64 KiB 注入预算失败、四个 publication response-loss 窗口、plan tamper 与 CLI 脱敏；真实 Docker sealed reconciliation `2/2` 通过。 +- 完整 Local Owner `262 total / 255 pass / 7 conditional skip / 0 fail`；18-package clean build/逐包测试退出 0；backend build 通过，tracked backend `1541 total / 1539 pass / 2 conditional skip / 0 fail`。 +- Edge import、Cluster dependency、package boundary、service-manager bridge、deployment lock surface 与 fresh readiness 定向门全部通过；workspace 保持 18 packages，`singleSourcePackages=[]`、`shallowSourcePackages=[]`。Local Admin 为 `46 source / 45 nested / 1 root public export`，Local Owner 为 `161 source / 160 nested / 1 root binary entry`，新增实现均在既有领域目录。 +- 十四档 Edge/Standalone artifact audit 全部 compatible。基础 Edge/Standalone 精确保持 `2,611,978 / 2,612,056 bytes`、319 files、58 modules；D-393 一次性 authority 没有进入低配常驻闭包。 +- GitNexus staged audit 为 `15 files / 56 symbols / 0 affected execution flows / LOW`；`next` 相对 `develop` 的 `3464 files / 48714 symbols / 269 flows / CRITICAL` 是 3.0 孵化分支累计风险，继续单独保留，不能归因于本切片。 +- 本切片不修改 SQL migration、PostgreSQL role/ACL、Pool、Cluster 拓扑或 HA 运维语义，因此不重复占有 PostgreSQL HA 证明；写前 backup/apply/rollback 切片引入 SQL 写语义时必须重新选择相应数据库门禁。 diff --git a/packages/ql3-local-admin/package.json b/packages/ql3-local-admin/package.json index 9af79f7e..a0b1ecca 100644 --- a/packages/ql3-local-admin/package.json +++ b/packages/ql3-local-admin/package.json @@ -40,6 +40,11 @@ "require": "./dist/legacy-adoption/legacyCrontabDecisionIssuer.js", "default": "./dist/legacy-adoption/legacyCrontabDecisionIssuer.js" }, + "./adoption-inspection": { + "types": "./dist/legacy-adoption/legacyCrontabInspection.d.ts", + "require": "./dist/legacy-adoption/legacyCrontabInspection.js", + "default": "./dist/legacy-adoption/legacyCrontabInspection.js" + }, "./package-staging": { "types": "./dist/plugin-package/pluginPackageStaging.d.ts", "require": "./dist/plugin-package/pluginPackageStaging.js", diff --git a/packages/ql3-local-admin/src/legacy-adoption/legacyCrontabInspection.ts b/packages/ql3-local-admin/src/legacy-adoption/legacyCrontabInspection.ts new file mode 100644 index 00000000..988fc8c7 --- /dev/null +++ b/packages/ql3-local-admin/src/legacy-adoption/legacyCrontabInspection.ts @@ -0,0 +1,2 @@ +export type { LegacyCrontabAdoptionInspection } from './legacyCrontabAdoption'; +export { visitLegacyCrontabAdoptionInspections } from './legacyCrontabAdoption'; diff --git a/packages/ql3-local-owner-cli/src/deployment/cutover/instanceLineage.ts b/packages/ql3-local-owner-cli/src/deployment/cutover/instanceLineage.ts index f98bc73c..dc1cb314 100644 --- a/packages/ql3-local-owner-cli/src/deployment/cutover/instanceLineage.ts +++ b/packages/ql3-local-owner-cli/src/deployment/cutover/instanceLineage.ts @@ -31,6 +31,7 @@ export type LocalCutoverInstanceHeadState = | 'reconciliation_reviewed' | 'reconciliation_application_prepared' | 'reconciliation_application_planned' + | 'reconciliation_automation_planned' | 'rollback_prepared' | 'legacy_restart_requested' | 'legacy_running' @@ -170,6 +171,7 @@ function parseHead(value: unknown): Readonly { head.state !== 'reconciliation_reviewed' && head.state !== 'reconciliation_application_prepared' && head.state !== 'reconciliation_application_planned' && + head.state !== 'reconciliation_automation_planned' && head.state !== 'rollback_prepared' && head.state !== 'legacy_restart_requested' && head.state !== 'legacy_running' && @@ -344,6 +346,7 @@ export function advanceLocalCutoverInstanceHead( | 'reconciliation_reviewed' | 'reconciliation_application_prepared' | 'reconciliation_application_planned' + | 'reconciliation_automation_planned' | 'rollback_prepared' | 'legacy_restart_requested' | 'legacy_running' @@ -390,6 +393,7 @@ export function advanceLocalCutoverInstanceHead( current.state === 'reconciliation_reviewed' || current.state === 'reconciliation_application_prepared' || current.state === 'reconciliation_application_planned' || + current.state === 'reconciliation_automation_planned' || current.state === 'legacy_restart_requested' || current.state === 'legacy_running' || current.state === 'legacy_ready') @@ -421,6 +425,8 @@ export function advanceLocalCutoverInstanceHead( current.state === 'reconciliation_reviewed') || (state === 'reconciliation_application_planned' && current.state === 'reconciliation_application_prepared') || + (state === 'reconciliation_automation_planned' && + current.state === 'reconciliation_application_planned') || (state === 'rollback_prepared' && current.state === 'target_stopped') || (state === 'legacy_restart_requested' && current.state === 'rollback_prepared') || diff --git a/packages/ql3-local-owner-cli/src/deployment/localDeployment.ts b/packages/ql3-local-owner-cli/src/deployment/localDeployment.ts index 9a7589f5..15e5d33c 100644 --- a/packages/ql3-local-owner-cli/src/deployment/localDeployment.ts +++ b/packages/ql3-local-owner-cli/src/deployment/localDeployment.ts @@ -131,6 +131,12 @@ import { verifyLocalReconciliationApplication, verifyLocalReconciliationApplicationCommandFile, } from './reconciliation/application/coordinator'; +import { + planLocalReconciliationAutomation, + planLocalReconciliationAutomationCommandFile, + verifyLocalReconciliationAutomationPlan, + verifyLocalReconciliationAutomationPlanCommandFile, +} from './reconciliation/application/automation/coordinator'; export { commitLocalReconciliationPlan, @@ -153,6 +159,10 @@ export { commitLocalReconciliationApplicationCommandFile, verifyLocalReconciliationApplication, verifyLocalReconciliationApplicationCommandFile, + planLocalReconciliationAutomation, + planLocalReconciliationAutomationCommandFile, + verifyLocalReconciliationAutomationPlan, + verifyLocalReconciliationAutomationPlanCommandFile, }; export { @@ -182,6 +192,28 @@ export { type LocalReconciliationApplicationPlan, type LocalReconciliationApplicationPlanReceipt, } from './reconciliation/application/plan'; +export { + normalizeLocalReconciliationAutomationPlanCommand, + normalizeLocalReconciliationAutomationVerifyCommand, + type LocalReconciliationAutomationOptions, + type LocalReconciliationAutomationPlanCommand, + type LocalReconciliationAutomationPlanResult, + type LocalReconciliationAutomationVerifyCommand, +} from './reconciliation/application/automation/contract'; +export { + type LocalReconciliationAutomationPlanDependencies, +} from './reconciliation/application/automation/coordinator'; +export { + MAX_EDGE_LOCAL_RECONCILIATION_AUTOMATION_PLAN_BYTES, + MAX_STANDALONE_LOCAL_RECONCILIATION_AUTOMATION_PLAN_BYTES, + normalizeLocalReconciliationAutomationPlanReceipt, + type LocalReconciliationAutomationPlanFooter, + type LocalReconciliationAutomationPlanHeader, + type LocalReconciliationAutomationPlanReceipt, + type LocalReconciliationAutomationPlanRow, + type LocalReconciliationAutomationPlanSummary, + type LocalReconciliationAutomationRowRequirement, +} from './reconciliation/application/automation/rowPlan'; export { LocalDeploymentConfigurationError, diff --git a/packages/ql3-local-owner-cli/src/deployment/localDeploymentCli.ts b/packages/ql3-local-owner-cli/src/deployment/localDeploymentCli.ts index abdc2dcd..59550213 100644 --- a/packages/ql3-local-owner-cli/src/deployment/localDeploymentCli.ts +++ b/packages/ql3-local-owner-cli/src/deployment/localDeploymentCli.ts @@ -25,6 +25,8 @@ import { prepareLocalReconciliationApplicationCommandFile, commitLocalReconciliationApplicationCommandFile, verifyLocalReconciliationApplicationCommandFile, + planLocalReconciliationAutomationCommandFile, + verifyLocalReconciliationAutomationPlanCommandFile, writeLocalReconciliationReviewDiagnosticsCommandFile, prepareLocalDeploymentCommandFile, proveLocalDeploymentLegacyReadinessCommandFile, @@ -40,7 +42,7 @@ import { } from './localDeployment'; const USAGE = - 'Usage: ql3-local-deploy --command-file /absolute/private-command.json'; + 'Usage: ql3-local-deploy --command-file /absolute/private-command.json'; async function main(argv: readonly string[]): Promise { if (argv.length === 1 && (argv[0] === '--help' || argv[0] === '-h')) { @@ -82,6 +84,8 @@ async function main(argv: readonly string[]): Promise { argv[0] !== 'reconciliation-application-prepare' && argv[0] !== 'reconciliation-application-commit' && argv[0] !== 'reconciliation-application-verify' && + argv[0] !== 'reconciliation-automation-plan' && + argv[0] !== 'reconciliation-automation-verify' && argv[0] !== 'compose-revision' && argv[0] !== 'compose-preflight' && argv[0] !== 'compose-apply' && @@ -178,6 +182,10 @@ async function main(argv: readonly string[]): Promise { ? commitLocalReconciliationApplicationCommandFile(argv[2]!) : argv[0] === 'reconciliation-application-verify' ? verifyLocalReconciliationApplicationCommandFile(argv[2]!) + : argv[0] === 'reconciliation-automation-plan' + ? planLocalReconciliationAutomationCommandFile(argv[2]!) + : argv[0] === 'reconciliation-automation-verify' + ? verifyLocalReconciliationAutomationPlanCommandFile(argv[2]!) : argv[0] === 'compose-revision' ? switchLocalDeploymentComposeRevisionCommandFile(argv[2]!) : argv[0] === 'compose-preflight' diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/automation/contract.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/automation/contract.ts new file mode 100644 index 00000000..28ba7636 --- /dev/null +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/automation/contract.ts @@ -0,0 +1,325 @@ +import path from 'node:path'; + +import { currentIdentity } from '../../../foundation/contract'; +import { LocalDeploymentConfigurationError } from '../../../foundation/error'; + +const DIGEST_PATTERN = /^[0-9a-f]{64}$/; +const UUID_V4_PATTERN = + /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/; +const SAFE_PATH_PATTERN = /^\/[A-Za-z0-9._/@-]+$/; +const MAX_PATH_BYTES = 4_096; + +export interface LocalReconciliationAutomationOptions { + readonly deploymentRoot: string; + readonly applicationRoot: string; + readonly automationRoot: string; + readonly allowRootService: boolean; +} + +export interface LocalReconciliationAutomationPlanCommand { + readonly schemaVersion: 1; + readonly operation: 'local.deployment.reconciliation.automation.plan'; + readonly options: Readonly; + readonly request: Readonly<{ + automationId: string; + applicationId: string; + expectedApplicationPlanDigest: string; + expectedHeadDigest: string; + decisionFilePath: string; + projectId: string; + legacyTimezone: string | null; + preparedAtMs: number; + }>; +} + +export interface LocalReconciliationAutomationVerifyCommand { + readonly schemaVersion: 1; + readonly operation: 'local.deployment.reconciliation.automation.verify'; + readonly options: Readonly; + readonly request: Readonly<{ + automationId: string; + expectedAutomationPlanDigest: string; + }>; +} + +export interface LocalReconciliationAutomationPlanResult { + readonly schemaVersion: 1; + readonly operation: + | 'local.deployment.reconciliation.automation.plan' + | 'local.deployment.reconciliation.automation.verify'; + readonly status: 'prepared' | 'existing' | 'verified'; + readonly state: 'reconciliation_automation_planned'; + readonly automationId: string; + readonly automationPlanDigest: string; + readonly outcome: 'ready' | 'manual_required' | 'no_effect'; + readonly rowCount: number; + readonly eligibleCount: number; + readonly conflictCount: number; + readonly instanceHeadDigest: string; +} + +function configurationError(message: string): never { + throw new LocalDeploymentConfigurationError( + `reconciliation automation ${message}`, + ); +} + +function object(value: unknown, label: string): Record { + if ( + !value || + typeof value !== 'object' || + Array.isArray(value) || + (Object.getPrototypeOf(value) !== Object.prototype && + Object.getPrototypeOf(value) !== null) + ) { + configurationError(`${label} must be an object`); + } + return value as Record; +} + +function exact( + value: Record, + keys: readonly string[], + label: string, +): void { + const actual = Object.keys(value).sort(); + const expected = [...keys].sort(); + if ( + actual.length !== expected.length || + actual.some((key, index) => key !== expected[index]) + ) { + configurationError(`${label} shape is invalid`); + } +} + +function safePath(value: unknown, label: string): string { + if ( + typeof value !== 'string' || + !path.isAbsolute(value) || + path.parse(value).root === value || + path.normalize(value) !== value || + value.includes('\0') || + value.includes('//') || + !SAFE_PATH_PATTERN.test(value) || + Buffer.byteLength(value, 'utf8') > MAX_PATH_BYTES + ) { + configurationError(`${label} must be a safe non-root absolute path`); + } + return value; +} + +function overlaps(left: string, right: string): boolean { + const relative = path.relative(left, right); + return ( + relative === '' || + (!relative.startsWith('..') && !path.isAbsolute(relative)) + ); +} + +function digest(value: unknown, label: string): string { + if (typeof value !== 'string' || !DIGEST_PATTERN.test(value)) { + configurationError(`${label} must be a SHA-256 digest`); + } + return value; +} + +function identifier(value: unknown, label: string): string { + if (typeof value !== 'string' || !UUID_V4_PATTERN.test(value)) { + configurationError(`${label} must be a lowercase UUID v4`); + } + return value; +} + +function projectId(value: unknown): string { + if ( + typeof value !== 'string' || + value.length < 1 || + value.length > 128 || + /[\u0000-\u001f\u007f]/u.test(value) + ) { + configurationError('projectId is invalid'); + } + return value; +} + +function timezone(value: unknown): string | null { + if (value === null) return null; + if ( + typeof value !== 'string' || + value.length < 1 || + Buffer.byteLength(value, 'utf8') > 128 || + value.includes('\0') + ) { + configurationError('legacyTimezone is invalid'); + } + try { + return new Intl.DateTimeFormat('en-US', { + timeZone: value, + }).resolvedOptions().timeZone; + } catch { + return configurationError('legacyTimezone is unsupported'); + } +} + +function normalizeOptions( + value: unknown, +): Readonly { + const options = object(value, 'options'); + exact( + options, + [ + 'allowRootService', + 'applicationRoot', + 'automationRoot', + 'deploymentRoot', + ], + 'options', + ); + const identity = currentIdentity(); + if ( + typeof options.allowRootService !== 'boolean' || + (identity.uid === 0) !== options.allowRootService + ) { + configurationError('command identity is invalid'); + } + const roots = [ + safePath(options.deploymentRoot, 'deploymentRoot'), + safePath(options.applicationRoot, 'applicationRoot'), + safePath(options.automationRoot, 'automationRoot'), + ]; + for (let left = 0; left < roots.length; left += 1) { + for (let right = left + 1; right < roots.length; right += 1) { + if ( + overlaps(roots[left]!, roots[right]!) || + overlaps(roots[right]!, roots[left]!) + ) { + configurationError('deployment, application and automation roots overlap'); + } + } + } + return Object.freeze({ + deploymentRoot: roots[0]!, + applicationRoot: roots[1]!, + automationRoot: roots[2]!, + allowRootService: options.allowRootService, + }); +} + +function command( + value: unknown, + operation: + | LocalReconciliationAutomationPlanCommand['operation'] + | LocalReconciliationAutomationVerifyCommand['operation'], +): Readonly<{ + options: Readonly; + request: Record; +}> { + const selected = object(value, 'command'); + exact( + selected, + ['operation', 'options', 'request', 'schemaVersion'], + 'command', + ); + if (selected.schemaVersion !== 1 || selected.operation !== operation) { + configurationError('command version or operation is invalid'); + } + return Object.freeze({ + options: normalizeOptions(selected.options), + request: object(selected.request, 'request'), + }); +} + +export function normalizeLocalReconciliationAutomationPlanCommand( + value: unknown, +): Readonly { + const selected = command( + value, + 'local.deployment.reconciliation.automation.plan', + ); + exact( + selected.request, + [ + 'applicationId', + 'automationId', + 'decisionFilePath', + 'expectedApplicationPlanDigest', + 'expectedHeadDigest', + 'legacyTimezone', + 'preparedAtMs', + 'projectId', + ], + 'request', + ); + const decisionFilePath = safePath( + selected.request.decisionFilePath, + 'decisionFilePath', + ); + if ( + [ + selected.options.deploymentRoot, + selected.options.applicationRoot, + selected.options.automationRoot, + ].some( + (root) => + overlaps(root, decisionFilePath) || overlaps(decisionFilePath, root), + ) + ) { + configurationError('decisionFilePath overlaps an authority root'); + } + if ( + !Number.isSafeInteger(selected.request.preparedAtMs) || + (selected.request.preparedAtMs as number) < 0 + ) { + configurationError('preparedAtMs is invalid'); + } + return Object.freeze({ + schemaVersion: 1, + operation: 'local.deployment.reconciliation.automation.plan', + options: selected.options, + request: Object.freeze({ + automationId: identifier(selected.request.automationId, 'automationId'), + applicationId: identifier( + selected.request.applicationId, + 'applicationId', + ), + expectedApplicationPlanDigest: digest( + selected.request.expectedApplicationPlanDigest, + 'expectedApplicationPlanDigest', + ), + expectedHeadDigest: digest( + selected.request.expectedHeadDigest, + 'expectedHeadDigest', + ), + decisionFilePath, + projectId: projectId(selected.request.projectId), + legacyTimezone: timezone(selected.request.legacyTimezone), + preparedAtMs: selected.request.preparedAtMs as number, + }), + }); +} + +export function normalizeLocalReconciliationAutomationVerifyCommand( + value: unknown, +): Readonly { + const selected = command( + value, + 'local.deployment.reconciliation.automation.verify', + ); + exact( + selected.request, + ['automationId', 'expectedAutomationPlanDigest'], + 'request', + ); + return Object.freeze({ + schemaVersion: 1, + operation: 'local.deployment.reconciliation.automation.verify', + options: selected.options, + request: Object.freeze({ + automationId: identifier(selected.request.automationId, 'automationId'), + expectedAutomationPlanDigest: digest( + selected.request.expectedAutomationPlanDigest, + 'expectedAutomationPlanDigest', + ), + }), + }); +} diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/automation/coordinator.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/automation/coordinator.ts new file mode 100644 index 00000000..d18c5884 --- /dev/null +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/automation/coordinator.ts @@ -0,0 +1,792 @@ +import fs from 'node:fs'; +import path from 'node:path'; + +import { readPrivateLocalCommandFile } from '@qinglong/local-command-file'; + +import { currentIdentity } from '../../../foundation/contract'; +import { LocalDeploymentConfigurationError } from '../../../foundation/error'; +import { + ensurePrivateDirectory, + publishExactFile, + syncPublishedDirectory, + validatePrivateDirectory, +} from '../../../foundation/files'; +import { + advanceLocalCutoverInstanceHead, + readLocalCutoverInstanceHead, + type LocalCutoverInstanceHead, +} from '../../../cutover/instanceLineage'; +import { readLocalReconciliationPlanTerminal } from '../../planning/preparation'; +import { + assertLocalReconciliationReviewDecisionMatchesFact, + withLocalReconciliationReviewDecisionFile, + type LocalReconciliationReviewDisposition, +} from '../../review/decisionFile'; +import { visitLocalReconciliationDiagnosticFacts } from '../../review/diagnostics'; +import { withLocalReconciliationSealedDatabase } from '../../sealed-bundle/reader'; +import { + readLocalReconciliationApplicationTerminal, + type LocalReconciliationApplicationTerminal, +} from '../coordinator'; +import { + normalizeLocalReconciliationAutomationPlanCommand, + normalizeLocalReconciliationAutomationVerifyCommand, + type LocalReconciliationAutomationPlanCommand, + type LocalReconciliationAutomationPlanResult, + type LocalReconciliationAutomationVerifyCommand, +} from './contract'; +import { + buildLocalReconciliationAutomationPlanReceipt, + hashLocalReconciliationAutomationPlanFile, + MAX_EDGE_LOCAL_RECONCILIATION_AUTOMATION_PLAN_BYTES, + MAX_STANDALONE_LOCAL_RECONCILIATION_AUTOMATION_PLAN_BYTES, + normalizeLocalReconciliationAutomationPlanReceipt, + writeLocalReconciliationAutomationPlan, + type LocalReconciliationAutomationPlanHeader, + type LocalReconciliationAutomationPlanReceipt, +} from './rowPlan'; + +const MAX_RECEIPT_BYTES = 64 * 1024; + +export interface LocalReconciliationAutomationPlanDependencies { + readonly beforeDatabaseOpen?: ( + kind: 'legacy' | 'target', + mode: 'main_only_immutable' | 'wal_shm_readonly', + cacheKiB: 2_048 | 8_192, + ) => void; + readonly afterDatabaseClose?: (kind: 'legacy' | 'target') => void; + readonly afterPlanPublished?: () => void; + readonly afterReceiptPublished?: () => void; + readonly afterTerminalSealed?: () => void; + readonly afterHeadAdvanced?: () => void; +} + +interface AutomationPaths { + readonly root: string; + readonly staging: string; + readonly plan: string; + readonly planStage: string; + readonly receipt: string; +} + +function configurationError(message: string, cause?: unknown): never { + throw new LocalDeploymentConfigurationError( + `reconciliation automation ${message}`, + { cause }, + ); +} + +function automationPaths( + automationRoot: string, + automationId: string, +): Readonly { + const root = path.join(automationRoot, automationId); + const staging = path.join(root, 'staging'); + return Object.freeze({ + root, + staging, + plan: path.join(root, 'plan.ndjson'), + planStage: path.join(staging, 'plan.ndjson.stage'), + receipt: path.join(root, 'receipt.json'), + }); +} + +function validateCatalog(selected: Readonly, terminal: boolean): void { + const allowed = new Set([ + 'plan.ndjson', + 'receipt.json', + 'staging', + ...(!terminal ? ['.receipt.json.ql3-deploy-stage'] : []), + ]); + for (const entry of fs.readdirSync(selected.root, { withFileTypes: true })) { + if (!allowed.has(entry.name) || entry.isSymbolicLink()) { + configurationError('plan root contains unknown material'); + } + } + const stagingEntries = fs.readdirSync(selected.staging); + if ( + terminal + ? stagingEntries.length !== 0 + : stagingEntries.some((entry) => entry !== 'plan.ndjson.stage') + ) { + configurationError('plan staging contains unknown material'); + } +} + +function validateDirectory( + directory: string, + uid: number, + modes: readonly number[], + label: string, +): number { + let stat: fs.Stats; + try { + stat = fs.lstatSync(directory); + } catch (error) { + return configurationError(`${label} is unavailable`, error); + } + const mode = stat.mode & 0o777; + if ( + !stat.isDirectory() || + stat.isSymbolicLink() || + stat.uid !== uid || + !modes.includes(mode) || + fs.realpathSync(directory) !== directory + ) { + configurationError(`${label} identity is invalid`); + } + return mode; +} + +function readReceipt( + filePath: string, + uid: number, + allowedModes: readonly number[], +): Readonly { + let descriptor: number | undefined; + let bytes: Buffer | undefined; + try { + const before = fs.lstatSync(filePath, { bigint: true }); + if ( + !before.isFile() || + before.isSymbolicLink() || + Number(before.uid) !== uid || + !allowedModes.includes(Number(before.mode) & 0o777) || + before.nlink !== 1n || + before.size < 2n || + before.size > BigInt(MAX_RECEIPT_BYTES) + ) { + configurationError('receipt identity is invalid'); + } + descriptor = fs.openSync( + filePath, + fs.constants.O_RDONLY | (fs.constants.O_NOFOLLOW ?? 0), + ); + const opened = fs.fstatSync(descriptor, { bigint: true }); + if ( + opened.dev !== before.dev || + opened.ino !== before.ino || + opened.size !== before.size + ) { + configurationError('receipt changed while opening'); + } + bytes = fs.readFileSync(descriptor); + const after = fs.fstatSync(descriptor, { bigint: true }); + const current = fs.lstatSync(filePath, { bigint: true }); + if ( + after.dev !== before.dev || + after.ino !== before.ino || + after.size !== before.size || + current.dev !== before.dev || + current.ino !== before.ino || + current.mtimeNs !== before.mtimeNs || + current.ctimeNs !== before.ctimeNs || + current.mode !== before.mode || + current.nlink !== before.nlink + ) { + configurationError('receipt changed while reading'); + } + return normalizeLocalReconciliationAutomationPlanReceipt( + JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(bytes)), + ); + } catch (error) { + if (error instanceof LocalDeploymentConfigurationError) throw error; + return configurationError('receipt cannot be read', error); + } finally { + bytes?.fill(0); + if (descriptor !== undefined) fs.closeSync(descriptor); + } +} + +function validatePlanFile( + filePath: string, + receipt: Readonly, + uid: number, + allowedModes: readonly number[], +): void { + let descriptor: number | undefined; + try { + const before = fs.lstatSync(filePath, { bigint: true }); + if ( + !before.isFile() || + before.isSymbolicLink() || + Number(before.uid) !== uid || + !allowedModes.includes(Number(before.mode) & 0o777) || + before.nlink !== 1n || + before.size !== BigInt(receipt.planFileBytes) + ) { + configurationError('plan file identity is invalid'); + } + descriptor = fs.openSync( + filePath, + fs.constants.O_RDONLY | (fs.constants.O_NOFOLLOW ?? 0), + ); + const opened = fs.fstatSync(descriptor, { bigint: true }); + if ( + opened.dev !== before.dev || + opened.ino !== before.ino || + opened.size !== before.size || + hashLocalReconciliationAutomationPlanFile( + descriptor, + receipt.planFileBytes, + ) !== receipt.planFileDigest + ) { + configurationError('plan file content drifted'); + } + const after = fs.fstatSync(descriptor, { bigint: true }); + const current = fs.lstatSync(filePath, { bigint: true }); + if ( + after.dev !== before.dev || + after.ino !== before.ino || + after.mtimeNs !== before.mtimeNs || + after.ctimeNs !== before.ctimeNs || + current.dev !== before.dev || + current.ino !== before.ino || + current.mtimeNs !== before.mtimeNs || + current.ctimeNs !== before.ctimeNs + ) { + configurationError('plan file changed while verifying'); + } + } catch (error) { + if (error instanceof LocalDeploymentConfigurationError) throw error; + configurationError('plan file cannot be verified', error); + } finally { + if (descriptor !== undefined) fs.closeSync(descriptor); + } +} + +function receiptContents( + receipt: Readonly, +): string { + const contents = `${JSON.stringify(receipt, null, 2)}\n`; + if (Buffer.byteLength(contents, 'utf8') > MAX_RECEIPT_BYTES) { + configurationError('receipt exceeds 64 KiB'); + } + return contents; +} + +function validateApplicationBinding( + command: Readonly, + terminal: Readonly, +): void { + if ( + terminal.intent.command.options.deploymentRoot !== + command.options.deploymentRoot || + terminal.intent.command.options.applicationRoot !== + command.options.applicationRoot || + terminal.plan.applicationId !== command.request.applicationId || + terminal.plan.applicationPlanDigest !== + command.request.expectedApplicationPlanDigest || + terminal.head.state !== 'reconciliation_application_planned' || + terminal.head.headDigest !== command.request.expectedHeadDigest || + terminal.head.sourceRecordDigest !== terminal.plan.applicationPlanDigest || + command.request.preparedAtMs < terminal.plan.committedAtMs + ) { + configurationError('plan is detached from its exact application head'); + } + const automation = terminal.plan.domains.find( + (selected) => selected.domain === 'automation', + ); + if ( + !automation || + (automation.action !== 'adapter_required' && + automation.action !== 'adapter_and_manual') + ) { + configurationError('application plan does not require an automation adapter'); + } +} + +function tableDisposition( + command: Readonly, + terminal: Readonly, + dependencies: LocalReconciliationAutomationPlanDependencies, + uid: number, +): Readonly<{ + disposition: 'adopt_legacy' | 'retain_both'; + decisionFileDigest: string; + confirmDecisionFileIdentity(): void; + planTerminal: ReturnType; +}> { + const planTerminal = readLocalReconciliationPlanTerminal( + terminal.review.intent.command.options.planRoot, + terminal.review.intent.command.request.planId, + uid, + ); + let selectedDisposition: LocalReconciliationReviewDisposition | undefined; + const reviewed = withLocalReconciliationReviewDecisionFile( + command.request.decisionFilePath, + { + reviewId: terminal.review.review.reviewId, + profile: terminal.plan.profile, + planDigest: planTerminal.plan.planDigest, + preparationDigest: terminal.review.intent.preparationDigest, + }, + (cursor) => { + for (const database of ['legacy', 'target'] as const) { + const opened = withLocalReconciliationSealedDatabase( + planTerminal.bundle, + database, + uid, + dependencies, + (client) => + visitLocalReconciliationDiagnosticFacts( + client, + database, + (fact) => { + if (fact.decisionRequirement === 'informational') return; + const decision = cursor.next(); + if (decision === null) { + configurationError('decision file omitted a canonical fact'); + } + assertLocalReconciliationReviewDecisionMatchesFact( + decision, + fact, + ); + if ( + fact.database === 'legacy' && + fact.domain === 'automation' && + fact.factKind === 'table' && + fact.tableName === 'Crontabs' + ) { + if (selectedDisposition !== undefined) { + configurationError('legacy Crontabs authority is ambiguous'); + } + selectedDisposition = decision.disposition; + } + }, + ), + ); + if (opened === null) { + configurationError('manual-required SQLite topology cannot be adapted'); + } + } + }, + ); + if ( + reviewed.evidence.fileDigest !== + terminal.review.authorization.decisionFileDigest || + reviewed.evidence.decisionCount !== + terminal.review.authorization.decisionCount || + (selectedDisposition !== 'adopt_legacy' && + selectedDisposition !== 'retain_both') + ) { + configurationError('signed review did not authorize legacy Crontabs adoption'); + } + return Object.freeze({ + disposition: selectedDisposition, + decisionFileDigest: reviewed.evidence.fileDigest, + confirmDecisionFileIdentity: reviewed.confirmIdentity, + planTerminal, + }); +} + +function maxPlanBytes(profile: 'edge' | 'standalone'): number { + return profile === 'edge' + ? MAX_EDGE_LOCAL_RECONCILIATION_AUTOMATION_PLAN_BYTES + : MAX_STANDALONE_LOCAL_RECONCILIATION_AUTOMATION_PLAN_BYTES; +} + +function publishPlan( + selected: Readonly, + command: Readonly, + terminal: Readonly, + authority: ReturnType, + dependencies: LocalReconciliationAutomationPlanDependencies, + uid: number, +): Readonly { + let descriptor: number | undefined; + let createdStage = false; + try { + descriptor = fs.openSync( + selected.planStage, + fs.constants.O_CREAT | + fs.constants.O_EXCL | + fs.constants.O_WRONLY | + (fs.constants.O_NOFOLLOW ?? 0), + 0o600, + ); + createdStage = true; + fs.fchmodSync(descriptor, 0o600); + const header: Omit = + Object.freeze({ + schemaVersion: 1, + kind: 'qinglong3-local-reconciliation-automation-plan-header', + automationId: command.request.automationId, + applicationId: command.request.applicationId, + applicationPlanDigest: terminal.plan.applicationPlanDigest, + reviewDigest: terminal.review.review.reviewDigest, + reviewAuthorizationDigest: + terminal.review.authorization.authorizationDigest, + reviewDecisionSetDigest: + terminal.review.authorization.decisionSetDigest, + reviewDecisionFileDigest: authority.decisionFileDigest, + bundleDigest: authority.planTerminal.bundle.receipt.bundleDigest, + bundleFingerprintDigest: authority.planTerminal.bundle.fingerprintDigest, + profile: terminal.plan.profile, + projectId: command.request.projectId, + legacyTimezone: command.request.legacyTimezone, + tableDisposition: authority.disposition, + preparedHeadDigest: terminal.head.headDigest, + preparedAtMs: command.request.preparedAtMs, + }); + const generated = withLocalReconciliationSealedDatabase( + authority.planTerminal.bundle, + 'target', + uid, + dependencies, + (target) => + withLocalReconciliationSealedDatabase( + authority.planTerminal.bundle, + 'legacy', + uid, + dependencies, + (legacy) => + writeLocalReconciliationAutomationPlan({ + descriptor: descriptor!, + maxBytes: maxPlanBytes(terminal.plan.profile), + header, + legacy, + target, + }), + ), + ); + if (generated === null || generated === undefined) { + configurationError('manual-required SQLite topology cannot be planned'); + } + fs.fsyncSync(descriptor); + fs.closeSync(descriptor); + descriptor = undefined; + authority.confirmDecisionFileIdentity(); + const receipt = buildLocalReconciliationAutomationPlanReceipt( + generated.header, + generated.footer, + generated.fileBytes, + generated.fileDigest, + ); + if (fs.existsSync(selected.plan)) { + validatePlanFile(selected.plan, receipt, uid, [0o600]); + } else { + try { + fs.linkSync(selected.planStage, selected.plan); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error; + validatePlanFile(selected.plan, receipt, uid, [0o600]); + } + syncPublishedDirectory(selected.root); + } + fs.unlinkSync(selected.planStage); + createdStage = false; + syncPublishedDirectory(selected.staging); + return receipt; + } catch (error) { + if (error instanceof LocalDeploymentConfigurationError) throw error; + return configurationError('plan cannot be published', error); + } finally { + if (descriptor !== undefined) fs.closeSync(descriptor); + if (createdStage) { + try { + fs.unlinkSync(selected.planStage); + } catch { + // A complete stage remains recoverable; partial stages fail closed. + } + } + } +} + +function sealFile(filePath: string, uid: number): void { + let descriptor: number | undefined; + try { + const before = fs.lstatSync(filePath, { bigint: true }); + if ( + !before.isFile() || + before.isSymbolicLink() || + Number(before.uid) !== uid || + ![0o600, 0o400].includes(Number(before.mode) & 0o777) || + before.nlink !== 1n + ) { + configurationError('terminal file cannot be sealed'); + } + descriptor = fs.openSync( + filePath, + fs.constants.O_RDONLY | (fs.constants.O_NOFOLLOW ?? 0), + ); + const opened = fs.fstatSync(descriptor, { bigint: true }); + if ( + opened.dev !== before.dev || + opened.ino !== before.ino || + opened.size !== before.size + ) { + configurationError('terminal file changed while sealing'); + } + if ((Number(opened.mode) & 0o777) !== 0o400) { + fs.fchmodSync(descriptor, 0o400); + } + fs.fsyncSync(descriptor); + } finally { + if (descriptor !== undefined) fs.closeSync(descriptor); + } +} + +function sealDirectory(directory: string, uid: number): void { + const mode = validateDirectory(directory, uid, [0o700, 0o500], 'terminal directory'); + const descriptor = fs.openSync(directory, fs.constants.O_RDONLY); + try { + if (mode !== 0o500) fs.fchmodSync(descriptor, 0o500); + fs.fsyncSync(descriptor); + } finally { + fs.closeSync(descriptor); + } +} + +function sealTerminal(selected: Readonly, uid: number): void { + if (fs.readdirSync(selected.staging).length !== 0) { + configurationError('staging must be empty before terminal seal'); + } + sealFile(selected.plan, uid); + sealFile(selected.receipt, uid); + sealDirectory(selected.staging, uid); + sealDirectory(selected.root, uid); + validateCatalog(selected, true); +} + +function advanceHead( + terminal: Readonly, + receipt: Readonly, + uid: number, +): Readonly { + return advanceLocalCutoverInstanceHead( + { + options: { + deploymentRoot: terminal.intent.command.options.deploymentRoot, + }, + request: { + cutoverId: terminal.intent.cutoverId, + profile: terminal.intent.profile, + instanceId: terminal.intent.instanceId, + expectedActivationDigest: terminal.intent.activationDigest, + requestedAtMs: receipt.preparedAtMs, + }, + }, + uid, + 'reconciliation_automation_planned', + terminal.intent.generation, + receipt.automationPlanDigest, + ); +} + +function result( + operation: LocalReconciliationAutomationPlanResult['operation'], + status: LocalReconciliationAutomationPlanResult['status'], + receipt: Readonly, + head: Readonly, +): Readonly { + return Object.freeze({ + schemaVersion: 1, + operation, + status, + state: 'reconciliation_automation_planned', + automationId: receipt.automationId, + automationPlanDigest: receipt.automationPlanDigest, + outcome: receipt.outcome, + rowCount: receipt.rowCount, + eligibleCount: receipt.eligibleCount, + conflictCount: receipt.conflictCount, + instanceHeadDigest: head.headDigest, + }); +} + +function validateTerminalBinding( + receipt: Readonly, + terminal: Readonly, + automationId: string, +): void { + if ( + receipt.automationId !== automationId || + receipt.applicationId !== terminal.plan.applicationId || + receipt.applicationPlanDigest !== terminal.plan.applicationPlanDigest + ) { + configurationError('terminal plan binding drifted'); + } +} + +export async function planLocalReconciliationAutomation( + value: unknown, + dependencies: LocalReconciliationAutomationPlanDependencies = {}, +): Promise> { + const command = normalizeLocalReconciliationAutomationPlanCommand(value); + const identity = currentIdentity(); + for (const [directory, label] of [ + [command.options.deploymentRoot, 'deploymentRoot'], + [command.options.applicationRoot, 'applicationRoot'], + [command.options.automationRoot, 'automationRoot'], + ] as const) { + validatePrivateDirectory(directory, identity.uid, label); + } + const terminal = await readLocalReconciliationApplicationTerminal( + command.options.applicationRoot, + command.request.applicationId, + identity.uid, + ); + const selected = automationPaths( + command.options.automationRoot, + command.request.automationId, + ); + if (fs.existsSync(selected.receipt)) { + validateDirectory( + selected.root, + identity.uid, + [0o700, 0o500], + 'automation plan root', + ); + validateDirectory( + selected.staging, + identity.uid, + [0o700, 0o500], + 'automation staging', + ); + validateCatalog(selected, false); + const receipt = readReceipt(selected.receipt, identity.uid, [0o600, 0o400]); + validateTerminalBinding(receipt, terminal, command.request.automationId); + if ( + receipt.applicationPlanDigest !== + command.request.expectedApplicationPlanDigest || + receipt.preparedHeadDigest !== command.request.expectedHeadDigest || + receipt.preparedAtMs !== command.request.preparedAtMs + ) { + configurationError('terminal plan is not an exact command replay'); + } + validatePlanFile(selected.plan, receipt, identity.uid, [0o600, 0o400]); + let head = readLocalCutoverInstanceHead( + command.options.deploymentRoot, + terminal.intent.instanceId, + identity.uid, + ); + const existing = head.state === 'reconciliation_automation_planned'; + if ( + !existing && + (head.state !== 'reconciliation_application_planned' || + head.headDigest !== command.request.expectedHeadDigest || + head.sourceRecordDigest !== terminal.plan.applicationPlanDigest) + ) { + configurationError('terminal plan lost instance head binding'); + } + if ( + existing && + head.sourceRecordDigest !== receipt.automationPlanDigest + ) { + configurationError('terminal plan head digest drifted'); + } + sealTerminal(selected, identity.uid); + dependencies.afterTerminalSealed?.(); + head = existing ? head : advanceHead(terminal, receipt, identity.uid); + dependencies.afterHeadAdvanced?.(); + return result( + command.operation, + existing ? 'existing' : 'prepared', + receipt, + head, + ); + } + validateApplicationBinding(command, terminal); + const authority = tableDisposition( + command, + terminal, + dependencies, + identity.uid, + ); + ensurePrivateDirectory(selected.root, identity.uid, 'automationPlanDirectory'); + ensurePrivateDirectory( + selected.staging, + identity.uid, + 'automationPlanStaging', + ); + validateCatalog(selected, false); + const receipt = publishPlan( + selected, + command, + terminal, + authority, + dependencies, + identity.uid, + ); + dependencies.afterPlanPublished?.(); + authority.confirmDecisionFileIdentity(); + publishExactFile( + selected.receipt, + receiptContents(receipt), + 0o600, + identity.uid, + 'reconciliation automation receipt', + ); + dependencies.afterReceiptPublished?.(); + sealTerminal(selected, identity.uid); + dependencies.afterTerminalSealed?.(); + const head = advanceHead(terminal, receipt, identity.uid); + dependencies.afterHeadAdvanced?.(); + return result(command.operation, 'prepared', receipt, head); +} + +export async function verifyLocalReconciliationAutomationPlan( + value: unknown, +): Promise> { + const command = normalizeLocalReconciliationAutomationVerifyCommand(value); + const identity = currentIdentity(); + for (const [directory, label] of [ + [command.options.deploymentRoot, 'deploymentRoot'], + [command.options.applicationRoot, 'applicationRoot'], + [command.options.automationRoot, 'automationRoot'], + ] as const) { + validatePrivateDirectory(directory, identity.uid, label); + } + const selected = automationPaths( + command.options.automationRoot, + command.request.automationId, + ); + validateDirectory(selected.root, identity.uid, [0o500], 'automation plan root'); + validateDirectory(selected.staging, identity.uid, [0o500], 'automation staging'); + validateCatalog(selected, true); + const receipt = readReceipt(selected.receipt, identity.uid, [0o400]); + if ( + receipt.automationPlanDigest !== + command.request.expectedAutomationPlanDigest + ) { + configurationError('expected automation plan digest drifted'); + } + const terminal = await readLocalReconciliationApplicationTerminal( + command.options.applicationRoot, + receipt.applicationId, + identity.uid, + ); + validateTerminalBinding(receipt, terminal, command.request.automationId); + validatePlanFile(selected.plan, receipt, identity.uid, [0o400]); + const head = readLocalCutoverInstanceHead( + command.options.deploymentRoot, + terminal.intent.instanceId, + identity.uid, + ); + if ( + head.state !== 'reconciliation_automation_planned' || + head.sourceRecordDigest !== receipt.automationPlanDigest + ) { + configurationError('automation plan is detached from the instance head'); + } + return result(command.operation, 'verified', receipt, head); +} + +export function planLocalReconciliationAutomationCommandFile( + filePath: string, + dependencies: LocalReconciliationAutomationPlanDependencies = {}, +): Promise> { + return planLocalReconciliationAutomation( + readPrivateLocalCommandFile(filePath), + dependencies, + ); +} + +export function verifyLocalReconciliationAutomationPlanCommandFile( + filePath: string, +): Promise> { + return verifyLocalReconciliationAutomationPlan( + readPrivateLocalCommandFile(filePath), + ); +} diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/automation/rowPlan.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/automation/rowPlan.ts new file mode 100644 index 00000000..e88bb3ad --- /dev/null +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/automation/rowPlan.ts @@ -0,0 +1,475 @@ +import { createHash } from 'node:crypto'; +import fs from 'node:fs'; +import type { DatabaseSync } from 'node:sqlite'; + +import { + visitLegacyCrontabAdoptionInspections, + type LegacyCrontabAdoptionInspection, +} from '@qinglong/local-admin/adoption-inspection'; + +import { LocalDeploymentConfigurationError } from '../../../foundation/error'; +import { cutoverDigest } from '../../../cutover/targetEvidence'; + +const HEADER_KIND = 'qinglong3-local-reconciliation-automation-plan-header'; +const ROW_KIND = 'qinglong3-local-reconciliation-automation-plan-row'; +const FOOTER_KIND = 'qinglong3-local-reconciliation-automation-plan-footer'; +const RECEIPT_SCHEMA = + 'qinglong3-local-reconciliation-automation-plan-receipt'; +const DIGEST_PATTERN = /^[0-9a-f]{64}$/; +const UUID_V4_PATTERN = + /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/; +const MAX_LINE_BYTES = 64 * 1024; +const HASH_BUFFER_BYTES = 64 * 1024; +export const MAX_EDGE_LOCAL_RECONCILIATION_AUTOMATION_PLAN_BYTES = + 8 * 1024 * 1024; +export const MAX_STANDALONE_LOCAL_RECONCILIATION_AUTOMATION_PLAN_BYTES = + 32 * 1024 * 1024; + +export type LocalReconciliationAutomationRowRequirement = + | 'review_adopt' + | 'review_skip_conflict' + | 'manual_required'; + +export interface LocalReconciliationAutomationPlanHeader { + readonly schemaVersion: 1; + readonly kind: typeof HEADER_KIND; + readonly automationId: string; + readonly applicationId: string; + readonly applicationPlanDigest: string; + readonly reviewDigest: string; + readonly reviewAuthorizationDigest: string; + readonly reviewDecisionSetDigest: string; + readonly reviewDecisionFileDigest: string; + readonly bundleDigest: string; + readonly bundleFingerprintDigest: string; + readonly profile: 'edge' | 'standalone'; + readonly projectId: string; + readonly legacyTimezone: string | null; + readonly tableDisposition: 'adopt_legacy' | 'retain_both'; + readonly preparedHeadDigest: string; + readonly preparedAtMs: number; + readonly headerDigest: string; +} + +export interface LocalReconciliationAutomationPlanRow { + readonly schemaVersion: 1; + readonly kind: typeof ROW_KIND; + readonly rowOrdinal: number; + readonly sourceDigest: string; + readonly classification: + | 'lossless' + | 'requires_shell_compatibility' + | 'requires_manual_action' + | 'malformed'; + readonly reasons: readonly string[]; + readonly proposedTaskId: string | null; + readonly enabled: boolean | null; + readonly triggerCount: number; + readonly candidateDigest: string | null; + readonly target: + | Readonly<{ state: 'absent' }> + | Readonly<{ + state: 'occupied'; + revision: number; + contentDigest: string; + }>; + readonly requirement: LocalReconciliationAutomationRowRequirement; + readonly rowPlanDigest: string; +} + +export interface LocalReconciliationAutomationPlanSummary { + readonly rowCount: number; + readonly eligibleCount: number; + readonly shellCompatibilityCount: number; + readonly manualCount: number; + readonly conflictCount: number; + readonly triggerCount: number; + readonly outcome: 'ready' | 'manual_required' | 'no_effect'; +} + +export interface LocalReconciliationAutomationPlanFooter + extends LocalReconciliationAutomationPlanSummary { + readonly schemaVersion: 1; + readonly kind: typeof FOOTER_KIND; + readonly automationId: string; + readonly legacyInventoryDigest: string; + readonly rowSetDigest: string; + readonly automationPlanDigest: string; +} + +export interface LocalReconciliationAutomationPlanReceipt + extends LocalReconciliationAutomationPlanSummary { + readonly schema: typeof RECEIPT_SCHEMA; + readonly schemaVersion: 1; + readonly state: 'reconciliation_automation_planned'; + readonly automationId: string; + readonly applicationId: string; + readonly applicationPlanDigest: string; + readonly preparedHeadDigest: string; + readonly legacyInventoryDigest: string; + readonly rowSetDigest: string; + readonly automationPlanDigest: string; + readonly planFileBytes: number; + readonly planFileDigest: string; + readonly preparedAtMs: number; + readonly receiptDigest: string; +} + +export interface WriteLocalReconciliationAutomationPlanOptions { + readonly descriptor: number; + readonly maxBytes: number; + readonly header: Omit; + readonly legacy: DatabaseSync; + readonly target: DatabaseSync; +} + +function configurationError(message: string, cause?: unknown): never { + throw new LocalDeploymentConfigurationError( + `reconciliation automation row plan ${message}`, + { cause }, + ); +} + +function exact( + value: unknown, + keys: readonly string[], + label: string, +): Record { + if (!value || typeof value !== 'object' || Array.isArray(value)) { + configurationError(`${label} must be an object`); + } + const record = value as Record; + const actual = Object.keys(record).sort(); + const expected = [...keys].sort(); + if ( + actual.length !== expected.length || + actual.some((key, index) => key !== expected[index]) + ) { + configurationError(`${label} shape is invalid`); + } + return record; +} + +function line(value: unknown): Buffer { + const bytes = Buffer.from(`${JSON.stringify(value)}\n`, 'utf8'); + if (bytes.byteLength < 3 || bytes.byteLength > MAX_LINE_BYTES + 1) { + bytes.fill(0); + configurationError('record exceeds its line bound'); + } + return bytes; +} + +function writeAll(descriptor: number, bytes: Buffer): void { + let offset = 0; + while (offset < bytes.byteLength) { + const written = fs.writeSync( + descriptor, + bytes, + offset, + bytes.byteLength - offset, + ); + if (written < 1) configurationError('write stalled'); + offset += written; + } +} + +function targetTask( + target: DatabaseSync, + projectId: string, + taskId: string, +): + | Readonly<{ state: 'absent' }> + | Readonly<{ state: 'occupied'; revision: number; contentDigest: string }> { + const row = target + .prepare( + `SELECT d."current_revision" AS revision, + r."content_digest" AS contentDigest + FROM "QingLong3TaskDefinitions" d + JOIN "QingLong3TaskDefinitionRevisions" r + ON r."project_id" = d."project_id" + AND r."task_id" = d."task_id" + AND r."revision" = d."current_revision" + WHERE d."project_id" = ? AND d."task_id" = ? LIMIT 1`, + ) + .get(projectId, taskId) as + | { readonly revision?: unknown; readonly contentDigest?: unknown } + | undefined; + if (!row) return Object.freeze({ state: 'absent' as const }); + if ( + !Number.isSafeInteger(row.revision) || + (row.revision as number) < 1 || + typeof row.contentDigest !== 'string' || + !DIGEST_PATTERN.test(row.contentDigest) + ) { + configurationError('target task projection drifted'); + } + return Object.freeze({ + state: 'occupied' as const, + revision: row.revision as number, + contentDigest: row.contentDigest, + }); +} + +function row( + inspection: Readonly, + target: DatabaseSync, + projectId: string, +): Readonly { + const diagnostic = inspection.diagnostic; + const selectedTarget = + diagnostic.taskId === null + ? Object.freeze({ state: 'absent' as const }) + : targetTask(target, projectId, diagnostic.taskId); + const candidateDigest = inspection.candidate + ? cutoverDigest({ + projectId, + sourceDigest: inspection.candidate.sourceDigest, + task: inspection.candidate.task, + triggers: inspection.candidate.triggers, + }) + : null; + const requirement: LocalReconciliationAutomationRowRequirement = + !inspection.candidate + ? 'manual_required' + : selectedTarget.state === 'occupied' + ? 'review_skip_conflict' + : 'review_adopt'; + const payload = Object.freeze({ + schemaVersion: 1 as const, + kind: ROW_KIND, + rowOrdinal: diagnostic.rowOrdinal, + sourceDigest: diagnostic.sourceDigest, + classification: diagnostic.classification, + reasons: diagnostic.reasons, + proposedTaskId: diagnostic.taskId, + enabled: diagnostic.enabled, + triggerCount: diagnostic.triggerCount, + candidateDigest, + target: selectedTarget, + requirement, + }); + return Object.freeze({ ...payload, rowPlanDigest: cutoverDigest(payload) }); +} + +export function writeLocalReconciliationAutomationPlan( + options: Readonly, +): Readonly<{ + header: Readonly; + footer: Readonly; + fileBytes: number; + fileDigest: string; +}> { + if ( + !Number.isSafeInteger(options.maxBytes) || + options.maxBytes < MAX_LINE_BYTES + ) { + configurationError('byte budget is invalid'); + } + const header = Object.freeze({ + ...options.header, + headerDigest: cutoverDigest(options.header), + }); + const fileHash = createHash('sha256'); + const rowHash = createHash('sha256').update( + 'qinglong3.local-reconciliation-automation-row-set.v1\0', + ); + let fileBytes = 0; + const append = (value: unknown, includeInRows = false): void => { + const bytes = line(value); + try { + if (fileBytes + bytes.byteLength > options.maxBytes) { + configurationError('exceeds profile byte budget'); + } + writeAll(options.descriptor, bytes); + fileHash.update(bytes); + if (includeInRows) rowHash.update(bytes); + fileBytes += bytes.byteLength; + } finally { + bytes.fill(0); + } + }; + append(header); + let eligibleCount = 0; + let shellCompatibilityCount = 0; + let manualCount = 0; + let conflictCount = 0; + let triggerCount = 0; + const inventory = visitLegacyCrontabAdoptionInspections( + options.legacy, + header.legacyTimezone, + (inspection) => { + const selected = row(inspection, options.target, header.projectId); + if (selected.requirement === 'review_adopt') eligibleCount += 1; + if (selected.classification === 'requires_shell_compatibility') { + shellCompatibilityCount += 1; + } + if (selected.requirement === 'manual_required') manualCount += 1; + if (selected.requirement === 'review_skip_conflict') conflictCount += 1; + triggerCount += selected.triggerCount; + append(selected, true); + }, + ); + const summary: LocalReconciliationAutomationPlanSummary = Object.freeze({ + rowCount: inventory.rowCount, + eligibleCount, + shellCompatibilityCount, + manualCount, + conflictCount, + triggerCount, + outcome: + inventory.rowCount === 0 + ? ('no_effect' as const) + : manualCount > 0 || conflictCount > 0 + ? ('manual_required' as const) + : ('ready' as const), + }); + const footerPayload = Object.freeze({ + schemaVersion: 1 as const, + kind: FOOTER_KIND, + automationId: header.automationId, + ...summary, + legacyInventoryDigest: inventory.inventoryDigest, + rowSetDigest: rowHash.digest('hex'), + }); + const footer = Object.freeze({ + ...footerPayload, + automationPlanDigest: cutoverDigest({ + headerDigest: header.headerDigest, + ...footerPayload, + }), + }); + append(footer); + return Object.freeze({ + header, + footer, + fileBytes, + fileDigest: fileHash.digest('hex'), + }); +} + +export function buildLocalReconciliationAutomationPlanReceipt( + header: Readonly, + footer: Readonly, + planFileBytes: number, + planFileDigest: string, +): Readonly { + const payload = Object.freeze({ + schema: RECEIPT_SCHEMA, + schemaVersion: 1 as const, + state: 'reconciliation_automation_planned' as const, + automationId: header.automationId, + applicationId: header.applicationId, + applicationPlanDigest: header.applicationPlanDigest, + preparedHeadDigest: header.preparedHeadDigest, + legacyInventoryDigest: footer.legacyInventoryDigest, + rowSetDigest: footer.rowSetDigest, + automationPlanDigest: footer.automationPlanDigest, + planFileBytes, + planFileDigest, + rowCount: footer.rowCount, + eligibleCount: footer.eligibleCount, + shellCompatibilityCount: footer.shellCompatibilityCount, + manualCount: footer.manualCount, + conflictCount: footer.conflictCount, + triggerCount: footer.triggerCount, + outcome: footer.outcome, + preparedAtMs: header.preparedAtMs, + }); + return Object.freeze({ ...payload, receiptDigest: cutoverDigest(payload) }); +} + +export function normalizeLocalReconciliationAutomationPlanReceipt( + value: unknown, +): Readonly { + const receipt = exact( + value, + [ + 'applicationId', + 'applicationPlanDigest', + 'automationId', + 'automationPlanDigest', + 'conflictCount', + 'eligibleCount', + 'legacyInventoryDigest', + 'manualCount', + 'outcome', + 'planFileBytes', + 'planFileDigest', + 'preparedHeadDigest', + 'preparedAtMs', + 'receiptDigest', + 'rowCount', + 'rowSetDigest', + 'schema', + 'schemaVersion', + 'shellCompatibilityCount', + 'state', + 'triggerCount', + ], + 'receipt', + ); + const { receiptDigest, ...payload } = receipt; + const counts = [ + receipt.rowCount, + receipt.eligibleCount, + receipt.shellCompatibilityCount, + receipt.manualCount, + receipt.conflictCount, + receipt.triggerCount, + receipt.planFileBytes, + receipt.preparedAtMs, + ]; + if ( + receipt.schema !== RECEIPT_SCHEMA || + receipt.schemaVersion !== 1 || + receipt.state !== 'reconciliation_automation_planned' || + typeof receipt.automationId !== 'string' || + !UUID_V4_PATTERN.test(receipt.automationId) || + typeof receipt.applicationId !== 'string' || + !UUID_V4_PATTERN.test(receipt.applicationId) || + !counts.every((count) => Number.isSafeInteger(count) && (count as number) >= 0) || + !['ready', 'manual_required', 'no_effect'].includes( + receipt.outcome as string, + ) || + [ + receipt.applicationPlanDigest, + receipt.preparedHeadDigest, + receipt.legacyInventoryDigest, + receipt.rowSetDigest, + receipt.automationPlanDigest, + receipt.planFileDigest, + receiptDigest, + ].some( + (candidate) => + typeof candidate !== 'string' || !DIGEST_PATTERN.test(candidate), + ) || + cutoverDigest(payload) !== receiptDigest + ) { + configurationError('receipt drifted'); + } + return Object.freeze( + receipt, + ) as unknown as Readonly; +} + +export function hashLocalReconciliationAutomationPlanFile( + descriptor: number, + expectedBytes: number, +): string { + const hash = createHash('sha256'); + const buffer = Buffer.allocUnsafe(HASH_BUFFER_BYTES); + let offset = 0; + while (offset < expectedBytes) { + const count = fs.readSync( + descriptor, + buffer, + 0, + Math.min(buffer.byteLength, expectedBytes - offset), + offset, + ); + if (count < 1) configurationError('plan file read stalled'); + hash.update(buffer.subarray(0, count)); + offset += count; + } + return hash.digest('hex'); +} diff --git a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/coordinator.ts b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/coordinator.ts index 8a9a2afd..efc260da 100644 --- a/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/coordinator.ts +++ b/packages/ql3-local-owner-cli/src/deployment/reconciliation/application/coordinator.ts @@ -81,6 +81,13 @@ interface TerminalApplication { readonly receipt: Readonly; } +export interface LocalReconciliationApplicationTerminal + extends TerminalApplication { + readonly intent: Readonly; + readonly review: Readonly; + readonly head: Readonly; +} + function configurationError(message: string, cause?: unknown): never { throw new LocalDeploymentConfigurationError(message, { cause }); } @@ -662,6 +669,61 @@ function readTerminalApplication( return terminal; } +/** + * Re-opens a sealed application terminal for a domain adapter. The adapter is + * given the exact signed review and current instance head rather than a loose + * plan file, so no downstream DML authority can be inferred from copied JSON. + */ +export async function readLocalReconciliationApplicationTerminal( + applicationRoot: string, + applicationId: string, + uid: number, +): Promise> { + const intent = readLocalReconciliationApplicationIntent( + applicationRoot, + applicationId, + ); + if ( + intent.command.options.applicationRoot !== applicationRoot || + intent.command.request.applicationId !== applicationId + ) { + configurationError('terminal reconciliation application path drifted'); + } + const selected = applicationPaths(applicationRoot, applicationId); + validateDirectory( + selected.root, + uid, + [0o500], + 'reconciliationApplicationDirectory', + ); + validateDirectory( + selected.staging, + uid, + [0o500], + 'reconciliationApplicationStaging', + ); + validateCatalog(selected, true); + const review = await readReview( + intent.command, + intent.command.request.reviewId, + uid, + ); + const terminal = readTerminalApplication( + selected, + intent, + review, + uid, + [0o400], + ); + const head = readLocalCutoverInstanceHead( + intent.command.options.deploymentRoot, + intent.instanceId, + uid, + ); + validateHeadIdentity(head, intent); + return Object.freeze({ intent, review, ...terminal, head }); +} + function sealFile(filePath: string, uid: number): void { let descriptor: number | undefined; try { diff --git a/packages/ql3-local-owner-cli/src/deployment/service-manager/serviceCutoverConsumer.ts b/packages/ql3-local-owner-cli/src/deployment/service-manager/serviceCutoverConsumer.ts index afd07f3e..45c7fc4a 100644 --- a/packages/ql3-local-owner-cli/src/deployment/service-manager/serviceCutoverConsumer.ts +++ b/packages/ql3-local-owner-cli/src/deployment/service-manager/serviceCutoverConsumer.ts @@ -755,7 +755,8 @@ function replayResult( head.state === 'reconciliation_review_prepared' || head.state === 'reconciliation_reviewed' || head.state === 'reconciliation_application_prepared' || - head.state === 'reconciliation_application_planned'); + head.state === 'reconciliation_application_planned' || + head.state === 'reconciliation_automation_planned'); if ( record.actionId !== intent.actionId || record.intentDigest !== intent.intentDigest || diff --git a/packages/ql3-local-owner-cli/test/reconciliationCapturePrepare.test.cjs b/packages/ql3-local-owner-cli/test/reconciliationCapturePrepare.test.cjs index a7033765..6384492b 100644 --- a/packages/ql3-local-owner-cli/test/reconciliationCapturePrepare.test.cjs +++ b/packages/ql3-local-owner-cli/test/reconciliationCapturePrepare.test.cjs @@ -14,10 +14,12 @@ const { commitLocalReconciliationReview, prepareLocalReconciliationCapture, prepareLocalReconciliationApplication, + planLocalReconciliationAutomation, prepareLocalReconciliationPlan, prepareLocalReconciliationReview, verifyLocalReconciliationCapture, verifyLocalReconciliationApplication, + verifyLocalReconciliationAutomationPlan, verifyLocalReconciliationPlan, verifyLocalReconciliationReview, writeLocalReconciliationReviewDiagnostics, @@ -25,6 +27,9 @@ const { const { normalizeLocalReconciliationCaptureManifest, } = require('../dist/deployment/reconciliation/bundle.js'); +const { + writeLocalReconciliationAutomationPlan, +} = require('../dist/deployment/reconciliation/application/automation/rowPlan.js'); const { createLocalDataDirectoryApplicationCommit, } = require('@qinglong/local-sqlite/data-directory-application-commit'); @@ -642,6 +647,75 @@ function planningDatabaseInitializer({ unknownTargetTable = false } = {}) { }; } +function automationDatabaseInitializer() { + return ({ legacySourcePath, recoveryPath, targetDatabasePath }) => { + const legacy = new DatabaseSync(legacySourcePath); + legacy.exec(` + CREATE TABLE "Crontabs" ( + id INTEGER PRIMARY KEY, + name TEXT, + command TEXT, + schedule TEXT, + saved INTEGER, + isSystem INTEGER, + isDisabled INTEGER, + isPinned INTEGER, + labels TEXT, + sub_id INTEGER, + extra_schedules TEXT, + task_before TEXT, + task_after TEXT, + log_name TEXT, + allow_multiple_instances INTEGER, + work_dir TEXT + ); + INSERT INTO "Crontabs" ( + id, name, command, schedule, saved, isSystem, isDisabled, isPinned + ) VALUES (1, 'nightly', 'task nightly.js', '0 0 * * *', 1, 0, 0, 0); + `); + legacy.close(); + fs.chmodSync(legacySourcePath, 0o600); + fs.copyFileSync(legacySourcePath, recoveryPath); + fs.chmodSync(recoveryPath, 0o600); + + const target = new DatabaseSync(targetDatabasePath); + target.exec(` + CREATE TABLE "QingLong3SchemaCapabilities" (id INTEGER PRIMARY KEY); + CREATE TABLE "QingLong3TaskDefinitions" ( + project_id TEXT NOT NULL, + task_id TEXT NOT NULL, + current_revision INTEGER NOT NULL, + PRIMARY KEY (project_id, task_id) + ); + CREATE TABLE "QingLong3TaskDefinitionRevisions" ( + project_id TEXT NOT NULL, + task_id TEXT NOT NULL, + revision INTEGER NOT NULL, + content_digest TEXT NOT NULL, + PRIMARY KEY (project_id, task_id, revision) + ); + `); + target.close(); + fs.chmodSync(targetDatabasePath, 0o600); + }; +} + +function mutateAutomationTarget({ targetDatabasePath }, occupied = false) { + const target = new DatabaseSync(targetDatabasePath); + if (occupied) { + target.exec(` + INSERT INTO "QingLong3TaskDefinitions" VALUES ('default', 'legacy-cron:1', 1); + INSERT INTO "QingLong3TaskDefinitionRevisions" VALUES ( + 'default', 'legacy-cron:1', 1, '${'a'.repeat(64)}' + ); + `); + } else { + target.exec('INSERT INTO "QingLong3SchemaCapabilities" (id) VALUES (1)'); + } + target.close(); + return Object.freeze({}); +} + function mutatePlanningTarget({ targetDatabasePath }) { const target = new DatabaseSync(targetDatabasePath); target.exec('INSERT INTO "QingLong3TaskDefinitions" (id) VALUES (1)'); @@ -2547,6 +2621,523 @@ test('application coordinator plans eight content-free domains and verifies with assert.equal(cli.stdout.includes('Crontabs'), false); }); +test('automation adapter builds a sealed row plan with bounded conflict evidence', async (t) => { + const state = await reviewedApplicationFixture(t, { + planId: '00000000-0000-4000-8000-000000000431', + reviewId: '00000000-0000-4000-8000-000000000432', + applicationId: '00000000-0000-4000-8000-000000000433', + reviewSuffix: 'automation-row-plan', + createDefaultSidecars: false, + initializeDatabases: automationDatabaseInitializer(), + mutateTarget(paths) { + return mutateAutomationTarget(paths); + }, + mutateDecisions(records) { + const selected = records.find( + (record) => + record.kind === 'qinglong3-local-reconciliation-review-decision' && + record.database === 'legacy' && + record.domain === 'automation' && + record.factKind === 'table' && + record.disposition === 'exclude_legacy', + ); + assert.ok(selected); + selected.disposition = 'adopt_legacy'; + selected.reason = 'prefer_legacy'; + }, + }); + const prepared = await prepareLocalReconciliationApplication( + state.prepareApplicationCommand, + ); + const application = await commitLocalReconciliationApplication( + applicationCommitCommand(state, prepared), + ); + const automationRoot = path.join(path.dirname(state.captureRoot), 'automation-root'); + fs.mkdirSync(automationRoot, { mode: 0o700 }); + const targetBefore = fs.readFileSync(state.targetDatabasePath); + const command = { + schemaVersion: 1, + operation: 'local.deployment.reconciliation.automation.plan', + options: { + deploymentRoot: state.deploymentRoot, + applicationRoot: state.applicationRoot, + automationRoot, + allowRootService: rootAcknowledgement(), + }, + request: { + automationId: '00000000-0000-4000-8000-000000000434', + applicationId: state.prepareApplicationCommand.request.applicationId, + expectedApplicationPlanDigest: application.applicationPlanDigest, + expectedHeadDigest: application.instanceHeadDigest, + decisionFilePath: state.reviewFile.filePath, + projectId: 'default', + legacyTimezone: 'Asia/Shanghai', + preparedAtMs: state.prepareApplicationCommand.request.preparedAtMs + 2, + }, + }; + const planned = await planLocalReconciliationAutomation(command); + assert.equal(planned.status, 'prepared'); + assert.equal(planned.state, 'reconciliation_automation_planned'); + assert.equal(planned.outcome, 'ready'); + assert.equal(planned.rowCount, 1); + assert.equal(planned.eligibleCount, 1); + assert.equal(planned.conflictCount, 0); + const automationDirectory = path.join( + automationRoot, + command.request.automationId, + ); + assert.deepEqual(fs.readdirSync(automationDirectory).sort(), [ + 'plan.ndjson', + 'receipt.json', + 'staging', + ]); + assert.equal(fs.statSync(automationDirectory).mode & 0o777, 0o500); + assert.equal( + fs.statSync(path.join(automationDirectory, 'staging')).mode & 0o777, + 0o500, + ); + for (const name of ['plan.ndjson', 'receipt.json']) { + assert.equal( + fs.statSync(path.join(automationDirectory, name)).mode & 0o777, + 0o400, + ); + } + const planText = fs.readFileSync( + path.join(automationDirectory, 'plan.ndjson'), + 'utf8', + ); + assert.equal(planText.includes('task nightly.js'), false); + assert.equal(planText.includes('nightly'), false); + assert.equal(planText.includes('legacy-cron:1'), true); + assert.equal(planText.includes('review-owner'), false); + assert.equal(planText.includes('"requirement":"review_adopt"'), true); + assert.equal(fs.readFileSync(state.targetDatabasePath).equals(targetBefore), true); + + const replay = await planLocalReconciliationAutomation(command); + assert.equal(replay.status, 'existing'); + assert.equal(replay.automationPlanDigest, planned.automationPlanDigest); + const verifyCommand = { + schemaVersion: 1, + operation: 'local.deployment.reconciliation.automation.verify', + options: command.options, + request: { + automationId: command.request.automationId, + expectedAutomationPlanDigest: planned.automationPlanDigest, + }, + }; + const verified = await verifyLocalReconciliationAutomationPlan(verifyCommand); + assert.equal(verified.status, 'verified'); + const head = readLocalCutoverInstanceHead( + state.deploymentRoot, + state.captureCommand.request.instanceId, + state.uid, + ); + assert.equal(head.state, 'reconciliation_automation_planned'); + assert.equal(head.sourceRecordDigest, planned.automationPlanDigest); + + const commandPath = path.join(state.deploymentRoot, 'automation-verify.json'); + fs.writeFileSync(commandPath, `${JSON.stringify(verifyCommand)}\n`, { + mode: 0o600, + }); + const cli = spawnSync( + process.execPath, + [ + path.join(__dirname, '../dist/deployment/localDeploymentCli.js'), + 'reconciliation-automation-verify', + '--command-file', + commandPath, + ], + { encoding: 'utf8' }, + ); + assert.equal(cli.status, 0, cli.stderr); + assert.equal(JSON.parse(cli.stdout).status, 'verified'); + assert.equal(cli.stdout.includes('legacy-cron:1'), false); +}); + +test('automation row plan fails closed to manual review on a target task collision', async (t) => { + const state = await reviewedApplicationFixture(t, { + planId: '00000000-0000-4000-8000-000000000435', + reviewId: '00000000-0000-4000-8000-000000000436', + applicationId: '00000000-0000-4000-8000-000000000437', + reviewSuffix: 'automation-row-conflict', + createDefaultSidecars: false, + initializeDatabases: automationDatabaseInitializer(), + mutateTarget(paths) { + return mutateAutomationTarget(paths, true); + }, + mutateDecisions(records) { + const selected = records.find( + (record) => + record.kind === 'qinglong3-local-reconciliation-review-decision' && + record.database === 'legacy' && + record.domain === 'automation' && + record.factKind === 'table' && + record.disposition === 'exclude_legacy', + ); + assert.ok(selected); + selected.disposition = 'retain_both'; + selected.reason = 'preserve_both'; + }, + }); + const prepared = await prepareLocalReconciliationApplication( + state.prepareApplicationCommand, + ); + const application = await commitLocalReconciliationApplication( + applicationCommitCommand(state, prepared), + ); + const automationRoot = path.join( + path.dirname(state.captureRoot), + 'automation-conflict-root', + ); + fs.mkdirSync(automationRoot, { mode: 0o700 }); + const planned = await planLocalReconciliationAutomation({ + schemaVersion: 1, + operation: 'local.deployment.reconciliation.automation.plan', + options: { + deploymentRoot: state.deploymentRoot, + applicationRoot: state.applicationRoot, + automationRoot, + allowRootService: rootAcknowledgement(), + }, + request: { + automationId: '00000000-0000-4000-8000-000000000438', + applicationId: state.prepareApplicationCommand.request.applicationId, + expectedApplicationPlanDigest: application.applicationPlanDigest, + expectedHeadDigest: application.instanceHeadDigest, + decisionFilePath: state.reviewFile.filePath, + projectId: 'default', + legacyTimezone: 'Asia/Shanghai', + preparedAtMs: state.prepareApplicationCommand.request.preparedAtMs + 2, + }, + }); + assert.equal(planned.outcome, 'manual_required'); + assert.equal(planned.eligibleCount, 0); + assert.equal(planned.conflictCount, 1); + const planText = fs.readFileSync( + path.join( + automationRoot, + '00000000-0000-4000-8000-000000000438', + 'plan.ndjson', + ), + 'utf8', + ); + assert.equal(planText.includes('"state":"occupied"'), true); + assert.equal(planText.includes('"requirement":"review_skip_conflict"'), true); + assert.equal(planText.includes('task nightly.js'), false); +}); + +test('automation row planner makes an empty table no-effect and a missing timezone manual', (t) => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'ql3-automation-row-unit-')); + t.after(() => removeFixtureRoot(root)); + const legacySourcePath = path.join(root, 'legacy.sqlite'); + const recoveryPath = path.join(root, 'recovery.sqlite'); + const targetDatabasePath = path.join(root, 'target.sqlite'); + automationDatabaseInitializer()({ + legacySourcePath, + recoveryPath, + targetDatabasePath, + }); + const legacy = new DatabaseSync(legacySourcePath); + const target = new DatabaseSync(targetDatabasePath); + t.after(() => { + legacy.close(); + target.close(); + }); + legacy.exec('DELETE FROM "Crontabs"'); + const header = { + schemaVersion: 1, + kind: 'qinglong3-local-reconciliation-automation-plan-header', + automationId: '00000000-0000-4000-8000-000000000439', + applicationId: '00000000-0000-4000-8000-00000000043a', + applicationPlanDigest: '1'.repeat(64), + reviewDigest: '2'.repeat(64), + reviewAuthorizationDigest: '3'.repeat(64), + reviewDecisionSetDigest: '4'.repeat(64), + reviewDecisionFileDigest: '5'.repeat(64), + bundleDigest: '6'.repeat(64), + bundleFingerprintDigest: '7'.repeat(64), + profile: 'edge', + projectId: 'default', + legacyTimezone: null, + tableDisposition: 'adopt_legacy', + preparedHeadDigest: '8'.repeat(64), + preparedAtMs: 1, + }; + const emptyPath = path.join(root, 'empty.ndjson'); + const emptyDescriptor = fs.openSync(emptyPath, 'wx', 0o600); + let empty; + try { + empty = writeLocalReconciliationAutomationPlan({ + descriptor: emptyDescriptor, + maxBytes: 64 * 1024, + header, + legacy, + target, + }); + } finally { + fs.closeSync(emptyDescriptor); + } + assert.equal(empty.footer.outcome, 'no_effect'); + assert.equal(empty.footer.rowCount, 0); + + legacy.exec(` + INSERT INTO "Crontabs" ( + id, name, command, schedule, saved, isSystem, isDisabled, isPinned + ) VALUES (2, 'timezone-required', 'task timezone.js', '0 0 * * *', 1, 0, 0, 0) + `); + const manualPath = path.join(root, 'manual.ndjson'); + const manualDescriptor = fs.openSync(manualPath, 'wx', 0o600); + let manual; + try { + manual = writeLocalReconciliationAutomationPlan({ + descriptor: manualDescriptor, + maxBytes: 64 * 1024, + header: { ...header, automationId: '00000000-0000-4000-8000-00000000043b' }, + legacy, + target, + }); + } finally { + fs.closeSync(manualDescriptor); + } + assert.equal(manual.footer.outcome, 'manual_required'); + assert.equal(manual.footer.manualCount, 1); + const manualText = fs.readFileSync(manualPath, 'utf8'); + assert.equal(manualText.includes('timezone_required'), true); + assert.equal(manualText.includes('task timezone.js'), false); +}); + +test('automation row planner fails closed at its byte budget', (t) => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'ql3-automation-budget-')); + t.after(() => removeFixtureRoot(root)); + const legacySourcePath = path.join(root, 'legacy.sqlite'); + const recoveryPath = path.join(root, 'recovery.sqlite'); + const targetDatabasePath = path.join(root, 'target.sqlite'); + automationDatabaseInitializer()({ + legacySourcePath, + recoveryPath, + targetDatabasePath, + }); + const legacy = new DatabaseSync(legacySourcePath); + const target = new DatabaseSync(targetDatabasePath); + t.after(() => { + legacy.close(); + target.close(); + }); + const insert = legacy.prepare(` + INSERT INTO "Crontabs" ( + id, name, command, schedule, saved, isSystem, isDisabled, isPinned + ) VALUES (?, ?, ?, '0 0 * * *', 1, 0, 0, 0) + `); + legacy.exec('BEGIN IMMEDIATE'); + try { + for (let id = 2; id <= 400; id += 1) { + insert.run(id, `task-${id}`, `task workload-${id}.js`); + } + legacy.exec('COMMIT'); + } catch (error) { + legacy.exec('ROLLBACK'); + throw error; + } + const outputPath = path.join(root, 'bounded.ndjson'); + const descriptor = fs.openSync(outputPath, 'wx', 0o600); + try { + assert.throws( + () => + writeLocalReconciliationAutomationPlan({ + descriptor, + maxBytes: 64 * 1024, + header: { + schemaVersion: 1, + kind: 'qinglong3-local-reconciliation-automation-plan-header', + automationId: '00000000-0000-4000-8000-00000000043c', + applicationId: '00000000-0000-4000-8000-00000000043d', + applicationPlanDigest: '1'.repeat(64), + reviewDigest: '2'.repeat(64), + reviewAuthorizationDigest: '3'.repeat(64), + reviewDecisionSetDigest: '4'.repeat(64), + reviewDecisionFileDigest: '5'.repeat(64), + bundleDigest: '6'.repeat(64), + bundleFingerprintDigest: '7'.repeat(64), + profile: 'edge', + projectId: 'default', + legacyTimezone: 'UTC', + tableDisposition: 'adopt_legacy', + preparedHeadDigest: '8'.repeat(64), + preparedAtMs: 1, + }, + legacy, + target, + }), + /exceeds profile byte budget/, + ); + } finally { + fs.closeSync(descriptor); + } + assert.equal(fs.statSync(outputPath).size <= 64 * 1024, true); +}); + +test('automation row planning replays every publication boundary and rejects drift', async (t) => { + const windows = [ + ['plan', '000000000441', '000000000442', '000000000443', '000000000444'], + ['receipt', '000000000445', '000000000446', '000000000447', '000000000448'], + ['seal', '000000000449', '00000000044a', '00000000044b', '00000000044c'], + ['head', '00000000044d', '00000000044e', '00000000044f', '000000000450'], + ]; + for (const [window, planTail, reviewTail, applicationTail, automationTail] of windows) { + const state = await reviewedApplicationFixture(t, { + planId: `00000000-0000-4000-8000-${planTail}`, + reviewId: `00000000-0000-4000-8000-${reviewTail}`, + applicationId: `00000000-0000-4000-8000-${applicationTail}`, + reviewSuffix: `automation-${window}-replay`, + createDefaultSidecars: false, + initializeDatabases: automationDatabaseInitializer(), + mutateTarget(paths) { + return mutateAutomationTarget(paths); + }, + mutateDecisions(records) { + const selected = records.find( + (record) => + record.kind === 'qinglong3-local-reconciliation-review-decision' && + record.database === 'legacy' && + record.domain === 'automation' && + record.factKind === 'table' && + record.disposition === 'exclude_legacy', + ); + assert.ok(selected); + selected.disposition = 'adopt_legacy'; + selected.reason = 'prefer_legacy'; + }, + }); + const prepared = await prepareLocalReconciliationApplication( + state.prepareApplicationCommand, + ); + const application = await commitLocalReconciliationApplication( + applicationCommitCommand(state, prepared), + ); + const automationRoot = path.join( + path.dirname(state.captureRoot), + `automation-${window}-root`, + ); + fs.mkdirSync(automationRoot, { mode: 0o700 }); + const command = { + schemaVersion: 1, + operation: 'local.deployment.reconciliation.automation.plan', + options: { + deploymentRoot: state.deploymentRoot, + applicationRoot: state.applicationRoot, + automationRoot, + allowRootService: rootAcknowledgement(), + }, + request: { + automationId: `00000000-0000-4000-8000-${automationTail}`, + applicationId: state.prepareApplicationCommand.request.applicationId, + expectedApplicationPlanDigest: application.applicationPlanDigest, + expectedHeadDigest: application.instanceHeadDigest, + decisionFilePath: state.reviewFile.filePath, + projectId: 'default', + legacyTimezone: 'Asia/Shanghai', + preparedAtMs: state.prepareApplicationCommand.request.preparedAtMs + 2, + }, + }; + const callback = + window === 'plan' + ? 'afterPlanPublished' + : window === 'receipt' + ? 'afterReceiptPublished' + : window === 'seal' + ? 'afterTerminalSealed' + : 'afterHeadAdvanced'; + await assert.rejects( + planLocalReconciliationAutomation(command, { + [callback]() { + throw new Error(`automation ${window} response loss`); + }, + }), + new RegExp(`automation ${window} response loss`), + ); + const replay = await planLocalReconciliationAutomation(command); + assert.equal(replay.state, 'reconciliation_automation_planned'); + if (window === 'head') assert.equal(replay.status, 'existing'); + } + + const drift = await reviewedApplicationFixture(t, { + planId: '00000000-0000-4000-8000-000000000451', + reviewId: '00000000-0000-4000-8000-000000000452', + applicationId: '00000000-0000-4000-8000-000000000453', + reviewSuffix: 'automation-drift', + createDefaultSidecars: false, + initializeDatabases: automationDatabaseInitializer(), + mutateTarget(paths) { + return mutateAutomationTarget(paths); + }, + mutateDecisions(records) { + const selected = records.find( + (record) => + record.kind === 'qinglong3-local-reconciliation-review-decision' && + record.database === 'legacy' && + record.domain === 'automation' && + record.factKind === 'table' && + record.disposition === 'exclude_legacy', + ); + assert.ok(selected); + selected.disposition = 'adopt_legacy'; + selected.reason = 'prefer_legacy'; + }, + }); + const prepared = await prepareLocalReconciliationApplication( + drift.prepareApplicationCommand, + ); + const application = await commitLocalReconciliationApplication( + applicationCommitCommand(drift, prepared), + ); + const automationRoot = path.join( + path.dirname(drift.captureRoot), + 'automation-drift-root', + ); + fs.mkdirSync(automationRoot, { mode: 0o700 }); + const command = { + schemaVersion: 1, + operation: 'local.deployment.reconciliation.automation.plan', + options: { + deploymentRoot: drift.deploymentRoot, + applicationRoot: drift.applicationRoot, + automationRoot, + allowRootService: rootAcknowledgement(), + }, + request: { + automationId: '00000000-0000-4000-8000-000000000454', + applicationId: drift.prepareApplicationCommand.request.applicationId, + expectedApplicationPlanDigest: application.applicationPlanDigest, + expectedHeadDigest: application.instanceHeadDigest, + decisionFilePath: drift.reviewFile.filePath, + projectId: 'default', + legacyTimezone: 'Asia/Shanghai', + preparedAtMs: drift.prepareApplicationCommand.request.preparedAtMs + 2, + }, + }; + const planned = await planLocalReconciliationAutomation(command); + const planPath = path.join( + automationRoot, + command.request.automationId, + 'plan.ndjson', + ); + fs.chmodSync(planPath, 0o600); + fs.appendFileSync(planPath, '{}\n'); + fs.chmodSync(planPath, 0o400); + await assert.rejects( + verifyLocalReconciliationAutomationPlan({ + schemaVersion: 1, + operation: 'local.deployment.reconciliation.automation.verify', + options: command.options, + request: { + automationId: command.request.automationId, + expectedAutomationPlanDigest: planned.automationPlanDigest, + }, + }), + /plan file identity is invalid|plan file content drifted/, + ); +}); + test('application coordinator resumes every publication window and fences competitors', async (t) => { const prepareCrash = await reviewedApplicationFixture(t, { planId: '00000000-0000-4000-8000-000000000410', diff --git a/scripts/ql3-cluster-dependency-audit.cjs b/scripts/ql3-cluster-dependency-audit.cjs index 1cc8ef68..30478607 100644 --- a/scripts/ql3-cluster-dependency-audit.cjs +++ b/scripts/ql3-cluster-dependency-audit.cjs @@ -2018,6 +2018,11 @@ function auditSourceImports(root, packagePath, findings) { '@qinglong/local-sqlite/authentication-read', ].includes(specifier) ) && + !( + path.relative(packageDirectory, filePath) === + 'src/deployment/reconciliation/application/automation/rowPlan.ts' && + specifier === '@qinglong/local-admin/adoption-inspection' + ) && !( [ 'src/deployment/compose/composeApply.ts', diff --git a/test/back/ql3PackageBoundaryAudit.test.cjs b/test/back/ql3PackageBoundaryAudit.test.cjs index e7979048..7c220cbb 100644 --- a/test/back/ql3PackageBoundaryAudit.test.cjs +++ b/test/back/ql3PackageBoundaryAudit.test.cjs @@ -97,10 +97,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( rootSourceFileRoles: localAdmin.rootSourceFileRoles, }, { - sourceFiles: 45, + sourceFiles: 46, rootSourceFiles: 1, rootSourceLines: 9, - nestedSourceFiles: 44, + nestedSourceFiles: 45, rootSourceFileRoles: { 'runtime.ts': 'public_export' }, }, ); @@ -207,10 +207,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( rootSourceFileRoles: localOwnerCli.rootSourceFileRoles, }, { - sourceFiles: 158, + sourceFiles: 161, rootSourceFiles: 1, rootSourceLines: 50, - nestedSourceFiles: 157, + nestedSourceFiles: 160, rootSourceFileRoles: { 'cli.ts': 'binary_entry' }, }, );