feat(ql3): plan sealed reconciliation bundles

This commit is contained in:
whyour
2026-08-21 18:06:27 +08:00
parent 4012f54a50
commit 46a7dc7496
16 changed files with 3078 additions and 43 deletions
+16 -13
View File
@@ -11,17 +11,20 @@
最新增量证据(2026-08-21):
- D-390/ADR-0483契约已冻结,待实现):`reconciliation_captured` 只证明原始字节可恢复,不授予自动回灌。下一切片先把 capture
manifest/receipt 升为 v2:逻辑名保持内容无关,payload 改用 SQLite 可识别的固定 `target.sqlite* / legacy.sqlite* /
recovery.sqlite` 名称,terminal asset/directory 收敛为 `0400/0500`,并把 activation source/target baseline SHA-256 纳入 lineage。
main-only 使用 immutable readonly;只有 WAL+SHM 完整配对且无 journal 时使用普通 readonly,打开前后必须重验全部 asset;hot
journal、不完整 sidecar 或漂移不执行 SQLite recovery,直接 `manual_required`。既有 Local Owner 增加独立
`reconciliation.plan.prepare|commit|verify`,以 `reconciliation_captured → reconciliation_plan_prepared → reconciliation_planned`
CAS fence 发布固定八领域的 bounded count/digest/disposition plan。plan 不保存 row value、command、Secret、credential、日志或业务
标识,不产生 `import_ready/rollback_ready/legacy_ready`unknown schema、不可逆 Run/history、Secret custody、target-only Package/AI
未映射 Legacy facts 必须保守。实现继续进入 `deployment/reconciliation/sealed-bundle|planning/`,单 handle、64 KiB builder、Edge/
Standalone 2/8 MiB cache,不新增 package/dependency/binary/daemon 或 Local SQLite mutation authority import。本条在 Linux/Docker
readonly hash-stability、crash replay、完整 package/backend、架构/release 与十四档 artifact 门完成前保持 Proposed。
- D-390/ADR-0483已接受):既有 Local Owner 已实现密封 capture 的严格只读消费与独立
`reconciliation.plan.prepare|commit|verify`。capture v2 使用 SQLite 可识别的固定 `target.sqlite* / legacy.sqlite* /
recovery.sqlite` 物理名和 `0400/0500` terminal sealmain-only 走 immutable readonlyWAL+SHM 完整配对走普通 readonlyhot
journal、不完整 sidecar 或任一 stat/hash/mode 漂移在 SQLite open 前稳定 `manual_required`。instance head 以
`reconciliation_captured → reconciliation_plan_prepared → reconciliation_planned` CAS fence 发布固定八领域的 bounded
count/digest/disposition planplan 不保存 row value、command、Secret、credential、日志、表名、路径或业务标识,也不产生
`import_ready/rollback_ready/legacy_ready`。实现内聚在 `deployment/reconciliation/sealed-bundle|planning/`,单 SQLite handle、
64 KiB hash/plan 上限、Edge/Standalone 2/8 MiB cache,不新增 package、dependency 或常驻对象。真实 Linux/Docker main-only
WAL+SHM readonly/hash-stability `2/2`,聚焦 `24 total / 22 pass / 2 conditional skip / 0 fail`Local Owner
`246 total / 239 pass / 7 conditional skip / 0 fail`tracked backend `1540 total / 1538 pass / 2 conditional skip / 0 fail`
18-package clean build/逐包测试、八项架构/发布审计和十四档 artifact audit 全通过。workspace 仍为 18 packagesLocal Owner
`146 source / 145 nested / 1 root binary entry`;基础 Edge/Standalone closure 仍为 319 files、58 modules,不含一次性 plan authority。
下一切片 D-391 应定义消费 exact plan digest 的私有逐对象诊断、人工裁决与审批协议;它仍不能借 plan 获得自动 import 或 rollback
authority。
- D-389/ADR-0482(已接受):target stopped 后的 `reconciliation_required` 不能直接逆迁移或覆盖 2.x source;既有 Local Owner
已实现独立 `reconciliation.capture.prepare|commit|verify`,只允许 exact stopped reconciliation head,以 instance CAS 建立唯一
capture fence,并把 target main/sidecars、Legacy source main/sidecars、activation recovery 与内容无关 lineage 以固定 64 KiB
@@ -36,8 +39,8 @@
`141 source / 140 nested / 1 root binary entry`;基础 Edge/Standalone artifact 仍为 `2,611,978 / 2,612,056` bytes、319 files、
58 modules,证明一次性 capture authority 未进入低配常驻闭包。Application+AI 为 `4,529,710 / 4,529,842` bytesMCP 为
`7,337,910 / 7,338,018` bytes,均 compatible。独立 PostgreSQL 18.6 arm64 HA Docker 基线继续以 timeline `1 → 2`、146
gates 和无 finding 的 evidence audit 通过,但不把本机 capture authority 带入 Cluster。后续 D-390 应定义消费 exact
bundle/head 的数据域 diff、冲突裁决与受审 plan,不能把 capture completion 当作 reconciliation completion。
gates 和无 finding 的 evidence audit 通过,但不把本机 capture authority 带入 Cluster。D-390 已完成 exact bundle/head 的只读
数据域 plancapture completion 与 plan completion 仍都不等于 reconciliation completion。
- D-388/ADR-0481(已接受):D-387 committed data receipt 只作为启动前置事实,不获得 activation/rollback authority。
adopted-only Application v4 在 signal、SQLite、Secret、Plugin、AI 前以 no-follow stable descriptor 验证
`commitPath/expectedCommitDigest/expectedReceiptDigest`;独立 `local.deployment.adopted.prepare|verify` 已覆盖 systemd、OpenRC、
@@ -1,6 +1,6 @@
# ADR-0483:密封 Reconciliation Bundle 的有界数据域计划
- 状态:ProposedD-390 契约冻结
- 状态:AcceptedD-390 已实现
- 日期:2026-08-21
- 关联 RFCQL-RFC-0001 D-05、D-06、D-17、D-64、D-87、D-184、D-259、D-383、D-389、D-390
- 关联 ADRADR-0064、ADR-0094、ADR-0095、ADR-0194、ADR-0201、ADR-0314、ADR-0315、ADR-0482
@@ -143,6 +143,29 @@ package、production dependency、binary、daemon、timer、watcher、listener
7. Edge 固定 2 MiB cache、Standalone 8 MiB cache;完整 Local Owner/backend/package、架构、release、十四档 artifact 与真实 Docker
readonly rehearsal 通过,基础 Edge closure 不增长。
## 实现与验证证据
D-390 已在既有 `@qinglong/local-owner-cli/src/deployment/reconciliation/` 内实现。`sealed-bundle/reader.ts` 只接受经 terminal
validator 验证的密封 capturemain-only 走 immutable readonlyWAL+SHM 完整配对走普通 readonlyhot journal、sidecar 不配对或
任一 stat/hash/mode 漂移均在 SQLite open 前失败关闭。`planning/` 以固定 contract、inventory、prepare/commit/verify 将 instance head
`reconciliation_captured` CAS 推进到 `reconciliation_plan_prepared`、再推进到 `reconciliation_planned`。plan 只包含固定八领域的
有界计数、digest、disposition 和保守 outcome,不保存表名、路径、row value、Secret、credential、命令或日志,也不产生 import authority。
实现没有新增 workspace package、production dependency、binary、daemon、listener、timer、watcher 或网络访问。workspace 仍为 18
packages`singleSourcePackages=[]``shallowSourcePackages=[]`Local Owner 为 `146 source / 145 nested / 1 root binary entry`,新增的
5 个源文件全部进入 `deployment/reconciliation/sealed-bundle|planning/`,没有回到 `src/` 根平铺。Edge/Standalone cache 固定为
2/8 MiBhash buffer 与 plan 上限均为 64 KiBschema/table 上限为 4,096/512。
验收结果:reconciliation 聚焦套件 `24 total / 22 pass / 2 conditional Docker skip / 0 fail`;真实 Linux/Docker main-only 与
WAL+SHM readonly/hash-stability rehearsal `2/2`;完整 Local Owner `246 total / 239 pass / 7 conditional skip / 0 fail`tracked
backend `1540 total / 1538 pass / 2 conditional skip / 0 fail`18-package clean build/逐包测试通过。Edge import、Cluster dependency、
package boundary、service-manager bridge、Local image、image release、release version 与 deployment-lock surface 审计均 compatible。
十四档 artifact audit 均 compatible。基础 Edge/Standalone 仍为 `2,611,978 / 2,612,056` bytes、319 files、58 modulesAdopted
仍为 `2,831,713 / 2,831,836` bytes、339 files、59 modulesApplication+AI 为 `4,529,710 / 4,529,842` bytes、516 files、
144 modulesMCP 为 `7,337,910 / 7,338,018` bytes、805 files、228 modules。一次性 plan authority 未进入基础常驻闭包,也没有被
Cluster/PostgreSQL 消费。
## 未包含
- 逐对象明细分页、人工冲突选择和审批;
+1 -1
View File
@@ -486,7 +486,7 @@
| [ADR-0480](./ADR-0480-authenticated-atomic-legacy-data-application.md) | 受认证、原子且可恢复的 Legacy Data Application | Accepted |
| [ADR-0481](./ADR-0481-committed-legacy-data-deployment-lineage.md) | Committed Legacy Data Receipt 的本机部署 Lineage | Accepted |
| [ADR-0482](./ADR-0482-stopped-local-sqlite-reconciliation-recovery-bundle.md) | 停止态 Local SQLite Reconciliation Recovery Bundle | Accepted |
| [ADR-0483](./ADR-0483-sealed-reconciliation-bundle-domain-plan.md) | 密封 Reconciliation Bundle 的有界数据域计划 | ProposedD-390 契约冻结) |
| [ADR-0483](./ADR-0483-sealed-reconciliation-bundle-domain-plan.md) | 密封 Reconciliation Bundle 的有界数据域计划 | Accepted |
## 规则
@@ -25,6 +25,8 @@ export type LocalCutoverInstanceHeadState =
| 'target_stopped'
| 'reconciliation_capture_prepared'
| 'reconciliation_captured'
| 'reconciliation_plan_prepared'
| 'reconciliation_planned'
| 'rollback_prepared'
| 'legacy_restart_requested'
| 'legacy_running'
@@ -158,6 +160,8 @@ function parseHead(value: unknown): Readonly<LocalCutoverInstanceHead> {
head.state !== 'target_stopped' &&
head.state !== 'reconciliation_capture_prepared' &&
head.state !== 'reconciliation_captured' &&
head.state !== 'reconciliation_plan_prepared' &&
head.state !== 'reconciliation_planned' &&
head.state !== 'rollback_prepared' &&
head.state !== 'legacy_restart_requested' &&
head.state !== 'legacy_running' &&
@@ -326,6 +330,8 @@ export function advanceLocalCutoverInstanceHead(
| 'target_stopped'
| 'reconciliation_capture_prepared'
| 'reconciliation_captured'
| 'reconciliation_plan_prepared'
| 'reconciliation_planned'
| 'rollback_prepared'
| 'legacy_restart_requested'
| 'legacy_running'
@@ -366,6 +372,8 @@ export function advanceLocalCutoverInstanceHead(
(current.state === 'rollback_prepared' ||
current.state === 'reconciliation_capture_prepared' ||
current.state === 'reconciliation_captured' ||
current.state === 'reconciliation_plan_prepared' ||
current.state === 'reconciliation_planned' ||
current.state === 'legacy_restart_requested' ||
current.state === 'legacy_running' ||
current.state === 'legacy_ready')
@@ -385,6 +393,10 @@ export function advanceLocalCutoverInstanceHead(
current.state === 'target_stopped') ||
(state === 'reconciliation_captured' &&
current.state === 'reconciliation_capture_prepared') ||
(state === 'reconciliation_plan_prepared' &&
current.state === 'reconciliation_captured') ||
(state === 'reconciliation_planned' &&
current.state === 'reconciliation_plan_prepared') ||
(state === 'rollback_prepared' && current.state === 'target_stopped') ||
(state === 'legacy_restart_requested' &&
current.state === 'rollback_prepared') ||
@@ -103,6 +103,23 @@ import {
verifyLocalReconciliationCapture,
verifyLocalReconciliationCaptureCommandFile,
} from './reconciliation/bundle';
import {
commitLocalReconciliationPlan,
commitLocalReconciliationPlanCommandFile,
prepareLocalReconciliationPlan,
prepareLocalReconciliationPlanCommandFile,
verifyLocalReconciliationPlan,
verifyLocalReconciliationPlanCommandFile,
} from './reconciliation/planning/preparation';
export {
commitLocalReconciliationPlan,
commitLocalReconciliationPlanCommandFile,
prepareLocalReconciliationPlan,
prepareLocalReconciliationPlanCommandFile,
verifyLocalReconciliationPlan,
verifyLocalReconciliationPlanCommandFile,
};
export {
LocalDeploymentConfigurationError,
@@ -143,6 +160,34 @@ export {
type LocalReconciliationCaptureVerifyCommand,
type LocalReconciliationStoppedAuthority,
} from './reconciliation/contract';
export {
LOCAL_RECONCILIATION_PLAN_DOMAINS,
normalizeLocalReconciliationPlanCommitCommand,
normalizeLocalReconciliationPlanPrepareCommand,
normalizeLocalReconciliationPlanVerifyCommand,
type LocalReconciliationPlanCommitCommand,
type LocalReconciliationPlanDisposition,
type LocalReconciliationPlanDomain,
type LocalReconciliationPlanPrepareCommand,
type LocalReconciliationPlanPrepareResult,
type LocalReconciliationPlanTerminalResult,
type LocalReconciliationPlanVerifyCommand,
} from './reconciliation/planning/contract';
export {
localReconciliationPlanDirectory,
normalizeLocalReconciliationPlanIntent,
readLocalReconciliationPlanIntent,
type LocalReconciliationPlanDependencies,
type LocalReconciliationPlanIntent,
} from './reconciliation/planning/preparation';
export {
normalizeLocalReconciliationPlan,
normalizeLocalReconciliationPlanReceipt,
type LocalReconciliationPlan,
type LocalReconciliationPlanDatabaseSummary,
type LocalReconciliationPlanDomainSummary,
type LocalReconciliationPlanReceipt,
} from './reconciliation/planning/plan';
export {
localReconciliationCaptureDirectory,
localReconciliationCaptureIntentPath,
@@ -14,8 +14,11 @@ import {
prepareLocalServiceManagerIntentCommandFile,
prepareLocalServiceManagerLegacyRollbackCommandFile,
prepareLocalReconciliationCaptureCommandFile,
prepareLocalReconciliationPlanCommandFile,
commitLocalReconciliationCaptureCommandFile,
commitLocalReconciliationPlanCommandFile,
verifyLocalReconciliationCaptureCommandFile,
verifyLocalReconciliationPlanCommandFile,
prepareLocalDeploymentCommandFile,
proveLocalDeploymentLegacyReadinessCommandFile,
restoreLocalDeploymentComposeCommitCommandFile,
@@ -30,7 +33,7 @@ import {
} from './localDeployment';
const USAGE =
'Usage: ql3-local-deploy <prepare|adopted-prepare|adopted-verify|status|service-intent-prepare|service-outcome-consume|service-cutover-consume|service-legacy-rollback-prepare|service-legacy-rollback-authorize|service-legacy-rollback-consume|cutover-legacy-stop|cutover-target-start|cutover-target-restart|cutover-target-stop|cutover-legacy-rollback-prepare|cutover-legacy-rollback-commit|cutover-legacy-readiness-probe|cutover-manual-diagnose|cutover-manual-resolution-prepare|cutover-manual-resolution-commit|reconciliation-capture-prepare|reconciliation-capture-commit|reconciliation-capture-verify|compose-revision|compose-preflight|compose-apply|compose-restore-prepare|compose-restore-commit|compose-evidence-collect-prepare|compose-evidence-collect-commit> --command-file /absolute/private-command.json';
'Usage: ql3-local-deploy <prepare|adopted-prepare|adopted-verify|status|service-intent-prepare|service-outcome-consume|service-cutover-consume|service-legacy-rollback-prepare|service-legacy-rollback-authorize|service-legacy-rollback-consume|cutover-legacy-stop|cutover-target-start|cutover-target-restart|cutover-target-stop|cutover-legacy-rollback-prepare|cutover-legacy-rollback-commit|cutover-legacy-readiness-probe|cutover-manual-diagnose|cutover-manual-resolution-prepare|cutover-manual-resolution-commit|reconciliation-capture-prepare|reconciliation-capture-commit|reconciliation-capture-verify|reconciliation-plan-prepare|reconciliation-plan-commit|reconciliation-plan-verify|compose-revision|compose-preflight|compose-apply|compose-restore-prepare|compose-restore-commit|compose-evidence-collect-prepare|compose-evidence-collect-commit> --command-file /absolute/private-command.json';
async function main(argv: readonly string[]): Promise<void> {
if (argv.length === 1 && (argv[0] === '--help' || argv[0] === '-h')) {
@@ -62,6 +65,9 @@ async function main(argv: readonly string[]): Promise<void> {
argv[0] !== 'reconciliation-capture-prepare' &&
argv[0] !== 'reconciliation-capture-commit' &&
argv[0] !== 'reconciliation-capture-verify' &&
argv[0] !== 'reconciliation-plan-prepare' &&
argv[0] !== 'reconciliation-plan-commit' &&
argv[0] !== 'reconciliation-plan-verify' &&
argv[0] !== 'compose-revision' &&
argv[0] !== 'compose-preflight' &&
argv[0] !== 'compose-apply' &&
@@ -138,6 +144,12 @@ async function main(argv: readonly string[]): Promise<void> {
? commitLocalReconciliationCaptureCommandFile(argv[2]!)
: argv[0] === 'reconciliation-capture-verify'
? verifyLocalReconciliationCaptureCommandFile(argv[2]!)
: argv[0] === 'reconciliation-plan-prepare'
? prepareLocalReconciliationPlanCommandFile(argv[2]!)
: argv[0] === 'reconciliation-plan-commit'
? commitLocalReconciliationPlanCommandFile(argv[2]!)
: argv[0] === 'reconciliation-plan-verify'
? verifyLocalReconciliationPlanCommandFile(argv[2]!)
: argv[0] === 'compose-revision'
? switchLocalDeploymentComposeRevisionCommandFile(argv[2]!)
: argv[0] === 'compose-preflight'
@@ -555,6 +555,17 @@ function readTerminal(
return Object.freeze({ manifest, receipt });
}
export function readLocalReconciliationCaptureTerminal(
captureRoot: string,
captureId: string,
uid: number,
): Readonly<{
manifest: Readonly<LocalReconciliationCaptureManifest>;
receipt: Readonly<LocalReconciliationCaptureReceipt>;
}> {
return readTerminal(capturePaths(captureRoot, captureId), uid);
}
function readPublishedManifest(
paths: ReturnType<typeof capturePaths>,
uid: number,
@@ -0,0 +1,328 @@
import path from 'node:path';
import { currentIdentity } from '../../foundation/contract';
import { LocalDeploymentConfigurationError } from '../../foundation/error';
const DIGEST_PATTERN = /^[0-9a-f]{64}$/;
const SAFE_PATH_PATTERN = /^\/[A-Za-z0-9._/@-]+$/;
const UUID_V4_PATTERN =
/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/;
const TIMEZONE_PATTERN = /^[A-Za-z0-9._+-]+(?:\/[A-Za-z0-9._+-]+)*$/;
const MAX_PATH_BYTES = 4_096;
const MAX_TIMEZONE_BYTES = 128;
export const LOCAL_RECONCILIATION_PLAN_DOMAINS = Object.freeze([
'schema_lineage',
'automation',
'secret_and_config',
'run_history',
'plugin_package',
'ai_and_tool',
'identity_policy_audit',
'unknown',
] as const);
export type LocalReconciliationPlanDomain =
(typeof LOCAL_RECONCILIATION_PLAN_DOMAINS)[number];
export type LocalReconciliationPlanDisposition =
| 'aligned'
| 'legacy_changed'
| 'target_changed'
| 'diverged'
| 'target_only'
| 'manual_required'
| 'unsupported';
export interface LocalReconciliationPlanOptions {
readonly deploymentRoot: string;
readonly captureRoot: string;
readonly planRoot: string;
readonly allowRootService: boolean;
}
export interface LocalReconciliationPlanPrepareCommand {
readonly schemaVersion: 1;
readonly operation: 'local.deployment.reconciliation.plan.prepare';
readonly options: Readonly<LocalReconciliationPlanOptions>;
readonly request: Readonly<{
planId: string;
captureId: string;
expectedBundleDigest: string;
expectedHeadDigest: string;
legacyTimezone: string | null;
preparedAtMs: number;
}>;
}
export interface LocalReconciliationPlanCommitCommand {
readonly schemaVersion: 1;
readonly operation: 'local.deployment.reconciliation.plan.commit';
readonly options: Readonly<LocalReconciliationPlanOptions>;
readonly request: Readonly<{
planId: string;
expectedPreparationDigest: string;
committedAtMs: number;
}>;
}
export interface LocalReconciliationPlanVerifyCommand {
readonly schemaVersion: 1;
readonly operation: 'local.deployment.reconciliation.plan.verify';
readonly options: Readonly<LocalReconciliationPlanOptions>;
readonly request: Readonly<{
planId: string;
expectedPlanDigest: string;
}>;
}
export interface LocalReconciliationPlanPrepareResult {
readonly schemaVersion: 1;
readonly operation: 'local.deployment.reconciliation.plan.prepare';
readonly status: 'prepared' | 'existing';
readonly state: 'reconciliation_plan_prepared';
readonly planId: string;
readonly preparationDigest: string;
readonly instanceHeadDigest: string;
}
export interface LocalReconciliationPlanTerminalResult {
readonly schemaVersion: 1;
readonly operation:
| 'local.deployment.reconciliation.plan.commit'
| 'local.deployment.reconciliation.plan.verify';
readonly status: 'prepared' | 'existing' | 'verified';
readonly state: 'reconciliation_planned';
readonly planId: string;
readonly planDigest: string;
readonly outcome: 'review_required' | 'manual_required';
readonly domainCount: 8;
readonly instanceHeadDigest: string;
}
function configurationError(message: string): never {
throw new LocalDeploymentConfigurationError(message);
}
function object(value: unknown, label: string): Record<string, unknown> {
if (
!value ||
typeof value !== 'object' ||
Array.isArray(value) ||
(Object.getPrototypeOf(value) !== Object.prototype &&
Object.getPrototypeOf(value) !== null)
) {
configurationError(`${label} must be an object`);
}
return value as Record<string, unknown>;
}
function exact(
value: Record<string, unknown>,
keys: readonly string[],
label: string,
): void {
const actual = Object.keys(value).sort();
const expected = [...keys].sort();
if (
actual.length !== expected.length ||
actual.some((key, index) => key !== expected[index])
) {
configurationError(`${label} shape is invalid`);
}
}
function safeAbsolutePath(value: unknown, label: string): string {
if (
typeof value !== 'string' ||
!path.isAbsolute(value) ||
path.normalize(value) !== value ||
path.parse(value).root === value ||
value.includes('\0') ||
value.includes('//') ||
!SAFE_PATH_PATTERN.test(value) ||
Buffer.byteLength(value, 'utf8') > MAX_PATH_BYTES
) {
configurationError(`${label} must be a safe non-root absolute path`);
}
return value;
}
function overlaps(left: string, right: string): boolean {
const relative = path.relative(left, right);
return relative === '' || (!relative.startsWith('..') && !path.isAbsolute(relative));
}
function digest(value: unknown, label: string): string {
if (typeof value !== 'string' || !DIGEST_PATTERN.test(value)) {
configurationError(`${label} must be a SHA-256 digest`);
}
return value;
}
function identifier(value: unknown, label: string): string {
if (typeof value !== 'string' || !UUID_V4_PATTERN.test(value)) {
configurationError(`${label} must be a UUID v4`);
}
return value;
}
function normalizeOptions(value: unknown): Readonly<LocalReconciliationPlanOptions> {
const options = object(value, 'options');
exact(
options,
['allowRootService', 'captureRoot', 'deploymentRoot', 'planRoot'],
'options',
);
const identity = currentIdentity();
if (
typeof options.allowRootService !== 'boolean' ||
(identity.uid === 0) !== options.allowRootService
) {
configurationError('reconciliation plan command identity is invalid');
}
const deploymentRoot = safeAbsolutePath(options.deploymentRoot, 'deploymentRoot');
const captureRoot = safeAbsolutePath(options.captureRoot, 'captureRoot');
const planRoot = safeAbsolutePath(options.planRoot, 'planRoot');
if (
overlaps(deploymentRoot, captureRoot) ||
overlaps(captureRoot, deploymentRoot) ||
overlaps(deploymentRoot, planRoot) ||
overlaps(planRoot, deploymentRoot) ||
overlaps(captureRoot, planRoot) ||
overlaps(planRoot, captureRoot)
) {
configurationError('deploymentRoot, captureRoot and planRoot must not overlap');
}
return Object.freeze({
deploymentRoot,
captureRoot,
planRoot,
allowRootService: options.allowRootService,
});
}
export function normalizeLocalReconciliationPlanPrepareCommand(
value: unknown,
): Readonly<LocalReconciliationPlanPrepareCommand> {
const command = object(value, 'reconciliation plan prepare command');
exact(command, ['operation', 'options', 'request', 'schemaVersion'], 'command');
const request = object(command.request, 'request');
exact(
request,
[
'captureId',
'expectedBundleDigest',
'expectedHeadDigest',
'legacyTimezone',
'planId',
'preparedAtMs',
],
'request',
);
if (
command.schemaVersion !== 1 ||
command.operation !== 'local.deployment.reconciliation.plan.prepare' ||
!Number.isSafeInteger(request.preparedAtMs) ||
(request.preparedAtMs as number) < 0 ||
(request.legacyTimezone !== null &&
(typeof request.legacyTimezone !== 'string' ||
!TIMEZONE_PATTERN.test(request.legacyTimezone) ||
Buffer.byteLength(request.legacyTimezone, 'utf8') > MAX_TIMEZONE_BYTES))
) {
configurationError('reconciliation plan prepare command is invalid');
}
return Object.freeze({
schemaVersion: 1 as const,
operation: 'local.deployment.reconciliation.plan.prepare' as const,
options: normalizeOptions(command.options),
request: Object.freeze({
planId: identifier(request.planId, 'planId'),
captureId: identifier(request.captureId, 'captureId'),
expectedBundleDigest: digest(
request.expectedBundleDigest,
'expectedBundleDigest',
),
expectedHeadDigest: digest(request.expectedHeadDigest, 'expectedHeadDigest'),
legacyTimezone: request.legacyTimezone as string | null,
preparedAtMs: request.preparedAtMs as number,
}),
});
}
function normalizeTerminalCommand(
value: unknown,
operation:
| 'local.deployment.reconciliation.plan.commit'
| 'local.deployment.reconciliation.plan.verify',
): Readonly<{
command: Record<string, unknown>;
options: Readonly<LocalReconciliationPlanOptions>;
request: Record<string, unknown>;
}> {
const command = object(value, 'reconciliation plan terminal command');
exact(command, ['operation', 'options', 'request', 'schemaVersion'], 'command');
if (command.schemaVersion !== 1 || command.operation !== operation) {
configurationError('reconciliation plan terminal command is invalid');
}
return Object.freeze({
command,
options: normalizeOptions(command.options),
request: object(command.request, 'request'),
});
}
export function normalizeLocalReconciliationPlanCommitCommand(
value: unknown,
): Readonly<LocalReconciliationPlanCommitCommand> {
const normalized = normalizeTerminalCommand(
value,
'local.deployment.reconciliation.plan.commit',
);
exact(
normalized.request,
['committedAtMs', 'expectedPreparationDigest', 'planId'],
'request',
);
if (
!Number.isSafeInteger(normalized.request.committedAtMs) ||
(normalized.request.committedAtMs as number) < 0
) {
configurationError('reconciliation plan commit command is invalid');
}
return Object.freeze({
schemaVersion: 1 as const,
operation: 'local.deployment.reconciliation.plan.commit' as const,
options: normalized.options,
request: Object.freeze({
planId: identifier(normalized.request.planId, 'planId'),
expectedPreparationDigest: digest(
normalized.request.expectedPreparationDigest,
'expectedPreparationDigest',
),
committedAtMs: normalized.request.committedAtMs as number,
}),
});
}
export function normalizeLocalReconciliationPlanVerifyCommand(
value: unknown,
): Readonly<LocalReconciliationPlanVerifyCommand> {
const normalized = normalizeTerminalCommand(
value,
'local.deployment.reconciliation.plan.verify',
);
exact(normalized.request, ['expectedPlanDigest', 'planId'], 'request');
return Object.freeze({
schemaVersion: 1 as const,
operation: 'local.deployment.reconciliation.plan.verify' as const,
options: normalized.options,
request: Object.freeze({
planId: identifier(normalized.request.planId, 'planId'),
expectedPlanDigest: digest(
normalized.request.expectedPlanDigest,
'expectedPlanDigest',
),
}),
});
}
@@ -0,0 +1,426 @@
import crypto from 'node:crypto';
import type { DatabaseSync } from 'node:sqlite';
import { LocalDeploymentConfigurationError } from '../../foundation/error';
import {
LOCAL_RECONCILIATION_PLAN_DOMAINS,
type LocalReconciliationPlanDomain,
} from './contract';
import {
withLocalReconciliationSealedDatabase,
type LocalReconciliationSealedBundle,
type LocalReconciliationSealedBundleReaderDependencies,
type LocalReconciliationSealedDatabaseKind,
type LocalReconciliationSealedDatabaseTopology,
} from '../sealed-bundle/reader';
const MAX_SCHEMA_OBJECTS = 4_096;
const MAX_TABLES = 512;
export interface LocalReconciliationDomainInventory {
readonly domain: LocalReconciliationPlanDomain;
readonly schemaObjectCount: number;
readonly tableCount: number;
readonly rowCount: number;
readonly rowCountComplete: boolean;
readonly inventoryDigest: string;
}
export interface LocalReconciliationDatabaseInventory {
readonly kind: LocalReconciliationSealedDatabaseKind;
readonly topology: LocalReconciliationSealedDatabaseTopology['mode'];
readonly baselineState: 'unchanged' | 'changed';
readonly opened: boolean;
readonly integrity: 'ok' | 'manual_required';
readonly foreignKeys: 'ok' | 'manual_required';
readonly schemaObjectCount: number;
readonly tableCount: number;
readonly rowCount: number;
readonly rowCountComplete: boolean;
readonly unsupportedObjectCount: number;
readonly domains: readonly Readonly<LocalReconciliationDomainInventory>[];
readonly inventoryDigest: string;
}
export interface LocalReconciliationBundleInventory {
readonly legacy: Readonly<LocalReconciliationDatabaseInventory>;
readonly target: Readonly<LocalReconciliationDatabaseInventory>;
readonly inventoryDigest: string;
}
interface MutableDomainInventory {
schemaObjectCount: number;
tableCount: number;
rowCount: number;
rowCountComplete: boolean;
factHash: crypto.Hash;
}
function configurationError(message: string, cause?: unknown): never {
throw new LocalDeploymentConfigurationError(message, { cause });
}
function digest(value: unknown): string {
return crypto
.createHash('sha256')
.update(JSON.stringify(value), 'utf8')
.digest('hex');
}
function legacyDomain(name: string): LocalReconciliationPlanDomain {
if (['Crontabs', 'CrontabViews', 'Subscriptions'].includes(name)) {
return 'automation';
}
if (['Envs', 'Configs'].includes(name)) return 'secret_and_config';
if (['Auths', 'Users'].includes(name)) return 'identity_policy_audit';
if (['Dependences', 'Dependencies', 'Apps'].includes(name)) {
return 'plugin_package';
}
if (['CrontabStats', 'Logs'].includes(name)) return 'run_history';
return 'unknown';
}
function targetDomain(name: string): LocalReconciliationPlanDomain {
if (name === 'QingLong3SchemaCapabilities' || name.includes('Migration')) {
return 'schema_lineage';
}
if (
name.includes('TaskDefinition') ||
name.includes('Trigger') ||
name.includes('Automation')
) {
return 'automation';
}
if (name.includes('Secret') || name.includes('DataDirectoryAdoption')) {
return 'secret_and_config';
}
if (
name === 'Runs' ||
name.startsWith('Run') ||
name.startsWith('StepRun') ||
name.includes('CompletionReceiptJournal')
) {
return 'run_history';
}
if (name.includes('PluginPackage') || name.includes('PackageProposal')) {
return 'plugin_package';
}
if (
name.includes('Tool') ||
name.includes('Prompt') ||
name.includes('Provider') ||
name.includes('Model')
) {
return 'ai_and_tool';
}
if (
name.includes('Project') ||
name.includes('RoleBinding') ||
name.includes('Identity') ||
name.includes('Credential') ||
name.includes('Pepper') ||
name.includes('Approval') ||
name.includes('ApprovedAction') ||
name.includes('SecurityAudit') ||
name.includes('LocalOwner')
) {
return 'identity_policy_audit';
}
return 'unknown';
}
function classify(
kind: LocalReconciliationSealedDatabaseKind,
name: string,
): LocalReconciliationPlanDomain {
return kind === 'legacy' ? legacyDomain(name) : targetDomain(name);
}
function quotedIdentifier(name: string): string {
return `"${name.replaceAll('"', '""')}"`;
}
function addCount(
current: number,
countText: string,
): Readonly<{ value: number; complete: boolean }> {
if (!/^(?:0|[1-9][0-9]*)$/.test(countText)) {
return configurationError('SQLite row count is invalid');
}
const count = BigInt(countText);
const next = BigInt(current) + count;
if (next > BigInt(Number.MAX_SAFE_INTEGER)) {
return Object.freeze({ value: Number.MAX_SAFE_INTEGER, complete: false });
}
return Object.freeze({ value: Number(next), complete: true });
}
function emptyDomains(): Map<LocalReconciliationPlanDomain, MutableDomainInventory> {
return new Map(
LOCAL_RECONCILIATION_PLAN_DOMAINS.map((domain) => [
domain,
{
schemaObjectCount: 0,
tableCount: 0,
rowCount: 0,
rowCountComplete: true,
factHash: crypto.createHash('sha256'),
},
]),
);
}
function finalizeDomains(
domains: Map<LocalReconciliationPlanDomain, MutableDomainInventory>,
): readonly Readonly<LocalReconciliationDomainInventory>[] {
return Object.freeze(
LOCAL_RECONCILIATION_PLAN_DOMAINS.map((domain) => {
const current = domains.get(domain)!;
const payload = Object.freeze({
domain,
schemaObjectCount: current.schemaObjectCount,
tableCount: current.tableCount,
rowCount: current.rowCount,
rowCountComplete: current.rowCountComplete,
factDigest: current.factHash.digest('hex'),
});
return Object.freeze({
domain,
schemaObjectCount: current.schemaObjectCount,
tableCount: current.tableCount,
rowCount: current.rowCount,
rowCountComplete: current.rowCountComplete,
inventoryDigest: digest(payload),
});
}),
);
}
function manualInventory(
kind: LocalReconciliationSealedDatabaseKind,
topology: LocalReconciliationSealedDatabaseTopology['mode'],
baselineState: 'unchanged' | 'changed',
): Readonly<LocalReconciliationDatabaseInventory> {
const mutableDomains = emptyDomains();
for (const domain of mutableDomains.values()) {
domain.rowCountComplete = false;
}
const domains = finalizeDomains(mutableDomains);
const payload = Object.freeze({
kind,
topology,
baselineState,
opened: false,
integrity: 'manual_required' as const,
foreignKeys: 'manual_required' as const,
schemaObjectCount: 0,
tableCount: 0,
rowCount: 0,
rowCountComplete: false,
unsupportedObjectCount: 0,
domains,
});
return Object.freeze({ ...payload, inventoryDigest: digest(payload) });
}
function inspectDatabase(
client: DatabaseSync,
kind: LocalReconciliationSealedDatabaseKind,
topology: LocalReconciliationSealedDatabaseTopology['mode'],
baselineState: 'unchanged' | 'changed',
): Readonly<LocalReconciliationDatabaseInventory> {
const quick = client.prepare('PRAGMA quick_check(1)').get() as
| { readonly quick_check?: unknown }
| undefined;
const foreignKeyViolation = client.prepare('PRAGMA foreign_key_check').get();
const schema = client.prepare(
`SELECT type, name, tbl_name AS tableName
FROM sqlite_schema
WHERE name NOT LIKE 'sqlite_%'
ORDER BY type, name
LIMIT ${MAX_SCHEMA_OBJECTS + 1}`,
);
const schemaRows = schema.iterate() as IterableIterator<{
readonly type?: unknown;
readonly name?: unknown;
readonly tableName?: unknown;
}>;
const tables = client.prepare(
`SELECT name, type
FROM pragma_table_list
WHERE schema = 'main' AND name NOT LIKE 'sqlite_%'
ORDER BY name
LIMIT ${MAX_TABLES + 1}`,
);
const tableRows = tables.iterate() as IterableIterator<{
readonly name?: unknown;
readonly type?: unknown;
}>;
const domains = emptyDomains();
let unsupportedObjectCount = 0;
let schemaObjectCount = 0;
for (const row of schemaRows) {
schemaObjectCount += 1;
if (schemaObjectCount > MAX_SCHEMA_OBJECTS) {
configurationError('sealed SQLite schema object budget is exceeded');
}
if (
typeof row.type !== 'string' ||
typeof row.name !== 'string' ||
typeof row.tableName !== 'string' ||
Buffer.byteLength(row.name, 'utf8') > 1_024 ||
Buffer.byteLength(row.tableName, 'utf8') > 1_024
) {
configurationError('sealed SQLite schema catalog drifted');
}
const domain = classify(kind, row.tableName);
const current = domains.get(domain)!;
current.schemaObjectCount += 1;
current.factHash.update(
`${JSON.stringify({
type: row.type,
name: row.name,
tableName: row.tableName,
})}\n`,
'utf8',
);
if (!['index', 'table', 'trigger', 'view'].includes(row.type)) {
unsupportedObjectCount += 1;
}
}
let totalRows = 0;
let totalRowsComplete = true;
let tableCount = 0;
for (const row of tableRows) {
tableCount += 1;
if (tableCount > MAX_TABLES) {
configurationError('sealed SQLite table budget is exceeded');
}
if (
typeof row.name !== 'string' ||
typeof row.type !== 'string' ||
Buffer.byteLength(row.name, 'utf8') > 1_024
) {
configurationError('sealed SQLite table catalog drifted');
}
const domain = classify(kind, row.name);
const current = domains.get(domain)!;
current.tableCount += 1;
if (row.type !== 'table') {
current.rowCountComplete = false;
totalRowsComplete = false;
unsupportedObjectCount += 1;
current.factHash.update(
`${JSON.stringify({ table: row.name, tableType: row.type })}\n`,
'utf8',
);
continue;
}
if (domain === 'unknown') {
current.rowCountComplete = false;
totalRowsComplete = false;
current.factHash.update(
`${JSON.stringify({ table: row.name, countSkipped: true })}\n`,
'utf8',
);
continue;
}
const counted = client
.prepare(
`SELECT CAST(COUNT(*) AS TEXT) AS countText FROM ${quotedIdentifier(
row.name,
)}`,
)
.get() as { readonly countText?: unknown } | undefined;
if (typeof counted?.countText !== 'string') {
configurationError('sealed SQLite table count drifted');
}
const domainCount = addCount(current.rowCount, counted.countText);
current.rowCount = domainCount.value;
current.rowCountComplete &&= domainCount.complete;
const totalCount = addCount(totalRows, counted.countText);
totalRows = totalCount.value;
totalRowsComplete &&= totalCount.complete;
current.factHash.update(
`${JSON.stringify({ table: row.name, countText: counted.countText })}\n`,
'utf8',
);
}
const finalized = finalizeDomains(domains);
const payload = Object.freeze({
kind,
topology,
baselineState,
opened: true,
integrity:
quick?.quick_check === 'ok'
? ('ok' as const)
: ('manual_required' as const),
foreignKeys:
foreignKeyViolation === undefined
? ('ok' as const)
: ('manual_required' as const),
schemaObjectCount,
tableCount,
rowCount: totalRows,
rowCountComplete: totalRowsComplete,
unsupportedObjectCount,
domains: finalized,
});
return Object.freeze({ ...payload, inventoryDigest: digest(payload) });
}
function baselineState(
bundle: Readonly<LocalReconciliationSealedBundle>,
kind: LocalReconciliationSealedDatabaseKind,
): 'unchanged' | 'changed' {
const main = bundle.manifest.assets.find(
(asset) => asset.logicalName === `${kind}-main`,
);
if (main === undefined) configurationError('sealed bundle main asset is absent');
const baseline =
kind === 'legacy'
? bundle.manifest.legacyBaselineSha256
: bundle.manifest.targetBaselineSha256;
const hasSidecar = bundle.manifest.assets.some((asset) =>
asset.logicalName.startsWith(`${kind}-`) && asset.logicalName !== `${kind}-main`,
);
return main.sha256 === baseline && !hasSidecar ? 'unchanged' : 'changed';
}
function databaseInventory(
bundle: Readonly<LocalReconciliationSealedBundle>,
kind: LocalReconciliationSealedDatabaseKind,
uid: number,
dependencies: LocalReconciliationSealedBundleReaderDependencies,
): Readonly<LocalReconciliationDatabaseInventory> {
const selected = kind === 'legacy' ? bundle.legacy : bundle.target;
const baseline = baselineState(bundle, kind);
if (selected.mode === 'manual_required') {
return manualInventory(kind, selected.mode, baseline);
}
const inspected = withLocalReconciliationSealedDatabase(
bundle,
kind,
uid,
dependencies,
(client) => inspectDatabase(client, kind, selected.mode, baseline),
);
if (inspected === null) return manualInventory(kind, selected.mode, baseline);
return inspected;
}
export function inventoryLocalReconciliationSealedBundle(
bundle: Readonly<LocalReconciliationSealedBundle>,
uid: number,
dependencies: LocalReconciliationSealedBundleReaderDependencies = {},
): Readonly<LocalReconciliationBundleInventory> {
const legacy = databaseInventory(bundle, 'legacy', uid, dependencies);
const target = databaseInventory(bundle, 'target', uid, dependencies);
const payload = Object.freeze({
bundleDigest: bundle.receipt.bundleDigest,
legacy,
target,
});
return Object.freeze({ legacy, target, inventoryDigest: digest(payload) });
}
@@ -0,0 +1,567 @@
import { LocalDeploymentConfigurationError } from '../../foundation/error';
import { cutoverDigest } from '../../cutover/targetEvidence';
import {
LOCAL_RECONCILIATION_PLAN_DOMAINS,
type LocalReconciliationPlanDisposition,
type LocalReconciliationPlanDomain,
} from './contract';
import type { LocalReconciliationBundleInventory } from './inventory';
import type { LocalReconciliationPlanIntent } from './preparation';
const PLAN_SCHEMA = 'qinglong3-local-reconciliation-plan';
const RECEIPT_SCHEMA = 'qinglong3-local-reconciliation-plan-receipt';
const DIGEST_PATTERN = /^[0-9a-f]{64}$/;
const UUID_V4_PATTERN =
/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/;
const TIMEZONE_PATTERN = /^[A-Za-z0-9._+-]+(?:\/[A-Za-z0-9._+-]+)*$/;
const MAX_PLAN_BYTES = 64 * 1024;
export interface LocalReconciliationPlanDomainSummary {
readonly domain: LocalReconciliationPlanDomain;
readonly legacySchemaObjects: number;
readonly targetSchemaObjects: number;
readonly legacyTables: number;
readonly targetTables: number;
readonly legacyRows: number;
readonly targetRows: number;
readonly rowCountsComplete: boolean;
readonly inventoryDigest: string;
readonly disposition: LocalReconciliationPlanDisposition;
}
export interface LocalReconciliationPlanDatabaseSummary {
readonly kind: 'legacy' | 'target';
readonly topology:
| 'main_only_immutable'
| 'wal_shm_readonly'
| 'manual_required';
readonly baselineState: 'unchanged' | 'changed';
readonly opened: boolean;
readonly integrity: 'ok' | 'manual_required';
readonly foreignKeys: 'ok' | 'manual_required';
readonly schemaObjectCount: number;
readonly tableCount: number;
readonly rowCount: number;
readonly rowCountComplete: boolean;
readonly unsupportedObjectCount: number;
readonly inventoryDigest: string;
}
export interface LocalReconciliationPlan {
readonly schema: typeof PLAN_SCHEMA;
readonly schemaVersion: 1;
readonly state: 'reconciliation_planned';
readonly planId: string;
readonly captureId: string;
readonly profile: 'edge' | 'standalone';
readonly preparationDigest: string;
readonly bundleDigest: string;
readonly legacyTimezone: string | null;
readonly committedAtMs: number;
readonly databases: readonly [
Readonly<LocalReconciliationPlanDatabaseSummary>,
Readonly<LocalReconciliationPlanDatabaseSummary>,
];
readonly domains: readonly Readonly<LocalReconciliationPlanDomainSummary>[];
readonly inventoryDigest: string;
readonly outcome: 'review_required' | 'manual_required';
readonly planDigest: string;
}
export interface LocalReconciliationPlanReceipt {
readonly schema: typeof RECEIPT_SCHEMA;
readonly schemaVersion: 1;
readonly state: 'reconciliation_planned';
readonly planId: string;
readonly captureId: string;
readonly preparationDigest: string;
readonly bundleDigest: string;
readonly planDigest: string;
readonly outcome: 'review_required' | 'manual_required';
readonly domainCount: 8;
readonly committedAtMs: number;
readonly receiptDigest: string;
}
function configurationError(message: string): never {
throw new LocalDeploymentConfigurationError(message);
}
function object(value: unknown, label: string): Record<string, unknown> {
if (
!value ||
typeof value !== 'object' ||
Array.isArray(value) ||
(Object.getPrototypeOf(value) !== Object.prototype &&
Object.getPrototypeOf(value) !== null)
) {
configurationError(`${label} must be an object`);
}
return value as Record<string, unknown>;
}
function exact(
value: Record<string, unknown>,
keys: readonly string[],
label: string,
): void {
const actual = Object.keys(value).sort();
const expected = [...keys].sort();
if (
actual.length !== expected.length ||
actual.some((key, index) => key !== expected[index])
) {
configurationError(`${label} shape is invalid`);
}
}
function safeCount(value: unknown): value is number {
return Number.isSafeInteger(value) && (value as number) >= 0;
}
function summary(
inventory: LocalReconciliationBundleInventory['legacy'],
): Readonly<LocalReconciliationPlanDatabaseSummary> {
return Object.freeze({
kind: inventory.kind,
topology: inventory.topology,
baselineState: inventory.baselineState,
opened: inventory.opened,
integrity: inventory.integrity,
foreignKeys: inventory.foreignKeys,
schemaObjectCount: inventory.schemaObjectCount,
tableCount: inventory.tableCount,
rowCount: inventory.rowCount,
rowCountComplete: inventory.rowCountComplete,
unsupportedObjectCount: inventory.unsupportedObjectCount,
inventoryDigest: inventory.inventoryDigest,
});
}
function domainDisposition(
domain: LocalReconciliationPlanDomain,
legacy: LocalReconciliationBundleInventory['legacy']['domains'][number],
target: LocalReconciliationBundleInventory['target']['domains'][number],
inventory: Readonly<LocalReconciliationBundleInventory>,
legacyTimezone: string | null,
): LocalReconciliationPlanDisposition {
const legacyFacts = legacy.schemaObjectCount + legacy.tableCount;
const targetFacts = target.schemaObjectCount + target.tableCount;
if (
inventory.legacy.topology === 'manual_required' ||
inventory.target.topology === 'manual_required' ||
inventory.legacy.integrity !== 'ok' ||
inventory.target.integrity !== 'ok' ||
inventory.legacy.foreignKeys !== 'ok' ||
inventory.target.foreignKeys !== 'ok'
) {
return 'manual_required';
}
if (
domain === 'unknown' &&
legacy.schemaObjectCount +
legacy.tableCount +
target.schemaObjectCount +
target.tableCount >
0
) {
return 'manual_required';
}
if (
!legacy.rowCountComplete ||
!target.rowCountComplete ||
inventory.legacy.unsupportedObjectCount > 0 ||
inventory.target.unsupportedObjectCount > 0
) {
return 'unsupported';
}
if (
(domain === 'secret_and_config' ||
domain === 'run_history' ||
domain === 'identity_policy_audit') &&
legacyFacts + targetFacts > 0
) {
return 'manual_required';
}
if (domain === 'automation' && legacyTimezone !== null && targetFacts > 0) {
return 'manual_required';
}
if (legacyFacts === 0 && targetFacts === 0) return 'aligned';
if (legacyFacts === 0) return 'target_only';
if (targetFacts === 0) return 'legacy_changed';
if (
inventory.legacy.baselineState === 'unchanged' &&
inventory.target.baselineState === 'unchanged'
) {
return 'aligned';
}
if (
inventory.legacy.baselineState === 'changed' &&
inventory.target.baselineState === 'unchanged'
) {
return 'legacy_changed';
}
if (
inventory.legacy.baselineState === 'unchanged' &&
inventory.target.baselineState === 'changed'
) {
return 'target_changed';
}
return 'diverged';
}
export function buildLocalReconciliationPlan(
intent: Readonly<LocalReconciliationPlanIntent>,
inventory: Readonly<LocalReconciliationBundleInventory>,
committedAtMs: number,
): Readonly<LocalReconciliationPlan> {
const domains = Object.freeze(
LOCAL_RECONCILIATION_PLAN_DOMAINS.map((domain, index) => {
const legacy = inventory.legacy.domains[index]!;
const target = inventory.target.domains[index]!;
if (legacy.domain !== domain || target.domain !== domain) {
configurationError('reconciliation inventory domain ordering drifted');
}
const disposition = domainDisposition(
domain,
legacy,
target,
inventory,
intent.command.request.legacyTimezone,
);
const inventoryDigest = cutoverDigest({
domain,
legacyInventoryDigest: legacy.inventoryDigest,
targetInventoryDigest: target.inventoryDigest,
});
return Object.freeze({
domain,
legacySchemaObjects: legacy.schemaObjectCount,
targetSchemaObjects: target.schemaObjectCount,
legacyTables: legacy.tableCount,
targetTables: target.tableCount,
legacyRows: legacy.rowCount,
targetRows: target.rowCount,
rowCountsComplete:
legacy.rowCountComplete && target.rowCountComplete,
inventoryDigest,
disposition,
});
}),
);
const outcome = domains.some(
(domain) =>
domain.disposition === 'manual_required' ||
domain.disposition === 'unsupported',
)
? ('manual_required' as const)
: ('review_required' as const);
const payload = Object.freeze({
schema: PLAN_SCHEMA,
schemaVersion: 1 as const,
state: 'reconciliation_planned' as const,
planId: intent.command.request.planId,
captureId: intent.command.request.captureId,
profile: intent.profile,
preparationDigest: intent.preparationDigest,
bundleDigest: intent.command.request.expectedBundleDigest,
legacyTimezone: intent.command.request.legacyTimezone,
committedAtMs,
databases: Object.freeze([
summary(inventory.legacy),
summary(inventory.target),
]) as LocalReconciliationPlan['databases'],
domains,
inventoryDigest: inventory.inventoryDigest,
outcome,
});
const plan = Object.freeze({ ...payload, planDigest: cutoverDigest(payload) });
if (Buffer.byteLength(`${JSON.stringify(plan, null, 2)}\n`, 'utf8') > MAX_PLAN_BYTES) {
configurationError('reconciliation plan exceeds the 64 KiB budget');
}
return plan;
}
function normalizeDatabaseSummary(
value: unknown,
expectedKind: 'legacy' | 'target',
): Readonly<LocalReconciliationPlanDatabaseSummary> {
const database = object(value, 'reconciliation plan database summary');
exact(
database,
[
'baselineState',
'foreignKeys',
'integrity',
'inventoryDigest',
'kind',
'opened',
'rowCount',
'rowCountComplete',
'schemaObjectCount',
'tableCount',
'topology',
'unsupportedObjectCount',
],
'reconciliation plan database summary',
);
if (
database.kind !== expectedKind ||
!['main_only_immutable', 'wal_shm_readonly', 'manual_required'].includes(
database.topology as string,
) ||
!['unchanged', 'changed'].includes(database.baselineState as string) ||
typeof database.opened !== 'boolean' ||
!['ok', 'manual_required'].includes(database.integrity as string) ||
!['ok', 'manual_required'].includes(database.foreignKeys as string) ||
!safeCount(database.schemaObjectCount) ||
!safeCount(database.tableCount) ||
!safeCount(database.rowCount) ||
typeof database.rowCountComplete !== 'boolean' ||
!safeCount(database.unsupportedObjectCount) ||
typeof database.inventoryDigest !== 'string' ||
!DIGEST_PATTERN.test(database.inventoryDigest)
) {
configurationError('reconciliation plan database summary drifted');
}
if (
(database.topology === 'manual_required' && database.opened) ||
(database.topology !== 'manual_required' && !database.opened)
) {
configurationError('reconciliation plan database open evidence drifted');
}
return Object.freeze(
database,
) as unknown as Readonly<LocalReconciliationPlanDatabaseSummary>;
}
function normalizeDomainSummary(
value: unknown,
expectedDomain: LocalReconciliationPlanDomain,
): Readonly<LocalReconciliationPlanDomainSummary> {
const domain = object(value, 'reconciliation plan domain summary');
exact(
domain,
[
'disposition',
'domain',
'inventoryDigest',
'legacyRows',
'legacySchemaObjects',
'legacyTables',
'rowCountsComplete',
'targetRows',
'targetSchemaObjects',
'targetTables',
],
'reconciliation plan domain summary',
);
if (
domain.domain !== expectedDomain ||
![
'aligned',
'legacy_changed',
'target_changed',
'diverged',
'target_only',
'manual_required',
'unsupported',
].includes(domain.disposition as string) ||
!safeCount(domain.legacySchemaObjects) ||
!safeCount(domain.targetSchemaObjects) ||
!safeCount(domain.legacyTables) ||
!safeCount(domain.targetTables) ||
!safeCount(domain.legacyRows) ||
!safeCount(domain.targetRows) ||
typeof domain.rowCountsComplete !== 'boolean' ||
typeof domain.inventoryDigest !== 'string' ||
!DIGEST_PATTERN.test(domain.inventoryDigest)
) {
configurationError('reconciliation plan domain summary drifted');
}
return Object.freeze(
domain,
) as unknown as Readonly<LocalReconciliationPlanDomainSummary>;
}
export function normalizeLocalReconciliationPlan(
value: unknown,
): Readonly<LocalReconciliationPlan> {
const plan = object(value, 'reconciliation plan');
exact(
plan,
[
'bundleDigest',
'captureId',
'committedAtMs',
'databases',
'domains',
'inventoryDigest',
'legacyTimezone',
'outcome',
'planDigest',
'planId',
'preparationDigest',
'profile',
'schema',
'schemaVersion',
'state',
],
'reconciliation plan',
);
const rawDatabases = plan.databases;
const rawDomains = plan.domains;
if (!Array.isArray(rawDatabases) || !Array.isArray(rawDomains)) {
configurationError('reconciliation plan summaries must be arrays');
}
const databases = Object.freeze([
normalizeDatabaseSummary(rawDatabases[0], 'legacy'),
normalizeDatabaseSummary(rawDatabases[1], 'target'),
]) as LocalReconciliationPlan['databases'];
const domains = Object.freeze(
LOCAL_RECONCILIATION_PLAN_DOMAINS.map((domain, index) =>
normalizeDomainSummary(rawDomains[index], domain),
),
);
const { planDigest, ...payload } = plan;
if (
plan.schema !== PLAN_SCHEMA ||
plan.schemaVersion !== 1 ||
plan.state !== 'reconciliation_planned' ||
typeof plan.planId !== 'string' ||
!UUID_V4_PATTERN.test(plan.planId) ||
typeof plan.captureId !== 'string' ||
!UUID_V4_PATTERN.test(plan.captureId) ||
rawDatabases.length !== 2 ||
rawDomains.length !== 8 ||
(plan.profile !== 'edge' && plan.profile !== 'standalone') ||
!Number.isSafeInteger(plan.committedAtMs) ||
(plan.committedAtMs as number) < 0 ||
(plan.legacyTimezone !== null &&
(typeof plan.legacyTimezone !== 'string' ||
!TIMEZONE_PATTERN.test(plan.legacyTimezone) ||
Buffer.byteLength(plan.legacyTimezone, 'utf8') > 128)) ||
(plan.outcome !== 'review_required' && plan.outcome !== 'manual_required') ||
[
plan.preparationDigest,
plan.bundleDigest,
plan.inventoryDigest,
planDigest,
].some(
(candidate) =>
typeof candidate !== 'string' || !DIGEST_PATTERN.test(candidate),
) ||
cutoverDigest(payload) !== planDigest
) {
configurationError('reconciliation plan drifted');
}
const derivedOutcome = domains.some(
(domain) =>
domain.disposition === 'manual_required' ||
domain.disposition === 'unsupported',
)
? 'manual_required'
: 'review_required';
if (plan.outcome !== derivedOutcome) {
configurationError('reconciliation plan outcome drifted');
}
const sum = (values: readonly number[]): bigint =>
values.reduce((total, value) => total + BigInt(value), 0n);
if (
sum(domains.map((domain) => domain.legacySchemaObjects)) !==
BigInt(databases[0].schemaObjectCount) ||
sum(domains.map((domain) => domain.targetSchemaObjects)) !==
BigInt(databases[1].schemaObjectCount) ||
sum(domains.map((domain) => domain.legacyTables)) !==
BigInt(databases[0].tableCount) ||
sum(domains.map((domain) => domain.targetTables)) !==
BigInt(databases[1].tableCount) ||
(databases[0].rowCountComplete &&
sum(domains.map((domain) => domain.legacyRows)) !==
BigInt(databases[0].rowCount)) ||
(databases[1].rowCountComplete &&
sum(domains.map((domain) => domain.targetRows)) !==
BigInt(databases[1].rowCount)) ||
(databases[0].rowCountComplete &&
databases[1].rowCountComplete &&
domains.some((domain) => !domain.rowCountsComplete))
) {
configurationError('reconciliation plan aggregate summary drifted');
}
return Object.freeze({
...(plan as unknown as LocalReconciliationPlan),
databases,
domains,
});
}
export function localReconciliationPlanReceipt(
plan: Readonly<LocalReconciliationPlan>,
): Readonly<LocalReconciliationPlanReceipt> {
const payload = Object.freeze({
schema: RECEIPT_SCHEMA,
schemaVersion: 1 as const,
state: 'reconciliation_planned' as const,
planId: plan.planId,
captureId: plan.captureId,
preparationDigest: plan.preparationDigest,
bundleDigest: plan.bundleDigest,
planDigest: plan.planDigest,
outcome: plan.outcome,
domainCount: 8 as const,
committedAtMs: plan.committedAtMs,
});
return Object.freeze({ ...payload, receiptDigest: cutoverDigest(payload) });
}
export function normalizeLocalReconciliationPlanReceipt(
value: unknown,
): Readonly<LocalReconciliationPlanReceipt> {
const receipt = object(value, 'reconciliation plan receipt');
exact(
receipt,
[
'bundleDigest',
'captureId',
'committedAtMs',
'domainCount',
'outcome',
'planDigest',
'planId',
'preparationDigest',
'receiptDigest',
'schema',
'schemaVersion',
'state',
],
'reconciliation plan receipt',
);
const { receiptDigest, ...payload } = receipt;
if (
receipt.schema !== RECEIPT_SCHEMA ||
receipt.schemaVersion !== 1 ||
receipt.state !== 'reconciliation_planned' ||
typeof receipt.planId !== 'string' ||
!UUID_V4_PATTERN.test(receipt.planId) ||
typeof receipt.captureId !== 'string' ||
!UUID_V4_PATTERN.test(receipt.captureId) ||
receipt.domainCount !== 8 ||
(receipt.outcome !== 'review_required' &&
receipt.outcome !== 'manual_required') ||
!Number.isSafeInteger(receipt.committedAtMs) ||
(receipt.committedAtMs as number) < 0 ||
[
receipt.preparationDigest,
receipt.bundleDigest,
receipt.planDigest,
receiptDigest,
].some(
(candidate) =>
typeof candidate !== 'string' || !DIGEST_PATTERN.test(candidate),
) ||
cutoverDigest(payload) !== receiptDigest
) {
configurationError('reconciliation plan receipt drifted');
}
return receipt as unknown as Readonly<LocalReconciliationPlanReceipt>;
}
@@ -0,0 +1,673 @@
import fs from 'node:fs';
import path from 'node:path';
import { readPrivateLocalCommandFile } from '@qinglong/local-command-file';
import { currentIdentity } from '../../foundation/contract';
import { LocalDeploymentConfigurationError } from '../../foundation/error';
import {
ensurePrivateDirectory,
preflightPublishedFile,
publishExactFile,
validatePrivateDirectory,
} from '../../foundation/files';
import {
advanceLocalCutoverInstanceHead,
readLocalCutoverInstanceHead,
type LocalCutoverInstanceHead,
} from '../../cutover/instanceLineage';
import { cutoverDigest } from '../../cutover/targetEvidence';
import {
inspectLocalReconciliationSealedBundle,
type LocalReconciliationSealedBundle,
type LocalReconciliationSealedBundleReaderDependencies,
} from '../sealed-bundle/reader';
import { readLocalReconciliationCaptureIntent } from '../preparation';
import {
normalizeLocalReconciliationPlanCommitCommand,
normalizeLocalReconciliationPlanPrepareCommand,
normalizeLocalReconciliationPlanVerifyCommand,
type LocalReconciliationPlanCommitCommand,
type LocalReconciliationPlanPrepareCommand,
type LocalReconciliationPlanPrepareResult,
type LocalReconciliationPlanTerminalResult,
} from './contract';
import { inventoryLocalReconciliationSealedBundle } from './inventory';
import {
buildLocalReconciliationPlan,
localReconciliationPlanReceipt,
normalizeLocalReconciliationPlan,
normalizeLocalReconciliationPlanReceipt,
type LocalReconciliationPlan,
type LocalReconciliationPlanReceipt,
} from './plan';
const INTENT_SCHEMA = 'qinglong3-local-reconciliation-plan-intent';
const DIGEST_PATTERN = /^[0-9a-f]{64}$/;
const MAX_PLANS = 64;
export interface LocalReconciliationPlanIntent {
readonly schema: typeof INTENT_SCHEMA;
readonly schemaVersion: 1;
readonly state: 'reconciliation_plan_prepared';
readonly command: Readonly<LocalReconciliationPlanPrepareCommand>;
readonly profile: 'edge' | 'standalone';
readonly instanceId: string;
readonly cutoverId: string;
readonly generation: number;
readonly activationDigest: string;
readonly captureManifestDigest: string;
readonly captureFingerprintDigest: string;
readonly capturedHeadDigest: string;
readonly preparationDigest: string;
}
export interface LocalReconciliationPlanDependencies
extends LocalReconciliationSealedBundleReaderDependencies {
readonly afterHeadPrepared?: () => void;
readonly afterPlanPublished?: () => void;
readonly afterReceiptPublished?: () => void;
readonly afterHeadAdvanced?: () => void;
}
interface LocalReconciliationPlanPaths {
readonly root: string;
readonly staging: string;
readonly intent: string;
readonly plan: string;
readonly receipt: string;
}
function configurationError(message: string): never {
throw new LocalDeploymentConfigurationError(message);
}
function object(value: unknown, label: string): Record<string, unknown> {
if (
!value ||
typeof value !== 'object' ||
Array.isArray(value) ||
(Object.getPrototypeOf(value) !== Object.prototype &&
Object.getPrototypeOf(value) !== null)
) {
configurationError(`${label} must be an object`);
}
return value as Record<string, unknown>;
}
function exact(
value: Record<string, unknown>,
keys: readonly string[],
label: string,
): void {
const actual = Object.keys(value).sort();
const expected = [...keys].sort();
if (
actual.length !== expected.length ||
actual.some((key, index) => key !== expected[index])
) {
configurationError(`${label} shape is invalid`);
}
}
export function localReconciliationPlanDirectory(
planRoot: string,
planId: string,
): string {
return path.join(planRoot, planId);
}
function planPaths(planRoot: string, planId: string): LocalReconciliationPlanPaths {
const root = localReconciliationPlanDirectory(planRoot, planId);
return Object.freeze({
root,
staging: path.join(root, 'staging'),
intent: path.join(root, 'intent.json'),
plan: path.join(root, 'plan.json'),
receipt: path.join(root, 'receipt.json'),
});
}
function ensurePlanDirectory(
planRoot: string,
planId: string,
uid: number,
): Readonly<LocalReconciliationPlanPaths> {
const paths = planPaths(planRoot, planId);
const entries = fs.readdirSync(planRoot, { withFileTypes: true });
for (const entry of entries) {
if (!entry.isDirectory() || entry.isSymbolicLink()) {
configurationError('reconciliation plan catalog contains drift');
}
}
if (entries.length >= MAX_PLANS && !fs.existsSync(paths.root)) {
configurationError('reconciliation plan retention limit is reached');
}
ensurePrivateDirectory(paths.root, uid, 'reconciliationPlanDirectory');
ensurePrivateDirectory(paths.staging, uid, 'reconciliationPlanStaging');
return paths;
}
function validateCatalog(
paths: Readonly<LocalReconciliationPlanPaths>,
terminal: boolean,
): void {
const allowed = new Set([
'intent.json',
'plan.json',
'receipt.json',
'staging',
...(!terminal
? [
'.intent.json.ql3-deploy-stage',
'.plan.json.ql3-deploy-stage',
'.receipt.json.ql3-deploy-stage',
]
: []),
]);
for (const entry of fs.readdirSync(paths.root, { withFileTypes: true })) {
if (!allowed.has(entry.name) || entry.isSymbolicLink()) {
configurationError('reconciliation plan root contains unknown material');
}
}
if (fs.readdirSync(paths.staging).length !== 0) {
configurationError('reconciliation plan staging contains unknown material');
}
}
function contents(value: unknown): string {
return `${JSON.stringify(value, null, 2)}\n`;
}
export function normalizeLocalReconciliationPlanIntent(
value: unknown,
): Readonly<LocalReconciliationPlanIntent> {
const intent = object(value, 'reconciliation plan intent');
exact(
intent,
[
'activationDigest',
'captureFingerprintDigest',
'captureManifestDigest',
'capturedHeadDigest',
'command',
'cutoverId',
'generation',
'instanceId',
'preparationDigest',
'profile',
'schema',
'schemaVersion',
'state',
],
'reconciliation plan intent',
);
const command = normalizeLocalReconciliationPlanPrepareCommand(intent.command);
const { preparationDigest, ...payload } = intent;
if (
intent.schema !== INTENT_SCHEMA ||
intent.schemaVersion !== 1 ||
intent.state !== 'reconciliation_plan_prepared' ||
(intent.profile !== 'edge' && intent.profile !== 'standalone') ||
typeof intent.instanceId !== 'string' ||
intent.instanceId.length < 1 ||
intent.instanceId.length > 128 ||
typeof intent.cutoverId !== 'string' ||
intent.cutoverId.length < 1 ||
intent.cutoverId.length > 128 ||
!Number.isSafeInteger(intent.generation) ||
(intent.generation as number) < 1 ||
[
intent.activationDigest,
intent.captureManifestDigest,
intent.captureFingerprintDigest,
intent.capturedHeadDigest,
preparationDigest,
].some(
(candidate) =>
typeof candidate !== 'string' || !DIGEST_PATTERN.test(candidate),
) ||
cutoverDigest(payload) !== preparationDigest
) {
configurationError('reconciliation plan intent drifted');
}
return Object.freeze({
...(intent as unknown as LocalReconciliationPlanIntent),
command,
});
}
export function readLocalReconciliationPlanIntent(
planRoot: string,
planId: string,
): Readonly<LocalReconciliationPlanIntent> {
return normalizeLocalReconciliationPlanIntent(
readPrivateLocalCommandFile(planPaths(planRoot, planId).intent),
);
}
function buildIntent(
command: Readonly<LocalReconciliationPlanPrepareCommand>,
bundle: Readonly<LocalReconciliationSealedBundle>,
): Readonly<LocalReconciliationPlanIntent> {
const captureIntent = readLocalReconciliationCaptureIntent(
command.options.captureRoot,
command.request.captureId,
);
if (
captureIntent.command.options.deploymentRoot !==
command.options.deploymentRoot ||
captureIntent.command.options.captureRoot !== command.options.captureRoot ||
captureIntent.command.options.allowRootService !==
command.options.allowRootService ||
bundle.receipt.captureId !== command.request.captureId ||
bundle.receipt.bundleDigest !== command.request.expectedBundleDigest ||
command.request.preparedAtMs < bundle.manifest.committedAtMs
) {
configurationError('reconciliation plan is detached from its capture');
}
const payload = Object.freeze({
schema: INTENT_SCHEMA,
schemaVersion: 1 as const,
state: 'reconciliation_plan_prepared' as const,
command,
profile: captureIntent.command.request.profile,
instanceId: captureIntent.command.request.instanceId,
cutoverId: captureIntent.command.request.cutoverId,
generation: captureIntent.command.request.generation,
activationDigest: captureIntent.command.request.expectedActivationDigest,
captureManifestDigest: bundle.manifest.manifestDigest,
captureFingerprintDigest: bundle.fingerprintDigest,
capturedHeadDigest: command.request.expectedHeadDigest,
});
return Object.freeze({ ...payload, preparationDigest: cutoverDigest(payload) });
}
function validateHeadIdentity(
head: Readonly<LocalCutoverInstanceHead>,
intent: Readonly<LocalReconciliationPlanIntent>,
): void {
if (
head.profile !== intent.profile ||
head.cutoverId !== intent.cutoverId ||
head.activationDigest !== intent.activationDigest ||
head.generation !== intent.generation
) {
configurationError('reconciliation plan instance head identity drifted');
}
}
function advancePlanHead(
intent: Readonly<LocalReconciliationPlanIntent>,
uid: number,
state: 'reconciliation_plan_prepared' | 'reconciliation_planned',
requestedAtMs: number,
sourceRecordDigest: string,
): Readonly<LocalCutoverInstanceHead> {
return advanceLocalCutoverInstanceHead(
{
options: { deploymentRoot: intent.command.options.deploymentRoot },
request: {
cutoverId: intent.cutoverId,
profile: intent.profile,
instanceId: intent.instanceId,
expectedActivationDigest: intent.activationDigest,
requestedAtMs,
},
},
uid,
state,
intent.generation,
sourceRecordDigest,
);
}
export function prepareLocalReconciliationPlan(
input: unknown,
dependencies: LocalReconciliationPlanDependencies = {},
): Readonly<LocalReconciliationPlanPrepareResult> {
const command = normalizeLocalReconciliationPlanPrepareCommand(input);
const identity = currentIdentity();
validatePrivateDirectory(command.options.deploymentRoot, identity.uid, 'deploymentRoot');
validatePrivateDirectory(command.options.captureRoot, identity.uid, 'captureRoot');
validatePrivateDirectory(command.options.planRoot, identity.uid, 'planRoot');
const bundle = inspectLocalReconciliationSealedBundle(
command.options.captureRoot,
command.request.captureId,
identity.uid,
);
const intent = buildIntent(command, bundle);
const head = readLocalCutoverInstanceHead(
command.options.deploymentRoot,
intent.instanceId,
identity.uid,
);
validateHeadIdentity(head, intent);
if (
(head.state === 'reconciliation_captured' &&
(head.headDigest !== command.request.expectedHeadDigest ||
head.sourceRecordDigest !== command.request.expectedBundleDigest)) ||
(head.state === 'reconciliation_plan_prepared' &&
head.sourceRecordDigest !== intent.preparationDigest) ||
(head.state !== 'reconciliation_captured' &&
head.state !== 'reconciliation_plan_prepared')
) {
configurationError('plan prepare lost the captured instance head compare-and-swap');
}
const paths = ensurePlanDirectory(
command.options.planRoot,
command.request.planId,
identity.uid,
);
const serialized = contents(intent);
preflightPublishedFile(
paths.intent,
serialized,
0o600,
identity.uid,
'reconciliation plan intent',
);
const nextHead =
head.state === 'reconciliation_plan_prepared'
? head
: advancePlanHead(
intent,
identity.uid,
'reconciliation_plan_prepared',
command.request.preparedAtMs,
intent.preparationDigest,
);
dependencies.afterHeadPrepared?.();
const status = publishExactFile(
paths.intent,
serialized,
0o600,
identity.uid,
'reconciliation plan intent',
);
validateCatalog(paths, false);
return Object.freeze({
schemaVersion: 1 as const,
operation: command.operation,
status,
state: 'reconciliation_plan_prepared' as const,
planId: command.request.planId,
preparationDigest: intent.preparationDigest,
instanceHeadDigest: nextHead.headDigest,
});
}
function validateTerminalBinding(
intent: Readonly<LocalReconciliationPlanIntent>,
plan: Readonly<LocalReconciliationPlan>,
receipt: Readonly<LocalReconciliationPlanReceipt>,
bundle: Readonly<LocalReconciliationSealedBundle>,
): void {
if (
plan.planId !== intent.command.request.planId ||
plan.captureId !== intent.command.request.captureId ||
plan.profile !== intent.profile ||
plan.preparationDigest !== intent.preparationDigest ||
plan.bundleDigest !== intent.command.request.expectedBundleDigest ||
plan.legacyTimezone !== intent.command.request.legacyTimezone ||
plan.committedAtMs < intent.command.request.preparedAtMs ||
receipt.planId !== plan.planId ||
receipt.captureId !== plan.captureId ||
receipt.preparationDigest !== plan.preparationDigest ||
receipt.bundleDigest !== plan.bundleDigest ||
receipt.planDigest !== plan.planDigest ||
receipt.outcome !== plan.outcome ||
receipt.committedAtMs !== plan.committedAtMs ||
bundle.receipt.bundleDigest !== plan.bundleDigest ||
bundle.manifest.manifestDigest !== intent.captureManifestDigest ||
bundle.fingerprintDigest !== intent.captureFingerprintDigest
) {
configurationError('terminal reconciliation plan binding drifted');
}
}
function readTerminal(
paths: Readonly<LocalReconciliationPlanPaths>,
intent: Readonly<LocalReconciliationPlanIntent>,
bundle: Readonly<LocalReconciliationSealedBundle>,
uid: number,
): Readonly<{
plan: Readonly<LocalReconciliationPlan>;
receipt: Readonly<LocalReconciliationPlanReceipt>;
}> {
validatePrivateDirectory(paths.root, uid, 'reconciliationPlanDirectory');
validatePrivateDirectory(paths.staging, uid, 'reconciliationPlanStaging');
const plan = normalizeLocalReconciliationPlan(
readPrivateLocalCommandFile(paths.plan),
);
const receipt = normalizeLocalReconciliationPlanReceipt(
readPrivateLocalCommandFile(paths.receipt),
);
validateTerminalBinding(intent, plan, receipt, bundle);
validateCatalog(paths, true);
return Object.freeze({ plan, receipt });
}
function result(
operation: LocalReconciliationPlanTerminalResult['operation'],
status: LocalReconciliationPlanTerminalResult['status'],
terminal: Readonly<{
plan: Readonly<LocalReconciliationPlan>;
receipt: Readonly<LocalReconciliationPlanReceipt>;
}>,
head: Readonly<LocalCutoverInstanceHead>,
): Readonly<LocalReconciliationPlanTerminalResult> {
return Object.freeze({
schemaVersion: 1 as const,
operation,
status,
state: 'reconciliation_planned' as const,
planId: terminal.plan.planId,
planDigest: terminal.plan.planDigest,
outcome: terminal.plan.outcome,
domainCount: 8 as const,
instanceHeadDigest: head.headDigest,
});
}
function validateCommitBinding(
command: Readonly<LocalReconciliationPlanCommitCommand>,
intent: Readonly<LocalReconciliationPlanIntent>,
): void {
if (
intent.command.options.deploymentRoot !== command.options.deploymentRoot ||
intent.command.options.captureRoot !== command.options.captureRoot ||
intent.command.options.planRoot !== command.options.planRoot ||
intent.command.options.allowRootService !== command.options.allowRootService ||
intent.command.request.planId !== command.request.planId ||
intent.preparationDigest !== command.request.expectedPreparationDigest ||
command.request.committedAtMs < intent.command.request.preparedAtMs
) {
configurationError('plan commit is not bound to its exact preparation');
}
}
export function commitLocalReconciliationPlan(
input: unknown,
dependencies: LocalReconciliationPlanDependencies = {},
): Readonly<LocalReconciliationPlanTerminalResult> {
const command = normalizeLocalReconciliationPlanCommitCommand(input);
const identity = currentIdentity();
validatePrivateDirectory(command.options.deploymentRoot, identity.uid, 'deploymentRoot');
validatePrivateDirectory(command.options.captureRoot, identity.uid, 'captureRoot');
validatePrivateDirectory(command.options.planRoot, identity.uid, 'planRoot');
const paths = planPaths(command.options.planRoot, command.request.planId);
validatePrivateDirectory(paths.root, identity.uid, 'reconciliationPlanDirectory');
validatePrivateDirectory(paths.staging, identity.uid, 'reconciliationPlanStaging');
const intent = readLocalReconciliationPlanIntent(
command.options.planRoot,
command.request.planId,
);
validateCommitBinding(command, intent);
const bundle = inspectLocalReconciliationSealedBundle(
command.options.captureRoot,
intent.command.request.captureId,
identity.uid,
);
const head = readLocalCutoverInstanceHead(
command.options.deploymentRoot,
intent.instanceId,
identity.uid,
);
validateHeadIdentity(head, intent);
if (fs.existsSync(paths.receipt)) {
const terminal = readTerminal(paths, intent, bundle, identity.uid);
if (
terminal.plan.committedAtMs !== command.request.committedAtMs ||
(head.state !== 'reconciliation_plan_prepared' &&
head.state !== 'reconciliation_planned') ||
(head.state === 'reconciliation_plan_prepared' &&
head.sourceRecordDigest !== intent.preparationDigest) ||
(head.state === 'reconciliation_planned' &&
head.sourceRecordDigest !== terminal.plan.planDigest)
) {
configurationError('terminal plan lost its instance head binding');
}
const terminalHead =
head.state === 'reconciliation_planned'
? head
: advancePlanHead(
intent,
identity.uid,
'reconciliation_planned',
terminal.plan.committedAtMs,
terminal.plan.planDigest,
);
dependencies.afterHeadAdvanced?.();
return result(
command.operation,
head.state === 'reconciliation_planned' ? 'existing' : 'prepared',
terminal,
terminalHead,
);
}
if (
head.state !== 'reconciliation_plan_prepared' ||
head.sourceRecordDigest !== intent.preparationDigest
) {
configurationError('plan commit lost the prepared instance head fence');
}
let plan: Readonly<LocalReconciliationPlan>;
if (fs.existsSync(paths.plan)) {
plan = normalizeLocalReconciliationPlan(
readPrivateLocalCommandFile(paths.plan),
);
if (
plan.planId !== intent.command.request.planId ||
plan.captureId !== intent.command.request.captureId ||
plan.preparationDigest !== intent.preparationDigest ||
plan.bundleDigest !== intent.command.request.expectedBundleDigest ||
plan.committedAtMs !== command.request.committedAtMs
) {
configurationError('published reconciliation plan lost its preparation');
}
} else {
const inventory = inventoryLocalReconciliationSealedBundle(
bundle,
identity.uid,
dependencies,
);
plan = buildLocalReconciliationPlan(
intent,
inventory,
command.request.committedAtMs,
);
publishExactFile(
paths.plan,
contents(plan),
0o600,
identity.uid,
'reconciliation plan',
);
dependencies.afterPlanPublished?.();
}
const receipt = localReconciliationPlanReceipt(plan);
publishExactFile(
paths.receipt,
contents(receipt),
0o600,
identity.uid,
'reconciliation plan receipt',
);
dependencies.afterReceiptPublished?.();
const terminal = readTerminal(paths, intent, bundle, identity.uid);
const terminalHead = advancePlanHead(
intent,
identity.uid,
'reconciliation_planned',
plan.committedAtMs,
plan.planDigest,
);
dependencies.afterHeadAdvanced?.();
return result(command.operation, 'prepared', terminal, terminalHead);
}
export function verifyLocalReconciliationPlan(
input: unknown,
dependencies: LocalReconciliationSealedBundleReaderDependencies = {},
): Readonly<LocalReconciliationPlanTerminalResult> {
void dependencies;
const command = normalizeLocalReconciliationPlanVerifyCommand(input);
const identity = currentIdentity();
validatePrivateDirectory(command.options.deploymentRoot, identity.uid, 'deploymentRoot');
validatePrivateDirectory(command.options.captureRoot, identity.uid, 'captureRoot');
validatePrivateDirectory(command.options.planRoot, identity.uid, 'planRoot');
const paths = planPaths(command.options.planRoot, command.request.planId);
const intent = readLocalReconciliationPlanIntent(
command.options.planRoot,
command.request.planId,
);
if (
intent.command.options.deploymentRoot !== command.options.deploymentRoot ||
intent.command.options.captureRoot !== command.options.captureRoot ||
intent.command.options.planRoot !== command.options.planRoot ||
intent.command.options.allowRootService !== command.options.allowRootService
) {
configurationError('plan verify is detached from preparation');
}
const bundle = inspectLocalReconciliationSealedBundle(
command.options.captureRoot,
intent.command.request.captureId,
identity.uid,
);
const terminal = readTerminal(paths, intent, bundle, identity.uid);
if (terminal.plan.planDigest !== command.request.expectedPlanDigest) {
configurationError('plan verify expected digest drifted');
}
const head = readLocalCutoverInstanceHead(
command.options.deploymentRoot,
intent.instanceId,
identity.uid,
);
validateHeadIdentity(head, intent);
if (
head.state !== 'reconciliation_planned' ||
head.sourceRecordDigest !== terminal.plan.planDigest
) {
configurationError('plan verify lost the terminal instance head');
}
return result(command.operation, 'verified', terminal, head);
}
export function prepareLocalReconciliationPlanCommandFile(
filePath: string,
): Readonly<LocalReconciliationPlanPrepareResult> {
return prepareLocalReconciliationPlan(readPrivateLocalCommandFile(filePath));
}
export function commitLocalReconciliationPlanCommandFile(
filePath: string,
): Readonly<LocalReconciliationPlanTerminalResult> {
return commitLocalReconciliationPlan(readPrivateLocalCommandFile(filePath));
}
export function verifyLocalReconciliationPlanCommandFile(
filePath: string,
): Readonly<LocalReconciliationPlanTerminalResult> {
return verifyLocalReconciliationPlan(readPrivateLocalCommandFile(filePath));
}
@@ -0,0 +1,339 @@
import crypto from 'node:crypto';
import fs from 'node:fs';
import path from 'node:path';
import { DatabaseSync } from 'node:sqlite';
import { LocalDeploymentConfigurationError } from '../../foundation/error';
import { validatePrivateDirectory } from '../../foundation/files';
import {
readLocalReconciliationCaptureTerminal,
type LocalReconciliationCaptureManifest,
type LocalReconciliationCaptureReceipt,
} from '../bundle';
import { localReconciliationCaptureDirectory } from '../preparation';
import { localReconciliationCaptureAssetFileName } from '../stableCopy';
const HASH_BUFFER_BYTES = 64 * 1024;
export type LocalReconciliationSealedDatabaseKind = 'legacy' | 'target';
export type LocalReconciliationSealedDatabaseMode =
| 'main_only_immutable'
| 'wal_shm_readonly'
| 'manual_required';
export interface LocalReconciliationSealedDatabaseTopology {
readonly kind: LocalReconciliationSealedDatabaseKind;
readonly mode: LocalReconciliationSealedDatabaseMode;
readonly reason:
| null
| 'hot_journal'
| 'unpaired_wal_shm';
}
export interface LocalReconciliationSealedBundle {
readonly captureRoot: string;
readonly captureDirectory: string;
readonly assetsDirectory: string;
readonly manifest: Readonly<LocalReconciliationCaptureManifest>;
readonly receipt: Readonly<LocalReconciliationCaptureReceipt>;
readonly fingerprintDigest: string;
readonly target: Readonly<LocalReconciliationSealedDatabaseTopology>;
readonly legacy: Readonly<LocalReconciliationSealedDatabaseTopology>;
}
export interface LocalReconciliationSealedBundleReaderDependencies {
readonly beforeDatabaseOpen?: (
kind: LocalReconciliationSealedDatabaseKind,
mode: Exclude<LocalReconciliationSealedDatabaseMode, 'manual_required'>,
cacheKiB: 2_048 | 8_192,
) => void;
readonly afterDatabaseClose?: (
kind: LocalReconciliationSealedDatabaseKind,
) => void;
}
function configurationError(message: string, cause?: unknown): never {
throw new LocalDeploymentConfigurationError(message, { cause });
}
function validateSealedDirectory(directory: string, uid: number): void {
let stat: fs.Stats;
try {
stat = fs.lstatSync(directory);
} catch (error) {
return configurationError('sealed capture assets are unavailable', error);
}
if (
!stat.isDirectory() ||
stat.isSymbolicLink() ||
stat.uid !== uid ||
(stat.mode & 0o777) !== 0o500 ||
fs.realpathSync(directory) !== directory
) {
configurationError('sealed capture assets identity drifted');
}
}
function hashDescriptor(descriptor: number, bytes: number): string {
const hash = crypto.createHash('sha256');
const buffer = Buffer.allocUnsafe(HASH_BUFFER_BYTES);
let offset = 0;
while (offset < bytes) {
const count = fs.readSync(
descriptor,
buffer,
0,
Math.min(buffer.byteLength, bytes - offset),
offset,
);
if (count < 1) configurationError('sealed capture asset read stalled');
hash.update(buffer.subarray(0, count));
offset += count;
}
return hash.digest('hex');
}
function assetFingerprint(
assetsDirectory: string,
asset: LocalReconciliationCaptureManifest['assets'][number],
uid: number,
): Readonly<Record<string, string | number>> {
const filePath = path.join(
assetsDirectory,
localReconciliationCaptureAssetFileName(asset.logicalName),
);
let descriptor: number | undefined;
try {
const pathStat = fs.lstatSync(filePath, { bigint: true });
descriptor = fs.openSync(
filePath,
fs.constants.O_RDONLY | (fs.constants.O_NOFOLLOW ?? 0),
);
const opened = fs.fstatSync(descriptor, { bigint: true });
if (
!pathStat.isFile() ||
pathStat.isSymbolicLink() ||
!opened.isFile() ||
pathStat.dev !== opened.dev ||
pathStat.ino !== opened.ino ||
opened.uid !== BigInt(uid) ||
opened.nlink !== 1n ||
(opened.mode & 0o777n) !== 0o400n ||
opened.size !== BigInt(asset.bytes) ||
opened.size > BigInt(Number.MAX_SAFE_INTEGER) ||
fs.realpathSync(filePath) !== filePath
) {
configurationError('sealed capture asset identity drifted');
}
const sha256 = hashDescriptor(descriptor, Number(opened.size));
const after = fs.fstatSync(descriptor, { bigint: true });
if (
sha256 !== asset.sha256 ||
opened.dev !== after.dev ||
opened.ino !== after.ino ||
opened.uid !== after.uid ||
opened.gid !== after.gid ||
opened.mode !== after.mode ||
opened.nlink !== after.nlink ||
opened.size !== after.size ||
opened.mtimeNs !== after.mtimeNs ||
opened.ctimeNs !== after.ctimeNs
) {
configurationError('sealed capture asset drifted while reading');
}
return Object.freeze({
logicalName: asset.logicalName,
device: opened.dev.toString(),
inode: opened.ino.toString(),
uid: Number(opened.uid),
gid: Number(opened.gid),
mode: Number(opened.mode),
links: Number(opened.nlink),
bytes: Number(opened.size),
modifiedAtNs: opened.mtimeNs.toString(),
changedAtNs: opened.ctimeNs.toString(),
sha256,
});
} catch (error) {
if (error instanceof LocalDeploymentConfigurationError) throw error;
return configurationError('sealed capture asset is unavailable', error);
} finally {
if (descriptor !== undefined) fs.closeSync(descriptor);
}
}
function fingerprint(
assetsDirectory: string,
manifest: Readonly<LocalReconciliationCaptureManifest>,
uid: number,
): string {
validateSealedDirectory(assetsDirectory, uid);
const assets = manifest.assets.map((asset) =>
assetFingerprint(assetsDirectory, asset, uid),
);
return crypto
.createHash('sha256')
.update(JSON.stringify(assets), 'utf8')
.digest('hex');
}
function topology(
manifest: Readonly<LocalReconciliationCaptureManifest>,
kind: LocalReconciliationSealedDatabaseKind,
): Readonly<LocalReconciliationSealedDatabaseTopology> {
const names = new Set(manifest.assets.map((asset) => asset.logicalName));
const wal = names.has(`${kind}-wal`);
const shm = names.has(`${kind}-shm`);
const journal = names.has(`${kind}-journal`);
if (journal) {
return Object.freeze({ kind, mode: 'manual_required', reason: 'hot_journal' });
}
if (wal !== shm) {
return Object.freeze({
kind,
mode: 'manual_required',
reason: 'unpaired_wal_shm',
});
}
return Object.freeze({
kind,
mode: wal ? 'wal_shm_readonly' : 'main_only_immutable',
reason: null,
});
}
export function inspectLocalReconciliationSealedBundle(
captureRoot: string,
captureId: string,
uid: number,
): Readonly<LocalReconciliationSealedBundle> {
validatePrivateDirectory(captureRoot, uid, 'captureRoot');
const captureDirectory = localReconciliationCaptureDirectory(
captureRoot,
captureId,
);
const assetsDirectory = path.join(captureDirectory, 'assets');
const terminal = readLocalReconciliationCaptureTerminal(
captureRoot,
captureId,
uid,
);
const fingerprintDigest = fingerprint(
assetsDirectory,
terminal.manifest,
uid,
);
return Object.freeze({
captureRoot,
captureDirectory,
assetsDirectory,
manifest: terminal.manifest,
receipt: terminal.receipt,
fingerprintDigest,
target: topology(terminal.manifest, 'target'),
legacy: topology(terminal.manifest, 'legacy'),
});
}
function databasePath(
bundle: Readonly<LocalReconciliationSealedBundle>,
kind: LocalReconciliationSealedDatabaseKind,
): string {
return path.join(bundle.assetsDirectory, `${kind}.sqlite`);
}
function configureReadOnlyDatabase(
client: DatabaseSync,
profile: LocalReconciliationCaptureManifest['profile'],
): void {
const cacheKiB = profile === 'edge' ? 2_048 : 8_192;
client.enableDefensive(true);
client.exec(
`PRAGMA trusted_schema = OFF; PRAGMA query_only = ON; PRAGMA temp_store = MEMORY; PRAGMA mmap_size = 0; PRAGMA cache_size = -${cacheKiB}`,
);
const trustedSchema = client.prepare('PRAGMA trusted_schema').get() as
| { readonly trusted_schema?: unknown }
| undefined;
const queryOnly = client.prepare('PRAGMA query_only').get() as
| { readonly query_only?: unknown }
| undefined;
const tempStore = client.prepare('PRAGMA temp_store').get() as
| { readonly temp_store?: unknown }
| undefined;
const mmapSize = client.prepare('PRAGMA mmap_size').get() as
| { readonly mmap_size?: unknown }
| undefined;
const cacheSize = client.prepare('PRAGMA cache_size').get() as
| { readonly cache_size?: unknown }
| undefined;
if (
trustedSchema?.trusted_schema !== 0 ||
queryOnly?.query_only !== 1 ||
tempStore?.temp_store !== 2 ||
mmapSize?.mmap_size !== 0 ||
cacheSize?.cache_size !== -cacheKiB
) {
configurationError('sealed SQLite read-only configuration drifted');
}
}
export function withLocalReconciliationSealedDatabase<T>(
bundle: Readonly<LocalReconciliationSealedBundle>,
kind: LocalReconciliationSealedDatabaseKind,
uid: number,
dependencies: LocalReconciliationSealedBundleReaderDependencies,
read: (client: DatabaseSync) => T,
): T | null {
const current = inspectLocalReconciliationSealedBundle(
bundle.captureRoot,
bundle.receipt.captureId,
uid,
);
if (
current.receipt.bundleDigest !== bundle.receipt.bundleDigest ||
current.fingerprintDigest !== bundle.fingerprintDigest
) {
configurationError('sealed capture bundle drifted before SQLite open');
}
const selected = kind === 'target' ? current.target : current.legacy;
if (selected.mode === 'manual_required') return null;
const cacheKiB = current.manifest.profile === 'edge' ? 2_048 : 8_192;
dependencies.beforeDatabaseOpen?.(kind, selected.mode, cacheKiB);
const mainPath = databasePath(current, kind);
const source =
selected.mode === 'main_only_immutable'
? `file:${mainPath}?immutable=1`
: mainPath;
let client: DatabaseSync | undefined;
let output: T;
try {
client = new DatabaseSync(source, {
allowExtension: false,
defensive: true,
enableDoubleQuotedStringLiterals: false,
enableForeignKeyConstraints: true,
readOnly: true,
timeout: 0,
});
configureReadOnlyDatabase(client, current.manifest.profile);
output = read(client);
} catch (error) {
if (error instanceof LocalDeploymentConfigurationError) throw error;
return configurationError('sealed SQLite inventory failed', error);
} finally {
if (client !== undefined) client.close();
dependencies.afterDatabaseClose?.(kind);
}
const after = inspectLocalReconciliationSealedBundle(
bundle.captureRoot,
bundle.receipt.captureId,
uid,
);
if (
after.receipt.bundleDigest !== current.receipt.bundleDigest ||
after.fingerprintDigest !== current.fingerprintDigest
) {
configurationError('sealed capture bundle drifted after SQLite close');
}
return output;
}
@@ -749,7 +749,9 @@ function replayResult(
const stoppedCaptureProgress =
record.state === 'target_stopped' &&
(head.state === 'reconciliation_capture_prepared' ||
head.state === 'reconciliation_captured');
head.state === 'reconciliation_captured' ||
head.state === 'reconciliation_plan_prepared' ||
head.state === 'reconciliation_planned');
if (
record.actionId !== intent.actionId ||
record.intentDigest !== intent.intentDigest ||
@@ -4,12 +4,16 @@ const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const { spawnSync } = require('node:child_process');
const { DatabaseSync } = require('node:sqlite');
const { test } = require('node:test');
const {
commitLocalReconciliationCapture,
commitLocalReconciliationPlan,
prepareLocalReconciliationCapture,
prepareLocalReconciliationPlan,
verifyLocalReconciliationCapture,
verifyLocalReconciliationPlan,
} = require('../dist/deployment/localDeployment.js');
const {
normalizeLocalReconciliationCaptureManifest,
@@ -79,6 +83,9 @@ function fixture(
{
reconciliationRequired = true,
stoppedAuthority = 'docker',
profile = 'edge',
createDefaultSidecars = true,
initializeDatabases,
mutateTarget,
} = {},
) {
@@ -112,9 +119,17 @@ function fixture(
deploymentRoot,
'local-application.json',
);
fs.writeFileSync(legacySourcePath, 'legacy-source\n', { mode: 0o600 });
fs.writeFileSync(targetDatabasePath, 'target-initial\n', { mode: 0o600 });
fs.writeFileSync(recoveryPath, 'legacy-source\n', { mode: 0o600 });
if (initializeDatabases === undefined) {
fs.writeFileSync(legacySourcePath, 'legacy-source\n', { mode: 0o600 });
fs.writeFileSync(targetDatabasePath, 'target-initial\n', { mode: 0o600 });
fs.writeFileSync(recoveryPath, 'legacy-source\n', { mode: 0o600 });
} else {
initializeDatabases({
legacySourcePath,
recoveryPath,
targetDatabasePath,
});
}
const manifestPayload = {
schemaVersion: 1,
kind: 'qinglong3-local-sqlite-adoption-manifest-fixture',
@@ -130,7 +145,7 @@ function fixture(
schemaVersion: 1,
kind: 'qinglong3-local-sqlite-activation',
state: 'prepared',
profile: 'edge',
profile,
createdAtMs: 1_000,
adoptionManifestDigest: manifestDigest,
planDigest: '2'.repeat(64),
@@ -172,19 +187,21 @@ function fixture(
mode: 0o600,
});
}
fs.writeFileSync(`${targetDatabasePath}-wal`, 'target-wal-facts\n', {
mode: 0o600,
});
fs.writeFileSync(`${legacySourcePath}-journal`, 'legacy-journal-state\n', {
mode: 0o600,
});
if (createDefaultSidecars) {
fs.writeFileSync(`${targetDatabasePath}-wal`, 'target-wal-facts\n', {
mode: 0o600,
});
fs.writeFileSync(`${legacySourcePath}-journal`, 'legacy-journal-state\n', {
mode: 0o600,
});
}
}
const commitmentPayload = {
schemaVersion: 1,
kind: 'qinglong3-local-legacy-silence-commitment',
state: 'legacy_stopped',
cutoverId,
profile: 'edge',
profile,
instanceId: 'edge-router-1',
activationDigest,
requestedAtMs: 1_100,
@@ -208,7 +225,7 @@ function fixture(
const dataCommit = createLocalDataDirectoryApplicationCommit({
mutationId: '00000000-0000-4000-8000-000000000301',
projectId: 'project-edge-router-1',
profile: 'edge',
profile,
sourceStageManifestDigest: '6'.repeat(64),
transformationDigest: '7'.repeat(64),
modelDigest: '8'.repeat(64),
@@ -228,7 +245,7 @@ function fixture(
const application = {
schema: 'qinglong/local-application-process@v4',
instanceId: 'edge-router-1',
profile: 'edge',
profile,
storage: {
mode: 'adopted',
sourcePath: legacySourcePath,
@@ -272,7 +289,7 @@ function fixture(
state: 'prepared',
bundleId: '00000000-0000-4000-8000-000000000d88',
preparedAtMs: 1_400,
profile: 'edge',
profile,
instanceId: 'edge-router-1',
cutoverId,
serviceKind: 'compose',
@@ -311,7 +328,7 @@ function fixture(
options: { deploymentRoot },
request: {
cutoverId,
profile: 'edge',
profile,
instanceId: 'edge-router-1',
expectedActivationDigest: activationDigest,
requestedAtMs: 2_000,
@@ -335,7 +352,7 @@ function fixture(
);
const reconciliation = readTargetDataReconciliationEvidenceForPaths(
{
profile: 'edge',
profile,
activationPath,
legacySourcePath,
targetDatabasePath,
@@ -346,7 +363,7 @@ function fixture(
const runCommand = {
request: {
cutoverId,
profile: 'edge',
profile,
instanceId: 'edge-router-1',
expectedActivationDigest: activationDigest,
generation: 1,
@@ -396,7 +413,7 @@ function fixture(
action: 'stop',
state: 'target_stopped',
cutoverId,
profile: 'edge',
profile,
instanceId: 'edge-router-1',
activationDigest,
generation: 1,
@@ -431,7 +448,7 @@ function fixture(
request: {
captureId: '00000000-0000-4000-8000-000000000101',
stoppedAuthority,
profile: 'edge',
profile,
instanceId: 'edge-router-1',
cutoverId,
generation: 1,
@@ -584,6 +601,131 @@ function preparedCapture(t, options) {
return { ...state, prepared, commitCommand };
}
function planningDatabaseInitializer({ unknownTargetTable = false } = {}) {
return ({ legacySourcePath, recoveryPath, targetDatabasePath }) => {
const legacy = new DatabaseSync(legacySourcePath);
legacy.exec(`
CREATE TABLE "Crontabs" (id INTEGER PRIMARY KEY, schedule TEXT NOT NULL);
CREATE TABLE "Envs" (id INTEGER PRIMARY KEY, name TEXT NOT NULL, value TEXT NOT NULL);
INSERT INTO "Crontabs" (id, schedule) VALUES (1, '0 0 * * *');
INSERT INTO "Envs" (id, name, value) VALUES (1, 'TOKEN', 'private-value');
`);
legacy.close();
fs.chmodSync(legacySourcePath, 0o600);
fs.copyFileSync(legacySourcePath, recoveryPath);
fs.chmodSync(recoveryPath, 0o600);
const target = new DatabaseSync(targetDatabasePath);
target.exec(`
CREATE TABLE "QingLong3SchemaCapabilities" (id INTEGER PRIMARY KEY);
CREATE TABLE "QingLong3TaskDefinitions" (id INTEGER PRIMARY KEY);
CREATE TABLE "Runs" (id INTEGER PRIMARY KEY);
${
unknownTargetTable
? 'CREATE TABLE "UnreviewedFacts" (id INTEGER PRIMARY KEY); INSERT INTO "UnreviewedFacts" (id) VALUES (1);'
: ''
}
`);
target.close();
fs.chmodSync(targetDatabasePath, 0o600);
};
}
function mutatePlanningTarget({ targetDatabasePath }) {
const target = new DatabaseSync(targetDatabasePath);
target.exec('INSERT INTO "QingLong3TaskDefinitions" (id) VALUES (1)');
target.close();
return Object.freeze({});
}
function preparedPlan(t, options = {}) {
const state = preparedCapture(t, {
createDefaultSidecars: options.createDefaultSidecars ?? false,
initializeDatabases:
options.initializeDatabases ?? planningDatabaseInitializer(options),
mutateTarget: options.mutateTarget ?? mutatePlanningTarget,
profile: options.profile ?? 'edge',
});
const captured = commitLocalReconciliationCapture(state.commitCommand);
const planRoot = path.join(path.dirname(state.captureRoot), 'plan-root');
fs.mkdirSync(planRoot, { mode: 0o700 });
const prepareCommand = {
schemaVersion: 1,
operation: 'local.deployment.reconciliation.plan.prepare',
options: {
deploymentRoot: state.deploymentRoot,
captureRoot: state.captureRoot,
planRoot,
allowRootService: rootAcknowledgement(),
},
request: {
planId: options.planId ?? '00000000-0000-4000-8000-000000000201',
captureId: state.command.request.captureId,
expectedBundleDigest: captured.bundleDigest,
expectedHeadDigest: captured.instanceHeadDigest,
legacyTimezone: options.legacyTimezone ?? null,
preparedAtMs: 6_000,
},
};
const planPrepared = prepareLocalReconciliationPlan(prepareCommand);
const planCommitCommand = {
schemaVersion: 1,
operation: 'local.deployment.reconciliation.plan.commit',
options: prepareCommand.options,
request: {
planId: prepareCommand.request.planId,
expectedPreparationDigest: planPrepared.preparationDigest,
committedAtMs: 7_000,
},
};
return {
...state,
captured,
planRoot,
prepareCommand,
planPrepared,
planCommitCommand,
};
}
function dockerReadSealedSqlite(assetsDirectory, mode) {
const source =
mode === 'main_only_immutable'
? 'file:/bundle/target.sqlite?immutable=1'
: '/bundle/target.sqlite';
const script = `
const crypto = require('node:crypto');
const fs = require('node:fs');
const { DatabaseSync } = require('node:sqlite');
const files = fs.readdirSync('/bundle').sort();
const snapshot = () => Object.fromEntries(files.map((name) => [name, crypto.createHash('sha256').update(fs.readFileSync('/bundle/' + name)).digest('hex')]));
const before = snapshot();
const client = new DatabaseSync(${JSON.stringify(source)}, { allowExtension: false, defensive: true, readOnly: true, timeout: 0 });
client.enableDefensive(true);
client.exec('PRAGMA trusted_schema = OFF; PRAGMA query_only = ON; PRAGMA temp_store = MEMORY; PRAGMA mmap_size = 0; PRAGMA cache_size = -2048');
const row = client.prepare('SELECT COUNT(*) AS count FROM "QingLong3TaskDefinitions"').get();
client.close();
const after = snapshot();
process.stdout.write(JSON.stringify({ count: row.count, unchanged: JSON.stringify(before) === JSON.stringify(after) }));
`;
const result = spawnSync(
'docker',
[
'run',
'--rm',
'--mount',
`type=bind,source=${assetsDirectory},target=/bundle,readonly`,
'node:24-bookworm-slim',
'node',
'-e',
script,
],
{ encoding: 'utf8' },
);
assert.equal(result.status, 0, result.stderr);
return JSON.parse(result.stdout);
}
function capturePath(state, name) {
return path.join(state.captureRoot, state.command.request.captureId, name);
}
@@ -992,6 +1134,419 @@ test('capture manifest schema v1 is rejected instead of silently upgraded', (t)
);
});
test('plan reads sealed main-only SQLite with fixed budgets and verifies without opening', (t) => {
const state = preparedPlan(t);
const beforeAssets = fs.readdirSync(
capturePath(state, 'assets'),
).map((name) => ({
name,
bytes: fs.readFileSync(capturePath(state, `assets/${name}`)),
stat: fs.statSync(capturePath(state, `assets/${name}`), { bigint: true }),
}));
const opens = [];
const committed = commitLocalReconciliationPlan(state.planCommitCommand, {
beforeDatabaseOpen(kind, mode, cacheKiB) {
opens.push({ kind, mode, cacheKiB });
},
});
assert.equal(committed.status, 'prepared');
assert.equal(committed.state, 'reconciliation_planned');
assert.deepEqual(opens, [
{ kind: 'legacy', mode: 'main_only_immutable', cacheKiB: 2048 },
{ kind: 'target', mode: 'main_only_immutable', cacheKiB: 2048 },
]);
const planPath = path.join(
state.planRoot,
state.prepareCommand.request.planId,
'plan.json',
);
const planText = fs.readFileSync(planPath, 'utf8');
const plan = JSON.parse(planText);
assert.equal(Buffer.byteLength(planText, 'utf8') <= 64 * 1024, true);
assert.deepEqual(
plan.domains.map((domain) => domain.domain),
[
'schema_lineage',
'automation',
'secret_and_config',
'run_history',
'plugin_package',
'ai_and_tool',
'identity_policy_audit',
'unknown',
],
);
assert.equal(plan.domains.length, 8);
assert.equal(plan.outcome, 'manual_required');
assert.equal(planText.includes('private-value'), false);
assert.equal(planText.includes(state.captureRoot), false);
assert.equal(planText.includes(state.legacySourcePath), false);
assert.equal(planText.includes('Crontabs'), false);
const head = readLocalCutoverInstanceHead(
state.deploymentRoot,
state.command.request.instanceId,
state.uid,
);
assert.equal(head.state, 'reconciliation_planned');
assert.equal(head.sourceRecordDigest, committed.planDigest);
const verifyCommand = {
schemaVersion: 1,
operation: 'local.deployment.reconciliation.plan.verify',
options: state.prepareCommand.options,
request: {
planId: state.prepareCommand.request.planId,
expectedPlanDigest: committed.planDigest,
},
};
let verifyOpens = 0;
assert.equal(
verifyLocalReconciliationPlan(verifyCommand, {
beforeDatabaseOpen() {
verifyOpens += 1;
},
}).status,
'verified',
);
assert.equal(verifyOpens, 0);
for (const before of beforeAssets) {
const assetPath = capturePath(state, `assets/${before.name}`);
const after = fs.statSync(assetPath, { bigint: true });
assert.equal(fs.readFileSync(assetPath).equals(before.bytes), true);
for (const key of [
'dev',
'ino',
'uid',
'gid',
'mode',
'nlink',
'size',
'mtimeNs',
'ctimeNs',
]) {
assert.equal(after[key], before.stat[key]);
}
}
});
test('plan reads a sealed WAL and SHM snapshot without changing either asset', (t) => {
let target;
const initializeDatabases = (paths) => {
planningDatabaseInitializer()(paths);
target = new DatabaseSync(paths.targetDatabasePath);
target.exec('PRAGMA journal_mode = WAL; PRAGMA wal_autocheckpoint = 0');
};
const mutateTarget = ({ targetDatabasePath }) => {
target.exec('INSERT INTO "QingLong3TaskDefinitions" (id) VALUES (1)');
fs.chmodSync(`${targetDatabasePath}-wal`, 0o600);
fs.chmodSync(`${targetDatabasePath}-shm`, 0o600);
return Object.freeze({});
};
t.after(() => {
try {
target?.close();
} catch {
// The fixture cleanup may already have invalidated the source handle.
}
});
const state = preparedPlan(t, {
initializeDatabases,
mutateTarget,
planId: '00000000-0000-4000-8000-000000000205',
});
const targetWal = captureAssetPath(state, 'target-wal');
const targetShm = captureAssetPath(state, 'target-shm');
const before = [targetWal, targetShm].map((assetPath) => ({
assetPath,
bytes: fs.readFileSync(assetPath),
stat: fs.statSync(assetPath, { bigint: true }),
}));
const opens = [];
commitLocalReconciliationPlan(state.planCommitCommand, {
beforeDatabaseOpen(kind, mode, cacheKiB) {
opens.push({ kind, mode, cacheKiB });
},
});
assert.deepEqual(opens, [
{ kind: 'legacy', mode: 'main_only_immutable', cacheKiB: 2048 },
{ kind: 'target', mode: 'wal_shm_readonly', cacheKiB: 2048 },
]);
for (const item of before) {
const after = fs.statSync(item.assetPath, { bigint: true });
assert.equal(fs.readFileSync(item.assetPath).equals(item.bytes), true);
assert.equal(after.mtimeNs, item.stat.mtimeNs);
assert.equal(after.ctimeNs, item.stat.ctimeNs);
assert.equal(after.mode, item.stat.mode);
}
});
test('standalone planning fixes each SQLite cache at 8 MiB', (t) => {
const state = preparedPlan(t, {
profile: 'standalone',
planId: '00000000-0000-4000-8000-000000000206',
});
const opens = [];
commitLocalReconciliationPlan(state.planCommitCommand, {
beforeDatabaseOpen(kind, mode, cacheKiB) {
opens.push({ kind, mode, cacheKiB });
},
});
assert.deepEqual(opens, [
{ kind: 'legacy', mode: 'main_only_immutable', cacheKiB: 8192 },
{ kind: 'target', mode: 'main_only_immutable', cacheKiB: 8192 },
]);
});
test('plan commit converges plan, receipt and head crash windows exactly', (t) => {
const planState = preparedPlan(t, {
planId: '00000000-0000-4000-8000-000000000211',
});
assert.throws(
() =>
commitLocalReconciliationPlan(planState.planCommitCommand, {
afterPlanPublished() {
throw new Error('plan crash');
},
}),
/plan crash/,
);
let replayOpens = 0;
const resumedPlan = commitLocalReconciliationPlan(
planState.planCommitCommand,
{
beforeDatabaseOpen() {
replayOpens += 1;
},
},
);
assert.equal(resumedPlan.status, 'prepared');
assert.equal(replayOpens, 0);
const receiptState = preparedPlan(t, {
planId: '00000000-0000-4000-8000-000000000212',
});
assert.throws(
() =>
commitLocalReconciliationPlan(receiptState.planCommitCommand, {
afterReceiptPublished() {
throw new Error('receipt crash');
},
}),
/receipt crash/,
);
replayOpens = 0;
assert.equal(
commitLocalReconciliationPlan(receiptState.planCommitCommand, {
beforeDatabaseOpen() {
replayOpens += 1;
},
}).status,
'prepared',
);
assert.equal(replayOpens, 0);
const headState = preparedPlan(t, {
planId: '00000000-0000-4000-8000-000000000213',
});
assert.throws(
() =>
commitLocalReconciliationPlan(headState.planCommitCommand, {
afterHeadAdvanced() {
throw new Error('head response loss');
},
}),
/head response loss/,
);
assert.equal(
commitLocalReconciliationPlan(headState.planCommitCommand).status,
'existing',
);
});
test('hot journal and unpaired sidecars become manual without SQLite open', (t) => {
const state = preparedPlan(t, {
createDefaultSidecars: true,
planId: '00000000-0000-4000-8000-000000000221',
});
let opens = 0;
const committed = commitLocalReconciliationPlan(state.planCommitCommand, {
beforeDatabaseOpen() {
opens += 1;
},
});
assert.equal(opens, 0);
assert.equal(committed.outcome, 'manual_required');
const plan = JSON.parse(
fs.readFileSync(
path.join(
state.planRoot,
state.prepareCommand.request.planId,
'plan.json',
),
'utf8',
),
);
assert.deepEqual(
plan.databases.map((database) => ({
kind: database.kind,
topology: database.topology,
opened: database.opened,
})),
[
{ kind: 'legacy', topology: 'manual_required', opened: false },
{ kind: 'target', topology: 'manual_required', opened: false },
],
);
assert.equal(
plan.domains.every(
(domain) => domain.disposition === 'manual_required',
),
true,
);
});
test('unknown target schema is summarized only by digest and requires manual review', (t) => {
const state = preparedPlan(t, {
unknownTargetTable: true,
planId: '00000000-0000-4000-8000-000000000231',
});
commitLocalReconciliationPlan(state.planCommitCommand);
const planText = fs.readFileSync(
path.join(
state.planRoot,
state.prepareCommand.request.planId,
'plan.json',
),
'utf8',
);
const plan = JSON.parse(planText);
const unknown = plan.domains.find((domain) => domain.domain === 'unknown');
assert.equal(unknown.targetTables, 1);
assert.equal(unknown.rowCountsComplete, false);
assert.equal(unknown.disposition, 'manual_required');
assert.equal(plan.outcome, 'manual_required');
assert.equal(planText.includes('UnreviewedFacts'), false);
});
test('one plan fence blocks a second plan and all rollback transitions', (t) => {
const state = preparedPlan(t, {
planId: '00000000-0000-4000-8000-000000000241',
});
const competing = structuredClone(state.prepareCommand);
competing.request.planId = '00000000-0000-4000-8000-000000000242';
assert.throws(
() => prepareLocalReconciliationPlan(competing),
/compare-and-swap/,
);
assert.throws(
() =>
advanceLocalCutoverInstanceHead(
state.identity,
state.uid,
'rollback_prepared',
1,
'f'.repeat(64),
),
/transition is invalid/,
);
assert.throws(
() =>
advanceLocalCutoverInstanceHead(
state.identity,
state.uid,
'target_active',
2,
'e'.repeat(64),
),
/transition is invalid/,
);
});
test('plan prepare recovers head response loss and CLI verify stays content-free', (t) => {
const capture = preparedCapture(t, {
createDefaultSidecars: false,
initializeDatabases: planningDatabaseInitializer(),
mutateTarget: mutatePlanningTarget,
});
const captured = commitLocalReconciliationCapture(capture.commitCommand);
const planRoot = path.join(path.dirname(capture.captureRoot), 'plan-root');
fs.mkdirSync(planRoot, { mode: 0o700 });
const prepareCommand = {
schemaVersion: 1,
operation: 'local.deployment.reconciliation.plan.prepare',
options: {
deploymentRoot: capture.deploymentRoot,
captureRoot: capture.captureRoot,
planRoot,
allowRootService: rootAcknowledgement(),
},
request: {
planId: '00000000-0000-4000-8000-000000000251',
captureId: capture.command.request.captureId,
expectedBundleDigest: captured.bundleDigest,
expectedHeadDigest: captured.instanceHeadDigest,
legacyTimezone: null,
preparedAtMs: 6_000,
},
};
assert.throws(
() =>
prepareLocalReconciliationPlan(prepareCommand, {
afterHeadPrepared() {
throw new Error('prepare response loss');
},
}),
/prepare response loss/,
);
const prepared = prepareLocalReconciliationPlan(prepareCommand);
assert.equal(prepared.status, 'prepared');
const committed = commitLocalReconciliationPlan({
schemaVersion: 1,
operation: 'local.deployment.reconciliation.plan.commit',
options: prepareCommand.options,
request: {
planId: prepareCommand.request.planId,
expectedPreparationDigest: prepared.preparationDigest,
committedAtMs: 7_000,
},
});
const verifyCommand = {
schemaVersion: 1,
operation: 'local.deployment.reconciliation.plan.verify',
options: prepareCommand.options,
request: {
planId: prepareCommand.request.planId,
expectedPlanDigest: committed.planDigest,
},
};
const commandPath = path.join(capture.deploymentRoot, 'plan-verify.json');
fs.writeFileSync(commandPath, `${JSON.stringify(verifyCommand)}\n`, {
mode: 0o600,
});
const cli = spawnSync(
process.execPath,
[
path.join(__dirname, '../dist/deployment/localDeploymentCli.js'),
'reconciliation-plan-verify',
'--command-file',
commandPath,
],
{ encoding: 'utf8' },
);
assert.equal(cli.status, 0, cli.stderr);
assert.equal(JSON.parse(cli.stdout).status, 'verified');
assert.equal(cli.stdout.includes(capture.captureRoot), false);
assert.equal(cli.stdout.includes(planRoot), false);
assert.equal(cli.stdout.includes(capture.targetDatabasePath), false);
const overlapping = structuredClone(prepareCommand);
overlapping.request.planId = '00000000-0000-4000-8000-000000000252';
overlapping.options.planRoot = capture.captureRoot;
assert.throws(
() => prepareLocalReconciliationPlan(overlapping),
/must not overlap/,
);
});
test(
'real stopped Docker target produces an independently verified bundle',
{ skip: process.env.QL3_RECONCILIATION_DOCKER_GATE !== '1' },
@@ -1003,6 +1558,8 @@ test(
});
});
const state = preparedCapture(t, {
createDefaultSidecars: false,
initializeDatabases: planningDatabaseInitializer(),
mutateTarget({ root }) {
const created = spawnSync(
'docker',
@@ -1015,7 +1572,7 @@ test(
'node:24-bookworm-slim',
'node',
'-e',
"require('node:fs').writeFileSync('/capture-fixture/database.ql3.sqlite','target-docker-mutated\\n')",
"const { DatabaseSync } = require('node:sqlite'); const db = new DatabaseSync('/capture-fixture/database.ql3.sqlite'); db.exec('INSERT INTO \\\"QingLong3TaskDefinitions\\\" (id) VALUES (1)'); db.close()",
],
{ encoding: 'utf8' },
);
@@ -1051,9 +1608,12 @@ test(
},
});
const committed = commitLocalReconciliationCapture(state.commitCommand);
assert.equal(
fs.readFileSync(captureAssetPath(state, 'target-main'), 'utf8'),
'target-docker-mutated\n',
assert.deepEqual(
dockerReadSealedSqlite(
capturePath(state, 'assets'),
'main_only_immutable',
),
{ count: 1, unchanged: true },
);
const verified = verifyLocalReconciliationCapture({
schemaVersion: 1,
@@ -1068,3 +1628,37 @@ test(
assert.equal(verified.bundleDigest, committed.bundleDigest);
},
);
test(
'real Docker reads sealed WAL and SHM without changing the bundle',
{ skip: process.env.QL3_RECONCILIATION_DOCKER_GATE !== '1' },
(t) => {
let target;
const state = preparedCapture(t, {
createDefaultSidecars: false,
initializeDatabases(paths) {
planningDatabaseInitializer()(paths);
target = new DatabaseSync(paths.targetDatabasePath);
target.exec('PRAGMA journal_mode = WAL; PRAGMA wal_autocheckpoint = 0');
},
mutateTarget({ targetDatabasePath }) {
target.exec('INSERT INTO "QingLong3TaskDefinitions" (id) VALUES (1)');
fs.chmodSync(`${targetDatabasePath}-wal`, 0o600);
fs.chmodSync(`${targetDatabasePath}-shm`, 0o600);
return Object.freeze({});
},
});
t.after(() => {
try {
target?.close();
} catch {
// The fixture cleanup may already have invalidated the source handle.
}
});
commitLocalReconciliationCapture(state.commitCommand);
assert.deepEqual(
dockerReadSealedSqlite(capturePath(state, 'assets'), 'wal_shm_readonly'),
{ count: 1, unchanged: true },
);
},
);
@@ -16,7 +16,7 @@ const test = spawnSync(
[
'--test',
'--test-name-pattern',
'real stopped Docker target',
'real .*Docker',
path.join(
root,
'packages/ql3-local-owner-cli/test/reconciliationCapturePrepare.test.cjs',
+2 -2
View File
@@ -207,10 +207,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
rootSourceFileRoles: localOwnerCli.rootSourceFileRoles,
},
{
sourceFiles: 141,
sourceFiles: 146,
rootSourceFiles: 1,
rootSourceLines: 50,
nestedSourceFiles: 140,
nestedSourceFiles: 145,
rootSourceFileRoles: { 'cli.ts': 'binary_entry' },
},
);