feat(local): preserve terminal run history

This commit is contained in:
whyour
2026-08-23 13:30:18 +08:00
parent 5f62f2f724
commit 42262094cc
13 changed files with 1938 additions and 43 deletions
+20 -2
View File
@@ -6,10 +6,28 @@
- 目标版本:QingLong 3.x
- 作者:QingLong Maintainers
- 创建日期:2026-07-17
- 最后更新:2026-08-22
- 最后更新:2026-08-23
- 讨论范围:架构与演进路线,不包含最终 UI 视觉方案
最新增量证据(2026-08-22):
最新增量证据(2026-08-23):
- D-396/ADR-0490(进行中):Run History 不再只有永久 `manual_external`,但也没有被错误实现为 Legacy 日志到 3.0 Run ledger 的回灌。
新的 Local adapter 以 ADR-0482 sealed capture bundle 作为 append-only 保全资产:Legacy history 必须逐事实选择 `retain_both`Target history
必须选择 `retain_target`receipt 只绑定 signed review、application、bundle fingerprint、领域 inventory 与有界 fact counts,不保存表名、Run ID、
状态值、日志、路径或 row body。Target 只有在所有 Run 已 `succeeded|failed|cancelled|timed_out` 且有完成时间、没有 active Attempt、所有
StepRun 已终态时才从 `blocked/historical_integrity_required` 提升为 `required/historical_preservation_required`active、waiting、retry、lost、
缺列或 schema drift 继续失败关闭,人工 decision 不能升级。
实现位于既有 `local-owner-cli/deployment/reconciliation/application/run-history/{contract,evidence,coordinator}`,新增短生命周期
`reconciliation-run-history-preserve|verify`,不新增 instance state、package、dependency、binary、SQL、daemon 或 `src/` 根平铺。preservation
receipt no-replace 发布并封为 `0400/0500`,绑定 exact application/plan/head;跨领域 completion 以兼容 v1 的 schema v2 消费
`run_history_preservation``adapterCount` 可为 `0|1|2`,因此 Automation 与 Run History 可以同时被证明而不产生 adapter 状态组合爆炸。
当前聚焦 Run History `2/2`、旧 completion v1 `2/2`,已覆盖 publication/seal response loss、active Run、review escalation、receipt tamper、v2
completion 与 content-free CLI。完整 reconciliation 为 `48/46/2/0`Local Owner 为 `273/266/7/0`tracked backend 为
`1561/1559/2/0`18-package clean build/test 为 `2919/2897/22/0`package/dependency/source boundary、service-manager bridge、Edge import
与十四档 Local artifact audit 全部 compatible。workspace 保持 18 packages、`singleSourcePackages=[]``shallowSourcePackages=[]`Local Owner
`175/174/1`source/nested/root binary entry);基础 Edge/Standalone 制品仍为 `2,611,978 / 2,612,056 bytes`、319 files、58 modules
一次性 Owner authority 未进入低资源常驻 runtime。远程 CI 尚未验证,ADR-0490 保持 Proposed。
- D-393/ADR-0486/ADR-0487(进行中):首个 Automation adapter 已先建立独立的有界逐行 plan fence,而没有提前取得 DML authority。既有
Legacy Crontab classifier 被复用于 exact sealed Legacy source;每行只记录 source/candidate digest、classification/reasons、proposed
@@ -0,0 +1,110 @@
# ADR-0490Run History 终态保全与跨领域完成证明
- 状态:Proposed(D-396 实现与本地完整门禁已完成,等待远程 CI)
- 日期:2026-08-23
- 决策:D-396
- 关联:ADR-0482、ADR-0483、ADR-0484、ADR-0485、ADR-0488、ADR-0489
## 背景
QingLong 2.x 的 `CrontabStats``Logs` 与 QingLong 3.0 的 `Runs``RunAttempts``RunEvents``StepRuns` 等对象都属于运行历史,但两代模型并不具备可证明的一一映射。把旧统计或日志行直接插入 3.0 Run ledger 会伪造 Run identity、event sequence、Attempt/Step 关系与执行来源;用 target 覆盖 Legacy 又会违反 append-only 历史边界。
D-391 因而把 Run History 全部标为 blockedD-394 completion 也只能拒绝该领域。这个保守默认能避免数据损坏,却使一个已经停稳、只要求保留两侧历史的迁移永远不能形成完成证明。D-396 必须提供真正可完成、但不冒充历史转换的领域 adapter。
## 决策
### 1. Run History adapter 是保全证明,不是数据导入器
Legacy 与 target 的历史继续保存在 ADR-0482 已封存的 capture bundle 中。adapter 不执行 INSERT、UPDATE、DELETE、checkpoint、日志复制、格式转换或外部上传,也不把旧行回灌到 3.0 Run ledger。
合法裁决固定为:
- Legacy Run History 的每个 required fact 必须选择 `retain_both / preserve_both`
- Target Run History 的每个 required fact 必须选择 `retain_target / preserve_target`
- `adopt_legacy``exclude_legacy``defer``manual_external` 或任意 blocked fact 都不能形成保全证明。
receipt 只绑定 bundle、review、application、领域 inventory 与 fact counts,不保存表名、Run ID、状态值、日志、路径或 row body。
### 2. 只有确定终态的 target history 可被提升为 required
Legacy capture 已由 stopped source fence 固定,因此其 Run History 可进入 `historical_preservation_required`。Target 必须从 exact sealed SQLite 重新证明:
- `Runs` 存在 `status``finished_at_ms`,所有 Run 只能是 `succeeded|failed|cancelled|timed_out`,且具备完成时间;
- 若存在 `RunAttempts`,不能有 `claimed|starting|running` Attempt
- 若存在 `StepRuns`,所有 Step 只能是 `succeeded|failed|skipped|cancelled|timed_out`,且具备完成时间;
- 缺少 canonical 列、active/waiting/retry/lost Run、active Attempt、pending/waiting/running/lost Step 或查询漂移继续保持 `blocked / historical_integrity_required`
该判断同时用于 diagnostics、review commit 和 adapter replayoperator 不能通过修改私有 decision file 把 active Run 提升为可保全。
### 3. 新增独立短生命周期 preserve/verify 命令
既有 `ql3-local-deploy` 增加:
```text
local.deployment.reconciliation.run-history.preserve
local.deployment.reconciliation.run-history.verify
```
preserve 重新打开 sealed application、plan、bundle 与 owner-private decision file,重放全部 canonical facts,并要求 review authorization 的 decision file digest/count 完全一致。它只允许 Run History domain 为 `adapter_required`,并绑定 exact `reconciliation_application_planned``reconciliation_automation_applied` source head。
每个 preservation directory 最多只含 `receipt.json`,以 no-replace publication、file/directory fsync 和 exact replay 收敛,最终文件/目录权限为 `0400/0500`catalog 上限 64。receipt 发布或封存后的 response loss 可由相同命令继续完成,不覆盖冲突证据。
preserve 不增加 instance head 状态。它本身不授予 restart;若在发布期间另一个 adapter 推进 head,该证明会因 source-head digest 脱离而不能被 completion 消费,调用方必须在新的终态 head 上使用新 preservation ID 重新证明。这避免了多个独立 adapter 状态的组合爆炸。
### 4. Completion schema v2 消费第二种 adapter evidence
completion command/receipt 保留 v1 兼容语义;只有携带 Run History preservation 时使用 schema v2。固定八领域证据新增:
```text
run_history + adapter_required + run_history_preservation
```
completion 重新验证 sealed preservation、原 decision stream、bundle fingerprint、application plan 与 source head;调用方只能提交 preservation ID 和 expected digest,不能自报 domain evidence。`adapterCount``0|1` 扩展为 `0|1|2`,允许 Automation apply 与 Run History preservation 同时作为已证明 adapter。
完成时间不得早于 application commit、Automation apply 或 history preservation 中任一最新证据。v1 receipt 仍按原规则读取和验证;v2 receipt 必须实际包含 Run History preservation,不能用版本号绕过领域证明。
### 5. 低资源与集群边界
实现内聚于既有 `@qinglong/local-owner-cli/src/deployment/reconciliation/application/run-history/`,不新增 workspace package、production dependency、binary、daemon、timer、watcher、listener、socket、SQL migration 或 `src/` 根平铺。
Edge/Standalone 每次最多打开一个 sealed SQLite handle,状态判定均为 `LIMIT 1`decision file 继续使用 64 KiB 流式缓冲;长期新增资产只有一个小型 content-free receipt,不复制数据库或日志。
本 ADR 只定义 Local SQLite 保全。Cluster/PostgreSQL 的 Run History 必须使用数据库 snapshot、事务一致性、租约/运行收敛与 HA retention 的独立证明,不能复用本机路径或把 Local capture bundle 当成集群历史 authority。
## 被拒绝的替代方案
### 把 Legacy 统计和日志导入 3.0 Run ledger
拒绝。没有 Run/Attempt/Event/Step identity 与顺序证明,导入会制造看似完整但语义虚假的运行记录。
### 只保存 source database 的 SHA-256
拒绝。整库 hash 会被其他 adapter 对无关表的合法写入破坏,也不能表达签名的 append-only 保留选择。D-396 绑定 sealed bundle 与 Run History domain inventory。
### 每增加一个 adapter 就新增 instance head 状态
拒绝。八领域会产生顺序与组合爆炸。无 DML 的 history proof 绑定当前 source head,最终由跨领域 completion 一次 CAS。
### 把 active/lost Run 当成历史
拒绝。`running|waiting_approval|retry_wait|lost` 都可能仍需执行、恢复或人工裁决,封存为完成历史会丢失运行 authority。
## 验收条件
1. terminal Legacy/Target history 从 blocked 精确提升为 requiredactive/inconclusive target history 始终 blocked。
2. 只有 `legacy retain_both + target retain_target` 的完整 signed canonical decision stream 可发布 receipt。
3. receipt 不包含名称、路径、Run ID、状态值、日志或 row body,且收敛为 `0400/0500`
4. receipt publication/seal response loss exact replay;替换、篡改、额外 catalog、stale head 与 decision drift 全部失败关闭。
5. completion v2 重新验证 preservation 并生成固定八领域 evidencecompletion v1 保持兼容。
6. preserve/verify 无 SQLite DML、数据库复制、service、Docker、network 或后台对象。
7. 聚焦、完整 Local Owner、tracked backend、18-package、package/dependency/source boundary、artifact/Edge import 与远程 CI 全部通过后,本 ADR 才能转为 Accepted。
## 当前证据
- 聚焦 Run History`2/2`,覆盖终态 preserve/verify、两处 response-loss replay、v2 completion、CLI content-free、active Run blocked、人工越权拒绝与 sealed receipt 篡改。
- 既有 completion v1 聚焦回归:`2/2`
- 完整 reconciliation`48 total / 46 pass / 2 conditional Docker skip / 0 fail`;完整 Local Owner`273 / 266 / 7 / 0`
- tracked backend`1561 total / 1559 pass / 2 conditional skip / 0 fail`18-package clean build/test`2919 / 2897 / 22 / 0`
- package/dependency/source boundary、service-manager bridge import、Edge import 与十四档 Local artifact audit 全部 compatibleworkspace 仍为 18 packages、`singleSourcePackages=[]``shallowSourcePackages=[]`Local Owner 为 `175 source / 174 nested / 1 root binary entry`
- 基础 Edge/Standalone 制品保持 `2,611,978 / 2,612,056 bytes`、319 files、58 loaded modules;本阶段未把一次性 Owner authority 带入常驻小设备 runtime。
- 远程 CI 尚未验证,因此本 ADR 仍为 Proposed。
@@ -154,6 +154,13 @@ import {
verifyLocalReconciliationAutomationApply,
verifyLocalReconciliationAutomationApplyCommandFile,
} from './reconciliation/application/automation/applyCoordinator';
import {
preserveLocalReconciliationRunHistory,
preserveLocalReconciliationRunHistoryCommandFile,
readLocalReconciliationRunHistoryTerminal,
verifyLocalReconciliationRunHistory,
verifyLocalReconciliationRunHistoryCommandFile,
} from './reconciliation/application/run-history/coordinator';
import {
completeLocalReconciliation,
completeLocalReconciliationCommandFile,
@@ -199,6 +206,11 @@ export {
verifyLocalReconciliationAutomationApplyCommandFile,
rollbackLocalReconciliationAutomationApply,
rollbackLocalReconciliationAutomationApplyCommandFile,
preserveLocalReconciliationRunHistory,
preserveLocalReconciliationRunHistoryCommandFile,
readLocalReconciliationRunHistoryTerminal,
verifyLocalReconciliationRunHistory,
verifyLocalReconciliationRunHistoryCommandFile,
completeLocalReconciliation,
completeLocalReconciliationCommandFile,
verifyLocalReconciliationCompletion,
@@ -212,6 +224,8 @@ export {
type LocalReconciliationCompletionAutomationBinding,
type LocalReconciliationCompletionAutomationOptions,
type LocalReconciliationCompletionOptions,
type LocalReconciliationCompletionRunHistoryBinding,
type LocalReconciliationCompletionRunHistoryOptions,
type LocalReconciliationCompletionResult,
type LocalReconciliationCompletionVerifyCommand,
} from './reconciliation/completion/contract';
@@ -234,6 +248,23 @@ export {
} from './reconciliation/application/automation/applyContract';
export { type LocalReconciliationAutomationApplyDependencies } from './reconciliation/application/automation/applyCoordinator';
export {
normalizeLocalReconciliationRunHistoryPreserveCommand,
normalizeLocalReconciliationRunHistoryVerifyCommand,
type LocalReconciliationRunHistoryOptions,
type LocalReconciliationRunHistoryPreserveCommand,
type LocalReconciliationRunHistoryResult,
type LocalReconciliationRunHistoryVerifyCommand,
} from './reconciliation/application/run-history/contract';
export {
type LocalReconciliationRunHistoryDependencies,
type LocalReconciliationRunHistoryTerminal,
} from './reconciliation/application/run-history/coordinator';
export {
normalizeLocalReconciliationRunHistoryPreservationReceipt,
type LocalReconciliationRunHistoryPreservationReceipt,
} from './reconciliation/application/run-history/evidence';
export {
normalizeLocalReconciliationApplicationCommitCommand,
normalizeLocalReconciliationApplicationPrepareCommand,
@@ -33,6 +33,8 @@ import {
applyLocalReconciliationAutomationCommandFile,
verifyLocalReconciliationAutomationApplyCommandFile,
rollbackLocalReconciliationAutomationApplyCommandFile,
preserveLocalReconciliationRunHistoryCommandFile,
verifyLocalReconciliationRunHistoryCommandFile,
completeLocalReconciliationCommandFile,
verifyLocalReconciliationCompletionCommandFile,
writeLocalReconciliationReviewDiagnosticsCommandFile,
@@ -50,7 +52,7 @@ import {
} from './localDeployment';
const USAGE =
'Usage: ql3-local-deploy <prepare|adopted-prepare|adopted-verify|status|service-intent-prepare|service-outcome-consume|service-cutover-consume|service-legacy-rollback-prepare|service-legacy-rollback-authorize|service-legacy-rollback-consume|cutover-legacy-stop|cutover-target-start|cutover-target-restart|cutover-target-stop|cutover-legacy-rollback-prepare|cutover-legacy-rollback-commit|cutover-legacy-readiness-probe|cutover-manual-diagnose|cutover-manual-resolution-prepare|cutover-manual-resolution-commit|reconciliation-capture-prepare|reconciliation-capture-commit|reconciliation-capture-verify|reconciliation-plan-prepare|reconciliation-plan-commit|reconciliation-plan-verify|reconciliation-review-prepare|reconciliation-review-diagnostics|reconciliation-review-commit|reconciliation-review-verify|reconciliation-application-prepare|reconciliation-application-commit|reconciliation-application-verify|reconciliation-automation-plan|reconciliation-automation-verify|reconciliation-automation-decision-prepare|reconciliation-automation-decision-commit|reconciliation-automation-decision-verify|reconciliation-automation-apply|reconciliation-automation-apply-verify|reconciliation-automation-apply-rollback|reconciliation-complete|reconciliation-complete-verify|compose-revision|compose-preflight|compose-apply|compose-restore-prepare|compose-restore-commit|compose-evidence-collect-prepare|compose-evidence-collect-commit> --command-file /absolute/private-command.json';
'Usage: ql3-local-deploy <prepare|adopted-prepare|adopted-verify|status|service-intent-prepare|service-outcome-consume|service-cutover-consume|service-legacy-rollback-prepare|service-legacy-rollback-authorize|service-legacy-rollback-consume|cutover-legacy-stop|cutover-target-start|cutover-target-restart|cutover-target-stop|cutover-legacy-rollback-prepare|cutover-legacy-rollback-commit|cutover-legacy-readiness-probe|cutover-manual-diagnose|cutover-manual-resolution-prepare|cutover-manual-resolution-commit|reconciliation-capture-prepare|reconciliation-capture-commit|reconciliation-capture-verify|reconciliation-plan-prepare|reconciliation-plan-commit|reconciliation-plan-verify|reconciliation-review-prepare|reconciliation-review-diagnostics|reconciliation-review-commit|reconciliation-review-verify|reconciliation-application-prepare|reconciliation-application-commit|reconciliation-application-verify|reconciliation-automation-plan|reconciliation-automation-verify|reconciliation-automation-decision-prepare|reconciliation-automation-decision-commit|reconciliation-automation-decision-verify|reconciliation-automation-apply|reconciliation-automation-apply-verify|reconciliation-automation-apply-rollback|reconciliation-run-history-preserve|reconciliation-run-history-verify|reconciliation-complete|reconciliation-complete-verify|compose-revision|compose-preflight|compose-apply|compose-restore-prepare|compose-restore-commit|compose-evidence-collect-prepare|compose-evidence-collect-commit> --command-file /absolute/private-command.json';
async function main(argv: readonly string[]): Promise<void> {
if (argv.length === 1 && (argv[0] === '--help' || argv[0] === '-h')) {
@@ -100,6 +102,8 @@ async function main(argv: readonly string[]): Promise<void> {
argv[0] !== 'reconciliation-automation-apply' &&
argv[0] !== 'reconciliation-automation-apply-verify' &&
argv[0] !== 'reconciliation-automation-apply-rollback' &&
argv[0] !== 'reconciliation-run-history-preserve' &&
argv[0] !== 'reconciliation-run-history-verify' &&
argv[0] !== 'reconciliation-complete' &&
argv[0] !== 'reconciliation-complete-verify' &&
argv[0] !== 'compose-revision' &&
@@ -214,6 +218,10 @@ async function main(argv: readonly string[]): Promise<void> {
? verifyLocalReconciliationAutomationApplyCommandFile(argv[2]!)
: argv[0] === 'reconciliation-automation-apply-rollback'
? rollbackLocalReconciliationAutomationApplyCommandFile(argv[2]!)
: argv[0] === 'reconciliation-run-history-preserve'
? preserveLocalReconciliationRunHistoryCommandFile(argv[2]!)
: argv[0] === 'reconciliation-run-history-verify'
? verifyLocalReconciliationRunHistoryCommandFile(argv[2]!)
: argv[0] === 'reconciliation-complete'
? completeLocalReconciliationCommandFile(argv[2]!)
: argv[0] === 'reconciliation-complete-verify'
@@ -0,0 +1,289 @@
import path from 'node:path';
import { currentIdentity } from '../../../foundation/contract';
import { LocalDeploymentConfigurationError } from '../../../foundation/error';
const DIGEST = /^[0-9a-f]{64}$/;
const UUID_V4 =
/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/;
const SAFE_PATH = /^\/[A-Za-z0-9._/@-]+$/;
export interface LocalReconciliationRunHistoryOptions {
readonly deploymentRoot: string;
readonly applicationRoot: string;
readonly runHistoryRoot: string;
readonly allowRootService: boolean;
}
export interface LocalReconciliationRunHistoryPreserveCommand {
readonly schemaVersion: 1;
readonly operation: 'local.deployment.reconciliation.run-history.preserve';
readonly options: Readonly<LocalReconciliationRunHistoryOptions>;
readonly request: Readonly<{
preservationId: string;
applicationId: string;
expectedApplicationPlanDigest: string;
expectedHeadDigest: string;
decisionFilePath: string;
preservedAtMs: number;
}>;
}
export interface LocalReconciliationRunHistoryVerifyCommand {
readonly schemaVersion: 1;
readonly operation: 'local.deployment.reconciliation.run-history.verify';
readonly options: Readonly<LocalReconciliationRunHistoryOptions>;
readonly request: Readonly<{
preservationId: string;
applicationId: string;
expectedPreservationDigest: string;
decisionFilePath: string;
}>;
}
export interface LocalReconciliationRunHistoryResult {
readonly schemaVersion: 1;
readonly operation:
| LocalReconciliationRunHistoryPreserveCommand['operation']
| LocalReconciliationRunHistoryVerifyCommand['operation'];
readonly status: 'preserved' | 'existing' | 'verified';
readonly state: 'reconciliation_run_history_preserved';
readonly preservationId: string;
readonly applicationId: string;
readonly preservationDigest: string;
readonly legacyFactCount: number;
readonly targetFactCount: number;
}
function fail(message: string): never {
throw new LocalDeploymentConfigurationError(
`reconciliation run history ${message}`,
);
}
function record(value: unknown, label: string): Record<string, unknown> {
if (!value || typeof value !== 'object' || Array.isArray(value)) {
fail(`${label} must be an object`);
}
return value as Record<string, unknown>;
}
function exact(
value: Record<string, unknown>,
keys: readonly string[],
label: string,
): void {
const actual = Object.keys(value).sort();
const expected = [...keys].sort();
if (
actual.length !== expected.length ||
actual.some((key, index) => key !== expected[index])
) {
fail(`${label} shape is invalid`);
}
}
function safePath(value: unknown, label: string): string {
if (
typeof value !== 'string' ||
!path.isAbsolute(value) ||
path.parse(value).root === value ||
path.normalize(value) !== value ||
value.includes('\0') ||
value.includes('//') ||
!SAFE_PATH.test(value) ||
Buffer.byteLength(value, 'utf8') > 4_096
) {
fail(`${label} must be a safe non-root absolute path`);
}
return value;
}
function overlaps(left: string, right: string): boolean {
const relative = path.relative(left, right);
return (
relative === '' ||
(!relative.startsWith('..') && !path.isAbsolute(relative))
);
}
function identifier(value: unknown, pattern: RegExp, label: string): string {
if (typeof value !== 'string' || !pattern.test(value))
fail(`${label} is invalid`);
return value;
}
function normalizeOptions(
value: unknown,
): Readonly<LocalReconciliationRunHistoryOptions> {
const selected = record(value, 'options');
exact(
selected,
['allowRootService', 'applicationRoot', 'deploymentRoot', 'runHistoryRoot'],
'options',
);
if (
typeof selected.allowRootService !== 'boolean' ||
(currentIdentity().uid === 0) !== selected.allowRootService
) {
fail('command identity is invalid');
}
const roots = [
safePath(selected.deploymentRoot, 'deploymentRoot'),
safePath(selected.applicationRoot, 'applicationRoot'),
safePath(selected.runHistoryRoot, 'runHistoryRoot'),
];
for (let left = 0; left < roots.length; left += 1) {
for (let right = left + 1; right < roots.length; right += 1) {
if (
overlaps(roots[left]!, roots[right]!) ||
overlaps(roots[right]!, roots[left]!)
) {
fail('authority roots overlap');
}
}
}
return Object.freeze({
deploymentRoot: roots[0]!,
applicationRoot: roots[1]!,
runHistoryRoot: roots[2]!,
allowRootService: selected.allowRootService,
});
}
function command(value: unknown, operation: string) {
const selected = record(value, 'command');
exact(
selected,
['operation', 'options', 'request', 'schemaVersion'],
'command',
);
if (selected.schemaVersion !== 1 || selected.operation !== operation) {
fail('command version or operation is invalid');
}
return Object.freeze({
options: normalizeOptions(selected.options),
request: record(selected.request, 'request'),
});
}
function decisionFilePath(
value: unknown,
options: Readonly<LocalReconciliationRunHistoryOptions>,
): string {
const selected = safePath(value, 'decisionFilePath');
for (const root of [
options.deploymentRoot,
options.applicationRoot,
options.runHistoryRoot,
]) {
if (overlaps(root, selected) || overlaps(selected, root)) {
fail('decisionFilePath overlaps an authority root');
}
}
return selected;
}
export function normalizeLocalReconciliationRunHistoryPreserveCommand(
value: unknown,
): Readonly<LocalReconciliationRunHistoryPreserveCommand> {
const selected = command(
value,
'local.deployment.reconciliation.run-history.preserve',
);
exact(
selected.request,
[
'applicationId',
'decisionFilePath',
'expectedApplicationPlanDigest',
'expectedHeadDigest',
'preservationId',
'preservedAtMs',
],
'request',
);
if (
!Number.isSafeInteger(selected.request.preservedAtMs) ||
(selected.request.preservedAtMs as number) < 0
) {
fail('preservedAtMs is invalid');
}
return Object.freeze({
schemaVersion: 1,
operation: 'local.deployment.reconciliation.run-history.preserve',
options: selected.options,
request: Object.freeze({
preservationId: identifier(
selected.request.preservationId,
UUID_V4,
'preservationId',
),
applicationId: identifier(
selected.request.applicationId,
UUID_V4,
'applicationId',
),
expectedApplicationPlanDigest: identifier(
selected.request.expectedApplicationPlanDigest,
DIGEST,
'expectedApplicationPlanDigest',
),
expectedHeadDigest: identifier(
selected.request.expectedHeadDigest,
DIGEST,
'expectedHeadDigest',
),
decisionFilePath: decisionFilePath(
selected.request.decisionFilePath,
selected.options,
),
preservedAtMs: selected.request.preservedAtMs as number,
}),
});
}
export function normalizeLocalReconciliationRunHistoryVerifyCommand(
value: unknown,
): Readonly<LocalReconciliationRunHistoryVerifyCommand> {
const selected = command(
value,
'local.deployment.reconciliation.run-history.verify',
);
exact(
selected.request,
[
'applicationId',
'decisionFilePath',
'expectedPreservationDigest',
'preservationId',
],
'request',
);
return Object.freeze({
schemaVersion: 1,
operation: 'local.deployment.reconciliation.run-history.verify',
options: selected.options,
request: Object.freeze({
preservationId: identifier(
selected.request.preservationId,
UUID_V4,
'preservationId',
),
applicationId: identifier(
selected.request.applicationId,
UUID_V4,
'applicationId',
),
expectedPreservationDigest: identifier(
selected.request.expectedPreservationDigest,
DIGEST,
'expectedPreservationDigest',
),
decisionFilePath: decisionFilePath(
selected.request.decisionFilePath,
selected.options,
),
}),
});
}
@@ -0,0 +1,623 @@
import fs from 'node:fs';
import path from 'node:path';
import { readPrivateLocalCommandFile } from '@qinglong/local-command-file';
import { currentIdentity } from '../../../foundation/contract';
import { LocalDeploymentConfigurationError } from '../../../foundation/error';
import {
preflightPublishedFile,
publishExactFile,
validatePrivateDirectory,
} from '../../../foundation/files';
import { readLocalCutoverInstanceHead } from '../../../cutover/instanceLineage';
import { readLocalReconciliationPlanTerminal } from '../../planning/preparation';
import {
assertLocalReconciliationReviewDecisionMatchesFact,
withLocalReconciliationReviewDecisionFile,
} from '../../review/decisionFile';
import { visitLocalReconciliationDiagnosticFacts } from '../../review/diagnostics';
import { readLocalReconciliationReviewTerminalJson } from '../../review/terminalEvidence';
import {
withLocalReconciliationSealedDatabase,
type LocalReconciliationSealedBundleReaderDependencies,
} from '../../sealed-bundle/reader';
import {
readLocalReconciliationApplicationTerminal,
type LocalReconciliationApplicationTerminal,
} from '../coordinator';
import {
normalizeLocalReconciliationRunHistoryPreserveCommand,
normalizeLocalReconciliationRunHistoryVerifyCommand,
type LocalReconciliationRunHistoryOptions,
type LocalReconciliationRunHistoryPreserveCommand,
type LocalReconciliationRunHistoryResult,
type LocalReconciliationRunHistoryVerifyCommand,
} from './contract';
import {
buildLocalReconciliationRunHistoryPreservationReceipt,
localReconciliationRunHistoryReceiptContents,
normalizeLocalReconciliationRunHistoryPreservationReceipt,
type LocalReconciliationRunHistoryPreservationReceipt,
} from './evidence';
const MAX_PRESERVATIONS = 64;
const MAX_RECEIPT_BYTES = 64 * 1024;
interface RunHistoryPaths {
readonly root: string;
readonly receipt: string;
}
interface RunHistoryAuthority {
readonly application: Readonly<LocalReconciliationApplicationTerminal>;
readonly runHistoryInventoryDigest: string;
readonly bundleDigest: string;
readonly bundleFingerprintDigest: string;
readonly decisionFileDigest: string;
readonly legacyFactCount: number;
readonly targetFactCount: number;
confirmDecisionFileIdentity(): void;
}
export interface LocalReconciliationRunHistoryDependencies
extends LocalReconciliationSealedBundleReaderDependencies {
readonly afterReceiptPublished?: () => void;
readonly afterTerminalSealed?: () => void;
}
export interface LocalReconciliationRunHistoryTerminal {
readonly receipt: Readonly<LocalReconciliationRunHistoryPreservationReceipt>;
readonly application: Readonly<LocalReconciliationApplicationTerminal>;
}
function fail(message: string, cause?: unknown): never {
throw new LocalDeploymentConfigurationError(
`reconciliation run history ${message}`,
{ cause },
);
}
function runHistoryPaths(
runHistoryRoot: string,
preservationId: string,
): Readonly<RunHistoryPaths> {
const root = path.join(runHistoryRoot, preservationId);
return Object.freeze({ root, receipt: path.join(root, 'receipt.json') });
}
function validateDirectory(
directory: string,
uid: number,
modes: readonly number[],
label: string,
): number {
let stat: fs.Stats;
try {
stat = fs.lstatSync(directory);
} catch (error) {
return fail(`${label} is unavailable`, error);
}
const mode = stat.mode & 0o777;
if (
!stat.isDirectory() ||
stat.isSymbolicLink() ||
stat.uid !== uid ||
!modes.includes(mode) ||
fs.realpathSync(directory) !== directory
) {
fail(`${label} identity is invalid`);
}
return mode;
}
function validateCatalog(selected: Readonly<RunHistoryPaths>, sealed: boolean) {
const allowed = new Set([
'receipt.json',
...(!sealed ? ['.receipt.json.ql3-deploy-stage'] : []),
]);
for (const entry of fs.readdirSync(selected.root, { withFileTypes: true })) {
if (!allowed.has(entry.name) || entry.isSymbolicLink()) {
fail('preservation catalog contains unknown material');
}
}
}
function ensurePreservationDirectory(
runHistoryRoot: string,
preservationId: string,
uid: number,
): Readonly<RunHistoryPaths> {
const selected = runHistoryPaths(runHistoryRoot, preservationId);
const entries = fs.readdirSync(runHistoryRoot, { withFileTypes: true });
if (entries.some((entry) => !entry.isDirectory() || entry.isSymbolicLink())) {
fail('preservation root catalog contains drift');
}
if (entries.length >= MAX_PRESERVATIONS && !fs.existsSync(selected.root)) {
fail('preservation retention limit is reached');
}
try {
fs.mkdirSync(selected.root, { mode: 0o700 });
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== 'EEXIST') {
fail('preservation directory cannot be created', error);
}
}
const mode = validateDirectory(
selected.root,
uid,
[0o700, 0o500],
'preservation directory',
);
validateCatalog(selected, mode === 0o500);
return selected;
}
function readReceipt(
selected: Readonly<RunHistoryPaths>,
uid: number,
modes: readonly number[],
): Readonly<LocalReconciliationRunHistoryPreservationReceipt> {
let stat: fs.Stats;
try {
stat = fs.lstatSync(selected.receipt);
} catch (error) {
return fail('preservation receipt is unavailable', error);
}
if (
!stat.isFile() ||
stat.isSymbolicLink() ||
stat.uid !== uid ||
!modes.includes(stat.mode & 0o777) ||
stat.nlink !== 1 ||
stat.size < 2 ||
stat.size > MAX_RECEIPT_BYTES
) {
fail('preservation receipt identity is invalid');
}
try {
return normalizeLocalReconciliationRunHistoryPreservationReceipt(
readLocalReconciliationReviewTerminalJson(selected.receipt, uid, modes),
);
} catch (error) {
if (error instanceof LocalDeploymentConfigurationError) throw error;
return fail('preservation receipt cannot be read', error);
}
}
function sealPreservation(
selected: Readonly<RunHistoryPaths>,
uid: number,
): void {
validateDirectory(
selected.root,
uid,
[0o700, 0o500],
'preservation directory',
);
readReceipt(selected, uid, [0o600, 0o400]);
let descriptor = fs.openSync(
selected.receipt,
fs.constants.O_RDONLY | (fs.constants.O_NOFOLLOW ?? 0),
);
try {
const stat = fs.fstatSync(descriptor);
if (!stat.isFile() || stat.uid !== uid || stat.nlink !== 1) {
fail('preservation receipt cannot be sealed');
}
if ((stat.mode & 0o777) !== 0o400) {
fs.fchmodSync(descriptor, 0o400);
fs.fsyncSync(descriptor);
}
} finally {
fs.closeSync(descriptor);
}
descriptor = fs.openSync(selected.root, fs.constants.O_RDONLY);
try {
const stat = fs.fstatSync(descriptor);
if ((stat.mode & 0o777) !== 0o500) fs.fchmodSync(descriptor, 0o500);
fs.fsyncSync(descriptor);
} finally {
fs.closeSync(descriptor);
}
validateCatalog(selected, true);
readReceipt(selected, uid, [0o400]);
}
function validateApplication(
application: Readonly<LocalReconciliationApplicationTerminal>,
options: Readonly<LocalReconciliationRunHistoryOptions>,
applicationId: string,
applicationPlanDigest: string,
): void {
const runHistory = application.plan.domains.find(
(domain) => domain.domain === 'run_history',
);
if (
application.intent.command.options.deploymentRoot !==
options.deploymentRoot ||
application.intent.command.options.applicationRoot !==
options.applicationRoot ||
application.plan.applicationId !== applicationId ||
application.plan.applicationPlanDigest !== applicationPlanDigest ||
!runHistory ||
runHistory.action !== 'adapter_required'
) {
fail('preservation authority is detached from the application plan');
}
}
function assertSourceHead(
application: Readonly<LocalReconciliationApplicationTerminal>,
expectedHeadDigest: string,
uid: number,
): void {
const head = readLocalCutoverInstanceHead(
application.intent.command.options.deploymentRoot,
application.intent.instanceId,
uid,
);
const automation = application.plan.domains.find(
(domain) => domain.domain === 'automation',
);
const expectedState =
automation?.action === 'no_effect'
? 'reconciliation_application_planned'
: automation?.action === 'adapter_required'
? 'reconciliation_automation_applied'
: null;
if (
expectedState === null ||
head.headDigest !== expectedHeadDigest ||
head.state !== expectedState ||
head.instanceId !== application.intent.instanceId ||
head.cutoverId !== application.intent.cutoverId ||
head.generation !== application.intent.generation ||
head.activationDigest !== application.intent.activationDigest
) {
fail('preservation lost source head compare-and-swap');
}
}
async function deriveAuthority(
options: Readonly<LocalReconciliationRunHistoryOptions>,
applicationId: string,
applicationPlanDigest: string,
decisionFilePath: string,
uid: number,
dependencies: LocalReconciliationRunHistoryDependencies,
): Promise<Readonly<RunHistoryAuthority>> {
const application = await readLocalReconciliationApplicationTerminal(
options.applicationRoot,
applicationId,
uid,
);
validateApplication(
application,
options,
applicationId,
applicationPlanDigest,
);
const planTerminal = readLocalReconciliationPlanTerminal(
application.review.intent.command.options.planRoot,
application.review.intent.command.request.planId,
uid,
);
const runHistory = planTerminal.plan.domains.find(
(domain) => domain.domain === 'run_history',
);
if (!runHistory) fail('run history inventory is absent');
let legacyFactCount = 0;
let targetFactCount = 0;
const reviewed = withLocalReconciliationReviewDecisionFile(
decisionFilePath,
{
reviewId: application.review.review.reviewId,
profile: application.plan.profile,
planDigest: planTerminal.plan.planDigest,
preparationDigest: application.review.intent.preparationDigest,
},
(cursor) => {
for (const database of ['legacy', 'target'] as const) {
const opened = withLocalReconciliationSealedDatabase(
planTerminal.bundle,
database,
uid,
dependencies,
(client) =>
visitLocalReconciliationDiagnosticFacts(
client,
database,
(fact) => {
if (fact.decisionRequirement === 'informational') return;
const decision = cursor.next();
if (decision === null) {
fail('decision file omitted a canonical fact');
}
assertLocalReconciliationReviewDecisionMatchesFact(
decision,
fact,
);
if (fact.domain !== 'run_history') return;
if (
fact.decisionRequirement !== 'required' ||
fact.reason !== 'historical_preservation_required' ||
(database === 'legacy' &&
decision.disposition !== 'retain_both') ||
(database === 'target' &&
decision.disposition !== 'retain_target')
) {
fail(
'signed review did not authorize append-only preservation',
);
}
if (database === 'legacy') legacyFactCount += 1;
else targetFactCount += 1;
},
),
);
if (opened === null) {
fail('manual-required SQLite topology cannot be preserved');
}
}
},
);
if (
reviewed.evidence.fileDigest !==
application.review.authorization.decisionFileDigest ||
reviewed.evidence.decisionCount !==
application.review.authorization.decisionCount ||
legacyFactCount < 1 ||
targetFactCount < 1
) {
fail('signed review authority drifted');
}
return Object.freeze({
application,
runHistoryInventoryDigest: runHistory.inventoryDigest,
bundleDigest: planTerminal.bundle.receipt.bundleDigest,
bundleFingerprintDigest: planTerminal.bundle.fingerprintDigest,
decisionFileDigest: reviewed.evidence.fileDigest,
legacyFactCount,
targetFactCount,
confirmDecisionFileIdentity: reviewed.confirmIdentity,
});
}
function validateReceiptBinding(
receipt: Readonly<LocalReconciliationRunHistoryPreservationReceipt>,
authority: Readonly<RunHistoryAuthority>,
preservationId: string,
): void {
const application = authority.application;
if (
receipt.preservationId !== preservationId ||
receipt.applicationId !== application.plan.applicationId ||
receipt.profile !== application.plan.profile ||
receipt.instanceId !== application.intent.instanceId ||
receipt.cutoverId !== application.intent.cutoverId ||
receipt.generation !== application.intent.generation ||
receipt.activationDigest !== application.intent.activationDigest ||
receipt.applicationPlanDigest !== application.plan.applicationPlanDigest ||
receipt.reviewDigest !== application.review.review.reviewDigest ||
receipt.reviewAuthorizationDigest !==
application.review.authorization.authorizationDigest ||
receipt.reviewDecisionSetDigest !==
application.review.authorization.decisionSetDigest ||
receipt.reviewDecisionFileDigest !== authority.decisionFileDigest ||
receipt.bundleDigest !== authority.bundleDigest ||
receipt.bundleFingerprintDigest !== authority.bundleFingerprintDigest ||
receipt.runHistoryInventoryDigest !== authority.runHistoryInventoryDigest ||
receipt.legacyFactCount !== authority.legacyFactCount ||
receipt.targetFactCount !== authority.targetFactCount
) {
fail('preservation receipt is detached from terminal authority');
}
}
function result(
operation: LocalReconciliationRunHistoryResult['operation'],
status: LocalReconciliationRunHistoryResult['status'],
receipt: Readonly<LocalReconciliationRunHistoryPreservationReceipt>,
): Readonly<LocalReconciliationRunHistoryResult> {
return Object.freeze({
schemaVersion: 1,
operation,
status,
state: 'reconciliation_run_history_preserved',
preservationId: receipt.preservationId,
applicationId: receipt.applicationId,
preservationDigest: receipt.preservationDigest,
legacyFactCount: receipt.legacyFactCount,
targetFactCount: receipt.targetFactCount,
});
}
export async function readLocalReconciliationRunHistoryTerminal(
options: Readonly<LocalReconciliationRunHistoryOptions>,
preservationId: string,
applicationId: string,
decisionFilePath: string,
uid: number,
dependencies: LocalReconciliationRunHistoryDependencies = {},
): Promise<Readonly<LocalReconciliationRunHistoryTerminal>> {
const selected = runHistoryPaths(options.runHistoryRoot, preservationId);
validateDirectory(selected.root, uid, [0o500], 'preservation directory');
validateCatalog(selected, true);
const receipt = readReceipt(selected, uid, [0o400]);
if (receipt.applicationId !== applicationId) {
fail('preservation path binding drifted');
}
const authority = await deriveAuthority(
options,
applicationId,
receipt.applicationPlanDigest,
decisionFilePath,
uid,
dependencies,
);
validateReceiptBinding(receipt, authority, preservationId);
authority.confirmDecisionFileIdentity();
return Object.freeze({ receipt, application: authority.application });
}
export async function preserveLocalReconciliationRunHistory(
value: unknown,
dependencies: LocalReconciliationRunHistoryDependencies = {},
): Promise<Readonly<LocalReconciliationRunHistoryResult>> {
const command = normalizeLocalReconciliationRunHistoryPreserveCommand(value);
const uid = currentIdentity().uid;
for (const [directory, label] of [
[command.options.deploymentRoot, 'deploymentRoot'],
[command.options.applicationRoot, 'applicationRoot'],
[command.options.runHistoryRoot, 'runHistoryRoot'],
] as const) {
validatePrivateDirectory(directory, uid, label);
}
const selected = ensurePreservationDirectory(
command.options.runHistoryRoot,
command.request.preservationId,
uid,
);
const authority = await deriveAuthority(
command.options,
command.request.applicationId,
command.request.expectedApplicationPlanDigest,
command.request.decisionFilePath,
uid,
dependencies,
);
let receipt: Readonly<LocalReconciliationRunHistoryPreservationReceipt>;
let status: 'preserved' | 'existing' = 'preserved';
if (fs.existsSync(selected.receipt)) {
status = 'existing';
receipt = readReceipt(selected, uid, [0o600, 0o400]);
validateReceiptBinding(receipt, authority, command.request.preservationId);
if (
receipt.sourceHeadDigest !== command.request.expectedHeadDigest ||
receipt.preservedAtMs !== command.request.preservedAtMs
) {
fail('preservation command is not an exact replay');
}
} else {
assertSourceHead(
authority.application,
command.request.expectedHeadDigest,
uid,
);
if (
command.request.preservedAtMs < authority.application.plan.committedAtMs
) {
fail('preservation timestamp precedes application evidence');
}
receipt = buildLocalReconciliationRunHistoryPreservationReceipt({
preservationId: command.request.preservationId,
applicationId: command.request.applicationId,
profile: authority.application.plan.profile,
instanceId: authority.application.intent.instanceId,
cutoverId: authority.application.intent.cutoverId,
generation: authority.application.intent.generation,
activationDigest: authority.application.intent.activationDigest,
applicationPlanDigest: authority.application.plan.applicationPlanDigest,
sourceHeadDigest: command.request.expectedHeadDigest,
reviewDigest: authority.application.review.review.reviewDigest,
reviewAuthorizationDigest:
authority.application.review.authorization.authorizationDigest,
reviewDecisionSetDigest:
authority.application.review.authorization.decisionSetDigest,
reviewDecisionFileDigest: authority.decisionFileDigest,
bundleDigest: authority.bundleDigest,
bundleFingerprintDigest: authority.bundleFingerprintDigest,
runHistoryInventoryDigest: authority.runHistoryInventoryDigest,
legacyFactCount: authority.legacyFactCount,
targetFactCount: authority.targetFactCount,
preservedAtMs: command.request.preservedAtMs,
});
const contents = localReconciliationRunHistoryReceiptContents(receipt);
preflightPublishedFile(
selected.receipt,
contents,
0o600,
uid,
'run history preservation receipt',
);
authority.confirmDecisionFileIdentity();
assertSourceHead(
authority.application,
command.request.expectedHeadDigest,
uid,
);
publishExactFile(
selected.receipt,
contents,
0o600,
uid,
'run history preservation receipt',
);
dependencies.afterReceiptPublished?.();
}
sealPreservation(selected, uid);
dependencies.afterTerminalSealed?.();
authority.confirmDecisionFileIdentity();
assertSourceHead(
authority.application,
command.request.expectedHeadDigest,
uid,
);
return result(command.operation, status, receipt);
}
export async function verifyLocalReconciliationRunHistory(
value: unknown,
dependencies: LocalReconciliationRunHistoryDependencies = {},
): Promise<Readonly<LocalReconciliationRunHistoryResult>> {
const command = normalizeLocalReconciliationRunHistoryVerifyCommand(value);
const uid = currentIdentity().uid;
for (const [directory, label] of [
[command.options.deploymentRoot, 'deploymentRoot'],
[command.options.applicationRoot, 'applicationRoot'],
[command.options.runHistoryRoot, 'runHistoryRoot'],
] as const) {
validatePrivateDirectory(directory, uid, label);
}
const terminal = await readLocalReconciliationRunHistoryTerminal(
command.options,
command.request.preservationId,
command.request.applicationId,
command.request.decisionFilePath,
uid,
dependencies,
);
if (
terminal.receipt.preservationDigest !==
command.request.expectedPreservationDigest
) {
fail('verify command is detached from preservation receipt');
}
return result(command.operation, 'verified', terminal.receipt);
}
export function preserveLocalReconciliationRunHistoryCommandFile(
filePath: string,
dependencies: LocalReconciliationRunHistoryDependencies = {},
) {
return preserveLocalReconciliationRunHistory(
readPrivateLocalCommandFile(filePath),
dependencies,
);
}
export function verifyLocalReconciliationRunHistoryCommandFile(
filePath: string,
dependencies: LocalReconciliationRunHistoryDependencies = {},
) {
return verifyLocalReconciliationRunHistory(
readPrivateLocalCommandFile(filePath),
dependencies,
);
}
export type {
LocalReconciliationRunHistoryPreserveCommand,
LocalReconciliationRunHistoryVerifyCommand,
};
@@ -0,0 +1,158 @@
import { LocalDeploymentConfigurationError } from '../../../foundation/error';
import { cutoverDigest } from '../../../cutover/targetEvidence';
const RECEIPT_SCHEMA =
'qinglong3-local-reconciliation-run-history-preservation-receipt';
const DIGEST = /^[0-9a-f]{64}$/;
const UUID_V4 =
/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/;
export interface LocalReconciliationRunHistoryPreservationReceipt {
readonly schema: typeof RECEIPT_SCHEMA;
readonly schemaVersion: 1;
readonly state: 'reconciliation_run_history_preserved';
readonly preservationId: string;
readonly applicationId: string;
readonly profile: 'edge' | 'standalone';
readonly instanceId: string;
readonly cutoverId: string;
readonly generation: number;
readonly activationDigest: string;
readonly applicationPlanDigest: string;
readonly sourceHeadDigest: string;
readonly reviewDigest: string;
readonly reviewAuthorizationDigest: string;
readonly reviewDecisionSetDigest: string;
readonly reviewDecisionFileDigest: string;
readonly bundleDigest: string;
readonly bundleFingerprintDigest: string;
readonly runHistoryInventoryDigest: string;
readonly legacyFactCount: number;
readonly targetFactCount: number;
readonly preservedAtMs: number;
readonly preservationDigest: string;
}
function fail(message: string): never {
throw new LocalDeploymentConfigurationError(
`reconciliation run history evidence ${message}`,
);
}
function exact(
value: unknown,
keys: readonly string[],
): Record<string, unknown> {
if (!value || typeof value !== 'object' || Array.isArray(value)) {
fail('receipt must be an object');
}
const selected = value as Record<string, unknown>;
const actual = Object.keys(selected).sort();
const expected = [...keys].sort();
if (
actual.length !== expected.length ||
actual.some((key, index) => key !== expected[index])
) {
fail('receipt shape is invalid');
}
return selected;
}
export function buildLocalReconciliationRunHistoryPreservationReceipt(
input: Omit<
LocalReconciliationRunHistoryPreservationReceipt,
'schema' | 'schemaVersion' | 'state' | 'preservationDigest'
>,
): Readonly<LocalReconciliationRunHistoryPreservationReceipt> {
const payload = Object.freeze({
schema: RECEIPT_SCHEMA,
schemaVersion: 1 as const,
state: 'reconciliation_run_history_preserved' as const,
...input,
});
return Object.freeze({
...payload,
preservationDigest: cutoverDigest(payload),
});
}
export function normalizeLocalReconciliationRunHistoryPreservationReceipt(
value: unknown,
): Readonly<LocalReconciliationRunHistoryPreservationReceipt> {
const selected = exact(value, [
'activationDigest',
'applicationId',
'applicationPlanDigest',
'bundleDigest',
'bundleFingerprintDigest',
'cutoverId',
'generation',
'instanceId',
'legacyFactCount',
'preservationDigest',
'preservationId',
'preservedAtMs',
'profile',
'reviewAuthorizationDigest',
'reviewDecisionFileDigest',
'reviewDecisionSetDigest',
'reviewDigest',
'runHistoryInventoryDigest',
'schema',
'schemaVersion',
'sourceHeadDigest',
'state',
'targetFactCount',
]);
const { preservationDigest, ...payload } = selected;
if (
selected.schema !== RECEIPT_SCHEMA ||
selected.schemaVersion !== 1 ||
selected.state !== 'reconciliation_run_history_preserved' ||
typeof selected.preservationId !== 'string' ||
!UUID_V4.test(selected.preservationId) ||
typeof selected.applicationId !== 'string' ||
!UUID_V4.test(selected.applicationId) ||
(selected.profile !== 'edge' && selected.profile !== 'standalone') ||
typeof selected.instanceId !== 'string' ||
selected.instanceId.length < 1 ||
typeof selected.cutoverId !== 'string' ||
selected.cutoverId.length < 1 ||
!Number.isSafeInteger(selected.generation) ||
(selected.generation as number) < 1 ||
![
selected.activationDigest,
selected.applicationPlanDigest,
selected.sourceHeadDigest,
selected.reviewDigest,
selected.reviewAuthorizationDigest,
selected.reviewDecisionSetDigest,
selected.reviewDecisionFileDigest,
selected.bundleDigest,
selected.bundleFingerprintDigest,
selected.runHistoryInventoryDigest,
preservationDigest,
].every(
(candidate) => typeof candidate === 'string' && DIGEST.test(candidate),
) ||
!Number.isSafeInteger(selected.legacyFactCount) ||
(selected.legacyFactCount as number) < 1 ||
!Number.isSafeInteger(selected.targetFactCount) ||
(selected.targetFactCount as number) < 1 ||
!Number.isSafeInteger(selected.preservedAtMs) ||
(selected.preservedAtMs as number) < 0 ||
cutoverDigest(payload) !== preservationDigest
) {
fail('receipt binding is invalid');
}
return Object.freeze({
...payload,
preservationDigest,
}) as unknown as Readonly<LocalReconciliationRunHistoryPreservationReceipt>;
}
export function localReconciliationRunHistoryReceiptContents(
receipt: Readonly<LocalReconciliationRunHistoryPreservationReceipt>,
): string {
return `${JSON.stringify(receipt, null, 2)}\n`;
}
@@ -22,17 +22,28 @@ export interface LocalReconciliationCompletionOptions {
readonly applicationRoot: string;
readonly completionRoot: string;
readonly automation: Readonly<LocalReconciliationCompletionAutomationOptions> | null;
readonly runHistory: Readonly<LocalReconciliationCompletionRunHistoryOptions> | null;
readonly allowRootService: boolean;
}
export interface LocalReconciliationCompletionRunHistoryOptions {
readonly runHistoryRoot: string;
readonly decisionFilePath: string;
}
export interface LocalReconciliationCompletionAutomationBinding {
readonly automationId: string;
readonly decisionId: string;
readonly expectedApplyDigest: string;
}
export interface LocalReconciliationCompletionRunHistoryBinding {
readonly preservationId: string;
readonly expectedPreservationDigest: string;
}
export interface LocalReconciliationCompleteCommand {
readonly schemaVersion: 1;
readonly schemaVersion: 1 | 2;
readonly operation: 'local.deployment.reconciliation.complete';
readonly options: Readonly<LocalReconciliationCompletionOptions>;
readonly request: Readonly<{
@@ -41,12 +52,13 @@ export interface LocalReconciliationCompleteCommand {
expectedApplicationPlanDigest: string;
expectedHeadDigest: string;
automation: Readonly<LocalReconciliationCompletionAutomationBinding> | null;
runHistory: Readonly<LocalReconciliationCompletionRunHistoryBinding> | null;
completedAtMs: number;
}>;
}
export interface LocalReconciliationCompletionVerifyCommand {
readonly schemaVersion: 1;
readonly schemaVersion: 1 | 2;
readonly operation: 'local.deployment.reconciliation.complete.verify';
readonly options: Readonly<LocalReconciliationCompletionOptions>;
readonly request: Readonly<{
@@ -54,6 +66,7 @@ export interface LocalReconciliationCompletionVerifyCommand {
applicationId: string;
expectedCompletionDigest: string;
automation: Readonly<LocalReconciliationCompletionAutomationBinding> | null;
runHistory: Readonly<LocalReconciliationCompletionRunHistoryBinding> | null;
}>;
}
@@ -68,7 +81,7 @@ export interface LocalReconciliationCompletionResult {
readonly applicationId: string;
readonly completionDigest: string;
readonly domainCount: 8;
readonly adapterCount: 0 | 1;
readonly adapterCount: 0 | 1 | 2;
readonly instanceHeadDigest: string;
}
@@ -169,17 +182,27 @@ function normalizeAutomationOptions(
function normalizeOptions(
value: unknown,
schemaVersion: 1 | 2,
): Readonly<LocalReconciliationCompletionOptions> {
const selected = record(value, 'options');
exact(
selected,
[
'allowRootService',
'applicationRoot',
'automation',
'completionRoot',
'deploymentRoot',
],
schemaVersion === 1
? [
'allowRootService',
'applicationRoot',
'automation',
'completionRoot',
'deploymentRoot',
]
: [
'allowRootService',
'applicationRoot',
'automation',
'completionRoot',
'deploymentRoot',
'runHistory',
],
'options',
);
if (
@@ -189,11 +212,16 @@ function normalizeOptions(
fail('command identity is invalid');
}
const automation = normalizeAutomationOptions(selected.automation);
const runHistory =
schemaVersion === 1
? null
: normalizeRunHistoryOptions(selected.runHistory);
const normalized = Object.freeze({
deploymentRoot: safePath(selected.deploymentRoot, 'deploymentRoot'),
applicationRoot: safePath(selected.applicationRoot, 'applicationRoot'),
completionRoot: safePath(selected.completionRoot, 'completionRoot'),
automation,
runHistory,
allowRootService: selected.allowRootService,
}) as Readonly<LocalReconciliationCompletionOptions>;
const roots = [
@@ -207,6 +235,7 @@ function normalizeOptions(
automation.automationDecisionRoot,
automation.automationApplyRoot,
]),
...(runHistory === null ? [] : [runHistory.runHistoryRoot]),
];
for (let left = 0; left < roots.length; left += 1) {
for (let right = left + 1; right < roots.length; right += 1) {
@@ -228,9 +257,34 @@ function normalizeOptions(
) {
fail('targetDatabasePath overlaps an authority root');
}
if (
runHistory !== null &&
roots.some(
(root) =>
overlaps(root, runHistory.decisionFilePath) ||
overlaps(runHistory.decisionFilePath, root),
)
) {
fail('decisionFilePath overlaps an authority root');
}
return normalized;
}
function normalizeRunHistoryOptions(
value: unknown,
): Readonly<LocalReconciliationCompletionRunHistoryOptions> {
const selected = record(value, 'run history options');
exact(
selected,
['decisionFilePath', 'runHistoryRoot'],
'run history options',
);
return Object.freeze({
runHistoryRoot: safePath(selected.runHistoryRoot, 'runHistoryRoot'),
decisionFilePath: safePath(selected.decisionFilePath, 'decisionFilePath'),
});
}
function normalizeAutomationBinding(
value: unknown,
): Readonly<LocalReconciliationCompletionAutomationBinding> | null {
@@ -251,6 +305,28 @@ function normalizeAutomationBinding(
});
}
function normalizeRunHistoryBinding(
value: unknown,
): Readonly<LocalReconciliationCompletionRunHistoryBinding> {
const selected = record(value, 'run history binding');
exact(
selected,
['expectedPreservationDigest', 'preservationId'],
'run history binding',
);
return Object.freeze({
preservationId: identifier(
selected.preservationId,
UUID_V4,
'preservationId',
),
expectedPreservationDigest: digest(
selected.expectedPreservationDigest,
'expectedPreservationDigest',
),
});
}
function command(value: unknown, operation: string) {
const selected = record(value, 'command');
exact(
@@ -258,11 +334,15 @@ function command(value: unknown, operation: string) {
['operation', 'options', 'request', 'schemaVersion'],
'command',
);
if (selected.schemaVersion !== 1 || selected.operation !== operation) {
if (
(selected.schemaVersion !== 1 && selected.schemaVersion !== 2) ||
selected.operation !== operation
) {
fail('command version or operation is invalid');
}
return Object.freeze({
options: normalizeOptions(selected.options),
schemaVersion: selected.schemaVersion,
options: normalizeOptions(selected.options, selected.schemaVersion),
request: record(selected.request, 'request'),
});
}
@@ -273,14 +353,24 @@ export function normalizeLocalReconciliationCompleteCommand(
const selected = command(value, 'local.deployment.reconciliation.complete');
exact(
selected.request,
[
'applicationId',
'automation',
'completedAtMs',
'completionId',
'expectedApplicationPlanDigest',
'expectedHeadDigest',
],
selected.schemaVersion === 1
? [
'applicationId',
'automation',
'completedAtMs',
'completionId',
'expectedApplicationPlanDigest',
'expectedHeadDigest',
]
: [
'applicationId',
'automation',
'completedAtMs',
'completionId',
'expectedApplicationPlanDigest',
'expectedHeadDigest',
'runHistory',
],
'request',
);
if (
@@ -290,7 +380,7 @@ export function normalizeLocalReconciliationCompleteCommand(
fail('completedAtMs is invalid');
}
return Object.freeze({
schemaVersion: 1,
schemaVersion: selected.schemaVersion,
operation: 'local.deployment.reconciliation.complete',
options: selected.options,
request: Object.freeze({
@@ -313,6 +403,10 @@ export function normalizeLocalReconciliationCompleteCommand(
'expectedHeadDigest',
),
automation: normalizeAutomationBinding(selected.request.automation),
runHistory:
selected.schemaVersion === 1
? null
: normalizeRunHistoryBinding(selected.request.runHistory),
completedAtMs: selected.request.completedAtMs as number,
}),
});
@@ -327,11 +421,24 @@ export function normalizeLocalReconciliationCompletionVerifyCommand(
);
exact(
selected.request,
['applicationId', 'automation', 'completionId', 'expectedCompletionDigest'],
selected.schemaVersion === 1
? [
'applicationId',
'automation',
'completionId',
'expectedCompletionDigest',
]
: [
'applicationId',
'automation',
'completionId',
'expectedCompletionDigest',
'runHistory',
],
'request',
);
return Object.freeze({
schemaVersion: 1,
schemaVersion: selected.schemaVersion,
operation: 'local.deployment.reconciliation.complete.verify',
options: selected.options,
request: Object.freeze({
@@ -350,6 +457,10 @@ export function normalizeLocalReconciliationCompletionVerifyCommand(
'expectedCompletionDigest',
),
automation: normalizeAutomationBinding(selected.request.automation),
runHistory:
selected.schemaVersion === 1
? null
: normalizeRunHistoryBinding(selected.request.runHistory),
}),
});
}
@@ -33,6 +33,11 @@ import type {
LocalReconciliationAutomationApplyReceipt,
} from '../application/automation/applyEvidence';
import { readLocalReconciliationAutomationDecisionTerminal } from '../application/automation/decisionCoordinator';
import {
readLocalReconciliationRunHistoryTerminal,
type LocalReconciliationRunHistoryDependencies,
} from '../application/run-history/coordinator';
import type { LocalReconciliationRunHistoryPreservationReceipt } from '../application/run-history/evidence';
import type { LocalReconciliationCompletionDomainEvidence } from './evidence';
import {
buildLocalReconciliationCompletionReceipt,
@@ -63,7 +68,12 @@ interface AutomationProof {
readonly paths: ReturnType<typeof localReconciliationAutomationApplyPaths>;
}
export interface LocalReconciliationCompletionDependencies {
interface RunHistoryProof {
readonly receipt: Readonly<LocalReconciliationRunHistoryPreservationReceipt>;
}
export interface LocalReconciliationCompletionDependencies
extends LocalReconciliationRunHistoryDependencies {
readonly inspectSnapshot?: typeof inspectLocalSqliteSnapshot;
readonly afterReceiptPublished?: () => void;
readonly afterTerminalSealed?: () => void;
@@ -71,6 +81,62 @@ export interface LocalReconciliationCompletionDependencies {
readonly afterBackupCollected?: () => void;
}
async function runHistoryProof(
command: Readonly<LocalReconciliationCompleteCommand>,
terminal: Readonly<LocalReconciliationApplicationTerminal>,
uid: number,
dependencies: LocalReconciliationCompletionDependencies,
): Promise<Readonly<RunHistoryProof> | null> {
const domain = terminal.plan.domains.find(
(selected) => selected.domain === 'run_history',
);
if (!domain) fail('run history domain is absent');
if (domain.action === 'no_effect') {
if (
command.options.runHistory !== null ||
command.request.runHistory !== null
) {
fail('no-effect completion must not carry run history authority');
}
return null;
}
if (
domain.action !== 'adapter_required' ||
command.options.runHistory === null ||
command.request.runHistory === null
) {
fail('run history domain is not terminally provable');
}
validatePrivateDirectory(
command.options.runHistory.runHistoryRoot,
uid,
'runHistoryRoot',
);
const history = await readLocalReconciliationRunHistoryTerminal(
{
deploymentRoot: command.options.deploymentRoot,
applicationRoot: command.options.applicationRoot,
runHistoryRoot: command.options.runHistory.runHistoryRoot,
allowRootService: command.options.allowRootService,
},
command.request.runHistory.preservationId,
command.request.applicationId,
command.options.runHistory.decisionFilePath,
uid,
dependencies,
);
if (
history.receipt.preservationDigest !==
command.request.runHistory.expectedPreservationDigest ||
history.receipt.applicationPlanDigest !==
terminal.plan.applicationPlanDigest ||
history.receipt.sourceHeadDigest !== command.request.expectedHeadDigest
) {
fail('run history preservation evidence is detached');
}
return Object.freeze({ receipt: history.receipt });
}
function fail(message: string, cause?: unknown): never {
throw new LocalDeploymentConfigurationError(
`reconciliation completion ${message}`,
@@ -393,6 +459,7 @@ async function automationProof(
function domainEvidence(
terminal: Readonly<LocalReconciliationApplicationTerminal>,
automation: Readonly<AutomationProof> | null,
runHistory: Readonly<RunHistoryProof> | null,
): readonly Readonly<LocalReconciliationCompletionDomainEvidence>[] {
return Object.freeze(
terminal.plan.domains.map((domain) => {
@@ -416,6 +483,18 @@ function domainEvidence(
evidenceDigest: automation.receipt.applyDigest,
});
}
if (
domain.domain === 'run_history' &&
domain.action === 'adapter_required' &&
runHistory !== null
) {
return Object.freeze({
domain: domain.domain,
action: 'adapter_required' as const,
evidenceKind: 'run_history_preservation' as const,
evidenceDigest: runHistory.receipt.preservationDigest,
});
}
return fail(`${domain.domain} is not terminally reconciled`);
}),
);
@@ -539,7 +618,13 @@ export async function completeLocalReconciliation(
uid,
dependencies,
);
const domains = domainEvidence(terminal, automation);
const runHistory = await runHistoryProof(
command,
terminal,
uid,
dependencies,
);
const domains = domainEvidence(terminal, automation, runHistory);
const selected = ensureCompletionDirectory(
command.options.completionRoot,
command.request.completionId,
@@ -572,10 +657,11 @@ export async function completeLocalReconciliation(
assertSourceHead(head, command.request.expectedHeadDigest, automation);
const adapterCount = domains.filter(
(domain) => domain.action === 'adapter_required',
).length as 0 | 1;
).length as 0 | 1 | 2;
const latestEvidenceAtMs = Math.max(
terminal.plan.committedAtMs,
automation?.receipt.appliedAtMs ?? 0,
runHistory?.receipt.preservedAtMs ?? 0,
);
if (command.request.completedAtMs < latestEvidenceAtMs) {
fail('completion timestamp precedes terminal evidence');
@@ -682,7 +768,7 @@ export async function verifyLocalReconciliationCompletion(
command.options.applicationRoot,
);
const syntheticCompleteCommand = Object.freeze({
schemaVersion: 1 as const,
schemaVersion: receipt.schemaVersion,
operation: 'local.deployment.reconciliation.complete' as const,
options: command.options,
request: Object.freeze({
@@ -691,6 +777,7 @@ export async function verifyLocalReconciliationCompletion(
expectedApplicationPlanDigest: receipt.applicationPlanDigest,
expectedHeadDigest: receipt.sourceHeadDigest,
automation: command.request.automation,
runHistory: command.request.runHistory,
completedAtMs: receipt.completedAtMs,
}),
});
@@ -700,7 +787,13 @@ export async function verifyLocalReconciliationCompletion(
uid,
dependencies,
);
const domains = domainEvidence(terminal, automation);
const runHistory = await runHistoryProof(
syntheticCompleteCommand,
terminal,
uid,
dependencies,
);
const domains = domainEvidence(terminal, automation, runHistory);
validateReceiptBinding(
receipt,
terminal,
@@ -13,13 +13,16 @@ const UUID_V4 =
export interface LocalReconciliationCompletionDomainEvidence {
readonly domain: LocalReconciliationPlanDomain;
readonly action: 'no_effect' | 'adapter_required';
readonly evidenceKind: 'application_summary' | 'automation_apply';
readonly evidenceKind:
| 'application_summary'
| 'automation_apply'
| 'run_history_preservation';
readonly evidenceDigest: string;
}
export interface LocalReconciliationCompletionReceipt {
readonly schema: typeof RECEIPT_SCHEMA;
readonly schemaVersion: 1;
readonly schemaVersion: 1 | 2;
readonly state: 'reconciliation_completed';
readonly completionId: string;
readonly applicationId: string;
@@ -31,7 +34,7 @@ export interface LocalReconciliationCompletionReceipt {
readonly applicationPlanDigest: string;
readonly sourceHeadDigest: string;
readonly domains: readonly Readonly<LocalReconciliationCompletionDomainEvidence>[];
readonly adapterCount: 0 | 1;
readonly adapterCount: 0 | 1 | 2;
readonly completedAtMs: number;
readonly completionDigest: string;
}
@@ -65,6 +68,7 @@ function exact(
function domainEvidence(
value: unknown,
expectedDomain: LocalReconciliationPlanDomain,
schemaVersion: 1 | 2,
): Readonly<LocalReconciliationCompletionDomainEvidence> {
const selected = exact(
value,
@@ -78,9 +82,14 @@ function domainEvidence(
expectedDomain === 'automation' &&
selected.action === 'adapter_required' &&
selected.evidenceKind === 'automation_apply';
const runHistory =
schemaVersion === 2 &&
expectedDomain === 'run_history' &&
selected.action === 'adapter_required' &&
selected.evidenceKind === 'run_history_preservation';
if (
selected.domain !== expectedDomain ||
(!noEffect && !automation) ||
(!noEffect && !automation && !runHistory) ||
typeof selected.evidenceDigest !== 'string' ||
!DIGEST.test(selected.evidenceDigest)
) {
@@ -100,9 +109,14 @@ export function buildLocalReconciliationCompletionReceipt(
'schema' | 'schemaVersion' | 'state' | 'completionDigest'
>,
): Readonly<LocalReconciliationCompletionReceipt> {
const schemaVersion = input.domains.some(
(domain) => domain.evidenceKind === 'run_history_preservation',
)
? (2 as const)
: (1 as const);
const payload = Object.freeze({
schema: RECEIPT_SCHEMA,
schemaVersion: 1 as const,
schemaVersion,
state: 'reconciliation_completed' as const,
...input,
});
@@ -140,10 +154,14 @@ export function normalizeLocalReconciliationCompletionReceipt(
if (!Array.isArray(selected.domains) || selected.domains.length !== 8) {
fail('receipt domain catalog is invalid');
}
if (selected.schemaVersion !== 1 && selected.schemaVersion !== 2) {
fail('receipt schema version is invalid');
}
const schemaVersion = selected.schemaVersion;
const rawDomains = selected.domains as unknown[];
const domains = Object.freeze(
LOCAL_RECONCILIATION_PLAN_DOMAINS.map((domain, index) =>
domainEvidence(rawDomains[index], domain),
domainEvidence(rawDomains[index], domain, schemaVersion),
),
);
const adapterCount = domains.filter(
@@ -153,7 +171,10 @@ export function normalizeLocalReconciliationCompletionReceipt(
const normalized = Object.freeze({ ...raw, domains });
if (
selected.schema !== RECEIPT_SCHEMA ||
selected.schemaVersion !== 1 ||
(schemaVersion === 2) !==
domains.some(
(domain) => domain.evidenceKind === 'run_history_preservation',
) ||
selected.state !== 'reconciliation_completed' ||
typeof selected.completionId !== 'string' ||
!UUID_V4.test(selected.completionId) ||
@@ -174,7 +195,7 @@ export function normalizeLocalReconciliationCompletionReceipt(
].every(
(candidate) => typeof candidate === 'string' && DIGEST.test(candidate),
) ||
(selected.adapterCount !== 0 && selected.adapterCount !== 1) ||
![0, 1, 2].includes(selected.adapterCount as number) ||
selected.adapterCount !== adapterCount ||
!Number.isSafeInteger(selected.completedAtMs) ||
(selected.completedAtMs as number) < 0 ||
@@ -31,6 +31,7 @@ export type LocalReconciliationDiagnosticReason =
| 'reviewable_fact'
| 'unknown_schema'
| 'secret_custody_required'
| 'historical_preservation_required'
| 'historical_integrity_required'
| 'identity_custody_required';
@@ -89,7 +90,83 @@ function quotedIdentifier(name: string): string {
return `"${name.replaceAll('"', '""')}"`;
}
function requirement(domain: LocalReconciliationPlanDomain): Readonly<{
function targetRunHistoryIsTerminal(client: DatabaseSync): boolean {
const tables = new Map<string, Set<string>>();
for (const tableName of ['Runs', 'RunAttempts', 'StepRuns'] as const) {
const table = client
.prepare(
`SELECT 1 AS present
FROM sqlite_schema
WHERE type = 'table' AND name = ?
LIMIT 1`,
)
.get(tableName) as { readonly present?: unknown } | undefined;
if (table?.present !== 1) continue;
const columns = new Set<string>();
const rows = client
.prepare(`SELECT name FROM pragma_table_info(?) ORDER BY cid LIMIT 257`)
.iterate(tableName) as IterableIterator<{ readonly name?: unknown }>;
let count = 0;
for (const row of rows) {
count += 1;
if (
count > 256 ||
typeof row.name !== 'string' ||
Buffer.byteLength(row.name, 'utf8') > MAX_NAME_BYTES
) {
return false;
}
columns.add(row.name);
}
tables.set(tableName, columns);
}
const runs = tables.get('Runs');
if (!runs?.has('status') || !runs.has('finished_at_ms')) return false;
const activeRun = client
.prepare(
`SELECT 1 AS present
FROM "Runs"
WHERE status NOT IN ('succeeded','failed','cancelled','timed_out')
OR finished_at_ms IS NULL
LIMIT 1`,
)
.get() as { readonly present?: unknown } | undefined;
if (activeRun?.present === 1) return false;
const attempts = tables.get('RunAttempts');
if (attempts !== undefined) {
if (!attempts.has('status')) return false;
const activeAttempt = client
.prepare(
`SELECT 1 AS present
FROM "RunAttempts"
WHERE status IN ('claimed','starting','running')
LIMIT 1`,
)
.get() as { readonly present?: unknown } | undefined;
if (activeAttempt?.present === 1) return false;
}
const steps = tables.get('StepRuns');
if (steps !== undefined) {
if (!steps.has('status') || !steps.has('finished_at_ms')) return false;
const activeStep = client
.prepare(
`SELECT 1 AS present
FROM "StepRuns"
WHERE status NOT IN ('succeeded','failed','skipped','cancelled','timed_out')
OR finished_at_ms IS NULL
LIMIT 1`,
)
.get() as { readonly present?: unknown } | undefined;
if (activeStep?.present === 1) return false;
}
return true;
}
function requirement(
client: DatabaseSync,
database: LocalReconciliationSealedDatabaseKind,
domain: LocalReconciliationPlanDomain,
): Readonly<{
decisionRequirement: LocalReconciliationDiagnosticDecisionRequirement;
reason: LocalReconciliationDiagnosticReason;
}> {
@@ -106,6 +183,12 @@ function requirement(domain: LocalReconciliationPlanDomain): Readonly<{
});
}
if (domain === 'run_history') {
if (database === 'legacy' || targetRunHistoryIsTerminal(client)) {
return Object.freeze({
decisionRequirement: 'required' as const,
reason: 'historical_preservation_required' as const,
});
}
return Object.freeze({
decisionRequirement: 'blocked' as const,
reason: 'historical_integrity_required' as const,
@@ -161,8 +244,11 @@ function fact(
name: string,
tableName: string,
count: string | null,
review: Readonly<{
decisionRequirement: LocalReconciliationDiagnosticDecisionRequirement;
reason: LocalReconciliationDiagnosticReason;
}>,
): Readonly<LocalReconciliationDiagnosticFact> {
const review = requirement(domain);
const payload = Object.freeze({
schema: FACT_SCHEMA,
schemaVersion: 1 as const,
@@ -190,6 +276,7 @@ function schemaFacts(
records: readonly Readonly<LocalReconciliationDiagnosticFact>[];
complete: boolean;
}> {
const review = requirement(client, database, domain);
const rows = client
.prepare(
`SELECT type, name, tbl_name AS tableName
@@ -240,6 +327,7 @@ function schemaFacts(
row.name,
row.tableName,
null,
review,
),
);
}
@@ -256,6 +344,7 @@ function tableFacts(
records: readonly Readonly<LocalReconciliationDiagnosticFact>[];
complete: boolean;
}> {
const review = requirement(client, database, domain);
const rows = client
.prepare(
`SELECT name, type
@@ -302,6 +391,7 @@ function tableFacts(
row.name,
row.name,
rowCount(client, domain, row.name, row.type),
review,
),
);
}
@@ -18,6 +18,7 @@ const {
prepareLocalReconciliationCapture,
prepareLocalReconciliationApplication,
prepareLocalReconciliationAutomationDecision,
preserveLocalReconciliationRunHistory,
readLocalReconciliationAutomationDecisionTerminal,
rollbackLocalReconciliationAutomationApply,
planLocalReconciliationAutomation,
@@ -31,6 +32,7 @@ const {
verifyLocalReconciliationCompletion,
verifyLocalReconciliationPlan,
verifyLocalReconciliationReview,
verifyLocalReconciliationRunHistory,
writeLocalReconciliationReviewDiagnostics,
} = require('../dist/deployment/localDeployment.js');
const {
@@ -710,6 +712,50 @@ function automationDatabaseInitializer() {
};
}
function runHistoryDatabaseInitializer() {
return ({ legacySourcePath, recoveryPath, targetDatabasePath }) => {
const legacy = new DatabaseSync(legacySourcePath);
legacy.exec(`
CREATE TABLE "CrontabStats" (
id INTEGER PRIMARY KEY,
timestamp INTEGER NOT NULL,
status INTEGER NOT NULL
);
INSERT INTO "CrontabStats" (id, timestamp, status)
VALUES (1, 1000, 0);
`);
legacy.close();
fs.chmodSync(legacySourcePath, 0o600);
fs.copyFileSync(legacySourcePath, recoveryPath);
fs.chmodSync(recoveryPath, 0o600);
const target = new DatabaseSync(targetDatabasePath);
target.exec(`
CREATE TABLE "Runs" (
id TEXT PRIMARY KEY,
status TEXT NOT NULL,
finished_at_ms INTEGER
);
INSERT INTO "Runs" (id, status, finished_at_ms)
VALUES ('baseline', 'succeeded', 1000);
`);
target.close();
fs.chmodSync(targetDatabasePath, 0o600);
};
}
function mutateRunHistoryTarget({ targetDatabasePath }, status = 'failed') {
const target = new DatabaseSync(targetDatabasePath);
target
.prepare(
`INSERT INTO "Runs" (id, status, finished_at_ms)
VALUES (?, ?, ?)`,
)
.run('captured', status, status === 'running' ? null : 2000);
target.close();
return Object.freeze({});
}
function automationReadyDatabaseInitializer() {
const initializeLegacy = automationDatabaseInitializer();
return (paths) => {
@@ -3244,6 +3290,303 @@ test('completion fence retains automation rollback backup while other domains re
);
});
test('run history preservation seals terminal histories and completes through v2 evidence', async (t) => {
const state = await reviewedApplicationFixture(t, {
planId: '00000000-0000-4000-8000-000000000501',
reviewId: '00000000-0000-4000-8000-000000000502',
applicationId: '00000000-0000-4000-8000-000000000503',
reviewSuffix: 'run-history-preservation',
createDefaultSidecars: false,
initializeDatabases: runHistoryDatabaseInitializer(),
mutateTarget(paths) {
return mutateRunHistoryTarget(paths);
},
mutateDecisions(records) {
const selected = records.filter(
(record) =>
record.kind === 'qinglong3-local-reconciliation-review-decision' &&
record.database === 'legacy' &&
record.domain === 'run_history',
);
assert.ok(selected.length > 0);
for (const record of selected) {
record.disposition = 'retain_both';
record.reason = 'preserve_both';
}
},
});
const prepared = await prepareLocalReconciliationApplication(
state.prepareApplicationCommand,
);
const application = await commitLocalReconciliationApplication(
applicationCommitCommand(state, prepared),
);
assert.equal(application.outcome, 'adapter_required');
const applicationPlan = JSON.parse(
fs.readFileSync(
path.join(state.applicationRoot, application.applicationId, 'plan.json'),
'utf8',
),
);
assert.equal(
applicationPlan.domains.find((domain) => domain.domain === 'run_history')
.action,
'adapter_required',
);
const runHistoryRoot = path.join(
path.dirname(state.captureRoot),
'run-history-preservation-root',
);
fs.mkdirSync(runHistoryRoot, { mode: 0o700 });
const preservationCommand = {
schemaVersion: 1,
operation: 'local.deployment.reconciliation.run-history.preserve',
options: {
deploymentRoot: state.deploymentRoot,
applicationRoot: state.applicationRoot,
runHistoryRoot,
allowRootService: rootAcknowledgement(),
},
request: {
preservationId: '00000000-0000-4000-8000-000000000504',
applicationId: application.applicationId,
expectedApplicationPlanDigest: application.applicationPlanDigest,
expectedHeadDigest: application.instanceHeadDigest,
decisionFilePath: state.reviewFile.filePath,
preservedAtMs: state.prepareApplicationCommand.request.preparedAtMs + 3,
},
};
for (const boundary of ['afterReceiptPublished', 'afterTerminalSealed']) {
await assert.rejects(
preserveLocalReconciliationRunHistory(preservationCommand, {
[boundary]() {
throw new Error(`run history ${boundary} response loss`);
},
}),
new RegExp(`run history ${boundary} response loss`),
);
}
const preserved = await preserveLocalReconciliationRunHistory(
preservationCommand,
);
assert.equal(preserved.status, 'existing');
assert.ok(preserved.legacyFactCount > 0);
assert.ok(preserved.targetFactCount > 0);
const preservationDirectory = path.join(
runHistoryRoot,
preservationCommand.request.preservationId,
);
assert.deepEqual(fs.readdirSync(preservationDirectory), ['receipt.json']);
assert.equal(fs.statSync(preservationDirectory).mode & 0o777, 0o500);
assert.equal(
fs.statSync(path.join(preservationDirectory, 'receipt.json')).mode & 0o777,
0o400,
);
const preservationReceiptText = fs.readFileSync(
path.join(preservationDirectory, 'receipt.json'),
'utf8',
);
assert.equal(preservationReceiptText.includes('CrontabStats'), false);
assert.equal(preservationReceiptText.includes('captured'), false);
assert.equal(
preservationReceiptText.includes(state.reviewFile.filePath),
false,
);
const preservationVerifyCommand = {
schemaVersion: 1,
operation: 'local.deployment.reconciliation.run-history.verify',
options: preservationCommand.options,
request: {
preservationId: preservationCommand.request.preservationId,
applicationId: preservationCommand.request.applicationId,
expectedPreservationDigest: preserved.preservationDigest,
decisionFilePath: state.reviewFile.filePath,
},
};
const preservationVerified = await verifyLocalReconciliationRunHistory(
preservationVerifyCommand,
);
assert.equal(preservationVerified.status, 'verified');
const completionRoot = path.join(
path.dirname(state.captureRoot),
'completion-run-history',
);
fs.mkdirSync(completionRoot, { mode: 0o700 });
const runHistory = {
preservationId: preservationCommand.request.preservationId,
expectedPreservationDigest: preserved.preservationDigest,
};
const completionCommand = {
schemaVersion: 2,
operation: 'local.deployment.reconciliation.complete',
options: {
deploymentRoot: state.deploymentRoot,
applicationRoot: state.applicationRoot,
completionRoot,
automation: null,
runHistory: {
runHistoryRoot,
decisionFilePath: state.reviewFile.filePath,
},
allowRootService: rootAcknowledgement(),
},
request: {
completionId: '00000000-0000-4000-8000-000000000505',
applicationId: application.applicationId,
expectedApplicationPlanDigest: application.applicationPlanDigest,
expectedHeadDigest: application.instanceHeadDigest,
automation: null,
runHistory,
completedAtMs: preservationCommand.request.preservedAtMs + 1,
},
};
const completed = await completeLocalReconciliation(completionCommand);
assert.equal(completed.adapterCount, 1);
const completionReceipt = JSON.parse(
fs.readFileSync(
path.join(
completionRoot,
completionCommand.request.completionId,
'receipt.json',
),
'utf8',
),
);
assert.equal(completionReceipt.schemaVersion, 2);
assert.deepEqual(
completionReceipt.domains.find((domain) => domain.domain === 'run_history'),
{
domain: 'run_history',
action: 'adapter_required',
evidenceKind: 'run_history_preservation',
evidenceDigest: preserved.preservationDigest,
},
);
const completionVerified = await verifyLocalReconciliationCompletion({
schemaVersion: 2,
operation: 'local.deployment.reconciliation.complete.verify',
options: completionCommand.options,
request: {
completionId: completionCommand.request.completionId,
applicationId: completionCommand.request.applicationId,
expectedCompletionDigest: completed.completionDigest,
automation: null,
runHistory,
},
});
assert.equal(completionVerified.status, 'verified');
const commandPath = path.join(
state.deploymentRoot,
'run-history-verify.json',
);
fs.writeFileSync(
commandPath,
`${JSON.stringify(preservationVerifyCommand)}\n`,
{ mode: 0o600 },
);
const cli = spawnSync(
process.execPath,
[
path.join(__dirname, '../dist/deployment/localDeploymentCli.js'),
'reconciliation-run-history-verify',
'--command-file',
commandPath,
],
{ encoding: 'utf8' },
);
assert.equal(cli.status, 0, cli.stderr);
assert.equal(JSON.parse(cli.stdout).status, 'verified');
assert.equal(cli.stdout.includes(runHistoryRoot), false);
assert.equal(cli.stdout.includes('CrontabStats'), false);
const preservationReceiptPath = path.join(
preservationDirectory,
'receipt.json',
);
const tamperedReceipt = JSON.parse(
fs.readFileSync(preservationReceiptPath, 'utf8'),
);
tamperedReceipt.legacyFactCount += 1;
fs.chmodSync(preservationDirectory, 0o700);
fs.chmodSync(preservationReceiptPath, 0o600);
fs.writeFileSync(
preservationReceiptPath,
`${JSON.stringify(tamperedReceipt, null, 2)}\n`,
);
fs.chmodSync(preservationReceiptPath, 0o400);
fs.chmodSync(preservationDirectory, 0o500);
await assert.rejects(
verifyLocalReconciliationRunHistory(preservationVerifyCommand),
/receipt binding is invalid/,
);
});
test('run history preservation rejects active target runs and review escalation', async (t) => {
const state = reviewCommitFixture(t, {
planId: '00000000-0000-4000-8000-000000000511',
reviewId: '00000000-0000-4000-8000-000000000512',
reviewSuffix: 'run-history-active',
createDefaultSidecars: false,
initializeDatabases: runHistoryDatabaseInitializer(),
mutateTarget(paths) {
return mutateRunHistoryTarget(paths, 'running');
},
});
const pageCommand = diagnosticCommand(state, state.prepared, {
database: 'target',
domain: 'run_history',
factKind: 'table',
outputName: 'active-run-history.json',
});
writeLocalReconciliationReviewDiagnostics(pageCommand);
const page = JSON.parse(
fs.readFileSync(pageCommand.request.outputPath, 'utf8'),
);
assert.ok(page.records.length > 0);
assert.equal(
page.records.every(
(fact) =>
fact.decisionRequirement === 'blocked' &&
fact.reason === 'historical_integrity_required',
),
true,
);
const selected = state.reviewFile.records.find(
(record) =>
record.kind === 'qinglong3-local-reconciliation-review-decision' &&
record.database === 'target' &&
record.domain === 'run_history',
);
assert.ok(selected);
selected.disposition = 'retain_target';
selected.reason = 'preserve_target';
fs.writeFileSync(
state.reviewFile.filePath,
`${state.reviewFile.records
.map((record) => JSON.stringify(record))
.join('\n')}\n`,
{ mode: 0o600 },
);
await assert.rejects(
commitLocalReconciliationReview(state.command, state.dependencies),
/decision disposition is not allowed for canonical fact/,
);
assert.equal(
fs.existsSync(
path.join(
state.reviewRoot,
state.reviewCommand.request.reviewId,
'review.json',
),
),
false,
);
});
test('automation adapter builds a sealed row plan with bounded conflict evidence', async (t) => {
const state = await reviewedApplicationFixture(t, {
planId: '00000000-0000-4000-8000-000000000431',
+2 -2
View File
@@ -207,10 +207,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
rootSourceFileRoles: localOwnerCli.rootSourceFileRoles,
},
{
sourceFiles: 172,
sourceFiles: 175,
rootSourceFiles: 1,
rootSourceLines: 50,
nestedSourceFiles: 171,
nestedSourceFiles: 174,
rootSourceFileRoles: { 'cli.ts': 'binary_entry' },
},
);