mirror of
https://github.com/whyour/qinglong.git
synced 2026-09-20 16:07:11 +08:00
feat(local): seal reconciliation apply storage
This commit is contained in:
@@ -11,7 +11,7 @@
|
||||
|
||||
最新增量证据(2026-08-22):
|
||||
|
||||
- D-393/ADR-0486(进行中):首个 Automation adapter 已先建立独立的有界逐行 plan fence,而没有提前取得 DML authority。既有
|
||||
- D-393/ADR-0486/ADR-0487(进行中):首个 Automation adapter 已先建立独立的有界逐行 plan fence,而没有提前取得 DML authority。既有
|
||||
Legacy Crontab classifier 被复用于 exact sealed Legacy source;每行只记录 source/candidate digest、classification/reasons、proposed
|
||||
Task ID、trigger count,以及 captured target 当前 Task revision/content digest 的 `absent|occupied` 冲突证据,不保存 command/spec、Task
|
||||
name、reviewer、credential、Secret、路径或 target row body。planner 重新验证 D-392 application terminal、D-391 signed review 所绑定的原始
|
||||
@@ -70,15 +70,28 @@
|
||||
apply receipt 绑定 publication digest、adopted Task/Trigger/skipped counters、写后完整 SQLite snapshot 和写前 backup;verify 同时检查 signed
|
||||
decision、instance head 与当前 target snapshot,stdout 不返回 reviewer、路径、row digest 或 command。rollback 不是裸文件覆盖:它先以当前 reviewer
|
||||
强认证和当前 Project Policy 重放已存在 publication,再要求 target 精确等于 apply snapshot,最后调用 `preserveDatabaseIdentity=true` 的 restore,
|
||||
保持 activation 所绑定的 SQLite inode。restore、rollback receipt 和 rollback head 的 response-loss 同样可恢复;可恢复 backup authority 因仍需支持
|
||||
apply 后显式 rollback,保持 current-UID `0700/0600`,不伪装成已经归档的 `0500/0400` terminal。聚焦 reconciliation 套件仍为
|
||||
保持 activation 所绑定的 SQLite inode。restore、rollback receipt 和 rollback head 的 response-loss 同样可恢复;该切片当时为继续支持显式
|
||||
rollback,暂将可恢复 backup authority 保持为 current-UID `0700/0600`,尚未声称 terminal retention 已完成。聚焦 reconciliation 套件仍为
|
||||
`44 total / 42 pass / 2 conditional Docker skip / 0 fail`,但成功场景现在额外覆盖首次 apply、mutation replay、不同 reviewer 拒绝、六个
|
||||
backup/apply/rollback response-loss 窗口、content-free CLI verify、identity-preserving rollback 与 rollback replay。完整 Local Owner 在受限沙箱为
|
||||
`266 total / 256 pass / 7 conditional skip / 3 loopback-listen EPERM`,两个对应文件在沙箱外 `15/15` 通过;package/dependency boundary 为
|
||||
`70/70`。workspace 仍为 18 packages、`singleSourcePackages=[]`、`shallowSourcePackages=[]`,Local Owner 为
|
||||
`168 source / 167 nested / 1 root binary entry`,没有新增 package、production dependency、daemon、listener、timer 或 `src/` 根平铺。
|
||||
D-393 仍未代表完整 reconciliation:Automation 之外的 Secret、Plugin、Identity、history 等领域 adapter 尚未 apply,target service 也未获得 restart
|
||||
authority;后续必须先完成 apply-root 的 retention/seal 策略和跨领域 completion fence,再进入 target restart/readiness。
|
||||
第四切片以 ADR-0487 收敛 apply-root 的 retention/seal,而没有引入后台 GC。每个 apply 现在把 immutable evidence 与 mutable
|
||||
`rollback-work/` 分离:apply receipt 与 applied head 完成后,root/backup 和 intent/receipt/backup 分别封为 `0500/0400`,只保留一个空的
|
||||
current-UID `0700` rollback workspace。rollback 先把只读 backup 复制到该隔离工作区,以固定 64 KiB buffer 复算 SHA-256,再复用
|
||||
identity-preserving restore;restore response loss 会先由当前 target snapshot 裁决,不会因临时副本残留错误退回 applied 路径。rollback receipt/head
|
||||
收敛后,同步删除 restore 临时材料和数据库等量 backup,只保留三个 `0400` receipt/intent 证据,并把三层目录封为 `0500`。Applied backup 在跨领域
|
||||
completion fence 接受前仍禁止自动删除,因此 rollback safety 与低配设备空间回收没有互相冒充。
|
||||
|
||||
新增实现是既有 Automation 目录中的 `applyStorage.ts`,没有新增 package、production dependency、daemon、timer、watcher、listener、SQL migration、
|
||||
PostgreSQL ACL/Pool 或 cluster workload;hash/read/copy 均为固定内存。聚焦 reconciliation 为
|
||||
`44 total / 42 pass / 2 conditional Docker skip / 0 fail`,覆盖 apply head/seal 与 rollback restore/receipt/head/seal 共十个 response-loss 窗口、
|
||||
mode/catalog、backup 回收和 terminal replay;完整 Local Owner 在真实 loopback 环境为 `266 total / 259 pass / 7 conditional skip / 0 fail`,18-package
|
||||
clean build/逐包测试退出 0,真实 Docker reconciliation `2/2`,package/dependency boundary `70/70`。workspace 仍为 18 packages、`singleSourcePackages=[]`、
|
||||
`shallowSourcePackages=[]`;Local Owner 为 `169 source / 168 nested / 1 root binary entry`。D-393 仍未代表完整 reconciliation:Automation 之外的
|
||||
Secret、Plugin、Identity、history 等领域 adapter 尚未 apply,target service 也未获得 restart authority;下一切片是跨领域 completion fence,完成后才可进入
|
||||
target restart/readiness。
|
||||
- D-392/ADR-0485(已接受):D-391 的 signed review 不能直接获得通用 DML authority;表级 `adopt_legacy/retain_both` 也不能证明
|
||||
Automation 行级 command/trigger 兼容,更不能覆盖 Secret custody、append-only history、Plugin/AI 外部资产与 Identity/Policy 语义。
|
||||
因此既有 Local Owner 新增 `reconciliation.application.prepare|commit|verify`,以
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
# ADR-0487:Reconciliation Apply 证据封存与回滚存储生命周期
|
||||
|
||||
- 状态:Accepted
|
||||
- 日期:2026-08-22
|
||||
- 决策:D-393
|
||||
- 关联:ADR-0309、ADR-0314、ADR-0482、ADR-0485、ADR-0486
|
||||
|
||||
## 背景
|
||||
|
||||
Automation apply 已具备写前 SQLite backup、原子 Task/Trigger adoption、写后 receipt、instance-head CAS 和显式全库 rollback,但第一版为了让 restore primitive 随时可写临时文件,把整个 apply root 和 backup 长期保留为 `0700/0600`。这同时产生两个问题:
|
||||
|
||||
- 已完成的 intent、receipt 和 backup 仍可写,terminal evidence 没有最小权限封存;
|
||||
- rollback 完成后仍保留一份与数据库同量级的 backup,低容量路由设备会永久承担无用写放大和磁盘占用。
|
||||
|
||||
直接把整棵目录封为 `0500/0400` 又不可行。既有 identity-preserving restore 要求 source、stage、replaced file 的父目录可写且为当前 UID 的 `0700` 私有目录;为 restore 临时解封整棵 evidence root 会扩大可变范围,并使 crash recovery 无法区分 immutable evidence 与 mutable work material。
|
||||
|
||||
## 决策
|
||||
|
||||
### 1. 把证据与回滚工作区分离
|
||||
|
||||
每个 Automation apply 使用固定布局:
|
||||
|
||||
```text
|
||||
apply/<automationId>/
|
||||
├── intent.json
|
||||
├── receipt.json
|
||||
├── backup/
|
||||
│ └── before.sqlite
|
||||
└── rollback-work/
|
||||
├── restore-source.sqlite
|
||||
├── restore-stage.sqlite
|
||||
├── replaced.sqlite
|
||||
└── receipt.json
|
||||
```
|
||||
|
||||
`intent.json`、apply `receipt.json` 与 `backup/before.sqlite` 是已认证的 immutable evidence;`rollback-work/` 是预先建立的短生命周期 mutable workspace。固定 exact catalog 拒绝额外文件、目录和 symlink,不允许调用方提供任意临时路径。
|
||||
|
||||
### 2. Applied 状态立即最小权限封存
|
||||
|
||||
apply receipt 与 `reconciliation_automation_applied` head 都已发布后:
|
||||
|
||||
- root、`backup/` 封为 `0500`;
|
||||
- intent、apply receipt、backup 封为 `0400`;
|
||||
- `rollback-work/` 保持当前 UID `0700` 且必须为空。
|
||||
|
||||
封存前以固定 64 KiB buffer 流式重算 backup SHA-256、字节数和 inode/owner/mode/link identity。JSON evidence 使用有界 stable-descriptor read;terminal 验证要求单 link `0400`,不会把遗留 hard-link stage 当作已封存成功。
|
||||
|
||||
封存是幂等、可恢复的。文件、backup directory、root 依次收紧;因此任一步骤掉电后只会形成受审的 `0700|0500`、`0600|0400` 中间组合,exact replay 可继续收紧,不能重新放宽 immutable evidence。
|
||||
|
||||
### 3. Rollback 只在隔离工作区恢复
|
||||
|
||||
rollback 不直接把 `0400` backup 交给 restore。当前 reviewer、Project Policy、publication、applied head 和 target-after snapshot 复验完成后,将 backup 复制为 `rollback-work/restore-source.sqlite`,以固定内存重算相同 SHA-256 并要求 `0600` 单 link。restore stage、replaced database 和 rollback receipt 都只出现在该 `0700` 工作区。
|
||||
|
||||
这样既保留既有 `preserveDatabaseIdentity=true` 的恢复语义,也不需要解封 apply intent、receipt、root 或原 backup。restore response loss 时,下一次调用先检查当前 target snapshot;若已经等于 backup,则直接发布/重放 rollback receipt 和 head,不会因临时副本尚在而错误要求 applied 工作区为空。
|
||||
|
||||
### 4. Rolled-back 状态同步回收重资产
|
||||
|
||||
rollback receipt 与 `reconciliation_automation_rolled_back` head 收敛后:
|
||||
|
||||
- 删除 restore source、stage、replaced 等临时材料;
|
||||
- 删除已完成职责的 `backup/before.sqlite`;
|
||||
- 保留 intent、apply receipt 和 rollback receipt,全部封为 `0400`;
|
||||
- root、空 `backup/` 和只含 rollback receipt 的 `rollback-work/` 全部封为 `0500`。
|
||||
|
||||
备份删除不是后台 GC,也不依赖 daemon、timer、watcher 或数据库表。它发生在已经用 restored snapshot、rollback receipt 和 instance head 证明回滚完成的同一次短生命周期 Owner 命令中,因此路由设备没有额外常驻成本,集群节点也共享同一确定性状态机。
|
||||
|
||||
Applied backup 在尚未 rollback 时不得自动删除。跨 Automation、Secret、Plugin、Identity、history 等领域的全局 completion fence 尚未建立;在该 fence 接受之前,系统不能自行推断 rollback authority 已过期。
|
||||
|
||||
### 5. Profile 与部署规模
|
||||
|
||||
- Edge/Standalone:hash/read/copy 使用 64 KiB 固定 buffer,不把数据库读入内存;rolled-back 后同步释放数据库等量 backup。
|
||||
- 较大单机/集群节点:不增加包、production dependency、连接、SQL migration、Pool 或 cluster workload;可由未来独立 adapter 复用状态语义,但 PostgreSQL 必须有自己的 snapshot/HA authority,不能把本机文件复制当作集群备份。
|
||||
- 所有 Profile:verify 只读且不修复 drift;apply/rollback 的恢复行为只在显式 mutation command 内发生。
|
||||
|
||||
## 被拒绝的替代方案
|
||||
|
||||
### 永久保留 `0700/0600`
|
||||
|
||||
拒绝。它扩大 terminal evidence 可变面,并让已回滚设备永久保留无用数据库副本。
|
||||
|
||||
### Applied 后立即删除 backup
|
||||
|
||||
拒绝。显式 rollback 仍是当前 D-393 的必要能力;在跨领域 completion fence 前删除会把成功 apply 变成不可恢复的单向操作。
|
||||
|
||||
### Restore 时解封整个 apply root
|
||||
|
||||
拒绝。restore 只需要三个临时路径。隔离 `rollback-work/` 可以保持 immutable evidence 全程只读,并缩小掉电恢复状态空间。
|
||||
|
||||
### 后台定时 GC
|
||||
|
||||
拒绝。它为低配设备增加 timer、扫描、写唤醒和新的竞态,也无法仅凭文件年龄安全判断跨领域 rollback 已失效。
|
||||
|
||||
## 验收证据
|
||||
|
||||
- reconciliation 聚焦套件 `44 total / 42 pass / 2 conditional Docker skip / 0 fail`,覆盖 apply head/seal 与 rollback restore/receipt/head/seal 共十个 response-loss 窗口、实体 mode/catalog、identity-preserving restore、backup 回收及 terminal replay。
|
||||
- 完整 Local Owner 在真实 loopback 环境 `266 total / 259 pass / 7 conditional skip / 0 fail`;18-package clean build/逐包测试退出 0;真实 stopped-target Docker reconciliation `2/2`。
|
||||
- cluster dependency 与 package boundary 组合门 `70/70`,workspace 保持 18 packages、`singleSourcePackages=[]`、`shallowSourcePackages=[]`;Local Owner 为 `169 source / 168 nested / 1 root binary entry`。
|
||||
- 不新增 workspace package、production dependency、daemon、timer、watcher、listener、SQL migration、PostgreSQL role/ACL、Pool 或 cluster workload。
|
||||
- D-393 仍未完成:下一切片必须建立跨领域 completion fence;Secret、Plugin、Identity、history 等 adapter 及 target restart/readiness authority 仍关闭。
|
||||
+96
-114
@@ -1,5 +1,4 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
import { applyReconciliationAutomationDecision } from '@qinglong/local-admin/reconciliation-automation-decision';
|
||||
import { readPrivateLocalCommandFile } from '@qinglong/local-command-file';
|
||||
@@ -18,7 +17,6 @@ import type { SecurityPrincipal } from '@qinglong/runtime-core/security';
|
||||
import { currentIdentity } from '../../../foundation/contract';
|
||||
import { LocalDeploymentConfigurationError } from '../../../foundation/error';
|
||||
import {
|
||||
ensurePrivateDirectory,
|
||||
preflightPublishedFile,
|
||||
publishExactFile,
|
||||
validatePrivateDirectory,
|
||||
@@ -48,28 +46,28 @@ import {
|
||||
buildLocalReconciliationAutomationApplyReceipt,
|
||||
buildLocalReconciliationAutomationRollbackReceipt,
|
||||
localReconciliationAutomationApplyEvidenceContents,
|
||||
normalizeLocalReconciliationAutomationApplyIntent,
|
||||
normalizeLocalReconciliationAutomationApplyReceipt,
|
||||
normalizeLocalReconciliationAutomationRollbackReceipt,
|
||||
type LocalReconciliationAutomationApplyIntent,
|
||||
type LocalReconciliationAutomationApplyReceipt,
|
||||
} from './applyEvidence';
|
||||
import { readLocalReconciliationAutomationDecisionTerminal } from './decisionCoordinator';
|
||||
import { createLocalReconciliationAutomationRequirementFactory } from './planReader';
|
||||
import {
|
||||
ensureLocalReconciliationAutomationApplyLayout,
|
||||
localReconciliationAutomationApplyPaths,
|
||||
prepareLocalReconciliationAutomationRollbackSource,
|
||||
readLocalReconciliationAutomationApplyIntent,
|
||||
readLocalReconciliationAutomationApplyReceipt,
|
||||
readLocalReconciliationAutomationRollbackReceipt,
|
||||
sealLocalReconciliationAutomationAppliedStorage,
|
||||
sealLocalReconciliationAutomationRolledBackStorage,
|
||||
validateLocalReconciliationAutomationAppliedStorage,
|
||||
validateLocalReconciliationAutomationApplyCatalog,
|
||||
validateLocalReconciliationAutomationApplyLayout,
|
||||
validateLocalReconciliationAutomationRolledBackStorage,
|
||||
} from './applyStorage';
|
||||
|
||||
const MAX_AUTHENTICATION_AGE_MS = 5 * 60 * 1_000;
|
||||
|
||||
interface ApplyPaths {
|
||||
root: string;
|
||||
backupRoot: string;
|
||||
intent: string;
|
||||
backup: string;
|
||||
receipt: string;
|
||||
rollback: string;
|
||||
restoreStage: string;
|
||||
replaced: string;
|
||||
}
|
||||
|
||||
type AuthenticationDatabase = Awaited<
|
||||
ReturnType<typeof openLocalSqliteAuthenticationReadDatabase>
|
||||
>;
|
||||
@@ -85,8 +83,11 @@ export interface LocalReconciliationAutomationApplyDependencies {
|
||||
readonly afterDatabaseCommit?: () => void;
|
||||
readonly afterReceiptPublished?: () => void;
|
||||
readonly afterAppliedHead?: () => void;
|
||||
readonly afterAppliedSeal?: () => void;
|
||||
readonly afterRestore?: () => void;
|
||||
readonly afterRollbackReceipt?: () => void;
|
||||
readonly afterRollbackHead?: () => void;
|
||||
readonly afterRollbackSeal?: () => void;
|
||||
}
|
||||
|
||||
function fail(message: string, cause?: unknown): never {
|
||||
@@ -96,21 +97,6 @@ function fail(message: string, cause?: unknown): never {
|
||||
);
|
||||
}
|
||||
|
||||
function paths(root: string, automationId: string): Readonly<ApplyPaths> {
|
||||
const selected = path.join(root, automationId);
|
||||
const backupRoot = path.join(selected, 'backup');
|
||||
return Object.freeze({
|
||||
root: selected,
|
||||
backupRoot,
|
||||
intent: path.join(selected, 'intent.json'),
|
||||
backup: path.join(backupRoot, 'before.sqlite'),
|
||||
receipt: path.join(selected, 'receipt.json'),
|
||||
rollback: path.join(selected, 'rollback.json'),
|
||||
restoreStage: path.join(backupRoot, 'restore-stage.sqlite'),
|
||||
replaced: path.join(backupRoot, 'replaced.sqlite'),
|
||||
});
|
||||
}
|
||||
|
||||
function decisionOptions(
|
||||
options: Readonly<LocalReconciliationAutomationApplyOptions>,
|
||||
) {
|
||||
@@ -123,50 +109,6 @@ function decisionOptions(
|
||||
});
|
||||
}
|
||||
|
||||
function readIntent(
|
||||
selected: Readonly<ApplyPaths>,
|
||||
): Readonly<LocalReconciliationAutomationApplyIntent> {
|
||||
return normalizeLocalReconciliationAutomationApplyIntent(
|
||||
readPrivateLocalCommandFile(selected.intent),
|
||||
);
|
||||
}
|
||||
|
||||
function readReceipt(
|
||||
selected: Readonly<ApplyPaths>,
|
||||
): Readonly<LocalReconciliationAutomationApplyReceipt> {
|
||||
return normalizeLocalReconciliationAutomationApplyReceipt(
|
||||
readPrivateLocalCommandFile(selected.receipt),
|
||||
);
|
||||
}
|
||||
|
||||
function validateCatalog(selected: Readonly<ApplyPaths>): void {
|
||||
const allowed = new Set([
|
||||
'backup',
|
||||
'intent.json',
|
||||
'receipt.json',
|
||||
'rollback.json',
|
||||
'.intent.json.ql3-deploy-stage',
|
||||
'.receipt.json.ql3-deploy-stage',
|
||||
'.rollback.json.ql3-deploy-stage',
|
||||
]);
|
||||
for (const entry of fs.readdirSync(selected.root, { withFileTypes: true })) {
|
||||
if (!allowed.has(entry.name) || entry.isSymbolicLink())
|
||||
fail('apply root contains unknown material');
|
||||
}
|
||||
const backupAllowed = new Set([
|
||||
'before.sqlite',
|
||||
'.before.sqlite.ql3-backup-stage',
|
||||
'restore-stage.sqlite',
|
||||
'replaced.sqlite',
|
||||
]);
|
||||
for (const entry of fs.readdirSync(selected.backupRoot, {
|
||||
withFileTypes: true,
|
||||
})) {
|
||||
if (!backupAllowed.has(entry.name) || entry.isSymbolicLink())
|
||||
fail('backup root contains unknown material');
|
||||
}
|
||||
}
|
||||
|
||||
function advance(
|
||||
intent: Readonly<LocalReconciliationAutomationApplyIntent>,
|
||||
uid: number,
|
||||
@@ -333,17 +275,12 @@ export async function applyLocalReconciliationAutomation(
|
||||
[command.options.automationApplyRoot, 'automationApplyRoot'],
|
||||
] as const)
|
||||
validatePrivateDirectory(directory, uid, label);
|
||||
const selected = paths(
|
||||
const selected = localReconciliationAutomationApplyPaths(
|
||||
command.options.automationApplyRoot,
|
||||
command.request.automationId,
|
||||
);
|
||||
ensurePrivateDirectory(selected.root, uid, 'automation apply root');
|
||||
ensurePrivateDirectory(
|
||||
selected.backupRoot,
|
||||
uid,
|
||||
'automation apply backup root',
|
||||
);
|
||||
validateCatalog(selected);
|
||||
ensureLocalReconciliationAutomationApplyLayout(selected, uid);
|
||||
validateLocalReconciliationAutomationApplyCatalog(selected);
|
||||
|
||||
const terminal = await readLocalReconciliationAutomationDecisionTerminal(
|
||||
decisionOptions(command.options),
|
||||
@@ -374,7 +311,7 @@ export async function applyLocalReconciliationAutomation(
|
||||
uid,
|
||||
);
|
||||
if (fs.existsSync(selected.intent)) {
|
||||
intent = readIntent(selected);
|
||||
intent = readLocalReconciliationAutomationApplyIntent(selected, uid);
|
||||
verifyIntentCommand(intent, command);
|
||||
} else {
|
||||
if (
|
||||
@@ -427,7 +364,10 @@ export async function applyLocalReconciliationAutomation(
|
||||
}
|
||||
|
||||
if (fs.existsSync(selected.receipt)) {
|
||||
const receipt = readReceipt(selected);
|
||||
const receipt = readLocalReconciliationAutomationApplyReceipt(
|
||||
selected,
|
||||
uid,
|
||||
);
|
||||
const current = await (
|
||||
dependencies.inspectSnapshot ?? inspectLocalSqliteSnapshot
|
||||
)({
|
||||
@@ -453,6 +393,13 @@ export async function applyLocalReconciliationAutomation(
|
||||
receipt.applyDigest !== head.sourceRecordDigest
|
||||
)
|
||||
fail('terminal apply receipt drifted');
|
||||
sealLocalReconciliationAutomationAppliedStorage(
|
||||
selected,
|
||||
intent,
|
||||
receipt,
|
||||
uid,
|
||||
);
|
||||
dependencies.afterAppliedSeal?.();
|
||||
return result(command.operation, 'existing', receipt, intent, head);
|
||||
}
|
||||
if (
|
||||
@@ -586,6 +533,13 @@ export async function applyLocalReconciliationAutomation(
|
||||
command.request.appliedAtMs,
|
||||
);
|
||||
dependencies.afterAppliedHead?.();
|
||||
sealLocalReconciliationAutomationAppliedStorage(
|
||||
selected,
|
||||
intent,
|
||||
receipt,
|
||||
uid,
|
||||
);
|
||||
dependencies.afterAppliedSeal?.();
|
||||
return result(
|
||||
command.operation,
|
||||
publication.status === 'existing' ? 'existing' : 'applied',
|
||||
@@ -601,7 +555,7 @@ export async function verifyLocalReconciliationAutomationApply(
|
||||
const command =
|
||||
normalizeLocalReconciliationAutomationApplyVerifyCommand(value);
|
||||
const uid = currentIdentity().uid;
|
||||
const selected = paths(
|
||||
const selected = localReconciliationAutomationApplyPaths(
|
||||
command.options.automationApplyRoot,
|
||||
command.request.automationId,
|
||||
);
|
||||
@@ -610,15 +564,10 @@ export async function verifyLocalReconciliationAutomationApply(
|
||||
uid,
|
||||
'automationApplyRoot',
|
||||
);
|
||||
validatePrivateDirectory(selected.root, uid, 'automation apply root');
|
||||
validatePrivateDirectory(
|
||||
selected.backupRoot,
|
||||
uid,
|
||||
'automation apply backup root',
|
||||
);
|
||||
validateCatalog(selected);
|
||||
const intent = readIntent(selected);
|
||||
const receipt = readReceipt(selected);
|
||||
validateLocalReconciliationAutomationApplyLayout(selected, uid);
|
||||
validateLocalReconciliationAutomationApplyCatalog(selected);
|
||||
const intent = readLocalReconciliationAutomationApplyIntent(selected, uid);
|
||||
const receipt = readLocalReconciliationAutomationApplyReceipt(selected, uid);
|
||||
if (
|
||||
intent.command.request.decisionId !== command.request.decisionId ||
|
||||
receipt.decisionId !== command.request.decisionId ||
|
||||
@@ -637,6 +586,7 @@ export async function verifyLocalReconciliationAutomationApply(
|
||||
uid,
|
||||
);
|
||||
if (head.state === 'reconciliation_automation_applied') {
|
||||
validateLocalReconciliationAutomationAppliedStorage(selected, intent, uid);
|
||||
if (head.sourceRecordDigest !== receipt.applyDigest)
|
||||
fail('applied head drifted');
|
||||
const current = await inspectLocalSqliteSnapshot({
|
||||
@@ -649,10 +599,12 @@ export async function verifyLocalReconciliationAutomationApply(
|
||||
}
|
||||
if (
|
||||
head.state === 'reconciliation_automation_rolled_back' &&
|
||||
fs.existsSync(selected.rollback)
|
||||
fs.existsSync(selected.rollbackReceipt)
|
||||
) {
|
||||
const rollback = normalizeLocalReconciliationAutomationRollbackReceipt(
|
||||
readPrivateLocalCommandFile(selected.rollback),
|
||||
validateLocalReconciliationAutomationRolledBackStorage(selected, uid);
|
||||
const rollback = readLocalReconciliationAutomationRollbackReceipt(
|
||||
selected,
|
||||
uid,
|
||||
);
|
||||
if (
|
||||
rollback.applyDigest !== receipt.applyDigest ||
|
||||
@@ -680,7 +632,7 @@ export async function rollbackLocalReconciliationAutomationApply(
|
||||
const command =
|
||||
normalizeLocalReconciliationAutomationApplyRollbackCommand(value);
|
||||
const uid = currentIdentity().uid;
|
||||
const selected = paths(
|
||||
const selected = localReconciliationAutomationApplyPaths(
|
||||
command.options.automationApplyRoot,
|
||||
command.request.automationId,
|
||||
);
|
||||
@@ -689,15 +641,10 @@ export async function rollbackLocalReconciliationAutomationApply(
|
||||
uid,
|
||||
'automationApplyRoot',
|
||||
);
|
||||
validatePrivateDirectory(selected.root, uid, 'automation apply root');
|
||||
validatePrivateDirectory(
|
||||
selected.backupRoot,
|
||||
uid,
|
||||
'automation apply backup root',
|
||||
);
|
||||
validateCatalog(selected);
|
||||
const intent = readIntent(selected);
|
||||
const receipt = readReceipt(selected);
|
||||
validateLocalReconciliationAutomationApplyLayout(selected, uid);
|
||||
validateLocalReconciliationAutomationApplyCatalog(selected);
|
||||
const intent = readLocalReconciliationAutomationApplyIntent(selected, uid);
|
||||
const receipt = readLocalReconciliationAutomationApplyReceipt(selected, uid);
|
||||
if (
|
||||
receipt.decisionId !== command.request.decisionId ||
|
||||
receipt.automationId !== command.request.automationId ||
|
||||
@@ -714,9 +661,10 @@ export async function rollbackLocalReconciliationAutomationApply(
|
||||
intent.instanceId,
|
||||
uid,
|
||||
);
|
||||
if (fs.existsSync(selected.rollback)) {
|
||||
const rollback = normalizeLocalReconciliationAutomationRollbackReceipt(
|
||||
readPrivateLocalCommandFile(selected.rollback),
|
||||
if (fs.existsSync(selected.rollbackReceipt)) {
|
||||
const rollback = readLocalReconciliationAutomationRollbackReceipt(
|
||||
selected,
|
||||
uid,
|
||||
);
|
||||
const current = await (
|
||||
dependencies.inspectSnapshot ?? inspectLocalSqliteSnapshot
|
||||
@@ -737,11 +685,20 @@ export async function rollbackLocalReconciliationAutomationApply(
|
||||
rollback.rollbackDigest,
|
||||
rollback.rolledBackAtMs,
|
||||
);
|
||||
dependencies.afterRollbackHead?.();
|
||||
} else if (
|
||||
head.state !== 'reconciliation_automation_rolled_back' ||
|
||||
head.sourceRecordDigest !== rollback.rollbackDigest
|
||||
)
|
||||
fail('rollback replay drifted');
|
||||
sealLocalReconciliationAutomationRolledBackStorage(
|
||||
selected,
|
||||
intent,
|
||||
receipt,
|
||||
rollback,
|
||||
uid,
|
||||
);
|
||||
dependencies.afterRollbackSeal?.();
|
||||
return result(command.operation, 'existing', receipt, intent, head);
|
||||
}
|
||||
if (
|
||||
@@ -765,7 +722,7 @@ export async function rollbackLocalReconciliationAutomationApply(
|
||||
rolledBackAtMs: command.request.rolledBackAtMs,
|
||||
});
|
||||
publishExactFile(
|
||||
selected.rollback,
|
||||
selected.rollbackReceipt,
|
||||
localReconciliationAutomationApplyEvidenceContents(rollback),
|
||||
0o600,
|
||||
uid,
|
||||
@@ -779,10 +736,25 @@ export async function rollbackLocalReconciliationAutomationApply(
|
||||
rollback.rollbackDigest,
|
||||
command.request.rolledBackAtMs,
|
||||
);
|
||||
dependencies.afterRollbackHead?.();
|
||||
sealLocalReconciliationAutomationRolledBackStorage(
|
||||
selected,
|
||||
intent,
|
||||
receipt,
|
||||
rollback,
|
||||
uid,
|
||||
);
|
||||
dependencies.afterRollbackSeal?.();
|
||||
return result(command.operation, 'existing', receipt, intent, head);
|
||||
}
|
||||
if (current.sha256 !== receipt.targetAfter.sha256)
|
||||
fail('rollback current target drifted');
|
||||
sealLocalReconciliationAutomationAppliedStorage(
|
||||
selected,
|
||||
intent,
|
||||
receipt,
|
||||
uid,
|
||||
);
|
||||
|
||||
const authenticatedScope = await authenticate(
|
||||
command.options,
|
||||
@@ -855,11 +827,12 @@ export async function rollbackLocalReconciliationAutomationApply(
|
||||
} finally {
|
||||
await authenticatedScope.database.close();
|
||||
}
|
||||
prepareLocalReconciliationAutomationRollbackSource(selected, intent, uid);
|
||||
const restored = await (
|
||||
dependencies.restoreSnapshot ?? restoreLocalSqliteSnapshot
|
||||
)({
|
||||
databasePath: command.options.targetDatabasePath,
|
||||
sourceSnapshotPath: selected.backup,
|
||||
sourceSnapshotPath: selected.rollbackSource,
|
||||
restoreStagePath: selected.restoreStage,
|
||||
replacedDatabasePath: selected.replaced,
|
||||
expectedCurrentSha256: receipt.targetAfter.sha256,
|
||||
@@ -885,7 +858,7 @@ export async function rollbackLocalReconciliationAutomationApply(
|
||||
rolledBackAtMs: command.request.rolledBackAtMs,
|
||||
});
|
||||
publishExactFile(
|
||||
selected.rollback,
|
||||
selected.rollbackReceipt,
|
||||
localReconciliationAutomationApplyEvidenceContents(rollback),
|
||||
0o600,
|
||||
uid,
|
||||
@@ -899,6 +872,15 @@ export async function rollbackLocalReconciliationAutomationApply(
|
||||
rollback.rollbackDigest,
|
||||
command.request.rolledBackAtMs,
|
||||
);
|
||||
dependencies.afterRollbackHead?.();
|
||||
sealLocalReconciliationAutomationRolledBackStorage(
|
||||
selected,
|
||||
intent,
|
||||
receipt,
|
||||
rollback,
|
||||
uid,
|
||||
);
|
||||
dependencies.afterRollbackSeal?.();
|
||||
return result(command.operation, 'rolled_back', receipt, intent, head);
|
||||
}
|
||||
|
||||
|
||||
+605
@@ -0,0 +1,605 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
import { LocalDeploymentConfigurationError } from '../../../foundation/error';
|
||||
import type {
|
||||
LocalReconciliationAutomationApplyIntent,
|
||||
LocalReconciliationAutomationApplyReceipt,
|
||||
LocalReconciliationAutomationRollbackReceipt,
|
||||
} from './applyEvidence';
|
||||
import {
|
||||
normalizeLocalReconciliationAutomationApplyIntent,
|
||||
normalizeLocalReconciliationAutomationApplyReceipt,
|
||||
normalizeLocalReconciliationAutomationRollbackReceipt,
|
||||
} from './applyEvidence';
|
||||
|
||||
const MAX_EVIDENCE_BYTES = 64 * 1024;
|
||||
const HASH_BUFFER_BYTES = 64 * 1024;
|
||||
|
||||
export interface LocalReconciliationAutomationApplyPaths {
|
||||
readonly root: string;
|
||||
readonly backupRoot: string;
|
||||
readonly rollbackRoot: string;
|
||||
readonly intent: string;
|
||||
readonly backup: string;
|
||||
readonly receipt: string;
|
||||
readonly rollbackReceipt: string;
|
||||
readonly rollbackSource: string;
|
||||
readonly restoreStage: string;
|
||||
readonly replaced: string;
|
||||
}
|
||||
|
||||
function fail(message: string, cause?: unknown): never {
|
||||
throw new LocalDeploymentConfigurationError(
|
||||
`reconciliation automation apply storage ${message}`,
|
||||
{ cause },
|
||||
);
|
||||
}
|
||||
|
||||
export function localReconciliationAutomationApplyPaths(
|
||||
root: string,
|
||||
automationId: string,
|
||||
): Readonly<LocalReconciliationAutomationApplyPaths> {
|
||||
const selected = path.join(root, automationId);
|
||||
const backupRoot = path.join(selected, 'backup');
|
||||
const rollbackRoot = path.join(selected, 'rollback-work');
|
||||
return Object.freeze({
|
||||
root: selected,
|
||||
backupRoot,
|
||||
rollbackRoot,
|
||||
intent: path.join(selected, 'intent.json'),
|
||||
backup: path.join(backupRoot, 'before.sqlite'),
|
||||
receipt: path.join(selected, 'receipt.json'),
|
||||
rollbackReceipt: path.join(rollbackRoot, 'receipt.json'),
|
||||
rollbackSource: path.join(rollbackRoot, 'restore-source.sqlite'),
|
||||
restoreStage: path.join(rollbackRoot, 'restore-stage.sqlite'),
|
||||
replaced: path.join(rollbackRoot, 'replaced.sqlite'),
|
||||
});
|
||||
}
|
||||
|
||||
function directoryMode(
|
||||
directory: string,
|
||||
uid: number,
|
||||
modes: readonly number[],
|
||||
label: string,
|
||||
): number {
|
||||
let stat: fs.Stats;
|
||||
try {
|
||||
stat = fs.lstatSync(directory);
|
||||
} catch (error) {
|
||||
return fail(`${label} is unavailable`, error);
|
||||
}
|
||||
const mode = stat.mode & 0o777;
|
||||
if (
|
||||
!stat.isDirectory() ||
|
||||
stat.isSymbolicLink() ||
|
||||
stat.uid !== uid ||
|
||||
!modes.includes(mode) ||
|
||||
fs.realpathSync(directory) !== directory
|
||||
) {
|
||||
fail(`${label} identity is invalid`);
|
||||
}
|
||||
return mode;
|
||||
}
|
||||
|
||||
function ensureDirectory(directory: string, uid: number, label: string): void {
|
||||
try {
|
||||
fs.mkdirSync(directory, { mode: 0o700 });
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code !== 'EEXIST') {
|
||||
fail(`${label} cannot be created`, error);
|
||||
}
|
||||
}
|
||||
directoryMode(directory, uid, [0o700], label);
|
||||
}
|
||||
|
||||
export function ensureLocalReconciliationAutomationApplyLayout(
|
||||
selected: Readonly<LocalReconciliationAutomationApplyPaths>,
|
||||
uid: number,
|
||||
): void {
|
||||
if (!fs.existsSync(selected.root)) {
|
||||
ensureDirectory(selected.root, uid, 'root');
|
||||
}
|
||||
const rootMode = directoryMode(selected.root, uid, [0o700, 0o500], 'root');
|
||||
if (rootMode === 0o700) {
|
||||
if (!fs.existsSync(selected.backupRoot)) {
|
||||
ensureDirectory(selected.backupRoot, uid, 'backup root');
|
||||
}
|
||||
if (!fs.existsSync(selected.rollbackRoot)) {
|
||||
ensureDirectory(selected.rollbackRoot, uid, 'rollback work root');
|
||||
}
|
||||
}
|
||||
directoryMode(selected.backupRoot, uid, [0o700, 0o500], 'backup root');
|
||||
directoryMode(
|
||||
selected.rollbackRoot,
|
||||
uid,
|
||||
[0o700, 0o500],
|
||||
'rollback work root',
|
||||
);
|
||||
}
|
||||
|
||||
export function validateLocalReconciliationAutomationApplyLayout(
|
||||
selected: Readonly<LocalReconciliationAutomationApplyPaths>,
|
||||
uid: number,
|
||||
): void {
|
||||
directoryMode(selected.root, uid, [0o700, 0o500], 'root');
|
||||
directoryMode(selected.backupRoot, uid, [0o700, 0o500], 'backup root');
|
||||
directoryMode(
|
||||
selected.rollbackRoot,
|
||||
uid,
|
||||
[0o700, 0o500],
|
||||
'rollback work root',
|
||||
);
|
||||
}
|
||||
|
||||
export function validateLocalReconciliationAutomationApplyCatalog(
|
||||
selected: Readonly<LocalReconciliationAutomationApplyPaths>,
|
||||
): void {
|
||||
const rootAllowed = new Set([
|
||||
'backup',
|
||||
'rollback-work',
|
||||
'intent.json',
|
||||
'receipt.json',
|
||||
'.intent.json.ql3-deploy-stage',
|
||||
'.receipt.json.ql3-deploy-stage',
|
||||
]);
|
||||
for (const entry of fs.readdirSync(selected.root, { withFileTypes: true })) {
|
||||
if (!rootAllowed.has(entry.name) || entry.isSymbolicLink()) {
|
||||
fail('root contains unknown material');
|
||||
}
|
||||
}
|
||||
const backupAllowed = new Set([
|
||||
'before.sqlite',
|
||||
'.before.sqlite.ql3-backup-stage',
|
||||
]);
|
||||
for (const entry of fs.readdirSync(selected.backupRoot, {
|
||||
withFileTypes: true,
|
||||
})) {
|
||||
if (!backupAllowed.has(entry.name) || entry.isSymbolicLink()) {
|
||||
fail('backup root contains unknown material');
|
||||
}
|
||||
}
|
||||
const rollbackAllowed = new Set([
|
||||
'receipt.json',
|
||||
'.receipt.json.ql3-deploy-stage',
|
||||
'restore-source.sqlite',
|
||||
'restore-stage.sqlite',
|
||||
'replaced.sqlite',
|
||||
]);
|
||||
for (const entry of fs.readdirSync(selected.rollbackRoot, {
|
||||
withFileTypes: true,
|
||||
})) {
|
||||
if (!rollbackAllowed.has(entry.name) || entry.isSymbolicLink()) {
|
||||
fail('rollback work root contains unknown material');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function stableJson(
|
||||
filePath: string,
|
||||
uid: number,
|
||||
modes: readonly number[],
|
||||
label: string,
|
||||
links: readonly bigint[] = [1n, 2n],
|
||||
): unknown {
|
||||
let descriptor: number | undefined;
|
||||
let bytes: Buffer | undefined;
|
||||
try {
|
||||
const before = fs.lstatSync(filePath, { bigint: true });
|
||||
if (
|
||||
!before.isFile() ||
|
||||
before.isSymbolicLink() ||
|
||||
Number(before.uid) !== uid ||
|
||||
!modes.includes(Number(before.mode) & 0o777) ||
|
||||
!links.includes(before.nlink) ||
|
||||
before.size < 2n ||
|
||||
before.size > BigInt(MAX_EVIDENCE_BYTES)
|
||||
) {
|
||||
fail(`${label} identity is invalid`);
|
||||
}
|
||||
descriptor = fs.openSync(
|
||||
filePath,
|
||||
fs.constants.O_RDONLY | (fs.constants.O_NOFOLLOW ?? 0),
|
||||
);
|
||||
const opened = fs.fstatSync(descriptor, { bigint: true });
|
||||
if (
|
||||
opened.dev !== before.dev ||
|
||||
opened.ino !== before.ino ||
|
||||
opened.size !== before.size ||
|
||||
opened.mtimeNs !== before.mtimeNs ||
|
||||
opened.ctimeNs !== before.ctimeNs ||
|
||||
opened.mode !== before.mode ||
|
||||
opened.nlink !== before.nlink
|
||||
) {
|
||||
fail(`${label} changed while opening`);
|
||||
}
|
||||
bytes = Buffer.alloc(Number(opened.size));
|
||||
let offset = 0;
|
||||
while (offset < bytes.length) {
|
||||
const count = fs.readSync(
|
||||
descriptor,
|
||||
bytes,
|
||||
offset,
|
||||
bytes.length - offset,
|
||||
offset,
|
||||
);
|
||||
if (count < 1) fail(`${label} read stalled`);
|
||||
offset += count;
|
||||
}
|
||||
const after = fs.fstatSync(descriptor, { bigint: true });
|
||||
if (
|
||||
after.dev !== opened.dev ||
|
||||
after.ino !== opened.ino ||
|
||||
after.size !== opened.size ||
|
||||
after.mtimeNs !== opened.mtimeNs ||
|
||||
after.ctimeNs !== opened.ctimeNs ||
|
||||
after.mode !== opened.mode ||
|
||||
after.nlink !== opened.nlink
|
||||
) {
|
||||
fail(`${label} drifted while reading`);
|
||||
}
|
||||
return JSON.parse(bytes.toString('utf8')) as unknown;
|
||||
} catch (error) {
|
||||
if (error instanceof LocalDeploymentConfigurationError) throw error;
|
||||
return fail(`${label} cannot be read`, error);
|
||||
} finally {
|
||||
bytes?.fill(0);
|
||||
if (descriptor !== undefined) fs.closeSync(descriptor);
|
||||
}
|
||||
}
|
||||
|
||||
export function readLocalReconciliationAutomationApplyIntent(
|
||||
selected: Readonly<LocalReconciliationAutomationApplyPaths>,
|
||||
uid: number,
|
||||
): Readonly<LocalReconciliationAutomationApplyIntent> {
|
||||
return normalizeLocalReconciliationAutomationApplyIntent(
|
||||
stableJson(selected.intent, uid, [0o600, 0o400], 'intent'),
|
||||
);
|
||||
}
|
||||
|
||||
export function readLocalReconciliationAutomationApplyReceipt(
|
||||
selected: Readonly<LocalReconciliationAutomationApplyPaths>,
|
||||
uid: number,
|
||||
): Readonly<LocalReconciliationAutomationApplyReceipt> {
|
||||
return normalizeLocalReconciliationAutomationApplyReceipt(
|
||||
stableJson(selected.receipt, uid, [0o600, 0o400], 'receipt'),
|
||||
);
|
||||
}
|
||||
|
||||
export function readLocalReconciliationAutomationRollbackReceipt(
|
||||
selected: Readonly<LocalReconciliationAutomationApplyPaths>,
|
||||
uid: number,
|
||||
): Readonly<LocalReconciliationAutomationRollbackReceipt> {
|
||||
return normalizeLocalReconciliationAutomationRollbackReceipt(
|
||||
stableJson(
|
||||
selected.rollbackReceipt,
|
||||
uid,
|
||||
[0o600, 0o400],
|
||||
'rollback receipt',
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
function syncDirectory(directory: string): void {
|
||||
const descriptor = fs.openSync(directory, fs.constants.O_RDONLY);
|
||||
try {
|
||||
fs.fsyncSync(descriptor);
|
||||
} finally {
|
||||
fs.closeSync(descriptor);
|
||||
}
|
||||
}
|
||||
|
||||
function syncFile(filePath: string): void {
|
||||
const descriptor = fs.openSync(
|
||||
filePath,
|
||||
fs.constants.O_RDONLY | (fs.constants.O_NOFOLLOW ?? 0),
|
||||
);
|
||||
try {
|
||||
fs.fsyncSync(descriptor);
|
||||
} finally {
|
||||
fs.closeSync(descriptor);
|
||||
}
|
||||
}
|
||||
|
||||
function stableFileSha256(
|
||||
filePath: string,
|
||||
uid: number,
|
||||
modes: readonly number[],
|
||||
expectedBytes: number,
|
||||
expectedSha256: string,
|
||||
label: string,
|
||||
): void {
|
||||
let descriptor: number | undefined;
|
||||
const buffer = Buffer.alloc(HASH_BUFFER_BYTES);
|
||||
try {
|
||||
const before = fs.lstatSync(filePath, { bigint: true });
|
||||
if (
|
||||
!before.isFile() ||
|
||||
before.isSymbolicLink() ||
|
||||
Number(before.uid) !== uid ||
|
||||
!modes.includes(Number(before.mode) & 0o777) ||
|
||||
before.nlink !== 1n ||
|
||||
before.size !== BigInt(expectedBytes) ||
|
||||
fs.realpathSync(filePath) !== filePath
|
||||
) {
|
||||
fail(`${label} identity is invalid`);
|
||||
}
|
||||
descriptor = fs.openSync(
|
||||
filePath,
|
||||
fs.constants.O_RDONLY | (fs.constants.O_NOFOLLOW ?? 0),
|
||||
);
|
||||
const opened = fs.fstatSync(descriptor, { bigint: true });
|
||||
if (
|
||||
opened.dev !== before.dev ||
|
||||
opened.ino !== before.ino ||
|
||||
opened.size !== before.size ||
|
||||
opened.mtimeNs !== before.mtimeNs ||
|
||||
opened.ctimeNs !== before.ctimeNs ||
|
||||
opened.mode !== before.mode ||
|
||||
opened.nlink !== before.nlink
|
||||
) {
|
||||
fail(`${label} changed while opening`);
|
||||
}
|
||||
const hash = createHash('sha256');
|
||||
let offset = 0;
|
||||
while (offset < expectedBytes) {
|
||||
const count = fs.readSync(
|
||||
descriptor,
|
||||
buffer,
|
||||
0,
|
||||
Math.min(buffer.length, expectedBytes - offset),
|
||||
offset,
|
||||
);
|
||||
if (count < 1) fail(`${label} read stalled`);
|
||||
hash.update(buffer.subarray(0, count));
|
||||
offset += count;
|
||||
}
|
||||
const after = fs.fstatSync(descriptor, { bigint: true });
|
||||
if (
|
||||
after.dev !== opened.dev ||
|
||||
after.ino !== opened.ino ||
|
||||
after.size !== opened.size ||
|
||||
after.mtimeNs !== opened.mtimeNs ||
|
||||
after.ctimeNs !== opened.ctimeNs ||
|
||||
after.mode !== opened.mode ||
|
||||
after.nlink !== opened.nlink ||
|
||||
hash.digest('hex') !== expectedSha256
|
||||
) {
|
||||
fail(`${label} content drifted`);
|
||||
}
|
||||
} catch (error) {
|
||||
if (error instanceof LocalDeploymentConfigurationError) throw error;
|
||||
fail(`${label} cannot be verified`, error);
|
||||
} finally {
|
||||
buffer.fill(0);
|
||||
if (descriptor !== undefined) fs.closeSync(descriptor);
|
||||
}
|
||||
}
|
||||
|
||||
function removeExactHardLink(
|
||||
targetPath: string,
|
||||
stagePath: string,
|
||||
uid: number,
|
||||
label: string,
|
||||
): void {
|
||||
if (!fs.existsSync(stagePath)) return;
|
||||
const target = fs.lstatSync(targetPath, { bigint: true });
|
||||
const stage = fs.lstatSync(stagePath, { bigint: true });
|
||||
if (
|
||||
!target.isFile() ||
|
||||
!stage.isFile() ||
|
||||
target.isSymbolicLink() ||
|
||||
stage.isSymbolicLink() ||
|
||||
Number(target.uid) !== uid ||
|
||||
Number(stage.uid) !== uid ||
|
||||
target.dev !== stage.dev ||
|
||||
target.ino !== stage.ino ||
|
||||
target.nlink !== 2n ||
|
||||
stage.nlink !== 2n
|
||||
) {
|
||||
fail(`${label} stage identity drifted`);
|
||||
}
|
||||
fs.unlinkSync(stagePath);
|
||||
syncDirectory(path.dirname(stagePath));
|
||||
}
|
||||
|
||||
function removeExactStage(
|
||||
targetPath: string,
|
||||
uid: number,
|
||||
label: string,
|
||||
): void {
|
||||
removeExactHardLink(
|
||||
targetPath,
|
||||
path.join(
|
||||
path.dirname(targetPath),
|
||||
`.${path.basename(targetPath)}.ql3-deploy-stage`,
|
||||
),
|
||||
uid,
|
||||
label,
|
||||
);
|
||||
}
|
||||
|
||||
function sealFile(filePath: string, uid: number, label: string): void {
|
||||
removeExactStage(filePath, uid, label);
|
||||
const before = fs.lstatSync(filePath);
|
||||
if (
|
||||
!before.isFile() ||
|
||||
before.isSymbolicLink() ||
|
||||
before.uid !== uid ||
|
||||
before.nlink !== 1 ||
|
||||
![0o600, 0o400].includes(before.mode & 0o777)
|
||||
) {
|
||||
fail(`${label} cannot be sealed`);
|
||||
}
|
||||
if ((before.mode & 0o777) !== 0o400) fs.chmodSync(filePath, 0o400);
|
||||
syncFile(filePath);
|
||||
}
|
||||
|
||||
function sealDirectory(directory: string, uid: number, label: string): void {
|
||||
const mode = directoryMode(directory, uid, [0o700, 0o500], label);
|
||||
if (mode !== 0o500) fs.chmodSync(directory, 0o500);
|
||||
syncDirectory(directory);
|
||||
}
|
||||
|
||||
function emptyDirectory(directory: string, label: string): void {
|
||||
if (fs.readdirSync(directory).length !== 0) {
|
||||
fail(`${label} must be empty`);
|
||||
}
|
||||
}
|
||||
|
||||
function validateBackup(
|
||||
selected: Readonly<LocalReconciliationAutomationApplyPaths>,
|
||||
intent: Readonly<LocalReconciliationAutomationApplyIntent>,
|
||||
uid: number,
|
||||
modes: readonly number[],
|
||||
): void {
|
||||
stableFileSha256(
|
||||
selected.backup,
|
||||
uid,
|
||||
modes,
|
||||
intent.backup.bytes,
|
||||
intent.backup.sha256,
|
||||
'backup',
|
||||
);
|
||||
}
|
||||
|
||||
export function sealLocalReconciliationAutomationAppliedStorage(
|
||||
selected: Readonly<LocalReconciliationAutomationApplyPaths>,
|
||||
intent: Readonly<LocalReconciliationAutomationApplyIntent>,
|
||||
_receipt: Readonly<LocalReconciliationAutomationApplyReceipt>,
|
||||
uid: number,
|
||||
): void {
|
||||
validateLocalReconciliationAutomationApplyLayout(selected, uid);
|
||||
validateLocalReconciliationAutomationApplyCatalog(selected);
|
||||
removeExactHardLink(
|
||||
selected.backup,
|
||||
path.join(selected.backupRoot, '.before.sqlite.ql3-backup-stage'),
|
||||
uid,
|
||||
'backup',
|
||||
);
|
||||
validateBackup(selected, intent, uid, [0o600, 0o400]);
|
||||
emptyDirectory(selected.rollbackRoot, 'rollback work root');
|
||||
sealFile(selected.intent, uid, 'intent');
|
||||
sealFile(selected.receipt, uid, 'receipt');
|
||||
sealFile(selected.backup, uid, 'backup');
|
||||
sealDirectory(selected.backupRoot, uid, 'backup root');
|
||||
sealDirectory(selected.root, uid, 'root');
|
||||
validateLocalReconciliationAutomationAppliedStorage(selected, intent, uid);
|
||||
}
|
||||
|
||||
export function validateLocalReconciliationAutomationAppliedStorage(
|
||||
selected: Readonly<LocalReconciliationAutomationApplyPaths>,
|
||||
intent: Readonly<LocalReconciliationAutomationApplyIntent>,
|
||||
uid: number,
|
||||
): void {
|
||||
directoryMode(selected.root, uid, [0o500], 'root');
|
||||
directoryMode(selected.backupRoot, uid, [0o500], 'backup root');
|
||||
directoryMode(selected.rollbackRoot, uid, [0o700], 'rollback work root');
|
||||
validateLocalReconciliationAutomationApplyCatalog(selected);
|
||||
emptyDirectory(selected.rollbackRoot, 'rollback work root');
|
||||
stableJson(selected.intent, uid, [0o400], 'intent', [1n]);
|
||||
stableJson(selected.receipt, uid, [0o400], 'receipt', [1n]);
|
||||
validateBackup(selected, intent, uid, [0o400]);
|
||||
}
|
||||
|
||||
export function prepareLocalReconciliationAutomationRollbackSource(
|
||||
selected: Readonly<LocalReconciliationAutomationApplyPaths>,
|
||||
intent: Readonly<LocalReconciliationAutomationApplyIntent>,
|
||||
uid: number,
|
||||
): void {
|
||||
directoryMode(selected.root, uid, [0o500], 'root');
|
||||
directoryMode(selected.backupRoot, uid, [0o500], 'backup root');
|
||||
directoryMode(selected.rollbackRoot, uid, [0o700], 'rollback work root');
|
||||
validateBackup(selected, intent, uid, [0o400]);
|
||||
if (!fs.existsSync(selected.rollbackSource)) {
|
||||
try {
|
||||
fs.copyFileSync(
|
||||
selected.backup,
|
||||
selected.rollbackSource,
|
||||
fs.constants.COPYFILE_EXCL,
|
||||
);
|
||||
fs.chmodSync(selected.rollbackSource, 0o600);
|
||||
syncFile(selected.rollbackSource);
|
||||
syncDirectory(selected.rollbackRoot);
|
||||
} catch (error) {
|
||||
if (fs.existsSync(selected.rollbackSource)) {
|
||||
try {
|
||||
fs.unlinkSync(selected.rollbackSource);
|
||||
syncDirectory(selected.rollbackRoot);
|
||||
} catch {
|
||||
// Deterministic residue remains fail-closed for exact replay.
|
||||
}
|
||||
}
|
||||
fail('rollback source cannot be prepared', error);
|
||||
}
|
||||
}
|
||||
stableFileSha256(
|
||||
selected.rollbackSource,
|
||||
uid,
|
||||
[0o600],
|
||||
intent.backup.bytes,
|
||||
intent.backup.sha256,
|
||||
'rollback source',
|
||||
);
|
||||
}
|
||||
|
||||
function unlinkIfPresent(filePath: string): void {
|
||||
if (!fs.existsSync(filePath)) return;
|
||||
fs.unlinkSync(filePath);
|
||||
syncDirectory(path.dirname(filePath));
|
||||
}
|
||||
|
||||
export function sealLocalReconciliationAutomationRolledBackStorage(
|
||||
selected: Readonly<LocalReconciliationAutomationApplyPaths>,
|
||||
intent: Readonly<LocalReconciliationAutomationApplyIntent>,
|
||||
_receipt: Readonly<LocalReconciliationAutomationApplyReceipt>,
|
||||
_rollback: Readonly<LocalReconciliationAutomationRollbackReceipt>,
|
||||
uid: number,
|
||||
): void {
|
||||
validateLocalReconciliationAutomationApplyLayout(selected, uid);
|
||||
validateLocalReconciliationAutomationApplyCatalog(selected);
|
||||
if (fs.existsSync(selected.backup)) {
|
||||
validateBackup(selected, intent, uid, [0o400]);
|
||||
}
|
||||
for (const temporary of [selected.restoreStage, selected.replaced]) {
|
||||
if (fs.existsSync(temporary)) {
|
||||
fail('rollback temporary evidence remains');
|
||||
}
|
||||
}
|
||||
unlinkIfPresent(selected.rollbackSource);
|
||||
if ((fs.statSync(selected.backupRoot).mode & 0o777) !== 0o700) {
|
||||
fs.chmodSync(selected.backupRoot, 0o700);
|
||||
syncDirectory(selected.root);
|
||||
}
|
||||
unlinkIfPresent(selected.backup);
|
||||
removeExactStage(selected.rollbackReceipt, uid, 'rollback receipt');
|
||||
sealFile(selected.intent, uid, 'intent');
|
||||
sealFile(selected.receipt, uid, 'receipt');
|
||||
sealFile(selected.rollbackReceipt, uid, 'rollback receipt');
|
||||
sealDirectory(selected.backupRoot, uid, 'backup root');
|
||||
sealDirectory(selected.rollbackRoot, uid, 'rollback work root');
|
||||
sealDirectory(selected.root, uid, 'root');
|
||||
validateLocalReconciliationAutomationRolledBackStorage(selected, uid);
|
||||
}
|
||||
|
||||
export function validateLocalReconciliationAutomationRolledBackStorage(
|
||||
selected: Readonly<LocalReconciliationAutomationApplyPaths>,
|
||||
uid: number,
|
||||
): void {
|
||||
directoryMode(selected.root, uid, [0o500], 'root');
|
||||
directoryMode(selected.backupRoot, uid, [0o500], 'backup root');
|
||||
directoryMode(selected.rollbackRoot, uid, [0o500], 'rollback work root');
|
||||
validateLocalReconciliationAutomationApplyCatalog(selected);
|
||||
emptyDirectory(selected.backupRoot, 'backup root');
|
||||
if (
|
||||
fs
|
||||
.readdirSync(selected.rollbackRoot)
|
||||
.some((name) => name !== 'receipt.json')
|
||||
) {
|
||||
fail('sealed rollback work root contains temporary material');
|
||||
}
|
||||
stableJson(selected.intent, uid, [0o400], 'intent', [1n]);
|
||||
stableJson(selected.receipt, uid, [0o400], 'receipt', [1n]);
|
||||
stableJson(selected.rollbackReceipt, uid, [0o400], 'rollback receipt', [1n]);
|
||||
}
|
||||
@@ -3288,7 +3288,12 @@ test('automation decision reauthenticates the same reviewer, seals exact row dec
|
||||
}),
|
||||
/current reviewer authentication is not strong or identical/,
|
||||
);
|
||||
for (const boundary of ['afterDatabaseCommit', 'afterReceiptPublished']) {
|
||||
for (const boundary of [
|
||||
'afterDatabaseCommit',
|
||||
'afterReceiptPublished',
|
||||
'afterAppliedHead',
|
||||
'afterAppliedSeal',
|
||||
]) {
|
||||
await assert.rejects(
|
||||
applyLocalReconciliationAutomation(applyCommand, {
|
||||
...applyDependencies,
|
||||
@@ -3307,6 +3312,34 @@ test('automation decision reauthenticates the same reviewer, seals exact row dec
|
||||
assert.equal(applied.state, 'reconciliation_automation_applied');
|
||||
assert.equal(applied.adoptedTaskCount, 1);
|
||||
assert.equal(fs.statSync(state.targetDatabasePath).ino, targetIdentity.ino);
|
||||
const applyEvidenceRoot = path.join(
|
||||
automationApplyRoot,
|
||||
state.automationCommand.request.automationId,
|
||||
);
|
||||
const applyBackupRoot = path.join(applyEvidenceRoot, 'backup');
|
||||
const rollbackWorkRoot = path.join(applyEvidenceRoot, 'rollback-work');
|
||||
assert.equal(fs.statSync(applyEvidenceRoot).mode & 0o777, 0o500);
|
||||
assert.equal(fs.statSync(applyBackupRoot).mode & 0o777, 0o500);
|
||||
assert.equal(fs.statSync(rollbackWorkRoot).mode & 0o777, 0o700);
|
||||
assert.equal(
|
||||
fs.statSync(path.join(applyEvidenceRoot, 'intent.json')).mode & 0o777,
|
||||
0o400,
|
||||
);
|
||||
assert.equal(
|
||||
fs.statSync(path.join(applyEvidenceRoot, 'receipt.json')).mode & 0o777,
|
||||
0o400,
|
||||
);
|
||||
assert.equal(
|
||||
fs.statSync(path.join(applyBackupRoot, 'before.sqlite')).mode & 0o777,
|
||||
0o400,
|
||||
);
|
||||
assert.deepEqual(fs.readdirSync(applyEvidenceRoot).sort(), [
|
||||
'backup',
|
||||
'intent.json',
|
||||
'receipt.json',
|
||||
'rollback-work',
|
||||
]);
|
||||
assert.deepEqual(fs.readdirSync(rollbackWorkRoot), []);
|
||||
const applyReplay = await applyLocalReconciliationAutomation(
|
||||
applyCommand,
|
||||
applyDependencies,
|
||||
@@ -3335,7 +3368,12 @@ test('automation decision reauthenticates the same reviewer, seals exact row dec
|
||||
rolledBackAtMs: appliedAtMs + 1,
|
||||
},
|
||||
};
|
||||
for (const boundary of ['afterRestore', 'afterRollbackReceipt']) {
|
||||
for (const boundary of [
|
||||
'afterRestore',
|
||||
'afterRollbackReceipt',
|
||||
'afterRollbackHead',
|
||||
'afterRollbackSeal',
|
||||
]) {
|
||||
await assert.rejects(
|
||||
rollbackLocalReconciliationAutomationApply(rollbackCommand, {
|
||||
...applyDependencies,
|
||||
@@ -3353,6 +3391,15 @@ test('automation decision reauthenticates the same reviewer, seals exact row dec
|
||||
assert.equal(rolledBack.status, 'existing');
|
||||
assert.equal(rolledBack.state, 'reconciliation_automation_rolled_back');
|
||||
assert.equal(fs.statSync(state.targetDatabasePath).ino, targetIdentity.ino);
|
||||
assert.equal(fs.statSync(applyEvidenceRoot).mode & 0o777, 0o500);
|
||||
assert.equal(fs.statSync(applyBackupRoot).mode & 0o777, 0o500);
|
||||
assert.equal(fs.statSync(rollbackWorkRoot).mode & 0o777, 0o500);
|
||||
assert.deepEqual(fs.readdirSync(applyBackupRoot), []);
|
||||
assert.deepEqual(fs.readdirSync(rollbackWorkRoot), ['receipt.json']);
|
||||
assert.equal(
|
||||
fs.statSync(path.join(rollbackWorkRoot, 'receipt.json')).mode & 0o777,
|
||||
0o400,
|
||||
);
|
||||
assert.equal(
|
||||
(
|
||||
await rollbackLocalReconciliationAutomationApply(
|
||||
|
||||
@@ -207,10 +207,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
|
||||
rootSourceFileRoles: localOwnerCli.rootSourceFileRoles,
|
||||
},
|
||||
{
|
||||
sourceFiles: 168,
|
||||
sourceFiles: 169,
|
||||
rootSourceFiles: 1,
|
||||
rootSourceLines: 50,
|
||||
nestedSourceFiles: 167,
|
||||
nestedSourceFiles: 168,
|
||||
rootSourceFileRoles: { 'cli.ts': 'binary_entry' },
|
||||
},
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user