mirror of
https://github.com/whyour/qinglong.git
synced 2026-09-20 16:07:11 +08:00
feat(ql3): apply legacy data transformation
This commit is contained in:
whyour
@@ -11,6 +11,26 @@
|
||||
|
||||
最新增量证据（2026-08-21）：
|
||||
|
||||
- D-387/ADR-0480（已接受）：把 D-386 prepared model 推进为受认证、Project-scoped、可审计且可恢复的原子 application。
|
||||
既有 `ql3-adoption` 增加 exact `local-data-directory.adoption.apply|apply.verify`；Owner credential 通过独立短生命周期 SQLite handle 建立
|
||||
`local_data_adoption` 强认证并要求目标 Project 的 `secret.manage`，publisher 在 `BEGIN IMMEDIATE` 内再次复验 credential、Project 与
|
||||
RoleBinding fence。SQLite contract v50 的 `0099/0100` 增加父 adoption ledger、逐 Secret binding、trigger、typed schema、readiness
|
||||
relation 和 `legacy_data_directory_adoption=1` capability；所有 AES-256-GCM Secret envelope、逐项 `secret.create` audit、父
|
||||
`legacy-data.apply` audit、disabled model 与 canonical receipt 在一个事务提交，任一目标冲突整批回滚。DB COMMIT 是逻辑提交点；随后
|
||||
`.commit-incomplete → model rename 为 .reclaiming-model → Secret 文件覆盖/fsync/unlink → content-free commit.json`，exact apply replay
|
||||
可从 durable receipt 收敛 COMMIT-response-loss 和清理中断，`apply.verify` 只验证完整终态。最终明确
|
||||
`physicalErasureGuaranteed=false`，不把闪存/CoW overwrite 冒充介质擦除，也不激活配置、SSH、Task 或 service cutover。实现内聚在既有
|
||||
Local Owner/Admin/SQLite 的 data-directory 子目录，不新增 package、依赖、binary、daemon、listener、timer、watcher、网络或常驻资源；
|
||||
Edge/Standalone 继续使用 128/512 Secret 上限。focused data-directory `13/13`、Local SQLite `236/236`、Local Owner
|
||||
`208 total / 203 pass / 5 conditional skip / 0 fail` 与 backend `1,535 total / 1,533 pass / 2 conditional skip / 0 fail` 已通过，覆盖解密一致、
|
||||
stdout 脱敏、exact replay/verify、renamed-model 崩溃恢复、第二个 Secret 冲突的整批回滚和 viewer 拒绝零发布。18-package clean build/逐包测试、
|
||||
schema readiness v50/100 migrations/83 tables、八项架构审计、本地镜像审计和十四档 artifact audit 全 compatible；workspace 仍为 18 packages，
|
||||
`singleSourcePackages=[]`、`shallowSourcePackages=[]`，Local Owner 为 `131 source / 130 nested / 1 root binary entry`。基础 Edge/Standalone
|
||||
`2,611,978 / 2,612,056` bytes、319 files、58 modules，Adopted `2,831,713 / 2,831,836` bytes、339 files、59 modules，Application+AI
|
||||
`4,515,571 / 4,515,703` bytes、514 files、142 modules，MCP `7,337,910 / 7,338,018` bytes、805 files、228 modules，均未放宽预算。
|
||||
GitNexus staged change audit 只允许本切片的预期存储/readiness/application 影响；`next` 相对 `develop` 的全分支 CRITICAL 差异另作为 3.0
|
||||
孵化累计风险保留，不能归因于 D-387。本切片不改变 PostgreSQL schema、ACL、role、Pool、连接或 HA 拓扑，因此不重新占有 PostgreSQL HA
|
||||
证明。D-388 应把 committed receipt 接入 systemd/OpenRC/Compose deployment lineage。
|
||||
- D-386/ADR-0479（已接受）：把 D-385 的 `config/db/ssh.d` 私有 snapshot 转成 no-replace、版本化、Project-bound 的 prepared model，
|
||||
新增 exact `local-data-directory.adoption.transform|transform.verify`。`config.sh` 永不执行，只把简单非空 export 写入独立
|
||||
`qinglong3-local-secret-value` 文件，非 export setting 退役，复杂/重复行转人工复核；Keyv 以 read-only defensive SQLite 和 reviewed
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
# ADR-0480：受认证、原子且可恢复的 Legacy Data Application
|
||||
|
||||
- 状态：Accepted
|
||||
- 日期：2026-08-21
|
||||
- 关联：QL-RFC-0001、ADR-0207、ADR-0395、ADR-0476、ADR-0478、ADR-0479
|
||||
|
||||
## 上下文
|
||||
|
||||
ADR-0479 只生成 `activation=disabled` 的 prepared model。模型内含待导入 Secret 明文，因此不能长期作为运行态事实，也不能由转换器顺便
|
||||
写入目标数据库。应用阶段还必须同时解决四个问题：目标 Project 的强认证与授权、多个 Secret 和模型收据的原子提交、数据库提交后的
|
||||
明文回收，以及进程在数据库 COMMIT 与文件清理之间崩溃时的确定性恢复。
|
||||
|
||||
QingLong 3.0 同时面向低配路由设备和集群节点。本地流程不能为了迁移增加 daemon、队列或第二套数据库；Cluster 又不能因本地简化而
|
||||
失去 Project/RoleBinding fence、审计和幂等事实。SQLite 事务只能原子覆盖数据库，不能原子覆盖文件系统，因此协议必须明确承认并闭合
|
||||
这个边界，而不是宣称跨介质事务。
|
||||
|
||||
## 决策
|
||||
|
||||
### 1. 延续既有领域，不新增 workspace package
|
||||
|
||||
既有 `ql3-adoption` 增加两个 exact、私有 command-file operation：
|
||||
|
||||
- `local-data-directory.adoption.apply`；
|
||||
- `local-data-directory.adoption.apply.verify`。
|
||||
|
||||
职责分别内聚在 `@qinglong/local-owner-cli/lifecycle/data-directory-adoption/application`、
|
||||
`@qinglong/local-admin/data-directory-adoption` 和 `@qinglong/local-sqlite/adoption/data-directory`。CLI 管文件 ceremony，Admin 管认证后的业务
|
||||
组合，SQLite adapter 管唯一写事务。它们不是独立部署单元，不按源码文件拆成微包。
|
||||
|
||||
### 2. 强认证和 Project Policy 是提交前提
|
||||
|
||||
命令只接受 deployment root 内的 Owner pepper keyring、credential presentation 和 Local Secret keyring 路径；敏感 credential 不进入
|
||||
command JSON 或 stdout。Owner credential 通过既有本地强认证建立 `local_data_adoption` authority，随后以目标 `projectId` 请求
|
||||
`secret.manage`。只有 `owner|admin` 的 active RoleBinding 可继续；publisher 在 `BEGIN IMMEDIATE` 内再次校验 credential、Project 和
|
||||
RoleBinding fence，消除预检到提交之间的权限漂移。
|
||||
|
||||
失败认证与拒绝授权写低敏 `legacy-data.apply` 审计，但不会读取 prepared Secret 或写入 adoption/Secret 表。
|
||||
|
||||
### 3. 一个 SQLite 事务提交完整批次
|
||||
|
||||
SQLite contract 从 v49 升到 v50，迁移 `0099/0100` 增加：
|
||||
|
||||
- `QingLong3LegacyDataDirectoryAdoptions`：唯一 mutation、transformation、disabled model、publication digest、receipt 和提交时间；
|
||||
- `QingLong3LegacyDataDirectoryAdoptionSecrets`：按 ordinal 绑定每个来源名称摘要、目标 Secret、value digest、Secret envelope 和 audit；
|
||||
- trigger、FK、唯一索引、typed schema、readiness relation 和 `legacy_data_directory_adoption=1` capability。
|
||||
|
||||
publisher 在一个 `BEGIN IMMEDIATE` 中依次验证 exact replay、Project/RoleBinding fence、所有目标 Secret 当前不存在，再写 encrypted Secret
|
||||
envelope、每个 `secret.create` audit、父 `legacy-data.apply` audit、disabled model、子项和 canonical receipt。任一目标冲突、外部 authority
|
||||
复验失败或约束失败都会回滚整个批次，不允许“部分 Secret 已导入”。同一 mutation 的 exact replay 读取 durable receipt，不再次读取明文或
|
||||
创建新版本；同一 transformation 不能用另一 mutation 重复提交。
|
||||
|
||||
### 4. 只应用 `ready` 模型，永不顺带激活
|
||||
|
||||
apply 在数据库 mutation 前后重新验证 D-385 stage、SQLite activation 和 D-386 transformation binding。`manual_required`、目标漂移、源
|
||||
漂移、Project/profile/digest 不匹配都失败关闭。写入的 config、Keyv、SSH 和 manual-review 模型仍固定 `activation=disabled`；本 ADR
|
||||
不授权执行旧 shell、恢复 `authInfo`、启用 SSH host、启动任务或切换服务。
|
||||
|
||||
Secret 明文只用于当前进程内的 AES-256-GCM envelope 生成。每个 Secret 使用批次 mutation 和目标 identity 推导的稳定 v4 mutation ID，
|
||||
并绑定独立 `secret.create` audit。key material 使用后立即覆盖；JavaScript 字符串的运行时内存擦除不作不可验证的保证。
|
||||
|
||||
### 5. 数据库 receipt 是恢复权威
|
||||
|
||||
数据库 COMMIT 是逻辑提交点。提交后 CLI 在 transformation root 写 `.commit-incomplete`，把 `model` 原子 rename 为
|
||||
`.reclaiming-model`，逐个以 no-follow、owner/mode/inode 检查打开 Secret value 文件，覆盖、fsync、unlink，再删除其余 prepared model
|
||||
文件。完成后写内容无关的 `commit.json`，最后删除 marker。成功根目录固定为：
|
||||
|
||||
```text
|
||||
transformationRoot/
|
||||
manifest.json
|
||||
commit.json
|
||||
```
|
||||
|
||||
如果进程在 COMMIT 后崩溃，同一 exact apply 先解析数据库 receipt，不依赖已被部分删除的 model；它只接受与 receipt 精确匹配的 marker 和
|
||||
`model|.reclaiming-model` 状态，然后继续清理并重建同一 `commit.json`。未知文件、双 model、marker 漂移或 commit 漂移全部失败关闭，保留
|
||||
现场供 operator 诊断。`apply.verify` 只验证完整终态，不隐式修复；恢复必须显式重放原 apply。
|
||||
|
||||
### 6. 明文回收不冒充物理擦除
|
||||
|
||||
协议保证 prepared model 中的明文文件被逻辑覆盖并 unlink，最终 transformation root 不再含 model 或 Secret value 文件。由于 SSD、闪存
|
||||
FTL、CoW、快照、备份和宿主文件系统可能保留旧块，receipt 固定声明 `physicalErasureGuaranteed=false`。需要物理销毁保证的部署者必须使用
|
||||
加密卷、销毁卷密钥或设备级擦除流程；不能把本地 overwrite 结果写成合规证明。
|
||||
|
||||
### 7. Profile 资源边界与部署边界
|
||||
|
||||
Edge/Standalone 继续使用 D-386 的 128/512 Secret 上限；单条 envelope 最高 16 KiB。事务只打开一个短生命周期 SQLite writer，认证使用
|
||||
独立短生命周期 handle 以避免嵌套 operation authority；没有网络、timer、watcher、后台 retry、缓存或常驻内存。Cluster 后续可以复用
|
||||
prepared-model/receipt 语义，但必须以 PostgreSQL、separation-of-duty 和 Cluster 专用 authority 实现，不能把本地 credential ceremony
|
||||
直接搬进控制面。
|
||||
|
||||
## 被拒绝的替代方案
|
||||
|
||||
### 每个 Secret 独立调用现有 `secret.put`
|
||||
|
||||
拒绝。第二个 Secret 冲突会留下第一个已提交，无法证明批次与 transformation receipt 一致。
|
||||
|
||||
### 先删除明文，再提交数据库
|
||||
|
||||
拒绝。数据库失败后会失去唯一待导入值，D-385 staging 也不能安全地代替已审核 transformation。
|
||||
|
||||
### 用文件锁或后台 daemon 模拟跨介质事务
|
||||
|
||||
拒绝。文件锁不能回滚 SQLite COMMIT；daemon 会给路由设备增加常驻资源和长期高权限。durable DB receipt 加调用方显式重放已经能闭合崩溃窗。
|
||||
|
||||
### 成功 apply 后直接启用配置、SSH 或服务
|
||||
|
||||
拒绝。数据提交、风险复核、运行时 activation 和 service cutover 是不同 authority；合并会让迁移 credential 获得过宽副作用。
|
||||
|
||||
### 声称 overwrite 等于安全物理擦除
|
||||
|
||||
拒绝。对现代闪存、CoW 和快照不可证明，会产生错误的安全与合规承诺。
|
||||
|
||||
## 影响
|
||||
|
||||
### 正面
|
||||
|
||||
- prepared model 首次具有强认证、Project-scoped、可审计的最终数据库提交路径；
|
||||
- Secret、子审计、disabled model 和 receipt 同事务提交，目标冲突不会留下部分状态；
|
||||
- COMMIT-response-loss 和清理中断可用同一命令收敛，不依赖残存明文；
|
||||
- 成功后高敏 model 被回收，stdout、manifest、commit 和 receipt 只保留低敏摘要；
|
||||
- workspace package 数量和低配常驻拓扑保持不变。
|
||||
|
||||
### 代价与限制
|
||||
|
||||
- SQLite contract 升到 v50，旧 v49 binary 不能写新库，部署镜像 label/readiness 必须同步升级；
|
||||
- DB COMMIT 后、文件清理前的命令可能以失败退出，但 durable receipt 表明逻辑提交已发生，operator 必须重放同一 apply；
|
||||
- 逻辑明文回收不保证介质级擦除；
|
||||
- 本 ADR 不迁移 `scripts/upload` 到最终 runtime，不完成 service cutover，也不实现 Cluster 数据提交。
|
||||
|
||||
## 验证结果
|
||||
|
||||
- focused data-directory `13/13`，覆盖成功提交、Secret 解密一致、stdout 脱敏、exact apply/apply.verify replay、COMMIT 后 renamed-model 恢复、
|
||||
第二个 Secret 冲突整批回滚和 viewer 拒绝零发布；
|
||||
- Local SQLite `236/236`，覆盖 v50 migration manifest、typed schema、readiness、unknown table drift 和既有 repository 回归；schema readiness
|
||||
以 mode-0600 Edge 数据库实测为 contract v50、100 migrations、83 tables、SQLite 3.53.3、DELETE journal；
|
||||
- Local Owner 为 `208 total / 203 pass / 5 conditional skip / 0 fail`；backend 为
|
||||
`1,535 total / 1,533 pass / 2 conditional skip / 0 fail`；`pnpm build:back` 和 18-package clean build/逐包测试退出 0；
|
||||
- package boundary 保持 18 packages、`singleSourcePackages=[]`、`shallowSourcePackages=[]`；Local Admin、Local Owner、Local SQLite 新源码均进入
|
||||
既有领域子目录，Local Owner 为 `131 source / 130 nested / 1 root binary entry`；
|
||||
- 八项架构审计、本地镜像审计和十四档 artifact audit 全 compatible；基础 Edge/Standalone 为 319 files/58 modules，最小 artifact
|
||||
2,611,978 bytes；Application+AI 为 514 files/142 modules，最大 4,515,703 bytes；MCP 为 805 files/228 modules，最大 7,338,018 bytes，
|
||||
全部保留既有预算；
|
||||
- GitNexus `compare develop` 如实返回整条 `next` 孵化分支的 CRITICAL 累计差异；提交前另以 staged diff 收窄并核验 D-387 的预期符号与流程影响，
|
||||
不用全分支风险替代切片审计；
|
||||
- 本切片不修改 PostgreSQL schema、ACL、role、Pool、连接或 HA 拓扑，不重新占有 PostgreSQL HA 证明。
|
||||
|
||||
## 后续
|
||||
|
||||
- D-388：把 committed transformation receipt 接入 systemd/OpenRC/Compose deployment lineage，并保持 activation 与 rollback 独立授权；
|
||||
- 在固定物理 Edge/NAS 上执行 RSS、I/O、磁盘峰值、ENOSPC、断电和加密卷销毁演练；
|
||||
- 为 Cluster 定义 PostgreSQL/SoD 版本的 prepared-model application，而不是复用本地 Owner credential 文件。
|
||||
@@ -483,6 +483,7 @@
|
||||
| [ADR-0477](./ADR-0477-bounded-legacy-data-directory-inventory.md) | 有界 Legacy Data Directory 盘点 | Accepted |
|
||||
| [ADR-0478](./ADR-0478-private-legacy-data-directory-staging.md) | 私有 Legacy Data Directory 暂存与稳定校验 | Accepted |
|
||||
| [ADR-0479](./ADR-0479-private-versioned-legacy-config-keyv-ssh-transformation.md) | 私有、版本化的 Legacy Config、Keyv 与 SSH 转换 | Accepted |
|
||||
| [ADR-0480](./ADR-0480-authenticated-atomic-legacy-data-application.md) | 受认证、原子且可恢复的 Legacy Data Application | Accepted |
|
||||
|
||||
## 规则
|
||||
|
||||
|
||||
@@ -222,17 +222,90 @@ Secret 明文，必须按高敏迁移材料保护，不能写入日志、提交
|
||||
成功返回 `status=verified`，evidence 应与 transform 完全相同。verify 会在目标校验前后重新验证 D-385 staging、当前 2.x source 和 SQLite
|
||||
activation，并检查目标 exact 文件集、私有 mode、Project/profile/path binding、Secret value digest 和完整模型树摘要。
|
||||
|
||||
## 10. 崩溃残留与恢复
|
||||
## 10. 受认证提交 prepared model
|
||||
|
||||
只有 transform 返回 `assessment=ready`、transform.verify 成功，且目标 Project 已存在 active Owner/Admin RoleBinding 时才能 apply。先准备：
|
||||
|
||||
- deployment root 内 `0700` 的 Owner pepper keyring；
|
||||
- deployment root 内 `0600` 的 Owner credential presentation；
|
||||
- deployment root 内 `0600` 的 Local Secret keyring；
|
||||
- 两个不同的 UUID v4：`mutationId` 与失败审计 `failureAuditEventId`。
|
||||
|
||||
credential token 和 Secret key material 不写入 command JSON。提交命令是在 transform.verify options 上增加以下 authority：
|
||||
|
||||
```json
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"operation": "local-data-directory.adoption.apply",
|
||||
"options": {
|
||||
"deploymentRoot": "/opt/qinglong3/adoption",
|
||||
"dataRoot": "/opt/qinglong/data",
|
||||
"stagingRoot": "/opt/qinglong3/adoption/staging/reviewed-data",
|
||||
"transformationRoot": "/opt/qinglong3/adoption/transformations/reviewed-data-v1",
|
||||
"projectId": "<target-project-id>",
|
||||
"profile": "edge",
|
||||
"expectedManifestDigest": "<64-hex-directory-manifest-digest>",
|
||||
"expectedTransformationDigest": "<64-hex-transformation-digest>",
|
||||
"ownerPepperKeyringDirectory": "/opt/qinglong3/adoption/owner-keys",
|
||||
"credentialFilePath": "/opt/qinglong3/adoption/owner-credential.json",
|
||||
"secretKeyringPath": "/opt/qinglong3/adoption/local-secret-keyring.json",
|
||||
"mutationId": "<uuid-v4>",
|
||||
"failureAuditEventId": "<different-uuid-v4>",
|
||||
"requestId": "legacy-data-apply-20260821",
|
||||
"sqlite": {
|
||||
"sourcePath": "/opt/qinglong/data/db/database.sqlite",
|
||||
"targetPath": "/opt/qinglong3/adoption/sqlite/qinglong3.sqlite",
|
||||
"recoveryPath": "/opt/qinglong3/adoption/sqlite/database.pre-ql3.sqlite",
|
||||
"manifestPath": "/opt/qinglong3/adoption/sqlite/adoption.json",
|
||||
"activationPath": "/opt/qinglong3/adoption/sqlite/activation.json",
|
||||
"expectedActivationDigest": "<64-hex-sqlite-activation-digest>"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
apply 会以 Owner credential 建立强认证，要求目标 Project 的 `secret.manage`，并在一个 SQLite `BEGIN IMMEDIATE` 中提交所有加密 Secret、
|
||||
逐项 `secret.create` 审计、父 `legacy-data.apply` 审计、disabled model 和 canonical receipt。任何一个目标 Secret 已存在都会回滚整个批次。
|
||||
成功返回 `status=committed`、`databaseStatus=inserted` 和低敏 digest/count evidence，不返回 Secret 名称、Project ID 或明文。
|
||||
|
||||
成功后 transformation root 只剩 `manifest.json` 与 `commit.json`；`model/` 和其中的明文文件已逻辑覆盖并删除。结果固定声明
|
||||
`physicalErasureGuaranteed=false`：闪存 FTL、CoW、快照或备份可能保留旧块。需要介质级保证时应使用加密卷并销毁卷密钥或执行设备级擦除。
|
||||
|
||||
apply 只把模型持久化为 `activation=disabled`，不会启用 SSH、执行配置、启动任务或切换服务。
|
||||
|
||||
## 11. 校验 committed application
|
||||
|
||||
保存 apply 返回的 `receiptDigest`。将原命令 operation 改为 `local-data-directory.adoption.apply.verify`，并在 options 增加：
|
||||
|
||||
```json
|
||||
{
|
||||
"expectedReceiptDigest": "<64-hex-receipt-digest>"
|
||||
}
|
||||
```
|
||||
|
||||
verify 使用相同强认证和 exact mutation，读取 durable receipt，验证 transformation manifest 与最终 `commit.json`，但不修复中间状态。成功返回
|
||||
`status=verified`、`databaseStatus=existing`。receipt、commit、Project/profile/transformation 或 root 文件集有任何漂移都会失败关闭。
|
||||
|
||||
## 12. 崩溃残留与恢复
|
||||
|
||||
stage 和 transform 都在创建目标根后立即写入 `.incomplete`。只有 payload/model 和 `manifest.json` 都持久化且静态校验通过后才删除。
|
||||
命令失败或进程崩溃后：
|
||||
这些阶段失败或进程崩溃后：
|
||||
|
||||
- 不要直接把残留目录当作恢复资产；
|
||||
- 不要在原路径重试，stage/transform 会 no-replace 拒绝；
|
||||
- 先保存现场用于诊断，再由 operator 显式移走残留目录，并使用一个新的空路径重试；
|
||||
- verify 遇到 `.incomplete`、额外文件或缺失文件一律失败关闭。
|
||||
|
||||
## 11. 常见失败
|
||||
apply 的数据库 COMMIT 是逻辑提交点。COMMIT 后清理使用 `.commit-incomplete` 和 `.reclaiming-model`。若 apply 失败且 transformation root 出现
|
||||
这些条目：
|
||||
|
||||
- 不要重新 transform，也不要手工删除 marker/model；
|
||||
- 查询错误和目标数据库 readiness，保留完整诊断现场；
|
||||
- 使用完全相同的 command、mutation ID 和 digest 重放 `local-data-directory.adoption.apply`；
|
||||
- exact replay 从 durable receipt 继续回收 model 并重建同一 `commit.json`，不会再次创建 Secret；
|
||||
- 未知文件、marker 漂移、双 model 或 receipt 不一致必须人工调查，不能用 apply.verify 或手改文件绕过。
|
||||
|
||||
## 13. 常见失败
|
||||
|
||||
- 根目录或条目 group/world 可写：修正 ownership/permission 后重新盘点；
|
||||
- symlink、硬链接或特殊文件：保留现场，确认来源和目标后人工处置；盘点不会跟随或读取；
|
||||
@@ -246,16 +319,22 @@ stage 和 transform 都在创建目标根后立即写入 `.incomplete`。只有
|
||||
- Keyv schema、row 或字节超限：不要把数据库当 JSON dump 绕过，先确认版本和未知数据归属；
|
||||
- `manual_required`：从 D-385 staging 审核相应原始输入；不要执行旧 shell、复用 `authInfo` 或启用旧 SSH config；
|
||||
- transform verify drift：停止后续 apply，保留 D-385 staging 和 transformation root，重新建立新的版本化转换路径。
|
||||
- credential rejected：核对 credential presentation、pepper keyring、有效期和目标数据库绑定，不要把 token 填进 command JSON；
|
||||
- `APPLICATION_FORBIDDEN`：当前主体不是目标 Project 的 active owner/admin；通过正式 Policy 管理流程修复 RoleBinding 后使用新失败审计 ID重试；
|
||||
- `ADOPTION_CONFLICT`：目标 Secret、mutation 或 transformation 已存在不一致事实；批次已回滚，禁止删除现有 Secret 强行导入；
|
||||
- apply 在数据库 COMMIT 后清理失败：重放完全相同的 apply；不要生成新 mutation，否则会与 transformation 唯一约束冲突；
|
||||
- apply.verify 报终态不完整：它不会恢复；改为重放原 apply。
|
||||
|
||||
## 12. 当前边界
|
||||
## 14. 当前边界
|
||||
|
||||
本流程已经提供 inspect、私有 stage、版本化 transform 和两级稳定 verify。它仍不：
|
||||
本流程已经提供 inspect、私有 stage、版本化 transform、受认证原子 apply、崩溃恢复和三级稳定 verify。它仍不：
|
||||
|
||||
- 删除或修改任何源资产；
|
||||
- 把 prepared Secret/settings/SSH binding 写入或激活到 3.0 目标数据库；
|
||||
- 激活已提交的 settings/SSH binding、启动任务或切换服务；
|
||||
- 把历史日志/备份复制到默认目标，或复用跨架构 repo/dependency cache；
|
||||
- 授权 service-manager/Compose cutover 或 Legacy rollback；
|
||||
- 证明固定物理路由器/NAS 上的耗时、RSS、I/O、磁盘峰值和断电恢复。
|
||||
|
||||
即使 transform 返回 `status=prepared` 且 transform.verify 返回 `status=verified`，仍不是 apply 或 cutover 授权。继续保留原始 2.x
|
||||
data directory、SQLite recovery、D-385 staging 和两份 manifest，等待受认证 apply/commit 与部署 lineage 完成。
|
||||
apply 成功只证明目标数据库中的 encrypted Secret、disabled model、audit 和 receipt 原子一致，并且 prepared 明文文件已逻辑回收；它仍不是
|
||||
cutover 或 activation 授权。继续保留原始 2.x data directory、SQLite recovery、D-385 staging、manifest 和 `commit.json`，直到独立部署
|
||||
lineage 与回滚门完成。
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
},
|
||||
{
|
||||
"kind": "ordered_ledger",
|
||||
"maxDirectSourceFiles": 99,
|
||||
"maxDirectSourceFiles": 101,
|
||||
"path": "packages/ql3-local-sqlite/src/migrations",
|
||||
"rationale": "SQLite migrations are an append-only version ledger whose ordering and discoverability are safer in one reviewed directory."
|
||||
}
|
||||
@@ -55,12 +55,12 @@
|
||||
"authorities": [
|
||||
"Kubernetes mutation",
|
||||
"PostgreSQL management",
|
||||
"optional mTLS/OIDC human Approval management",
|
||||
"caller-driven Approval management client",
|
||||
"loopback-only read-only Copilot Console",
|
||||
"one-shot maintenance"
|
||||
"optional mTLS/OIDC human Approval management",
|
||||
"caller-driven Approval management client",
|
||||
"loopback-only read-only Copilot Console",
|
||||
"one-shot maintenance"
|
||||
],
|
||||
"rationale": "独立管理进程、一次性高权限命令和 operator-workstation Console 必须排除在常驻 cluster-control 之外；Approval 的 service/transport/process/client 与只读 Copilot Console 均属于同一 Cluster Admin 制品，分别使用内聚领域目录而不是新增微包。Console 的宿主启动器、配置模板与发布验签脚本随已签名的多架构 Admin OCI 分发，不形成第 19 个 workspace package 或第二套依赖树。"
|
||||
"rationale": "独立管理进程、一次性高权限命令和 operator-workstation Console 必须排除在常驻 cluster-control 之外；Approval 的 service/transport/process/client 与只读 Copilot Console 均属于同一 Cluster Admin 制品，分别使用内聚领域目录而不是新增微包。Console 的宿主启动器、配置模板与发布验签脚本随已签名的多架构 Admin OCI 分发，不形成第 19 个 workspace package 或第二套依赖树。"
|
||||
},
|
||||
{
|
||||
"path": "packages/ql3-cluster-control",
|
||||
|
||||
@@ -115,6 +115,11 @@
|
||||
"require": "./dist/security-administration/secretAdministration.js",
|
||||
"default": "./dist/security-administration/secretAdministration.js"
|
||||
},
|
||||
"./data-directory-adoption": {
|
||||
"types": "./dist/data-directory-adoption/dataDirectoryAdoption.d.ts",
|
||||
"require": "./dist/data-directory-adoption/dataDirectoryAdoption.js",
|
||||
"default": "./dist/data-directory-adoption/dataDirectoryAdoption.js"
|
||||
},
|
||||
"./task-definition-administration": {
|
||||
"types": "./dist/automation-administration/taskDefinitionAdministration.d.ts",
|
||||
"require": "./dist/automation-administration/taskDefinitionAdministration.js",
|
||||
|
||||
@@ -0,0 +1,547 @@
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
import {
|
||||
createLocalDataDirectoryAdoptionReceipt,
|
||||
createLocalDataDirectoryAdoptionSecretItem,
|
||||
createLocalDataDirectorySourceNameDigest,
|
||||
openLocalSqliteDataDirectoryAdoptionDatabase,
|
||||
type LocalDataDirectoryAdoptionRecord,
|
||||
type LocalDataDirectoryAdoptionSecretPublication,
|
||||
type LocalDataDirectoryAppliedModel,
|
||||
} from '@qinglong/local-sqlite/data-directory-adoption';
|
||||
import {
|
||||
encryptLocalSecretEnvelope,
|
||||
ownedLocalSecretKeyMaterial,
|
||||
} from '@qinglong/local-secret';
|
||||
import {
|
||||
LOCAL_SECRET_ALGORITHM,
|
||||
assertLocalSecretName,
|
||||
assertLocalSecretPlaintext,
|
||||
assertLocalSecretProjectId,
|
||||
type LocalSecretKeyProvider,
|
||||
} from '@qinglong/runtime-core/local-secret';
|
||||
import {
|
||||
ProjectPolicyEngine,
|
||||
ProjectPolicyUnavailableError,
|
||||
} from '@qinglong/runtime-core/project-policy';
|
||||
import {
|
||||
normalizeSecurityPrincipal,
|
||||
type SecurityPolicyDecision,
|
||||
type SecurityPrincipal,
|
||||
} from '@qinglong/runtime-core/security';
|
||||
import {
|
||||
normalizeSecurityAuditRecord,
|
||||
type SecurityAuditRecord,
|
||||
} from '@qinglong/runtime-core/security-audit';
|
||||
|
||||
const DIGEST_PATTERN = /^[0-9a-f]{64}$/;
|
||||
const UUID_V4_PATTERN =
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/;
|
||||
const REQUEST_ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$/;
|
||||
const VALUE_FILE_PATTERN = /^secret-values\/[0-9a-f]{64}\.json$/;
|
||||
const STRONG_USER_ASSURANCES = new Set([
|
||||
'multi_factor',
|
||||
'hardware',
|
||||
'local_console',
|
||||
]);
|
||||
|
||||
export interface PreparedLocalDataDirectoryAdoptionSecret {
|
||||
readonly kind: 'environment' | 'ssh_private_key';
|
||||
readonly sourceName: string;
|
||||
readonly targetName: string;
|
||||
readonly expectedCurrentVersion: 0;
|
||||
readonly valueFile: string;
|
||||
readonly valueDigest: string;
|
||||
readonly plaintext: string;
|
||||
}
|
||||
|
||||
export interface PreparedLocalDataDirectoryAdoptionModel {
|
||||
readonly model: Readonly<LocalDataDirectoryAppliedModel>;
|
||||
readonly secrets: readonly Readonly<PreparedLocalDataDirectoryAdoptionSecret>[];
|
||||
}
|
||||
|
||||
export interface ApplyReviewedLocalDataDirectoryAdoptionOptions {
|
||||
readonly databasePath: string;
|
||||
readonly profile: 'edge' | 'standalone';
|
||||
readonly projectId: string;
|
||||
readonly mutationId: string;
|
||||
readonly failureAuditEventId: string;
|
||||
readonly requestId: string;
|
||||
readonly sourceStageManifestDigest: string;
|
||||
readonly transformationDigest: string;
|
||||
readonly modelDigest: string;
|
||||
readonly principal: Readonly<SecurityPrincipal>;
|
||||
readonly keyProvider: LocalSecretKeyProvider;
|
||||
readonly observedAtMs: number;
|
||||
readonly busyTimeoutMs?: number;
|
||||
readonly loadPreparedModel: () =>
|
||||
| Readonly<PreparedLocalDataDirectoryAdoptionModel>
|
||||
| Promise<Readonly<PreparedLocalDataDirectoryAdoptionModel>>;
|
||||
readonly confirmAuthenticationAuthority: () => void | Promise<void>;
|
||||
readonly confirmPreparedAuthority: () => void | Promise<void>;
|
||||
}
|
||||
|
||||
export interface ApplyReviewedLocalDataDirectoryAdoptionResult {
|
||||
readonly status: 'inserted' | 'existing';
|
||||
readonly adoption: Readonly<LocalDataDirectoryAdoptionRecord>;
|
||||
}
|
||||
|
||||
export class LocalDataDirectoryAdoptionApplicationConfigurationError extends TypeError {
|
||||
readonly code = 'LOCAL_DATA_DIRECTORY_ADOPTION_APPLICATION_INVALID';
|
||||
|
||||
constructor(message: string, readonly cause?: unknown) {
|
||||
super(`Local data directory adoption application is invalid: ${message}`);
|
||||
this.name = 'LocalDataDirectoryAdoptionApplicationConfigurationError';
|
||||
}
|
||||
}
|
||||
|
||||
export class LocalDataDirectoryAdoptionApplicationAuthenticationError extends Error {
|
||||
readonly code =
|
||||
'LOCAL_DATA_DIRECTORY_ADOPTION_APPLICATION_AUTHENTICATION_REQUIRED';
|
||||
|
||||
constructor() {
|
||||
super(
|
||||
'Local data directory adoption application requires a strong principal',
|
||||
);
|
||||
this.name = 'LocalDataDirectoryAdoptionApplicationAuthenticationError';
|
||||
}
|
||||
}
|
||||
|
||||
export class LocalDataDirectoryAdoptionApplicationAuthorizationError extends Error {
|
||||
readonly code = 'LOCAL_DATA_DIRECTORY_ADOPTION_APPLICATION_FORBIDDEN';
|
||||
|
||||
constructor() {
|
||||
super('Local data directory adoption application is not authorized');
|
||||
this.name = 'LocalDataDirectoryAdoptionApplicationAuthorizationError';
|
||||
}
|
||||
}
|
||||
|
||||
export class LocalDataDirectoryAdoptionApplicationUnavailableError extends Error {
|
||||
readonly code = 'LOCAL_DATA_DIRECTORY_ADOPTION_APPLICATION_UNAVAILABLE';
|
||||
|
||||
constructor(readonly cause?: unknown) {
|
||||
super('Local data directory adoption application is unavailable');
|
||||
this.name = 'LocalDataDirectoryAdoptionApplicationUnavailableError';
|
||||
}
|
||||
}
|
||||
|
||||
function exactKeys(value: object, expected: readonly string[]): boolean {
|
||||
const actual = Object.keys(value).sort();
|
||||
const canonical = [...expected].sort();
|
||||
return (
|
||||
actual.length === canonical.length &&
|
||||
actual.every((key, index) => key === canonical[index])
|
||||
);
|
||||
}
|
||||
|
||||
function assertOptions(
|
||||
options: ApplyReviewedLocalDataDirectoryAdoptionOptions,
|
||||
): void {
|
||||
if (
|
||||
!options ||
|
||||
typeof options !== 'object' ||
|
||||
Array.isArray(options) ||
|
||||
!UUID_V4_PATTERN.test(options.mutationId) ||
|
||||
!UUID_V4_PATTERN.test(options.failureAuditEventId) ||
|
||||
options.failureAuditEventId === options.mutationId ||
|
||||
!REQUEST_ID_PATTERN.test(options.requestId) ||
|
||||
(options.profile !== 'edge' && options.profile !== 'standalone') ||
|
||||
![
|
||||
options.sourceStageManifestDigest,
|
||||
options.transformationDigest,
|
||||
options.modelDigest,
|
||||
].every((digest) => DIGEST_PATTERN.test(digest)) ||
|
||||
!Number.isSafeInteger(options.observedAtMs) ||
|
||||
options.observedAtMs < 0 ||
|
||||
typeof options.loadPreparedModel !== 'function' ||
|
||||
typeof options.confirmAuthenticationAuthority !== 'function' ||
|
||||
typeof options.confirmPreparedAuthority !== 'function' ||
|
||||
!options.keyProvider ||
|
||||
typeof options.keyProvider.active !== 'function' ||
|
||||
typeof options.keyProvider.resolve !== 'function' ||
|
||||
(options.busyTimeoutMs !== undefined &&
|
||||
(!Number.isSafeInteger(options.busyTimeoutMs) ||
|
||||
options.busyTimeoutMs < 100 ||
|
||||
options.busyTimeoutMs > 30_000))
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionApplicationConfigurationError(
|
||||
'options are invalid',
|
||||
);
|
||||
}
|
||||
try {
|
||||
assertLocalSecretProjectId(options.projectId);
|
||||
} catch (error) {
|
||||
throw new LocalDataDirectoryAdoptionApplicationConfigurationError(
|
||||
'projectId is invalid',
|
||||
error,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function strongPrincipal(
|
||||
value: Readonly<SecurityPrincipal>,
|
||||
nowMs: number,
|
||||
): Readonly<SecurityPrincipal> {
|
||||
let principal: Readonly<SecurityPrincipal>;
|
||||
try {
|
||||
principal = normalizeSecurityPrincipal(value, nowMs);
|
||||
} catch {
|
||||
throw new LocalDataDirectoryAdoptionApplicationAuthenticationError();
|
||||
}
|
||||
const human =
|
||||
principal.subject.type === 'user' &&
|
||||
STRONG_USER_ASSURANCES.has(principal.assurance);
|
||||
const system =
|
||||
principal.subject.type === 'system' && principal.assurance === 'service';
|
||||
if (!human && !system) {
|
||||
throw new LocalDataDirectoryAdoptionApplicationAuthenticationError();
|
||||
}
|
||||
return principal;
|
||||
}
|
||||
|
||||
function auditRecord(options: {
|
||||
readonly eventId: string;
|
||||
readonly requestId: string;
|
||||
readonly projectId: string;
|
||||
readonly principal: Readonly<SecurityPrincipal> | null;
|
||||
readonly operationId: string;
|
||||
readonly outcome: SecurityAuditRecord['outcome'];
|
||||
readonly reasons: readonly string[];
|
||||
readonly decision: Readonly<SecurityPolicyDecision> | null;
|
||||
readonly occurredAtMs: number;
|
||||
}): Readonly<SecurityAuditRecord> {
|
||||
return normalizeSecurityAuditRecord({
|
||||
eventId: options.eventId,
|
||||
requestId: options.requestId,
|
||||
operationId: options.operationId,
|
||||
projectId: options.projectId,
|
||||
subject: options.principal?.subject ?? null,
|
||||
authenticationId: options.principal?.authenticationId ?? null,
|
||||
outcome: options.outcome,
|
||||
reasons: options.reasons,
|
||||
fence: options.decision?.fence ?? null,
|
||||
occurredAtMs: options.occurredAtMs,
|
||||
});
|
||||
}
|
||||
|
||||
function deterministicMutationId(
|
||||
batchMutationId: string,
|
||||
identity: string,
|
||||
): string {
|
||||
const bytes = createHash('sha256')
|
||||
.update('qinglong3.legacy-data-directory-secret-mutation.v1\0')
|
||||
.update(batchMutationId)
|
||||
.update('\0')
|
||||
.update(identity)
|
||||
.digest()
|
||||
.subarray(0, 16);
|
||||
bytes[6] = ((bytes[6] as number) & 0x0f) | 0x40;
|
||||
bytes[8] = ((bytes[8] as number) & 0x3f) | 0x80;
|
||||
const hex = bytes.toString('hex');
|
||||
return `${hex.slice(0, 8)}-${hex.slice(8, 12)}-${hex.slice(
|
||||
12,
|
||||
16,
|
||||
)}-${hex.slice(16, 20)}-${hex.slice(20)}`;
|
||||
}
|
||||
|
||||
function assertPreparedModel(
|
||||
value: Readonly<PreparedLocalDataDirectoryAdoptionModel>,
|
||||
profile: 'edge' | 'standalone',
|
||||
): void {
|
||||
if (
|
||||
!value ||
|
||||
typeof value !== 'object' ||
|
||||
Array.isArray(value) ||
|
||||
!exactKeys(value, ['model', 'secrets']) ||
|
||||
!Array.isArray(value.secrets) ||
|
||||
value.secrets.length > (profile === 'edge' ? 128 : 512)
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionApplicationConfigurationError(
|
||||
'prepared model shape is invalid',
|
||||
);
|
||||
}
|
||||
const targets = new Set<string>();
|
||||
for (const entry of value.secrets) {
|
||||
if (
|
||||
!entry ||
|
||||
typeof entry !== 'object' ||
|
||||
Array.isArray(entry) ||
|
||||
!exactKeys(entry, [
|
||||
'expectedCurrentVersion',
|
||||
'kind',
|
||||
'plaintext',
|
||||
'sourceName',
|
||||
'targetName',
|
||||
'valueDigest',
|
||||
'valueFile',
|
||||
]) ||
|
||||
(entry.kind !== 'environment' && entry.kind !== 'ssh_private_key') ||
|
||||
typeof entry.sourceName !== 'string' ||
|
||||
entry.sourceName.length < 1 ||
|
||||
entry.sourceName.includes('\0') ||
|
||||
entry.expectedCurrentVersion !== 0 ||
|
||||
!VALUE_FILE_PATTERN.test(entry.valueFile) ||
|
||||
!DIGEST_PATTERN.test(entry.valueDigest) ||
|
||||
createHash('sha256').update(entry.plaintext, 'utf8').digest('hex') !==
|
||||
entry.valueDigest ||
|
||||
targets.has(entry.targetName)
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionApplicationConfigurationError(
|
||||
'prepared Secret entry is invalid',
|
||||
);
|
||||
}
|
||||
try {
|
||||
assertLocalSecretName(entry.targetName);
|
||||
assertLocalSecretPlaintext(entry.plaintext);
|
||||
} catch (error) {
|
||||
throw new LocalDataDirectoryAdoptionApplicationConfigurationError(
|
||||
'prepared Secret value is invalid',
|
||||
error,
|
||||
);
|
||||
}
|
||||
targets.add(entry.targetName);
|
||||
}
|
||||
}
|
||||
|
||||
function sameExpected(
|
||||
record: Readonly<LocalDataDirectoryAdoptionRecord>,
|
||||
options: Readonly<ApplyReviewedLocalDataDirectoryAdoptionOptions>,
|
||||
): boolean {
|
||||
return (
|
||||
record.mutationId === options.mutationId &&
|
||||
record.projectId === options.projectId &&
|
||||
record.profile === options.profile &&
|
||||
record.sourceStageManifestDigest === options.sourceStageManifestDigest &&
|
||||
record.transformationDigest === options.transformationDigest &&
|
||||
record.modelDigest === options.modelDigest
|
||||
);
|
||||
}
|
||||
|
||||
export async function applyReviewedLocalDataDirectoryAdoption(
|
||||
options: Readonly<ApplyReviewedLocalDataDirectoryAdoptionOptions>,
|
||||
): Promise<Readonly<ApplyReviewedLocalDataDirectoryAdoptionResult>> {
|
||||
assertOptions(options);
|
||||
const database = await openLocalSqliteDataDirectoryAdoptionDatabase({
|
||||
databasePath: options.databasePath,
|
||||
profile: options.profile,
|
||||
...(options.busyTimeoutMs === undefined
|
||||
? {}
|
||||
: { busyTimeoutMs: options.busyTimeoutMs }),
|
||||
});
|
||||
try {
|
||||
let principal: Readonly<SecurityPrincipal>;
|
||||
try {
|
||||
principal = strongPrincipal(options.principal, options.observedAtMs);
|
||||
await options.confirmAuthenticationAuthority();
|
||||
} catch (error) {
|
||||
try {
|
||||
await database.securityAudit.record(
|
||||
auditRecord({
|
||||
eventId: options.failureAuditEventId,
|
||||
requestId: options.requestId,
|
||||
projectId: options.projectId,
|
||||
principal: null,
|
||||
operationId: 'legacy-data.apply',
|
||||
outcome: 'authentication_rejected',
|
||||
reasons: ['strong_authentication_required'],
|
||||
decision: null,
|
||||
occurredAtMs: options.observedAtMs,
|
||||
}),
|
||||
);
|
||||
} catch {
|
||||
throw new LocalDataDirectoryAdoptionApplicationUnavailableError();
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
|
||||
const policy = new ProjectPolicyEngine(database.projectPolicy);
|
||||
let decision: Readonly<SecurityPolicyDecision>;
|
||||
try {
|
||||
decision = await policy.authorize(
|
||||
principal,
|
||||
options.projectId,
|
||||
'secret.manage',
|
||||
);
|
||||
} catch (error) {
|
||||
if (!(error instanceof ProjectPolicyUnavailableError)) {
|
||||
throw new LocalDataDirectoryAdoptionApplicationUnavailableError(error);
|
||||
}
|
||||
try {
|
||||
await database.securityAudit.record(
|
||||
auditRecord({
|
||||
eventId: options.failureAuditEventId,
|
||||
requestId: options.requestId,
|
||||
projectId: options.projectId,
|
||||
principal,
|
||||
operationId: 'legacy-data.apply',
|
||||
outcome: 'authorization_unavailable',
|
||||
reasons: ['policy_unavailable'],
|
||||
decision: null,
|
||||
occurredAtMs: options.observedAtMs,
|
||||
}),
|
||||
);
|
||||
} catch {
|
||||
throw new LocalDataDirectoryAdoptionApplicationUnavailableError();
|
||||
}
|
||||
throw new LocalDataDirectoryAdoptionApplicationUnavailableError();
|
||||
}
|
||||
if (decision.effect !== 'allow') {
|
||||
try {
|
||||
await database.securityAudit.record(
|
||||
auditRecord({
|
||||
eventId: options.failureAuditEventId,
|
||||
requestId: options.requestId,
|
||||
projectId: options.projectId,
|
||||
principal,
|
||||
operationId: 'legacy-data.apply',
|
||||
outcome:
|
||||
decision.effect === 'require_approval'
|
||||
? 'approval_required'
|
||||
: 'denied',
|
||||
reasons: decision.reasons,
|
||||
decision,
|
||||
occurredAtMs: options.observedAtMs,
|
||||
}),
|
||||
);
|
||||
} catch {
|
||||
throw new LocalDataDirectoryAdoptionApplicationUnavailableError();
|
||||
}
|
||||
throw new LocalDataDirectoryAdoptionApplicationAuthorizationError();
|
||||
}
|
||||
if (!decision.fence || decision.fence.bindingVersion === null) {
|
||||
throw new LocalDataDirectoryAdoptionApplicationUnavailableError();
|
||||
}
|
||||
|
||||
const existing = await database.publisher.resolve(options.mutationId);
|
||||
if (existing) {
|
||||
if (!sameExpected(existing, options)) {
|
||||
throw new LocalDataDirectoryAdoptionApplicationConfigurationError(
|
||||
'mutation replay does not match the committed adoption',
|
||||
);
|
||||
}
|
||||
await options.confirmAuthenticationAuthority();
|
||||
return Object.freeze({ status: 'existing', adoption: existing });
|
||||
}
|
||||
|
||||
const prepared = await options.loadPreparedModel();
|
||||
assertPreparedModel(prepared, options.profile);
|
||||
const material = ownedLocalSecretKeyMaterial(
|
||||
await options.keyProvider.active(),
|
||||
);
|
||||
const publications: LocalDataDirectoryAdoptionSecretPublication[] = [];
|
||||
try {
|
||||
for (const [index, secret] of prepared.secrets.entries()) {
|
||||
const ordinal = index + 1;
|
||||
const secretMutationId = deterministicMutationId(
|
||||
options.mutationId,
|
||||
`${secret.kind}\0${secret.targetName}`,
|
||||
);
|
||||
const envelope = encryptLocalSecretEnvelope(
|
||||
{
|
||||
projectId: options.projectId,
|
||||
name: secret.targetName,
|
||||
version: 1,
|
||||
mutationId: secretMutationId,
|
||||
keyId: material.keyId,
|
||||
algorithm: LOCAL_SECRET_ALGORITHM,
|
||||
createdAtMs: options.observedAtMs,
|
||||
},
|
||||
secret.plaintext,
|
||||
material.key,
|
||||
);
|
||||
const item = createLocalDataDirectoryAdoptionSecretItem({
|
||||
projectId: options.projectId,
|
||||
ordinal,
|
||||
kind: secret.kind,
|
||||
sourceNameDigest: createLocalDataDirectorySourceNameDigest(
|
||||
secret.kind,
|
||||
secret.sourceName,
|
||||
),
|
||||
secretName: secret.targetName,
|
||||
secretMutationId,
|
||||
valueFile: secret.valueFile,
|
||||
valueDigest: secret.valueDigest,
|
||||
});
|
||||
publications.push(
|
||||
Object.freeze({
|
||||
ordinal,
|
||||
kind: secret.kind,
|
||||
sourceNameDigest: item.sourceNameDigest,
|
||||
valueFile: secret.valueFile,
|
||||
valueDigest: secret.valueDigest,
|
||||
envelope,
|
||||
secretRef: item.secretRef,
|
||||
itemDigest: item.itemDigest,
|
||||
audit: auditRecord({
|
||||
eventId: secretMutationId,
|
||||
requestId: options.requestId,
|
||||
projectId: options.projectId,
|
||||
principal,
|
||||
operationId: 'secret.create',
|
||||
outcome: 'allowed',
|
||||
reasons: decision.reasons,
|
||||
decision,
|
||||
occurredAtMs: options.observedAtMs,
|
||||
}),
|
||||
}),
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
material.key.fill(0);
|
||||
}
|
||||
const receipt = createLocalDataDirectoryAdoptionReceipt({
|
||||
mutationId: options.mutationId,
|
||||
projectId: options.projectId,
|
||||
profile: options.profile,
|
||||
sourceStageManifestDigest: options.sourceStageManifestDigest,
|
||||
transformationDigest: options.transformationDigest,
|
||||
modelDigest: options.modelDigest,
|
||||
secrets: publications.map(({ envelope, ...item }) =>
|
||||
Object.freeze({
|
||||
ordinal: item.ordinal,
|
||||
kind: item.kind,
|
||||
sourceNameDigest: item.sourceNameDigest,
|
||||
secretName: envelope.name,
|
||||
secretVersion: 1 as const,
|
||||
secretMutationId: envelope.mutationId,
|
||||
valueFile: item.valueFile,
|
||||
valueDigest: item.valueDigest,
|
||||
secretRef: item.secretRef,
|
||||
itemDigest: item.itemDigest,
|
||||
}),
|
||||
),
|
||||
committedAtMs: options.observedAtMs,
|
||||
});
|
||||
const applied = await database.publisher.publish({
|
||||
mutationId: options.mutationId,
|
||||
projectId: options.projectId,
|
||||
profile: options.profile,
|
||||
sourceStageManifestDigest: options.sourceStageManifestDigest,
|
||||
transformationDigest: options.transformationDigest,
|
||||
modelDigest: options.modelDigest,
|
||||
model: prepared.model,
|
||||
subject: principal.subject,
|
||||
fence: decision.fence,
|
||||
audit: auditRecord({
|
||||
eventId: options.mutationId,
|
||||
requestId: options.requestId,
|
||||
projectId: options.projectId,
|
||||
principal,
|
||||
operationId: 'legacy-data.apply',
|
||||
outcome: 'allowed',
|
||||
reasons: decision.reasons,
|
||||
decision,
|
||||
occurredAtMs: options.observedAtMs,
|
||||
}),
|
||||
secrets: Object.freeze(publications),
|
||||
receipt,
|
||||
async confirmExternalAuthority() {
|
||||
await options.confirmPreparedAuthority();
|
||||
await options.confirmAuthenticationAuthority();
|
||||
},
|
||||
});
|
||||
return Object.freeze(applied);
|
||||
} finally {
|
||||
await database.close();
|
||||
}
|
||||
}
|
||||
+272
@@ -0,0 +1,272 @@
|
||||
import {
|
||||
applyReviewedLocalDataDirectoryAdoption,
|
||||
LocalDataDirectoryAdoptionApplicationConfigurationError,
|
||||
} from '@qinglong/local-admin/data-directory-adoption';
|
||||
import { LocalSecretKeyringFileProvider } from '@qinglong/local-secret';
|
||||
import { openLocalSqliteBootstrapDatabase } from '@qinglong/local-sqlite/bootstrap';
|
||||
import {
|
||||
establishAuthenticatedLocalCommand,
|
||||
type AuthenticatedLocalCommand,
|
||||
} from '@qinglong/local-owner-console/authenticated-command';
|
||||
|
||||
import {
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_OPERATION,
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_VERIFY_OPERATION,
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_VERIFY_OPERATION,
|
||||
LocalDataDirectoryAdoptionConfigurationError,
|
||||
type ApplyLocalDataDirectoryAdoptionCommand,
|
||||
type VerifyLocalDataDirectoryAdoptionApplicationCommand,
|
||||
type VerifyLocalDataDirectoryAdoptionCommand,
|
||||
} from '../contract';
|
||||
import { verifyLocalDataDirectoryAdoption } from '../staging';
|
||||
import { transformationAuthority } from '../transformation/files';
|
||||
import {
|
||||
loadStaticTransformation,
|
||||
verifyTransformationManifestBinding,
|
||||
} from '../transformation/manifest';
|
||||
import {
|
||||
reclaimCommittedTransformationModel,
|
||||
verifyCommittedTransformationModel,
|
||||
} from './cleanup';
|
||||
|
||||
export interface LocalDataDirectoryApplicationResult {
|
||||
readonly schemaVersion: 1;
|
||||
readonly operation:
|
||||
| typeof LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_OPERATION
|
||||
| typeof LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_VERIFY_OPERATION;
|
||||
readonly status: 'committed' | 'verified';
|
||||
readonly evidence: Readonly<{
|
||||
profile: 'edge' | 'standalone';
|
||||
databaseStatus: 'inserted' | 'existing';
|
||||
sourceStageManifestDigest: string;
|
||||
transformationDigest: string;
|
||||
modelDigest: string;
|
||||
publicationDigest: string;
|
||||
receiptDigest: string;
|
||||
commitDigest: string;
|
||||
secretCount: number;
|
||||
environmentSecretCount: number;
|
||||
sshSecretCount: number;
|
||||
committedAtMs: number;
|
||||
modelReclaimed: true;
|
||||
plaintextFilesRemoved: true;
|
||||
physicalErasureGuaranteed: false;
|
||||
}>;
|
||||
}
|
||||
|
||||
type ApplicationCommand =
|
||||
| Readonly<ApplyLocalDataDirectoryAdoptionCommand>
|
||||
| Readonly<VerifyLocalDataDirectoryAdoptionApplicationCommand>;
|
||||
|
||||
function sourceVerificationCommand(
|
||||
command: ApplicationCommand,
|
||||
): Readonly<VerifyLocalDataDirectoryAdoptionCommand> {
|
||||
return Object.freeze({
|
||||
schemaVersion: 1,
|
||||
operation: LOCAL_DATA_DIRECTORY_ADOPTION_VERIFY_OPERATION,
|
||||
options: Object.freeze({
|
||||
deploymentRoot: command.options.deploymentRoot,
|
||||
dataRoot: command.options.dataRoot,
|
||||
stagingRoot: command.options.stagingRoot,
|
||||
profile: command.options.profile,
|
||||
sqlite: command.options.sqlite,
|
||||
expectedManifestDigest: command.options.expectedManifestDigest,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
async function loadPreparedModel(
|
||||
command: ApplicationCommand,
|
||||
authority: ReturnType<typeof transformationAuthority>,
|
||||
) {
|
||||
const before = await verifyLocalDataDirectoryAdoption(
|
||||
sourceVerificationCommand(command),
|
||||
);
|
||||
const loaded = loadStaticTransformation({
|
||||
authority,
|
||||
profile: command.options.profile,
|
||||
projectId: command.options.projectId,
|
||||
sourceStageManifestDigest: command.options.expectedManifestDigest,
|
||||
expectedTransformationDigest: command.options.expectedTransformationDigest,
|
||||
});
|
||||
if (
|
||||
loaded.manifest.assessment !== 'ready' ||
|
||||
loaded.manifest.model.manualCategories !== 0 ||
|
||||
(loaded.prepared.model.manualReview as { readonly required?: unknown })
|
||||
.required !== false
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionApplicationConfigurationError(
|
||||
'manual review must be resolved in a new transformation before apply',
|
||||
);
|
||||
}
|
||||
const after = await verifyLocalDataDirectoryAdoption(
|
||||
sourceVerificationCommand(command),
|
||||
);
|
||||
if (JSON.stringify(before.evidence) !== JSON.stringify(after.evidence)) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'staged source changed while preparing the application transaction',
|
||||
);
|
||||
}
|
||||
return loaded.prepared;
|
||||
}
|
||||
|
||||
function result(
|
||||
operation: LocalDataDirectoryApplicationResult['operation'],
|
||||
status: LocalDataDirectoryApplicationResult['status'],
|
||||
databaseStatus: 'inserted' | 'existing',
|
||||
adoption: Awaited<
|
||||
ReturnType<typeof applyReviewedLocalDataDirectoryAdoption>
|
||||
>['adoption'],
|
||||
commit: ReturnType<typeof reclaimCommittedTransformationModel>,
|
||||
): Readonly<LocalDataDirectoryApplicationResult> {
|
||||
return Object.freeze({
|
||||
schemaVersion: 1,
|
||||
operation,
|
||||
status,
|
||||
evidence: Object.freeze({
|
||||
profile: adoption.profile,
|
||||
databaseStatus,
|
||||
sourceStageManifestDigest: adoption.sourceStageManifestDigest,
|
||||
transformationDigest: adoption.transformationDigest,
|
||||
modelDigest: adoption.modelDigest,
|
||||
publicationDigest: adoption.publicationDigest,
|
||||
receiptDigest: adoption.receiptDigest,
|
||||
commitDigest: commit.commitDigest,
|
||||
secretCount: adoption.receipt.secretCount,
|
||||
environmentSecretCount: adoption.receipt.environmentSecretCount,
|
||||
sshSecretCount: adoption.receipt.sshSecretCount,
|
||||
committedAtMs: adoption.committedAtMs,
|
||||
modelReclaimed: true,
|
||||
plaintextFilesRemoved: true,
|
||||
physicalErasureGuaranteed: false,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
async function authenticatedAuthority(command: ApplicationCommand): Promise<
|
||||
Readonly<{
|
||||
authenticated: Readonly<AuthenticatedLocalCommand>;
|
||||
close(): Promise<void>;
|
||||
}>
|
||||
> {
|
||||
const database = await openLocalSqliteBootstrapDatabase({
|
||||
databasePath: command.options.sqlite.targetPath,
|
||||
profile: command.options.profile,
|
||||
...(command.options.busyTimeoutMs === undefined
|
||||
? {}
|
||||
: { busyTimeoutMs: command.options.busyTimeoutMs }),
|
||||
});
|
||||
try {
|
||||
const authenticated = await establishAuthenticatedLocalCommand(database, {
|
||||
deploymentRoot: command.options.deploymentRoot,
|
||||
databasePath: command.options.sqlite.targetPath,
|
||||
ownerPepperKeyringDirectory: command.options.ownerPepperKeyringDirectory,
|
||||
credentialFilePath: command.options.credentialFilePath,
|
||||
authenticationNamespace: 'local_data_adoption',
|
||||
});
|
||||
return Object.freeze({
|
||||
authenticated,
|
||||
close: () => database.close(),
|
||||
});
|
||||
} catch (error) {
|
||||
await database.close();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function apply(
|
||||
command: ApplicationCommand,
|
||||
verifyOnly: boolean,
|
||||
): Promise<Readonly<LocalDataDirectoryApplicationResult>> {
|
||||
const authority = transformationAuthority(command.options, false);
|
||||
const manifest = verifyTransformationManifestBinding({
|
||||
authority,
|
||||
profile: command.options.profile,
|
||||
projectId: command.options.projectId,
|
||||
sourceStageManifestDigest: command.options.expectedManifestDigest,
|
||||
expectedTransformationDigest: command.options.expectedTransformationDigest,
|
||||
});
|
||||
const authentication = await authenticatedAuthority(command);
|
||||
try {
|
||||
const applied = await applyReviewedLocalDataDirectoryAdoption({
|
||||
databasePath: command.options.sqlite.targetPath,
|
||||
profile: command.options.profile,
|
||||
projectId: command.options.projectId,
|
||||
mutationId: command.options.mutationId,
|
||||
failureAuditEventId: command.options.failureAuditEventId,
|
||||
requestId: command.options.requestId,
|
||||
sourceStageManifestDigest: command.options.expectedManifestDigest,
|
||||
transformationDigest: command.options.expectedTransformationDigest,
|
||||
modelDigest: manifest.model.digest,
|
||||
principal: authentication.authenticated.principal,
|
||||
keyProvider: new LocalSecretKeyringFileProvider(
|
||||
command.options.secretKeyringPath,
|
||||
),
|
||||
observedAtMs: Date.now(),
|
||||
...(command.options.busyTimeoutMs === undefined
|
||||
? {}
|
||||
: { busyTimeoutMs: command.options.busyTimeoutMs }),
|
||||
loadPreparedModel: verifyOnly
|
||||
? () => {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'application receipt does not exist',
|
||||
);
|
||||
}
|
||||
: () => loadPreparedModel(command, authority),
|
||||
confirmAuthenticationAuthority: () =>
|
||||
authentication.authenticated.confirm(),
|
||||
async confirmPreparedAuthority() {
|
||||
await loadPreparedModel(command, authority);
|
||||
},
|
||||
});
|
||||
if (
|
||||
applied.adoption.modelDigest !== manifest.model.digest ||
|
||||
applied.adoption.transformationDigest !== manifest.transformationDigest
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'database application does not match the transformation manifest',
|
||||
);
|
||||
}
|
||||
if (verifyOnly) {
|
||||
const verifyCommand =
|
||||
command as Readonly<VerifyLocalDataDirectoryAdoptionApplicationCommand>;
|
||||
const commit = verifyCommittedTransformationModel({
|
||||
authority,
|
||||
adoption: applied.adoption,
|
||||
expectedReceiptDigest: verifyCommand.options.expectedReceiptDigest,
|
||||
});
|
||||
return result(
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_VERIFY_OPERATION,
|
||||
'verified',
|
||||
applied.status,
|
||||
applied.adoption,
|
||||
commit,
|
||||
);
|
||||
}
|
||||
const commit = reclaimCommittedTransformationModel({
|
||||
authority,
|
||||
adoption: applied.adoption,
|
||||
});
|
||||
return result(
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_OPERATION,
|
||||
'committed',
|
||||
applied.status,
|
||||
applied.adoption,
|
||||
commit,
|
||||
);
|
||||
} finally {
|
||||
await authentication.close();
|
||||
}
|
||||
}
|
||||
|
||||
export function applyLocalDataDirectoryAdoption(
|
||||
command: Readonly<ApplyLocalDataDirectoryAdoptionCommand>,
|
||||
): Promise<Readonly<LocalDataDirectoryApplicationResult>> {
|
||||
return apply(command, false);
|
||||
}
|
||||
|
||||
export function verifyLocalDataDirectoryAdoptionApplication(
|
||||
command: Readonly<VerifyLocalDataDirectoryAdoptionApplicationCommand>,
|
||||
): Promise<Readonly<LocalDataDirectoryApplicationResult>> {
|
||||
return apply(command, true);
|
||||
}
|
||||
+468
@@ -0,0 +1,468 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
import {
|
||||
createLocalDataDirectorySourceNameDigest,
|
||||
type LocalDataDirectoryAdoptionRecord,
|
||||
} from '@qinglong/local-sqlite/data-directory-adoption';
|
||||
|
||||
import { LocalDataDirectoryAdoptionConfigurationError } from '../contract';
|
||||
import {
|
||||
assertPrivateDirectory,
|
||||
sameStat,
|
||||
sortedNames,
|
||||
syncDirectory,
|
||||
} from '../filesystem';
|
||||
import { sha256Text } from '../manifest';
|
||||
import {
|
||||
readStablePrivateUtf8File,
|
||||
writePrivateJson,
|
||||
type TransformationAuthority,
|
||||
} from '../transformation/files';
|
||||
import { verifyTransformationModel } from '../transformation/model';
|
||||
import { verifyTransformationManifestBinding } from '../transformation/manifest';
|
||||
|
||||
export const APPLICATION_COMMIT_NAME = 'commit.json';
|
||||
const COMMIT_INCOMPLETE_NAME = '.commit-incomplete';
|
||||
const RECLAIMING_MODEL_NAME = '.reclaiming-model';
|
||||
const MODEL_NAME = 'model';
|
||||
const MAX_JSON_BYTES = 1024 * 1024;
|
||||
const ZERO_CHUNK = Buffer.alloc(64 * 1024);
|
||||
|
||||
interface LocalDataDirectoryApplicationCommitPayload {
|
||||
readonly schemaVersion: 1;
|
||||
readonly kind: 'qinglong3-legacy-data-directory-application';
|
||||
readonly state: 'committed';
|
||||
readonly mutationId: string;
|
||||
readonly profile: 'edge' | 'standalone';
|
||||
readonly projectIdDigest: string;
|
||||
readonly sourceStageManifestDigest: string;
|
||||
readonly transformationDigest: string;
|
||||
readonly modelDigest: string;
|
||||
readonly publicationDigest: string;
|
||||
readonly receiptDigest: string;
|
||||
readonly secretCount: number;
|
||||
readonly environmentSecretCount: number;
|
||||
readonly sshSecretCount: number;
|
||||
readonly committedAtMs: number;
|
||||
readonly reclamation: Readonly<{
|
||||
modelRemoved: true;
|
||||
plaintextFilesRemoved: true;
|
||||
physicalErasureGuaranteed: false;
|
||||
}>;
|
||||
}
|
||||
|
||||
export interface LocalDataDirectoryApplicationCommit
|
||||
extends LocalDataDirectoryApplicationCommitPayload {
|
||||
readonly commitDigest: string;
|
||||
}
|
||||
|
||||
function exactKeys(value: object, expected: readonly string[]): boolean {
|
||||
const actual = Object.keys(value).sort();
|
||||
const canonical = [...expected].sort();
|
||||
return (
|
||||
actual.length === canonical.length &&
|
||||
actual.every((key, index) => key === canonical[index])
|
||||
);
|
||||
}
|
||||
|
||||
function expectedCommit(
|
||||
adoption: Readonly<LocalDataDirectoryAdoptionRecord>,
|
||||
): Readonly<LocalDataDirectoryApplicationCommit> {
|
||||
const payload: LocalDataDirectoryApplicationCommitPayload = {
|
||||
schemaVersion: 1,
|
||||
kind: 'qinglong3-legacy-data-directory-application',
|
||||
state: 'committed',
|
||||
mutationId: adoption.mutationId,
|
||||
profile: adoption.profile,
|
||||
projectIdDigest: sha256Text(adoption.projectId),
|
||||
sourceStageManifestDigest: adoption.sourceStageManifestDigest,
|
||||
transformationDigest: adoption.transformationDigest,
|
||||
modelDigest: adoption.modelDigest,
|
||||
publicationDigest: adoption.publicationDigest,
|
||||
receiptDigest: adoption.receiptDigest,
|
||||
secretCount: adoption.receipt.secretCount,
|
||||
environmentSecretCount: adoption.receipt.environmentSecretCount,
|
||||
sshSecretCount: adoption.receipt.sshSecretCount,
|
||||
committedAtMs: adoption.committedAtMs,
|
||||
reclamation: Object.freeze({
|
||||
modelRemoved: true,
|
||||
plaintextFilesRemoved: true,
|
||||
physicalErasureGuaranteed: false,
|
||||
}),
|
||||
};
|
||||
return Object.freeze({
|
||||
...payload,
|
||||
commitDigest: sha256Text(JSON.stringify(payload)),
|
||||
});
|
||||
}
|
||||
|
||||
function readJson(filePath: string, uid: number, label: string): unknown {
|
||||
try {
|
||||
return JSON.parse(
|
||||
readStablePrivateUtf8File(filePath, uid, MAX_JSON_BYTES, label),
|
||||
);
|
||||
} catch (error) {
|
||||
if (error instanceof LocalDataDirectoryAdoptionConfigurationError) {
|
||||
throw error;
|
||||
}
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
`${label} JSON is invalid`,
|
||||
error,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function sameJson(left: unknown, right: unknown): boolean {
|
||||
return JSON.stringify(left) === JSON.stringify(right);
|
||||
}
|
||||
|
||||
function verifyMarker(
|
||||
root: string,
|
||||
uid: number,
|
||||
adoption: Readonly<LocalDataDirectoryAdoptionRecord>,
|
||||
): void {
|
||||
const marker = readJson(
|
||||
path.join(root, COMMIT_INCOMPLETE_NAME),
|
||||
uid,
|
||||
'application recovery marker',
|
||||
);
|
||||
if (
|
||||
!marker ||
|
||||
typeof marker !== 'object' ||
|
||||
Array.isArray(marker) ||
|
||||
!exactKeys(marker, [
|
||||
'kind',
|
||||
'mutationId',
|
||||
'receiptDigest',
|
||||
'schemaVersion',
|
||||
'transformationDigest',
|
||||
])
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'application recovery marker shape is invalid',
|
||||
);
|
||||
}
|
||||
const candidate = marker as Record<string, unknown>;
|
||||
if (
|
||||
candidate.schemaVersion !== 1 ||
|
||||
candidate.kind !==
|
||||
'qinglong3-legacy-data-directory-application-incomplete' ||
|
||||
candidate.mutationId !== adoption.mutationId ||
|
||||
candidate.transformationDigest !== adoption.transformationDigest ||
|
||||
candidate.receiptDigest !== adoption.receiptDigest
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'application recovery marker does not match the committed database',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function verifyCommit(
|
||||
authority: Readonly<TransformationAuthority>,
|
||||
adoption: Readonly<LocalDataDirectoryAdoptionRecord>,
|
||||
): Readonly<LocalDataDirectoryApplicationCommit> {
|
||||
const actual = readJson(
|
||||
path.join(authority.transformationRoot, APPLICATION_COMMIT_NAME),
|
||||
authority.uid,
|
||||
'application commit',
|
||||
);
|
||||
const expected = expectedCommit(adoption);
|
||||
if (!sameJson(actual, expected)) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'application commit does not match the durable database receipt',
|
||||
);
|
||||
}
|
||||
return expected;
|
||||
}
|
||||
|
||||
function assertPreparedMatches(
|
||||
modelRoot: string,
|
||||
authority: Readonly<TransformationAuthority>,
|
||||
adoption: Readonly<LocalDataDirectoryAdoptionRecord>,
|
||||
): void {
|
||||
const manifest = verifyTransformationManifestBinding({
|
||||
authority,
|
||||
profile: adoption.profile,
|
||||
projectId: adoption.projectId,
|
||||
sourceStageManifestDigest: adoption.sourceStageManifestDigest,
|
||||
expectedTransformationDigest: adoption.transformationDigest,
|
||||
});
|
||||
const prepared = verifyTransformationModel({
|
||||
modelRoot,
|
||||
uid: authority.uid,
|
||||
projectId: adoption.projectId,
|
||||
profile: adoption.profile,
|
||||
expected: manifest.model,
|
||||
});
|
||||
if (
|
||||
manifest.assessment !== 'ready' ||
|
||||
manifest.model.manualCategories !== 0 ||
|
||||
!sameJson(prepared.model, adoption.model) ||
|
||||
prepared.secrets.length !== adoption.secrets.length ||
|
||||
prepared.secrets.some((secret, index) => {
|
||||
const stored = adoption.secrets[index];
|
||||
return (
|
||||
!stored ||
|
||||
stored.ordinal !== index + 1 ||
|
||||
stored.kind !== secret.kind ||
|
||||
stored.sourceNameDigest !==
|
||||
createLocalDataDirectorySourceNameDigest(
|
||||
secret.kind,
|
||||
secret.sourceName,
|
||||
) ||
|
||||
stored.secretName !== secret.targetName ||
|
||||
stored.valueFile !== secret.valueFile ||
|
||||
stored.valueDigest !== secret.valueDigest
|
||||
);
|
||||
})
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'prepared model does not match the committed database receipt',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function privateEntry(
|
||||
entryPath: string,
|
||||
uid: number,
|
||||
kind: 'file' | 'directory',
|
||||
): fs.BigIntStats {
|
||||
const stat = fs.lstatSync(entryPath, { bigint: true });
|
||||
if (
|
||||
stat.isSymbolicLink() ||
|
||||
stat.uid !== BigInt(uid) ||
|
||||
(stat.mode & 0o777n) !== (kind === 'file' ? 0o600n : 0o700n) ||
|
||||
(kind === 'file'
|
||||
? !stat.isFile() || stat.nlink !== 1n
|
||||
: !stat.isDirectory())
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'reclaiming model identity or mode is invalid',
|
||||
);
|
||||
}
|
||||
return stat;
|
||||
}
|
||||
|
||||
function overwriteAndUnlink(filePath: string, uid: number): void {
|
||||
const expected = privateEntry(filePath, uid, 'file');
|
||||
if (expected.size < 0n || expected.size > BigInt(32 * 1024)) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'reclaiming Secret file size is invalid',
|
||||
);
|
||||
}
|
||||
const descriptor = fs.openSync(
|
||||
filePath,
|
||||
fs.constants.O_RDWR | (fs.constants.O_NOFOLLOW ?? 0),
|
||||
);
|
||||
try {
|
||||
const opened = fs.fstatSync(descriptor, { bigint: true });
|
||||
if (!sameStat(expected, opened)) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'reclaiming Secret identity changed before overwrite',
|
||||
);
|
||||
}
|
||||
let remaining = Number(opened.size);
|
||||
let offset = 0;
|
||||
while (remaining > 0) {
|
||||
const count = Math.min(remaining, ZERO_CHUNK.length);
|
||||
const written = fs.writeSync(descriptor, ZERO_CHUNK, 0, count, offset);
|
||||
if (written < 1) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'reclaiming Secret overwrite made no progress',
|
||||
);
|
||||
}
|
||||
remaining -= written;
|
||||
offset += written;
|
||||
}
|
||||
fs.fsyncSync(descriptor);
|
||||
} finally {
|
||||
fs.closeSync(descriptor);
|
||||
}
|
||||
fs.unlinkSync(filePath);
|
||||
}
|
||||
|
||||
function reclaimRenamedModel(
|
||||
root: string,
|
||||
uid: number,
|
||||
adoption: Readonly<LocalDataDirectoryAdoptionRecord>,
|
||||
): void {
|
||||
const modelRoot = path.join(root, RECLAIMING_MODEL_NAME);
|
||||
privateEntry(modelRoot, uid, 'directory');
|
||||
const allowedRootFiles = new Set([
|
||||
'config.json',
|
||||
'keyv.json',
|
||||
'manual-review.json',
|
||||
'secret-imports.json',
|
||||
'ssh.json',
|
||||
'secret-values',
|
||||
]);
|
||||
if (sortedNames(modelRoot).some((name) => !allowedRootFiles.has(name))) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'reclaiming model contains an unexpected entry',
|
||||
);
|
||||
}
|
||||
const secretRoot = path.join(modelRoot, 'secret-values');
|
||||
if (fs.existsSync(secretRoot)) {
|
||||
privateEntry(secretRoot, uid, 'directory');
|
||||
const expectedFiles = new Set(
|
||||
adoption.secrets.map(({ valueFile }) => path.basename(valueFile)),
|
||||
);
|
||||
const actualFiles = sortedNames(secretRoot);
|
||||
if (actualFiles.some((name) => !expectedFiles.has(name))) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'reclaiming Secret directory contains an unexpected entry',
|
||||
);
|
||||
}
|
||||
for (const name of actualFiles) {
|
||||
overwriteAndUnlink(path.join(secretRoot, name), uid);
|
||||
}
|
||||
fs.rmdirSync(secretRoot);
|
||||
syncDirectory(modelRoot);
|
||||
}
|
||||
for (const name of [
|
||||
'config.json',
|
||||
'keyv.json',
|
||||
'ssh.json',
|
||||
'manual-review.json',
|
||||
'secret-imports.json',
|
||||
]) {
|
||||
const filePath = path.join(modelRoot, name);
|
||||
if (!fs.existsSync(filePath)) continue;
|
||||
privateEntry(filePath, uid, 'file');
|
||||
fs.unlinkSync(filePath);
|
||||
}
|
||||
syncDirectory(modelRoot);
|
||||
if (sortedNames(modelRoot).length !== 0) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'reclaiming model could not be emptied',
|
||||
);
|
||||
}
|
||||
fs.rmdirSync(modelRoot);
|
||||
syncDirectory(root);
|
||||
}
|
||||
|
||||
export function reclaimCommittedTransformationModel(options: {
|
||||
readonly authority: Readonly<TransformationAuthority>;
|
||||
readonly adoption: Readonly<LocalDataDirectoryAdoptionRecord>;
|
||||
}): Readonly<LocalDataDirectoryApplicationCommit> {
|
||||
const { authority, adoption } = options;
|
||||
const root = authority.transformationRoot;
|
||||
const before = assertPrivateDirectory(
|
||||
root,
|
||||
authority.uid,
|
||||
'transformationRoot',
|
||||
);
|
||||
verifyTransformationManifestBinding({
|
||||
authority,
|
||||
profile: adoption.profile,
|
||||
projectId: adoption.projectId,
|
||||
sourceStageManifestDigest: adoption.sourceStageManifestDigest,
|
||||
expectedTransformationDigest: adoption.transformationDigest,
|
||||
});
|
||||
const initialNames = sortedNames(root);
|
||||
const allowedNames = new Set([
|
||||
'manifest.json',
|
||||
APPLICATION_COMMIT_NAME,
|
||||
COMMIT_INCOMPLETE_NAME,
|
||||
MODEL_NAME,
|
||||
RECLAIMING_MODEL_NAME,
|
||||
]);
|
||||
if (
|
||||
initialNames.some((name) => !allowedNames.has(name)) ||
|
||||
!initialNames.includes('manifest.json') ||
|
||||
(initialNames.includes(MODEL_NAME) &&
|
||||
initialNames.includes(RECLAIMING_MODEL_NAME))
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'application recovery root shape is invalid',
|
||||
);
|
||||
}
|
||||
if (!initialNames.includes(COMMIT_INCOMPLETE_NAME)) {
|
||||
if (
|
||||
initialNames.includes(APPLICATION_COMMIT_NAME) &&
|
||||
!initialNames.includes(MODEL_NAME) &&
|
||||
!initialNames.includes(RECLAIMING_MODEL_NAME)
|
||||
) {
|
||||
return verifyCommit(authority, adoption);
|
||||
}
|
||||
writePrivateJson(path.join(root, COMMIT_INCOMPLETE_NAME), {
|
||||
schemaVersion: 1,
|
||||
kind: 'qinglong3-legacy-data-directory-application-incomplete',
|
||||
mutationId: adoption.mutationId,
|
||||
transformationDigest: adoption.transformationDigest,
|
||||
receiptDigest: adoption.receiptDigest,
|
||||
});
|
||||
syncDirectory(root);
|
||||
}
|
||||
verifyMarker(root, authority.uid, adoption);
|
||||
|
||||
const modelRoot = path.join(root, MODEL_NAME);
|
||||
const reclaimingRoot = path.join(root, RECLAIMING_MODEL_NAME);
|
||||
if (fs.existsSync(modelRoot)) {
|
||||
assertPreparedMatches(modelRoot, authority, adoption);
|
||||
if (fs.existsSync(reclaimingRoot)) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'multiple application model recovery roots exist',
|
||||
);
|
||||
}
|
||||
fs.renameSync(modelRoot, reclaimingRoot);
|
||||
syncDirectory(root);
|
||||
}
|
||||
if (fs.existsSync(reclaimingRoot)) {
|
||||
reclaimRenamedModel(root, authority.uid, adoption);
|
||||
}
|
||||
|
||||
const commitPath = path.join(root, APPLICATION_COMMIT_NAME);
|
||||
if (!fs.existsSync(commitPath)) {
|
||||
writePrivateJson(commitPath, expectedCommit(adoption));
|
||||
syncDirectory(root);
|
||||
}
|
||||
const commit = verifyCommit(authority, adoption);
|
||||
fs.unlinkSync(path.join(root, COMMIT_INCOMPLETE_NAME));
|
||||
syncDirectory(root);
|
||||
if (
|
||||
JSON.stringify(sortedNames(root)) !==
|
||||
JSON.stringify([APPLICATION_COMMIT_NAME, 'manifest.json'].sort())
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'committed transformation root contains unexpected entries',
|
||||
);
|
||||
}
|
||||
const after = fs.lstatSync(root, { bigint: true });
|
||||
if (
|
||||
before.dev !== after.dev ||
|
||||
before.ino !== after.ino ||
|
||||
before.uid !== after.uid ||
|
||||
(after.mode & 0o777n) !== 0o700n
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'transformation root identity changed during application cleanup',
|
||||
);
|
||||
}
|
||||
return commit;
|
||||
}
|
||||
|
||||
export function verifyCommittedTransformationModel(options: {
|
||||
readonly authority: Readonly<TransformationAuthority>;
|
||||
readonly adoption: Readonly<LocalDataDirectoryAdoptionRecord>;
|
||||
readonly expectedReceiptDigest: string;
|
||||
}): Readonly<LocalDataDirectoryApplicationCommit> {
|
||||
if (
|
||||
options.adoption.receiptDigest !== options.expectedReceiptDigest ||
|
||||
JSON.stringify(sortedNames(options.authority.transformationRoot)) !==
|
||||
JSON.stringify([APPLICATION_COMMIT_NAME, 'manifest.json'].sort())
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'committed transformation evidence is incomplete',
|
||||
);
|
||||
}
|
||||
verifyTransformationManifestBinding({
|
||||
authority: options.authority,
|
||||
profile: options.adoption.profile,
|
||||
projectId: options.adoption.projectId,
|
||||
sourceStageManifestDigest: options.adoption.sourceStageManifestDigest,
|
||||
expectedTransformationDigest: options.adoption.transformationDigest,
|
||||
});
|
||||
return verifyCommit(options.authority, options.adoption);
|
||||
}
|
||||
@@ -1,10 +1,17 @@
|
||||
import {
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_OPERATION,
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_VERIFY_OPERATION,
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_INSPECT_OPERATION,
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_STAGE_OPERATION,
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_OPERATION,
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_VERIFY_OPERATION,
|
||||
normalizeLocalDataDirectoryAdoptionCommand,
|
||||
} from './contract';
|
||||
import {
|
||||
applyLocalDataDirectoryAdoption,
|
||||
verifyLocalDataDirectoryAdoptionApplication,
|
||||
type LocalDataDirectoryApplicationResult,
|
||||
} from './application/application';
|
||||
import {
|
||||
inspectLocalDataDirectoryAdoption,
|
||||
type LocalDataDirectoryAdoptionInspectResult,
|
||||
@@ -23,7 +30,8 @@ import {
|
||||
export type LocalDataDirectoryAdoptionProductCommandResult =
|
||||
| LocalDataDirectoryAdoptionInspectResult
|
||||
| LocalDataDirectoryAdoptionMutationResult
|
||||
| LocalDataDirectoryTransformationResult;
|
||||
| LocalDataDirectoryTransformationResult
|
||||
| LocalDataDirectoryApplicationResult;
|
||||
|
||||
export async function runLocalDataDirectoryAdoptionProductCommand(
|
||||
value: unknown,
|
||||
@@ -41,5 +49,13 @@ export async function runLocalDataDirectoryAdoptionProductCommand(
|
||||
if (command.operation === LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_OPERATION) {
|
||||
return transformLocalDataDirectoryAdoption(command);
|
||||
}
|
||||
if (command.operation === LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_OPERATION) {
|
||||
return applyLocalDataDirectoryAdoption(command);
|
||||
}
|
||||
if (
|
||||
command.operation === LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_VERIFY_OPERATION
|
||||
) {
|
||||
return verifyLocalDataDirectoryAdoptionApplication(command);
|
||||
}
|
||||
return verifyLocalDataDirectoryAdoptionTransformation(command);
|
||||
}
|
||||
|
||||
@@ -3,6 +3,9 @@ import path from 'node:path';
|
||||
const MAX_PATH_BYTES = 4_096;
|
||||
const DIGEST_PATTERN = /^[0-9a-f]{64}$/;
|
||||
const PROJECT_ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$/;
|
||||
const UUID_V4_PATTERN =
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/;
|
||||
const REQUEST_ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$/;
|
||||
|
||||
export const LOCAL_DATA_DIRECTORY_ADOPTION_INSPECT_OPERATION =
|
||||
'local-data-directory.adoption.inspect' as const;
|
||||
@@ -14,13 +17,19 @@ export const LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_OPERATION =
|
||||
'local-data-directory.adoption.transform' as const;
|
||||
export const LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_VERIFY_OPERATION =
|
||||
'local-data-directory.adoption.transform.verify' as const;
|
||||
export const LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_OPERATION =
|
||||
'local-data-directory.adoption.apply' as const;
|
||||
export const LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_VERIFY_OPERATION =
|
||||
'local-data-directory.adoption.apply.verify' as const;
|
||||
|
||||
export type LocalDataDirectoryAdoptionOperation =
|
||||
| typeof LOCAL_DATA_DIRECTORY_ADOPTION_INSPECT_OPERATION
|
||||
| typeof LOCAL_DATA_DIRECTORY_ADOPTION_STAGE_OPERATION
|
||||
| typeof LOCAL_DATA_DIRECTORY_ADOPTION_VERIFY_OPERATION
|
||||
| typeof LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_OPERATION
|
||||
| typeof LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_VERIFY_OPERATION;
|
||||
| typeof LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_VERIFY_OPERATION
|
||||
| typeof LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_OPERATION
|
||||
| typeof LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_VERIFY_OPERATION;
|
||||
|
||||
export interface InspectLocalDataDirectoryAdoptionCommand {
|
||||
readonly schemaVersion: 1;
|
||||
@@ -85,12 +94,40 @@ export interface VerifyLocalDataDirectoryAdoptionTransformationCommand {
|
||||
};
|
||||
}
|
||||
|
||||
interface LocalDataDirectoryAdoptionApplicationOptions
|
||||
extends LocalDataDirectoryAdoptionTransformationOptions {
|
||||
readonly expectedTransformationDigest: string;
|
||||
readonly ownerPepperKeyringDirectory: string;
|
||||
readonly credentialFilePath: string;
|
||||
readonly secretKeyringPath: string;
|
||||
readonly mutationId: string;
|
||||
readonly failureAuditEventId: string;
|
||||
readonly requestId: string;
|
||||
readonly busyTimeoutMs?: number;
|
||||
}
|
||||
|
||||
export interface ApplyLocalDataDirectoryAdoptionCommand {
|
||||
readonly schemaVersion: 1;
|
||||
readonly operation: typeof LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_OPERATION;
|
||||
readonly options: LocalDataDirectoryAdoptionApplicationOptions;
|
||||
}
|
||||
|
||||
export interface VerifyLocalDataDirectoryAdoptionApplicationCommand {
|
||||
readonly schemaVersion: 1;
|
||||
readonly operation: typeof LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_VERIFY_OPERATION;
|
||||
readonly options: LocalDataDirectoryAdoptionApplicationOptions & {
|
||||
readonly expectedReceiptDigest: string;
|
||||
};
|
||||
}
|
||||
|
||||
export type LocalDataDirectoryAdoptionCommand =
|
||||
| InspectLocalDataDirectoryAdoptionCommand
|
||||
| StageLocalDataDirectoryAdoptionCommand
|
||||
| VerifyLocalDataDirectoryAdoptionCommand
|
||||
| TransformLocalDataDirectoryAdoptionCommand
|
||||
| VerifyLocalDataDirectoryAdoptionTransformationCommand;
|
||||
| VerifyLocalDataDirectoryAdoptionTransformationCommand
|
||||
| ApplyLocalDataDirectoryAdoptionCommand
|
||||
| VerifyLocalDataDirectoryAdoptionApplicationCommand;
|
||||
|
||||
export class LocalDataDirectoryAdoptionConfigurationError extends TypeError {
|
||||
readonly code = 'LOCAL_DATA_DIRECTORY_ADOPTION_CONFIGURATION_INVALID';
|
||||
@@ -129,7 +166,19 @@ export function isLocalDataDirectoryAdoptionOperation(
|
||||
value === LOCAL_DATA_DIRECTORY_ADOPTION_STAGE_OPERATION ||
|
||||
value === LOCAL_DATA_DIRECTORY_ADOPTION_VERIFY_OPERATION ||
|
||||
value === LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_OPERATION ||
|
||||
value === LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_VERIFY_OPERATION
|
||||
value === LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_VERIFY_OPERATION ||
|
||||
value === LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_OPERATION ||
|
||||
value === LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_VERIFY_OPERATION
|
||||
);
|
||||
}
|
||||
|
||||
function descendant(root: string, candidate: string): boolean {
|
||||
const relative = path.relative(root, candidate);
|
||||
return (
|
||||
relative.length > 0 &&
|
||||
relative !== '..' &&
|
||||
!relative.startsWith(`..${path.sep}`) &&
|
||||
!path.isAbsolute(relative)
|
||||
);
|
||||
}
|
||||
|
||||
@@ -219,6 +268,10 @@ export function normalizeLocalDataDirectoryAdoptionCommand(
|
||||
);
|
||||
}
|
||||
const options = candidate.options as Record<string, unknown>;
|
||||
const application =
|
||||
candidate.operation === LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_OPERATION ||
|
||||
candidate.operation ===
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_VERIFY_OPERATION;
|
||||
let expectedKeys: readonly string[];
|
||||
if (candidate.operation === LOCAL_DATA_DIRECTORY_ADOPTION_INSPECT_OPERATION) {
|
||||
expectedKeys = ['dataRoot', 'profile'];
|
||||
@@ -245,9 +298,24 @@ export function normalizeLocalDataDirectoryAdoptionCommand(
|
||||
? []
|
||||
: ['projectId', 'transformationRoot']),
|
||||
...(candidate.operation ===
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_VERIFY_OPERATION
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_VERIFY_OPERATION || application
|
||||
? ['expectedTransformationDigest']
|
||||
: []),
|
||||
...(application
|
||||
? [
|
||||
'credentialFilePath',
|
||||
'failureAuditEventId',
|
||||
'mutationId',
|
||||
'ownerPepperKeyringDirectory',
|
||||
'requestId',
|
||||
'secretKeyringPath',
|
||||
...(options.busyTimeoutMs === undefined ? [] : ['busyTimeoutMs']),
|
||||
]
|
||||
: []),
|
||||
...(candidate.operation ===
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_VERIFY_OPERATION
|
||||
? ['expectedReceiptDigest']
|
||||
: []),
|
||||
];
|
||||
}
|
||||
if (
|
||||
@@ -282,7 +350,8 @@ export function normalizeLocalDataDirectoryAdoptionCommand(
|
||||
candidate.operation ===
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_OPERATION ||
|
||||
candidate.operation ===
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_VERIFY_OPERATION
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_VERIFY_OPERATION ||
|
||||
application
|
||||
) {
|
||||
if (
|
||||
!normalizedAbsolutePath(options.transformationRoot) ||
|
||||
@@ -294,8 +363,9 @@ export function normalizeLocalDataDirectoryAdoptionCommand(
|
||||
);
|
||||
}
|
||||
if (
|
||||
candidate.operation ===
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_VERIFY_OPERATION &&
|
||||
(candidate.operation ===
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_TRANSFORM_VERIFY_OPERATION ||
|
||||
application) &&
|
||||
(typeof options.expectedTransformationDigest !== 'string' ||
|
||||
!DIGEST_PATTERN.test(options.expectedTransformationDigest))
|
||||
) {
|
||||
@@ -303,6 +373,47 @@ export function normalizeLocalDataDirectoryAdoptionCommand(
|
||||
'transformation digest is invalid',
|
||||
);
|
||||
}
|
||||
if (application) {
|
||||
for (const key of [
|
||||
'ownerPepperKeyringDirectory',
|
||||
'credentialFilePath',
|
||||
'secretKeyringPath',
|
||||
]) {
|
||||
if (
|
||||
!normalizedAbsolutePath(options[key]) ||
|
||||
!descendant(
|
||||
options.deploymentRoot as string,
|
||||
options[key] as string,
|
||||
)
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'application authority path is invalid',
|
||||
);
|
||||
}
|
||||
}
|
||||
if (
|
||||
options.credentialFilePath === options.secretKeyringPath ||
|
||||
typeof options.mutationId !== 'string' ||
|
||||
!UUID_V4_PATTERN.test(options.mutationId) ||
|
||||
typeof options.failureAuditEventId !== 'string' ||
|
||||
!UUID_V4_PATTERN.test(options.failureAuditEventId) ||
|
||||
options.failureAuditEventId === options.mutationId ||
|
||||
typeof options.requestId !== 'string' ||
|
||||
!REQUEST_ID_PATTERN.test(options.requestId) ||
|
||||
(options.busyTimeoutMs !== undefined &&
|
||||
(!Number.isSafeInteger(options.busyTimeoutMs) ||
|
||||
(options.busyTimeoutMs as number) < 100 ||
|
||||
(options.busyTimeoutMs as number) > 30_000)) ||
|
||||
(candidate.operation ===
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_APPLY_VERIFY_OPERATION &&
|
||||
(typeof options.expectedReceiptDigest !== 'string' ||
|
||||
!DIGEST_PATTERN.test(options.expectedReceiptDigest)))
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'application authority binding is invalid',
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return Object.freeze(value as LocalDataDirectoryAdoptionCommand);
|
||||
|
||||
+51
-2
@@ -13,6 +13,7 @@ import {
|
||||
type LocalDataDirectoryTransformationManifest,
|
||||
type TransformationModelEvidence,
|
||||
type TransformationSourceEvidence,
|
||||
type VerifiedTransformationModel,
|
||||
} from './model';
|
||||
|
||||
export const TRANSFORMATION_MANIFEST_NAME = 'manifest.json';
|
||||
@@ -210,6 +211,54 @@ export function verifyStaticTransformation(options: {
|
||||
readonly sourceStageManifestDigest: string;
|
||||
readonly expectedTransformationDigest: string;
|
||||
}): Readonly<LocalDataDirectoryTransformationManifest> {
|
||||
return loadStaticTransformation(options).manifest;
|
||||
}
|
||||
|
||||
export function verifyTransformationManifestBinding(options: {
|
||||
readonly authority: Readonly<TransformationAuthority>;
|
||||
readonly profile: 'edge' | 'standalone';
|
||||
readonly projectId: string;
|
||||
readonly sourceStageManifestDigest: string;
|
||||
readonly expectedTransformationDigest: string;
|
||||
}): Readonly<LocalDataDirectoryTransformationManifest> {
|
||||
const before = assertPrivateDirectory(
|
||||
options.authority.transformationRoot,
|
||||
options.authority.uid,
|
||||
'transformationRoot',
|
||||
);
|
||||
const manifest = readManifest(
|
||||
options.authority.transformationRoot,
|
||||
options.authority.uid,
|
||||
);
|
||||
if (
|
||||
manifest.transformationDigest !== options.expectedTransformationDigest ||
|
||||
manifest.profile !== options.profile ||
|
||||
manifest.projectIdDigest !== sha256Text(options.projectId) ||
|
||||
manifest.sourceStageManifestDigest !== options.sourceStageManifestDigest ||
|
||||
manifest.transformationRootPathDigest !==
|
||||
sha256Text(options.authority.transformationRoot) ||
|
||||
!sameStat(
|
||||
before,
|
||||
fs.lstatSync(options.authority.transformationRoot, { bigint: true }),
|
||||
)
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConfigurationError(
|
||||
'transformation manifest authority binding is invalid',
|
||||
);
|
||||
}
|
||||
return manifest;
|
||||
}
|
||||
|
||||
export function loadStaticTransformation(options: {
|
||||
readonly authority: Readonly<TransformationAuthority>;
|
||||
readonly profile: 'edge' | 'standalone';
|
||||
readonly projectId: string;
|
||||
readonly sourceStageManifestDigest: string;
|
||||
readonly expectedTransformationDigest: string;
|
||||
}): Readonly<{
|
||||
manifest: Readonly<LocalDataDirectoryTransformationManifest>;
|
||||
prepared: Readonly<VerifiedTransformationModel>;
|
||||
}> {
|
||||
const before = assertPrivateDirectory(
|
||||
options.authority.transformationRoot,
|
||||
options.authority.uid,
|
||||
@@ -239,7 +288,7 @@ export function verifyStaticTransformation(options: {
|
||||
'transformation manifest authority binding is invalid',
|
||||
);
|
||||
}
|
||||
verifyTransformationModel({
|
||||
const prepared = verifyTransformationModel({
|
||||
modelRoot: path.join(options.authority.transformationRoot, 'model'),
|
||||
uid: options.authority.uid,
|
||||
projectId: options.projectId,
|
||||
@@ -256,5 +305,5 @@ export function verifyStaticTransformation(options: {
|
||||
'transformation root changed during verification',
|
||||
);
|
||||
}
|
||||
return manifest;
|
||||
return Object.freeze({ manifest, prepared });
|
||||
}
|
||||
|
||||
+49
-4
@@ -58,6 +58,22 @@ interface SecretImportEntry {
|
||||
readonly valueDigest: string;
|
||||
}
|
||||
|
||||
export interface VerifiedTransformationSecret extends SecretImportEntry {
|
||||
readonly plaintext: string;
|
||||
}
|
||||
|
||||
export interface VerifiedTransformationModel {
|
||||
readonly model: Readonly<{
|
||||
schema: 'qinglong/legacy-data-directory-applied-model@v1';
|
||||
activation: 'disabled';
|
||||
config: Readonly<Record<string, unknown>>;
|
||||
keyv: Readonly<Record<string, unknown>>;
|
||||
ssh: Readonly<Record<string, unknown>>;
|
||||
manualReview: Readonly<Record<string, unknown>>;
|
||||
}>;
|
||||
readonly secrets: readonly Readonly<VerifiedTransformationSecret>[];
|
||||
}
|
||||
|
||||
function exactKeys(value: object, expected: readonly string[]): boolean {
|
||||
const actual = Object.keys(value).sort();
|
||||
const canonical = [...expected].sort();
|
||||
@@ -252,7 +268,7 @@ export function verifyTransformationModel(options: {
|
||||
readonly projectId: string;
|
||||
readonly profile: 'edge' | 'standalone';
|
||||
readonly expected: Readonly<TransformationModelEvidence>;
|
||||
}): void {
|
||||
}): Readonly<VerifiedTransformationModel> {
|
||||
if (
|
||||
JSON.stringify(sortedNames(options.modelRoot)) !==
|
||||
JSON.stringify(
|
||||
@@ -270,17 +286,17 @@ export function verifyTransformationModel(options: {
|
||||
'transformation model root contains unexpected entries',
|
||||
);
|
||||
}
|
||||
assertSchemaFile(
|
||||
const config = assertSchemaFile(
|
||||
path.join(options.modelRoot, 'config.json'),
|
||||
options.uid,
|
||||
'qinglong/legacy-config-transformation@v1',
|
||||
);
|
||||
assertSchemaFile(
|
||||
const keyv = assertSchemaFile(
|
||||
path.join(options.modelRoot, 'keyv.json'),
|
||||
options.uid,
|
||||
'qinglong/legacy-keyv-transformation@v1',
|
||||
);
|
||||
assertSchemaFile(
|
||||
const ssh = assertSchemaFile(
|
||||
path.join(options.modelRoot, 'ssh.json'),
|
||||
options.uid,
|
||||
'qinglong/legacy-ssh-transformation@v1',
|
||||
@@ -329,6 +345,7 @@ export function verifyTransformationModel(options: {
|
||||
}
|
||||
const expectedFiles: string[] = [];
|
||||
const targets = new Set<string>();
|
||||
const secrets: VerifiedTransformationSecret[] = [];
|
||||
let environmentSecrets = 0;
|
||||
let sshSecrets = 0;
|
||||
for (const value of candidate.imports) {
|
||||
@@ -353,7 +370,13 @@ export function verifyTransformationModel(options: {
|
||||
if (
|
||||
(entry.kind !== 'environment' && entry.kind !== 'ssh_private_key') ||
|
||||
typeof entry.sourceName !== 'string' ||
|
||||
entry.sourceName.length < 1 ||
|
||||
Buffer.byteLength(entry.sourceName, 'utf8') > 255 ||
|
||||
/[\u0000-\u001f\u007f]/.test(entry.sourceName) ||
|
||||
typeof entry.targetName !== 'string' ||
|
||||
entry.targetName.length < 1 ||
|
||||
Buffer.byteLength(entry.targetName, 'utf8') > 128 ||
|
||||
/[\u0000-\u001f\u007f]/.test(entry.targetName) ||
|
||||
entry.expectedCurrentVersion !== 0 ||
|
||||
typeof entry.valueFile !== 'string' ||
|
||||
!SECRET_FILE_PATTERN.test(entry.valueFile) ||
|
||||
@@ -396,6 +419,17 @@ export function verifyTransformationModel(options: {
|
||||
}
|
||||
if (entry.kind === 'environment') environmentSecrets += 1;
|
||||
else sshSecrets += 1;
|
||||
secrets.push(
|
||||
Object.freeze({
|
||||
kind: entry.kind,
|
||||
sourceName: entry.sourceName,
|
||||
targetName: entry.targetName,
|
||||
expectedCurrentVersion: 0,
|
||||
valueFile: entry.valueFile,
|
||||
valueDigest: entry.valueDigest,
|
||||
plaintext: secretValue.value,
|
||||
}) as Readonly<VerifiedTransformationSecret>,
|
||||
);
|
||||
}
|
||||
expectedFiles.sort();
|
||||
if (
|
||||
@@ -427,4 +461,15 @@ export function verifyTransformationModel(options: {
|
||||
'transformation model no longer matches the manifest',
|
||||
);
|
||||
}
|
||||
return Object.freeze({
|
||||
model: Object.freeze({
|
||||
schema: 'qinglong/legacy-data-directory-applied-model@v1' as const,
|
||||
activation: 'disabled' as const,
|
||||
config: Object.freeze(config),
|
||||
keyv: Object.freeze(keyv),
|
||||
ssh: Object.freeze(ssh),
|
||||
manualReview: Object.freeze(manual),
|
||||
}),
|
||||
secrets: Object.freeze(secrets),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -6,6 +6,19 @@ const path = require('node:path');
|
||||
const { DatabaseSync } = require('node:sqlite');
|
||||
const { test } = require('node:test');
|
||||
|
||||
const {
|
||||
provisionLocalOwnerPepperKey,
|
||||
} = require('@qinglong/local-owner-console');
|
||||
const {
|
||||
LocalSecretKeyringFileProvider,
|
||||
decryptLocalSecretEnvelopeToBuffer,
|
||||
provisionLocalSecretKeyring,
|
||||
} = require('@qinglong/local-secret');
|
||||
const {
|
||||
apiCredentialSecretDigest,
|
||||
formatApiCredentialToken,
|
||||
} = require('@qinglong/runtime-core/api-credential-token');
|
||||
|
||||
const BINARY = path.join(__dirname, '../dist/lifecycle/adoptionCli.js');
|
||||
const DIRECTORY_INSPECT = 'local-data-directory.adoption.inspect';
|
||||
const DIRECTORY_STAGE = 'local-data-directory.adoption.stage';
|
||||
@@ -13,6 +26,18 @@ const DIRECTORY_VERIFY = 'local-data-directory.adoption.verify';
|
||||
const DIRECTORY_TRANSFORM = 'local-data-directory.adoption.transform';
|
||||
const DIRECTORY_TRANSFORM_VERIFY =
|
||||
'local-data-directory.adoption.transform.verify';
|
||||
const DIRECTORY_APPLY = 'local-data-directory.adoption.apply';
|
||||
const DIRECTORY_APPLY_VERIFY = 'local-data-directory.adoption.apply.verify';
|
||||
const APPLICATION_CREDENTIAL_ID = 'data-adoption-owner';
|
||||
const APPLICATION_PEPPER_KEY_ID = 'data-adoption-owner-v1';
|
||||
const APPLICATION_PEPPER = Buffer.alloc(32, 121).toString('base64url');
|
||||
const APPLICATION_CREDENTIAL_SECRET = Buffer.alloc(32, 122).toString(
|
||||
'base64url',
|
||||
);
|
||||
const APPLICATION_TOKEN = formatApiCredentialToken(
|
||||
APPLICATION_CREDENTIAL_ID,
|
||||
APPLICATION_CREDENTIAL_SECRET,
|
||||
);
|
||||
|
||||
function privateDirectory(directoryPath) {
|
||||
fs.mkdirSync(directoryPath, { recursive: true, mode: 0o700 });
|
||||
@@ -25,6 +50,15 @@ function privateFile(filePath, content) {
|
||||
fs.chmodSync(filePath, 0o600);
|
||||
}
|
||||
|
||||
function privatizeTree(root) {
|
||||
fs.chmodSync(root, 0o700);
|
||||
for (const entry of fs.readdirSync(root, { withFileTypes: true })) {
|
||||
const entryPath = path.join(root, entry.name);
|
||||
if (entry.isDirectory()) privatizeTree(entryPath);
|
||||
else fs.chmodSync(entryPath, 0o600);
|
||||
}
|
||||
}
|
||||
|
||||
function createLegacyDatabase(sourcePath) {
|
||||
const source = new DatabaseSync(sourcePath);
|
||||
source.exec(`
|
||||
@@ -371,6 +405,184 @@ function transformationOptions(value, prepared, staged, projectId) {
|
||||
};
|
||||
}
|
||||
|
||||
async function provisionApplicationAuthority(value, role = 'owner') {
|
||||
const ownerPepperKeyringDirectory = path.join(
|
||||
value.deploymentRoot,
|
||||
'owner-keys',
|
||||
);
|
||||
const credentialFilePath = path.join(
|
||||
value.deploymentRoot,
|
||||
'owner-credential.json',
|
||||
);
|
||||
const secretKeyringPath = path.join(
|
||||
value.deploymentRoot,
|
||||
'local-secret-keyring.json',
|
||||
);
|
||||
privateDirectory(ownerPepperKeyringDirectory);
|
||||
await provisionLocalSecretKeyring(secretKeyringPath);
|
||||
const pepper = provisionLocalOwnerPepperKey({
|
||||
keyringDirectory: ownerPepperKeyringDirectory,
|
||||
pepperKeyId: APPLICATION_PEPPER_KEY_ID,
|
||||
randomBytes: () => Buffer.alloc(32, 121),
|
||||
});
|
||||
const now = Date.now();
|
||||
const database = new DatabaseSync(value.targetPath);
|
||||
try {
|
||||
database
|
||||
.prepare(
|
||||
`INSERT INTO "QingLong3LocalOwnerPepperKeys" (
|
||||
"pepper_key_id", "material_digest", "backup_digest", "state",
|
||||
"version", "register_mutation_id", "activate_mutation_id",
|
||||
"registered_at_ms", "activated_at_ms"
|
||||
) VALUES (?, ?, ?, 'active', 2, ?, ?, ?, ?)`,
|
||||
)
|
||||
.run(
|
||||
APPLICATION_PEPPER_KEY_ID,
|
||||
pepper.digest,
|
||||
'e'.repeat(64),
|
||||
'd3870000-0000-4000-8000-000000000001',
|
||||
'd3870000-0000-4000-8000-000000000002',
|
||||
now - 2_000,
|
||||
now - 1_500,
|
||||
);
|
||||
database
|
||||
.prepare(
|
||||
`INSERT INTO "QingLong3LocalOwnerPepperActivations" (
|
||||
"generation", "mutation_id", "expected_generation",
|
||||
"previous_pepper_key_id", "active_pepper_key_id",
|
||||
"material_digest", "backup_digest", "activated_at_ms"
|
||||
) VALUES (1, ?, 0, NULL, ?, ?, ?, ?)`,
|
||||
)
|
||||
.run(
|
||||
'd3870000-0000-4000-8000-000000000002',
|
||||
APPLICATION_PEPPER_KEY_ID,
|
||||
pepper.digest,
|
||||
'e'.repeat(64),
|
||||
now - 1_500,
|
||||
);
|
||||
database
|
||||
.prepare(
|
||||
`INSERT INTO "QingLong3IdentitySubjects" (
|
||||
"subject_type", "subject_id", "status", "version",
|
||||
"created_at_ms", "updated_at_ms"
|
||||
) VALUES ('user', 'data-adoption-owner', 'active', 1, ?, ?)`,
|
||||
)
|
||||
.run(now - 1_000, now - 1_000);
|
||||
database
|
||||
.prepare(
|
||||
`INSERT INTO "QingLong3ApiCredentials" (
|
||||
"credential_id", "version", "state", "subject_type",
|
||||
"subject_id", "secret_digest", "created_at_ms",
|
||||
"not_before_at_ms", "expires_at_ms"
|
||||
) VALUES (?, 1, 'active', 'user', 'data-adoption-owner', ?, ?, ?, ?)`,
|
||||
)
|
||||
.run(
|
||||
APPLICATION_CREDENTIAL_ID,
|
||||
apiCredentialSecretDigest(
|
||||
APPLICATION_PEPPER,
|
||||
APPLICATION_CREDENTIAL_ID,
|
||||
APPLICATION_CREDENTIAL_SECRET,
|
||||
),
|
||||
now - 1_000,
|
||||
now - 1_000,
|
||||
now + 10 * 60 * 1_000,
|
||||
);
|
||||
database
|
||||
.prepare(
|
||||
`INSERT INTO "QingLong3ApiCredentialPepperBindings" (
|
||||
"credential_id", "credential_version", "pepper_key_id"
|
||||
) VALUES (?, 1, ?)`,
|
||||
)
|
||||
.run(APPLICATION_CREDENTIAL_ID, APPLICATION_PEPPER_KEY_ID);
|
||||
database
|
||||
.prepare(
|
||||
`INSERT INTO "QingLong3ProjectRoleBindings" (
|
||||
"project_id", "subject_type", "subject_id", "version", "state",
|
||||
"role", "mutation_id", "changed_by_type", "changed_by_id",
|
||||
"created_at_ms"
|
||||
) VALUES (
|
||||
'default', 'user', 'data-adoption-owner', 1, 'active', ?,
|
||||
'd387-owner-binding', 'user', 'data-adoption-owner', ?
|
||||
)`,
|
||||
)
|
||||
.run(role, now - 500);
|
||||
} finally {
|
||||
database.close();
|
||||
}
|
||||
fs.chmodSync(value.targetPath, 0o600);
|
||||
privateFile(
|
||||
credentialFilePath,
|
||||
`${JSON.stringify({
|
||||
schemaVersion: 1,
|
||||
kind: 'qinglong3-local-identity-credential-presentation',
|
||||
token: APPLICATION_TOKEN,
|
||||
})}\n`,
|
||||
);
|
||||
return {
|
||||
ownerPepperKeyringDirectory,
|
||||
credentialFilePath,
|
||||
secretKeyringPath,
|
||||
};
|
||||
}
|
||||
|
||||
function applicationOptions(
|
||||
value,
|
||||
prepared,
|
||||
staged,
|
||||
authority,
|
||||
transformationDigest,
|
||||
) {
|
||||
return {
|
||||
...transformationOptions(value, prepared, staged, 'default'),
|
||||
expectedTransformationDigest: transformationDigest,
|
||||
...authority,
|
||||
mutationId: 'd3870000-0000-4000-8000-000000000010',
|
||||
failureAuditEventId: 'd3870000-0000-4000-8000-000000000011',
|
||||
requestId: 'd387-data-directory-apply',
|
||||
};
|
||||
}
|
||||
|
||||
function applicationDatabaseRows(databasePath) {
|
||||
const database = new DatabaseSync(databasePath, { readOnly: true });
|
||||
try {
|
||||
return {
|
||||
adoptions: database
|
||||
.prepare(
|
||||
`SELECT mutation_id AS "mutationId", model_json AS "modelJson",
|
||||
receipt_digest AS "receiptDigest"
|
||||
FROM "QingLong3LegacyDataDirectoryAdoptions"`,
|
||||
)
|
||||
.all(),
|
||||
items: database
|
||||
.prepare(
|
||||
`SELECT secret_name AS "secretName", value_digest AS "valueDigest"
|
||||
FROM "QingLong3LegacyDataDirectoryAdoptionSecrets"
|
||||
ORDER BY ordinal`,
|
||||
)
|
||||
.all(),
|
||||
envelopes: database
|
||||
.prepare(
|
||||
`SELECT secret_name AS "secretName", version, mutation_id AS "mutationId",
|
||||
key_id AS "keyId", algorithm, nonce, ciphertext,
|
||||
auth_tag AS "authTag", created_at_ms AS "createdAtMs"
|
||||
FROM "QingLong3LocalSecretEnvelopes"
|
||||
ORDER BY secret_name`,
|
||||
)
|
||||
.all(),
|
||||
audits: database
|
||||
.prepare(
|
||||
`SELECT operation_id AS "operationId", outcome
|
||||
FROM "QingLong3SecurityAuditEvents"
|
||||
WHERE operation_id IN ('legacy-data.apply', 'secret.create')
|
||||
ORDER BY operation_id, event_id`,
|
||||
)
|
||||
.all(),
|
||||
};
|
||||
} finally {
|
||||
database.close();
|
||||
}
|
||||
}
|
||||
|
||||
test('stages only reviewed payloads behind the real SQLite activation fence', (t) => {
|
||||
const value = fixture(t);
|
||||
const prepared = prepare(value);
|
||||
@@ -927,3 +1139,285 @@ test('widened transformation commands fail closed before source access', (t) =>
|
||||
'LOCAL_DATA_DIRECTORY_ADOPTION_CONFIGURATION_INVALID',
|
||||
);
|
||||
});
|
||||
|
||||
test('atomically applies a ready transformation, reclaims plaintext and exactly replays', async (t) => {
|
||||
const value = fixture(t);
|
||||
const secrets = configureTransformationInput(value);
|
||||
const { prepared, staged } = stageForTransformation(value);
|
||||
const authority = await provisionApplicationAuthority(value);
|
||||
const transformed = run(
|
||||
value,
|
||||
'directory-transform-before-apply',
|
||||
DIRECTORY_TRANSFORM,
|
||||
transformationOptions(value, prepared, staged, 'default'),
|
||||
).result;
|
||||
const options = applicationOptions(
|
||||
value,
|
||||
prepared,
|
||||
staged,
|
||||
authority,
|
||||
transformed.evidence.transformationDigest,
|
||||
);
|
||||
const recoveryModel = path.join(value.deploymentRoot, 'recovery-model');
|
||||
fs.cpSync(path.join(value.transformationRoot, 'model'), recoveryModel, {
|
||||
recursive: true,
|
||||
preserveTimestamps: true,
|
||||
});
|
||||
privatizeTree(recoveryModel);
|
||||
|
||||
const applied = run(value, 'directory-apply', DIRECTORY_APPLY, options);
|
||||
assert.equal(applied.result.status, 'committed');
|
||||
assert.equal(applied.result.evidence.databaseStatus, 'inserted');
|
||||
assert.equal(applied.result.evidence.secretCount, 2);
|
||||
assert.equal(applied.result.evidence.environmentSecretCount, 1);
|
||||
assert.equal(applied.result.evidence.sshSecretCount, 1);
|
||||
assert.equal(applied.result.evidence.modelReclaimed, true);
|
||||
assert.equal(applied.result.evidence.plaintextFilesRemoved, true);
|
||||
assert.equal(applied.result.evidence.physicalErasureGuaranteed, false);
|
||||
for (const sensitive of [
|
||||
secrets.environmentValue,
|
||||
secrets.sshValue,
|
||||
secrets.authValue,
|
||||
secrets.environmentName,
|
||||
secrets.sshAlias,
|
||||
]) {
|
||||
assert.equal(applied.child.stdout.includes(sensitive), false);
|
||||
}
|
||||
assert.deepEqual(fs.readdirSync(value.transformationRoot).sort(), [
|
||||
'commit.json',
|
||||
'manifest.json',
|
||||
]);
|
||||
const commit = JSON.parse(
|
||||
fs.readFileSync(path.join(value.transformationRoot, 'commit.json'), 'utf8'),
|
||||
);
|
||||
assert.equal(commit.reclamation.modelRemoved, true);
|
||||
assert.equal(commit.reclamation.plaintextFilesRemoved, true);
|
||||
assert.equal(commit.reclamation.physicalErasureGuaranteed, false);
|
||||
assert.equal(commit.commitDigest, applied.result.evidence.commitDigest);
|
||||
|
||||
const rows = applicationDatabaseRows(value.targetPath);
|
||||
assert.equal(rows.adoptions.length, 1);
|
||||
assert.equal(rows.items.length, 2);
|
||||
assert.equal(rows.envelopes.length, 2);
|
||||
assert.deepEqual(
|
||||
rows.audits.map(({ operationId, outcome }) => [operationId, outcome]),
|
||||
[
|
||||
['legacy-data.apply', 'allowed'],
|
||||
['secret.create', 'allowed'],
|
||||
['secret.create', 'allowed'],
|
||||
],
|
||||
);
|
||||
const durableModel = JSON.parse(rows.adoptions[0].modelJson);
|
||||
assert.equal(durableModel.activation, 'disabled');
|
||||
assert.equal(durableModel.config.activation, 'disabled');
|
||||
assert.equal(durableModel.keyv.activation, 'disabled');
|
||||
assert.equal(durableModel.ssh.activation, 'disabled');
|
||||
|
||||
const provider = new LocalSecretKeyringFileProvider(
|
||||
authority.secretKeyringPath,
|
||||
);
|
||||
const plaintexts = [];
|
||||
for (const envelope of rows.envelopes) {
|
||||
const material = await provider.resolve(envelope.keyId);
|
||||
assert.ok(material);
|
||||
const plaintext = decryptLocalSecretEnvelopeToBuffer(
|
||||
{
|
||||
projectId: 'default',
|
||||
name: envelope.secretName,
|
||||
version: envelope.version,
|
||||
mutationId: envelope.mutationId,
|
||||
keyId: envelope.keyId,
|
||||
algorithm: envelope.algorithm,
|
||||
nonce: Buffer.from(envelope.nonce).toString('base64url'),
|
||||
ciphertext: Buffer.from(envelope.ciphertext).toString('base64url'),
|
||||
authTag: Buffer.from(envelope.authTag).toString('base64url'),
|
||||
createdAtMs: envelope.createdAtMs,
|
||||
},
|
||||
material.key,
|
||||
);
|
||||
try {
|
||||
plaintexts.push(plaintext.toString('utf8'));
|
||||
} finally {
|
||||
plaintext.fill(0);
|
||||
material.key.fill(0);
|
||||
}
|
||||
}
|
||||
assert.deepEqual(
|
||||
plaintexts.sort(),
|
||||
[secrets.environmentValue, secrets.sshValue].sort(),
|
||||
);
|
||||
|
||||
const replayed = run(
|
||||
value,
|
||||
'directory-apply-replay',
|
||||
DIRECTORY_APPLY,
|
||||
options,
|
||||
).result;
|
||||
assert.equal(replayed.evidence.databaseStatus, 'existing');
|
||||
assert.equal(
|
||||
replayed.evidence.receiptDigest,
|
||||
applied.result.evidence.receiptDigest,
|
||||
);
|
||||
assert.equal(
|
||||
replayed.evidence.commitDigest,
|
||||
applied.result.evidence.commitDigest,
|
||||
);
|
||||
const verified = run(
|
||||
value,
|
||||
'directory-apply-verify',
|
||||
DIRECTORY_APPLY_VERIFY,
|
||||
{
|
||||
...options,
|
||||
expectedReceiptDigest: applied.result.evidence.receiptDigest,
|
||||
},
|
||||
).result;
|
||||
assert.equal(verified.status, 'verified');
|
||||
assert.equal(verified.evidence.databaseStatus, 'existing');
|
||||
assert.equal(
|
||||
verified.evidence.commitDigest,
|
||||
applied.result.evidence.commitDigest,
|
||||
);
|
||||
|
||||
fs.unlinkSync(path.join(value.transformationRoot, 'commit.json'));
|
||||
fs.renameSync(
|
||||
recoveryModel,
|
||||
path.join(value.transformationRoot, '.reclaiming-model'),
|
||||
);
|
||||
privateFile(
|
||||
path.join(value.transformationRoot, '.commit-incomplete'),
|
||||
`${JSON.stringify({
|
||||
schemaVersion: 1,
|
||||
kind: 'qinglong3-legacy-data-directory-application-incomplete',
|
||||
mutationId: options.mutationId,
|
||||
transformationDigest: options.expectedTransformationDigest,
|
||||
receiptDigest: applied.result.evidence.receiptDigest,
|
||||
})}\n`,
|
||||
);
|
||||
const recovered = run(
|
||||
value,
|
||||
'directory-apply-recover-renamed-model',
|
||||
DIRECTORY_APPLY,
|
||||
options,
|
||||
).result;
|
||||
assert.equal(recovered.evidence.databaseStatus, 'existing');
|
||||
assert.equal(
|
||||
recovered.evidence.commitDigest,
|
||||
applied.result.evidence.commitDigest,
|
||||
);
|
||||
assert.deepEqual(fs.readdirSync(value.transformationRoot).sort(), [
|
||||
'commit.json',
|
||||
'manifest.json',
|
||||
]);
|
||||
});
|
||||
|
||||
test('rolls back every new Secret when one application target already exists', async (t) => {
|
||||
const value = fixture(t);
|
||||
configureTransformationInput(value);
|
||||
const { prepared, staged } = stageForTransformation(value);
|
||||
const authority = await provisionApplicationAuthority(value);
|
||||
const transformed = run(
|
||||
value,
|
||||
'directory-transform-before-conflict',
|
||||
DIRECTORY_TRANSFORM,
|
||||
transformationOptions(value, prepared, staged, 'default'),
|
||||
).result;
|
||||
const imports = JSON.parse(
|
||||
fs.readFileSync(
|
||||
path.join(value.transformationRoot, 'model', 'secret-imports.json'),
|
||||
'utf8',
|
||||
),
|
||||
).imports;
|
||||
assert.equal(imports.length, 2);
|
||||
const conflictingName = imports[1].targetName;
|
||||
const database = new DatabaseSync(value.targetPath);
|
||||
try {
|
||||
database
|
||||
.prepare(
|
||||
`INSERT INTO "QingLong3LocalSecretEnvelopes" (
|
||||
"project_id", "secret_name", "version", "mutation_id", "key_id",
|
||||
"algorithm", "nonce", "ciphertext", "auth_tag", "created_at_ms"
|
||||
) VALUES ('default', ?, 1, ?, 'preexisting-v1', 'aes-256-gcm', ?, ?, ?, ?)`,
|
||||
)
|
||||
.run(
|
||||
conflictingName,
|
||||
'd3870000-0000-4000-8000-000000000020',
|
||||
Buffer.alloc(12, 1),
|
||||
Buffer.alloc(0),
|
||||
Buffer.alloc(16, 2),
|
||||
Date.now(),
|
||||
);
|
||||
} finally {
|
||||
database.close();
|
||||
}
|
||||
|
||||
const conflict = runRaw(
|
||||
value,
|
||||
'directory-apply-conflict',
|
||||
DIRECTORY_APPLY,
|
||||
applicationOptions(
|
||||
value,
|
||||
prepared,
|
||||
staged,
|
||||
authority,
|
||||
transformed.evidence.transformationDigest,
|
||||
),
|
||||
);
|
||||
assert.equal(conflict.status, 1);
|
||||
assert.equal(conflict.stdout, '');
|
||||
assert.equal(
|
||||
JSON.parse(conflict.stderr).code,
|
||||
'LOCAL_DATA_DIRECTORY_ADOPTION_CONFLICT',
|
||||
);
|
||||
const rows = applicationDatabaseRows(value.targetPath);
|
||||
assert.equal(rows.adoptions.length, 0);
|
||||
assert.equal(rows.items.length, 0);
|
||||
assert.equal(rows.envelopes.length, 1);
|
||||
assert.equal(rows.envelopes[0].secretName, conflictingName);
|
||||
assert.equal(
|
||||
fs.existsSync(path.join(value.transformationRoot, 'model')),
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
test('rejects a non-admin application without publishing or reclaiming the model', async (t) => {
|
||||
const value = fixture(t);
|
||||
configureTransformationInput(value);
|
||||
const { prepared, staged } = stageForTransformation(value);
|
||||
const authority = await provisionApplicationAuthority(value, 'viewer');
|
||||
const transformed = run(
|
||||
value,
|
||||
'directory-transform-before-denial',
|
||||
DIRECTORY_TRANSFORM,
|
||||
transformationOptions(value, prepared, staged, 'default'),
|
||||
).result;
|
||||
const denied = runRaw(
|
||||
value,
|
||||
'directory-apply-denied',
|
||||
DIRECTORY_APPLY,
|
||||
applicationOptions(
|
||||
value,
|
||||
prepared,
|
||||
staged,
|
||||
authority,
|
||||
transformed.evidence.transformationDigest,
|
||||
),
|
||||
);
|
||||
assert.equal(denied.status, 1);
|
||||
assert.equal(denied.stdout, '');
|
||||
assert.equal(
|
||||
JSON.parse(denied.stderr).code,
|
||||
'LOCAL_DATA_DIRECTORY_ADOPTION_APPLICATION_FORBIDDEN',
|
||||
);
|
||||
const rows = applicationDatabaseRows(value.targetPath);
|
||||
assert.equal(rows.adoptions.length, 0);
|
||||
assert.equal(rows.items.length, 0);
|
||||
assert.equal(rows.envelopes.length, 0);
|
||||
assert.deepEqual(
|
||||
rows.audits.map(({ operationId, outcome }) => [operationId, outcome]),
|
||||
[['legacy-data.apply', 'denied']],
|
||||
);
|
||||
assert.equal(
|
||||
fs.existsSync(path.join(value.transformationRoot, 'model')),
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -475,9 +475,9 @@ function composeDockerHarness(
|
||||
'/opt/qinglong/node_modules/@qinglong/local-application/dist/cli.js',
|
||||
],
|
||||
Labels: {
|
||||
'io.qinglong.local.sqlite-contract-min': '49',
|
||||
'io.qinglong.local.sqlite-contract-max': '49',
|
||||
'io.qinglong.local.sqlite-write-contract': '49',
|
||||
'io.qinglong.local.sqlite-contract-min': '50',
|
||||
'io.qinglong.local.sqlite-contract-max': '50',
|
||||
'io.qinglong.local.sqlite-write-contract': '50',
|
||||
'io.qinglong.local.application-config': '2',
|
||||
'io.qinglong.local.compose-selection': '1',
|
||||
'io.qinglong.ai': 'excluded',
|
||||
@@ -1172,9 +1172,9 @@ test('preflights exact local image, Compose merge and SQLite capability', async
|
||||
'/opt/qinglong/node_modules/@qinglong/local-application/dist/cli.js',
|
||||
],
|
||||
Labels: {
|
||||
'io.qinglong.local.sqlite-contract-min': '49',
|
||||
'io.qinglong.local.sqlite-contract-max': '49',
|
||||
'io.qinglong.local.sqlite-write-contract': '49',
|
||||
'io.qinglong.local.sqlite-contract-min': '50',
|
||||
'io.qinglong.local.sqlite-contract-max': '50',
|
||||
'io.qinglong.local.sqlite-write-contract': '50',
|
||||
'io.qinglong.local.application-config': '2',
|
||||
'io.qinglong.local.compose-selection': '1',
|
||||
'io.qinglong.ai': 'excluded',
|
||||
@@ -1226,7 +1226,7 @@ test('preflights exact local image, Compose merge and SQLite capability', async
|
||||
assert.equal(result.status, 'ready');
|
||||
assert.equal(result.generation, 1);
|
||||
assert.equal(result.profile, 'edge');
|
||||
assert.equal(result.sqlite.contractVersion, 49);
|
||||
assert.equal(result.sqlite.contractVersion, 50);
|
||||
assert.equal(result.image.architecture, 'arm64');
|
||||
assert.equal(calls.length, 2);
|
||||
assert.deepEqual(calls[0].slice(0, 2), ['image', 'inspect']);
|
||||
@@ -1326,8 +1326,8 @@ test('applies one Compose generation and exactly replays its health receipt', as
|
||||
assert.equal(mode(receiptPath), 0o600);
|
||||
const receipt = JSON.parse(fs.readFileSync(receiptPath, 'utf8'));
|
||||
assert.deepEqual(receipt.sqlite, {
|
||||
contractVersion: 49,
|
||||
writeContractVersion: 49,
|
||||
contractVersion: 50,
|
||||
writeContractVersion: 50,
|
||||
writeObservation: 'unchanged',
|
||||
backup: null,
|
||||
});
|
||||
@@ -1628,8 +1628,8 @@ test('rolls a failed Compose candidate forward to a healthy prior digest', async
|
||||
`${command.request.rolloutId}.sqlite`,
|
||||
);
|
||||
assert.equal(mode(backupPath), 0o600);
|
||||
assert.equal(receipt.sqlite.contractVersion, 49);
|
||||
assert.equal(receipt.sqlite.writeContractVersion, 49);
|
||||
assert.equal(receipt.sqlite.contractVersion, 50);
|
||||
assert.equal(receipt.sqlite.writeContractVersion, 50);
|
||||
assert.equal(receipt.sqlite.writeObservation, 'changed');
|
||||
assert.match(receipt.sqlite.backup.sha256, /^[0-9a-f]{64}$/);
|
||||
assert.equal(receipt.sqlite.backup.bytes > 0, true);
|
||||
|
||||
@@ -34,8 +34,8 @@ test('inspects the exact fresh Profile schema without exposing its path', async
|
||||
assert.equal(result.status, 'ready');
|
||||
assert.equal(result.profile, 'edge');
|
||||
assert.equal(result.storage.contractName, 'local-control-core');
|
||||
assert.equal(result.storage.contractVersion, 49);
|
||||
assert.equal(result.storage.migrationCount, 98);
|
||||
assert.equal(result.storage.contractVersion, 50);
|
||||
assert.equal(result.storage.migrationCount, 100);
|
||||
assert.equal(result.storage.journalMode, 'delete');
|
||||
assert.equal(JSON.stringify(result).includes(state.directory), false);
|
||||
});
|
||||
|
||||
@@ -75,6 +75,11 @@
|
||||
"require": "./dist/adoption/legacyAdoptionDatabase.js",
|
||||
"default": "./dist/adoption/legacyAdoptionDatabase.js"
|
||||
},
|
||||
"./data-directory-adoption": {
|
||||
"types": "./dist/adoption/data-directory/dataDirectoryAdoptionDatabase.d.ts",
|
||||
"require": "./dist/adoption/data-directory/dataDirectoryAdoptionDatabase.js",
|
||||
"default": "./dist/adoption/data-directory/dataDirectoryAdoptionDatabase.js"
|
||||
},
|
||||
"./plugin-package-install": {
|
||||
"types": "./dist/plugin-package/pluginPackageInstallRepository.d.ts",
|
||||
"require": "./dist/plugin-package/pluginPackageInstallRepository.js",
|
||||
|
||||
+994
@@ -0,0 +1,994 @@
|
||||
import { createHash } from 'node:crypto';
|
||||
import type { DatabaseSync } from 'node:sqlite';
|
||||
|
||||
import {
|
||||
createLocalSecretRef,
|
||||
normalizeLocalSecretEnvelope,
|
||||
type LocalSecretEnvelope,
|
||||
} from '@qinglong/runtime-core/local-secret';
|
||||
import {
|
||||
normalizeProjectPolicySubject,
|
||||
type ProjectPolicyRepository,
|
||||
} from '@qinglong/runtime-core/project-policy';
|
||||
import type {
|
||||
SecurityPolicyFence,
|
||||
SecuritySubject,
|
||||
} from '@qinglong/runtime-core/security';
|
||||
import {
|
||||
normalizeSecurityAuditRecord,
|
||||
type SecurityAuditRecord,
|
||||
type SecurityAuditSink,
|
||||
} from '@qinglong/runtime-core/security-audit';
|
||||
|
||||
import { LocalSqliteOperationAuthority } from '../../authority/operationAuthority';
|
||||
import {
|
||||
auditLocalSqliteReadiness,
|
||||
type LocalSqliteReadinessEvidence,
|
||||
} from '../../readiness/readiness';
|
||||
import { LocalSqliteSecurityAuthorityStore } from '../../security/securityAuthorityStore';
|
||||
import {
|
||||
assertLocalSqliteOptions,
|
||||
assertLocalSqlitePathBoundary,
|
||||
openLocalSqliteClient,
|
||||
type LocalSqliteDatabaseOptions,
|
||||
type LocalSqliteProfile,
|
||||
} from '../../storage/config';
|
||||
|
||||
const DIGEST_PATTERN = /^[0-9a-f]{64}$/;
|
||||
const UUID_V4_PATTERN =
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/;
|
||||
const VALUE_FILE_PATTERN = /^secret-values\/[0-9a-f]{64}\.json$/;
|
||||
|
||||
export interface LocalDataDirectoryAppliedModel {
|
||||
readonly schema: 'qinglong/legacy-data-directory-applied-model@v1';
|
||||
readonly activation: 'disabled';
|
||||
readonly config: Readonly<Record<string, unknown>>;
|
||||
readonly keyv: Readonly<Record<string, unknown>>;
|
||||
readonly ssh: Readonly<Record<string, unknown>>;
|
||||
readonly manualReview: Readonly<Record<string, unknown>>;
|
||||
}
|
||||
|
||||
export interface LocalDataDirectoryAdoptionSecretPublication {
|
||||
readonly ordinal: number;
|
||||
readonly kind: 'environment' | 'ssh_private_key';
|
||||
readonly sourceNameDigest: string;
|
||||
readonly valueFile: string;
|
||||
readonly valueDigest: string;
|
||||
readonly envelope: Readonly<LocalSecretEnvelope>;
|
||||
readonly secretRef: string;
|
||||
readonly itemDigest: string;
|
||||
readonly audit: Readonly<SecurityAuditRecord>;
|
||||
}
|
||||
|
||||
export interface LocalDataDirectoryAdoptionSecretRecord {
|
||||
readonly ordinal: number;
|
||||
readonly kind: 'environment' | 'ssh_private_key';
|
||||
readonly sourceNameDigest: string;
|
||||
readonly secretName: string;
|
||||
readonly secretVersion: 1;
|
||||
readonly secretMutationId: string;
|
||||
readonly valueFile: string;
|
||||
readonly valueDigest: string;
|
||||
readonly secretRef: string;
|
||||
readonly itemDigest: string;
|
||||
}
|
||||
|
||||
export interface LocalDataDirectoryAdoptionReceiptPayload {
|
||||
readonly schema: 'qinglong/legacy-data-directory-adoption-receipt@v1';
|
||||
readonly mutationId: string;
|
||||
readonly projectId: string;
|
||||
readonly profile: LocalSqliteProfile;
|
||||
readonly sourceStageManifestDigest: string;
|
||||
readonly transformationDigest: string;
|
||||
readonly modelDigest: string;
|
||||
readonly secretCount: number;
|
||||
readonly environmentSecretCount: number;
|
||||
readonly sshSecretCount: number;
|
||||
readonly items: readonly Readonly<{
|
||||
ordinal: number;
|
||||
kind: LocalDataDirectoryAdoptionSecretRecord['kind'];
|
||||
sourceNameDigest: string;
|
||||
secretRef: string;
|
||||
valueFile: string;
|
||||
valueDigest: string;
|
||||
itemDigest: string;
|
||||
}>[];
|
||||
readonly publicationDigest: string;
|
||||
readonly auditEventId: string;
|
||||
readonly committedAtMs: number;
|
||||
}
|
||||
|
||||
export interface LocalDataDirectoryAdoptionReceipt
|
||||
extends LocalDataDirectoryAdoptionReceiptPayload {
|
||||
readonly receiptDigest: string;
|
||||
}
|
||||
|
||||
export interface LocalDataDirectoryAdoptionRecord {
|
||||
readonly mutationId: string;
|
||||
readonly projectId: string;
|
||||
readonly profile: LocalSqliteProfile;
|
||||
readonly sourceStageManifestDigest: string;
|
||||
readonly transformationDigest: string;
|
||||
readonly modelDigest: string;
|
||||
readonly model: Readonly<LocalDataDirectoryAppliedModel>;
|
||||
readonly publicationDigest: string;
|
||||
readonly auditEventId: string;
|
||||
readonly committedAtMs: number;
|
||||
readonly receiptDigest: string;
|
||||
readonly receipt: Readonly<LocalDataDirectoryAdoptionReceipt>;
|
||||
readonly secrets: readonly Readonly<LocalDataDirectoryAdoptionSecretRecord>[];
|
||||
}
|
||||
|
||||
export interface PublishLocalDataDirectoryAdoptionCommand {
|
||||
readonly mutationId: string;
|
||||
readonly projectId: string;
|
||||
readonly profile: LocalSqliteProfile;
|
||||
readonly sourceStageManifestDigest: string;
|
||||
readonly transformationDigest: string;
|
||||
readonly modelDigest: string;
|
||||
readonly model: Readonly<LocalDataDirectoryAppliedModel>;
|
||||
readonly subject: Readonly<SecuritySubject>;
|
||||
readonly fence: Readonly<SecurityPolicyFence>;
|
||||
readonly audit: Readonly<SecurityAuditRecord>;
|
||||
readonly secrets: readonly Readonly<LocalDataDirectoryAdoptionSecretPublication>[];
|
||||
readonly receipt: Readonly<LocalDataDirectoryAdoptionReceipt>;
|
||||
readonly confirmExternalAuthority: () => void | Promise<void>;
|
||||
}
|
||||
|
||||
export interface PublishLocalDataDirectoryAdoptionResult {
|
||||
readonly status: 'inserted' | 'existing';
|
||||
readonly adoption: Readonly<LocalDataDirectoryAdoptionRecord>;
|
||||
}
|
||||
|
||||
export class LocalDataDirectoryAdoptionConflictError extends Error {
|
||||
readonly code = 'LOCAL_DATA_DIRECTORY_ADOPTION_CONFLICT';
|
||||
|
||||
constructor() {
|
||||
super('Local data directory adoption conflicts with durable state');
|
||||
this.name = 'LocalDataDirectoryAdoptionConflictError';
|
||||
}
|
||||
}
|
||||
|
||||
export class LocalDataDirectoryAdoptionAuthorizationFenceConflictError extends Error {
|
||||
readonly code = 'LOCAL_DATA_DIRECTORY_ADOPTION_AUTHORIZATION_FENCE_CONFLICT';
|
||||
|
||||
constructor() {
|
||||
super('Local data directory adoption authorization fence changed');
|
||||
this.name = 'LocalDataDirectoryAdoptionAuthorizationFenceConflictError';
|
||||
}
|
||||
}
|
||||
|
||||
export class LocalDataDirectoryAdoptionUnavailableError extends Error {
|
||||
readonly code = 'LOCAL_DATA_DIRECTORY_ADOPTION_UNAVAILABLE';
|
||||
|
||||
constructor(readonly cause?: unknown) {
|
||||
super('Local data directory adoption storage is unavailable');
|
||||
this.name = 'LocalDataDirectoryAdoptionUnavailableError';
|
||||
}
|
||||
}
|
||||
|
||||
type Row = Record<string, unknown>;
|
||||
|
||||
function sha256(domain: string, value: string): string {
|
||||
return createHash('sha256')
|
||||
.update(domain, 'utf8')
|
||||
.update(value, 'utf8')
|
||||
.digest('hex');
|
||||
}
|
||||
|
||||
function exactKeys(value: object, expected: readonly string[]): boolean {
|
||||
const actual = Object.keys(value).sort();
|
||||
const canonical = [...expected].sort();
|
||||
return (
|
||||
actual.length === canonical.length &&
|
||||
actual.every((key, index) => key === canonical[index])
|
||||
);
|
||||
}
|
||||
|
||||
function safeInteger(value: unknown, minimum = 0): value is number {
|
||||
return Number.isSafeInteger(value) && (value as number) >= minimum;
|
||||
}
|
||||
|
||||
function text(row: Row, key: string): string {
|
||||
const value = row[key];
|
||||
if (typeof value !== 'string') {
|
||||
throw new LocalDataDirectoryAdoptionUnavailableError();
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function integer(row: Row, key: string): number {
|
||||
const value = row[key];
|
||||
if (!Number.isSafeInteger(value)) {
|
||||
throw new LocalDataDirectoryAdoptionUnavailableError();
|
||||
}
|
||||
return value as number;
|
||||
}
|
||||
|
||||
function json(row: Row, key: string): unknown {
|
||||
try {
|
||||
return JSON.parse(text(row, key));
|
||||
} catch (error) {
|
||||
throw new LocalDataDirectoryAdoptionUnavailableError(error);
|
||||
}
|
||||
}
|
||||
|
||||
function assertModel(
|
||||
value: unknown,
|
||||
): asserts value is LocalDataDirectoryAppliedModel {
|
||||
if (
|
||||
!value ||
|
||||
typeof value !== 'object' ||
|
||||
Array.isArray(value) ||
|
||||
!exactKeys(value, [
|
||||
'activation',
|
||||
'config',
|
||||
'keyv',
|
||||
'manualReview',
|
||||
'schema',
|
||||
'ssh',
|
||||
])
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConflictError();
|
||||
}
|
||||
const model = value as Record<string, unknown>;
|
||||
const entries = [model.config, model.keyv, model.ssh, model.manualReview];
|
||||
if (
|
||||
model.schema !== 'qinglong/legacy-data-directory-applied-model@v1' ||
|
||||
model.activation !== 'disabled' ||
|
||||
entries.some(
|
||||
(entry) => !entry || typeof entry !== 'object' || Array.isArray(entry),
|
||||
) ||
|
||||
(model.config as Row).schema !==
|
||||
'qinglong/legacy-config-transformation@v1' ||
|
||||
(model.config as Row).activation !== 'disabled' ||
|
||||
(model.keyv as Row).schema !== 'qinglong/legacy-keyv-transformation@v1' ||
|
||||
(model.keyv as Row).activation !== 'disabled' ||
|
||||
(model.ssh as Row).schema !== 'qinglong/legacy-ssh-transformation@v1' ||
|
||||
(model.ssh as Row).activation !== 'disabled' ||
|
||||
(model.manualReview as Row).schema !==
|
||||
'qinglong/legacy-data-directory-manual-review@v1' ||
|
||||
(model.manualReview as Row).required !== false ||
|
||||
(model.manualReview as Row).activation !== 'disabled' ||
|
||||
Buffer.byteLength(JSON.stringify(value), 'utf8') > 1024 * 1024
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConflictError();
|
||||
}
|
||||
}
|
||||
|
||||
function itemSemantic(
|
||||
item: Omit<LocalDataDirectoryAdoptionSecretRecord, 'itemDigest'>,
|
||||
): string {
|
||||
return JSON.stringify(item);
|
||||
}
|
||||
|
||||
export function createLocalDataDirectorySourceNameDigest(
|
||||
kind: LocalDataDirectoryAdoptionSecretRecord['kind'],
|
||||
sourceName: string,
|
||||
): string {
|
||||
if (
|
||||
(kind !== 'environment' && kind !== 'ssh_private_key') ||
|
||||
typeof sourceName !== 'string' ||
|
||||
sourceName.length < 1 ||
|
||||
sourceName.includes('\0')
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConflictError();
|
||||
}
|
||||
return createHash('sha256')
|
||||
.update('qinglong3.legacy-data-directory-source-name.v1\0')
|
||||
.update(kind)
|
||||
.update('\0')
|
||||
.update(sourceName)
|
||||
.digest('hex');
|
||||
}
|
||||
|
||||
export function createLocalDataDirectoryAdoptionSecretItem(options: {
|
||||
readonly projectId: string;
|
||||
readonly ordinal: number;
|
||||
readonly kind: LocalDataDirectoryAdoptionSecretRecord['kind'];
|
||||
readonly sourceNameDigest: string;
|
||||
readonly secretName: string;
|
||||
readonly secretMutationId: string;
|
||||
readonly valueFile: string;
|
||||
readonly valueDigest: string;
|
||||
}): Readonly<LocalDataDirectoryAdoptionSecretRecord> {
|
||||
return createSecretRecord(options);
|
||||
}
|
||||
|
||||
function createSecretRecord(options: {
|
||||
readonly projectId: string;
|
||||
readonly ordinal: number;
|
||||
readonly kind: LocalDataDirectoryAdoptionSecretRecord['kind'];
|
||||
readonly sourceNameDigest: string;
|
||||
readonly secretName: string;
|
||||
readonly secretMutationId: string;
|
||||
readonly valueFile: string;
|
||||
readonly valueDigest: string;
|
||||
}): Readonly<LocalDataDirectoryAdoptionSecretRecord> {
|
||||
const secretRef = createLocalSecretRef({
|
||||
projectId: options.projectId,
|
||||
name: options.secretName,
|
||||
version: 1,
|
||||
});
|
||||
const semantic = Object.freeze({
|
||||
ordinal: options.ordinal,
|
||||
kind: options.kind,
|
||||
sourceNameDigest: options.sourceNameDigest,
|
||||
secretName: options.secretName,
|
||||
secretVersion: 1 as const,
|
||||
secretMutationId: options.secretMutationId,
|
||||
valueFile: options.valueFile,
|
||||
valueDigest: options.valueDigest,
|
||||
secretRef,
|
||||
});
|
||||
return Object.freeze({
|
||||
...semantic,
|
||||
itemDigest: sha256(
|
||||
'qinglong3.legacy-data-directory-adoption-secret-item.v1\0',
|
||||
itemSemantic(semantic),
|
||||
),
|
||||
});
|
||||
}
|
||||
|
||||
function publicationDigest(options: {
|
||||
readonly mutationId: string;
|
||||
readonly projectId: string;
|
||||
readonly sourceStageManifestDigest: string;
|
||||
readonly transformationDigest: string;
|
||||
readonly modelDigest: string;
|
||||
readonly secrets: readonly Readonly<LocalDataDirectoryAdoptionSecretRecord>[];
|
||||
}): string {
|
||||
const hash = createHash('sha256')
|
||||
.update('qinglong3.legacy-data-directory-adoption-publication.v1\0')
|
||||
.update(options.mutationId)
|
||||
.update('\0')
|
||||
.update(options.projectId)
|
||||
.update('\0')
|
||||
.update(options.sourceStageManifestDigest)
|
||||
.update('\0')
|
||||
.update(options.transformationDigest)
|
||||
.update('\0')
|
||||
.update(options.modelDigest);
|
||||
for (const item of options.secrets) hash.update('\0').update(item.itemDigest);
|
||||
return hash.digest('hex');
|
||||
}
|
||||
|
||||
export function createLocalDataDirectoryAdoptionReceipt(options: {
|
||||
readonly mutationId: string;
|
||||
readonly projectId: string;
|
||||
readonly profile: LocalSqliteProfile;
|
||||
readonly sourceStageManifestDigest: string;
|
||||
readonly transformationDigest: string;
|
||||
readonly modelDigest: string;
|
||||
readonly secrets: readonly Readonly<LocalDataDirectoryAdoptionSecretRecord>[];
|
||||
readonly committedAtMs: number;
|
||||
}): Readonly<LocalDataDirectoryAdoptionReceipt> {
|
||||
const environmentSecretCount = options.secrets.filter(
|
||||
({ kind }) => kind === 'environment',
|
||||
).length;
|
||||
const sshSecretCount = options.secrets.length - environmentSecretCount;
|
||||
const publication = publicationDigest(options);
|
||||
const payload: LocalDataDirectoryAdoptionReceiptPayload = {
|
||||
schema: 'qinglong/legacy-data-directory-adoption-receipt@v1',
|
||||
mutationId: options.mutationId,
|
||||
projectId: options.projectId,
|
||||
profile: options.profile,
|
||||
sourceStageManifestDigest: options.sourceStageManifestDigest,
|
||||
transformationDigest: options.transformationDigest,
|
||||
modelDigest: options.modelDigest,
|
||||
secretCount: options.secrets.length,
|
||||
environmentSecretCount,
|
||||
sshSecretCount,
|
||||
items: Object.freeze(
|
||||
options.secrets.map(
|
||||
({
|
||||
ordinal,
|
||||
kind,
|
||||
sourceNameDigest,
|
||||
secretRef,
|
||||
valueFile,
|
||||
valueDigest,
|
||||
itemDigest,
|
||||
}) =>
|
||||
Object.freeze({
|
||||
ordinal,
|
||||
kind,
|
||||
sourceNameDigest,
|
||||
secretRef,
|
||||
valueFile,
|
||||
valueDigest,
|
||||
itemDigest,
|
||||
}),
|
||||
),
|
||||
),
|
||||
publicationDigest: publication,
|
||||
auditEventId: options.mutationId,
|
||||
committedAtMs: options.committedAtMs,
|
||||
};
|
||||
return Object.freeze({
|
||||
...payload,
|
||||
receiptDigest: sha256(
|
||||
'qinglong3.legacy-data-directory-adoption-receipt.v1\0',
|
||||
JSON.stringify(payload),
|
||||
),
|
||||
});
|
||||
}
|
||||
|
||||
function sameJson(left: unknown, right: unknown): boolean {
|
||||
return JSON.stringify(left) === JSON.stringify(right);
|
||||
}
|
||||
|
||||
function parseRecord(
|
||||
client: DatabaseSync,
|
||||
row: Row,
|
||||
): LocalDataDirectoryAdoptionRecord {
|
||||
const profile = text(row, 'profile');
|
||||
if (profile !== 'edge' && profile !== 'standalone') {
|
||||
throw new LocalDataDirectoryAdoptionUnavailableError();
|
||||
}
|
||||
const mutationId = text(row, 'mutationId');
|
||||
const projectId = text(row, 'projectId');
|
||||
const model = json(row, 'modelJson');
|
||||
assertModel(model);
|
||||
const itemRows = client
|
||||
.prepare(
|
||||
`SELECT item."ordinal" AS "ordinal", item."kind" AS "kind",
|
||||
item."source_name_digest" AS "sourceNameDigest",
|
||||
item."secret_name" AS "secretName",
|
||||
item."secret_version" AS "secretVersion",
|
||||
item."secret_mutation_id" AS "secretMutationId",
|
||||
item."value_file" AS "valueFile",
|
||||
item."value_digest" AS "valueDigest",
|
||||
item."secret_ref" AS "secretRef",
|
||||
item."item_digest" AS "itemDigest"
|
||||
FROM "QingLong3LegacyDataDirectoryAdoptionSecrets" AS item
|
||||
JOIN "QingLong3LocalSecretEnvelopes" AS secret
|
||||
ON secret."project_id" = item."project_id"
|
||||
AND secret."secret_name" = item."secret_name"
|
||||
AND secret."version" = item."secret_version"
|
||||
AND secret."mutation_id" = item."secret_mutation_id"
|
||||
JOIN "QingLong3SecurityAuditEvents" AS audit
|
||||
ON audit."event_id" = item."secret_mutation_id"
|
||||
AND audit."project_id" = item."project_id"
|
||||
AND audit."operation_id" = 'secret.create'
|
||||
AND audit."outcome" = 'allowed'
|
||||
WHERE item."adoption_mutation_id" = ?
|
||||
ORDER BY item."ordinal"`,
|
||||
)
|
||||
.all(mutationId) as Row[];
|
||||
const secrets = Object.freeze(
|
||||
itemRows.map((item, index) => {
|
||||
const kind = text(item, 'kind');
|
||||
const candidate = createSecretRecord({
|
||||
projectId,
|
||||
ordinal: integer(item, 'ordinal'),
|
||||
kind:
|
||||
kind === 'environment' || kind === 'ssh_private_key'
|
||||
? kind
|
||||
: (() => {
|
||||
throw new LocalDataDirectoryAdoptionUnavailableError();
|
||||
})(),
|
||||
sourceNameDigest: text(item, 'sourceNameDigest'),
|
||||
secretName: text(item, 'secretName'),
|
||||
secretMutationId: text(item, 'secretMutationId'),
|
||||
valueFile: text(item, 'valueFile'),
|
||||
valueDigest: text(item, 'valueDigest'),
|
||||
});
|
||||
if (
|
||||
candidate.ordinal !== index + 1 ||
|
||||
candidate.secretVersion !== integer(item, 'secretVersion') ||
|
||||
candidate.secretRef !== text(item, 'secretRef') ||
|
||||
candidate.itemDigest !== text(item, 'itemDigest')
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionUnavailableError();
|
||||
}
|
||||
return candidate;
|
||||
}),
|
||||
);
|
||||
const sourceStageManifestDigest = text(row, 'sourceStageManifestDigest');
|
||||
const transformationDigest = text(row, 'transformationDigest');
|
||||
const modelDigest = text(row, 'modelDigest');
|
||||
const committedAtMs = integer(row, 'committedAtMs');
|
||||
const receipt = createLocalDataDirectoryAdoptionReceipt({
|
||||
mutationId,
|
||||
projectId,
|
||||
profile,
|
||||
sourceStageManifestDigest,
|
||||
transformationDigest,
|
||||
modelDigest,
|
||||
secrets,
|
||||
committedAtMs,
|
||||
});
|
||||
const storedReceipt = json(row, 'receiptJson');
|
||||
if (
|
||||
secrets.length !== integer(row, 'secretCount') ||
|
||||
receipt.environmentSecretCount !== integer(row, 'environmentSecretCount') ||
|
||||
receipt.sshSecretCount !== integer(row, 'sshSecretCount') ||
|
||||
receipt.publicationDigest !== text(row, 'publicationDigest') ||
|
||||
receipt.auditEventId !== text(row, 'auditEventId') ||
|
||||
receipt.receiptDigest !== text(row, 'receiptDigest') ||
|
||||
!sameJson(receipt, storedReceipt)
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionUnavailableError();
|
||||
}
|
||||
return Object.freeze({
|
||||
mutationId,
|
||||
projectId,
|
||||
profile,
|
||||
sourceStageManifestDigest,
|
||||
transformationDigest,
|
||||
modelDigest,
|
||||
model,
|
||||
publicationDigest: receipt.publicationDigest,
|
||||
auditEventId: receipt.auditEventId,
|
||||
committedAtMs,
|
||||
receiptDigest: receipt.receiptDigest,
|
||||
receipt,
|
||||
secrets,
|
||||
});
|
||||
}
|
||||
|
||||
const RECORD_SELECT = `
|
||||
adoption."mutation_id" AS "mutationId",
|
||||
adoption."project_id" AS "projectId",
|
||||
adoption."profile" AS "profile",
|
||||
adoption."source_stage_manifest_digest" AS "sourceStageManifestDigest",
|
||||
adoption."transformation_digest" AS "transformationDigest",
|
||||
adoption."model_digest" AS "modelDigest",
|
||||
adoption."secret_count" AS "secretCount",
|
||||
adoption."environment_secret_count" AS "environmentSecretCount",
|
||||
adoption."ssh_secret_count" AS "sshSecretCount",
|
||||
adoption."model_json" AS "modelJson",
|
||||
adoption."publication_digest" AS "publicationDigest",
|
||||
adoption."audit_event_id" AS "auditEventId",
|
||||
adoption."committed_at_ms" AS "committedAtMs",
|
||||
adoption."receipt_digest" AS "receiptDigest",
|
||||
adoption."receipt_json" AS "receiptJson"`;
|
||||
|
||||
function findRecord(
|
||||
client: DatabaseSync,
|
||||
mutationId: string,
|
||||
transformationDigest?: string,
|
||||
): LocalDataDirectoryAdoptionRecord | null {
|
||||
const rows = transformationDigest
|
||||
? (client
|
||||
.prepare(
|
||||
`SELECT ${RECORD_SELECT}
|
||||
FROM "QingLong3LegacyDataDirectoryAdoptions" AS adoption
|
||||
JOIN "QingLong3SecurityAuditEvents" AS audit
|
||||
ON audit."event_id" = adoption."audit_event_id"
|
||||
AND audit."project_id" = adoption."project_id"
|
||||
AND audit."operation_id" = 'legacy-data.apply'
|
||||
AND audit."outcome" = 'allowed'
|
||||
WHERE adoption."mutation_id" = ?
|
||||
OR adoption."transformation_digest" = ?
|
||||
LIMIT 2`,
|
||||
)
|
||||
.all(mutationId, transformationDigest) as Row[])
|
||||
: (client
|
||||
.prepare(
|
||||
`SELECT ${RECORD_SELECT}
|
||||
FROM "QingLong3LegacyDataDirectoryAdoptions" AS adoption
|
||||
JOIN "QingLong3SecurityAuditEvents" AS audit
|
||||
ON audit."event_id" = adoption."audit_event_id"
|
||||
AND audit."project_id" = adoption."project_id"
|
||||
AND audit."operation_id" = 'legacy-data.apply'
|
||||
AND audit."outcome" = 'allowed'
|
||||
WHERE adoption."mutation_id" = ?
|
||||
LIMIT 2`,
|
||||
)
|
||||
.all(mutationId) as Row[]);
|
||||
if (rows.length === 0) return null;
|
||||
if (rows.length !== 1) throw new LocalDataDirectoryAdoptionConflictError();
|
||||
return parseRecord(client, rows[0]!);
|
||||
}
|
||||
|
||||
function insertAudit(client: DatabaseSync, audit: SecurityAuditRecord): void {
|
||||
client
|
||||
.prepare(
|
||||
`INSERT INTO "QingLong3SecurityAuditEvents" (
|
||||
"event_id", "request_id", "operation_id", "project_id",
|
||||
"subject_type", "subject_id", "authentication_id", "outcome",
|
||||
"reasons_json", "fence_project_version", "fence_binding_version",
|
||||
"occurred_at_ms"
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
)
|
||||
.run(
|
||||
audit.eventId,
|
||||
audit.requestId,
|
||||
audit.operationId,
|
||||
audit.projectId,
|
||||
audit.subject?.type ?? null,
|
||||
audit.subject?.id ?? null,
|
||||
audit.authenticationId,
|
||||
audit.outcome,
|
||||
JSON.stringify(audit.reasons),
|
||||
audit.fence?.projectVersion ?? null,
|
||||
audit.fence?.bindingVersion ?? null,
|
||||
audit.occurredAtMs,
|
||||
);
|
||||
}
|
||||
|
||||
function assertCommand(command: PublishLocalDataDirectoryAdoptionCommand): {
|
||||
readonly subject: Readonly<SecuritySubject>;
|
||||
readonly audit: Readonly<SecurityAuditRecord>;
|
||||
readonly secrets: readonly Readonly<{
|
||||
publication: LocalDataDirectoryAdoptionSecretPublication;
|
||||
record: LocalDataDirectoryAdoptionSecretRecord;
|
||||
envelope: LocalSecretEnvelope;
|
||||
audit: SecurityAuditRecord;
|
||||
}>[];
|
||||
} {
|
||||
if (
|
||||
!command ||
|
||||
typeof command !== 'object' ||
|
||||
!UUID_V4_PATTERN.test(command.mutationId) ||
|
||||
(command.profile !== 'edge' && command.profile !== 'standalone') ||
|
||||
![
|
||||
command.sourceStageManifestDigest,
|
||||
command.transformationDigest,
|
||||
command.modelDigest,
|
||||
].every((value) => DIGEST_PATTERN.test(value)) ||
|
||||
!Array.isArray(command.secrets) ||
|
||||
command.secrets.length > (command.profile === 'edge' ? 128 : 512) ||
|
||||
typeof command.confirmExternalAuthority !== 'function'
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConflictError();
|
||||
}
|
||||
assertModel(command.model);
|
||||
const subject = normalizeProjectPolicySubject(command.subject);
|
||||
const audit = normalizeSecurityAuditRecord(command.audit);
|
||||
if (
|
||||
!command.fence ||
|
||||
!safeInteger(command.fence.projectVersion, 1) ||
|
||||
!safeInteger(command.fence.bindingVersion, 1) ||
|
||||
audit.eventId !== command.mutationId ||
|
||||
audit.operationId !== 'legacy-data.apply' ||
|
||||
audit.projectId !== command.projectId ||
|
||||
audit.subject?.type !== subject.type ||
|
||||
audit.subject.id !== subject.id ||
|
||||
audit.outcome !== 'allowed' ||
|
||||
audit.fence?.projectVersion !== command.fence.projectVersion ||
|
||||
audit.fence.bindingVersion !== command.fence.bindingVersion ||
|
||||
audit.occurredAtMs !== command.receipt.committedAtMs
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConflictError();
|
||||
}
|
||||
const names = new Set<string>();
|
||||
const mutations = new Set<string>([command.mutationId]);
|
||||
const secrets = command.secrets.map((publication, index) => {
|
||||
const envelope = normalizeLocalSecretEnvelope(publication.envelope);
|
||||
const itemAudit = normalizeSecurityAuditRecord(publication.audit);
|
||||
const record = createSecretRecord({
|
||||
projectId: command.projectId,
|
||||
ordinal: publication.ordinal,
|
||||
kind: publication.kind,
|
||||
sourceNameDigest: publication.sourceNameDigest,
|
||||
secretName: envelope.name,
|
||||
secretMutationId: envelope.mutationId,
|
||||
valueFile: publication.valueFile,
|
||||
valueDigest: publication.valueDigest,
|
||||
});
|
||||
if (
|
||||
publication.ordinal !== index + 1 ||
|
||||
envelope.projectId !== command.projectId ||
|
||||
envelope.version !== 1 ||
|
||||
!UUID_V4_PATTERN.test(envelope.mutationId) ||
|
||||
names.has(envelope.name) ||
|
||||
mutations.has(envelope.mutationId) ||
|
||||
!DIGEST_PATTERN.test(publication.sourceNameDigest) ||
|
||||
!DIGEST_PATTERN.test(publication.valueDigest) ||
|
||||
!VALUE_FILE_PATTERN.test(publication.valueFile) ||
|
||||
publication.secretRef !== record.secretRef ||
|
||||
publication.itemDigest !== record.itemDigest ||
|
||||
itemAudit.eventId !== envelope.mutationId ||
|
||||
itemAudit.operationId !== 'secret.create' ||
|
||||
itemAudit.projectId !== command.projectId ||
|
||||
itemAudit.subject?.type !== subject.type ||
|
||||
itemAudit.subject.id !== subject.id ||
|
||||
itemAudit.outcome !== 'allowed' ||
|
||||
itemAudit.fence?.projectVersion !== command.fence.projectVersion ||
|
||||
itemAudit.fence.bindingVersion !== command.fence.bindingVersion ||
|
||||
itemAudit.occurredAtMs !== command.receipt.committedAtMs
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConflictError();
|
||||
}
|
||||
names.add(envelope.name);
|
||||
mutations.add(envelope.mutationId);
|
||||
return Object.freeze({ publication, record, envelope, audit: itemAudit });
|
||||
});
|
||||
const expectedReceipt = createLocalDataDirectoryAdoptionReceipt({
|
||||
mutationId: command.mutationId,
|
||||
projectId: command.projectId,
|
||||
profile: command.profile,
|
||||
sourceStageManifestDigest: command.sourceStageManifestDigest,
|
||||
transformationDigest: command.transformationDigest,
|
||||
modelDigest: command.modelDigest,
|
||||
secrets: secrets.map(({ record }) => record),
|
||||
committedAtMs: command.receipt.committedAtMs,
|
||||
});
|
||||
if (!sameJson(expectedReceipt, command.receipt)) {
|
||||
throw new LocalDataDirectoryAdoptionConflictError();
|
||||
}
|
||||
return Object.freeze({ subject, audit, secrets: Object.freeze(secrets) });
|
||||
}
|
||||
|
||||
function exactReplay(
|
||||
existing: Readonly<LocalDataDirectoryAdoptionRecord>,
|
||||
command: Readonly<PublishLocalDataDirectoryAdoptionCommand>,
|
||||
): boolean {
|
||||
return (
|
||||
existing.mutationId === command.mutationId &&
|
||||
existing.projectId === command.projectId &&
|
||||
existing.profile === command.profile &&
|
||||
existing.sourceStageManifestDigest === command.sourceStageManifestDigest &&
|
||||
existing.transformationDigest === command.transformationDigest &&
|
||||
existing.modelDigest === command.modelDigest &&
|
||||
sameJson(existing.model, command.model) &&
|
||||
sameJson(existing.receipt, command.receipt)
|
||||
);
|
||||
}
|
||||
|
||||
export class LocalSqliteDataDirectoryAdoptionPublisher {
|
||||
constructor(private readonly authority: LocalSqliteOperationAuthority) {}
|
||||
|
||||
resolve(
|
||||
mutationId: string,
|
||||
): Promise<Readonly<LocalDataDirectoryAdoptionRecord> | null> {
|
||||
if (!UUID_V4_PATTERN.test(mutationId)) {
|
||||
return Promise.reject(new LocalDataDirectoryAdoptionConflictError());
|
||||
}
|
||||
return this.authority.enqueue(
|
||||
async () => findRecord(this.authority.client, mutationId),
|
||||
() => new LocalDataDirectoryAdoptionUnavailableError(),
|
||||
);
|
||||
}
|
||||
|
||||
publish(
|
||||
command: Readonly<PublishLocalDataDirectoryAdoptionCommand>,
|
||||
): Promise<PublishLocalDataDirectoryAdoptionResult> {
|
||||
const normalized = assertCommand(command);
|
||||
return this.authority.enqueue(
|
||||
async () => {
|
||||
const client = this.authority.client;
|
||||
let began = false;
|
||||
try {
|
||||
client.exec('BEGIN IMMEDIATE');
|
||||
began = true;
|
||||
const replay = findRecord(
|
||||
client,
|
||||
command.mutationId,
|
||||
command.transformationDigest,
|
||||
);
|
||||
if (replay) {
|
||||
if (!exactReplay(replay, command)) {
|
||||
throw new LocalDataDirectoryAdoptionConflictError();
|
||||
}
|
||||
await command.confirmExternalAuthority();
|
||||
client.exec('COMMIT');
|
||||
began = false;
|
||||
return Object.freeze({
|
||||
status: 'existing' as const,
|
||||
adoption: replay,
|
||||
});
|
||||
}
|
||||
|
||||
const project = client
|
||||
.prepare(
|
||||
`SELECT "version", "status" FROM "QingLong3Projects"
|
||||
WHERE "id" = ? LIMIT 1`,
|
||||
)
|
||||
.get(command.projectId) as Row | undefined;
|
||||
if (
|
||||
!project ||
|
||||
integer(project, 'version') !== command.fence.projectVersion ||
|
||||
text(project, 'status') !== 'active'
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionAuthorizationFenceConflictError();
|
||||
}
|
||||
const binding = client
|
||||
.prepare(
|
||||
`SELECT "version", "state", "role"
|
||||
FROM "QingLong3ProjectRoleBindings"
|
||||
WHERE "project_id" = ? AND "subject_type" = ? AND "subject_id" = ?
|
||||
ORDER BY "version" DESC LIMIT 1`,
|
||||
)
|
||||
.get(
|
||||
command.projectId,
|
||||
normalized.subject.type,
|
||||
normalized.subject.id,
|
||||
) as Row | undefined;
|
||||
if (
|
||||
!binding ||
|
||||
integer(binding, 'version') !== command.fence.bindingVersion ||
|
||||
text(binding, 'state') !== 'active' ||
|
||||
!['owner', 'admin'].includes(text(binding, 'role'))
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionAuthorizationFenceConflictError();
|
||||
}
|
||||
|
||||
for (const { envelope, audit } of normalized.secrets) {
|
||||
const current = client
|
||||
.prepare(
|
||||
`SELECT MAX("version") AS "version"
|
||||
FROM "QingLong3LocalSecretEnvelopes"
|
||||
WHERE "project_id" = ? AND "secret_name" = ?`,
|
||||
)
|
||||
.get(envelope.projectId, envelope.name) as Row;
|
||||
if (current.version !== null) {
|
||||
throw new LocalDataDirectoryAdoptionConflictError();
|
||||
}
|
||||
const nonce = Buffer.from(envelope.nonce, 'base64url');
|
||||
const ciphertext = Buffer.from(envelope.ciphertext, 'base64url');
|
||||
const authTag = Buffer.from(envelope.authTag, 'base64url');
|
||||
try {
|
||||
client
|
||||
.prepare(
|
||||
`INSERT INTO "QingLong3LocalSecretEnvelopes" (
|
||||
"project_id", "secret_name", "version", "mutation_id",
|
||||
"key_id", "algorithm", "nonce", "ciphertext", "auth_tag",
|
||||
"created_at_ms"
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
)
|
||||
.run(
|
||||
envelope.projectId,
|
||||
envelope.name,
|
||||
envelope.version,
|
||||
envelope.mutationId,
|
||||
envelope.keyId,
|
||||
envelope.algorithm,
|
||||
nonce,
|
||||
ciphertext,
|
||||
authTag,
|
||||
envelope.createdAtMs,
|
||||
);
|
||||
} finally {
|
||||
nonce.fill(0);
|
||||
ciphertext.fill(0);
|
||||
authTag.fill(0);
|
||||
}
|
||||
insertAudit(client, audit);
|
||||
}
|
||||
insertAudit(client, normalized.audit);
|
||||
client
|
||||
.prepare(
|
||||
`INSERT INTO "QingLong3LegacyDataDirectoryAdoptions" (
|
||||
"mutation_id", "project_id", "profile",
|
||||
"source_stage_manifest_digest", "transformation_digest",
|
||||
"model_digest", "secret_count", "environment_secret_count",
|
||||
"ssh_secret_count", "model_json", "publication_digest",
|
||||
"audit_event_id", "committed_at_ms", "receipt_digest",
|
||||
"receipt_json"
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
)
|
||||
.run(
|
||||
command.mutationId,
|
||||
command.projectId,
|
||||
command.profile,
|
||||
command.sourceStageManifestDigest,
|
||||
command.transformationDigest,
|
||||
command.modelDigest,
|
||||
command.receipt.secretCount,
|
||||
command.receipt.environmentSecretCount,
|
||||
command.receipt.sshSecretCount,
|
||||
JSON.stringify(command.model),
|
||||
command.receipt.publicationDigest,
|
||||
command.mutationId,
|
||||
command.receipt.committedAtMs,
|
||||
command.receipt.receiptDigest,
|
||||
JSON.stringify(command.receipt),
|
||||
);
|
||||
for (const { record } of normalized.secrets) {
|
||||
client
|
||||
.prepare(
|
||||
`INSERT INTO "QingLong3LegacyDataDirectoryAdoptionSecrets" (
|
||||
"adoption_mutation_id", "ordinal", "project_id", "kind",
|
||||
"source_name_digest", "secret_name", "secret_version",
|
||||
"secret_mutation_id", "value_file", "value_digest",
|
||||
"secret_ref", "item_digest"
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
)
|
||||
.run(
|
||||
command.mutationId,
|
||||
record.ordinal,
|
||||
command.projectId,
|
||||
record.kind,
|
||||
record.sourceNameDigest,
|
||||
record.secretName,
|
||||
record.secretVersion,
|
||||
record.secretMutationId,
|
||||
record.valueFile,
|
||||
record.valueDigest,
|
||||
record.secretRef,
|
||||
record.itemDigest,
|
||||
);
|
||||
}
|
||||
const stored = findRecord(client, command.mutationId);
|
||||
if (!stored || !exactReplay(stored, command)) {
|
||||
throw new LocalDataDirectoryAdoptionUnavailableError();
|
||||
}
|
||||
await command.confirmExternalAuthority();
|
||||
client.exec('COMMIT');
|
||||
began = false;
|
||||
return Object.freeze({
|
||||
status: 'inserted' as const,
|
||||
adoption: stored,
|
||||
});
|
||||
} catch (error) {
|
||||
if (began && client.isTransaction) {
|
||||
try {
|
||||
client.exec('ROLLBACK');
|
||||
} catch {
|
||||
// Preserve the original failure.
|
||||
}
|
||||
}
|
||||
if (
|
||||
error instanceof LocalDataDirectoryAdoptionConflictError ||
|
||||
error instanceof
|
||||
LocalDataDirectoryAdoptionAuthorizationFenceConflictError ||
|
||||
error instanceof LocalDataDirectoryAdoptionUnavailableError
|
||||
) {
|
||||
throw error;
|
||||
}
|
||||
if (
|
||||
error &&
|
||||
typeof error === 'object' &&
|
||||
'code' in error &&
|
||||
typeof error.code === 'string' &&
|
||||
error.code.startsWith('SQLITE_CONSTRAINT')
|
||||
) {
|
||||
throw new LocalDataDirectoryAdoptionConflictError();
|
||||
}
|
||||
throw new LocalDataDirectoryAdoptionUnavailableError(error);
|
||||
}
|
||||
},
|
||||
() => new LocalDataDirectoryAdoptionUnavailableError(),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
export interface LocalSqliteDataDirectoryAdoptionDatabase {
|
||||
readonly profile: LocalSqliteProfile;
|
||||
readonly readiness: LocalSqliteReadinessEvidence;
|
||||
readonly projectPolicy: ProjectPolicyRepository;
|
||||
readonly securityAudit: SecurityAuditSink;
|
||||
readonly publisher: LocalSqliteDataDirectoryAdoptionPublisher;
|
||||
close(): Promise<void>;
|
||||
}
|
||||
|
||||
/** Short-lived data-directory adoption authority; runtime hosts must not import it. */
|
||||
export async function openLocalSqliteDataDirectoryAdoptionDatabase(
|
||||
options: LocalSqliteDatabaseOptions,
|
||||
): Promise<LocalSqliteDataDirectoryAdoptionDatabase> {
|
||||
assertLocalSqliteOptions(options);
|
||||
assertLocalSqlitePathBoundary(options.databasePath, false);
|
||||
const client = openLocalSqliteClient(options, false);
|
||||
try {
|
||||
const readiness = await auditLocalSqliteReadiness(client);
|
||||
const authority = new LocalSqliteOperationAuthority(client);
|
||||
const securityAuthority = new LocalSqliteSecurityAuthorityStore(authority);
|
||||
const projectPolicy: ProjectPolicyRepository = Object.freeze({
|
||||
resolve: (
|
||||
...[projectId, subject]: Parameters<ProjectPolicyRepository['resolve']>
|
||||
) => securityAuthority.resolve(projectId, subject),
|
||||
append: (...[command]: Parameters<ProjectPolicyRepository['append']>) =>
|
||||
securityAuthority.append(command),
|
||||
});
|
||||
let closePromise: Promise<void> | undefined;
|
||||
return Object.freeze({
|
||||
profile: options.profile,
|
||||
readiness,
|
||||
projectPolicy,
|
||||
securityAudit: securityAuthority,
|
||||
publisher: new LocalSqliteDataDirectoryAdoptionPublisher(authority),
|
||||
close() {
|
||||
if (closePromise) return closePromise;
|
||||
closePromise = authority.close();
|
||||
return closePromise;
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
if (client.isOpen) client.close();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
export const LOCAL_DATA_DIRECTORY_ADOPTION_SECRET_GUARD_TRIGGER_SQL = `
|
||||
CREATE TRIGGER "ql3_legacy_data_directory_adoption_secret_guard"
|
||||
BEFORE INSERT ON "QingLong3LegacyDataDirectoryAdoptionSecrets"
|
||||
BEGIN
|
||||
SELECT CASE WHEN NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM "QingLong3LegacyDataDirectoryAdoptions" AS adoption
|
||||
JOIN "QingLong3LocalSecretEnvelopes" AS secret
|
||||
ON secret."project_id" = NEW."project_id"
|
||||
AND secret."secret_name" = NEW."secret_name"
|
||||
AND secret."version" = NEW."secret_version"
|
||||
WHERE adoption."mutation_id" = NEW."adoption_mutation_id"
|
||||
AND adoption."project_id" = NEW."project_id"
|
||||
AND secret."mutation_id" = NEW."secret_mutation_id"
|
||||
) THEN RAISE(ABORT, 'legacy data directory adoption Secret authority mismatch') END;
|
||||
END
|
||||
`.trim();
|
||||
@@ -108,6 +108,8 @@ import { local0095PluginPackageSecretBindingTargetGuardMigration } from '../migr
|
||||
import { local0096CapabilityV48Migration } from '../migrations/0096-capability-v48';
|
||||
import { local0097PluginPackageSecretBindingTransitionReceiptsMigration } from '../migrations/0097-plugin-package-secret-binding-transition-receipts';
|
||||
import { local0098CapabilityV49Migration } from '../migrations/0098-capability-v49';
|
||||
import { local0099LegacyDataDirectoryAdoptionsMigration } from '../migrations/0099-legacy-data-directory-adoptions';
|
||||
import { local0100CapabilityV50Migration } from '../migrations/0100-capability-v50';
|
||||
import type { LocalSqliteMigrationContext } from '../migrations/sqlMigration';
|
||||
import {
|
||||
LOCAL_SQLITE_MIGRATION_STREAM_ID,
|
||||
@@ -228,6 +230,8 @@ export const localSqliteMigrationDefinition: MigrationStreamDefinition<LocalSqli
|
||||
local0096CapabilityV48Migration,
|
||||
local0097PluginPackageSecretBindingTransitionReceiptsMigration,
|
||||
local0098CapabilityV49Migration,
|
||||
local0099LegacyDataDirectoryAdoptionsMigration,
|
||||
local0100CapabilityV50Migration,
|
||||
]),
|
||||
});
|
||||
|
||||
|
||||
@@ -502,5 +502,15 @@ export const localSqliteMigrationManifest: MigrationStreamManifest =
|
||||
checksum:
|
||||
'133bdb78900256971bb6e13de8a129024d09f9d4c9d9290137eca3ff6e8b30eb',
|
||||
}),
|
||||
Object.freeze({
|
||||
id: '0099-legacy-data-directory-adoptions',
|
||||
checksum:
|
||||
'5a9edabd6a3e13cd8d71be6e85e3f211269b1909295aea27573c567c15112fd2',
|
||||
}),
|
||||
Object.freeze({
|
||||
id: '0100-capability-v50',
|
||||
checksum:
|
||||
'ea4ef39fe237d8db032da89c8f79dfda631b7201d1e5ac8c743b67e75aad5b07',
|
||||
}),
|
||||
]),
|
||||
});
|
||||
|
||||
@@ -0,0 +1,179 @@
|
||||
import { defineLocalSqliteMigration } from './sqlMigration';
|
||||
import { LOCAL_DATA_DIRECTORY_ADOPTION_SECRET_GUARD_TRIGGER_SQL } from '../adoption/data-directory/dataDirectoryAdoptionSchemaContract';
|
||||
|
||||
export const local0099LegacyDataDirectoryAdoptionsMigration =
|
||||
defineLocalSqliteMigration({
|
||||
id: '0099-legacy-data-directory-adoptions',
|
||||
statements: [
|
||||
`
|
||||
CREATE TABLE "QingLong3LegacyDataDirectoryAdoptions" (
|
||||
"mutation_id" TEXT PRIMARY KEY NOT NULL,
|
||||
"project_id" TEXT NOT NULL,
|
||||
"profile" TEXT NOT NULL,
|
||||
"source_stage_manifest_digest" TEXT NOT NULL,
|
||||
"transformation_digest" TEXT NOT NULL,
|
||||
"model_digest" TEXT NOT NULL,
|
||||
"secret_count" INTEGER NOT NULL,
|
||||
"environment_secret_count" INTEGER NOT NULL,
|
||||
"ssh_secret_count" INTEGER NOT NULL,
|
||||
"model_json" TEXT NOT NULL,
|
||||
"publication_digest" TEXT NOT NULL,
|
||||
"audit_event_id" TEXT NOT NULL,
|
||||
"committed_at_ms" INTEGER NOT NULL,
|
||||
"receipt_digest" TEXT NOT NULL,
|
||||
"receipt_json" TEXT NOT NULL,
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_project_fk
|
||||
FOREIGN KEY ("project_id")
|
||||
REFERENCES "QingLong3Projects" ("id")
|
||||
ON DELETE RESTRICT ON UPDATE RESTRICT,
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_audit_fk
|
||||
FOREIGN KEY ("audit_event_id")
|
||||
REFERENCES "QingLong3SecurityAuditEvents" ("event_id")
|
||||
ON DELETE RESTRICT ON UPDATE RESTRICT,
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_identity_check CHECK (
|
||||
length("mutation_id") = 36 AND
|
||||
substr("mutation_id", 15, 1) = '4' AND
|
||||
replace("mutation_id", '-', '') NOT GLOB '*[^0-9a-f]*' AND
|
||||
"audit_event_id" = "mutation_id" AND
|
||||
length("project_id") BETWEEN 1 AND 128
|
||||
),
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_profile_check CHECK (
|
||||
"profile" IN ('edge', 'standalone')
|
||||
),
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_digest_check CHECK (
|
||||
length("source_stage_manifest_digest") = 64 AND
|
||||
"source_stage_manifest_digest" NOT GLOB '*[^0-9a-f]*' AND
|
||||
length("transformation_digest") = 64 AND
|
||||
"transformation_digest" NOT GLOB '*[^0-9a-f]*' AND
|
||||
length("model_digest") = 64 AND
|
||||
"model_digest" NOT GLOB '*[^0-9a-f]*' AND
|
||||
length("publication_digest") = 64 AND
|
||||
"publication_digest" NOT GLOB '*[^0-9a-f]*' AND
|
||||
length("receipt_digest") = 64 AND
|
||||
"receipt_digest" NOT GLOB '*[^0-9a-f]*'
|
||||
),
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_count_check CHECK (
|
||||
"secret_count" BETWEEN 0 AND CASE "profile" WHEN 'edge' THEN 128 ELSE 512 END AND
|
||||
"environment_secret_count" BETWEEN 0 AND "secret_count" AND
|
||||
"ssh_secret_count" BETWEEN 0 AND "secret_count" AND
|
||||
"environment_secret_count" + "ssh_secret_count" = "secret_count"
|
||||
),
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_model_check CHECK (
|
||||
length(CAST("model_json" AS BLOB)) BETWEEN 2 AND 1048576 AND
|
||||
json_valid("model_json") AND json_type("model_json") = 'object' AND
|
||||
json_extract("model_json", '$.schema') = 'qinglong/legacy-data-directory-applied-model@v1' AND
|
||||
json_extract("model_json", '$.activation') = 'disabled' AND
|
||||
json_extract("model_json", '$.config.schema') = 'qinglong/legacy-config-transformation@v1' AND
|
||||
json_extract("model_json", '$.config.activation') = 'disabled' AND
|
||||
json_extract("model_json", '$.keyv.schema') = 'qinglong/legacy-keyv-transformation@v1' AND
|
||||
json_extract("model_json", '$.keyv.activation') = 'disabled' AND
|
||||
json_extract("model_json", '$.ssh.schema') = 'qinglong/legacy-ssh-transformation@v1' AND
|
||||
json_extract("model_json", '$.ssh.activation') = 'disabled' AND
|
||||
json_extract("model_json", '$.manualReview.schema') = 'qinglong/legacy-data-directory-manual-review@v1' AND
|
||||
json_extract("model_json", '$.manualReview.required') = 0 AND
|
||||
json_extract("model_json", '$.manualReview.activation') = 'disabled'
|
||||
),
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_receipt_check CHECK (
|
||||
length(CAST("receipt_json" AS BLOB)) BETWEEN 2 AND 1048576 AND
|
||||
json_valid("receipt_json") AND json_type("receipt_json") = 'object' AND
|
||||
json_extract("receipt_json", '$.schema') = 'qinglong/legacy-data-directory-adoption-receipt@v1' AND
|
||||
json_extract("receipt_json", '$.mutationId') = "mutation_id" AND
|
||||
json_extract("receipt_json", '$.projectId') = "project_id" AND
|
||||
json_extract("receipt_json", '$.profile') = "profile" AND
|
||||
json_extract("receipt_json", '$.sourceStageManifestDigest') = "source_stage_manifest_digest" AND
|
||||
json_extract("receipt_json", '$.transformationDigest') = "transformation_digest" AND
|
||||
json_extract("receipt_json", '$.modelDigest') = "model_digest" AND
|
||||
json_extract("receipt_json", '$.secretCount') = "secret_count" AND
|
||||
json_extract("receipt_json", '$.environmentSecretCount') = "environment_secret_count" AND
|
||||
json_extract("receipt_json", '$.sshSecretCount') = "ssh_secret_count" AND
|
||||
json_extract("receipt_json", '$.publicationDigest') = "publication_digest" AND
|
||||
json_extract("receipt_json", '$.auditEventId') = "audit_event_id" AND
|
||||
json_extract("receipt_json", '$.committedAtMs') = "committed_at_ms" AND
|
||||
json_extract("receipt_json", '$.receiptDigest') = "receipt_digest"
|
||||
),
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_time_check CHECK (
|
||||
"committed_at_ms" >= 0
|
||||
)
|
||||
)
|
||||
`,
|
||||
`
|
||||
CREATE UNIQUE INDEX "ql3_legacy_data_directory_adoption_transformation_uidx"
|
||||
ON "QingLong3LegacyDataDirectoryAdoptions" ("transformation_digest")
|
||||
`,
|
||||
`
|
||||
CREATE UNIQUE INDEX "ql3_legacy_data_directory_adoption_receipt_uidx"
|
||||
ON "QingLong3LegacyDataDirectoryAdoptions" ("receipt_digest")
|
||||
`,
|
||||
`
|
||||
CREATE INDEX "ql3_legacy_data_directory_adoption_project_time_idx"
|
||||
ON "QingLong3LegacyDataDirectoryAdoptions" (
|
||||
"project_id", "committed_at_ms" DESC, "mutation_id" DESC
|
||||
)
|
||||
`,
|
||||
`
|
||||
CREATE TABLE "QingLong3LegacyDataDirectoryAdoptionSecrets" (
|
||||
"adoption_mutation_id" TEXT NOT NULL,
|
||||
"ordinal" INTEGER NOT NULL,
|
||||
"project_id" TEXT NOT NULL,
|
||||
"kind" TEXT NOT NULL,
|
||||
"source_name_digest" TEXT NOT NULL,
|
||||
"secret_name" TEXT NOT NULL,
|
||||
"secret_version" INTEGER NOT NULL,
|
||||
"secret_mutation_id" TEXT NOT NULL,
|
||||
"value_file" TEXT NOT NULL,
|
||||
"value_digest" TEXT NOT NULL,
|
||||
"secret_ref" TEXT NOT NULL,
|
||||
"item_digest" TEXT NOT NULL,
|
||||
PRIMARY KEY ("adoption_mutation_id", "ordinal"),
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_secret_parent_fk
|
||||
FOREIGN KEY ("adoption_mutation_id")
|
||||
REFERENCES "QingLong3LegacyDataDirectoryAdoptions" ("mutation_id")
|
||||
ON DELETE RESTRICT ON UPDATE RESTRICT,
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_secret_envelope_fk
|
||||
FOREIGN KEY ("project_id", "secret_name", "secret_version")
|
||||
REFERENCES "QingLong3LocalSecretEnvelopes" ("project_id", "secret_name", "version")
|
||||
ON DELETE RESTRICT ON UPDATE RESTRICT,
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_secret_audit_fk
|
||||
FOREIGN KEY ("secret_mutation_id")
|
||||
REFERENCES "QingLong3SecurityAuditEvents" ("event_id")
|
||||
ON DELETE RESTRICT ON UPDATE RESTRICT,
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_secret_identity_check CHECK (
|
||||
"ordinal" BETWEEN 1 AND 512 AND
|
||||
length("project_id") BETWEEN 1 AND 128 AND
|
||||
"kind" IN ('environment', 'ssh_private_key') AND
|
||||
length("secret_name") BETWEEN 1 AND 128 AND
|
||||
"secret_version" = 1 AND
|
||||
length("secret_mutation_id") = 36 AND
|
||||
substr("secret_mutation_id", 15, 1) = '4' AND
|
||||
replace("secret_mutation_id", '-', '') NOT GLOB '*[^0-9a-f]*' AND
|
||||
length("value_file") = 83 AND
|
||||
"value_file" GLOB 'secret-values/[0-9a-f]*.json' AND
|
||||
length("secret_ref") BETWEEN 1 AND 512
|
||||
),
|
||||
CONSTRAINT ql3_legacy_data_directory_adoption_secret_digest_check CHECK (
|
||||
length("source_name_digest") = 64 AND
|
||||
"source_name_digest" NOT GLOB '*[^0-9a-f]*' AND
|
||||
length("value_digest") = 64 AND
|
||||
"value_digest" NOT GLOB '*[^0-9a-f]*' AND
|
||||
length("item_digest") = 64 AND
|
||||
"item_digest" NOT GLOB '*[^0-9a-f]*'
|
||||
)
|
||||
)
|
||||
`,
|
||||
`
|
||||
CREATE UNIQUE INDEX "ql3_legacy_data_directory_adoption_secret_name_uidx"
|
||||
ON "QingLong3LegacyDataDirectoryAdoptionSecrets" (
|
||||
"adoption_mutation_id", "secret_name"
|
||||
)
|
||||
`,
|
||||
`
|
||||
CREATE UNIQUE INDEX "ql3_legacy_data_directory_adoption_secret_mutation_uidx"
|
||||
ON "QingLong3LegacyDataDirectoryAdoptionSecrets" ("secret_mutation_id")
|
||||
`,
|
||||
`
|
||||
CREATE UNIQUE INDEX "ql3_legacy_data_directory_adoption_secret_item_uidx"
|
||||
ON "QingLong3LegacyDataDirectoryAdoptionSecrets" ("item_digest")
|
||||
`,
|
||||
LOCAL_DATA_DIRECTORY_ADOPTION_SECRET_GUARD_TRIGGER_SQL,
|
||||
],
|
||||
});
|
||||
@@ -0,0 +1,14 @@
|
||||
import { CAPABILITIES_V49 } from './0098-capability-v49';
|
||||
import { defineLocalSqliteMigration } from './sqlMigration';
|
||||
|
||||
export const CAPABILITIES_V50 = CAPABILITIES_V49.replace(
|
||||
'"legacy_adoption_ledger":1,',
|
||||
'"legacy_adoption_ledger":1,"legacy_data_directory_adoption":1,',
|
||||
);
|
||||
|
||||
export const local0100CapabilityV50Migration = defineLocalSqliteMigration({
|
||||
id: '0100-capability-v50',
|
||||
statements: [
|
||||
`UPDATE "QingLong3SchemaCapabilities" SET contract_version = 50, migration_id = '0099-legacy-data-directory-adoptions', capabilities = '${CAPABILITIES_V50}', updated_at_ms = CAST(unixepoch('subsec') * 1000 AS INTEGER) WHERE contract_name = 'local-control-core' AND contract_version = 49 AND migration_id = '0097-plugin-package-secret-binding-transition-receipts' AND capabilities = '${CAPABILITIES_V49}'`,
|
||||
],
|
||||
});
|
||||
@@ -2,6 +2,7 @@ import { auditMigrationStreamHistory } from '@qinglong/runtime-core/migration-st
|
||||
import type { DatabaseSync } from 'node:sqlite';
|
||||
import { localSqliteMigrationManifest } from '../migration/migrationManifest';
|
||||
import { LocalSqliteMigrationStreamStore } from '../migration/migrationStreamStore';
|
||||
import { LOCAL_DATA_DIRECTORY_ADOPTION_SECRET_GUARD_TRIGGER_SQL } from '../adoption/data-directory/dataDirectoryAdoptionSchemaContract';
|
||||
import { LOCAL_PLUGIN_PACKAGE_SECRET_MATERIALIZATION_TRIGGER_SQL } from '../plugin-package/pluginPackageSecretMaterializationSchemaContract';
|
||||
import { LOCAL_PLUGIN_PACKAGE_SECRET_BINDING_TARGET_TRIGGER_SQL } from '../plugin-package/secret-binding/pluginPackageSecretBindingTargetSchemaContract';
|
||||
import { LOCAL_PLUGIN_PACKAGE_SECRET_BINDING_TRANSITION_RECEIPT_TRIGGER_SQL } from '../plugin-package/secret-binding/transitionReceiptSchemaContract';
|
||||
@@ -11,7 +12,15 @@ import {
|
||||
} from '../run/stepRunSchemaContract';
|
||||
|
||||
export const LOCAL_SQLITE_CONTRACT_NAME = 'local-control-core';
|
||||
export const LOCAL_SQLITE_CONTRACT_VERSION = 49;
|
||||
export const LOCAL_SQLITE_CONTRACT_VERSION = 50;
|
||||
|
||||
const LEGACY_DATA_DIRECTORY_ADOPTION_TRIGGERS = Object.freeze([
|
||||
Object.freeze({
|
||||
name: 'ql3_legacy_data_directory_adoption_secret_guard',
|
||||
tableName: 'QingLong3LegacyDataDirectoryAdoptionSecrets',
|
||||
sql: LOCAL_DATA_DIRECTORY_ADOPTION_SECRET_GUARD_TRIGGER_SQL,
|
||||
}),
|
||||
]);
|
||||
|
||||
const PLUGIN_PACKAGE_SECRET_BINDING_TARGET_TRIGGERS = Object.freeze([
|
||||
Object.freeze({
|
||||
@@ -1389,6 +1398,51 @@ const REQUIRED_SCHEMA = Object.freeze({
|
||||
'ql3_legacy_adoptions_project_time_idx',
|
||||
]),
|
||||
}),
|
||||
QingLong3LegacyDataDirectoryAdoptions: Object.freeze({
|
||||
columns: Object.freeze([
|
||||
'mutation_id',
|
||||
'project_id',
|
||||
'profile',
|
||||
'source_stage_manifest_digest',
|
||||
'transformation_digest',
|
||||
'model_digest',
|
||||
'secret_count',
|
||||
'environment_secret_count',
|
||||
'ssh_secret_count',
|
||||
'model_json',
|
||||
'publication_digest',
|
||||
'audit_event_id',
|
||||
'committed_at_ms',
|
||||
'receipt_digest',
|
||||
'receipt_json',
|
||||
]),
|
||||
indexes: Object.freeze([
|
||||
'ql3_legacy_data_directory_adoption_transformation_uidx',
|
||||
'ql3_legacy_data_directory_adoption_receipt_uidx',
|
||||
'ql3_legacy_data_directory_adoption_project_time_idx',
|
||||
]),
|
||||
}),
|
||||
QingLong3LegacyDataDirectoryAdoptionSecrets: Object.freeze({
|
||||
columns: Object.freeze([
|
||||
'adoption_mutation_id',
|
||||
'ordinal',
|
||||
'project_id',
|
||||
'kind',
|
||||
'source_name_digest',
|
||||
'secret_name',
|
||||
'secret_version',
|
||||
'secret_mutation_id',
|
||||
'value_file',
|
||||
'value_digest',
|
||||
'secret_ref',
|
||||
'item_digest',
|
||||
]),
|
||||
indexes: Object.freeze([
|
||||
'ql3_legacy_data_directory_adoption_secret_name_uidx',
|
||||
'ql3_legacy_data_directory_adoption_secret_mutation_uidx',
|
||||
'ql3_legacy_data_directory_adoption_secret_item_uidx',
|
||||
]),
|
||||
}),
|
||||
QingLong3IdentitySubjects: Object.freeze({
|
||||
columns: Object.freeze([
|
||||
'subject_type',
|
||||
@@ -1793,6 +1847,7 @@ function assertRequiredSchema(client: DatabaseSync): number {
|
||||
...PLUGIN_PACKAGE_AUTOMATION_DISPOSITION_TRIGGERS,
|
||||
...PLUGIN_PACKAGE_SECRET_BINDING_TARGET_TRIGGERS,
|
||||
...PLUGIN_PACKAGE_SECRET_MATERIALIZATION_TRIGGERS,
|
||||
...LEGACY_DATA_DIRECTORY_ADOPTION_TRIGGERS,
|
||||
].sort((left, right) => left.name.localeCompare(right.name));
|
||||
if (
|
||||
triggerRows.length !== expectedTriggers.length ||
|
||||
@@ -2603,11 +2658,10 @@ export async function auditLocalSqliteReadiness(
|
||||
!capability ||
|
||||
capability.contract_name !== LOCAL_SQLITE_CONTRACT_NAME ||
|
||||
capability.contract_version !== LOCAL_SQLITE_CONTRACT_VERSION ||
|
||||
capability.migration_id !==
|
||||
'0097-plugin-package-secret-binding-transition-receipts' ||
|
||||
capability.migration_id !== '0099-legacy-data-directory-adoptions' ||
|
||||
typeof capability.capabilities !== 'string' ||
|
||||
capability.capabilities !==
|
||||
'{"run_core":1,"run_retry_policy":1,"completion_receipt_journal":1,"local_dispatch_plan":1,"local_secret_envelope":1,"local_project_policy":1,"local_project_administration":1,"local_security_audit":1,"local_security_audit_compaction":1,"local_secret_authorized_mutation":1,"local_identity":1,"local_api_credential":1,"local_identity_provisioning":1,"local_identity_credential_administration":1,"local_owner_bootstrap":1,"local_owner_delivery_acknowledgement":1,"api_credential_pepper_binding":1,"local_owner_pepper_catalog":1,"local_owner_credential_recovery":1,"local_owner_pepper_reference_inspection":1,"local_owner_pepper_material_gc":1,"local_owner_delivery_acknowledgement_gc":1,"task_definition":1,"local_execution_revision_digest":1,"trigger_definition":1,"legacy_adoption_ledger":1,"local_scheduler_admission":1,"plugin_package_install":1,"approved_action":1,"plugin_package_admission":1,"approved_action_execution":1,"plugin_package_proposal":1,"plugin_package_materialized_revision":1,"plugin_package_secret_binding":1,"plugin_package_secret_binding_transition":1,"plugin_package_secret_binding_transition_receipt":1,"plugin_package_secret_materialization":1,"plugin_package_task_reconciliation":1,"project_tool_definition_snapshot":1,"step_run":1,"tool_execution_evidence":1,"tool_execution_start_barrier":1,"tool_invocation_artifact":1,"tool_execution_artifact_binding":1,"tool_execution_completion":1,"tool_execution_failure_completion":1,"tool_result_key_catalog":1,"tool_result_rekey":1,"plugin_package_quarantine":1,"plugin_package_lifecycle":1,"plugin_package_automation_publication":1,"plugin_package_automation_security_withdrawal":1,"plugin_package_workflow_admission":1,"plugin_package_workflow_run_list":1,"run_attempt_log_retention":1,"plugin_package_workflow_task_attempt_admission":1}' ||
|
||||
'{"run_core":1,"run_retry_policy":1,"completion_receipt_journal":1,"local_dispatch_plan":1,"local_secret_envelope":1,"local_project_policy":1,"local_project_administration":1,"local_security_audit":1,"local_security_audit_compaction":1,"local_secret_authorized_mutation":1,"local_identity":1,"local_api_credential":1,"local_identity_provisioning":1,"local_identity_credential_administration":1,"local_owner_bootstrap":1,"local_owner_delivery_acknowledgement":1,"api_credential_pepper_binding":1,"local_owner_pepper_catalog":1,"local_owner_credential_recovery":1,"local_owner_pepper_reference_inspection":1,"local_owner_pepper_material_gc":1,"local_owner_delivery_acknowledgement_gc":1,"task_definition":1,"local_execution_revision_digest":1,"trigger_definition":1,"legacy_adoption_ledger":1,"legacy_data_directory_adoption":1,"local_scheduler_admission":1,"plugin_package_install":1,"approved_action":1,"plugin_package_admission":1,"approved_action_execution":1,"plugin_package_proposal":1,"plugin_package_materialized_revision":1,"plugin_package_secret_binding":1,"plugin_package_secret_binding_transition":1,"plugin_package_secret_binding_transition_receipt":1,"plugin_package_secret_materialization":1,"plugin_package_task_reconciliation":1,"project_tool_definition_snapshot":1,"step_run":1,"tool_execution_evidence":1,"tool_execution_start_barrier":1,"tool_invocation_artifact":1,"tool_execution_artifact_binding":1,"tool_execution_completion":1,"tool_execution_failure_completion":1,"tool_result_key_catalog":1,"tool_result_rekey":1,"plugin_package_quarantine":1,"plugin_package_lifecycle":1,"plugin_package_automation_publication":1,"plugin_package_automation_security_withdrawal":1,"plugin_package_workflow_admission":1,"plugin_package_workflow_run_list":1,"run_attempt_log_retention":1,"plugin_package_workflow_task_attempt_admission":1}' ||
|
||||
typeof capability.updated_at_ms !== 'number' ||
|
||||
!Number.isSafeInteger(capability.updated_at_ms) ||
|
||||
capability.updated_at_ms < 0
|
||||
|
||||
@@ -3328,6 +3328,144 @@ export const legacyAdoptions = sqliteTable(
|
||||
],
|
||||
);
|
||||
|
||||
export const legacyDataDirectoryAdoptions = sqliteTable(
|
||||
'QingLong3LegacyDataDirectoryAdoptions',
|
||||
{
|
||||
mutationId: text('mutation_id').primaryKey(),
|
||||
projectId: text('project_id')
|
||||
.notNull()
|
||||
.references(() => localProjects.id, {
|
||||
onDelete: 'restrict',
|
||||
onUpdate: 'restrict',
|
||||
}),
|
||||
profile: text('profile').notNull(),
|
||||
sourceStageManifestDigest: text('source_stage_manifest_digest').notNull(),
|
||||
transformationDigest: text('transformation_digest').notNull(),
|
||||
modelDigest: text('model_digest').notNull(),
|
||||
secretCount: integer('secret_count').notNull(),
|
||||
environmentSecretCount: integer('environment_secret_count').notNull(),
|
||||
sshSecretCount: integer('ssh_secret_count').notNull(),
|
||||
modelJson: text('model_json', { mode: 'json' })
|
||||
.$type<Record<string, unknown>>()
|
||||
.notNull(),
|
||||
publicationDigest: text('publication_digest').notNull(),
|
||||
auditEventId: text('audit_event_id')
|
||||
.notNull()
|
||||
.references(() => localSecurityAuditEvents.eventId, {
|
||||
onDelete: 'restrict',
|
||||
onUpdate: 'restrict',
|
||||
}),
|
||||
committedAtMs: integer('committed_at_ms').notNull(),
|
||||
receiptDigest: text('receipt_digest').notNull(),
|
||||
receiptJson: text('receipt_json', { mode: 'json' })
|
||||
.$type<Record<string, unknown>>()
|
||||
.notNull(),
|
||||
},
|
||||
(table) => [
|
||||
check(
|
||||
'ql3_legacy_data_directory_adoption_identity_check',
|
||||
sql`length(${table.mutationId}) = 36 and substr(${table.mutationId}, 15, 1) = '4' and replace(${table.mutationId}, '-', '') not glob '*[^0-9a-f]*' and ${table.auditEventId} = ${table.mutationId} and length(${table.projectId}) between 1 and 128`,
|
||||
),
|
||||
check(
|
||||
'ql3_legacy_data_directory_adoption_profile_check',
|
||||
sql`${table.profile} in ('edge', 'standalone')`,
|
||||
),
|
||||
check(
|
||||
'ql3_legacy_data_directory_adoption_digest_check',
|
||||
sql`length(${table.sourceStageManifestDigest}) = 64 and ${table.sourceStageManifestDigest} not glob '*[^0-9a-f]*' and length(${table.transformationDigest}) = 64 and ${table.transformationDigest} not glob '*[^0-9a-f]*' and length(${table.modelDigest}) = 64 and ${table.modelDigest} not glob '*[^0-9a-f]*' and length(${table.publicationDigest}) = 64 and ${table.publicationDigest} not glob '*[^0-9a-f]*' and length(${table.receiptDigest}) = 64 and ${table.receiptDigest} not glob '*[^0-9a-f]*'`,
|
||||
),
|
||||
check(
|
||||
'ql3_legacy_data_directory_adoption_count_check',
|
||||
sql`${table.secretCount} between 0 and case ${table.profile} when 'edge' then 128 else 512 end and ${table.environmentSecretCount} between 0 and ${table.secretCount} and ${table.sshSecretCount} between 0 and ${table.secretCount} and ${table.environmentSecretCount} + ${table.sshSecretCount} = ${table.secretCount}`,
|
||||
),
|
||||
check(
|
||||
'ql3_legacy_data_directory_adoption_model_check',
|
||||
sql`length(cast(${table.modelJson} as blob)) between 2 and 1048576 and json_valid(${table.modelJson}) and json_type(${table.modelJson}) = 'object' and json_extract(${table.modelJson}, '$.schema') = 'qinglong/legacy-data-directory-applied-model@v1' and json_extract(${table.modelJson}, '$.activation') = 'disabled' and json_extract(${table.modelJson}, '$.config.schema') = 'qinglong/legacy-config-transformation@v1' and json_extract(${table.modelJson}, '$.config.activation') = 'disabled' and json_extract(${table.modelJson}, '$.keyv.schema') = 'qinglong/legacy-keyv-transformation@v1' and json_extract(${table.modelJson}, '$.keyv.activation') = 'disabled' and json_extract(${table.modelJson}, '$.ssh.schema') = 'qinglong/legacy-ssh-transformation@v1' and json_extract(${table.modelJson}, '$.ssh.activation') = 'disabled' and json_extract(${table.modelJson}, '$.manualReview.schema') = 'qinglong/legacy-data-directory-manual-review@v1' and json_extract(${table.modelJson}, '$.manualReview.required') = 0 and json_extract(${table.modelJson}, '$.manualReview.activation') = 'disabled'`,
|
||||
),
|
||||
check(
|
||||
'ql3_legacy_data_directory_adoption_receipt_check',
|
||||
sql`length(cast(${table.receiptJson} as blob)) between 2 and 1048576 and json_valid(${table.receiptJson}) and json_type(${table.receiptJson}) = 'object' and json_extract(${table.receiptJson}, '$.schema') = 'qinglong/legacy-data-directory-adoption-receipt@v1' and json_extract(${table.receiptJson}, '$.mutationId') = ${table.mutationId} and json_extract(${table.receiptJson}, '$.projectId') = ${table.projectId} and json_extract(${table.receiptJson}, '$.profile') = ${table.profile} and json_extract(${table.receiptJson}, '$.sourceStageManifestDigest') = ${table.sourceStageManifestDigest} and json_extract(${table.receiptJson}, '$.transformationDigest') = ${table.transformationDigest} and json_extract(${table.receiptJson}, '$.modelDigest') = ${table.modelDigest} and json_extract(${table.receiptJson}, '$.secretCount') = ${table.secretCount} and json_extract(${table.receiptJson}, '$.environmentSecretCount') = ${table.environmentSecretCount} and json_extract(${table.receiptJson}, '$.sshSecretCount') = ${table.sshSecretCount} and json_extract(${table.receiptJson}, '$.publicationDigest') = ${table.publicationDigest} and json_extract(${table.receiptJson}, '$.auditEventId') = ${table.auditEventId} and json_extract(${table.receiptJson}, '$.committedAtMs') = ${table.committedAtMs} and json_extract(${table.receiptJson}, '$.receiptDigest') = ${table.receiptDigest}`,
|
||||
),
|
||||
check(
|
||||
'ql3_legacy_data_directory_adoption_time_check',
|
||||
sql`${table.committedAtMs} >= 0`,
|
||||
),
|
||||
uniqueIndex('ql3_legacy_data_directory_adoption_transformation_uidx').on(
|
||||
table.transformationDigest,
|
||||
),
|
||||
uniqueIndex('ql3_legacy_data_directory_adoption_receipt_uidx').on(
|
||||
table.receiptDigest,
|
||||
),
|
||||
index('ql3_legacy_data_directory_adoption_project_time_idx').on(
|
||||
table.projectId,
|
||||
sql`${table.committedAtMs} desc`,
|
||||
sql`${table.mutationId} desc`,
|
||||
),
|
||||
],
|
||||
);
|
||||
|
||||
export const legacyDataDirectoryAdoptionSecrets = sqliteTable(
|
||||
'QingLong3LegacyDataDirectoryAdoptionSecrets',
|
||||
{
|
||||
adoptionMutationId: text('adoption_mutation_id').notNull(),
|
||||
ordinal: integer('ordinal').notNull(),
|
||||
projectId: text('project_id').notNull(),
|
||||
kind: text('kind').notNull(),
|
||||
sourceNameDigest: text('source_name_digest').notNull(),
|
||||
secretName: text('secret_name').notNull(),
|
||||
secretVersion: integer('secret_version').notNull(),
|
||||
secretMutationId: text('secret_mutation_id')
|
||||
.notNull()
|
||||
.references(() => localSecurityAuditEvents.eventId, {
|
||||
onDelete: 'restrict',
|
||||
onUpdate: 'restrict',
|
||||
}),
|
||||
valueFile: text('value_file').notNull(),
|
||||
valueDigest: text('value_digest').notNull(),
|
||||
secretRef: text('secret_ref').notNull(),
|
||||
itemDigest: text('item_digest').notNull(),
|
||||
},
|
||||
(table) => [
|
||||
primaryKey({ columns: [table.adoptionMutationId, table.ordinal] }),
|
||||
foreignKey({
|
||||
columns: [table.adoptionMutationId],
|
||||
foreignColumns: [legacyDataDirectoryAdoptions.mutationId],
|
||||
name: 'ql3_legacy_data_directory_adoption_secret_parent_fk',
|
||||
})
|
||||
.onDelete('restrict')
|
||||
.onUpdate('restrict'),
|
||||
foreignKey({
|
||||
columns: [table.projectId, table.secretName, table.secretVersion],
|
||||
foreignColumns: [
|
||||
localSecretEnvelopes.projectId,
|
||||
localSecretEnvelopes.name,
|
||||
localSecretEnvelopes.version,
|
||||
],
|
||||
name: 'ql3_legacy_data_directory_adoption_secret_envelope_fk',
|
||||
})
|
||||
.onDelete('restrict')
|
||||
.onUpdate('restrict'),
|
||||
check(
|
||||
'ql3_legacy_data_directory_adoption_secret_identity_check',
|
||||
sql`${table.ordinal} between 1 and 512 and length(${table.projectId}) between 1 and 128 and ${table.kind} in ('environment', 'ssh_private_key') and length(${table.secretName}) between 1 and 128 and ${table.secretVersion} = 1 and length(${table.secretMutationId}) = 36 and substr(${table.secretMutationId}, 15, 1) = '4' and replace(${table.secretMutationId}, '-', '') not glob '*[^0-9a-f]*' and length(${table.valueFile}) = 83 and ${table.valueFile} glob 'secret-values/[0-9a-f]*.json' and length(${table.secretRef}) between 1 and 512`,
|
||||
),
|
||||
check(
|
||||
'ql3_legacy_data_directory_adoption_secret_digest_check',
|
||||
sql`length(${table.sourceNameDigest}) = 64 and ${table.sourceNameDigest} not glob '*[^0-9a-f]*' and length(${table.valueDigest}) = 64 and ${table.valueDigest} not glob '*[^0-9a-f]*' and length(${table.itemDigest}) = 64 and ${table.itemDigest} not glob '*[^0-9a-f]*'`,
|
||||
),
|
||||
uniqueIndex('ql3_legacy_data_directory_adoption_secret_name_uidx').on(
|
||||
table.adoptionMutationId,
|
||||
table.secretName,
|
||||
),
|
||||
uniqueIndex('ql3_legacy_data_directory_adoption_secret_mutation_uidx').on(
|
||||
table.secretMutationId,
|
||||
),
|
||||
uniqueIndex('ql3_legacy_data_directory_adoption_secret_item_uidx').on(
|
||||
table.itemDigest,
|
||||
),
|
||||
],
|
||||
);
|
||||
|
||||
export const localIdentitySubjects = sqliteTable(
|
||||
'QingLong3IdentitySubjects',
|
||||
{
|
||||
@@ -5042,6 +5180,8 @@ export const localSqliteSchema = Object.freeze({
|
||||
toolExecutionResultRekeyHeads,
|
||||
toolResultKeyRetirementReceipts,
|
||||
legacyAdoptions,
|
||||
legacyDataDirectoryAdoptions,
|
||||
legacyDataDirectoryAdoptionSecrets,
|
||||
localIdentitySubjects,
|
||||
localApiCredentials,
|
||||
localApiCredentialPepperBindings,
|
||||
|
||||
@@ -148,9 +148,11 @@ test('creates a reviewed edge database and opens runtime only after readiness',
|
||||
'0096-capability-v48',
|
||||
'0097-plugin-package-secret-binding-transition-receipts',
|
||||
'0098-capability-v49',
|
||||
'0099-legacy-data-directory-adoptions',
|
||||
'0100-capability-v50',
|
||||
]);
|
||||
assert.equal(migrated.readiness.contractName, 'local-control-core');
|
||||
assert.equal(migrated.readiness.contractVersion, 49);
|
||||
assert.equal(migrated.readiness.contractVersion, 50);
|
||||
assert.equal(migrated.readiness.journalMode, 'delete');
|
||||
assert.equal(fs.statSync(databasePath).mode & 0o777, 0o600);
|
||||
|
||||
@@ -596,8 +598,8 @@ test('backfills v14 execution revisions with a verified independent digest', asy
|
||||
.get(),
|
||||
},
|
||||
{
|
||||
contract_version: 49,
|
||||
migration_id: '0097-plugin-package-secret-binding-transition-receipts',
|
||||
contract_version: 50,
|
||||
migration_id: '0099-legacy-data-directory-adoptions',
|
||||
},
|
||||
);
|
||||
} finally {
|
||||
@@ -784,19 +786,19 @@ test('excludes reviewed optional feature tables while preserving unknown table d
|
||||
const options = { databasePath, profile: 'edge' };
|
||||
await migrateLocalSqlitePath(options);
|
||||
const client = new DatabaseSync(databasePath);
|
||||
assert.equal((await auditLocalSqlitePath(options)).tableCount, 81);
|
||||
assert.equal((await auditLocalSqlitePath(options)).tableCount, 83);
|
||||
client.exec(
|
||||
'CREATE TABLE "ModelInvocationFeatureHead" (feature_id TEXT PRIMARY KEY)',
|
||||
);
|
||||
client.close();
|
||||
|
||||
assert.equal((await auditLocalSqlitePath(options)).tableCount, 81);
|
||||
assert.equal((await auditLocalSqlitePath(options)).tableCount, 83);
|
||||
|
||||
const unknownClient = new DatabaseSync(databasePath);
|
||||
unknownClient.exec('CREATE TABLE "UserExtensionData" (id TEXT PRIMARY KEY)');
|
||||
unknownClient.close();
|
||||
|
||||
assert.equal((await auditLocalSqlitePath(options)).tableCount, 82);
|
||||
assert.equal((await auditLocalSqlitePath(options)).tableCount, 84);
|
||||
|
||||
const triggerClient = new DatabaseSync(databasePath);
|
||||
triggerClient.exec(`
|
||||
|
||||
@@ -156,7 +156,7 @@ test('atomically admits one generation-bound Workflow Run and exactly replays it
|
||||
},
|
||||
{ runs: 1, steps: 2, events: 3, mutations: 2, admissions: 1 },
|
||||
);
|
||||
assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 49);
|
||||
assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 50);
|
||||
});
|
||||
|
||||
test('runs an optional authorization guard inside new and replay transactions', async (t) => {
|
||||
@@ -288,7 +288,7 @@ test('exactly replays immutable admission after the Workflow StepRun advances',
|
||||
},
|
||||
{ status: 'running', version: 5, eventSequence: 5 },
|
||||
);
|
||||
assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 49);
|
||||
assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 50);
|
||||
});
|
||||
|
||||
test('fails closed before writing when the exact installation is not active', async (t) => {
|
||||
|
||||
+1
-1
@@ -231,7 +231,7 @@ test('atomically admits the exact reconciled local Task revision and replays it'
|
||||
stepAttemptCount: 0,
|
||||
},
|
||||
);
|
||||
assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 49);
|
||||
assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 50);
|
||||
});
|
||||
|
||||
test('bounds candidate paging before SQL and fences cancellation', async (t) => {
|
||||
|
||||
@@ -40,9 +40,9 @@ test('creates and exactly replays a reviewed rollout backup', async (t) => {
|
||||
await migrateLocalSqlitePath(state);
|
||||
const prepared = await createLocalSqliteRolloutBackup(state);
|
||||
assert.equal(prepared.status, 'prepared');
|
||||
assert.equal(prepared.contractVersion, 49);
|
||||
assert.equal(prepared.writeContractVersion, 49);
|
||||
assert.equal(LOCAL_SQLITE_WRITE_CONTRACT_VERSION, 49);
|
||||
assert.equal(prepared.contractVersion, 50);
|
||||
assert.equal(prepared.writeContractVersion, 50);
|
||||
assert.equal(LOCAL_SQLITE_WRITE_CONTRACT_VERSION, 50);
|
||||
assert.match(prepared.sha256, /^[0-9a-f]{64}$/);
|
||||
assert.equal(prepared.bytes > 0, true);
|
||||
assert.equal(prepared.pageCount > 0, true);
|
||||
|
||||
@@ -1767,6 +1767,20 @@ function auditSourceImports(root, packagePath, findings) {
|
||||
'src/lifecycle/data-directory-adoption/staging.ts' &&
|
||||
specifier === '@qinglong/local-admin/runtime'
|
||||
) &&
|
||||
!(
|
||||
path.relative(packageDirectory, filePath) ===
|
||||
'src/lifecycle/data-directory-adoption/application/application.ts' &&
|
||||
[
|
||||
'@qinglong/local-admin/data-directory-adoption',
|
||||
'@qinglong/local-owner-console/authenticated-command',
|
||||
'@qinglong/local-sqlite/bootstrap',
|
||||
].includes(specifier)
|
||||
) &&
|
||||
!(
|
||||
path.relative(packageDirectory, filePath) ===
|
||||
'src/lifecycle/data-directory-adoption/application/cleanup.ts' &&
|
||||
specifier === '@qinglong/local-sqlite/data-directory-adoption'
|
||||
) &&
|
||||
!(
|
||||
path.relative(packageDirectory, filePath) ===
|
||||
'src/plugin-package/pluginPackageCommand.ts' &&
|
||||
@@ -2093,6 +2107,9 @@ function auditSourceImports(root, packagePath, findings) {
|
||||
(path.relative(packageDirectory, filePath) ===
|
||||
'src/legacy-adoption/legacyCrontabPublisher.ts' &&
|
||||
specifier === '@qinglong/local-sqlite/adoption') ||
|
||||
(path.relative(packageDirectory, filePath) ===
|
||||
'src/data-directory-adoption/dataDirectoryAdoption.ts' &&
|
||||
specifier === '@qinglong/local-sqlite/data-directory-adoption') ||
|
||||
(path.relative(packageDirectory, filePath) ===
|
||||
'src/adopted-profile/localAdoptedProfile.ts' &&
|
||||
specifier === '@qinglong/local-sqlite/profile') ||
|
||||
@@ -2229,6 +2246,14 @@ function auditSourceImports(root, packagePath, findings) {
|
||||
'@qinglong/runtime-core/security',
|
||||
'@qinglong/runtime-core/security-audit',
|
||||
].includes(specifier)) ||
|
||||
(path.relative(packageDirectory, filePath) ===
|
||||
'src/data-directory-adoption/dataDirectoryAdoption.ts' &&
|
||||
[
|
||||
'@qinglong/runtime-core/local-secret',
|
||||
'@qinglong/runtime-core/project-policy',
|
||||
'@qinglong/runtime-core/security',
|
||||
'@qinglong/runtime-core/security-audit',
|
||||
].includes(specifier)) ||
|
||||
(path.relative(packageDirectory, filePath) ===
|
||||
'src/security-administration/projectPolicyAdministration.ts' &&
|
||||
[
|
||||
@@ -2812,6 +2837,14 @@ function auditSourceImports(root, packagePath, findings) {
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
if (
|
||||
packagePath === 'packages/ql3-local-owner-cli' &&
|
||||
path.relative(packageDirectory, filePath) ===
|
||||
'src/lifecycle/data-directory-adoption/application/application.ts' &&
|
||||
specifier === '@qinglong/local-secret'
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
if (
|
||||
packagePath === 'packages/ql3-local-owner-cli' &&
|
||||
path.relative(packageDirectory, filePath) ===
|
||||
|
||||
@@ -1914,6 +1914,110 @@ test('confines adoption and Package command authorities to owner CLI subpaths',
|
||||
);
|
||||
});
|
||||
|
||||
test('confines data-directory application authority to exact reviewed owners', (t) => {
|
||||
const root = fs.mkdtempSync(
|
||||
path.join(os.tmpdir(), 'ql3-data-directory-application-boundary-'),
|
||||
);
|
||||
const ownerApplicationDirectory = path.join(
|
||||
root,
|
||||
'packages/ql3-local-owner-cli/src/lifecycle/data-directory-adoption/application',
|
||||
);
|
||||
const adminAdoptionDirectory = path.join(
|
||||
root,
|
||||
'packages/ql3-local-admin/src/data-directory-adoption',
|
||||
);
|
||||
fs.mkdirSync(ownerApplicationDirectory, { recursive: true });
|
||||
fs.mkdirSync(adminAdoptionDirectory, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(ownerApplicationDirectory, 'application.ts'),
|
||||
[
|
||||
"import { application } from '@qinglong/local-admin/data-directory-adoption';",
|
||||
"import { secrets } from '@qinglong/local-secret';",
|
||||
"import { authenticated } from '@qinglong/local-owner-console/authenticated-command';",
|
||||
"import { bootstrap } from '@qinglong/local-sqlite/bootstrap';",
|
||||
].join('\n'),
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(ownerApplicationDirectory, 'cleanup.ts'),
|
||||
"import { cleanup } from '@qinglong/local-sqlite/data-directory-adoption';",
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(ownerApplicationDirectory, 'neighbor.ts'),
|
||||
[
|
||||
"import { application } from '@qinglong/local-admin/data-directory-adoption';",
|
||||
"import { secrets } from '@qinglong/local-secret';",
|
||||
"import { cleanup } from '@qinglong/local-sqlite/data-directory-adoption';",
|
||||
].join('\n'),
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(adminAdoptionDirectory, 'dataDirectoryAdoption.ts'),
|
||||
[
|
||||
"import { database } from '@qinglong/local-sqlite/data-directory-adoption';",
|
||||
"import type { secret } from '@qinglong/runtime-core/local-secret';",
|
||||
"import { policy } from '@qinglong/runtime-core/project-policy';",
|
||||
"import type { decision } from '@qinglong/runtime-core/security';",
|
||||
"import type { audit } from '@qinglong/runtime-core/security-audit';",
|
||||
].join('\n'),
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(adminAdoptionDirectory, 'neighbor.ts'),
|
||||
[
|
||||
"import { database } from '@qinglong/local-sqlite/data-directory-adoption';",
|
||||
"import type { secret } from '@qinglong/runtime-core/local-secret';",
|
||||
].join('\n'),
|
||||
);
|
||||
t.after(() => fs.rmSync(root, { recursive: true, force: true }));
|
||||
|
||||
const ownerFindings = [];
|
||||
auditSourceImports(root, 'packages/ql3-local-owner-cli', ownerFindings);
|
||||
assert.deepEqual(
|
||||
ownerFindings.map(({ code, file, specifier }) => ({
|
||||
code,
|
||||
file,
|
||||
specifier,
|
||||
})),
|
||||
[
|
||||
{
|
||||
code: 'FORBIDDEN_LOCAL_ADOPTION_CLI_AUTHORITY_IMPORT',
|
||||
file: 'packages/ql3-local-owner-cli/src/lifecycle/data-directory-adoption/application/neighbor.ts',
|
||||
specifier: '@qinglong/local-admin/data-directory-adoption',
|
||||
},
|
||||
{
|
||||
code: 'FORBIDDEN_PACKAGE_SOURCE_IMPORT',
|
||||
file: 'packages/ql3-local-owner-cli/src/lifecycle/data-directory-adoption/application/neighbor.ts',
|
||||
specifier: '@qinglong/local-secret',
|
||||
},
|
||||
{
|
||||
code: 'FORBIDDEN_LOCAL_ADOPTION_CLI_AUTHORITY_IMPORT',
|
||||
file: 'packages/ql3-local-owner-cli/src/lifecycle/data-directory-adoption/application/neighbor.ts',
|
||||
specifier: '@qinglong/local-sqlite/data-directory-adoption',
|
||||
},
|
||||
],
|
||||
);
|
||||
|
||||
const adminFindings = [];
|
||||
auditSourceImports(root, 'packages/ql3-local-admin', adminFindings);
|
||||
assert.deepEqual(
|
||||
adminFindings.map(({ code, file, specifier }) => ({
|
||||
code,
|
||||
file,
|
||||
specifier,
|
||||
})),
|
||||
[
|
||||
{
|
||||
code: 'FORBIDDEN_LOCAL_ADMIN_SQLITE_ENTRYPOINT',
|
||||
file: 'packages/ql3-local-admin/src/data-directory-adoption/neighbor.ts',
|
||||
specifier: '@qinglong/local-sqlite/data-directory-adoption',
|
||||
},
|
||||
{
|
||||
code: 'FORBIDDEN_LOCAL_ADMIN_RUNTIME_CORE_ENTRYPOINT',
|
||||
file: 'packages/ql3-local-admin/src/data-directory-adoption/neighbor.ts',
|
||||
specifier: '@qinglong/runtime-core/local-secret',
|
||||
},
|
||||
],
|
||||
);
|
||||
});
|
||||
|
||||
test('confines fresh setup authority to the reviewed owner CLI subpath', (t) => {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'ql3-setup-boundary-'));
|
||||
const sourceDirectory = path.join(root, 'packages/ql3-local-owner-cli/src');
|
||||
|
||||
@@ -79,7 +79,7 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
|
||||
},
|
||||
{
|
||||
directory: 'packages/ql3-local-sqlite/src/migrations',
|
||||
directSourceFiles: 99,
|
||||
directSourceFiles: 101,
|
||||
reviewKind: 'ordered_ledger',
|
||||
},
|
||||
],
|
||||
@@ -97,10 +97,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
|
||||
rootSourceFileRoles: localAdmin.rootSourceFileRoles,
|
||||
},
|
||||
{
|
||||
sourceFiles: 44,
|
||||
sourceFiles: 45,
|
||||
rootSourceFiles: 1,
|
||||
rootSourceLines: 9,
|
||||
nestedSourceFiles: 43,
|
||||
nestedSourceFiles: 44,
|
||||
rootSourceFileRoles: { 'runtime.ts': 'public_export' },
|
||||
},
|
||||
);
|
||||
@@ -207,10 +207,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
|
||||
rootSourceFileRoles: localOwnerCli.rootSourceFileRoles,
|
||||
},
|
||||
{
|
||||
sourceFiles: 129,
|
||||
sourceFiles: 131,
|
||||
rootSourceFiles: 1,
|
||||
rootSourceLines: 50,
|
||||
nestedSourceFiles: 128,
|
||||
nestedSourceFiles: 130,
|
||||
rootSourceFileRoles: { 'cli.ts': 'binary_entry' },
|
||||
},
|
||||
);
|
||||
@@ -540,10 +540,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
|
||||
rootSourceFileRoles: localSqlite.rootSourceFileRoles,
|
||||
},
|
||||
{
|
||||
sourceFiles: 197,
|
||||
sourceFiles: 201,
|
||||
rootSourceFiles: 1,
|
||||
rootSourceLines: 31,
|
||||
nestedSourceFiles: 196,
|
||||
nestedSourceFiles: 200,
|
||||
rootSourceFileRoles: { 'index.ts': 'public_export' },
|
||||
},
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user