mirror of
https://github.com/whyour/qinglong.git
synced 2026-09-20 16:07:11 +08:00
feat(ql3): add cluster legacy env migration plan ledger
This commit is contained in:
whyour
@@ -93,6 +93,33 @@
|
||||
readiness、观察窗和 rollback retention 之后另行强认证。Cluster 必须使用 PostgreSQL SERIALIZABLE ledger、外部 KMS/Secret provider 与 HA evidence，
|
||||
不复用 Local SQLite/POSIX authority，也不得把明文写入 PostgreSQL、ConfigMap、Pod env 或 Job command。
|
||||
|
||||
- D-400/ADR-0495（已验收）：第十一个 Secret/Config 切片补上 Cluster Legacy Env migration 的
|
||||
**content-free plan ledger baseline**，但没有把计划冒充实际迁移。profile-neutral
|
||||
`qinglong/cluster-legacy-env-migration-plan@v1` 只保存 reconciliation bundle、reviewed
|
||||
decision、candidate set、Task/Trigger revision set 和 plan 的 SHA-256 摘要，外加守恒计数、
|
||||
同 Project 且固定 version 的 canonical SecretRef、effective bytes 与数据库时间；不保存源
|
||||
Env name/value、row body、plaintext/ciphertext、key ID、Task/Trigger spec 或 credential。
|
||||
上限固定为 100,000 source rows、100,000 Tasks、500,000 Triggers、64 KiB effective
|
||||
Secret 和 8 KiB JSON。
|
||||
|
||||
PostgreSQL `pg-0070-cluster-legacy-env-migration-plans` 将 control contract 推进到 v69，
|
||||
以 Project FK、plan/mutation/digest 唯一性和六组 named constraints 创建 append-only 表；
|
||||
`plan_json` 必须与列值重建的 canonical JSONB 精确相等，因此额外 Env 字段不能藏入 durable
|
||||
JSON。只有 `ql3_automation_manager` 拥有 `SELECT, INSERT`，无 UPDATE/DELETE；runtime、
|
||||
admin 和其余角色均无权限。repository 只从显式
|
||||
`cluster-legacy-env-migration-plan` subpath 导出，以数据库时间和短 SERIALIZABLE transaction
|
||||
完成 exact mutation replay、active Project snapshot、三次有界 serialization retry 与可选同事务
|
||||
hook；普通 SERIALIZABLE read 避免 `FOR SHARE` 隐含的 Project UPDATE 权限扩张。
|
||||
|
||||
runtime contract 4/4、repository 5/5、migration/schema/readiness/entrypoint 92/92 通过，两包
|
||||
完整测试零失败。真实 PostgreSQL 18.4 已成功应用 70 条 migration，并证明 create/replay/read、
|
||||
automation UPDATE `42501`、runtime/admin SELECT `42501`，以及 widened `envName` JSON 被精确
|
||||
constraint 以 `23514` 拒绝。该切片复用既有两个 package，未增加 dependency、daemon 或
|
||||
Edge/Standalone import。它只关闭 plan ledger baseline；Cluster source scan/signed decision
|
||||
装配、逐项 Task/Trigger current-head revalidation 与 revision mutation、Secret material custody、
|
||||
migration receipt、HA promotion replay 和固定低性能设备证据仍是后续门禁。ADR-0491 因此继续
|
||||
Proposed。
|
||||
|
||||
- D-396/ADR-0490（已验收）：Run History 不再只有永久 `manual_external`，但也没有被错误实现为 Legacy 日志到 3.0 Run ledger 的回灌。
|
||||
新的 Local adapter 以 ADR-0482 sealed capture bundle 作为 append-only 保全资产：Legacy history 必须逐事实选择 `retain_both`，Target history
|
||||
必须选择 `retain_target`；receipt 只绑定 signed review、application、bundle fingerprint、领域 inventory 与有界 fact counts，不保存表名、Run ID、
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# ADR-0491：有界 Secret/Config Reconciliation 与任务环境绑定
|
||||
|
||||
- 状态：Proposed（D-397 已实现 Legacy Env inspection、私有有界 row plan、durable plan publication、独立 signed decision、逐项 Automation adoption provenance、Local SQLite 原子 application publisher、Owner prepared/apply/rollback 编排、ADR-0492 completion v3，并由 ADR-0494 完成 Cluster mounted-files provider live 子门；真实 Edge 空间证据、Cluster migration ledger 与直接外部 custody gate 尚未完成）
|
||||
- 状态：Proposed（D-397 已实现 Legacy Env inspection、私有有界 row plan、durable plan publication、独立 signed decision、逐项 Automation adoption provenance、Local SQLite 原子 application publisher、Owner prepared/apply/rollback 编排、ADR-0492 completion v3；ADR-0494 完成 Cluster mounted-files provider live 子门，ADR-0495 完成 content-free Cluster plan ledger baseline；真实 Edge 空间证据、Cluster Task/Trigger mutation/receipt、HA replay 与直接外部 custody gate 尚未完成）
|
||||
- 日期：2026-08-23
|
||||
- 决策：D-397
|
||||
- 关联：ADR-0073、ADR-0074、ADR-0092、ADR-0094、ADR-0480、ADR-0482、ADR-0483、ADR-0484、ADR-0485、ADR-0486、ADR-0487、ADR-0488、ADR-0490
|
||||
@@ -147,4 +147,4 @@ D-397 当前八切片已经实现：absent、unsupported、Edge over-budget、2.
|
||||
|
||||
ADR-0494 已完成 Cluster `mounted-files` provider live 子门：真实三节点 K3s 中两个 management replica、direct exact-key executor 和两个跨节点 provider observer 完成 PostgreSQL durable approval/binding、Kubernetes atomic projection rotation、无 Secret API 权限/ServiceAccount token、只读 `0440`、内容脱敏及删除后 fail-closed；v2 私有报告 24/24 gates 为 true，并保持 v1 verifier 兼容。该门不增加 Edge 闭包，也不等于直接 Vault/KMS/HSM custody。
|
||||
|
||||
转为 Accepted 前仍必须完成：固定低性能 Edge 设备的真实空间/写放大/断电恢复证据，以及 Cluster Legacy Env migration 的专用 PostgreSQL SERIALIZABLE ledger、Task/Trigger revision mutation、外部 custody adapter 和 HA promotion 后 receipt replay。ADR-0492 已完成本机 completion schema 演进和 completed-head 后 rollback material 回收，ADR-0493 又让没有 Legacy 身份输入的 fresh v52 目标身份经 signed `retain_target` 正确形成 no-effect，并精确消除六张已知目标表的 `unknown` 误判。Legacy `Auths/Users` 或真正未知表仍保持 manual；本切片的 Local Owner 编排、ADR-0494 的 mounted-files gate 或 PostgreSQL HA 证据都不得冒充完整 Cluster migration 与外部密钥托管。
|
||||
转为 Accepted 前仍必须完成：固定低性能 Edge 设备的真实空间/写放大/断电恢复证据，以及 Cluster Legacy Env migration 的逐项 Task/Trigger current-head revalidation、revision mutation/receipt、外部 custody adapter 和 HA promotion 后 receipt replay。ADR-0495 已完成专用 PostgreSQL SERIALIZABLE plan ledger baseline，但它只保存摘要、计数和 pinned SecretRef，不执行 Task/Trigger DML，也不接触 Secret material。ADR-0492 已完成本机 completion schema 演进和 completed-head 后 rollback material 回收，ADR-0493 又让没有 Legacy 身份输入的 fresh v52 目标身份经 signed `retain_target` 正确形成 no-effect，并精确消除六张已知目标表的 `unknown` 误判。Legacy `Auths/Users` 或真正未知表仍保持 manual；本切片的 Local Owner 编排、ADR-0494 的 mounted-files gate、ADR-0495 的 plan ledger 或 PostgreSQL HA 证据都不得冒充完整 Cluster migration 与外部密钥托管。
|
||||
|
||||
@@ -0,0 +1,140 @@
|
||||
# ADR-0495：无敏感内容的 Cluster Legacy Env 迁移计划账本
|
||||
|
||||
- 状态：Accepted
|
||||
- 日期：2026-08-24
|
||||
- 决策：D-400
|
||||
- 关联：ADR-0104、ADR-0233、ADR-0259、ADR-0491、ADR-0494
|
||||
|
||||
## 背景
|
||||
|
||||
ADR-0491 已完成 Local SQLite 上的 Legacy Env 检查、人工裁决、Secret application、
|
||||
Task/Trigger revision mutation、回滚和 completion；ADR-0494 又证明了 Cluster
|
||||
`mounted-files` provider 的在线轮换。Cluster 仍缺少一个独立于 Local SQLite/POSIX
|
||||
authority 的持久计划边界，无法在真正写 Secret、Task 或 Trigger 之前冻结源证据、目标
|
||||
revision 集合和幂等 mutation 身份。
|
||||
|
||||
该边界不能把 2.x `Envs` 的名称、值、行内容搬进 PostgreSQL，也不能为了“先跑起来”
|
||||
让 runtime、admin 或通用 root export 取得迁移写权限。小型路由设备不应安装或加载本
|
||||
Cluster 能力；集群侧也必须有明确的行数、Task、Trigger 和有效载荷预算。
|
||||
|
||||
## 决策
|
||||
|
||||
### 1. 账本只保存不可逆摘要、计数和目标引用
|
||||
|
||||
新增 profile-neutral `qinglong/cluster-legacy-env-migration-plan@v1`。计划只包含：
|
||||
|
||||
- plan、mutation、Project 身份；
|
||||
- reconciliation bundle、reviewed decision、candidate set 三个 SHA-256 摘要；
|
||||
- source/active/disabled/effective binding 计数；
|
||||
- 同 Project、精确固定 version 的 canonical `SecretRef`；
|
||||
- Task 与 Trigger revision set 摘要和计数；
|
||||
- effective Secret 总字节数、数据库计划时间和 plan digest。
|
||||
|
||||
它不得包含源 Env 名称/值、row body、目标 plaintext/ciphertext、key ID、credential、
|
||||
Task spec、Trigger spec 或迁移日志。`SecretRef` 是既有目标路由引用，不是源 Env 名称；
|
||||
其 payload 仍受 canonical encoding、同 Project 和固定 version 校验。
|
||||
|
||||
硬上限为 100,000 个源行、100,000 个 Task、500,000 个 Trigger、64 KiB effective
|
||||
Secret 和 8 KiB plan JSON。active/disabled/source 计数必须守恒，至少存在一个 active
|
||||
行、effective binding 和 Task。
|
||||
|
||||
### 2. PostgreSQL 只提供 append-only v69 计划表
|
||||
|
||||
`pg-0070-cluster-legacy-env-migration-plans` 将 control contract 推进到 v69，并新增
|
||||
`cluster_legacy_env_migration_plans`：
|
||||
|
||||
- `plan_id` 为主键，`mutation_id` 与 `plan_digest` 各自唯一；
|
||||
- Project 外键使用 `ON DELETE/UPDATE RESTRICT`；
|
||||
- identity、digest、source、target、time、JSON 六组 named constraints；
|
||||
- `plan_json` 必须与列值构造出的完整 canonical JSONB **精确相等**，不是包含关系，
|
||||
因而任何额外 Env 字段也会被数据库拒绝；
|
||||
- 不提供 UPDATE、DELETE、TRUNCATE 或后台 GC。
|
||||
|
||||
迁移能力值新增 `cluster_legacy_env_migration_plan: 1`。Drizzle schema、readiness
|
||||
contract、固定 migration checksum 和实际 SQL stream 必须保持同锁步。
|
||||
|
||||
### 3. 只有 Automation Manager 可以读取和追加
|
||||
|
||||
新表从 PUBLIC 撤销全部权限，只向 `ql3_automation_manager` 授予 `SELECT, INSERT`。
|
||||
runtime、admin、Worker ingress、Approval/Run/Package/Worker Credential 等其他角色均为
|
||||
零权限；migration owner 只用于 schema 管理与集成验证。
|
||||
|
||||
具体 repository 只由
|
||||
`@qinglong/cluster-postgres/cluster-legacy-env-migration-plan` 显式子路径发布，不进入
|
||||
package root、runtime 或 admin entrypoint。runtime-core 也只通过
|
||||
`@qinglong/runtime-core/cluster-legacy-env-migration-plan` 发布数据契约。
|
||||
|
||||
### 4. 发布使用数据库时间、SERIALIZABLE 和精确重放
|
||||
|
||||
repository 每次使用一个短 `SERIALIZABLE` transaction，设置 statement、lock 和 idle
|
||||
transaction timeout；先按 `mutation_id` 查 exact replay，再验证 Project 在该可序列化
|
||||
快照中为 active，使用 `transaction_timestamp()` 生成 `plannedAtMs`，最后追加计划。
|
||||
|
||||
相同 mutation 与相同 intent 返回 durable `existing`；mutation drift、plan identity
|
||||
占用、inactive Project 和 named constraint 冲突失败关闭。`40001`、`40P01`、`55P03`
|
||||
最多重试三次。可选 transaction hook 只用于后续在**同一 Automation Manager 事务**中
|
||||
追加 Task/Trigger revision mutation；本 ADR 不调用它来声称那些 mutation 已实现。
|
||||
|
||||
Project 检查使用普通 SERIALIZABLE read，而不是 `FOR SHARE`。后者在 PostgreSQL 中还
|
||||
要求 UPDATE privilege，会错误扩大 Automation Manager 权限；可序列化顺序与外键已足以
|
||||
表达“计划发生在并发归档之前”的合法历史。
|
||||
|
||||
### 5. 不扩大 Edge/Standalone 闭包
|
||||
|
||||
实现复用既有 `runtime-core` 与 `cluster-postgres` package，不新增 workspace package、
|
||||
生产依赖、daemon、timer、watcher、controller 或缓存全集。Edge/Standalone 不导入
|
||||
PostgreSQL repository 或 migration；计数上限是 Cluster 计划输入预算，不是把集群表或
|
||||
Task 集合加载进小型设备。
|
||||
|
||||
## 被拒绝的替代方案
|
||||
|
||||
### 在 PostgreSQL 保存 Env 名称、值、密文或 key ID
|
||||
|
||||
拒绝。密文仍会扩大 custody、rotation、backup 和 HA 泄漏面；计划账本只需要摘要和目标
|
||||
引用即可证明后续 mutation 的输入身份。
|
||||
|
||||
### 在同一切片直接改写全部 Task/Trigger head
|
||||
|
||||
拒绝。计划发布和实际 DML 是两个可独立审计的 authority 阶段。没有 current-head
|
||||
revalidation、逐项 mutation receipt 与完整 response-loss 测试前，不得把 plan 冒充迁移。
|
||||
|
||||
### 复用 Local SQLite 或 Plugin Package Secret Binding 表
|
||||
|
||||
拒绝。Local authority 不属于 Cluster；Plugin Package binding 的 installation/generation
|
||||
身份也不能表达 Legacy Env reconciliation bundle、disabled preservation 和 Task/Trigger
|
||||
revision set。
|
||||
|
||||
### 为一个 contract 和一个 repository 新建 package
|
||||
|
||||
拒绝。它们分别属于既有 runtime migration contract 与 Cluster reconciliation storage
|
||||
能力，新增微包会再次制造用户已经指出的单文件 package 问题。
|
||||
|
||||
## 当前验证
|
||||
|
||||
2026-08-24 已完成：
|
||||
|
||||
- runtime contract 定向测试 4/4；
|
||||
- PostgreSQL repository 定向测试 5/5；
|
||||
- `@qinglong/runtime-core` 与 `@qinglong/cluster-postgres` 完整 package 测试零失败；
|
||||
- PostgreSQL migration/schema/readiness/entrypoint 定向门 92/92；
|
||||
- 真实 PostgreSQL 18.4 应用 70 条 migration 成功；
|
||||
- 实际 automation-manager create/replay/read 成功，UPDATE 被 `42501` 拒绝；
|
||||
- 实际 runtime/admin SELECT 被 `42501` 拒绝；
|
||||
- 带源 `envName` 的 widened JSON 被
|
||||
`ql3_cluster_legacy_env_plan_json_check` 以 `23514` 拒绝。
|
||||
|
||||
## 边界与后续门禁
|
||||
|
||||
本 ADR 关闭 ADR-0491 的 **Cluster plan ledger baseline**，但不把 ADR-0491 整体转为
|
||||
Accepted，也不声明：
|
||||
|
||||
1. Legacy source scan、sealed bundle 或 signed decision 已在 Cluster 中生产装配；
|
||||
2. Task/Trigger current head 已逐项复验或 revision mutation 已提交；
|
||||
3. Secret material 已写入 Kubernetes/Vault/KMS/HSM；
|
||||
4. migration Job、短期身份、receipt 与 promotion 后 exact replay 已完成；
|
||||
5. PostgreSQL physical failover、CloudNativePG promotion 或多架构结果已由本切片重新证明；
|
||||
6. 固定低性能路由设备的空间、写放大、断电和恢复证据已完成。
|
||||
|
||||
下一切片应在 Automation Manager 同一事务内绑定 current Task/Trigger revision set、逐项
|
||||
mutation 与 receipt，同时保持计划表 append-only；随后再接入外部 custody 和 HA promotion
|
||||
后的 exact replay。任何一步都不能回写源 Env 明文或放宽其他数据库角色。
|
||||
@@ -498,6 +498,7 @@
|
||||
| [ADR-0492](./ADR-0492-compatible-secret-config-completion-v3.md) | 兼容 Secret/Config 的 Reconciliation Completion v3 | Accepted |
|
||||
| [ADR-0493](./ADR-0493-target-native-reconciliation-domain-preservation.md) | 目标原生 Reconciliation 域分类与身份保留 | Accepted |
|
||||
| [ADR-0494](./ADR-0494-postgresql-secret-binding-and-mounted-provider-live-rotation.md) | PostgreSQL Secret Binding 与 Mounted Provider 在线轮换门 | Accepted |
|
||||
| [ADR-0495](./ADR-0495-content-free-cluster-legacy-env-migration-plan-ledger.md) | 无敏感内容的 Cluster Legacy Env 迁移计划账本 | Accepted |
|
||||
|
||||
## 规则
|
||||
|
||||
|
||||
@@ -217,6 +217,7 @@ function runtimePrivileges() {
|
||||
plugin_package_workflow_admissions: [true, true, false, false],
|
||||
plugin_package_workflow_admission_steps: [true, true, false, false],
|
||||
plugin_package_workflow_task_attempt_admissions: [true, true, false, false],
|
||||
cluster_legacy_env_migration_plans: [false, false, false, false],
|
||||
worker_execution_attestations: [true, false, false, false],
|
||||
run_events: [true, true, false, false],
|
||||
run_cancellation_dispatches: [true, true, true, false],
|
||||
|
||||
@@ -131,6 +131,7 @@ function runtimePrivileges() {
|
||||
plugin_package_workflow_admissions: [true, true, false, false],
|
||||
plugin_package_workflow_admission_steps: [true, true, false, false],
|
||||
plugin_package_workflow_task_attempt_admissions: [true, true, false, false],
|
||||
cluster_legacy_env_migration_plans: [false, false, false, false],
|
||||
worker_execution_attestations: [true, false, false, false],
|
||||
run_events: [true, true, false, false],
|
||||
run_cancellation_dispatches: [true, true, true, false],
|
||||
|
||||
@@ -65,6 +65,11 @@
|
||||
"require": "./dist/entrypoints/automationManager.js",
|
||||
"default": "./dist/entrypoints/automationManager.js"
|
||||
},
|
||||
"./cluster-legacy-env-migration-plan": {
|
||||
"types": "./dist/reconciliation/clusterLegacyEnvMigrationPlanRepository.d.ts",
|
||||
"require": "./dist/reconciliation/clusterLegacyEnvMigrationPlanRepository.js",
|
||||
"default": "./dist/reconciliation/clusterLegacyEnvMigrationPlanRepository.js"
|
||||
},
|
||||
"./task-start": {
|
||||
"types": "./dist/task-start/taskStartRepository.d.ts",
|
||||
"require": "./dist/task-start/taskStartRepository.js",
|
||||
|
||||
@@ -353,5 +353,10 @@ export const postgresqlMainMigrationManifest: MigrationStreamManifest =
|
||||
checksum:
|
||||
'1191255575589abc2686b391827607abddb4edb78007245dbaaf45dc1c4e5e8b',
|
||||
}),
|
||||
Object.freeze({
|
||||
id: 'pg-0070-cluster-legacy-env-migration-plans',
|
||||
checksum:
|
||||
'7cd6d993f48e7bcebcd62c93571a738d5117c9bcde33b974c5ac8962e2a03fe4',
|
||||
}),
|
||||
]),
|
||||
});
|
||||
|
||||
@@ -72,6 +72,7 @@ import { pg0066CancellationDispatchMigration } from '../run/migrations/pg-0066-c
|
||||
import { pg0067CancellationDispatchManagementMigration } from '../run-management/pg-0067-cancellation-dispatch-management';
|
||||
import { pg0068CancellationDispatchProjectKeysetMigration } from '../run-management/pg-0068-cancellation-dispatch-project-keyset';
|
||||
import { pg0069WorkerSessionManagementObservationMigration } from '../remote-execution/pg-0069-worker-session-management-observation';
|
||||
import { pg0070ClusterLegacyEnvMigrationPlansMigration } from '../reconciliation/pg-0070-cluster-legacy-env-migration-plans';
|
||||
|
||||
export const postgresqlMainMigrationStream: MigrationStreamDefinition<PostgresMigrationContext> =
|
||||
Object.freeze({
|
||||
@@ -149,5 +150,6 @@ export const postgresqlMainMigrationStream: MigrationStreamDefinition<PostgresMi
|
||||
pg0067CancellationDispatchManagementMigration,
|
||||
pg0068CancellationDispatchProjectKeysetMigration,
|
||||
pg0069WorkerSessionManagementObservationMigration,
|
||||
pg0070ClusterLegacyEnvMigrationPlansMigration,
|
||||
]),
|
||||
});
|
||||
|
||||
+301
@@ -0,0 +1,301 @@
|
||||
import {
|
||||
ClusterLegacyEnvMigrationPlanConflictError,
|
||||
ClusterLegacyEnvMigrationPlanUnavailableError,
|
||||
assertClusterLegacyEnvMigrationPlanIdentifier,
|
||||
clusterLegacyEnvMigrationPlanMatchesIntent,
|
||||
createClusterLegacyEnvMigrationPlan,
|
||||
normalizeClusterLegacyEnvMigrationPlan,
|
||||
normalizeClusterLegacyEnvMigrationPlanIntent,
|
||||
type ClusterLegacyEnvMigrationPlan,
|
||||
type ClusterLegacyEnvMigrationPlanIntent,
|
||||
type ClusterLegacyEnvMigrationPlanRepository,
|
||||
} from '@qinglong/runtime-core/cluster-legacy-env-migration-plan';
|
||||
import type { PostgresClient, PostgresPool } from '@qinglong/runtime-core';
|
||||
|
||||
import {
|
||||
POSTGRES_DEFINITION_RETRYABLE_SQL_STATES,
|
||||
POSTGRES_DEFINITION_TRANSACTION_ATTEMPTS,
|
||||
configurePostgresDefinitionTransaction,
|
||||
postgresRequiredInteger,
|
||||
postgresRequiredJsonObject,
|
||||
postgresSqlState,
|
||||
rollbackPostgresDefinitionTransaction,
|
||||
} from '../repository/definitionRepositorySupport';
|
||||
|
||||
type Row = Record<string, unknown>;
|
||||
type Queryable = Pick<PostgresPool, 'query'> | Pick<PostgresClient, 'query'>;
|
||||
|
||||
export interface PostgresClusterLegacyEnvMigrationPlanTransactionContext {
|
||||
readonly intent: Readonly<ClusterLegacyEnvMigrationPlanIntent>;
|
||||
readonly replay: Readonly<ClusterLegacyEnvMigrationPlan> | null;
|
||||
readonly plan: Readonly<ClusterLegacyEnvMigrationPlan>;
|
||||
}
|
||||
|
||||
export type PostgresClusterLegacyEnvMigrationPlanTransactionHook = (
|
||||
client: PostgresClient,
|
||||
context: Readonly<PostgresClusterLegacyEnvMigrationPlanTransactionContext>,
|
||||
) => Promise<void>;
|
||||
|
||||
function unavailable(): ClusterLegacyEnvMigrationPlanUnavailableError {
|
||||
return new ClusterLegacyEnvMigrationPlanUnavailableError();
|
||||
}
|
||||
|
||||
function planFromRow(row: Row): Readonly<ClusterLegacyEnvMigrationPlan> {
|
||||
try {
|
||||
return normalizeClusterLegacyEnvMigrationPlan(
|
||||
postgresRequiredJsonObject(
|
||||
row.planJson,
|
||||
unavailable,
|
||||
) as unknown as ClusterLegacyEnvMigrationPlan,
|
||||
);
|
||||
} catch (error) {
|
||||
if (error instanceof ClusterLegacyEnvMigrationPlanUnavailableError) {
|
||||
throw error;
|
||||
}
|
||||
throw unavailable();
|
||||
}
|
||||
}
|
||||
|
||||
async function findByPlanId(
|
||||
queryable: Queryable,
|
||||
planId: string,
|
||||
): Promise<Readonly<ClusterLegacyEnvMigrationPlan> | null> {
|
||||
const result = await queryable.query<Row>(
|
||||
`SELECT plan_json AS "planJson"
|
||||
FROM "ql3"."cluster_legacy_env_migration_plans"
|
||||
WHERE plan_id = $1
|
||||
LIMIT 2`,
|
||||
[planId],
|
||||
);
|
||||
if (result.rows.length === 0) return null;
|
||||
if (result.rows.length !== 1) throw unavailable();
|
||||
const plan = planFromRow(result.rows[0]!);
|
||||
if (plan.planId !== planId) throw unavailable();
|
||||
return plan;
|
||||
}
|
||||
|
||||
async function findByMutationId(
|
||||
queryable: Queryable,
|
||||
mutationId: string,
|
||||
): Promise<Readonly<ClusterLegacyEnvMigrationPlan> | null> {
|
||||
const result = await queryable.query<Row>(
|
||||
`SELECT plan_json AS "planJson"
|
||||
FROM "ql3"."cluster_legacy_env_migration_plans"
|
||||
WHERE mutation_id = $1
|
||||
LIMIT 2`,
|
||||
[mutationId],
|
||||
);
|
||||
if (result.rows.length === 0) return null;
|
||||
if (result.rows.length !== 1) throw unavailable();
|
||||
const plan = planFromRow(result.rows[0]!);
|
||||
if (plan.mutationId !== mutationId) throw unavailable();
|
||||
return plan;
|
||||
}
|
||||
|
||||
function mappedError(error: unknown): Error {
|
||||
if (
|
||||
error instanceof ClusterLegacyEnvMigrationPlanConflictError ||
|
||||
error instanceof ClusterLegacyEnvMigrationPlanUnavailableError
|
||||
) {
|
||||
return error;
|
||||
}
|
||||
const state = postgresSqlState(error);
|
||||
if (state === '23503' || state === '23505' || state === '23514') {
|
||||
return new ClusterLegacyEnvMigrationPlanConflictError();
|
||||
}
|
||||
return unavailable();
|
||||
}
|
||||
|
||||
/**
|
||||
* Automation-manager-only append authority for content-free Cluster Legacy Env
|
||||
* migration plans. It does not materialize Secrets or mutate Task/Trigger heads.
|
||||
*/
|
||||
export class PostgresClusterLegacyEnvMigrationPlanRepository
|
||||
implements ClusterLegacyEnvMigrationPlanRepository
|
||||
{
|
||||
constructor(private readonly pool: PostgresPool) {
|
||||
if (
|
||||
!pool ||
|
||||
typeof pool.query !== 'function' ||
|
||||
typeof pool.connect !== 'function'
|
||||
) {
|
||||
throw new TypeError(
|
||||
'PostgreSQL Cluster Legacy Env migration pool is invalid',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async findByPlanId(
|
||||
planIdValue: string,
|
||||
): Promise<Readonly<ClusterLegacyEnvMigrationPlan> | null> {
|
||||
const planId = assertClusterLegacyEnvMigrationPlanIdentifier(
|
||||
planIdValue,
|
||||
'planId',
|
||||
);
|
||||
try {
|
||||
return await findByPlanId(this.pool, planId);
|
||||
} catch (error) {
|
||||
throw mappedError(error);
|
||||
}
|
||||
}
|
||||
|
||||
async publish(
|
||||
intentValue: Readonly<ClusterLegacyEnvMigrationPlanIntent>,
|
||||
transactionHook?: PostgresClusterLegacyEnvMigrationPlanTransactionHook,
|
||||
): Promise<
|
||||
Readonly<{
|
||||
status: 'created' | 'existing';
|
||||
plan: Readonly<ClusterLegacyEnvMigrationPlan>;
|
||||
}>
|
||||
> {
|
||||
if (
|
||||
transactionHook !== undefined &&
|
||||
typeof transactionHook !== 'function'
|
||||
) {
|
||||
throw new TypeError(
|
||||
'Cluster Legacy Env migration transaction hook is invalid',
|
||||
);
|
||||
}
|
||||
const intent = normalizeClusterLegacyEnvMigrationPlanIntent(intentValue);
|
||||
|
||||
for (
|
||||
let attempt = 0;
|
||||
attempt < POSTGRES_DEFINITION_TRANSACTION_ATTEMPTS;
|
||||
attempt += 1
|
||||
) {
|
||||
let client: PostgresClient;
|
||||
try {
|
||||
client = await this.pool.connect();
|
||||
} catch {
|
||||
throw unavailable();
|
||||
}
|
||||
let began = false;
|
||||
let transactionHookError: unknown;
|
||||
try {
|
||||
await configurePostgresDefinitionTransaction(client);
|
||||
began = true;
|
||||
|
||||
const replay = await findByMutationId(client, intent.mutationId);
|
||||
if (replay) {
|
||||
if (!clusterLegacyEnvMigrationPlanMatchesIntent(replay, intent)) {
|
||||
throw new ClusterLegacyEnvMigrationPlanConflictError();
|
||||
}
|
||||
if (transactionHook) {
|
||||
try {
|
||||
const hookResult = await transactionHook(
|
||||
client,
|
||||
Object.freeze({ intent, replay, plan: replay }),
|
||||
);
|
||||
if (hookResult !== undefined) {
|
||||
throw new TypeError(
|
||||
'Cluster Legacy Env migration transaction hook must not return a value',
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
transactionHookError = error;
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
await client.query('COMMIT');
|
||||
began = false;
|
||||
return Object.freeze({ status: 'existing', plan: replay });
|
||||
}
|
||||
|
||||
const project = await client.query<{ status: unknown }>(
|
||||
`SELECT status
|
||||
FROM "ql3"."projects"
|
||||
WHERE id = $1`,
|
||||
[intent.projectId],
|
||||
);
|
||||
if (project.rows.length !== 1 || project.rows[0]?.status !== 'active') {
|
||||
throw new ClusterLegacyEnvMigrationPlanConflictError();
|
||||
}
|
||||
|
||||
const occupied = await findByPlanId(client, intent.planId);
|
||||
if (occupied) throw new ClusterLegacyEnvMigrationPlanConflictError();
|
||||
|
||||
const clock = await client.query<{ plannedAtMs: unknown }>(
|
||||
`SELECT floor(extract(epoch FROM transaction_timestamp()) * 1000)::bigint AS "plannedAtMs"`,
|
||||
);
|
||||
if (clock.rows.length !== 1) throw unavailable();
|
||||
const plan = createClusterLegacyEnvMigrationPlan(
|
||||
intent,
|
||||
postgresRequiredInteger(clock.rows[0]?.plannedAtMs, unavailable),
|
||||
);
|
||||
|
||||
await client.query(
|
||||
`INSERT INTO "ql3"."cluster_legacy_env_migration_plans" (
|
||||
plan_id, mutation_id, project_id, plan_digest,
|
||||
reconciliation_bundle_digest, decision_digest,
|
||||
candidate_set_digest, source_row_count, active_row_count,
|
||||
disabled_row_count, effective_binding_count, secret_ref,
|
||||
task_revision_set_digest, trigger_revision_set_digest,
|
||||
task_count, trigger_count, total_effective_bytes,
|
||||
planned_at_ms, plan_json
|
||||
) VALUES (
|
||||
$1, $2, $3, $4, $5, $6, $7, $8, $9, $10,
|
||||
$11, $12, $13, $14, $15, $16, $17, $18, $19::jsonb
|
||||
)`,
|
||||
[
|
||||
plan.planId,
|
||||
plan.mutationId,
|
||||
plan.projectId,
|
||||
plan.planDigest,
|
||||
plan.source.reconciliationBundleDigest,
|
||||
plan.source.decisionDigest,
|
||||
plan.source.candidateSetDigest,
|
||||
plan.source.sourceRowCount,
|
||||
plan.source.activeRowCount,
|
||||
plan.source.disabledRowCount,
|
||||
plan.source.effectiveBindingCount,
|
||||
plan.target.secretRef,
|
||||
plan.target.taskRevisionSetDigest,
|
||||
plan.target.triggerRevisionSetDigest,
|
||||
plan.target.taskCount,
|
||||
plan.target.triggerCount,
|
||||
plan.target.totalEffectiveBytes,
|
||||
plan.plannedAtMs,
|
||||
JSON.stringify(plan),
|
||||
],
|
||||
);
|
||||
|
||||
if (transactionHook) {
|
||||
try {
|
||||
const hookResult = await transactionHook(
|
||||
client,
|
||||
Object.freeze({ intent, replay: null, plan }),
|
||||
);
|
||||
if (hookResult !== undefined) {
|
||||
throw new TypeError(
|
||||
'Cluster Legacy Env migration transaction hook must not return a value',
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
transactionHookError = error;
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
await client.query('COMMIT');
|
||||
began = false;
|
||||
return Object.freeze({ status: 'created', plan });
|
||||
} catch (error) {
|
||||
if (began) await rollbackPostgresDefinitionTransaction(client);
|
||||
const state = postgresSqlState(error);
|
||||
if (
|
||||
state &&
|
||||
POSTGRES_DEFINITION_RETRYABLE_SQL_STATES.has(state) &&
|
||||
attempt + 1 < POSTGRES_DEFINITION_TRANSACTION_ATTEMPTS
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
if (error === transactionHookError && error instanceof Error) {
|
||||
throw error;
|
||||
}
|
||||
throw mappedError(error);
|
||||
} finally {
|
||||
client.release();
|
||||
}
|
||||
}
|
||||
throw unavailable();
|
||||
}
|
||||
}
|
||||
+105
@@ -0,0 +1,105 @@
|
||||
import { CAPABILITIES_V68 } from '../remote-execution/pg-0069-worker-session-management-observation';
|
||||
import { definePostgresSqlMigration } from '../migrations/sqlMigration';
|
||||
|
||||
export const CAPABILITIES_V69 = CAPABILITIES_V68.replace(
|
||||
'"cluster_execution_revision":1,',
|
||||
'"cluster_execution_revision":1,"cluster_legacy_env_migration_plan":1,',
|
||||
);
|
||||
|
||||
export const pg0070ClusterLegacyEnvMigrationPlansMigration =
|
||||
definePostgresSqlMigration({
|
||||
id: 'pg-0070-cluster-legacy-env-migration-plans',
|
||||
statements: [
|
||||
`
|
||||
CREATE TABLE "ql3"."cluster_legacy_env_migration_plans" (
|
||||
plan_id varchar(128) PRIMARY KEY,
|
||||
mutation_id varchar(128) NOT NULL,
|
||||
project_id varchar(128) NOT NULL,
|
||||
plan_digest char(64) NOT NULL,
|
||||
reconciliation_bundle_digest char(64) NOT NULL,
|
||||
decision_digest char(64) NOT NULL,
|
||||
candidate_set_digest char(64) NOT NULL,
|
||||
source_row_count integer NOT NULL,
|
||||
active_row_count integer NOT NULL,
|
||||
disabled_row_count integer NOT NULL,
|
||||
effective_binding_count integer NOT NULL,
|
||||
secret_ref varchar(512) NOT NULL,
|
||||
task_revision_set_digest char(64) NOT NULL,
|
||||
trigger_revision_set_digest char(64) NOT NULL,
|
||||
task_count integer NOT NULL,
|
||||
trigger_count integer NOT NULL,
|
||||
total_effective_bytes integer NOT NULL,
|
||||
planned_at_ms bigint NOT NULL,
|
||||
plan_json jsonb NOT NULL,
|
||||
CONSTRAINT ql3_cluster_legacy_env_plan_project_fk
|
||||
FOREIGN KEY (project_id) REFERENCES "ql3"."projects" (id)
|
||||
ON DELETE RESTRICT ON UPDATE RESTRICT,
|
||||
CONSTRAINT ql3_cluster_legacy_env_plan_identity_check CHECK (
|
||||
plan_id ~ '^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$' AND
|
||||
mutation_id ~ '^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$' AND
|
||||
project_id ~ '^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$'
|
||||
),
|
||||
CONSTRAINT ql3_cluster_legacy_env_plan_digest_check CHECK (
|
||||
plan_digest ~ '^[0-9a-f]{64}$' AND
|
||||
reconciliation_bundle_digest ~ '^[0-9a-f]{64}$' AND
|
||||
decision_digest ~ '^[0-9a-f]{64}$' AND
|
||||
candidate_set_digest ~ '^[0-9a-f]{64}$' AND
|
||||
task_revision_set_digest ~ '^[0-9a-f]{64}$' AND
|
||||
trigger_revision_set_digest ~ '^[0-9a-f]{64}$'
|
||||
),
|
||||
CONSTRAINT ql3_cluster_legacy_env_plan_source_check CHECK (
|
||||
source_row_count BETWEEN 1 AND 100000 AND
|
||||
active_row_count BETWEEN 1 AND 100000 AND
|
||||
disabled_row_count BETWEEN 0 AND 100000 AND
|
||||
source_row_count = active_row_count + disabled_row_count AND
|
||||
effective_binding_count BETWEEN 1 AND active_row_count
|
||||
),
|
||||
CONSTRAINT ql3_cluster_legacy_env_plan_target_check CHECK (
|
||||
secret_ref ~ '^qlsecret:v1:[A-Za-z0-9_-]+$' AND
|
||||
octet_length(secret_ref) BETWEEN 14 AND 512 AND
|
||||
task_count BETWEEN 1 AND 100000 AND
|
||||
trigger_count BETWEEN 0 AND 500000 AND
|
||||
total_effective_bytes BETWEEN 1 AND 65536
|
||||
),
|
||||
CONSTRAINT ql3_cluster_legacy_env_plan_time_check CHECK (
|
||||
planned_at_ms >= 0
|
||||
),
|
||||
CONSTRAINT ql3_cluster_legacy_env_plan_json_check CHECK (
|
||||
jsonb_typeof(plan_json) = 'object' AND
|
||||
octet_length(plan_json::text) BETWEEN 2 AND 8192 AND
|
||||
plan_json = jsonb_build_object(
|
||||
'schema', 'qinglong/cluster-legacy-env-migration-plan@v1',
|
||||
'planId', plan_id,
|
||||
'mutationId', mutation_id,
|
||||
'projectId', project_id,
|
||||
'source', jsonb_build_object(
|
||||
'reconciliationBundleDigest', reconciliation_bundle_digest,
|
||||
'decisionDigest', decision_digest,
|
||||
'candidateSetDigest', candidate_set_digest,
|
||||
'sourceRowCount', source_row_count,
|
||||
'activeRowCount', active_row_count,
|
||||
'disabledRowCount', disabled_row_count,
|
||||
'effectiveBindingCount', effective_binding_count
|
||||
),
|
||||
'target', jsonb_build_object(
|
||||
'secretRef', secret_ref,
|
||||
'taskRevisionSetDigest', task_revision_set_digest,
|
||||
'triggerRevisionSetDigest', trigger_revision_set_digest,
|
||||
'taskCount', task_count,
|
||||
'triggerCount', trigger_count,
|
||||
'totalEffectiveBytes', total_effective_bytes
|
||||
),
|
||||
'plannedAtMs', planned_at_ms,
|
||||
'planDigest', plan_digest
|
||||
)
|
||||
)
|
||||
)
|
||||
`.trim(),
|
||||
`CREATE UNIQUE INDEX ql3_cluster_legacy_env_plan_mutation_uidx ON "ql3"."cluster_legacy_env_migration_plans" (mutation_id)`,
|
||||
`CREATE UNIQUE INDEX ql3_cluster_legacy_env_plan_digest_uidx ON "ql3"."cluster_legacy_env_migration_plans" (plan_digest)`,
|
||||
`CREATE INDEX ql3_cluster_legacy_env_plan_project_idx ON "ql3"."cluster_legacy_env_migration_plans" (project_id, planned_at_ms, plan_id)`,
|
||||
`REVOKE ALL ON "ql3"."cluster_legacy_env_migration_plans" FROM PUBLIC`,
|
||||
`GRANT SELECT, INSERT ON "ql3"."cluster_legacy_env_migration_plans" TO ql3_automation_manager`,
|
||||
`DO $ql3$ BEGIN UPDATE "ql3"."schema_capabilities" SET contract_version = 69, migration_id = 'pg-0070-cluster-legacy-env-migration-plans', capabilities = '${CAPABILITIES_V69}'::jsonb, updated_at_ms = floor(extract(epoch FROM transaction_timestamp()) * 1000)::bigint WHERE contract_name = 'control-core' AND contract_version = 68 AND migration_id = 'pg-0069-worker-session-management-observation' AND capabilities = '${CAPABILITIES_V68}'::jsonb; IF NOT FOUND THEN RAISE EXCEPTION 'control-core capability is not at version 68' USING ERRCODE = 'check_violation'; END IF; END $ql3$`,
|
||||
],
|
||||
});
|
||||
@@ -109,6 +109,77 @@ export const projects = ql3Schema.table(
|
||||
],
|
||||
);
|
||||
|
||||
export const clusterLegacyEnvMigrationPlans = ql3Schema.table(
|
||||
'cluster_legacy_env_migration_plans',
|
||||
{
|
||||
planId: varchar('plan_id', { length: 128 }).primaryKey(),
|
||||
mutationId: varchar('mutation_id', { length: 128 }).notNull(),
|
||||
projectId: varchar('project_id', { length: 128 }).notNull(),
|
||||
planDigest: char('plan_digest', { length: 64 }).notNull(),
|
||||
reconciliationBundleDigest: char('reconciliation_bundle_digest', {
|
||||
length: 64,
|
||||
}).notNull(),
|
||||
decisionDigest: char('decision_digest', { length: 64 }).notNull(),
|
||||
candidateSetDigest: char('candidate_set_digest', { length: 64 }).notNull(),
|
||||
sourceRowCount: integer('source_row_count').notNull(),
|
||||
activeRowCount: integer('active_row_count').notNull(),
|
||||
disabledRowCount: integer('disabled_row_count').notNull(),
|
||||
effectiveBindingCount: integer('effective_binding_count').notNull(),
|
||||
secretRef: varchar('secret_ref', { length: 512 }).notNull(),
|
||||
taskRevisionSetDigest: char('task_revision_set_digest', {
|
||||
length: 64,
|
||||
}).notNull(),
|
||||
triggerRevisionSetDigest: char('trigger_revision_set_digest', {
|
||||
length: 64,
|
||||
}).notNull(),
|
||||
taskCount: integer('task_count').notNull(),
|
||||
triggerCount: integer('trigger_count').notNull(),
|
||||
totalEffectiveBytes: integer('total_effective_bytes').notNull(),
|
||||
plannedAtMs: bigint('planned_at_ms', { mode: 'number' }).notNull(),
|
||||
planJson: jsonb('plan_json').$type<Record<string, unknown>>().notNull(),
|
||||
},
|
||||
(table) => [
|
||||
foreignKey({
|
||||
name: 'ql3_cluster_legacy_env_plan_project_fk',
|
||||
columns: [table.projectId],
|
||||
foreignColumns: [projects.id],
|
||||
}).onDelete('restrict'),
|
||||
check(
|
||||
'ql3_cluster_legacy_env_plan_identity_check',
|
||||
sql`${table.planId} ~ '^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$' and ${table.mutationId} ~ '^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$' and ${table.projectId} ~ '^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$'`,
|
||||
),
|
||||
check(
|
||||
'ql3_cluster_legacy_env_plan_digest_check',
|
||||
sql`${table.planDigest} ~ '^[0-9a-f]{64}$' and ${table.reconciliationBundleDigest} ~ '^[0-9a-f]{64}$' and ${table.decisionDigest} ~ '^[0-9a-f]{64}$' and ${table.candidateSetDigest} ~ '^[0-9a-f]{64}$' and ${table.taskRevisionSetDigest} ~ '^[0-9a-f]{64}$' and ${table.triggerRevisionSetDigest} ~ '^[0-9a-f]{64}$'`,
|
||||
),
|
||||
check(
|
||||
'ql3_cluster_legacy_env_plan_source_check',
|
||||
sql`${table.sourceRowCount} between 1 and 100000 and ${table.activeRowCount} between 1 and 100000 and ${table.disabledRowCount} between 0 and 100000 and ${table.sourceRowCount} = ${table.activeRowCount} + ${table.disabledRowCount} and ${table.effectiveBindingCount} between 1 and ${table.activeRowCount}`,
|
||||
),
|
||||
check(
|
||||
'ql3_cluster_legacy_env_plan_target_check',
|
||||
sql`${table.secretRef} ~ '^qlsecret:v1:[A-Za-z0-9_-]+$' and octet_length(${table.secretRef}) between 14 and 512 and ${table.taskCount} between 1 and 100000 and ${table.triggerCount} between 0 and 500000 and ${table.totalEffectiveBytes} between 1 and 65536`,
|
||||
),
|
||||
check(
|
||||
'ql3_cluster_legacy_env_plan_time_check',
|
||||
sql`${table.plannedAtMs} >= 0`,
|
||||
),
|
||||
check(
|
||||
'ql3_cluster_legacy_env_plan_json_check',
|
||||
sql`jsonb_typeof(${table.planJson}) = 'object' and octet_length(${table.planJson}::text) between 2 and 8192 and ${table.planJson} = jsonb_build_object('schema', 'qinglong/cluster-legacy-env-migration-plan@v1', 'planId', ${table.planId}, 'mutationId', ${table.mutationId}, 'projectId', ${table.projectId}, 'source', jsonb_build_object('reconciliationBundleDigest', ${table.reconciliationBundleDigest}, 'decisionDigest', ${table.decisionDigest}, 'candidateSetDigest', ${table.candidateSetDigest}, 'sourceRowCount', ${table.sourceRowCount}, 'activeRowCount', ${table.activeRowCount}, 'disabledRowCount', ${table.disabledRowCount}, 'effectiveBindingCount', ${table.effectiveBindingCount}), 'target', jsonb_build_object('secretRef', ${table.secretRef}, 'taskRevisionSetDigest', ${table.taskRevisionSetDigest}, 'triggerRevisionSetDigest', ${table.triggerRevisionSetDigest}, 'taskCount', ${table.taskCount}, 'triggerCount', ${table.triggerCount}, 'totalEffectiveBytes', ${table.totalEffectiveBytes}), 'plannedAtMs', ${table.plannedAtMs}, 'planDigest', ${table.planDigest})`,
|
||||
),
|
||||
uniqueIndex('ql3_cluster_legacy_env_plan_mutation_uidx').on(
|
||||
table.mutationId,
|
||||
),
|
||||
uniqueIndex('ql3_cluster_legacy_env_plan_digest_uidx').on(table.planDigest),
|
||||
index('ql3_cluster_legacy_env_plan_project_idx').on(
|
||||
table.projectId,
|
||||
table.plannedAtMs,
|
||||
table.planId,
|
||||
),
|
||||
],
|
||||
);
|
||||
|
||||
export const pluginPackageInstalls = ql3Schema.table(
|
||||
'plugin_package_installs',
|
||||
{
|
||||
@@ -520,7 +591,9 @@ export const pluginPackageSecretBindingTransitionApprovalPlans =
|
||||
'plugin_package_secret_binding_transition_approval_plans',
|
||||
{
|
||||
actionRef: varchar('action_ref', { length: 255 }).primaryKey(),
|
||||
approvalPlanDigest: char('approval_plan_digest', { length: 64 }).notNull(),
|
||||
approvalPlanDigest: char('approval_plan_digest', {
|
||||
length: 64,
|
||||
}).notNull(),
|
||||
transitionDigest: char('transition_digest', { length: 64 }).notNull(),
|
||||
generationDigest: char('generation_digest', { length: 64 }).notNull(),
|
||||
projectId: varchar('project_id', { length: 128 }).notNull(),
|
||||
@@ -1045,7 +1118,9 @@ export const approvedActionManualRecoveryResolutions = ql3Schema.table(
|
||||
resolvedById: varchar('resolved_by_id', { length: 255 }).notNull(),
|
||||
authenticationId: varchar('authentication_id', { length: 128 }).notNull(),
|
||||
assurance: varchar('assurance', { length: 32 }).notNull(),
|
||||
authenticatedAtMs: bigint('authenticated_at_ms', { mode: 'number' }).notNull(),
|
||||
authenticatedAtMs: bigint('authenticated_at_ms', {
|
||||
mode: 'number',
|
||||
}).notNull(),
|
||||
projectVersion: integer('project_version').notNull(),
|
||||
bindingVersion: integer('binding_version').notNull(),
|
||||
auditEventId: uuid('audit_event_id').notNull(),
|
||||
@@ -6213,6 +6288,7 @@ export const ql3PostgresTables = [
|
||||
schemaMigrations,
|
||||
schemaCapabilities,
|
||||
projects,
|
||||
clusterLegacyEnvMigrationPlans,
|
||||
pluginPackageInstalls,
|
||||
pluginPackageInstallHeads,
|
||||
pluginPackageInstallMutations,
|
||||
|
||||
@@ -21,8 +21,8 @@ export interface PostgresSchemaContractTrigger {
|
||||
export interface PostgresSchemaContract {
|
||||
readonly schema: 'ql3';
|
||||
readonly contractName: 'control-core';
|
||||
readonly contractVersion: 68;
|
||||
readonly migrationId: 'pg-0069-worker-session-management-observation';
|
||||
readonly contractVersion: 69;
|
||||
readonly migrationId: 'pg-0070-cluster-legacy-env-migration-plans';
|
||||
readonly minimumServerMajor: 16;
|
||||
readonly maximumServerMajor: 18;
|
||||
readonly capabilities: Readonly<{
|
||||
@@ -50,6 +50,7 @@ export interface PostgresSchemaContract {
|
||||
cluster_scheduler_admission: 1;
|
||||
database_role_grants: 1;
|
||||
cluster_execution_revision: 1;
|
||||
cluster_legacy_env_migration_plan: 1;
|
||||
identity_admin: 1;
|
||||
plugin_package_admission: 1;
|
||||
plugin_package_authority_split: 1;
|
||||
@@ -122,8 +123,8 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract =
|
||||
Object.freeze({
|
||||
schema: 'ql3',
|
||||
contractName: 'control-core',
|
||||
contractVersion: 68,
|
||||
migrationId: 'pg-0069-worker-session-management-observation',
|
||||
contractVersion: 69,
|
||||
migrationId: 'pg-0070-cluster-legacy-env-migration-plans',
|
||||
minimumServerMajor: 16,
|
||||
maximumServerMajor: 18,
|
||||
capabilities: Object.freeze({
|
||||
@@ -137,6 +138,7 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract =
|
||||
automation_management_boundary: 1,
|
||||
automation_management_identity_keyset_ledger: 1,
|
||||
cluster_execution_revision: 1,
|
||||
cluster_legacy_env_migration_plan: 1,
|
||||
cluster_recovery: 1,
|
||||
cluster_recovery_claim: 1,
|
||||
cluster_scheduler_admission: 1,
|
||||
@@ -228,6 +230,27 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract =
|
||||
'created_at_ms',
|
||||
'updated_at_ms',
|
||||
]),
|
||||
table('cluster_legacy_env_migration_plans', [
|
||||
'plan_id',
|
||||
'mutation_id',
|
||||
'project_id',
|
||||
'plan_digest',
|
||||
'reconciliation_bundle_digest',
|
||||
'decision_digest',
|
||||
'candidate_set_digest',
|
||||
'source_row_count',
|
||||
'active_row_count',
|
||||
'disabled_row_count',
|
||||
'effective_binding_count',
|
||||
'secret_ref',
|
||||
'task_revision_set_digest',
|
||||
'trigger_revision_set_digest',
|
||||
'task_count',
|
||||
'trigger_count',
|
||||
'total_effective_bytes',
|
||||
'planned_at_ms',
|
||||
'plan_json',
|
||||
]),
|
||||
table('plugin_package_installs', [
|
||||
'installation_id',
|
||||
'project_id',
|
||||
@@ -1539,6 +1562,10 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract =
|
||||
'schema_capabilities_pkey',
|
||||
'projects_pkey',
|
||||
'ql3_projects_slug_uidx',
|
||||
'cluster_legacy_env_migration_plans_pkey',
|
||||
'ql3_cluster_legacy_env_plan_mutation_uidx',
|
||||
'ql3_cluster_legacy_env_plan_digest_uidx',
|
||||
'ql3_cluster_legacy_env_plan_project_idx',
|
||||
'plugin_package_installs_pkey',
|
||||
'ql3_plugin_package_installs_quarantine_target_key',
|
||||
'ql3_plugin_package_installs_recovery_idx',
|
||||
@@ -1863,6 +1890,12 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract =
|
||||
'ql3_projects_version_check',
|
||||
'ql3_projects_created_at_check',
|
||||
'ql3_projects_updated_at_check',
|
||||
'ql3_cluster_legacy_env_plan_identity_check',
|
||||
'ql3_cluster_legacy_env_plan_digest_check',
|
||||
'ql3_cluster_legacy_env_plan_source_check',
|
||||
'ql3_cluster_legacy_env_plan_target_check',
|
||||
'ql3_cluster_legacy_env_plan_time_check',
|
||||
'ql3_cluster_legacy_env_plan_json_check',
|
||||
'ql3_plugin_package_installs_identity_check',
|
||||
'ql3_plugin_package_installs_operation_check',
|
||||
'ql3_plugin_package_installs_state_check',
|
||||
@@ -2342,6 +2375,7 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract =
|
||||
]),
|
||||
foreignKeys: Object.freeze([
|
||||
'ql3_schema_capabilities_migration_fk',
|
||||
'ql3_cluster_legacy_env_plan_project_fk',
|
||||
'ql3_plugin_package_installs_project_fk',
|
||||
'ql3_plugin_package_install_heads_project_fk',
|
||||
'ql3_plugin_package_install_heads_install_fk',
|
||||
|
||||
@@ -150,6 +150,12 @@ const REQUIRED_RUNTIME_PRIVILEGES = Object.freeze({
|
||||
update: true,
|
||||
delete: false,
|
||||
}),
|
||||
cluster_legacy_env_migration_plans: Object.freeze({
|
||||
select: false,
|
||||
insert: false,
|
||||
update: false,
|
||||
delete: false,
|
||||
}),
|
||||
plugin_package_installs: Object.freeze({
|
||||
select: false,
|
||||
insert: false,
|
||||
@@ -699,6 +705,12 @@ const REQUIRED_ADMIN_PRIVILEGES = Object.freeze({
|
||||
update: false,
|
||||
delete: false,
|
||||
}),
|
||||
cluster_legacy_env_migration_plans: Object.freeze({
|
||||
select: false,
|
||||
insert: false,
|
||||
update: false,
|
||||
delete: false,
|
||||
}),
|
||||
plugin_package_installs: Object.freeze({
|
||||
select: false,
|
||||
insert: false,
|
||||
@@ -1417,6 +1429,7 @@ const REQUIRED_AUTOMATION_MANAGER_PRIVILEGES: RequiredPrivileges =
|
||||
update: true,
|
||||
}
|
||||
: name === 'security_audit_events' ||
|
||||
name === 'cluster_legacy_env_migration_plans' ||
|
||||
name === 'task_definition_revisions' ||
|
||||
name === 'task_execution_revisions' ||
|
||||
name === 'trigger_revisions'
|
||||
|
||||
@@ -0,0 +1,286 @@
|
||||
const assert = require('node:assert/strict');
|
||||
const { test } = require('node:test');
|
||||
|
||||
const {
|
||||
ClusterLegacyEnvMigrationPlanConflictError,
|
||||
ClusterLegacyEnvMigrationPlanUnavailableError,
|
||||
} = require('@qinglong/runtime-core/cluster-legacy-env-migration-plan');
|
||||
const { createSecretRef } = require('@qinglong/runtime-core/secret-reference');
|
||||
const {
|
||||
PostgresClusterLegacyEnvMigrationPlanRepository,
|
||||
} = require('@qinglong/cluster-postgres/cluster-legacy-env-migration-plan');
|
||||
|
||||
function intent(overrides = {}) {
|
||||
const projectId = overrides.projectId ?? 'project-a';
|
||||
return {
|
||||
planId: 'legacy-env-plan-a',
|
||||
mutationId: 'legacy-env-mutation-a',
|
||||
projectId,
|
||||
source: {
|
||||
reconciliationBundleDigest: '1'.repeat(64),
|
||||
decisionDigest: '2'.repeat(64),
|
||||
candidateSetDigest: '3'.repeat(64),
|
||||
sourceRowCount: 3,
|
||||
activeRowCount: 2,
|
||||
disabledRowCount: 1,
|
||||
effectiveBindingCount: 2,
|
||||
},
|
||||
target: {
|
||||
secretRef: createSecretRef({
|
||||
projectId,
|
||||
name: 'legacy-env-bundle',
|
||||
version: 7,
|
||||
}),
|
||||
taskRevisionSetDigest: '4'.repeat(64),
|
||||
triggerRevisionSetDigest: '5'.repeat(64),
|
||||
taskCount: 2,
|
||||
triggerCount: 3,
|
||||
totalEffectiveBytes: 1024,
|
||||
},
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function fixture(options = {}) {
|
||||
const plansById = new Map();
|
||||
const plansByMutation = new Map();
|
||||
const queries = [];
|
||||
let serializationFailures = options.serializationFailures ?? 0;
|
||||
let connections = 0;
|
||||
|
||||
const pool = {
|
||||
async query(text, values) {
|
||||
queries.push({ scope: 'pool', text, values });
|
||||
if (text.includes('WHERE plan_id')) {
|
||||
const plan = plansById.get(values[0]);
|
||||
return {
|
||||
rows: plan ? [{ planJson: plan }] : [],
|
||||
rowCount: plan ? 1 : 0,
|
||||
};
|
||||
}
|
||||
throw new Error('unexpected pool query');
|
||||
},
|
||||
async connect() {
|
||||
connections += 1;
|
||||
return {
|
||||
async query(text, values) {
|
||||
queries.push({ scope: 'client', text, values });
|
||||
if (
|
||||
text === 'BEGIN ISOLATION LEVEL SERIALIZABLE' ||
|
||||
text === 'COMMIT' ||
|
||||
text === 'ROLLBACK' ||
|
||||
text.includes("set_config('")
|
||||
) {
|
||||
return { rows: [], rowCount: 0 };
|
||||
}
|
||||
if (text.includes('WHERE mutation_id')) {
|
||||
if (serializationFailures > 0) {
|
||||
serializationFailures -= 1;
|
||||
throw Object.assign(new Error('serialization retry'), {
|
||||
code: '40001',
|
||||
});
|
||||
}
|
||||
const plan = plansByMutation.get(values[0]);
|
||||
return {
|
||||
rows: plan ? [{ planJson: plan }] : [],
|
||||
rowCount: plan ? 1 : 0,
|
||||
};
|
||||
}
|
||||
if (text.includes('FROM "ql3"."projects"')) {
|
||||
return options.projectStatus === 'archived'
|
||||
? { rows: [{ status: 'archived' }], rowCount: 1 }
|
||||
: { rows: [{ status: 'active' }], rowCount: 1 };
|
||||
}
|
||||
if (text.includes('WHERE plan_id')) {
|
||||
const plan = plansById.get(values[0]);
|
||||
return {
|
||||
rows: plan ? [{ planJson: plan }] : [],
|
||||
rowCount: plan ? 1 : 0,
|
||||
};
|
||||
}
|
||||
if (text.includes('transaction_timestamp')) {
|
||||
return { rows: [{ plannedAtMs: '12345' }], rowCount: 1 };
|
||||
}
|
||||
if (text.includes('INSERT INTO')) {
|
||||
const plan = JSON.parse(values[18]);
|
||||
plansById.set(plan.planId, plan);
|
||||
plansByMutation.set(plan.mutationId, plan);
|
||||
return { rows: [], rowCount: 1 };
|
||||
}
|
||||
if (text === 'SELECT hook_boundary') {
|
||||
return { rows: [], rowCount: 0 };
|
||||
}
|
||||
throw new Error(`unexpected client query: ${text}`);
|
||||
},
|
||||
release() {
|
||||
queries.push({ scope: 'client', text: 'RELEASE' });
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
return {
|
||||
repository: new PostgresClusterLegacyEnvMigrationPlanRepository(pool),
|
||||
plansById,
|
||||
plansByMutation,
|
||||
queries,
|
||||
connectionCount: () => connections,
|
||||
};
|
||||
}
|
||||
|
||||
test('publishes and exactly replays one content-free plan in serializable transactions', async () => {
|
||||
const state = fixture();
|
||||
const hookContexts = [];
|
||||
const hook = async (client, context) => {
|
||||
hookContexts.push(context);
|
||||
await client.query('SELECT hook_boundary');
|
||||
};
|
||||
const created = await state.repository.publish(intent(), hook);
|
||||
const replay = await state.repository.publish(intent(), hook);
|
||||
|
||||
assert.equal(created.status, 'created');
|
||||
assert.equal(replay.status, 'existing');
|
||||
assert.deepEqual(replay.plan, created.plan);
|
||||
assert.equal(state.plansById.size, 1);
|
||||
assert.equal(hookContexts[0].replay, null);
|
||||
assert.deepEqual(hookContexts[1].replay, created.plan);
|
||||
assert.equal(
|
||||
state.queries.filter(
|
||||
({ text }) => text === 'BEGIN ISOLATION LEVEL SERIALIZABLE',
|
||||
).length,
|
||||
2,
|
||||
);
|
||||
assert.equal(state.queries.filter(({ text }) => text === 'COMMIT').length, 2);
|
||||
|
||||
const insert = state.queries.find(({ text }) => text.includes('INSERT INTO'));
|
||||
const encodedPlan = insert.values[18];
|
||||
assert.doesNotMatch(encodedPlan, /TOKEN|secretValue|ciphertext|keyId/i);
|
||||
assert.equal(
|
||||
JSON.parse(encodedPlan).target.secretRef,
|
||||
intent().target.secretRef,
|
||||
);
|
||||
});
|
||||
|
||||
test('rejects mutation replay drift and inactive Projects without writing', async () => {
|
||||
const replayState = fixture();
|
||||
await replayState.repository.publish(intent());
|
||||
await assert.rejects(
|
||||
replayState.repository.publish(
|
||||
intent({
|
||||
target: { ...intent().target, taskCount: 3 },
|
||||
}),
|
||||
),
|
||||
ClusterLegacyEnvMigrationPlanConflictError,
|
||||
);
|
||||
assert.equal(
|
||||
replayState.queries.filter(({ text }) => text.includes('INSERT INTO'))
|
||||
.length,
|
||||
1,
|
||||
);
|
||||
assert.equal(
|
||||
replayState.queries.some(({ text }) => text === 'ROLLBACK'),
|
||||
true,
|
||||
);
|
||||
|
||||
const archivedState = fixture({ projectStatus: 'archived' });
|
||||
await assert.rejects(
|
||||
archivedState.repository.publish(intent()),
|
||||
ClusterLegacyEnvMigrationPlanConflictError,
|
||||
);
|
||||
assert.equal(
|
||||
archivedState.queries.some(({ text }) => text.includes('INSERT INTO')),
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
test('retries bounded serializable failures and preserves the transaction hook error', async () => {
|
||||
const state = fixture({ serializationFailures: 1 });
|
||||
const created = await state.repository.publish(intent());
|
||||
assert.equal(created.status, 'created');
|
||||
assert.equal(state.connectionCount(), 2);
|
||||
assert.equal(
|
||||
state.queries.filter(({ text }) => text === 'ROLLBACK').length,
|
||||
1,
|
||||
);
|
||||
|
||||
const hookError = new Error('caller hook failed');
|
||||
await assert.rejects(
|
||||
fixture().repository.publish(intent(), async () => {
|
||||
throw hookError;
|
||||
}),
|
||||
(error) => error === hookError,
|
||||
);
|
||||
});
|
||||
|
||||
test('fails closed on malformed durable JSON and hides raw storage errors', async () => {
|
||||
const malformedPool = {
|
||||
async query() {
|
||||
return { rows: [{ planJson: { schema: 'wrong' } }], rowCount: 1 };
|
||||
},
|
||||
async connect() {
|
||||
throw new Error('unused');
|
||||
},
|
||||
};
|
||||
await assert.rejects(
|
||||
new PostgresClusterLegacyEnvMigrationPlanRepository(
|
||||
malformedPool,
|
||||
).findByPlanId('legacy-env-plan-a'),
|
||||
ClusterLegacyEnvMigrationPlanUnavailableError,
|
||||
);
|
||||
|
||||
const identityDrift = fixture();
|
||||
const created = await identityDrift.repository.publish(intent());
|
||||
identityDrift.plansById.set('legacy-env-plan-b', created.plan);
|
||||
await assert.rejects(
|
||||
identityDrift.repository.findByPlanId('legacy-env-plan-b'),
|
||||
ClusterLegacyEnvMigrationPlanUnavailableError,
|
||||
);
|
||||
identityDrift.plansByMutation.set('legacy-env-mutation-b', created.plan);
|
||||
await assert.rejects(
|
||||
identityDrift.repository.publish(
|
||||
intent({
|
||||
planId: 'legacy-env-plan-b',
|
||||
mutationId: 'legacy-env-mutation-b',
|
||||
}),
|
||||
),
|
||||
ClusterLegacyEnvMigrationPlanUnavailableError,
|
||||
);
|
||||
|
||||
const failedPool = {
|
||||
async query() {
|
||||
throw new Error('password=do-not-leak');
|
||||
},
|
||||
async connect() {
|
||||
throw new Error('unused');
|
||||
},
|
||||
};
|
||||
await assert.rejects(
|
||||
new PostgresClusterLegacyEnvMigrationPlanRepository(
|
||||
failedPool,
|
||||
).findByPlanId('legacy-env-plan-a'),
|
||||
(error) => {
|
||||
assert.ok(error instanceof ClusterLegacyEnvMigrationPlanUnavailableError);
|
||||
assert.doesNotMatch(error.message, /password|do-not-leak/i);
|
||||
return true;
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test('keeps the append authority behind its explicit package subpath', () => {
|
||||
const root = require('@qinglong/cluster-postgres');
|
||||
const runtime = require('@qinglong/cluster-postgres/runtime');
|
||||
const admin = require('@qinglong/cluster-postgres/admin');
|
||||
const authority = require('@qinglong/cluster-postgres/cluster-legacy-env-migration-plan');
|
||||
assert.equal(root.PostgresClusterLegacyEnvMigrationPlanRepository, undefined);
|
||||
assert.equal(
|
||||
runtime.PostgresClusterLegacyEnvMigrationPlanRepository,
|
||||
undefined,
|
||||
);
|
||||
assert.equal(
|
||||
admin.PostgresClusterLegacyEnvMigrationPlanRepository,
|
||||
undefined,
|
||||
);
|
||||
assert.equal(
|
||||
typeof authority.PostgresClusterLegacyEnvMigrationPlanRepository,
|
||||
'function',
|
||||
);
|
||||
});
|
||||
@@ -71,6 +71,9 @@ const {
|
||||
const {
|
||||
resolveClusterScheduleDecision,
|
||||
} = require('@qinglong/runtime-core/cluster-scheduler');
|
||||
const {
|
||||
PostgresClusterLegacyEnvMigrationPlanRepository,
|
||||
} = require('../dist/reconciliation/clusterLegacyEnvMigrationPlanRepository');
|
||||
|
||||
function nextMinute(schedule, afterMs) {
|
||||
if (schedule.expression !== '* * * * *' || schedule.timezone !== 'UTC') {
|
||||
@@ -877,7 +880,10 @@ if (!migrationConnectionString) {
|
||||
assert.equal(competing?.status, 'leased');
|
||||
assert.equal(claimed.dispatch.version, 1);
|
||||
assert.equal(claimed.dispatch.dispatchCount, 1);
|
||||
assert.equal(claimed.dispatch.createdAtMs >= Number(before.rows[0].nowMs), true);
|
||||
assert.equal(
|
||||
claimed.dispatch.createdAtMs >= Number(before.rows[0].nowMs),
|
||||
true,
|
||||
);
|
||||
const rawLeaseToken = claimed.leaseToken;
|
||||
const stored = await migrationDatabase.pool.query(
|
||||
`SELECT lease_token_digest AS "leaseTokenDigest",
|
||||
@@ -963,11 +969,13 @@ if (!migrationConnectionString) {
|
||||
assert.equal(retry.dispatch.status, 'retry_wait');
|
||||
assert.equal(retry.event.type, 'run.cancel_dispatch_failed');
|
||||
assert.equal(
|
||||
(await firstRepository.claim({
|
||||
...candidate,
|
||||
owner: 'primary-a',
|
||||
leaseToken: 'lease-a-retry',
|
||||
})).status,
|
||||
(
|
||||
await firstRepository.claim({
|
||||
...candidate,
|
||||
owner: 'primary-a',
|
||||
leaseToken: 'lease-a-retry',
|
||||
})
|
||||
).status,
|
||||
'not_due',
|
||||
);
|
||||
await migrationDatabase.pool.query(
|
||||
@@ -2199,9 +2207,7 @@ if (!migrationConnectionString) {
|
||||
const migrationDatabase = await open('migration');
|
||||
try {
|
||||
await runPostgresMigrations({ pool: migrationDatabase.pool });
|
||||
await migrationDatabase.pool.query(
|
||||
'TRUNCATE TABLE "ql3"."runs" CASCADE',
|
||||
);
|
||||
await migrationDatabase.pool.query('TRUNCATE TABLE "ql3"."runs" CASCADE');
|
||||
await observeContractPublisherTrust(migrationDatabase.pool);
|
||||
await migrationDatabase.pool.query(
|
||||
`INSERT INTO "ql3"."projects" (
|
||||
@@ -3847,12 +3853,13 @@ if (!migrationConnectionString) {
|
||||
const executions = new PostgresApprovedActionExecutionRepository(
|
||||
executorDatabase.pool,
|
||||
);
|
||||
const pendingSecretActions =
|
||||
await executions.listReconciliableExecutions({
|
||||
const pendingSecretActions = await executions.listReconciliableExecutions(
|
||||
{
|
||||
nowMs: claimedAtMs,
|
||||
limit: 1,
|
||||
actionTypes: [consumed.dispatch.action.actionType],
|
||||
});
|
||||
},
|
||||
);
|
||||
assert.equal(pendingSecretActions.truncated, false);
|
||||
assert.equal(pendingSecretActions.executions.length, 1);
|
||||
assert.equal(
|
||||
@@ -6532,4 +6539,136 @@ if (!migrationConnectionString) {
|
||||
await database.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('persists one content-free Legacy Env migration plan with isolated authority', async () => {
|
||||
const projectId = `legacy-env-project-${process.pid}`;
|
||||
const planId = `legacy-env-plan-${process.pid}`;
|
||||
const mutationId = `legacy-env-mutation-${process.pid}`;
|
||||
const migrationDatabase = await open('migration');
|
||||
const automationDatabase = await open('automation-manager');
|
||||
const runtimeDatabase = await open('runtime');
|
||||
const adminDatabase = await open('admin');
|
||||
try {
|
||||
await runPostgresMigrations({ pool: migrationDatabase.pool });
|
||||
await migrationDatabase.pool.query(
|
||||
`INSERT INTO "ql3"."projects"
|
||||
(id, name, slug, status, version, created_at_ms, updated_at_ms)
|
||||
VALUES ($1, $1, $1, 'active', 1, 1, 1)
|
||||
ON CONFLICT (id) DO NOTHING`,
|
||||
[projectId],
|
||||
);
|
||||
const intent = {
|
||||
planId,
|
||||
mutationId,
|
||||
projectId,
|
||||
source: {
|
||||
reconciliationBundleDigest: '1'.repeat(64),
|
||||
decisionDigest: '2'.repeat(64),
|
||||
candidateSetDigest: '3'.repeat(64),
|
||||
sourceRowCount: 3,
|
||||
activeRowCount: 2,
|
||||
disabledRowCount: 1,
|
||||
effectiveBindingCount: 2,
|
||||
},
|
||||
target: {
|
||||
secretRef: createSecretRef({
|
||||
projectId,
|
||||
name: 'legacy-env-bundle',
|
||||
version: 1,
|
||||
}),
|
||||
taskRevisionSetDigest: '4'.repeat(64),
|
||||
triggerRevisionSetDigest: '5'.repeat(64),
|
||||
taskCount: 2,
|
||||
triggerCount: 1,
|
||||
totalEffectiveBytes: 1024,
|
||||
},
|
||||
};
|
||||
const repository = new PostgresClusterLegacyEnvMigrationPlanRepository(
|
||||
automationDatabase.pool,
|
||||
);
|
||||
const created = await repository.publish(intent);
|
||||
const replay = await repository.publish(intent);
|
||||
assert.equal(created.status, 'created');
|
||||
assert.equal(replay.status, 'existing');
|
||||
assert.deepEqual(replay.plan, created.plan);
|
||||
assert.deepEqual(await repository.findByPlanId(planId), created.plan);
|
||||
|
||||
const stored = await automationDatabase.pool.query(
|
||||
`SELECT plan_json AS "planJson"
|
||||
FROM "ql3"."cluster_legacy_env_migration_plans"
|
||||
WHERE plan_id = $1`,
|
||||
[planId],
|
||||
);
|
||||
assert.equal(stored.rowCount, 1);
|
||||
assert.deepEqual(stored.rows[0].planJson, created.plan);
|
||||
assert.doesNotMatch(
|
||||
JSON.stringify(stored.rows[0].planJson),
|
||||
/TOKEN|secretValue|ciphertext|keyId/i,
|
||||
);
|
||||
|
||||
const invalidPlanId = `${planId}-widened`;
|
||||
const invalidMutationId = `${mutationId}-widened`;
|
||||
const invalidDigest = 'f'.repeat(64);
|
||||
await assert.rejects(
|
||||
migrationDatabase.pool.query(
|
||||
`INSERT INTO "ql3"."cluster_legacy_env_migration_plans" (
|
||||
plan_id, mutation_id, project_id, plan_digest,
|
||||
reconciliation_bundle_digest, decision_digest,
|
||||
candidate_set_digest, source_row_count, active_row_count,
|
||||
disabled_row_count, effective_binding_count, secret_ref,
|
||||
task_revision_set_digest, trigger_revision_set_digest,
|
||||
task_count, trigger_count, total_effective_bytes,
|
||||
planned_at_ms, plan_json
|
||||
)
|
||||
SELECT $2::varchar, $3::varchar, project_id, $4::varchar,
|
||||
reconciliation_bundle_digest, decision_digest,
|
||||
candidate_set_digest, source_row_count, active_row_count,
|
||||
disabled_row_count, effective_binding_count, secret_ref,
|
||||
task_revision_set_digest, trigger_revision_set_digest,
|
||||
task_count, trigger_count, total_effective_bytes,
|
||||
planned_at_ms,
|
||||
plan_json || jsonb_build_object(
|
||||
'planId', $2::varchar,
|
||||
'mutationId', $3::varchar,
|
||||
'planDigest', $4::varchar,
|
||||
'envName', 'TOKEN'
|
||||
)
|
||||
FROM "ql3"."cluster_legacy_env_migration_plans"
|
||||
WHERE plan_id = $1`,
|
||||
[planId, invalidPlanId, invalidMutationId, invalidDigest],
|
||||
),
|
||||
(error) =>
|
||||
error?.code === '23514' &&
|
||||
error?.constraint === 'ql3_cluster_legacy_env_plan_json_check',
|
||||
);
|
||||
|
||||
await assert.rejects(
|
||||
automationDatabase.pool.query(
|
||||
`UPDATE "ql3"."cluster_legacy_env_migration_plans"
|
||||
SET planned_at_ms = planned_at_ms
|
||||
WHERE plan_id = $1`,
|
||||
[planId],
|
||||
),
|
||||
(error) => error?.code === '42501',
|
||||
);
|
||||
for (const database of [runtimeDatabase, adminDatabase]) {
|
||||
await assert.rejects(
|
||||
database.pool.query(
|
||||
`SELECT plan_id
|
||||
FROM "ql3"."cluster_legacy_env_migration_plans"
|
||||
WHERE plan_id = $1`,
|
||||
[planId],
|
||||
),
|
||||
(error) => error?.code === '42501',
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
await Promise.all([
|
||||
adminDatabase.close(),
|
||||
runtimeDatabase.close(),
|
||||
automationDatabase.close(),
|
||||
migrationDatabase.close(),
|
||||
]);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
@@ -120,6 +120,7 @@ test('defines the immutable PostgreSQL capability and Run core stream', async ()
|
||||
'pg-0067-cancellation-dispatch-management',
|
||||
'pg-0068-cancellation-dispatch-project-keyset',
|
||||
'pg-0069-worker-session-management-observation',
|
||||
'pg-0070-cluster-legacy-env-migration-plans',
|
||||
],
|
||||
);
|
||||
for (const migration of postgresqlMainMigrationStream.migrations) {
|
||||
@@ -603,6 +604,11 @@ test('freezes every published PostgreSQL migration checksum', () => {
|
||||
checksum:
|
||||
'1191255575589abc2686b391827607abddb4edb78007245dbaaf45dc1c4e5e8b',
|
||||
},
|
||||
{
|
||||
id: 'pg-0070-cluster-legacy-env-migration-plans',
|
||||
checksum:
|
||||
'7cd6d993f48e7bcebcd62c93571a738d5117c9bcde33b974c5ac8962e2a03fe4',
|
||||
},
|
||||
];
|
||||
assert.deepEqual(
|
||||
postgresqlMainMigrationStream.migrations.map(({ id, checksum }) => ({
|
||||
@@ -2241,11 +2247,11 @@ test('advances capability v63 with manager-only immutable Secret transition plan
|
||||
sql,
|
||||
/GRANT SELECT ON "ql3"\."plugin_package_secret_binding_transition_approval_plans" TO ql3_package_manager, ql3_package_executor/,
|
||||
);
|
||||
assert.match(
|
||||
assert.match(sql, /GRANT EXECUTE ON FUNCTION [^;]+ TO ql3_package_manager/);
|
||||
assert.doesNotMatch(
|
||||
sql,
|
||||
/GRANT EXECUTE ON FUNCTION [^;]+ TO ql3_package_manager/,
|
||||
/GRANT EXECUTE ON FUNCTION [^;]+ TO ql3_package_executor/,
|
||||
);
|
||||
assert.doesNotMatch(sql, /GRANT EXECUTE ON FUNCTION [^;]+ TO ql3_package_executor/);
|
||||
assert.match(sql, /contract_version = 63/);
|
||||
assert.match(
|
||||
sql,
|
||||
@@ -2321,10 +2327,7 @@ test('advances capability v65 with database-timed fenced cancellation dispatch',
|
||||
sql,
|
||||
/CREATE UNIQUE INDEX ql3_run_attempts_run_id_uidx ON "ql3"\."run_attempts" \(run_id, id\)/,
|
||||
);
|
||||
assert.match(
|
||||
sql,
|
||||
/CREATE TABLE "ql3"\."run_cancellation_dispatches"/,
|
||||
);
|
||||
assert.match(sql, /CREATE TABLE "ql3"\."run_cancellation_dispatches"/);
|
||||
assert.match(
|
||||
sql,
|
||||
/FOREIGN KEY \(run_id, attempt_id\)[\s\S]+REFERENCES "ql3"\."run_attempts" \(run_id, id\)/,
|
||||
@@ -2342,16 +2345,11 @@ test('advances capability v65 with database-timed fenced cancellation dispatch',
|
||||
assert.match(sql, /contract_version = 65/);
|
||||
assert.match(sql, /"run_cancellation_dispatch":1/);
|
||||
assert.match(sql, /contract_version = 64/);
|
||||
assert.match(
|
||||
sql,
|
||||
/migration_id = 'pg-0065-approved-action-manual-recovery'/,
|
||||
);
|
||||
assert.match(sql, /migration_id = 'pg-0065-approved-action-manual-recovery'/);
|
||||
});
|
||||
|
||||
test('advances capability v66 with least-privilege cancellation diagnostics and rearm', async () => {
|
||||
const migration = migrationById(
|
||||
'pg-0067-cancellation-dispatch-management',
|
||||
);
|
||||
const migration = migrationById('pg-0067-cancellation-dispatch-management');
|
||||
const statements = [];
|
||||
await migration.up({
|
||||
async query(statement) {
|
||||
@@ -2398,10 +2396,7 @@ test('advances capability v67 with a Project-scoped blocked keyset', async () =>
|
||||
},
|
||||
});
|
||||
const sql = statements.join('\n');
|
||||
assert.match(
|
||||
sql,
|
||||
/ADD COLUMN project_id varchar\(128\)/,
|
||||
);
|
||||
assert.match(sql, /ADD COLUMN project_id varchar\(128\)/);
|
||||
assert.match(
|
||||
sql,
|
||||
/SET project_id = run\.project_id FROM "ql3"\."runs" AS run/,
|
||||
@@ -2460,3 +2455,37 @@ test('advances capability v68 with read-only Worker session observation', async
|
||||
/migration_id = 'pg-0068-cancellation-dispatch-project-keyset'/,
|
||||
);
|
||||
});
|
||||
|
||||
test('advances capability v69 with a content-free Legacy Env plan ledger', async () => {
|
||||
const migration = migrationById('pg-0070-cluster-legacy-env-migration-plans');
|
||||
const statements = [];
|
||||
await migration.up({
|
||||
async query(statement) {
|
||||
statements.push(statement);
|
||||
return { rows: [] };
|
||||
},
|
||||
});
|
||||
const sql = statements.join('\n');
|
||||
assert.match(sql, /CREATE TABLE "ql3"\."cluster_legacy_env_migration_plans"/);
|
||||
assert.match(sql, /source_row_count BETWEEN 1 AND 100000/);
|
||||
assert.match(sql, /total_effective_bytes BETWEEN 1 AND 65536/);
|
||||
assert.match(
|
||||
sql,
|
||||
/plan_json = jsonb_build_object\([\s\S]+qinglong\/cluster-legacy-env-migration-plan@v1/,
|
||||
);
|
||||
assert.match(
|
||||
sql,
|
||||
/GRANT SELECT, INSERT ON "ql3"\."cluster_legacy_env_migration_plans" TO ql3_automation_manager/,
|
||||
);
|
||||
assert.doesNotMatch(
|
||||
sql,
|
||||
/GRANT (?:UPDATE|DELETE|TRUNCATE)[^;]+cluster_legacy_env_migration_plans/,
|
||||
);
|
||||
assert.match(sql, /contract_version = 69/);
|
||||
assert.match(sql, /"cluster_legacy_env_migration_plan":1/);
|
||||
assert.match(sql, /contract_version = 68/);
|
||||
assert.match(
|
||||
sql,
|
||||
/migration_id = 'pg-0069-worker-session-management-observation'/,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -33,6 +33,7 @@ function validPrivileges() {
|
||||
schema_migrations: [true, false, false, false],
|
||||
schema_capabilities: [true, false, false, false],
|
||||
projects: [true, true, true, false],
|
||||
cluster_legacy_env_migration_plans: [false, false, false, false],
|
||||
task_definitions: [true, false, false, false],
|
||||
task_definition_revisions: [true, false, false, false],
|
||||
task_execution_revisions: [true, false, false, false],
|
||||
@@ -167,6 +168,7 @@ function validAdminPrivileges() {
|
||||
schema_migrations: [true, false, false, false],
|
||||
schema_capabilities: [true, false, false, false],
|
||||
projects: [true, false, false, false],
|
||||
cluster_legacy_env_migration_plans: [false, false, false, false],
|
||||
task_definitions: [false, false, false, false],
|
||||
task_definition_revisions: [false, false, false, false],
|
||||
task_execution_revisions: [false, false, false, false],
|
||||
@@ -454,6 +456,7 @@ function automationManagerPrivileges() {
|
||||
'project_role_bindings',
|
||||
'plugin_package_task_ownerships',
|
||||
'plugin_package_identity_keyset_ledger',
|
||||
'cluster_legacy_env_migration_plans',
|
||||
'security_audit_events',
|
||||
'task_definitions',
|
||||
'task_definition_revisions',
|
||||
@@ -471,6 +474,7 @@ function automationManagerPrivileges() {
|
||||
'trigger_revisions',
|
||||
'trigger_schedules',
|
||||
'plugin_package_identity_keyset_ledger',
|
||||
'cluster_legacy_env_migration_plans',
|
||||
]);
|
||||
return postgresqlControlSchemaContract.tables.map(({ name: tableName }) => ({
|
||||
tableName,
|
||||
@@ -791,9 +795,7 @@ function queryable(overrides = {}) {
|
||||
: 'runs';
|
||||
assert.match(
|
||||
text,
|
||||
new RegExp(
|
||||
`format\\('%I\\.%I', \\$1::text, '${tableName}'\\)`,
|
||||
),
|
||||
new RegExp(`format\\('%I\\.%I', \\$1::text, '${tableName}'\\)`),
|
||||
);
|
||||
const columns = contract.tables.find(
|
||||
({ name }) => name === tableName,
|
||||
@@ -836,7 +838,7 @@ test('accepts the exact PostgreSQL control schema and least-privilege runtime ro
|
||||
serverMajor: 16,
|
||||
currentUser: 'ql3_runtime',
|
||||
contractName: 'control-core',
|
||||
contractVersion: 68,
|
||||
contractVersion: 69,
|
||||
migrationIds: [
|
||||
'pg-0001-schema-capability',
|
||||
'pg-0002-run-core',
|
||||
@@ -907,6 +909,7 @@ test('accepts the exact PostgreSQL control schema and least-privilege runtime ro
|
||||
'pg-0067-cancellation-dispatch-management',
|
||||
'pg-0068-cancellation-dispatch-project-keyset',
|
||||
'pg-0069-worker-session-management-observation',
|
||||
'pg-0070-cluster-legacy-env-migration-plans',
|
||||
],
|
||||
});
|
||||
});
|
||||
@@ -937,10 +940,10 @@ test('accepts the exact schema and isolated least-privilege admin role', async (
|
||||
}),
|
||||
);
|
||||
assert.equal(report.currentUser, 'ql3_admin');
|
||||
assert.equal(report.contractVersion, 68);
|
||||
assert.equal(report.contractVersion, 69);
|
||||
assert.equal(
|
||||
report.migrationIds.at(-1),
|
||||
'pg-0069-worker-session-management-observation',
|
||||
'pg-0070-cluster-legacy-env-migration-plans',
|
||||
);
|
||||
});
|
||||
|
||||
@@ -953,10 +956,10 @@ test('accepts the isolated least-privilege automation manager role', async () =>
|
||||
}),
|
||||
);
|
||||
assert.equal(report.currentUser, 'ql3_automation_manager');
|
||||
assert.equal(report.contractVersion, 68);
|
||||
assert.equal(report.contractVersion, 69);
|
||||
assert.equal(
|
||||
report.migrationIds.at(-1),
|
||||
'pg-0069-worker-session-management-observation',
|
||||
'pg-0070-cluster-legacy-env-migration-plans',
|
||||
);
|
||||
|
||||
const widened = automationManagerPrivileges();
|
||||
@@ -985,10 +988,10 @@ test('accepts the isolated least-privilege human Approval manager role', async (
|
||||
}),
|
||||
);
|
||||
assert.equal(report.currentUser, 'ql3_approval_manager');
|
||||
assert.equal(report.contractVersion, 68);
|
||||
assert.equal(report.contractVersion, 69);
|
||||
assert.equal(
|
||||
report.migrationIds.at(-1),
|
||||
'pg-0069-worker-session-management-observation',
|
||||
'pg-0070-cluster-legacy-env-migration-plans',
|
||||
);
|
||||
|
||||
const widened = approvalManagerPrivileges();
|
||||
@@ -1019,10 +1022,10 @@ test('accepts the isolated least-privilege Run manager role', async () => {
|
||||
}),
|
||||
);
|
||||
assert.equal(report.currentUser, 'ql3_run_manager');
|
||||
assert.equal(report.contractVersion, 68);
|
||||
assert.equal(report.contractVersion, 69);
|
||||
assert.equal(
|
||||
report.migrationIds.at(-1),
|
||||
'pg-0069-worker-session-management-observation',
|
||||
'pg-0070-cluster-legacy-env-migration-plans',
|
||||
);
|
||||
|
||||
const widened = runManagerPrivileges();
|
||||
@@ -1183,10 +1186,10 @@ test('accepts the exact schema and isolated Worker ingress role', async () => {
|
||||
}),
|
||||
);
|
||||
assert.equal(report.currentUser, 'ql3_worker_ingress');
|
||||
assert.equal(report.contractVersion, 68);
|
||||
assert.equal(report.contractVersion, 69);
|
||||
assert.equal(
|
||||
report.migrationIds.at(-1),
|
||||
'pg-0069-worker-session-management-observation',
|
||||
'pg-0070-cluster-legacy-env-migration-plans',
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -260,6 +260,9 @@
|
||||
"builtin-run-log-excerpt-projection": [
|
||||
"dist/tool-execution/builtin-run-log-excerpt/builtInRunLogExcerptProjection.d.ts"
|
||||
],
|
||||
"cluster-legacy-env-migration-plan": [
|
||||
"dist/migration/clusterLegacyEnvMigrationPlan.d.ts"
|
||||
],
|
||||
"run": [
|
||||
"dist/run/run.d.ts"
|
||||
],
|
||||
@@ -727,6 +730,11 @@
|
||||
"require": "./dist/secret/secretReference.js",
|
||||
"default": "./dist/secret/secretReference.js"
|
||||
},
|
||||
"./cluster-legacy-env-migration-plan": {
|
||||
"types": "./dist/migration/clusterLegacyEnvMigrationPlan.d.ts",
|
||||
"require": "./dist/migration/clusterLegacyEnvMigrationPlan.js",
|
||||
"default": "./dist/migration/clusterLegacyEnvMigrationPlan.js"
|
||||
},
|
||||
"./cluster-execution-revision": {
|
||||
"types": "./dist/task-definition/clusterExecutionRevision.d.ts",
|
||||
"require": "./dist/task-definition/clusterExecutionRevision.js",
|
||||
|
||||
@@ -0,0 +1,434 @@
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
import { parseSecretRef } from '../secret/secretReference';
|
||||
|
||||
export const CLUSTER_LEGACY_ENV_MIGRATION_PLAN_SCHEMA =
|
||||
'qinglong/cluster-legacy-env-migration-plan@v1' as const;
|
||||
export const MAX_CLUSTER_LEGACY_ENV_SOURCE_ROWS = 100_000;
|
||||
export const MAX_CLUSTER_LEGACY_ENV_TASKS = 100_000;
|
||||
export const MAX_CLUSTER_LEGACY_ENV_TRIGGERS = 500_000;
|
||||
export const MAX_CLUSTER_LEGACY_ENV_EFFECTIVE_BYTES = 64 * 1024;
|
||||
export const MAX_CLUSTER_LEGACY_ENV_MIGRATION_PLAN_JSON_BYTES = 8 * 1024;
|
||||
|
||||
export interface ClusterLegacyEnvMigrationSourceEvidence {
|
||||
readonly reconciliationBundleDigest: string;
|
||||
readonly decisionDigest: string;
|
||||
readonly candidateSetDigest: string;
|
||||
readonly sourceRowCount: number;
|
||||
readonly activeRowCount: number;
|
||||
readonly disabledRowCount: number;
|
||||
readonly effectiveBindingCount: number;
|
||||
}
|
||||
|
||||
export interface ClusterLegacyEnvMigrationTarget {
|
||||
readonly secretRef: string;
|
||||
readonly taskRevisionSetDigest: string;
|
||||
readonly triggerRevisionSetDigest: string;
|
||||
readonly taskCount: number;
|
||||
readonly triggerCount: number;
|
||||
readonly totalEffectiveBytes: number;
|
||||
}
|
||||
|
||||
export interface ClusterLegacyEnvMigrationPlanIntent {
|
||||
readonly planId: string;
|
||||
readonly mutationId: string;
|
||||
readonly projectId: string;
|
||||
readonly source: Readonly<ClusterLegacyEnvMigrationSourceEvidence>;
|
||||
readonly target: Readonly<ClusterLegacyEnvMigrationTarget>;
|
||||
}
|
||||
|
||||
export interface ClusterLegacyEnvMigrationPlan
|
||||
extends ClusterLegacyEnvMigrationPlanIntent {
|
||||
readonly schema: typeof CLUSTER_LEGACY_ENV_MIGRATION_PLAN_SCHEMA;
|
||||
readonly plannedAtMs: number;
|
||||
readonly planDigest: string;
|
||||
}
|
||||
|
||||
export interface ClusterLegacyEnvMigrationPlanRepository {
|
||||
publish(intent: Readonly<ClusterLegacyEnvMigrationPlanIntent>): Promise<
|
||||
Readonly<{
|
||||
status: 'created' | 'existing';
|
||||
plan: Readonly<ClusterLegacyEnvMigrationPlan>;
|
||||
}>
|
||||
>;
|
||||
findByPlanId(
|
||||
planId: string,
|
||||
): Promise<Readonly<ClusterLegacyEnvMigrationPlan> | null>;
|
||||
}
|
||||
|
||||
export class InvalidClusterLegacyEnvMigrationPlanError extends TypeError {
|
||||
readonly code = 'CLUSTER_LEGACY_ENV_MIGRATION_PLAN_INVALID';
|
||||
|
||||
constructor(message: string) {
|
||||
super(`Cluster Legacy Env migration plan is invalid: ${message}`);
|
||||
this.name = 'InvalidClusterLegacyEnvMigrationPlanError';
|
||||
}
|
||||
}
|
||||
|
||||
export class ClusterLegacyEnvMigrationPlanConflictError extends Error {
|
||||
readonly code = 'CLUSTER_LEGACY_ENV_MIGRATION_PLAN_CONFLICT';
|
||||
|
||||
constructor() {
|
||||
super('Cluster Legacy Env migration plan conflicts with durable state');
|
||||
this.name = 'ClusterLegacyEnvMigrationPlanConflictError';
|
||||
}
|
||||
}
|
||||
|
||||
export class ClusterLegacyEnvMigrationPlanUnavailableError extends Error {
|
||||
readonly code = 'CLUSTER_LEGACY_ENV_MIGRATION_PLAN_UNAVAILABLE';
|
||||
|
||||
constructor(options?: ErrorOptions) {
|
||||
super('Cluster Legacy Env migration plan is unavailable', options);
|
||||
this.name = 'ClusterLegacyEnvMigrationPlanUnavailableError';
|
||||
}
|
||||
}
|
||||
|
||||
const ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$/;
|
||||
const DIGEST_PATTERN = /^[0-9a-f]{64}$/;
|
||||
const PLAN_DIGEST_DOMAIN = Buffer.from(
|
||||
'qinglong/cluster-legacy-env-migration-plan-digest@v1\0',
|
||||
'utf8',
|
||||
);
|
||||
|
||||
function invalid(message: string): never {
|
||||
throw new InvalidClusterLegacyEnvMigrationPlanError(message);
|
||||
}
|
||||
|
||||
function record(value: unknown, label: string): Record<string, unknown> {
|
||||
if (
|
||||
!value ||
|
||||
typeof value !== 'object' ||
|
||||
Array.isArray(value) ||
|
||||
(Object.getPrototypeOf(value) !== Object.prototype &&
|
||||
Object.getPrototypeOf(value) !== null)
|
||||
) {
|
||||
return invalid(`${label} must be an object`);
|
||||
}
|
||||
const descriptors = Object.getOwnPropertyDescriptors(value);
|
||||
if (
|
||||
Object.values(descriptors).some(
|
||||
(descriptor) =>
|
||||
descriptor.get !== undefined ||
|
||||
descriptor.set !== undefined ||
|
||||
descriptor.enumerable !== true,
|
||||
)
|
||||
) {
|
||||
return invalid(`${label} must contain enumerable data properties`);
|
||||
}
|
||||
return value as Record<string, unknown>;
|
||||
}
|
||||
|
||||
function exactKeys(
|
||||
value: object,
|
||||
expected: readonly string[],
|
||||
label: string,
|
||||
): void {
|
||||
const keys = Reflect.ownKeys(value);
|
||||
const canonical = [...expected].sort();
|
||||
if (
|
||||
keys.some((key) => typeof key !== 'string') ||
|
||||
keys.length !== canonical.length ||
|
||||
keys
|
||||
.map(String)
|
||||
.sort()
|
||||
.some((key, index) => key !== canonical[index])
|
||||
) {
|
||||
invalid(`${label} shape is invalid`);
|
||||
}
|
||||
}
|
||||
|
||||
export function assertClusterLegacyEnvMigrationPlanIdentifier(
|
||||
value: unknown,
|
||||
label: 'mutationId' | 'planId' | 'projectId',
|
||||
): string {
|
||||
if (typeof value !== 'string' || !ID_PATTERN.test(value)) {
|
||||
return invalid(`${label} is invalid`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function digest(value: unknown, label: string): string {
|
||||
if (typeof value !== 'string' || !DIGEST_PATTERN.test(value)) {
|
||||
return invalid(`${label} is invalid`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function count(value: unknown, label: string, maximum: number): number {
|
||||
if (
|
||||
!Number.isSafeInteger(value) ||
|
||||
(value as number) < 0 ||
|
||||
(value as number) > maximum
|
||||
) {
|
||||
return invalid(`${label} is invalid`);
|
||||
}
|
||||
return value as number;
|
||||
}
|
||||
|
||||
function timestamp(value: unknown): number {
|
||||
if (!Number.isSafeInteger(value) || (value as number) < 0) {
|
||||
return invalid('plannedAtMs is invalid');
|
||||
}
|
||||
return value as number;
|
||||
}
|
||||
|
||||
function sourceEvidence(
|
||||
value: ClusterLegacyEnvMigrationSourceEvidence,
|
||||
): Readonly<ClusterLegacyEnvMigrationSourceEvidence> {
|
||||
const candidate = record(value, 'source');
|
||||
exactKeys(
|
||||
candidate,
|
||||
[
|
||||
'activeRowCount',
|
||||
'candidateSetDigest',
|
||||
'decisionDigest',
|
||||
'disabledRowCount',
|
||||
'effectiveBindingCount',
|
||||
'reconciliationBundleDigest',
|
||||
'sourceRowCount',
|
||||
],
|
||||
'source',
|
||||
);
|
||||
const sourceRowCount = count(
|
||||
value.sourceRowCount,
|
||||
'sourceRowCount',
|
||||
MAX_CLUSTER_LEGACY_ENV_SOURCE_ROWS,
|
||||
);
|
||||
const activeRowCount = count(
|
||||
value.activeRowCount,
|
||||
'activeRowCount',
|
||||
MAX_CLUSTER_LEGACY_ENV_SOURCE_ROWS,
|
||||
);
|
||||
const disabledRowCount = count(
|
||||
value.disabledRowCount,
|
||||
'disabledRowCount',
|
||||
MAX_CLUSTER_LEGACY_ENV_SOURCE_ROWS,
|
||||
);
|
||||
const effectiveBindingCount = count(
|
||||
value.effectiveBindingCount,
|
||||
'effectiveBindingCount',
|
||||
MAX_CLUSTER_LEGACY_ENV_SOURCE_ROWS,
|
||||
);
|
||||
if (
|
||||
sourceRowCount < 1 ||
|
||||
sourceRowCount !== activeRowCount + disabledRowCount ||
|
||||
activeRowCount < 1 ||
|
||||
effectiveBindingCount < 1 ||
|
||||
effectiveBindingCount > activeRowCount
|
||||
) {
|
||||
return invalid('source row counts are inconsistent');
|
||||
}
|
||||
return Object.freeze({
|
||||
reconciliationBundleDigest: digest(
|
||||
value.reconciliationBundleDigest,
|
||||
'reconciliationBundleDigest',
|
||||
),
|
||||
decisionDigest: digest(value.decisionDigest, 'decisionDigest'),
|
||||
candidateSetDigest: digest(value.candidateSetDigest, 'candidateSetDigest'),
|
||||
sourceRowCount,
|
||||
activeRowCount,
|
||||
disabledRowCount,
|
||||
effectiveBindingCount,
|
||||
});
|
||||
}
|
||||
|
||||
function target(
|
||||
value: ClusterLegacyEnvMigrationTarget,
|
||||
projectId: string,
|
||||
): Readonly<ClusterLegacyEnvMigrationTarget> {
|
||||
const candidate = record(value, 'target');
|
||||
exactKeys(
|
||||
candidate,
|
||||
[
|
||||
'secretRef',
|
||||
'taskCount',
|
||||
'taskRevisionSetDigest',
|
||||
'totalEffectiveBytes',
|
||||
'triggerCount',
|
||||
'triggerRevisionSetDigest',
|
||||
],
|
||||
'target',
|
||||
);
|
||||
let parsedSecretRef;
|
||||
try {
|
||||
parsedSecretRef = parseSecretRef(value.secretRef);
|
||||
} catch {
|
||||
return invalid('secretRef is invalid');
|
||||
}
|
||||
if (
|
||||
parsedSecretRef.projectId !== projectId ||
|
||||
parsedSecretRef.version === undefined
|
||||
) {
|
||||
return invalid('secretRef must pin a version in the same Project');
|
||||
}
|
||||
const taskCount = count(
|
||||
value.taskCount,
|
||||
'taskCount',
|
||||
MAX_CLUSTER_LEGACY_ENV_TASKS,
|
||||
);
|
||||
if (taskCount < 1) return invalid('taskCount is invalid');
|
||||
const triggerCount = count(
|
||||
value.triggerCount,
|
||||
'triggerCount',
|
||||
MAX_CLUSTER_LEGACY_ENV_TRIGGERS,
|
||||
);
|
||||
const totalEffectiveBytes = count(
|
||||
value.totalEffectiveBytes,
|
||||
'totalEffectiveBytes',
|
||||
MAX_CLUSTER_LEGACY_ENV_EFFECTIVE_BYTES,
|
||||
);
|
||||
if (totalEffectiveBytes < 1) {
|
||||
return invalid('totalEffectiveBytes is invalid');
|
||||
}
|
||||
return Object.freeze({
|
||||
secretRef: value.secretRef,
|
||||
taskRevisionSetDigest: digest(
|
||||
value.taskRevisionSetDigest,
|
||||
'taskRevisionSetDigest',
|
||||
),
|
||||
triggerRevisionSetDigest: digest(
|
||||
value.triggerRevisionSetDigest,
|
||||
'triggerRevisionSetDigest',
|
||||
),
|
||||
taskCount,
|
||||
triggerCount,
|
||||
totalEffectiveBytes,
|
||||
});
|
||||
}
|
||||
|
||||
export function normalizeClusterLegacyEnvMigrationPlanIntent(
|
||||
value: ClusterLegacyEnvMigrationPlanIntent,
|
||||
): Readonly<ClusterLegacyEnvMigrationPlanIntent> {
|
||||
const candidate = record(value, 'plan intent');
|
||||
exactKeys(
|
||||
candidate,
|
||||
['mutationId', 'planId', 'projectId', 'source', 'target'],
|
||||
'plan intent',
|
||||
);
|
||||
const projectId = assertClusterLegacyEnvMigrationPlanIdentifier(
|
||||
value.projectId,
|
||||
'projectId',
|
||||
);
|
||||
return Object.freeze({
|
||||
planId: assertClusterLegacyEnvMigrationPlanIdentifier(
|
||||
value.planId,
|
||||
'planId',
|
||||
),
|
||||
mutationId: assertClusterLegacyEnvMigrationPlanIdentifier(
|
||||
value.mutationId,
|
||||
'mutationId',
|
||||
),
|
||||
projectId,
|
||||
source: sourceEvidence(value.source),
|
||||
target: target(value.target, projectId),
|
||||
});
|
||||
}
|
||||
|
||||
function unsignedFields(
|
||||
value: Omit<ClusterLegacyEnvMigrationPlan, 'planDigest'>,
|
||||
): object {
|
||||
return {
|
||||
schema: value.schema,
|
||||
planId: value.planId,
|
||||
mutationId: value.mutationId,
|
||||
projectId: value.projectId,
|
||||
source: value.source,
|
||||
target: value.target,
|
||||
plannedAtMs: value.plannedAtMs,
|
||||
};
|
||||
}
|
||||
|
||||
export function clusterLegacyEnvMigrationPlanDigest(
|
||||
value: Omit<ClusterLegacyEnvMigrationPlan, 'planDigest'>,
|
||||
): string {
|
||||
return createHash('sha256')
|
||||
.update(PLAN_DIGEST_DOMAIN)
|
||||
.update(JSON.stringify(unsignedFields(value)), 'utf8')
|
||||
.digest('hex');
|
||||
}
|
||||
|
||||
function bounded(
|
||||
value: Readonly<ClusterLegacyEnvMigrationPlan>,
|
||||
): Readonly<ClusterLegacyEnvMigrationPlan> {
|
||||
if (
|
||||
Buffer.byteLength(JSON.stringify(value), 'utf8') >
|
||||
MAX_CLUSTER_LEGACY_ENV_MIGRATION_PLAN_JSON_BYTES
|
||||
) {
|
||||
return invalid('encoded plan exceeds the size limit');
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
export function createClusterLegacyEnvMigrationPlan(
|
||||
intentValue: ClusterLegacyEnvMigrationPlanIntent,
|
||||
plannedAtMsValue: number,
|
||||
): Readonly<ClusterLegacyEnvMigrationPlan> {
|
||||
const intent = normalizeClusterLegacyEnvMigrationPlanIntent(intentValue);
|
||||
const unsigned = Object.freeze({
|
||||
schema: CLUSTER_LEGACY_ENV_MIGRATION_PLAN_SCHEMA,
|
||||
...intent,
|
||||
plannedAtMs: timestamp(plannedAtMsValue),
|
||||
});
|
||||
return bounded(
|
||||
Object.freeze({
|
||||
...unsigned,
|
||||
planDigest: clusterLegacyEnvMigrationPlanDigest(unsigned),
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
export function normalizeClusterLegacyEnvMigrationPlan(
|
||||
value: ClusterLegacyEnvMigrationPlan,
|
||||
): Readonly<ClusterLegacyEnvMigrationPlan> {
|
||||
const candidate = record(value, 'plan');
|
||||
exactKeys(
|
||||
candidate,
|
||||
[
|
||||
'mutationId',
|
||||
'planDigest',
|
||||
'planId',
|
||||
'plannedAtMs',
|
||||
'projectId',
|
||||
'schema',
|
||||
'source',
|
||||
'target',
|
||||
],
|
||||
'plan',
|
||||
);
|
||||
if (value.schema !== CLUSTER_LEGACY_ENV_MIGRATION_PLAN_SCHEMA) {
|
||||
return invalid('schema is invalid');
|
||||
}
|
||||
const expected = createClusterLegacyEnvMigrationPlan(
|
||||
{
|
||||
planId: value.planId,
|
||||
mutationId: value.mutationId,
|
||||
projectId: value.projectId,
|
||||
source: value.source,
|
||||
target: value.target,
|
||||
},
|
||||
value.plannedAtMs,
|
||||
);
|
||||
if (
|
||||
typeof value.planDigest !== 'string' ||
|
||||
!DIGEST_PATTERN.test(value.planDigest) ||
|
||||
value.planDigest !== expected.planDigest
|
||||
) {
|
||||
return invalid('planDigest does not match plan');
|
||||
}
|
||||
return expected;
|
||||
}
|
||||
|
||||
export function clusterLegacyEnvMigrationPlanMatchesIntent(
|
||||
planValue: ClusterLegacyEnvMigrationPlan,
|
||||
intentValue: ClusterLegacyEnvMigrationPlanIntent,
|
||||
): boolean {
|
||||
const plan = normalizeClusterLegacyEnvMigrationPlan(planValue);
|
||||
const intent = normalizeClusterLegacyEnvMigrationPlanIntent(intentValue);
|
||||
return (
|
||||
plan.planId === intent.planId &&
|
||||
plan.mutationId === intent.mutationId &&
|
||||
plan.projectId === intent.projectId &&
|
||||
JSON.stringify(plan.source) === JSON.stringify(intent.source) &&
|
||||
JSON.stringify(plan.target) === JSON.stringify(intent.target)
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,177 @@
|
||||
const assert = require('node:assert/strict');
|
||||
const { test } = require('node:test');
|
||||
|
||||
const {
|
||||
CLUSTER_LEGACY_ENV_MIGRATION_PLAN_SCHEMA,
|
||||
MAX_CLUSTER_LEGACY_ENV_EFFECTIVE_BYTES,
|
||||
MAX_CLUSTER_LEGACY_ENV_SOURCE_ROWS,
|
||||
MAX_CLUSTER_LEGACY_ENV_TASKS,
|
||||
MAX_CLUSTER_LEGACY_ENV_TRIGGERS,
|
||||
InvalidClusterLegacyEnvMigrationPlanError,
|
||||
clusterLegacyEnvMigrationPlanMatchesIntent,
|
||||
createClusterLegacyEnvMigrationPlan,
|
||||
normalizeClusterLegacyEnvMigrationPlan,
|
||||
normalizeClusterLegacyEnvMigrationPlanIntent,
|
||||
} = require('@qinglong/runtime-core/cluster-legacy-env-migration-plan');
|
||||
const { createSecretRef } = require('@qinglong/runtime-core/secret-reference');
|
||||
|
||||
const DIGESTS = Object.freeze({
|
||||
reconciliationBundleDigest: '1'.repeat(64),
|
||||
decisionDigest: '2'.repeat(64),
|
||||
candidateSetDigest: '3'.repeat(64),
|
||||
taskRevisionSetDigest: '4'.repeat(64),
|
||||
triggerRevisionSetDigest: '5'.repeat(64),
|
||||
});
|
||||
|
||||
function intent(overrides = {}) {
|
||||
const projectId = overrides.projectId ?? 'project-a';
|
||||
return {
|
||||
planId: 'legacy-env-plan-a',
|
||||
mutationId: 'legacy-env-mutation-a',
|
||||
projectId,
|
||||
source: {
|
||||
reconciliationBundleDigest: DIGESTS.reconciliationBundleDigest,
|
||||
decisionDigest: DIGESTS.decisionDigest,
|
||||
candidateSetDigest: DIGESTS.candidateSetDigest,
|
||||
sourceRowCount: 4,
|
||||
activeRowCount: 3,
|
||||
disabledRowCount: 1,
|
||||
effectiveBindingCount: 2,
|
||||
},
|
||||
target: {
|
||||
secretRef: createSecretRef({
|
||||
projectId,
|
||||
name: 'legacy-env-bundle',
|
||||
version: 7,
|
||||
}),
|
||||
taskRevisionSetDigest: DIGESTS.taskRevisionSetDigest,
|
||||
triggerRevisionSetDigest: DIGESTS.triggerRevisionSetDigest,
|
||||
taskCount: 2,
|
||||
triggerCount: 3,
|
||||
totalEffectiveBytes: 1024,
|
||||
},
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
test('creates one frozen content-free migration plan with a stable digest', () => {
|
||||
const value = createClusterLegacyEnvMigrationPlan(intent(), 12_345);
|
||||
assert.equal(value.schema, CLUSTER_LEGACY_ENV_MIGRATION_PLAN_SCHEMA);
|
||||
assert.match(value.planDigest, /^[0-9a-f]{64}$/);
|
||||
assert.equal(Object.isFrozen(value), true);
|
||||
assert.equal(Object.isFrozen(value.source), true);
|
||||
assert.equal(Object.isFrozen(value.target), true);
|
||||
assert.deepEqual(normalizeClusterLegacyEnvMigrationPlan(value), value);
|
||||
assert.equal(
|
||||
clusterLegacyEnvMigrationPlanMatchesIntent(value, intent()),
|
||||
true,
|
||||
);
|
||||
assert.deepEqual(Object.keys(value).sort(), [
|
||||
'mutationId',
|
||||
'planDigest',
|
||||
'planId',
|
||||
'plannedAtMs',
|
||||
'projectId',
|
||||
'schema',
|
||||
'source',
|
||||
'target',
|
||||
]);
|
||||
assert.equal(JSON.stringify(value).includes('legacy-env-bundle'), false);
|
||||
});
|
||||
|
||||
test('requires a canonical version-pinned SecretRef in the same Project', () => {
|
||||
const unversioned = intent();
|
||||
unversioned.target.secretRef = createSecretRef({
|
||||
projectId: unversioned.projectId,
|
||||
name: 'legacy-env-bundle',
|
||||
});
|
||||
assert.throws(
|
||||
() => normalizeClusterLegacyEnvMigrationPlanIntent(unversioned),
|
||||
InvalidClusterLegacyEnvMigrationPlanError,
|
||||
);
|
||||
|
||||
const crossProject = intent();
|
||||
crossProject.target.secretRef = createSecretRef({
|
||||
projectId: 'project-b',
|
||||
name: 'legacy-env-bundle',
|
||||
version: 1,
|
||||
});
|
||||
assert.throws(
|
||||
() => normalizeClusterLegacyEnvMigrationPlanIntent(crossProject),
|
||||
InvalidClusterLegacyEnvMigrationPlanError,
|
||||
);
|
||||
});
|
||||
|
||||
test('enforces source consistency and router-safe bounded targets', () => {
|
||||
const invalidValues = [
|
||||
{ source: { ...intent().source, sourceRowCount: 5 } },
|
||||
{
|
||||
source: {
|
||||
...intent().source,
|
||||
sourceRowCount: MAX_CLUSTER_LEGACY_ENV_SOURCE_ROWS + 1,
|
||||
},
|
||||
},
|
||||
{
|
||||
target: {
|
||||
...intent().target,
|
||||
taskCount: MAX_CLUSTER_LEGACY_ENV_TASKS + 1,
|
||||
},
|
||||
},
|
||||
{
|
||||
target: {
|
||||
...intent().target,
|
||||
triggerCount: MAX_CLUSTER_LEGACY_ENV_TRIGGERS + 1,
|
||||
},
|
||||
},
|
||||
{
|
||||
target: {
|
||||
...intent().target,
|
||||
totalEffectiveBytes: MAX_CLUSTER_LEGACY_ENV_EFFECTIVE_BYTES + 1,
|
||||
},
|
||||
},
|
||||
];
|
||||
for (const invalidValue of invalidValues) {
|
||||
assert.throws(
|
||||
() => normalizeClusterLegacyEnvMigrationPlanIntent(intent(invalidValue)),
|
||||
InvalidClusterLegacyEnvMigrationPlanError,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects widened shapes, accessors, symbols and tampered durable digests', () => {
|
||||
const widened = intent();
|
||||
widened.source.envName = 'TOKEN';
|
||||
assert.throws(
|
||||
() => normalizeClusterLegacyEnvMigrationPlanIntent(widened),
|
||||
InvalidClusterLegacyEnvMigrationPlanError,
|
||||
);
|
||||
|
||||
const symbol = intent();
|
||||
symbol.target[Symbol('secretValue')] = 'not-storable';
|
||||
assert.throws(
|
||||
() => normalizeClusterLegacyEnvMigrationPlanIntent(symbol),
|
||||
InvalidClusterLegacyEnvMigrationPlanError,
|
||||
);
|
||||
|
||||
const accessor = intent();
|
||||
Object.defineProperty(accessor.source, 'sourceRowCount', {
|
||||
enumerable: true,
|
||||
get() {
|
||||
return 4;
|
||||
},
|
||||
});
|
||||
assert.throws(
|
||||
() => normalizeClusterLegacyEnvMigrationPlanIntent(accessor),
|
||||
InvalidClusterLegacyEnvMigrationPlanError,
|
||||
);
|
||||
|
||||
const plan = createClusterLegacyEnvMigrationPlan(intent(), 12_345);
|
||||
assert.throws(
|
||||
() =>
|
||||
normalizeClusterLegacyEnvMigrationPlan({
|
||||
...plan,
|
||||
planDigest: 'f'.repeat(64),
|
||||
}),
|
||||
InvalidClusterLegacyEnvMigrationPlanError,
|
||||
);
|
||||
});
|
||||
@@ -299,10 +299,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
|
||||
rootSourceFileRoles: runtimeCore.rootSourceFileRoles,
|
||||
},
|
||||
{
|
||||
sourceFiles: 171,
|
||||
sourceFiles: 172,
|
||||
rootSourceFiles: 1,
|
||||
rootSourceLines: 160,
|
||||
nestedSourceFiles: 170,
|
||||
nestedSourceFiles: 171,
|
||||
rootSourceFileRoles: { 'index.ts': 'public_export' },
|
||||
},
|
||||
);
|
||||
@@ -421,10 +421,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
|
||||
rootSourceFileRoles: clusterPostgres.rootSourceFileRoles,
|
||||
},
|
||||
{
|
||||
sourceFiles: 175,
|
||||
sourceFiles: 177,
|
||||
rootSourceFiles: 1,
|
||||
rootSourceLines: 126,
|
||||
nestedSourceFiles: 174,
|
||||
nestedSourceFiles: 176,
|
||||
rootSourceFileRoles: { 'index.ts': 'public_export' },
|
||||
},
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user