diff --git a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md index 48ffb428..5aea3ef3 100644 --- a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md +++ b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md @@ -26,6 +26,7 @@ `e7c1743e932f2d7c35dc9153cdf5bc4a03356a38d93fce5507354652aa207a05`，独立审计与 Docker 清理通过。完整验证证据见 ADR-0391。 - D-304/ADR-0392（已接受）：Plugin Package 进入安全 quarantine 时，Workflow/Prompt automation publication 不再仅依赖运行时 start guard 间接拒绝，而是与 quarantine event、Package-owned Task disabled revisions、Project Tool snapshot 和 withdrawal receipt 在同一 SQLite/PostgreSQL 事务中收敛为 `withdrawn`。原先仅能引用普通 lifecycle event 的外键升级为 append-only disposition-event 联合引用，历史 migration 与 publication digest schema 保持不变；SQLite edge/standalone 崩溃矩阵覆盖 automation publication insert 后、event/task/receipt/COMMIT 前后，PostgreSQL 通过触发器在既有 `SECURITY DEFINER` quarantine commit 内登记 disposition，再由同一外层 SERIALIZABLE transaction CAS publication head。能力位为 `plugin_package_automation_security_withdrawal@1`；不新增 package、daemon、timer、连接或常驻缓存，适用于低配路由设备和集群节点。SQLite 全量 228/228；PostgreSQL package 311 pass/1 条外部 URL 条件 skip；完整 18-package build/test 退出 0，backend 1,188 pass/2 skip，package/dependency boundary 零 finding；PostgreSQL 18.4 arm64 HA 125 项 gate 全绿、timeline `1→2`，报告 SHA-256 为 `ab156901b9c96ec5a62259c44d83d24ded011e0616dc827d928f3e13efd11786`。 +- D-305/ADR-0393（已接受）：Plugin Package Manifest 的逻辑 Secret requirement 获得按 resource generation 固定的不可变 binding。binding 精确覆盖 Manifest requirements，只保存同 Project、显式 version 的 `qlsecret://` 引用与 `approved-action-execution|local-owner-confirmation` authority evidence digest，不保存 Secret 明文；required 不可为空，optional 可显式为 `null`。发布由当前 active installation head、lock、generation 与 Manifest digest 联合 fencing，相同事实幂等、不同事实冲突；domain-separated digest、最多 64 项和 64 KiB 单行预算同时约束 Local 与 Cluster。SQLite 追加 `0091` ledger 与 capability v46，PostgreSQL 追加 `pg-0059`、capability v58，并只向 package executor 授予 `SELECT, INSERT`。不新增 package、daemon、timer、watcher、连接、缓存或集群 workload，低配路由设备只承担一个有界表和三个索引。D-305 不冒充 Secret 已进入执行路径：现有 materialization 拒绝仍保留，D-306 再完成用户授权、Secret resolution、runtime consumption 与 lifecycle/rebinding 语义。core 509/509、SQLite 232/232、PostgreSQL 316 pass/1 条件 skip；完整 18-package clean build/test 退出 0，backend 1,188 pass/2 skip，五项边界审计零 finding，workspace 仍无 single-source/shallow-source package。PostgreSQL 18.4 arm64 HA 125 项 gate 全绿、timeline `1→2`，报告 SHA-256 为 `acf0fea7ca7699989dfe70f5dd0061cdf5fb1968c691094331fea06ce01b96dc`。 - D-302/ADR-0390（已接受） Cluster operator context 增加无网络、无 mutation 的内建 `ql3-cluster-admin context validate` 预检。它先复用 owner-private context reader，再让每个 entry 经过与真实请求相同的 production HTTPS/Kubernetes configuration preparation，验证精确 route、hostname、CA、 diff --git a/docs/adr/ADR-0393-generation-bound-plugin-package-secret-binding-ledger.md b/docs/adr/ADR-0393-generation-bound-plugin-package-secret-binding-ledger.md new file mode 100644 index 00000000..90484d3d --- /dev/null +++ b/docs/adr/ADR-0393-generation-bound-plugin-package-secret-binding-ledger.md @@ -0,0 +1,75 @@ +# ADR-0393：按 Generation 固定的 Plugin Package Secret 绑定账本 + +- 状态：Accepted +- 日期：2026-08-13 +- 关联 RFC：QL-RFC-0001 D-305 +- 关联 ADR：ADR-0216、ADR-0221、ADR-0392 + +## 问题 + +Plugin Package Manifest 已能声明逻辑 Secret 需求，但 materialization 仍会拒绝任何 Secret requirement。仅把 Secret 名称传入执行器会留下三类不确定性：Secret rotation 会让同一 generation 在不同时间解析到不同值，跨 Project 引用可能越权，安装确认与实际执行也缺少可审计的同一事实。 + +QingLong 3.0 同时面向低配路由设备与集群节点。解决方案不能为 Local/Edge 引入 Secret watcher、常驻缓存、额外数据库连接或独立 daemon，也不能在集群数据库中向宽权限角色暴露 Secret 绑定事实。 + +## 决策 + +1. 引入 `qinglong/plugin-package-secret-binding@v1`。一个不可变 binding 精确绑定一个 Package resource generation、Manifest digest 与 lock digest，并完整覆盖 Manifest 声明的所有 Secret requirement。 +2. binding 只保存 `qlsecret://` 引用，不保存 Secret 明文。引用必须与 Package installation 位于同一 Project，并固定显式 version；required requirement 不允许为空，optional requirement 可以显式记录 `null`。 +3. binding 必须携带 `approved-action-execution` 或 `local-owner-confirmation` authority kind 及其 evidence digest。D-305 只定义并持久化 authority 事实，不新增绕过既有审批或 owner-confirmation 的入口。 +4. 发布时必须证明目标仍是当前 active installation head，且 installation、Project、Package、lock、generation 与 Manifest digest 全部一致。相同 generation 的完全相同内容幂等返回 existing；不同内容冲突并失败关闭。 +5. binding 使用 domain-separated SHA-256 摘要，持久 JSON 上限 64 KiB，最多沿用 Manifest 的 64 个 Secret requirement 上限。SQLite 与 PostgreSQL 分别追加同构 ledger，不改写历史 migration。 +6. Local 使用既有单写者 SQLite authority；Cluster 仅向 `ql3_package_executor` 授予 ledger 的 `SELECT, INSERT`，runtime、admin、package manager 与 worker ingress 均无权限。实现不新增 package、进程、timer、watcher、连接或常驻缓存。 +7. ledger 是历史事实，不直接成为 Package lifecycle 的 active-consumer blocker。D-306 在消费 binding 并产生实际 materialization/执行依赖时，再定义 Secret resolution、撤回与 rebinding 的生命周期规则。 + +## 方言映射 + +| 项目 | Local / Edge / Standalone | Cluster | +| --- | --- | --- | +| migration | `0091-plugin-package-secret-bindings` + `0092-capability-v46` | `pg-0059-plugin-package-secret-bindings` | +| capability | `plugin_package_secret_binding@1`，contract v46 | `plugin_package_secret_binding@1`，contract v58 | +| 写入串行化 | 既有 SQLite 单写者 authority | PostgreSQL 唯一约束与 active-head 条件写入 | +| 资源增量 | 一个有界 append-only 表、三个索引 | 一个有界 append-only 表、三个索引 | +| 常驻开销 | 无新增进程、连接、timer、watcher、cache | 无新增 Pod、worker、连接、timer、watcher、cache | + +## 明确不在 D-305 中完成 + +- 不读取或解密 Secret 明文，不把 Secret 注入 Task 环境。 +- 不提供面向用户的 bind/rebind 命令或远程管理 API。 +- 不声明 Secret rotation 自动改变既有 generation；新版本必须由后续显式授权和新事实消费。 +- 不改变现有“Manifest 含 Secret requirement 时 materialization 拒绝”的执行边界。 + +## 替代方案 + +### 执行时解析浮动 Secret 名称 + +拒绝。它让同一 generation 的行为随 rotation 漂移，无法复现，也无法把授权证据与实际版本绑定。 + +### 把加密 Secret 值复制进 Package ledger + +拒绝。它扩大密文托管面、备份恢复面与轮换复杂度，也不必要地增加路由设备存储和集群权限面。 + +### 只在产品层保存临时绑定 + +拒绝。崩溃、重启或主库切换后无法证明执行消费的是安装时确认的同一绑定。 + +## 影响 + +- 相同 Package generation 的 Secret 选择成为可复现、可摘要、可审计的不可变事实。 +- Secret 明文仍留在既有 Secret custody 边界内；ledger 泄露只暴露有界引用元数据，不暴露值。 +- D-305 建立的是消费前置条件，不代表 Secret-aware Package 已可运行；产品授权与 runtime consumption 仍属于 D-306。 +- Local 与 Cluster 保持同一领域契约、不同存储 authority，低配设备不承担集群组件成本。 + +## 验证 + +- core contract：精确 Manifest coverage、required/optional、同 Project、固定 version、canonical order、authority/digest/shape drift 与 generation/Manifest 重校验。 +- SQLite：幂等、冲突、active generation fencing、tamper fail-closed、migration checksum、typed schema、readiness 与 rollout safety。 +- PostgreSQL：同构 repository contract、migration checksum、schema/readiness 与 package-executor 最小权限。 +- 阶段完成前运行完整 18-package build/test、backend 回归、package/dependency boundary 审计和 PostgreSQL 18 physical-streaming HA gate。 + +## 当前证据 + +- runtime-core 全量 509/509；local-sqlite 全量 232/232。 +- cluster-postgres 317 项：316 pass、1 条仅在未提供独立测试数据库 URL 时条件 skip、0 fail。 +- 完整 18-package clean build/test 退出 0；Worker 133/133，AI 209 pass/3 条件 skip，Cluster Admin 302 pass/2 条件 skip，Local Owner 163 pass/5 条件 skip，其余 package 同样 0 fail。 +- backend 1,190 项：1,188 pass、2 条件 skip、0 fail。package boundary、cluster dependency boundary、edge import、service bridge import 与 local image 五项审计均为 0 finding；workspace 仍为 18 package、无 single-source/shallow-source package，PostgreSQL 60 个与 SQLite 93 个 migration 文件继续作为显式 `ordered_ledger` 管理。 +- PostgreSQL 18.4 arm64 physical-streaming HA 125 项 gate 全绿，timeline `1→2`，旧主完成 fence 与 `pg_rewind` 只读重入；报告 SHA-256 为 `acf0fea7ca7699989dfe70f5dd0061cdf5fb1968c691094331fea06ce01b96dc`。 diff --git a/docs/adr/README.md b/docs/adr/README.md index 1f673e51..cc93fe15 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -396,6 +396,7 @@ | [ADR-0390](./ADR-0390-offline-cluster-operator-context-preflight.md) | Offline Cluster Operator Context Preflight | Accepted | | [ADR-0391](./ADR-0391-read-only-cluster-operator-context-readiness.md) | Read-only Cluster Operator Context Readiness | Accepted | | [ADR-0392](./ADR-0392-atomic-plugin-package-automation-security-withdrawal.md) | Plugin Package Automation 安全隔离原子撤回 | Accepted | +| [ADR-0393](./ADR-0393-generation-bound-plugin-package-secret-binding-ledger.md) | 按 Generation 固定的 Plugin Package Secret 绑定账本 | Accepted | ## 规则 diff --git a/docs/ql3-package-boundaries.json b/docs/ql3-package-boundaries.json index 5a2a1a84..b4c91765 100644 --- a/docs/ql3-package-boundaries.json +++ b/docs/ql3-package-boundaries.json @@ -6,13 +6,13 @@ "reviewedDenseDirectories": [ { "kind": "ordered_ledger", - "maxDirectSourceFiles": 59, + "maxDirectSourceFiles": 60, "path": "packages/ql3-cluster-postgres/src/migrations", "rationale": "PostgreSQL migrations are an append-only version ledger whose ordering and discoverability are safer in one reviewed directory." }, { "kind": "ordered_ledger", - "maxDirectSourceFiles": 91, + "maxDirectSourceFiles": 93, "path": "packages/ql3-local-sqlite/src/migrations", "rationale": "SQLite migrations are an append-only version ledger whose ordering and discoverability are safer in one reviewed directory." } @@ -84,7 +84,7 @@ "path": "packages/ql3-cluster-postgres", "name": "@qinglong/cluster-postgres", "rootSourceFileHardCap": 1, - "rootSourceLineHardCap": 125, + "rootSourceLineHardCap": 126, "rootSourceFileRoles": { "index.ts": "public_export" }, diff --git a/packages/ql3-cluster-admin/test/pluginPackageRecovery.test.cjs b/packages/ql3-cluster-admin/test/pluginPackageRecovery.test.cjs index c744a4d1..f686c57d 100644 --- a/packages/ql3-cluster-admin/test/pluginPackageRecovery.test.cjs +++ b/packages/ql3-cluster-admin/test/pluginPackageRecovery.test.cjs @@ -34,6 +34,7 @@ function executorPrivileges() { 'plugin_package_materialized_revisions', 'plugin_package_automation_publications', 'plugin_package_automation_publication_heads', + 'plugin_package_secret_bindings', 'project_tool_definition_snapshots', 'project_tool_definition_snapshot_sources', 'plugin_package_publisher_provenance', diff --git a/packages/ql3-cluster-control/test/application.test.cjs b/packages/ql3-cluster-control/test/application.test.cjs index 07849366..ee3ad16c 100644 --- a/packages/ql3-cluster-control/test/application.test.cjs +++ b/packages/ql3-cluster-control/test/application.test.cjs @@ -199,6 +199,7 @@ function runtimePrivileges() { plugin_package_automation_publications: [true, false, false, false], plugin_package_automation_disposition_events: [false, false, false, false], plugin_package_automation_publication_heads: [true, false, false, false], + plugin_package_secret_bindings: [false, false, false, false], plugin_package_workflow_admissions: [true, true, false, false], plugin_package_workflow_admission_steps: [true, true, false, false], plugin_package_workflow_task_attempt_admissions: [true, true, false, false], diff --git a/packages/ql3-cluster-control/test/bootstrap.test.cjs b/packages/ql3-cluster-control/test/bootstrap.test.cjs index 9348093b..26f980ea 100644 --- a/packages/ql3-cluster-control/test/bootstrap.test.cjs +++ b/packages/ql3-cluster-control/test/bootstrap.test.cjs @@ -113,6 +113,7 @@ function runtimePrivileges() { plugin_package_automation_publications: [true, false, false, false], plugin_package_automation_disposition_events: [false, false, false, false], plugin_package_automation_publication_heads: [true, false, false, false], + plugin_package_secret_bindings: [false, false, false, false], plugin_package_workflow_admissions: [true, true, false, false], plugin_package_workflow_admission_steps: [true, true, false, false], plugin_package_workflow_task_attempt_admissions: [true, true, false, false], diff --git a/packages/ql3-cluster-postgres/package.json b/packages/ql3-cluster-postgres/package.json index aae3f4ad..81398dc7 100644 --- a/packages/ql3-cluster-postgres/package.json +++ b/packages/ql3-cluster-postgres/package.json @@ -140,6 +140,11 @@ "require": "./dist/plugin-package/installation/pluginPackageMaterializedRevisionRepository.js", "default": "./dist/plugin-package/installation/pluginPackageMaterializedRevisionRepository.js" }, + "./plugin-package-secret-binding": { + "types": "./dist/plugin-package/installation/pluginPackageSecretBindingRepository.d.ts", + "require": "./dist/plugin-package/installation/pluginPackageSecretBindingRepository.js", + "default": "./dist/plugin-package/installation/pluginPackageSecretBindingRepository.js" + }, "./plugin-package-automation-publication": { "types": "./dist/plugin-package/publication/pluginPackageAutomationPublicationRepository.d.ts", "require": "./dist/plugin-package/publication/pluginPackageAutomationPublicationRepository.js", diff --git a/packages/ql3-cluster-postgres/src/entrypoints/packageExecutor.ts b/packages/ql3-cluster-postgres/src/entrypoints/packageExecutor.ts index 74ee5f50..df41aebd 100644 --- a/packages/ql3-cluster-postgres/src/entrypoints/packageExecutor.ts +++ b/packages/ql3-cluster-postgres/src/entrypoints/packageExecutor.ts @@ -40,6 +40,7 @@ export { } from '../schema/schemaReadiness'; export { PostgresPluginPackageMaterializedRevisionRepository } from '../plugin-package/installation/pluginPackageMaterializedRevisionRepository'; +export { PostgresPluginPackageSecretBindingRepository } from '../plugin-package/installation/pluginPackageSecretBindingRepository'; export { PostgresPluginPackageAutomationPublicationRepository } from '../plugin-package/publication/pluginPackageAutomationPublicationRepository'; export { CLUSTER_PLUGIN_PACKAGE_QUARANTINE_TARGET_LIMIT, diff --git a/packages/ql3-cluster-postgres/src/index.ts b/packages/ql3-cluster-postgres/src/index.ts index 5cce93f2..50bcf50d 100644 --- a/packages/ql3-cluster-postgres/src/index.ts +++ b/packages/ql3-cluster-postgres/src/index.ts @@ -25,6 +25,7 @@ export { pluginPackageIdentityKeysetLedger, pluginPackageManagementQuotaBuckets, pluginPackageMaterializedRevisions, + pluginPackageSecretBindings, pluginPackageQuarantineEvents, pluginPackageWorkflowAdmissions, pluginPackageWorkflowAdmissionSteps, diff --git a/packages/ql3-cluster-postgres/src/migration/migrationManifest.ts b/packages/ql3-cluster-postgres/src/migration/migrationManifest.ts index 0851f18e..db482705 100644 --- a/packages/ql3-cluster-postgres/src/migration/migrationManifest.ts +++ b/packages/ql3-cluster-postgres/src/migration/migrationManifest.ts @@ -298,5 +298,10 @@ export const postgresqlMainMigrationManifest: MigrationStreamManifest = checksum: 'd184324909f1e450f3c1b58d422796e3869a1360df60c3f2dfe4af0bacc37471', }), + Object.freeze({ + id: 'pg-0059-plugin-package-secret-bindings', + checksum: + '87582d256c868bd7f5af352c4b052fdab9f3714e1e7179e35d33bfa5d62957be', + }), ]), }); diff --git a/packages/ql3-cluster-postgres/src/migrations/index.ts b/packages/ql3-cluster-postgres/src/migrations/index.ts index 0f0fcc88..1bf9afc4 100644 --- a/packages/ql3-cluster-postgres/src/migrations/index.ts +++ b/packages/ql3-cluster-postgres/src/migrations/index.ts @@ -61,6 +61,7 @@ import { pg0055RunAttemptLogRetentionMigration } from './pg-0055-run-attempt-log import { pg0056RunManagementBoundaryMigration } from '../run-management/pg-0056-run-management-boundary'; import { pg0057RunManagementStopBoundaryMigration } from '../run-management/pg-0057-run-management-stop-boundary'; import { pg0058PluginPackageAutomationDispositionEventsMigration } from './pg-0058-plugin-package-automation-disposition-events'; +import { pg0059PluginPackageSecretBindingsMigration } from './pg-0059-plugin-package-secret-bindings'; export const postgresqlMainMigrationStream: MigrationStreamDefinition = Object.freeze({ @@ -127,5 +128,6 @@ export const postgresqlMainMigrationStream: MigrationStreamDefinition= 0 + ), + CONSTRAINT ql3_plugin_package_secret_binding_digest_check CHECK ( + generation_digest ~ '^[0-9a-f]{64}$' AND + lock_digest ~ '^[0-9a-f]{64}$' AND + manifest_digest ~ '^[0-9a-f]{64}$' AND + evidence_digest ~ '^[0-9a-f]{64}$' AND + binding_digest ~ '^[0-9a-f]{64}$' + ), + CONSTRAINT ql3_plugin_package_secret_binding_json_check CHECK ( + jsonb_typeof(binding_json) = 'object' AND + octet_length(binding_json::text) BETWEEN 2 AND 65536 AND + binding_json @> jsonb_build_object( + 'schema', 'qinglong/plugin-package-secret-binding@v1', + 'target', jsonb_build_object( + 'generationDigest', generation_digest, + 'projectId', project_id, + 'packageName', package_name, + 'installationId', installation_id, + 'lockDigest', lock_digest, + 'generation', generation, + 'manifestDigest', manifest_digest + ), + 'authority', jsonb_build_object( + 'kind', authority_kind, + 'evidenceDigest', evidence_digest + ), + 'boundAtMs', bound_at_ms, + 'bindingDigest', binding_digest + ) AND + jsonb_typeof(binding_json -> 'entries') = 'array' AND + jsonb_array_length(binding_json -> 'entries') BETWEEN 1 AND 64 + ) +) + `.trim(), + `CREATE UNIQUE INDEX ql3_plugin_package_secret_binding_generation_uidx ON "ql3"."plugin_package_secret_bindings" (project_id, package_name, generation)`, + `CREATE UNIQUE INDEX ql3_plugin_package_secret_binding_digest_uidx ON "ql3"."plugin_package_secret_bindings" (binding_digest)`, + `CREATE INDEX ql3_plugin_package_secret_binding_install_idx ON "ql3"."plugin_package_secret_bindings" (installation_id, generation_digest)`, + `REVOKE ALL ON "ql3"."plugin_package_secret_bindings" FROM PUBLIC, ql3_runtime, ql3_admin, ql3_package_manager, ql3_package_executor, ql3_worker_ingress`, + `GRANT SELECT, INSERT ON "ql3"."plugin_package_secret_bindings" TO ql3_package_executor`, + `DO $ql3$ BEGIN UPDATE "ql3"."schema_capabilities" SET contract_version = 58, migration_id = 'pg-0059-plugin-package-secret-bindings', capabilities = '${CAPABILITIES_V58}'::jsonb, updated_at_ms = floor(extract(epoch FROM transaction_timestamp()) * 1000)::bigint WHERE contract_name = 'control-core' AND contract_version = 57 AND migration_id = 'pg-0058-plugin-package-automation-disposition-events' AND capabilities = '${CAPABILITIES_V57}'::jsonb; IF NOT FOUND THEN RAISE EXCEPTION 'control-core capability is not at version 57' USING ERRCODE = 'check_violation'; END IF; END $ql3$`, + ], + }); diff --git a/packages/ql3-cluster-postgres/src/plugin-package/installation/pluginPackageSecretBindingRepository.ts b/packages/ql3-cluster-postgres/src/plugin-package/installation/pluginPackageSecretBindingRepository.ts new file mode 100644 index 00000000..45bfa8b4 --- /dev/null +++ b/packages/ql3-cluster-postgres/src/plugin-package/installation/pluginPackageSecretBindingRepository.ts @@ -0,0 +1,236 @@ +import type { PostgresPool } from '@qinglong/runtime-core'; +import { + InvalidPluginPackageSecretBindingError, + MAX_PLUGIN_PACKAGE_SECRET_BINDING_JSON_BYTES, + PluginPackageSecretBindingConflictError, + PluginPackageSecretBindingUnavailableError, + normalizePluginPackageSecretBinding, + type PluginPackageSecretBinding, + type PluginPackageSecretBindingRepository, +} from '@qinglong/runtime-core/plugin-package-secret-binding'; + +import { + postgresRequiredInteger, + postgresRequiredJsonObject, + postgresRequiredString, + postgresSqlState, +} from '../../repository/definitionRepositorySupport'; + +type Row = Record; + +const DIGEST = /^[0-9a-f]{64}$/; + +function invalid(message: string): never { + throw new InvalidPluginPackageSecretBindingError(message); +} + +function unavailable(): PluginPackageSecretBindingUnavailableError { + return new PluginPackageSecretBindingUnavailableError(); +} + +function generationDigest(value: unknown): string { + if (typeof value !== 'string' || !DIGEST.test(value)) { + return invalid('generation digest is invalid'); + } + return value; +} + +function serialize(binding: Readonly): string { + const value = JSON.stringify(binding); + if ( + Buffer.byteLength(value, 'utf8') > + MAX_PLUGIN_PACKAGE_SECRET_BINDING_JSON_BYTES + ) { + return invalid('durable JSON byte budget exceeded'); + } + return value; +} + +function mapStorageError(error: unknown): Error { + if ( + error instanceof InvalidPluginPackageSecretBindingError || + error instanceof PluginPackageSecretBindingConflictError || + error instanceof PluginPackageSecretBindingUnavailableError + ) { + return error; + } + const state = postgresSqlState(error); + if (state === '23503' || state === '23505' || state === '23514') { + return new PluginPackageSecretBindingConflictError( + 'durable binding identity is already bound', + ); + } + return new PluginPackageSecretBindingUnavailableError({ + cause: error instanceof Error ? error : undefined, + }); +} + +export class PostgresPluginPackageSecretBindingRepository + implements PluginPackageSecretBindingRepository +{ + constructor(private readonly pool: Pick) { + if (!pool || typeof pool.query !== 'function') { + throw new TypeError( + 'PostgreSQL Plugin Package Secret binding repository options are invalid', + ); + } + } + + private parse(row: Row): Readonly { + try { + const binding = normalizePluginPackageSecretBinding( + postgresRequiredJsonObject(row.bindingJson, unavailable), + ); + if ( + binding.target.generationDigest !== + postgresRequiredString(row.generationDigest, unavailable) || + binding.target.projectId !== + postgresRequiredString(row.projectId, unavailable) || + binding.target.packageName !== + postgresRequiredString(row.packageName, unavailable) || + binding.target.installationId !== + postgresRequiredString(row.installationId, unavailable) || + binding.target.lockDigest !== + postgresRequiredString(row.lockDigest, unavailable) || + binding.target.generation !== + postgresRequiredInteger(row.generation, unavailable) || + binding.target.manifestDigest !== + postgresRequiredString(row.manifestDigest, unavailable) || + binding.authority.kind !== + postgresRequiredString(row.authorityKind, unavailable) || + binding.authority.evidenceDigest !== + postgresRequiredString(row.evidenceDigest, unavailable) || + binding.boundAtMs !== + postgresRequiredInteger(row.boundAtMs, unavailable) || + binding.bindingDigest !== + postgresRequiredString(row.bindingDigest, unavailable) + ) { + throw unavailable(); + } + return binding; + } catch (error) { + if (error instanceof PluginPackageSecretBindingUnavailableError) { + throw error; + } + throw unavailable(); + } + } + + private async findStored( + digest: string, + ): Promise | null> { + const result = await this.pool.query( + `SELECT generation_digest AS "generationDigest", + project_id AS "projectId", + package_name AS "packageName", + installation_id AS "installationId", + lock_digest AS "lockDigest", + generation, + manifest_digest AS "manifestDigest", + authority_kind AS "authorityKind", + evidence_digest AS "evidenceDigest", + bound_at_ms AS "boundAtMs", + binding_digest AS "bindingDigest", + binding_json AS "bindingJson" + FROM "ql3"."plugin_package_secret_bindings" + WHERE generation_digest = $1 + LIMIT 2`, + [digest], + ); + if (result.rows.length === 0) return null; + if (result.rows.length !== 1) throw unavailable(); + return this.parse(result.rows[0]!); + } + + async find( + digest: string, + ): Promise | null> { + try { + return await this.findStored(generationDigest(digest)); + } catch (error) { + throw mapStorageError(error); + } + } + + async publish(value: Readonly): Promise< + Readonly<{ + status: 'created' | 'existing'; + binding: Readonly; + }> + > { + const binding = normalizePluginPackageSecretBinding(value); + const bindingJson = serialize(binding); + try { + const existing = await this.findStored(binding.target.generationDigest); + if (existing) { + if (JSON.stringify(existing) !== bindingJson) { + throw new PluginPackageSecretBindingConflictError( + 'generation digest is bound to another Secret mapping', + ); + } + return Object.freeze({ + status: 'existing' as const, + binding: existing, + }); + } + + const inserted = await this.pool.query( + `INSERT INTO "ql3"."plugin_package_secret_bindings" ( + generation_digest, project_id, package_name, installation_id, + lock_digest, generation, manifest_digest, authority_kind, + evidence_digest, bound_at_ms, binding_digest, binding_json + ) + SELECT $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12::jsonb + FROM "ql3"."plugin_package_installs" AS install + INNER JOIN "ql3"."plugin_package_install_heads" AS head + ON head.installation_id = install.installation_id + AND head.project_id = install.project_id + AND head.package_name = install.package_name + WHERE install.installation_id = $4 + AND install.project_id = $2 + AND install.package_name = $3 + AND install.lock_digest = $5 + AND install.active_lock_digest = $5 + AND install.target_generation = $6 + AND install.state = 'active' + AND install.lock_json ->> 'manifestDigest' = $7 + ON CONFLICT (generation_digest) DO NOTHING + RETURNING generation_digest`, + [ + binding.target.generationDigest, + binding.target.projectId, + binding.target.packageName, + binding.target.installationId, + binding.target.lockDigest, + binding.target.generation, + binding.target.manifestDigest, + binding.authority.kind, + binding.authority.evidenceDigest, + binding.boundAtMs, + binding.bindingDigest, + bindingJson, + ], + ); + const stored = await this.findStored(binding.target.generationDigest); + if (!stored) { + throw new PluginPackageSecretBindingConflictError( + 'binding target is not the current active Package generation', + ); + } + if (JSON.stringify(stored) !== bindingJson) { + throw new PluginPackageSecretBindingConflictError( + 'generation digest is bound to another Secret mapping', + ); + } + return Object.freeze({ + status: + inserted.rows.length === 1 + ? ('created' as const) + : ('existing' as const), + binding: stored, + }); + } catch (error) { + throw mapStorageError(error); + } + } +} diff --git a/packages/ql3-cluster-postgres/src/schema/schema.ts b/packages/ql3-cluster-postgres/src/schema/schema.ts index 0a76cb46..8f5fc586 100644 --- a/packages/ql3-cluster-postgres/src/schema/schema.ts +++ b/packages/ql3-cluster-postgres/src/schema/schema.ts @@ -326,6 +326,66 @@ export const pluginPackageMaterializedRevisions = ql3Schema.table( ], ); +export const pluginPackageSecretBindings = ql3Schema.table( + 'plugin_package_secret_bindings', + { + generationDigest: char('generation_digest', { length: 64 }).primaryKey(), + projectId: varchar('project_id', { length: 128 }).notNull(), + packageName: varchar('package_name', { length: 63 }).notNull(), + installationId: varchar('installation_id', { length: 128 }).notNull(), + lockDigest: char('lock_digest', { length: 64 }).notNull(), + generation: integer('generation').notNull(), + manifestDigest: char('manifest_digest', { length: 64 }).notNull(), + authorityKind: varchar('authority_kind', { length: 32 }).notNull(), + evidenceDigest: char('evidence_digest', { length: 64 }).notNull(), + boundAtMs: bigint('bound_at_ms', { mode: 'number' }).notNull(), + bindingDigest: char('binding_digest', { length: 64 }).notNull(), + bindingJson: jsonb('binding_json') + .$type>() + .notNull(), + }, + (table) => [ + foreignKey({ + name: 'ql3_plugin_package_secret_binding_project_fk', + columns: [table.projectId], + foreignColumns: [projects.id], + }) + .onDelete('restrict') + .onUpdate('restrict'), + foreignKey({ + name: 'ql3_plugin_package_secret_binding_install_fk', + columns: [table.installationId], + foreignColumns: [pluginPackageInstalls.installationId], + }) + .onDelete('restrict') + .onUpdate('restrict'), + check( + 'ql3_plugin_package_secret_binding_identity_check', + sql`${table.projectId} ~ '^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$' and ${table.packageName} ~ '^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$' and ${table.installationId} ~ '^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$' and ${table.generation} between 1 and 2147483647 and ${table.authorityKind} in ('approved-action-execution','local-owner-confirmation') and ${table.boundAtMs} >= 0`, + ), + check( + 'ql3_plugin_package_secret_binding_digest_check', + sql`${table.generationDigest} ~ '^[0-9a-f]{64}$' and ${table.lockDigest} ~ '^[0-9a-f]{64}$' and ${table.manifestDigest} ~ '^[0-9a-f]{64}$' and ${table.evidenceDigest} ~ '^[0-9a-f]{64}$' and ${table.bindingDigest} ~ '^[0-9a-f]{64}$'`, + ), + check( + 'ql3_plugin_package_secret_binding_json_check', + sql`jsonb_typeof(${table.bindingJson}) = 'object' and octet_length(${table.bindingJson}::text) between 2 and 65536 and ${table.bindingJson} @> jsonb_build_object('schema', 'qinglong/plugin-package-secret-binding@v1', 'target', jsonb_build_object('generationDigest', ${table.generationDigest}, 'projectId', ${table.projectId}, 'packageName', ${table.packageName}, 'installationId', ${table.installationId}, 'lockDigest', ${table.lockDigest}, 'generation', ${table.generation}, 'manifestDigest', ${table.manifestDigest}), 'authority', jsonb_build_object('kind', ${table.authorityKind}, 'evidenceDigest', ${table.evidenceDigest}), 'boundAtMs', ${table.boundAtMs}, 'bindingDigest', ${table.bindingDigest}) and jsonb_typeof(${table.bindingJson} -> 'entries') = 'array' and jsonb_array_length(${table.bindingJson} -> 'entries') between 1 and 64`, + ), + uniqueIndex('ql3_plugin_package_secret_binding_generation_uidx').on( + table.projectId, + table.packageName, + table.generation, + ), + uniqueIndex('ql3_plugin_package_secret_binding_digest_uidx').on( + table.bindingDigest, + ), + index('ql3_plugin_package_secret_binding_install_idx').on( + table.installationId, + table.generationDigest, + ), + ], +); + export const projectToolDefinitionSnapshots = ql3Schema.table( 'project_tool_definition_snapshots', { @@ -5812,6 +5872,7 @@ export const ql3PostgresTables = [ pluginPackageInstallHeads, pluginPackageInstallMutations, pluginPackageMaterializedRevisions, + pluginPackageSecretBindings, projectToolDefinitionSnapshots, projectToolDefinitionSnapshotSources, pluginPackageQuarantineEvents, diff --git a/packages/ql3-cluster-postgres/src/schema/schemaContract.ts b/packages/ql3-cluster-postgres/src/schema/schemaContract.ts index 8e414f3e..c8b21c97 100644 --- a/packages/ql3-cluster-postgres/src/schema/schemaContract.ts +++ b/packages/ql3-cluster-postgres/src/schema/schemaContract.ts @@ -15,8 +15,8 @@ export interface PostgresSchemaContractFunction { export interface PostgresSchemaContract { readonly schema: 'ql3'; readonly contractName: 'control-core'; - readonly contractVersion: 57; - readonly migrationId: 'pg-0058-plugin-package-automation-disposition-events'; + readonly contractVersion: 58; + readonly migrationId: 'pg-0059-plugin-package-secret-bindings'; readonly minimumServerMajor: 16; readonly maximumServerMajor: 18; readonly capabilities: Readonly<{ @@ -55,6 +55,7 @@ export interface PostgresSchemaContract { plugin_package_lifecycle_plan: 1; plugin_package_management_quota: 1; plugin_package_materialized_revision: 1; + plugin_package_secret_binding: 1; plugin_package_proposal: 1; plugin_package_publisher_provenance: 1; plugin_package_publisher_trust_authority: 1; @@ -104,8 +105,8 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract = Object.freeze({ schema: 'ql3', contractName: 'control-core', - contractVersion: 57, - migrationId: 'pg-0058-plugin-package-automation-disposition-events', + contractVersion: 58, + migrationId: 'pg-0059-plugin-package-secret-bindings', minimumServerMajor: 16, maximumServerMajor: 18, capabilities: Object.freeze({ @@ -137,6 +138,7 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract = plugin_package_lifecycle_plan: 1, plugin_package_management_quota: 1, plugin_package_materialized_revision: 1, + plugin_package_secret_binding: 1, plugin_package_proposal: 1, plugin_package_publisher_provenance: 1, plugin_package_publisher_trust_authority: 1, @@ -242,6 +244,20 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract = 'revision_json', 'created_at_ms', ]), + table('plugin_package_secret_bindings', [ + 'generation_digest', + 'project_id', + 'package_name', + 'installation_id', + 'lock_digest', + 'generation', + 'manifest_digest', + 'authority_kind', + 'evidence_digest', + 'bound_at_ms', + 'binding_digest', + 'binding_json', + ]), table('project_tool_definition_snapshots', [ 'project_id', 'active_vector_digest', @@ -1417,6 +1433,10 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract = 'ql3_plugin_package_materialized_revision_generation_uidx', 'ql3_plugin_package_materialized_revision_lock_idx', 'ql3_plugin_package_materialized_revision_snapshot_source_uidx', + 'plugin_package_secret_bindings_pkey', + 'ql3_plugin_package_secret_binding_generation_uidx', + 'ql3_plugin_package_secret_binding_digest_uidx', + 'ql3_plugin_package_secret_binding_install_idx', 'project_tool_definition_snapshots_pkey', 'ql3_project_tool_snapshot_withdrawal_key', 'ql3_project_tool_definition_snapshot_digest_uidx', @@ -1708,6 +1728,9 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract = 'ql3_plugin_package_installs_version_check', 'ql3_plugin_package_installs_digest_check', 'ql3_plugin_package_installs_record_check', + 'ql3_plugin_package_secret_binding_identity_check', + 'ql3_plugin_package_secret_binding_digest_check', + 'ql3_plugin_package_secret_binding_json_check', 'ql3_plugin_package_quarantine_identity_check', 'ql3_plugin_package_quarantine_state_check', 'ql3_plugin_package_quarantine_subject_check', @@ -2161,6 +2184,8 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract = 'ql3_plugin_package_install_heads_install_fk', 'ql3_plugin_package_install_mutations_install_fk', 'ql3_plugin_package_materialized_revision_project_fk', + 'ql3_plugin_package_secret_binding_project_fk', + 'ql3_plugin_package_secret_binding_install_fk', 'ql3_project_tool_definition_snapshot_project_fk', 'ql3_project_tool_definition_snapshot_source_snapshot_fk', 'ql3_project_tool_definition_snapshot_source_install_fk', diff --git a/packages/ql3-cluster-postgres/src/schema/schemaReadiness.ts b/packages/ql3-cluster-postgres/src/schema/schemaReadiness.ts index 120a75a2..867a96f1 100644 --- a/packages/ql3-cluster-postgres/src/schema/schemaReadiness.ts +++ b/packages/ql3-cluster-postgres/src/schema/schemaReadiness.ts @@ -167,6 +167,12 @@ const REQUIRED_RUNTIME_PRIVILEGES = Object.freeze({ update: false, delete: false, }), + plugin_package_secret_bindings: Object.freeze({ + select: false, + insert: false, + update: false, + delete: false, + }), project_tool_definition_snapshots: Object.freeze({ select: false, insert: false, @@ -680,6 +686,12 @@ const REQUIRED_ADMIN_PRIVILEGES = Object.freeze({ update: false, delete: false, }), + plugin_package_secret_bindings: Object.freeze({ + select: false, + insert: false, + update: false, + delete: false, + }), project_tool_definition_snapshots: Object.freeze({ select: false, insert: false, @@ -1230,6 +1242,7 @@ const REQUIRED_PACKAGE_EXECUTOR_PRIVILEGES: RequiredPrivileges = Object.freeze( name === 'approved_action_dispatches' || name === 'plugin_package_install_mutations' || name === 'plugin_package_materialized_revisions' || + name === 'plugin_package_secret_bindings' || name === 'project_tool_definition_snapshots' || name === 'project_tool_definition_snapshot_sources' || name === 'plugin_package_lifecycle_plans' || diff --git a/packages/ql3-cluster-postgres/test/pluginPackageSecretBindingRepository.test.cjs b/packages/ql3-cluster-postgres/test/pluginPackageSecretBindingRepository.test.cjs new file mode 100644 index 00000000..cc728a76 --- /dev/null +++ b/packages/ql3-cluster-postgres/test/pluginPackageSecretBindingRepository.test.cjs @@ -0,0 +1,189 @@ +const assert = require('node:assert/strict'); +const { test } = require('node:test'); + +const { + PluginPackageSecretBindingConflictError, + PluginPackageSecretBindingUnavailableError, + createPluginPackageSecretBinding, +} = require('@qinglong/runtime-core/plugin-package-secret-binding'); +const { + createPluginPackageResourceGeneration, +} = require('@qinglong/runtime-core/plugin-package-resource-generation'); +const { createSecretRef } = require('@qinglong/runtime-core/secret-reference'); +const { + PostgresPluginPackageSecretBindingRepository, +} = require('../dist/plugin-package/installation/pluginPackageSecretBindingRepository'); + +const MANIFEST = { + apiVersion: 'qinglong.io/v1alpha1', + kind: 'Package', + metadata: { + name: 'example-monitor', + displayName: 'Example Monitor', + version: '1.0.0', + description: 'Secret binding repository fixture', + license: 'Apache-2.0', + }, + spec: { + compatibility: { + qinglong: '>=3.0.0-0 <4.0.0', + architectures: ['arm64'], + deploymentProfiles: ['cluster-control'], + }, + runtimes: [], + resources: { + memory: { recommended: '32Mi' }, + disk: { install: '4Mi', working: '8Mi' }, + }, + permissions: { + network: { allowedHosts: [] }, + secrets: [{ name: 'TOKEN', required: true }], + tools: [], + }, + contents: { tasks: [], workflows: [], prompts: [], tools: [] }, + }, +}; + +function fixture(boundAtMs = 100) { + const generation = createPluginPackageResourceGeneration({ + installationId: 'install-1', + projectId: 'project-1', + packageName: 'example-monitor', + lockDigest: 'a'.repeat(64), + generation: 1, + previousActiveLockDigest: null, + contentDigest: 'b'.repeat(64), + contents: MANIFEST.spec.contents, + }); + return createPluginPackageSecretBinding({ + generation, + manifest: MANIFEST, + assignments: [ + { + name: 'TOKEN', + secretRef: createSecretRef({ + projectId: 'project-1', + name: 'runtime-token', + version: 2, + }), + }, + ], + authority: { + kind: 'approved-action-execution', + evidenceDigest: 'c'.repeat(64), + }, + boundAtMs, + }); +} + +function fakePool(active = true) { + let row; + const queries = []; + return { + queries, + pool: { + async query(text, values = []) { + queries.push({ text, values }); + if (text.startsWith('SELECT')) return { rows: row ? [{ ...row }] : [] }; + if (text.startsWith('INSERT')) { + if (!active || row) return { rows: [] }; + const binding = JSON.parse(values[11]); + row = { + generationDigest: values[0], + projectId: values[1], + packageName: values[2], + installationId: values[3], + lockDigest: values[4], + generation: values[5], + manifestDigest: values[6], + authorityKind: values[7], + evidenceDigest: values[8], + boundAtMs: values[9], + bindingDigest: values[10], + bindingJson: binding, + }; + return { rows: [{ generation_digest: values[0] }] }; + } + throw new Error(`unexpected SQL: ${text}`); + }, + }, + corrupt() { + row.bindingJson.boundAtMs = 999; + }, + }; +} + +test('publishes, exact-replays and finds one binding', async () => { + const value = fakePool(); + const repository = new PostgresPluginPackageSecretBindingRepository( + value.pool, + ); + const binding = fixture(); + assert.equal((await repository.publish(binding)).status, 'created'); + assert.equal((await repository.publish(binding)).status, 'existing'); + assert.deepEqual( + await repository.find(binding.target.generationDigest), + binding, + ); + assert.match( + value.queries.find(({ text }) => text.startsWith('INSERT')).text, + /install\.state = 'active'/, + ); +}); + +test('rejects inactive targets and conflicting content', async () => { + const inactive = fakePool(false); + await assert.rejects( + new PostgresPluginPackageSecretBindingRepository(inactive.pool).publish( + fixture(), + ), + PluginPackageSecretBindingConflictError, + ); + const active = fakePool(); + const repository = new PostgresPluginPackageSecretBindingRepository( + active.pool, + ); + await repository.publish(fixture()); + await assert.rejects( + repository.publish(fixture(101)), + PluginPackageSecretBindingConflictError, + ); +}); + +test('fails closed on corrupted JSON and maps PostgreSQL constraints', async () => { + const value = fakePool(); + const repository = new PostgresPluginPackageSecretBindingRepository( + value.pool, + ); + const binding = fixture(); + await repository.publish(binding); + value.corrupt(); + await assert.rejects( + repository.find(binding.target.generationDigest), + PluginPackageSecretBindingUnavailableError, + ); + const failing = new PostgresPluginPackageSecretBindingRepository({ + async query() { + const error = new Error('duplicate'); + error.code = '23505'; + throw error; + }, + }); + await assert.rejects( + failing.find('a'.repeat(64)), + PluginPackageSecretBindingConflictError, + ); +}); + +test('publishes storage through package-executor and explicit subpath', () => { + assert.equal( + require('@qinglong/cluster-postgres/plugin-package-secret-binding') + .PostgresPluginPackageSecretBindingRepository, + PostgresPluginPackageSecretBindingRepository, + ); + assert.equal( + require('../dist/entrypoints/packageExecutor') + .PostgresPluginPackageSecretBindingRepository, + PostgresPluginPackageSecretBindingRepository, + ); +}); diff --git a/packages/ql3-cluster-postgres/test/postgresqlMigrationDefinitions.test.cjs b/packages/ql3-cluster-postgres/test/postgresqlMigrationDefinitions.test.cjs index 3ee2c8e9..4d940ab2 100644 --- a/packages/ql3-cluster-postgres/test/postgresqlMigrationDefinitions.test.cjs +++ b/packages/ql3-cluster-postgres/test/postgresqlMigrationDefinitions.test.cjs @@ -109,6 +109,7 @@ test('defines the immutable PostgreSQL capability and Run core stream', async () 'pg-0056-run-management-boundary', 'pg-0057-run-management-stop-boundary', 'pg-0058-plugin-package-automation-disposition-events', + 'pg-0059-plugin-package-secret-bindings', ], ); for (const migration of postgresqlMainMigrationStream.migrations) { @@ -160,6 +161,7 @@ test('keeps local-only and legacy tables out of the cluster baseline', async () 'plugin_package_install_heads', 'plugin_package_install_mutations', 'plugin_package_materialized_revisions', + 'plugin_package_secret_bindings', 'plugin_package_lifecycle_events', 'plugin_package_lifecycle_heads', 'plugin_package_lifecycle_receipts', @@ -534,6 +536,11 @@ test('freezes every published PostgreSQL migration checksum', () => { checksum: 'd184324909f1e450f3c1b58d422796e3869a1360df60c3f2dfe4af0bacc37471', }, + { + id: 'pg-0059-plugin-package-secret-bindings', + checksum: + '87582d256c868bd7f5af352c4b052fdab9f3714e1e7179e35d33bfa5d62957be', + }, ]; assert.deepEqual( postgresqlMainMigrationStream.migrations.map(({ id, checksum }) => ({ @@ -1958,3 +1965,26 @@ test('advances capability v56 with column-scoped Run stop authority', async () = assert.match(sql, /contract_version = 55/); assert.match(sql, /migration_id = 'pg-0056-run-management-boundary'/); }); + +test('advances capability v58 with immutable generation-bound Package Secret bindings', async () => { + const migration = migrationById('pg-0059-plugin-package-secret-bindings'); + const statements = []; + await migration.up({ + async query(statement) { + statements.push(statement); + return { rows: [] }; + }, + }); + const sql = statements.join('\n'); + assert.match(sql, /CREATE TABLE "ql3"\."plugin_package_secret_bindings"/); + assert.match(sql, /qinglong\/plugin-package-secret-binding@v1/); + assert.match(sql, /GRANT SELECT, INSERT[^;]+TO ql3_package_executor/); + assert.doesNotMatch(sql, /GRANT UPDATE|GRANT DELETE/); + assert.match(sql, /contract_version = 58/); + assert.match(sql, /"plugin_package_secret_binding":1/); + assert.match(sql, /contract_version = 57/); + assert.match( + sql, + /migration_id = 'pg-0058-plugin-package-automation-disposition-events'/, + ); +}); diff --git a/packages/ql3-cluster-postgres/test/postgresqlSchemaReadiness.test.cjs b/packages/ql3-cluster-postgres/test/postgresqlSchemaReadiness.test.cjs index 570accde..84eb2d48 100644 --- a/packages/ql3-cluster-postgres/test/postgresqlSchemaReadiness.test.cjs +++ b/packages/ql3-cluster-postgres/test/postgresqlSchemaReadiness.test.cjs @@ -74,6 +74,7 @@ function validPrivileges() { plugin_package_install_heads: [false, false, false, false], plugin_package_install_mutations: [false, false, false, false], plugin_package_materialized_revisions: [false, false, false, false], + plugin_package_secret_bindings: [false, false, false, false], project_tool_definition_snapshots: [false, false, false, false], project_tool_definition_snapshot_sources: [false, false, false, false], plugin_package_quarantine_events: [false, false, false, false], @@ -192,6 +193,7 @@ function validAdminPrivileges() { plugin_package_install_heads: [false, false, false, false], plugin_package_install_mutations: [false, false, false, false], plugin_package_materialized_revisions: [false, false, false, false], + plugin_package_secret_bindings: [false, false, false, false], project_tool_definition_snapshots: [false, false, false, false], project_tool_definition_snapshot_sources: [false, false, false, false], plugin_package_quarantine_events: [false, false, false, false], @@ -302,6 +304,7 @@ function packagePrivileges(kind) { 'plugin_package_install_heads', 'plugin_package_install_mutations', 'plugin_package_materialized_revisions', + 'plugin_package_secret_bindings', 'project_tool_definition_snapshots', 'project_tool_definition_snapshot_sources', 'plugin_package_admission_receipts', @@ -344,6 +347,7 @@ function packagePrivileges(kind) { 'plugin_package_install_heads', 'plugin_package_install_mutations', 'plugin_package_materialized_revisions', + 'plugin_package_secret_bindings', 'plugin_package_automation_publications', 'plugin_package_automation_publication_heads', 'project_tool_definition_snapshots', @@ -750,7 +754,7 @@ test('accepts the exact PostgreSQL control schema and least-privilege runtime ro serverMajor: 16, currentUser: 'ql3_runtime', contractName: 'control-core', - contractVersion: 57, + contractVersion: 58, migrationIds: [ 'pg-0001-schema-capability', 'pg-0002-run-core', @@ -810,6 +814,7 @@ test('accepts the exact PostgreSQL control schema and least-privilege runtime ro 'pg-0056-run-management-boundary', 'pg-0057-run-management-stop-boundary', 'pg-0058-plugin-package-automation-disposition-events', + 'pg-0059-plugin-package-secret-bindings', ], }); }); @@ -840,10 +845,10 @@ test('accepts the exact schema and isolated least-privilege admin role', async ( }), ); assert.equal(report.currentUser, 'ql3_admin'); - assert.equal(report.contractVersion, 57); + assert.equal(report.contractVersion, 58); assert.equal( report.migrationIds.at(-1), - 'pg-0058-plugin-package-automation-disposition-events', + 'pg-0059-plugin-package-secret-bindings', ); }); @@ -856,10 +861,10 @@ test('accepts the isolated least-privilege automation manager role', async () => }), ); assert.equal(report.currentUser, 'ql3_automation_manager'); - assert.equal(report.contractVersion, 57); + assert.equal(report.contractVersion, 58); assert.equal( report.migrationIds.at(-1), - 'pg-0058-plugin-package-automation-disposition-events', + 'pg-0059-plugin-package-secret-bindings', ); const widened = automationManagerPrivileges(); @@ -888,10 +893,10 @@ test('accepts the isolated least-privilege human Approval manager role', async ( }), ); assert.equal(report.currentUser, 'ql3_approval_manager'); - assert.equal(report.contractVersion, 57); + assert.equal(report.contractVersion, 58); assert.equal( report.migrationIds.at(-1), - 'pg-0058-plugin-package-automation-disposition-events', + 'pg-0059-plugin-package-secret-bindings', ); const widened = approvalManagerPrivileges(); @@ -922,10 +927,10 @@ test('accepts the isolated least-privilege Run manager role', async () => { }), ); assert.equal(report.currentUser, 'ql3_run_manager'); - assert.equal(report.contractVersion, 57); + assert.equal(report.contractVersion, 58); assert.equal( report.migrationIds.at(-1), - 'pg-0058-plugin-package-automation-disposition-events', + 'pg-0059-plugin-package-secret-bindings', ); const widened = runManagerPrivileges(); @@ -1057,10 +1062,10 @@ test('accepts the exact schema and isolated Worker ingress role', async () => { }), ); assert.equal(report.currentUser, 'ql3_worker_ingress'); - assert.equal(report.contractVersion, 57); + assert.equal(report.contractVersion, 58); assert.equal( report.migrationIds.at(-1), - 'pg-0058-plugin-package-automation-disposition-events', + 'pg-0059-plugin-package-secret-bindings', ); }); diff --git a/packages/ql3-local-owner-cli/test/localDeployment.test.cjs b/packages/ql3-local-owner-cli/test/localDeployment.test.cjs index 94cb06b8..7343a0fa 100644 --- a/packages/ql3-local-owner-cli/test/localDeployment.test.cjs +++ b/packages/ql3-local-owner-cli/test/localDeployment.test.cjs @@ -363,9 +363,9 @@ function composeDockerHarness( '/opt/qinglong/node_modules/@qinglong/local-application/dist/cli.js', ], Labels: { - 'io.qinglong.local.sqlite-contract-min': '45', - 'io.qinglong.local.sqlite-contract-max': '45', - 'io.qinglong.local.sqlite-write-contract': '45', + 'io.qinglong.local.sqlite-contract-min': '46', + 'io.qinglong.local.sqlite-contract-max': '46', + 'io.qinglong.local.sqlite-write-contract': '46', 'io.qinglong.local.application-config': '2', 'io.qinglong.local.compose-selection': '1', 'io.qinglong.ai': 'excluded', @@ -975,9 +975,9 @@ test('preflights exact local image, Compose merge and SQLite capability', async '/opt/qinglong/node_modules/@qinglong/local-application/dist/cli.js', ], Labels: { - 'io.qinglong.local.sqlite-contract-min': '45', - 'io.qinglong.local.sqlite-contract-max': '45', - 'io.qinglong.local.sqlite-write-contract': '45', + 'io.qinglong.local.sqlite-contract-min': '46', + 'io.qinglong.local.sqlite-contract-max': '46', + 'io.qinglong.local.sqlite-write-contract': '46', 'io.qinglong.local.application-config': '2', 'io.qinglong.local.compose-selection': '1', 'io.qinglong.ai': 'excluded', @@ -1029,7 +1029,7 @@ test('preflights exact local image, Compose merge and SQLite capability', async assert.equal(result.status, 'ready'); assert.equal(result.generation, 1); assert.equal(result.profile, 'edge'); - assert.equal(result.sqlite.contractVersion, 45); + assert.equal(result.sqlite.contractVersion, 46); assert.equal(result.image.architecture, 'arm64'); assert.equal(calls.length, 2); assert.deepEqual(calls[0].slice(0, 2), ['image', 'inspect']); @@ -1129,8 +1129,8 @@ test('applies one Compose generation and exactly replays its health receipt', as assert.equal(mode(receiptPath), 0o600); const receipt = JSON.parse(fs.readFileSync(receiptPath, 'utf8')); assert.deepEqual(receipt.sqlite, { - contractVersion: 45, - writeContractVersion: 45, + contractVersion: 46, + writeContractVersion: 46, writeObservation: 'unchanged', backup: null, }); @@ -1427,8 +1427,8 @@ test('rolls a failed Compose candidate forward to a healthy prior digest', async `${command.request.rolloutId}.sqlite`, ); assert.equal(mode(backupPath), 0o600); - assert.equal(receipt.sqlite.contractVersion, 45); - assert.equal(receipt.sqlite.writeContractVersion, 45); + assert.equal(receipt.sqlite.contractVersion, 46); + assert.equal(receipt.sqlite.writeContractVersion, 46); assert.equal(receipt.sqlite.writeObservation, 'changed'); assert.match(receipt.sqlite.backup.sha256, /^[0-9a-f]{64}$/); assert.equal(receipt.sqlite.backup.bytes > 0, true); diff --git a/packages/ql3-local-owner-cli/test/localReadiness.test.cjs b/packages/ql3-local-owner-cli/test/localReadiness.test.cjs index 749661c8..83779ab4 100644 --- a/packages/ql3-local-owner-cli/test/localReadiness.test.cjs +++ b/packages/ql3-local-owner-cli/test/localReadiness.test.cjs @@ -34,8 +34,8 @@ test('inspects the exact fresh Profile schema without exposing its path', async assert.equal(result.status, 'ready'); assert.equal(result.profile, 'edge'); assert.equal(result.storage.contractName, 'local-control-core'); - assert.equal(result.storage.contractVersion, 45); - assert.equal(result.storage.migrationCount, 90); + assert.equal(result.storage.contractVersion, 46); + assert.equal(result.storage.migrationCount, 92); assert.equal(result.storage.journalMode, 'delete'); assert.equal(JSON.stringify(result).includes(state.directory), false); }); diff --git a/packages/ql3-local-sqlite/package.json b/packages/ql3-local-sqlite/package.json index fa2574ff..424ffc06 100644 --- a/packages/ql3-local-sqlite/package.json +++ b/packages/ql3-local-sqlite/package.json @@ -85,6 +85,11 @@ "require": "./dist/plugin-package/pluginPackageMaterializedRevisionRepository.js", "default": "./dist/plugin-package/pluginPackageMaterializedRevisionRepository.js" }, + "./plugin-package-secret-binding": { + "types": "./dist/plugin-package/pluginPackageSecretBindingRepository.d.ts", + "require": "./dist/plugin-package/pluginPackageSecretBindingRepository.js", + "default": "./dist/plugin-package/pluginPackageSecretBindingRepository.js" + }, "./plugin-package-task-reconciliation": { "types": "./dist/plugin-package/pluginPackageTaskReconciliationRepository.d.ts", "require": "./dist/plugin-package/pluginPackageTaskReconciliationRepository.js", diff --git a/packages/ql3-local-sqlite/src/migration/migration.ts b/packages/ql3-local-sqlite/src/migration/migration.ts index 12058fad..1d2804c0 100644 --- a/packages/ql3-local-sqlite/src/migration/migration.ts +++ b/packages/ql3-local-sqlite/src/migration/migration.ts @@ -100,6 +100,8 @@ import { local0087RunAttemptLogRetentionMigration } from '../migrations/0087-run import { local0088CapabilityV44Migration } from '../migrations/0088-capability-v44'; import { local0089PluginPackageAutomationDispositionEventsMigration } from '../migrations/0089-plugin-package-automation-disposition-events'; import { local0090CapabilityV45Migration } from '../migrations/0090-capability-v45'; +import { local0091PluginPackageSecretBindingsMigration } from '../migrations/0091-plugin-package-secret-bindings'; +import { local0092CapabilityV46Migration } from '../migrations/0092-capability-v46'; import type { LocalSqliteMigrationContext } from '../migrations/sqlMigration'; import { LOCAL_SQLITE_MIGRATION_STREAM_ID, @@ -212,6 +214,8 @@ export const localSqliteMigrationDefinition: MigrationStreamDefinition= 0 + ), + CONSTRAINT ql3_plugin_package_secret_binding_digest_check CHECK ( + length(generation_digest) = 64 AND generation_digest NOT GLOB '*[^0-9a-f]*' AND + length(lock_digest) = 64 AND lock_digest NOT GLOB '*[^0-9a-f]*' AND + length(manifest_digest) = 64 AND manifest_digest NOT GLOB '*[^0-9a-f]*' AND + length(evidence_digest) = 64 AND evidence_digest NOT GLOB '*[^0-9a-f]*' AND + length(binding_digest) = 64 AND binding_digest NOT GLOB '*[^0-9a-f]*' + ), + CONSTRAINT ql3_plugin_package_secret_binding_json_check CHECK ( + length(CAST(binding_json AS BLOB)) BETWEEN 2 AND 65536 AND + json_valid(binding_json) AND json_type(binding_json) = 'object' AND + json_extract(binding_json, '$.schema') = 'qinglong/plugin-package-secret-binding@v1' AND + json_extract(binding_json, '$.target.generationDigest') = generation_digest AND + json_extract(binding_json, '$.target.projectId') = project_id AND + json_extract(binding_json, '$.target.packageName') = package_name AND + json_extract(binding_json, '$.target.installationId') = installation_id AND + json_extract(binding_json, '$.target.lockDigest') = lock_digest AND + json_extract(binding_json, '$.target.generation') = generation AND + json_extract(binding_json, '$.target.manifestDigest') = manifest_digest AND + json_extract(binding_json, '$.authority.kind') = authority_kind AND + json_extract(binding_json, '$.authority.evidenceDigest') = evidence_digest AND + json_extract(binding_json, '$.boundAtMs') = bound_at_ms AND + json_extract(binding_json, '$.bindingDigest') = binding_digest AND + json_type(binding_json, '$.entries') = 'array' AND + json_array_length(json_extract(binding_json, '$.entries')) BETWEEN 1 AND 64 + ) +) + `, + `CREATE UNIQUE INDEX ql3_plugin_package_secret_binding_generation_uidx ON "QingLong3PluginPackageSecretBindings" (project_id, package_name, generation)`, + `CREATE UNIQUE INDEX ql3_plugin_package_secret_binding_digest_uidx ON "QingLong3PluginPackageSecretBindings" (binding_digest)`, + `CREATE INDEX ql3_plugin_package_secret_binding_install_idx ON "QingLong3PluginPackageSecretBindings" (installation_id, generation_digest)`, + ], + }); diff --git a/packages/ql3-local-sqlite/src/migrations/0092-capability-v46.ts b/packages/ql3-local-sqlite/src/migrations/0092-capability-v46.ts new file mode 100644 index 00000000..fa780e6f --- /dev/null +++ b/packages/ql3-local-sqlite/src/migrations/0092-capability-v46.ts @@ -0,0 +1,14 @@ +import { CAPABILITIES_V45 } from './0090-capability-v45'; +import { defineLocalSqliteMigration } from './sqlMigration'; + +export const CAPABILITIES_V46 = CAPABILITIES_V45.replace( + '"plugin_package_task_reconciliation":1,', + '"plugin_package_secret_binding":1,"plugin_package_task_reconciliation":1,', +); + +export const local0092CapabilityV46Migration = defineLocalSqliteMigration({ + id: '0092-capability-v46', + statements: [ + `UPDATE "QingLong3SchemaCapabilities" SET contract_version = 46, migration_id = '0091-plugin-package-secret-bindings', capabilities = '${CAPABILITIES_V46}', updated_at_ms = CAST(unixepoch('subsec') * 1000 AS INTEGER) WHERE contract_name = 'local-control-core' AND contract_version = 45 AND migration_id = '0089-plugin-package-automation-disposition-events' AND capabilities = '${CAPABILITIES_V45}'`, + ], +}); diff --git a/packages/ql3-local-sqlite/src/plugin-package/pluginPackageSecretBindingRepository.ts b/packages/ql3-local-sqlite/src/plugin-package/pluginPackageSecretBindingRepository.ts new file mode 100644 index 00000000..6d01e513 --- /dev/null +++ b/packages/ql3-local-sqlite/src/plugin-package/pluginPackageSecretBindingRepository.ts @@ -0,0 +1,250 @@ +import type { DatabaseSync } from 'node:sqlite'; + +import { + InvalidPluginPackageSecretBindingError, + MAX_PLUGIN_PACKAGE_SECRET_BINDING_JSON_BYTES, + PluginPackageSecretBindingConflictError, + PluginPackageSecretBindingUnavailableError, + normalizePluginPackageSecretBinding, + type PluginPackageSecretBinding, + type PluginPackageSecretBindingRepository, +} from '@qinglong/runtime-core/plugin-package-secret-binding'; + +import { LocalSqliteOperationAuthority } from '../authority/operationAuthority'; + +type Row = Record; + +const DIGEST = /^[0-9a-f]{64}$/; + +function invalid(message: string): never { + throw new InvalidPluginPackageSecretBindingError(message); +} + +function text(row: Row, key: string): string { + const value = row[key]; + if (typeof value !== 'string') { + throw new PluginPackageSecretBindingUnavailableError(); + } + return value; +} + +function integer(row: Row, key: string): number { + const value = row[key]; + if (!Number.isSafeInteger(value) || (value as number) < 0) { + throw new PluginPackageSecretBindingUnavailableError(); + } + return value as number; +} + +function generationDigest(value: unknown): string { + if (typeof value !== 'string' || !DIGEST.test(value)) { + return invalid('generation digest is invalid'); + } + return value; +} + +function serialize(binding: Readonly): string { + const value = JSON.stringify(binding); + if ( + Buffer.byteLength(value, 'utf8') > + MAX_PLUGIN_PACKAGE_SECRET_BINDING_JSON_BYTES + ) { + return invalid('durable JSON byte budget exceeded'); + } + return value; +} + +function mapStorageError(error: unknown): Error { + if ( + error instanceof InvalidPluginPackageSecretBindingError || + error instanceof PluginPackageSecretBindingConflictError || + error instanceof PluginPackageSecretBindingUnavailableError + ) { + return error; + } + if ( + error && + typeof error === 'object' && + 'code' in error && + typeof error.code === 'string' && + error.code.startsWith('SQLITE_CONSTRAINT') + ) { + return new PluginPackageSecretBindingConflictError( + 'durable binding identity is already bound', + ); + } + return new PluginPackageSecretBindingUnavailableError({ + cause: error instanceof Error ? error : undefined, + }); +} + +export class LocalSqlitePluginPackageSecretBindingRepository + implements PluginPackageSecretBindingRepository +{ + private readonly authority: LocalSqliteOperationAuthority; + + constructor(authority: LocalSqliteOperationAuthority | DatabaseSync) { + this.authority = + authority instanceof LocalSqliteOperationAuthority + ? authority + : new LocalSqliteOperationAuthority(authority); + } + + private parse(row: Row): Readonly { + try { + const binding = normalizePluginPackageSecretBinding( + JSON.parse(text(row, 'bindingJson')), + ); + if ( + binding.target.generationDigest !== text(row, 'generationDigest') || + binding.target.projectId !== text(row, 'projectId') || + binding.target.packageName !== text(row, 'packageName') || + binding.target.installationId !== text(row, 'installationId') || + binding.target.lockDigest !== text(row, 'lockDigest') || + binding.target.generation !== integer(row, 'generation') || + binding.target.manifestDigest !== text(row, 'manifestDigest') || + binding.authority.kind !== text(row, 'authorityKind') || + binding.authority.evidenceDigest !== text(row, 'evidenceDigest') || + binding.boundAtMs !== integer(row, 'boundAtMs') || + binding.bindingDigest !== text(row, 'bindingDigest') + ) { + throw new PluginPackageSecretBindingUnavailableError(); + } + return binding; + } catch (error) { + if (error instanceof PluginPackageSecretBindingUnavailableError) { + throw error; + } + throw new PluginPackageSecretBindingUnavailableError(); + } + } + + private findStored( + digest: string, + ): Readonly | null { + const row = this.authority.client + .prepare( + `SELECT generation_digest AS "generationDigest", + project_id AS "projectId", + package_name AS "packageName", + installation_id AS "installationId", + lock_digest AS "lockDigest", + generation, + manifest_digest AS "manifestDigest", + authority_kind AS "authorityKind", + evidence_digest AS "evidenceDigest", + bound_at_ms AS "boundAtMs", + binding_digest AS "bindingDigest", + binding_json AS "bindingJson" + FROM "QingLong3PluginPackageSecretBindings" + WHERE generation_digest = ?`, + ) + .get(digest) as Row | undefined; + return row ? this.parse(row) : null; + } + + private enqueue(work: () => T): Promise { + return this.authority.enqueue( + async () => { + try { + return work(); + } catch (error) { + throw mapStorageError(error); + } + }, + () => new PluginPackageSecretBindingUnavailableError(), + ); + } + + async find( + digest: string, + ): Promise | null> { + const normalized = generationDigest(digest); + return await this.enqueue(() => this.findStored(normalized)); + } + + publish(value: Readonly): Promise< + Readonly<{ + status: 'created' | 'existing'; + binding: Readonly; + }> + > { + const binding = normalizePluginPackageSecretBinding(value); + const bindingJson = serialize(binding); + return this.enqueue(() => { + const existing = this.findStored(binding.target.generationDigest); + if (existing) { + if (JSON.stringify(existing) !== bindingJson) { + throw new PluginPackageSecretBindingConflictError( + 'generation digest is bound to another Secret mapping', + ); + } + return Object.freeze({ + status: 'existing' as const, + binding: existing, + }); + } + + const result = this.authority.client + .prepare( + `INSERT INTO "QingLong3PluginPackageSecretBindings" ( + generation_digest, project_id, package_name, installation_id, + lock_digest, generation, manifest_digest, authority_kind, + evidence_digest, bound_at_ms, binding_digest, binding_json + ) + SELECT ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? + FROM "QingLong3PluginPackageInstalls" AS install + INNER JOIN "QingLong3PluginPackageInstallHeads" AS head + ON head.installation_id = install.installation_id + AND head.project_id = install.project_id + AND head.package_name = install.package_name + WHERE install.installation_id = ? + AND install.project_id = ? + AND install.package_name = ? + AND install.lock_digest = ? + AND install.active_lock_digest = ? + AND install.target_generation = ? + AND install.state = 'active' + AND json_extract(install.lock_json, '$.manifestDigest') = ? + ON CONFLICT (generation_digest) DO NOTHING`, + ) + .run( + binding.target.generationDigest, + binding.target.projectId, + binding.target.packageName, + binding.target.installationId, + binding.target.lockDigest, + binding.target.generation, + binding.target.manifestDigest, + binding.authority.kind, + binding.authority.evidenceDigest, + binding.boundAtMs, + binding.bindingDigest, + bindingJson, + binding.target.installationId, + binding.target.projectId, + binding.target.packageName, + binding.target.lockDigest, + binding.target.lockDigest, + binding.target.generation, + binding.target.manifestDigest, + ); + const stored = this.findStored(binding.target.generationDigest); + if (!stored) { + throw new PluginPackageSecretBindingConflictError( + 'binding target is not the current active Package generation', + ); + } + if (JSON.stringify(stored) !== bindingJson) { + throw new PluginPackageSecretBindingConflictError( + 'generation digest is bound to another Secret mapping', + ); + } + return Object.freeze({ + status: + result.changes === 1 ? ('created' as const) : ('existing' as const), + binding: stored, + }); + }); + } +} diff --git a/packages/ql3-local-sqlite/src/readiness/readiness.ts b/packages/ql3-local-sqlite/src/readiness/readiness.ts index 7ffde8a8..5c947750 100644 --- a/packages/ql3-local-sqlite/src/readiness/readiness.ts +++ b/packages/ql3-local-sqlite/src/readiness/readiness.ts @@ -8,7 +8,7 @@ import { } from '../run/stepRunSchemaContract'; export const LOCAL_SQLITE_CONTRACT_NAME = 'local-control-core'; -export const LOCAL_SQLITE_CONTRACT_VERSION = 45; +export const LOCAL_SQLITE_CONTRACT_VERSION = 46; const PLUGIN_PACKAGE_AUTOMATION_DISPOSITION_TRIGGERS = Object.freeze([ Object.freeze({ @@ -1142,6 +1142,27 @@ const REQUIRED_SCHEMA = Object.freeze({ 'ql3_plugin_package_materialized_revision_snapshot_source_uidx', ]), }), + QingLong3PluginPackageSecretBindings: Object.freeze({ + columns: Object.freeze([ + 'generation_digest', + 'project_id', + 'package_name', + 'installation_id', + 'lock_digest', + 'generation', + 'manifest_digest', + 'authority_kind', + 'evidence_digest', + 'bound_at_ms', + 'binding_digest', + 'binding_json', + ]), + indexes: Object.freeze([ + 'ql3_plugin_package_secret_binding_generation_uidx', + 'ql3_plugin_package_secret_binding_digest_uidx', + 'ql3_plugin_package_secret_binding_install_idx', + ]), + }), QingLong3ProjectToolDefinitionSnapshots: Object.freeze({ columns: Object.freeze([ 'project_id', @@ -2533,10 +2554,10 @@ export async function auditLocalSqliteReadiness( capability.contract_name !== LOCAL_SQLITE_CONTRACT_NAME || capability.contract_version !== LOCAL_SQLITE_CONTRACT_VERSION || capability.migration_id !== - '0089-plugin-package-automation-disposition-events' || + '0091-plugin-package-secret-bindings' || typeof capability.capabilities !== 'string' || capability.capabilities !== - '{"run_core":1,"run_retry_policy":1,"completion_receipt_journal":1,"local_dispatch_plan":1,"local_secret_envelope":1,"local_project_policy":1,"local_project_administration":1,"local_security_audit":1,"local_security_audit_compaction":1,"local_secret_authorized_mutation":1,"local_identity":1,"local_api_credential":1,"local_identity_provisioning":1,"local_identity_credential_administration":1,"local_owner_bootstrap":1,"local_owner_delivery_acknowledgement":1,"api_credential_pepper_binding":1,"local_owner_pepper_catalog":1,"local_owner_credential_recovery":1,"local_owner_pepper_reference_inspection":1,"local_owner_pepper_material_gc":1,"local_owner_delivery_acknowledgement_gc":1,"task_definition":1,"local_execution_revision_digest":1,"trigger_definition":1,"legacy_adoption_ledger":1,"local_scheduler_admission":1,"plugin_package_install":1,"approved_action":1,"plugin_package_admission":1,"approved_action_execution":1,"plugin_package_proposal":1,"plugin_package_materialized_revision":1,"plugin_package_task_reconciliation":1,"project_tool_definition_snapshot":1,"step_run":1,"tool_execution_evidence":1,"tool_execution_start_barrier":1,"tool_invocation_artifact":1,"tool_execution_artifact_binding":1,"tool_execution_completion":1,"tool_execution_failure_completion":1,"tool_result_key_catalog":1,"tool_result_rekey":1,"plugin_package_quarantine":1,"plugin_package_lifecycle":1,"plugin_package_automation_publication":1,"plugin_package_automation_security_withdrawal":1,"plugin_package_workflow_admission":1,"plugin_package_workflow_run_list":1,"run_attempt_log_retention":1,"plugin_package_workflow_task_attempt_admission":1}' || + '{"run_core":1,"run_retry_policy":1,"completion_receipt_journal":1,"local_dispatch_plan":1,"local_secret_envelope":1,"local_project_policy":1,"local_project_administration":1,"local_security_audit":1,"local_security_audit_compaction":1,"local_secret_authorized_mutation":1,"local_identity":1,"local_api_credential":1,"local_identity_provisioning":1,"local_identity_credential_administration":1,"local_owner_bootstrap":1,"local_owner_delivery_acknowledgement":1,"api_credential_pepper_binding":1,"local_owner_pepper_catalog":1,"local_owner_credential_recovery":1,"local_owner_pepper_reference_inspection":1,"local_owner_pepper_material_gc":1,"local_owner_delivery_acknowledgement_gc":1,"task_definition":1,"local_execution_revision_digest":1,"trigger_definition":1,"legacy_adoption_ledger":1,"local_scheduler_admission":1,"plugin_package_install":1,"approved_action":1,"plugin_package_admission":1,"approved_action_execution":1,"plugin_package_proposal":1,"plugin_package_materialized_revision":1,"plugin_package_secret_binding":1,"plugin_package_task_reconciliation":1,"project_tool_definition_snapshot":1,"step_run":1,"tool_execution_evidence":1,"tool_execution_start_barrier":1,"tool_invocation_artifact":1,"tool_execution_artifact_binding":1,"tool_execution_completion":1,"tool_execution_failure_completion":1,"tool_result_key_catalog":1,"tool_result_rekey":1,"plugin_package_quarantine":1,"plugin_package_lifecycle":1,"plugin_package_automation_publication":1,"plugin_package_automation_security_withdrawal":1,"plugin_package_workflow_admission":1,"plugin_package_workflow_run_list":1,"run_attempt_log_retention":1,"plugin_package_workflow_task_attempt_admission":1}' || typeof capability.updated_at_ms !== 'number' || !Number.isSafeInteger(capability.updated_at_ms) || capability.updated_at_ms < 0 diff --git a/packages/ql3-local-sqlite/src/storage/schema.ts b/packages/ql3-local-sqlite/src/storage/schema.ts index b9f112e8..ccde7ea9 100644 --- a/packages/ql3-local-sqlite/src/storage/schema.ts +++ b/packages/ql3-local-sqlite/src/storage/schema.ts @@ -1217,6 +1217,62 @@ export const pluginPackageMaterializedRevisions = sqliteTable( ], ); +export const pluginPackageSecretBindings = sqliteTable( + 'QingLong3PluginPackageSecretBindings', + { + generationDigest: text('generation_digest').primaryKey(), + projectId: text('project_id') + .notNull() + .references(() => localProjects.id, { + onDelete: 'restrict', + onUpdate: 'restrict', + }), + packageName: text('package_name').notNull(), + installationId: text('installation_id') + .notNull() + .references(() => pluginPackageInstalls.installationId, { + onDelete: 'restrict', + onUpdate: 'restrict', + }), + lockDigest: text('lock_digest').notNull(), + generation: integer('generation').notNull(), + manifestDigest: text('manifest_digest').notNull(), + authorityKind: text('authority_kind').notNull(), + evidenceDigest: text('evidence_digest').notNull(), + boundAtMs: integer('bound_at_ms').notNull(), + bindingDigest: text('binding_digest').notNull(), + bindingJson: text('binding_json', { mode: 'json' }) + .$type>() + .notNull(), + }, + (table) => [ + check( + 'ql3_plugin_package_secret_binding_identity_check', + sql`length(${table.projectId}) between 1 and 128 and length(${table.packageName}) between 1 and 63 and length(${table.installationId}) between 1 and 128 and ${table.generation} between 1 and 2147483647 and ${table.authorityKind} in ('approved-action-execution','local-owner-confirmation') and ${table.boundAtMs} >= 0`, + ), + check( + 'ql3_plugin_package_secret_binding_digest_check', + sql`length(${table.generationDigest}) = 64 and ${table.generationDigest} not glob '*[^0-9a-f]*' and length(${table.lockDigest}) = 64 and ${table.lockDigest} not glob '*[^0-9a-f]*' and length(${table.manifestDigest}) = 64 and ${table.manifestDigest} not glob '*[^0-9a-f]*' and length(${table.evidenceDigest}) = 64 and ${table.evidenceDigest} not glob '*[^0-9a-f]*' and length(${table.bindingDigest}) = 64 and ${table.bindingDigest} not glob '*[^0-9a-f]*'`, + ), + check( + 'ql3_plugin_package_secret_binding_json_check', + sql`length(cast(${table.bindingJson} as blob)) between 2 and 65536 and json_valid(${table.bindingJson}) and json_type(${table.bindingJson}) = 'object' and json_extract(${table.bindingJson}, '$.schema') = 'qinglong/plugin-package-secret-binding@v1' and json_extract(${table.bindingJson}, '$.target.generationDigest') = ${table.generationDigest} and json_extract(${table.bindingJson}, '$.target.projectId') = ${table.projectId} and json_extract(${table.bindingJson}, '$.target.packageName') = ${table.packageName} and json_extract(${table.bindingJson}, '$.target.installationId') = ${table.installationId} and json_extract(${table.bindingJson}, '$.target.lockDigest') = ${table.lockDigest} and json_extract(${table.bindingJson}, '$.target.generation') = ${table.generation} and json_extract(${table.bindingJson}, '$.target.manifestDigest') = ${table.manifestDigest} and json_extract(${table.bindingJson}, '$.authority.kind') = ${table.authorityKind} and json_extract(${table.bindingJson}, '$.authority.evidenceDigest') = ${table.evidenceDigest} and json_extract(${table.bindingJson}, '$.boundAtMs') = ${table.boundAtMs} and json_extract(${table.bindingJson}, '$.bindingDigest') = ${table.bindingDigest} and json_type(${table.bindingJson}, '$.entries') = 'array' and json_array_length(json_extract(${table.bindingJson}, '$.entries')) between 1 and 64`, + ), + uniqueIndex('ql3_plugin_package_secret_binding_generation_uidx').on( + table.projectId, + table.packageName, + table.generation, + ), + uniqueIndex('ql3_plugin_package_secret_binding_digest_uidx').on( + table.bindingDigest, + ), + index('ql3_plugin_package_secret_binding_install_idx').on( + table.installationId, + table.generationDigest, + ), + ], +); + export const projectToolDefinitionSnapshots = sqliteTable( 'QingLong3ProjectToolDefinitionSnapshots', { @@ -4888,6 +4944,7 @@ export const localSqliteSchema = Object.freeze({ pluginPackageInstallHeads, pluginPackageInstallMutations, pluginPackageMaterializedRevisions, + pluginPackageSecretBindings, pluginPackageQuarantineEvents, pluginPackageWithdrawalReceipts, pluginPackageWithdrawalTasks, diff --git a/packages/ql3-local-sqlite/test/database.test.cjs b/packages/ql3-local-sqlite/test/database.test.cjs index 2c8f07d8..fdb86aa4 100644 --- a/packages/ql3-local-sqlite/test/database.test.cjs +++ b/packages/ql3-local-sqlite/test/database.test.cjs @@ -140,9 +140,11 @@ test('creates a reviewed edge database and opens runtime only after readiness', '0088-capability-v44', '0089-plugin-package-automation-disposition-events', '0090-capability-v45', + '0091-plugin-package-secret-bindings', + '0092-capability-v46', ]); assert.equal(migrated.readiness.contractName, 'local-control-core'); - assert.equal(migrated.readiness.contractVersion, 45); + assert.equal(migrated.readiness.contractVersion, 46); assert.equal(migrated.readiness.journalMode, 'delete'); assert.equal(fs.statSync(databasePath).mode & 0o777, 0o600); @@ -505,8 +507,8 @@ test('backfills v14 execution revisions with a verified independent digest', asy .get(), }, { - contract_version: 45, - migration_id: '0089-plugin-package-automation-disposition-events', + contract_version: 46, + migration_id: '0091-plugin-package-secret-bindings', }, ); } finally { @@ -693,19 +695,19 @@ test('excludes reviewed optional feature tables while preserving unknown table d const options = { databasePath, profile: 'edge' }; await migrateLocalSqlitePath(options); const client = new DatabaseSync(databasePath); - assert.equal((await auditLocalSqlitePath(options)).tableCount, 79); + assert.equal((await auditLocalSqlitePath(options)).tableCount, 80); client.exec( 'CREATE TABLE "ModelInvocationFeatureHead" (feature_id TEXT PRIMARY KEY)', ); client.close(); - assert.equal((await auditLocalSqlitePath(options)).tableCount, 79); + assert.equal((await auditLocalSqlitePath(options)).tableCount, 80); const unknownClient = new DatabaseSync(databasePath); unknownClient.exec('CREATE TABLE "UserExtensionData" (id TEXT PRIMARY KEY)'); unknownClient.close(); - assert.equal((await auditLocalSqlitePath(options)).tableCount, 80); + assert.equal((await auditLocalSqlitePath(options)).tableCount, 81); const triggerClient = new DatabaseSync(databasePath); triggerClient.exec(` diff --git a/packages/ql3-local-sqlite/test/pluginPackageSecretBindingRepository.test.cjs b/packages/ql3-local-sqlite/test/pluginPackageSecretBindingRepository.test.cjs new file mode 100644 index 00000000..bdbc475c --- /dev/null +++ b/packages/ql3-local-sqlite/test/pluginPackageSecretBindingRepository.test.cjs @@ -0,0 +1,208 @@ +const assert = require('node:assert/strict'); +const { DatabaseSync } = require('node:sqlite'); +const { test } = require('node:test'); + +const { + PluginPackageSecretBindingConflictError, + PluginPackageSecretBindingUnavailableError, + createPluginPackageSecretBinding, +} = require('@qinglong/runtime-core/plugin-package-secret-binding'); +const { + createPluginPackageResourceGeneration, +} = require('@qinglong/runtime-core/plugin-package-resource-generation'); +const { createSecretRef } = require('@qinglong/runtime-core/secret-reference'); +const { + LocalSqlitePluginPackageSecretBindingRepository, +} = require('../dist/plugin-package/pluginPackageSecretBindingRepository'); +const { migrateLocalSqliteDatabase } = require('../dist/migration/migration'); + +const LOCK_DIGEST = 'a'.repeat(64); +const MANIFEST = { + apiVersion: 'qinglong.io/v1alpha1', + kind: 'Package', + metadata: { + name: 'example-monitor', + displayName: 'Example Monitor', + version: '1.0.0', + description: 'Secret binding repository fixture', + license: 'Apache-2.0', + }, + spec: { + compatibility: { + qinglong: '>=3.0.0-0 <4.0.0', + architectures: ['arm64'], + deploymentProfiles: ['edge'], + }, + runtimes: [], + resources: { + memory: { recommended: '32Mi' }, + disk: { install: '4Mi', working: '8Mi' }, + }, + permissions: { + network: { allowedHosts: [] }, + secrets: [{ name: 'TOKEN', required: true }], + tools: [], + }, + contents: { tasks: [], workflows: [], prompts: [], tools: [] }, + }, +}; + +function fixture(boundAtMs = 100) { + const generation = createPluginPackageResourceGeneration({ + installationId: 'install-1', + projectId: 'project-1', + packageName: 'example-monitor', + lockDigest: LOCK_DIGEST, + generation: 1, + previousActiveLockDigest: null, + contentDigest: 'b'.repeat(64), + contents: MANIFEST.spec.contents, + }); + const binding = createPluginPackageSecretBinding({ + generation, + manifest: MANIFEST, + assignments: [ + { + name: 'TOKEN', + secretRef: createSecretRef({ + projectId: 'project-1', + name: 'runtime-token', + version: 2, + }), + }, + ], + authority: { + kind: 'local-owner-confirmation', + evidenceDigest: 'c'.repeat(64), + }, + boundAtMs, + }); + return { binding, generation }; +} + +async function harness(active = true) { + const client = new DatabaseSync(':memory:'); + client.exec('PRAGMA foreign_keys = ON'); + await migrateLocalSqliteDatabase(client); + const { binding, generation } = fixture(); + client + .prepare( + `INSERT INTO "QingLong3Projects" + (id, name, slug, status, version, created_at_ms, updated_at_ms) + VALUES ('project-1', 'Project 1', 'project-1', 'active', 1, 1, 1)`, + ) + .run(); + const recordDigest = 'd'.repeat(64); + const lockJson = JSON.stringify({ + lockDigest: LOCK_DIGEST, + projectId: 'project-1', + packageName: 'example-monitor', + manifestDigest: binding.target.manifestDigest, + }); + const recordJson = JSON.stringify({ + installationId: 'install-1', + projectId: 'project-1', + packageName: 'example-monitor', + lockDigest: LOCK_DIGEST, + state: active ? 'active' : 'failed', + version: 1, + recordDigest, + }); + client + .prepare( + `INSERT INTO "QingLong3PluginPackageInstalls" ( + installation_id, project_id, package_name, package_version, + operation, lock_digest, target_generation, + previous_active_lock_digest, active_lock_digest, state, version, + last_mutation_id, last_mutation_digest, lock_json, record_json, + record_digest, created_at_ms, updated_at_ms + ) VALUES (?, ?, ?, '1.0.0', 'install', ?, 1, NULL, ?, ?, 1, + 'mutation-1', ?, ?, ?, ?, 1, 1)`, + ) + .run( + 'install-1', + 'project-1', + 'example-monitor', + LOCK_DIGEST, + active ? LOCK_DIGEST : null, + active ? 'active' : 'failed', + 'e'.repeat(64), + lockJson, + recordJson, + recordDigest, + ); + client + .prepare( + `INSERT INTO "QingLong3PluginPackageInstallHeads" + (project_id, package_name, installation_id) + VALUES ('project-1', 'example-monitor', 'install-1')`, + ) + .run(); + return { + client, + binding, + generation, + repository: new LocalSqlitePluginPackageSecretBindingRepository(client), + }; +} + +test('publishes and exact-replays one active generation binding', async (t) => { + const value = await harness(); + t.after(() => value.client.close()); + const created = await value.repository.publish(value.binding); + assert.equal(created.status, 'created'); + assert.deepEqual(created.binding, value.binding); + const replay = await value.repository.publish(value.binding); + assert.equal(replay.status, 'existing'); + assert.deepEqual( + await value.repository.find(value.generation.generationDigest), + value.binding, + ); +}); + +test('rejects inactive targets and conflicting content', async (t) => { + const inactive = await harness(false); + t.after(() => inactive.client.close()); + await assert.rejects( + inactive.repository.publish(inactive.binding), + PluginPackageSecretBindingConflictError, + ); + + const active = await harness(); + t.after(() => active.client.close()); + await active.repository.publish(active.binding); + await assert.rejects( + active.repository.publish(fixture(101).binding), + PluginPackageSecretBindingConflictError, + ); +}); + +test('fails closed when durable binding JSON is changed in place', async (t) => { + const value = await harness(); + t.after(() => value.client.close()); + await value.repository.publish(value.binding); + value.client.exec('PRAGMA ignore_check_constraints = ON'); + value.client + .prepare( + `UPDATE "QingLong3PluginPackageSecretBindings" + SET binding_json = json_set(binding_json, '$.boundAtMs', 999) + WHERE generation_digest = ?`, + ) + .run(value.generation.generationDigest); + await assert.rejects( + value.repository.find(value.generation.generationDigest), + PluginPackageSecretBindingUnavailableError, + ); +}); + +test('publishes storage only through the explicit subpath', () => { + assert.equal( + require('@qinglong/local-sqlite/plugin-package-secret-binding') + .LocalSqlitePluginPackageSecretBindingRepository, + LocalSqlitePluginPackageSecretBindingRepository, + ); + assert.equal( + require('../dist').LocalSqlitePluginPackageSecretBindingRepository, + undefined, + ); +}); diff --git a/packages/ql3-local-sqlite/test/pluginPackageWorkflowAdmissionRepository.test.cjs b/packages/ql3-local-sqlite/test/pluginPackageWorkflowAdmissionRepository.test.cjs index a0125914..34dc256b 100644 --- a/packages/ql3-local-sqlite/test/pluginPackageWorkflowAdmissionRepository.test.cjs +++ b/packages/ql3-local-sqlite/test/pluginPackageWorkflowAdmissionRepository.test.cjs @@ -156,7 +156,7 @@ test('atomically admits one generation-bound Workflow Run and exactly replays it }, { runs: 1, steps: 2, events: 3, mutations: 2, admissions: 1 }, ); - assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 45); + assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 46); }); test('runs an optional authorization guard inside new and replay transactions', async (t) => { @@ -288,7 +288,7 @@ test('exactly replays immutable admission after the Workflow StepRun advances', }, { status: 'running', version: 5, eventSequence: 5 }, ); - assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 45); + assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 46); }); test('fails closed before writing when the exact installation is not active', async (t) => { diff --git a/packages/ql3-local-sqlite/test/pluginPackageWorkflowTaskAttemptAdmissionRepository.test.cjs b/packages/ql3-local-sqlite/test/pluginPackageWorkflowTaskAttemptAdmissionRepository.test.cjs index f07d05d3..ca700f0b 100644 --- a/packages/ql3-local-sqlite/test/pluginPackageWorkflowTaskAttemptAdmissionRepository.test.cjs +++ b/packages/ql3-local-sqlite/test/pluginPackageWorkflowTaskAttemptAdmissionRepository.test.cjs @@ -231,7 +231,7 @@ test('atomically admits the exact reconciled local Task revision and replays it' stepAttemptCount: 0, }, ); - assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 45); + assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 46); }); test('bounds candidate paging before SQL and fences cancellation', async (t) => { diff --git a/packages/ql3-local-sqlite/test/rolloutSafety.test.cjs b/packages/ql3-local-sqlite/test/rolloutSafety.test.cjs index cf12d66c..7beb2f6b 100644 --- a/packages/ql3-local-sqlite/test/rolloutSafety.test.cjs +++ b/packages/ql3-local-sqlite/test/rolloutSafety.test.cjs @@ -40,9 +40,9 @@ test('creates and exactly replays a reviewed rollout backup', async (t) => { await migrateLocalSqlitePath(state); const prepared = await createLocalSqliteRolloutBackup(state); assert.equal(prepared.status, 'prepared'); - assert.equal(prepared.contractVersion, 45); - assert.equal(prepared.writeContractVersion, 45); - assert.equal(LOCAL_SQLITE_WRITE_CONTRACT_VERSION, 45); + assert.equal(prepared.contractVersion, 46); + assert.equal(prepared.writeContractVersion, 46); + assert.equal(LOCAL_SQLITE_WRITE_CONTRACT_VERSION, 46); assert.match(prepared.sha256, /^[0-9a-f]{64}$/); assert.equal(prepared.bytes > 0, true); assert.equal(prepared.pageCount > 0, true); diff --git a/packages/ql3-runtime-core/package.json b/packages/ql3-runtime-core/package.json index 1cbcd983..682d9b93 100644 --- a/packages/ql3-runtime-core/package.json +++ b/packages/ql3-runtime-core/package.json @@ -83,6 +83,9 @@ "plugin-package-resource-materialization": [ "dist/plugin-package/pluginPackageResourceMaterialization.d.ts" ], + "plugin-package-secret-binding": [ + "dist/plugin-package/pluginPackageSecretBinding.d.ts" + ], "plugin-package-task-reconciliation": [ "dist/plugin-package/pluginPackageTaskReconciliation.d.ts" ], @@ -371,6 +374,11 @@ "require": "./dist/plugin-package/pluginPackageResourceMaterialization.js", "default": "./dist/plugin-package/pluginPackageResourceMaterialization.js" }, + "./plugin-package-secret-binding": { + "types": "./dist/plugin-package/pluginPackageSecretBinding.d.ts", + "require": "./dist/plugin-package/pluginPackageSecretBinding.js", + "default": "./dist/plugin-package/pluginPackageSecretBinding.js" + }, "./plugin-package-task-reconciliation": { "types": "./dist/plugin-package/pluginPackageTaskReconciliation.d.ts", "require": "./dist/plugin-package/pluginPackageTaskReconciliation.js", diff --git a/packages/ql3-runtime-core/src/plugin-package/pluginPackageSecretBinding.ts b/packages/ql3-runtime-core/src/plugin-package/pluginPackageSecretBinding.ts new file mode 100644 index 00000000..c01c52b5 --- /dev/null +++ b/packages/ql3-runtime-core/src/plugin-package/pluginPackageSecretBinding.ts @@ -0,0 +1,497 @@ +import { createHash } from 'node:crypto'; + +import { + MAX_PLUGIN_PACKAGE_SECRETS, + normalizePluginPackageManifest, + type PluginPackageManifest, + type PluginPackageSecretRequirement, +} from './pluginPackage'; +import { pluginPackageManifestDigest } from './installation/pluginPackageInstall'; +import { + normalizePluginPackageResourceGeneration, + type PluginPackageResourceGeneration, +} from './pluginPackageResourceGeneration'; +import { parseSecretRef } from '../secret/secretReference'; + +export const PLUGIN_PACKAGE_SECRET_BINDING_SCHEMA = + 'qinglong/plugin-package-secret-binding@v1' as const; +export const PLUGIN_PACKAGE_SECRET_BINDING_AUTHORITY_KINDS = [ + 'approved-action-execution', + 'local-owner-confirmation', +] as const; +export const MAX_PLUGIN_PACKAGE_SECRET_BINDING_JSON_BYTES = 64 * 1024; + +export type PluginPackageSecretBindingAuthorityKind = + (typeof PLUGIN_PACKAGE_SECRET_BINDING_AUTHORITY_KINDS)[number]; + +export interface PluginPackageSecretBindingTarget { + readonly installationId: string; + readonly projectId: string; + readonly packageName: string; + readonly lockDigest: string; + readonly generation: number; + readonly generationDigest: string; + readonly manifestDigest: string; +} + +export interface PluginPackageSecretBindingEntry { + readonly name: string; + readonly required: boolean; + readonly secretRef: string | null; +} + +export interface PluginPackageSecretBindingAuthority { + readonly kind: PluginPackageSecretBindingAuthorityKind; + readonly evidenceDigest: string; +} + +export interface PluginPackageSecretBinding { + readonly schema: typeof PLUGIN_PACKAGE_SECRET_BINDING_SCHEMA; + readonly target: Readonly; + readonly entries: readonly Readonly[]; + readonly authority: Readonly; + readonly boundAtMs: number; + readonly bindingDigest: string; +} + +export interface PluginPackageSecretBindingAssignment { + readonly name: string; + readonly secretRef: string | null; +} + +export interface CreatePluginPackageSecretBindingInput { + readonly generation: Readonly; + readonly manifest: Readonly; + readonly assignments: readonly Readonly[]; + readonly authority: Readonly; + readonly boundAtMs: number; +} + +export interface PluginPackageSecretBindingRepository { + find( + generationDigest: string, + ): Promise | null>; + publish(binding: Readonly): Promise< + Readonly<{ + status: 'created' | 'existing'; + binding: Readonly; + }> + >; +} + +export class InvalidPluginPackageSecretBindingError extends TypeError { + readonly code = 'PLUGIN_PACKAGE_SECRET_BINDING_INVALID'; + + constructor(message: string) { + super(`Plugin Package Secret binding is invalid: ${message}`); + this.name = 'InvalidPluginPackageSecretBindingError'; + } +} + +export class PluginPackageSecretBindingConflictError extends Error { + readonly code = 'PLUGIN_PACKAGE_SECRET_BINDING_CONFLICT'; + + constructor(message: string) { + super(`Plugin Package Secret binding conflicts with state: ${message}`); + this.name = 'PluginPackageSecretBindingConflictError'; + } +} + +export class PluginPackageSecretBindingUnavailableError extends Error { + readonly code = 'PLUGIN_PACKAGE_SECRET_BINDING_UNAVAILABLE'; + + constructor(options?: ErrorOptions) { + super('Plugin Package Secret binding is unavailable', options); + this.name = 'PluginPackageSecretBindingUnavailableError'; + } +} + +const DIGEST = /^[0-9a-f]{64}$/; +const IDENTIFIER = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$/; +const PACKAGE_NAME = /^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$/; +const SECRET_NAME = /^[A-Z_][A-Z0-9_]{0,127}$/; +const BINDING_DIGEST_DOMAIN = Buffer.from( + 'qinglong/plugin-package-secret-binding-digest@v1\0', + 'utf8', +); + +function invalid(message: string): never { + throw new InvalidPluginPackageSecretBindingError(message); +} + +function dataRecord(value: unknown, label: string): Record { + if ( + !value || + typeof value !== 'object' || + Array.isArray(value) || + (Object.getPrototypeOf(value) !== Object.prototype && + Object.getPrototypeOf(value) !== null) + ) { + return invalid(`${label} must be an object`); + } + const descriptors = Object.getOwnPropertyDescriptors(value); + if ( + Object.values(descriptors).some( + (descriptor) => + descriptor.get !== undefined || + descriptor.set !== undefined || + descriptor.enumerable !== true, + ) + ) { + return invalid(`${label} must contain enumerable data properties`); + } + return value as Record; +} + +function exactKeys( + value: object, + expected: readonly string[], + label: string, +): void { + const actual = Reflect.ownKeys(value); + const stringKeys = actual.filter( + (key): key is string => typeof key === 'string', + ); + const canonical = [...expected].sort(); + if ( + actual.length !== canonical.length || + stringKeys.length !== canonical.length || + stringKeys.sort().some((key, index) => key !== canonical[index]) + ) { + invalid(`${label} shape is invalid`); + } +} + +function denseArray(value: unknown, label: string): readonly unknown[] { + if (!Array.isArray(value) || value.length > MAX_PLUGIN_PACKAGE_SECRETS) { + return invalid(`${label} is invalid`); + } + const keys = Object.keys(value); + const ownKeys = Reflect.ownKeys(value); + if ( + ownKeys.length !== value.length + 1 || + !ownKeys.includes('length') || + keys.length !== value.length || + keys.some((key, index) => key !== String(index)) + ) { + return invalid(`${label} must be a dense data array`); + } + const descriptors = Object.getOwnPropertyDescriptors(value); + if ( + keys.some((key) => { + const descriptor = descriptors[key]; + return ( + descriptor === undefined || + descriptor.get !== undefined || + descriptor.set !== undefined || + descriptor.enumerable !== true + ); + }) + ) { + return invalid(`${label} must be a dense data array`); + } + return value; +} + +function digest(value: unknown, label: string): string { + if (typeof value !== 'string' || !DIGEST.test(value)) { + return invalid(`${label} is invalid`); + } + return value; +} + +function identifier(value: unknown, label: string): string { + if (typeof value !== 'string' || !IDENTIFIER.test(value)) { + return invalid(`${label} is invalid`); + } + return value; +} + +function packageName(value: unknown): string { + if (typeof value !== 'string' || !PACKAGE_NAME.test(value)) { + return invalid('Package name is invalid'); + } + return value; +} + +function generation(value: unknown): number { + if ( + !Number.isSafeInteger(value) || + (value as number) < 1 || + (value as number) > 2_147_483_647 + ) { + return invalid('generation is invalid'); + } + return value as number; +} + +function timestamp(value: unknown): number { + if (!Number.isSafeInteger(value) || (value as number) < 0) { + return invalid('boundAtMs is invalid'); + } + return value as number; +} + +function secretName(value: unknown): string { + if (typeof value !== 'string' || !SECRET_NAME.test(value)) { + return invalid('Secret requirement name is invalid'); + } + return value; +} + +function normalizeAuthority( + value: unknown, +): Readonly { + const authority = dataRecord(value, 'authority'); + exactKeys(authority, ['evidenceDigest', 'kind'], 'authority'); + if ( + !PLUGIN_PACKAGE_SECRET_BINDING_AUTHORITY_KINDS.includes( + authority.kind as PluginPackageSecretBindingAuthorityKind, + ) + ) { + return invalid('authority kind is invalid'); + } + return Object.freeze({ + kind: authority.kind as PluginPackageSecretBindingAuthorityKind, + evidenceDigest: digest( + authority.evidenceDigest, + 'authority evidence digest', + ), + }); +} + +function normalizeEntries( + value: unknown, + projectId: string, +): readonly Readonly[] { + const values = denseArray(value, 'entries'); + if (values.length === 0) return invalid('entries must not be empty'); + const seen = new Set(); + const entries = values.map((entryValue) => { + const entry = dataRecord(entryValue, 'entry'); + exactKeys(entry, ['name', 'required', 'secretRef'], 'entry'); + const name = secretName(entry.name); + if (seen.has(name)) return invalid('Secret requirement is duplicated'); + seen.add(name); + if (typeof entry.required !== 'boolean') { + return invalid('Secret requirement required flag is invalid'); + } + if (entry.secretRef === null) { + if (entry.required) return invalid(`required Secret ${name} is unbound`); + return Object.freeze({ name, required: false, secretRef: null }); + } + let reference; + try { + reference = parseSecretRef(entry.secretRef); + } catch { + return invalid(`Secret ${name} reference is invalid`); + } + if (reference.projectId !== projectId) { + return invalid(`Secret ${name} reference crosses Project boundary`); + } + if (reference.version === undefined) { + return invalid(`Secret ${name} reference must pin an explicit version`); + } + return Object.freeze({ + name, + required: entry.required, + secretRef: entry.secretRef as string, + }); + }); + const sorted = [...entries].sort((left, right) => + left.name.localeCompare(right.name), + ); + if (entries.some((entry, index) => entry !== sorted[index])) { + return invalid('entries are not in canonical order'); + } + return Object.freeze(entries); +} + +function targetFromGeneration( + generation: Readonly, + manifestDigest: string, +): Readonly { + return Object.freeze({ + installationId: generation.installationId, + projectId: generation.projectId, + packageName: generation.packageName, + lockDigest: generation.lockDigest, + generation: generation.generation, + generationDigest: generation.generationDigest, + manifestDigest, + }); +} + +function unsignedBinding( + target: Readonly, + entries: readonly Readonly[], + authority: Readonly, + boundAtMs: number, +): Omit { + return { + schema: PLUGIN_PACKAGE_SECRET_BINDING_SCHEMA, + target, + entries, + authority, + boundAtMs, + }; +} + +function bindingDigest( + value: Omit, +): string { + return createHash('sha256') + .update(BINDING_DIGEST_DOMAIN) + .update(JSON.stringify(value), 'utf8') + .digest('hex'); +} + +function entriesFromAssignments( + requirements: readonly Readonly[], + assignmentsValue: unknown, + projectId: string, +): readonly Readonly[] { + if (requirements.length === 0) { + return invalid('Manifest does not declare Secret requirements'); + } + const assignments = denseArray(assignmentsValue, 'assignments'); + const mapped = new Map(); + for (const assignmentValue of assignments) { + const assignment = dataRecord(assignmentValue, 'assignment'); + exactKeys(assignment, ['name', 'secretRef'], 'assignment'); + const name = secretName(assignment.name); + if (mapped.has(name)) return invalid('Secret assignment is duplicated'); + if ( + assignment.secretRef !== null && + typeof assignment.secretRef !== 'string' + ) { + return invalid(`Secret ${name} assignment is invalid`); + } + mapped.set(name, assignment.secretRef as string | null); + } + if ( + mapped.size !== requirements.length || + requirements.some((requirement) => !mapped.has(requirement.name)) + ) { + return invalid('assignments do not exactly match Manifest requirements'); + } + return normalizeEntries( + requirements.map((requirement) => ({ + name: requirement.name, + required: requirement.required, + secretRef: mapped.get(requirement.name) ?? null, + })), + projectId, + ); +} + +export function createPluginPackageSecretBinding( + input: CreatePluginPackageSecretBindingInput, +): Readonly { + const generation = normalizePluginPackageResourceGeneration(input.generation); + const manifest = normalizePluginPackageManifest(input.manifest); + if (manifest.metadata.name !== generation.packageName) { + return invalid('Manifest Package does not match generation'); + } + const target = targetFromGeneration( + generation, + pluginPackageManifestDigest(manifest), + ); + const entries = entriesFromAssignments( + manifest.spec.permissions.secrets, + input.assignments, + generation.projectId, + ); + const authority = normalizeAuthority(input.authority); + const boundAtMs = timestamp(input.boundAtMs); + const unsigned = unsignedBinding(target, entries, authority, boundAtMs); + return Object.freeze({ ...unsigned, bindingDigest: bindingDigest(unsigned) }); +} + +export function normalizePluginPackageSecretBinding( + value: unknown, +): Readonly { + const binding = dataRecord(value, 'binding'); + exactKeys( + binding, + ['authority', 'bindingDigest', 'boundAtMs', 'entries', 'schema', 'target'], + 'binding', + ); + if (binding.schema !== PLUGIN_PACKAGE_SECRET_BINDING_SCHEMA) { + return invalid('schema is unsupported'); + } + const targetValue = dataRecord(binding.target, 'target'); + exactKeys( + targetValue, + [ + 'generation', + 'generationDigest', + 'installationId', + 'lockDigest', + 'manifestDigest', + 'packageName', + 'projectId', + ], + 'target', + ); + const target = Object.freeze({ + installationId: identifier(targetValue.installationId, 'installation ID'), + projectId: identifier(targetValue.projectId, 'Project ID'), + packageName: packageName(targetValue.packageName), + lockDigest: digest(targetValue.lockDigest, 'lock digest'), + generation: generation(targetValue.generation), + generationDigest: digest(targetValue.generationDigest, 'generation digest'), + manifestDigest: digest(targetValue.manifestDigest, 'Manifest digest'), + }); + const entries = normalizeEntries(binding.entries, target.projectId); + const authority = normalizeAuthority(binding.authority); + const boundAtMs = timestamp(binding.boundAtMs); + const unsigned = unsignedBinding(target, entries, authority, boundAtMs); + if ( + digest(binding.bindingDigest, 'binding digest') !== bindingDigest(unsigned) + ) { + return invalid('binding digest does not match content'); + } + const normalized = Object.freeze({ + ...unsigned, + bindingDigest: binding.bindingDigest as string, + }); + if ( + Buffer.byteLength(JSON.stringify(normalized), 'utf8') > + MAX_PLUGIN_PACKAGE_SECRET_BINDING_JSON_BYTES + ) { + return invalid('durable JSON byte budget exceeded'); + } + return normalized; +} + +export function assertPluginPackageSecretBindingMatches( + value: unknown, + generationValue: Readonly, + manifestValue: Readonly, +): Readonly { + const binding = normalizePluginPackageSecretBinding(value); + const generation = normalizePluginPackageResourceGeneration(generationValue); + const manifest = normalizePluginPackageManifest(manifestValue); + const expectedTarget = targetFromGeneration( + generation, + pluginPackageManifestDigest(manifest), + ); + if (JSON.stringify(binding.target) !== JSON.stringify(expectedTarget)) { + return invalid('target does not match generation and Manifest'); + } + if ( + binding.entries.length !== manifest.spec.permissions.secrets.length || + binding.entries.some((entry, index) => { + const requirement = manifest.spec.permissions.secrets[index]; + return ( + requirement === undefined || + entry.name !== requirement.name || + entry.required !== requirement.required + ); + }) + ) { + return invalid('entries do not exactly match Manifest requirements'); + } + return binding; +} diff --git a/packages/ql3-runtime-core/test/pluginPackageSecretBinding.test.cjs b/packages/ql3-runtime-core/test/pluginPackageSecretBinding.test.cjs new file mode 100644 index 00000000..f2a17a13 --- /dev/null +++ b/packages/ql3-runtime-core/test/pluginPackageSecretBinding.test.cjs @@ -0,0 +1,249 @@ +const assert = require('node:assert/strict'); +const { test } = require('node:test'); + +const { + InvalidPluginPackageSecretBindingError, + PLUGIN_PACKAGE_SECRET_BINDING_SCHEMA, + assertPluginPackageSecretBindingMatches, + createPluginPackageSecretBinding, + normalizePluginPackageSecretBinding, +} = require('../dist/plugin-package/pluginPackageSecretBinding'); +const { + createPluginPackageResourceGeneration, +} = require('../dist/plugin-package/pluginPackageResourceGeneration'); +const { createSecretRef } = require('../dist/secret/secretReference'); +const { + PLUGIN_PACKAGE_API_VERSION, + PLUGIN_PACKAGE_KIND, +} = require('../dist/plugin-package/pluginPackage'); + +const LOCK_DIGEST = 'a'.repeat(64); +const EVIDENCE_DIGEST = 'b'.repeat(64); + +function manifest( + secretRequirements = [ + { name: 'OPTIONAL_TOKEN', required: false }, + { name: 'REQUIRED_TOKEN', required: true }, + ], +) { + return { + apiVersion: PLUGIN_PACKAGE_API_VERSION, + kind: PLUGIN_PACKAGE_KIND, + metadata: { + name: 'example-monitor', + displayName: 'Example Monitor', + version: '1.2.0', + description: 'Tests durable Secret binding', + license: 'Apache-2.0', + }, + spec: { + compatibility: { + qinglong: '>=3.0.0-0 <4.0.0', + architectures: ['arm64'], + deploymentProfiles: ['edge'], + }, + runtimes: [], + resources: { + memory: { recommended: '32Mi' }, + disk: { install: '4Mi', working: '8Mi' }, + }, + permissions: { + network: { allowedHosts: [] }, + secrets: secretRequirements, + tools: [], + }, + contents: { tasks: [], workflows: [], prompts: [], tools: [] }, + }, + }; +} + +function generation(projectId = 'project-1') { + return createPluginPackageResourceGeneration({ + installationId: 'install-1', + projectId, + packageName: 'example-monitor', + lockDigest: LOCK_DIGEST, + generation: 1, + previousActiveLockDigest: null, + contentDigest: 'c'.repeat(64), + contents: { tasks: [], workflows: [], prompts: [], tools: [] }, + }); +} + +function versionedRef(projectId, name, version = 3) { + return createSecretRef({ projectId, name, version }); +} + +function binding(overrides = {}) { + return createPluginPackageSecretBinding({ + generation: generation(), + manifest: manifest(), + assignments: [ + { name: 'OPTIONAL_TOKEN', secretRef: null }, + { + name: 'REQUIRED_TOKEN', + secretRef: versionedRef('project-1', 'runtime-token'), + }, + ], + authority: { + kind: 'local-owner-confirmation', + evidenceDigest: EVIDENCE_DIGEST, + }, + boundAtMs: 1_700_000_000_000, + ...overrides, + }); +} + +test('creates one canonical generation-bound Secret binding without plaintext', () => { + const value = binding(); + assert.equal(value.schema, PLUGIN_PACKAGE_SECRET_BINDING_SCHEMA); + assert.deepEqual(value.entries, [ + { name: 'OPTIONAL_TOKEN', required: false, secretRef: null }, + { + name: 'REQUIRED_TOKEN', + required: true, + secretRef: versionedRef('project-1', 'runtime-token'), + }, + ]); + assert.match(value.bindingDigest, /^[0-9a-f]{64}$/); + assert.equal(JSON.stringify(value).includes('secret-value'), false); + assert.deepEqual(normalizePluginPackageSecretBinding(value), value); + assert.equal(Object.isFrozen(value), true); + assert.equal(Object.isFrozen(value.entries), true); +}); + +test('requires assignments to exactly cover the Manifest contract', () => { + assert.throws( + () => binding({ assignments: [] }), + /exactly match Manifest requirements/, + ); + assert.throws( + () => + binding({ + assignments: [ + { name: 'OPTIONAL_TOKEN', secretRef: null }, + { name: 'UNDECLARED_TOKEN', secretRef: null }, + ], + }), + InvalidPluginPackageSecretBindingError, + ); + assert.throws( + () => + binding({ + assignments: [ + { name: 'OPTIONAL_TOKEN', secretRef: null }, + { name: 'REQUIRED_TOKEN', secretRef: null }, + ], + }), + /required Secret REQUIRED_TOKEN is unbound/, + ); +}); + +test('rejects floating and cross-Project Secret references', () => { + assert.throws( + () => + binding({ + assignments: [ + { name: 'OPTIONAL_TOKEN', secretRef: null }, + { + name: 'REQUIRED_TOKEN', + secretRef: createSecretRef({ + projectId: 'project-1', + name: 'runtime-token', + }), + }, + ], + }), + /must pin an explicit version/, + ); + assert.throws( + () => + binding({ + assignments: [ + { name: 'OPTIONAL_TOKEN', secretRef: null }, + { + name: 'REQUIRED_TOKEN', + secretRef: versionedRef('project-2', 'runtime-token'), + }, + ], + }), + /crosses Project boundary/, + ); +}); + +test('fails closed on authority, target, order and digest drift', () => { + assert.throws( + () => + binding({ + authority: { kind: 'operator', evidenceDigest: EVIDENCE_DIGEST }, + }), + /authority kind is invalid/, + ); + const value = binding(); + assert.throws( + () => + normalizePluginPackageSecretBinding({ + ...value, + entries: [...value.entries].reverse(), + }), + /canonical order/, + ); + assert.throws( + () => + normalizePluginPackageSecretBinding({ + ...value, + target: { ...value.target, generationDigest: 'd'.repeat(64) }, + }), + /binding digest does not match/, + ); + assert.throws( + () => normalizePluginPackageSecretBinding({ ...value, extra: true }), + /binding shape is invalid/, + ); +}); + +test('revalidates durable binding against the complete generation and Manifest', () => { + const value = binding(); + assert.deepEqual( + assertPluginPackageSecretBindingMatches(value, generation(), manifest()), + value, + ); + assert.throws( + () => + assertPluginPackageSecretBindingMatches( + value, + generation('project-2'), + manifest(), + ), + /target does not match/, + ); + assert.throws( + () => + assertPluginPackageSecretBindingMatches( + value, + generation(), + manifest([{ name: 'REQUIRED_TOKEN', required: true }]), + ), + /target does not match|entries do not exactly match/, + ); +}); + +test('rejects a binding when the Package declares no Secret requirement', () => { + assert.throws( + () => + binding({ + manifest: manifest([]), + assignments: [], + }), + /does not declare Secret requirements/, + ); +}); + +test('exports Secret binding only through its explicit subpath', () => { + assert.equal(require('../dist').createPluginPackageSecretBinding, undefined); + assert.equal( + require('@qinglong/runtime-core/plugin-package-secret-binding') + .createPluginPackageSecretBinding, + createPluginPackageSecretBinding, + ); +}); diff --git a/test/back/ql3PackageBoundaryAudit.test.cjs b/test/back/ql3PackageBoundaryAudit.test.cjs index 48994991..c4a15653 100644 --- a/test/back/ql3PackageBoundaryAudit.test.cjs +++ b/test/back/ql3PackageBoundaryAudit.test.cjs @@ -74,12 +74,12 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( [ { directory: 'packages/ql3-cluster-postgres/src/migrations', - directSourceFiles: 59, + directSourceFiles: 60, reviewKind: 'ordered_ledger', }, { directory: 'packages/ql3-local-sqlite/src/migrations', - directSourceFiles: 91, + directSourceFiles: 93, reviewKind: 'ordered_ledger', }, ], @@ -299,10 +299,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( rootSourceFileRoles: runtimeCore.rootSourceFileRoles, }, { - sourceFiles: 152, + sourceFiles: 153, rootSourceFiles: 1, rootSourceLines: 160, - nestedSourceFiles: 151, + nestedSourceFiles: 152, rootSourceFileRoles: { 'index.ts': 'public_export' }, }, ); @@ -421,10 +421,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( rootSourceFileRoles: clusterPostgres.rootSourceFileRoles, }, { - sourceFiles: 154, + sourceFiles: 156, rootSourceFiles: 1, - rootSourceLines: 125, - nestedSourceFiles: 153, + rootSourceLines: 126, + nestedSourceFiles: 155, rootSourceFileRoles: { 'index.ts': 'public_export' }, }, ); @@ -540,10 +540,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( rootSourceFileRoles: localSqlite.rootSourceFileRoles, }, { - sourceFiles: 181, + sourceFiles: 184, rootSourceFiles: 1, rootSourceLines: 31, - nestedSourceFiles: 180, + nestedSourceFiles: 183, rootSourceFileRoles: { 'index.ts': 'public_export' }, }, );