diff --git a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md index 0c826302..48ffb428 100644 --- a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md +++ b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md @@ -24,6 +24,8 @@ `10001:10001`、network none、read-only root、drop ALL、128 MiB/32 PIDs 下完成本地 TLS readiness 契约。14 个 Local Profile artifact 与 D-302 对应字节数一致；PostgreSQL 18.4 arm64 HA 123 项 gate 全绿、timeline `1→2`，证据 SHA-256 为 `e7c1743e932f2d7c35dc9153cdf5bc4a03356a38d93fce5507354652aa207a05`，独立审计与 Docker 清理通过。完整验证证据见 ADR-0391。 + +- D-304/ADR-0392（已接受）：Plugin Package 进入安全 quarantine 时，Workflow/Prompt automation publication 不再仅依赖运行时 start guard 间接拒绝，而是与 quarantine event、Package-owned Task disabled revisions、Project Tool snapshot 和 withdrawal receipt 在同一 SQLite/PostgreSQL 事务中收敛为 `withdrawn`。原先仅能引用普通 lifecycle event 的外键升级为 append-only disposition-event 联合引用，历史 migration 与 publication digest schema 保持不变；SQLite edge/standalone 崩溃矩阵覆盖 automation publication insert 后、event/task/receipt/COMMIT 前后，PostgreSQL 通过触发器在既有 `SECURITY DEFINER` quarantine commit 内登记 disposition，再由同一外层 SERIALIZABLE transaction CAS publication head。能力位为 `plugin_package_automation_security_withdrawal@1`；不新增 package、daemon、timer、连接或常驻缓存，适用于低配路由设备和集群节点。SQLite 全量 228/228；PostgreSQL package 311 pass/1 条外部 URL 条件 skip；完整 18-package build/test 退出 0，backend 1,188 pass/2 skip，package/dependency boundary 零 finding；PostgreSQL 18.4 arm64 HA 125 项 gate 全绿、timeline `1→2`，报告 SHA-256 为 `ab156901b9c96ec5a62259c44d83d24ded011e0616dc827d928f3e13efd11786`。 - D-302/ADR-0390（已接受） Cluster operator context 增加无网络、无 mutation 的内建 `ql3-cluster-admin context validate` 预检。它先复用 owner-private context reader，再让每个 entry 经过与真实请求相同的 production HTTPS/Kubernetes configuration preparation，验证精确 route、hostname、CA、 diff --git a/docs/adr/ADR-0392-atomic-plugin-package-automation-security-withdrawal.md b/docs/adr/ADR-0392-atomic-plugin-package-automation-security-withdrawal.md new file mode 100644 index 00000000..60b2ccc8 --- /dev/null +++ b/docs/adr/ADR-0392-atomic-plugin-package-automation-security-withdrawal.md @@ -0,0 +1,41 @@ +# ADR-0392：Plugin Package Automation 安全隔离原子撤回 + +- 状态：Accepted +- 日期：2026-08-13 +- 关联 RFC：QL-RFC-0001 D-304 +- 关联 ADR：ADR-0221、ADR-0222 + +## 问题 + +现有 quarantine 会原子撤回 Package-owned Task 与 Tool snapshot，并由 Run、Tool、Workflow start guard 阻止新执行；但 Workflow/Prompt automation publication head 仍可能保持 `active`。这会让产品可见状态、安全状态与恢复扫描语义不一致。 + +早期 automation publication 的 `lifecycle_event_digest` 又通过外键只允许引用普通 lifecycle event，因此 quarantine event 无法成为合法撤回证据。修改历史 migration 或发布摘要会破坏已部署数据库与 append-only 证据链。 + +## 决策 + +1. 新增 append-only automation disposition event 投影，统一登记 `lifecycle|quarantine`，但保留 publication v1 字段名与 digest 计算。 +2. SQLite 以追加 migration 重建 publication 外键，运行期先插入 quarantine event，由 trigger 登记 disposition，再以窄接口只允许 `active -> withdrawn`，全部处于同一 `BEGIN IMMEDIATE`。迁移表重建窗口临时关闭 foreign key，提交后恢复并立即执行 readiness、`foreign_key_check` 与 schema lockstep。 +3. PostgreSQL 以 `pg-0058` 追加表、触发器、外键与能力位。既有 quarantine `SECURITY DEFINER` function 写入事件时触发 disposition，repository 随后在同一 SERIALIZABLE transaction 内 CAS automation publication；任一失败整体回滚。 +4. `absent` 或已 `withdrawn` 的 publication 不制造无意义版本。publication target 与 quarantine install generation 不一致时 fail closed。 +5. receipt replay 校验目标 generation 在隔离提交点之后不存在 active automation publication；缺失或被回拨的撤回证据返回 unavailable。 +6. 能力位为 `plugin_package_automation_security_withdrawal@1`。实现不得增加 package、常驻进程、定时器、连接或缓存。 + +## 资源与部署影响 + +- edge/standalone：新增一个两列小表和两个 INSERT trigger；每次 lifecycle/quarantine 仅追加一行，常驻内存与连接数不变。 +- cluster：新增同构表、两个 statement-local trigger 和一个只供 trigger 调用的函数；disposition 投影不向任何运行角色开放读取，无新 Pod、worker 或数据库连接。 +- 历史 publication JSON、digest、lifecycle event 与 quarantine receipt 均不改写。 + +## 验证 + +- SQLite local package 全量测试、typed schema/readiness、备份恢复及 edge/standalone quarantine crash matrix。 +- PostgreSQL package 全量测试、migration checksum、Drizzle/schema/readiness/least-privilege contract。 +- PostgreSQL 18 physical-streaming HA Docker gate，验证 quarantine、automation withdrawn、COMMIT-response-loss、promotion 与 rewind 后证据存活。 + +## 当前证据 + +- SQLite 全量 228/228；quarantine crash matrix 覆盖 edge/standalone × 6 个 event、automation withdrawal、Task/Tool、receipt 与 COMMIT 前后窗口。 +- PostgreSQL package 312 项：311 pass、1 条仅在未提供独立测试数据库 URL 时条件 skip、0 fail；migration checksum、Drizzle/schema/readiness 与最小权限契约全绿。 +- Cluster Admin 304 项：302 pass/2 条件 skip；Cluster Control 232 项：230 pass/2 条件 skip；Local Owner CLI 168 项：163 pass/5 条件 skip；完整 18-package build/test 退出 0。 +- backend 1,190 项：1,188 pass/2 条件 skip/0 fail；package boundary 保持精确 18 package、无 single-source/shallow package，两个 `ordered_ledger` 目录分别锁定为 PostgreSQL 59 与 SQLite 91 个 migration source；cluster dependency audit 零 finding。 +- PostgreSQL 18.4 arm64 physical-streaming HA 125 项 gate 全绿，timeline `1→2`；包含 quarantine COMMIT-response-loss、standby 上 disposition/withdrawn head、promotion 后精确 publication chain 与旧主 rewind/rejoin。私有报告 SHA-256 为 `ab156901b9c96ec5a62259c44d83d24ded011e0616dc827d928f3e13efd11786`，测试容器、网络与卷均已清理。 diff --git a/docs/adr/README.md b/docs/adr/README.md index 115a2600..1f673e51 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -395,6 +395,7 @@ | [ADR-0389](./ADR-0389-owner-private-cluster-operator-context.md) | Owner-private Cluster Operator Context | Accepted | | [ADR-0390](./ADR-0390-offline-cluster-operator-context-preflight.md) | Offline Cluster Operator Context Preflight | Accepted | | [ADR-0391](./ADR-0391-read-only-cluster-operator-context-readiness.md) | Read-only Cluster Operator Context Readiness | Accepted | +| [ADR-0392](./ADR-0392-atomic-plugin-package-automation-security-withdrawal.md) | Plugin Package Automation 安全隔离原子撤回 | Accepted | ## 规则 diff --git a/docs/ql3-package-boundaries.json b/docs/ql3-package-boundaries.json index 98962fbe..5a2a1a84 100644 --- a/docs/ql3-package-boundaries.json +++ b/docs/ql3-package-boundaries.json @@ -6,13 +6,13 @@ "reviewedDenseDirectories": [ { "kind": "ordered_ledger", - "maxDirectSourceFiles": 58, + "maxDirectSourceFiles": 59, "path": "packages/ql3-cluster-postgres/src/migrations", "rationale": "PostgreSQL migrations are an append-only version ledger whose ordering and discoverability are safer in one reviewed directory." }, { "kind": "ordered_ledger", - "maxDirectSourceFiles": 89, + "maxDirectSourceFiles": 91, "path": "packages/ql3-local-sqlite/src/migrations", "rationale": "SQLite migrations are an append-only version ledger whose ordering and discoverability are safer in one reviewed directory." } diff --git a/packages/ql3-cluster-admin/test/bootstrap.test.cjs b/packages/ql3-cluster-admin/test/bootstrap.test.cjs index 6f68d350..4fcb1df7 100644 --- a/packages/ql3-cluster-admin/test/bootstrap.test.cjs +++ b/packages/ql3-cluster-admin/test/bootstrap.test.cjs @@ -170,6 +170,7 @@ function database(serverVersionNum = '160014') { 'plugin_package_tool_start_allowed', 'plugin_package_workflow_admission_snapshot', 'plugin_package_workflow_task_attempt_snapshot', + 'register_plugin_package_automation_disposition_event', ].includes(functionName), isOwner: false, })), diff --git a/packages/ql3-cluster-admin/test/pluginPackageRecovery.test.cjs b/packages/ql3-cluster-admin/test/pluginPackageRecovery.test.cjs index 27ccf501..c744a4d1 100644 --- a/packages/ql3-cluster-admin/test/pluginPackageRecovery.test.cjs +++ b/packages/ql3-cluster-admin/test/pluginPackageRecovery.test.cjs @@ -198,6 +198,7 @@ function database(serverVersionNum = '160014') { 'plugin_package_workflow_admission_snapshot', 'plugin_package_workflow_task_attempt_snapshot', 'lock_run_management_policy_fence', + 'register_plugin_package_automation_disposition_event', ].includes(functionName), isOwner: false, })), diff --git a/packages/ql3-cluster-control/test/application.test.cjs b/packages/ql3-cluster-control/test/application.test.cjs index c5e931b6..07849366 100644 --- a/packages/ql3-cluster-control/test/application.test.cjs +++ b/packages/ql3-cluster-control/test/application.test.cjs @@ -197,15 +197,11 @@ function runtimePrivileges() { plugin_package_lifecycle_tasks: [false, false, false, false], plugin_package_lifecycle_plans: [false, false, false, false], plugin_package_automation_publications: [true, false, false, false], + plugin_package_automation_disposition_events: [false, false, false, false], plugin_package_automation_publication_heads: [true, false, false, false], plugin_package_workflow_admissions: [true, true, false, false], plugin_package_workflow_admission_steps: [true, true, false, false], - plugin_package_workflow_task_attempt_admissions: [ - true, - true, - false, - false, - ], + plugin_package_workflow_task_attempt_admissions: [true, true, false, false], worker_execution_attestations: [true, false, false, false], run_events: [true, true, false, false], run_retry_policies: [true, true, true, false], diff --git a/packages/ql3-cluster-control/test/bootstrap.test.cjs b/packages/ql3-cluster-control/test/bootstrap.test.cjs index 46dee65e..9348093b 100644 --- a/packages/ql3-cluster-control/test/bootstrap.test.cjs +++ b/packages/ql3-cluster-control/test/bootstrap.test.cjs @@ -111,6 +111,7 @@ function runtimePrivileges() { plugin_package_lifecycle_tasks: [false, false, false, false], plugin_package_lifecycle_plans: [false, false, false, false], plugin_package_automation_publications: [true, false, false, false], + plugin_package_automation_disposition_events: [false, false, false, false], plugin_package_automation_publication_heads: [true, false, false, false], plugin_package_workflow_admissions: [true, true, false, false], plugin_package_workflow_admission_steps: [true, true, false, false], diff --git a/packages/ql3-cluster-postgres/src/migration/migrationManifest.ts b/packages/ql3-cluster-postgres/src/migration/migrationManifest.ts index 4774d936..0851f18e 100644 --- a/packages/ql3-cluster-postgres/src/migration/migrationManifest.ts +++ b/packages/ql3-cluster-postgres/src/migration/migrationManifest.ts @@ -293,5 +293,10 @@ export const postgresqlMainMigrationManifest: MigrationStreamManifest = checksum: 'ab2d0eee3d85a937e1e87243b1fd1e75181529122b64026303488404162e4ba7', }), + Object.freeze({ + id: 'pg-0058-plugin-package-automation-disposition-events', + checksum: + 'd184324909f1e450f3c1b58d422796e3869a1360df60c3f2dfe4af0bacc37471', + }), ]), }); diff --git a/packages/ql3-cluster-postgres/src/migrations/index.ts b/packages/ql3-cluster-postgres/src/migrations/index.ts index 45531ded..0f0fcc88 100644 --- a/packages/ql3-cluster-postgres/src/migrations/index.ts +++ b/packages/ql3-cluster-postgres/src/migrations/index.ts @@ -60,6 +60,7 @@ import { pg0054ApprovalManagementBoundaryMigration } from './pg-0054-approval-ma import { pg0055RunAttemptLogRetentionMigration } from './pg-0055-run-attempt-log-retention'; import { pg0056RunManagementBoundaryMigration } from '../run-management/pg-0056-run-management-boundary'; import { pg0057RunManagementStopBoundaryMigration } from '../run-management/pg-0057-run-management-stop-boundary'; +import { pg0058PluginPackageAutomationDispositionEventsMigration } from './pg-0058-plugin-package-automation-disposition-events'; export const postgresqlMainMigrationStream: MigrationStreamDefinition = Object.freeze({ @@ -125,5 +126,6 @@ export const postgresqlMainMigrationStream: MigrationStreamDefinition; type Queryable = Pick; export const CLUSTER_PLUGIN_PACKAGE_QUARANTINE_TARGET_LIMIT = 128; -function unavailable( - cause?: unknown, -): PluginPackageQuarantineUnavailableError { +function unavailable(cause?: unknown): PluginPackageQuarantineUnavailableError { return new PluginPackageQuarantineUnavailableError({ cause: cause instanceof Error ? cause : undefined, }); @@ -189,8 +189,10 @@ export class PostgresPluginPackageQuarantineRepository if (result.rows.length !== 1) throw unavailable(); try { return normalizePluginPackageQuarantineEvent( - recordJson(result.rows[0]!, 'eventJson') as unknown as - PluginPackageQuarantineEvent, + recordJson( + result.rows[0]!, + 'eventJson', + ) as unknown as PluginPackageQuarantineEvent, ); } catch (error) { if (error instanceof PluginPackageQuarantineUnavailableError) { @@ -215,8 +217,10 @@ export class PostgresPluginPackageQuarantineRepository if (result.rows.length !== 1) throw unavailable(); try { const receipt = normalizePluginPackageWithdrawalReceipt( - recordJson(result.rows[0]!, 'receiptJson') as unknown as - PluginPackageWithdrawalReceipt, + recordJson( + result.rows[0]!, + 'receiptJson', + ) as unknown as PluginPackageWithdrawalReceipt, ); assertPluginPackageWithdrawalMatchesEvent(event, receipt); await this.#assertReceiptRelations(queryable, receipt); @@ -296,8 +300,10 @@ export class PostgresPluginPackageQuarantineRepository if (snapshots.rows.length !== 1) throw unavailable(); try { const snapshot = normalizeProjectToolDefinitionSnapshot( - recordJson(snapshots.rows[0]!, 'snapshotJson') as unknown as - ProjectToolDefinitionSnapshot, + recordJson( + snapshots.rows[0]!, + 'snapshotJson', + ) as unknown as ProjectToolDefinitionSnapshot, ); if ( snapshot.sources.length !== receipt.capability.retainedSourceCount || @@ -316,6 +322,31 @@ export class PostgresPluginPackageQuarantineRepository } throw unavailable(error); } + const automation = await queryable.query( + `SELECT state, + lifecycle_event_digest AS "lifecycleEventDigest" + FROM "ql3"."plugin_package_automation_publications" + WHERE project_id = $1 AND package_name = $2 + AND installation_id = $3 AND lock_digest = $4 + ORDER BY version DESC + LIMIT 1`, + [ + receipt.target.projectId, + receipt.target.packageName, + receipt.target.installationId, + receipt.target.lockDigest, + ], + ); + if (automation.rows.length > 1) throw unavailable(); + const publication = automation.rows[0]; + if ( + publication && + (text(publication, 'state') === 'active' || + (publication.lifecycleEventDigest === receipt.eventDigest && + text(publication, 'state') !== 'withdrawn')) + ) { + throw unavailable(); + } } async #findStored( @@ -345,9 +376,7 @@ export class PostgresPluginPackageQuarantineRepository LIMIT $2`, [lockDigest, CLUSTER_PLUGIN_PACKAGE_QUARANTINE_TARGET_LIMIT + 1], ); - if ( - result.rows.length > CLUSTER_PLUGIN_PACKAGE_QUARANTINE_TARGET_LIMIT - ) { + if (result.rows.length > CLUSTER_PLUGIN_PACKAGE_QUARANTINE_TARGET_LIMIT) { throw new PluginPackageQuarantineConflictError( 'matching install targets exceed the Cluster limit', ); @@ -355,8 +384,10 @@ export class PostgresPluginPackageQuarantineRepository return Object.freeze( result.rows.map((row) => { const record = normalizePluginPackageInstallRecord( - recordJson(row, 'recordJson') as unknown as - PluginPackageInstallRecord, + recordJson( + row, + 'recordJson', + ) as unknown as PluginPackageInstallRecord, ); return Object.freeze({ projectId: record.projectId, @@ -411,8 +442,10 @@ export class PostgresPluginPackageQuarantineRepository let record: Readonly; try { record = normalizePluginPackageInstallRecord( - recordJson(result.rows[0]!, 'recordJson') as unknown as - PluginPackageInstallRecord, + recordJson( + result.rows[0]!, + 'recordJson', + ) as unknown as PluginPackageInstallRecord, ); } catch (error) { if (error instanceof PluginPackageQuarantineUnavailableError) { @@ -436,12 +469,45 @@ export class PostgresPluginPackageQuarantineRepository return record; } + async #withdrawAutomation( + client: PostgresClient, + event: Readonly, + record: Readonly, + committedAtMs: number, + ): Promise { + const publications = + new PostgresPluginPackageAutomationPublicationRepository(this.pool); + const current = await publications.findCurrentInTransaction( + client, + event.target.projectId, + event.target.packageName, + ); + if (!current) return; + if ( + current.target.installationId !== event.target.installationId || + current.target.lockDigest !== event.target.lockDigest || + current.target.generation !== record.targetGeneration + ) { + throw new PluginPackageQuarantineConflictError( + 'Workflow/Prompt publication does not match the quarantined Package generation', + ); + } + if (current.state === 'absent' || current.state === 'withdrawn') return; + await publications.publishSecurityWithdrawalInTransaction( + client, + createPluginPackageAutomationLifecyclePublication({ + previous: current, + state: 'withdrawn', + lifecycleEventDigest: event.eventDigest, + publishedAtMs: committedAtMs, + }), + ); + } + async #activeContributions( queryable: Queryable, projectId: string, - ): Promise< - readonly Readonly[] - > { + ): Promise[]> { const result = await queryable.query( `SELECT revision.revision_json AS "revisionJson" FROM "ql3"."plugin_package_install_heads" AS head @@ -532,9 +598,7 @@ export class PostgresPluginPackageQuarantineRepository MAX_PLUGIN_PACKAGE_QUARANTINE_TASK_WITHDRAWALS + 1, ], ); - if ( - result.rows.length > MAX_PLUGIN_PACKAGE_QUARANTINE_TASK_WITHDRAWALS - ) { + if (result.rows.length > MAX_PLUGIN_PACKAGE_QUARANTINE_TASK_WITHDRAWALS) { throw new PluginPackageQuarantineConflictError( 'owned Tasks exceed the quarantine withdrawal limit', ); @@ -608,14 +672,11 @@ export class PostgresPluginPackageQuarantineRepository 'event digest is bound to another quarantine', ); } - const existingReceipt = await this.#receiptByEvent( - client, - existingEvent, - ); + const existingReceipt = await this.#receiptByEvent(client, existingEvent); if (!existingReceipt) throw unavailable(); return Object.freeze({ created: false, receipt: existingReceipt }); } - await this.#install(client, event); + const install = await this.#install(client, event); const committedAtMs = Math.max( await this.#databaseNowMs(client), event.occurredAtMs, @@ -706,6 +767,9 @@ export class PostgresPluginPackageQuarantineRepository ) { throw unavailable(); } + if (event.target.installState === 'active') { + await this.#withdrawAutomation(client, event, install, committedAtMs); + } const stored = await this.#findStored(client, event.eventDigest); if (!stored || !same(stored, receipt)) throw unavailable(); return Object.freeze({ diff --git a/packages/ql3-cluster-postgres/src/plugin-package/publication/pluginPackageAutomationPublicationRepository.ts b/packages/ql3-cluster-postgres/src/plugin-package/publication/pluginPackageAutomationPublicationRepository.ts index f8add033..a43798ff 100644 --- a/packages/ql3-cluster-postgres/src/plugin-package/publication/pluginPackageAutomationPublicationRepository.ts +++ b/packages/ql3-cluster-postgres/src/plugin-package/publication/pluginPackageAutomationPublicationRepository.ts @@ -1,8 +1,5 @@ // PostgreSQL adapter owned by Plugin Package publication and recovery. -import type { - PostgresClient, - PostgresPool, -} from '@qinglong/runtime-core'; +import type { PostgresClient, PostgresPool } from '@qinglong/runtime-core'; import { InvalidPluginPackageAutomationPublicationError, MAX_PLUGIN_PACKAGE_AUTOMATION_PUBLICATION_BYTES, @@ -49,7 +46,10 @@ function unavailable( }); } -function targetIdentity(projectId: unknown, packageName: unknown): { +function targetIdentity( + projectId: unknown, + packageName: unknown, +): { readonly projectId: string; readonly packageName: string; } { @@ -147,8 +147,7 @@ export class PostgresPluginPackageAutomationPublicationRepository postgresRequiredString(row.generationDigest, unavailable) || publication.target.materializedRevisionDigest !== postgresRequiredString(row.materializedRevisionDigest, unavailable) || - publication.state !== - postgresRequiredString(row.state, unavailable) || + publication.state !== postgresRequiredString(row.state, unavailable) || publication.version !== postgresRequiredInteger(row.version, unavailable) || publication.publishedAtMs !== @@ -158,9 +157,7 @@ export class PostgresPluginPackageAutomationPublicationRepository } return publication; } catch (error) { - if ( - error instanceof PluginPackageAutomationPublicationUnavailableError - ) { + if (error instanceof PluginPackageAutomationPublicationUnavailableError) { throw error; } throw unavailable(error); @@ -393,9 +390,10 @@ export class PostgresPluginPackageAutomationPublicationRepository return this.#findCurrent(client, projectId, packageName, true); } - async publishInTransaction( + async #publishInTransaction( client: PostgresClient, value: Readonly, + securityWithdrawal: boolean, ): Promise< Readonly<{ status: 'created' | 'existing'; @@ -443,10 +441,7 @@ export class PostgresPluginPackageAutomationPublicationRepository ); } try { - assertPluginPackageAutomationPublicationSuccessor( - current, - publication, - ); + assertPluginPackageAutomationPublicationSuccessor(current, publication); } catch (error) { if (error instanceof InvalidPluginPackageAutomationPublicationError) { throw new PluginPackageAutomationPublicationConflictError( @@ -469,20 +464,14 @@ export class PostgresPluginPackageAutomationPublicationRepository ); if ( revision.rows.length !== 1 || - postgresRequiredString( - revision.rows[0]!.revisionDigest, - unavailable, - ) !== publication.target.materializedRevisionDigest || + postgresRequiredString(revision.rows[0]!.revisionDigest, unavailable) !== + publication.target.materializedRevisionDigest || postgresRequiredString(revision.rows[0]!.projectId, unavailable) !== publication.target.projectId || - postgresRequiredString( - revision.rows[0]!.packageName, - unavailable, - ) !== publication.target.packageName || - postgresRequiredInteger( - revision.rows[0]!.generation, - unavailable, - ) !== publication.target.generation || + postgresRequiredString(revision.rows[0]!.packageName, unavailable) !== + publication.target.packageName || + postgresRequiredInteger(revision.rows[0]!.generation, unavailable) !== + publication.target.generation || postgresRequiredString(revision.rows[0]!.lockDigest, unavailable) !== publication.target.lockDigest ) { @@ -490,41 +479,43 @@ export class PostgresPluginPackageAutomationPublicationRepository 'materialized revision fence does not match publication target', ); } - const securityFence = await client.query( - `SELECT - EXISTS ( - SELECT 1 - FROM "ql3"."plugin_package_quarantine_events" AS quarantine - WHERE quarantine.project_id = $1 - AND quarantine.package_name = $2 - AND quarantine.installation_id = $3 - AND quarantine.lock_digest = $4 - ) OR EXISTS ( - SELECT 1 - FROM "ql3"."plugin_package_publisher_provenance" AS provenance - JOIN "ql3"."plugin_package_publisher_revocation_receipts" AS revoked - ON revoked.publisher = provenance.publisher - AND revoked.key_id = provenance.key_id - WHERE provenance.installation_id = $3 - AND provenance.lock_digest = $4 - ) AS "blocked"`, - [ - publication.target.projectId, - publication.target.packageName, - publication.target.installationId, - publication.target.lockDigest, - ], - ); - if ( - securityFence.rows.length !== 1 || - typeof securityFence.rows[0]?.blocked !== 'boolean' - ) { - throw unavailable(); - } - if (securityFence.rows[0].blocked) { - throw new PluginPackageAutomationPublicationConflictError( - 'security-fenced Package generation cannot publish automation', + if (!securityWithdrawal) { + const securityFence = await client.query( + `SELECT + EXISTS ( + SELECT 1 + FROM "ql3"."plugin_package_quarantine_events" AS quarantine + WHERE quarantine.project_id = $1 + AND quarantine.package_name = $2 + AND quarantine.installation_id = $3 + AND quarantine.lock_digest = $4 + ) OR EXISTS ( + SELECT 1 + FROM "ql3"."plugin_package_publisher_provenance" AS provenance + JOIN "ql3"."plugin_package_publisher_revocation_receipts" AS revoked + ON revoked.publisher = provenance.publisher + AND revoked.key_id = provenance.key_id + WHERE provenance.installation_id = $3 + AND provenance.lock_digest = $4 + ) AS "blocked"`, + [ + publication.target.projectId, + publication.target.packageName, + publication.target.installationId, + publication.target.lockDigest, + ], ); + if ( + securityFence.rows.length !== 1 || + typeof securityFence.rows[0]?.blocked !== 'boolean' + ) { + throw unavailable(); + } + if (securityFence.rows[0].blocked) { + throw new PluginPackageAutomationPublicationConflictError( + 'security-fenced Package generation cannot publish automation', + ); + } } await client.query( `INSERT INTO "ql3"."plugin_package_automation_publications" ( @@ -601,13 +592,41 @@ export class PostgresPluginPackageAutomationPublicationRepository }); } - async publish( + publishInTransaction( + client: PostgresClient, value: Readonly, ): Promise< Readonly<{ status: 'created' | 'existing'; publication: Readonly; }> + > { + return this.#publishInTransaction(client, value, false); + } + + publishSecurityWithdrawalInTransaction( + client: PostgresClient, + value: Readonly, + ): Promise< + Readonly<{ + status: 'created' | 'existing'; + publication: Readonly; + }> + > { + const publication = normalizePluginPackageAutomationPublication(value); + if (publication.state !== 'withdrawn') { + throw new PluginPackageAutomationPublicationConflictError( + 'security withdrawal must narrow automation state', + ); + } + return this.#publishInTransaction(client, publication, true); + } + + async publish(value: Readonly): Promise< + Readonly<{ + status: 'created' | 'existing'; + publication: Readonly; + }> > { for ( let attempt = 0; diff --git a/packages/ql3-cluster-postgres/src/schema/schema.ts b/packages/ql3-cluster-postgres/src/schema/schema.ts index 616d556d..0a76cb46 100644 --- a/packages/ql3-cluster-postgres/src/schema/schema.ts +++ b/packages/ql3-cluster-postgres/src/schema/schema.ts @@ -1707,6 +1707,24 @@ export const pluginPackageLifecyclePlans = ql3Schema.table( ], ); +export const pluginPackageAutomationDispositionEvents = ql3Schema.table( + 'plugin_package_automation_disposition_events', + { + eventDigest: char('event_digest', { length: 64 }).primaryKey(), + eventKind: varchar('event_kind', { length: 16 }).notNull(), + }, + (table) => [ + check( + 'ql3_plugin_package_automation_disposition_kind_check', + sql`${table.eventKind} in ('lifecycle','quarantine')`, + ), + check( + 'ql3_plugin_package_automation_disposition_digest_check', + sql`${table.eventDigest} ~ '^[0-9a-f]{64}$'`, + ), + ], +); + export const pluginPackageAutomationPublications = ql3Schema.table( 'plugin_package_automation_publications', { @@ -1743,9 +1761,9 @@ export const pluginPackageAutomationPublications = ql3Schema.table( foreignColumns: [table.publicationDigest], }).onDelete('restrict'), foreignKey({ - name: 'ql3_plugin_package_automation_publication_lifecycle_fk', + name: 'ql3_plugin_package_automation_publication_disposition_fk', columns: [table.lifecycleEventDigest], - foreignColumns: [pluginPackageLifecycleEvents.eventDigest], + foreignColumns: [pluginPackageAutomationDispositionEvents.eventDigest], }).onDelete('restrict'), uniqueIndex('ql3_plugin_package_automation_publication_version_key').on( table.projectId, @@ -5653,9 +5671,11 @@ export const pluginPackageWorkflowTaskAttemptAdmissions = ql3Schema.table( uniqueIndex( 'plugin_package_workflow_task_attempt_admissions_event_id_key', ).on(table.eventId), - uniqueIndex( - 'plugin_package_workflow_task_attempt_admissions_epoch_key', - ).on(table.runId, table.stepRunId, table.stepRunVersion), + uniqueIndex('plugin_package_workflow_task_attempt_admissions_epoch_key').on( + table.runId, + table.stepRunId, + table.stepRunVersion, + ), uniqueIndex( 'plugin_package_workflow_task_attempt_admissions_number_key', ).on(table.runId, table.attemptNumber), @@ -5684,10 +5704,7 @@ export const pluginPackageWorkflowTaskAttemptAdmissions = ql3Schema.table( }).onDelete('restrict'), foreignKey({ name: 'ql3_pp_workflow_task_attempt_reconciliation_fk', - columns: [ - table.generationDigest, - table.taskReconciliationReceiptDigest, - ], + columns: [table.generationDigest, table.taskReconciliationReceiptDigest], foreignColumns: [ pluginPackageTaskReconciliations.generationDigest, pluginPackageTaskReconciliations.receiptDigest, @@ -5724,9 +5741,11 @@ export const pluginPackageWorkflowTaskAttemptAdmissions = ql3Schema.table( 'ql3_plugin_package_workflow_task_attempt_admission_json_check', sql`jsonb_typeof(${table.receiptJson}) = 'object' and octet_length(${table.receiptJson}::text) between 2 and 16384 and ${table.receiptJson} @> jsonb_build_object('schema', 'qinglong/plugin-package-workflow-task-attempt-admission@v1', 'receiptDigest', ${table.receiptDigest}, 'attemptId', ${table.attemptId}, 'planDigest', ${table.planDigest}, 'runId', ${table.runId}, 'stepRunId', ${table.stepRunId}, 'stepRunVersion', ${table.stepRunVersion}, 'stepRunDigest', ${table.stepRunDigest}, 'resourceTaskId', ${table.resourceTaskId}, 'taskReconciliationReceiptDigest', ${table.taskReconciliationReceiptDigest}, 'taskId', ${table.taskId}, 'taskRevision', ${table.taskRevision}, 'taskDefinitionDigest', ${table.taskDefinitionDigest}, 'executorType', ${table.executorType}, 'executionDigest', ${table.executionDigest}, 'attemptNumber', ${table.attemptNumber}, 'eventId', ${table.eventId}, 'runVersion', ${table.runVersion}, 'runEventSequence', ${table.runEventSequence}, 'admittedAtMs', ${table.admittedAtMs})`, ), - index( - 'ql3_pp_workflow_task_attempt_candidate_idx', - ).on(table.runId, table.stepRunId, table.admittedAtMs), + index('ql3_pp_workflow_task_attempt_candidate_idx').on( + table.runId, + table.stepRunId, + table.admittedAtMs, + ), ], ); @@ -5803,6 +5822,7 @@ export const ql3PostgresTables = [ pluginPackageLifecycleReceipts, pluginPackageLifecycleTasks, pluginPackageLifecyclePlans, + pluginPackageAutomationDispositionEvents, pluginPackageAutomationPublications, pluginPackageAutomationPublicationHeads, pluginPackageWorkflowAdmissions, diff --git a/packages/ql3-cluster-postgres/src/schema/schemaContract.ts b/packages/ql3-cluster-postgres/src/schema/schemaContract.ts index faa32eb7..8e414f3e 100644 --- a/packages/ql3-cluster-postgres/src/schema/schemaContract.ts +++ b/packages/ql3-cluster-postgres/src/schema/schemaContract.ts @@ -15,8 +15,8 @@ export interface PostgresSchemaContractFunction { export interface PostgresSchemaContract { readonly schema: 'ql3'; readonly contractName: 'control-core'; - readonly contractVersion: 56; - readonly migrationId: 'pg-0057-run-management-stop-boundary'; + readonly contractVersion: 57; + readonly migrationId: 'pg-0058-plugin-package-automation-disposition-events'; readonly minimumServerMajor: 16; readonly maximumServerMajor: 18; readonly capabilities: Readonly<{ @@ -44,6 +44,7 @@ export interface PostgresSchemaContract { plugin_package_admission: 1; plugin_package_authority_split: 1; plugin_package_automation_publication: 1; + plugin_package_automation_security_withdrawal: 1; plugin_package_automation_start_guard: 1; plugin_package_workflow_admission: 1; plugin_package_workflow_run_list: 1; @@ -103,8 +104,8 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract = Object.freeze({ schema: 'ql3', contractName: 'control-core', - contractVersion: 56, - migrationId: 'pg-0057-run-management-stop-boundary', + contractVersion: 57, + migrationId: 'pg-0058-plugin-package-automation-disposition-events', minimumServerMajor: 16, maximumServerMajor: 18, capabilities: Object.freeze({ @@ -125,6 +126,7 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract = plugin_package_admission: 1, plugin_package_authority_split: 1, plugin_package_automation_publication: 1, + plugin_package_automation_security_withdrawal: 1, plugin_package_automation_start_guard: 1, plugin_package_workflow_admission: 1, plugin_package_workflow_run_list: 1, @@ -382,6 +384,10 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract = 'expires_at_ms', 'plan_json', ]), + table('plugin_package_automation_disposition_events', [ + 'event_digest', + 'event_kind', + ]), table('plugin_package_automation_publications', [ 'publication_digest', 'project_id', @@ -1445,6 +1451,7 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract = 'ql3_plugin_package_lifecycle_plan_impact_key', 'ql3_plugin_package_lifecycle_plan_expiry_idx', 'plugin_package_automation_publications_pkey', + 'plugin_package_automation_disposition_events_pkey', 'ql3_plugin_package_automation_publication_version_key', 'ql3_plugin_package_automation_publication_previous_key', 'ql3_plugin_package_automation_publication_generation_idx', @@ -1736,6 +1743,8 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract = 'ql3_plugin_package_automation_publication_digest_check', 'ql3_plugin_package_automation_publication_json_check', 'ql3_plugin_package_automation_publication_head_state_check', + 'ql3_plugin_package_automation_disposition_kind_check', + 'ql3_plugin_package_automation_disposition_digest_check', 'ql3_plugin_package_workflow_admission_identity_check', 'ql3_plugin_package_workflow_admission_digest_check', 'ql3_plugin_package_workflow_admission_json_check', @@ -2175,7 +2184,7 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract = 'ql3_plugin_package_lifecycle_plan_install_fk', 'ql3_plugin_package_automation_publication_revision_fk', 'ql3_plugin_package_automation_publication_previous_fk', - 'ql3_plugin_package_automation_publication_lifecycle_fk', + 'ql3_plugin_package_automation_publication_disposition_fk', 'ql3_plugin_package_automation_publication_head_publication_fk', 'ql3_plugin_package_workflow_admission_run_fk', 'ql3_plugin_package_workflow_admission_publication_fk', @@ -2302,6 +2311,14 @@ export const postgresqlControlSchemaContract: PostgresSchemaContract = 'ql3_run_retry_policies_run_fk', ]), functions: Object.freeze([ + Object.freeze({ + name: 'register_plugin_package_automation_disposition_event', + identityArguments: '', + owner: 'ql3_migration', + securityDefiner: true, + volatility: 'volatile', + configuration: Object.freeze(['search_path=pg_catalog, ql3']), + }), Object.freeze({ name: 'enforce_plugin_package_stage_provenance', identityArguments: '', diff --git a/packages/ql3-cluster-postgres/src/schema/schemaReadiness.ts b/packages/ql3-cluster-postgres/src/schema/schemaReadiness.ts index ac09942f..120a75a2 100644 --- a/packages/ql3-cluster-postgres/src/schema/schemaReadiness.ts +++ b/packages/ql3-cluster-postgres/src/schema/schemaReadiness.ts @@ -233,6 +233,12 @@ const REQUIRED_RUNTIME_PRIVILEGES = Object.freeze({ update: false, delete: false, }), + plugin_package_automation_disposition_events: Object.freeze({ + select: false, + insert: false, + update: false, + delete: false, + }), plugin_package_automation_publication_heads: Object.freeze({ select: true, insert: false, @@ -740,6 +746,12 @@ const REQUIRED_ADMIN_PRIVILEGES = Object.freeze({ update: false, delete: false, }), + plugin_package_automation_disposition_events: Object.freeze({ + select: false, + insert: false, + update: false, + delete: false, + }), plugin_package_automation_publication_heads: Object.freeze({ select: false, insert: false, @@ -1492,6 +1504,7 @@ const REQUIRED_RUNTIME_FUNCTION_PRIVILEGES: RequiredFunctionPrivileges = plugin_package_lifecycle_blocking_runs: false, plugin_package_run_start_allowed: true, plugin_package_tool_start_allowed: true, + register_plugin_package_automation_disposition_event: false, }); const REQUIRED_PACKAGE_MANAGER_FUNCTION_PRIVILEGES: RequiredFunctionPrivileges = @@ -1509,6 +1522,7 @@ const REQUIRED_PACKAGE_MANAGER_FUNCTION_PRIVILEGES: RequiredFunctionPrivileges = plugin_package_lifecycle_blocking_runs: false, plugin_package_run_start_allowed: false, plugin_package_tool_start_allowed: false, + register_plugin_package_automation_disposition_event: false, }); const REQUIRED_PACKAGE_EXECUTOR_FUNCTION_PRIVILEGES: RequiredFunctionPrivileges = @@ -1526,6 +1540,7 @@ const REQUIRED_PACKAGE_EXECUTOR_FUNCTION_PRIVILEGES: RequiredFunctionPrivileges plugin_package_lifecycle_blocking_runs: true, plugin_package_run_start_allowed: false, plugin_package_tool_start_allowed: false, + register_plugin_package_automation_disposition_event: false, }); const REQUIRED_WORKER_CREDENTIAL_FUNCTION_PRIVILEGES: RequiredFunctionPrivileges = diff --git a/packages/ql3-cluster-postgres/test/postgresqlMigrationDefinitions.test.cjs b/packages/ql3-cluster-postgres/test/postgresqlMigrationDefinitions.test.cjs index 460c2d17..3ee2c8e9 100644 --- a/packages/ql3-cluster-postgres/test/postgresqlMigrationDefinitions.test.cjs +++ b/packages/ql3-cluster-postgres/test/postgresqlMigrationDefinitions.test.cjs @@ -108,6 +108,7 @@ test('defines the immutable PostgreSQL capability and Run core stream', async () 'pg-0055-run-attempt-log-retention', 'pg-0056-run-management-boundary', 'pg-0057-run-management-stop-boundary', + 'pg-0058-plugin-package-automation-disposition-events', ], ); for (const migration of postgresqlMainMigrationStream.migrations) { @@ -528,6 +529,11 @@ test('freezes every published PostgreSQL migration checksum', () => { checksum: 'ab2d0eee3d85a937e1e87243b1fd1e75181529122b64026303488404162e4ba7', }, + { + id: 'pg-0058-plugin-package-automation-disposition-events', + checksum: + 'd184324909f1e450f3c1b58d422796e3869a1360df60c3f2dfe4af0bacc37471', + }, ]; assert.deepEqual( postgresqlMainMigrationStream.migrations.map(({ id, checksum }) => ({ diff --git a/packages/ql3-cluster-postgres/test/postgresqlSchemaReadiness.test.cjs b/packages/ql3-cluster-postgres/test/postgresqlSchemaReadiness.test.cjs index 16588cfc..570accde 100644 --- a/packages/ql3-cluster-postgres/test/postgresqlSchemaReadiness.test.cjs +++ b/packages/ql3-cluster-postgres/test/postgresqlSchemaReadiness.test.cjs @@ -85,6 +85,7 @@ function validPrivileges() { plugin_package_lifecycle_tasks: [false, false, false, false], plugin_package_lifecycle_plans: [false, false, false, false], plugin_package_automation_publications: [true, false, false, false], + plugin_package_automation_disposition_events: [false, false, false, false], plugin_package_automation_publication_heads: [true, false, false, false], plugin_package_workflow_admissions: [true, true, false, false], plugin_package_workflow_admission_steps: [true, true, false, false], @@ -202,6 +203,7 @@ function validAdminPrivileges() { plugin_package_lifecycle_tasks: [false, false, false, false], plugin_package_lifecycle_plans: [false, false, false, false], plugin_package_automation_publications: [false, false, false, false], + plugin_package_automation_disposition_events: [false, false, false, false], plugin_package_automation_publication_heads: [false, false, false, false], plugin_package_workflow_admissions: [false, false, false, false], plugin_package_workflow_admission_steps: [false, false, false, false], @@ -748,7 +750,7 @@ test('accepts the exact PostgreSQL control schema and least-privilege runtime ro serverMajor: 16, currentUser: 'ql3_runtime', contractName: 'control-core', - contractVersion: 56, + contractVersion: 57, migrationIds: [ 'pg-0001-schema-capability', 'pg-0002-run-core', @@ -807,6 +809,7 @@ test('accepts the exact PostgreSQL control schema and least-privilege runtime ro 'pg-0055-run-attempt-log-retention', 'pg-0056-run-management-boundary', 'pg-0057-run-management-stop-boundary', + 'pg-0058-plugin-package-automation-disposition-events', ], }); }); @@ -837,10 +840,10 @@ test('accepts the exact schema and isolated least-privilege admin role', async ( }), ); assert.equal(report.currentUser, 'ql3_admin'); - assert.equal(report.contractVersion, 56); + assert.equal(report.contractVersion, 57); assert.equal( report.migrationIds.at(-1), - 'pg-0057-run-management-stop-boundary', + 'pg-0058-plugin-package-automation-disposition-events', ); }); @@ -853,10 +856,10 @@ test('accepts the isolated least-privilege automation manager role', async () => }), ); assert.equal(report.currentUser, 'ql3_automation_manager'); - assert.equal(report.contractVersion, 56); + assert.equal(report.contractVersion, 57); assert.equal( report.migrationIds.at(-1), - 'pg-0057-run-management-stop-boundary', + 'pg-0058-plugin-package-automation-disposition-events', ); const widened = automationManagerPrivileges(); @@ -885,10 +888,10 @@ test('accepts the isolated least-privilege human Approval manager role', async ( }), ); assert.equal(report.currentUser, 'ql3_approval_manager'); - assert.equal(report.contractVersion, 56); + assert.equal(report.contractVersion, 57); assert.equal( report.migrationIds.at(-1), - 'pg-0057-run-management-stop-boundary', + 'pg-0058-plugin-package-automation-disposition-events', ); const widened = approvalManagerPrivileges(); @@ -919,10 +922,10 @@ test('accepts the isolated least-privilege Run manager role', async () => { }), ); assert.equal(report.currentUser, 'ql3_run_manager'); - assert.equal(report.contractVersion, 56); + assert.equal(report.contractVersion, 57); assert.equal( report.migrationIds.at(-1), - 'pg-0057-run-management-stop-boundary', + 'pg-0058-plugin-package-automation-disposition-events', ); const widened = runManagerPrivileges(); @@ -1054,10 +1057,10 @@ test('accepts the exact schema and isolated Worker ingress role', async () => { }), ); assert.equal(report.currentUser, 'ql3_worker_ingress'); - assert.equal(report.contractVersion, 56); + assert.equal(report.contractVersion, 57); assert.equal( report.migrationIds.at(-1), - 'pg-0057-run-management-stop-boundary', + 'pg-0058-plugin-package-automation-disposition-events', ); }); diff --git a/packages/ql3-local-owner-cli/test/localDeployment.test.cjs b/packages/ql3-local-owner-cli/test/localDeployment.test.cjs index 5fd61887..94cb06b8 100644 --- a/packages/ql3-local-owner-cli/test/localDeployment.test.cjs +++ b/packages/ql3-local-owner-cli/test/localDeployment.test.cjs @@ -363,9 +363,9 @@ function composeDockerHarness( '/opt/qinglong/node_modules/@qinglong/local-application/dist/cli.js', ], Labels: { - 'io.qinglong.local.sqlite-contract-min': '44', - 'io.qinglong.local.sqlite-contract-max': '44', - 'io.qinglong.local.sqlite-write-contract': '44', + 'io.qinglong.local.sqlite-contract-min': '45', + 'io.qinglong.local.sqlite-contract-max': '45', + 'io.qinglong.local.sqlite-write-contract': '45', 'io.qinglong.local.application-config': '2', 'io.qinglong.local.compose-selection': '1', 'io.qinglong.ai': 'excluded', @@ -975,9 +975,9 @@ test('preflights exact local image, Compose merge and SQLite capability', async '/opt/qinglong/node_modules/@qinglong/local-application/dist/cli.js', ], Labels: { - 'io.qinglong.local.sqlite-contract-min': '44', - 'io.qinglong.local.sqlite-contract-max': '44', - 'io.qinglong.local.sqlite-write-contract': '44', + 'io.qinglong.local.sqlite-contract-min': '45', + 'io.qinglong.local.sqlite-contract-max': '45', + 'io.qinglong.local.sqlite-write-contract': '45', 'io.qinglong.local.application-config': '2', 'io.qinglong.local.compose-selection': '1', 'io.qinglong.ai': 'excluded', @@ -1029,7 +1029,7 @@ test('preflights exact local image, Compose merge and SQLite capability', async assert.equal(result.status, 'ready'); assert.equal(result.generation, 1); assert.equal(result.profile, 'edge'); - assert.equal(result.sqlite.contractVersion, 44); + assert.equal(result.sqlite.contractVersion, 45); assert.equal(result.image.architecture, 'arm64'); assert.equal(calls.length, 2); assert.deepEqual(calls[0].slice(0, 2), ['image', 'inspect']); @@ -1129,8 +1129,8 @@ test('applies one Compose generation and exactly replays its health receipt', as assert.equal(mode(receiptPath), 0o600); const receipt = JSON.parse(fs.readFileSync(receiptPath, 'utf8')); assert.deepEqual(receipt.sqlite, { - contractVersion: 44, - writeContractVersion: 44, + contractVersion: 45, + writeContractVersion: 45, writeObservation: 'unchanged', backup: null, }); @@ -1427,8 +1427,8 @@ test('rolls a failed Compose candidate forward to a healthy prior digest', async `${command.request.rolloutId}.sqlite`, ); assert.equal(mode(backupPath), 0o600); - assert.equal(receipt.sqlite.contractVersion, 44); - assert.equal(receipt.sqlite.writeContractVersion, 44); + assert.equal(receipt.sqlite.contractVersion, 45); + assert.equal(receipt.sqlite.writeContractVersion, 45); assert.equal(receipt.sqlite.writeObservation, 'changed'); assert.match(receipt.sqlite.backup.sha256, /^[0-9a-f]{64}$/); assert.equal(receipt.sqlite.backup.bytes > 0, true); diff --git a/packages/ql3-local-owner-cli/test/localReadiness.test.cjs b/packages/ql3-local-owner-cli/test/localReadiness.test.cjs index ad39af01..749661c8 100644 --- a/packages/ql3-local-owner-cli/test/localReadiness.test.cjs +++ b/packages/ql3-local-owner-cli/test/localReadiness.test.cjs @@ -34,8 +34,8 @@ test('inspects the exact fresh Profile schema without exposing its path', async assert.equal(result.status, 'ready'); assert.equal(result.profile, 'edge'); assert.equal(result.storage.contractName, 'local-control-core'); - assert.equal(result.storage.contractVersion, 44); - assert.equal(result.storage.migrationCount, 88); + assert.equal(result.storage.contractVersion, 45); + assert.equal(result.storage.migrationCount, 90); assert.equal(result.storage.journalMode, 'delete'); assert.equal(JSON.stringify(result).includes(state.directory), false); }); diff --git a/packages/ql3-local-owner-cli/test/pluginPackageCommand.test.cjs b/packages/ql3-local-owner-cli/test/pluginPackageCommand.test.cjs index b672ec34..eba009ed 100644 --- a/packages/ql3-local-owner-cli/test/pluginPackageCommand.test.cjs +++ b/packages/ql3-local-owner-cli/test/pluginPackageCommand.test.cjs @@ -193,12 +193,10 @@ function packageArtifact(manifest) { path: 'package.json', body: Buffer.from(serializePluginPackageManifest(manifest)), }, - ...CLI_PACKAGE_RESOURCES - .map(({ reference, body }) => ({ - path: reference.path, - body, - })) - .sort((left, right) => left.path.localeCompare(right.path)), + ...CLI_PACKAGE_RESOURCES.map(({ reference, body }) => ({ + path: reference.path, + body, + })).sort((left, right) => left.path.localeCompare(right.path)), ]); } @@ -259,13 +257,11 @@ function actionInput() { artifactDigest, artifactBytes: artifact.byteLength, contentDigest: pluginPackageContentTreeDigest( - CLI_PACKAGE_RESOURCES - .map(({ reference, body }) => ({ - path: reference.path, - bytes: body.byteLength, - digest: digest(body), - })) - .sort((left, right) => left.path.localeCompare(right.path)), + CLI_PACKAGE_RESOURCES.map(({ reference, body }) => ({ + path: reference.path, + bytes: body.byteLength, + digest: digest(body), + })).sort((left, right) => left.path.localeCompare(right.path)), ), }, architecture: 'arm64', @@ -478,9 +474,7 @@ async function activatePackageAutomation(databasePath, lock, manifest) { const revision = materializePluginPackageResources({ generation, lock, - manifestBytes: Buffer.from( - serializePluginPackageManifest(manifest), - ), + manifestBytes: Buffer.from(serializePluginPackageManifest(manifest)), resources: generation.resources.map((reference) => ({ reference, bytes: bodies.get(`${reference.kind}\0${reference.path}`), @@ -521,10 +515,8 @@ function publisherTrustRunnerWithOneSnapshotFault() { inspect: inspectLocalPluginPackagePublisherTrust, publish: publishLocalPluginPackagePublisherTrust, retire: retireLocalPluginPackagePublisherKey, - analyzePublisherKey: - analyzeLocalPluginPackageRecoveryCatalogPublisherKey, - proposeRevocation: - proposeLocalPluginPackagePublisherKeyRevocation, + analyzePublisherKey: analyzeLocalPluginPackageRecoveryCatalogPublisherKey, + proposeRevocation: proposeLocalPluginPackagePublisherKeyRevocation, async confirmRevocation(options) { return confirmLocalPluginPackagePublisherKeyRevocation({ ...options, @@ -896,10 +888,7 @@ test('runs the private command-file package lifecycle with replay-safe IDs', asy const catalogHelp = spawnSync( process.execPath, [ - path.join( - __dirname, - '../dist/plugin-package/pluginPackageCatalogCli.js', - ), + path.join(__dirname, '../dist/plugin-package/pluginPackageCatalogCli.js'), '--help', ], { encoding: 'utf8' }, @@ -1183,15 +1172,29 @@ test('runs the private command-file package lifecycle with replay-safe IDs', asy WHERE project_id = ? AND package_name = ?`, ) .get(lock.projectId, lock.packageName); - assert.deepEqual({ ...automationHead }, { - publicationDigest: - activeAutomation.publication.publicationDigest, - state: 'active', - }); - assert.equal( - await new LocalSqlitePluginPackageAutomationPublicationRepository( + const automationRepository = + new LocalSqlitePluginPackageAutomationPublicationRepository( quarantineAuthority, - ).isStartAllowed( + ); + const withdrawnAutomation = await automationRepository.findCurrent( + lock.projectId, + lock.packageName, + ); + assert.ok(withdrawnAutomation); + assert.equal(withdrawnAutomation.state, 'withdrawn'); + assert.equal( + withdrawnAutomation.previousPublicationDigest, + activeAutomation.publication.publicationDigest, + ); + assert.deepEqual( + { ...automationHead }, + { + publicationDigest: withdrawnAutomation.publicationDigest, + state: 'withdrawn', + }, + ); + assert.equal( + await automationRepository.isStartAllowed( lock.projectId, lock.packageName, activeAutomation.publication.publicationDigest, diff --git a/packages/ql3-local-sqlite/src/migration/migration.ts b/packages/ql3-local-sqlite/src/migration/migration.ts index 27d56e64..12058fad 100644 --- a/packages/ql3-local-sqlite/src/migration/migration.ts +++ b/packages/ql3-local-sqlite/src/migration/migration.ts @@ -98,6 +98,8 @@ import { local0085PluginPackageWorkflowRunListIndexMigration } from '../migratio import { local0086CapabilityV43Migration } from '../migrations/0086-capability-v43'; import { local0087RunAttemptLogRetentionMigration } from '../migrations/0087-run-attempt-log-retention'; import { local0088CapabilityV44Migration } from '../migrations/0088-capability-v44'; +import { local0089PluginPackageAutomationDispositionEventsMigration } from '../migrations/0089-plugin-package-automation-disposition-events'; +import { local0090CapabilityV45Migration } from '../migrations/0090-capability-v45'; import type { LocalSqliteMigrationContext } from '../migrations/sqlMigration'; import { LOCAL_SQLITE_MIGRATION_STREAM_ID, @@ -208,6 +210,8 @@ export const localSqliteMigrationDefinition: MigrationStreamDefinition { + async findById(migrationId: string): Promise { const row = this.client .prepare( `SELECT migration_id, stream_id, dialect, checksum, applied_at_ms @@ -88,6 +86,11 @@ CREATE TABLE IF NOT EXISTS "QingLong3SchemaMigrations" ( transaction: MigrationStreamTransaction, ) => Promise, ): Promise { + const foreignKeys = this.client.prepare('PRAGMA foreign_keys').get() as + | { foreign_keys?: unknown } + | undefined; + const restoreForeignKeys = foreignKeys?.foreign_keys === 1; + if (restoreForeignKeys) this.client.exec('PRAGMA foreign_keys = OFF'); this.client.exec('BEGIN IMMEDIATE'); try { const result = await work({ @@ -116,9 +119,11 @@ CREATE TABLE IF NOT EXISTS "QingLong3SchemaMigrations" ( }, }); this.client.exec('COMMIT'); + if (restoreForeignKeys) this.client.exec('PRAGMA foreign_keys = ON'); return result; } catch (error) { if (this.client.isTransaction) this.client.exec('ROLLBACK'); + if (restoreForeignKeys) this.client.exec('PRAGMA foreign_keys = ON'); throw error; } } diff --git a/packages/ql3-local-sqlite/src/migrations/0089-plugin-package-automation-disposition-events.ts b/packages/ql3-local-sqlite/src/migrations/0089-plugin-package-automation-disposition-events.ts new file mode 100644 index 00000000..270d96b7 --- /dev/null +++ b/packages/ql3-local-sqlite/src/migrations/0089-plugin-package-automation-disposition-events.ts @@ -0,0 +1,105 @@ +import { defineLocalSqliteMigration } from './sqlMigration'; + +export const local0089PluginPackageAutomationDispositionEventsMigration = + defineLocalSqliteMigration({ + id: '0089-plugin-package-automation-disposition-events', + statements: [ + ` +CREATE TABLE "QingLong3PluginPackageAutomationDispositionEvents" ( + event_digest TEXT PRIMARY KEY NOT NULL, + event_kind TEXT NOT NULL + CONSTRAINT ql3_plugin_package_automation_disposition_kind_check + CHECK (event_kind IN ('lifecycle','quarantine')), + CONSTRAINT ql3_plugin_package_automation_disposition_digest_check CHECK ( + length(event_digest) = 64 AND + event_digest NOT GLOB '*[^0-9a-f]*' + ) +) + `, + `INSERT INTO "QingLong3PluginPackageAutomationDispositionEvents" (event_digest, event_kind) SELECT event_digest, 'lifecycle' FROM "QingLong3PluginPackageLifecycleEvents"`, + `INSERT OR IGNORE INTO "QingLong3PluginPackageAutomationDispositionEvents" (event_digest, event_kind) SELECT event_digest, 'quarantine' FROM "QingLong3PluginPackageQuarantineEvents"`, + `CREATE TRIGGER ql3_plugin_package_automation_lifecycle_disposition_insert AFTER INSERT ON "QingLong3PluginPackageLifecycleEvents" BEGIN INSERT OR IGNORE INTO "QingLong3PluginPackageAutomationDispositionEvents" (event_digest, event_kind) VALUES (NEW.event_digest, 'lifecycle'); END`, + `CREATE TRIGGER ql3_plugin_package_automation_quarantine_disposition_insert AFTER INSERT ON "QingLong3PluginPackageQuarantineEvents" BEGIN INSERT OR IGNORE INTO "QingLong3PluginPackageAutomationDispositionEvents" (event_digest, event_kind) VALUES (NEW.event_digest, 'quarantine'); END`, + `PRAGMA defer_foreign_keys = ON`, + `PRAGMA legacy_alter_table = ON`, + `ALTER TABLE "QingLong3PluginPackageAutomationPublications" RENAME TO "QingLong3PluginPackageAutomationPublicationsBeforeDisposition"`, + ` +CREATE TABLE "QingLong3PluginPackageAutomationPublications" ( + publication_digest TEXT PRIMARY KEY NOT NULL, + project_id TEXT NOT NULL, + package_name TEXT NOT NULL, + installation_id TEXT NOT NULL, + lock_digest TEXT NOT NULL, + generation INTEGER NOT NULL, + generation_digest TEXT NOT NULL, + materialized_revision_digest TEXT NOT NULL, + state TEXT NOT NULL, + version INTEGER NOT NULL, + previous_publication_digest TEXT, + lifecycle_event_digest TEXT, + published_at_ms INTEGER NOT NULL, + publication_json TEXT NOT NULL, + CONSTRAINT ql3_plugin_package_automation_publication_revision_fk + FOREIGN KEY (generation_digest) + REFERENCES "QingLong3PluginPackageMaterializedRevisions" (generation_digest) + ON DELETE RESTRICT ON UPDATE RESTRICT, + CONSTRAINT ql3_plugin_package_automation_publication_previous_fk + FOREIGN KEY (previous_publication_digest) + REFERENCES "QingLong3PluginPackageAutomationPublications" (publication_digest) + ON DELETE RESTRICT ON UPDATE RESTRICT, + CONSTRAINT ql3_plugin_package_automation_publication_disposition_fk + FOREIGN KEY (lifecycle_event_digest) + REFERENCES "QingLong3PluginPackageAutomationDispositionEvents" (event_digest) + ON DELETE RESTRICT ON UPDATE RESTRICT, + CONSTRAINT ql3_plugin_package_automation_publication_identity_check CHECK ( + length(project_id) BETWEEN 1 AND 128 AND + length(package_name) BETWEEN 1 AND 63 AND + length(installation_id) BETWEEN 1 AND 128 AND + generation BETWEEN 1 AND 2147483647 AND + state IN ('active','withdrawn','absent') AND + version BETWEEN 1 AND 2147483647 AND published_at_ms >= 0 AND + (version = 1 AND state IN ('active','absent') AND + previous_publication_digest IS NULL AND lifecycle_event_digest IS NULL OR + version > 1 AND previous_publication_digest IS NOT NULL) AND + (state <> 'withdrawn' OR lifecycle_event_digest IS NOT NULL) + ), + CONSTRAINT ql3_plugin_package_automation_publication_digest_check CHECK ( + length(publication_digest) = 64 AND publication_digest NOT GLOB '*[^0-9a-f]*' AND + length(lock_digest) = 64 AND lock_digest NOT GLOB '*[^0-9a-f]*' AND + length(generation_digest) = 64 AND generation_digest NOT GLOB '*[^0-9a-f]*' AND + length(materialized_revision_digest) = 64 AND materialized_revision_digest NOT GLOB '*[^0-9a-f]*' AND + (previous_publication_digest IS NULL OR length(previous_publication_digest) = 64 AND previous_publication_digest NOT GLOB '*[^0-9a-f]*') AND + (lifecycle_event_digest IS NULL OR length(lifecycle_event_digest) = 64 AND lifecycle_event_digest NOT GLOB '*[^0-9a-f]*') + ), + CONSTRAINT ql3_plugin_package_automation_publication_json_check CHECK ( + length(CAST(publication_json AS BLOB)) BETWEEN 2 AND 12582912 AND + json_valid(publication_json) AND json_type(publication_json) = 'object' AND + json_extract(publication_json, '$.schema') = 'qinglong/plugin-package-automation-publication@v1' AND + json_extract(publication_json, '$.target.projectId') = project_id AND + json_extract(publication_json, '$.target.packageName') = package_name AND + json_extract(publication_json, '$.target.installationId') = installation_id AND + json_extract(publication_json, '$.target.lockDigest') = lock_digest AND + json_extract(publication_json, '$.target.generation') = generation AND + json_extract(publication_json, '$.target.generationDigest') = generation_digest AND + json_extract(publication_json, '$.target.materializedRevisionDigest') = materialized_revision_digest AND + json_extract(publication_json, '$.state') = state AND + json_extract(publication_json, '$.version') = version AND + (previous_publication_digest IS NULL AND json_type(publication_json, '$.previousPublicationDigest') = 'null' OR json_extract(publication_json, '$.previousPublicationDigest') = previous_publication_digest) AND + (lifecycle_event_digest IS NULL AND json_type(publication_json, '$.lifecycleEventDigest') = 'null' OR json_extract(publication_json, '$.lifecycleEventDigest') = lifecycle_event_digest) AND + json_extract(publication_json, '$.publishedAtMs') = published_at_ms AND + json_extract(publication_json, '$.publicationDigest') = publication_digest AND + json_type(publication_json, '$.definitions.workflows') = 'array' AND + json_type(publication_json, '$.definitions.prompts') = 'array' AND + (state = 'absent' AND json_array_length(json_extract(publication_json, '$.definitions.workflows')) + json_array_length(json_extract(publication_json, '$.definitions.prompts')) = 0 OR + state <> 'absent' AND json_array_length(json_extract(publication_json, '$.definitions.workflows')) + json_array_length(json_extract(publication_json, '$.definitions.prompts')) > 0) + ) +) + `, + `INSERT INTO "QingLong3PluginPackageAutomationPublications" SELECT * FROM "QingLong3PluginPackageAutomationPublicationsBeforeDisposition" ORDER BY project_id, package_name, version`, + `DROP TABLE "QingLong3PluginPackageAutomationPublicationsBeforeDisposition"`, + `CREATE UNIQUE INDEX ql3_plugin_package_automation_publication_version_uidx ON "QingLong3PluginPackageAutomationPublications" (project_id, package_name, version)`, + `CREATE UNIQUE INDEX ql3_plugin_package_automation_publication_previous_uidx ON "QingLong3PluginPackageAutomationPublications" (previous_publication_digest) WHERE previous_publication_digest IS NOT NULL`, + `CREATE INDEX ql3_plugin_package_automation_publication_generation_idx ON "QingLong3PluginPackageAutomationPublications" (generation_digest, publication_digest)`, + `PRAGMA legacy_alter_table = OFF`, + ], + }); diff --git a/packages/ql3-local-sqlite/src/migrations/0090-capability-v45.ts b/packages/ql3-local-sqlite/src/migrations/0090-capability-v45.ts new file mode 100644 index 00000000..40f3b22f --- /dev/null +++ b/packages/ql3-local-sqlite/src/migrations/0090-capability-v45.ts @@ -0,0 +1,14 @@ +import { CAPABILITIES_V44 } from './0088-capability-v44'; +import { defineLocalSqliteMigration } from './sqlMigration'; + +export const CAPABILITIES_V45 = CAPABILITIES_V44.replace( + '"plugin_package_automation_publication":1,', + '"plugin_package_automation_publication":1,"plugin_package_automation_security_withdrawal":1,', +); + +export const local0090CapabilityV45Migration = defineLocalSqliteMigration({ + id: '0090-capability-v45', + statements: [ + `UPDATE "QingLong3SchemaCapabilities" SET contract_version = 45, migration_id = '0089-plugin-package-automation-disposition-events', capabilities = '${CAPABILITIES_V45}', updated_at_ms = CAST(unixepoch('subsec') * 1000 AS INTEGER) WHERE contract_name = 'local-control-core' AND contract_version = 44 AND migration_id = '0087-run-attempt-log-retention' AND capabilities = '${CAPABILITIES_V44}'`, + ], +}); diff --git a/packages/ql3-local-sqlite/src/plugin-package/pluginPackageAutomationPublicationRepository.ts b/packages/ql3-local-sqlite/src/plugin-package/pluginPackageAutomationPublicationRepository.ts index 8ef7c777..45e9b6c2 100644 --- a/packages/ql3-local-sqlite/src/plugin-package/pluginPackageAutomationPublicationRepository.ts +++ b/packages/ql3-local-sqlite/src/plugin-package/pluginPackageAutomationPublicationRepository.ts @@ -44,7 +44,10 @@ function integer(row: Row, key: string): number { return value as number; } -function targetIdentity(projectId: unknown, packageName: unknown): { +function targetIdentity( + projectId: unknown, + packageName: unknown, +): { readonly projectId: string; readonly packageName: string; } { @@ -119,19 +122,18 @@ export class LocalSqlitePluginPackageAutomationPublicationRepository #parse(row: Row): Readonly { try { const publication = normalizePluginPackageAutomationPublication( - JSON.parse(text(row, 'publicationJson')) as - PluginPackageAutomationPublication, + JSON.parse( + text(row, 'publicationJson'), + ) as PluginPackageAutomationPublication, ); if ( - publication.publicationDigest !== - text(row, 'publicationDigest') || + publication.publicationDigest !== text(row, 'publicationDigest') || publication.target.projectId !== text(row, 'projectId') || publication.target.packageName !== text(row, 'packageName') || publication.target.installationId !== text(row, 'installationId') || publication.target.lockDigest !== text(row, 'lockDigest') || publication.target.generation !== integer(row, 'generation') || - publication.target.generationDigest !== - text(row, 'generationDigest') || + publication.target.generationDigest !== text(row, 'generationDigest') || publication.target.materializedRevisionDigest !== text(row, 'materializedRevisionDigest') || publication.state !== text(row, 'state') || @@ -142,9 +144,7 @@ export class LocalSqlitePluginPackageAutomationPublicationRepository } return publication; } catch (error) { - if ( - error instanceof PluginPackageAutomationPublicationUnavailableError - ) { + if (error instanceof PluginPackageAutomationPublicationUnavailableError) { throw error; } throw new PluginPackageAutomationPublicationUnavailableError(); @@ -389,8 +389,9 @@ export class LocalSqlitePluginPackageAutomationPublicationRepository return this.#findCurrent(projectId, packageName); } - publishInTransaction( + #publishInTransaction( value: Readonly, + securityWithdrawal: boolean, ): Readonly<{ status: 'created' | 'existing'; publication: Readonly; @@ -430,10 +431,7 @@ export class LocalSqlitePluginPackageAutomationPublicationRepository ); } try { - assertPluginPackageAutomationPublicationSuccessor( - current, - publication, - ); + assertPluginPackageAutomationPublicationSuccessor(current, publication); } catch (error) { if (error instanceof InvalidPluginPackageAutomationPublicationError) { throw new PluginPackageAutomationPublicationConflictError( @@ -467,33 +465,35 @@ export class LocalSqlitePluginPackageAutomationPublicationRepository 'materialized revision fence does not match publication target', ); } - const securityFence = client - .prepare( - `SELECT EXISTS ( - SELECT 1 - FROM "QingLong3PluginPackageQuarantineEvents" AS quarantine - WHERE quarantine.project_id = ? - AND quarantine.package_name = ? - AND quarantine.installation_id = ? - AND quarantine.lock_digest = ? - ) AS "blocked"`, - ) - .get( - publication.target.projectId, - publication.target.packageName, - publication.target.installationId, - publication.target.lockDigest, - ) as Row | undefined; - if ( - !securityFence || - (securityFence.blocked !== 0 && securityFence.blocked !== 1) - ) { - throw new PluginPackageAutomationPublicationUnavailableError(); - } - if (securityFence.blocked === 1) { - throw new PluginPackageAutomationPublicationConflictError( - 'quarantined Package generation cannot publish automation', - ); + if (!securityWithdrawal) { + const securityFence = client + .prepare( + `SELECT EXISTS ( + SELECT 1 + FROM "QingLong3PluginPackageQuarantineEvents" AS quarantine + WHERE quarantine.project_id = ? + AND quarantine.package_name = ? + AND quarantine.installation_id = ? + AND quarantine.lock_digest = ? + ) AS "blocked"`, + ) + .get( + publication.target.projectId, + publication.target.packageName, + publication.target.installationId, + publication.target.lockDigest, + ) as Row | undefined; + if ( + !securityFence || + (securityFence.blocked !== 0 && securityFence.blocked !== 1) + ) { + throw new PluginPackageAutomationPublicationUnavailableError(); + } + if (securityFence.blocked === 1) { + throw new PluginPackageAutomationPublicationConflictError( + 'quarantined Package generation cannot publish automation', + ); + } } client .prepare( @@ -570,9 +570,31 @@ export class LocalSqlitePluginPackageAutomationPublicationRepository }); } - publish( + publishInTransaction( value: Readonly, - ): Promise< + ): Readonly<{ + status: 'created' | 'existing'; + publication: Readonly; + }> { + return this.#publishInTransaction(value, false); + } + + publishSecurityWithdrawalInTransaction( + value: Readonly, + ): Readonly<{ + status: 'created' | 'existing'; + publication: Readonly; + }> { + const publication = normalizePluginPackageAutomationPublication(value); + if (publication.state !== 'withdrawn') { + throw new PluginPackageAutomationPublicationConflictError( + 'security withdrawal must narrow automation state', + ); + } + return this.#publishInTransaction(publication, true); + } + + publish(value: Readonly): Promise< Readonly<{ status: 'created' | 'existing'; publication: Readonly; diff --git a/packages/ql3-local-sqlite/src/plugin-package/pluginPackageQuarantineRepository.ts b/packages/ql3-local-sqlite/src/plugin-package/pluginPackageQuarantineRepository.ts index dcb6abff..0644ca73 100644 --- a/packages/ql3-local-sqlite/src/plugin-package/pluginPackageQuarantineRepository.ts +++ b/packages/ql3-local-sqlite/src/plugin-package/pluginPackageQuarantineRepository.ts @@ -20,6 +20,7 @@ import { normalizePluginPackageInstallRecord, type PluginPackageInstallRecord, } from '@qinglong/runtime-core/plugin-package-install'; +import { createPluginPackageAutomationLifecyclePublication } from '@qinglong/runtime-core/plugin-package-automation-publication'; import { createProjectToolDefinitionSnapshot, normalizeProjectToolDefinitionSnapshot, @@ -40,6 +41,7 @@ import { } from '@qinglong/runtime-core/task-spec-semantic'; import { LocalSqliteOperationAuthority } from '../authority/operationAuthority'; +import { LocalSqlitePluginPackageAutomationPublicationRepository } from './pluginPackageAutomationPublicationRepository'; type Row = Record; @@ -314,6 +316,30 @@ export class LocalSqlitePluginPackageQuarantineRepository if (error instanceof PluginPackageQuarantineUnavailableError) throw error; throw new PluginPackageQuarantineUnavailableError(); } + const automation = this.#authority.client + .prepare( + `SELECT state, + lifecycle_event_digest AS "lifecycleEventDigest" + FROM "QingLong3PluginPackageAutomationPublications" + WHERE project_id = ? AND package_name = ? + AND installation_id = ? AND lock_digest = ? + ORDER BY version DESC + LIMIT 1`, + ) + .get( + receipt.target.projectId, + receipt.target.packageName, + receipt.target.installationId, + receipt.target.lockDigest, + ) as Row | undefined; + if ( + automation && + (text(automation, 'state') === 'active' || + (automation.lifecycleEventDigest === receipt.eventDigest && + text(automation, 'state') !== 'withdrawn')) + ) { + throw new PluginPackageQuarantineUnavailableError(); + } } #findStored( @@ -433,6 +459,40 @@ export class LocalSqlitePluginPackageQuarantineRepository return record; } + #withdrawAutomation( + event: Readonly, + record: Readonly, + committedAtMs: number, + ): void { + const publications = + new LocalSqlitePluginPackageAutomationPublicationRepository( + this.#authority, + ); + const current = publications.findCurrentInTransaction( + event.target.projectId, + event.target.packageName, + ); + if (!current) return; + if ( + current.target.installationId !== event.target.installationId || + current.target.lockDigest !== event.target.lockDigest || + current.target.generation !== record.targetGeneration + ) { + throw new PluginPackageQuarantineConflictError( + 'Workflow/Prompt publication does not match the quarantined Package generation', + ); + } + if (current.state === 'absent' || current.state === 'withdrawn') return; + publications.publishSecurityWithdrawalInTransaction( + createPluginPackageAutomationLifecyclePublication({ + previous: current, + state: 'withdrawn', + lifecycleEventDigest: event.eventDigest, + publishedAtMs: committedAtMs, + }), + ); + } + #activeContributions( projectId: string, ): readonly Readonly[] { @@ -792,7 +852,7 @@ export class LocalSqlitePluginPackageQuarantineRepository 'target lock is already quarantined by another event', ); } - this.#install(event); + const install = this.#install(event); const clock = client .prepare( `SELECT CAST(unixepoch('subsec') * 1000 AS INTEGER) AS "nowMs"`, @@ -857,6 +917,7 @@ export class LocalSqlitePluginPackageQuarantineRepository ), ); this.#insertEvent(event); + this.#withdrawAutomation(event, install, committedAtMs); this.#publishSnapshot(snapshot, committedAtMs); const receipt = createPluginPackageWithdrawalReceipt({ eventDigest: event.eventDigest, diff --git a/packages/ql3-local-sqlite/src/readiness/readiness.ts b/packages/ql3-local-sqlite/src/readiness/readiness.ts index 9e1fdf4e..7ffde8a8 100644 --- a/packages/ql3-local-sqlite/src/readiness/readiness.ts +++ b/packages/ql3-local-sqlite/src/readiness/readiness.ts @@ -8,7 +8,20 @@ import { } from '../run/stepRunSchemaContract'; export const LOCAL_SQLITE_CONTRACT_NAME = 'local-control-core'; -export const LOCAL_SQLITE_CONTRACT_VERSION = 44; +export const LOCAL_SQLITE_CONTRACT_VERSION = 45; + +const PLUGIN_PACKAGE_AUTOMATION_DISPOSITION_TRIGGERS = Object.freeze([ + Object.freeze({ + name: 'ql3_plugin_package_automation_lifecycle_disposition_insert', + tableName: 'QingLong3PluginPackageLifecycleEvents', + sql: `CREATE TRIGGER ql3_plugin_package_automation_lifecycle_disposition_insert AFTER INSERT ON "QingLong3PluginPackageLifecycleEvents" BEGIN INSERT OR IGNORE INTO "QingLong3PluginPackageAutomationDispositionEvents" (event_digest, event_kind) VALUES (NEW.event_digest, 'lifecycle'); END`, + }), + Object.freeze({ + name: 'ql3_plugin_package_automation_quarantine_disposition_insert', + tableName: 'QingLong3PluginPackageQuarantineEvents', + sql: `CREATE TRIGGER ql3_plugin_package_automation_quarantine_disposition_insert AFTER INSERT ON "QingLong3PluginPackageQuarantineEvents" BEGIN INSERT OR IGNORE INTO "QingLong3PluginPackageAutomationDispositionEvents" (event_digest, event_kind) VALUES (NEW.event_digest, 'quarantine'); END`, + }), +]); const OPTIONAL_FEATURE_TABLE_NAMES = new Set([ 'QingLong3AiSchemaMigrations', @@ -1007,6 +1020,10 @@ const REQUIRED_SCHEMA = Object.freeze({ 'ql3_plugin_package_automation_publication_generation_idx', ]), }), + QingLong3PluginPackageAutomationDispositionEvents: Object.freeze({ + columns: Object.freeze(['event_digest', 'event_kind']), + indexes: Object.freeze([]), + }), QingLong3PluginPackageAutomationPublicationHeads: Object.freeze({ columns: Object.freeze([ 'project_id', @@ -1702,9 +1719,10 @@ function assertRequiredSchema(client: DatabaseSync): number { ORDER BY name`, ) .all(...ownedTableNames) as unknown as TriggerRow[]; - const expectedTriggers = [...LOCAL_STEP_RUN_REFERENCE_TRIGGERS].sort( - (left, right) => left.name.localeCompare(right.name), - ); + const expectedTriggers = [ + ...LOCAL_STEP_RUN_REFERENCE_TRIGGERS, + ...PLUGIN_PACKAGE_AUTOMATION_DISPOSITION_TRIGGERS, + ].sort((left, right) => left.name.localeCompare(right.name)); if ( triggerRows.length !== expectedTriggers.length || triggerRows.some((row, index) => { @@ -2097,8 +2115,15 @@ function assertPluginPackageAutomationPublicationIntegrity( LEFT JOIN "QingLong3PluginPackageAutomationPublications" AS previous ON previous.publication_digest = publication.previous_publication_digest + LEFT JOIN "QingLong3PluginPackageAutomationDispositionEvents" + AS disposition + ON disposition.event_digest = publication.lifecycle_event_digest LEFT JOIN "QingLong3PluginPackageLifecycleEvents" AS lifecycle ON lifecycle.event_digest = publication.lifecycle_event_digest + AND disposition.event_kind = 'lifecycle' + LEFT JOIN "QingLong3PluginPackageQuarantineEvents" AS quarantine + ON quarantine.event_digest = publication.lifecycle_event_digest + AND disposition.event_kind = 'quarantine' WHERE materialized.generation_digest IS NULL OR materialized.project_id <> publication.project_id OR materialized.package_name <> publication.package_name @@ -2129,14 +2154,25 @@ function assertPluginPackageAutomationPublicationIntegrity( ) OR ( publication.lifecycle_event_digest IS NOT NULL AND ( - lifecycle.event_digest IS NULL OR - lifecycle.project_id <> publication.project_id OR - lifecycle.package_name <> publication.package_name OR - lifecycle.installation_id <> publication.installation_id OR - lifecycle.lock_digest <> publication.lock_digest OR - lifecycle.generation_digest <> publication.generation_digest OR - lifecycle.materialized_revision_digest <> - publication.materialized_revision_digest OR + disposition.event_digest IS NULL OR + disposition.event_kind = 'lifecycle' AND ( + lifecycle.event_digest IS NULL OR + lifecycle.project_id <> publication.project_id OR + lifecycle.package_name <> publication.package_name OR + lifecycle.installation_id <> publication.installation_id OR + lifecycle.lock_digest <> publication.lock_digest OR + lifecycle.generation_digest <> publication.generation_digest OR + lifecycle.materialized_revision_digest <> + publication.materialized_revision_digest + ) OR + disposition.event_kind = 'quarantine' AND ( + quarantine.event_digest IS NULL OR + quarantine.project_id <> publication.project_id OR + quarantine.package_name <> publication.package_name OR + quarantine.installation_id <> publication.installation_id OR + quarantine.lock_digest <> publication.lock_digest OR + publication.state <> 'withdrawn' + ) OR previous.installation_id <> publication.installation_id OR previous.lock_digest <> publication.lock_digest OR previous.generation_digest <> publication.generation_digest OR @@ -2496,10 +2532,11 @@ export async function auditLocalSqliteReadiness( !capability || capability.contract_name !== LOCAL_SQLITE_CONTRACT_NAME || capability.contract_version !== LOCAL_SQLITE_CONTRACT_VERSION || - capability.migration_id !== '0087-run-attempt-log-retention' || + capability.migration_id !== + '0089-plugin-package-automation-disposition-events' || typeof capability.capabilities !== 'string' || capability.capabilities !== - '{"run_core":1,"run_retry_policy":1,"completion_receipt_journal":1,"local_dispatch_plan":1,"local_secret_envelope":1,"local_project_policy":1,"local_project_administration":1,"local_security_audit":1,"local_security_audit_compaction":1,"local_secret_authorized_mutation":1,"local_identity":1,"local_api_credential":1,"local_identity_provisioning":1,"local_identity_credential_administration":1,"local_owner_bootstrap":1,"local_owner_delivery_acknowledgement":1,"api_credential_pepper_binding":1,"local_owner_pepper_catalog":1,"local_owner_credential_recovery":1,"local_owner_pepper_reference_inspection":1,"local_owner_pepper_material_gc":1,"local_owner_delivery_acknowledgement_gc":1,"task_definition":1,"local_execution_revision_digest":1,"trigger_definition":1,"legacy_adoption_ledger":1,"local_scheduler_admission":1,"plugin_package_install":1,"approved_action":1,"plugin_package_admission":1,"approved_action_execution":1,"plugin_package_proposal":1,"plugin_package_materialized_revision":1,"plugin_package_task_reconciliation":1,"project_tool_definition_snapshot":1,"step_run":1,"tool_execution_evidence":1,"tool_execution_start_barrier":1,"tool_invocation_artifact":1,"tool_execution_artifact_binding":1,"tool_execution_completion":1,"tool_execution_failure_completion":1,"tool_result_key_catalog":1,"tool_result_rekey":1,"plugin_package_quarantine":1,"plugin_package_lifecycle":1,"plugin_package_automation_publication":1,"plugin_package_workflow_admission":1,"plugin_package_workflow_run_list":1,"run_attempt_log_retention":1,"plugin_package_workflow_task_attempt_admission":1}' || + '{"run_core":1,"run_retry_policy":1,"completion_receipt_journal":1,"local_dispatch_plan":1,"local_secret_envelope":1,"local_project_policy":1,"local_project_administration":1,"local_security_audit":1,"local_security_audit_compaction":1,"local_secret_authorized_mutation":1,"local_identity":1,"local_api_credential":1,"local_identity_provisioning":1,"local_identity_credential_administration":1,"local_owner_bootstrap":1,"local_owner_delivery_acknowledgement":1,"api_credential_pepper_binding":1,"local_owner_pepper_catalog":1,"local_owner_credential_recovery":1,"local_owner_pepper_reference_inspection":1,"local_owner_pepper_material_gc":1,"local_owner_delivery_acknowledgement_gc":1,"task_definition":1,"local_execution_revision_digest":1,"trigger_definition":1,"legacy_adoption_ledger":1,"local_scheduler_admission":1,"plugin_package_install":1,"approved_action":1,"plugin_package_admission":1,"approved_action_execution":1,"plugin_package_proposal":1,"plugin_package_materialized_revision":1,"plugin_package_task_reconciliation":1,"project_tool_definition_snapshot":1,"step_run":1,"tool_execution_evidence":1,"tool_execution_start_barrier":1,"tool_invocation_artifact":1,"tool_execution_artifact_binding":1,"tool_execution_completion":1,"tool_execution_failure_completion":1,"tool_result_key_catalog":1,"tool_result_rekey":1,"plugin_package_quarantine":1,"plugin_package_lifecycle":1,"plugin_package_automation_publication":1,"plugin_package_automation_security_withdrawal":1,"plugin_package_workflow_admission":1,"plugin_package_workflow_run_list":1,"run_attempt_log_retention":1,"plugin_package_workflow_task_attempt_admission":1}' || typeof capability.updated_at_ms !== 'number' || !Number.isSafeInteger(capability.updated_at_ms) || capability.updated_at_ms < 0 diff --git a/packages/ql3-local-sqlite/src/storage/schema.ts b/packages/ql3-local-sqlite/src/storage/schema.ts index 42ba7400..b9f112e8 100644 --- a/packages/ql3-local-sqlite/src/storage/schema.ts +++ b/packages/ql3-local-sqlite/src/storage/schema.ts @@ -4486,6 +4486,24 @@ export const pluginPackageLifecycleTasks = sqliteTable( ], ); +export const pluginPackageAutomationDispositionEvents = sqliteTable( + 'QingLong3PluginPackageAutomationDispositionEvents', + { + eventDigest: text('event_digest').primaryKey(), + eventKind: text('event_kind').notNull(), + }, + (table) => [ + check( + 'ql3_plugin_package_automation_disposition_kind_check', + sql`${table.eventKind} in ('lifecycle','quarantine')`, + ), + check( + 'ql3_plugin_package_automation_disposition_digest_check', + sql`length(${table.eventDigest}) = 64 and ${table.eventDigest} not glob '*[^0-9a-f]*'`, + ), + ], +); + export const pluginPackageAutomationPublications = sqliteTable( 'QingLong3PluginPackageAutomationPublications', { @@ -4510,7 +4528,7 @@ export const pluginPackageAutomationPublications = sqliteTable( { onDelete: 'restrict', onUpdate: 'restrict' }, ), lifecycleEventDigest: text('lifecycle_event_digest').references( - () => pluginPackageLifecycleEvents.eventDigest, + () => pluginPackageAutomationDispositionEvents.eventDigest, { onDelete: 'restrict', onUpdate: 'restrict' }, ), publishedAtMs: integer('published_at_ms').notNull(), @@ -4877,6 +4895,7 @@ export const localSqliteSchema = Object.freeze({ pluginPackageLifecycleHeads, pluginPackageLifecycleReceipts, pluginPackageLifecycleTasks, + pluginPackageAutomationDispositionEvents, pluginPackageAutomationPublications, pluginPackageAutomationPublicationHeads, pluginPackageWorkflowAdmissions, diff --git a/packages/ql3-local-sqlite/test/database.test.cjs b/packages/ql3-local-sqlite/test/database.test.cjs index 5f86d712..2c8f07d8 100644 --- a/packages/ql3-local-sqlite/test/database.test.cjs +++ b/packages/ql3-local-sqlite/test/database.test.cjs @@ -138,9 +138,11 @@ test('creates a reviewed edge database and opens runtime only after readiness', '0086-capability-v43', '0087-run-attempt-log-retention', '0088-capability-v44', + '0089-plugin-package-automation-disposition-events', + '0090-capability-v45', ]); assert.equal(migrated.readiness.contractName, 'local-control-core'); - assert.equal(migrated.readiness.contractVersion, 44); + assert.equal(migrated.readiness.contractVersion, 45); assert.equal(migrated.readiness.journalMode, 'delete'); assert.equal(fs.statSync(databasePath).mode & 0o777, 0o600); @@ -503,8 +505,8 @@ test('backfills v14 execution revisions with a verified independent digest', asy .get(), }, { - contract_version: 44, - migration_id: '0087-run-attempt-log-retention', + contract_version: 45, + migration_id: '0089-plugin-package-automation-disposition-events', }, ); } finally { @@ -691,19 +693,19 @@ test('excludes reviewed optional feature tables while preserving unknown table d const options = { databasePath, profile: 'edge' }; await migrateLocalSqlitePath(options); const client = new DatabaseSync(databasePath); - assert.equal((await auditLocalSqlitePath(options)).tableCount, 78); + assert.equal((await auditLocalSqlitePath(options)).tableCount, 79); client.exec( 'CREATE TABLE "ModelInvocationFeatureHead" (feature_id TEXT PRIMARY KEY)', ); client.close(); - assert.equal((await auditLocalSqlitePath(options)).tableCount, 78); + assert.equal((await auditLocalSqlitePath(options)).tableCount, 79); const unknownClient = new DatabaseSync(databasePath); unknownClient.exec('CREATE TABLE "UserExtensionData" (id TEXT PRIMARY KEY)'); unknownClient.close(); - assert.equal((await auditLocalSqlitePath(options)).tableCount, 79); + assert.equal((await auditLocalSqlitePath(options)).tableCount, 80); const triggerClient = new DatabaseSync(databasePath); triggerClient.exec(` diff --git a/packages/ql3-local-sqlite/test/fixtures/pluginPackageQuarantineCrashMatrixFixture.cjs b/packages/ql3-local-sqlite/test/fixtures/pluginPackageQuarantineCrashMatrixFixture.cjs index 93d42548..29a656df 100644 --- a/packages/ql3-local-sqlite/test/fixtures/pluginPackageQuarantineCrashMatrixFixture.cjs +++ b/packages/ql3-local-sqlite/test/fixtures/pluginPackageQuarantineCrashMatrixFixture.cjs @@ -4,6 +4,9 @@ const { DatabaseSync } = require('node:sqlite'); const { createPluginPackageQuarantineEvent, } = require('@qinglong/runtime-core/plugin-package-quarantine'); +const { + createInitialPluginPackageAutomationPublication, +} = require('@qinglong/runtime-core/plugin-package-automation-publication'); const { activateInstall, pluginPackageTaskReconciliationFixture, @@ -14,6 +17,9 @@ const { const { LocalSqlitePluginPackageInstallRepository, } = require('../../dist/plugin-package/pluginPackageInstallRepository'); +const { + LocalSqlitePluginPackageAutomationPublicationRepository, +} = require('../../dist/plugin-package/pluginPackageAutomationPublicationRepository'); const { LocalSqlitePluginPackageMaterializedRevisionRepository, } = require('../../dist/plugin-package/pluginPackageMaterializedRevisionRepository'); @@ -30,6 +36,11 @@ const DIGEST_D = 'd'.repeat(64); const DIGEST_E = 'e'.repeat(64); const CRASH_POINTS = Object.freeze({ + after_automation_withdrawal: Object.freeze({ + timing: 'afterRun', + sql: 'INSERT INTO "QingLong3PluginPackageAutomationPublications"', + durable: false, + }), after_task_disable: Object.freeze({ timing: 'afterRun', sql: 'INSERT INTO "QingLong3TaskDefinitionRevisions"', @@ -60,6 +71,24 @@ const CRASH_POINTS = Object.freeze({ function fixture(profile) { return pluginPackageTaskReconciliationFixture(`quarantine-crash-${profile}`, { profile, + workflows: [ + { + schema: 'qinglong/plugin-package-workflow-resource@v1', + id: 'daily', + name: 'Daily workflow', + enabled: true, + steps: [{ id: 'run', task: 'alpha', needs: [] }], + }, + ], + prompts: [ + { + schema: 'qinglong/plugin-package-prompt-resource@v1', + id: 'operator', + name: 'Operator prompt', + template: 'Run {{task}}', + parameters: [{ name: 'task', required: true }], + }, + ], }); } @@ -119,6 +148,15 @@ async function setupScenario({ databasePath, profile }) { ); await activateInstall(install, value); await materialized.publish(value.revision); + await new LocalSqlitePluginPackageAutomationPublicationRepository( + authority, + ).publish( + createInitialPluginPackageAutomationPublication( + value.revision, + value.registry, + value.install.active.updatedAtMs, + ), + ); await reconciliation.reconcile(value.revision, { async findActiveResourceGeneration() { return value.revision.generation; @@ -259,6 +297,24 @@ async function verifyScenario({ databasePath, pointName, profile }) { ) { throw new Error(`${profile}/${pointName} Task withdrawal is incomplete`); } + const automation = database + .prepare( + `SELECT publication.state, + publication.lifecycle_event_digest AS "lifecycleEventDigest" + FROM "QingLong3PluginPackageAutomationPublicationHeads" AS head + JOIN "QingLong3PluginPackageAutomationPublications" AS publication + ON publication.publication_digest = head.publication_digest + WHERE head.project_id = ? AND head.package_name = ?`, + ) + .get(value.projectId, value.packageName); + if ( + automation?.state !== 'withdrawn' || + automation.lifecycleEventDigest !== quarantineEvent.eventDigest + ) { + throw new Error( + `${profile}/${pointName} automation withdrawal is incomplete`, + ); + } await auditLocalSqliteReadiness(database); const integrity = database.prepare('PRAGMA integrity_check').get(); const foreignKey = database diff --git a/packages/ql3-local-sqlite/test/pluginPackageQuarantineCrashMatrix.test.cjs b/packages/ql3-local-sqlite/test/pluginPackageQuarantineCrashMatrix.test.cjs index a1d09cd6..2628ce5e 100644 --- a/packages/ql3-local-sqlite/test/pluginPackageQuarantineCrashMatrix.test.cjs +++ b/packages/ql3-local-sqlite/test/pluginPackageQuarantineCrashMatrix.test.cjs @@ -60,10 +60,10 @@ test( assert.equal(reports.at(-1).durableAfterCrash, point.durable); } } - assert.equal(reports.length, 10); + assert.equal(reports.length, 12); assert.equal( reports.filter(({ crashBeforeCommit }) => crashBeforeCommit).length, - 8, + 10, ); assert.equal( reports.filter(({ durableAfterCrash }) => durableAfterCrash).length, diff --git a/packages/ql3-local-sqlite/test/pluginPackageQuarantineRepository.test.cjs b/packages/ql3-local-sqlite/test/pluginPackageQuarantineRepository.test.cjs index 6cecf273..c467fe64 100644 --- a/packages/ql3-local-sqlite/test/pluginPackageQuarantineRepository.test.cjs +++ b/packages/ql3-local-sqlite/test/pluginPackageQuarantineRepository.test.cjs @@ -7,6 +7,9 @@ const { PluginPackageQuarantineUnavailableError, createPluginPackageQuarantineEvent, } = require('@qinglong/runtime-core/plugin-package-quarantine'); +const { + createInitialPluginPackageAutomationPublication, +} = require('@qinglong/runtime-core/plugin-package-automation-publication'); const { RunRepositoryConstraintError, } = require('@qinglong/runtime-core/run-repository'); @@ -17,13 +20,18 @@ const { activateInstall, pluginPackageTaskReconciliationFixture, } = require('../../../test/contracts/pluginPackageTaskReconciliationRepositoryContract.cjs'); -const { LocalSqliteOperationAuthority } = require('../dist/authority/operationAuthority'); +const { + LocalSqliteOperationAuthority, +} = require('../dist/authority/operationAuthority'); const { LocalSqlitePluginPackageInstallRepository, } = require('../dist/plugin-package/pluginPackageInstallRepository'); const { LocalSqlitePluginPackageMaterializedRevisionRepository, } = require('../dist/plugin-package/pluginPackageMaterializedRevisionRepository'); +const { + LocalSqlitePluginPackageAutomationPublicationRepository, +} = require('../dist/plugin-package/pluginPackageAutomationPublicationRepository'); const { LocalSqlitePluginPackageTaskReconciliationRepository, } = require('../dist/plugin-package/pluginPackageTaskReconciliationRepository'); @@ -46,6 +54,24 @@ const digest = (value) => value.repeat(64); async function harness(t, namespace) { const fixture = pluginPackageTaskReconciliationFixture(namespace, { profile: 'edge', + workflows: [ + { + schema: 'qinglong/plugin-package-workflow-resource@v1', + id: 'daily', + name: 'Daily workflow', + enabled: true, + steps: [{ id: 'run', task: 'alpha', needs: [] }], + }, + ], + prompts: [ + { + schema: 'qinglong/plugin-package-prompt-resource@v1', + id: 'operator', + name: 'Operator prompt', + template: 'Run {{task}}', + parameters: [{ name: 'task', required: true }], + }, + ], }); const client = new DatabaseSync(':memory:'); client.exec('PRAGMA foreign_keys = ON'); @@ -67,6 +93,9 @@ async function harness(t, namespace) { authority, fixture.registry, ), + automation: new LocalSqlitePluginPackageAutomationPublicationRepository( + authority, + ), reconciliation: new LocalSqlitePluginPackageTaskReconciliationRepository( authority, fixture.registry, @@ -108,6 +137,13 @@ function quarantineEvent(fixture, record = fixture.install.active) { async function publishActivePackage(value) { await activateInstall(value.install, value.fixture); await value.materialized.publish(value.fixture.revision); + await value.automation.publish( + createInitialPluginPackageAutomationPublication( + value.fixture.revision, + value.fixture.registry, + value.fixture.install.active.updatedAtMs, + ), + ); await value.reconciliation.reconcile(value.fixture.revision, { async findActiveResourceGeneration() { return value.fixture.revision.generation; @@ -199,6 +235,13 @@ test('withdraws active Package Tasks and Tool source in one exact replayable tra .snapshotDigest, created.receipt.capability.currentToolSnapshotDigest, ); + const automation = await value.automation.findCurrent( + value.fixture.projectId, + value.fixture.packageName, + ); + assert.equal(automation.state, 'withdrawn'); + assert.equal(automation.lifecycleEventDigest, event.eventDigest); + assert.equal(automation.version, 2); const replay = await value.quarantine.quarantine(event, () => { authorizationChecks += 1; @@ -264,6 +307,51 @@ test('rolls back every withdrawal fact when the target install advanced', async .get(value.fixture.projectId).count, 0, ); + assert.equal( + ( + await value.automation.findCurrent( + value.fixture.projectId, + value.fixture.packageName, + ) + ).state, + 'active', + ); +}); + +test('fails closed when quarantine automation withdrawal evidence is rewound', async (t) => { + const value = await harness(t, 'sqlite-quarantine-automation-corrupt'); + await publishActivePackage(value); + const event = quarantineEvent(value.fixture); + await value.quarantine.quarantine(event, () => {}); + const active = value.client + .prepare( + `SELECT publication_digest AS "publicationDigest" + FROM "QingLong3PluginPackageAutomationPublications" + WHERE project_id = ? AND package_name = ? AND state = 'active'`, + ) + .get(value.fixture.projectId, value.fixture.packageName); + value.client.exec('PRAGMA foreign_keys = OFF'); + value.client + .prepare( + `UPDATE "QingLong3PluginPackageAutomationPublicationHeads" + SET publication_digest = ?, state = 'active', version = 1 + WHERE project_id = ? AND package_name = ?`, + ) + .run( + active.publicationDigest, + value.fixture.projectId, + value.fixture.packageName, + ); + value.client + .prepare( + `DELETE FROM "QingLong3PluginPackageAutomationPublications" + WHERE project_id = ? AND package_name = ? AND state = 'withdrawn'`, + ) + .run(value.fixture.projectId, value.fixture.packageName); + await assert.rejects( + value.quarantine.findByEventDigest(event.eventDigest), + PluginPackageQuarantineUnavailableError, + ); }); test('rolls back withdrawal when the in-transaction Owner fence changes before commit', async (t) => { diff --git a/packages/ql3-local-sqlite/test/pluginPackageWorkflowAdmissionRepository.test.cjs b/packages/ql3-local-sqlite/test/pluginPackageWorkflowAdmissionRepository.test.cjs index ff97e0c4..a0125914 100644 --- a/packages/ql3-local-sqlite/test/pluginPackageWorkflowAdmissionRepository.test.cjs +++ b/packages/ql3-local-sqlite/test/pluginPackageWorkflowAdmissionRepository.test.cjs @@ -156,7 +156,7 @@ test('atomically admits one generation-bound Workflow Run and exactly replays it }, { runs: 1, steps: 2, events: 3, mutations: 2, admissions: 1 }, ); - assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 44); + assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 45); }); test('runs an optional authorization guard inside new and replay transactions', async (t) => { @@ -288,7 +288,7 @@ test('exactly replays immutable admission after the Workflow StepRun advances', }, { status: 'running', version: 5, eventSequence: 5 }, ); - assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 44); + assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 45); }); test('fails closed before writing when the exact installation is not active', async (t) => { diff --git a/packages/ql3-local-sqlite/test/pluginPackageWorkflowTaskAttemptAdmissionRepository.test.cjs b/packages/ql3-local-sqlite/test/pluginPackageWorkflowTaskAttemptAdmissionRepository.test.cjs index d6a676c5..f07d05d3 100644 --- a/packages/ql3-local-sqlite/test/pluginPackageWorkflowTaskAttemptAdmissionRepository.test.cjs +++ b/packages/ql3-local-sqlite/test/pluginPackageWorkflowTaskAttemptAdmissionRepository.test.cjs @@ -231,7 +231,7 @@ test('atomically admits the exact reconciled local Task revision and replays it' stepAttemptCount: 0, }, ); - assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 44); + assert.equal((await auditLocalSqliteReadiness(client)).contractVersion, 45); }); test('bounds candidate paging before SQL and fences cancellation', async (t) => { diff --git a/packages/ql3-local-sqlite/test/rolloutSafety.test.cjs b/packages/ql3-local-sqlite/test/rolloutSafety.test.cjs index 248c3f6c..cf12d66c 100644 --- a/packages/ql3-local-sqlite/test/rolloutSafety.test.cjs +++ b/packages/ql3-local-sqlite/test/rolloutSafety.test.cjs @@ -40,9 +40,9 @@ test('creates and exactly replays a reviewed rollout backup', async (t) => { await migrateLocalSqlitePath(state); const prepared = await createLocalSqliteRolloutBackup(state); assert.equal(prepared.status, 'prepared'); - assert.equal(prepared.contractVersion, 44); - assert.equal(prepared.writeContractVersion, 44); - assert.equal(LOCAL_SQLITE_WRITE_CONTRACT_VERSION, 44); + assert.equal(prepared.contractVersion, 45); + assert.equal(prepared.writeContractVersion, 45); + assert.equal(LOCAL_SQLITE_WRITE_CONTRACT_VERSION, 45); assert.match(prepared.sha256, /^[0-9a-f]{64}$/); assert.equal(prepared.bytes > 0, true); assert.equal(prepared.pageCount > 0, true); diff --git a/scripts/ql3-postgres-ha-contract.cjs b/scripts/ql3-postgres-ha-contract.cjs index 7e5f9315..259c11b7 100644 --- a/scripts/ql3-postgres-ha-contract.cjs +++ b/scripts/ql3-postgres-ha-contract.cjs @@ -7705,6 +7705,17 @@ async function runPluginPackageQuarantineMatrix(options) { ), false, ); + const withdrawnAutomation = await automationRepository.findCurrent( + fixture.projectId, + fixture.packageName, + ); + assert.ok(withdrawnAutomation); + assert.equal(withdrawnAutomation.state, 'withdrawn'); + assert.equal(withdrawnAutomation.lifecycleEventDigest, event.eventDigest); + assert.equal( + withdrawnAutomation.previousPublicationDigest, + lifecycle.automationPublication.publicationDigest, + ); const currentSnapshot = await snapshotRepository.findCurrent( fixture.projectId, ); @@ -7740,6 +7751,20 @@ async function runPluginPackageQuarantineMatrix(options) { FROM "ql3"."project_tool_definition_snapshot_sources" WHERE project_id = $2 AND active_vector_digest = $3) AS "retainedSources", + (SELECT count(*)::integer + FROM "ql3"."plugin_package_automation_disposition_events" + WHERE event_digest = $1 AND event_kind = 'quarantine') + AS "automationDispositionEvents", + (SELECT count(*)::integer + FROM "ql3"."plugin_package_automation_publication_heads" AS head + JOIN "ql3"."plugin_package_automation_publications" AS publication + ON publication.project_id = head.project_id + AND publication.package_name = head.package_name + AND publication.publication_digest = head.publication_digest + WHERE head.project_id = $2 + AND publication.lifecycle_event_digest = $1 + AND publication.state = 'withdrawn') + AS "withdrawnAutomationHeads", (SELECT count(*)::integer FROM "ql3"."plugin_package_workflow_admissions" WHERE plan_digest = $4) AS "workflowAdmissions", @@ -7791,6 +7816,8 @@ async function runPluginPackageQuarantineMatrix(options) { row?.taskCount === 2 && row?.disabledTasks === 2 && row?.retainedSources === 0 && + row?.automationDispositionEvents === 1 && + row?.withdrawnAutomationHeads === 1 && row?.workflowAdmissions === 1 && row?.workflowEvents === 5 && row?.workflowStepMutations === 3 && @@ -7857,6 +7884,7 @@ async function runPluginPackageQuarantineMatrix(options) { automationStartAllowedBeforeRevocationReceipt: true, automationStartAllowedAfterRevocationReceipt: false, automationStartAllowedAfterQuarantine: false, + automationPublicationWithdrawnAtomically: true, taskWithdrawals: responseLossResult.receipt.capability.taskWithdrawals.length, retainedSources: @@ -7868,6 +7896,7 @@ async function runPluginPackageQuarantineMatrix(options) { automationRecoverySourceConverged: true, automationStartGuardRuntimeOnly: false, automationStartFenceSurvivedPromotion: false, + automationPublicationSurvivedPromotion: false, workflowAdmissionCreatedAtomically: true, workflowAuthorizedAdmissionAtomic: true, workflowAdmissionExactReplay: true, @@ -8312,6 +8341,20 @@ async function verifyPluginPackageQuarantineAfterPromotion(options) { ), false, ); + const withdrawnAutomation = + await new PostgresPluginPackageAutomationPublicationRepository( + packageDatabase.pool, + ).findCurrent(quarantine.report.projectId, quarantine.report.packageName); + assert.ok(withdrawnAutomation); + assert.equal(withdrawnAutomation.state, 'withdrawn'); + assert.equal( + withdrawnAutomation.lifecycleEventDigest, + quarantine.event.eventDigest, + ); + assert.equal( + withdrawnAutomation.previousPublicationDigest, + quarantine.lifecycle.automationPublication.publicationDigest, + ); const workflowAdmissions = new PostgresAuthorizedPluginPackageWorkflowAdmissionRepository( runtimeDatabase.pool, @@ -8568,6 +8611,7 @@ async function verifyPluginPackageQuarantineAfterPromotion(options) { assert.equal(executorWorkflowTaskAttemptDenied, true); quarantine.report.automationStartGuardRuntimeOnly = true; quarantine.report.automationStartFenceSurvivedPromotion = true; + quarantine.report.automationPublicationSurvivedPromotion = true; quarantine.report.workflowAdmissionRuntimeOnly = true; quarantine.report.workflowAdmissionSurvivedPromotion = true; quarantine.report.workflowAuthorizedAdmissionSurvivedPromotion = true; @@ -8592,6 +8636,20 @@ async function verifyPluginPackageQuarantineAfterPromotion(options) { (SELECT count(*)::integer FROM "ql3"."plugin_package_withdrawal_tasks" WHERE event_digest = $1) AS "taskCount", + (SELECT count(*)::integer + FROM "ql3"."plugin_package_automation_disposition_events" + WHERE event_digest = $1 AND event_kind = 'quarantine') + AS "automationDispositionEventCount", + (SELECT count(*)::integer + FROM "ql3"."plugin_package_automation_publication_heads" AS head + JOIN "ql3"."plugin_package_automation_publications" AS publication + ON publication.project_id = head.project_id + AND publication.package_name = head.package_name + AND publication.publication_digest = head.publication_digest + WHERE head.project_id = $7 + AND publication.lifecycle_event_digest = $1 + AND publication.state = 'withdrawn') + AS "withdrawnAutomationHeadCount", (SELECT count(*)::integer FROM "ql3"."plugin_package_publisher_revocation_receipts" WHERE receipt_digest = $2) AS "revocationReceiptCount", @@ -8637,6 +8695,7 @@ async function verifyPluginPackageQuarantineAfterPromotion(options) { quarantine.report.proposalDigest, quarantine.workflowAdmission.plan.planDigest, quarantine.workflowAdmission.plan.runId, + quarantine.report.projectId, ], ); assert.deepEqual(facts.rows, [ @@ -8644,6 +8703,8 @@ async function verifyPluginPackageQuarantineAfterPromotion(options) { eventCount: 1, receiptCount: 1, taskCount: 2, + automationDispositionEventCount: 1, + withdrawnAutomationHeadCount: 1, revocationReceiptCount: 1, impactCount: 1, impactItemCount: 1, @@ -8696,9 +8757,8 @@ async function verifyPluginPackageLifecycleAfterPromotion(options) { lifecycle.enableReceipt.lifecycle, ); assert.deepEqual( - await automations.findCurrent( - lifecycle.report.projectId, - lifecycle.report.packageName, + await automations.findByDigest( + lifecycle.automationPublication.publicationDigest, ), lifecycle.automationPublication, ); @@ -8716,6 +8776,10 @@ async function verifyPluginPackageLifecycleAfterPromotion(options) { (SELECT count(*)::integer FROM "ql3"."plugin_package_lifecycle_tasks" WHERE event_digest IN ($1, $2)) AS "taskCount", + (SELECT count(*)::integer + FROM "ql3"."plugin_package_automation_publications" + WHERE publication_digest = $5) + AS "lifecycleAutomationPublicationCount", (SELECT count(*)::integer FROM "ql3"."plugin_package_automation_publications" WHERE project_id = $3 AND package_name = $4) @@ -8737,6 +8801,7 @@ async function verifyPluginPackageLifecycleAfterPromotion(options) { lifecycle.enableEvent.eventDigest, lifecycle.report.projectId, lifecycle.report.packageName, + lifecycle.automationPublication.publicationDigest, ], ); assert.deepEqual(facts.rows, [ @@ -8744,9 +8809,10 @@ async function verifyPluginPackageLifecycleAfterPromotion(options) { eventCount: 2, receiptCount: 2, taskCount: 4, - automationPublicationCount: 5, - automationState: 'active', - automationVersion: 5, + lifecycleAutomationPublicationCount: 1, + automationPublicationCount: 6, + automationState: 'withdrawn', + automationVersion: 6, version: lifecycle.report.lifecycleVersion, disposition: 'active', }, @@ -11356,10 +11422,13 @@ async function main(argv = process.argv.slice(2)) { .keyId, ), }; - assert.deepEqual( - promotedPromptExecution, - pluginPackageQuarantine.promptExecution.beforePromotion, - ); + assert.deepEqual(promotedPromptExecution, { + ...pluginPackageQuarantine.promptExecution.beforePromotion, + catalog: { + ...pluginPackageQuarantine.promptExecution.beforePromotion.catalog, + publicationState: 'withdrawn', + }, + }); pluginPackageQuarantine.promptExecution.afterPromotion = promotedPromptExecution; pluginPackageQuarantine.promptExecution.survivedPromotion = true; @@ -12391,6 +12460,11 @@ async function main(argv = process.argv.slice(2)) { pluginPackageLifecycle.report.automationPublicationSurvivedPromotion, pluginPackageAutomationRecoverySourceConverges: pluginPackageQuarantine.report.automationRecoverySourceConverged, + pluginPackageQuarantineWithdrawsAutomationAtomically: + pluginPackageQuarantine.report + .automationPublicationWithdrawnAtomically, + pluginPackageQuarantineAutomationWithdrawalSurvivesPromotion: + pluginPackageQuarantine.report.automationPublicationSurvivedPromotion, pluginPackagePublisherRevocationImmediatelyFencesAutomation: pluginPackageQuarantine.report .automationStartAllowedBeforeRevocationReceipt && diff --git a/test/back/ql3PackageBoundaryAudit.test.cjs b/test/back/ql3PackageBoundaryAudit.test.cjs index be394ab7..48994991 100644 --- a/test/back/ql3PackageBoundaryAudit.test.cjs +++ b/test/back/ql3PackageBoundaryAudit.test.cjs @@ -74,12 +74,12 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( [ { directory: 'packages/ql3-cluster-postgres/src/migrations', - directSourceFiles: 58, + directSourceFiles: 59, reviewKind: 'ordered_ledger', }, { directory: 'packages/ql3-local-sqlite/src/migrations', - directSourceFiles: 89, + directSourceFiles: 91, reviewKind: 'ordered_ledger', }, ], @@ -421,10 +421,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( rootSourceFileRoles: clusterPostgres.rootSourceFileRoles, }, { - sourceFiles: 153, + sourceFiles: 154, rootSourceFiles: 1, rootSourceLines: 125, - nestedSourceFiles: 152, + nestedSourceFiles: 153, rootSourceFileRoles: { 'index.ts': 'public_export' }, }, ); @@ -540,10 +540,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( rootSourceFileRoles: localSqlite.rootSourceFileRoles, }, { - sourceFiles: 179, + sourceFiles: 181, rootSourceFiles: 1, rootSourceLines: 31, - nestedSourceFiles: 178, + nestedSourceFiles: 180, rootSourceFileRoles: { 'index.ts': 'public_export' }, }, );