diff --git a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md index 30786ee4..7ce2ad21 100644 --- a/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md +++ b/docs/QINGLONG_3_0_ARCHITECTURE_RFC.md @@ -11,6 +11,7 @@ 最新增量证据(2026-08-13): +- D-310/ADR-0402(已接受):新增 `qinglong.task.runs.compare@1.0.0`,把“最近成功/失败 Run”的选择从模型无界分页收回服务端。输入只接受 Task ID,固定读取按 created/id 倒序的 64 条 Project-scoped Task Run,第 65 条仅证明窗口截断且协议不返回 cursor;选择 succeeded baseline 与 failed candidate 后按固定顺序执行最多两个低敏点查并复用共享差值算法,输出明确区分 complete 与窗口内未找到,consistency 固定为 `bounded_task_window_then_ordered_point_reads`。实现没有扩大 CRITICAL/HIGH 的通用 SQLite/PostgreSQL Run Reader,而是在双方既有 `run/outcome-comparison/` 中提供窄 adapter;不新增 package、依赖、migration、索引、连接、timer、listener、watcher 或 cache,默认 Edge/Standalone 制品字节数保持 2,589,812。定向 Runtime Core 10/10、SQLite 1/1(真实 query plan 命中既有 Task 时间索引)、PostgreSQL adapter 2/2、Local MCP 47/47、dependency firewall 53/53;最终 18-package clean build/test 与 backend 1,206 pass/2 条件 skip/0 fail,package/dependency/Edge/Cluster deployment 审计零 finding。Edge-MCP 为 7,237,187 bytes/795 files/220 modules/RSS 38,699,008 bytes,Standalone-MCP 为 7,237,295 bytes/795 files/220 modules/RSS 38,600,704 bytes,均在门内。PostgreSQL 18.4 arm64 HA 125/125 Gate、timeline `1→2`,报告 SHA-256 为 `229c7cac328ee960f667f92868374264a10cb75090ef93d644da1326385d8774`。 - D-309/ADR-0401(已接受):`qinglong.run.compare@1.0.0` 进入可选 `ql3-mcp` stdio 产品入口,复用 Runtime Core 的共享 Definition/projection 和既有 MCP 静态注册循环。每次调用重新执行 Owner credential 认证、`tool.call:qinglong.run.compare` + `run.read` Policy、durable allowed audit、credential fence confirm,再按 baseline→candidate 串行执行两个 Project-scoped SQLite 点查;错误稳定收敛为 `run_compare_unavailable`。该入口是 ADR-0347 的交互式只读 surface,只持久化安全 admission,不冒充 StepRun、encrypted Tool completion 或模型 Trace;内部 Copilot 的受信执行仍必须走完整 completion 链。实现不新增 package、依赖、migration、表、索引、连接、timer、listener、watcher、cache 或网络 endpoint,默认 Edge/Standalone 继续裁掉 MCP package,仅显式 `edge-mcp|standalone-mcp` 承担调用成本。Local MCP 46/46、最终 18-package clean build/test 退出 0、backend 1,206 pass/2 条件 skip/0 fail,package/dependency/Edge/Cluster deployment 审计零 finding;默认 Edge 为 2,589,812 bytes/315 files/56 modules/RSS 11,091,968 bytes,Edge-MCP 为 7,219,977 bytes/792 files/217 modules/RSS 38,649,856 bytes,Standalone-MCP 为 7,220,085 bytes/792 files/217 modules/RSS 38,043,648 bytes,均在各自门内。 - D-308/ADR-0400(已接受):首个只读 Copilot 的“最近成功/失败运行对比”不再依赖 Prompt 自由拼接两个查询。`qinglong.run.compare@1.0.0` 作为第二个受信内建 Tool,固定 `read/low`、`run.read`、`database.read` 和 5 秒 deadline;只有当前 Project Tool snapshot 显式包含 reviewed Definition、产品 composition 显式绑定 adapter 后才可执行。它按 baseline→candidate 串行复用两次有界 Run 点查,absent 与 cross-Project 均为 `found:false`,只输出低敏 Run projection、固定 changed fields 和可证明的 queue/execution/total duration delta;任一时间戳不完整或结束早于开始时不生成对应差值。输出明确标记 `ordered_independent_point_reads`,不冒充数据库事务快照。实现位于既有 `runtime-core/tool-execution/builtin-run-compare/`,只提供两个显式 subpath,不从 package root 导出,不新增 workspace package、依赖、表、migration、连接、timer、listener、watcher、cache 或低配设备常驻开销。最终 18-package clean build/test 退出 0;backend 1,208 项为 1,206 pass/2 条件 skip/0 fail;package/dependency/Edge/Cluster deployment 审计零 finding。Edge、Edge AI、Edge MCP 制品为 2,589,812 / 3,121,108 / 7,209,862 bytes,均低于各自上限且未装配 subpath 被发布投影裁掉。 - D-307/ADR-0399(已接受):物理 Edge release archive Gate 使用外部 Ed25519 签发、QingLong verify-only 的两阶段协议。`prepare` exact 重建 owner-private 统一物理报告,要求 direct release service start 已通过,并把 repository、40 位 Git revision、设备/boot、物理报告、release archive、实机 artifact tree/metadata/entrypoint 与 Node digest/version 编入无换行 canonical payload;私钥始终位于 HSM/KMS/离线 operator。`finalize` 以 operator-pinned SPKI 公钥重算 fingerprint,稳定读取并复核所有输入后验证 64-byte detached signature,任一 source/archive/evidence 漂移均失败关闭,输出 `0600` no-replace envelope。通过只把 `release_archive_signature` 替换为 `release_archive_signature_or_attestation`,`supported:false` 与其余 firmware、整机 flash、migration、断电、固定实机采集和 Cluster 容量 Gate 全部保留。基础 importer 同时把 Edge、SQLite 与 Plugin Package 三个 workload 的 platform/architecture 精确绑定到统一物理观测,release verifier 再要求 observed/Edge/SQLite 为完整 recorder shape 且 Node identity 相同,拒绝跨主机拼接与重算外层摘要后的最小伪造。实现不新增 package、依赖、daemon、listener、timer、watcher 或设备常驻负担;18-package clean build/test 退出 0,backend 1,208 项为 1,206 pass/2 条件 skip/0 fail,package/dependency/Edge/service bridge/Cluster deployment 审计零 finding。 @@ -6906,8 +6907,13 @@ Copilot 默认不能直接提交修改或运行命令。写操作通过 Tool Reg credential 后才可发现,并且每次调用都经过 Tool Policy、durable Security Audit 和 credential fence confirm;它没有进入默认 Edge/Standalone application,也未向 HTTP、Cluster 或任意模型 授予 ambient capability。该交互式 MCP 读取不冒充内部 Trusted Tool 的 StepRun/encrypted -completion。“最近成功与失败”的有界选择、日志 range/redaction 和最终 Copilot Prompt 仍需 -后续产品 Gate。 +completion。ADR-0402 已完成“最近成功与失败”的有界选择: +`qinglong.task.runs.compare@1.0.0` 只接受 Task ID,在一次固定 64+1 的 Project-scoped +Task Run 窗口中选择最新 succeeded/failed,并复用共享低敏对比算法。协议没有 cursor; +窗口截断且缺少任一 outcome 时必须返回 `complete=false`,禁止模型把局部缺失解释成全历史 +不存在。可选本机 MCP 已接入同一认证/Policy/Audit/confirm 链;PostgreSQL 仅提供窄 adapter, +Cluster 产品入口和完整 Trusted Tool completion 仍需独立门禁。日志 range/redaction 和最终 +Copilot Prompt 仍需后续产品 Gate。 ## 17. Tool Registry diff --git a/docs/adr/ADR-0402-bounded-latest-task-run-outcome-comparison.md b/docs/adr/ADR-0402-bounded-latest-task-run-outcome-comparison.md new file mode 100644 index 00000000..96d1e270 --- /dev/null +++ b/docs/adr/ADR-0402-bounded-latest-task-run-outcome-comparison.md @@ -0,0 +1,103 @@ +# ADR-0402:有界的 Task 最近成功/失败 Run 对比 + +- 状态:Accepted +- 日期:2026-08-14 +- 关联 RFC:QL-RFC-0001 D-310、Phase 2 +- 关联 ADR:ADR-0347、ADR-0400、ADR-0401 + +## 问题 + +ADR-0400/0401 已提供按两个显式 Run ID 对比的共享受信 Tool 和可选本机 MCP 入口,但“选择 +最近一次成功与失败运行”仍可能被留给模型。模型如果循环调用 `run.list`、翻页并自行过滤, +搜索次数、历史范围和停止条件都不可证明;在低配路由设备上会形成不可控 SQLite I/O,在 +Cluster 上也会放大数据库读取和模型上下文,同时给 Prompt 注入制造更大的输入面。 + +直接把 Task/status 过滤继续塞入通用 Run Reader 也不合理。GitNexus 显示 +`LocalSqliteRunReader` 为 CRITICAL(14 个直接、29 个总影响、1 条产品启动流程), +`PostgresRunReader` 为 HIGH(7 个直接、15 个总影响)。为了一个 AI 只读用例扩大这两个基础 +Reader 的职责,会让本地恢复、API、Cluster recovery 和写事务继承树承担无关回归风险。 + +## 决策 + +1. 增加 `qinglong.task.runs.compare@1.0.0`。输入精确为一个不超过 255 字符的 `taskId`, + 没有 `limit`、cursor、status、SQL、排序或 Project 参数;Project 只来自受信 composition。 + Definition 固定为 `read/low`、`run.read`、5 秒 deadline。 +2. Runtime Core 在 `run/outcome-comparison/` 定义窄 + `TaskRunOutcomeWindowReader`。repository 一次只返回按 + `(created_at_ms DESC, id DESC)` 排列的最小五字段记录:Run/Project/Task identity、status 和 + created time。Tool 固定请求 65 条,其中前 64 条是搜索窗口,第 65 条只证明还有更老记录; + 协议永远不返回 cursor,模型不能扩大窗口。 +3. 前 64 条中第一个 `succeeded` 是 baseline,第一个 `failed` 是 candidate。两者都找到, + 或存储明确返回不超过 64 条时,`selection.complete=true`;如果窗口已满且缺少任一结果, + 输出 `complete=false`,不能把“窗口内未找到”伪装成历史上不存在。 +4. 选择后按 succeeded → failed 固定顺序执行最多两个既有 Project-scoped Run 点查,并复用 + ADR-0400 的低敏 projection、changed fields 和时长差值算法。选择出的 Run 如果消失、变更 + Task/outcome、跨 Project 或损坏,整个调用稳定 unavailable;不回退到继续搜索。 +5. 输出携带 + `consistency=bounded_task_window_then_ordered_point_reads`。它明确表示一次有界选择查询后跟 + 两次独立点查,不声称数据库事务快照。baseline 永远代表 succeeded,candidate 永远代表 + failed,差值方向固定为 failed − succeeded。 +6. SQLite 与 PostgreSQL 各自新增独立、二级目录内的窄 adapter,不修改现有通用 Run Reader + 或 Repository 继承树。SQLite 真实 `EXPLAIN QUERY PLAN` 必须命中既有 + `ql3_local_runs_task_created_idx`;PostgreSQL adapter 使用同样的 Project/Task/order/`LIMIT 65` + 查询。复用既有索引避免新增 migration、索引空间和每次 Run 写放大。 +7. 可选 `ql3-mcp` 复用既有逐调用 Owner credential → Tool Policy → durable allowed audit → + credential confirm 链,审计理由固定为 `tool_qinglong_task_runs_compare`。默认 Edge/Standalone + 仍不装载 MCP;Cluster 只获得显式 PostgreSQL adapter subpath,不开放 ambient MCP/HTTP + endpoint,内部 Copilot 仍须走完整 Trusted Tool completion 链。 + +## 低配、集群与 package 布局 + +- 默认低配设备没有新增进程、连接、timer、listener、watcher、cache、migration 或索引;启用 + MCP 时每次调用固定为一个 65 行上限的最小列查询和最多两个串行点查。 +- Cluster adapter 复用调用方现有 queryable/Pool,不创建第二连接池;driver 错误收敛为稳定 + repository operation error,畸形 bigint/status 收敛为 constraint error。 +- package 数保持 18。能力落入 Runtime Core 既有 `run/outcome-comparison/` 与 + `tool-execution/builtin-run-compare/`,持久化实现落入两端既有 + `run/outcome-comparison/`;不为一两个文件创建微型 package,也不回到 package `src` 根平铺。 +- Runtime Core、Local SQLite、Cluster PostgreSQL 只提供显式 subpath,package 根入口不扩大; + Local MCP dependency firewall 只放行两个新 Runtime Core 精确 subpath。 + +## 被否决方案 + +1. **让模型分页 `run.list`**:查询次数、停止条件和上下文大小不可证明。 +2. **把 task/status/cursor 加入通用 Run list**:会把 AI 用例扩散到 HTTP/API 和基础 Reader, + 且仍允许模型无界翻页。 +3. **分别查询所有成功和失败历史**:即使只返回一条,缺少固定窗口时存储扫描和语义仍无界。 +4. **为两种 outcome 新增索引**:会增加所有设备的磁盘与 Run 写放大;固定 64 条窗口可复用 + 已有 Task 时间索引,当前收益不足以支付 migration 和低配常驻成本。 +5. **修改 `LocalSqliteRunReader`/`PostgresRunReader`**:GitNexus 已证明 HIGH/CRITICAL blast + radius,独立 reader 能用更小权限和故障域完成目标。 +6. **只返回两个 Run ID**:仍迫使调用方再次自由组合字段和差值;本 Tool 应交付完整的有界 + 低敏对比结果。 + +## 当前验证 + +1. Runtime Core 定向 10/10:覆盖成功/失败选择顺序、差值、固定 64+1 窗口、无 cursor、 + incomplete 语义、跨 Project/乱序/畸形/消失/存储失败、Definition/binding 漂移、显式 + subpath 和根入口零导出。 +2. Local SQLite 1/1:fresh production migration 后写入多 Project/Task/状态 Run,经 MCP 的 + 单 authority 读取精确窗口,并用真实 `EXPLAIN QUERY PLAN` 证明命中既有 Task 时间索引。 + PostgreSQL adapter 2/2:精确 SQL/参数/顺序/上限、bigint 映射、畸形状态和 driver 错误 + 收敛全部通过。 +3. Local MCP 47/47:新增 Tool discovery、固定 read-only annotations、双 permission、durable + audit reason、credential confirm、真实 stdio + fresh SQLite 的选择和两次点查均通过。 +4. dependency firewall 53/53,完整 dependency audit 与 package boundary audit 零 finding; + package 保持 18 个,`singleSourcePackages=[]`、`shallowSourcePackages=[]`。 +5. 最终 18-package clean build/test 退出 0;backend 1,208 项为 1,206 pass、2 条平台条件 + skip、0 fail。Edge import 与 Cluster deployment 审计零 finding。Runtime Core 为 165/164 + nested、Local SQLite 为 197/196 nested、Cluster PostgreSQL 为 168/167 nested,三者 package + 根入口均未增长。 +6. 默认 Edge artifact 保持 2,589,812 bytes/315 files/56 modules,证明未装配能力被完全裁掉; + RSS 增量 11,173,888 bytes,低于 16 MiB 门。Edge-MCP 为 7,237,187 bytes/795 files/ + 220 modules/RSS 38,699,008 bytes,Standalone-MCP 为 7,237,295 bytes/795 files/ + 220 modules/RSS 38,600,704 bytes,均低于 16 MiB/1,536 files/48 MiB 门。 +7. PostgreSQL 18.4 arm64 HA 125/125 Gate 通过,timeline `1→2`,报告 SHA-256 为 + `229c7cac328ee960f667f92868374264a10cb75090ef93d644da1326385d8774`;容器、网络与卷零残留。 + +## 后续门禁 + +1. Cluster 产品 composition 装配 Definition/binding,并以真实 PostgreSQL 完成加密结果和 + StepRun/Trace completion; +2. 日志解释另建 Artifact range、redaction、prompt-injection 和字节预算协议; +3. 固定物理 Edge MCP 单次选择/点查延迟与 RSS,并验证撤权竞态。 diff --git a/docs/adr/README.md b/docs/adr/README.md index 883a7613..40c386dc 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -405,6 +405,7 @@ | [ADR-0399](./ADR-0399-external-release-attestation-for-physical-edge-evidence.md) | 物理 Edge 证据的外部发布归档证明 | Accepted(exact recorder 重建与离线验签已实现;固定实机 ceremony 待执行) | | [ADR-0400](./ADR-0400-bounded-trusted-run-comparison-tool.md) | 有界、受信的 Run 对比 Tool | Accepted | | [ADR-0401](./ADR-0401-optional-local-mcp-run-comparison-surface.md) | 可选本机 MCP Run 对比产品入口 | Accepted | +| [ADR-0402](./ADR-0402-bounded-latest-task-run-outcome-comparison.md) | 有界的 Task 最近成功/失败 Run 对比 | Accepted | ## 规则 diff --git a/packages/ql3-cluster-postgres/package.json b/packages/ql3-cluster-postgres/package.json index 7ee645f1..db98f27c 100644 --- a/packages/ql3-cluster-postgres/package.json +++ b/packages/ql3-cluster-postgres/package.json @@ -75,6 +75,11 @@ "require": "./dist/run-management/runManualRetryRepository.js", "default": "./dist/run-management/runManualRetryRepository.js" }, + "./task-run-outcome-window": { + "types": "./dist/run/outcome-comparison/taskRunOutcomeWindowReader.d.ts", + "require": "./dist/run/outcome-comparison/taskRunOutcomeWindowReader.js", + "default": "./dist/run/outcome-comparison/taskRunOutcomeWindowReader.js" + }, "./run-manager": { "types": "./dist/entrypoints/runManager.d.ts", "require": "./dist/entrypoints/runManager.js", diff --git a/packages/ql3-cluster-postgres/src/run/outcome-comparison/taskRunOutcomeWindowReader.ts b/packages/ql3-cluster-postgres/src/run/outcome-comparison/taskRunOutcomeWindowReader.ts new file mode 100644 index 00000000..a0115c93 --- /dev/null +++ b/packages/ql3-cluster-postgres/src/run/outcome-comparison/taskRunOutcomeWindowReader.ts @@ -0,0 +1,84 @@ +import { RUN_STATUSES, type RunStatus } from '@qinglong/runtime-core/run'; +import { + RunRepositoryConstraintError, + RunRepositoryOperationError, +} from '@qinglong/runtime-core/run-repository'; +import { + normalizeTaskRunOutcomeWindowQuery, + normalizeTaskRunOutcomeWindowRecord, + type TaskRunOutcomeWindowQuery, + type TaskRunOutcomeWindowReader, + type TaskRunOutcomeWindowRecord, +} from '@qinglong/runtime-core/task-run-outcome-window'; +import type { PostgresQueryable } from '@qinglong/runtime-core'; + +type QueryRow = Readonly>; + +function requiredString(row: QueryRow, property: string): string { + const value = row[property]; + if (typeof value !== 'string' || value.length === 0) { + throw new RunRepositoryConstraintError( + `PostgreSQL Task Run outcome row has an invalid ${property}`, + ); + } + return value; +} + +function requiredInteger(row: QueryRow, property: string): number { + const value = row[property]; + if (typeof value === 'number' && Number.isSafeInteger(value)) return value; + if (typeof value === 'string' && /^(0|[1-9]\d*)$/.test(value)) { + const parsed = Number(value); + if (Number.isSafeInteger(parsed)) return parsed; + } + throw new RunRepositoryConstraintError( + `PostgreSQL Task Run outcome row has an invalid ${property}`, + ); +} + +function record(row: QueryRow): Readonly { + const status = requiredString(row, 'status'); + if (!RUN_STATUSES.includes(status as RunStatus)) { + throw new RunRepositoryConstraintError( + 'PostgreSQL Task Run outcome row has an invalid status', + ); + } + return normalizeTaskRunOutcomeWindowRecord({ + id: requiredString(row, 'id'), + projectId: requiredString(row, 'projectId'), + taskId: requiredString(row, 'taskId'), + status: status as RunStatus, + createdAtMs: requiredInteger(row, 'createdAtMs'), + }); +} + +export class PostgresTaskRunOutcomeWindowReader + implements TaskRunOutcomeWindowReader +{ + constructor(private readonly queryable: PostgresQueryable) {} + + async listRecentRunsByTask( + value: Readonly, + ): Promise[]> { + const query = normalizeTaskRunOutcomeWindowQuery(value); + try { + const result = await this.queryable.query( + `SELECT + "id" AS "id", + "project_id" AS "projectId", + "task_id" AS "taskId", + "status" AS "status", + "created_at_ms" AS "createdAtMs" + FROM "ql3"."runs" + WHERE "project_id" = $1 AND "task_id" = $2 + ORDER BY "created_at_ms" DESC, "id" DESC + LIMIT $3`, + [query.projectId, query.taskId, query.limit], + ); + return Object.freeze(result.rows.map(record)); + } catch (error) { + if (error instanceof RunRepositoryConstraintError) throw error; + throw new RunRepositoryOperationError(error); + } + } +} diff --git a/packages/ql3-cluster-postgres/test/taskRunOutcomeWindowReader.test.cjs b/packages/ql3-cluster-postgres/test/taskRunOutcomeWindowReader.test.cjs new file mode 100644 index 00000000..a5ac9e51 --- /dev/null +++ b/packages/ql3-cluster-postgres/test/taskRunOutcomeWindowReader.test.cjs @@ -0,0 +1,114 @@ +const assert = require('node:assert/strict'); +const { test } = require('node:test'); + +const { + RunRepositoryConstraintError, + RunRepositoryOperationError, +} = require('@qinglong/runtime-core/run-repository'); +const { + PostgresTaskRunOutcomeWindowReader, +} = require('@qinglong/cluster-postgres/task-run-outcome-window'); + +function harness(rows) { + const queries = []; + return { + queries, + queryable: { + async query(text, values) { + queries.push({ text, values }); + return { rows, rowCount: rows.length }; + }, + }, + }; +} + +test('reads one bounded PostgreSQL Project Task window with bigint normalization', async () => { + assert.equal( + require('@qinglong/cluster-postgres').PostgresTaskRunOutcomeWindowReader, + undefined, + ); + const value = harness([ + { + id: 'run-failed', + projectId: 'project-a', + taskId: 'task-a', + status: 'failed', + createdAtMs: '2000', + }, + { + id: 'run-succeeded', + projectId: 'project-a', + taskId: 'task-a', + status: 'succeeded', + createdAtMs: '1000', + }, + ]); + const rows = await new PostgresTaskRunOutcomeWindowReader( + value.queryable, + ).listRecentRunsByTask({ + projectId: 'project-a', + taskId: 'task-a', + limit: 65, + }); + + assert.deepEqual(rows, [ + { + id: 'run-failed', + projectId: 'project-a', + taskId: 'task-a', + status: 'failed', + createdAtMs: 2000, + }, + { + id: 'run-succeeded', + projectId: 'project-a', + taskId: 'task-a', + status: 'succeeded', + createdAtMs: 1000, + }, + ]); + assert.deepEqual(value.queries[0].values, ['project-a', 'task-a', 65]); + assert.match( + value.queries[0].text, + /WHERE "project_id" = \$1 AND "task_id" = \$2[\s\S]*ORDER BY "created_at_ms" DESC, "id" DESC[\s\S]*LIMIT \$3/u, + ); +}); + +test('rejects invalid windows and maps driver failures without leaking details', async () => { + const invalid = harness([ + { + id: 'run-invalid', + projectId: 'project-a', + taskId: 'task-a', + status: 'invented', + createdAtMs: '1000', + }, + ]); + await assert.rejects( + new PostgresTaskRunOutcomeWindowReader( + invalid.queryable, + ).listRecentRunsByTask({ + projectId: 'project-a', + taskId: 'task-a', + limit: 65, + }), + RunRepositoryConstraintError, + ); + + await assert.rejects( + new PostgresTaskRunOutcomeWindowReader({ + async query() { + throw new Error('postgresql://private-host/secret'); + }, + }).listRecentRunsByTask({ + projectId: 'project-a', + taskId: 'task-a', + limit: 65, + }), + (error) => { + assert.ok(error instanceof RunRepositoryOperationError); + assert.equal(error.message, 'Run repository operation failed'); + return true; + }, + ); +}); diff --git a/packages/ql3-local-mcp-server/src/application-runtime/mcpServer.ts b/packages/ql3-local-mcp-server/src/application-runtime/mcpServer.ts index 517d083f..331bfa60 100644 --- a/packages/ql3-local-mcp-server/src/application-runtime/mcpServer.ts +++ b/packages/ql3-local-mcp-server/src/application-runtime/mcpServer.ts @@ -50,6 +50,11 @@ import { BUILTIN_RUN_COMPARE_TOOL_DEFINITION, executeBuiltInRunCompareTool, } from '@qinglong/runtime-core/builtin-run-compare-projection'; +import { + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL, + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION, + executeBuiltInTaskRunOutcomeCompareTool, +} from '@qinglong/runtime-core/builtin-task-run-outcome-compare-projection'; import { BUILTIN_RUN_READ_TOOL, BUILTIN_RUN_READ_TOOL_DEFINITION, @@ -58,6 +63,7 @@ import { import type { RunRepositoryReader } from '@qinglong/runtime-core/run-repository'; import type { StepRunRepository } from '@qinglong/runtime-core/step-run'; import type { ProjectRunListReader } from '@qinglong/runtime-core/project-run-list'; +import type { TaskRunOutcomeWindowReader } from '@qinglong/runtime-core/task-run-outcome-window'; import type { SecurityPrincipal } from '@qinglong/runtime-core/security'; import type { TaskDefinitionSource } from '@qinglong/runtime-core/task-definition'; import type { TriggerSource } from '@qinglong/runtime-core/trigger'; @@ -112,7 +118,8 @@ type LocalMcpRunReader = Pick< RunRepositoryReader, 'findRunById' | 'listEvents' > & - ProjectRunListReader; + ProjectRunListReader & + TaskRunOutcomeWindowReader; type LocalMcpTaskReader = Pick< TaskDefinitionSource, @@ -194,6 +201,24 @@ const LOCAL_MCP_READ_TOOLS: readonly LocalMcpReadToolDescriptor[] = input: ToolJsonValue, ) => executeBuiltInRunCompareTool(authority.runs, projectId, input), }), + Object.freeze({ + tool: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL, + definition: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION, + title: 'Compare Latest QingLong Task Run Outcomes', + auditReason: 'tool_qinglong_task_runs_compare', + unavailableCode: 'task_run_outcome_compare_unavailable', + execute: ( + authority: LocalMcpReadAuthority, + projectId: string, + input: ToolJsonValue, + ) => + executeBuiltInTaskRunOutcomeCompareTool( + authority.runs, + authority.runs, + projectId, + input, + ), + }), Object.freeze({ tool: BUILTIN_RUN_EVENT_LIST_TOOL, definition: BUILTIN_RUN_EVENT_LIST_TOOL_DEFINITION, diff --git a/packages/ql3-local-mcp-server/test/mcpServer.test.cjs b/packages/ql3-local-mcp-server/test/mcpServer.test.cjs index 8824c6a7..8f001aa4 100644 --- a/packages/ql3-local-mcp-server/test/mcpServer.test.cjs +++ b/packages/ql3-local-mcp-server/test/mcpServer.test.cjs @@ -165,6 +165,7 @@ function fixture(options = {}) { const audits = []; let reads = 0; let listReads = 0; + let outcomeWindowReads = 0; let eventReads = 0; let taskListReads = 0; let triggerListReads = 0; @@ -181,6 +182,15 @@ function fixture(options = {}) { startedAtMs: 24, finishedAtMs: 30, }); + const failedRun = Object.freeze({ + ...candidateRun, + id: 'run-failed', + status: 'failed', + createdAtMs: 30, + queuedAtMs: 31, + startedAtMs: 34, + finishedAtMs: 40, + }); const server = createQingLongLocalMcpServer({ projectId: 'default', now: () => NOW, @@ -222,11 +232,37 @@ function fixture(options = {}) { const values = [candidateRun, run()]; return values.slice(0, query.limit); }, + async listRecentRunsByTask(query) { + events.push('read-outcome-window'); + outcomeWindowReads += 1; + assert.deepEqual(query, { + projectId: 'default', + taskId: 'task-1', + limit: 65, + }); + return [ + { + id: failedRun.id, + projectId: failedRun.projectId, + taskId: failedRun.taskId, + status: failedRun.status, + createdAtMs: failedRun.createdAtMs, + }, + { + id: 'run-1', + projectId: 'default', + taskId: 'task-1', + status: 'succeeded', + createdAtMs: 10, + }, + ]; + }, async findRunById(runId) { events.push('read'); reads += 1; if (runId === 'run-1') return run(options.runProjectId); - return runId === 'run-2' ? candidateRun : null; + if (runId === 'run-2') return candidateRun; + return runId === 'run-failed' ? failedRun : null; }, async listEvents(runId, query) { events.push('read-events'); @@ -328,6 +364,7 @@ function fixture(options = {}) { counters: () => ({ reads, listReads, + outcomeWindowReads, eventReads, taskListReads, triggerListReads, @@ -393,6 +430,7 @@ test('advertises bounded read-only Run Tools and executes auth -> Policy -> Audi 'qinglong.run.list', 'qinglong.run.get', 'qinglong.run.compare', + 'qinglong.task.runs.compare', 'qinglong.run.events.list', 'qinglong.run.steps.list', 'qinglong.task.get', @@ -451,6 +489,7 @@ test('advertises bounded read-only Run Tools and executes auth -> Policy -> Audi assert.deepEqual(value.counters(), { reads: 1, listReads: 0, + outcomeWindowReads: 0, eventReads: 0, taskListReads: 0, triggerListReads: 0, @@ -533,6 +572,98 @@ test('compares two Project Runs through the same fenced admission', async (t) => assert.deepEqual(value.counters(), { reads: 2, listReads: 0, + outcomeWindowReads: 0, + eventReads: 0, + taskListReads: 0, + triggerListReads: 0, + approvalListReads: 0, + approvalDetailReads: 0, + confirmations: 1, + }); +}); + +test('selects and compares latest Task outcomes through the same fenced admission', async (t) => { + const value = fixture(); + const connected = await client(value.server, t); + const response = await connected.request('tools/call', { + name: 'qinglong.task.runs.compare', + arguments: { taskId: 'task-1' }, + }); + assert.equal(response.result.isError, undefined); + assert.deepEqual(response.result.structuredContent, { + taskId: 'task-1', + baselineOutcome: 'succeeded', + candidateOutcome: 'failed', + baseline: { + found: true, + id: 'run-1', + taskId: 'task-1', + taskRevision: 'revision-1', + status: 'succeeded', + version: 3, + eventSequence: 4, + priority: 0, + executionOrigin: 'manual', + executionOwner: 'runtime', + createdAtMs: 10, + queuedAtMs: 11, + startedAtMs: 12, + finishedAtMs: 13, + }, + candidate: { + found: true, + id: 'run-failed', + taskId: 'task-1', + taskRevision: 'revision-2', + status: 'failed', + version: 3, + eventSequence: 4, + priority: 1, + executionOrigin: 'manual', + executionOwner: 'runtime', + createdAtMs: 30, + queuedAtMs: 31, + startedAtMs: 34, + finishedAtMs: 40, + }, + comparable: true, + sameTask: true, + sameTaskRevision: false, + changedFields: ['taskRevision', 'status', 'priority'], + queueDelayDeltaMs: 0, + executionDurationDeltaMs: 5, + totalDurationDeltaMs: 7, + consistency: 'bounded_task_window_then_ordered_point_reads', + selection: { + windowLimit: 64, + searchedRunCount: 2, + hasOlderRuns: false, + complete: true, + order: 'created_at_desc_id_desc', + }, + }); + assert.deepEqual(value.permissions, [ + 'tool.call:qinglong.task.runs.compare', + 'run.read', + ]); + assert.deepEqual(value.events, [ + 'authenticate', + 'policy:tool.call:qinglong.task.runs.compare', + 'policy:run.read', + 'audit:allowed', + 'confirm', + 'read-outcome-window', + 'read', + 'read', + ]); + assert.deepEqual(value.audits[0].reasons, [ + 'tool_invocation_allowed', + 'tool_qinglong_task_runs_compare', + ]); + assert.deepEqual(value.counters(), { + reads: 2, + listReads: 0, + outcomeWindowReads: 1, eventReads: 0, taskListReads: 0, triggerListReads: 0, @@ -590,6 +721,7 @@ test('discovers recent Project Runs through the same fenced admission', async (t assert.deepEqual(value.counters(), { reads: 0, listReads: 1, + outcomeWindowReads: 0, eventReads: 0, taskListReads: 0, triggerListReads: 0, @@ -642,6 +774,7 @@ test('lists a payload-free Run event page through the same fenced admission', as assert.deepEqual(value.counters(), { reads: 1, listReads: 0, + outcomeWindowReads: 0, eventReads: 1, taskListReads: 0, triggerListReads: 0, @@ -694,6 +827,7 @@ test('discovers low-sensitive Tasks through task.read admission', async (t) => { assert.deepEqual(value.counters(), { reads: 0, listReads: 0, + outcomeWindowReads: 0, eventReads: 0, taskListReads: 1, triggerListReads: 0, @@ -743,6 +877,7 @@ test('reads one current Task fence through task.read admission', async (t) => { assert.deepEqual(value.counters(), { reads: 0, listReads: 0, + outcomeWindowReads: 0, eventReads: 0, taskListReads: 1, triggerListReads: 0, @@ -803,6 +938,7 @@ test('discovers low-sensitive Triggers through trigger.read admission', async (t assert.deepEqual(value.counters(), { reads: 0, listReads: 0, + outcomeWindowReads: 0, eventReads: 0, taskListReads: 0, triggerListReads: 1, @@ -859,6 +995,7 @@ test('discovers low-sensitive Approvals through approval.read admission', async assert.deepEqual(value.counters(), { reads: 0, listReads: 0, + outcomeWindowReads: 0, eventReads: 0, taskListReads: 0, triggerListReads: 0, @@ -930,6 +1067,7 @@ test('reads one redacted Approval preview through approval.read and artifact.rea assert.deepEqual(value.counters(), { reads: 0, listReads: 0, + outcomeWindowReads: 0, eventReads: 0, taskListReads: 0, triggerListReads: 0, diff --git a/packages/ql3-local-mcp-server/test/stdio.e2e.test.cjs b/packages/ql3-local-mcp-server/test/stdio.e2e.test.cjs index 47892bc3..da1a139c 100644 --- a/packages/ql3-local-mcp-server/test/stdio.e2e.test.cjs +++ b/packages/ql3-local-mcp-server/test/stdio.e2e.test.cjs @@ -90,6 +90,44 @@ async function fixture(t) { priority: 1, createdAtMs: NOW - 3_000, }); + await transaction.insertRun({ + id: 'run-mcp-e2e-success', + projectId: 'default', + taskId: 'task-mcp', + taskRevision: 'revision-3', + taskName: 'MCP successful baseline', + triggerType: 'manual', + executionOrigin: 'manual', + executionOwner: 'runtime', + triggeredBy: 'user:mcp-owner', + status: 'succeeded', + version: 0, + eventSequence: 0, + priority: 0, + createdAtMs: NOW - 4_000, + queuedAtMs: NOW - 3_900, + startedAtMs: NOW - 3_800, + finishedAtMs: NOW - 3_600, + }); + await transaction.insertRun({ + id: 'run-mcp-e2e-failure', + projectId: 'default', + taskId: 'task-mcp', + taskRevision: 'revision-4', + taskName: 'MCP failed candidate', + triggerType: 'manual', + executionOrigin: 'manual', + executionOwner: 'runtime', + triggeredBy: 'user:mcp-owner', + status: 'failed', + version: 0, + eventSequence: 0, + priority: 2, + createdAtMs: NOW - 5_000, + queuedAtMs: NOW - 4_850, + startedAtMs: NOW - 4_700, + finishedAtMs: NOW - 4_300, + }); await transaction.appendEvent({ id: 'mcp-e2e-event-1', runId: 'run-mcp-e2e', @@ -473,6 +511,7 @@ test('serves the authenticated Run Tool over the real stdio protocol and persist 'qinglong.run.list', 'qinglong.run.get', 'qinglong.run.compare', + 'qinglong.task.runs.compare', 'qinglong.run.events.list', 'qinglong.run.steps.list', 'qinglong.task.get', @@ -684,6 +723,67 @@ test('serves the authenticated Run Tool over the real stdio protocol and persist changedFields: ['taskRevision', 'priority'], consistency: 'ordered_independent_point_reads', }); + const comparedOutcomes = await request('tools/call', { + name: 'qinglong.task.runs.compare', + arguments: { taskId: 'task-mcp' }, + }); + assert.equal( + comparedOutcomes.result.isError, + undefined, + JSON.stringify(comparedOutcomes), + ); + assert.deepEqual(comparedOutcomes.result.structuredContent, { + taskId: 'task-mcp', + baselineOutcome: 'succeeded', + candidateOutcome: 'failed', + baseline: { + found: true, + id: 'run-mcp-e2e-success', + taskId: 'task-mcp', + taskRevision: 'revision-3', + status: 'succeeded', + version: 0, + eventSequence: 0, + priority: 0, + executionOrigin: 'manual', + executionOwner: 'runtime', + createdAtMs: NOW - 4_000, + queuedAtMs: NOW - 3_900, + startedAtMs: NOW - 3_800, + finishedAtMs: NOW - 3_600, + }, + candidate: { + found: true, + id: 'run-mcp-e2e-failure', + taskId: 'task-mcp', + taskRevision: 'revision-4', + status: 'failed', + version: 0, + eventSequence: 0, + priority: 2, + executionOrigin: 'manual', + executionOwner: 'runtime', + createdAtMs: NOW - 5_000, + queuedAtMs: NOW - 4_850, + startedAtMs: NOW - 4_700, + finishedAtMs: NOW - 4_300, + }, + comparable: true, + sameTask: true, + sameTaskRevision: false, + changedFields: ['taskRevision', 'status', 'priority'], + queueDelayDeltaMs: 50, + executionDurationDeltaMs: 200, + totalDurationDeltaMs: 300, + consistency: 'bounded_task_window_then_ordered_point_reads', + selection: { + windowLimit: 64, + searchedRunCount: 4, + hasOlderRuns: false, + complete: true, + order: 'created_at_desc_id_desc', + }, + }); const events = await request('tools/call', { name: 'qinglong.run.events.list', arguments: { runId: 'run-mcp-e2e', limit: 1 }, @@ -765,6 +865,11 @@ test('serves the authenticated Run Tool over the real stdio protocol and persist outcome: 'allowed', subjectId: 'mcp-user', }, + { + operationId: 'mcp.tool.call', + outcome: 'allowed', + subjectId: 'mcp-user', + }, ], ); } finally { diff --git a/packages/ql3-local-sqlite/package.json b/packages/ql3-local-sqlite/package.json index 37b73484..1c7271ea 100644 --- a/packages/ql3-local-sqlite/package.json +++ b/packages/ql3-local-sqlite/package.json @@ -205,6 +205,11 @@ "require": "./dist/tool-execution/localMcpReadDatabase.js", "default": "./dist/tool-execution/localMcpReadDatabase.js" }, + "./task-run-outcome-window": { + "types": "./dist/run/outcome-comparison/taskRunOutcomeWindowReader.d.ts", + "require": "./dist/run/outcome-comparison/taskRunOutcomeWindowReader.js", + "default": "./dist/run/outcome-comparison/taskRunOutcomeWindowReader.js" + }, "./approved-action": { "types": "./dist/approved-action/approvalRequestRepository.d.ts", "require": "./dist/approved-action/approvalRequestRepository.js", diff --git a/packages/ql3-local-sqlite/src/run/outcome-comparison/taskRunOutcomeWindowReader.ts b/packages/ql3-local-sqlite/src/run/outcome-comparison/taskRunOutcomeWindowReader.ts new file mode 100644 index 00000000..88d1bc38 --- /dev/null +++ b/packages/ql3-local-sqlite/src/run/outcome-comparison/taskRunOutcomeWindowReader.ts @@ -0,0 +1,49 @@ +import { RUN_STATUSES } from '@qinglong/runtime-core/run'; +import { + normalizeTaskRunOutcomeWindowQuery, + type TaskRunOutcomeWindowQuery, + type TaskRunOutcomeWindowReader, + type TaskRunOutcomeWindowRecord, +} from '@qinglong/runtime-core/task-run-outcome-window'; +import type { DatabaseSync } from 'node:sqlite'; + +import { + queryRows, + requiredEnum, + requiredInteger, + requiredString, +} from '../runPersistence'; + +export class LocalSqliteTaskRunOutcomeWindowReader + implements TaskRunOutcomeWindowReader +{ + constructor(private readonly client: DatabaseSync) {} + + async listRecentRunsByTask( + value: Readonly, + ): Promise[]> { + const query = normalizeTaskRunOutcomeWindowQuery(value); + return queryRows( + this.client, + `SELECT + "id" AS "id", + "project_id" AS "projectId", + "task_id" AS "taskId", + "status" AS "status", + "created_at_ms" AS "createdAtMs" + FROM "Runs" + WHERE "project_id" = ? AND "task_id" = ? + ORDER BY "created_at_ms" DESC, "id" DESC + LIMIT ?`, + [query.projectId, query.taskId, query.limit], + ).map((row) => + Object.freeze({ + id: requiredString(row, 'id'), + projectId: requiredString(row, 'projectId'), + taskId: requiredString(row, 'taskId'), + status: requiredEnum(row, 'status', RUN_STATUSES), + createdAtMs: requiredInteger(row, 'createdAtMs'), + }), + ); + } +} diff --git a/packages/ql3-local-sqlite/src/tool-execution/localMcpReadDatabase.ts b/packages/ql3-local-sqlite/src/tool-execution/localMcpReadDatabase.ts index 0d892c7d..cd365348 100644 --- a/packages/ql3-local-sqlite/src/tool-execution/localMcpReadDatabase.ts +++ b/packages/ql3-local-sqlite/src/tool-execution/localMcpReadDatabase.ts @@ -5,6 +5,10 @@ import type { ProjectRunListQuery, ProjectRunListReader, } from '@qinglong/runtime-core/project-run-list'; +import type { + TaskRunOutcomeWindowQuery, + TaskRunOutcomeWindowReader, +} from '@qinglong/runtime-core/task-run-outcome-window'; import { RunRepositoryBusyError, RunRepositoryOperationError, @@ -26,6 +30,7 @@ import { type LocalSqliteReadinessEvidence, } from '../readiness/readiness'; import { LocalSqliteRunReader } from '../run/runReader'; +import { LocalSqliteTaskRunOutcomeWindowReader } from '../run/outcome-comparison/taskRunOutcomeWindowReader'; import { LocalSqliteStepRunRepository } from '../run/stepRunRepository'; import { assertLocalSqliteOptions, @@ -44,7 +49,8 @@ export interface LocalSqliteMcpReadDatabase { readonly profile: LocalSqliteProfile; readonly readiness: LocalSqliteReadinessEvidence; readonly runs: Pick & - ProjectRunListReader; + ProjectRunListReader & + TaskRunOutcomeWindowReader; readonly stepRuns: Pick; readonly taskDefinitions: Pick< TaskDefinitionSource, @@ -76,13 +82,17 @@ export async function openLocalSqliteMcpReadDatabase( const readiness = await auditLocalSqliteReadiness(client); const authority = new LocalSqliteOperationAuthority(client); const reader = new LocalSqliteRunReader(client); + const outcomeWindowReader = new LocalSqliteTaskRunOutcomeWindowReader( + client, + ); const stepRunRepository = new LocalSqliteStepRunRepository(authority); const taskRepository = new LocalSqliteTaskDefinitionRepository(authority); const triggerRepository = new LocalSqliteTriggerRepository(authority); const approvalSource = new LocalSqliteApprovalRequestSource(authority); const security = new LocalSqliteSecurityAuthorityStore(authority); const runs: Pick & - ProjectRunListReader = Object.freeze({ + ProjectRunListReader & + TaskRunOutcomeWindowReader = Object.freeze({ listRunsByProject(query: Readonly) { return authority.enqueue( () => reader.listRunsByProject(query), @@ -94,6 +104,17 @@ export async function openLocalSqliteMcpReadDatabase( ), ); }, + listRecentRunsByTask(query: Readonly) { + return authority.enqueue( + () => outcomeWindowReader.listRecentRunsByTask(query), + (reason) => + reason === 'busy' + ? new RunRepositoryBusyError() + : new RunRepositoryOperationError( + new Error('Local SQLite MCP read database is closed'), + ), + ); + }, findRunById(runId: string) { return authority.enqueue( () => reader.findRunById(runId), diff --git a/packages/ql3-local-sqlite/test/taskRunOutcomeWindowReader.test.cjs b/packages/ql3-local-sqlite/test/taskRunOutcomeWindowReader.test.cjs new file mode 100644 index 00000000..b484b44f --- /dev/null +++ b/packages/ql3-local-sqlite/test/taskRunOutcomeWindowReader.test.cjs @@ -0,0 +1,125 @@ +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const os = require('node:os'); +const path = require('node:path'); +const { DatabaseSync } = require('node:sqlite'); +const { test } = require('node:test'); + +const { + migrateLocalSqlitePath, + openLocalSqliteRuntimeDatabase, +} = require('../dist'); +const { + openLocalSqliteMcpReadDatabase, +} = require('@qinglong/local-sqlite/mcp-read-database'); +const { + LocalSqliteTaskRunOutcomeWindowReader, +} = require('@qinglong/local-sqlite/task-run-outcome-window'); + +function run(id, projectId, taskId, status, createdAtMs) { + return { + id, + projectId, + taskId, + taskRevision: 'revision-1', + taskName: id, + triggerType: 'manual', + executionOrigin: 'manual', + executionOwner: 'runtime', + triggeredBy: 'test', + status, + version: 0, + eventSequence: 0, + priority: 0, + createdAtMs, + }; +} + +test('reads one fixed Project Task outcome window through the existing indexed SQLite authority', async (t) => { + assert.equal(typeof LocalSqliteTaskRunOutcomeWindowReader, 'function'); + assert.equal( + require('@qinglong/local-sqlite').LocalSqliteTaskRunOutcomeWindowReader, + undefined, + ); + const directory = fs.mkdtempSync( + path.join(os.tmpdir(), 'ql3-outcome-window-'), + ); + t.after(() => fs.rmSync(directory, { recursive: true, force: true })); + const options = { + databasePath: path.join(directory, 'qinglong3.sqlite'), + profile: 'edge', + }; + await migrateLocalSqlitePath(options); + const runtime = await openLocalSqliteRuntimeDatabase(options); + await runtime.runRepository.transaction(async (transaction) => { + for (const value of [ + run('run-old-success', 'default', 'task-a', 'succeeded', 10), + run('run-failure', 'default', 'task-a', 'failed', 20), + run('run-running', 'default', 'task-a', 'running', 30), + run('run-other-task', 'default', 'task-b', 'failed', 40), + run('run-other-project', 'other', 'task-a', 'failed', 50), + ]) { + await transaction.insertRun(value); + } + }); + await runtime.close(); + + const database = await openLocalSqliteMcpReadDatabase(options); + t.after(() => database.close()); + const rows = await database.runs.listRecentRunsByTask({ + projectId: 'default', + taskId: 'task-a', + limit: 3, + }); + assert.deepEqual(rows, [ + { + id: 'run-running', + projectId: 'default', + taskId: 'task-a', + status: 'running', + createdAtMs: 30, + }, + { + id: 'run-failure', + projectId: 'default', + taskId: 'task-a', + status: 'failed', + createdAtMs: 20, + }, + { + id: 'run-old-success', + projectId: 'default', + taskId: 'task-a', + status: 'succeeded', + createdAtMs: 10, + }, + ]); + await assert.rejects( + database.runs.listRecentRunsByTask({ + projectId: 'default', + taskId: 'task-a', + limit: 66, + }), + TypeError, + ); + + await database.close(); + const client = new DatabaseSync(options.databasePath, { readOnly: true }); + t.after(() => { + if (client.isOpen) client.close(); + }); + const plan = client + .prepare( + `EXPLAIN QUERY PLAN + SELECT "id", "project_id", "task_id", "status", "created_at_ms" + FROM "Runs" + WHERE "project_id" = ? AND "task_id" = ? + ORDER BY "created_at_ms" DESC, "id" DESC + LIMIT ?`, + ) + .all('default', 'task-a', 65); + assert.match( + plan.map((entry) => entry.detail).join('\n'), + /ql3_local_runs_task_created_idx/u, + ); +}); diff --git a/packages/ql3-runtime-core/package.json b/packages/ql3-runtime-core/package.json index 86e3d755..801c57ea 100644 --- a/packages/ql3-runtime-core/package.json +++ b/packages/ql3-runtime-core/package.json @@ -242,6 +242,12 @@ "builtin-run-compare-projection": [ "dist/tool-execution/builtin-run-compare/builtInRunCompareProjection.d.ts" ], + "builtin-task-run-outcome-compare-tool": [ + "dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareTool.d.ts" + ], + "builtin-task-run-outcome-compare-projection": [ + "dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareProjection.d.ts" + ], "run": [ "dist/run/run.d.ts" ], @@ -254,6 +260,9 @@ "project-run-list": [ "dist/run/projectRunList.d.ts" ], + "task-run-outcome-window": [ + "dist/run/outcome-comparison/taskRunOutcomeWindow.d.ts" + ], "bounded-run-read-projection": [ "dist/run/projection/boundedRunReadProjection.d.ts" ], @@ -318,6 +327,11 @@ "require": "./dist/run/projectRunList.js", "default": "./dist/run/projectRunList.js" }, + "./task-run-outcome-window": { + "types": "./dist/run/outcome-comparison/taskRunOutcomeWindow.d.ts", + "require": "./dist/run/outcome-comparison/taskRunOutcomeWindow.js", + "default": "./dist/run/outcome-comparison/taskRunOutcomeWindow.js" + }, "./bounded-run-read-projection": { "types": "./dist/run/projection/boundedRunReadProjection.d.ts", "require": "./dist/run/projection/boundedRunReadProjection.js", @@ -668,6 +682,16 @@ "require": "./dist/tool-execution/builtin-run-compare/builtInRunCompareProjection.js", "default": "./dist/tool-execution/builtin-run-compare/builtInRunCompareProjection.js" }, + "./builtin-task-run-outcome-compare-tool": { + "types": "./dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareTool.d.ts", + "require": "./dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareTool.js", + "default": "./dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareTool.js" + }, + "./builtin-task-run-outcome-compare-projection": { + "types": "./dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareProjection.d.ts", + "require": "./dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareProjection.js", + "default": "./dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareProjection.js" + }, "./secret-reference": { "types": "./dist/secret/secretReference.d.ts", "require": "./dist/secret/secretReference.js", diff --git a/packages/ql3-runtime-core/src/run/outcome-comparison/taskRunOutcomeWindow.ts b/packages/ql3-runtime-core/src/run/outcome-comparison/taskRunOutcomeWindow.ts new file mode 100644 index 00000000..97b94ac4 --- /dev/null +++ b/packages/ql3-runtime-core/src/run/outcome-comparison/taskRunOutcomeWindow.ts @@ -0,0 +1,91 @@ +import { RUN_STATUSES, type RunStatus } from '../run'; + +export const MAX_TASK_RUN_OUTCOME_WINDOW_STORAGE_LIMIT = 65; + +const CONTROL_PATTERN = /[\u0000-\u001f\u007f]/; + +export interface TaskRunOutcomeWindowQuery { + readonly projectId: string; + readonly taskId: string; + readonly limit: number; +} + +export interface TaskRunOutcomeWindowRecord { + readonly id: string; + readonly projectId: string; + readonly taskId: string; + readonly status: RunStatus; + readonly createdAtMs: number; +} + +export interface TaskRunOutcomeWindowReader { + listRecentRunsByTask( + query: Readonly, + ): Promise[]>; +} + +function boundedText(value: unknown, maximum: number): value is string { + return ( + typeof value === 'string' && + value.length > 0 && + value.length <= maximum && + !CONTROL_PATTERN.test(value) + ); +} + +export function normalizeTaskRunOutcomeWindowQuery( + value: Readonly, +): Readonly { + if ( + !value || + typeof value !== 'object' || + Array.isArray(value) || + Reflect.ownKeys(value).length !== 3 || + !Object.hasOwn(value, 'projectId') || + !Object.hasOwn(value, 'taskId') || + !Object.hasOwn(value, 'limit') || + !boundedText(value.projectId, 128) || + !boundedText(value.taskId, 255) || + !Number.isSafeInteger(value.limit) || + value.limit < 1 || + value.limit > MAX_TASK_RUN_OUTCOME_WINDOW_STORAGE_LIMIT + ) { + throw new TypeError('Task Run outcome window query is invalid'); + } + return Object.freeze({ + projectId: value.projectId, + taskId: value.taskId, + limit: value.limit, + }); +} + +export function normalizeTaskRunOutcomeWindowRecord( + value: Readonly, +): Readonly { + if ( + !value || + typeof value !== 'object' || + Array.isArray(value) || + Reflect.ownKeys(value).length !== 5 || + !Object.hasOwn(value, 'id') || + !Object.hasOwn(value, 'projectId') || + !Object.hasOwn(value, 'taskId') || + !Object.hasOwn(value, 'status') || + !Object.hasOwn(value, 'createdAtMs') || + !boundedText(value.id, 128) || + !boundedText(value.projectId, 128) || + !boundedText(value.taskId, 255) || + !RUN_STATUSES.includes(value.status) || + !Number.isSafeInteger(value.createdAtMs) || + value.createdAtMs < 0 + ) { + throw new TypeError('Task Run outcome window record is invalid'); + } + return Object.freeze({ + id: value.id, + projectId: value.projectId, + taskId: value.taskId, + status: value.status, + createdAtMs: value.createdAtMs, + }); +} diff --git a/packages/ql3-runtime-core/src/tool-execution/builtin-run-compare/builtInRunCompareProjection.ts b/packages/ql3-runtime-core/src/tool-execution/builtin-run-compare/builtInRunCompareProjection.ts index 2a04695f..75c8006c 100644 --- a/packages/ql3-runtime-core/src/tool-execution/builtin-run-compare/builtInRunCompareProjection.ts +++ b/packages/ql3-runtime-core/src/tool-execution/builtin-run-compare/builtInRunCompareProjection.ts @@ -19,7 +19,7 @@ export const BUILTIN_RUN_COMPARE_TIMEOUT_SECONDS = 5; const MAX_INT = 2_147_483_647; const MIN_INT = -2_147_483_648; const CONTROL_PATTERN = /[\u0000-\u001f\u007f]/; -const COMPARABLE_FIELDS = Object.freeze([ +export const BUILTIN_RUN_COMPARABLE_FIELDS = Object.freeze([ 'taskId', 'taskRevision', 'status', @@ -28,7 +28,7 @@ const COMPARABLE_FIELDS = Object.freeze([ 'executionOwner', ] as const); -const RUN_PROJECTION_SCHEMA = Object.freeze({ +export const BUILTIN_RUN_PROJECTION_SCHEMA = Object.freeze({ type: 'object' as const, properties: { found: { type: 'boolean' as const }, @@ -103,8 +103,8 @@ export const BUILTIN_RUN_COMPARE_TOOL_DEFINITION = normalizeToolDefinition({ outputSchema: { type: 'object', properties: { - baseline: RUN_PROJECTION_SCHEMA, - candidate: RUN_PROJECTION_SCHEMA, + baseline: BUILTIN_RUN_PROJECTION_SCHEMA, + candidate: BUILTIN_RUN_PROJECTION_SCHEMA, comparable: { type: 'boolean' }, sameTask: { type: 'boolean' }, sameTaskRevision: { type: 'boolean' }, @@ -113,9 +113,9 @@ export const BUILTIN_RUN_COMPARE_TOOL_DEFINITION = normalizeToolDefinition({ items: { type: 'string', maxLength: 32, - enum: COMPARABLE_FIELDS, + enum: BUILTIN_RUN_COMPARABLE_FIELDS, }, - maxItems: COMPARABLE_FIELDS.length, + maxItems: BUILTIN_RUN_COMPARABLE_FIELDS.length, }, queueDelayDeltaMs: { type: 'integer', @@ -212,14 +212,21 @@ function timestampDelta( return Number.isSafeInteger(delta) ? delta : undefined; } -function compareProjections( +export type BuiltInRunComparisonConsistency = + | 'ordered_independent_point_reads' + | 'bounded_task_window_then_ordered_point_reads'; + +export function compareBoundedRunProjections( baseline: BoundedRunReadProjection, candidate: BoundedRunReadProjection, + consistency: BuiltInRunComparisonConsistency, ): Readonly> { const comparable = baseline.found === true && candidate.found === true; const changedFields = Object.freeze( comparable - ? COMPARABLE_FIELDS.filter((field) => baseline[field] !== candidate[field]) + ? BUILTIN_RUN_COMPARABLE_FIELDS.filter( + (field) => baseline[field] !== candidate[field], + ) : [], ); const queueDelayDeltaMs = comparable @@ -259,7 +266,7 @@ function compareProjections( ? {} : { executionDurationDeltaMs }), ...(totalDurationDeltaMs === undefined ? {} : { totalDurationDeltaMs }), - consistency: 'ordered_independent_point_reads', + consistency, }); } @@ -295,7 +302,11 @@ export async function executeBuiltInRunCompareTool( projectId, inputRecord.candidateRunId, ); - return compareProjections(baseline, candidate); + return compareBoundedRunProjections( + baseline, + candidate, + 'ordered_independent_point_reads', + ); } catch (error) { if (!(error instanceof BoundedRunReadProjectionUnavailableError)) { return invalid('execution context or input is invalid'); diff --git a/packages/ql3-runtime-core/src/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareProjection.ts b/packages/ql3-runtime-core/src/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareProjection.ts new file mode 100644 index 00000000..8de469e7 --- /dev/null +++ b/packages/ql3-runtime-core/src/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareProjection.ts @@ -0,0 +1,360 @@ +import type { RunRepositoryReader } from '../../run/runRepository'; +import { + BoundedRunReadProjectionUnavailableError, + executeBoundedRunReadProjection, + type BoundedRunReadProjection, +} from '../../run/projection/boundedRunReadProjection'; +import { + MAX_TASK_RUN_OUTCOME_WINDOW_STORAGE_LIMIT, + normalizeTaskRunOutcomeWindowQuery, + normalizeTaskRunOutcomeWindowRecord, + type TaskRunOutcomeWindowReader, + type TaskRunOutcomeWindowRecord, +} from '../../run/outcome-comparison/taskRunOutcomeWindow'; +import { + normalizeToolDefinition, + type ToolJsonValue, +} from '../tool-registry/toolRegistry'; +import { + BUILTIN_RUN_COMPARABLE_FIELDS, + BUILTIN_RUN_PROJECTION_SCHEMA, + compareBoundedRunProjections, +} from './builtInRunCompareProjection'; + +export const BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL = Object.freeze({ + name: 'qinglong.task.runs.compare', + version: '1.0.0', +}); +export const BUILTIN_TASK_RUN_OUTCOME_COMPARE_TIMEOUT_SECONDS = 5; +export const TASK_RUN_OUTCOME_SEARCH_LIMIT = 64; + +const MAX_INT = 2_147_483_647; +const MIN_INT = -2_147_483_648; +const CONTROL_PATTERN = /[\u0000-\u001f\u007f]/; + +export const BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION = + normalizeToolDefinition({ + name: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.name, + version: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.version, + description: + 'Compare the latest succeeded and failed Runs found in one fixed bounded Task history window', + inputSchema: { + type: 'object', + properties: { + taskId: { type: 'string', minLength: 1, maxLength: 255 }, + }, + required: ['taskId'], + additionalProperties: false, + }, + outputSchema: { + type: 'object', + properties: { + taskId: { type: 'string', minLength: 1, maxLength: 255 }, + baselineOutcome: { + type: 'string', + maxLength: 16, + enum: ['succeeded'] as const, + }, + candidateOutcome: { + type: 'string', + maxLength: 16, + enum: ['failed'] as const, + }, + baseline: BUILTIN_RUN_PROJECTION_SCHEMA, + candidate: BUILTIN_RUN_PROJECTION_SCHEMA, + comparable: { type: 'boolean' }, + sameTask: { type: 'boolean' }, + sameTaskRevision: { type: 'boolean' }, + changedFields: { + type: 'array', + items: { + type: 'string', + maxLength: 32, + enum: BUILTIN_RUN_COMPARABLE_FIELDS, + }, + maxItems: BUILTIN_RUN_COMPARABLE_FIELDS.length, + }, + queueDelayDeltaMs: { + type: 'integer', + minimum: -Number.MAX_SAFE_INTEGER, + maximum: Number.MAX_SAFE_INTEGER, + }, + executionDurationDeltaMs: { + type: 'integer', + minimum: -Number.MAX_SAFE_INTEGER, + maximum: Number.MAX_SAFE_INTEGER, + }, + totalDurationDeltaMs: { + type: 'integer', + minimum: -Number.MAX_SAFE_INTEGER, + maximum: Number.MAX_SAFE_INTEGER, + }, + selection: { + type: 'object', + properties: { + windowLimit: { + type: 'integer', + minimum: TASK_RUN_OUTCOME_SEARCH_LIMIT, + maximum: TASK_RUN_OUTCOME_SEARCH_LIMIT, + }, + searchedRunCount: { + type: 'integer', + minimum: 0, + maximum: TASK_RUN_OUTCOME_SEARCH_LIMIT, + }, + hasOlderRuns: { type: 'boolean' }, + complete: { type: 'boolean' }, + order: { + type: 'string', + maxLength: 32, + enum: ['created_at_desc_id_desc'] as const, + }, + }, + required: [ + 'windowLimit', + 'searchedRunCount', + 'hasOlderRuns', + 'complete', + 'order', + ], + additionalProperties: false, + }, + consistency: { + type: 'string', + maxLength: 64, + enum: ['bounded_task_window_then_ordered_point_reads'] as const, + }, + }, + required: [ + 'taskId', + 'baselineOutcome', + 'candidateOutcome', + 'baseline', + 'candidate', + 'comparable', + 'sameTask', + 'sameTaskRevision', + 'changedFields', + 'selection', + 'consistency', + ], + additionalProperties: false, + }, + effect: 'read', + risk: 'low', + requiredPermissions: ['run.read'], + timeoutSeconds: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TIMEOUT_SECONDS, + }); + +export class InvalidBuiltInTaskRunOutcomeCompareToolError extends TypeError { + readonly code = 'BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_INVALID'; + + constructor(message: string) { + super(`Built-in Task Run outcome compare Tool is invalid: ${message}`); + this.name = 'InvalidBuiltInTaskRunOutcomeCompareToolError'; + } +} + +export class BuiltInTaskRunOutcomeCompareToolUnavailableError extends Error { + readonly code = 'BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_UNAVAILABLE'; + + constructor() { + super('Built-in Task Run outcome compare Tool is unavailable'); + this.name = 'BuiltInTaskRunOutcomeCompareToolUnavailableError'; + } +} + +function invalid(message: string): never { + throw new InvalidBuiltInTaskRunOutcomeCompareToolError(message); +} + +function boundedText(value: unknown, maximum: number): value is string { + return ( + typeof value === 'string' && + value.length > 0 && + value.length <= maximum && + !CONTROL_PATTERN.test(value) + ); +} + +function isStrictlyOlder( + value: Readonly, + previous: Readonly, +): boolean { + return ( + value.createdAtMs < previous.createdAtMs || + (value.createdAtMs === previous.createdAtMs && value.id < previous.id) + ); +} + +async function selectOutcomeWindow( + windows: TaskRunOutcomeWindowReader, + projectId: string, + taskId: string, +): Promise< + Readonly<{ + latestSucceededRunId?: string; + latestFailedRunId?: string; + searchedRunCount: number; + hasOlderRuns: boolean; + complete: boolean; + }> +> { + let rows: readonly Readonly[]; + try { + rows = await windows.listRecentRunsByTask( + normalizeTaskRunOutcomeWindowQuery({ + projectId, + taskId, + limit: MAX_TASK_RUN_OUTCOME_WINDOW_STORAGE_LIMIT, + }), + ); + } catch { + throw new BuiltInTaskRunOutcomeCompareToolUnavailableError(); + } + if ( + !Array.isArray(rows) || + rows.length > MAX_TASK_RUN_OUTCOME_WINDOW_STORAGE_LIMIT + ) { + throw new BuiltInTaskRunOutcomeCompareToolUnavailableError(); + } + + let previous: Readonly | undefined; + let latestSucceededRunId: string | undefined; + let latestFailedRunId: string | undefined; + for (const row of rows) { + let normalized: Readonly; + try { + normalized = normalizeTaskRunOutcomeWindowRecord(row); + } catch { + throw new BuiltInTaskRunOutcomeCompareToolUnavailableError(); + } + if ( + normalized.projectId !== projectId || + normalized.taskId !== taskId || + (previous !== undefined && !isStrictlyOlder(normalized, previous)) + ) { + throw new BuiltInTaskRunOutcomeCompareToolUnavailableError(); + } + previous = normalized; + } + + const window = rows.slice(0, TASK_RUN_OUTCOME_SEARCH_LIMIT); + for (const row of window) { + if (row.status === 'succeeded' && latestSucceededRunId === undefined) { + latestSucceededRunId = row.id; + } + if (row.status === 'failed' && latestFailedRunId === undefined) { + latestFailedRunId = row.id; + } + if (latestSucceededRunId !== undefined && latestFailedRunId !== undefined) { + break; + } + } + const hasOlderRuns = rows.length > TASK_RUN_OUTCOME_SEARCH_LIMIT; + const complete = + !hasOlderRuns || + (latestSucceededRunId !== undefined && latestFailedRunId !== undefined); + return Object.freeze({ + ...(latestSucceededRunId === undefined ? {} : { latestSucceededRunId }), + ...(latestFailedRunId === undefined ? {} : { latestFailedRunId }), + searchedRunCount: window.length, + hasOlderRuns, + complete, + }); +} + +async function selectedProjection( + runs: Pick, + projectId: string, + taskId: string, + expectedStatus: 'succeeded' | 'failed', + runId: string | undefined, +): Promise> { + if (runId === undefined) return Object.freeze({ found: false }); + const projection = await executeBoundedRunReadProjection( + runs, + projectId, + runId, + ); + if ( + projection.found !== true || + projection.taskId !== taskId || + projection.status !== expectedStatus + ) { + throw new BuiltInTaskRunOutcomeCompareToolUnavailableError(); + } + return projection; +} + +export async function executeBuiltInTaskRunOutcomeCompareTool( + windows: TaskRunOutcomeWindowReader, + runs: Pick, + projectId: string, + input: ToolJsonValue, +): Promise>> { + const inputRecord = + input && typeof input === 'object' && !Array.isArray(input) + ? (input as Readonly>) + : null; + if ( + !windows || + typeof windows.listRecentRunsByTask !== 'function' || + !runs || + typeof runs.findRunById !== 'function' || + !boundedText(projectId, 128) || + !inputRecord || + Reflect.ownKeys(inputRecord).length !== 1 || + !boundedText(inputRecord.taskId, 255) + ) { + return invalid('execution context or input is invalid'); + } + + try { + const selected = await selectOutcomeWindow( + windows, + projectId, + inputRecord.taskId, + ); + const baseline = await selectedProjection( + runs, + projectId, + inputRecord.taskId, + 'succeeded', + selected.latestSucceededRunId, + ); + const candidate = await selectedProjection( + runs, + projectId, + inputRecord.taskId, + 'failed', + selected.latestFailedRunId, + ); + const comparison = compareBoundedRunProjections( + baseline, + candidate, + 'bounded_task_window_then_ordered_point_reads', + ); + return Object.freeze({ + taskId: inputRecord.taskId, + baselineOutcome: 'succeeded', + candidateOutcome: 'failed', + ...comparison, + selection: Object.freeze({ + windowLimit: TASK_RUN_OUTCOME_SEARCH_LIMIT, + searchedRunCount: selected.searchedRunCount, + hasOlderRuns: selected.hasOlderRuns, + complete: selected.complete, + order: 'created_at_desc_id_desc', + }), + }); + } catch (error) { + if ( + error instanceof BuiltInTaskRunOutcomeCompareToolUnavailableError || + error instanceof BoundedRunReadProjectionUnavailableError + ) { + throw new BuiltInTaskRunOutcomeCompareToolUnavailableError(); + } + return invalid('execution context or input is invalid'); + } +} diff --git a/packages/ql3-runtime-core/src/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareTool.ts b/packages/ql3-runtime-core/src/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareTool.ts new file mode 100644 index 00000000..4c2c6e40 --- /dev/null +++ b/packages/ql3-runtime-core/src/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareTool.ts @@ -0,0 +1,181 @@ +import type { DeploymentProfile } from '../../cluster-control/clusterControlActivation'; +import type { RunRepositoryReader } from '../../run/runRepository'; +import type { TaskRunOutcomeWindowReader } from '../../run/outcome-comparison/taskRunOutcomeWindow'; +import { + normalizeProjectToolDefinitionSnapshot, + type ProjectToolDefinitionSnapshot, +} from '../tool-registry/projectToolDefinitionSnapshot'; +import { + ToolDefinitionRegistry, + type ToolJsonValue, +} from '../tool-registry/toolRegistry'; +import { + createTrustedToolHandlerBinding, + normalizeTrustedToolHandlerBinding, + type TrustedToolHandlerBinding, +} from '../trustedToolInvocation'; +import type { + TrustedToolExecutionAdapter, + TrustedToolExecutionAdapterContext, +} from '../trustedToolExecution'; +import { + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TIMEOUT_SECONDS, + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL, + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION, + InvalidBuiltInTaskRunOutcomeCompareToolError, + executeBuiltInTaskRunOutcomeCompareTool, +} from './builtInTaskRunOutcomeCompareProjection'; + +export { + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TIMEOUT_SECONDS, + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL, + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION, + TASK_RUN_OUTCOME_SEARCH_LIMIT, + BuiltInTaskRunOutcomeCompareToolUnavailableError, + InvalidBuiltInTaskRunOutcomeCompareToolError, + executeBuiltInTaskRunOutcomeCompareTool, +} from './builtInTaskRunOutcomeCompareProjection'; + +export const BUILTIN_TASK_RUN_OUTCOME_COMPARE_ADAPTER = Object.freeze({ + id: 'builtin.qinglong.task-runs-compare', + version: '1.0.0', +}); +export const BUILTIN_TASK_RUN_OUTCOME_COMPARE_REDACTION_CONTRACT = + Object.freeze({ + id: 'redaction.qinglong.task-runs-compare', + version: '1.0.0', + }); +export const BUILTIN_TASK_RUN_OUTCOME_COMPARE_AUDIT_CONTRACT = Object.freeze({ + id: 'audit.qinglong.tool-call', + version: '1.0.0', +}); + +const CONTROL_PATTERN = /[\u0000-\u001f\u007f]/; + +function invalid(message: string): never { + throw new InvalidBuiltInTaskRunOutcomeCompareToolError(message); +} + +function sameValue(left: unknown, right: unknown): boolean { + return JSON.stringify(left) === JSON.stringify(right); +} + +function boundedText(value: unknown, maximum: number): value is string { + return ( + typeof value === 'string' && + value.length > 0 && + value.length <= maximum && + !CONTROL_PATTERN.test(value) + ); +} + +export function createBuiltInTaskRunOutcomeCompareToolHandlerBinding( + snapshotValue: ProjectToolDefinitionSnapshot, + profiles: readonly DeploymentProfile[], +): Readonly { + const snapshot = normalizeProjectToolDefinitionSnapshot(snapshotValue); + const definition = snapshot.definitions.find( + (entry) => + entry.definition.name === BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.name && + entry.definition.version === + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.version, + )?.definition; + if ( + !definition || + !sameValue(definition, BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION) + ) { + return invalid('reviewed Tool definition is absent or changed'); + } + return createTrustedToolHandlerBinding(snapshot, { + tool: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL, + adapter: BUILTIN_TASK_RUN_OUTCOME_COMPARE_ADAPTER, + executionClass: 'builtin_in_process', + profiles, + authorities: ['database.read'], + timeoutSeconds: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TIMEOUT_SECONDS, + redactionContract: BUILTIN_TASK_RUN_OUTCOME_COMPARE_REDACTION_CONTRACT, + auditContract: BUILTIN_TASK_RUN_OUTCOME_COMPARE_AUDIT_CONTRACT, + }); +} + +export class BuiltInTaskRunOutcomeCompareToolAdapter + implements TrustedToolExecutionAdapter +{ + readonly binding!: Readonly; + readonly profile!: DeploymentProfile; + readonly recoveryMode = 'retry_safe_read' as const; + readonly #windows!: TaskRunOutcomeWindowReader; + readonly #runs!: Pick; + + constructor( + bindingValue: TrustedToolHandlerBinding, + profile: DeploymentProfile, + definitions: ToolDefinitionRegistry, + windows: TaskRunOutcomeWindowReader, + runs: Pick, + ) { + const binding = normalizeTrustedToolHandlerBinding(bindingValue); + if (!(definitions instanceof ToolDefinitionRegistry)) { + return invalid('Tool Definition registry is invalid'); + } + let definition; + try { + definition = definitions.resolve( + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.name, + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.version, + ); + } catch { + return invalid('reviewed Tool definition is unavailable'); + } + if ( + !sameValue(binding.tool, BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL) || + !sameValue(binding.adapter, BUILTIN_TASK_RUN_OUTCOME_COMPARE_ADAPTER) || + binding.executionClass !== 'builtin_in_process' || + !sameValue(binding.authorities, ['database.read']) || + binding.timeoutSeconds !== + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TIMEOUT_SECONDS || + !sameValue( + binding.redactionContract, + BUILTIN_TASK_RUN_OUTCOME_COMPARE_REDACTION_CONTRACT, + ) || + !sameValue( + binding.auditContract, + BUILTIN_TASK_RUN_OUTCOME_COMPARE_AUDIT_CONTRACT, + ) || + !binding.profiles.includes(profile) || + !sameValue(definition, BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION) + ) { + return invalid('binding does not match the reviewed adapter contract'); + } + if (!windows || typeof windows.listRecentRunsByTask !== 'function') { + return invalid('Task Run outcome window reader is invalid'); + } + if (!runs || typeof runs.findRunById !== 'function') { + return invalid('Run repository is invalid'); + } + this.binding = binding; + this.profile = profile; + this.#windows = windows; + this.#runs = runs; + Object.freeze(this); + } + + async execute( + context: Readonly, + input: ToolJsonValue, + ): Promise { + if ( + !context || + typeof context !== 'object' || + !boundedText(context.projectId, 128) + ) { + return invalid('execution context or input is invalid'); + } + return executeBuiltInTaskRunOutcomeCompareTool( + this.#windows, + this.#runs, + context.projectId, + input, + ); + } +} diff --git a/packages/ql3-runtime-core/test/builtInTaskRunOutcomeCompareTool.test.cjs b/packages/ql3-runtime-core/test/builtInTaskRunOutcomeCompareTool.test.cjs new file mode 100644 index 00000000..319cf9c0 --- /dev/null +++ b/packages/ql3-runtime-core/test/builtInTaskRunOutcomeCompareTool.test.cjs @@ -0,0 +1,328 @@ +const assert = require('node:assert/strict'); +const { test } = require('node:test'); + +const { + BUILTIN_TASK_RUN_OUTCOME_COMPARE_ADAPTER, + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL, + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION, + BuiltInTaskRunOutcomeCompareToolAdapter, + BuiltInTaskRunOutcomeCompareToolUnavailableError, + InvalidBuiltInTaskRunOutcomeCompareToolError, + TASK_RUN_OUTCOME_SEARCH_LIMIT, + createBuiltInTaskRunOutcomeCompareToolHandlerBinding, + executeBuiltInTaskRunOutcomeCompareTool, +} = require('../dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareTool'); +const { + createPluginPackageResourceGenerationFromReferences, +} = require('../dist/plugin-package/pluginPackageResourceGeneration'); +const { + createProjectToolDefinitionSnapshot, + projectToolDefinitionRegistry, +} = require('../dist/tool-execution/tool-registry/projectToolDefinitionSnapshot'); + +const DIGEST_A = 'a'.repeat(64); +const DIGEST_B = 'b'.repeat(64); +const DIGEST_C = 'c'.repeat(64); + +function run(id, overrides = {}) { + return { + id, + projectId: 'project-outcomes', + taskId: 'task-backup', + taskRevision: 'task-backup@4', + triggerType: 'schedule', + executionOrigin: 'scheduled_system', + executionOwner: 'runtime', + status: 'succeeded', + version: 4, + eventSequence: 8, + priority: 10, + createdAtMs: 1_000, + queuedAtMs: 1_020, + startedAtMs: 1_050, + finishedAtMs: 1_150, + requestId: 'must-not-cross-tool-output', + ...overrides, + }; +} + +function windowRecord(value) { + return { + id: value.id, + projectId: value.projectId, + taskId: value.taskId, + status: value.status, + createdAtMs: value.createdAtMs, + }; +} + +function fixture(records, window, calls = []) { + return { + windows: { + async listRecentRunsByTask(query) { + calls.push({ type: 'window', query }); + return window; + }, + }, + runs: { + async findRunById(runId) { + calls.push({ type: 'point', runId }); + return records.get(runId) ?? null; + }, + }, + }; +} + +function snapshot( + definition = BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION, +) { + const generation = createPluginPackageResourceGenerationFromReferences({ + installationId: 'install-qinglong-outcome-compare', + projectId: 'project-outcomes', + packageName: 'qinglong', + lockDigest: DIGEST_A, + generation: 1, + previousActiveLockDigest: null, + contentDigest: DIGEST_B, + resources: [], + }); + return createProjectToolDefinitionSnapshot({ + projectId: 'project-outcomes', + contributions: [ + { + generation, + revisionDigest: DIGEST_C, + definitions: [definition], + }, + ], + }); +} + +test('selects and compares the latest succeeded and failed Runs in one fixed Task window', async () => { + const calls = []; + const succeeded = run('run-success'); + const failed = run('run-failure', { + taskRevision: 'task-backup@5', + status: 'failed', + version: 6, + eventSequence: 12, + createdAtMs: 2_000, + queuedAtMs: 2_040, + startedAtMs: 2_100, + finishedAtMs: 2_350, + }); + const ignored = run('run-running', { + status: 'running', + createdAtMs: 3_000, + }); + const value = fixture( + new Map([ + [succeeded.id, succeeded], + [failed.id, failed], + ]), + [windowRecord(ignored), windowRecord(failed), windowRecord(succeeded)], + calls, + ); + const output = await executeBuiltInTaskRunOutcomeCompareTool( + value.windows, + value.runs, + 'project-outcomes', + { taskId: 'task-backup' }, + ); + + assert.deepEqual(calls, [ + { + type: 'window', + query: { + projectId: 'project-outcomes', + taskId: 'task-backup', + limit: 65, + }, + }, + { type: 'point', runId: 'run-success' }, + { type: 'point', runId: 'run-failure' }, + ]); + assert.equal(output.taskId, 'task-backup'); + assert.equal(output.baselineOutcome, 'succeeded'); + assert.equal(output.candidateOutcome, 'failed'); + assert.equal(output.baseline.id, 'run-success'); + assert.equal(output.candidate.id, 'run-failure'); + assert.deepEqual(output.changedFields, ['taskRevision', 'status']); + assert.equal(output.queueDelayDeltaMs, 20); + assert.equal(output.executionDurationDeltaMs, 150); + assert.equal(output.totalDurationDeltaMs, 200); + assert.deepEqual(output.selection, { + windowLimit: 64, + searchedRunCount: 3, + hasOlderRuns: false, + complete: true, + order: 'created_at_desc_id_desc', + }); + assert.equal( + output.consistency, + 'bounded_task_window_then_ordered_point_reads', + ); + assert.equal(output.baseline.requestId, undefined); + + const registry = projectToolDefinitionRegistry(snapshot()); + assert.deepEqual( + registry.normalizeOutput( + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.name, + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.version, + output, + ), + output, + ); +}); + +test('reports an incomplete fixed window without exposing pagination', async () => { + const calls = []; + const rows = Array.from( + { length: TASK_RUN_OUTCOME_SEARCH_LIMIT + 1 }, + (_, index) => + windowRecord( + run(`run-${String(100 - index).padStart(3, '0')}`, { + status: 'running', + createdAtMs: 10_000 - index, + }), + ), + ); + const value = fixture(new Map(), rows, calls); + const output = await executeBuiltInTaskRunOutcomeCompareTool( + value.windows, + value.runs, + 'project-outcomes', + { taskId: 'task-backup' }, + ); + + assert.deepEqual(output.baseline, { found: false }); + assert.deepEqual(output.candidate, { found: false }); + assert.equal(output.comparable, false); + assert.deepEqual(output.selection, { + windowLimit: 64, + searchedRunCount: 64, + hasOlderRuns: true, + complete: false, + order: 'created_at_desc_id_desc', + }); + assert.equal(calls.length, 1); + assert.equal( + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION.inputSchema.properties + .after, + undefined, + ); + assert.equal( + BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION.inputSchema.properties + .limit, + undefined, + ); +}); + +test('fails closed on foreign, unordered, corrupt, disappearing, and unavailable records', async () => { + const valid = windowRecord(run('run-valid')); + for (const window of [ + [{ ...valid, projectId: 'other-project' }], + [valid, { ...valid, id: 'run-newer', createdAtMs: 2_000 }], + [{ ...valid, status: 'invented' }], + ]) { + const value = fixture(new Map(), window); + await assert.rejects( + executeBuiltInTaskRunOutcomeCompareTool( + value.windows, + value.runs, + 'project-outcomes', + { taskId: 'task-backup' }, + ), + BuiltInTaskRunOutcomeCompareToolUnavailableError, + ); + } + + const missing = fixture(new Map(), [valid]); + await assert.rejects( + executeBuiltInTaskRunOutcomeCompareTool( + missing.windows, + missing.runs, + 'project-outcomes', + { taskId: 'task-backup' }, + ), + BuiltInTaskRunOutcomeCompareToolUnavailableError, + ); + + await assert.rejects( + executeBuiltInTaskRunOutcomeCompareTool( + { + async listRecentRunsByTask() { + throw new Error('private DSN must not escape'); + }, + }, + fixture(new Map(), []).runs, + 'project-outcomes', + { taskId: 'task-backup' }, + ), + BuiltInTaskRunOutcomeCompareToolUnavailableError, + ); +}); + +test('rejects aliases and binds the reviewed retry-safe database-read adapter', async () => { + const empty = fixture(new Map(), []); + await assert.rejects( + executeBuiltInTaskRunOutcomeCompareTool( + empty.windows, + empty.runs, + 'project-outcomes', + { taskId: 'task-backup', after: 'cursor' }, + ), + InvalidBuiltInTaskRunOutcomeCompareToolError, + ); + + const currentSnapshot = snapshot(); + const binding = createBuiltInTaskRunOutcomeCompareToolHandlerBinding( + currentSnapshot, + ['edge', 'standalone', 'cluster-control'], + ); + assert.deepEqual(binding.tool, BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL); + assert.deepEqual(binding.adapter, BUILTIN_TASK_RUN_OUTCOME_COMPARE_ADAPTER); + assert.deepEqual(binding.authorities, ['database.read']); + + const adapter = new BuiltInTaskRunOutcomeCompareToolAdapter( + binding, + 'edge', + projectToolDefinitionRegistry(currentSnapshot), + empty.windows, + empty.runs, + ); + assert.equal(adapter.recoveryMode, 'retry_safe_read'); + const output = await adapter.execute( + { projectId: 'project-outcomes' }, + { taskId: 'task-backup' }, + ); + assert.equal(output.selection.complete, true); + + assert.throws( + () => + new BuiltInTaskRunOutcomeCompareToolAdapter( + { ...binding, authorities: ['database.read', 'network.client'] }, + 'edge', + projectToolDefinitionRegistry(currentSnapshot), + empty.windows, + empty.runs, + ), + /handler authority is invalid/, + ); +}); + +test('publishes only explicit outcome comparison subpaths and keeps the root unchanged', () => { + const tool = require('@qinglong/runtime-core/builtin-task-run-outcome-compare-tool'); + const projection = require('@qinglong/runtime-core/builtin-task-run-outcome-compare-projection'); + const window = require('@qinglong/runtime-core/task-run-outcome-window'); + const root = require('@qinglong/runtime-core'); + + assert.equal( + tool.BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.name, + 'qinglong.task.runs.compare', + ); + assert.equal(projection.TASK_RUN_OUTCOME_SEARCH_LIMIT, 64); + assert.equal(window.MAX_TASK_RUN_OUTCOME_WINDOW_STORAGE_LIMIT, 65); + assert.equal(root.BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL, undefined); + assert.equal(root.executeBuiltInTaskRunOutcomeCompareTool, undefined); +}); diff --git a/scripts/ql3-cluster-dependency-audit.cjs b/scripts/ql3-cluster-dependency-audit.cjs index 22978c52..5996ee77 100644 --- a/scripts/ql3-cluster-dependency-audit.cjs +++ b/scripts/ql3-cluster-dependency-audit.cjs @@ -1014,10 +1014,12 @@ function auditSourceImports(root, packagePath, findings) { '@qinglong/runtime-core/bounded-run-list-projection', '@qinglong/runtime-core/bounded-run-step-list-projection', '@qinglong/runtime-core/builtin-run-compare-projection', + '@qinglong/runtime-core/builtin-task-run-outcome-compare-projection', '@qinglong/runtime-core/builtin-run-read-projection', '@qinglong/runtime-core/bounded-task-read-projection', '@qinglong/runtime-core/bounded-task-list-projection', '@qinglong/runtime-core/project-run-list', + '@qinglong/runtime-core/task-run-outcome-window', '@qinglong/runtime-core/project-policy', '@qinglong/runtime-core/run', '@qinglong/runtime-core/run-repository', diff --git a/test/back/ql3ClusterDependencyAudit.test.cjs b/test/back/ql3ClusterDependencyAudit.test.cjs index 3b58cffd..5ffb1150 100644 --- a/test/back/ql3ClusterDependencyAudit.test.cjs +++ b/test/back/ql3ClusterDependencyAudit.test.cjs @@ -193,7 +193,9 @@ test('confines local MCP to its reviewed protocol and read-authority subpaths', "import { runs } from '@qinglong/runtime-core/bounded-run-list-projection';", "import { run } from '@qinglong/runtime-core/run';", "import { compare } from '@qinglong/runtime-core/builtin-run-compare-projection';", + "import { outcomes } from '@qinglong/runtime-core/builtin-task-run-outcome-compare-projection';", "import { tool } from '@qinglong/runtime-core/builtin-run-read-projection';", + "import { window } from '@qinglong/runtime-core/task-run-outcome-window';", "import { tasks } from '@qinglong/runtime-core/bounded-task-list-projection';", "import { task } from '@qinglong/runtime-core/bounded-task-read-projection';", "import { trigger } from '@qinglong/runtime-core/trigger';", diff --git a/test/back/ql3PackageBoundaryAudit.test.cjs b/test/back/ql3PackageBoundaryAudit.test.cjs index 14d8f44d..4a0ca309 100644 --- a/test/back/ql3PackageBoundaryAudit.test.cjs +++ b/test/back/ql3PackageBoundaryAudit.test.cjs @@ -299,10 +299,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( rootSourceFileRoles: runtimeCore.rootSourceFileRoles, }, { - sourceFiles: 162, + sourceFiles: 165, rootSourceFiles: 1, rootSourceLines: 160, - nestedSourceFiles: 161, + nestedSourceFiles: 164, rootSourceFileRoles: { 'index.ts': 'public_export' }, }, ); @@ -421,10 +421,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( rootSourceFileRoles: clusterPostgres.rootSourceFileRoles, }, { - sourceFiles: 167, + sourceFiles: 168, rootSourceFiles: 1, rootSourceLines: 126, - nestedSourceFiles: 166, + nestedSourceFiles: 167, rootSourceFileRoles: { 'index.ts': 'public_export' }, }, ); @@ -540,10 +540,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', ( rootSourceFileRoles: localSqlite.rootSourceFileRoles, }, { - sourceFiles: 196, + sourceFiles: 197, rootSourceFiles: 1, rootSourceLines: 31, - nestedSourceFiles: 195, + nestedSourceFiles: 196, rootSourceFileRoles: { 'index.ts': 'public_export' }, }, );