feat(ql3): compare bounded task run outcomes

This commit is contained in:
whyour
2026-08-14 17:18:25 +08:00
parent a55613afaa
commit 828370c60b
22 changed files with 1802 additions and 22 deletions
+8 -2
View File
@@ -11,6 +11,7 @@
最新增量证据(2026-08-13):
- D-310/ADR-0402(已接受):新增 `qinglong.task.runs.compare@1.0.0`,把“最近成功/失败 Run”的选择从模型无界分页收回服务端。输入只接受 Task ID,固定读取按 created/id 倒序的 64 条 Project-scoped Task Run,第 65 条仅证明窗口截断且协议不返回 cursor;选择 succeeded baseline 与 failed candidate 后按固定顺序执行最多两个低敏点查并复用共享差值算法,输出明确区分 complete 与窗口内未找到,consistency 固定为 `bounded_task_window_then_ordered_point_reads`。实现没有扩大 CRITICAL/HIGH 的通用 SQLite/PostgreSQL Run Reader,而是在双方既有 `run/outcome-comparison/` 中提供窄 adapter;不新增 package、依赖、migration、索引、连接、timer、listener、watcher 或 cache,默认 Edge/Standalone 制品字节数保持 2,589,812。定向 Runtime Core 10/10、SQLite 1/1(真实 query plan 命中既有 Task 时间索引)、PostgreSQL adapter 2/2、Local MCP 47/47、dependency firewall 53/53;最终 18-package clean build/test 与 backend 1,206 pass/2 条件 skip/0 failpackage/dependency/Edge/Cluster deployment 审计零 finding。Edge-MCP 为 7,237,187 bytes/795 files/220 modules/RSS 38,699,008 bytesStandalone-MCP 为 7,237,295 bytes/795 files/220 modules/RSS 38,600,704 bytes,均在门内。PostgreSQL 18.4 arm64 HA 125/125 Gate、timeline `1→2`,报告 SHA-256 为 `229c7cac328ee960f667f92868374264a10cb75090ef93d644da1326385d8774`
- D-309/ADR-0401(已接受):`qinglong.run.compare@1.0.0` 进入可选 `ql3-mcp` stdio 产品入口,复用 Runtime Core 的共享 Definition/projection 和既有 MCP 静态注册循环。每次调用重新执行 Owner credential 认证、`tool.call:qinglong.run.compare` + `run.read` Policy、durable allowed audit、credential fence confirm,再按 baseline→candidate 串行执行两个 Project-scoped SQLite 点查;错误稳定收敛为 `run_compare_unavailable`。该入口是 ADR-0347 的交互式只读 surface,只持久化安全 admission,不冒充 StepRun、encrypted Tool completion 或模型 Trace;内部 Copilot 的受信执行仍必须走完整 completion 链。实现不新增 package、依赖、migration、表、索引、连接、timer、listener、watcher、cache 或网络 endpoint,默认 Edge/Standalone 继续裁掉 MCP package,仅显式 `edge-mcp|standalone-mcp` 承担调用成本。Local MCP 46/46、最终 18-package clean build/test 退出 0、backend 1,206 pass/2 条件 skip/0 failpackage/dependency/Edge/Cluster deployment 审计零 finding;默认 Edge 为 2,589,812 bytes/315 files/56 modules/RSS 11,091,968 bytesEdge-MCP 为 7,219,977 bytes/792 files/217 modules/RSS 38,649,856 bytesStandalone-MCP 为 7,220,085 bytes/792 files/217 modules/RSS 38,043,648 bytes,均在各自门内。
- D-308/ADR-0400(已接受):首个只读 Copilot 的“最近成功/失败运行对比”不再依赖 Prompt 自由拼接两个查询。`qinglong.run.compare@1.0.0` 作为第二个受信内建 Tool,固定 `read/low``run.read``database.read` 和 5 秒 deadline;只有当前 Project Tool snapshot 显式包含 reviewed Definition、产品 composition 显式绑定 adapter 后才可执行。它按 baseline→candidate 串行复用两次有界 Run 点查,absent 与 cross-Project 均为 `found:false`,只输出低敏 Run projection、固定 changed fields 和可证明的 queue/execution/total duration delta;任一时间戳不完整或结束早于开始时不生成对应差值。输出明确标记 `ordered_independent_point_reads`,不冒充数据库事务快照。实现位于既有 `runtime-core/tool-execution/builtin-run-compare/`,只提供两个显式 subpath,不从 package root 导出,不新增 workspace package、依赖、表、migration、连接、timer、listener、watcher、cache 或低配设备常驻开销。最终 18-package clean build/test 退出 0backend 1,208 项为 1,206 pass/2 条件 skip/0 failpackage/dependency/Edge/Cluster deployment 审计零 finding。Edge、Edge AI、Edge MCP 制品为 2,589,812 / 3,121,108 / 7,209,862 bytes,均低于各自上限且未装配 subpath 被发布投影裁掉。
- D-307/ADR-0399(已接受):物理 Edge release archive Gate 使用外部 Ed25519 签发、QingLong verify-only 的两阶段协议。`prepare` exact 重建 owner-private 统一物理报告,要求 direct release service start 已通过,并把 repository、40 位 Git revision、设备/boot、物理报告、release archive、实机 artifact tree/metadata/entrypoint 与 Node digest/version 编入无换行 canonical payload;私钥始终位于 HSM/KMS/离线 operator。`finalize` 以 operator-pinned SPKI 公钥重算 fingerprint,稳定读取并复核所有输入后验证 64-byte detached signature,任一 source/archive/evidence 漂移均失败关闭,输出 `0600` no-replace envelope。通过只把 `release_archive_signature` 替换为 `release_archive_signature_or_attestation``supported:false` 与其余 firmware、整机 flash、migration、断电、固定实机采集和 Cluster 容量 Gate 全部保留。基础 importer 同时把 Edge、SQLite 与 Plugin Package 三个 workload 的 platform/architecture 精确绑定到统一物理观测,release verifier 再要求 observed/Edge/SQLite 为完整 recorder shape 且 Node identity 相同,拒绝跨主机拼接与重算外层摘要后的最小伪造。实现不新增 package、依赖、daemon、listener、timer、watcher 或设备常驻负担;18-package clean build/test 退出 0backend 1,208 项为 1,206 pass/2 条件 skip/0 failpackage/dependency/Edge/service bridge/Cluster deployment 审计零 finding。
@@ -6906,8 +6907,13 @@ Copilot 默认不能直接提交修改或运行命令。写操作通过 Tool Reg
credential 后才可发现,并且每次调用都经过 Tool Policy、durable Security Audit 和 credential
fence confirm;它没有进入默认 Edge/Standalone application,也未向 HTTP、Cluster 或任意模型
授予 ambient capability。该交互式 MCP 读取不冒充内部 Trusted Tool 的 StepRun/encrypted
completion。“最近成功与失败”的有界选择、日志 range/redaction 和最终 Copilot Prompt 仍需
后续产品 Gate。
completion。ADR-0402 已完成“最近成功与失败”的有界选择
`qinglong.task.runs.compare@1.0.0` 只接受 Task ID,在一次固定 64+1 的 Project-scoped
Task Run 窗口中选择最新 succeeded/failed,并复用共享低敏对比算法。协议没有 cursor;
窗口截断且缺少任一 outcome 时必须返回 `complete=false`,禁止模型把局部缺失解释成全历史
不存在。可选本机 MCP 已接入同一认证/Policy/Audit/confirm 链;PostgreSQL 仅提供窄 adapter
Cluster 产品入口和完整 Trusted Tool completion 仍需独立门禁。日志 range/redaction 和最终
Copilot Prompt 仍需后续产品 Gate。
## 17. Tool Registry
@@ -0,0 +1,103 @@
# ADR-0402:有界的 Task 最近成功/失败 Run 对比
- 状态:Accepted
- 日期:2026-08-14
- 关联 RFCQL-RFC-0001 D-310、Phase 2
- 关联 ADRADR-0347、ADR-0400、ADR-0401
## 问题
ADR-0400/0401 已提供按两个显式 Run ID 对比的共享受信 Tool 和可选本机 MCP 入口,但“选择
最近一次成功与失败运行”仍可能被留给模型。模型如果循环调用 `run.list`、翻页并自行过滤,
搜索次数、历史范围和停止条件都不可证明;在低配路由设备上会形成不可控 SQLite I/O,在
Cluster 上也会放大数据库读取和模型上下文,同时给 Prompt 注入制造更大的输入面。
直接把 Task/status 过滤继续塞入通用 Run Reader 也不合理。GitNexus 显示
`LocalSqliteRunReader` 为 CRITICAL(14 个直接、29 个总影响、1 条产品启动流程),
`PostgresRunReader` 为 HIGH(7 个直接、15 个总影响)。为了一个 AI 只读用例扩大这两个基础
Reader 的职责,会让本地恢复、API、Cluster recovery 和写事务继承树承担无关回归风险。
## 决策
1. 增加 `qinglong.task.runs.compare@1.0.0`。输入精确为一个不超过 255 字符的 `taskId`
没有 `limit`、cursor、status、SQL、排序或 Project 参数;Project 只来自受信 composition。
Definition 固定为 `read/low``run.read`、5 秒 deadline。
2. Runtime Core 在 `run/outcome-comparison/` 定义窄
`TaskRunOutcomeWindowReader`。repository 一次只返回按
`(created_at_ms DESC, id DESC)` 排列的最小五字段记录:Run/Project/Task identity、status 和
created time。Tool 固定请求 65 条,其中前 64 条是搜索窗口,第 65 条只证明还有更老记录;
协议永远不返回 cursor,模型不能扩大窗口。
3. 前 64 条中第一个 `succeeded` 是 baseline,第一个 `failed` 是 candidate。两者都找到,
或存储明确返回不超过 64 条时,`selection.complete=true`;如果窗口已满且缺少任一结果,
输出 `complete=false`,不能把“窗口内未找到”伪装成历史上不存在。
4. 选择后按 succeeded → failed 固定顺序执行最多两个既有 Project-scoped Run 点查,并复用
ADR-0400 的低敏 projection、changed fields 和时长差值算法。选择出的 Run 如果消失、变更
Task/outcome、跨 Project 或损坏,整个调用稳定 unavailable;不回退到继续搜索。
5. 输出携带
`consistency=bounded_task_window_then_ordered_point_reads`。它明确表示一次有界选择查询后跟
两次独立点查,不声称数据库事务快照。baseline 永远代表 succeededcandidate 永远代表
failed,差值方向固定为 failed succeeded。
6. SQLite 与 PostgreSQL 各自新增独立、二级目录内的窄 adapter,不修改现有通用 Run Reader
或 Repository 继承树。SQLite 真实 `EXPLAIN QUERY PLAN` 必须命中既有
`ql3_local_runs_task_created_idx`PostgreSQL adapter 使用同样的 Project/Task/order/`LIMIT 65`
查询。复用既有索引避免新增 migration、索引空间和每次 Run 写放大。
7. 可选 `ql3-mcp` 复用既有逐调用 Owner credential → Tool Policy → durable allowed audit →
credential confirm 链,审计理由固定为 `tool_qinglong_task_runs_compare`。默认 Edge/Standalone
仍不装载 MCPCluster 只获得显式 PostgreSQL adapter subpath,不开放 ambient MCP/HTTP
endpoint,内部 Copilot 仍须走完整 Trusted Tool completion 链。
## 低配、集群与 package 布局
- 默认低配设备没有新增进程、连接、timer、listener、watcher、cache、migration 或索引;启用
MCP 时每次调用固定为一个 65 行上限的最小列查询和最多两个串行点查。
- Cluster adapter 复用调用方现有 queryable/Pool,不创建第二连接池;driver 错误收敛为稳定
repository operation error,畸形 bigint/status 收敛为 constraint error。
- package 数保持 18。能力落入 Runtime Core 既有 `run/outcome-comparison/`
`tool-execution/builtin-run-compare/`,持久化实现落入两端既有
`run/outcome-comparison/`;不为一两个文件创建微型 package,也不回到 package `src` 根平铺。
- Runtime Core、Local SQLite、Cluster PostgreSQL 只提供显式 subpathpackage 根入口不扩大;
Local MCP dependency firewall 只放行两个新 Runtime Core 精确 subpath。
## 被否决方案
1. **让模型分页 `run.list`**:查询次数、停止条件和上下文大小不可证明。
2. **把 task/status/cursor 加入通用 Run list**:会把 AI 用例扩散到 HTTP/API 和基础 Reader
且仍允许模型无界翻页。
3. **分别查询所有成功和失败历史**:即使只返回一条,缺少固定窗口时存储扫描和语义仍无界。
4. **为两种 outcome 新增索引**:会增加所有设备的磁盘与 Run 写放大;固定 64 条窗口可复用
已有 Task 时间索引,当前收益不足以支付 migration 和低配常驻成本。
5. **修改 `LocalSqliteRunReader`/`PostgresRunReader`**GitNexus 已证明 HIGH/CRITICAL blast
radius,独立 reader 能用更小权限和故障域完成目标。
6. **只返回两个 Run ID**:仍迫使调用方再次自由组合字段和差值;本 Tool 应交付完整的有界
低敏对比结果。
## 当前验证
1. Runtime Core 定向 10/10:覆盖成功/失败选择顺序、差值、固定 64+1 窗口、无 cursor、
incomplete 语义、跨 Project/乱序/畸形/消失/存储失败、Definition/binding 漂移、显式
subpath 和根入口零导出。
2. Local SQLite 1/1fresh production migration 后写入多 Project/Task/状态 Run,经 MCP 的
单 authority 读取精确窗口,并用真实 `EXPLAIN QUERY PLAN` 证明命中既有 Task 时间索引。
PostgreSQL adapter 2/2:精确 SQL/参数/顺序/上限、bigint 映射、畸形状态和 driver 错误
收敛全部通过。
3. Local MCP 47/47:新增 Tool discovery、固定 read-only annotations、双 permission、durable
audit reason、credential confirm、真实 stdio + fresh SQLite 的选择和两次点查均通过。
4. dependency firewall 53/53,完整 dependency audit 与 package boundary audit 零 finding
package 保持 18 个,`singleSourcePackages=[]``shallowSourcePackages=[]`
5. 最终 18-package clean build/test 退出 0backend 1,208 项为 1,206 pass、2 条平台条件
skip、0 fail。Edge import 与 Cluster deployment 审计零 finding。Runtime Core 为 165/164
nested、Local SQLite 为 197/196 nested、Cluster PostgreSQL 为 168/167 nested,三者 package
根入口均未增长。
6. 默认 Edge artifact 保持 2,589,812 bytes/315 files/56 modules,证明未装配能力被完全裁掉;
RSS 增量 11,173,888 bytes,低于 16 MiB 门。Edge-MCP 为 7,237,187 bytes/795 files/
220 modules/RSS 38,699,008 bytesStandalone-MCP 为 7,237,295 bytes/795 files/
220 modules/RSS 38,600,704 bytes,均低于 16 MiB/1,536 files/48 MiB 门。
7. PostgreSQL 18.4 arm64 HA 125/125 Gate 通过,timeline `1→2`,报告 SHA-256 为
`229c7cac328ee960f667f92868374264a10cb75090ef93d644da1326385d8774`;容器、网络与卷零残留。
## 后续门禁
1. Cluster 产品 composition 装配 Definition/binding,并以真实 PostgreSQL 完成加密结果和
StepRun/Trace completion
2. 日志解释另建 Artifact range、redaction、prompt-injection 和字节预算协议;
3. 固定物理 Edge MCP 单次选择/点查延迟与 RSS,并验证撤权竞态。
+1
View File
@@ -405,6 +405,7 @@
| [ADR-0399](./ADR-0399-external-release-attestation-for-physical-edge-evidence.md) | 物理 Edge 证据的外部发布归档证明 | Acceptedexact recorder 重建与离线验签已实现;固定实机 ceremony 待执行) |
| [ADR-0400](./ADR-0400-bounded-trusted-run-comparison-tool.md) | 有界、受信的 Run 对比 Tool | Accepted |
| [ADR-0401](./ADR-0401-optional-local-mcp-run-comparison-surface.md) | 可选本机 MCP Run 对比产品入口 | Accepted |
| [ADR-0402](./ADR-0402-bounded-latest-task-run-outcome-comparison.md) | 有界的 Task 最近成功/失败 Run 对比 | Accepted |
## 规则
@@ -75,6 +75,11 @@
"require": "./dist/run-management/runManualRetryRepository.js",
"default": "./dist/run-management/runManualRetryRepository.js"
},
"./task-run-outcome-window": {
"types": "./dist/run/outcome-comparison/taskRunOutcomeWindowReader.d.ts",
"require": "./dist/run/outcome-comparison/taskRunOutcomeWindowReader.js",
"default": "./dist/run/outcome-comparison/taskRunOutcomeWindowReader.js"
},
"./run-manager": {
"types": "./dist/entrypoints/runManager.d.ts",
"require": "./dist/entrypoints/runManager.js",
@@ -0,0 +1,84 @@
import { RUN_STATUSES, type RunStatus } from '@qinglong/runtime-core/run';
import {
RunRepositoryConstraintError,
RunRepositoryOperationError,
} from '@qinglong/runtime-core/run-repository';
import {
normalizeTaskRunOutcomeWindowQuery,
normalizeTaskRunOutcomeWindowRecord,
type TaskRunOutcomeWindowQuery,
type TaskRunOutcomeWindowReader,
type TaskRunOutcomeWindowRecord,
} from '@qinglong/runtime-core/task-run-outcome-window';
import type { PostgresQueryable } from '@qinglong/runtime-core';
type QueryRow = Readonly<Record<string, unknown>>;
function requiredString(row: QueryRow, property: string): string {
const value = row[property];
if (typeof value !== 'string' || value.length === 0) {
throw new RunRepositoryConstraintError(
`PostgreSQL Task Run outcome row has an invalid ${property}`,
);
}
return value;
}
function requiredInteger(row: QueryRow, property: string): number {
const value = row[property];
if (typeof value === 'number' && Number.isSafeInteger(value)) return value;
if (typeof value === 'string' && /^(0|[1-9]\d*)$/.test(value)) {
const parsed = Number(value);
if (Number.isSafeInteger(parsed)) return parsed;
}
throw new RunRepositoryConstraintError(
`PostgreSQL Task Run outcome row has an invalid ${property}`,
);
}
function record(row: QueryRow): Readonly<TaskRunOutcomeWindowRecord> {
const status = requiredString(row, 'status');
if (!RUN_STATUSES.includes(status as RunStatus)) {
throw new RunRepositoryConstraintError(
'PostgreSQL Task Run outcome row has an invalid status',
);
}
return normalizeTaskRunOutcomeWindowRecord({
id: requiredString(row, 'id'),
projectId: requiredString(row, 'projectId'),
taskId: requiredString(row, 'taskId'),
status: status as RunStatus,
createdAtMs: requiredInteger(row, 'createdAtMs'),
});
}
export class PostgresTaskRunOutcomeWindowReader
implements TaskRunOutcomeWindowReader
{
constructor(private readonly queryable: PostgresQueryable) {}
async listRecentRunsByTask(
value: Readonly<TaskRunOutcomeWindowQuery>,
): Promise<readonly Readonly<TaskRunOutcomeWindowRecord>[]> {
const query = normalizeTaskRunOutcomeWindowQuery(value);
try {
const result = await this.queryable.query<QueryRow>(
`SELECT
"id" AS "id",
"project_id" AS "projectId",
"task_id" AS "taskId",
"status" AS "status",
"created_at_ms" AS "createdAtMs"
FROM "ql3"."runs"
WHERE "project_id" = $1 AND "task_id" = $2
ORDER BY "created_at_ms" DESC, "id" DESC
LIMIT $3`,
[query.projectId, query.taskId, query.limit],
);
return Object.freeze(result.rows.map(record));
} catch (error) {
if (error instanceof RunRepositoryConstraintError) throw error;
throw new RunRepositoryOperationError(error);
}
}
}
@@ -0,0 +1,114 @@
const assert = require('node:assert/strict');
const { test } = require('node:test');
const {
RunRepositoryConstraintError,
RunRepositoryOperationError,
} = require('@qinglong/runtime-core/run-repository');
const {
PostgresTaskRunOutcomeWindowReader,
} = require('@qinglong/cluster-postgres/task-run-outcome-window');
function harness(rows) {
const queries = [];
return {
queries,
queryable: {
async query(text, values) {
queries.push({ text, values });
return { rows, rowCount: rows.length };
},
},
};
}
test('reads one bounded PostgreSQL Project Task window with bigint normalization', async () => {
assert.equal(
require('@qinglong/cluster-postgres').PostgresTaskRunOutcomeWindowReader,
undefined,
);
const value = harness([
{
id: 'run-failed',
projectId: 'project-a',
taskId: 'task-a',
status: 'failed',
createdAtMs: '2000',
},
{
id: 'run-succeeded',
projectId: 'project-a',
taskId: 'task-a',
status: 'succeeded',
createdAtMs: '1000',
},
]);
const rows = await new PostgresTaskRunOutcomeWindowReader(
value.queryable,
).listRecentRunsByTask({
projectId: 'project-a',
taskId: 'task-a',
limit: 65,
});
assert.deepEqual(rows, [
{
id: 'run-failed',
projectId: 'project-a',
taskId: 'task-a',
status: 'failed',
createdAtMs: 2000,
},
{
id: 'run-succeeded',
projectId: 'project-a',
taskId: 'task-a',
status: 'succeeded',
createdAtMs: 1000,
},
]);
assert.deepEqual(value.queries[0].values, ['project-a', 'task-a', 65]);
assert.match(
value.queries[0].text,
/WHERE "project_id" = \$1 AND "task_id" = \$2[\s\S]*ORDER BY "created_at_ms" DESC, "id" DESC[\s\S]*LIMIT \$3/u,
);
});
test('rejects invalid windows and maps driver failures without leaking details', async () => {
const invalid = harness([
{
id: 'run-invalid',
projectId: 'project-a',
taskId: 'task-a',
status: 'invented',
createdAtMs: '1000',
},
]);
await assert.rejects(
new PostgresTaskRunOutcomeWindowReader(
invalid.queryable,
).listRecentRunsByTask({
projectId: 'project-a',
taskId: 'task-a',
limit: 65,
}),
RunRepositoryConstraintError,
);
await assert.rejects(
new PostgresTaskRunOutcomeWindowReader({
async query() {
throw new Error('postgresql://private-host/secret');
},
}).listRecentRunsByTask({
projectId: 'project-a',
taskId: 'task-a',
limit: 65,
}),
(error) => {
assert.ok(error instanceof RunRepositoryOperationError);
assert.equal(error.message, 'Run repository operation failed');
return true;
},
);
});
@@ -50,6 +50,11 @@ import {
BUILTIN_RUN_COMPARE_TOOL_DEFINITION,
executeBuiltInRunCompareTool,
} from '@qinglong/runtime-core/builtin-run-compare-projection';
import {
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL,
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION,
executeBuiltInTaskRunOutcomeCompareTool,
} from '@qinglong/runtime-core/builtin-task-run-outcome-compare-projection';
import {
BUILTIN_RUN_READ_TOOL,
BUILTIN_RUN_READ_TOOL_DEFINITION,
@@ -58,6 +63,7 @@ import {
import type { RunRepositoryReader } from '@qinglong/runtime-core/run-repository';
import type { StepRunRepository } from '@qinglong/runtime-core/step-run';
import type { ProjectRunListReader } from '@qinglong/runtime-core/project-run-list';
import type { TaskRunOutcomeWindowReader } from '@qinglong/runtime-core/task-run-outcome-window';
import type { SecurityPrincipal } from '@qinglong/runtime-core/security';
import type { TaskDefinitionSource } from '@qinglong/runtime-core/task-definition';
import type { TriggerSource } from '@qinglong/runtime-core/trigger';
@@ -112,7 +118,8 @@ type LocalMcpRunReader = Pick<
RunRepositoryReader,
'findRunById' | 'listEvents'
> &
ProjectRunListReader;
ProjectRunListReader &
TaskRunOutcomeWindowReader;
type LocalMcpTaskReader = Pick<
TaskDefinitionSource,
@@ -194,6 +201,24 @@ const LOCAL_MCP_READ_TOOLS: readonly LocalMcpReadToolDescriptor[] =
input: ToolJsonValue,
) => executeBuiltInRunCompareTool(authority.runs, projectId, input),
}),
Object.freeze({
tool: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL,
definition: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION,
title: 'Compare Latest QingLong Task Run Outcomes',
auditReason: 'tool_qinglong_task_runs_compare',
unavailableCode: 'task_run_outcome_compare_unavailable',
execute: (
authority: LocalMcpReadAuthority,
projectId: string,
input: ToolJsonValue,
) =>
executeBuiltInTaskRunOutcomeCompareTool(
authority.runs,
authority.runs,
projectId,
input,
),
}),
Object.freeze({
tool: BUILTIN_RUN_EVENT_LIST_TOOL,
definition: BUILTIN_RUN_EVENT_LIST_TOOL_DEFINITION,
@@ -165,6 +165,7 @@ function fixture(options = {}) {
const audits = [];
let reads = 0;
let listReads = 0;
let outcomeWindowReads = 0;
let eventReads = 0;
let taskListReads = 0;
let triggerListReads = 0;
@@ -181,6 +182,15 @@ function fixture(options = {}) {
startedAtMs: 24,
finishedAtMs: 30,
});
const failedRun = Object.freeze({
...candidateRun,
id: 'run-failed',
status: 'failed',
createdAtMs: 30,
queuedAtMs: 31,
startedAtMs: 34,
finishedAtMs: 40,
});
const server = createQingLongLocalMcpServer({
projectId: 'default',
now: () => NOW,
@@ -222,11 +232,37 @@ function fixture(options = {}) {
const values = [candidateRun, run()];
return values.slice(0, query.limit);
},
async listRecentRunsByTask(query) {
events.push('read-outcome-window');
outcomeWindowReads += 1;
assert.deepEqual(query, {
projectId: 'default',
taskId: 'task-1',
limit: 65,
});
return [
{
id: failedRun.id,
projectId: failedRun.projectId,
taskId: failedRun.taskId,
status: failedRun.status,
createdAtMs: failedRun.createdAtMs,
},
{
id: 'run-1',
projectId: 'default',
taskId: 'task-1',
status: 'succeeded',
createdAtMs: 10,
},
];
},
async findRunById(runId) {
events.push('read');
reads += 1;
if (runId === 'run-1') return run(options.runProjectId);
return runId === 'run-2' ? candidateRun : null;
if (runId === 'run-2') return candidateRun;
return runId === 'run-failed' ? failedRun : null;
},
async listEvents(runId, query) {
events.push('read-events');
@@ -328,6 +364,7 @@ function fixture(options = {}) {
counters: () => ({
reads,
listReads,
outcomeWindowReads,
eventReads,
taskListReads,
triggerListReads,
@@ -393,6 +430,7 @@ test('advertises bounded read-only Run Tools and executes auth -> Policy -> Audi
'qinglong.run.list',
'qinglong.run.get',
'qinglong.run.compare',
'qinglong.task.runs.compare',
'qinglong.run.events.list',
'qinglong.run.steps.list',
'qinglong.task.get',
@@ -451,6 +489,7 @@ test('advertises bounded read-only Run Tools and executes auth -> Policy -> Audi
assert.deepEqual(value.counters(), {
reads: 1,
listReads: 0,
outcomeWindowReads: 0,
eventReads: 0,
taskListReads: 0,
triggerListReads: 0,
@@ -533,6 +572,98 @@ test('compares two Project Runs through the same fenced admission', async (t) =>
assert.deepEqual(value.counters(), {
reads: 2,
listReads: 0,
outcomeWindowReads: 0,
eventReads: 0,
taskListReads: 0,
triggerListReads: 0,
approvalListReads: 0,
approvalDetailReads: 0,
confirmations: 1,
});
});
test('selects and compares latest Task outcomes through the same fenced admission', async (t) => {
const value = fixture();
const connected = await client(value.server, t);
const response = await connected.request('tools/call', {
name: 'qinglong.task.runs.compare',
arguments: { taskId: 'task-1' },
});
assert.equal(response.result.isError, undefined);
assert.deepEqual(response.result.structuredContent, {
taskId: 'task-1',
baselineOutcome: 'succeeded',
candidateOutcome: 'failed',
baseline: {
found: true,
id: 'run-1',
taskId: 'task-1',
taskRevision: 'revision-1',
status: 'succeeded',
version: 3,
eventSequence: 4,
priority: 0,
executionOrigin: 'manual',
executionOwner: 'runtime',
createdAtMs: 10,
queuedAtMs: 11,
startedAtMs: 12,
finishedAtMs: 13,
},
candidate: {
found: true,
id: 'run-failed',
taskId: 'task-1',
taskRevision: 'revision-2',
status: 'failed',
version: 3,
eventSequence: 4,
priority: 1,
executionOrigin: 'manual',
executionOwner: 'runtime',
createdAtMs: 30,
queuedAtMs: 31,
startedAtMs: 34,
finishedAtMs: 40,
},
comparable: true,
sameTask: true,
sameTaskRevision: false,
changedFields: ['taskRevision', 'status', 'priority'],
queueDelayDeltaMs: 0,
executionDurationDeltaMs: 5,
totalDurationDeltaMs: 7,
consistency: 'bounded_task_window_then_ordered_point_reads',
selection: {
windowLimit: 64,
searchedRunCount: 2,
hasOlderRuns: false,
complete: true,
order: 'created_at_desc_id_desc',
},
});
assert.deepEqual(value.permissions, [
'tool.call:qinglong.task.runs.compare',
'run.read',
]);
assert.deepEqual(value.events, [
'authenticate',
'policy:tool.call:qinglong.task.runs.compare',
'policy:run.read',
'audit:allowed',
'confirm',
'read-outcome-window',
'read',
'read',
]);
assert.deepEqual(value.audits[0].reasons, [
'tool_invocation_allowed',
'tool_qinglong_task_runs_compare',
]);
assert.deepEqual(value.counters(), {
reads: 2,
listReads: 0,
outcomeWindowReads: 1,
eventReads: 0,
taskListReads: 0,
triggerListReads: 0,
@@ -590,6 +721,7 @@ test('discovers recent Project Runs through the same fenced admission', async (t
assert.deepEqual(value.counters(), {
reads: 0,
listReads: 1,
outcomeWindowReads: 0,
eventReads: 0,
taskListReads: 0,
triggerListReads: 0,
@@ -642,6 +774,7 @@ test('lists a payload-free Run event page through the same fenced admission', as
assert.deepEqual(value.counters(), {
reads: 1,
listReads: 0,
outcomeWindowReads: 0,
eventReads: 1,
taskListReads: 0,
triggerListReads: 0,
@@ -694,6 +827,7 @@ test('discovers low-sensitive Tasks through task.read admission', async (t) => {
assert.deepEqual(value.counters(), {
reads: 0,
listReads: 0,
outcomeWindowReads: 0,
eventReads: 0,
taskListReads: 1,
triggerListReads: 0,
@@ -743,6 +877,7 @@ test('reads one current Task fence through task.read admission', async (t) => {
assert.deepEqual(value.counters(), {
reads: 0,
listReads: 0,
outcomeWindowReads: 0,
eventReads: 0,
taskListReads: 1,
triggerListReads: 0,
@@ -803,6 +938,7 @@ test('discovers low-sensitive Triggers through trigger.read admission', async (t
assert.deepEqual(value.counters(), {
reads: 0,
listReads: 0,
outcomeWindowReads: 0,
eventReads: 0,
taskListReads: 0,
triggerListReads: 1,
@@ -859,6 +995,7 @@ test('discovers low-sensitive Approvals through approval.read admission', async
assert.deepEqual(value.counters(), {
reads: 0,
listReads: 0,
outcomeWindowReads: 0,
eventReads: 0,
taskListReads: 0,
triggerListReads: 0,
@@ -930,6 +1067,7 @@ test('reads one redacted Approval preview through approval.read and artifact.rea
assert.deepEqual(value.counters(), {
reads: 0,
listReads: 0,
outcomeWindowReads: 0,
eventReads: 0,
taskListReads: 0,
triggerListReads: 0,
@@ -90,6 +90,44 @@ async function fixture(t) {
priority: 1,
createdAtMs: NOW - 3_000,
});
await transaction.insertRun({
id: 'run-mcp-e2e-success',
projectId: 'default',
taskId: 'task-mcp',
taskRevision: 'revision-3',
taskName: 'MCP successful baseline',
triggerType: 'manual',
executionOrigin: 'manual',
executionOwner: 'runtime',
triggeredBy: 'user:mcp-owner',
status: 'succeeded',
version: 0,
eventSequence: 0,
priority: 0,
createdAtMs: NOW - 4_000,
queuedAtMs: NOW - 3_900,
startedAtMs: NOW - 3_800,
finishedAtMs: NOW - 3_600,
});
await transaction.insertRun({
id: 'run-mcp-e2e-failure',
projectId: 'default',
taskId: 'task-mcp',
taskRevision: 'revision-4',
taskName: 'MCP failed candidate',
triggerType: 'manual',
executionOrigin: 'manual',
executionOwner: 'runtime',
triggeredBy: 'user:mcp-owner',
status: 'failed',
version: 0,
eventSequence: 0,
priority: 2,
createdAtMs: NOW - 5_000,
queuedAtMs: NOW - 4_850,
startedAtMs: NOW - 4_700,
finishedAtMs: NOW - 4_300,
});
await transaction.appendEvent({
id: 'mcp-e2e-event-1',
runId: 'run-mcp-e2e',
@@ -473,6 +511,7 @@ test('serves the authenticated Run Tool over the real stdio protocol and persist
'qinglong.run.list',
'qinglong.run.get',
'qinglong.run.compare',
'qinglong.task.runs.compare',
'qinglong.run.events.list',
'qinglong.run.steps.list',
'qinglong.task.get',
@@ -684,6 +723,67 @@ test('serves the authenticated Run Tool over the real stdio protocol and persist
changedFields: ['taskRevision', 'priority'],
consistency: 'ordered_independent_point_reads',
});
const comparedOutcomes = await request('tools/call', {
name: 'qinglong.task.runs.compare',
arguments: { taskId: 'task-mcp' },
});
assert.equal(
comparedOutcomes.result.isError,
undefined,
JSON.stringify(comparedOutcomes),
);
assert.deepEqual(comparedOutcomes.result.structuredContent, {
taskId: 'task-mcp',
baselineOutcome: 'succeeded',
candidateOutcome: 'failed',
baseline: {
found: true,
id: 'run-mcp-e2e-success',
taskId: 'task-mcp',
taskRevision: 'revision-3',
status: 'succeeded',
version: 0,
eventSequence: 0,
priority: 0,
executionOrigin: 'manual',
executionOwner: 'runtime',
createdAtMs: NOW - 4_000,
queuedAtMs: NOW - 3_900,
startedAtMs: NOW - 3_800,
finishedAtMs: NOW - 3_600,
},
candidate: {
found: true,
id: 'run-mcp-e2e-failure',
taskId: 'task-mcp',
taskRevision: 'revision-4',
status: 'failed',
version: 0,
eventSequence: 0,
priority: 2,
executionOrigin: 'manual',
executionOwner: 'runtime',
createdAtMs: NOW - 5_000,
queuedAtMs: NOW - 4_850,
startedAtMs: NOW - 4_700,
finishedAtMs: NOW - 4_300,
},
comparable: true,
sameTask: true,
sameTaskRevision: false,
changedFields: ['taskRevision', 'status', 'priority'],
queueDelayDeltaMs: 50,
executionDurationDeltaMs: 200,
totalDurationDeltaMs: 300,
consistency: 'bounded_task_window_then_ordered_point_reads',
selection: {
windowLimit: 64,
searchedRunCount: 4,
hasOlderRuns: false,
complete: true,
order: 'created_at_desc_id_desc',
},
});
const events = await request('tools/call', {
name: 'qinglong.run.events.list',
arguments: { runId: 'run-mcp-e2e', limit: 1 },
@@ -765,6 +865,11 @@ test('serves the authenticated Run Tool over the real stdio protocol and persist
outcome: 'allowed',
subjectId: 'mcp-user',
},
{
operationId: 'mcp.tool.call',
outcome: 'allowed',
subjectId: 'mcp-user',
},
],
);
} finally {
+5
View File
@@ -205,6 +205,11 @@
"require": "./dist/tool-execution/localMcpReadDatabase.js",
"default": "./dist/tool-execution/localMcpReadDatabase.js"
},
"./task-run-outcome-window": {
"types": "./dist/run/outcome-comparison/taskRunOutcomeWindowReader.d.ts",
"require": "./dist/run/outcome-comparison/taskRunOutcomeWindowReader.js",
"default": "./dist/run/outcome-comparison/taskRunOutcomeWindowReader.js"
},
"./approved-action": {
"types": "./dist/approved-action/approvalRequestRepository.d.ts",
"require": "./dist/approved-action/approvalRequestRepository.js",
@@ -0,0 +1,49 @@
import { RUN_STATUSES } from '@qinglong/runtime-core/run';
import {
normalizeTaskRunOutcomeWindowQuery,
type TaskRunOutcomeWindowQuery,
type TaskRunOutcomeWindowReader,
type TaskRunOutcomeWindowRecord,
} from '@qinglong/runtime-core/task-run-outcome-window';
import type { DatabaseSync } from 'node:sqlite';
import {
queryRows,
requiredEnum,
requiredInteger,
requiredString,
} from '../runPersistence';
export class LocalSqliteTaskRunOutcomeWindowReader
implements TaskRunOutcomeWindowReader
{
constructor(private readonly client: DatabaseSync) {}
async listRecentRunsByTask(
value: Readonly<TaskRunOutcomeWindowQuery>,
): Promise<readonly Readonly<TaskRunOutcomeWindowRecord>[]> {
const query = normalizeTaskRunOutcomeWindowQuery(value);
return queryRows(
this.client,
`SELECT
"id" AS "id",
"project_id" AS "projectId",
"task_id" AS "taskId",
"status" AS "status",
"created_at_ms" AS "createdAtMs"
FROM "Runs"
WHERE "project_id" = ? AND "task_id" = ?
ORDER BY "created_at_ms" DESC, "id" DESC
LIMIT ?`,
[query.projectId, query.taskId, query.limit],
).map((row) =>
Object.freeze({
id: requiredString(row, 'id'),
projectId: requiredString(row, 'projectId'),
taskId: requiredString(row, 'taskId'),
status: requiredEnum(row, 'status', RUN_STATUSES),
createdAtMs: requiredInteger(row, 'createdAtMs'),
}),
);
}
}
@@ -5,6 +5,10 @@ import type {
ProjectRunListQuery,
ProjectRunListReader,
} from '@qinglong/runtime-core/project-run-list';
import type {
TaskRunOutcomeWindowQuery,
TaskRunOutcomeWindowReader,
} from '@qinglong/runtime-core/task-run-outcome-window';
import {
RunRepositoryBusyError,
RunRepositoryOperationError,
@@ -26,6 +30,7 @@ import {
type LocalSqliteReadinessEvidence,
} from '../readiness/readiness';
import { LocalSqliteRunReader } from '../run/runReader';
import { LocalSqliteTaskRunOutcomeWindowReader } from '../run/outcome-comparison/taskRunOutcomeWindowReader';
import { LocalSqliteStepRunRepository } from '../run/stepRunRepository';
import {
assertLocalSqliteOptions,
@@ -44,7 +49,8 @@ export interface LocalSqliteMcpReadDatabase {
readonly profile: LocalSqliteProfile;
readonly readiness: LocalSqliteReadinessEvidence;
readonly runs: Pick<RunRepositoryReader, 'findRunById' | 'listEvents'> &
ProjectRunListReader;
ProjectRunListReader &
TaskRunOutcomeWindowReader;
readonly stepRuns: Pick<StepRunRepository, 'listByRun'>;
readonly taskDefinitions: Pick<
TaskDefinitionSource,
@@ -76,13 +82,17 @@ export async function openLocalSqliteMcpReadDatabase(
const readiness = await auditLocalSqliteReadiness(client);
const authority = new LocalSqliteOperationAuthority(client);
const reader = new LocalSqliteRunReader(client);
const outcomeWindowReader = new LocalSqliteTaskRunOutcomeWindowReader(
client,
);
const stepRunRepository = new LocalSqliteStepRunRepository(authority);
const taskRepository = new LocalSqliteTaskDefinitionRepository(authority);
const triggerRepository = new LocalSqliteTriggerRepository(authority);
const approvalSource = new LocalSqliteApprovalRequestSource(authority);
const security = new LocalSqliteSecurityAuthorityStore(authority);
const runs: Pick<RunRepositoryReader, 'findRunById' | 'listEvents'> &
ProjectRunListReader = Object.freeze({
ProjectRunListReader &
TaskRunOutcomeWindowReader = Object.freeze({
listRunsByProject(query: Readonly<ProjectRunListQuery>) {
return authority.enqueue(
() => reader.listRunsByProject(query),
@@ -94,6 +104,17 @@ export async function openLocalSqliteMcpReadDatabase(
),
);
},
listRecentRunsByTask(query: Readonly<TaskRunOutcomeWindowQuery>) {
return authority.enqueue(
() => outcomeWindowReader.listRecentRunsByTask(query),
(reason) =>
reason === 'busy'
? new RunRepositoryBusyError()
: new RunRepositoryOperationError(
new Error('Local SQLite MCP read database is closed'),
),
);
},
findRunById(runId: string) {
return authority.enqueue(
() => reader.findRunById(runId),
@@ -0,0 +1,125 @@
const assert = require('node:assert/strict');
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const { DatabaseSync } = require('node:sqlite');
const { test } = require('node:test');
const {
migrateLocalSqlitePath,
openLocalSqliteRuntimeDatabase,
} = require('../dist');
const {
openLocalSqliteMcpReadDatabase,
} = require('@qinglong/local-sqlite/mcp-read-database');
const {
LocalSqliteTaskRunOutcomeWindowReader,
} = require('@qinglong/local-sqlite/task-run-outcome-window');
function run(id, projectId, taskId, status, createdAtMs) {
return {
id,
projectId,
taskId,
taskRevision: 'revision-1',
taskName: id,
triggerType: 'manual',
executionOrigin: 'manual',
executionOwner: 'runtime',
triggeredBy: 'test',
status,
version: 0,
eventSequence: 0,
priority: 0,
createdAtMs,
};
}
test('reads one fixed Project Task outcome window through the existing indexed SQLite authority', async (t) => {
assert.equal(typeof LocalSqliteTaskRunOutcomeWindowReader, 'function');
assert.equal(
require('@qinglong/local-sqlite').LocalSqliteTaskRunOutcomeWindowReader,
undefined,
);
const directory = fs.mkdtempSync(
path.join(os.tmpdir(), 'ql3-outcome-window-'),
);
t.after(() => fs.rmSync(directory, { recursive: true, force: true }));
const options = {
databasePath: path.join(directory, 'qinglong3.sqlite'),
profile: 'edge',
};
await migrateLocalSqlitePath(options);
const runtime = await openLocalSqliteRuntimeDatabase(options);
await runtime.runRepository.transaction(async (transaction) => {
for (const value of [
run('run-old-success', 'default', 'task-a', 'succeeded', 10),
run('run-failure', 'default', 'task-a', 'failed', 20),
run('run-running', 'default', 'task-a', 'running', 30),
run('run-other-task', 'default', 'task-b', 'failed', 40),
run('run-other-project', 'other', 'task-a', 'failed', 50),
]) {
await transaction.insertRun(value);
}
});
await runtime.close();
const database = await openLocalSqliteMcpReadDatabase(options);
t.after(() => database.close());
const rows = await database.runs.listRecentRunsByTask({
projectId: 'default',
taskId: 'task-a',
limit: 3,
});
assert.deepEqual(rows, [
{
id: 'run-running',
projectId: 'default',
taskId: 'task-a',
status: 'running',
createdAtMs: 30,
},
{
id: 'run-failure',
projectId: 'default',
taskId: 'task-a',
status: 'failed',
createdAtMs: 20,
},
{
id: 'run-old-success',
projectId: 'default',
taskId: 'task-a',
status: 'succeeded',
createdAtMs: 10,
},
]);
await assert.rejects(
database.runs.listRecentRunsByTask({
projectId: 'default',
taskId: 'task-a',
limit: 66,
}),
TypeError,
);
await database.close();
const client = new DatabaseSync(options.databasePath, { readOnly: true });
t.after(() => {
if (client.isOpen) client.close();
});
const plan = client
.prepare(
`EXPLAIN QUERY PLAN
SELECT "id", "project_id", "task_id", "status", "created_at_ms"
FROM "Runs"
WHERE "project_id" = ? AND "task_id" = ?
ORDER BY "created_at_ms" DESC, "id" DESC
LIMIT ?`,
)
.all('default', 'task-a', 65);
assert.match(
plan.map((entry) => entry.detail).join('\n'),
/ql3_local_runs_task_created_idx/u,
);
});
+24
View File
@@ -242,6 +242,12 @@
"builtin-run-compare-projection": [
"dist/tool-execution/builtin-run-compare/builtInRunCompareProjection.d.ts"
],
"builtin-task-run-outcome-compare-tool": [
"dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareTool.d.ts"
],
"builtin-task-run-outcome-compare-projection": [
"dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareProjection.d.ts"
],
"run": [
"dist/run/run.d.ts"
],
@@ -254,6 +260,9 @@
"project-run-list": [
"dist/run/projectRunList.d.ts"
],
"task-run-outcome-window": [
"dist/run/outcome-comparison/taskRunOutcomeWindow.d.ts"
],
"bounded-run-read-projection": [
"dist/run/projection/boundedRunReadProjection.d.ts"
],
@@ -318,6 +327,11 @@
"require": "./dist/run/projectRunList.js",
"default": "./dist/run/projectRunList.js"
},
"./task-run-outcome-window": {
"types": "./dist/run/outcome-comparison/taskRunOutcomeWindow.d.ts",
"require": "./dist/run/outcome-comparison/taskRunOutcomeWindow.js",
"default": "./dist/run/outcome-comparison/taskRunOutcomeWindow.js"
},
"./bounded-run-read-projection": {
"types": "./dist/run/projection/boundedRunReadProjection.d.ts",
"require": "./dist/run/projection/boundedRunReadProjection.js",
@@ -668,6 +682,16 @@
"require": "./dist/tool-execution/builtin-run-compare/builtInRunCompareProjection.js",
"default": "./dist/tool-execution/builtin-run-compare/builtInRunCompareProjection.js"
},
"./builtin-task-run-outcome-compare-tool": {
"types": "./dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareTool.d.ts",
"require": "./dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareTool.js",
"default": "./dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareTool.js"
},
"./builtin-task-run-outcome-compare-projection": {
"types": "./dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareProjection.d.ts",
"require": "./dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareProjection.js",
"default": "./dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareProjection.js"
},
"./secret-reference": {
"types": "./dist/secret/secretReference.d.ts",
"require": "./dist/secret/secretReference.js",
@@ -0,0 +1,91 @@
import { RUN_STATUSES, type RunStatus } from '../run';
export const MAX_TASK_RUN_OUTCOME_WINDOW_STORAGE_LIMIT = 65;
const CONTROL_PATTERN = /[\u0000-\u001f\u007f]/;
export interface TaskRunOutcomeWindowQuery {
readonly projectId: string;
readonly taskId: string;
readonly limit: number;
}
export interface TaskRunOutcomeWindowRecord {
readonly id: string;
readonly projectId: string;
readonly taskId: string;
readonly status: RunStatus;
readonly createdAtMs: number;
}
export interface TaskRunOutcomeWindowReader {
listRecentRunsByTask(
query: Readonly<TaskRunOutcomeWindowQuery>,
): Promise<readonly Readonly<TaskRunOutcomeWindowRecord>[]>;
}
function boundedText(value: unknown, maximum: number): value is string {
return (
typeof value === 'string' &&
value.length > 0 &&
value.length <= maximum &&
!CONTROL_PATTERN.test(value)
);
}
export function normalizeTaskRunOutcomeWindowQuery(
value: Readonly<TaskRunOutcomeWindowQuery>,
): Readonly<TaskRunOutcomeWindowQuery> {
if (
!value ||
typeof value !== 'object' ||
Array.isArray(value) ||
Reflect.ownKeys(value).length !== 3 ||
!Object.hasOwn(value, 'projectId') ||
!Object.hasOwn(value, 'taskId') ||
!Object.hasOwn(value, 'limit') ||
!boundedText(value.projectId, 128) ||
!boundedText(value.taskId, 255) ||
!Number.isSafeInteger(value.limit) ||
value.limit < 1 ||
value.limit > MAX_TASK_RUN_OUTCOME_WINDOW_STORAGE_LIMIT
) {
throw new TypeError('Task Run outcome window query is invalid');
}
return Object.freeze({
projectId: value.projectId,
taskId: value.taskId,
limit: value.limit,
});
}
export function normalizeTaskRunOutcomeWindowRecord(
value: Readonly<TaskRunOutcomeWindowRecord>,
): Readonly<TaskRunOutcomeWindowRecord> {
if (
!value ||
typeof value !== 'object' ||
Array.isArray(value) ||
Reflect.ownKeys(value).length !== 5 ||
!Object.hasOwn(value, 'id') ||
!Object.hasOwn(value, 'projectId') ||
!Object.hasOwn(value, 'taskId') ||
!Object.hasOwn(value, 'status') ||
!Object.hasOwn(value, 'createdAtMs') ||
!boundedText(value.id, 128) ||
!boundedText(value.projectId, 128) ||
!boundedText(value.taskId, 255) ||
!RUN_STATUSES.includes(value.status) ||
!Number.isSafeInteger(value.createdAtMs) ||
value.createdAtMs < 0
) {
throw new TypeError('Task Run outcome window record is invalid');
}
return Object.freeze({
id: value.id,
projectId: value.projectId,
taskId: value.taskId,
status: value.status,
createdAtMs: value.createdAtMs,
});
}
@@ -19,7 +19,7 @@ export const BUILTIN_RUN_COMPARE_TIMEOUT_SECONDS = 5;
const MAX_INT = 2_147_483_647;
const MIN_INT = -2_147_483_648;
const CONTROL_PATTERN = /[\u0000-\u001f\u007f]/;
const COMPARABLE_FIELDS = Object.freeze([
export const BUILTIN_RUN_COMPARABLE_FIELDS = Object.freeze([
'taskId',
'taskRevision',
'status',
@@ -28,7 +28,7 @@ const COMPARABLE_FIELDS = Object.freeze([
'executionOwner',
] as const);
const RUN_PROJECTION_SCHEMA = Object.freeze({
export const BUILTIN_RUN_PROJECTION_SCHEMA = Object.freeze({
type: 'object' as const,
properties: {
found: { type: 'boolean' as const },
@@ -103,8 +103,8 @@ export const BUILTIN_RUN_COMPARE_TOOL_DEFINITION = normalizeToolDefinition({
outputSchema: {
type: 'object',
properties: {
baseline: RUN_PROJECTION_SCHEMA,
candidate: RUN_PROJECTION_SCHEMA,
baseline: BUILTIN_RUN_PROJECTION_SCHEMA,
candidate: BUILTIN_RUN_PROJECTION_SCHEMA,
comparable: { type: 'boolean' },
sameTask: { type: 'boolean' },
sameTaskRevision: { type: 'boolean' },
@@ -113,9 +113,9 @@ export const BUILTIN_RUN_COMPARE_TOOL_DEFINITION = normalizeToolDefinition({
items: {
type: 'string',
maxLength: 32,
enum: COMPARABLE_FIELDS,
enum: BUILTIN_RUN_COMPARABLE_FIELDS,
},
maxItems: COMPARABLE_FIELDS.length,
maxItems: BUILTIN_RUN_COMPARABLE_FIELDS.length,
},
queueDelayDeltaMs: {
type: 'integer',
@@ -212,14 +212,21 @@ function timestampDelta(
return Number.isSafeInteger(delta) ? delta : undefined;
}
function compareProjections(
export type BuiltInRunComparisonConsistency =
| 'ordered_independent_point_reads'
| 'bounded_task_window_then_ordered_point_reads';
export function compareBoundedRunProjections(
baseline: BoundedRunReadProjection,
candidate: BoundedRunReadProjection,
consistency: BuiltInRunComparisonConsistency,
): Readonly<Record<string, ToolJsonValue>> {
const comparable = baseline.found === true && candidate.found === true;
const changedFields = Object.freeze(
comparable
? COMPARABLE_FIELDS.filter((field) => baseline[field] !== candidate[field])
? BUILTIN_RUN_COMPARABLE_FIELDS.filter(
(field) => baseline[field] !== candidate[field],
)
: [],
);
const queueDelayDeltaMs = comparable
@@ -259,7 +266,7 @@ function compareProjections(
? {}
: { executionDurationDeltaMs }),
...(totalDurationDeltaMs === undefined ? {} : { totalDurationDeltaMs }),
consistency: 'ordered_independent_point_reads',
consistency,
});
}
@@ -295,7 +302,11 @@ export async function executeBuiltInRunCompareTool(
projectId,
inputRecord.candidateRunId,
);
return compareProjections(baseline, candidate);
return compareBoundedRunProjections(
baseline,
candidate,
'ordered_independent_point_reads',
);
} catch (error) {
if (!(error instanceof BoundedRunReadProjectionUnavailableError)) {
return invalid('execution context or input is invalid');
@@ -0,0 +1,360 @@
import type { RunRepositoryReader } from '../../run/runRepository';
import {
BoundedRunReadProjectionUnavailableError,
executeBoundedRunReadProjection,
type BoundedRunReadProjection,
} from '../../run/projection/boundedRunReadProjection';
import {
MAX_TASK_RUN_OUTCOME_WINDOW_STORAGE_LIMIT,
normalizeTaskRunOutcomeWindowQuery,
normalizeTaskRunOutcomeWindowRecord,
type TaskRunOutcomeWindowReader,
type TaskRunOutcomeWindowRecord,
} from '../../run/outcome-comparison/taskRunOutcomeWindow';
import {
normalizeToolDefinition,
type ToolJsonValue,
} from '../tool-registry/toolRegistry';
import {
BUILTIN_RUN_COMPARABLE_FIELDS,
BUILTIN_RUN_PROJECTION_SCHEMA,
compareBoundedRunProjections,
} from './builtInRunCompareProjection';
export const BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL = Object.freeze({
name: 'qinglong.task.runs.compare',
version: '1.0.0',
});
export const BUILTIN_TASK_RUN_OUTCOME_COMPARE_TIMEOUT_SECONDS = 5;
export const TASK_RUN_OUTCOME_SEARCH_LIMIT = 64;
const MAX_INT = 2_147_483_647;
const MIN_INT = -2_147_483_648;
const CONTROL_PATTERN = /[\u0000-\u001f\u007f]/;
export const BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION =
normalizeToolDefinition({
name: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.name,
version: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.version,
description:
'Compare the latest succeeded and failed Runs found in one fixed bounded Task history window',
inputSchema: {
type: 'object',
properties: {
taskId: { type: 'string', minLength: 1, maxLength: 255 },
},
required: ['taskId'],
additionalProperties: false,
},
outputSchema: {
type: 'object',
properties: {
taskId: { type: 'string', minLength: 1, maxLength: 255 },
baselineOutcome: {
type: 'string',
maxLength: 16,
enum: ['succeeded'] as const,
},
candidateOutcome: {
type: 'string',
maxLength: 16,
enum: ['failed'] as const,
},
baseline: BUILTIN_RUN_PROJECTION_SCHEMA,
candidate: BUILTIN_RUN_PROJECTION_SCHEMA,
comparable: { type: 'boolean' },
sameTask: { type: 'boolean' },
sameTaskRevision: { type: 'boolean' },
changedFields: {
type: 'array',
items: {
type: 'string',
maxLength: 32,
enum: BUILTIN_RUN_COMPARABLE_FIELDS,
},
maxItems: BUILTIN_RUN_COMPARABLE_FIELDS.length,
},
queueDelayDeltaMs: {
type: 'integer',
minimum: -Number.MAX_SAFE_INTEGER,
maximum: Number.MAX_SAFE_INTEGER,
},
executionDurationDeltaMs: {
type: 'integer',
minimum: -Number.MAX_SAFE_INTEGER,
maximum: Number.MAX_SAFE_INTEGER,
},
totalDurationDeltaMs: {
type: 'integer',
minimum: -Number.MAX_SAFE_INTEGER,
maximum: Number.MAX_SAFE_INTEGER,
},
selection: {
type: 'object',
properties: {
windowLimit: {
type: 'integer',
minimum: TASK_RUN_OUTCOME_SEARCH_LIMIT,
maximum: TASK_RUN_OUTCOME_SEARCH_LIMIT,
},
searchedRunCount: {
type: 'integer',
minimum: 0,
maximum: TASK_RUN_OUTCOME_SEARCH_LIMIT,
},
hasOlderRuns: { type: 'boolean' },
complete: { type: 'boolean' },
order: {
type: 'string',
maxLength: 32,
enum: ['created_at_desc_id_desc'] as const,
},
},
required: [
'windowLimit',
'searchedRunCount',
'hasOlderRuns',
'complete',
'order',
],
additionalProperties: false,
},
consistency: {
type: 'string',
maxLength: 64,
enum: ['bounded_task_window_then_ordered_point_reads'] as const,
},
},
required: [
'taskId',
'baselineOutcome',
'candidateOutcome',
'baseline',
'candidate',
'comparable',
'sameTask',
'sameTaskRevision',
'changedFields',
'selection',
'consistency',
],
additionalProperties: false,
},
effect: 'read',
risk: 'low',
requiredPermissions: ['run.read'],
timeoutSeconds: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TIMEOUT_SECONDS,
});
export class InvalidBuiltInTaskRunOutcomeCompareToolError extends TypeError {
readonly code = 'BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_INVALID';
constructor(message: string) {
super(`Built-in Task Run outcome compare Tool is invalid: ${message}`);
this.name = 'InvalidBuiltInTaskRunOutcomeCompareToolError';
}
}
export class BuiltInTaskRunOutcomeCompareToolUnavailableError extends Error {
readonly code = 'BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_UNAVAILABLE';
constructor() {
super('Built-in Task Run outcome compare Tool is unavailable');
this.name = 'BuiltInTaskRunOutcomeCompareToolUnavailableError';
}
}
function invalid(message: string): never {
throw new InvalidBuiltInTaskRunOutcomeCompareToolError(message);
}
function boundedText(value: unknown, maximum: number): value is string {
return (
typeof value === 'string' &&
value.length > 0 &&
value.length <= maximum &&
!CONTROL_PATTERN.test(value)
);
}
function isStrictlyOlder(
value: Readonly<TaskRunOutcomeWindowRecord>,
previous: Readonly<TaskRunOutcomeWindowRecord>,
): boolean {
return (
value.createdAtMs < previous.createdAtMs ||
(value.createdAtMs === previous.createdAtMs && value.id < previous.id)
);
}
async function selectOutcomeWindow(
windows: TaskRunOutcomeWindowReader,
projectId: string,
taskId: string,
): Promise<
Readonly<{
latestSucceededRunId?: string;
latestFailedRunId?: string;
searchedRunCount: number;
hasOlderRuns: boolean;
complete: boolean;
}>
> {
let rows: readonly Readonly<TaskRunOutcomeWindowRecord>[];
try {
rows = await windows.listRecentRunsByTask(
normalizeTaskRunOutcomeWindowQuery({
projectId,
taskId,
limit: MAX_TASK_RUN_OUTCOME_WINDOW_STORAGE_LIMIT,
}),
);
} catch {
throw new BuiltInTaskRunOutcomeCompareToolUnavailableError();
}
if (
!Array.isArray(rows) ||
rows.length > MAX_TASK_RUN_OUTCOME_WINDOW_STORAGE_LIMIT
) {
throw new BuiltInTaskRunOutcomeCompareToolUnavailableError();
}
let previous: Readonly<TaskRunOutcomeWindowRecord> | undefined;
let latestSucceededRunId: string | undefined;
let latestFailedRunId: string | undefined;
for (const row of rows) {
let normalized: Readonly<TaskRunOutcomeWindowRecord>;
try {
normalized = normalizeTaskRunOutcomeWindowRecord(row);
} catch {
throw new BuiltInTaskRunOutcomeCompareToolUnavailableError();
}
if (
normalized.projectId !== projectId ||
normalized.taskId !== taskId ||
(previous !== undefined && !isStrictlyOlder(normalized, previous))
) {
throw new BuiltInTaskRunOutcomeCompareToolUnavailableError();
}
previous = normalized;
}
const window = rows.slice(0, TASK_RUN_OUTCOME_SEARCH_LIMIT);
for (const row of window) {
if (row.status === 'succeeded' && latestSucceededRunId === undefined) {
latestSucceededRunId = row.id;
}
if (row.status === 'failed' && latestFailedRunId === undefined) {
latestFailedRunId = row.id;
}
if (latestSucceededRunId !== undefined && latestFailedRunId !== undefined) {
break;
}
}
const hasOlderRuns = rows.length > TASK_RUN_OUTCOME_SEARCH_LIMIT;
const complete =
!hasOlderRuns ||
(latestSucceededRunId !== undefined && latestFailedRunId !== undefined);
return Object.freeze({
...(latestSucceededRunId === undefined ? {} : { latestSucceededRunId }),
...(latestFailedRunId === undefined ? {} : { latestFailedRunId }),
searchedRunCount: window.length,
hasOlderRuns,
complete,
});
}
async function selectedProjection(
runs: Pick<RunRepositoryReader, 'findRunById'>,
projectId: string,
taskId: string,
expectedStatus: 'succeeded' | 'failed',
runId: string | undefined,
): Promise<Readonly<BoundedRunReadProjection>> {
if (runId === undefined) return Object.freeze({ found: false });
const projection = await executeBoundedRunReadProjection(
runs,
projectId,
runId,
);
if (
projection.found !== true ||
projection.taskId !== taskId ||
projection.status !== expectedStatus
) {
throw new BuiltInTaskRunOutcomeCompareToolUnavailableError();
}
return projection;
}
export async function executeBuiltInTaskRunOutcomeCompareTool(
windows: TaskRunOutcomeWindowReader,
runs: Pick<RunRepositoryReader, 'findRunById'>,
projectId: string,
input: ToolJsonValue,
): Promise<Readonly<Record<string, ToolJsonValue>>> {
const inputRecord =
input && typeof input === 'object' && !Array.isArray(input)
? (input as Readonly<Record<string, ToolJsonValue>>)
: null;
if (
!windows ||
typeof windows.listRecentRunsByTask !== 'function' ||
!runs ||
typeof runs.findRunById !== 'function' ||
!boundedText(projectId, 128) ||
!inputRecord ||
Reflect.ownKeys(inputRecord).length !== 1 ||
!boundedText(inputRecord.taskId, 255)
) {
return invalid('execution context or input is invalid');
}
try {
const selected = await selectOutcomeWindow(
windows,
projectId,
inputRecord.taskId,
);
const baseline = await selectedProjection(
runs,
projectId,
inputRecord.taskId,
'succeeded',
selected.latestSucceededRunId,
);
const candidate = await selectedProjection(
runs,
projectId,
inputRecord.taskId,
'failed',
selected.latestFailedRunId,
);
const comparison = compareBoundedRunProjections(
baseline,
candidate,
'bounded_task_window_then_ordered_point_reads',
);
return Object.freeze({
taskId: inputRecord.taskId,
baselineOutcome: 'succeeded',
candidateOutcome: 'failed',
...comparison,
selection: Object.freeze({
windowLimit: TASK_RUN_OUTCOME_SEARCH_LIMIT,
searchedRunCount: selected.searchedRunCount,
hasOlderRuns: selected.hasOlderRuns,
complete: selected.complete,
order: 'created_at_desc_id_desc',
}),
});
} catch (error) {
if (
error instanceof BuiltInTaskRunOutcomeCompareToolUnavailableError ||
error instanceof BoundedRunReadProjectionUnavailableError
) {
throw new BuiltInTaskRunOutcomeCompareToolUnavailableError();
}
return invalid('execution context or input is invalid');
}
}
@@ -0,0 +1,181 @@
import type { DeploymentProfile } from '../../cluster-control/clusterControlActivation';
import type { RunRepositoryReader } from '../../run/runRepository';
import type { TaskRunOutcomeWindowReader } from '../../run/outcome-comparison/taskRunOutcomeWindow';
import {
normalizeProjectToolDefinitionSnapshot,
type ProjectToolDefinitionSnapshot,
} from '../tool-registry/projectToolDefinitionSnapshot';
import {
ToolDefinitionRegistry,
type ToolJsonValue,
} from '../tool-registry/toolRegistry';
import {
createTrustedToolHandlerBinding,
normalizeTrustedToolHandlerBinding,
type TrustedToolHandlerBinding,
} from '../trustedToolInvocation';
import type {
TrustedToolExecutionAdapter,
TrustedToolExecutionAdapterContext,
} from '../trustedToolExecution';
import {
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TIMEOUT_SECONDS,
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL,
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION,
InvalidBuiltInTaskRunOutcomeCompareToolError,
executeBuiltInTaskRunOutcomeCompareTool,
} from './builtInTaskRunOutcomeCompareProjection';
export {
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TIMEOUT_SECONDS,
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL,
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION,
TASK_RUN_OUTCOME_SEARCH_LIMIT,
BuiltInTaskRunOutcomeCompareToolUnavailableError,
InvalidBuiltInTaskRunOutcomeCompareToolError,
executeBuiltInTaskRunOutcomeCompareTool,
} from './builtInTaskRunOutcomeCompareProjection';
export const BUILTIN_TASK_RUN_OUTCOME_COMPARE_ADAPTER = Object.freeze({
id: 'builtin.qinglong.task-runs-compare',
version: '1.0.0',
});
export const BUILTIN_TASK_RUN_OUTCOME_COMPARE_REDACTION_CONTRACT =
Object.freeze({
id: 'redaction.qinglong.task-runs-compare',
version: '1.0.0',
});
export const BUILTIN_TASK_RUN_OUTCOME_COMPARE_AUDIT_CONTRACT = Object.freeze({
id: 'audit.qinglong.tool-call',
version: '1.0.0',
});
const CONTROL_PATTERN = /[\u0000-\u001f\u007f]/;
function invalid(message: string): never {
throw new InvalidBuiltInTaskRunOutcomeCompareToolError(message);
}
function sameValue(left: unknown, right: unknown): boolean {
return JSON.stringify(left) === JSON.stringify(right);
}
function boundedText(value: unknown, maximum: number): value is string {
return (
typeof value === 'string' &&
value.length > 0 &&
value.length <= maximum &&
!CONTROL_PATTERN.test(value)
);
}
export function createBuiltInTaskRunOutcomeCompareToolHandlerBinding(
snapshotValue: ProjectToolDefinitionSnapshot,
profiles: readonly DeploymentProfile[],
): Readonly<TrustedToolHandlerBinding> {
const snapshot = normalizeProjectToolDefinitionSnapshot(snapshotValue);
const definition = snapshot.definitions.find(
(entry) =>
entry.definition.name === BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.name &&
entry.definition.version ===
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.version,
)?.definition;
if (
!definition ||
!sameValue(definition, BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION)
) {
return invalid('reviewed Tool definition is absent or changed');
}
return createTrustedToolHandlerBinding(snapshot, {
tool: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL,
adapter: BUILTIN_TASK_RUN_OUTCOME_COMPARE_ADAPTER,
executionClass: 'builtin_in_process',
profiles,
authorities: ['database.read'],
timeoutSeconds: BUILTIN_TASK_RUN_OUTCOME_COMPARE_TIMEOUT_SECONDS,
redactionContract: BUILTIN_TASK_RUN_OUTCOME_COMPARE_REDACTION_CONTRACT,
auditContract: BUILTIN_TASK_RUN_OUTCOME_COMPARE_AUDIT_CONTRACT,
});
}
export class BuiltInTaskRunOutcomeCompareToolAdapter
implements TrustedToolExecutionAdapter
{
readonly binding!: Readonly<TrustedToolHandlerBinding>;
readonly profile!: DeploymentProfile;
readonly recoveryMode = 'retry_safe_read' as const;
readonly #windows!: TaskRunOutcomeWindowReader;
readonly #runs!: Pick<RunRepositoryReader, 'findRunById'>;
constructor(
bindingValue: TrustedToolHandlerBinding,
profile: DeploymentProfile,
definitions: ToolDefinitionRegistry,
windows: TaskRunOutcomeWindowReader,
runs: Pick<RunRepositoryReader, 'findRunById'>,
) {
const binding = normalizeTrustedToolHandlerBinding(bindingValue);
if (!(definitions instanceof ToolDefinitionRegistry)) {
return invalid('Tool Definition registry is invalid');
}
let definition;
try {
definition = definitions.resolve(
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.name,
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.version,
);
} catch {
return invalid('reviewed Tool definition is unavailable');
}
if (
!sameValue(binding.tool, BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL) ||
!sameValue(binding.adapter, BUILTIN_TASK_RUN_OUTCOME_COMPARE_ADAPTER) ||
binding.executionClass !== 'builtin_in_process' ||
!sameValue(binding.authorities, ['database.read']) ||
binding.timeoutSeconds !==
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TIMEOUT_SECONDS ||
!sameValue(
binding.redactionContract,
BUILTIN_TASK_RUN_OUTCOME_COMPARE_REDACTION_CONTRACT,
) ||
!sameValue(
binding.auditContract,
BUILTIN_TASK_RUN_OUTCOME_COMPARE_AUDIT_CONTRACT,
) ||
!binding.profiles.includes(profile) ||
!sameValue(definition, BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION)
) {
return invalid('binding does not match the reviewed adapter contract');
}
if (!windows || typeof windows.listRecentRunsByTask !== 'function') {
return invalid('Task Run outcome window reader is invalid');
}
if (!runs || typeof runs.findRunById !== 'function') {
return invalid('Run repository is invalid');
}
this.binding = binding;
this.profile = profile;
this.#windows = windows;
this.#runs = runs;
Object.freeze(this);
}
async execute(
context: Readonly<TrustedToolExecutionAdapterContext>,
input: ToolJsonValue,
): Promise<unknown> {
if (
!context ||
typeof context !== 'object' ||
!boundedText(context.projectId, 128)
) {
return invalid('execution context or input is invalid');
}
return executeBuiltInTaskRunOutcomeCompareTool(
this.#windows,
this.#runs,
context.projectId,
input,
);
}
}
@@ -0,0 +1,328 @@
const assert = require('node:assert/strict');
const { test } = require('node:test');
const {
BUILTIN_TASK_RUN_OUTCOME_COMPARE_ADAPTER,
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL,
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION,
BuiltInTaskRunOutcomeCompareToolAdapter,
BuiltInTaskRunOutcomeCompareToolUnavailableError,
InvalidBuiltInTaskRunOutcomeCompareToolError,
TASK_RUN_OUTCOME_SEARCH_LIMIT,
createBuiltInTaskRunOutcomeCompareToolHandlerBinding,
executeBuiltInTaskRunOutcomeCompareTool,
} = require('../dist/tool-execution/builtin-run-compare/builtInTaskRunOutcomeCompareTool');
const {
createPluginPackageResourceGenerationFromReferences,
} = require('../dist/plugin-package/pluginPackageResourceGeneration');
const {
createProjectToolDefinitionSnapshot,
projectToolDefinitionRegistry,
} = require('../dist/tool-execution/tool-registry/projectToolDefinitionSnapshot');
const DIGEST_A = 'a'.repeat(64);
const DIGEST_B = 'b'.repeat(64);
const DIGEST_C = 'c'.repeat(64);
function run(id, overrides = {}) {
return {
id,
projectId: 'project-outcomes',
taskId: 'task-backup',
taskRevision: 'task-backup@4',
triggerType: 'schedule',
executionOrigin: 'scheduled_system',
executionOwner: 'runtime',
status: 'succeeded',
version: 4,
eventSequence: 8,
priority: 10,
createdAtMs: 1_000,
queuedAtMs: 1_020,
startedAtMs: 1_050,
finishedAtMs: 1_150,
requestId: 'must-not-cross-tool-output',
...overrides,
};
}
function windowRecord(value) {
return {
id: value.id,
projectId: value.projectId,
taskId: value.taskId,
status: value.status,
createdAtMs: value.createdAtMs,
};
}
function fixture(records, window, calls = []) {
return {
windows: {
async listRecentRunsByTask(query) {
calls.push({ type: 'window', query });
return window;
},
},
runs: {
async findRunById(runId) {
calls.push({ type: 'point', runId });
return records.get(runId) ?? null;
},
},
};
}
function snapshot(
definition = BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION,
) {
const generation = createPluginPackageResourceGenerationFromReferences({
installationId: 'install-qinglong-outcome-compare',
projectId: 'project-outcomes',
packageName: 'qinglong',
lockDigest: DIGEST_A,
generation: 1,
previousActiveLockDigest: null,
contentDigest: DIGEST_B,
resources: [],
});
return createProjectToolDefinitionSnapshot({
projectId: 'project-outcomes',
contributions: [
{
generation,
revisionDigest: DIGEST_C,
definitions: [definition],
},
],
});
}
test('selects and compares the latest succeeded and failed Runs in one fixed Task window', async () => {
const calls = [];
const succeeded = run('run-success');
const failed = run('run-failure', {
taskRevision: 'task-backup@5',
status: 'failed',
version: 6,
eventSequence: 12,
createdAtMs: 2_000,
queuedAtMs: 2_040,
startedAtMs: 2_100,
finishedAtMs: 2_350,
});
const ignored = run('run-running', {
status: 'running',
createdAtMs: 3_000,
});
const value = fixture(
new Map([
[succeeded.id, succeeded],
[failed.id, failed],
]),
[windowRecord(ignored), windowRecord(failed), windowRecord(succeeded)],
calls,
);
const output = await executeBuiltInTaskRunOutcomeCompareTool(
value.windows,
value.runs,
'project-outcomes',
{ taskId: 'task-backup' },
);
assert.deepEqual(calls, [
{
type: 'window',
query: {
projectId: 'project-outcomes',
taskId: 'task-backup',
limit: 65,
},
},
{ type: 'point', runId: 'run-success' },
{ type: 'point', runId: 'run-failure' },
]);
assert.equal(output.taskId, 'task-backup');
assert.equal(output.baselineOutcome, 'succeeded');
assert.equal(output.candidateOutcome, 'failed');
assert.equal(output.baseline.id, 'run-success');
assert.equal(output.candidate.id, 'run-failure');
assert.deepEqual(output.changedFields, ['taskRevision', 'status']);
assert.equal(output.queueDelayDeltaMs, 20);
assert.equal(output.executionDurationDeltaMs, 150);
assert.equal(output.totalDurationDeltaMs, 200);
assert.deepEqual(output.selection, {
windowLimit: 64,
searchedRunCount: 3,
hasOlderRuns: false,
complete: true,
order: 'created_at_desc_id_desc',
});
assert.equal(
output.consistency,
'bounded_task_window_then_ordered_point_reads',
);
assert.equal(output.baseline.requestId, undefined);
const registry = projectToolDefinitionRegistry(snapshot());
assert.deepEqual(
registry.normalizeOutput(
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.name,
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.version,
output,
),
output,
);
});
test('reports an incomplete fixed window without exposing pagination', async () => {
const calls = [];
const rows = Array.from(
{ length: TASK_RUN_OUTCOME_SEARCH_LIMIT + 1 },
(_, index) =>
windowRecord(
run(`run-${String(100 - index).padStart(3, '0')}`, {
status: 'running',
createdAtMs: 10_000 - index,
}),
),
);
const value = fixture(new Map(), rows, calls);
const output = await executeBuiltInTaskRunOutcomeCompareTool(
value.windows,
value.runs,
'project-outcomes',
{ taskId: 'task-backup' },
);
assert.deepEqual(output.baseline, { found: false });
assert.deepEqual(output.candidate, { found: false });
assert.equal(output.comparable, false);
assert.deepEqual(output.selection, {
windowLimit: 64,
searchedRunCount: 64,
hasOlderRuns: true,
complete: false,
order: 'created_at_desc_id_desc',
});
assert.equal(calls.length, 1);
assert.equal(
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION.inputSchema.properties
.after,
undefined,
);
assert.equal(
BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL_DEFINITION.inputSchema.properties
.limit,
undefined,
);
});
test('fails closed on foreign, unordered, corrupt, disappearing, and unavailable records', async () => {
const valid = windowRecord(run('run-valid'));
for (const window of [
[{ ...valid, projectId: 'other-project' }],
[valid, { ...valid, id: 'run-newer', createdAtMs: 2_000 }],
[{ ...valid, status: 'invented' }],
]) {
const value = fixture(new Map(), window);
await assert.rejects(
executeBuiltInTaskRunOutcomeCompareTool(
value.windows,
value.runs,
'project-outcomes',
{ taskId: 'task-backup' },
),
BuiltInTaskRunOutcomeCompareToolUnavailableError,
);
}
const missing = fixture(new Map(), [valid]);
await assert.rejects(
executeBuiltInTaskRunOutcomeCompareTool(
missing.windows,
missing.runs,
'project-outcomes',
{ taskId: 'task-backup' },
),
BuiltInTaskRunOutcomeCompareToolUnavailableError,
);
await assert.rejects(
executeBuiltInTaskRunOutcomeCompareTool(
{
async listRecentRunsByTask() {
throw new Error('private DSN must not escape');
},
},
fixture(new Map(), []).runs,
'project-outcomes',
{ taskId: 'task-backup' },
),
BuiltInTaskRunOutcomeCompareToolUnavailableError,
);
});
test('rejects aliases and binds the reviewed retry-safe database-read adapter', async () => {
const empty = fixture(new Map(), []);
await assert.rejects(
executeBuiltInTaskRunOutcomeCompareTool(
empty.windows,
empty.runs,
'project-outcomes',
{ taskId: 'task-backup', after: 'cursor' },
),
InvalidBuiltInTaskRunOutcomeCompareToolError,
);
const currentSnapshot = snapshot();
const binding = createBuiltInTaskRunOutcomeCompareToolHandlerBinding(
currentSnapshot,
['edge', 'standalone', 'cluster-control'],
);
assert.deepEqual(binding.tool, BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL);
assert.deepEqual(binding.adapter, BUILTIN_TASK_RUN_OUTCOME_COMPARE_ADAPTER);
assert.deepEqual(binding.authorities, ['database.read']);
const adapter = new BuiltInTaskRunOutcomeCompareToolAdapter(
binding,
'edge',
projectToolDefinitionRegistry(currentSnapshot),
empty.windows,
empty.runs,
);
assert.equal(adapter.recoveryMode, 'retry_safe_read');
const output = await adapter.execute(
{ projectId: 'project-outcomes' },
{ taskId: 'task-backup' },
);
assert.equal(output.selection.complete, true);
assert.throws(
() =>
new BuiltInTaskRunOutcomeCompareToolAdapter(
{ ...binding, authorities: ['database.read', 'network.client'] },
'edge',
projectToolDefinitionRegistry(currentSnapshot),
empty.windows,
empty.runs,
),
/handler authority is invalid/,
);
});
test('publishes only explicit outcome comparison subpaths and keeps the root unchanged', () => {
const tool = require('@qinglong/runtime-core/builtin-task-run-outcome-compare-tool');
const projection = require('@qinglong/runtime-core/builtin-task-run-outcome-compare-projection');
const window = require('@qinglong/runtime-core/task-run-outcome-window');
const root = require('@qinglong/runtime-core');
assert.equal(
tool.BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL.name,
'qinglong.task.runs.compare',
);
assert.equal(projection.TASK_RUN_OUTCOME_SEARCH_LIMIT, 64);
assert.equal(window.MAX_TASK_RUN_OUTCOME_WINDOW_STORAGE_LIMIT, 65);
assert.equal(root.BUILTIN_TASK_RUN_OUTCOME_COMPARE_TOOL, undefined);
assert.equal(root.executeBuiltInTaskRunOutcomeCompareTool, undefined);
});
+2
View File
@@ -1014,10 +1014,12 @@ function auditSourceImports(root, packagePath, findings) {
'@qinglong/runtime-core/bounded-run-list-projection',
'@qinglong/runtime-core/bounded-run-step-list-projection',
'@qinglong/runtime-core/builtin-run-compare-projection',
'@qinglong/runtime-core/builtin-task-run-outcome-compare-projection',
'@qinglong/runtime-core/builtin-run-read-projection',
'@qinglong/runtime-core/bounded-task-read-projection',
'@qinglong/runtime-core/bounded-task-list-projection',
'@qinglong/runtime-core/project-run-list',
'@qinglong/runtime-core/task-run-outcome-window',
'@qinglong/runtime-core/project-policy',
'@qinglong/runtime-core/run',
'@qinglong/runtime-core/run-repository',
@@ -193,7 +193,9 @@ test('confines local MCP to its reviewed protocol and read-authority subpaths',
"import { runs } from '@qinglong/runtime-core/bounded-run-list-projection';",
"import { run } from '@qinglong/runtime-core/run';",
"import { compare } from '@qinglong/runtime-core/builtin-run-compare-projection';",
"import { outcomes } from '@qinglong/runtime-core/builtin-task-run-outcome-compare-projection';",
"import { tool } from '@qinglong/runtime-core/builtin-run-read-projection';",
"import { window } from '@qinglong/runtime-core/task-run-outcome-window';",
"import { tasks } from '@qinglong/runtime-core/bounded-task-list-projection';",
"import { task } from '@qinglong/runtime-core/bounded-task-read-projection';",
"import { trigger } from '@qinglong/runtime-core/trigger';",
+6 -6
View File
@@ -299,10 +299,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
rootSourceFileRoles: runtimeCore.rootSourceFileRoles,
},
{
sourceFiles: 162,
sourceFiles: 165,
rootSourceFiles: 1,
rootSourceLines: 160,
nestedSourceFiles: 161,
nestedSourceFiles: 164,
rootSourceFileRoles: { 'index.ts': 'public_export' },
},
);
@@ -421,10 +421,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
rootSourceFileRoles: clusterPostgres.rootSourceFileRoles,
},
{
sourceFiles: 167,
sourceFiles: 168,
rootSourceFiles: 1,
rootSourceLines: 126,
nestedSourceFiles: 166,
nestedSourceFiles: 167,
rootSourceFileRoles: { 'index.ts': 'public_export' },
},
);
@@ -540,10 +540,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
rootSourceFileRoles: localSqlite.rootSourceFileRoles,
},
{
sourceFiles: 196,
sourceFiles: 197,
rootSourceFiles: 1,
rootSourceLines: 31,
nestedSourceFiles: 195,
nestedSourceFiles: 196,
rootSourceFileRoles: { 'index.ts': 'public_export' },
},
);