feat(ql3): expose cancellation status card

This commit is contained in:
whyour
2026-08-19 09:21:50 +08:00
parent 5a979e510e
commit 095683a4cb
15 changed files with 741 additions and 32 deletions
+17 -2
View File
@@ -11,7 +11,22 @@
最新增量证据(2026-08-19):
- D-366/ADR-0459(已接受;产品视觉入口待完成):在Run management plane 增加 `run.cancellation.summary`,由强认证 User 以
- D-367/ADR-0460(已接受;有界 blocked drill-down 与 Console 可选接入待完成):在`ql3 run`/`ql3-run-client` 增加一次性
`status --config=... --assertion=... --project=... [--format=text|json]` 产品入口。它在内存生成固定 `run.cancellation.summary` 命令,仍经同一
exact codec、TLS 1.3、Run 专用 mTLS/OIDC、固定 management route 和响应交叉不变量校验;原 `--command` 私有文件模式保持兼容。默认 text 是无 ANSI
的确定性 Project 状态卡,JSON 使用 `qinglong/run-cancellation-status@v1`;两者只含 D-366 低敏计数与结论。告警映射固定为
`clear→ok/0``converging→warning/10``attention_required→critical/20`,查询失败仍为 1、用法错误仍为 64,使外部 supervisor 无需解析自然语言。
命令一次请求后退出,不轮询、不重试、不缓存、不保持 socket,也不改变 Cluster readiness。实现只在已有 `cluster-admin/run-management`
`management-support` 内增加真实职责,不新增 package、binary、依赖、服务、端口、timer、queue、cache、数据库权限、migration 或 Kubernetes 对象;
Edge/Standalone 闭包不变化。未知 blocked Run 的发现仍须后续独立的有界 cursor 契约,不能把 status 扩成无界列表或把第二组管理 authority 默认塞进
Copilot Console。聚焦门 `12/12`、CLI/product 真实进程门 `15/15`Cluster Admin 全量为 `407 total / 404 pass / 3 conditional skip / 0 fail`
backend 为 `1,489 total / 1,487 pass / 2 conditional skip / 0 fail`18-package clean build/test 退出 0。四项架构审计与 `14/14` Local artifact audit
全部 compatibleworkspace 保持 18 packages、无 single/shallow packageCluster Admin 为 123 个 source、122 个 nested source、仅 1 个受审 binary root
entry。基础 Edge/Standalone 保持 `2,589,998 / 2,590,076` bytesApplication+AI 保持 `4,493,151 / 4,493,283` bytesMCP 保持
`7,315,930 / 7,316,038` bytes。PostgreSQL 18.6 arm64 HA `145/145`、timeline `1→2`,报告 SHA-256 为
`59a568d0511cde671946ebf6df09f88868a3d591c5021c90bc27d4715411091e`,独立 evidence audit compatible 且零 finding。
- D-366/ADR-0459(已接受;一次性产品 CLI 由 D-367 完成,Console 可选接入待完成):在既有 Run management plane 增加 `run.cancellation.summary`,由强认证 User 以
`run.read` 按需读取 Project 级 PostgreSQL 快照。响应只有五态 dispatch 计数、due/expired-lease 信号、四种 blocking-result 计数、最早 blocked
时间和 `clear|converging|attention_required`/`none|wait|inspect` 固定结论,不返回 Run/Attempt/Worker identity 或 lease capability。blocked 触发
`attention_required`,但不错误撤回整个 Cluster readinessdue/expired 只作为 caller-driven 收敛信号。查询与 allowed audit 位于同一 5 秒
@@ -22,7 +37,7 @@
`145/145` 执行 blocked summary→inspect→rearm→production delivery→WAL→promotiontimeline `1→2`,报告 SHA-256 为
`d763157b3a781e305add3c6f0c5080820b1d65b5feefe60be6fa7006c0050107`
- D-365/ADR-0458(已接受;聚合出口由 D-366 完成,产品视觉入口待完成):在既有隔离 `cluster-admin` Run management plane 上增加
- D-365/ADR-0458(已接受;聚合出口由 D-366、一次性产品 CLI 由 D-367 完成):在既有隔离 `cluster-admin` Run management plane 上增加
`run.cancellation.inspect``run.cancellation.rearm`,不新建 package、服务、端口、timer、连接池或 Kubernetes 对象。inspect 要求强认证 User
`run.read`viewer 可读取 Run/dispatch 的固定低敏投影,但永不返回 lease owner、raw token 或 token digestrearm 要求 `run.stop`,只接受
`blocked + expected dispatch version + expected blocking result` 的精确 CASretry delay 固定为 1 秒至 24 小时并由 PostgreSQL
@@ -155,9 +155,11 @@ PID 可复用,可能终止无关进程,禁止。
ADR-0459 进一步增加 Project-scoped、caller-driven summary,以固定计数和三态 assessment 提供 blocked/availability 告警出口,不改变全局 readiness,也不新增采集器。
ADR-0460 已在现有 `ql3 run` 增加 one-shot status 产品入口:同一强认证 summary 被投影为低敏 text/JSON 状态卡,并以 `0/10/20` 区分 clear、converging 与 attention-required;没有新增轮询器、临时 command 文件或常驻 authority。
HTTP worker 已通过默认关闭的 manual-only manifest bootstrap 接入 Local Supervisor:只有 accepted 且全部 gate 通过时才启动,失败或 shutdown 时有界停止。以下工作仍未完成,因此它仍只允许显式 canary,不得扩大到默认生产流量:
- 产品控制台中的 Project 状态卡、告警路由和有界 blocked drill-downProject 聚合出口私有 operator 处置协议已经实现。
- Copilot Console 的可选 Project 状态卡有界 blocked drill-downProject 聚合出口、一次性产品状态卡、告警退出码和私有 operator 处置协议已经实现。
- 固定 edge 设备的数据库写放大、RSS、时延和磁盘基准。
- 固定 Project allowlist 的外部时序指标适配;数据库事实驱动的按需 availability/blocked 汇总已经实现。
- 首次真实目标实例完整激活/回滚仪式与共享 config 多写者 authority。
@@ -180,3 +182,4 @@ HTTP worker 已通过默认关闭的 manual-only manifest bootstrap 接入 Local
14. 强认证 viewer 只能读取无 lease capability 的低敏诊断;只有 `run.stop` authority 可按 blocked/version/result 精确 rearm。
15. rearm、RunEvent 与 allowed audit 原子提交,数据库时间决定 retry duestale fence、授权漂移与 mutation drift 均失败关闭。
16. Project summary 的五态与 blocking-result 计数必须交叉守恒,blocked 只产生 `attention_required` 告警而不撤回全局 readiness,且响应不包含 Run/Attempt/lease identity。
17. `ql3 run status` 必须只发送一次固定 summarytext/JSON 事实一致,`0/10/20` 与三态严格映射;原 command-file 模式和 Edge/Standalone 闭包不得变化。
@@ -62,4 +62,4 @@ QingLong 同时面向低配路由器与集群。该能力只属于 Cluster opera
## 后续
ADR-0459 已用既有管理面完成数据库事实驱动的 Project 级 blocked/availability 汇总和固定告警结论;产品视觉入口、告警路由和有界 drill-down 仍待实现。CloudNativePG live failover、多副本容量压力、固定 Linux x64/arm64 与物理 Edge 资源证据继续作为发布最终化门,不由单机 Docker HA 结果替代。
ADR-0459 已用既有管理面完成数据库事实驱动的 Project 级 blocked/availability 汇总;ADR-0460 已增加一次性产品状态卡、稳定 JSON 与告警退出码。有界 blocked drill-down 和 Console 可选接入仍待实现。CloudNativePG live failover、多副本容量压力、固定 Linux x64/arm64 与物理 Edge 资源证据继续作为发布最终化门,不由单机 Docker HA 结果替代。
@@ -58,4 +58,4 @@ QingLong 同时面对小型路由设备和多副本 Cluster。Local/Edge 不应
## 后续
产品控制台可把该 summary 渲染为 Project 状态卡和告警入口,并通过单 Run inspect/rearm 完成 drill-down。CloudNativePG live failover、多副本容量压力、固定 Linux x64/arm64 和物理 Edge 资源证据仍是独立发布门;本 ADR 不把 Docker HA 或按需汇总冒充这些现场证据。
ADR-0460 已用现有 `ql3 run status` 完成一次性 Project 状态卡、稳定 JSON 和 `0/10/20` 告警退出码,不增加常驻组件或混合 Copilot Console authority。未知 blocked Run 的有界发现与 Console 可选接入仍需独立设计。CloudNativePG live failover、多副本容量压力、固定 Linux x64/arm64 和物理 Edge 资源证据仍是独立发布门;本 ADR 不把 Docker HA 或按需汇总冒充这些现场证据。
@@ -0,0 +1,62 @@
# ADR-0460:一次性 Cancellation 可用性状态卡与告警退出码
- 状态:Accepted
- 日期:2026-08-19
- 关联 RFCQL-RFC-0001 D-367、PR-5、PR-7
- 关联 ADRADR-0005、ADR-0458、ADR-0459
- AmendsADR-0459 的产品入口与告警路由边界
## 上下文
ADR-0459 已让 Run management plane 返回数据库事实驱动的 Project 级 cancellation summary,但现有 `ql3 run` 仍要求 operator 手工准备完整 command JSON,再自行解释三态 assessment。该协议可以被脚本调用,却还不是可直接使用的产品入口,也没有稳定的告警退出语义。
QingLong 的部署跨度很大。Cluster operator 需要可读状态卡和可供 systemd、CronJob、CI 或外部告警器路由的机器结果;路由器级 Edge/Standalone 则不能为 Cluster 可观测性引入 daemon、轮询、PostgreSQL 客户端或新的制品闭包。Copilot Console 又只有普通 Project API Bearer authority,直接把 Run management mTLS/OIDC 凭据并入该常驻本机 BFF 会混合两种权限边界。
## 决策
1. 在现有 `ql3 run`/`ql3-run-client` 增加一次性 `status` 模式:`ql3 run status --config=... --assertion=... --project=... [--format=text|json]`。它继续由统一产品 CLI 无 shell 委派同一 binary,不新增 workspace package、binary、服务或端口。
2. `status` 只在一次调用内生成 request/audit UUID,并构造固定 `run.cancellation.summary` 命令。Project、配置文件和短生命周期 assertion 仍显式提供;operator context 只可补充稳定的 Run management config,不保存 assertion。调用方不能提供 Run/Attempt identity、状态过滤、时间窗口、计数或服务端时间。
3. 通用认证管理客户端允许包内调用方传入内存命令,但仍执行同一 exact-shape normalizer、TLS 1.3、Run 专用 mTLS、OIDC Bearer、固定 management path、128 KiB 响应上限、单连接和响应校验。既有 `--command` 私有文件模式保持字节读取、权限检查、错误语义和输出格式不变。
4. 默认 `text` 输出是无 ANSI 控制字符、确定性、适合终端与日志的状态卡;`json` 输出使用固定 `qinglong/run-cancellation-status@v1` schema。两者只包含 request/Project、数据库观察时间、assessment/action、固定 dispatch/signal/blocking 计数和可选最早 blocked 时间,不增加 Run/Attempt/Worker、lease、PID、命令、环境、Secret、日志或错误原文。
5. 产品告警映射固定为 `clear → ok/0``converging → warning/10``attention_required → critical/20`。配置/网络/协议失败仍为 `1`CLI 用法错误仍为 `64`;因此外部 supervisor 可区分“等待 caller-driven 收敛”“需要 inspect”与“查询本身失败”,而无需解析自然语言。
6. 命令严格 one-shot:一次 summary、一次 allowed/denied audit,然后退出;不重试、不轮询、不缓存、不保持 socket,也不改变 `/readyz`。blocked 仍不撤回整个 Cluster readiness。
7. 该入口只进入 Cluster Admin 制品。Edge/Standalone 依赖图和制品不得包含 `cluster-admin``cluster-postgres``pg`、状态 CLI 或 TLS/OIDC 管理凭据。
8. D-367 不返回 blocked Run 列表。用户从已知 Run 继续使用既有 inspect/rearm;从聚合发现未知 blocked Run 的能力必须以后用独立、有界、稳定 cursor、低敏 identity 和索引证明的 drill-down 契约完成,不能让 status 偷偷退化为无界扫描。
## 被拒绝的替代方案
### 在 Copilot Console 直接复用普通 Project API 凭据
拒绝。summary 位于独立 Run management authority,普通 Bearer 不应绕过 mTLS、OIDC purpose 和专用 Policy/audit。把第二组高权限凭据强制塞进 Console 也会扩大常驻 BFF 的秘密与故障面;Console 接入应在未来以显式可选 authority 单独设计。
### 新建 exporter、告警 daemon 或后台轮询器
拒绝。它会增加常驻进程、连接和 cadence,并让小规模 Cluster 为无人查看的状态持续付费。一次性命令可由现有部署侧调度器按自身策略调用。
### 客户端从列表或本地缓存推导 assessment
拒绝。列表可能分页且跨页漂移,本地缓存也会在 failover 后失真。状态卡只投影经过服务器和客户端双重交叉不变量校验的单事务 summary。
### 所有非 clear 状态返回同一个退出码
拒绝。converging 需要等待,attention_required 需要 inspect;合并后外部告警器只能解析 JSON 或把正常收敛当成阻塞故障。
## 资源、安全与部署影响
- Cluster 每次调用只建立一个短生命周期 mTLS 连接,使用既有 Run manager pool 完成一个最长 5 秒的 SERIALIZABLE summary/audit 事务;客户端不创建 Agent keep-alive、缓存、timer 或临时 command 文件。
- 文本卡和 JSON 使用同一已校验 projectionProject/时间/计数均有既有协议上限,不存在动态指标 label 或任意服务端路径。
- 旧命令文件模式继续支持 retry、stop、summary、inspect 和 rearm,升级不要求重写现有自动化。
- 新源码保留在已有 `cluster-admin/run-management` 内;通用内存命令入口位于已有 `management-support`,没有单文件 package 或 `src` 根平铺。
## 验证
- `runCancellationStatus` 与 management-client 聚焦门 `12/12`CLI/product 真实进程门 `15/15`;覆盖内存 summary 命令、三态 severity/退出码、低敏文本卡、错误 operation 拒绝和原 command-file/mTLS route 兼容。
- 真实本机 TLS 1.3/mTLS CLI 集成测试验证只向 `/api/v3/runs/management` POST 一次 `run.cancellation.summary`,不带 Run ID,并以 `attention_required` 返回 JSON 与退出码 20help 与非法 Project 在 I/O 前失败关闭。
- Cluster Admin 全量 `407 total / 404 pass / 3 conditional skip / 0 fail`backend 全量 `1,489 total / 1,487 pass / 2 conditional skip / 0 fail`18-package clean build/test 退出 0。
- package boundary、Cluster dependency、Edge import、Cluster deployment 四项审计均 compatibleworkspace 保持 18 packages、无 single/shallow packageCluster Admin 为 123 个 source、122 个 nested source、仅 1 个受审 binary root entry。
- `14/14` Local artifact audit 均 compatible;基础 Edge/Standalone 为 `2,589,998 / 2,590,076` bytesApplication+AI 为 `4,493,151 / 4,493,283` bytesMCP 为 `7,315,930 / 7,316,038` bytes,证明 Cluster-only 入口未进入低配设备闭包。
- PostgreSQL 18.6 arm64 HA `145/145`timeline `1→2`,报告 SHA-256 `59a568d0511cde671946ebf6df09f88868a3d591c5021c90bc27d4715411091e`;独立 evidence audit 为 `compatible=true`、零 finding。
## 后续
D-368 可设计 Project-scoped blocked drill-down:固定小页、稳定数据库 cursor、只返回继续 inspect 所需的最低 identity,并证明索引、事务一致性、Policy/audit 和多副本 HA。Copilot Console 状态卡应复用本 ADR 的 projection 和显式 Run management authority,但不得默认持有该 authority 或建立轮询。
@@ -49,6 +49,12 @@ export interface ClusterPluginPackageManagementClientPaths {
readonly assertionFile: string;
}
export interface ClusterAuthenticatedManagementCommandExecution<Command> {
readonly configFile: string;
readonly assertionFile: string;
readonly command: Command;
}
export interface ClusterPluginPackageManagementClientResult {
readonly schemaVersion: 1;
readonly requestId: string;
@@ -1119,11 +1125,23 @@ export async function executeClusterAuthenticatedManagementClient<
Command,
Result,
>(
paths: ClusterPluginPackageManagementClientPaths,
execution:
| ClusterPluginPackageManagementClientPaths
| ClusterAuthenticatedManagementCommandExecution<Command>,
protocol: ClusterAuthenticatedManagementClientProtocol<Command, Result>,
connectionOptions?: ClusterPluginPackageManagementClientConnectionOptions,
): Promise<Readonly<ClusterAuthenticatedManagementClientResult<Result>>> {
exactObject(paths, ['configFile', 'commandFile', 'assertionFile']);
const inlineCommand =
execution !== null &&
typeof execution === 'object' &&
!Array.isArray(execution) &&
Object.hasOwn(execution, 'command');
exactObject(
execution,
inlineCommand
? ['configFile', 'command', 'assertionFile']
: ['configFile', 'commandFile', 'assertionFile'],
);
if (
!protocol ||
typeof protocol !== 'object' ||
@@ -1160,17 +1178,19 @@ export async function executeClusterAuthenticatedManagementClient<
| undefined;
try {
prepared = prepareClusterAuthenticatedManagementClientConfiguration(
paths.configFile,
execution.configFile,
protocol.managementPath,
protocol.clientCertificate,
);
commandBytes = readCanonicalFile(
paths.commandFile,
MAX_COMMAND_BYTES,
'private',
);
if (!inlineCommand) {
commandBytes = readCanonicalFile(
(execution as ClusterPluginPackageManagementClientPaths).commandFile,
MAX_COMMAND_BYTES,
'private',
);
}
assertionBytes = readCanonicalFile(
paths.assertionFile,
execution.assertionFile,
MAX_ASSERTION_BYTES,
'private',
);
@@ -1183,7 +1203,12 @@ export async function executeClusterAuthenticatedManagementClient<
clientCertificateBytes,
clientPrivateKeyBytes,
} = prepared;
const command = protocol.normalizeCommand(parseJson(commandBytes));
const command = protocol.normalizeCommand(
inlineCommand
? (execution as ClusterAuthenticatedManagementCommandExecution<Command>)
.command
: parseJson(commandBytes!),
);
const assertion = assertionBytes.toString('ascii');
if (
assertionBytes.some((byte) => byte > 0x7f) ||
@@ -87,7 +87,7 @@ export const QINGLONG3_CLUSTER_PRODUCT_COMMANDS: readonly QingLong3ClusterProduc
name: 'run',
binary: 'ql3-run-client',
target: 'run-management/runManagementClientCli.js',
description: 'retry or stop Runs under strong authentication',
description: 'observe, retry or stop Runs under strong authentication',
}),
Object.freeze({
name: 'automation',
@@ -0,0 +1,127 @@
import { randomUUID } from 'node:crypto';
import type { ClusterRunManagementClientResult } from './runManagementClient';
import {
RUN_CANCELLATION_DISPATCH_SUMMARY_REQUEST_SCHEMA,
normalizeClusterRunManagementCommand,
type ClusterRunManagementCancellationSummaryCommand,
type ClusterRunManagementCancellationSummaryTransportResult,
} from './runManagementTransport';
export const RUN_CANCELLATION_STATUS_SCHEMA =
'qinglong/run-cancellation-status@v1' as const;
export type RunCancellationStatusExitCode = 0 | 10 | 20;
export type RunCancellationStatusSeverity = 'ok' | 'warning' | 'critical';
type CancellationSummary =
ClusterRunManagementCancellationSummaryTransportResult['summary'];
export interface RunCancellationStatusObservation {
readonly schemaVersion: 1;
readonly schema: typeof RUN_CANCELLATION_STATUS_SCHEMA;
readonly component: 'qinglong3-run-management-client';
readonly event: 'cancellation_status_observed';
readonly requestId: string;
readonly projectId: string;
readonly observedAtMs: number;
readonly assessment: CancellationSummary['assessment'];
readonly operatorAction: CancellationSummary['operatorAction'];
readonly severity: RunCancellationStatusSeverity;
readonly exitCode: RunCancellationStatusExitCode;
readonly dispatches: CancellationSummary['dispatches'];
readonly signals: CancellationSummary['signals'];
readonly blockingResults: CancellationSummary['blockingResults'];
readonly oldestBlockedAtMs?: number;
}
export function createRunCancellationStatusCommand(
projectId: string,
createUuid: () => string = randomUUID,
): Readonly<ClusterRunManagementCancellationSummaryCommand> {
const requestId = createUuid();
const auditEventId = createUuid();
let failureAuditEventId = createUuid();
for (
let attempts = 0;
failureAuditEventId === auditEventId && attempts < 3;
attempts += 1
) {
failureAuditEventId = createUuid();
}
return normalizeClusterRunManagementCommand({
schemaVersion: 1,
operation: 'run.cancellation.summary',
request: {
projectId,
requestId,
auditEventId,
failureAuditEventId,
body: { schema: RUN_CANCELLATION_DISPATCH_SUMMARY_REQUEST_SCHEMA },
},
}) as Readonly<ClusterRunManagementCancellationSummaryCommand>;
}
export function projectRunCancellationStatus(
result: Readonly<ClusterRunManagementClientResult>,
): Readonly<RunCancellationStatusObservation> {
if (result.result.operation !== 'run.cancellation.summary') {
throw new TypeError('Run cancellation status requires a summary result');
}
const summary = result.result.summary;
const severity: RunCancellationStatusSeverity =
summary.assessment === 'clear'
? 'ok'
: summary.assessment === 'converging'
? 'warning'
: 'critical';
const exitCode: RunCancellationStatusExitCode =
severity === 'ok' ? 0 : severity === 'warning' ? 10 : 20;
return Object.freeze({
schemaVersion: 1,
schema: RUN_CANCELLATION_STATUS_SCHEMA,
component: 'qinglong3-run-management-client',
event: 'cancellation_status_observed',
requestId: result.requestId,
projectId: summary.projectId,
observedAtMs: summary.observedAtMs,
assessment: summary.assessment,
operatorAction: summary.operatorAction,
severity,
exitCode,
dispatches: summary.dispatches,
signals: summary.signals,
blockingResults: summary.blockingResults,
...(summary.oldestBlockedAtMs === undefined
? {}
: { oldestBlockedAtMs: summary.oldestBlockedAtMs }),
});
}
function label(value: string): string {
return value.replaceAll('_', ' ').toUpperCase();
}
export function formatRunCancellationStatusCard(
status: Readonly<RunCancellationStatusObservation>,
): string {
const dispatch = status.dispatches;
const blocking = status.blockingResults;
return [
'QingLong 3.0 / Cancellation Availability',
`PROJECT ${status.projectId}`,
`ASSESSMENT ${label(status.assessment)}`,
`ACTION ${label(status.operatorAction)}`,
`ALERT ${status.severity.toUpperCase()} (exit ${status.exitCode})`,
`OBSERVED ${new Date(status.observedAtMs).toISOString()}`,
`DISPATCHES total=${dispatch.total} pending=${dispatch.pending} leased=${dispatch.leased} retry_wait=${dispatch.retryWait} dispatched=${dispatch.dispatched} blocked=${dispatch.blocked}`,
`SIGNALS due=${status.signals.due} expired_lease=${status.signals.expiredLease}`,
`BLOCKING identity_mismatch=${blocking.identityMismatch} pid_mismatch=${blocking.pidMismatch} unsupported=${blocking.unsupported} invalid=${blocking.invalid}`,
`OLDEST_BLOCK ${
status.oldestBlockedAtMs === undefined
? '-'
: new Date(status.oldestBlockedAtMs).toISOString()
}`,
`REQUEST ${status.requestId}`,
].join('\n');
}
@@ -14,6 +14,7 @@ import {
import {
ClusterPluginPackageManagementClientRequestError,
executeClusterAuthenticatedManagementClient,
type ClusterAuthenticatedManagementCommandExecution,
type ClusterAuthenticatedManagementClientResult,
type ClusterPluginPackageManagementClientConnectionOptions,
type ClusterPluginPackageManagementClientPaths,
@@ -35,6 +36,8 @@ export type ClusterRunManagementClientConnectionOptions =
ClusterPluginPackageManagementClientConnectionOptions;
export type ClusterRunManagementClientResult =
ClusterAuthenticatedManagementClientResult<ClusterRunManagementTransportResult>;
export type ClusterRunManagementCommandExecution =
ClusterAuthenticatedManagementCommandExecution<ClusterRunManagementCommand>;
function invalid(): never {
throw new ClusterPluginPackageManagementClientRequestError();
@@ -427,3 +430,14 @@ export function executeClusterRunManagementClient(
connectionOptions,
);
}
export function executeClusterRunManagementCommand(
execution: ClusterRunManagementCommandExecution,
connectionOptions?: ClusterRunManagementClientConnectionOptions,
): Promise<Readonly<ClusterRunManagementClientResult>> {
return executeClusterAuthenticatedManagementClient(
execution,
PROTOCOL,
connectionOptions,
);
}
@@ -1,16 +1,73 @@
#!/usr/bin/env node
import { ClusterPluginPackageManagementClientRemoteError } from '../management-support/pluginPackageManagementClient';
import { executeClusterRunManagementClient } from './runManagementClient';
import {
executeClusterRunManagementClient,
executeClusterRunManagementCommand,
} from './runManagementClient';
import {
createRunCancellationStatusCommand,
formatRunCancellationStatusCard,
projectRunCancellationStatus,
} from './runCancellationStatus';
const USAGE =
'Usage: ql3-run-client --config=/absolute/client.json --command=/absolute/command.json --assertion=/absolute/assertion.jwt';
const USAGE = [
'Usage: ql3-run-client --config=/absolute/client.json --command=/absolute/command.json --assertion=/absolute/assertion.jwt',
' ql3-run-client status --config=/absolute/client.json --assertion=/absolute/assertion.jwt --project=PROJECT [--format=text|json]',
'',
'Status exit codes: 0=clear, 10=converging, 20=attention_required.',
].join('\n');
const PROJECT_ID = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$/;
function argumentsFrom(argv: readonly string[]): Readonly<{
configFile: string;
commandFile: string;
assertionFile: string;
}> | null {
type RunManagementClientArguments =
| Readonly<{
kind: 'command';
configFile: string;
commandFile: string;
assertionFile: string;
}>
| Readonly<{
kind: 'status';
configFile: string;
assertionFile: string;
projectId: string;
format: 'text' | 'json';
}>;
function argumentsFrom(
argv: readonly string[],
): Readonly<RunManagementClientArguments> | null {
const statusCount = argv.filter((argument) => argument === 'status').length;
if (statusCount > 0) {
if (statusCount !== 1 || argv.length < 4 || argv.length > 5) return null;
const values = new Map<string, string>();
for (const argument of argv) {
if (argument === 'status') continue;
const match = /^--(config|assertion|project|format)=(.+)$/.exec(argument);
if (!match || values.has(match[1]!)) return null;
values.set(match[1]!, match[2]!);
}
if (
!values.has('config') ||
!values.get('config')!.startsWith('/') ||
!values.has('assertion') ||
!values.get('assertion')!.startsWith('/') ||
!values.has('project') ||
!PROJECT_ID.test(values.get('project')!) ||
(values.has('format') &&
values.get('format') !== 'text' &&
values.get('format') !== 'json')
) {
return null;
}
return Object.freeze({
kind: 'status',
configFile: values.get('config')!,
assertionFile: values.get('assertion')!,
projectId: values.get('project')!,
format: (values.get('format') ?? 'text') as 'text' | 'json',
});
}
if (argv.length !== 3) return null;
const values = new Map<string, string>();
for (const argument of argv) {
@@ -18,8 +75,14 @@ function argumentsFrom(argv: readonly string[]): Readonly<{
if (!match || values.has(match[1]!)) return null;
values.set(match[1]!, match[2]!);
}
if (!values.has('config') || !values.has('command') || !values.has('assertion')) return null;
if (
!values.has('config') ||
!values.has('command') ||
!values.has('assertion')
)
return null;
return Object.freeze({
kind: 'command',
configFile: values.get('config')!,
commandFile: values.get('command')!,
assertionFile: values.get('assertion')!,
@@ -32,13 +95,18 @@ function failureFact(error: unknown): Readonly<Record<string, unknown>> {
schemaVersion: 1,
component: 'qinglong3-run-management-client',
event: 'command_failed',
code: typeof candidate?.code === 'string' ? candidate.code : 'QL3_RUN_MANAGEMENT_CLIENT_FAILED',
code:
typeof candidate?.code === 'string'
? candidate.code
: 'QL3_RUN_MANAGEMENT_CLIENT_FAILED',
...(error instanceof ClusterPluginPackageManagementClientRemoteError
? {
statusCode: error.statusCode,
responseCode: error.responseCode,
requestId: error.requestId,
...(error.retryAfterSeconds === null ? {} : { retryAfterSeconds: error.retryAfterSeconds }),
...(error.retryAfterSeconds === null
? {}
: { retryAfterSeconds: error.retryAfterSeconds }),
}
: {}),
});
@@ -51,13 +119,43 @@ async function run(argv: readonly string[]): Promise<void> {
}
const paths = argumentsFrom(argv);
if (!paths) {
process.stderr.write(`${JSON.stringify({ schemaVersion: 1, component: 'qinglong3-run-management-client', event: 'usage_invalid', code: 'QL3_RUN_MANAGEMENT_CLIENT_USAGE_INVALID' })}\n`);
process.stderr.write(
`${JSON.stringify({
schemaVersion: 1,
component: 'qinglong3-run-management-client',
event: 'usage_invalid',
code: 'QL3_RUN_MANAGEMENT_CLIENT_USAGE_INVALID',
})}\n`,
);
process.exitCode = 64;
return;
}
try {
if (paths.kind === 'status') {
const result = await executeClusterRunManagementCommand({
configFile: paths.configFile,
assertionFile: paths.assertionFile,
command: createRunCancellationStatusCommand(paths.projectId),
});
const status = projectRunCancellationStatus(result);
process.stdout.write(
paths.format === 'json'
? `${JSON.stringify(status)}\n`
: `${formatRunCancellationStatusCard(status)}\n`,
);
process.exitCode = status.exitCode;
return;
}
const result = await executeClusterRunManagementClient(paths);
process.stdout.write(`${JSON.stringify({ schemaVersion: 1, component: 'qinglong3-run-management-client', event: 'command_completed', requestId: result.requestId, result: result.result })}\n`);
process.stdout.write(
`${JSON.stringify({
schemaVersion: 1,
component: 'qinglong3-run-management-client',
event: 'command_completed',
requestId: result.requestId,
result: result.result,
})}\n`,
);
} catch (error) {
process.stderr.write(`${JSON.stringify(failureFact(error))}\n`);
process.exitCode = 1;
@@ -375,7 +375,7 @@ test('catalog exposes only reviewed product entrypoints from the same package',
test('help and version are bounded installation-derived product facts', () => {
const help = qingLong3ClusterProductHelp();
assert.match(help, /^Usage: ql3-cluster-admin <command> \[arguments\]/);
assert.match(help, /\n run\s+retry or stop Runs/);
assert.match(help, /\n run\s+observe, retry or stop Runs/);
assert.match(help, /\n copilot\s+diagnose, inspect, read or cancel Runs/);
assert.match(help, /\n copilot-mcp\s+serve the bounded Cluster Copilot MCP/);
assert.match(help, /\n copilot-console\s+open the loopback-only read-only/);
@@ -0,0 +1,121 @@
'use strict';
const assert = require('node:assert/strict');
const test = require('node:test');
const {
createRunCancellationStatusCommand,
formatRunCancellationStatusCard,
projectRunCancellationStatus,
} = require('../dist/run-management/runCancellationStatus.js');
const uuids = [
'019f9500-0000-4000-8000-000000000001',
'019f9500-0000-4000-8000-000000000002',
'019f9500-0000-4000-8000-000000000003',
];
function result(assessment) {
const blocked = assessment === 'attention_required' ? 1 : 0;
const pending = assessment === 'converging' ? 1 : 0;
return {
schemaVersion: 1,
requestId: 'request-summary-1',
result: {
schemaVersion: 1,
operation: 'run.cancellation.summary',
summary: {
schema: 'qinglong/run-cancellation-dispatch-summary@v1',
projectId: 'project-1',
observedAtMs: 1_700_000_000_000,
assessment,
operatorAction:
assessment === 'clear'
? 'none'
: assessment === 'converging'
? 'wait'
: 'inspect',
dispatches: {
total: blocked + pending,
pending,
leased: 0,
retryWait: 0,
dispatched: 0,
blocked,
},
signals: { due: pending, expiredLease: 0 },
blockingResults: {
identityMismatch: blocked,
pidMismatch: 0,
unsupported: 0,
invalid: 0,
},
...(blocked === 0 ? {} : { oldestBlockedAtMs: 1_699_999_999_000 }),
},
},
};
}
test('builds one exact Project summary command without a command file', () => {
let index = 0;
assert.deepEqual(
createRunCancellationStatusCommand('project-1', () => uuids[index++]),
{
schemaVersion: 1,
operation: 'run.cancellation.summary',
request: {
projectId: 'project-1',
requestId: uuids[0],
auditEventId: uuids[1],
failureAuditEventId: uuids[2],
body: {
schema: 'qinglong/run-cancellation-dispatch-summary-request@v1',
},
},
},
);
});
test('maps clear, converging and attention assessments to stable alert exits', () => {
const cases = [
['clear', 'ok', 0],
['converging', 'warning', 10],
['attention_required', 'critical', 20],
];
for (const [assessment, severity, exitCode] of cases) {
const status = projectRunCancellationStatus(result(assessment));
assert.equal(status.schema, 'qinglong/run-cancellation-status@v1');
assert.equal(status.assessment, assessment);
assert.equal(status.severity, severity);
assert.equal(status.exitCode, exitCode);
assert.equal(status.projectId, 'project-1');
assert.equal(Object.hasOwn(status, 'oldestBlockedAtMs'), exitCode === 20);
}
});
test('renders a deterministic low-sensitive operator card', () => {
const card = formatRunCancellationStatusCard(
projectRunCancellationStatus(result('attention_required')),
);
assert.match(card, /^QingLong 3\.0 \/ Cancellation Availability\n/);
assert.match(card, /ASSESSMENT ATTENTION REQUIRED/);
assert.match(card, /ALERT CRITICAL \(exit 20\)/);
assert.match(card, /DISPATCHES total=1 .* blocked=1/);
assert.match(card, /BLOCKING identity_mismatch=1/);
assert.doesNotMatch(
card,
/runId|attemptId|leaseOwner|leaseToken|command|environment|secret/i,
);
});
test('refuses to project a non-summary management result', () => {
assert.throws(
() =>
projectRunCancellationStatus({
schemaVersion: 1,
requestId: 'request-1',
result: { schemaVersion: 1, operation: 'run.stop', stop: {} },
}),
/requires a summary result/,
);
});
@@ -15,6 +15,7 @@ const { afterEach, test } = require('node:test');
const {
executeClusterRunManagementClient,
executeClusterRunManagementCommand,
validateClusterRunManagementClientResult,
} = require('@qinglong/cluster-admin/run-management-client');
const {
@@ -217,6 +218,32 @@ test('accepts only the exact Run route before opening one mTLS connection', asyn
assert.equal(connects, 1);
});
test('accepts a normalized in-memory summary command without a command file', async () => {
const paths = clientFiles();
let connects = 0;
await assert.rejects(
executeClusterRunManagementCommand(
{
configFile: paths.configFile,
assertionFile: paths.assertionFile,
command: summaryCommand,
},
{
async connect(target) {
connects += 1;
assert.deepEqual(target, {
hostname: 'run.example.test',
port: 8448,
});
throw new Error('expected-connect-stop');
},
},
),
{ code: 'QL3_PLUGIN_PACKAGE_MANAGEMENT_CLIENT_REQUEST_FAILED' },
);
assert.equal(connects, 1);
});
test('rejects response target, execution placement and shape drift', () => {
for (const candidate of [
response({ projectId: 'project-2' }),
@@ -0,0 +1,217 @@
'use strict';
const assert = require('node:assert/strict');
const { spawn } = require('node:child_process');
const fs = require('node:fs');
const { createServer } = require('node:https');
const os = require('node:os');
const path = require('node:path');
const test = require('node:test');
const packageRoot = path.resolve(__dirname, '..');
const cliPath = path.join(
packageRoot,
'dist',
'run-management',
'runManagementClientCli.js',
);
const tlsFixture = path.resolve(
packageRoot,
'../ql3-cluster-control/test/fixtures/mtls',
);
function privateFile(directory, name, contents) {
const filePath = path.join(directory, name);
fs.writeFileSync(filePath, contents, { mode: 0o600 });
return fs.realpathSync(filePath);
}
function runCli(args) {
return new Promise((resolve, reject) => {
const child = spawn(process.execPath, [cliPath, ...args], {
cwd: packageRoot,
stdio: ['ignore', 'pipe', 'pipe'],
});
const stdout = [];
const stderr = [];
child.stdout.on('data', (chunk) => stdout.push(chunk));
child.stderr.on('data', (chunk) => stderr.push(chunk));
child.once('error', reject);
child.once('close', (status, signal) => {
resolve({
status,
signal,
stdout: Buffer.concat(stdout).toString('utf8'),
stderr: Buffer.concat(stderr).toString('utf8'),
});
});
});
}
test('status mode calls only the summary operation and emits an alert exit', async (t) => {
const directory = fs.realpathSync(
fs.mkdtempSync(path.join(os.tmpdir(), 'ql3-run-status-cli-')),
);
t.after(() => fs.rmSync(directory, { recursive: true, force: true }));
const caFile = privateFile(
directory,
'ca.pem',
fs.readFileSync(path.join(tlsFixture, 'ca-cert.pem')),
);
const clientCertificateFile = privateFile(
directory,
'client.crt',
fs.readFileSync(path.join(tlsFixture, 'client-cert.pem')),
);
const clientPrivateKeyFile = privateFile(
directory,
'client.key',
fs.readFileSync(path.join(tlsFixture, 'client-key.pem')),
);
const requests = [];
const server = createServer(
{
key: fs.readFileSync(path.join(tlsFixture, 'server-key.pem')),
cert: fs.readFileSync(path.join(tlsFixture, 'server-cert.pem')),
ca: fs.readFileSync(path.join(tlsFixture, 'ca-cert.pem')),
requestCert: true,
rejectUnauthorized: true,
minVersion: 'TLSv1.3',
maxVersion: 'TLSv1.3',
},
(request, response) => {
const chunks = [];
request.on('data', (chunk) => chunks.push(chunk));
request.on('end', () => {
const command = JSON.parse(Buffer.concat(chunks).toString('utf8'));
requests.push({
method: request.method,
path: request.url,
authorization: request.headers.authorization,
authorized: request.socket.authorized,
command,
});
const bytes = Buffer.from(
JSON.stringify({
schemaVersion: 1,
requestId: command.request.requestId,
result: {
schemaVersion: 1,
operation: 'run.cancellation.summary',
summary: {
schema: 'qinglong/run-cancellation-dispatch-summary@v1',
projectId: 'project-1',
observedAtMs: 1_700_000_000_000,
assessment: 'attention_required',
operatorAction: 'inspect',
dispatches: {
total: 1,
pending: 0,
leased: 0,
retryWait: 0,
dispatched: 0,
blocked: 1,
},
signals: { due: 0, expiredLease: 0 },
blockingResults: {
identityMismatch: 1,
pidMismatch: 0,
unsupported: 0,
invalid: 0,
},
oldestBlockedAtMs: 1_699_999_999_000,
},
},
}),
'utf8',
);
response.writeHead(200, {
'content-type': 'application/json; charset=utf-8',
'content-length': String(bytes.byteLength),
});
response.end(bytes);
});
},
);
await new Promise((resolve, reject) => {
server.once('error', reject);
server.listen(0, '127.0.0.1', resolve);
});
t.after(
() =>
new Promise((resolve) => {
server.close(() => resolve());
}),
);
const address = server.address();
assert.notEqual(address, null);
assert.notEqual(typeof address, 'string');
const configFile = privateFile(
directory,
'client.json',
JSON.stringify({
schemaVersion: 1,
endpoint: `https://localhost:${address.port}/api/v3/runs/management`,
servername: 'localhost',
caFile,
clientCertificateFile,
clientPrivateKeyFile,
requestTimeoutMs: 2_000,
}),
);
const assertionFile = privateFile(
directory,
'assertion.jwt',
'eyJhbGciOiJFZERTQSJ9.eyJzdWIiOiJvcGVyYXRvci0xIn0.c2lnbmF0dXJl',
);
const result = await runCli([
`--config=${configFile}`,
'status',
`--assertion=${assertionFile}`,
'--project=project-1',
'--format=json',
]);
assert.equal(result.status, 20, result.stderr);
assert.equal(result.signal, null);
assert.equal(result.stderr, '');
const output = JSON.parse(result.stdout);
assert.equal(output.schema, 'qinglong/run-cancellation-status@v1');
assert.equal(output.assessment, 'attention_required');
assert.equal(output.severity, 'critical');
assert.equal(output.exitCode, 20);
assert.equal(output.dispatches.blocked, 1);
assert.equal(requests.length, 1);
assert.equal(requests[0].method, 'POST');
assert.equal(requests[0].path, '/api/v3/runs/management');
assert.equal(requests[0].authorized, true);
assert.match(requests[0].authorization, /^Bearer [A-Za-z0-9_-]+\./);
assert.equal(requests[0].command.operation, 'run.cancellation.summary');
assert.deepEqual(requests[0].command.request.body, {
schema: 'qinglong/run-cancellation-dispatch-summary-request@v1',
});
assert.equal(Object.hasOwn(requests[0].command.request, 'runId'), false);
});
test('help documents status routing and invalid projects fail before I/O', async () => {
const help = await runCli(['--help']);
assert.equal(help.status, 0);
assert.match(help.stdout, /ql3-run-client status/);
assert.match(help.stdout, /0=clear, 10=converging, 20=attention_required/);
const rejected = await runCli([
'status',
'--config=/private/client.json',
'--assertion=/private/assertion.jwt',
'--project=../escape',
]);
assert.equal(rejected.status, 64);
assert.equal(rejected.stdout, '');
assert.deepEqual(JSON.parse(rejected.stderr), {
schemaVersion: 1,
component: 'qinglong3-run-management-client',
event: 'usage_invalid',
code: 'QL3_RUN_MANAGEMENT_CLIENT_USAGE_INVALID',
});
});
+2 -2
View File
@@ -340,10 +340,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
rootSourceFileRoles: clusterAdmin.rootSourceFileRoles,
},
{
sourceFiles: 122,
sourceFiles: 123,
rootSourceFiles: 1,
rootSourceLines: 61,
nestedSourceFiles: 121,
nestedSourceFiles: 122,
rootSourceFileRoles: {
'modelInvocationMigrationCli.ts': 'binary_entry',
},