mirror of
https://github.com/whyour/qinglong.git
synced 2026-09-20 16:07:11 +08:00
feat(ql3): version worker support tier admission
This commit is contained in:
whyour
@@ -11,6 +11,25 @@
|
||||
|
||||
最新增量证据（2026-08-20）：
|
||||
|
||||
- D-372/ADR-0465（已接受）：把 D-371 的发布架构分层推进到 Remote Worker 的实际注册与调度边界。Worker canonical
|
||||
capability 现在必须声明 `protocolVersion` 与 `supportTier`，架构/Tier 必须与根 release identity v2 一致，Node
|
||||
`x64/ppc64/ia32/arm` 会归一化到发布词汇并与实际 Worker 进程复验。普通 Task 即使未声明 Placement 也只匹配 Tier 1
|
||||
与控制面 `>=1.0.0 <2.0.0`；candidate、experimental、legacy-only 必须由新 immutable revision 显式选择，且任务不能
|
||||
放宽控制面协议范围。新默认只在调度判定时应用，不写回历史 execution revision，因而不改既有 digest。Session register
|
||||
在 Repository 前拒绝未版本化、非 canonical 或 Tier 漂移快照；生产 Worker 同时必须声明 `remote-worker`，关闭此前
|
||||
`local_process` 节点可注册却永远无法领取任务的假健康状态。本切片没有新增 package、dependency、数据库对象、服务、
|
||||
端口、timer 或 Edge/Standalone 常驻闭包；legacy-only 只建立未来受限 adapter 的准入槽位，不表示 2.x adapter、
|
||||
Plugin/Tool/DB 权限或实机支持已经完成。runtime-core 全量 `574/574`、Worker Runtime 全量 `134/134`；完整 backend
|
||||
工作区为 `1,503 total / 1,501 pass / 2 conditional skip / 0 fail`（含一条不会提交的既有用户测试），18-package clean
|
||||
build/逐包测试退出 0。PostgreSQL 18.6 arm64 HA 为 `146/146`、timeline `1→2`，报告 SHA-256 为
|
||||
`e94d48f0bbe5d5af6f6fd18f94572ead40d7155c38d194dce958712d968efeed`；真实 Linux Worker + PostgreSQL 门同时通过
|
||||
TLS 1.3 mTLS、证书/credential rotation、同 Session、remote execution、Artifact completion 与数据库最小权限。
|
||||
本机 arm64 Worker edge/node active RSS 为 `72,073,216 / 71,991,296` bytes、max RSS 为
|
||||
`72,466,432 / 72,417,280` bytes，仅作为回归基线，不构成 ARM32 路由器实机证明。七项架构/部署审计与 14 档
|
||||
Local artifact audit 全部 compatible；基础 Edge/Standalone 为 `2,598,669 / 2,598,747` bytes，Application+AI 为
|
||||
`4,501,822 / 4,501,954` bytes，MCP 为 `7,324,601 / 7,324,709` bytes，workspace 仍为 18 packages、无
|
||||
single/shallow package，也没有新增外部 dependency。
|
||||
|
||||
- D-371/ADR-0464（已接受并机器化）：接受 ADR-0006 的 Node 24 与多架构分层，并把根 `ql3-release.json` 升级为
|
||||
`qinglong/release-identity@v2` 唯一事实源。3.0 Tier 1 精确为 `amd64`、`arm64`；`ppc64le`、`s390x` 是通过固定
|
||||
Node 24 与同等级原生门禁前不可发布的候选；`arm/v7` 在缺少 owner、可重复 toolchain 与设备报告时为 experimental blocked；
|
||||
@@ -6430,7 +6449,11 @@ D-371/ADR-0464 已接受 D-14 与 D-16 的一致解，当前分层为：
|
||||
| legacy-only | ARMv6、386 | 继续使用显式 2.x legacy line；不能把 Node 20/22 镜像标为满足 D-16 的完整 3.0 |
|
||||
|
||||
这个分层不等于放弃小设备。2.x legacy line 继续承担约定兼容窗口，旧设备参与 3.0 集群的受限 Worker 协议仍是
|
||||
独立后续决策；但不能因此把没有受支持 Node 24、数据库恢复与原生资源证据的设备标成完整 3.0 ql-core。若
|
||||
独立后续决策；D-372/ADR-0465 已先把 support Tier 与 Worker protocol v1 变成 Session register 和 Scheduler 的
|
||||
机器准入边界。默认任务只匹配 Tier 1；candidate、experimental 和 legacy-only 需要新的 immutable Placement 显式
|
||||
选择，且 capability architecture/Tier 必须与本表一致。该能力只为未来 legacy adapter 保留受限调度槽位，不表示
|
||||
ARMv6/386 的 2.x 进程已经能直接连接 3.0，也不授予 Plugin Host、任意 Tool 或控制面数据库权力。但不能因此把没有
|
||||
受支持 Node 24、数据库恢复与原生资源证据的设备标成完整 3.0 ql-core。若
|
||||
Maintainers 要求 ARMv6、ARMv7 或 386 成为 3.0 正式支持项，必须先修改 D-16 或长期维护对应 Node 24 构建链，
|
||||
升级 release identity schema，并通过与现有 Tier 1 同等级的发布门禁。
|
||||
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
# ADR-0465：版本化 Worker 支持等级准入
|
||||
|
||||
- 状态：Accepted
|
||||
- 日期：2026-08-20
|
||||
- 关联 RFC：QL-RFC-0001 D-372、D-14、D-16、D-107
|
||||
- 关联 ADR：ADR-0006、ADR-0012、ADR-0108、ADR-0464
|
||||
- Amends：ADR-0012 的 Worker capability 与 Placement 契约
|
||||
|
||||
## 上下文
|
||||
|
||||
D-371 已把 QingLong 3.0 完整核心的架构支持分为 Tier 1、candidate、experimental blocked 与 legacy-only，
|
||||
但 Worker Session 快照仍只声明 architecture、executor、runtime 和资源。控制面无法判断节点使用哪个 Worker
|
||||
协议，也无法区分一个完整支持节点、候选节点与只允许受限兼容的旧设备。默认 Placement 因此只检查
|
||||
`remote-worker`，不能机器化执行 ADR-0006 要求的 support Tier 与 protocol version 边界。
|
||||
|
||||
审计还发现生产 Worker fixture 与两个资源/部署门只声明 `local_process`，而 Cluster execution revision 的默认
|
||||
Placement 精确要求 `remote-worker`。这种配置可以启动并注册，却永远不能领取任务；同时部分 Worker 使用 Node
|
||||
的 `x64` 名称，发布身份使用 OCI 的 `amd64`，架构词汇也会漂移。
|
||||
|
||||
3.0 尚未正式发布，可以在当前孵化分支升级 Worker Session capability schema；但已经持久化的 immutable
|
||||
execution revision 摘要不能因默认政策变化而被静默重写。
|
||||
|
||||
## 决策
|
||||
|
||||
1. 在既有 `@qinglong/runtime-core` 的 `remote-execution` 领域目录内增加
|
||||
`remoteWorkerCompatibility.ts`，不新增 workspace package。该文件拥有 Worker protocol、support Tier、发布架构
|
||||
词汇与 Node runtime architecture 映射；避免把一个小契约再次拆成单文件微包。
|
||||
2. Worker capability 现在必须精确包含 `protocolVersion` 与 `supportTier`。当前协议版本为 `1.0.0`，控制面支持
|
||||
范围为 `>=1.0.0 <2.0.0`。未知字段、缺失字段、非法 SemVer、未知 Tier 或非 canonical JSON 在 Session register
|
||||
进入 Repository 前失败关闭。
|
||||
3. 架构词汇与 `qinglong/release-identity@v2` 保持同一映射：Tier 1 为 `amd64/arm64`，candidate 为
|
||||
`ppc64le/s390x`，experimental 为 `arm/v7`，legacy-only 为 `arm/v6/386`。测试直接读取根 release identity
|
||||
比较映射，阻止两份政策静默漂移；capability 声明的 architecture 与 supportTier 不一致时拒绝。
|
||||
4. Node runtime 名称必须先归一化为发布名称：`x64→amd64`、`ppc64→ppc64le`、`ia32→386`，ARM32 必须有明确
|
||||
`arm_version=6|7`，未知或含糊架构拒绝。生产 Worker 配置把 capability architecture 与当前 Node 进程再次
|
||||
比较，不能通过文件把一台机器伪报成另一种架构。
|
||||
5. Placement `required` 新增可选 `supportTiers` 与 `protocolVersionRange`。未声明 `supportTiers` 的普通任务只匹配
|
||||
Tier 1；candidate、experimental 或 legacy-only 必须由新的 immutable Task revision 显式选择。无论任务是否
|
||||
声明范围，Worker protocol 都必须先满足控制面 v1 全局范围，任务不能放宽控制面兼容政策。
|
||||
6. Tier 1 与 protocol v1 的默认策略只在 `evaluateRemoteWorkerPlacement` 中执行，不写回
|
||||
`effectiveRemoteWorkerPlacement`。因此旧 execution revision 的 canonical content 与 digest 保持不变；只有新任务
|
||||
显式声明 support/protocol 要求时才产生新的 revision digest。
|
||||
7. 生产 Worker capability 必须包含 `remote-worker` executor。仅声明 `local_process` 的配置在启动读取阶段拒绝，
|
||||
不再允许“注册成功但永远领不到任务”的假健康状态。资源基准、PostgreSQL live contract 与 Kubernetes rollout
|
||||
fixture 一并迁移到同一契约。
|
||||
8. legacy-only 的显式 Placement 只建立调度准入槽位，不表示 2.x Worker adapter 已实现，也不授予 Plugin Host、
|
||||
任意 Tool、控制面数据库、管理 API 或 Secret 明文权力。未来 adapter 必须继续使用现有 mTLS identity、Session、
|
||||
Run Lease、bounded Pull 与 Artifact/Completion fence，并提交独立 ADR、EOL 和资源证据。
|
||||
9. 本变更不新增 dependency、服务、端口、timer、连接池、数据库表或后台扫描。兼容性检查只发生在 Worker 配置
|
||||
读取、Session register 和已有 Placement 计算上；禁用 Worker 的 Edge/Standalone 路径仍不读取配置或加载运行时。
|
||||
|
||||
## 升级与回滚
|
||||
|
||||
- 这是 Worker capability 的有意破坏性升级。滚动升级必须先 drain 旧 Worker，完成所有 control replica 升级，再
|
||||
以带 v1/Tier 的新 capability 建立 Session；不能让旧控制面副本与新旧 capability 长期混跑。
|
||||
- 旧 Session 行不做数据库重写。未替换记录会在 lease 到期后离线；新控制面不会把未版本化快照调度为可用节点。
|
||||
- 回滚时先 drain v1 Worker，再整体回滚 control replica 和 Worker artifact。不能只回滚一侧并继续领取新任务。
|
||||
- execution revision 无需 migration；本 ADR 特意不把新默认值物化到历史 Placement，回滚不会改变既有 digest。
|
||||
|
||||
## 被拒绝的替代方案
|
||||
|
||||
### 根据 architecture 在 UI 临时推断 Tier
|
||||
|
||||
拒绝。UI 推断不能约束 Scheduler，也不能证明 Worker 协议与实际 Node runtime；控制面副本仍可能把任务发给不兼容节点。
|
||||
|
||||
### 把 Tier 与 protocol 作为普通 label
|
||||
|
||||
拒绝。label 可由任务任意匹配、没有固定词汇或 SemVer 语义，也不能表达不可被 Task 放宽的全局协议范围。
|
||||
|
||||
### 把新的默认字段写入所有 execution revision
|
||||
|
||||
拒绝。这会改变已持久化 revision 的 canonical content 与 digest，违反 append-only immutable execution authority。
|
||||
|
||||
### 保留只声明 `local_process` 的 Worker
|
||||
|
||||
拒绝。Cluster Planner 要求 `remote-worker`；接受不可能命中的节点只会制造错误健康信号和无界排障成本。
|
||||
|
||||
### 立即实现一个完整 2.x legacy Worker
|
||||
|
||||
拒绝。身份、最小命令集、Secret、日志、Artifact、EOL 和 ARM32 资源预算需要独立设计与实机证据。本切片先关闭
|
||||
控制面会默认误调度的协议漏洞，不能用一个适配器名义扩大旧设备权限。
|
||||
|
||||
## 验证与证据
|
||||
|
||||
- runtime-core 聚焦测试覆盖默认 Tier 1、显式 legacy-only、全局 protocol v1、任务范围、未知/重复 Tier、非法
|
||||
SemVer、未版本化快照、architecture/Tier 漂移、release identity 映射和历史 non-canonical snapshot。
|
||||
- Worker Runtime 聚焦测试覆盖 Node runtime architecture 绑定、`remote-worker` executor 必需条件、禁用路径零读取、
|
||||
edge/node 配置和 Session canonical register。
|
||||
- Cluster ingress 聚焦测试证明未版本化 capability 在 Repository 调用前返回固定 400，Dispatcher 只为匹配 Tier/
|
||||
protocol 的 Session claim offer。
|
||||
- runtime-core 全量 `574/574`、Worker Runtime 全量 `134/134`、完整 backend 工作区
|
||||
`1,503 total / 1,501 pass / 2 conditional skip / 0 fail`，18-package clean build 与逐包测试退出 0。backend 总数包含
|
||||
一条不会进入本阶段提交的既有用户测试。
|
||||
- PostgreSQL 18.6 arm64 HA `146/146`、timeline `1→2`，报告 SHA-256 为
|
||||
`e94d48f0bbe5d5af6f6fd18f94572ead40d7155c38d194dce958712d968efeed`。真实 Linux Worker + PostgreSQL 门进一步证明
|
||||
TLS 1.3 mTLS、证书与 credential rotation、同 Session 保持、remote execution、Artifact completion 与最小数据库权限；
|
||||
迁移前角色 fixture 和 cancellation dispatch owner 的既有漂移在本切片一并修复。
|
||||
- 本机 arm64 Worker 回归基准通过：edge active/max RSS 为 `72,073,216 / 72,466,432` bytes，node 为
|
||||
`71,991,296 / 72,417,280` bytes；它们是本地回归基线，不是 ARM32 路由器实机支持证明。
|
||||
- release version、package boundary、cluster dependency、Edge import、cluster deployment、Worker deployment 与 image release
|
||||
七项审计全部 compatible。14 档 Local artifact audit 全部 compatible；基础 Edge/Standalone 为
|
||||
`2,598,669 / 2,598,747` bytes，Application+AI 为 `4,501,822 / 4,501,954` bytes，MCP 为
|
||||
`7,324,601 / 7,324,709` bytes，均保留充足预算且没有新增 package 或外部 dependency。
|
||||
|
||||
## 后续边界
|
||||
|
||||
- 设计真正的 legacy Worker adapter：固定最小 command schema、无 Plugin/Tool/DB authority、显式 EOL、可撤销身份、
|
||||
资源预算以及 ARMv6/386/ARMv7 实机证据。
|
||||
- 在 Worker 管理只读面显示 architecture、supportTier、protocolVersion、Session/generation、runtime 与 capacity；
|
||||
operator 必须能区分“不兼容”“显式 legacy”“离线”和“容量为零”。
|
||||
- protocol v2 必须采用并行兼容窗口或新 endpoint/schema，并先证明混合 control replica 的 rollout/rollback；不能只
|
||||
修改范围字符串让旧实现接收新语义。
|
||||
@@ -465,6 +465,10 @@
|
||||
| [ADR-0459](./ADR-0459-project-scoped-cancellation-availability-summary.md) | Project-scoped Cancellation 可用性汇总 | Accepted |
|
||||
| [ADR-0460](./ADR-0460-one-shot-cancellation-status-product-entry.md) | 一次性 Cancellation 状态产品入口 | Accepted |
|
||||
| [ADR-0461](./ADR-0461-project-scoped-blocked-cancellation-keyset.md) | Project-scoped Blocked Cancellation 固定键集分页 | Accepted |
|
||||
| [ADR-0462](./ADR-0462-optional-console-run-management-drilldown.md) | 可选 Console Run Management Drill-down | Accepted |
|
||||
| [ADR-0463](./ADR-0463-native-console-capacity-and-assertion-lifecycle-evidence.md) | 原生双架构 Console 容量与 Assertion 生命周期证据 | Accepted（首份原生 CI 双架构报告待实际运行） |
|
||||
| [ADR-0464](./ADR-0464-machine-enforced-node24-architecture-support-tiers.md) | 机器化 Node 24 架构支持分层 | Accepted |
|
||||
| [ADR-0465](./ADR-0465-versioned-worker-support-tier-admission.md) | 版本化 Worker 支持等级准入 | Accepted |
|
||||
|
||||
## 规则
|
||||
|
||||
|
||||
@@ -527,6 +527,8 @@ if (!migrationConnectionString || !runtimeConnectionString) {
|
||||
const capabilities = canonicalRemoteWorkerCapabilities({
|
||||
architecture: 'arm64',
|
||||
executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0',
|
||||
supportTier: 'tier1',
|
||||
labels: { region: 'cn-east' },
|
||||
capacity: { memoryBytes: 536870912 },
|
||||
});
|
||||
@@ -1227,7 +1229,7 @@ if (!migrationConnectionString || !runtimeConnectionString) {
|
||||
);
|
||||
const sessionId = '018f0000-0000-7000-8000-000000000021';
|
||||
const capabilitiesJson =
|
||||
'{"architecture":"arm64","executors":["remote-worker"]}';
|
||||
'{"architecture":"arm64","executors":["remote-worker"],"protocolVersion":"1.0.0","supportTier":"tier1"}';
|
||||
const response = await workerRequest(
|
||||
application.address,
|
||||
`/api/v3/worker-ingress/workers/worker-http-a/sessions/${sessionId}/register`,
|
||||
|
||||
@@ -43,6 +43,7 @@ function revision() {
|
||||
function worker() {
|
||||
const capabilities = canonicalRemoteWorkerCapabilities({
|
||||
architecture: 'arm64', executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0', supportTier: 'tier1',
|
||||
});
|
||||
return {
|
||||
workerId: 'edge-1', sessionId: SESSION, generation: 2,
|
||||
|
||||
@@ -36,6 +36,10 @@ const {
|
||||
} = require('@qinglong/runtime-core/remote-worker-lease-control');
|
||||
|
||||
const SESSION_ID = '018f5c64-9b9d-7f1a-8c2d-1234567890ac';
|
||||
const CAPABILITIES_JSON =
|
||||
'{"architecture":"arm64","executors":["remote-worker"],"protocolVersion":"1.0.0","supportTier":"tier1"}';
|
||||
const CAPABILITIES_HASH =
|
||||
'b3d79017d91c477ffdf4a4dcc4ce9135ca053c921922ce0221920f905d8a2aa4';
|
||||
|
||||
function metadata(operation, workerId = 'edge-1') {
|
||||
return {
|
||||
@@ -80,9 +84,8 @@ function fixture(overrides = {}) {
|
||||
generation: command.generation,
|
||||
version: command.expectedVersion + 1,
|
||||
status: 'online',
|
||||
capabilitiesJson: '{}',
|
||||
capabilitiesHash:
|
||||
'44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a',
|
||||
capabilitiesJson: CAPABILITIES_JSON,
|
||||
capabilitiesHash: CAPABILITIES_HASH,
|
||||
maxConcurrentRuns: 1,
|
||||
availableSlots: command.availableSlots,
|
||||
registeredAtMs: 1,
|
||||
@@ -150,9 +153,8 @@ test('uses exact versioned register and transition contracts', async () => {
|
||||
generation: 1,
|
||||
version: 0,
|
||||
status: 'online',
|
||||
capabilitiesJson: '{}',
|
||||
capabilitiesHash:
|
||||
'44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a',
|
||||
capabilitiesJson: CAPABILITIES_JSON,
|
||||
capabilitiesHash: CAPABILITIES_HASH,
|
||||
maxConcurrentRuns: 1,
|
||||
availableSlots: 1,
|
||||
registeredAtMs: 1,
|
||||
@@ -188,7 +190,7 @@ test('uses exact versioned register and transition contracts', async () => {
|
||||
await context.pipeline.prepare(metadata('register'))
|
||||
).handle({
|
||||
schema: WORKER_SESSION_REGISTER_SCHEMA,
|
||||
capabilitiesJson: '{}',
|
||||
capabilitiesJson: CAPABILITIES_JSON,
|
||||
capabilitiesHash: baseRecord.capabilitiesHash,
|
||||
maxConcurrentRuns: 1,
|
||||
availableSlots: 1,
|
||||
@@ -208,6 +210,31 @@ test('uses exact versioned register and transition contracts', async () => {
|
||||
assert.equal(transition.body.status, 'draining');
|
||||
});
|
||||
|
||||
test('rejects unversioned Worker capabilities before repository registration', async () => {
|
||||
let registered = false;
|
||||
const context = fixture({
|
||||
workers: {
|
||||
async register() { registered = true; throw new Error('must not register'); },
|
||||
async heartbeatAuthenticated() { throw new Error('not used'); },
|
||||
async transitionAuthenticated() { throw new Error('not used'); },
|
||||
async findById() { return null; },
|
||||
async listAvailable() { throw new Error('not used'); },
|
||||
},
|
||||
});
|
||||
const prepared = await context.pipeline.prepare(metadata('register'));
|
||||
await assert.rejects(prepared.handle({
|
||||
schema: WORKER_SESSION_REGISTER_SCHEMA,
|
||||
capabilitiesJson: '{}',
|
||||
capabilitiesHash:
|
||||
'44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a',
|
||||
maxConcurrentRuns: 1,
|
||||
availableSlots: 1,
|
||||
leaseDurationMs: 30_000,
|
||||
}), (error) =>
|
||||
error.statusCode === 400 && error.code === 'invalid_worker_request');
|
||||
assert.equal(registered, false);
|
||||
});
|
||||
|
||||
test('binds the credential Worker to the path and rejects before body access', async () => {
|
||||
const { pipeline, events } = fixture();
|
||||
await assert.rejects(
|
||||
|
||||
@@ -5851,7 +5851,7 @@ if (!migrationConnectionString) {
|
||||
assert.equal(published.state, 'published');
|
||||
|
||||
const capabilitiesJson =
|
||||
'{"architecture":"arm64","executors":["remote-worker"]}';
|
||||
'{"architecture":"arm64","executors":["remote-worker"],"protocolVersion":"1.0.0","supportTier":"tier1"}';
|
||||
const capabilitiesHash = createHash('sha256')
|
||||
.update(capabilitiesJson)
|
||||
.digest('hex');
|
||||
@@ -6168,7 +6168,7 @@ if (!migrationConnectionString) {
|
||||
runtimeDatabase.pool,
|
||||
);
|
||||
const capabilitiesJson =
|
||||
'{"architecture":"arm64","executors":["remote-worker"]}';
|
||||
'{"architecture":"arm64","executors":["remote-worker"],"protocolVersion":"1.0.0","supportTier":"tier1"}';
|
||||
const capabilitiesHash = createHash('sha256')
|
||||
.update(capabilitiesJson)
|
||||
.digest('hex');
|
||||
@@ -6371,7 +6371,7 @@ if (!migrationConnectionString) {
|
||||
'worker-ingress-a',
|
||||
);
|
||||
const capabilitiesJson =
|
||||
'{"architecture":"arm64","executors":["remote-worker"]}';
|
||||
'{"architecture":"arm64","executors":["remote-worker"],"protocolVersion":"1.0.0","supportTier":"tier1"}';
|
||||
const workers = new PostgresWorkerSessionRepository(
|
||||
ingressDatabase.pool,
|
||||
);
|
||||
|
||||
+4
-2
@@ -10,7 +10,9 @@ const {
|
||||
const SESSION_ID = '019f7094-a853-72f3-82ab-dfa08e6bd1c1';
|
||||
const DELIVERY_ID = '123e4567-e89b-42d3-a456-426614174601';
|
||||
const CAPABILITIES_HASH =
|
||||
'44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a';
|
||||
'b3d79017d91c477ffdf4a4dcc4ce9135ca053c921922ce0221920f905d8a2aa4';
|
||||
const CAPABILITIES_JSON =
|
||||
'{"architecture":"arm64","executors":["remote-worker"],"protocolVersion":"1.0.0","supportTier":"tier1"}';
|
||||
|
||||
function session(version, overrides = {}) {
|
||||
return {
|
||||
@@ -19,7 +21,7 @@ function session(version, overrides = {}) {
|
||||
generation: 2,
|
||||
status: 'online',
|
||||
version,
|
||||
capabilitiesJson: '{}',
|
||||
capabilitiesJson: CAPABILITIES_JSON,
|
||||
capabilitiesHash: CAPABILITIES_HASH,
|
||||
maxConcurrentRuns: 2,
|
||||
availableSlots: 1,
|
||||
|
||||
@@ -6,7 +6,9 @@ const {
|
||||
|
||||
const SESSION_ID = '019f7094-a853-72f3-82ab-dfa08e6bd1c1';
|
||||
const CAPABILITIES_HASH =
|
||||
'44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a';
|
||||
'b3d79017d91c477ffdf4a4dcc4ce9135ca053c921922ce0221920f905d8a2aa4';
|
||||
const CAPABILITIES_JSON =
|
||||
'{"architecture":"arm64","executors":["remote-worker"],"protocolVersion":"1.0.0","supportTier":"tier1"}';
|
||||
|
||||
test('pins the shared transition status parameter to PostgreSQL varchar', async () => {
|
||||
const events = [];
|
||||
@@ -16,7 +18,7 @@ test('pins the shared transition status parameter to PostgreSQL varchar', async
|
||||
generation: 2,
|
||||
status: 'online',
|
||||
version: 3,
|
||||
capabilitiesJson: '{}',
|
||||
capabilitiesJson: CAPABILITIES_JSON,
|
||||
capabilitiesHash: CAPABILITIES_HASH,
|
||||
maxConcurrentRuns: 2,
|
||||
availableSlots: 1,
|
||||
|
||||
@@ -0,0 +1,144 @@
|
||||
import { semver } from '../versioning/pinnedSemver';
|
||||
|
||||
export const REMOTE_WORKER_PROTOCOL_VERSION = '1.0.0';
|
||||
export const REMOTE_WORKER_PROTOCOL_RANGE = '>=1.0.0 <2.0.0';
|
||||
|
||||
export const REMOTE_WORKER_SUPPORT_TIERS = [
|
||||
'tier1',
|
||||
'candidate',
|
||||
'experimental',
|
||||
'legacy-only',
|
||||
] as const;
|
||||
|
||||
export type RemoteWorkerSupportTier =
|
||||
(typeof REMOTE_WORKER_SUPPORT_TIERS)[number];
|
||||
|
||||
export const REMOTE_WORKER_ARCHITECTURES_BY_SUPPORT_TIER = Object.freeze({
|
||||
tier1: Object.freeze(['amd64', 'arm64'] as const),
|
||||
candidate: Object.freeze(['ppc64le', 's390x'] as const),
|
||||
experimental: Object.freeze(['arm/v7'] as const),
|
||||
'legacy-only': Object.freeze(['arm/v6', '386'] as const),
|
||||
});
|
||||
|
||||
export type RemoteWorkerArchitecture =
|
||||
(typeof REMOTE_WORKER_ARCHITECTURES_BY_SUPPORT_TIER)[RemoteWorkerSupportTier][number];
|
||||
|
||||
const REMOTE_WORKER_ARCHITECTURES = Object.freeze(
|
||||
Object.values(REMOTE_WORKER_ARCHITECTURES_BY_SUPPORT_TIER).flat(),
|
||||
);
|
||||
|
||||
function invalid(message: string): never {
|
||||
throw new TypeError(
|
||||
`Remote Worker compatibility value is invalid: ${message}`,
|
||||
);
|
||||
}
|
||||
|
||||
export function normalizeRemoteWorkerProtocolVersion(value: unknown): string {
|
||||
if (
|
||||
typeof value !== 'string' ||
|
||||
value.length < 1 ||
|
||||
value.length > 64 ||
|
||||
semver().valid(value) === null
|
||||
) {
|
||||
invalid('protocolVersion is not semver');
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
export function normalizeRemoteWorkerArchitecture(
|
||||
value: unknown,
|
||||
): RemoteWorkerArchitecture {
|
||||
if (
|
||||
typeof value !== 'string' ||
|
||||
!REMOTE_WORKER_ARCHITECTURES.includes(value as RemoteWorkerArchitecture)
|
||||
) {
|
||||
invalid('architecture is outside the release support policy');
|
||||
}
|
||||
return value as RemoteWorkerArchitecture;
|
||||
}
|
||||
|
||||
export function remoteWorkerSupportTierForArchitecture(
|
||||
value: unknown,
|
||||
): RemoteWorkerSupportTier {
|
||||
const architecture = normalizeRemoteWorkerArchitecture(value);
|
||||
for (const supportTier of REMOTE_WORKER_SUPPORT_TIERS) {
|
||||
if (
|
||||
(
|
||||
REMOTE_WORKER_ARCHITECTURES_BY_SUPPORT_TIER[
|
||||
supportTier
|
||||
] as readonly string[]
|
||||
).includes(architecture)
|
||||
) {
|
||||
return supportTier;
|
||||
}
|
||||
}
|
||||
return invalid('architecture has no supportTier');
|
||||
}
|
||||
|
||||
export function remoteWorkerArchitectureForNodeRuntime(
|
||||
nodeArchitecture: string,
|
||||
armVersion?: unknown,
|
||||
): RemoteWorkerArchitecture {
|
||||
if (nodeArchitecture === 'x64') return 'amd64';
|
||||
if (nodeArchitecture === 'arm64') return 'arm64';
|
||||
if (nodeArchitecture === 'ppc64') return 'ppc64le';
|
||||
if (nodeArchitecture === 's390x') return 's390x';
|
||||
if (nodeArchitecture === 'ia32') return '386';
|
||||
if (nodeArchitecture === 'arm' && (armVersion === 6 || armVersion === '6')) {
|
||||
return 'arm/v6';
|
||||
}
|
||||
if (nodeArchitecture === 'arm' && (armVersion === 7 || armVersion === '7')) {
|
||||
return 'arm/v7';
|
||||
}
|
||||
return invalid('Node runtime architecture is unsupported or ambiguous');
|
||||
}
|
||||
|
||||
export function normalizeRemoteWorkerProtocolVersionRange(
|
||||
value: unknown,
|
||||
): string {
|
||||
if (
|
||||
typeof value !== 'string' ||
|
||||
value.length < 1 ||
|
||||
value.length > 128 ||
|
||||
semver().validRange(value) === null
|
||||
) {
|
||||
invalid('protocolVersionRange is not semver');
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
export function normalizeRemoteWorkerSupportTier(
|
||||
value: unknown,
|
||||
): RemoteWorkerSupportTier {
|
||||
if (
|
||||
typeof value !== 'string' ||
|
||||
!REMOTE_WORKER_SUPPORT_TIERS.includes(value as RemoteWorkerSupportTier)
|
||||
) {
|
||||
invalid('supportTier is unknown');
|
||||
}
|
||||
return value as RemoteWorkerSupportTier;
|
||||
}
|
||||
|
||||
export function assertRemoteWorkerCompatibilityCapability(
|
||||
value: unknown,
|
||||
): void {
|
||||
if (!value || typeof value !== 'object' || Array.isArray(value)) {
|
||||
invalid('capabilities must be an object');
|
||||
}
|
||||
const source = value as Record<string, unknown>;
|
||||
const architecture = normalizeRemoteWorkerArchitecture(source.architecture);
|
||||
normalizeRemoteWorkerProtocolVersion(source.protocolVersion);
|
||||
const supportTier = normalizeRemoteWorkerSupportTier(source.supportTier);
|
||||
if (remoteWorkerSupportTierForArchitecture(architecture) !== supportTier) {
|
||||
invalid('architecture does not belong to supportTier');
|
||||
}
|
||||
}
|
||||
|
||||
export function remoteWorkerProtocolIsCompatible(
|
||||
protocolVersion: string,
|
||||
requiredRange = REMOTE_WORKER_PROTOCOL_RANGE,
|
||||
): boolean {
|
||||
const version = normalizeRemoteWorkerProtocolVersion(protocolVersion);
|
||||
const range = normalizeRemoteWorkerProtocolVersionRange(requiredRange);
|
||||
return semver().satisfies(version, range, { includePrerelease: true });
|
||||
}
|
||||
@@ -2,6 +2,20 @@ import { createHash } from 'node:crypto';
|
||||
import type { WorkerSessionRecord } from '../worker/workerSession';
|
||||
import { assertWorkerSessionRecord } from '../worker/workerSession';
|
||||
import { semver } from '../versioning/pinnedSemver';
|
||||
import {
|
||||
REMOTE_WORKER_PROTOCOL_RANGE,
|
||||
REMOTE_WORKER_SUPPORT_TIERS,
|
||||
normalizeRemoteWorkerArchitecture,
|
||||
normalizeRemoteWorkerProtocolVersion,
|
||||
normalizeRemoteWorkerProtocolVersionRange,
|
||||
normalizeRemoteWorkerSupportTier,
|
||||
remoteWorkerProtocolIsCompatible,
|
||||
remoteWorkerSupportTierForArchitecture,
|
||||
type RemoteWorkerArchitecture,
|
||||
type RemoteWorkerSupportTier,
|
||||
} from './remoteWorkerCompatibility';
|
||||
|
||||
export * from './remoteWorkerCompatibility';
|
||||
|
||||
export const REMOTE_WORKER_EXECUTOR_CAPABILITY = 'remote-worker';
|
||||
export const MAX_REMOTE_PLACEMENT_VALUES = 16;
|
||||
@@ -20,8 +34,10 @@ export interface RemoteWorkerRuntimeCapability {
|
||||
}
|
||||
|
||||
export interface RemoteWorkerCapabilities {
|
||||
readonly architecture: string;
|
||||
readonly architecture: RemoteWorkerArchitecture;
|
||||
readonly executors: readonly string[];
|
||||
readonly protocolVersion: string;
|
||||
readonly supportTier: RemoteWorkerSupportTier;
|
||||
readonly operatingSystem?: string;
|
||||
readonly runtimes?: readonly RemoteWorkerRuntimeCapability[];
|
||||
readonly labels?: Readonly<Record<string, string>>;
|
||||
@@ -45,10 +61,12 @@ export interface RemoteWorkerRuntimeRequirement {
|
||||
|
||||
export interface RemoteWorkerPlacementSpec {
|
||||
readonly required?: Readonly<{
|
||||
readonly architectures?: readonly string[];
|
||||
readonly architectures?: readonly RemoteWorkerArchitecture[];
|
||||
readonly operatingSystems?: readonly string[];
|
||||
readonly executors?: readonly string[];
|
||||
readonly protocolVersionRange?: string;
|
||||
readonly runtimes?: readonly RemoteWorkerRuntimeRequirement[];
|
||||
readonly supportTiers?: readonly RemoteWorkerSupportTier[];
|
||||
readonly labels?: Readonly<Record<string, string>>;
|
||||
readonly minMemoryBytes?: number;
|
||||
readonly minDiskBytes?: number;
|
||||
@@ -63,6 +81,8 @@ export interface RemoteWorkerPlacementSpec {
|
||||
|
||||
export type RemoteWorkerPlacementMismatch =
|
||||
| 'worker_unavailable'
|
||||
| 'support_tier'
|
||||
| 'protocol_version'
|
||||
| 'architecture'
|
||||
| 'operating_system'
|
||||
| 'executor'
|
||||
@@ -186,17 +206,19 @@ export function normalizeRemoteWorkerCapabilities(
|
||||
const source = object(value, 'capabilities');
|
||||
exactKeys(
|
||||
source,
|
||||
['architecture', 'executors'],
|
||||
['architecture', 'executors', 'protocolVersion', 'supportTier'],
|
||||
['capacity', 'features', 'labels', 'operatingSystem', 'runtimes'],
|
||||
'capabilities',
|
||||
);
|
||||
const architecture = boundedString(
|
||||
source.architecture,
|
||||
'architecture',
|
||||
32,
|
||||
CAPABILITY_NAME,
|
||||
);
|
||||
const architecture = normalizeRemoteWorkerArchitecture(source.architecture);
|
||||
const executors = sortedStrings(source.executors, 'executors', 16, false);
|
||||
const protocolVersion = normalizeRemoteWorkerProtocolVersion(
|
||||
source.protocolVersion,
|
||||
);
|
||||
const supportTier = normalizeRemoteWorkerSupportTier(source.supportTier);
|
||||
if (remoteWorkerSupportTierForArchitecture(architecture) !== supportTier) {
|
||||
invalid('architecture does not belong to supportTier');
|
||||
}
|
||||
const operatingSystem =
|
||||
source.operatingSystem === undefined
|
||||
? undefined
|
||||
@@ -330,6 +352,8 @@ export function normalizeRemoteWorkerCapabilities(
|
||||
return Object.freeze({
|
||||
architecture,
|
||||
executors,
|
||||
protocolVersion,
|
||||
supportTier,
|
||||
...(operatingSystem === undefined ? {} : { operatingSystem }),
|
||||
...(runtimes === undefined ? {} : { runtimes }),
|
||||
...(labels === undefined ? {} : { labels }),
|
||||
@@ -377,6 +401,21 @@ function optionalStringList(
|
||||
: sortedStrings(value, label, MAX_REMOTE_PLACEMENT_VALUES);
|
||||
}
|
||||
|
||||
function optionalArchitectureList(
|
||||
value: unknown,
|
||||
label: string,
|
||||
): readonly RemoteWorkerArchitecture[] | undefined {
|
||||
if (value === undefined) return undefined;
|
||||
if (!Array.isArray(value) || value.length > MAX_REMOTE_PLACEMENT_VALUES) {
|
||||
invalid(`${label} is invalid`);
|
||||
}
|
||||
const result = value.map((item) => normalizeRemoteWorkerArchitecture(item));
|
||||
if (new Set(result).size !== result.length) {
|
||||
invalid(`${label} contains duplicates`);
|
||||
}
|
||||
return Object.freeze([...result].sort());
|
||||
}
|
||||
|
||||
export function normalizeRemoteWorkerPlacement(
|
||||
value: unknown,
|
||||
): RemoteWorkerPlacementSpec {
|
||||
@@ -397,11 +436,13 @@ export function normalizeRemoteWorkerPlacement(
|
||||
'minDiskBytes',
|
||||
'minMemoryBytes',
|
||||
'operatingSystems',
|
||||
'protocolVersionRange',
|
||||
'runtimes',
|
||||
'supportTiers',
|
||||
],
|
||||
'placement.required',
|
||||
);
|
||||
const architectures = optionalStringList(
|
||||
const architectures = optionalArchitectureList(
|
||||
candidate.architectures,
|
||||
'placement.required.architectures',
|
||||
);
|
||||
@@ -413,6 +454,23 @@ export function normalizeRemoteWorkerPlacement(
|
||||
candidate.executors,
|
||||
'placement.required.executors',
|
||||
);
|
||||
const protocolVersionRange = candidate.protocolVersionRange === undefined
|
||||
? undefined
|
||||
: normalizeRemoteWorkerProtocolVersionRange(
|
||||
candidate.protocolVersionRange,
|
||||
);
|
||||
let supportTiers: readonly RemoteWorkerSupportTier[] | undefined;
|
||||
if (candidate.supportTiers !== undefined) {
|
||||
if (
|
||||
!Array.isArray(candidate.supportTiers) ||
|
||||
candidate.supportTiers.length > REMOTE_WORKER_SUPPORT_TIERS.length
|
||||
) invalid('placement.required.supportTiers is invalid');
|
||||
const mapped = candidate.supportTiers.map((value) =>
|
||||
normalizeRemoteWorkerSupportTier(value));
|
||||
if (new Set(mapped).size !== mapped.length)
|
||||
invalid('placement.required.supportTiers contains duplicates');
|
||||
supportTiers = Object.freeze([...mapped].sort());
|
||||
}
|
||||
const features = optionalStringList(
|
||||
candidate.features,
|
||||
'placement.required.features',
|
||||
@@ -468,7 +526,9 @@ export function normalizeRemoteWorkerPlacement(
|
||||
...(architectures === undefined ? {} : { architectures }),
|
||||
...(operatingSystems === undefined ? {} : { operatingSystems }),
|
||||
...(executors === undefined ? {} : { executors }),
|
||||
...(protocolVersionRange === undefined ? {} : { protocolVersionRange }),
|
||||
...(runtimes === undefined ? {} : { runtimes }),
|
||||
...(supportTiers === undefined ? {} : { supportTiers }),
|
||||
...(candidate.labels === undefined
|
||||
? {}
|
||||
: {
|
||||
@@ -594,6 +654,20 @@ export function evaluateRemoteWorkerPlacement(
|
||||
worker.leaseExpiresAtMs <= observedAtMs
|
||||
)
|
||||
mismatches.push('worker_unavailable');
|
||||
const requiredSupportTiers = required.supportTiers ?? ['tier1'];
|
||||
if (!requiredSupportTiers.includes(capabilities.supportTier))
|
||||
mismatches.push('support_tier');
|
||||
if (
|
||||
!remoteWorkerProtocolIsCompatible(
|
||||
capabilities.protocolVersion,
|
||||
REMOTE_WORKER_PROTOCOL_RANGE,
|
||||
) ||
|
||||
(required.protocolVersionRange !== undefined &&
|
||||
!remoteWorkerProtocolIsCompatible(
|
||||
capabilities.protocolVersion,
|
||||
required.protocolVersionRange,
|
||||
))
|
||||
) mismatches.push('protocol_version');
|
||||
if (
|
||||
required.architectures?.length &&
|
||||
!required.architectures.includes(capabilities.architecture)
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { createHash } from 'node:crypto';
|
||||
import { assertRemoteWorkerCompatibilityCapability } from '../remote-execution/remoteWorkerCompatibility';
|
||||
|
||||
export const WORKER_SESSION_STATUSES = ['online', 'draining', 'offline'] as const;
|
||||
export type WorkerSessionStatus = (typeof WORKER_SESSION_STATUSES)[number];
|
||||
@@ -141,6 +142,11 @@ export function assertWorkerCapabilitiesSnapshot(
|
||||
if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {
|
||||
invalid('Worker capabilities snapshot must be an object');
|
||||
}
|
||||
try {
|
||||
assertRemoteWorkerCompatibilityCapability(parsed);
|
||||
} catch {
|
||||
invalid('Worker capabilities compatibility contract is invalid');
|
||||
}
|
||||
if (
|
||||
createHash('sha256').update(capabilitiesJson, 'utf8').digest('hex') !==
|
||||
capabilitiesHash
|
||||
|
||||
@@ -13,6 +13,7 @@ import {
|
||||
type WorkerSessionRecord,
|
||||
type WorkerSessionStatus,
|
||||
} from './workerSession';
|
||||
import { canonicalRemoteWorkerCapabilities } from '../remote-execution/remoteWorkerPlacement';
|
||||
|
||||
export const WORKER_SESSION_REGISTER_SCHEMA =
|
||||
'qinglong/worker-session-register@v1';
|
||||
@@ -153,6 +154,13 @@ function validateRegister(
|
||||
command?.capabilitiesJson,
|
||||
command?.capabilitiesHash,
|
||||
);
|
||||
const canonical = canonicalRemoteWorkerCapabilities(
|
||||
JSON.parse(command.capabilitiesJson) as unknown,
|
||||
);
|
||||
if (
|
||||
canonical.json !== command.capabilitiesJson ||
|
||||
canonical.hash !== command.capabilitiesHash
|
||||
) throw new TypeError('capabilities snapshot is not canonical');
|
||||
assertWorkerConcurrency(
|
||||
command?.maxConcurrentRuns,
|
||||
command?.availableSlots,
|
||||
|
||||
@@ -20,6 +20,8 @@ function worker() {
|
||||
const snapshot = canonicalRemoteWorkerCapabilities({
|
||||
architecture: 'arm64',
|
||||
executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0',
|
||||
supportTier: 'tier1',
|
||||
operatingSystem: 'linux',
|
||||
runtimes: [{ name: 'node', version: '24.18.0' }],
|
||||
labels: { region: 'cn-east', tier: 'edge' },
|
||||
@@ -90,7 +92,10 @@ test('rejects non-canonical snapshots and reports bounded mismatch classes', ()
|
||||
const value = worker();
|
||||
const reordered = {
|
||||
...value,
|
||||
capabilitiesJson: JSON.stringify({ executors: ['remote-worker'], architecture: 'arm64' }),
|
||||
capabilitiesJson: JSON.stringify({
|
||||
executors: ['remote-worker'], architecture: 'arm64',
|
||||
protocolVersion: '1.0.0', supportTier: 'tier1',
|
||||
}),
|
||||
};
|
||||
reordered.capabilitiesHash = require('node:crypto')
|
||||
.createHash('sha256')
|
||||
@@ -102,7 +107,7 @@ test('rejects non-canonical snapshots and reports bounded mismatch classes', ()
|
||||
);
|
||||
const decision = evaluateRemoteWorkerPlacement(
|
||||
value,
|
||||
{ required: { architectures: ['x64'], labels: { region: 'eu' } } },
|
||||
{ required: { architectures: ['amd64'], labels: { region: 'eu' } } },
|
||||
20_000,
|
||||
);
|
||||
assert.deepEqual(decision.mismatches, ['architecture', 'label']);
|
||||
|
||||
@@ -36,6 +36,8 @@ test('uses pinned SemVer for remote runtime range admission', () => {
|
||||
const candidate = worker({
|
||||
architecture: 'arm64',
|
||||
executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0',
|
||||
supportTier: 'tier1',
|
||||
runtimes: [{ name: 'node', version: '24.18.0' }],
|
||||
});
|
||||
|
||||
@@ -54,3 +56,51 @@ test('uses pinned SemVer for remote runtime range admission', () => {
|
||||
/versionRange is not semver/,
|
||||
);
|
||||
});
|
||||
|
||||
test('defaults to Tier 1 protocol v1 and requires explicit legacy placement', () => {
|
||||
const legacy = worker({
|
||||
architecture: 'arm/v6', executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0', supportTier: 'legacy-only',
|
||||
});
|
||||
assert.deepEqual(evaluateRemoteWorkerPlacement(legacy, {}, 500), {
|
||||
matches: false, score: 0, mismatches: ['support_tier'],
|
||||
});
|
||||
assert.deepEqual(evaluateRemoteWorkerPlacement(
|
||||
legacy, { required: { supportTiers: ['legacy-only'] } }, 500,
|
||||
), { matches: true, score: 0, mismatches: [] });
|
||||
const incompatible = worker({
|
||||
architecture: 'arm64', executors: ['remote-worker'],
|
||||
protocolVersion: '2.0.0', supportTier: 'tier1',
|
||||
});
|
||||
assert.deepEqual(evaluateRemoteWorkerPlacement(incompatible, {}, 500), {
|
||||
matches: false, score: 0, mismatches: ['protocol_version'],
|
||||
});
|
||||
});
|
||||
|
||||
test('rejects unversioned capabilities and unknown support policy', () => {
|
||||
assert.throws(() => canonicalRemoteWorkerCapabilities({
|
||||
architecture: 'arm64', executors: ['remote-worker'],
|
||||
}), /shape is invalid/);
|
||||
assert.throws(() => normalizeRemoteWorkerPlacement({
|
||||
required: { supportTiers: ['unsupported'] },
|
||||
}), /supportTier is unknown/);
|
||||
assert.throws(() => normalizeRemoteWorkerPlacement({
|
||||
required: { protocolVersionRange: 'not-a-range' },
|
||||
}), /protocolVersionRange is not semver/);
|
||||
assert.throws(() => canonicalRemoteWorkerCapabilities({
|
||||
architecture: 'arm/v7', executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0', supportTier: 'tier1',
|
||||
}), /does not belong to supportTier/);
|
||||
});
|
||||
|
||||
test('keeps Worker architecture tiers aligned with the release identity', () => {
|
||||
const { REMOTE_WORKER_ARCHITECTURES_BY_SUPPORT_TIER } =
|
||||
require('../dist/remote-execution/remoteWorkerCompatibility');
|
||||
const release = require('../../../ql3-release.json');
|
||||
assert.deepEqual(REMOTE_WORKER_ARCHITECTURES_BY_SUPPORT_TIER, {
|
||||
tier1: release.architectureSupport.tier1,
|
||||
candidate: release.architectureSupport.candidates,
|
||||
experimental: release.architectureSupport.experimentalBlocked,
|
||||
'legacy-only': release.architectureSupport.legacyOnly,
|
||||
});
|
||||
});
|
||||
|
||||
@@ -8,7 +8,7 @@ const {
|
||||
} = require('../dist');
|
||||
|
||||
function capabilities() {
|
||||
const json = '{"architecture":"arm64","executors":["remote-worker"]}';
|
||||
const json = '{"architecture":"arm64","executors":["remote-worker"],"protocolVersion":"1.0.0","supportTier":"tier1"}';
|
||||
return {
|
||||
json,
|
||||
hash: createHash('sha256').update(json).digest('hex'),
|
||||
@@ -42,6 +42,10 @@ test('rejects forged capabilities, partial status capacity and invalid time', ()
|
||||
() => assertWorkerCapabilitiesSnapshot(snapshot.json, '0'.repeat(64)),
|
||||
InvalidWorkerSessionValueError,
|
||||
);
|
||||
const unversioned = '{"architecture":"arm64","executors":["remote-worker"]}';
|
||||
assert.throws(() => assertWorkerCapabilitiesSnapshot(
|
||||
unversioned, createHash('sha256').update(unversioned).digest('hex'),
|
||||
), /compatibility contract is invalid/);
|
||||
assert.throws(
|
||||
() =>
|
||||
assertWorkerSessionRecord({
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
'use strict';
|
||||
|
||||
const assert = require('node:assert/strict');
|
||||
const { createHash } = require('node:crypto');
|
||||
const { test } = require('node:test');
|
||||
const {
|
||||
createWorkerSessionHeartbeatRequestBody,
|
||||
@@ -21,9 +22,10 @@ const authority = Object.freeze({
|
||||
workerId: 'edge-1',
|
||||
sessionId: '018f5c64-9b9d-7f1a-8c2d-1234567890ac',
|
||||
});
|
||||
const capabilitiesJson = '{}';
|
||||
const capabilitiesHash =
|
||||
'44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a';
|
||||
const capabilitiesJson =
|
||||
'{"architecture":"arm64","executors":["remote-worker"],"protocolVersion":"1.0.0","supportTier":"tier1"}';
|
||||
const capabilitiesHash = createHash('sha256')
|
||||
.update(capabilitiesJson).digest('hex');
|
||||
|
||||
function record(overrides = {}) {
|
||||
return {
|
||||
|
||||
@@ -8,6 +8,8 @@ import {
|
||||
} from 'node:path';
|
||||
import {
|
||||
canonicalRemoteWorkerCapabilities,
|
||||
REMOTE_WORKER_EXECUTOR_CAPABILITY,
|
||||
remoteWorkerArchitectureForNodeRuntime,
|
||||
type RemoteWorkerCapabilities,
|
||||
} from '@qinglong/runtime-core/remote-dispatch';
|
||||
|
||||
@@ -206,7 +208,18 @@ async function capabilities(path: string): Promise<RemoteWorkerCapabilities> {
|
||||
const parsed = JSON.parse(
|
||||
new TextDecoder('utf-8', { fatal: true }).decode(bytes),
|
||||
) as unknown;
|
||||
return canonicalRemoteWorkerCapabilities(parsed).capabilities;
|
||||
const normalized = canonicalRemoteWorkerCapabilities(parsed).capabilities;
|
||||
if (!normalized.executors.includes(REMOTE_WORKER_EXECUTOR_CAPABILITY))
|
||||
throw new Error('remote-worker executor capability is required');
|
||||
const runtimeArchitecture = remoteWorkerArchitectureForNodeRuntime(
|
||||
process.arch,
|
||||
(process.config.variables as Readonly<Record<string, unknown>>)[
|
||||
'arm_version'
|
||||
],
|
||||
);
|
||||
if (normalized.architecture !== runtimeArchitecture)
|
||||
throw new Error('capability architecture does not match the Node runtime');
|
||||
return normalized;
|
||||
} catch (error) {
|
||||
throw new WorkerProcessConfigError(
|
||||
'QL3_WORKER_CAPABILITIES_FILE is unavailable or invalid',
|
||||
|
||||
@@ -40,9 +40,11 @@ async function temporaryStorage() {
|
||||
|
||||
function capabilities() {
|
||||
return {
|
||||
architecture: 'x64',
|
||||
architecture: 'amd64',
|
||||
operatingSystem: 'linux',
|
||||
executors: ['local_process'],
|
||||
executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0',
|
||||
supportTier: 'tier1',
|
||||
runtimes: [{ name: 'node', version: '24.14.0' }],
|
||||
labels: {},
|
||||
capacity: { cpuCores: 1, memoryBytes: 256 * 1024 * 1024 },
|
||||
|
||||
@@ -16,15 +16,24 @@ const {
|
||||
WorkerProcessError,
|
||||
runProductionWorkerProcess,
|
||||
} = require('@qinglong/worker-runtime/process');
|
||||
const {
|
||||
remoteWorkerArchitectureForNodeRuntime,
|
||||
remoteWorkerSupportTierForArchitecture,
|
||||
} = require('@qinglong/runtime-core/remote-dispatch');
|
||||
|
||||
async function environment(t) {
|
||||
const root = await mkdtemp(path.join(os.tmpdir(), 'ql3-worker-process-'));
|
||||
t.after(() => rm(root, { recursive: true, force: true }));
|
||||
const capabilities = path.join(root, 'capabilities.json');
|
||||
const architecture = remoteWorkerArchitectureForNodeRuntime(
|
||||
process.arch, process.config.variables.arm_version,
|
||||
);
|
||||
await writeFile(capabilities, JSON.stringify({
|
||||
architecture: 'x64',
|
||||
architecture,
|
||||
operatingSystem: 'linux',
|
||||
executors: ['local_process'],
|
||||
executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0',
|
||||
supportTier: remoteWorkerSupportTierForArchitecture(architecture),
|
||||
}));
|
||||
await chmod(capabilities, 0o444);
|
||||
return {
|
||||
|
||||
@@ -14,6 +14,16 @@ const {
|
||||
WorkerProcessConfigError,
|
||||
loadWorkerProcessConfig,
|
||||
} = require('@qinglong/worker-runtime/process-config');
|
||||
const {
|
||||
remoteWorkerArchitectureForNodeRuntime,
|
||||
remoteWorkerSupportTierForArchitecture,
|
||||
} = require('@qinglong/runtime-core/remote-dispatch');
|
||||
|
||||
const RUNTIME_ARCHITECTURE = remoteWorkerArchitectureForNodeRuntime(
|
||||
process.arch, process.config.variables.arm_version,
|
||||
);
|
||||
const RUNTIME_SUPPORT_TIER =
|
||||
remoteWorkerSupportTierForArchitecture(RUNTIME_ARCHITECTURE);
|
||||
|
||||
async function fixture(t) {
|
||||
const root = await mkdtemp(path.join(os.tmpdir(), 'ql3-worker-config-'));
|
||||
@@ -22,9 +32,11 @@ async function fixture(t) {
|
||||
await writeFile(
|
||||
capabilitiesFile,
|
||||
JSON.stringify({
|
||||
architecture: 'arm64',
|
||||
architecture: RUNTIME_ARCHITECTURE,
|
||||
operatingSystem: 'linux',
|
||||
executors: ['local_process'],
|
||||
executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0',
|
||||
supportTier: RUNTIME_SUPPORT_TIER,
|
||||
runtimes: [{ name: 'node', version: '24.18.0' }],
|
||||
labels: { site: 'edge-a' },
|
||||
capacity: {
|
||||
@@ -97,8 +109,10 @@ test('loads canonical edge defaults and bounded node overrides', async (t) => {
|
||||
assert.equal(edge.workerId, 'router-worker-1');
|
||||
assert.equal(edge.origin, 'https://control.example.internal:5801');
|
||||
assert.deepEqual(edge.capabilities, {
|
||||
architecture: 'arm64',
|
||||
executors: ['local_process'],
|
||||
architecture: RUNTIME_ARCHITECTURE,
|
||||
executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0',
|
||||
supportTier: RUNTIME_SUPPORT_TIER,
|
||||
operatingSystem: 'linux',
|
||||
runtimes: [{ name: 'node', version: '24.18.0' }],
|
||||
labels: { site: 'edge-a' },
|
||||
@@ -182,3 +196,19 @@ test('rejects widened profiles, origins, heartbeat and filesystem configuration'
|
||||
WorkerProcessConfigError,
|
||||
);
|
||||
});
|
||||
|
||||
test('rejects a Worker capability file that cannot satisfy remote placement', async (t) => {
|
||||
const current = await fixture(t);
|
||||
await chmod(current.capabilitiesFile, 0o644);
|
||||
await writeFile(current.capabilitiesFile, JSON.stringify({
|
||||
architecture: RUNTIME_ARCHITECTURE,
|
||||
executors: ['local_process'],
|
||||
protocolVersion: '1.0.0',
|
||||
supportTier: RUNTIME_SUPPORT_TIER,
|
||||
}));
|
||||
await chmod(current.capabilitiesFile, 0o444);
|
||||
await assert.rejects(
|
||||
loadWorkerProcessConfig(current.environment),
|
||||
WorkerProcessConfigError,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -17,9 +17,11 @@ const SESSION_ID = '018f5c64-9b9d-7f1a-8c2d-1234567890ac';
|
||||
|
||||
function capabilities() {
|
||||
return {
|
||||
architecture: 'x64',
|
||||
architecture: 'amd64',
|
||||
operatingSystem: 'linux',
|
||||
executors: ['local_process'],
|
||||
executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0',
|
||||
supportTier: 'tier1',
|
||||
runtimes: [{ name: 'node', version: '24.14.0' }],
|
||||
labels: {},
|
||||
capacity: { cpuCores: 1, memoryBytes: 256 * 1024 * 1024 },
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
'use strict';
|
||||
|
||||
const assert = require('node:assert/strict');
|
||||
const { createHash } = require('node:crypto');
|
||||
const { test } = require('node:test');
|
||||
const {
|
||||
WORKER_SESSION_HEARTBEAT_SCHEMA,
|
||||
@@ -19,9 +20,10 @@ const authority = {
|
||||
workerId: 'edge-1',
|
||||
sessionId: '018f5c64-9b9d-7f1a-8c2d-1234567890ac',
|
||||
};
|
||||
const capabilitiesJson = '{}';
|
||||
const capabilitiesHash =
|
||||
'44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a';
|
||||
const capabilitiesJson =
|
||||
'{"architecture":"arm64","executors":["remote-worker"],"protocolVersion":"1.0.0","supportTier":"tier1"}';
|
||||
const capabilitiesHash = createHash('sha256')
|
||||
.update(capabilitiesJson).digest('hex');
|
||||
|
||||
function client(exchange) {
|
||||
const transport = new WorkerIngressHttpsClient({
|
||||
|
||||
@@ -5501,7 +5501,7 @@ async function runCredentialDeliveryCommitResponseLossMatrix(options) {
|
||||
|
||||
const sessions = new PostgresWorkerSessionRepository(ingressDatabase.pool);
|
||||
const capabilitiesJson =
|
||||
'{"architecture":"arm64","executors":["remote-worker"]}';
|
||||
'{"architecture":"arm64","executors":["remote-worker"],"protocolVersion":"1.0.0","supportTier":"tier1"}';
|
||||
const capabilitiesHash = createHash('sha256')
|
||||
.update(capabilitiesJson)
|
||||
.digest('hex');
|
||||
|
||||
@@ -15,6 +15,10 @@ const { createMutualTlsPki } = require('./lib/ql3-live-pki.cjs');
|
||||
const {
|
||||
validateWorkerKubernetesRolloutLiveReport,
|
||||
} = require('./ql3-worker-kubernetes-rollout-live-audit.cjs');
|
||||
const {
|
||||
remoteWorkerArchitectureForNodeRuntime,
|
||||
remoteWorkerSupportTierForArchitecture,
|
||||
} = require('../packages/ql3-runtime-core/dist/remote-execution/remoteWorkerCompatibility.js');
|
||||
|
||||
const ROOT = path.resolve(__dirname, '..');
|
||||
const K3S_IMAGE = 'rancher/k3s:v1.34.3-k3s1';
|
||||
@@ -1613,6 +1617,9 @@ async function main(argv = process.argv.slice(2)) {
|
||||
'tls.crt': pkiMaterial.oldClientCertificate,
|
||||
},
|
||||
});
|
||||
const workerArchitecture = remoteWorkerArchitectureForNodeRuntime(
|
||||
process.arch, process.config.variables.arm_version,
|
||||
);
|
||||
apply({
|
||||
apiVersion: 'v1',
|
||||
kind: 'ConfigMap',
|
||||
@@ -1621,9 +1628,12 @@ async function main(argv = process.argv.slice(2)) {
|
||||
'worker-id': WORKER_ID,
|
||||
'control-origin': `https://${ingressServername}:5801`,
|
||||
'capabilities.json': `${JSON.stringify({
|
||||
architecture: process.arch,
|
||||
architecture: workerArchitecture,
|
||||
operatingSystem: 'linux',
|
||||
executors: ['local_process'],
|
||||
executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0',
|
||||
supportTier:
|
||||
remoteWorkerSupportTierForArchitecture(workerArchitecture),
|
||||
runtimes: [{ name: 'node', version: '24.18.0' }],
|
||||
labels: { contract: 'kubernetes-product-live' },
|
||||
capacity: { cpuCores: 1, memoryBytes: 256 * 1024 * 1024 },
|
||||
|
||||
@@ -62,6 +62,10 @@ const {
|
||||
const {
|
||||
createCertificateAuthority,
|
||||
} = require('../packages/ql3-worker-runtime/test/helpers/certificateAuthority.cjs');
|
||||
const {
|
||||
remoteWorkerArchitectureForNodeRuntime,
|
||||
remoteWorkerSupportTierForArchitecture,
|
||||
} = require('../packages/ql3-runtime-core/dist/remote-execution/remoteWorkerCompatibility.js');
|
||||
|
||||
const IMAGE = process.env.QL3_WORKER_POSTGRES_IMAGE ?? 'postgres:18';
|
||||
const LINUX_NODE_IMAGE =
|
||||
@@ -80,6 +84,8 @@ const AUTOMATION_MANAGER_USER = 'ql3_automation_manager';
|
||||
const AUTOMATION_MANAGER_PASSWORD = 'ql3_automation_manager_live';
|
||||
const APPROVAL_MANAGER_USER = 'ql3_approval_manager';
|
||||
const APPROVAL_MANAGER_PASSWORD = 'ql3_approval_manager_live';
|
||||
const RUN_MANAGER_USER = 'ql3_run_manager';
|
||||
const RUN_MANAGER_PASSWORD = 'ql3_run_manager_live';
|
||||
const PACKAGE_MANAGER_USER = 'ql3_package_manager';
|
||||
const PACKAGE_MANAGER_PASSWORD = 'ql3_package_manager_live';
|
||||
const PACKAGE_EXECUTOR_USER = 'ql3_package_executor';
|
||||
@@ -655,13 +661,18 @@ async function prepareWorkerFiles(root) {
|
||||
const certificateFile = path.join(authority, 'tls.crt');
|
||||
const privateKeyFile = path.join(authority, 'tls.key');
|
||||
const tokenFile = path.join(authority, 'credential-token');
|
||||
const architecture = remoteWorkerArchitectureForNodeRuntime(
|
||||
process.arch, process.config.variables.arm_version,
|
||||
);
|
||||
await Promise.all([
|
||||
writePrivate(
|
||||
capabilitiesFile,
|
||||
`${JSON.stringify({
|
||||
architecture: process.arch,
|
||||
architecture,
|
||||
operatingSystem: process.platform,
|
||||
executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0',
|
||||
supportTier: remoteWorkerSupportTierForArchitecture(architecture),
|
||||
runtimes: [{ name: 'node', version: process.versions.node }],
|
||||
labels: { contract: 'postgres-live' },
|
||||
capacity: {
|
||||
@@ -862,6 +873,9 @@ async function main() {
|
||||
await superuserDatabase.pool.query(
|
||||
`CREATE ROLE ${APPROVAL_MANAGER_USER} LOGIN PASSWORD '${APPROVAL_MANAGER_PASSWORD}'`,
|
||||
);
|
||||
await superuserDatabase.pool.query(
|
||||
`CREATE ROLE ${RUN_MANAGER_USER} LOGIN PASSWORD '${RUN_MANAGER_PASSWORD}'`,
|
||||
);
|
||||
await superuserDatabase.pool.query(
|
||||
`CREATE ROLE ${PACKAGE_MANAGER_USER} LOGIN PASSWORD '${PACKAGE_MANAGER_PASSWORD}'`,
|
||||
);
|
||||
@@ -962,9 +976,11 @@ async function main() {
|
||||
QL3_WORKER_INGRESS_TLS_CLIENT_CA_FILE: files.ingressClientCaFile,
|
||||
});
|
||||
assert.equal(ingressConfig.enabled, true);
|
||||
const runtimePort = createClusterWorkerRuntimePort(runtimeDatabase.pool, {
|
||||
artifactStore,
|
||||
});
|
||||
const runtimePort = createClusterWorkerRuntimePort(
|
||||
runtimeDatabase.pool,
|
||||
{ artifactStore },
|
||||
{ cancellationDispatchOwnerId: 'ql3-worker-postgres-live' },
|
||||
);
|
||||
const runtime = Object.freeze({
|
||||
...runtimePort,
|
||||
activation: Object.freeze({
|
||||
|
||||
@@ -14,6 +14,10 @@ const fixtures = path.resolve(
|
||||
__dirname,
|
||||
'../packages/ql3-cluster-control/test/fixtures/mtls',
|
||||
);
|
||||
const {
|
||||
remoteWorkerArchitectureForNodeRuntime,
|
||||
remoteWorkerSupportTierForArchitecture,
|
||||
} = require('../packages/ql3-runtime-core/dist/remote-execution/remoteWorkerCompatibility.js');
|
||||
|
||||
function argumentsMap() {
|
||||
return new Map(
|
||||
@@ -69,13 +73,18 @@ async function child() {
|
||||
const certificateFile = path.join(authority, 'tls.crt');
|
||||
const privateKeyFile = path.join(authority, 'tls.key');
|
||||
const tokenFile = path.join(authority, 'credential-token');
|
||||
const architecture = remoteWorkerArchitectureForNodeRuntime(
|
||||
process.arch, process.config.variables.arm_version,
|
||||
);
|
||||
await Promise.all([
|
||||
writePrivate(
|
||||
capabilitiesFile,
|
||||
`${JSON.stringify({
|
||||
architecture: process.arch,
|
||||
architecture,
|
||||
operatingSystem: process.platform,
|
||||
executors: ['local_process'],
|
||||
executors: ['remote-worker'],
|
||||
protocolVersion: '1.0.0',
|
||||
supportTier: remoteWorkerSupportTierForArchitecture(architecture),
|
||||
runtimes: [{ name: 'node', version: process.versions.node }],
|
||||
labels: {},
|
||||
capacity: {
|
||||
|
||||
@@ -299,10 +299,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
|
||||
rootSourceFileRoles: runtimeCore.rootSourceFileRoles,
|
||||
},
|
||||
{
|
||||
sourceFiles: 169,
|
||||
sourceFiles: 170,
|
||||
rootSourceFiles: 1,
|
||||
rootSourceLines: 160,
|
||||
nestedSourceFiles: 168,
|
||||
nestedSourceFiles: 169,
|
||||
rootSourceFileRoles: { 'index.ts': 'public_export' },
|
||||
},
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user